Data Management Integrated Subsystems

1. The "data acquisition" subsystem is responsible for retrieving the data from sensor packages and observation platforms in the network. Because of the wide variety of communications links and platform hardware used for measuring environmental data, the data acquisition subsystem is actually accomplished by a small number of independent software modules that implement communication with specific platform hardware or specific communications media. As new platform hardware configurations or communications channels are added to the observation network, it is easy to develop the corresponding acquisition component and integrate it into the existing acquisition subsystem.

2. The "data archival/decoding" subsystem is responsible for maintaining an archive of the source data. All incoming data are placed in a special directory known as the "inbox", where an automated "clerk" process picks up the data for archiving and decoding. The clerk process then calls individual decoder programs that convert the raw data files into a standardized interchange format for storage in the MySQL database. The clerk and its decoder programs also perform quality-control data checks to prevent erroneous data from entering the online database system. Any errors found in the incoming data are flagged and the source data files are left in the inbox until an operator takes corrective actions to eliminate the errors.

3. The MySQL database contains the decoded data as received from the observation platforms after they have passed through the initial decoder quality-control checks. Any further corrections or adjustments to the data are recorded in the database as separate "correction" records that are automatically applied to the data whenever they are extracted from the observations database.

4. The "data extraction" subsystem provides an interface to allow users to query the observation database. Currently the Hypertext Transfer Protocol (HTTP) and the Apache Web Server are the primary query interface to the database. The query interface has the ability to retrieve data from the observation database in a number of different formats (e.g., graphs, text) and can automatically apply various transformations to the data to present them in a form most useful to the end-user.

5. Surrounding the data extraction subsystem are a number of World Wide Web front ends that provide different views of the available data as well as ancillary information to assist the end-user in understanding the system.