Data Management Design Principles

1. Preserve source data and annotate data instead of modifying or deleting. Because CBI-DNR's products may be critically reviewed in legal proceedings, CBI-DNR discards as little information as possible. All source records from instruments are kept and archived in their original format, and these serve as the basis for all "derived" products. All changes (adjustments and deletions) are performed as annotations to the data sets without modifying the source data. In fact, it is possible for DNR to automatically regenerate the entire observation network database from the archived source documents.

2. Automate as much as possible. Automation is key to all CBI-DNR operations. Automation increases the capacity to manage data, reduces the opportunities for human error, ensures a consistent method for managing and processing data, and documents the procedures used (in the form of programs) to derive the data products.

3. Maintain a standard data interchange format. TCOON's early data management systems explicitly recognized the importance of a standardized data format that is easy to produce and easy for humans and automated systems to process. All data entering the TCOON data management system are first converted into a standardized ASCII representation with common time references and units. Also, all data files are tagged with sufficient metadata information to allow a consumer to determine the source and processing that has been performed on the data.

4. Avoid complex or proprietary components. Loose integration of simple components provides a more flexible and robust system than a tight integration of more complex components. Since 1991 all TCOON data management components have been Unix based; since 1996 all user interfaces have been Web based, and since 1998 all TCOON data management components have utilized standard PC-based hardware systems and open-source software packages such as Linux and MySQL. Using widely available and standardized system components has provided flexibility and stability in the data management environment.

5. Emphasize long-term reliability over short-term costs. Investments made in infrastructure design and implementation reap long-term benefits in the form of reduced operations costs and improved reliability. Maintenance and operation of a long-term observation network requires a long-term perspective for all of the day-to-day decisions that must be made. Software and procedural "quick fixes" are to be avoided, because a poor decision made in haste today often leads to an expensive repair in the future.