Managing Data for a CUAHSI Water Data Service

The CUAHSI HIS Server has been designed to enhance the capability for actively managing data within an environmental observatory or experimental watershed. The data workflow process, centered on the observations data model (ODM) relational database is schematically illustrated below.

Raw data is first captured through it being loaded into ODM. This achieves a key requirement to first preserve the raw data. The standard format of ODM has enabled the development of tools designed to provide additional data processing, quality control and analysis capability, thereby providing justification and benefit to a user for putting the data in the system. The data management system is designed to provide the following functionality:

• First capture and preserve the raw data.
• Facilitate data processing, including quality control checking, perhaps adjusting or deleting spurious or incorrect values, and deriving higher level information from raw or lower level data.
• Preserve metadata so that there is a traceable heritage from raw measurements to useable information.
• Store sufficient metadata so that data can be unambiguously interpreted and used.
• Maintain semantically consistent controlled vocabulary metadata to enhance consistency between data stored in different HIS Servers.

ODM Data Management Tools and Resources

• Observations Data Model (ODM) schema and documentation
• ODM Tools— an application that allows data managers and ODM users to query, export, visualize, summarize, and edit data that is stored in an ODM database.
• ODM Streaming Data Loader— a tool for for streaming continuously measured sensor data into an ODM database. The Streaming Data Loader takes as input datalogger files and loads them into ODM in near real time as they are updated by a telemetry system.
• ODM Data Loader— an application for loading data from simple text or spreadsheet files into an ODM database.

Other Data Managment/Maintenance Topics

• Loading Data — some of the tools and techniques for getting your data into an ODM.
• Updating the Master Controlled Vocabulary System— promotes consistency between different instances of the ODM through a moderated system for requesting changes to the master ODM controlled vocabularies. Once changes are accepted, ODM tools provides functionality to reconcile local controlled vocabularies in local ODM instances with master controlled vocabularies.