System Design

The CUAHSI-Hydrologic Information System is a distributed network of computers, linked together via the internet, some of them run by the CUAHSI-HIS project, some of them run by independent investigators, some of them run by 3rd-party organizations, such as the USGS and the EPA, and anyone with access to the internet can add their computer into this system. These various pieces function together as a single integrated system, where investigators publish their data and everyone has easy access to both investigator data as well as a variety of large data repositories in a standard way.

The Big Picture

The design for the CUAHSI HIS is illustrated in the adjacent figure. HIS has established foundational concepts and standards such as the observations data model (ODM) for the storage of point observations data, WaterML for the transmission of water data using web services and an ontology that associates semantically similar concepts to aid in information organization and data discovery. These foundational concepts support a variety of data storage and web service data transmission capabilities. WaterML is the XML based language used by HIS web services for the transmission of data. Variants of WaterML have also been adopted by some 3rd party agencies (USGS, NCDC) for their web services. HIS has developed a number of tools for working with data from ODM and WaterML based web services, both for data managers and for general users. These are grouped into web and client applications. Web applications run on a HIS server and are accessed using a browser, while client applications run on a users own computer. The applications are also separated into those that can be accessed by all and provide general access to HIS data, and those that are designed for data managers and may require authentication. The design diagram also groups components into those that are centrally maintained, those that are part of the distributable HIS server, and those that are distributable HIS client software. In developing HIS we have used a number of commercial software systems, like Microsoft SQL server and ArcGIS server. These allow us to focus on hydrology specific data system needs, without developing (or re-inventing) general purpose capabilities. The design diagram does not show these dependencies, which are indicated in the detailed descriptions of each component.

There are a number of different types of server systems in the HIS-CUAHSI system as well as a number of client applications.

Servers

Server Types

The server types are:

HIS Central Servers

These are the servers maintained by the HIS project to perform central system functions. The main centralized function is the hosting of WaterOneFlow web services, for a number of National databases identified to be of value to the hydrologic community.  Generally the data underlying these web services are maintained by the originating organization in their own format and system.  HIS has arrangements to extract data from these systems, transform it into WaterML and share it using WaterOneFlow web services.  Metadata catalogs have been developed and are hosted at HIS Central to index and support some of these web services. HIS Central servers also support the Central HIS WSDL Registry and other centralized system functions. Within HIS Central a number of servers are used to balance load and provide redundancy.

HIS Observatory Servers

These are the servers maintained by user groups for the publishing of data. The 11 WATERS test beds each have one of these. HIS Observatory Servers come in two flavors, standard and lite.

HIS OBSERVATORY SERVER (STANDARD)

The regular version uses the Standard versions of a number of commercial software products. The cost of the licenses required can be significant unless the organization owning the server is able to subscribe to Microsoft’s Academic Alliance program.

HIS OBSERVATORY SERVER (LITE)

The lite version of the HIS Observatory Server uses free or low cost (express) versions of much of the software, or in some cases, some of the pieces may even be missing. This version of the server costs little or nothing in licensing fees, but may have some reduced capabilities.

3rd-party WaterML Data Servers

CUAHSI HIS encourages the use of WaterML as a standard for sharing of data using web services. A number of organizations have started doing this. These 3rd party WaterOneFlow data services directly allow the HIS community access to their data.

3rd-party Custom WaterOneFlow Data Servers

There are a number of 3rd-party data servers that the HIS project felt were of significant value to the community, and therefore, worth the time and effort necessary to bring them into the HIS system. To do this, the HIS team has negotiated access arrangements and developed custom software that link these data servers directly to one or more of the HIS Central Servers and provided access to this data via custom WaterOneFlow web services on the central server. The list of WaterOneFlow web services available on HIS Central is given on the WaterOneFlow web page.

Server Configurations

HIS Central Servers

HIS Central Servers are comprised of the following components, which may be spread out over one or more physical servers with potentially redundant physical servers, as well:

• Microsoft Windows Server 2003
  • Microsoft Internet Information Server (IIS)
    • Web Services
      • 3rd-party data sources (Custom WaterOneFlow)
      • Central ODM databases (ODM WaterOneFlow)
      • HydroSeek native
      • Master controlled vocabulary
    • Web Sites
      • Network registry
      • Master controlled vocabulary registry
      • HydroSeek
      • HydroTagger
      • Main
    • Microsoft Visual Studio
      • SSIS
    • Microsoft SQL Server
      • Databases
        • 3rd-party metadata
        • HydroSeek
        • Master controlled vocabulary
        • Central ODM databases
    • ArcGIS Server
      • DASH (Data Access System for Hydrology - for 3rd-party databases and local ODM databases)

CUAHIS-HIS Servers

A CUAHSI-HIS Observatory Server is designed to function as the tool for a single investigator, or a small group of investigators, to publish their data. It comes in 2 main configurations, standard and lite. The standard version has the full compliment of features and has been optimized for an organization that has a Microsoft Developer Network Academic Alliance (MSDNAA) membership and also requires an ESRI ArcGIS Server license. The lite version uses free versions of much of the base software and does not have a map server.

CUAHSI-HIS SERVER (Standard)

This is a list of the base components that comprise a standard HIS observatory server and can be implemented on a Windows Server. This is the configuration deployed at WATERS Network test beds. This uses SQL Server and ArcGIS server functionality.

• Microsoft Windows Server 2003
  • Microsoft Internet Information Server (IIS)
    • Web Services:
      • Local ODM databases
  • Microsoft Visual Studio
    • SSIS
  • Microsoft SQL Server
    • Databases:
      • Local ODM databases
  • ArcGIS Server
    • DASH (Data Access System for Hydrology) for local ODM databases
      • Layers:
        • Local ODM layers

CUAHSI-HIS SERVER (Lite)

This is a list of the logical components that comprise a lite HIS observatory server and can be implemented on an XP or Windows Server with no additional software purchase required.

• Microsoft Windows XP or Server 2003
  • Microsoft Internet Information Server (IIS)
    • Web Services:
      • Local ODM databases
  • Microsoft Visual Studio Express
  • Microsoft SQL Server Express
    • Databases:
      • Local ODM network databases

3rd-party Data Servers

Both 3rd party custom and WaterOneFlow data servers may be part of the HIS system. Each has a custom configuration depending on the circumstances and data system used by its owner/operator, which can not be given in this design.

HIS Clients

HIS clients can be categorized into a number of groups, based on the combination of:

• What type of user is the client designed for? Is it for the general user trying to access data, or is it for a data manager that has more advanced needs but also has an account/password or other special authorization for advanced access to the system.
• Is the application installed locally on the client's computer, or is it accessed over the internet using a web browser?

Data Manager Clients

These are the tools that can be accessed by data managers that require the use of often-restricted ports (local access) and/or authentication.

LOCAL APPLICATIONS

These applications are installed locally on the user’s computer:

• Data Loader
• Streaming Data Loader
• ODM Tools
• Time Series Analyst (only conceptualized, at this time)

BROWSER APPLICATIONS

These applications run on one of the servers in the HIS system, but are accessed via web browser by the user:

• HydroTagger

General User Clients

These are the tools that can be used by individual general users.

BROWSER APPLICATIONS

These applications run on one of the servers in the HIS system, but are accessed via web browser by the user:

• HydroSeek
• DASH

LOCAL APPLICATIoNS

These are applications run locally on the user’s computer and use web services to access the data on one or more of the data servers in the system. As more and more commercial packages are upgraded to include accessing web services, this list is growing rapidly.

• Programming Languages-FORTRAN, C, VB
• GIS-ArcGIS,
• Excel
• Matlab
• IDL
• Statistics-Splus, R

In addition, there are a number of add-ons to these packages that expand there abilities to access CUAHSI-HIS data:

• WaterOneFlow Excel Client
• HydroGET
• HydroObjects