OHD/HL - papers: AMS98-2

IMPLEMENTATION OF MODERNIZED HYDROLOGIC OPERATIONS AND SERVICES IN THE NATIONAL WEATHER SERVICE: OVERVIEW AND STATUS

1.   INTRODUCTION

     The National Weather Service (NWS) is currently undergoing a major modernization effort, incorporating
new observing technologies and integrating the operations into a single forecasting environment.  One of the
goals of the modernization is to provide significantly improved hydrologic forecasts and warnings.  The NWS
Office of Hydrology is developing a set of capabilities to help meet this challenge.  These capabilities, referred
to as the Weather Forecast Office (WFO) Hydrologic Forecast System (WHFS), are being deployed nationally
as part of the Advanced Weather Interactive Processing System (AWIPS).

     The WHFS has been deployed in a number of field sites, serving as beta-test sites, since November,
1994.  Additionally, WHFS was deployed as part of the AWIPS Build 1 delivery in August 1996.  The WHFS
features an integrated data management approach using a relational database.  The individual applications
deployed with the initial AWIPS implementation include:  a series of graphical user interfaces for managing
the vast amount of reference and real-time data, a geographic display for monitoring and further analysis of
real time data, and a product generation application which automatically formats river forecasts and warnings
based on the observed and forecast data in the database.  An upgrade to the WHFS (Version 2.0), deployed
with AWIPS Build 3.1 in the Fall of 1997, includes the Weather Surveillance Radar - 1988 Doppler (WSR-88D)
Stage II precipitation processor, which combines the radar-based, hourly precipitation estimates (Stage I) with
hourly gage data to create gridded precipitation estimates.

     Future upgrades to the WHFS focus on enhancing the hydrologic forecasting capabilities.  These
forecasting capabilities include:  an area-wide flash flood prediction system which compares flash flood
guidance to precipitation in order to assess the potential for flash flooding over a given area, a site-specific
headwater modeling system to generate forecasts for specific headwater points, a simplified dam break
procedure to assess the impacts of dam failures on downstream locations, a set of improved data quality
control/monitoring techniques, and additional report generation capabilities.  These upgrades will be fielded
as part of the regular, incremental enhancements to be made to the AWIPS at all WFOs.

     Those offices currently using the WHFS have been active in providing feedback to the Office of Hydrology
regarding existing capabilities and suggesting future enhancements.  As development continues, this feedback
has helped ensure that the fielded system meets the needs of the hydrologic program.  Support of field
operations also involves ongoing efforts by the Office of Hydrology in developing and implementing enhanced
training and maintenance/support plans.  These efforts will be necessary as AWIPS moves forward into its
deployment phase and the WHFS is implemented at all 119 WFOs and all 13 River Forecast Centers (RFCs)
in the NWS.

2.   WHFS ORIGINS

     From the late-1980s to 1992, collaborative efforts between the NWS Office of Hydrology and the National
Oceanic and Atmospheric Administration's (NOAA) Forecast Systems Laboratory led to a set of prototype
hydrologic applications.  During this same time, the requirements for the AWIPS hydrometeorologic
applications were being formulated and formally documented by the NWS (NOAA, 1992).  The development
of the prototype applications and the requirements were done hand-in-hand, resulting in mutual benefits to
these simultaneous activities.

     By mid-1993, the Office of Hydrology assumed responsibility for developing these requirements into the
WHFS applications supporting the Hydrologic Services Program at the WFO (Shelton and May, 1996).  These
application requirements have been refined considerably as the development and deployment of the WHFS
continues through the AWIPS era.  Additional functionality has been added, particularly in the area of data
processing and data management functions.  The existing and future functions of the WHFS are summarized
in the following sections.

3.   WHFS CURRENT CAPABILITIES

3.1  Data Management Approach

     The WHFS applications operate against a single relational database known as the Integrated Hydrologic
Forecast System (IHFS) Database or IHFS_DB (Roe, et.al., 1998).  This database is managed by a
commercial relational database management system.  There is no passing of data between application
programs nor between major functions within a single application program.  Each major application function
directly queries or updates the database as necessary so that all functions are looking at the same data. 
Extensive use is made of foreign key relationships to maintain referential integrity.  Most data attributes that
take on values from specific valid lists (e.g., WFO identifiers) are connected to lookup tables via foreign keys
to make sure that invalid values are not permitted for those data attributes.

     The IHFS_DB contains six major categories of data that support the WHFS application programs and
other non-WHFS application programs that exist at a WFO or RFC.
The major data categories are:

-  operational observations and forecasts for stations
-  reference data for stations
-  radar and precipitation analysis grids
-  geographic reference data for points, vectors, and polygons
-  application and database control parameters
-  valid value lookup lists

     The relational database tables hold all data except grid data values.  Grids are stored using a combination
of the relational database tables and host files.  Each grid has its attributes stored in a relational database
table including an attribute which points to the host file that stores the actual grid data values.

     The design and documentation for the IHFS_DB is provided and maintained through the use of a modern
Computer-Aided Software Engineering (CASE) tool that implements the Chen entity-relationship approach to
data modeling (Chen, 1977).  The CASE tool repository of diagrams and definitions serves as the environment
for conducting future design and enhancement of the IHFS_DB and can be used  as an information resource
to aid local software development activities (Office of Hydrology, 1997).

3.2  Hydrologic Database Manager

     The hydrologic database manager program (HydroBase) is an interactive application that allows the user
to display, update, add, and delete all of the various types of parametric reference data (as opposed to
operational observations and forecasts) stored in the IHFS_DB for a forecast office's area of responsibility. 
It is essentially a database administrator tool primarily used by Service Hydrologists and Hydrologic Focal
Points (rather than forecasters).  The types of data managed by HydroBase include:

- river station reference information (Form E-19)
- meteorological station reference information
- radar station reference information
- station data sources
- data ingest control parameters
- valid lists (e.g., counties, states, zones, cities, etc.)
- application control parameters

     HydroBase also produces various reports such as a historical flood diagram, a Form E-19 report, a Form
B-44 report, a station list report, and a station classification report.

     The Version 2.0 release of the WHFS  includes many enhancements to HydroBase.  The menu hierarchy
has been simplified and leveled to make it much easier to navigate through the application.  The report
generation capability is new and includes choices to view, print, e-mail, and save reports.  Many new dialogs
have been added to  allow management of data sets for states, counties, forecast zones, cities, radars, purge
parameters, data quality tolerance range checks, Stage II Processor application parameters, the operational
data ingest filter, and areal and vector geographic plotting objects for HydroView.

     Plans for enhancements in the near future for HydroBase include: the ability to edit Dam Catalog
information, the addition of a Form E-3 flood report, enhancements to the existing Form E-19 report, and
support for management of forecast service information for river forecast points.

3.3  Hydrologic Data Viewer

     The hydrologic data viewer (HydroView) provides the user with a comprehensive set of displays to monitor
and analyze the hydrometeorological situation.  These displays include the geographic display of a WFO's
Hydrologic Service Area (HSA), and the various data overlays available for this display.  Also available are
X-Y time-series plots for graphical review of data,  numerous textual displays, lists, and reports of both
observational data and static reference data for the stations in the HSA.

     Within the HydroView geographic displays, a forecaster can display a variety of hydrometeorological
overlays with user-controlled zoom levels.  The forecaster can toggle displays of rivers, basins, zones,
counties, cities, and highways.  Operational data overlays are available and are categorized as either point,
areal, or gridded data displays.

     Point data for river stage, precipitation, snow, and temperature stations can be overlaid on geographic
displays.  The stations can be shown as icons only, or both the station icon and the selected data element can
be displayed.  The stations can be filtered according to their "source", thereby allowing data from different data
networks to be analyzed.  The river stage stations are shown as color-coded icons corresponding to their
relation to action stage and flood stage; these thresholds stages or the departure from them can be annotated. 
The precipitation data can be displayed for various durations ranging from 1 to 72 hours, and for the specified
ending time.  A sample HydroView display showing point data and the control window for it is given in Figure
1.


                      Figure 1. The HydroView Main Map and Display Control Dialog

     Areal data consisting of Flash Flood Guidance (FFG) and Mean Areal Precipitation (MAP) can be
displayed for counties, zones, and basins.  The MAP data is based on the Stage II gage-only grid and the
Stage II gage-radar grid (Shedd and Fulton, 1993).  These two data sets can also be displayed in grid form,
as can the Stage I grid.  The MAP data is displayable for durations ranging from 1 to 24 hours.

     HydroView can display information for the currently selected station in graphical and tabular form.  An X-Y
time-series plot of any of the operational data for the station is available to allow detailed review of the
observed and forecast data for a station.  The same information can be viewed in tabular form, from which
data can be edited, deleted, or inserted.  Assorted other displays are available for the selected station that
provide access to the detailed reference data associated with the station, such as flood impacts, a list of
contacts, historical crest data, etc.

     For WHFS Version 2.0,  an across-the-board review of HydroView functions resulted in significant
enhancements to its existing functionality.  In addition, features new to this release include display of areal
data and gridded data sets, distinction between rivers and streams, cities and towns, and highways and roads
in the overlays, and display and review of data quality attributes.


3.4  River Product Formatter

     The River Product Formatter (RiverPro) is an automated formatter for the following products: River
Statement (RVS), Flood Statement (FLS), and Flood Warning (FLW) (Office of Hydrology, 1996).  When
initiated, it compares observed and forecast river stage data with threshold stages, and tracks the history of
recently issued products, and then determines a recommended product  and the forecast points the product
should include.  The forecaster can accept these recommendations and generate the product using predefined
templates that control the product format and content.

     Alternatively, the forecaster can customize the product extensively - e.g., a different product can be
created with different forecast points included.  Also, the forecaster can select from the predefined templates
for each section of the product, and thereby control the precise wording and appearance of the product.  A
default set of predefined phrase templates is provided, and each office is able to modify or add to these
templates  to meet their local needs.

     In addition to providing the functions necessary to customize, generate, and edit a product, RiverPro
provides textual displays of information to support the forecaster in the decision-making process for product
issuance.  This information repeats some of the information available from the HydroView application which
may be running in a separate window; however, the information is presented from a product-generation
perspective.  The forecaster can view tabular summaries of the stage data and Form E-19 reference data for
stations, and can review information about previously issued products, including the product itself.  After the
product has been tailored as necessary, and reviewed, RiverPro can issue the product to  the appropriate data
dissemination circuits. In Version 2.0 of RiverPro, the entire user interface was enhanced to improve readability
and facilitate user control of the product generation process.

3.5  WSR-88D Stage II Processing

     The Stage II Processor is part of a three-step precipitation processing subsystem that is used to compute
hourly gridded precipitation accumulations  using rain gage data and radar data from the WSR-88D (Shedd
and Fulton, 1993).  The Stage II computed hourly precipitation accumulations consist of a rain gage-only
analysis grid and a gage-radar analysis grid that combines rain gage data with radar data.  The input radar
data is in the form of the Hourly Digital Precipitation (HDP) product, received from the Stage I processor
executing on the Radar Product Generator within the WSR-88D.

     The Stage II Processor and supporting software has been added to the WHFS suite of applications as a
background batch process.  Radar data and rain gage data for those gages under the radar umbrella are used
together in the analysis, which is performed under control of application control parameters. After the two grids
are created, an MAP processor forms MAP values from the two types of grids for river basins, forecast zones,
and counties, resulting in six types of hourly MAP amounts (from two types of grid analyses for three types
of areas).  Finally, other precipitation durations are derived from this data to create 3-, 6-, 12-, and 24-hour
accumulations of MAP for basins, zones, and counties.

     There are several data products  involved in the Stage II process that are viewable with the HydroView
data viewer discussed in Section 3.3.  HydroView allows the user to view the Stage I HDP radar grid, the Stage
II hourly gage-only analysis grid, the Stage II hourly gage-radar analysis grid, and the mean areal precipitation
values for basins, zones, and counties for durations of 1, 3, 6, 12, and 24 hours, based on the gage-only or
the gage-radar grids.

4.   WHFS FUTURE CAPABILITIES

     During the current development cycle of  the WHFS,  many field user requests were received for
enhancement to the existing capabilities discussed above.  Some examples are adding access to precipitation
data for formatted river products and thereby allowing additional product types to be generated, adding tabular
interfaces to the forecast data, and incorporating additional data quality control mechanisms in the overall data
system.  In addition, larger increments of new functionality are being incorporated as described below.

4.1  Area-Wide Flash Flood Prediction System

     A new increment of functionality referred to as the Area-Wide Flash Flood Prediction System (AWFFPS)
is being added to the WHFS set of applications.  Its purpose  is to assist the WFO forecaster in assessing the
threat of flash flooding by comparing precipitation data with FFG values for the same area and duration as
prepared by RFCs.  The AWFFPS functionality is being implemented as a series of coordinated functional
enhancements to the HydroView application, not as a stand-alone application.  The AWFFPS functionality
within HydroView will grow over time as new releases of WHFS are fielded.  The initial AWFFPS capability,
delivered with WHFS Version 2.1, is described below.

     The precipitation data used in comparisons are limited to observations - forecasted or projected
precipitation are not considered - and will come from three sources: precipitation point gage data, Stage II
gage-only analysis grids, and Stage II gage-radar analysis grids.  The FFG data used in comparisons are
generated by  the RFCs and can be for any of the three area entities: counties, zones, or basins.  If the
precipitation and FFG data are available, the comparison displays can be generated for 1-, 3-, 6-, 12-, and 24-
hour durations.  The comparisons are presented as either a difference value (i.e., precipitation - FFG) or as
a ratio (i.e., precipitation/FFG).  Displays are graphical with tabular summaries also provided.

     Future releases of the WHFS will enhance the AWFFPS functionality by adding features such as forecast
precipitation data, ingest of gridded FFG data from RFCs, and use of the WSR-88D Flash Flood Potential
algorithm, which projects the radar estimated precipitation up to one hour into the future using storm motion
characteristics.

4.2  Site-Specific Headwater Modeling System

     A future release of  the WHFS will provide hydrologic modeling capabilities at the WFO for headwater
basins.  This modeling system will generate time-series of forecast stage and discharge for small, fast-
response basins which will supplement the RFC forecasts for the RFC-modeled basin, or provide forecast
information for basins which are not currently modeled at the RFC or for when the RFC is not staffed.  Like
most hydrologic models, this model will use precipitation data and current river stage data as the primary
input, and using the initial soil moisture conditions and various geomorphologic properties of the basin, will
determine the runoff from the precipitation.  This runoff will then be converted to stream discharge using a unit
hydrograph transformation, and then the river stage will be determined from the stream rating curve.

     The Office of Hydrology is currently designing the Site-Specific Headwater Modeling System with a view
toward its implementation within the HydroView application of the WHFS.  This has included investigation of
the candidate rainfall-runoff models to use in the modeling system (Johnson et. al., 1998).  Because of the
hydrologic variability in the United States, it has been decided that the system must provide alternative models
in order to compute runoff under different regimes.  After much consideration, the runoff models initially
selected include the Sacramento Soil Moisture Accounting Model used in the NWS River Forecast System
(RFS), and a second model based on the National Resorce Conservation Service curve number methodology. 
The NWS RFS implementation for WFO headwater modeling has been developed, although it was not
operationally deployed (Glaudemans, 1997).

     Each model will have a precipitation pre-processor that determines the observed and forecast precipitation
for the basin; these values can be interactively modified to analyze different precipitation scenarios.  The user
can also adjust the model state initial parameters, or the parameter values can be accepted as is.  Interactive
review of the model output will allow the different models and the different precipitation scenarios to be
considered.  The models used for the system will be relatively simple, easy to parameterize and calibrate, and
must produce accurate forecasts of these fast-response streams, while using minimal computational
resources.  The forecast time-series generated by the model(s) would then be stored in the database and
subsequently used in public products formatted by RiverPro or some other formatter.

4.3  Dam Break Modeling

     A new large set of data is being added to the IHFS_DB database that includes static reference data and
forecast failure scenario data for dams across the United States.  The full catalog of dam information contains
in excess of 65,000 dams.  The static reference data for these dams comes from the "National Inventory of
Dams, Updated 1993-94", compiled by the United States Army Corps of Engineers and the Federal Emergency
Management Agency, and  includes nearly all dams in the United States that are higher than 7.62 meters (25
feet) or that impound more than 61,674 cubic meters (50 acre-feet) of water.

     The failure scenario data has been computed using methods, developed by the Office of Hydrology, which
extracted reference data from the catalog regarding the physical characteristics of the dam and reservoir.  This
data was used by a specialized version of the Office of Hydrology's Simplified Dam Break Model, which made
critical assumptions of particular data values and physical conditions to account for the data not available in
the catalog, but normally required by the model.  These assumptions decreased the accuracy of the flow
forecasts, but do provide a first-estimate of the failure scenario.

     The ability to search for and display information for dams in the catalog is being added to the HydroView
interactive application.  Editing capabilities are being added to HydroBase.  Future implementations will
integrate the full Simplified Dam Break Model so that when required input data are available, the more
representative results of this model can be used and  incorporated into the catalog.

4.4  System Monitoring

     A function, referred to as "HydroMon", is being added to provide a graphical interface from which the
system aspects of the WHFS can be monitored.  This function will provide user-friendly access to the many
log files that are generated by the collection of background programs that provide data decoding, posting, and
processing, in  support of the WHFS interactive application programs.  Examples are log files from the
Standard Hydrometeorological Exchange Format (SHEF) product decoder, the HDP product decoder, the
Stage II Processor, the mean areal precipitation computation process, and the precipitation accumulation
processes.

     HydroMon will not only provide convenient access to log information previously available only through
operating system file access commands, but will provide value-added information by including displays that
aid the user in determining the status of all data flows into and out of WHFS.  HydroMon will generate periodic
and on-demand HTML Web pages to be viewed using the Netscape Navigator commercial Web browser.  This
approach to status monitoring is already being used by the other components within AWIPS, with which the
HydroMon function will be integrated.

5.   WHFS FIELD DEPLOYMENT

     The WHFS has been deployed and in operational use for more than three years at offices throughout the
country.  After the initial development effort, the WHFS was installed at the NWSFO in Norman, OK in the fall
of 1994.  The WHFS was subsequently deployed to six additional offices as beta-test sites between the fall
of 1995 and the spring of 1997 in an effort to provide exposure to different climatic regimes and obtain greater
forecaster involvement.  Its use by the operational forecasters has yielded invaluable feedback that is being
used to identify enhancements needed in the system.

     Beginning in August 1996, the WHFS was deployed as part of the standard AWIPS delivery.  This initial
AWIPS deployment consisted of seven WFOs, none of which had been beta sites.  These AWIPS offices have
been active in using the system and providing feedback regarding its utility.  They have provided a unique
perspective in that the WHFS applications are running as part of the larger, formal AWIPS environment.  The
Office of Hydrology maintains a database of user-requested enhancements that are continually evaluated and
prioritized for development.

     Another AWIPS deployment sequence running from December 1997 through March 1998 is resulting in
an additional 11 WFOs using the WHFS, including two of the beta sites.  The status of the WHFS deployment,
as projected through March 1998, is shown in Figure 2.  In addition to the WFOs, the WHFS is being deployed
to the NWS Training Center, 4 of the 6 NWS Regional Headquarters Offices, and 8 of the 13 RFCs.


                 Figure 2. NWS Field Deployment of the WHFS Through March 1988

6.   SUMMARY

     The WHFS system provides the NWS hydrologic program with an invaluable set of tools for performing
its mission.  Using its relational database as the repository for all data sets, the applications provide data
management functions for control of the entire system, data analysis and display functions using a geographic-
based display, and an automated product formatting capability for issuance of public hydrologic products. 
Supporting these interactive applications are a collection of data decoding, data posting, and data processing
functions, including the Stage II Processor which generates gridded precipitation estimates using radar
estimates and gage reports.

     The recently delivered Version 2.0 of the WHFS has provided significant improvements to existing
functions and  incorporated new applications, in addition to using the IHFS_DB database, which establishes
a robust database structure upon which to develop new functionality.  Included in the future enhancements
are flash flood monitoring capabilities, dam-break catalog and modeling capabilities, and headwater modeling
capabilities.  As the WHFS continues to grow and be improved, based heavily on the feedback from
operational deployments of the existing system, it will provide the NWS hydrologic program the functions
required to modernize and improve services.

7.   REFERENCES

Chen, Peter, 1977:  The Entity-Relationship Approach to Logical Database Design.  Q E D Publishing Co.

Glaudemans, M.J., 1997:  A Local Headwater Model For Operational Use in the Modernized National Weather
     Service, 13th Conference on Hydrology, Amer. Meteor. Soc., Long Beach, California.

Johnson, D., E. Welles, and M. Smith, 1998:  Site Specific Modeling for National Weather Service Forecast
     Offices, Special Symposium on Hydrology, Phoenix, AZ, Amer. Meteor. Soc., Jan 11-16, 1998.

Office Of Hydrology, 1996:  RiverPro Reference Manual, April 1996.

Office of Hydrology, 1997:  IHFS_DB Version 1.0 Database Design Model (maintained in CASE tool).  National
     Weather Service Office of Hydrology, August 1997.

NOAA, 1992: AWIPS System Requirements Specifications.

Roe, J., G. Bonnin, M. Glaudemans, C. Gobs, and P. Tilles, 1998:  Recent Database Developments at the
     National Weather Service Office of Hydrology.  Special Symposium on Hydrology, Phoenix, AZ, Amer.
     Meteor. Soc., Jan 11-16, 1998.

Shedd, R.C., and R.A. Fulton, 1993:  WSR-88D Precipitation Processing and its use in National Weather
     Service Hydrologic Forecasting.  Proc. of the International Symposium on Engineering Hydrology, San
     Francisco, CA, ASCE, July 25-29, 1993

Shelton, D. R. and Edwin L. May, 1996:  Modernized Hydrologic Forecast Operations at National Weather
     Service Forecast Offices.  12th International Conference on Interactive Information and Processing
     Systems for Meteorology, Oceanography, and Hydrology, Atlanta, GA, Amer. Meteor. Soc., Jan 28-Feb
     2, 1996, pp 359-364.

As required by 17 U.S.C. 403, third parties producing works consisting predominantly of the material appearing in NWS Web pages must provide notice with such subsequently produced work(s) identifying such incorporated material and stating that such material is not subject to copyright protection.

Return to HRL Publications