A Verification System for Probabilistic Hydrograph Forecasts

Many water resource managers depend upon National Weather Service (NWS) long- lead river flow forecasts to make planning decisions. These forecasts, ranging from a week to several months into the future, are generated using the Extended Streamflow Prediction (ESP) technique. As a part of the Water Resources Forecasting System (WARFS) initiative, the NWS is enhancing these ESP forecasts.

The ESP Forecasting Technique

The ESP system produces probabilistic forecasts through a statistical analysis of an ensemble of model generated possible future runoff time series. These possible runoff time series are computed using historically observed precipitation and temperature time series as input to hydrologic models which are initialized with the current states of the hydrologic system. It is assumed that the input time series for ESP forecasts are the result of stochastic processes, with previous time periods being realizations of those processes.

Verification of ESP Products

One of the enhancements to ESP forecasts that WARFS will provide is a forecast verification system. The results of the statistical analysis performed for the forecast verifications will be provided as an ESP forecast product. These verification statistics will assist users with the difficult task of integrating multiple data sources into a single decision. Forecast verification data will lead to more effective resource management and more effective natural disaster mitigation.

Verification System

The first step in the verification process is to compute forecasts using historical data. Because the input processes are assumed stochastic, the years which occur after a year of interest may be considered possible input time series for that earlier year.

The ESP verification system will access the parametric information that is used for ESP forecasts in the NWSRFS. With the verification information forecasters will be able to compare the quality of ESP forecasts at different forecast points across their forecast area, and focus improvement efforts, such as model recalibrations, in those areas which appear to be the weakest. Results of the verification will be displayed via the ESP Analysis and Display Program (ESPADP), another software development effort under way as a WARFS activity. ESPADP will read the traces and apply various verification tests that users will select interactively. Verification results will then be written back to the NWSRFS parametric data bases for display when ESP forecasts are made. Figure 1 is a simple schematic description of the system.

Figure 1.

Verification Tests

Probabilistic forecasts cannot be tested in the same manner as deterministic forecast because there is no single forecast value that can be compared to the observations. Various verification tests have been recommended by different users. (Day, 1992). Some ideas for improving verification include:

• Test whether or not the moments of conditional distribution are statistically different than marginal ones, and if so, how much different

• Express forecast versus observed results in terms of correlation (R square), Root Mean Square Error, average bias, maximum bias, information transfer, and other

• Compute the distribution of quantiles where the observed flow falls on conditional distribution

• Test whether the elements of empirical cumulative distribution function of historical streamflows are uniformly distributed (Kolmogorov-Smirnov and 2 tests)

Summary

An enhanced set of statistical tests for ESP accuracy will be incorporated into the NWSRFS. Through a graphical interface within ESPADP, the tests will become easy-to- use and available in the every-day practice of the River Forecast Centers. The results of the verification system will contribute to efforts to reduce the damage caused by natural disasters, in particular floods and droughts.

References

Day, G.N., Brazil, L., McCarthy C.S., and Laurine, D.P., Verification of the National Weather Service Extended Streamflow Prediction Procedure, 28th Conference and Symposium on Managing Water Resources During Global Change, Reno, Nevada, November 1992.