AWIPS Build OB9.0 Release Notes

NWS/Office of Hydrologic Development (OHD) - Hydrologic Software Engineering Branch

November  21, 2008

The OHD Common software is delivered to both WFOs and RFCs.  It includes the:

         WHFS (WFO Hydrologic Forecast System) components

         Precipitation Processing System (e.g. MPE)  components

         Data Ingest (e.g. SHEF_decode) components

The release notes and related information are available at: http://ocwws.weather.gov/intranet/whfs/

A. Hydrologic IHFS Database

1.  Converted the IHFS Database from Version OB8.3 to Version OB9.   The DCP and Telem tables have been modified to accommodate a new field.  Other modified tables include the OfficeNotes table and the tables which store Forecast Service information.   Added the Velocity table for tracking storm motion and the VerifRespType table as a lookup table for verification response types.  Please refer to the database changes web page for details on specific table changes.

B. Precipitation Processing Operations 

B.1  MPE_Fieldgen

1. Modified satellite data grid analysis to smooth grids at the boundary between the local-bias corrected radar field and the local-bias corrected satellite field.  The Satellite-Radar grid and Satellite-Radar-Gage grid are effected by this change.

B.2 MPE Editor General

1. New Gage Table:  Implemented a new version of the Gage Table, which for each of the precipitation gages, lists the gage value and their values for the gage location as estimated by all the various gridded and multi-sensor fields.  Previously, the table presented an incomplete set of grids.  Now it contains all the values except for the raw satellite estimate value, although the satellite-radar and satellite-radar-gage value are presented.  The gage table initially places the columns in the same order as the items shown in the MPE Editor Precip Fields menu.  The rows of the Gage Table are initially sorted by the location id; rows can be sorted based on the value in a column by clicking on the column header. Both the column placement and the row value order can be interactively adjusted.

2. Modified Multi-hour Accumulation Display Features to add an option to choose the field.  Previously, the field accumulated was always the Best Estimate QPE field. 

3. Modified the Display7x7 window to contain an option button which allows the specific MPE product displayed in the window to be selected.

4.  Fixed bug in which edited precipitation values were saved as rounded integers when re-running the MPE Field Generator.   The edited value is now saved properly to the RawPP table.

B.3  MPE/DailyQC

1.  Fixed problem where the application would shut down when requesting display of freezing level if there were too few freezing level datasets available.    This also occurred for display of some temperature data sets.

2. Fixed problem where the Edit Precipitation Stations, Edit Temperature Stations, and Edit Freezing Stations windows each would creep a few pixels to the southeast when a new station is selected for editing.  Eventually, the windows reached the corner of the screen and the forecaster had to reposition them.   Also added logic so these windows close after editing a station, and when another station is selected for editing, the edit window always reappears in the same place on the screen.

3. Modified behavior regarding estimated gage values.  Previously, when a user set a station to bad using the Edit Precipitation Station window or the Edit Temperature Station window, an estimated gage value is shown.  Then, when the precipitation grids were rendered, the station's value is estimated to something different.  The station value was getting re-estimated once missing stations were assigned estimated values.  This was changed to always use the estimated value shown in the Edit Precipitation or Edit Temperature window.

4. Changed behavior of application so that when displaying the topographic image, it will display the topographic image below all other data sets and overlays.

B.4 Precipitation PC Derivations

1. Implemented multiple changes in how PC (i.e. cumulative counter) precipitation data is derived for given durations in order to adapt to characteristics of ALERT station reports.  These changes affect RiverPro, HydroView, and RiverMonitor operations which relate to the derivation of PC-based precipitation data. The changes are:
a) Longer periods of raw reports are retrieved to account for possibly longer periods of missing or unreported data, as defined by the token adjust_PC_startingtime, which defaults to 4 hours. 
b) If the station telemetry type is defined as “ALERT”, it now assumes data unreported for up to a specified number of hours as representing zero rainfall.  This period is also defined by the adjustable token adjust_PC_startingtime.  Furthermore, reported non-zero rainfall is assumed to occur at the time of the report rather than being evenly distributed starting at the time of the previous report.  This better accommodates the ALERT method for reporting bucket tip increments of precipitation.  Lastly, for any gages, the two PC values can now be outside the period of interest by a certain number of hours, and a derived value can still be determined.  It no longer insists that both PC values be within the desired period

B.5 Hi-Resolution Precipitation Nowcaster (HPN)

1. HPN introduced into the High-resolution Precipitation Estimator (HPE).  In OB8.3, HPE provided high resolution (~1km x 1km) mosaics of rain rate and 1-hour accumulation for multiple radars updated approximately every 5 minutes.  In OB9, the High-resolution Precipitation Nowcaster (HPN) routine adds forecasts of instantaneous rain rate at four 15 minute intervals (of 15, 30, 45, and 60 minutes) in the future and a forecast of 1-hour accumulated precipitation, without/with bias. 

The locally-generated HPN forecast grids are derived from the HPE and are generated by using a primarily advection algorithm.  Specifically, to compute the HPN grids, the most recent gridded HPE rain rate field is compared with the one about 15 minutes earlier to estimate the local motion of areas of echoes on a 20 km grid scale using a standard local pattern-matching scheme.  The gridded rain rates are then projected forward in time using the most-recent estimated gridded motion vectors at a time step small enough to prevent pixel jumping (aliasing) of the echoes.  The algorithm then computes smoothed forecasted rain rate fields.  This approach uses a variable, adaptable rate of smoothing with time that is computed based on the current observed rain rate fields as well as observed changes in echo structure over the past 15 minutes.

More information on this function is available in the AWIPS Operational Build 9 Release Notes. The primary purpose of HPE and HPN are for use in FFMP-A; to configure FFMP-A to use HPE and HPN, see the “How to Add HPE/BiasHPE” document found on the FFMP page at http://www.weather.gov/mdl/ffmp

C. RiverPro Product Formatter

1. New CAP tags in Products:  Adjusted product generation process by inserting Common Alert Protocol (CAP) “tags” in watch/warning/advisory products to identify the “call-to-action” (CTA) section of the product.  These tags will facilitate later processing of the following products generated by RiverPro: Flood Warning (FLW), Flood Statement (FLS), Point-specific Flood Watch (FFA), Point-specific Flood Advisory (FLS).  The CAP tags were added on new lines before and after the call-to-action section with the following text, respectively: “PRECAUTIONARY/PREPAREDNESS ACTIONS” and “&&”.

2. Arithmetic Operations on Template Variables: Added new feature to templates to allow basic arithmetic operations to be performed on numeric data retrieved from database.  Users can construct addition, subtraction, multiplication, and division operations.  These operations are performed on either a set of variables or a combination of variables and fixed values.  The feature is available for both regular variables and PE variables, assuming that the returned value is either a numeric or time value.  For example, the variables can be given as <ObsStg,+3.0>, <ObsStg,*-4>, <ObsStg,MET,/5.0>, <BLUO2,PC,1001,RG,Z,NOW,MATH+5.0>.  The arithmetic operations can be used anywhere in a template where traditional variables are supported, such as after keywords: “phrasestr:”, “varlist:”, “condition:”, “bulletstr:”, “indentstr:”.

3. Precipitation derivations for PC data usage in RiverPro PE variables are enhanced, as described in the Precipitation PC Derivations section above.

4. Added “NOW” ending time specifier to template PE value to complement the existing “LATEST” specifier.  LATEST implies the top of the current hour, while the new NOW specifier implies the current time.  This is useful for precise specification of precipitation duration periods.

5. Adjusted latitude, longitude variables (<LocLat>, <LocLon>) so that they use the coordinates from the Location table, rather than the RiverStat table.

6. Changed threshold for TRACE_THRESHOLD from 0.00999 to 0.0011 which can result in “T” being displayed for the “PE” variable values.

7. Corrected bug with the way the VTEC timeseries graph mishandled edits involving “missing” or undefined times.

8. Increased size allocated for the user time (i.e. T_Uxxx) format in RiverPro templates from 20 to 40 characters.

9. Fixed bug for the VTEC rise/fall/crest template variables, where it did not assign the value for them when they are missing. The bug caused any associated template condition check to fail.

D. Hydro Time Series Display 

1.  Modified color for flood categories in TimeSeries and TimeSeriesLite applications. These applications now use magenta-red-orange-yellow for major-moderate-flood/minor-action stages, respectively, as per the color usage in  AHPS graphics, SiteSpecific, and the RiverPro VTEC graph.

E. HydroView Data Viewer

1. Precipitation derivations for PC data are enhanced, as described in the Precipitation PC Derivations section above.  This affects display of precipitation data via the point display control feature of HydroView.


F. Site Specific Hydrologic Predictor

1. New Variational Data Assimilation Function: Added Variational Data Assimilation (VAR) for use with the Sacramento Soil Moisture Accounting Model (SAC-SMA) rainfall-runoff model.  This new function does not affect use of the Antecedent Precipitation Index (API) rainfall-runoff model. 

VAR uses estimated precipitation and observed stream measurements to alter the state variables and prior-computed runoff inputs to the SAC-SMA rainfall-runoff model such that the forecast created more closely matches the latest streamflow observations.   State variables represent the current state of the soil in the basin, while prior-computed runoff represents the water that has been added to the channel within the basin, but has not yet flowed out of the basin.  The VAR enhancement includes changes to the Analysis Window and a background preprocessor.  For details, please refer to the Site Specific Hydrologic Predictor (SSHP) Reference Guide.

2.  Added SiteSpecific interface to QPF data generated by the HPN process.  The HPN (Hi-Resolution Precipitation Nowcaster) is new to OB9 and provides one hour basin-averaged QPF values for use in SiteSpecific.

3. Save Crest Info: Added ability to Insert new record into the IngestFilter table when writing a crest extremum value for the first time for a location.  This is needed so that if a crest value is saved, the TimeSeries application will recognize the presence of the generated forecast data.   The saving of the crest value is controlled by the token: sshp_should_save_fcst_crest, which has values of true and false and defaults to true. The feature is enabled/disabled for all forecast points – i.e. it can not be turned on for some and off for others.

4. Updated the Data Transfer operation to disallow 6-hour runoff time series.

G. RiverMon Monitoring Tool

1.  Modified behavior regarding how the user selects which stations to monitor, using the select/filter controls on the left margin of the window.  This includes the following changes:
a) Fixed bug in which locations not in a designated RiverMonGroup groups are not displayed.
b) Adjusted so that groups or HSAs with no locations are not displayed in the left margin.

c) Removed function which reset the HSA of a group based on the HSA of the first location in the group, which overrode whatever the user had decided was the preferred HSA for that group.
d) Fixed bug in which the location’s HSA in the displayed column was incorrectly shown as its group's hsa, not the HSA explicitly defined for the location.


2. Modified the RiverMonLocation window so that it sorts the available location identifiers.


3. Precipitation derivations for PC data are enhanced, as described in the Precipitation PC Derivations section above.  This affects the precipitation monitoring mode of HydroView.

H. HydroBase Data Manager

1. Modified Editing Capabilities Of Flood Reports (DR #18989) – Raytheon modified the Hydrobase Flood Report display of separate flood events, where each event contains individual observations of river stage/discharge.  a) Added ability for users to delete individual observations within a given flood event; currently users can only delete flood events.  b)  Added ability for users to delete individual observations within a flood event and have the application re-compute rise above flood stage time, flood crest time, and fall below flood stage time based on the deletion of individual observations. 

2. Added Information To Hydro Database On Observation Frequencies (DR #19033) – Raytheon added ability to store and manage information regarding how frequently observed data is reported.  This includes information on a) the observation frequency of observed data elements to the IHFS database; b) whether the location receives random reports in addition to the defined observation frequency.

I. Utility Programs

A. HydroGen Web Data Extraction

1. Modified data extraction method for the gage zero, or zero datum, value from the zd field of the rivertstat IHFS database table in the IHFS database so that it no longer rounds the value to the nearest tenth of a foot.

B. Alert/Alarm Report Generation

1. Report Format Trimming and Control: The Alert/Alarm report generator was modified to present a more concise, user-readable format.  The default mode is the summary mode or the user can specify the verbose mode (“-v”).  With the verbose option, the thresholds are also listed with each station data set and a descriptive block of information is given at the end of the report.  To enable the option, add the “-v” option in the script /awips/hydroapps/whfs/bin/run_report_alarm script.  In the section “#run the application”,  add “-v” as in:

$WHFS_BIN_DIR/report_alarm$OS_SUFFIX -d$DB_NAME -p$PRODUCT_ID -s$SUFFIX            –rNEAREST -v >> $LOGFILE 2>&1

A sample non-verbose report is:


CREATED CCCACRXXX AT:  Thu Jul 10 14:48:22 2008

Obs River Stage    25.0 at Thu-Jul10-13:30 Chg=    9.1/hr > alarm 5.0/hr
Obs River Stage    15.9 at Thu-Jul10-13:30 > upper alert 15.0
Obs River Stage    15.9 at Thu-Jul10-13:00 > upper alert 15.0

Obs Water Temperature    83.1 at Thu-Jul10-14:00 > upper alert 80.0
Obs Water Temperature    83.3 at Thu-Jul10-13:00 > upper alert 80.0

C. SHEF Decoder

1.  Corrected Posting Of Probability Values (DR #19454) - Raytheon fixed two problems with posting the SHEF alphanumeric probability character into the database as a floating point number.  a) some probability values were posted with a rounding error; e.g. “A” posted as 0.001999 instead of 0.002.  b) some probability values were posted incorrectly; e.g. “7” posted is 0.500781 instead of 0.7.

D. Metar-to-SHEF Translator

1.  Fixed Handling Of  “No Special” Argument (DR #18525) - Raytheon fixed problem with usage of the -nospeci argument in the metar2shef_option token..  This argument instructs the translator not to process those MTRs with the SPECI notation at the beginning of the report but resulted in no METAR data being processed. 

2. Fixed Handling Of Reports With TORNADO Remark (DR #19347) -  Raytheon fixed problem where the translator was failing when it encountered a MTR report which contained remarks which include the word TORNADO. 

E. Rate-of-Change Checker

1. The roc_checker program now references a token (qc_questorbad) which indicates whether data which fails the ROC QC check should be labeled as bad (token='B') or questionable (token='Q').  The default value is 'B'.  Failed data are still left in their original table (e.g. Height) but are labeled as Bad, so they will not be used by applications such as Riverpro.  The failed data are left in their original table;  they are not moved to the RejectedData table.  This change does not affect the checks for a value against rate-of-change alert/alarm levels.

F. Forecast Services Station Management

 (This software is discussed in these OHD-Common release notes even though it is for RFCs only.)

1. Adjusted the user interface for accessing logistical verification forecast services information in the following ways:
a)  When deleting an entry, the confirmation window pop-up is displayed but the second acknowledgement pop-up is now displayed only if deletion was unsuccessful.
b)  When changing the Water Supply, Deterministic, or ESP data, if the user changes a database key field, an update occurs instead of creating a new entry and forcing the user to delete the old entry.
c)  Reversed order of messages in the database message box so new messages are now listed at the top.
d) Dashes displayed for NULL values no longer cause problems when updating records.
e) Added a Refresh button to the main window for use after updates to the location or river gage information.

2. Updated database tables as follows:
a) Added an "Other" option for all lookup tables.
b) Added a "Verification Response Type" and “Drainage Area” column to the FcstptServices table.
c) Added a "VAR Usage" column to the FcstptDeterm and FcstptESP tables.
d) Added lookup table for eligible response types:  VerifRespType.

J. System Features

1. Various applications, including SHEFdecoder, RiverPro, and HydroView were modified to remove their usage of the CURRENT_TIMESTAMP SQL keyword to avoid possible problems with timezone complications.