OHD-Common
AWIPS Build OB8.2 Release Notes
NWS/Office
of Hydrologic Development (OHD) - Hydrologic Software Engineering Branch
October 12, 2007
The OHD Common software is delivered to both WFOs and
RFCs. It
includes the:
·
WHFS
(WFO Hydrologic Forecast System) components
·
Precipitation
Processing System (e.g. MPE) components
·
Data
Ingest (e.g. SHEF_decode) components
The release notes are available at: https://ocwws.weather.gov/intranet/whfs/
A. Precipitation Processing Operations
A.1 MPE_Fieldgen
1. Added generation of three new gridded fields making use of satellite based QPE:
-- SRMOSAIC – satellite radar mosaic
-- SGMOSAIC – satellite gage mosaic
-- SRGMOSAIC – satellite radar gage mosaic
To include these in the list
of automatically generated products, the .Apps_defaults_site
token “mpe_generate_list” must be
adjusted. If the office wishes one of
these three new products to be their “best estimate” product by
default, the token “mpe_qpe_fieldtype” needs to be reset. For these token changes, use the upper case
name for the product(s).
2. Added operations to transmit the
precipitation mean field bias data currently generated in MPE Fieldgen to WFOs and on to
ORPG. This transfer of this bias
requires a sequence of steps and I intended to be used at RFCs
only. The bias values are sent from the
RFC, used at the WFO, and passed onto the ORPG. The bias transfer operations are mentioned
in Section C.
3. Added generation of two new gridded fields based on the RFC–provided mean field
precipitation bias value. This feature
is intended for WFO usage; RFCs already account for
their bias information internally. The
information is used to create these two new grids:
-- RFCBMOSAIC - RFC Mean Field Bias Corrected Radar Mosaic
-- RFCMMOSAIC - RFC Mean Field Bias Corrected Multi-sensor Mosaic
At a WFO, a new field in the database defines which RFC provides the bias
values for these fields; this is discussed in Section D.2.
4. Fixed a problem with edit polygons being applied incorrectly to base
fields.
5. Improved the method for saving the Best QPE grid
in NetCDF format.
Now it uses the coord_xxx.dat file to
determine the lat/lon coords
of the corners of the MPE area instead of tokens which may be different that
the coordinate file. Also,
fixed a potential problem in the swapping of the rows and columns of the QPE
product.
6. Removed the call to the build_hourly
program from the rerun_mpe_fieldgen script. This improves re-run performance. Build_hourly rarely needs to be done repetitiously for each
rerun.
7. Fixed
problem which caused the LMOSAIC and LOC_BIAS fields to be set to missing when mpe_fieldgen is rerun.
A.2 MPE Editor (General Functions)
1. Added three new satellite fields to the
sub-menu list shown for the “PrecipFields”
option and the “Edit Precipitation” window under the “Draw
Polygons” option. The new fields
are labeled as: “Satellite Gage Mosaic”, “Satellite Radar
Mosaic”, “Satellite Gage Radar
Mosaic”.
2 Provided new color selection interface to
manage colors shown in grids. Essentially the same dialog is used in HydroView and MPE_editor. For MPE_editor,
this is accessed from the Tools menu, by selecting MPE Color Manager.
3. The Transfer RFC Bias
option has been added to the MPEcontrol menu in MPE Editor. This function is intended for the RFCs. It allows the
user to control the transmittal of the bias information to the WFO. This menu item is selectable only if the mpe_transmit_bias token is set to ON. Selecting this menu option causes mean field
bias information for the hour to be sent across the WAN. The bias transfer operations are described
further below.
4. Fixed a bug in the grid_to_area.LX application, which produces the files that
MPE Editor uses to determine the county and basin located under the mouse
pointer.
5. Moved the Clear data
button to the more prominent location, in the MPEControl
Menu. It was previously located under
the Tools menu.
6.
Changed PRISM display options under the Climo menu to
allow PRISM data to be displayed and undisplayed
similar to an overlay, using a toggle button.
7. Fixed
problems causing the "Save Best Estimate" buttons to not be
selectable.
8.
Changed the edit precipitation window to prevent it from being resized too
small, which caused some of the option buttons to not be visible.
9. Fixed
problem that occurred when polygons are applied to a base product, causing them
to appear in the wrong place on the derived products.
10. Fixed
problem in the Edit Precipitation window which was causing the "Snow"
button to be partially cut-off.
11. Fixed
problem where the bias table could be resized too small, "cuting off" some of the right-most text fields.
12. Fixed
problem that occurred when in manual time lapse mode in which the date on the
displayed MPE product did not update when manually stepping forward or backward
through time.
13. Fixed
problem where the county and basin information was not displaying in the MPE
Info Legend when moving the mouse across the MPE Editor Map.
A.3 MPE Editor (DailyQC functions)
1. Added
topography data for
2.
Added flexibility to define the temperature and freezing level QC periods. The
CBRFC uses a period of 6Z to 6Z. The NWRFC QCs data later and would prefer a
12Z to 12Z period.-- Added the token dqc_preprocess_basetime
to allow user to select the 6hr starting period; if the token is set as 18Z,
then run_dqc_preproessor will create
temperature_1_xxx_point data starting from 18Z.
When changing the token, files need to be removed from the temperature
and freezing level point directories to prevent mixed datasets.
3. Added
feature to allow the max and min gridded temperature
legends to be different from the 6hr temperature legend.
4. Added a mpe_dqc_num_days token to
control number of days to process. The NWRFC usually processes only one day.
Currently, MPE is fixed at 10.. The default value for this token is 10 days.
5.
Added tokens allowing the user to control the following operations:
a) The
number of surrounding stations used to estimate precip
and temperature values (currently hard-coded as 30 for precip
estimates and 20 for temperature estimates):
mpe_dqc_max_precip_neighbors : 30
mpe_dqc_max_temp_neighbors : 20
b)
Deviations:
mpe_dqc_precip_deviation: 3.0
mpe_dqc_temperature_deviation: 10.0
c) Number
of good stations to find before jumping out of estimate loop;
mpe_dqc_min_good_stations : 5
d) Default
number of days shown in the Choose Data Period window;
mpe_dqc_num_days : 10
e) Copying Level2
data files to directory monitored by IHFS db shefdecoder;
mpe_copy_level2_dqc_to_ihfs_shef : OFF
f) Copying
Level2 data files to directory monitored by the archive db shefdecoder.
mpe_copy_level2_dqc_to_archive_shef : OFF
6. Fixed
problem where the colors for the grid legend did not agree with the point
values at the lower ranges. This was
related to problem in which temperature PRISM data did not display correctly.
7.
Corrected precipitation time consistency function. Some stations which appeared
in the inconsistency check mode also appeared in the consistency check mode.
8. Fixed
problem where in temperature mode, the button in the point display when set to
partial displays nonsense data.
9.
Fixed a problem where a change to a freezing level station prevented the 00Z
and 06Z freezing level grids. Once a manual edit was made, the 00Z and 06Z
grids could not be rendered. To be consistent with the precip and temp functions, if the user edits a time period,
all four time periods are changed back to "Point" and the
"Render Grids+MAZs" button is sensitized.
None of the four freezing level grids will display until they are rerendered by the user.
10.
Corrected problem of allowing date to advance so partial period can be QCd. During events
there is a need to be able to QC the most current 6 hour period of data. Previously one could not advance the date to
gain access to the current 6 hour data. Modified the
preprocessor so that if the current Zulu time is past 12:00Z, then generate
currentday+1 precipitation and temperature.
11. Fixed
the DailyQC precipitation, temperature, and freezing
data colormaps so that their definitions are not hardcoded. This was
changed to use the ColorValue table management method
like the rest of the MPE products. This
allows the user to dynamically change them from the Color Manager under the MPE
Editor Tools menu.
12. Fixed a
problem in which there appeared a dialog box stating that the freezing level
data were not edited, even if the freezing level data were edited. If the box
was closed and the user tried to save the changes, the program terminated. Modified behavior so the warning dialogs
always appear when closing MPE Editor in the instance where DailyQC
data have not been QC'd or saved to Level2.
13. Fixed
problem where in some cases, no data are being output from MPE. The level 2
data were not being written out.
14. Fixed a
problem which occurred after rendering grids or mean area fields
(precipitation/temperature/freezing level), in which manually editing one 6
hour time period affected the other three 6 hour time periods in the day. Now,
manually editing data on one 6 hour time frame does not affect the data on the
other three time frames.
15. Changed
behavior so that when clicking "render grids+MATs"
(or MAZs or MAPs), the
cursor is changed to a WATCH shape
A.4 Purge MPE Files
1. Adjusted syntax of purge commands to prevent
error messages from being generated. Also, adjusted
so that it does not delete directories if it is empty.
B. RFC Radar Precipitation Bias Transfer Operations
RFCs can now transmit their MPE-generated mean field bias
information for each of the radars in their forecast areas to co-located WFOs. The WFOs have the option to
apply these RFC mean field biases to the ORPG and MPE generated precipitation
fields. By doing this, WFOs are benefiting from the extensive QC performed by the
RFC HAS staff. Whether or not the RFC
mean field biases are used is determined on a radar by radar basis in Hydrobase (see Section D.2).
Instructions
for setting up the RFC Bias Transfer Process are provided in the OB82 AWIPS
Installation Notes.
1. New tokens control this process and can be edited in the Apps_defaults_site
file. The default values are defined for
a WFO configuration and should not be changed at a WFO.
a) mpe_transmit_bias (ON/OFF) –
If set to ON, RFC mean field bias information will be sent across the WAN to
the WFOs specified in the transmit_rfc_bias
script. If set to OFF, then RFC mean field bias information will not be
sent. By default this token is set to
OFF.
b) transmit_bias_on_rerun
(YES/NO) – If set to YES, RFC mean field bias information will be sent
whenever MPE Fieldgen is manually rerun from MPE
Editor. If this token is set to NO, then
information is not sent on a rerun. By
default this token is set to NO.
c) transmit_bias_on_save
(YES/NO) – If set to YES, RFC mean field bias information will be sent
whenever the Best Estimate QPE is manually saved from MPE Editor. If this token is set to NO, information is
not sent on a save. By default this
token is set to NO.
d) send_local_bias_when_rfc_bias_missing
(YES/NO) – This token is used by the bias message generation operation (Biasmesgen) which sends the bias information from a WFO to
the ORPG. When the token is set to YES,
it will use WFO-generated mean field bias values if RFC mean field biases are
not available. If this token is to NO, Biasmesgen will not send bias information to a
radar’s ORPG if the RFC bias information was not available. By default this token is set to NO.
B.2. Bias
Message Generation
This
application generaties the actual bias message sent
from a WFO to the ORPG.
1.
Modified to read the IHFS RadarLoc table to determine
from which office (the WFO or an RFC) to retrieve mean field bias information
for use in the bias message.
2. Set
up to use the send_local_bias_when_rfc_bias_missing
token to determine what to do if an RFC’s mean
field bias is not available for the current hour or the past hour. If this token is set to YES, then the WFO generated
mean field bias from the current hour or the past hour will be sent in place of
the missing RFC bias. If this token is
set to NO, then bias information for the radar will not be included in the bias
message.
3.
Updated the BiasMesgen logs to reveal the source of
the mean field bias chosen for each of the radars for which the WFO is
responsible.
C. HydroBase Database Manager
1. Added functionality to copy the set of
database tables used for the OB81 Forecast Service operations when copying information
for one location to another. Previously
these tables were not considered for the copy operation.
2. Updated the Hydrobase/Setup/Radar
Locations window to add a field for the office_id.
The BiasMesgen application uses this field when
creating the bias message to determine the source of the mean field bias
information. By default, this field is initialized to the local office
id. WFOs need
to enter the office_id of the RFC from which they'd
like to get mean field bias information for a given radar. RFCs don't need to change anything.
D. HydroView
1. Provided new color selection dialog to manage
colors for point data and for gridded overlays.
Essentially the same dialog is used in HydroView and MPE_editor. In HydroView, there Tools menu provides access to the HydroView Color Manager option. The MPE Color Manager option provides color
management for the Best Estimate QPE available for display in HydroView. The
legacy color manager is still available in HydroView,
but should not be needed, and it is labeled as deprecated.
2. In the Live Data point data display, fixed bug in which 6 hour precip accumulation values occasionally appeared in the
wrong period.
E. Site-Specific Hydrologic Prediction Headwater Model
1.
Added ability to save QPE and QPF defined interactively
within SSHP, and to load those in for later runs of the model. The file system is used for this; these saved
values are not stored in the database.
2.
Added ability to select an alternate unit hydrograph per
event. These multiple unit hydrographs
are stored in the database using the HydroBase
application, and are uniquely identified by the value for the
“model” field for the given location and upstream basin.
3.
Added ability for SSHP to generate “file-only”
forecasts in the background for monitoring use by the RiverMonitor
application. These forecasts are not
stored in the database; they are generated for the sole purpose of allowing
automated model runs to be monitored automatically in RiverMonitor. The files are generated by the run_SSHP_SAC_state_update script, which might not currently
be run at each location. For best
results, it should be run hourly from the cron. The resulting files are written to the /awips/hydroapps/whfs/local/data/sshp/forecast
directory. To monitor these in RiverMonitor, include the columns that use data extracted
from the file-only time series for the modeled locations. The columns include basistime
of crest, validtime of crest, crest value, and time
above flood stage.
4.
Added feature to allow for the choice of 1/3/6 hour FFG
values when using the API-MKC Rainfall-Runoff Model.
5.
Added display of hourly precipitation text amounts on
graphical MAP display, with display controlled by a toggle button. Also
added display of 6-hour and 24-hour totals on the main graph every 6 hours.
6.
Added ability to use up/right and down/left arrow keys to change the
precipitation amounts to get precise amounts on the graph.
7.
Added ability to see prior issued forecast. It is colored magenta, while the current
forecast is green.
8.
Modified to prevent the editing of observed precipitation for
periods before the model run start time, since it is confusing to allow edits
of something which has no effect on the model computations. If the user must edit precipitation at that
time, the model run start time can be moved back in time far enough to
accommodate this.
9.
Modified to allow double click on Control Window’s
table to invoke the Analysis Window for a particular location.
10.
Added ability to see
moderate and major flood line displayed. Previously, only Action Stage and
Flood Stage are displayed now.
11.
Added Sortable HSA column to the Control Window. The user can click on the lid column header
and then the HSA column header to make it sorted by HSA and then lid. Also, added Rainfall-Runoff Model
Preference (RR Model Pref)
column to the Control Window so one can see and sort the SAC-SMA points easily.
This field is also sortable.
12.
Added the following
tokens to control the existing forecast adjustment feature, which numerically
adjusts the generated forecast based on available observed data:
a) sshp_adjustment_pairing_minutes = 70 (time window
between observed and forecast pairs; if observed data is within the window, the
forecast is assigned the observed value)
b) sshp_adjustment_interpolation_hours = 3 (if the
observed and forecast values are outside this pairing time window, but within
the interpolation time window, then the forecast value is interpolated between
the observed values.)
No tokens were added for the related blending feature, since it is configurable
in the database and the GUI. Lastly,
fixed some bugs with the forecast adjustment feature.
13.
Added ability to see
the unadjusted simulated stage time series.
The initial behavior for showing this time series is based on the token sshp_show_simulated_timeseries; the user can change this
interactively. The time series is
displayed as blue when it varies from the forecast, otherwise, it is obscured
by the green forecast time series.
14.
Added feature which saves
the crest value when user saves a forecast.
F. RiverMonitor Data Monitor
1. Added ability to monitor SSHP-generated data (see
discussion in SiteSpecific section above).
2. Added ability to start SSHP from RiverMonitor. This is useful when the user sees that a SSHP
monitoring forecast run suggests flooding will occur.
3. Fixed
bug with occasionally incorrect flood and action stage departures (due to
precision errors)
4. Added basistime filter for forecast values. It now
shows fcst basis time column in the main
window. Added lookback
window for fcstbasis time and read the data that is
in the window and displays only those data passing time filter. No color is used for basis & valid time
cells, but it
does color the max fcst value red/yellow/white - no
gray color is usedsince the old records are filtered
out.
5. Fixed
sorting bug which appeared during refreshes.
6. Reorganized some of the menu bar selections.
7. Fixed some miscellaneous issues:
a) resorting issue when add a new column; b) slight sorting irregularity; c)
irregular coloring of SSHP fcst values,
8. Fix problemed
in which highlighting discontinuous rows doesn’t set the view port (the
visible area of the scrolled list) to
the first selected row.
G. RFC Forecast Services Data Manager
1. The
Data Services window now allows the user to enter “Analysis Start
Date” without entering a value for the “Analysis End Date”.
2.
Adjusted “Exceed Probability” field in the data services window so
that it no longer requires a value.
3.
Updated the message in the popup window which shows “data entry
violation” warning.
4.
Adjusted behavior of the “Console” window so that it prints one
message per line and new lines at the bottom. A “no records found”
message is printed when there are no rows found.
5.
Column sorting/searching bug is now fixed.
H. Batch/Utility/Data Programs
H.1. Purge_files
1. Adjusted syntax of purge commands to
prevent error messages from being generated.
Also,
adjusted so that it does not try and delete directories when the relevant
directory is empty.
H.2. System-Wide Features
1. Note, for offices that want background SiteSpecific forecasts to be generated,
run_SSHP_SAC_state_update on the cron,
at hourly intervals.
H.3. The IHFS
Database
1. Adjusted the process for loading the observed
river status values so that they do not load in missing (-9999.) values. These values are used by RiverPro
and HydroView
2.
Adjusted the delete_location function so that the
OB8.1 FcstService tables are deleted when a location
is deleted.