DATA USED IN DEVELOPMENT AND VALIDATION
DERIVATION OF STORM MOTION VECTORS
EXPRESSIONS FOR RAINFALL PROBABILITIES
SKILL OF CATEGORICAL (YES/NO) FORECASTS DERIVED FROM PROBABILITIES
CONVERTING PROBABILISTIC FORECASTS TO ISOHYETS
GRIDDED RAINFALL FORECAST DISPLAYS
FORECASTS OF 1-INCH RAINFALL AMOUNTS ASSOCIATED WITH INDIVIDUAL STORM CELLS
IMPLICATIONS FOR OPERATIONAL USE
The System for Convection Analysis and Nowcasting (SCAN), a suite of
algorithms within the Advanced Weather Interactive Processing System (AWIPS),
features a variety of automated radar interpretation products. The SCAN
0-1 Hour Rainfall algorithm (SRAIN1) produces the following products for
nowcasting convective rainfall:
gridded probabilities that 0-1 h rainfall will reach or exceed 0.1,
0.25, 0.5, and 0.75 inch (approximately 2.5, 6.25, 12.5, and 18.75 mm);
a 0-1 h gridded forecast rainfall amount;
storm-cell-based probabilities that 0-1 h rainfall will exceed one inch
anywhere in the immediate vicinity of convective storms.
The gridded probabilities and amounts are valid within each box of a 4-km grid
covering the central portion of the radar umbrella. The nowcasting algorithm
is based on an extrapolative-statistical method, first tested conceptually by
Saffle and Elvander (1981).
This technique was developed by correlating purely extrapolative forecasts
of radar reflectivity and vertically-integrated liquid (VIL) with analyses of
observed rainfall over many historical cases. The relationships between the
purely extrapolative forecasts and the observed amounts are now used in interpreting real-time extrapolative forecasts in terms of rainfall probability and
amount. The historical development data consist of base reflectivity observations from WSR-88D units in Oklahoma and Kansas, and coincident WSR-88D
Stage III radar-gauge rainfall estimate fields.
The resulting extrapolative-statistical approach to rainfall forecasting
implicitly accounts for echo decay and uncertainty in the extrapolation
process, and is a form of the Model Output Statistics technique often used to
produce forecasts of sensible weather from numerical weather prediction model
output. Because it incorporates input from a variety of statistical predictors, the extrapolative-statistical approach yields improvements over a purely
extrapolative one, at the expense of a modest increase in computing time. The
improvement is greatest for rainfall amounts of 0.5 inch and greater.
This note contains a brief explanation of the development methodology and
operational skill of the algorithm. A more complete summary of the methodology, as applied to an earlier version of the technique, was prepared by
Kitzmiller (1996). A summary of the development for the SCAN version is
contained in Kitzmiller and Churma (1999).
The radar data used in development were collected at the WSR-88D sites at
Twin Lakes, Oklahoma (KTLX), Tulsa, Oklahoma (KINX), and Wichita, Kansas
(KICT) during the 1994 and 1995 warm seasons. Each case featured at least
some convective storms, since our primary interest was forecasting the
location of the heaviest rainfall within regions affected by thunderstorms.
Base reflectivity data were objectively interpolated to a 4-km local grid
oriented north-south and covering a 230-km umbrella centered on the radar
site. The default WSR-88D Z-R and Z-M relationships were used to derive
rainrates from the reflectivity field at the lowest antenna elevation, and VIL
from volumetric reflectivity.
Our development effort was focused on this area because it was among the
first to be covered by WSR-88D Stage III precipitation analyses (Breidenbach
et al. 1998), prepared operationally at the NWS River Forecast Center in
Tulsa, Oklahoma. These analyses incorporate both radar and gauge data. One-hour rainfall estimates are interpolated to a polar stereographic grid with a
spacing of approximately 4 km over the central United States. Some manual
editing of the analyses is done to remove erroneous gauge reports and other
artefacts such as hail contamination of rainfall estimates.
An independent sample was collected from 72 events during the 1996 warm
season, at 27 WSR-88D sites in the United States east of the Rocky Mountains.
The 1996 data were not used in the initial development effort itself because
it includes some radar precipitation estimates with no gauge bias adjustment.
These data are still useful in testing the reliability of the forecast
algorithm in different geographic regions.
To prepare data for the development or training sample, we obtained 113 sequences of volumetric radar scans that were continuous for at least 25 minutes, and included a scan beginning 54-59 minutes into the hour. For each
sequence, a single storm motion vector (SMV) was derived by using a binary
pattern-matching method. Forecasts of the reflectivity and VIL fields were
made at 10-minute intervals through 60 minutes. For the sake of simplicity,
reflectivity was converted to nominal rainrates and integrated to create
rainfall amounts. The VIL forecasts were temporally averaged over the 1-hour
forecast period.
The statistical predictor/predictand sample was created by drawing the
predictor values and the verifying rainfall value from every ninth box in each
of the analysis grids. Only boxes between 20 and 90 nm from the radar were
included, since the radar is unable to detect higher-altitude echoes close to
the radar and lower-altitude echoes at longer ranges. We obtained a development sample of forecasts and verifying observations from 6886 individual grid
boxes.
A binary-correlation pattern-matching technique was used to estimate the
storm motion vector (SMV) for each historical case. This method, described by
Saffle and Elvander (1981) and by Ciccione and Pircher (1984), is computationally efficient and works well when the entire echo region does not change
size or shape appreciably between images. The method reduces two radar grid
maps to all "0" or "1" points according to some reflectivity criterion, in
this case 30 dBZ. The grids are separated by 20-35 minutes in time. The
earlier grid is shifted relative to the later one until the binary correlation
coefficient between the two is maximized; the vector displacement divided by
the time lag represents a velocity estimate.
In real-time operations, the lag-correlation method described above is used
in stratiform rainfall situations; in situations with convective storms
present, the average cell motion output in the Storm Track Information (STI)
product is used. The SMV is set equal to the 700-mb wind vector when the echo
pattern is too small or is evolving too rapidly for either of these approaches
to work effectively.
We obtained expressions relating the various predictors to the probability of reaching rainfall thresholds by a forward-selection linear screening regression procedure. In the expressions below, rainfall amounts are in mm and VIL is in kg m-2. For the 0.1-in predictand, the best fit to the dependent data was realized by the following expression:
P( 0.1) = 0.412 + 3.517 [VILFCST]AVG3 + 0.704 [VILINIT]MAX7
+ 0.98 [NREFL24]3X3 (1)
where P is probability in per cent. The VIL-based predictors are defined:
[VILFCST]AVG3 is the local average of the 0-60 minute time-average VIL forecast, [VILINIT]MAX7 is the maximum of VIL within the local 7x7 box region. The
final predictor, [NREFL24]3X3, is defined as the sum of the number of occurrences of 24-dBZ reflectivity within the local 3x3 box region over all 7
forecast intervals within the 0-60 minute period. This predictor ranges in
value from 0 to 63 (9 boxes times 7 forecast intervals beginning at 0 minutes). It indicates the fraction of the local area and time that is forecasted to experience 24-dBZ or greater reflectivity. The 0.1-inch rainfall
threshold was exceeded in 16.3% of the cases. This expression explained 47%
of the predictand variance within the dependent data sample.
A similar expression was derived for the probability of rain exceeding 0.25 inch:
P( 0.25) = -0.331 + 2.701 [VILFCST]AVG3 + 0.469 [VILINIT]MAX7
+ 0.230 [NREFL24]3X3 + 0.090 [RAINFCST]AVG3 (2)
where [RAINFCST]AVG3 is the local average of the 0-30 minute extrapolation
rainfall forecast. The 0.25-inch rainfall threshold was exceeded in 7.6% of
the cases, and this regression equation explains 34% of the predictand
variance.
For the two highest thresholds, the following expressions were derived:
P( 0.5) = 0.144 + 2.074 [VILFCST]AVG3 + 0.063 [RAINFCST]AVG3 (3)
P( 0.75) = 0.112 + 1.125 [VILFCST]AVG3 + 1.905 [NVIL10]3X3 (4)
where [NVIL10]3X3 is the number of boxes in the local 3x3 box region in which
initial-time VIL exceeds 10 kg m-2. This last predictor indicates the presence of larger convection cells. Within the dependent sample, 2.8% of the
cases had rainfall 0.5 inch, and 1.1% had rainfall 0.75 inch. The expression in (3) explained 19.4% of the predictand variance, and the expression in
(4) explained 9.5%
An informative method of assessing the skill of the probability forecasts
obtained from (1)-(4) involves scoring categorical (yes/no) forecasts derived
from the probabilities. The most common method of deriving categorical
forecasts from probabilities is by applying a threshold value: all probabilities below the threshold are interpreted as "no," those at and above the
threshold as "yes." Such forecasts can be scored in terms of probability of
detection (POD), false alarm ratio (FAR), and critical success index (CSI).
The calculation and characteristics of these scores have been discussed by
Donaldson et al. (1975) and by Schaefer (1990). Another measure of forecast
utility is bias, which is the ratio of the number of "yes" forecasts to "yes"
observations.
Scores for each of the four probability equations appear in Figs. 1-4. The
POD, FAR, CSI, and bias are shown for all probability thresholds from 1% to
30% or higher. For example, for the P( 0.1) equation (Fig. 1),
a yes/no threshold of 20% yields a POD of 0.82 and an FAR of
0.42. That is, 82% of all observed 0.1-inch precipitation events were covered
by "yes" forecasts, and 42% of all "yes" forecasts were false alarms in which
the observed rainfall was less than 0.1 inch. The bias at this threshold is
about 1.6, meaning that here were 60% more "yes" forecasts than were observed.
Alternatively, this bias indicates that on average the forecasted 0.1-inch
isohyets enclose 60% more area than do the observed isohyets. Of course, all
of the forecast scores are based on performance over many different events,
and will vary substantially from case to case.
As is apparent from the POD and FAR curves, it is possible to achieve large
POD's at the expense of high FAR and bias, or to achieve very low FAR's by
accepting low POD's. The CSI represents a measure of how well the FAR and POD
can be balanced between too many false alarms and too few event detections.
For the 0.1-inch forecasts, the highest CSI, 0.56, is reached at a threshold
of 35%, where the POD is 0.7, the FAR is 0.3., and the bias is 1.1.
Forecast skill is lower for the higher rainfall thresholds. The peak CSI
values are lower: 0.43 for 0.25 inch (Fig. 2), 0.30 for 0.5 inch
(Fig. 3), and 0.14 for 0.75 inch (Fig. 4). It
is also apparent that, as higher rainfall amounts are considered, it is
necessary to accept a higher FAR and bias in order to achieve any value of the
POD. For example, in order to detect 70% of events (POD=0.7), one must accept
an FAR of 0.3 for the 0.1-inch threshold, but an FAR of 0.78 for the 0.5-inch
threshold and an FAR of 0.89 for the 0.75-inch threshold.
We verified that the categorical forecast skill was similar within the 1996
independent case sample. In Figs. 5-8, POD and FAR are shown for both the
dependent and independent samples, with bold lines for the dependent data and
fine lines for the independent. For the three lower thresholds (Fig. 5
for 0.1 inch, Fig. 6 for 0.25 inch, and Fig. 7
for 0.5 inch), the POD's are very similar for both samples
except at threshold probabilities > 40%, while the FAR in the independent
sample is somewhat larger than in the dependent. This is logical, since the
probability equations on which the categorical forecasts were based were tuned
for skill in the dependent sample.
At higher threshold probabilities, the POD's were actually higher in the
independent sample. Also, skill for forecasting 0.75-inch amounts was
actually higher in the independent sample, i.e., POD was generally higher and
FAR lower (Fig. 8). This result is likely to be a consequence
of the rarity of the event, and thus due to chance. However, the verification
experiment strongly suggests that the forecast algorithm will perform stably
in most trials with a reasonably large number of cases.
The discussion in the previous section focused on simple yes/no forecasts, considering only whether one rainfall threshold would be exceeded. For many purposes, it is desireable to derive a forecast isohyet field from the probabilities. A simple method for carrying out this conversion is to assign a rainfall amount to a grid box after considering the probabilities for all threshold amounts.
In the present application, this is done by applying a fixed set of thresholds, and forecasting the rainfall category to be the highest one for which
the threshold is exceeded.
The set of probability thresholds were selected to be those which produced
the highest CSI within the dependent sample. These probabilities were 33%,
30%, 22%, and 14%, for the 0.1-, 0.25-, 0.5-, and 0.75-inch thresholds respectively. Thus for any grid box, if P( 0.1) is < 33%, the categorical rainfall
amount is forecasted to be less the 0.1 inch. If P( 0.1) > 32%, the categorical amount is forecasted to be at least 0.1 inch. If P( 0.1) > 32% and
P( 0.25) > 29%, the amount is forecasted to be at least 0.25 inch, and so on.
We have found that this procedure yields forecast fields with a noticeable
bias toward higher amounts, but which capture a substantial portion of the
higher rainfall observations and forecast the shape of the isohyet field well.
A tabulation of forecasts and obervations by category for the dependent sample
appears in Table 1. The columns represent forecast categories
and the rows, observed categories. Of the 6886 cases, 1370 were forecasted to
have rainfall 0.1 inch, while 1125 actually did so, indicating a bias of 1.2
for the precipitation area as a whole. For all cases in which rain was
forecasted, 34% fell into the correct observed category and 86% fell within
one category of the correct one.
Within the independent 1996 sample (Table 2) we found similar
results. The bias in overall rainfall area 0.1 inch was somewhat higher
(1.44 compared to 1.2), while 23% of the rainfall forecasts fell into the
correct category and 79% fell within one category of the correct one.
In practice, the categorical forecasts are transformed to continuous
rainfall amounts in the range 0.1-0.7 inch, and contoured at 0.1-inch intervals in the output graphical display. This was done by treating the boxes at
boundaries between categories as if they contained the lowest rainfall amount
within the category; the amounts within all remaining boxes were then estimated by interpolation.
These products are displayable through the local radar product menu within
D2D. To be seen at the proper scale, they should be accessed through the WFO-scale (local-scale) map display.
In AWIPS Build 4.1, the following command sequence accesses SCAN quantitative precipitation forecast (QPF) products:
1) Left click on the WSR-88D identifier on the menu bar above the map
display;
2) Left click and hold on the QPF products line of the local radar product
pull-down menu;
3) Drag the cursor to the right into the QPF products menu, down to the
desired product, then release the left button.
The product (if available) will then appear. When the full-sized WFO map is
displayed, only every other probability isopleth will appear (i.e. 10%, 30%,
50%, ...). On magnification, intermediate isopleths will appear. Similarly,
at full scale only a limited range of forecast isohyets will appear, with
isohyets appearing at every 0.1 inch on magnification.
The relationship between rainfall forecasts and initial-time base reflectivity fields is illustrated in Figs. 11-12. In the situation shown here
(1724 UTC, 18 August 1998), the storms over southern Pennsylvania were moving
slowly east; thus the highest probabilities for 0.1 inch (Fig. 11)
and the highest rainfall amount forecasts (Fig. 12)
lie just east of the convective cell centroids.
In another similar situation (7-8 September 1998), the forecasts showed
basically good agreement with the mesoscale organization of the observed
precipitation. Radar observations were from the WSR-88D unit at Sterling, VA.
At 2357 UTC, thunderstorms and showers over southern Pennsylvania, western
Maryland, and northwestern Virginia were moving slowly to the east-southeast.
The likeliest areas for significant rainfall during the subsequent 0000-0100
UTC period are indicated by the probabilities of 0.1 (Fig. 13),
0.25 (Fig. 14),
and 0.5 inch (Fig. 15).
The very highest probabilities were over
south-central Pennsylvania, as were the highest rainfall amount forecasts
(Fig. 16).
The Stage I radar-estimated rainfall field for the period 0000-0100 UTC
(Fig. 17) indicated that the heaviest
amounts did occur there, with a few values of 0.4 inch. The overall shape of
the forecast fields agrees with that of the observed, though the forecasts did
not completely capture the precipitation over and to the west of
Baltimore City.
We found that within the development data sample, our techniques were
inadequate to skillfully forecast the location of 1-hour rainfall amounts in
excess of one inch. These are especially rare event (in terms of specific
locations within a 4-km grid) and our predictors could not differentiate
between rainfall amounts of 0.5-0.99 inch and amounts 1 inch.
However, the extrapolative-statistical method yields useful indications as
to whether or not a storm cell will produce 1-inch rainfall at some place near
the path of the cell centroid. Formally, these cell-associated probabilities
are valid within a square region 28 km on a side, immediately in the path of
the storm reflectivity centroid. Such forecasts are analogous to the output
of the SCAN Severe Weather Detection and SCAN Severe Hail Detection algorithms
(Kitzmiller and Breidenbach 1993 and 1995; Kitzmiller et al. 1999).
The probability that an individual storm cell will produce one inch of rain is given by:
P( 1.0) = 2.19 x MXVILFCST - 5.76 (5)
where P( 1.0) is the probability in percent and MXVILFCST is the maximum
value of the time-average VIL forecast (kg m-2) in the path of the associated
storm cell centroid. Operationally, the probability is truncated at 40%,
since we found that the observed event relative frequency remains near that
value even when (5) evaluates to larger values.
In practice, we must determine P(1.0) for storm cells already defined in the
STI product. Therefore, local maxima in the MXVILFCST field are associated
with current cell centroids by finding the centroid closest to the point
reached when the VIL maximum is moved backward 30 minutes at the current
average cell velocity.
Categorical (yes/no) forecast skill scores within the dependent data sample
are shown in Fig. 18. The CSI is maximized near a threshold
probability of 17%, where the POD is 0.55 and the bias is about 1.7.
The output of this algorithm appears in the Thunderstorm Product popup box,
along with other storm characteristics such as maximum VIL, severe weather and
large hail probability, and lightning strike count. The probability number is
located under "HVY" (Fig. 19).
To obtain the popup box for any identified storm cell in the Thunderstorm
Product display, in AWIPS Build 4.1:
1) Select Storm Cells/Site Storm Threat from the local radar menu;
2) Position the cursor over the "Thunderstorm Popup" text identifier at the
lower right part of the screen, and middle click. This makes the product
"editable";
3) Position the cursor within a cell circle and right-click to bring up the
box for that cell.
In order for the rainfall forecasts to be generated in real time, AWIPS must
ingest the following products from the WSR-88D Radar Product Generator (RPG):
Z (base reflectivity, product 19, 0.5 elevation, 1-km resolution, 16 data levels)
VIL (product 57)
STI (Storm Track Index, product 58)
These must be included in the Routine Product Set (RPS) list.
The system also ingests numerical model input data for the 700-mb wind
vector. These data could be from the Eta or the RUC models.
Users should note that the gridded products forecast rainfall as averaged
over 16-km2 regions, the resolution of the Digital Precipitation Array (DPA)
product. This resolution is coarser than that of the One Hour Precipitation
(OHP) product, which shows averages for 1 x 2 km regions. Therefore the peak
amounts appearing in the forecasts are generally lower than those in the
corresponding OHP. However, any numerical value appearing in one box of the
DPA represents a substantially larger volume of water than does the same value
appearing in one box of the OHP. Stage III estimates were used during
development in preference to higher-resolution data because they had already
received quality control, were spatially continuous over fairly large areas,
and were likely to feature fewer range degradation effects than were OHP
products taken one umbrella at a time.
The cell-based 1-inch rainfall probability algorithm monitors storm development and motion to identify cells that have particularly large water loads and
are moving slowly enough to produce locally-heavy rainfall. Of course, the
heavy rainfall potential of a storm is often significantly different from its
severe weather potential. The SCAN suite has been constructed to identify
both types of events.
Robert Saffle and Robert Elvander provided the initial impetus for this
work, and gave helpful advice about getting it underway. Jeffrey Ator and
Bryon Lawrence created the computer code for binary pattern matching and
extrapolation forecasting. Mark Lilly selected the validation cases and
provided assistance in data processing. The WSR-88D Operational Support
Facility provided a copy of the algorithm for calculating gridded VIL from the
Archive Level II radar data, which was then coded by Douglas Rankin.
Breidenbach, J. P., D. J. Seo, and R. A. Fulton, 1998: Stage II and Stage III
post processing of NEXRAD precipitation estimates in the modernized
Weather Service. Preprints, 14th International Conf. on Interactive
Information Processing Systems, Phoenix, Amer. Meteor. Soc., 263-266.
Ciccione, M., and V. Pircher, 1984: Preliminary assessment of very short term
forecasting of rain from single radar data. Proceedings Second
International Symposium on Nowcasting, Norrkoping, European Space Agency,
241-246.
Donaldson, R. J., R. M. Dyer and M. J. Kraus, 1975: An objective evaluator of
techniques for predicting severe weather events. Preprints Ninth Conference on Severe Local Storms, Norman, Amer. Meteor. Soc., 321-326.
Kitzmiller, D. H., and J. P. Breidenbach, 1993: Probabilistic nowcasts of
large hail based on volumetric reflectivity and storm environment characteristics. Preprints 26th International Conference on Radar Meteorology,
Norman, Amer. Meteor. Soc., 157-159.
_____, and _____, 1995: Detection of severe local storm phenomena by automated interpretation of radar and storm environment data. NOAA Technical
Memorandum NWS TDL 82, National Weather Service, NOAA, U.S. Department of
Commerce, 33 pp. [Available from Techniques Development Laboratory,
W/OSD2, National Weather Service, 1325 East West Highway, Silver Spring,
Md.]
_____, and M. E. Churma, 1999: The AWIPS 0-1 hour rainfall forecast
algorithm. Preprints, 15th International Conf. on Interactive Information
Processing Systems, Dallas, Amer. Meteor. Soc.
_____, _____, and M. T. Filiaggi, 1999: Severe local storm and large hail
probability algorithms in the System for Convection Analysis and
Nowcasting. [Accessible at http://www.nws.noaa.gov/tdl/scan/scan2.html]
Saffle, R. E., and R. C. Elvander, 1981: Use of digital radar in automated
short range estimates of severe weather probability and radar reflectivity. Preprints Seventh Conference on Probability and Statistics in
Atmospheric Sciences, Monterey, Amer. Meteor. Soc., 192-199.
Schaefer, J. T., 1990: The critical success index as an indicator of warning
skill. Wea. Forecasting, 5, 570-575.
