EVALUATION OF THE AWIPS THUNDERSTORM PRODUCT
Michael E. Churma*
Research and Data Systems Corporation
Silver Spring, Maryland
Stephan B. Smith
Techniques Development Laboratory
Office of Systems Development
National Weather Service, NOAA
Silver Spring, MarylandĚ
The AWIPS Thunderstorm Product is designed to provide automatic thunderstorm detection and severity guidance for airports within the WSR-88D surveillance network (Smith and Churma 1996). The original algorithm was developed by forecasters at the Washington D.C.-Baltimore forecast office in Sterling, Virginia, as part of a risk reduction exercise initiated by NWS Eastern Region Headquarters (Stern et al. 1994). The Techniques Development Laboratory (TDL) was given the task of implementing the Sterling algorithm on AWIPS-compatible workstations by substituting volumetric, WSR-88D radar data for the single elevation scan WSR-74S data. During the 1996 convective season, this Phase-I AWIPS product was tested at sites (mostly airports) located within the umbrella of the radar at Sterling, Virginia (Fig.1). The results were compared to manual observations of thunderstorms in order to calculate a probability of detection (POD), false alarm ratio (FAR), and critical success index (CSI; see Schaefer 1990). This paper presents the results of this evaluation.
The algorithm is run at the completion of each radar volume scan (approximately every 56min in precipitation mode). Grids (4 km) of Vertically Integrated Liquid (VIL; see Greene and Clark 1972) and base reflectivity are ingested along with storm cell locations and cloudtoground (CG) lightning flashes accumulated since the previous volume scan. For each airport within the radar umbrella, a decision tree is followed to determine whether a thunderstorm threat is present at that airport.
Figure 2 displays the Phase-I (additional Phases II and III are described in section 4) thunderstorm algorithm decision tree. The conditions for determining a thunderstorm threat are organized from most definitive to least definitive. If a storm cell is identified within 10 nm of an airport with associated lightning, a thunderstorm threat is deemed to be present at the airport. If this condition is not met, the algorithm then checks to see whether widespread, light VIL (<10kgm2) and lightning are found near the airport. If true, a thunderstorm threat is again flagged. If false, the algorithm checks for a cell within 10 nm but with no lightning. If a thunderstorm threat is still not identified, a check is made for high reflectivity (>40 dBZ) within 10 nm of the airport. This additional check was incorporated onto the original Sterling algorithm to take into account situations where strong reflectivity at one level might indicate the existence of convective downdrafts that could seriously impact aircraft operations, even if associated VIL values were weak. Finally, if a CG flash and any VIL is found within 10 nm of an airport, a threat is identified.
It is important to note that the absence of CG flashes within a 56 min ingest window is not unusual even for severe thunderstorms (Smith 1996). Therefore, conditions in which a thunderstorm threat is flagged in the absence of CG lightning does not represent a major broadening of the definition of a thunderstorm. Nevertheless, the true aim of this product is to automatically identify conditions which present a convective hazard to aviation, i.e. convective downdrafts and strong low-level wind shear. Although the presence of CG lightning is a strong indicator of these conditions, its absence does not preclude them. In the event that radar or lightning data are not available, contingency logic trees, essentially subsets of that shown in Fig. 2, are invoked to provide the thunderstorm decision. The algorithm also has a simple consistency check to identify high reflectivity generated by ground clutter or anomalous propagation where there is no supporting VIL and/or nearby lightning.
For this evaluation, the algorithm was run at over 40 sites (Fig. 1) representing a variety of elevations, geographic areas, and distances from the radar. The thunderstorm threat decision was compared to corresponding manual observations of thunder at fully manual or augmented ASOS sites. Only convectively active days were chosen for comparison. On clear, stable days, the algorithm performs flawlessly in detecting the absence of thunderstorms at airports, volume scan after volume scan, for hours on end. A sample of 1539 manual observations was identified for comparison with the algorithm output. Tests were carried out to examine the algorithm's sensitivity to different combinations of data input. Specifically, the manual observations were compared to algorithm output produced with the full complement of radar and lightning data, radar data only, and lightning data only.
We wish to point out that manual observations of thunderstorms (our ground truth) are not free from error. The observer may not be able to hear audible thunder under certain conditions. Lightning flashes are not always easily visible during daylight hours, and thus tend to be underreported. Because our algorithm does not suffer from these problems, it will tend to report more thunderstorms than manual observers. As a result, what is normally considered to be a false alarm (algorithm detects a thunderstorm, but the human observer does not report one) may, in fact, simply be a case where the human observer missed a thunderstorm while the algorithm did not.
A POD of 85%, a FAR of 32%, and a CSI of 60% were obtained with the algorithm running with the full set of radar and lightning inputs. This represents a significant improvement over the results obtained by Stern et al. (1994) using coarser WSR-74S radar data, single elevation scan storm tracking, and 20 min temporal resolution. Figure 3 shows that the POD drops from 85% to 74% when the algorithm is run in either the radar only or lightning only mode. This attests to the complementary nature of different remote sensors in detecting thunderstorms. A CG lightning only algorithm has been proposed as an adequate replacement for manual reporting of thunder at ASOS sites. These results suggest that such an algorithm would significantly underreport the presence of thunderstorms near airports.
The Phase-I product is currently being implemented as a part of the System for Convection Analysis and Nowcasting (SCAN; Smith et al. 1998) in AWIPS in 1998.
A Phase-II product incorporates additional conditions into the decision tree based on cold cloudtop temperatures estimated from GOES infrared satellite data, as well as a cumulocongestus (pre-thunderstorm) detection algorithm. The former is designed to compensate for degradation in radar information resulting from mountainous terrain. The latter locates strong cloudtop cooling in GOES infrared imagery as a signal of rapidly growing thunderstorms. The Phase-II algorithm underwent real-time testing for the Sterling radar umbrella during the 1997 convective season and is scheduled for AWIPS implementation in 1999.
For a Phase-III product development, the automated site based thunderstorm detection algorithm will be converted to a gridded one in order to provide more complete and uniform thunderstorm surveillance over the continental United States. An estimation of detection confidence, based on multi-sensor redundancy checks, is also planned for this product enhancement.
The skilled assistance of David Kitzmiller on this project is gratefully acknowledged. We thank Andy Stern for providing the original code developed at Sterling.
Greene, D. R., and R. A. Clark, 1972: Vertically integrated liquid A new analysis tool. Mon. Wea. Rev., 100, 548-552.
Schaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting, 5, 570-575.
Smith, S. B., 1996: How soon can a thunderstorm be identified? A comparison of satelliteobserved cloudtop cooling and the onset of cloudtoground lightning. Preprints 18th Conf. on Severe Local Storms, San Francisco, Amer. Meteor. Soc., 479-482.
_____, and M. E. Churma, 1996: An overview of the AWIPS Thunderstorm Product. Preprints 15th Conf. on Weather Analysis and Forecasting. Norfolk, Amer. Meteor. Soc., 297-300.
_____, T. M. Graziano, R. A. Lane, W. O. Alexander, M.D. Eilts, J. T. Johnson, J. W. Wilson, R. D. Roberts, D. W. Burgess, D. H. Kitzmiller, R. E. Saffle, R. C. Elvander, S. M. Zubrick, J. T. Schaefer, S. J. Weiss, and D. A. Imy, 1998: The System for Convection Analysis and Nowcasting (SCAN). Preprints 16th Conf. on Weather Analysis and Forecasting, Phoenix, Amer. Meteor. Soc., (this volume).
Stern, A. J., R. H. Brady, P. D. Moore, and G. M. Carter, 1994: Identification of aviation weather hazards based on the integration of radar and lightning data. Bull. Amer. Meteor. Soc., 12, 2269-2280.