Local forecast by
"City, St" 


7. Evaluation of anomalous propagation echo detection in WSR88D data: a large sample case study
Witold F. Krajewski and Bertrand Vignal
Iowa Institute of Hydraulic Research
The University of Iowa
Iowa City, Iowa 52242, USA
Technical Note
submitted to
Journal of Oceanic and Atmospheric Technology
June 2000
Corresponding author:
Witold F. Krajewski
Iowa Institute of Hydraulic Research
Iowa City, Iowa 52242, USA
witoldkrajewski@uiowa.edu

Abstract
We evaluate a method of detecting anomalous propagation echo in volume scan radar reflectivity data. The method is based on a neural network approach and is suitable for operational implementation. It performs a classification of the base scan data on a pixelbypixel basis into two classes: rain and norain. We describe the results of applying the method to a large sample of WSR88D level II archive data. The data consist of over 10,000 volume scans collected in 1994 and 1995 by the Tulsa, Oklahoma WSR88D. Our evaluation includes analyses based on radar data only and on various comparisons of radar and rain gauge data. The rain gauge data are from the Oklahoma Mesonet. The results clearly show the
effectiveness of the procedure as indicated by reduced bias in rainfall accumulation and improved behavior in other statistics.
7.1 Introduction
One can safely say that quality control (QC) of radar reflectivity data is the most important step in
the overall process of radar rainfall estimation. The crucial aspect of such QC lies in the detection of ground clutter and echoes caused by anomalous propagation (AP) of radar waves (Moszkowicz et al. 1994). Recently, Grecu and Krajewski (2000) proposed a method of AP detection based on a neural network approach. The method classifies the base scan radar reflectivity data into rain or norain echo on a pixelbypixel basis. With this method, several characteristics of the reflectivity field are computed in the neighborhood of the pixel under investigation. A trained (i.e. calibrated) neural network uses these characteristics as inputs and performs the classification. A unique aspect of the method is the selection of the training data set required by the neural network approach. The authors advocate selecting only the "clear cut" cases, both for rain and norain echo. This allows fast and efficient preparation of the training sample, and thus, rapid implementation of the methodology. The selected set is used for random drawing of the training and validation samples. Thus, the training and the validation do not include the challenging cases where AP and rain might be colocated.
The Grecu and Krajewski (2000) methodology includes selfevaluation through repeated resampling and
crossvalidation. Performance was monitored in terms of the number of misclassified pixels. In this short communication, we expand the evaluation methodology. We include several analyses based on radar data only, as well as various comparisons with rain gauge observations.
7.2 Summary of the data
We used the same, but somewhat expanded database as Grecu and Krajewski
(2000). The data were collected by the Tulsa, Oklahoma, Weather Surveillance Radar1988 Doppler version (WSR88D). In Oklahoma, the rainfall regime is dominated by midlatitude convective systems
(Houze et al. 1990). The data cover mostly the warm season months of 1994 and 1995. Since the study of Grecu and Krajewski (2000), we filled several gaps in the radar data, and as a result had available over 10,000 volume scans (see Figure 1 for the histogram). These radar data were converted from the Archive level II format (Klazura and Imy 1993) to the efficient format ASCIIRLE (Kruger and Krajewski 1997) allowing the rapid access required for such a large sample study. The rain gauge data we used are from the Oklahoma Mesonet (Brock et al. 1995). Some 49 rain gauges are located within the Tulsa radar domain (Figure 7).
7.3 Results
We used the neural network trained by Grecu and Krajewski (2000), applying the same network configuration for the entire data set. Our analyses are divided into two parts: that based on radar data only and that based on both radar and rain gauge observations.
a. Radaronly analyses
First, let us consider the probability of detection (POD) of an echo stronger than a certain threshold, calculated on a pixelbypixel basis. Clearly, the possible values range from 0, if no echo is ever detected at the given pixel (this may happen if the view of the pixel is completely blocked by an object such as a building or mountain), to 1, if an echo is always detected (as in the case
of reflections off a mountain). If only raincaused echoes were detected, the expected range of values would be around 0.05 but the exact numbers are unknown. Furthermore, if the rainfall under the radar umbrella were climatologically and statistically homogeneous, and we had a sufficiently large
sample, the range of values of the POD would be very narrow a single
spike in the limit.
How does the POD pattern look for the Tulsa WSR88D? Figures 2 and 3 provide
the answer. In Figure 2 we show the effect of various thresholds (T) on the pattern of POD. Clearly, for T=0 dBZ the pattern displays circular artifacts which result from a combination of ground clutter effects (near the radar), and AP further out. The slight shift of the pattern towards the southeast reflects the rainfall climatology of the region. As the threshold increases, the POD pattern becomes more uniform. This is because by using thresholds, we eliminate much of the ground clutter and AP.
However, we also eliminate some of the rainfall. For example, simple backoftheenvelope calculations indicate that for T=20 dBZ we may be cutting as much as 10% of the areaaveraged rainfall accumulation. Thus, while using thresholds may be considered the simplest QC method, it introduces the risk of eliminating climatologically significant rainfall.
Now let us compare the corresponding patterns of POD calculated from quality controlled data (Figure 3). Clearly, the POD is now consistently lower and the patterns more uniform. If, in addition, we consider the corresponding histograms for both sets of the POD patterns (Figure 4), it becomes clear that the applied QC is more effective than simple thresholds. For the qualitycontrolled
data, there is little effect of applying thresholds on the histogram of POD, which indicates that most false echoes were removed by the QC procedure.
b. Radar and rain gauge analysis
Our analysis now includes hourly estimates of rainfall calculated by applying the "standard" NEXRAD ZR relationship Z=300R^{1.4} to the base scan data (antenna elevation angle of 0.47º). We also use hourly rain gauge data from the Oklahoma Mesonet.
First, consider the conditional probability that radar observes reflectivity (Z) greater than 10 dBZ given that a colocated rain gauge observes measurable rainfall (R>0.1 mm). This statistic is useful as the study by Grecu and Krajewski (2000) did not address the performance of AP detection in
the presence of rain. Theoretically, the probability P(Z > 10dBZ  R > 0.1 mm) should be high, but an AP procedure that eliminates too much rain would decrease it below the true (but unknown) level. As evident from Figure 6, the qualitycontrolled data results in the conditional probability more uniformly distributed along the distance from the radar, in line with data at far ranges where the QC procedure does little.
The effect of quality control is much more dramatic if we consider the conditional probability that radar observes significant echo given that rain gauges observe no rainfall, i.e. P(Z > 10dBZ  R < 0.1 mm). Theoretically, this probability should be only slightly greater than zero as radar "sees" larger
areas. Indeed, as we show in Figure 7, this probability seems reasonable for the qualitycontrolled data, but seems too high for the original data. Most importantly, this probability is independent of the distance from radara strong indication of the effectiveness of the QC procedure.
Finally, consider the total rainfall accumulation both from the rain gauges and the radar. We calculated the field of rainfall accumulation for the entire duration of our data set by interpolating between the locations of the gauges. The interpolation method is based on four quadrant nearneighbors and inverse distance weightsit is the same method used by the National Weather Service in its hydrologic forecasting. We applied it to both the radar and the rain gauge fields (Figure 8). We show the accumulated field within the boundaries of the state of Oklahoma only to avoid the possible inconsistency of using a rain gauge data set outside the Oklahoma Mesonet. To make the comparison easier, we also show a map of the biasdefined as difference between the twointerpolated the same way. It is clear that the accumulation field displays lower bias when calculated on the quality controlled data. We summarize the quantitative results in Table 1. The table shows that the effect of the QC procedure is smallest at distances higher than 100150 kilometers from the radar. This is understandable, as the influence of AP diminishes farther out from the radar. It is the nonuniformity of the vertical profile of reflectivity that is mostly responsible for the discrepancy at those
far ranges (Vignal et al. 1999).
7.4 Conclusions
The QC procedure developed by Grecu and Krajewski (2000) for detecting AP signal in WSR88D data proved very effective when applied to the Tulsa, Oklahoma radar. The agreement of longterm accumulation between the radar and rain gauge data is remarkable, especially considering no fitting of the ZR or any other parameters of the rainfall estimation algorithm. The algorithm we applied is the simplest it can beit does not include rainfall classification, vertical integration, or pattern
extrapolationall elements that improve radar rainfall estimation as shown in previous studies (e.g. Ciach et al. 1997; Anagnostou and Krajewski 1999). We used the standard ZR relationship that
proved adequate.
Perhaps the most important aspect of our study is that we used a large sample of volume scan reflectivity data and mimicked an operational environment to the degree possible. We are convinced that, due to the highly variable nature of rainfall processes, large sample studies are critically important in evaluating new technologies of radar rainfall estimation, including the "solveitall" polarimetric methods (e.g. Zrnic and Ryzkov 1999). The traditional mode of testing radar methods using only a few selected events is simply misguided. Many statistics calculated from radar and rain gauge observations, e.g. bias, are meaningful and/or statistically stable only for accumulations over a long period.
We have assembled large samples of radar reflectivity data for several WSR88D locations, including Davenport, Iowa, Grand Forks, North Dakota, Memphis, Tennessee, and Melbourne, Florida. We are planning to use these data sets for a variety of comparative studies and analyses, and we invite collaboration.
Acknowledgements. This study was supported by the National Weather Service Office of Hydrology under Cooperative Agreement with the Iowa Institute of Hydraulic Research (NA47WH0495). We would like to thank Dr. Mircea Grecu for his assistance with the computer code and Dr. Grzegorz Ciach for many helpful discussions.
References
Anagnostou, E.N. and W.F. Krajewski, Realtime radar rainfall estimation. Part 1: algorithm formulation, Journal of Atmospheric and Oceanic Technology, 16(2), 189197, 1999.
Brock, F.V., K.C. Crawford, R.L. Elliot, G.W. Cuperus, S.J. Stadler, H.L. Johnson and
M.D. Eilts: The Oklahoma Mesonet A technical overview, Journal of Atmospheric and Oceanic
Technology, 12(1), 519. , 1995
Ciach, G.J., W.F. Krajewski, E.N. Anagnostou, J.R. McCollum, M.L. Baeck, J.A. Smith,
and A. Kruger: Radar Rainfall Estimation for Ground Validation Studies of the
Tropical Rainfall Measuring Mission, Journal of Applied Meteorology, 36(6),
735747, 1997.
Grecu, M. and W.F. Krajewski: An efficient methodology for detection of anomalous propagation echoes in radar reflectivity data using neural networks, Journal of Oceanic and Atmospheric Technology, 17(2), 121129, 2000.
Houze Jr., R.A., Smull, B.F. and P. Dodge: Mesoscale organization of springtime
rainstorms in Oklahoma. Monthly Weather Rev., 117, 613654, 1990.
Kruger A. and W.F. Krajewski: Efficient storage of weather radar data. Software Practice and Experience, 27, 623635, 1997.
Klazura G.E. and D.A. Imy: A description of the initial set of analysis products
available from the NEXRAD WSR88D system. Bull. Amer. Meteor. Soc., 74(7),
12931311, 1993.
Moszkowicz, S., G.J. Ciach, and W.F. Krajewski, Statistical detection of anomalous propagation in radar reflectivity patterns, Journal of Atmospheric and Oceanic Technology, 11(4), 10261034, 1994.
Vignal B., Andrieu H. and J.D. Creutin, 1999: Identification of vertical profiles of
reflectivity from voluminal radar data. J. Appl. Meteor., 38(8), 12141228.
Zrnic, D.S., and A.V. Ryzhkov, 1999: Polarimetry for weather surveillance radars.Bull.
Amer. Meteor. Soc., 80(3), 389406.
Table 1: Evaluation of the QC procedure using total rainfall accumulations. The
criterion is the relative difference between accumulations from radar and rain
gauge, i.e. bias in percent.

Distance from Radar (km)

0200

050

50100

100150

150200

Accumulation from gauge (mm)

375

391

427

356

336

Bias before QC

26

34

9

32

35

Bias after QC

14

12

5

20

26

LIST OF FIGURES
Fig. 1. Histogram of the radar reflectivity data used in the study. The dark shaded
region corresponds to the data after QC.
Fig. 2. Probability of detection (POD) before QC for different thresholds (from 0 dBZ to 20 dBZ) based on 10,000 volume scans collected by the Tulsa, Oklahoma WSR88D between 1994 and 1995. The rings, centered on the radar, denote 100 and 200 km ranges.
Fig. 3. Same as Figure 1, but after QC.
Fig. 4. Histogram of POD based on the 360×200 polar pixels of the radar domain.
Fig. 5. Probability of detection conditional on the gauge measurements, P(Z > 10 dBZ  R > 0.1 MM).
Fig. 6. Probability of false detection conditional on the gauge measurements, P(Z > 10 dBZ  R > 0.1 MM).
Fig. 7. Total rainfall accumulation interpolated from the Oklahoma Mesonet rain gauge data and from radar, both before QC and after QC. Dots denote the gauge locations; rings are every 100 km from radar.
Fig. 8. Bias between radar and gauge total rainfall accumulation, considering radar data before and after QC. Dots denote the gauge locations; rings are every 100 km from the Tulsa WSR88D radar.
Fig. 1. Histogram of the radar reflectivity data used in the study. The dark shaded region corresponds to the data after QC.
Fig. 2. Probability of detection (POD) before QC for different thresholds (from 0 dBZ to 20 dBZ) based on 10,000 volume scans collected by the Tulsa, Oklahoma WSR88D between 1994 and 1995. The rings, centered on the radar, denote 100 and 200 km ranges.
Fig. 3. Same as Figure 2, but after QC.
Fig. 4. Histogram of POD based on the 360×200 polar pixels of the radar domain.
Fig. 5. Probability of detection conditional on the gauge measurements, P(Z > 10 dBZ  R > 0.1 mm).
Fig. 6. Probability of false detection conditional on the gauge measurements, P(Z > 10 dBZ  R > 0.1 mm).
Fig. 7. Total rainfall accumulation interpolated from the Oklahoma Mesonet rain gauge data and from radar, both before QC and after QC. Dots denote the gauge locations; rings are every 100 km from radar.
Fig. 8. Bias between radar and gauge total rainfall accumulation, considering radar data before and after QC. Dots denote the gauge locations; rings are every 100 km from the Tulsa WSR88D radar.

