7. Evaluation of anomalous propagation echo detection in WSR-88D data: a large sample case studyWitold F. Krajewski and Bertrand Vignal Iowa Institute of Hydraulic Research Technical Note submitted to Corresponding author: |
|||||||||||||||||||||||||||||||||||||||
|
Abstract We evaluate a method of detecting anomalous propagation echo in volume scan radar reflectivity data. The method is based on a neural network approach and is suitable for operational implementation. It performs a classification of the base scan data on a pixel-by-pixel basis into two classes: rain and no-rain. We describe the results of applying the method to a large sample of WSR-88D level II archive data. The data consist of over 10,000 volume scans collected in 1994 and 1995 by the Tulsa, Oklahoma WSR-88D. Our evaluation includes analyses based on radar data only and on various comparisons of radar and rain gauge data. The rain gauge data are from the Oklahoma Mesonet. The results clearly show the effectiveness of the procedure as indicated by reduced bias in rainfall accumulation and improved behavior in other statistics. 7.1 Introduction One can safely say that quality control (QC) of radar reflectivity data is the most important step in the overall process of radar rainfall estimation. The crucial aspect of such QC lies in the detection of ground clutter and echoes caused by anomalous propagation (AP) of radar waves (Moszkowicz et al. 1994). Recently, Grecu and Krajewski (2000) proposed a method of AP detection based on a neural network approach. The method classifies the base scan radar reflectivity data into rain or no-rain echo on a pixel-by-pixel basis. With this method, several characteristics of the reflectivity field are computed in the neighborhood of the pixel under investigation. A trained (i.e. calibrated) neural network uses these characteristics as inputs and performs the classification. A unique aspect of the method is the selection of the training data set required by the neural network approach. The authors advocate selecting only the "clear cut" cases, both for rain and no-rain echo. This allows fast and efficient preparation of the training sample, and thus, rapid implementation of the methodology. The selected set is used for random drawing of the training and validation samples. Thus, the training and the validation do not include the challenging cases where AP and rain might be co-located. The Grecu and Krajewski (2000) methodology includes self-evaluation through repeated re-sampling and cross-validation. Performance was monitored in terms of the number of misclassified pixels. In this short communication, we expand the evaluation methodology. We include several analyses based on radar data only, as well as various comparisons with rain gauge observations. 7.2 Summary of the data We used the same, but somewhat expanded database as Grecu and Krajewski (2000). The data were collected by the Tulsa, Oklahoma, Weather Surveillance Radar-1988 Doppler version (WSR-88D). In Oklahoma, the rainfall regime is dominated by mid-latitude convective systems (Houze et al. 1990). The data cover mostly the warm season months of 1994 and 1995. Since the study of Grecu and Krajewski (2000), we filled several gaps in the radar data, and as a result had available over 10,000 volume scans (see Figure 1 for the histogram). These radar data were converted from the Archive level II format (Klazura and Imy 1993) to the efficient format ASCII-RLE (Kruger and Krajewski 1997) allowing the rapid access required for such a large sample study. The rain gauge data we used are from the Oklahoma Mesonet (Brock et al. 1995). Some 49 rain gauges are located within the Tulsa radar domain (Figure 7). 7.3 Results We used the neural network trained by Grecu and Krajewski (2000), applying the same network configuration for the entire data set. Our analyses are divided into two parts: that based on radar data only and that based on both radar and rain gauge observations. a. Radar-only analyses First, let us consider the probability of detection (POD) of an echo stronger than a certain threshold, calculated on a pixel-by-pixel basis. Clearly, the possible values range from 0, if no echo is ever detected at the given pixel (this may happen if the view of the pixel is completely blocked by an object such as a building or mountain), to 1, if an echo is always detected (as in the case of reflections off a mountain). If only rain-caused echoes were detected, the expected range of values would be around 0.05 but the exact numbers are unknown. Furthermore, if the rainfall under the radar umbrella were climatologically and statistically homogeneous, and we had a sufficiently large sample, the range of values of the POD would be very narrow a single spike in the limit. How does the POD pattern look for the Tulsa WSR-88D? Figures 2 and 3 provide the answer. In Figure 2 we show the effect of various thresholds (T) on the pattern of POD. Clearly, for T=0 dBZ the pattern displays circular artifacts which result from a combination of ground clutter effects (near the radar), and AP further out. The slight shift of the pattern towards the southeast reflects the rainfall climatology of the region. As the threshold increases, the POD pattern becomes more uniform. This is because by using thresholds, we eliminate much of the ground clutter and AP. However, we also eliminate some of the rainfall. For example, simple back-of-the-envelope calculations indicate that for T=20 dBZ we may be cutting as much as 10% of the area-averaged rainfall accumulation. Thus, while using thresholds may be considered the simplest QC method, it introduces the risk of eliminating climatologically significant rainfall. Now let us compare the corresponding patterns of POD calculated from quality controlled data (Figure 3). Clearly, the POD is now consistently lower and the patterns more uniform. If, in addition, we consider the corresponding histograms for both sets of the POD patterns (Figure 4), it becomes clear that the applied QC is more effective than simple thresholds. For the quality-controlled data, there is little effect of applying thresholds on the histogram of POD, which indicates that most false echoes were removed by the QC procedure. b. Radar and rain gauge analysis Our analysis now includes hourly estimates of rainfall calculated by applying the "standard" NEXRAD Z-R relationship Z=300R1.4 to the base scan data (antenna elevation angle of 0.47º). We also use hourly rain gauge data from the Oklahoma Mesonet. First, consider the conditional probability that radar observes reflectivity (Z) greater than 10 dBZ given that a co-located rain gauge observes measurable rainfall (R>0.1 mm). This statistic is useful as the study by Grecu and Krajewski (2000) did not address the performance of AP detection in the presence of rain. Theoretically, the probability P(Z > 10dBZ | R > 0.1 mm) should be high, but an AP procedure that eliminates too much rain would decrease it below the true (but unknown) level. As evident from Figure 6, the quality-controlled data results in the conditional probability more uniformly distributed along the distance from the radar, in line with data at far ranges where the QC procedure does little. The effect of quality control is much more dramatic if we consider the conditional probability that radar observes significant echo given that rain gauges observe no rainfall, i.e. P(Z > 10dBZ | R < 0.1 mm). Theoretically, this probability should be only slightly greater than zero as radar "sees" larger areas. Indeed, as we show in Figure 7, this probability seems reasonable for the quality-controlled data, but seems too high for the original data. Most importantly, this probability is independent of the distance from radar-a strong indication of the effectiveness of the QC procedure. Finally, consider the total rainfall accumulation both from the rain gauges and the radar. We calculated the field of rainfall accumulation for the entire duration of our data set by interpolating between the locations of the gauges. The interpolation method is based on four quadrant near-neighbors and inverse distance weights-it is the same method used by the National Weather Service in its hydrologic forecasting. We applied it to both the radar and the rain gauge fields (Figure 8). We show the accumulated field within the boundaries of the state of Oklahoma only to avoid the possible inconsistency of using a rain gauge data set outside the Oklahoma Mesonet. To make the comparison easier, we also show a map of the bias-defined as difference between the two-interpolated the same way. It is clear that the accumulation field displays lower bias when calculated on the quality controlled data. We summarize the quantitative results in Table 1. The table shows that the effect of the QC procedure is smallest at distances higher than 100-150 kilometers from the radar. This is understandable, as the influence of AP diminishes farther out from the radar. It is the non-uniformity of the vertical profile of reflectivity that is mostly responsible for the discrepancy at those far ranges (Vignal et al. 1999). 7.4 Conclusions The QC procedure developed by Grecu and Krajewski (2000) for detecting AP signal in WSR-88D data proved very effective when applied to the Tulsa, Oklahoma radar. The agreement of long-term accumulation between the radar and rain gauge data is remarkable, especially considering no fitting of the Z-R or any other parameters of the rainfall estimation algorithm. The algorithm we applied is the simplest it can be-it does not include rainfall classification, vertical integration, or pattern extrapolation-all elements that improve radar rainfall estimation as shown in previous studies (e.g. Ciach et al. 1997; Anagnostou and Krajewski 1999). We used the standard Z-R relationship that proved adequate. Perhaps the most important aspect of our study is that we used a large sample of volume scan reflectivity data and mimicked an operational environment to the degree possible. We are convinced that, due to the highly variable nature of rainfall processes, large sample studies are critically important in evaluating new technologies of radar rainfall estimation, including the "solve-it-all" polarimetric methods (e.g. Zrnic and Ryzkov 1999). The traditional mode of testing radar methods using only a few selected events is simply misguided. Many statistics calculated from radar and rain gauge observations, e.g. bias, are meaningful and/or statistically stable only for accumulations over a long period. We have assembled large samples of radar reflectivity data for several WSR-88D locations, including Davenport, Iowa, Grand Forks, North Dakota, Memphis, Tennessee, and Melbourne, Florida. We are planning to use these data sets for a variety of comparative studies and analyses, and we invite collaboration. Acknowledgements. This study was supported by the National Weather Service Office of Hydrology under Cooperative Agreement with the Iowa Institute of Hydraulic Research (NA47WH0495). We would like to thank Dr. Mircea Grecu for his assistance with the computer code and Dr. Grzegorz Ciach for many helpful discussions. References Anagnostou, E.N. and W.F. Krajewski, Real-time radar rainfall estimation. Part 1: algorithm formulation, Journal of Atmospheric and Oceanic Technology, 16(2), 189-197, 1999. Brock, F.V., K.C. Crawford, R.L. Elliot, G.W. Cuperus, S.J. Stadler, H.L. Johnson and M.D. Eilts: The Oklahoma Mesonet A technical overview, Journal of Atmospheric and Oceanic Technology, 12(1), 5-19. , 1995 Ciach, G.J., W.F. Krajewski, E.N. Anagnostou, J.R. McCollum, M.L. Baeck, J.A. Smith, and A. Kruger: Radar Rainfall Estimation for Ground Validation Studies of the Tropical Rainfall Measuring Mission, Journal of Applied Meteorology, 36(6), 735-747, 1997. Grecu, M. and W.F. Krajewski: An efficient methodology for detection of anomalous propagation echoes in radar reflectivity data using neural networks, Journal of Oceanic and Atmospheric Technology, 17(2), 121-129, 2000. Houze Jr., R.A., Smull, B.F. and P. Dodge: Mesoscale organization of springtime rainstorms in Oklahoma. Monthly Weather Rev., 117, 613-654, 1990. Kruger A. and W.F. Krajewski: Efficient storage of weather radar data. Software Practice and Experience, 27, 623-635, 1997. Klazura G.E. and D.A. Imy: A description of the initial set of analysis products available from the NEXRAD WSR-88D system. Bull. Amer. Meteor. Soc., 74(7), 1293-1311, 1993. Moszkowicz, S., G.J. Ciach, and W.F. Krajewski, Statistical detection of anomalous propagation in radar reflectivity patterns, Journal of Atmospheric and Oceanic Technology, 11(4), 1026-1034, 1994. Vignal B., Andrieu H. and J.D. Creutin, 1999: Identification of vertical profiles of reflectivity from voluminal radar data. J. Appl. Meteor., 38(8), 1214-1228. Zrnic, D.S., and A.V. Ryzhkov, 1999: Polarimetry for weather surveillance radars.Bull. Amer. Meteor. Soc., 80(3), 389-406. Table 1: Evaluation of the QC procedure using total rainfall accumulations. The criterion is the relative difference between accumulations from radar and rain gauge, i.e. bias in percent.
Fig. 1. Histogram of the radar reflectivity data used in the study. The dark shaded region corresponds to the data after QC. Fig. 2. Probability of detection (POD) before QC for different thresholds (from 0 dBZ to 20 dBZ) based on 10,000 volume scans collected by the Tulsa, Oklahoma WSR-88D between 1994 and 1995. The rings, centered on the radar, denote 100 and 200 km ranges. Fig. 3. Same as Figure 1, but after QC. Fig. 4. Histogram of POD based on the 360×200 polar pixels of the radar domain. Fig. 5. Probability of detection conditional on the gauge measurements, P(Z > 10 dBZ | R > 0.1 MM). Fig. 6. Probability of false detection conditional on the gauge measurements, P(Z > 10 dBZ | R > 0.1 MM). Fig. 7. Total rainfall accumulation interpolated from the Oklahoma Mesonet rain gauge data and from radar, both before QC and after QC. Dots denote the gauge locations; rings are every 100 km from radar. Fig. 8. Bias between radar and gauge total rainfall accumulation, considering radar data before and after QC. Dots denote the gauge locations; rings are every 100 km from the Tulsa WSR-88D radar.  Fig. 1. Histogram of the radar reflectivity data used in the study. The dark shaded region corresponds to the data after QC.
 Fig. 2. Probability of detection (POD) before QC for different thresholds (from 0 dBZ to 20 dBZ) based on 10,000 volume scans collected by the Tulsa, Oklahoma WSR-88D between 1994 and 1995. The rings, centered on the radar, denote 100 and 200 km ranges.
 Fig. 3. Same as Figure 2, but after QC.            
 Fig. 8. Bias between radar and gauge total rainfall accumulation, considering radar data before and after QC. Dots denote the gauge locations; rings are every 100 km from the Tulsa WSR-88D radar. |
|||||||||||||||||||||||||||||||||||||||
|
Main Link Categories: Home | OHD | NWS |
Follow us on Twitter
Follow us on Facebook
Follow us on YouTube
OWP RSS Feed