1. Evaluation of Sterling, Virginia WSR-88D (KLWX) Rainfall Estimates Post-Calibration
1.1 Introduction During the period February 14-17, 2000 a team from the WSR-88D Operational Support Facility (OSF) visited the Sterling, Virginia WSR-88D radar site (KLWX) to perform newly developed and enhanced calibration procedures to improve the radar measurements of reflectivity and resulting derived products such as rainfall and VIL. This effort was motivated in part by the recent poor performance of the KLWX rainfall products relative to rain gauges for the Hurricane Floyd heavy rain event of 14-17 September 1999 where significant radar rainfall underestimation was typical (NWS 2000). This section describes the quantitative validation effort performed by the Hydrologic Research Laboratory (HRL) in cooperation with the OSF Applications Branch (Tim O'Bannon), Sterling Weather Forecast Office (SOO Steve Zubrick), and Eastern Region Scientific Services Division (Julie Gaddy) to compare radar rainfall estimates from the WSR-88D Precipitation Processing System (PPS) rainfall algorithm with corresponding rain gauge data to evaluate the integrity of the reflectivity measurements after the new calibration procedures were performed at LWX. 1.2 Calibration Results Details of the enhanced calibration procedures can be obtained from the OSF Engineering Branch. Briefly, the new procedures aim to calibrate the Internal Noise Source of the radar receiver, something which has previously been difficult to do, which then permits accurate calibration of the antenna gain using solar calibration procedures. Power measurement errors can also be corrected. Beta testing of the new procedures was performed at four WSR-88D radars in 1999, and in all cases the adjustments derived from the procedures resulted in an increase in the derived rainfall estimates (i.e., a negative reflectivity bias (dB) existed). Comparisons with a limited rain gauge dataset demonstrated improvements relative to the gauge measurements after the procedures were run. Prior to the February OSF visit to Sterling, HRL's experience in casual monitoring of LWX radar rainfall estimates relative to rain gauges and quantitative runs of the Stages II and III Precipitation Processing algorithms (which compute real-time gauge-radar mean field biases) for the Mid-Atlantic River Forecast Center region at HRL indicated that the Sterling radar was underestimating by a factor of about two compared to rain gauges. Such a G/R bias of 2.0 implies a reflectivity measurement that is too low by 4.2 dB if using the convective Z-R relation (Z=300 R1.4) and 3.6 dB if using the tropical relation (Z=250 R1.2) for example. During the Sterling calibration in mid-February, the OSF determined a net reflectivity calibration correction of +4.3 dB was needed. This is in approximate agreement with our experience of radar rainfall underestimation prior to that date. 1.3 Gauge-Radar Analysis Procedures Storm-total rainfall estimates from the WSR-88D PPS algorithm were compared with corresponding operational IFLOWS (Integrated Flood Observing and Warning System) rain gauge measurements for all rainfall events affecting the Maryland, northern Virginia, eastern West Virginia, and southern Pennsylvania regions, i.e., those areas within 230 km of KLWX. Table 1 lists the rain events analyzed in this study. Table 1. List of the ending dates of the 25 analyzed rain events. The dominant type of rainfall (stratiform vs. convective) is also listed as well as the particular Z-R relation being used by the PPS algorithm for that event. 3/17/2000 stratiform Z=130 R2.0 3/22 stratiform Z=130 R2.0 3/26 convective Z=300 R1.4 3/28 stratiform Z=300 R1.4 4/4 stratiform Z=200 R1.6 4/9 stratiform Z=200 R1.6 4/19 stratiform Z=200 R1.6 4/23 convective Z=300 R1.4 4/26 stratiform Z=300 R1.4 5/6 convective Z=300 R1.4 5/11 convective Z=300 R1.4 5/14 convective Z=300 R1.4 5/21 stratiform Z=300 R1.4 5/23 stratiform Z=300 R1.4 5/24 stratiform Z=300 R1.4 5/25 convective Z=300 R1.4 5/28 stratiform Z=300 R1.4 5/29 stratiform Z=300 R1.4 6/3 convective Z=300 R1.4 6/7 stratiform Z=300 R1.4 6/14 convective Z=300 R1.4 6/18 convective Z=300 R1.4 6/20 convective Z=300 R1.4 6/22 convective Z=300 R1.4 6/26 convective Z=300 R1.4
Storm-total rainfall estimates from the PPS algorithm running in real-time on the Warning Decision Support System (WDSS) computer at the Sterling forecast office were used in this study. Permission to FTP into the WDSS computer via the Internet was provided by Steve Zubrick, and an automated Unix shell script was set up on HRL computers to transfer the PPS's HYPROD.DAT and PPS.INPUT files once per hour at the top of every hour around the clock. The first file contains the digital storm-total rainfall array used by the PPS to produce the PUP's graphic Storm-Total Precipitation (STP) product. This digital array is a polar-gridded rainfall product with spatial resolution of 1 deg in azimuth by 2 km in range out to 230 km. The data resolution is in hundredths of inches of rainfall converted from the internal resolution within the PPS algorithm of 0.1 mm (see Fulton et. al., 1998 for details). The second file, PPS.INPUT, containing all of the PPS adaptable parameters, was also FTP'ed and archived so that the parameter settings in use on the WDSS were documented and saved for each event. Arguably the most important adaptable parameters for the PPS are the two Z-R coefficients (multiplicative factor, A, and exponent, b, in Z=A Rb) and the maximum rain rate threshold (hail cap threshold) above which rain rates, or equivalently reflectivity, are capped. The Sterling WFO varied the Z-R coefficients between three OSF-defined relations for the rain events (see Table 1): 1) Convective (Z=300 R1.4), 2) Marshall-Palmer Stratiform (Z=200 R1.6), and 3) Eastern U.S. Winter Stratiform (Z=130 R2.0). The maximum rain rate threshold remained unchanged for all events at a fixed value of 103.8 mm hr-1 which corresponds to 53 dBZ if the convective Z-R relationship is used. The decision was made to access the PPS's HYPROD.DAT file as run on the semi-operational WDSS computer instead of the operational Concurrent RPG because FTP access is not available on this closed system. To do these gauge-radar analyses using RPG rainfall products would have required the manual dumping of the WSR-88D HYPROD.DAT files onto SCSI tapes and shipment to HRL for analysis at the end of each rain event, something which is neither practical nor efficient. An alternative option of accessing via FTP the HYPROD.DAT file from the existing Open RPG running in a beta-test operational mode in the Silver Spring Metro Center at NWS Headquarters using a real-time base data input feed from KLWX was investigated as well, however attempts to decode the ORPG's HYPROD.DAT file using existing software at HRL was not successful. Therefore the use of the PPS output files from the Sterling WDSS was deemed the most practical and easiest approach. The WDSS is running the Build 9 version of the PPS which is not the most current version; however, changes to the software in the current Build 10 were not deemed to impact this quantitative analysis. "Storm-total" rainfall estimates were defined as the rainfall estimates from the time when
the PPS first started accumulating rainfall until just prior to when the internal rainfall arrays are re-initialized back to zeros an hour after the end of the event. This time period is recorded in the header data within the HYPROD.DAT file and is determined by the WSR-88D Precipitation Detection Function (PDF). Two of the three relevant PDF parameters for the "Significant Precipitation" category (Precipitation Rate Threshold, and Nominal Clutter area) are included in the PPS.INPUT file (the third relevant parameter, Precipitation Area Threshold, is not listed in the PPS.INPUT file and is assumed here to default to 0 km2 in the WDSS implementation of the PDF/PPS). The setting of these parameters determines when rainfall is first detected and thus when the PPS starts to accumulate rainfall from an initial zero-valued array. For all rainfall events, these parameters were set to reasonable values of -2.0 dBR and 50 km2. With these settings, the PPS starts accumulating rainfall when reflectivity echo (or alternatively rain rate assuming some given Z-R relationship) in any of the four lowest elevation angles exceeds -2.0 dBR (22.0 dBZ assuming the convective Z-R relation) when averaged over an area of 50 km2. The rainfall event ends (i.e., the rainfall arrays are reset) when an hour without reflectivity echo of this area and intensity passes within the radar scanning domain.
Rain gauge data were obtained in near-real-time from all IFLOWS gauges within 230 km range of KLWX. As many as 238 gauges reported for any given event, with typically around 190 gauges reporting. Many of these gauges were located in the mountainous western half of the radar scanning domain, and none were located in the azimuth sector from about 120 to 220 degrees. IFLOWS gauges are tipping bucket gauges with 1 mm (0.04 inches) resolution. The individual tips are accumulated into 15-minute accumulations by the IFLOWS data processing software at each of several collection sites. Other rain gauge data such as from ASOS or cooperative observers were not utilized due to the large amount of manual work needed to access and decode/reformat the data for input into the analysis routines. The IFLOWS data was easily accessible in digital form using automated data transfer procedures. Originally when this evaluation project was started, the near-real-time IFLOWS gauge data was downloaded as ASCII listings from the IFLOWS web site (http://www.afws.net/ and, for example for Maryland, at the end of each rain event, but a fair amount of manual intervention was necessary to make the listings useable by the analysis programs. In order to lessen this workload, I contacted the IFLOWS data manager, John Bollinger, in Kentucky, and he graciously agreed to provide the 15-minute IFLOWS gauge data in digital form suitable for FTP access on a daily basis for all gauges in Maryland, Virginia, West Virginia, and Pennsylvania. A decoding and processing program (prepiflows15.f) was written to decode, sum over the rain event, and format the gauge data for input into the gauge-radar analysis program. In addition, I obtained the IFLOWS Master Listing file containing metadata such as the gauge IDs, latitude/longitude locations, and altitude from Stan Campbell with the IFLOWS program. Another program (azraniflows.f) was written to compute the range and azimuth of each gauge relative to the LWX radar and extract all gauges that fall within 230 km of the radar. A third program (gagradscat.f) is the main gage-radar analysis program. It decodes the HYPROD.DAT file to extract the storm-total rainfall array, and then the single polar grid bin of rainfall overlying each gauge was paired with the corresponding gauge measurement knowing the latitude and longitude (and therefore range/azimuth) of each gauge as determined from the azraniflows.f program. This information was then read into MATLAB, a commercial data visualization program, and plotted using a program script. The MATLAB script performs an additional quality-control step to remove gauge-radar pairs for which the gauge measured no rain and the radar measured rain. This is to remove gauge measurements that are likely bad due to a clogged funnel or other mechanical gauge problems. No attempt is made to search neighboring polar radar bins surrounding the bin overlying the gauge in order to find a "better" match with the gauge. Because a single polar grid bin is paired with the gage measurement regardless of radar range, one can expect increased gauge-radar scatter for the gauges at far range compared to close-in gauges due to the much larger beam width at far ranges compared with the fixed, relatively-tiny gauge orifice size. This is the long-standing sampling issue between gauge and radar. No other manual gauge quality control was peformed.
After analyzing a few rain events in March, it became obvious that there was a need to isolate gauge-radar pairs for which the radar beam was near or above the melting level in the mixed phase or ice region from those in which the beam was sensing liquid drops. Often the rain gauges at the farthest ranges were reporting precipitation while the radar estimated little to no precipitation due to overshooting and/or beam widening effects (Fulton et al., 1998). This often caused an alignment of points along the gauge rainfall axis (with radar rainfall values of 0.0 mm)in gauge-radar scatter plots. Additionally bright band contamination (overestimation) was common particularly in the cool events with low freezing levels which caused gauge-radar points to appear near the other extreme in the plots. The bright band contamination and range degradation effects of the radar estimates always showed up clearly in the storm-total radar rainfall images, and this information was used to identify whether the event was predominantly stratiform or convective (see Table 1). Because inclusion of such gauge-radar pairs would significantly bias the results, an additional quality control procedure was implemented using atmospheric sounding data. In order to condition the gauge-radar analyses based on whether the radar beam above a gauge was sensing liquid or otherwise, the observed atmospheric sounding data for the Dulles International Airport (IAD) sounding site, located coincident with the KLWX radar, was obtained for all rain events from the NOAA Forecast Systems Laboratory (FSL) web site (http://www.fsl.noaa.gov/fsl/docs/data/fsl-data.html The height of the 0 deg C level was determined and averaged from each observed sounding between the time the rain event started and ended based on the PPS start and end times recorded in the WSR-88D's HYPROD.DAT file. Since some rain events lasted several days, the actual freezing level height measured at the fixed 00 or 12 UTC times may have varied from the mean value over that period. In addition to assuming a fixed, mean freezing level height over time, an assumption is made that the freezing level height is constant in space over the 230 range of the radar. Although both of these assumptions are clearly not realistic beyond first order, attempting to account for spatial and temporal changes in the freezing level height in these gauge-radar comparisons would have added significantly to the complexity of the analysis programs and probably not changed the conclusions appreciably. The height of the radar beam was computed using standard equations found in Doviak and Zrnic (1993) assuming fixed beam elevation angles of 0.5, 1.5, 2.4 and 3.4 degrees and standard atmosphere beam refraction. The static terrain-based hybrid scan sector file for the KLWX radar was decoded and used to determine what elevation angle was used by the PPS algorithm to compute rainfall for each 1 deg x 2 km polar grid bin. Knowing the elevation and azimuth angles and range, the height above mean sea level for each polar grid bin was computed within the gagradscat.f program and compared with the observed mean height of the melting level from the sounding data. Gauge-radar pairs were flagged according to whether the TOP of the radar beam above the gauge (obtained by adding 0.5 degrees to the elevation angles given above) was above or below this melting level. Pairs identified as lying below the melting level were termed "QC'ed" gauge-radar pairs implying they were quality controlled based on the melting level height where the radar was likely sensing liquid rain drops. Several simple statistics were computed for the gauge-radar pairs. The mean gauge-radar ratio (G/R) is defined as: where the summations are over all storm-total gauge-radar pairs for a given rainfall event. The correlation coefficient between gauge and radar rainfall estimates was also computed. The pairs were either all aggregated together regardless of freezing level height or conditioned on the freezing level height above the gauge relative to the top of the beam overlying that gauge that provided the reflectivity data to compute the rainfall estimates. Also, the gauge-radar pairs were stratified into three equal-width range intervals (0-77 km; 77-153 km; 153-230 km) in order to evaluate the contribution of well-known range degradation effects on the radar estimates. 1.4 Overall Results For All Events All gauge-radar storm-total rainfall pairs for 23 of the 25 rain events spanning about three months were combined together so that general conclusions could be drawn(1). Individual events will be presented in the Section V. Scatter plots of gauge vs. radar estimates are shown in Figure 1. These plots only include the QC'ed gauge-radar pairs, i.e., the ones for which the top of the radar beam above the gauge was below the 0 deg C level. These gauges are necessarily the ones at the innermost ranges of the radar scanning domain. Figure 1a combines all events together while Figs. 1b and 1c are plots for just the convective and stratiform events, respectively. The large scatter in these plots is disappointing but not unexpected given the variety of error sources associated with estimating rainfall from reflectivity measurements as well as errors in gauge measurement of rainfall. The mean gauge-radar ratio G/R is 0.82, or a radar overestimation by 22% on average. However, any given location for a given rainfall event can be either significantly over- or underestimated as illustrated in the figure. Keep in mind that these results depend critically upon which particular Z-R relation was used in generating the rainfall estimates (this varies depending on the event; see Table 1). Use of other relations would result in different statistics. For all convective events, the Convective Z-R relation (Z=300 R1.4) was used in the PPS. However, the Eastern U.S. Winter Stratiform (Z=130 R2.0), the Marshall-Palmer Stratiform (Z=200 R1.6), as well as the Convective Z-R relations were all used for at least one of the events defined as stratiform. Evaluation of the sensitivity to the choice of Z-R relation is being pursued for a sample of these events, but results are not yet available. Figure 1b presents results for the 11 convective events. It is clear that radar overestimation is common. The mean G/R ratio of 0.58 translates to radar overestimation relative to the gauges by a factor of 1.7 (=0.58-1). As an illustration, this bias would translate to a reflectivity measurement bias of +3.3 dB, i.e., a radar that is too "hot" by 3.3 dB, assuming use of the Convective Z-R relation, for example, and assuming that all of this mean gauge-radar bias can be attributed solely to reflectivity calibration error. The stratiform events are included in the scatter plot of Fig. 1c. There are more widely scattered pairs (with a correspondingly lower correlation coefficient) compared to the convective events in Fig. 1b. The G/R ratio is 1.10 implying radar estimates that are underestimated by a factor of 0.9 on average compared to the rain gauges. Note that there is clear dependence of radar performance (with respect to rainfall estimates) on the type of rainfall, convective vs. stratiform. These results show that the LWX radar tends to overestimate for convective events and underestimate for stratiform events. These results could change, however, if different Z-R relations were used. Since the "right" Z-R relation is not known a priori for each event, it is not possible to easily separate out the influence of reflectivity calibration errors from errors associated with use of an inappropriate Z-R relation. The Convective Z-R relation may actually perform better than one of the stratiform relations for a stratiform event, and vice-versa. Figure 1a. Gauge-radar scatter plot of storm-total rainfall for 11 convective and 12 stratiform events. Only QC'ed pairs are included here. Figure 1b. Gauge-radar scatter plot of storm-total rainfall for 11 convective events. Only QC'ed pairs are included here. Figure 1c. Gauge-radar scatter plot of storm-total rainfall for 12 stratiform events. Only QC'ed pairs are included here. 1.5 Individual Event Results The results for each of the 25 rainfall events is presented in this section with figures and tables specific to each case. Table 2 summarizes various information for each event such as the gauge-radar ratio, rain type, number of gauge-radar pairs, etc. Figs. 2a-y show the plots for each event. Table 2. List of rainfall events along with the dominant rain type (S stratiform or C convective), mean freezing level (FZL) height, number of gauge-radar pairs, G/R ratio and correlation coefficient for the three cases of a) all pairs, b) QC'ed pairs, and c) all pairs separated into three range intervals (inner, middle, outer). Asterisked freezing level heights are conservative (low) estimates when sounding data was unavailable.
Figure 2 (to follow over the coming pages). Storm-total radar rainfall (mm), contoured IFLOWS gauge rainfall (mm), gauge-radar scatter plot for all gauges and segregated by range, QC'ed gauge-radar scatter plot, location of IFLOWS gauges relative to LWX radar (bold dots indicate QC'ed gauges), and range dependence of gauge and radar rainfall derived from available gauge-radar pairs stratified into 23 km-wide range bands. The start and end dates and times of the event is listed at the top of the third and fourth panels.
Figure 2a. March 13, 2000
Figure 2b. March 22, 2000
Figure 2c. March 26, 2000.
Figure 2d. March 28, 2000
Figure 2e. April 4, 2000
Figure 2f. April 9, 2000. No QC'ed image is generated since no QC'ed pairs existed.
Figure 2g. April 19, 2000.
Figure 2h. April 23, 2000.
Figure 2i. April 26, 2000.
Figure 2j. May 6, 2000.
Figure 2k. May 11, 2000.
Figure 2l. May 14, 2000.
Figure 2m. May 21, 2000.
Figure 2n. May 23, 2000.
Figure 2o. May 24, 2000.
Figure 2p. May 25, 2000.
Figure 2q. May 28, 2000.
Figure 2r. May 29, 2000.
Figure 2s. June 3, 2000.
Figure 2t. June 7, 2000.
Figure 2u. June 14, 2000.
Figure 2v. June 18, 2000.
Figure 2w. June 20, 2000.
Figure 2x. June 22, 2000.
Figure 2y. June 26, 2000.
References Doviak, R. and D. Zrnic, 1993: Doppler Radar and Weather Observations. Academic Press, Inc. United States, 562 pp. Fulton, R., J. Breidenbach, D.-J. Seo, D. Miller, and T. O'Bannon, 1998: The WSR-88D Rainfall Algorithm. Wea. Forecasting, 13, 377-395. National Weather Service, 2000: Hurricane Floyd Floods of September 1999. Service Assessment. Available from NWS Office of Hydrology, 1325 East-West Highway, Silver Spring, MD 20910. 28 pp.1. Two rain events were excluded: 26 April and 22 June. The first event had severe bright band contamination, and the second event had problems with widespread anomalous propagation contamination of the rainfall estimates. Both cases produced gauge-radar pairs that were obvious outliers, and therefore these cases were excluded from further analyses. |
Main Link Categories: Home | OHD | NWS |