NAM-CMAQ Ozone Forecast Verification for the Summer of 2005 at Knoxville, Tennessee

I. Introduction

In the summer of 2004, the National Weather Service (NWS) launched experimental ozone forecasts for the northeastern quadrant of the Continental United States (CONUS). The domain of this experimental ozone forecast capability encompassed most of Tennessee. In the summer of 2005, this domain was expanded to include the entire eastern part of the CONUS through the Mississippi Valley. The portion of the domain that had been in development during 2004, and experimentally tested in 2005 was approved for operational deployment in August, 2005. Thus, all of Tennessee is now part of the operational domain (Davidson 2005).

The National Centers for Environmental Prediction's (NCEP) North American Mesoscale (NAM) and Community Multi-scale Air Quality (CMAQ) models form the backbone of the new NOAA-EPA air quality forecast capability. The NCEP runs the linked model system to provide the ozone forecast guidance data to state and local air quality forecasters and the public. State and local air quality forecasters issue the official air quality forecasts for specific regions of their respective states: some 300 cities nationwide issue air quality alerts based on ozone forecasts, and about 100 of them include information on airborne particulate matter. State air quality forecasters have historically used statistical models to provide air quality forecasts. These statistical approaches relate forecasted temperatures and sky cover to forecasted ambient ozone levels. (Personal communication with State of Tennessee air quality forecasters.)

As described by Otte, et al., (2005) the CMAQ model is coupled to the NAM model (12 km grid spacing) to provide forecast guidance for ground-level ozone. The NAM model provides forecast meteorological parameters to the air quality modules PREMAQ and CMAQ, including temperature, winds, mixing heights, and cloud cover. A pre-processor, called PREMAQ, modifies estimated emissions (based on EPA?s National Emission Inventory) for weather-dependence and, with other interface processors, converts the NAM output to a form that can be used by the chemical transport model, CMAQ. The CMAQ then performs atmospheric reactive chemical transport simulations, which are used to predict ozone concentrations (McQueen et al. 2005).

The ground-level ozone is commonly formed when nitrogen dioxide (NO₂), which is one of several nitrogen oxide species collectively called NO_x, photo-dissociates into nitric oxide (NO) and atomic oxygen (O) (Cooper and Alley 2002). The resulting atomic oxygen combines with molecular oxygen (O₂) to form ozone (O₃). However, the NO that remains from the initial photo-dissociation immediately reacts with ambient O₃ to reform NO₂ and O₂. Thus, there is no net ozone production from the photo-dissociation of NO₂. Volatile Organic Compounds (VOCs) play a significant role in ground-level ozone production in that atmospheric decomposition of VOCs produces peroxy radicals (RO₂ and HO₂). These peroxy radicals oxidize NO (another component of NO_x) into NO₂, resulting in more NO₂ for O₃ formation and less NO for O₃ depletion (Davidson, 2003). Relatively high ambient levels of ground-level ozone are considered a human health hazard in that O₃ causes inflammation of lung tissues. The Environmental Protection Agency (EPA) has set a maximum National Ambient Air Quality Standard (NAAQS) of 0.08 parts per million (ppm) 8-hour average (U.S. Environmental Protection Agency National Ambient Air Quality Standards). A maximum 1-hour value of 0.12 ppm was used prior to implementation of the 8-hour average, and is still used for areas classified as Early Action Compact (EAC) areas (EPA Green Book).

A study was conducted for the 2005 ozone season, which is nominally between May 1 and September 30, to assess performance of the NAM-CMAQ ozone forecast guidance. The city of Knoxville, Tennessee, was used as the forecast point. The NAM-CMAQ ozone predictions for Knoxville were compared to observed data for the corresponding days. Maximum one-hour and eight-hour average ozone forecast guidance concentrations were used. Ozone concentrations are given in parts per billion (ppb). Due to the photochemical nature of ozone production diurnal, in addition to seasonal cycles, are typical. Diurnal maximum 1-hour and 8-hour ozone levels typically occur during the afternoon hours.

II. Procedure

Ozone forecast guidance for Knoxville, Tennessee, was obtained each day through the study period from the NWS ozone forecast guidance data base on the operational Web site (https://www.weather.gov/aq/). The grid point used for Knoxville corresponded to latitude: 36.00 N, longitude: -83.90 W. Generally, data from the 1200 UTC run were used, although other model run times were used in isolated instances when 1200 UTC data were not available or where circumstances prevented obtaining the 1200 UTC data. The ozone forecast guidance analyzed and the highest predicted one-hour and eight-hour values were determined. These values were then compiled for each day through the study period (see Table 1). A graphical depiction of the maximum 8-hour predicted vs. actual ozone values is provided in Figure 1. Forecast guidance data were occasionally not recorded (on a total of 16 days). An entry of "MM" was made for each of those missing values.

Figure 1. Variation of predicted and observed maximum 8-hour average ozone values through the study period.

Official ozone monitoring data were obtained from the State of Tennessee Department of Environment and Conservation (TDEC) and the Knox County Department of Air Quality Management. There were two monitors in the Knoxville area, one monitor located at 9315 Rutledge Pike, Mascot, Tennessee (36.01944 N, -83.87361 W), and the other located at 4625 Mildred Drive, Knoxville, Tennessee (36.084722 N, -83.764722 W). Monitoring values for the two sites were averaged. The monitoring data were also included in the matrix (Table 1) on the corresponding days. The NAM-CMAQ ozone forecast verification was based on the data from these two monitors.

The predicted ozone values were compared to the monitoring results. Mean Absolute Error (MAE) and Mean Bias Error (MBE) were subsequently calculated for each predicted value, for both 1-hour and 8-hours.

III. Results

The MAE for the entire study period was approximately 10.3 ppb for both the 1-hour and 8-hour maximum average values. The maximum 1-hour ambient ozone levels averaged 66.9 ppb, and the maximum 8-hour ambient ozone levels averaged 57.6 ppb through the study period. Therefore the 10.3 ppb MAE equates to approximately 15% and 18% forecast errors, respectively.

The NAM-CMAQ forecast guidance for Knoxville showed a high bias, which suggests a tendency to overpredict ozone levels. The MBE for the maximum 1-hour average values was a +2.91 ppb. The MBE for the maximum 8-hour average values was +5.91 ppb. This overprediction bias has been reported previously in statistical evaluations for the entire Eastern domain conducted in a previous study of the NAM-CMAQ during developmental testing in 2004 (Otte, et al. 2002).

The NAM-CMAQ forecast that 10 days would exceed the maximum 8-hour NAAQS limit for ozone, which is 0.084 ppm (or 84 ppb) through the study period. Only one day actually exceeded the 8-hour value, based on the average of the two Knoxville ozone monitors. That occurred on July 26, when the average concentration of the ozone monitors was 92 ppb. The NMA-CMAQ predicted a value of 96 ppb for that day. There were no days which exceeded the now-defunct (except for EAC areas) maximum 1-hour NAAQS value of 0.124 ppm (124 ppb) and none were predicted by the NAM-CMAQ.

Some spatial variability was noted between the two ozone monitors. There were two instances through the study period when one of the monitors, but not the other, exceeded the 8-hour ozone NAAQS, and the average of the two monitors was at or below 84 ppb. These instances occurred on June 24, when the Mildred Drive monitor recorded 88 ppb, and on August 12, when the same monitor recorded 86 ppb.

The maximum 1-hour observed values ranged from 29 ppb on July 7 to 103 ppb on July 26. The maximum 8-hour observed values ranged from 18 ppb on September 26 to 92 ppb on July 26. The maximum forecast error occurred on June 9 when the predicted ozone value was missed by 60 ppb for 1-hour and 46 ppb for 8-hours. The NAM-CMAQ predicted a maximum 1-hour value of 119 and a maximum 8-hour value of 94 ppb, while the actual ambient ozone concentrations were 59 ppb and 48 ppb, respectively, based on the ozone monitoring data. These errors were caused by a bug in the interface processing between the newly updated ground-level global forecast system NWP model (GFS) that provides boundary and initial conditions to the NAM. The bug was diagnosed as a result of its impacts on predicted ground-level ozone, and the resulting crisis fix corrected the situation by mid-June Paula Davidson, personal communication).

Another large high bias error was noted on August 14, when NAM-CMAQ missed the 8-hour ozone value by 40 ppb (91 ppb predicted, 51 ppb actual). The large error that resulted on this day was probably related to unusual conditions not predicted by the NAM model. A review of the weather conditions reported at the Knoxville, Tennessee, airport (TYS) revealed temperatures were near the climatological normal for the date, but with fog reported until 1000 LST, and broken to overcast sky conditions in all but one observation during the remaining daylight hours.

The only observation that did not report a ceiling occurred at 1600 LST, which is fairly late in the afternoon, after the typical peak diurnal ozone production time. Based on raw NAM model data from that date, the predicted temperature (91⁰F) was close to the actual temperature (89⁰F). However, the predicted relative humidity (RH) values were generally below 70% at all levels (925 millibars to 300 millibars), suggesting that clouds were likely not predicted by the model through the peak ozone-generating daylight hours, thus leading to the relatively high ozone values forecast by NAM-CMAQ.

At any rate, it is clear that weather conditions, particularly cloud cover, play a major role in ambient ozone production. Since ozone production is a photochemical process, any errors in the NAM predicted cloud cover would have a significant effect on CMAQ forecast ozone concentrations.

The largest low bias occurred on August 20 for both the 1-hour and 8-hour categories. The NAM-CMAQ predicted 51 ppb (1 hour maximum) and 48 ppb (8 hour maximum) while the corresponding observed values were 85 ppb and 71 ppb, respectively. The weather observations for TYS on that day indicated few to scattered clouds through the daylight hours. The raw NAM data show quite high RH values (>85%) at 850 millibars throughout the daylight hours, suggesting the possibility that the NAM overpredicted clouds at that level. This would have caused the CMAQ to significantly underpredict the ozone levels.

The most accurate ozone forecasts occurred for May 24 and July 15 when the NAM-CMAQ maximum 8-hour values were 50 ppb and 54 ppb, respectively, the same as the observed values. The 1-hour forecast guidance was only 1 ppb from the observed values on those days.

A breakdown of the NAM-CMAQ forecast errors by percentage in various categories is provided in Table 2. The NAM-CMAQ had a low bias for both 1-hour and 8-hour ozone levels early in the season (May). There were 19 days and 16 days, respectively, where the NAM-CMAQ underpredicted ozone levels during May, while only seven days (out of 26 days sampled or 27%) and nine days (35%), respectively, were overpredicted.

The NAM-CMAQ tended to persistently overpredict ozone levels during June - September. The months with the greatest number of days in which NAM-CMAQ overpredicted ozone levels were July for 1-hour levels (21 out of 28 days sampled, or 75%), and August for the 8-hour levels (25 out of 29 days sampled, or 86%).

IV. Conclusion

In general, the NAM-CMAQ did a good job in predicting maximum ambient ozone concentrations for the following day, and could provide useful guidance of air quality. There was a tendency for the NAM-CMAQ to overpredict ozone levels, with a few days when large errors were noted. Those errors were most likely related to errors in the underlying NAM weather prediction, and could potentially be mitigated by incorporating knowledge of the expected performance of the NAM.

V. Acknowledgments

The author thanks the staff of the State of Tennessee Department of Environment and Conservation and the Knox County Department of Air Quality Management for providing the ozone monitoring data. Thanks also go to the staff of the National Weather Service Forecast Office at Morristown, Tennessee, especially David Hotz, Science and Operations Officer, and to the staff of the National Weather Service Office of Science and Technology, especially Dr. Paula Davidson, National Weather Service Program Manager for Air Quality Forecasting, for helpful suggestions with the study and manuscript.

VI. References

Cooper, C. David and F. C. Alley, 2002: Air Pollution Control; A Design Approach, 3d ed. Waveland Press, 739 pp.

Davidson, Paula M., 2003: National Air Quality Forecast Capability: First Steps Toward Implementation, May 7, 2003, Office of Science and Technology, National Weather Service, 31pp.

________________, 2005: National Air Quality Forecasting Capability: Progress and Plans, June 17, 2005, 18pp.

McQueen, Jeffrey T., Pius Lee, Marina Tsidulko, Geoff DiMego, Rohit Mather, Tanya Otte, Jon Pleim, George Pouliot, Daiwen Kang, Ken Schere, Jerry Gorline, Michael Schenk, and Paula Davidson, 2005: Update of the Eta-CMAQ forecast model run at NCEP operations and its performance for the summer 2004. Preprints, Third Annual Models-3 User's Conference, Chapel Hill, NC, Environ. Protection Agency, [Available online at https://www.cmascenter.org/conference/2004/abstracts/Forecasting/mcqueen_abstract.pdf.]

Otte, Tanya L., George Pouliot, Jonathan E. Pleim, Jeffrey O. Young, Kenneth L. Schere, David C. Wong, Pius C. S. Lee, Marina Tsidulko, Jeffery T. McQueen, Paula Davidson, Rohit Mathur, Hui-Ya Chuang, Geoff DiMego, and Nelson L. Seaman, 2005: Linking the Eta Model with the Community Multiscale Air Quality (CMAQ) Modeling System to Build a National Air Quality Forecasting System, Weather and Forecasting, Vol. 20, No. 3, pp. 367?384.

U.S. Environmental Protection Agency, National Ambient Air Quality Standards (NAAQS).

_______________________________ Green Book Nonattainment Areas for Criteria Pollutants 1-Hour Ozone Information.

* since 8-hour average exceeded 1-hour average, data were suspect and were not used

Note - Bold numbers indicate values exceeding the NAAQS limit for ozone