Science and Technology Infusion Climate Bulletin
NOAA’s National Weather Service
7th NOAA Annual Climate Prediction Application Science Workshop
Norman, OK, 24-27 March 2009                                                                                                                            [Print Version]

Accuracy of 6-10 Day Precipitation Forecasts and Its Improvement in the Past Six Years

Qi S. Hu and Kari Skaggs

University of Nebraska-Lincoln

A major factor weakening people’s intention to use weather and climate predictions in decision-making is the lack of knowledge of the accuracy of the predictions. Although some medium- and long-range predictions are provided with probability, there is no information of how the probability may have been validated for specific areas and regions of the users’ interest. The lack of this information prevents potential users of the predictions from developing their personal knowledge and self efficacy of using the forecasts effectively in decision-making, resulting in the overall low attitude toward use of the forecasts. It is thus important to inform users the accuracy along with weather and climate predictions so users can start building skills, experience, and confidence to use the forecasts in decision-making.

Different methods of measuring the accuracy of weather and climate predictions convey slightly different information about the accuracy to the users. The current method that CPC is using, the Heidke Skill Score, compares the forecast to the observation and then takes a step further to compare how this accuracy is compared to that of a random forecast, the climatology. The second comparison in this procedure gives a measure of the “skill” of the forecasts. This skill may be a good mathematical measure of the forecast accuracy but it is no longer a direct measure of the accuracy and, thus, could confuse the forecast users whose sole concern is “how the forecasts for my location have been compared to the observations in the past,” i.e., the plain accuracy of the forecasts. Thus, for the interest of users and for improving use of the forecasts it is necessary to provide the accuracy that measures how the forecasts have been compared to the observations for different seasons and for various locations.

We examined the accuracy of the 6-10 day (accumulated) precipitation forecasts issued by the NOAA CPC for the contiguous United States (see an example in Fig. 1). This forecast is identified by farmers and extension agents as the most useful forecast for nearly all farming decisions during the growing season. Actual forecasts made every day for 50 locations in the U.S. from Dec. 2001-Dec. 2007 were digitized from the forecast maps (whose electronic archives began in Dec. 2001). These forecasts were made in three terciles: “median” (“N” in Fig. 1), “above median” (“A”), and “below median” (“B”). Meanwhile, observed daily precipitation from these locations was used to calculate the actual 6-10 day precipitation and then categorized it by the climatological terciles at the location. The observed precipitation category was compared to the predicted, and the difference was used to define the accuracy of the forecasts. Specifically, we assigned an error value: error=0, if the forecast matched with the observation at the location; error=+1 or +2, if the predicted precipitation was one or two terciles above the observed; and error=-1 or -2, if the predicted precipitation was one or two terciles below the observed. Using this scale we calculated the accuracy of the 6-10 day precipitation forecasts at the 50 stations in the United States and evaluated the improvement of the accuracy in the past 6 years.

The results from the accuracy analysis show that the forecast has been correct about 40% of the time at each location, which is higher than the expected 33% correct from a random tercile forecasting. The majority of remaining 60% incorrect forecasts was in the same direction as the observations, i.e., forecasts of median when below or above median occurred and forecasts of below or above median when median occurred). The forecast was only in the opposite direction as the observations (±2 categories) an average of 14% of the time at each location, which is lower than the 22% from the random tercile forecasting. With fewer forecasts verifying in the ±2 categories and more correct forecasts than would be expected randomly, these results confirm the usefulness of these forecasts. Users can be confident that with adequate skills these forecasts can be useful in helping them avoid the harmful impacts of extreme wet (flood) or dry (drought) situations.

The improvement of accuracy of the forecasts in the period from 2002-2007 is measured by 6-year trend of annual frequency of correct forecasts at each of the 50 locations in the United States. These trends are shown in Fig. 2. The blue triangles show improvement of accuracy of forecasts at a location and red triangles show deteriorating accuracy of the forecast in the 6-year period. The larger triangles show statistically significant trends at 90% confidence level. The distributions of the blue and red triangles in Fig. 2 indicate that the accuracy of 6-10 day precipitation forecasts for the western U.S., the northern Great Plains, and most of the eastern U.S., has improved over the 6 years. Significant improvements are in the western and northeastern U.S. However, the forecast significantly decreased in accuracy for one station in the northern Midwest, while showing little improvement or slight worsening for the eastern-central and southern U.S.

Figure 1

Figure 2

While showing the accuracy improvement of the forecasts, the spatial variation of the forecast accuracy also indicates that some regions in the country have become “easier” to predict for their 6-10 day precipitation, or the methods used in making the predictions are more suitable for those regions. Some regions in the country are more difficult to predict for their precipitation or the methods need to be improved in order to improve the forecast accuracy.

Contact  Qi. S. Hu