Predictability of Coupled GCM Forecasts:


                                                                Emilia K. Jin                                         [Print Version]

Center for Ocean-Land-Atmosphere Studies

George Mason University

To understand the predictability of coupled general circulation models (CGCMs), the sea surface temperature (SST) predictability of CGCM hindcasts is investigated by analyzing the structure of systematic error and estimating the growth of forecast error from small initial perturbations. In addition, the CGCM’s behavior in a long simulation is analyzed to understand the cause of forecast error with respect to lead time in short-term forecasts. Focusing on the NCEP Coupled Forecast System (CFS) having 9-month integrations for all 12 calendar months, the CGCM datasets that come from the CliPAS (Climate Prediction and its Application to Society) and DEMETER (European Multimodel Ensemble system for seasonal to inTERannual prediction) projects are used. The 12 models used are fully coupled ocean-land-atmosphere dynamical seasonal prediction systems with 5- to 9-month integrations for 3 to 15 different initial conditions for summer and winter seasons in the common 23 years from 1981 to 2003. As an observational counterpart, the HadISST1.1 SST is used for comparison with the hindcasts. It should be noted that using an independent SST analysis tends to produce lower skill scores than using an SST analysis that contributed to the initial state used to make hindcasts. For example, the CPC SST analysis is assimilated as part of the input data for constructing the ocean initial conditions in the CFS hindcasts.

First, the overall forecast skill of the state-of-the-art CGCMs is assessed. Focusing on the tropical Pacific region, the forecast annual mean, annual cycle, and its influence on forecast skill are analyzed with respect to lead month. Figure 1 shows the anomaly correlation coefficient for each of the 12 CGCMs with respect to lead month during 1980-2001. In particular, the SNU and MPI models, having lower interannual forecast skill, are also outliers in the sense of climatology including annual mean and annual cycle. After removing the model mean bias, the predictability with respect to ENSO phase shows that the phase locking of the ENSO to the mean annual cycle has an influence on the seasonal dependence of skill. The growth phase of both warm and cold events is more predictable than the decay phase and normal events are far less predictable than warm and cold events. Accordingly, the forecast skill curves for August and November initial conditions, which include most of the growth phase, show slower decline of skill than those for February and May initial conditions.

The error growth and its implication on seasonal predictability are investigated focusing on the NCEP CFS model. Because this model has a large sample with 9-month integrations of 15 members during 23 years, it provides a better data set in which to examine how the errors grow with respect to the lead month. For up to two forecast months, the SST systematic error more than doubles over the whole global ocean. Beyond two months lead time, the subsequent increase shows a clear seasonal and regional dependence irrespective of lead time. After removing the systematic error, the root-mean-square error of the SST anomaly also shows a clear seasonality distinct from that of the systematic error coincident with various models’ results.

From the initial state, the growth of forecast error and the lower limit of error in the forecast system are investigated with respect to lead time. Fig. 2 shows the forecast error and the Lorenz curve for the ensemble mean and mean of individual member. Forecast error is the difference between simulated and observed anomalies. To calculate the error growth for CFS following Lorenz (Lorenz curves), we take the one-month forecast and two-month forecast validated at the same time and see how the error grows with time. Accordingly, the Lorenz curve is estimated from monthly mean data by assembling the locus of the RMS difference between the one-month and two-month lead forecasts for the first target month, the RMS difference between the two-month and three-month lead forecasts for the second target month, and so on.  In CGCMs, the initial error growth is saturated within two months and then error growth follows the identical error model for all initial cases. Therefore, the Lorenz curve of the ensemble mean is fairly constant with lead time. Nevertheless, the Lorenz curve of the mean of individual members in the CFS ensemble grows as fast as the forecast error, because there is a large ensemble spread due to the instability of the coupled system. We draw the same conclusion as Lorenz did for weather forecasting, which is that the best way to improve the weather forecast beyond day 1 is by improving the first day forecast (Lorenz 1982). Similarly, the biggest improvement of ENSO prediction can be obtained by reducing the first month forecast error.

The behavior of multiple CGCMs in long simulations is also investigated as an important source of forecast error in short-term forecasts with respect to lead time. The main analysis focuses on the CFS. The SINTEX-F, SNU and UKMO models are analyzed also, since they provide both more than 50-year control simulations and 23-year hindcast data sets. Figure 3 shows the warm minus cold composite of the NINO3 index during 1980-2001. The black curve is the observation and the dashed lines show the reconstructed NCEP CFS forecast data with respect to lead time. Four cases are selected for both El Niño (82/83, 86/87, 91/92, 97/98) and La Niña (84/85, 88/89, 98/99, 99/00). For the ENSO forecasts made with the CFS model, a constant phase shift with respect to lead month is clear, using monthly forecast composite data. This feature is related to the model property that ENSO has a long life cycle with a summer peak, as shown in the long run case, which differs from observations. By computing the warm minus cold composite, the model ENSO cycle can be compared with the observed (Fig. 4). As expected, the simulated ENSO cycle shows early and slow evolution. The model produces an incorrect peak in summer and the winter peak is weaker than observed. The decay phase looks more similar to observations but it is also progresses more slowly than observed, because the predicted peak of ENSO is smaller than the observed.  On the basis of this analysis, the ENSO forecast can be considered in the sense of the model’s ENSO properties. For the first month, the simulated ENSO agrees with the observed. However, by the ninth month, the slow evolution of this model’s ENSO mode is clear and it generates the phase shifted feature in previous plot.


Figure 1












Figure 2










Figure 3



Figure 4

For other models as well, the systematic errors in the long run - for example, mean bias, phase shift, weak amplitude, and wrong seasonal cycle - are reflected in the forecast skill as a major factor limiting predictability. Accordingly, the influence of coupled model errors on real forecasts is an important factor degrading the predictability after the influence of initial conditions fades out with respect to lead time.

Thanks to Drs. James L. Kinter, Jagadish Shukla, Bin Wang, and In-Sik Kang. Also, a special thanks to NCEP Environmental Modeling Center (EMC) for providing NCEP CFS data.

Contact  Emilia K. Jin