Geographical separation of seasonal prediction skill between statistical tool and dynamical model

Statistical tool: Constructed Analog (CA) (van den Dool 1992, 2007)

i) Data: HAD SST (45S-45N, 1948-1980)

ii) Ensemble size: 24 members (1-4 seasons data in ICs, 6 EOF cutoffs (35, 40, 45, 50, 55, 60)

Dynamical model: North American Multi-Model Ensemble (NMME) (Kirtman et al. 2014)

Initial condition (IC) season: MAM, JJA, SON, DJF

Forecast lead-time: 1 and 5 months

Skill metrics: Anomaly Correlation (AC)

Assessment time period: CA ~ 1981-2015; NMME ~ 1981-2010

Verification data: NOAA-OI-v2 SST

Puzzle: It was found distinct geographical separation of seasonal prediction skill with decent skill shown over the tropical western Pacific and Indian Ocean by CA and that over the tropical central-eastern Pacific by NMME (e.g. Fig. 1). A summary of all cases (varied initial seasons and lead times) is given by Table 1.

Challenges: The Constructed Analog (CA), a statistical tool, clearly revealed appreciable predictability over the tropical western Pacific and Indian Ocean, where dynamical models had little skill; pointing to possibly missing of important process(es) common to dynamical models, whose development efforts more focused on improving ENSO forecast historically.

More Details

A case of week-2 forecast running to the opposite of the observation

Date of initial condition : 11/21/2006

Forecast target period: 11/29-12/5/2006

Predictant:  Mean surface temperature anomaly

Model:  NCEP GFS


i)  The model ensemble forecast was totally out of phase compared with the observation.

ii)  Further examined the near-range forecast and found that the 6-10 day forecast from 11/26 initial condition can correctly capture the 500 hPa height observed pattern, while that from 11/25 initial condition cannot.

More Details

Skill disparity between temperature and precipitation in 2010-2011 seasonal forecasts

The fall, winter and spring of late 2010 and 2011 were characterized by a moderate to strong La Nia across the tropical Pacific Ocean, which shaped CPCs seasonal outlooks for those seasons.


i)  Precipitation forecasts for September November 2010 through April June 2011 scored at least 30% better than a climatological forecast, the longest streak (eight) of successful forecasts since CPC began issuing seasonal forecasts in 1995

ii) In contrast, the temperature forecasts during the heart of the winter (November January, December February, and January March) were not as successful, with Heidke skill scores near or below zero. What caused the disparity in skill between the temperature and precipitation forecasts? The answer could be the seasonally dependent influence of unpredictable factors, i.e. AO, PNA et al.

More Details

Predicting 2011/12 La Nina onset by models     Where was the early warning?

In CPC Sanity Check of November 2011, the failure of Nino 3.4 SST forecast from June initial condition by almost all models was brought to attention with two exceptions of good forecasts by ESSIC Intermediate Coupled Model (ICM) and by Japan Frontier Research Center for Global Change Coupled General Circulation Model. Looking at individual model performance, a common problem of phase delay in prediction of ENSO transition is clearly shown.

Question What key physical processes were missed or misrepresented?

More Details

Difficulties in prediction of ENSO phase changes and impact on outlooks of 2006 North Atlantic hurricane season & 2006/07 DJF US drought

The predictive ENSO condition is very important information for forecasters when making seasonal outlooks.  Forecasters found that all tools are too much like persistence, according to which too late phase transition would also occur in the forecast, the events starting too late and then lasting beyond the time they should.  Following is a case, showing forecasts of ENSO phase transitions in late 2006 and early 2007 and the influence on the outlooks of 2006 North Atlantic hurricane season and 2006/07 DJF US precipitation. The focus is on the NCEP Climate Forecast System (CFS), though other dynamical and statistical models have similar problem as seen in the ENSO prediction plume graph produced routinely by the International Research Institute for Climate and Society (IRI).

More Details

Outstanding Operational Forecast Challenges

Research to Operation

This board is established to publicize science-related forecast challenges identified by operational forecasters and end-users, inviting our partners in the research community to work together on improvement of national climate prediction services.