US National Oceanic and Atmospheric Administration
Climate Test Bed Joint Seminar Series
NCEP, Camp Springs, Maryland, 10 February 2009                                                                                                   [Print Version]

Seamless Prediction of Weather and Climate:
A New Paradigm for Modeling and Prediction Research

J. Shukla

George Mason University, Fairfax, Virginia
Institute for Global Environment and Society, Calverton, Maryland

Abstract

This paper describes the concept of seamless prediction and its evolution during the establishment of the COPES (Coordinated Observation and Prediction of the Earth System) framework of the World Climate Research Program (WCRP), where the concept was first presented in 2005.

1. Introduction

There is considerable historical evidence that major scientific and technical discoveries are often followed by the creation of institutions that can take advantage of those discoveries for the betterment of society. The breakthrough in our understanding of atmospheric dynamics that was developed during and after the Second World War, accompanied by the technological breakthrough of fast automatic computing devices, led to the rapid development of numerical weather prediction, a capability that has been institutionalized by many governments around the world and commercialized into a multi-billion dollar enterprise worldwide.

A second example is the development of our scientific understanding of chaos in nonlinear dynamical systems and the potential for predictability at seasonal time scales in the midst of that chaos. The application of that capability for seasonal climate prediction led to the creation of a number of institutions for dynamical seasonal prediction and research.

Now we have before us, thanks to the Intergovernmental Panel on Climate Change (IPCC), a third discovery: humans are affecting the Earth’s climate. Beyond IPCC, this discovery will inevitably lead to the establishment of new institutions, or a transformation of current institutions, whose goal will be to help the peoples, governments and corporations of the world manage the consequences of climate change wisely, economically and effectively.

In this paper we present an outline of the emerging paradigm of seamless prediction of weather and climate and present a strategy to revolutionize weather and climate prediction. There is no scientific basis to draw artificial boundaries between meso-scale prediction, synoptic scale prediction, seasonal prediction, ENSO prediction, decadal prediction and climate change. However, practical considerations of computing and of model complexity may require different prediction systems for different time scales. The simulation and prediction of meso-scale systems, synoptic scale disturbances, intra-seasonal, seasonal and inter-annual variations are intimately linked, and therefore, it is suggested that future research on prediction of weather and climate be carried out in a unified framework.

For reliable prediction of regional climate change it is essential that climate models accurately simulate the modes of natural variability from diurnal to seasonal and decadal. Utilization of the insights gained from operational weather and seasonal prediction, and of the synergy between the weather and climate prediction communities is essential for the development of next-generation seamless prediction systems.

One of the scientific implications of the seamless framework is that decadal and multi-decadal prediction using IPCC class models should correctly initialize the state of the ocean-land- atmosphere-cryosphere system. Just as the 1 day NWP forecast is critical in determining the 10 day forecast, it is likely that 10-30 year forecasts will be determined by one season to one year forecast. Institutionally the seamless framework requires the weather and climate communities to work as an integral part of a single scientific enterprise.

2. Weather prediction

It was nearly 100 years a go that V. Bjerknes made a prophetic statement that by using the “knowledge of the state of the atmosphere at the initial time,” and “knowledge of the laws according to which one state of the atmosphere develops another,” if we were able to make accurate forecasts for the future, “meteorology would then have become an exact science, a true physics of the atmosphere.” The seminal work of V. Bjerknes (1904) laid the foundation for mathematical modeling of atmospheric dynamics based on the laws of physics. Subsequent works by Richardson, Rossby, Charney and Phillips lead to a successful demonstration of prediction of short-term changes in the large scale atmospheric flow pattern by using dynamical models and heralded the modern era of numerical prediction of weather. Needless to say, that the advances in observational technologies which can accurately describe the state of the atmosphere, data analysis and data assimilation methodologies, and, advances in modeling of the atmosphere have made the dream of V. Bjerknes a reality. The steady advance in the accuracy of numerical weather prediction during the past 30 years is one of the most outstanding examples of scientific progress in the twentieth century. Global society is reaping huge benefits from improved weather forecasts. The number of deaths caused by weather and climate disasters has been reduced. In the past 20 years, the forecast error at day one has been reduced by more than 50%. This has occurred due to steady progress in improving the initial conditions of the atmosphere (which is mainly due to better data quality procedures, better assimilation techniques, and higher resolution models) and improving the accuracy and parameterizations in weather prediction models.

Having already made steady progress during the past 30 years, there is no indication yet that we have reached the limit of weather prediction. After examining the predictability of one of the operational weather prediction models of the European Centre for Medium Range Weather Forecasts (ECMWF), Lorenz (1982) made the following remarks:

“Better than guesswork forecasts of instantaneous weather patterns nearly two weeks in advance appear to be possible, and efforts to establish numerical-prediction models which are potentially capable of making such forecasts, and observing systems which enable to models to realize their potentialities, should continue.”

This statement is as valid today as it was 27 years ago. After publication of Lorenz’s paper, it could have been argued that as we continue to increase the resolution of models and resolve fast growing instabilities, the growth rate of initial errors becomes so large that no additional advantage will be gained by increasing the resolution of models. This did not happen at all. The skill of weather forecasts has continued to increase. While it is true that as the one-day forecast error has been reduced by 50%, the growth rate of initial error has increased from the doubling time of about 2 days to 1.5 days, however there still remains unrealized weather predictability for days 1-10.

In a classic paper Lorenz (1969) demonstrated how in a two-dimensional barotropic system an initial error in small scales induces error growth in larger scales. He showed that the predictability limit depended sensitively on the equilibrium energy spectrum. For a -5/3 slope, predictability is intrinsically limited no matter how small the initial error. However, for spectral slope of -3 or greater, increases in predictability were possible with the reduction in the scale and size of the initial error. Experience with operational numerical weather prediction (NWP) during the past 25 years has shown that as we continue to reduce the resolution of NWP models and improve the initial conditions of the atmosphere with better data analysis and assimilation schemes, we continue to improve medium range weather forecasts. Does this mean that we are still in the -3 regime? Or, is it possible that although our current high resolution models are beginning to resolve the upper portions of the -5/3 spectral range, the reason we continue to improve forecasts is because the scaling arguments on which intrinsic limits of predictability were based are not valid for the small scale of the real atmosphere? Recent observations of the atmospheric energy spectrum and numerical experiments with extremely high resolution models suggest that further improvements in medium range weather forecasts with improved high resolution models and improved observations is indeed possible.

A few examples of very high-resolution regional weather prediction models clearly show that local weather phenomena forced by local orography can be resolved and simulated better by higher resolution models. There are also a few examples of global very high-resolution models that show a far more realistic simulation of global weather by eliminating the parameterization of moist convection. There is also well-documented evidence to show that ensemble forecasting has improved the reliability of weather forecasts. In summary, there is overwhelming evidence to suggest that substantial improvements in the accuracy and the reliability of medium range weather forecasts beneficial to society is still possible if we can improve the initial state by using accurate observations and extremely high resolution weather prediction models that can explicitly resolve deep convective cloud systems. The improvements in the medium-range weather forecasts will directly translate into improvements in seasonal forecasts and vice versa.

3. Dynamical seasonal prediction

Twenty-five years ago, dynamical seasonal prediction was inconceivable. Today, it is a reality. This has become possible because of another major breakthrough in our field with the realization that, although the atmosphere is a chaotic dynamical system, there is predictability in the midst of chaos (Shukla 1998). Certain regions of the global atmosphere are so strongly determined by the underlying boundary conditions (viz sea surface temperature) that they do not show sensitive dependence on initial conditions of the atmosphere, and it should be possible to predict the atmospheric circulation (and rainfall) for as long as the lower boundary forcing can be predicted. Researchers have demonstrated that interactions among the global atmosphere, global oceans and global land surface, can produce fluctuations that are potentially predictable beyond the deterministic limit of predictability for weather. Researchers have further demonstrated that the short-term fluctuations of climate at seasonal and interannual timescales are indeed predictable. Just like the advances in weather prediction, this has also been made possible because of steady improvements in the prediction models which could be integrated for hundreds of days, and could simulate the observed covariability between the slowly varying boundary conditions at the earth’s surface and the atmosphere, and, improvements in coupled models which could predict these boundary conditions.

However, operational dynamical seasonal prediction is still in its infancy. Differences in the estimates of seasonal predictability by various “state-of-the-art” models in the world today are quite large, just as large, or even larger than the difference in the estimates of weather predictability 30 years ago. There is no evidence to suggest that the current limitations in the skill of dynamical seasonal forecasts is due to some fundamental predictability limit, on the other hand, there is overwhelming evidence that improvements in models and the initial conditions of ocean, land and atmosphere will improve accuracy of dynamical seasonal predictions.

When several state-of-the-art models were forced with the same SST, they produced very large differences in the tropical convection and the associated seasonal mean precipitation. It is our considered opinion that difference in the parameterizations of moist convection is the main reason for differences in the tropical precipitation and the associated global circulation. Therefore, in order to make better estimate of predictability of seasonal mean circulation and rainfall, it is essential to utilize high resolution models which can resolve and describe organized cloud systems with deep moist convection in the atmosphere, small scale orographic and landscape features overland, and energetic eddies in the oceans.

4. Sub-seasonal prediction

In spite of significant progress in weather forecasting for days 1-10, and some progress in dynamical seasonal prediction for days 1-100 averages, there is no comparable progress in understanding and predicting evolution of weather events during a season. It is well known that there are large week-to-week variations within a season, and even if it is not possible to predict the actual sequence of day-to-day weather, if it were possible to predict statistics of intra-seasonal variability, it would be highly beneficial to society. Examples of Intraseasonal variability which affect seasonal mean as well as produce large societal impacts during a season are given below:

1. Active and break cycles of Asian summer monsoon: the large weekly and monthly variations of regional monsoon rainfall are important for agriculture, energy and water supply planning.

2. Madden-Julian Oscillations: they produce a large tropical as well as extra-tropical variations during a season.

3. Tracks of tropical disturbances: even if individual hurricanes, typhoons and other tropical disturbances cannot be predicted beyond a few days, if it were possible to predict the statistics of these disturbances (changes in tracks, number and intensity), it would be extremely beneficial to society.

4. Extra-tropical storm tracks modulated by slowly varying large-scale flow.

5. Statistics of extreme events within slowly varying large scales flow.

In order to address the question of predictability of the statistics of high impact local weather events it is required that the models used for seasonal prediction resolve and simulate the local weather events. It is therefore essential that the models for predicting daily, weekly, monthly and seasonal variation be unified. In other words, very high-resolution models without parameterizations of deep moist convection must be extended at least up to a season to predict the statistics of high impact weather events within a season. It can be conjectured that since parameterizations of moist convection in coarse-resolution models is the main source of uncertainty in simulation and predicting seasonal means, models that can explicitly simulate organized could systems with moist convection will enhance the predictability of weekly and monthly averages during a season. A recent paper by Miura et al. (2007) supports this conjecture.

5. Decadal prediction and projection of climate change

Thanks to the IPCC, there is vast literature on projections of climate change. It is well known that the most recent assessment of the IPCC (2007) shows a large model dependent range in the projected global warming. We do not know, and there is no simple way to find out, that even for identical external forcing what parts of these differences is due to deficiencies in model physics and model resolution and what part is due to the intrinsic chaotic nature of the coupled climate system. A comparison of the model’s ability to simulate the climate of the twentieth century and global warming in the twenty-first century showed (Shukla et al., 2006) that global warming in the twenty-first century was large (4°-5°C) for those climate models that had the least error in the simulation of the twentieth century climate, and models with the largest error in simulating the current climate had small (2°-3°C) warming. Such results, although by no means definitive, strongly suggest the need to improve the fidelity of climate models.

There is also a great deal of scientific and societal interest in quantifying the predictability of decadal variations. Climate models should be able to simulate the regional weather fluctuations, intra-seasonal, seasonal and interannual variations as well as the decadal and multi-decadal variations to be able to be a reliable tool to predict changes in regional climate for 10-30 year time scales. The challenges of making reliable predictions of decadal variations require that the models are able to distinguish between the “unforced” decadal variations due to the internal dynamics of the coupled climate system and the “human forced” external component due to changes in greenhouse gases and land surface conditions. Decadal variations can be caused by changes in the amplitude and frequency of El Niño events, which can be caused by changes in the intra-seasonal (MJO, westerly wind bursts etc) variations which can, in turn, be caused by changes in the organized tropical convection. It is therefore important that climate models can resolve and simulate the meso-scale cloud systems with organized deep convection, intra-seasonal variations, ENSO, as well as decadal meridional oceanic overturning. This does not mean that climate models must correctly predict the phase evolution of each phenomenon for a long time (that will be impossible because of chaos; each phenomena will have its own intrinsic limit of deterministic prediction), but it does mean that the statistics of each phenomenon is correctly simulated by the climate models. If a model cannot simulate a phenomena, it cannot predict that phenomena.

6. Evolution of the concept of seamless prediction in WCRP

It is in the context described above that the author proposed to the WCRP in 2002 (WCRP, 2007) that a World Climate Experiment be launched to assess the predictability of climate system at all time scales. In response, the Joint Scientific Committee of WCRP established a task force (members: B. Hoskins, J. Church, J. Shukla), which, for the first time, introduced the concept of seamless prediction to WCRP. The task force on predictability assessment was later expanded by the WCRP to a much larger task force to develop a strategic framework for the future of WCRP. The newly developed WCRP strategic framework for 2005-2015 was named: Coordinated Observation and Prediction of the Earth System (COPES). The main aim of COPES is to facilitate analysis and prediction of Earth system variability and change for use in an increasing range of practical application of direct relevance, benefit and value to society.

The concept of seamless prediction as articulated by the WCRP-COPES strategic framework is quoted below (WCRP, 2005).

1. “There is now a new perspective of a continuum of prediction problems, with a blurring of the distinction between shorter-term predictions and longer-term climate projections. Increasingly, decadal and century-long climate projection will become an initial-value problem requiring knowledge of the current observed state of the atmosphere, the oceans, cryosphere, and land surface (including soil moisture, vegetation, etc.) in order to produce the best climate projections as well as state-of-the-art decadal and interannual predictions.”

2. “The shorter time-scales and weather are known to be important in influencing the longer-time-scale behavior. In addition, the regional impacts of longer-time scale changes will be felt by society mainly through the resulting changes in the character of the shorter time-scales, including extreme events. In recognition of this, climate models are being run with the highest possible resolution; resolutions there were employed in the best weather forecast models only a few years ago.”

3. “Even though the prediction problem itself is seamless, the best practical approach to it may be described as unified: models aimed at different time-scales and phenomena may have large communality but place emphasis on different aspects of the system.”

The WCRP strategic framework and the WCRP Modeling Panel adopted the concept of seamless prediction as the organizing principle for the future of WCRP modeling.

Since climate in a region is an ensemble of weather events, understanding and prediction of regional climate variability and climate change, including changes in extreme events, will require a unified initial value approach that encompasses weather, blocking, intra-seasonal oscillations, MJO, PNA, NAO, ENSO PDO, THC, etc. and climate change, in a seamless framework.

The seamless prediction concept implies seamlessness across space and time scales (multi-scale interactions); across scientific disciplines (physical climate system, biogeochemical cycles, socioeconomic systems); across institutions (academic, government, corporations); and across geographical boundaries (local, state, national, international).

Considering that the existing institutions have been established separately for weather prediction and climate research, it will be indeed a challenge to integrate and synthesize the activities of different weather and climate institutions in a seamless framework.

7. A proposal to revolutionize weather and climate prediction

About 150 scientists from the major modeling centers of the world gathered at the World Modelling Summit for Climate Prediction (held in Reading, England [6-9 May, 2008]) organized by the World Climate Research Program to discuss ways to revolutionize weather and climate prediction.

The deliberations at the Summit lead to a statement published in BAMS (Shukla et al., 2009). An important statement of the Summit was the need for a world climate research facility for climate prediction:

“The central component of this world facility will be one or more dedicated high-end computing facilities that will enable climate prediction at the model resolutions and levels of complexity considered essential for the most advanced and reliable representations of the climate system that technology and our scientific understanding of the problem can deliver. This computing capability acceleration, leading to systems at least a thousand times more powerful than the currently available computers, will permit scientists to strive towards kilometer-scale modeling of the global climate system which is crucial to more reliable prediction of the change of convective precipitation especially in the tropics.”

In this paper, I take the Summit statement one step further and propose the establishment of an international center for climate prediction consisting of three interconnected advanced computing facilities for climate research and prediction. This international center will help global society for adaptation, mitigation and sustainable development.

8. International center for climate prediction

The world recognizes that humans are contributing to climate change. The impending threat of global change is one of the most important and urgent problems facing humanity. The nations of the world are engaged in serious discussions about the most desirable and practical national and international strategies to achieve the dual objectives of reducing the emissions of greenhouse gases and ensuring sustainable development.

One important aspect of the problem that is not getting sufficient attention, especially in international negotiations, is the inevitability of climate change due to emissions that have already taken place, and the emissions that will certainly take place in the coming decades. Global society, especially the developing world, will require accurate, reliable and quantitative prediction of inevitable regional climate change for sustainable and resilient development. To adapt to, and to cope with, the dire consequences of inevitable climate change will require investments of trillions of dollars.

Although the current generation of climate models has shown convincingly that human activities can produce large changes in the climate, these models are not adequate to make accurate and reliable predictions of regional climate change and extreme events that are required for science-based adaptation strategies. The major effort of the international scientific community during the past 20 years has been focused on convincing a handful of powerful skeptics that global warming is real, rather than conducting the appropriate research necessary to build sophisticated climate models suitable for providing useful guidance to the policy makers for adaptation and mitigation. Therefore a large number of members of the scientific community, including climate scientists in India, are highly skeptical, and correctly so, about the current projections of regional climate change in India. We do not have confidence in the estimates of projected climate change for different regions of India, for example, identifying which regions will be affected by severe droughts or severe floods, the number and intensity of tropical cyclones, etc.

It is therefore necessary to develop and build accurate and advanced global models for climate prediction so that reliable and quantitative predictions of regional climate change can be provided to global society for science-based adaptation strategies. It is both possible and necessary to have a revolution in climate prediction. It is possible because of the major scientific and technological advances, and it is necessary because adaptation strategies in response to climate change require the most accurate and reliable regional predictions of climate. Sustainable development and the security of the future of life on Earth over the next century demand the best possible information both on the planet’s life-support system, in particular the availability of water and food, and possible socio-economic catastrophes due to climate change. The necessary scientific expertise and the computational-technological capabilities to produce reliable regional climate predictions are not available in any single nation.

Regional climate prediction does not mean we should use regional models to predict climate change because regional models require lateral boundary conditions which are currently obtained from low-resolution climate models. It has been pointed out by the WCRP Modeling Panel that, “Use of high resolution regional models to downscale regional climate change is questionable if the global models from which lateral boundary conditions for regional models are prescribed do not have reliable simulation of planetary waves and statistics of storms and blocking.” Global climate models at ultra fine resolution will require a huge increase in computational capability. Table 1 gives some estimates of computing capability required for different model grid resolutions and throughput rate (simulated days per wall clock hour). We argue that comprehensive multi-national efforts are essential to provide reliable climate predictions, and assessing their impact, with the level of confidence required by society.

 

Table 1

Climate change is a global problem and the solution to the problem of providing governments and society with reliable climate predictions should also be addressed at the global level. Many countries of the world will be making adaptation decisions costing sizable fractions of GDP in the billions (perhaps trillions) of dollars. It is highly desirable that these decisions are based on the most accurate climate predictions made by the most advanced scientific tools and infrastructure possible. It is proposed that an International Center for Climate Prediction be established to provide all the nations of the world the most accurate and reliable description of the current global environment, and prediction of climate change and its impacts based on the best science and the most advanced technology. The international center will foster research and training for building global capacity, developing a trained scientific workforce and engaging the global user community.

There are several examples of successful international institutions. The Consultative Group on International Agricultural Research (CGIAR) is credited with launching the global Green Revolution towards sustainable food security and poverty reduction. The Hubble Telescope, the Human Genome Project, ECMWF and the high-energy particle accelerator at CERN are outstanding successful examples of internationally funded infrastructures, and the ITER nuclear fusion facility may prove to be another. We assert that the problem of climate prediction is so important and urgent that it needs international collaboration and international infrastructure to provide society accurate and reliable climate information.

We propose that the international center will consist of at least three international nodes of advanced computing facilities for climate prediction. Considering the probabilistic nature of climate prediction and the need for creative competition among the scientific groups, and the large investments needed to create such facilities, the establishment of three independent but inter-connected facilities was considered to be optimal. Once the three international facilities are in place they should be connected to multiple regional climate prediction and adaptation research centers worldwide which will take the prediction from the international center and add value for local adaptation. The three interconnected computing and prediction facilities will engage scientists from all over the world. In particular, it must guarantee that climate-modeling groups from the developing countries contributed with predictions for future climate change. We propose that the new climate prediction facilities should be truly international and include developing countries. Each of the three international facilities will have a core scientific and technical staff of about 300 persons, and computer capability of about 20 Petaflops in the near future and about 200 Petaflops by the end of the next decade. Each facility will work closely with more than 500 scientists who will have high-level connectivity and access to the facility. The cost of such a single facility will be about $200 million per year; about $100 million per year for computational and data facilities, and about $100 million per year for scientific and technical staff, capacity building and research. The funding for the center can come from both adaptation funds under the UNFCCC and national contributions. The benefits to society will be enormous; some of the benefits are listed below:

1. To provide reliable quantitative predictions of regional climate change required for developing cost-effective adaptation and mitigation strategies, and to cope with the dire consequences of climate change.

2. All the nations of the world will have access to the most accurate and reliable climate information to plan their respective national adaptation strategies.

3. Build global capacity, develop a trained scientific workforce and engage the global user community.

4. Provide computational capacity to scientists worldwide that is unavailable at the national level. This has the potential to revolutionize climate prediction worldwide.

5. Make fundamental advances in climate modeling and prediction which is currently not possible because of insufficient computational capability and lack of a critical mass of scientific workforce.

Acknowledgements.  The author is thankful to Drs. Peter Lemke and John Church who, as chairs of JSC, encouraged and supported my proposal to develop a new strategy for WCRP in which all the components of WCRP (CLIVAR, GEWEX, CLiC, SPARC) will work synergistically to address the question of predictability of the total climate system. I thank Dr. Brian Hoskins, an officer of JSC, for his support and encouragement, and his contribution to the development of the concept of seamless prediction.  The author is thankful to Dr. Tim Palmer for numerous stimulating discussions and bringing to his attention a paper by Palmer and Webster (1995) in which they had independently proposed the concept of seamless prediction. Thanks are due to Drs. David Straus of the Center for Ocean-Land-Atmosphere Studies (COLA) for many stimulating discussions on energy spectra and limits of predictability. Many thanks to Dr. James Kinter of COLA and Mr. J. Abeles of IBM for calculating the computational capacity needed for various resolutions shown in Table 1.

REFERENCES

Bjerknes, V., 1904:  The problem of weather forecasting as a problem in mechanics and physics. Meteorologische Zeitschrift, 21, 1-7.

IPCC, 2007:  Climate change 2007: The physical science basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change.

Lorenz, E.N., 1969:  The predictability of a flow which possesses many scales of motion. Tellus, 21, 289-307.

Lorenz, E.N., 1982:  Atmospheric predictability experiments with a large numerical model. Tellus, 34, 505-513.

Miura, H. et.al., 2007:  A Madden-Julian Oscillation event realistically simulated by a global cloud-resolving model. Science, 318, 1763.

Palmer. T.N. and P.J. Webster, 1995:  Towards a unified approach to climate and weather prediction. Global Change: Proc. of the First Demetra Meeting Held at Chianciano Terme Italy, European Commission, EUR 15158 EN, 427pp. ISBN 92-826-7757-5.

Shukla, J., 1998:  Predictability in the midst of chaos: A scientific basis for climate forecasting.  Science, 282, 728-731.

Shukla, J., R. Hagedorn, B. Hoskins, J. Kinter, J. Marotzke, M. Miller, T.N. Palmer, and J. Slingo, 2009:  Strategies: Revolution in climate prediction is both necessary and possible: A declaration at the world modeling summit for climate prediction.  Bull. Amer. Meteor. Soc., 90, 175-178.

Shukla, J., T. DelSole, M. Fennessy, J. Kinter and D. Paolino, 2006:  Climate model fidelity and projections of climate change.  Geophys. Res. Lett., 33, doi:10.1029-2005GL025579.

WCRP, 2005:  The world climate research programme strategic framework 2005-2015. WMO/TD-No. 1291.

WCRP, 2007:  Report of the twenty-eighth session of the joint scientific committee, Zanzibar, Tanzania, 26-30 March 2007. WMO/TD-No. 1395, 173-176.

Contact  J. Shukla