Please view a summary of the case study and a list of relevant resources (LSE Research Impacts website).
Summary of the impact
As the realities of climate change have become more widely accepted over the last decade, decision makers have requested projections of future changes and impacts. Founded in 2000, the Centre for Analysis of Time Series (CATS) has conducted research revealing how the limited fidelity of climate models reduces the relevance of cost-benefit style management in this context: actions based on ill-founded projections (including probabilistic projections) can lead to maladaptation and poor policy choice. CATS’ conclusions were noted in the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) report and led in turn to the toning down of the UK Climate Projections 2009 and the 2012 UK Climate Change Risk Assessment. Members of the insurance sector, energy sector, national security agencies, scientific bodies and governments have modified their approaches to climate risk management as a direct result of understanding CATS’ research. Attempts to reinterpret climate model output and design computer experiments for more effective decision support have also resulted.
Underpinning research
Research insights and outputs:
The LSE's climate simulation research derives in large part from a broader research program on nonlinear dynamical systems, a key focus of CATS.
Mathematical research in Judd and Smith [1] established consequences of model imperfections for probability forecasting from noisy observations and models of chaotic systems. These notions were further developed in the context of climate models [See 2,3,4] and underpin the impact detailed here, specifically:
◾ The first stresses the fundamental limitations on interpreting the output from collections of today’s climate models as if they reflected the probability of future climate change [2];
◾ The second applies the insights of the first in the interpretation of what was then the largest ensemble of simulations with a complex climate model; this ensemble was generated by the climateprediction.net project of which Stainforth was co-founder and Smith Co-Investigator and a key player in its conceptual design [3];
◾ The third provides additional analysis while illustrating the practical and conceptual limitations in the probabilistic interpretation of such ensembles [4].
The basic insight is that the limited fidelity of a generation of models places an a priori cut-off on the quantitative informativeness of models, and therefore ensembles of models, from that generation. Simulation models are, of course, qualitatively different from the real-world system they attempt to represent. This is a particular problem in climate studies where the object of interest (the future state of the climate system) is expected to be qualitatively different to the state of the system for which we have some, limited, observations with which to assess our models. Technological and knowledge constraints impose shared weaknesses on all today’s models which limit the lead time and spatial scales on which simulations are realistic, or can be made informative by statistical post-processing. There is no statistical fix, just as a collection of simulations using Newtonian physics cannot be expected to account for non-Newtonian phenomena (like the orbit of Mercury) that require knowledge of general relativity, unless the information on those phenomena are in the observational data. Inasmuch as climate is an extrapolation problem, historical data are of limited use. The research provided a foundation for resisting the oversell of climate projections and protecting the credibility of science-based policy and decision making.
Key researchers:
Leonard Smith has been at LSE since 2000; David Stainforth has been at LSE since 2009; Edward R Tredger was a graduate student at LSE, 2006-2009; Ana Lopez, 2009-present; Erica Thompson, 2012-present.
References to the research
1. K Judd & LA Smith (2004) Indistinguishable States II: The Imperfect Model Scenario. Physica D 196: 224-242. DOI: 10.1016/j.physd.2004.03.020. http://eprints.lse.ac.uk/22175/
2. LA Smith, (2002) What Might We Learn from Climate Forecasts? Proc. National Acad. Sci. USA 4 (99): 2487-2492. http://eprints.lse.ac.uk/16905/
3. DA Stainforth, et al (2005) Uncertainty in the Predictions of the Climate Response to Rising Levels of Greenhouse Gases Nature 433 (7024): 403-406. DOI: 10.1038/nature03301.
4. DA Stainforth, MR Allen, ER Tredger & LA Smith (2007) Confidence, uncertainty and decision-support relevance in climate predictions, Phil. Trans. R. Soc. A, 365, 2145-2161. DOI:10.1098/rsta.2007.2074. http://eprints.lse.ac.uk/22222/
Evidence of Quality: publications in top-ranked journals, plus research grants as follows:
◾ EC Marie Curie Postdoctoral Fellowship - Dr Antje Weisheimer, EVK2-CT-2001-50012, Feb 02 – Jul 03.
◾ Climate Variability. Funded by University of California, San Diego. Grant # 10255373. Grant holder: Prof. L. Smith. £16,026. 01/11/2005 - 30/06/2006.
◾ Climateprediction.net: A practical platform for ensemble Earth System Modelling. NERC grant # NE/C515747/. Grant holder: Myles Allen, Oxford University. Co-Investigator: Prof L. Smith. ~£283k
◾ Ensemble-based Predictions of Climate Changes and their Impacts (ENSEMBLES). EU 6th framework programme / Integrated project. Grant # GOCE-CT-2003-505539-ENSEMBLES. Grant holder: Prof L. Smith £108,306. 01/09/2004 - 31/12/2009.
◾ Integrated Ocean Observing Systems (IOOS), NOAA, October 2007-October 2013. (~$500,000).
◾ Evaluating the economics of climate risks and opportunities in the insurance sector, Munich Re, October 2008 - September 2013. (£2.9M) PI: Prof L. Smith
Details of the impact
Impacts on public policy and services: Smith and Stainforth’s research has stimulated and informed policy debate on climate change since 2007. The Intergovernmental Panel on Climate Change (IPCC) Fourth Assessment Report cites Smith (Section 3, reference 2) as the sole article in noting the significant impact of structural model error on its “probability” distributions [A: pg. 797].
Their direct engagement in the policy development process is evidenced by their involvement with the pre-release criticism of UK Climate Projections 2007 (UKCP07). In response to a request from Defra for more information, Smith and Stainforth expressed concerns regarding the fidelity of UKCP07 [B]. Their subsequent involvement in post-study pre-release interactions with the Defra Chief Scientist and Sir Brian Hoskins contributed to the formation of an international review of UKCP07. There are two notable outcomes, firstly, the details of the review of UKCP07 have never been released, and secondly, UKCP07 became UKCP09. Smith’s research and views were quoted extensively in post-release criticism in both scientific and mainstream press [C].
The UK government leads the world in the search for climate forecasts at scales relevant for adaptation decisions. UK Climate Projections 2009 (UKCP09) initially claimed to provide detailed predictions (“post code” in space, daily in time) for this century. UKCP09 probabilities, described as best-available and dubbed “Bayesian”, provided core information which underlies the 2012 Climate Change Risk Assessment (CCRA).
The Royal Commission on Environmental Pollution, led by Sir John Lawton, invited both Smith and Stainforth to give evidence for their report [D]. Smith and Stainforth also contributed to the Treasury’s underpinning research on the economics of climate change, altering the framing of the Stern Review [E].
Impacts on the environment (policy debate on the environment have been stimulated or informed by research and research evidence): Smith and Stainforth’s improved interpretation of climate-model simulations has contributed to changes in how the UKCP09 is presented and government policy on climate change, allowing for better deployment of government funding. For example, the Climate Change Act 2008 committed the government to significant reductions in greenhouse gas emissions. Their work also affected the 2012 Climate Change Risk Assessment (CCRA) which represents a key part of the Government’s response to the Climate Change Act 2008, itself the first of a series of regular assessments required by law.
Having been severe critics of UKCP09, Smith and Stainforth were, at Defra’s request, both members of the initial framing meeting before beginning the CCRA, as well as reviewers of the draft report. They objected to the violation of methodological restrictions on the use of UKCP09 probabilities in the CCRA that had been agreed at the initial meeting, illustrating the intense pressure to over-interpret the output of climate models.
CATS’s distinct attention to the shift in the rational interpretation of climate predictions away from an optimization approach to a risk based approach founded on broad scientific insights and known vulnerabilities can be seen in the disclaimers of several significant publications (IPCC, UKCP09 program, the UKCP09 user guidance and other government reports), the focus on the Dutch alternative approaches to climate risk management, and the Treasury’s approach to the Stern Review [E].
Consequently, national meteorological services wishing to avoid UKCP09-like approaches within their own borders have been more effective at pressing alternative approaches. For example, Smith and Stainforth provided information to the Dutch meteorological office and Dutch government scientists [F] in support of their successful attempt to avoid a similar process to UKCP09. More recently, members of CATS were invited by the Dutch Government to a closed door meeting on the presentation of uncertainty in the 2013 IPCC Report [G].
Smith and Stainforth continue to engage with Defra, DECC, and a now independent UKCIP to improve how the level and coherence of UK climate information is evidenced and acknowledged.
Economic impacts (where performance of an existing business has been improved): The procurement of climate change research by industry provides further evidence of reach beyond public sector. CATS’ research has been used to clarify the limits on climate simulation for decision making and to improve the performance of EDF, EON, and NG. Smith provided a review of a UK Met Office commercial project purporting to provide high resolution meteorological information for “climate proofing” new long-lived energy infrastructure. In addition to written advice, Smith also represented industry in discussions with the Met Office to clarify the assumptions underlying the proposed study. Smith’s continued engagement with industry partners, Munich Re and Lloyd’s in particular, have allowed them to better interpret climate information [H, I].
International Reach: Smith’s involvement in the Baker Committee [J] for the US Central Intelligence Agency (CIA), focused on the extent to which the risks of climate change can be quantified. As a key participant in a series of meetings and reviews that led to the preparation of the Harvard Report, Professor Smith contributed his expertise, provided short turn-around calculations on the impact of station distribution, provided an in-depth review of the report and is named as one of the major reviewers. Dr James Baker states, “Professor Smith understands better than most climate scientists what the limitations of the science are and how to use statistical and physical analysis to draw robust conclusions for policy makers” [K].
Smith has represented the American Statistical Association at all three annual American Association for the Advancement of Science’s “Climate Day on Capitol Hill”; engaged with individual Senators’ and Congressmen’s offices (eight/year), and was invited to assist in developing both the American Statistical Association position on climate change and redrafting the American Geophysical Union’s position statement.
Wider Implications: Quantifying the financial value of this case study is nontrivial given the long forward shadow that today’s decisions on climate mitigation and adaptation will cast. Industrial sectors with large-scale infrastructure decisions (energy and ports in particular), and national security agencies familiar with unquantified risk (the US Central Intelligence Agency) have reconsidered their view of the fidelity and robustness of model-based projections, significantly reducing the likelihood of maladaptation through overconfidence of quantitative predictions. Direct costs/spend of particular studies reflect far too low a value (hundreds of thousands to millions of pounds). While the value at risk in the longer term is truly immense, attribution of any fraction of it to our actions is arbitrary. It is estimated that close to £11m was spent on UKCP09, which Smith and Stainforth show to be fundamentally flawed, furthermore, the ill-advised use of products from UKCP09 could cost orders of magnitude more.
Climate change is perhaps the greatest risk that humans will face in this century and the next, and alongside intervention, Smith contributes to public discourse [L]. Smith has publicly argued that effective adaptation to climate change costs a fraction of GDP with long term savings significantly greater. He and colleagues have established that the "probabilities" of UKCP09 are not a reliable foundation, either for adaptation planning or for risk assessment. The value at risk dwarfs the £10m spent on the studies themselves and the inopportune exposure of the weakness in the science base of UK adaptation plans risks a loss of public credibility in science based policy. UKCP09 and the CCRA are stronger than they would have been without the LSE's impact. Other countries including the Netherlands and the USA are more clearly aware of the mathematical shortcomings. The Netherlands have rejected the UK methodology and adopted an entirely different approach.
Sources to corroborate the impact
All Sources listed below can also be seen at: https://apps.lse.ac.uk/impact/case-study/view/7
A. Solomon, S., D. Qin, M. Manning, Z. Chen, M. Marquis, K.B. Averyt, M. Tignor and H.L. Miller (eds.). 2007. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. http://www.ipcc.ch/pdf/assessment-report/ar4/wg1/ar4_wg1_full_report.pdf
B. Professor Leonard Smith quoted in analysis by Pallab Ghosh, BBC News, 18 June 2009: https://apps.lse.ac.uk/impact/download/file/1556
C. Royal Commission on Environmental Pollution. (2010) Adapting Institutions to Climate Change, Twenty-eighth report. David Stainforth is acknowledged in the report as a key contributor. http://webarchive.nationalarchives.gov.uk/20110112040753/http://www.rcep.org.uk/reports/28-adaptation/documents/adaptation_final_report.pdf
https://apps.lse.ac.uk/impact/download/file/1557
D. HMT. (2010) The Stern Review. http://webarchive.nationalarchives.gov.uk/+/http:/www.hm-treasury.gov.uk/sternreview_index.htm
https://apps.lse.ac.uk/impact/download/file/1558
E. Testimonial from PBL Netherlands. This source is confidential
F. Invitation from IPCC. https://apps.lse.ac.uk/impact/download/file/1561
G. https://apps.lse.ac.uk/impact/download/file/1562
https://apps.lse.ac.uk/impact/download/file/1564
H. Munich Re, press release research collaboration. https://apps.lse.ac.uk/impact/download/file/1565
I. Harvard Climate Extremes Report (Baker Committee Report for the CIA, 2011-2013) http://environment.harvard.edu/sites/default/files/climate_extremes_report_2012-12-04.pdf
https://apps.lse.ac.uk/impact/download/file/1566
J. New Scientist. https://apps.lse.ac.uk/impact/download/file/1568