The Climate Change Commission has drawn on several models to help shape its advice to government, which is due to be made public tomorrow. Two external reviewers explain more about the models – and how the results compare with overseas work.
Toon Vandyck and Matthias Weitzel, climate modellers from the Joint Research Centre, the in-house research branch of the European Commission, reviewed NZ’s climate policy modelling in December 2020.
The SMC asked how modelling has contributed to the Climate Change Commission’s work.
Toon Vandyck and Matthias Weitzel, reviewers of the New Zealand Climate Change Commission’s modelling, Joint Research Centre, European Commission, comment:
Note: Toon and Matthias speak in their own name, and their statements do not represent an official position of the European Commission.
How have you been involved with the Climate Change Commission’s modelling?
“We were consulted by the NZ Climate Change Commission to provide a review of the modelling framework used. We received model descriptions and/or presentations about the models, including preliminary results, as well as key draft chapters of the report that rely on modelling output. In addition, we had the chance to interact with the modelling team and other reviewers during two open, constructive and in-depth discussion sessions. Our evaluation is published on the NZ Climate Change Commission website, and provides a very positive overall view on the data used, as well as on the methods and modelling approaches employed.”
What do you see as the main strengths and limitations of the modelling that’s been used?
“A main strength of the modelling used by the NZ Climate Change Commission is the use of multiple, complementary models to assess consequences from multiple angles in a consistent way. We strongly believe in the usefulness of combining complementary models in a broader modelling toolbox. Using varying types of individual models – with different strengths – can highlight important aspects in more detail.
“Also, the modelling builds on established and state-of-the-art methods while reflecting the detail of specific emission trends in New Zealand. This includes a representation of the agricultural sector in a more detailed way than in other international models. The specific situation of sectors that are represented by a single or very few firms is taken into account, which illustrate that both baseline and scenarios cater to the particular conditions of the country. Overall, we have been impressed by both the scope and detail of the modelling efforts, and believe these provide a robust quantitative framework to support ambitious climate policy proposals.
“One caveat is that the potential consequences of the Covid-19 crisis are not fully taken into account in the modelling carried out by the NZ Climate Commission. Naturally, the uncertainty surrounding the duration of the economic crisis, the speed and the shape of the recovery are stark, therefore any modelling would likely be highly unclear. This especially holds for potential trends in persistent behavioural change, such as reduced transport demand due to more use of videoconferences to replace business trips, or increased teleworking. Clearly, it makes a lot of sense to align climate targets and recovery packages, and modelling work can inform this debate.
“The economic crisis following the pandemic also emphasises questions around equity and distributional aspects. There may be room to strengthen the modelling work in that direction in the future, as recent evidence suggests that while overall economy-wide impacts of climate policy may be limited, more substantial changes may occur within the economy. A complete picture of distributional impacts would account for effects on the income side (jobs) and the expenditure side (consumption patterns, including food and energy), and could highlight how additional revenue from carbon pricing can be used to facilitate the transition for vulnerable groups in society.”
What is the purpose of modelling in this context?
“Modelling can be an important tool to help identify consequences arising from climate policies. Most models used for policy advice do not try to prescribe what the optimal policy is, but rather provide answers of likely effects of potential policies that are under discussion. Modelling is most often used to compare different options or scenarios, such as achieving different targets, or using different policy instruments to achieve a target. These scenarios can then be compared to a baseline case, which depicts a situation under current policies and expected trends for economic growth and technological change.
“So these models should not be regarded as crystal balls aiming to predict the future; instead, they serve to compare alternative scenarios in a coherent, consistent way to evaluate the overall impacts of a policy proposal compared with a ‘do-nothing’ option. Consequently, the model runs also help to identify how regions, households and industries would be affected by a given set of policy measures.
“Importantly, models are simplifications of the real world. They build on assumptions – about future technology cost, growth of population and the economy, for example – which are inherently uncertain as they are projecting several decades into the future. A seemingly reasonable set of assumptions is taken for a baseline, and the alternative policy scenarios analysed with the models, so only the policy impact is measured when comparing the scenarios.
“For climate change mitigation scenarios, ‘techno-economic’ assumptions are crucial. Historically, global projections for 2020 emissions that were carried out several years or decades ago grossly underestimated cost reductions for renewables like wind and solar – and thus underestimated the role these technologies could play in avoiding emissions. On the other hand, early modelling efforts may not have anticipated the fast economic growth of China. Obviously, none of the climate policy models foresaw the pandemic the world is facing today. Both policies and modelling efforts should therefore strive to take new information on board when it becomes available.
“A model is a useful instrument to show trade-offs, and interactions, resulting from various policy options. For example, trade-offs might exist between efficiency and equity. Modelling can also show the interplay between different sectors and consequences of policies on international competitiveness. In revealing – and quantifying – the trade-offs and synergies, modelling studies can illustrate consequences of particular policy choices, and in this way help in mapping out the road to a clean, fair and competitive economy.”
What’s your assessment of the quality of the NZ Climate Change Commission’s model, compared to international standards?
“In our view, the models used appear up to international standards. The economy-wide model C-PLAN, for instance, derives from a well-established model developed at MIT, which is well-documented and used regularly in peer-reviewed publications. Importantly, we have the impression that the modellers did not use tools that were available off-the-shelf, but made substantial efforts to capture the important elements that are specific to the situation of New Zealand.
“Furthermore, the NZ Climate Change Commission has used not one, but three models. The approach to use a modelling toolbox, combining models with different focus areas, is considered best practice in multi-disciplinary research on climate action, and is applied to inform climate policy in the EU. Of course, these models and the underlying databases require frequent updates and enhancements to incorporate new scientific insights and policy developments. A broad societal debate, as well as more detailed and technical feedback about the models, data and assumptions, can help to shape and prioritise future model developments.”
The modelling estimated the cost of NZ’s climate change response as less than 1 percent of GDP through to 2050. How realistic do you think this is – and how does it compare with projections for other countries or regions?
“A number of determinants in modelling contribute to the policy cost. First, an important element is how the baseline (sometimes referred to as ‘business-as-usual’) is characterised, which serves as a benchmark for comparison. Quantitative policy analysis typically presents differences between a scenario with a new policy and a baseline that captures policies already in place. This comparison isolates the effects of the new policy from other developments, such as economic growth, changes in demographic structure, technological progress over time, or other policies already in place. Accounting for ongoing trends – such as technological progress – is crucial, and may limit the cost estimates of climate policy.
“Including options to abate emissions into the modelling is another key determinant of policy cost. Without these options, models might need to revert to output reduction to meet an emission target – a measure that is very costly in terms of GDP. ‘Computable general equilibrium’ models were first developed to analyse smaller changes around the status quo. However, transitioning the economy to net zero emissions might not be a small one in terms of changes to the production structure, so economic models can be further developed to cater to new policy questions. For instance, models now account for alternative technologies going beyond the gradual improvement of fuel efficiency in conventional vehicles.
“Once you factor in the available technology options, as done by the C-PLAN model (e.g. electric vehicles, various types of renewable electricity, and abatement options for biogenic methane), macro-economic impacts may appear moderate or small. Also, when expressed in terms of changes in annual economic growth rates, it becomes clear that economic effects are very small when compared to, say, the impacts of the pandemic. Research also indicates that this will be particularly the case with early climate action, while delayed action implies drastic emission cuts in later years, with correspondingly higher costs.
“To compare, the cost for achieving net zero emissions in the EU in 2050 was reported as between 0.3 and 1.3% of GDP in 2050, with having a higher reduction in gross emissions than New Zealand due to a proportionally lower carbon sink. This was calculated by a model that is similar to C-PLAN. The range in the EU figures further highlights that the GDP implications depend not only on the emission level to be reached, but also on how the policy is implemented, and what is assumed for policies in the rest of the world. The transition may increase aggregate employment, in particular if it is pursued through policies like switching taxation from labour to emissions.
“Also, other studies also seem to indicate minor impacts on the aggregate level, while suggesting that important structural changes may occur in terms of shifting economic activity and jobs in specific sectors and regions. And, the long-term nature of the projections brings some uncertainty. The European Commission therefore also presented economic estimates from a different model type, a so-called ‘macro-econometric’ model. This model type allows for investments to take place now, while being financed partly in the future. Through a climate policy-led investment boost, GDP then even exceeds levels of the baseline, resulting in macro-economic gains in the short run.”
Did the Commission’s modelling take into account potential issues with the supply of electric vehicles as global demand rises over the next decade?
“This question is best directed at the Commission. However, if global demand for electric vehicles or batteries exceed production capacities, this could result in a period of higher prices – but temporarily higher prices would also stimulate further investment and development in these technologies. A global scale-up of technologies (e.g. renewables like solar or wind, batteries, etc.) typically leads to declining costs from learning and economies of scale. If global vehicle production was set to have a higher share of electric vehicles than expected, this would likely reduce the cost of batteries and hence electric vehicles, making the transition easier.”
Is it typical for the actual model used in climate modelling to be released to the public?
“Transparency of modelling and analysis is clearly desirable for successfully engaging stakeholders. This can take several forms. The European Commission, for example, uses a combination of in-house modelling and modelling performed by external contractors. Some of the models used are privately owned by contractors, others are successive versions of public software specifically modified for a given scenario-analysis exercise. Among those modelling tools used, public release of the model’s source code is not the norm.”
“At the same time, extensive documentation – such as model descriptions, including mathematical formulas – is available, and the European Commission has published the data underlying the charts in its 2030 and 2050 assessments as a separate annex. Impact assessments can be quite lengthy and typically provide a great deal of numbers that allow for detailed scrutiny of model outputs.
“Further, the baseline, which serves as a starting point of policy analysis, is made available publicly periodically. In the case of the studies to inform the CCC advice regarding NZ’s climate policy, we understand that the models will be documented at open source. This may further support a transparent and constructive debate around the transition to a low-carbon economy.”
What types of modelling have been done to inform the NZ Climate Change Commission’s advice?
“Ambitious climate policies can only be effective when they are comprehensive in their coverage. This poses challenges for research, as typically it is not straightforward (nor desirable) to develop one model that captures all the complexities related to climate policy. Essentially, expertise is needed from a range of disciplines, to cover at least three aspects: technology, economy and fairness.
“Different modelling tools can inform different aspects of climate policy consequences. For example, some models are more detailed in depicting the technology options in the energy sector. These models might be better suited to highlight likely changes in the energy sector. Other models capture the entire economy – while they are more aggregate in representing different technologies, they take into account important interactions between sectors, and between countries through international trade.
“The NZ Climate Change Commission relies on three models, which have been linked to provide a consistent assessment. The three models consist of a model with greater sectoral and technological detail (ENZ), a model capturing economic interactions (C-PLAN), and a distributional tool (DIM-E) that uses the results of the economy-wide C-PLAN model. This combination of modelling tools allows for a richer analysis of the various aspects that are at the heart of any successful transition to a low-carbon, climate-neutral economy.”
How did the modelling assess differential impacts of climate policy? (e.g. for vulnerable workers, businesses, ethnic groups)
“To assess policy impacts for different groups, the Commission used results from all three modelling tools. Results from ENZ and C-PLAN typically work with more aggregate data, such that fine-grained impacts in different regions or for different groups of society are not explicitly represented. However, the DIM-E model can downscale results that are reported in C-PLAN for an average, representative household. This is of high policy relevance as it allows finding winners and losers, enabling policymakers to anticipate challenges and work on plans that can facilitate the transition in a socially-fair manner. The distributional modelling with the DIM-E model is particularly useful to indicate vulnerable workers and businesses that may be affected particularly by climate policy across regions, sectors and ethnic groups. Revealing concentrated impacts is essential to ensure a just transition to a low-carbon economy.”
No conflict of interest declared.