Model-land is a hypothetical world in which our simulations are perfect, an attractive fairy-tale state of mind in which optimising a simulation invariably reflects desirable pathways in the real world. Decision-support in model-land implies taking the output of model simulations at face value (perhaps using some form of statistical post-processing to account for blatant inconsistencies), and then interpreting frequencies in model-land to represent probabilities in the real-world.
The following is something I see nearly every day in the media – where the model output is presented as the real world (even when the real world is different):
As a trivial example, when writing about forecasts of household consumption, energy prices, or global average surface temperature, many authors will use the same name and the same phrasing to refer to effects seen in the simulation as those used for the real world. It may not be the case that these authors are actually confused about which is which, the point is that readers of conclusions would benefit from a clear distinction being made, especially where such results are presented as if they have relevance to real-world phenomena and decision-making.
…
For what we term “climate-like” models, the realms of sophisticated statistical processing which variously “identify the best model”, “calibrate the parameters of the model”, “form a probability distribution from the ensemble”, “calculate the size of the discrepancy” etc., are castles in the air built on a single assumption which is known to be incorrect: that the model is perfect.
….
It is not clear why multi-model ensembles are taken to represent a probability distribution at all; the distributions from each imperfect model in the ensemble will differ from the desired perfect model probability distribution (if such a thing exists); it is not clear how combining them might lead to a relevant, much less precise, distribution for the real-world target of interest.
Source: 1662970102.pdf
The last paragraph quoted above is something that has long bothered me about model ensembles.
This paper is a good read. Click on the link above to read the full paper.
Thompson, Erica L.; Smith, Leonard A. (2019) : Escape from model-land, Economics Discussion Papers, No. 2019-23, Kiel Institute for the World Economy (IfW), Kiel. Retrieved from: https://www.econstor.eu/bitstream/10419/194875/1/1662970102.pdf
