Comments on Held: There are a lot of simple models because they are used for a lot of different purposes.
As tools/machines/data advance, you should re-evaluate your problem to see how you can move further along in understanding.
Use of MIPS:
Sometimes the results are simply tuning. [Individual models are tuned to observations.]
Sometimes the MIPS result in the identification of systematic biases.
Isaac’s hypothesis/model for improving comprehensive models. Build elegant model to understand and this knowledge will give a more efficient path to improving the comprehensive models. Problems:
are you sure the idealized models relevant to the real world?
Are the results of the idealized models used to confront the models? Why or why not?
Is the enterprise too difuse to progress efficiently in the skill of simulation and in building knowledge of the system (ie, constructing a theory of climate)?
Should all simple models be designed to confront the comprehensive models?
Simple models can be used to understand, or to constrain ….
Simple models can be used to shape, define, articulate ideas …
To move the modeling progress forward, do you need new administrative or programmatic elements or incentives for scientists?
Building knowledge using this give and take approach with comprehensive models and conceptual models is time consuming and people intensive. How can you afford to do it, in practice?
Perhaps all conceptual models should have disclaimer statements and honest assessments of limitations/potential show-stoppers…
If you have only a few conceptual models that are going to be the research horses, then who defines what these models will look like?
Isn’t it important to test simple models against each other?
Student Comments (delivered prior to class)
I completely agree with the idea of a model hierarchy. Models of intermediate complexity are definitely lacking. More models at the high complexity end are needed, but I think that more high end models is not necessarily the main issue there.
When building these top end models I feel like the design is insestuous. What I mean is that these high end models are all built with the more or less the same assumptions. This seems to be a completely practical symptom of the problem of building a complicated model with limited resources. Thus, comparisons between model results to determine the 'uncertainty' in present day model completely ignores the fact that all these guys are likely building models based on the same assumptions. It is erroneous to treat these models as a true distribution of thought.
People that think significantly differently from the mainstream modelers will likely have trouble getting funded to build a competing model because their proposals will be reviewed by mainstream modelers. This heads off the much of the 'distribution' in the model output at the pass. I'm sure that the review process is worthwhile here, but I feel that there is a certain amount of peer pressure to jump on board with some ideas rather than pursuing others.
I suppose this is where a model of intermediate complexity would be useful in proving that one assumption or other in a higher end model may be questionable. Thus, proving the need for a top end model of new structure.
Of course, I'm not a modeler, so this is an outsider's view of the situation.
It is my impression that Held's view of how biology developed it's intermediate models is a little simplified. I think that biologist go through the same growing pains of picking test cases that Held feels is holding atmospheric sciences back. There are many different options in biology to choose from (plant, animal, reproduction rate, genome complexity, ...). At least meteorologists have to spend less effort keeping their models alive. But biologists have an inherent humility in their scientific culture which allows them to swallow their pride for a bit and test some ideas on a simple plant that they think should apply to corn or soy. It is a respect and understanding of the complexity of the problem.
This is a great lesson on how to determine whether a problem has been proven or not. This gave me more ideas for my 'how to solve it: complex problems' list. did you prove that your theory is the only explanation, or does your theory just fit the data? is your theory only a mathematical explanation (a translation into another language) or does it include a physical explanation?
It seems like the ideas of stability and instability, attractors, etc... from chaos science are pretty crucial for us to address a theory of climate. Why one stable situation and not another. Where do the regime shifts/bifurcations happen? What triggers a shift? Why is the trigger as large as it is?
Jessep: You want answers?
Kaffee: I think I'm entitled to them.
J: You want answers?
K: I want the truth!
J: You can't handle the truth! (yet???)
A few good men (1994)
To some extent, I think that this is what Held is telling us. He implies that our knowledge about the climate system is not mature enough to handle the full complexity of the real system, so that it would probably be more profitable to really understand a much simpler system (e.g. the 2-layer model) and hope that what is learnt from it can be used to build an understanding for the more complex system. As he points out, biologists are fortunate in that evolution provided a natural hierarchy of complex systems, with the more complex systems built on top of simpler ones. In climate sciences we try to reverse-engineer the man to get the fly, and there's no obvious way of determining what is superfluous and what essential for understanding a particular aspect. Of course, we can not strip a man of all his complexity and still call him a man. Therefore, any simpler model will have a limited applicability and is only meaningful in the context of the specific question being addressed.
But Held goes beyond that. He advocates the selection of just a few simple models, on which a scientific community would be focused. This is in contrast to creating short-lived models that serve to support an idea and are then discarded. I think that the underlying issue is: should our immediate * object of study * be the full climate system or an idealized system on its own right, which may or may not correspond to nature? An extreme and illustrating example of the latter is Lorenz's butterfly system, whose study may have been used to build a theory of weather predicatability, yet at no point we claim that this system is a model for weather. This kind of studies, in which we develop ways of thinking about problems rather than actually solving real-life problems is probably the necessary first step for dealing with complex systems. The question would then shift from "how can I simplify my real-world system in order to make it manageable without affecting its representativeness?" (emphasizing realism) to "what system is simple enough to be tractable but that has the potential of exhibiting behaviour as rich as the real thing?" (emphasizing elegance).
Thoughts from this weeks readings:
Lorenz directly goes over the interaction of theoretical versus
observational ways to look at critical aspects of earths circulation. 40
years later, there is still something of this divide in how atmospheric
science can be viewed and/or presented to different audiences. In my
even when they are pretty easily accessible and/or statistically robust
(that seems to be the story of my research career thus far).
I'm left with the fact that determining how complex a model needs to be
useful theoretically in thinking about particular problems is tough- we
can't expect people to get that right most of the time. It's a good test
to think about when looking at a particular problem though. And it's good
practice to try to explain the salient features of your theoretical model
of choice in a reasonably short paper. (which too few people do)
Theoretical folks need to make their work accessible and be willing to
look at observations and applied folks need to be willing to think about
This paper is right on. In theory.
However; who will decide which simpler models to re-use. And how will we do this without overlap. Held does say that some overlap is good, so maybe that won't be a problem.
To some degree, isn't this what happened with the Eady model?
p. 413 ...:why do we have secondary circulations at all?
p.418: In a sense, then, these global currents are explained; they are demanded by the system of equations which governs the atmosphere.
p.418: What is lacking in this instance is a real physical insight into the mechanism through which the troughs and ridges acquire their typical orientation
Sense then, hasn't this insight been provived? Once the mathematics 'corners' an observed phenomenon, isn't it just a matter of time until we piece together the correct set of thought to explain the phenomenon with 'insight'. Why isn't math enough? Because we can't discuss it in pubs?
I think a very important point made by Held is the importance of elegance to understanding. We cannot cram everything imaginable into a model and expect to gain insight, or better understanding of the physics. Big GCMs may not be the best place to look for understanding a problem.
Held argues for fewer idealized models that are more widely applicable and more ‘kitchen sink’ models to better sample the space of possible models. If as we have posited, questions cannot be separated from the model used, will focusing on a few idealized models limit the type of questions we are asking in our field? Is that a problem or just a focusing of our energies. Is creating an ensemble of big climate models worth the resources and energy?
The Lorenz reading seemed quite relevant despite being written 40 years ago. That’s probably and indication that it contains some useful ideas…Our models will always have to include approximations and parameterizations. Lorenz shows that in a system with instability will diverge from reality. How can we estimate the magnitude of such an effect. Lorenz says maybe we’ll forecast as well a week in advance as was done for 1-2 days in advance. What are the analogous numbers for climate?.
On choosing the problems, I think there is no recipe. The great freedom that we have in complex problems is to pick/sense/define the smaller problems. There cannot be a general algorithm for disassembling the bigger problem. That is where the intuition, brainstorming, creativity, luck, or beer come in.
I think you are right. We are rarely disciplined enough to follow through on the more boring smaller problems. I guess it highlights the importance of properly characterizing the meaning of what you have done - how the smaller problem fits in, and what remains.
More common too, is the fact that bigger problems are so vaguely/imprecisely stated that it is not clear what would actually formally count as a solution!
Have fun tomorrow, look forward to catching up.