New York Times, February 20, 2007

THE PROBLEMS IN MODELING NATURE, WITH ITS UNRULY NATURAL TENDENCIES

[Rachel's introduction: Mathematical models often provide the basis for numerical risk assessments, which are government and industry's main tools for assuring the public that chemicals, pesticides, food additives, biotechnology, nanotechnoology and radiation are all "safe." But mathematical models are often based on faulty data and assumptions.]

By Cornelia Dean

When coastal engineers decide whether to dredge sand and pump it onto an eroded beach, they use mathematical models to predict how much sand they will need, when and where they must apply it, the rate it will move and how long the project will survive in the face of coastal storms and erosion.

Orrin H. Pilkey, a coastal geologist and emeritus professor at Duke, recommends another approach: just dredge up a lot of sand and dump it on the beach willy-nilly. This "kamikaze engineering" might not last very long, he says, but projects built according to models do not usually last very long either, and at least his approach would not lull anyone into false mathematical certitude.

Now Dr. Pilkey and his daughter Linda Pilkey-Jarvis, a geologist in the Washington State Department of Geology, have expanded this view into an overall attack on the use of computer programs to model nature. Nature is too complex, they say, and depends on too many processes that are poorly understood or little monitored -- whether the process is the feedback effects of cloud cover on global warming or the movement of grains of sand on a beach.

Their book, Useless Arithmetic: Why Environmental Scientists Can't Predict the Future, originated in a seminar Dr. Pilkey organized at Duke to look into the performance of mathematical models used in coastal geology. Among other things, participants concluded that beach modelers applied too many fixed values to phenomena that actually change quite a lot. For example, "assumed average wave height," a variable crucial for many models, assumes that all waves hit the beach in the same way, that they are all the same height and that their patterns will not change over time. But, the authors say, that's not the way things work.

Also, modelers' formulas may include coefficients (the authors call them "fudge factors") to ensure that they come out right. And the modelers may not check to see whether projects performed as predicted.

Eventually, the seminar participants widened the project, concluding that erroneous assumptions, fudge factors and the reluctance to check predictions against unruly natural outcomes produce models with, as the authors put it, "no demonstrable basis in nature." Among other problems, they cite much-modeled but nevertheless collapsed North Atlantic fishing stocks, poisonous pools unexpectedly produced by open pit mining, and invasive plants and animals that routinely outflank their modelers.

Two issues, the authors say, illustrate other problems with modeling. One is climate change, in which, they say, experts' justifiable caution about model uncertainties can encourage them to ignore accumulating evidence from the real world. The other is the movement of nuclear waste through an underground storage site at Yucca Mountain in Nevada, not because it has failed -- it has yet to be built -- but because they say it is unreasonable to expect accurate predictions of what will happen far into the future -- in this extreme case, tens or even hundreds of thousands of years from now.

Along the way, Dr. Pilkey and Ms. Pilkey-Jarvis describe and explain a host of modeling terms, including quantitative and qualitative models (models that seek to answer precise questions with more or less precise numbers, as against models that seek to discern environmental trends).

They also discuss concepts like model sensitivity -- the analysis of parameters included in a model to see which ones, if changed, are most likely to change model results.

But, the authors say it is important to remember that model sensitivity assesses the parameter's importance in the model, not necessarily in nature. If a model itself is "a poor representation of reality," they write, "determining the sensitivity of an individual parameter in the model is a meaningless pursuit."

Given the problems with models, should we abandon them altogether? Perhaps, the authors say. Their favored alternative seems to be adaptive management, in which policymakers may start with a model of how a given ecosystem works, but make constant observations in the field, altering their policies as conditions change. But that approach has drawbacks, among them requirements for assiduous monitoring, flexible planning and a willingness to change courses in midstream. For practical and political reasons, all are hard to achieve.

Besides, they acknowledge, people seem to have such a powerful desire to defend policies with formulas (or "fig leaves," as the authors call them), that managers keep applying them, long after their utility has been called into question.

So the authors offer some suggestions for using models better. We could, for example, pay more attention to nature, monitoring our streams, beaches, forests or fields to accumulate information on how living things and their environments interact. That kind of data is crucial for models. Modeling should be transparent. That is, any interested person should be able to see and understand how the model works -- what factors it weighs heaviest, what coefficients it includes, what phenomena it leaves out, and so on. Also, modelers should say explicitly what assumptions they make.

And instead of demanding to know exactly how high seas will rise or how many fish will be left in them or what the average global temperature will be in 20 years, they argue, we should seek to discern simply whether seas are rising, fish stocks are falling and average temperatures are increasing. And we should couple these models with observations from the field. Models should be regarded as producing "ballpark figures," they write, not accurate impact forecasts.

"If we wish to stay within the bounds of reality we must look to a more qualitative future," the authors write, "a future where there will be no certain answers to many of the important questions we have about the future of human interactions with the earth."