The Weather as algorithmic compression

Ian Murray seamus2001 at home.com
Tue Jul 3 12:03:40 PDT 2001


[from the NYT] July 3, 2001 Climate Research: The Devil Is in the Details By ANDREW C. REVKIN

In 1922, Dr. Lewis Fry Richardson, a British physicist with a penchant for grand ideas, described how to forecast the behavior of the atmosphere.

He had details wrong but the basic concept right: a suite of equations that, when applied to measurements of heat, cloudiness, humidity and the like, could project how those factors would change over time.

There was one grand problem. To predict weather 24 hours in advance, he said, 64,000 people with adding machines would have to work nonstop - for 24 hours.

Dr. Richardson pined for a day "in the dim future" when it might be possible to calculate conditions faster than they evolved.

That dim future is now. But while much has changed, much remains the same.

Supercomputers have answered Dr. Richardson's plea. Weeklong weather forecasts are generally reliable. But long-term climate predictions are still limited by the range of processes that affect the earth's atmosphere, from the chemistry of the microscopic particles that form cloud droplets to the decades-long stirrings of the seas.

With its oceans, shifting clouds, volcanoes and human emissions of heat-trapping gases and sun-blocking haze, earth remains a puzzle, said Dr. Michael E. Schlesinger, who directs climate research at the University of Illinois at Urbana- Champaign.

"If you were going to pick a planet to model, this is the last planet you would choose," he said.

So even as the evidence grows that earth's climate is warming and that people are responsible for at least part of the change, the toughness of the modeling problem is often cited by those who oppose international action to cut the emissions of heat-trapping gases.

And while American research centers once dominated this effort, they have recently fallen behind others overseas.

By many accounts, the dominant research effort is now at the Hadley Center for Climate Prediction and Research, 30 miles west of London. More than 100 scientists there are using extremely powerful computers just to explore long-term questions. Several recent studies by the National Academy of Sciences found that other countries had provided superior supercomputers for advanced climate research.

The academy found that efforts in the United States were hurt in the 1990's by a Commerce Department tariff of 450 percent on Japanese supercomputers. The tariff was lifted this spring.

The results are vexing for American scientists, said Dr. Maurice Blackmon, director of climate studies at the National Center for Atmospheric Research in Boulder, Colo.

Last week, Dr. Blackmon said in an interview, he met a climatologist from a Swiss university who was preparing to run a copy of the Boulder laboratory's most sophisticated model on a supercomputer in Bern "six to eight times faster than we can here."

"That's the definition of frustration," Dr. Blackmon said.

Even with the best computers, though, important parts of the climate puzzle still elude both the machines and the theoreticians, although progress is being made.

Dozens of mathematical models of the atmosphere and things that affect it are being applied to the problem. The most ambitious of these - about 20 or so around the world - simulate not only the air but also the oceans and, increasingly, other dynamic features of the planet: its shifting sea ice and glaciers, its cloak of vegetation, its soils.

These imagined earths are generated by supercomputers that tear through decades in a day, creating a compressed view of how the climate might behave if one influencing force or another changed.

The biggest models have improved substantially in the last few years, with many no longer requiring "flux adjustments" - essentially fudge factors - that were once needed to prevent the machine-generated, theoretical climates from drifting out of the realm of the possible.

The signal achievement in recent years has been the accumulation of evidence, much of it from advanced models, that rising levels of greenhouse gases in the air have discernibly warmed the planet.

But moving beyond that general conclusion presents enormous problems.

"We will of course improve our models," said Dr. Mojib Latif, the deputy director of the Max Planck Institute for Meteorology in Hamburg, Germany, "but I don't really see the biggest or most important results changing in the next 10 years."

"In terms of policy," Dr. Latif said, "the models have done their job."

But the models have not clearly answered a pivotal question: how sensitive is the climate to the intensifying greenhouse effect? In other words, how big is any coming climatic disruption likely to be?

The models still predict essentially the same wide range that was calculated nearly 30 years ago: roughly an average rise of 3 to 8 degrees Fahrenheit if greenhouse gases double from the concentrations measured before coal and oil burning and forest cutting significantly altered the atmosphere.

And that is a global prediction. When asked to predict local effects of global warming - say, on the Southwest or Europe - the margins of error grow, and competing models stray far and wide.

For example, the change in climate in particular places in the models still varies markedly depending on how programmers start the simulation - what values they pick for the initial conditions on earth.

The first set of numbers plugged into the matrix of equations is always an educated guess, said Dr. Curtis C. Covey, a physicist at the Lawrence Livermore National Laboratory who compares the performance of various models.

"Can you tell me what the initial conditions were in 1850? Can anybody?" Dr. Covey asked.

In fact, some top modelers say even the most powerful simulations can be pushed only so far before they reach limits of usefulness.

Dr. Syukuro Manabe, who in 1969 helped create the first model coupling the atmosphere and oceans, said in an interview that the most advanced versions had already gone too far.

"People are mixing up qualitative realism with quantitative realism," said Dr. Manabe, who did most of his work at the Commerce Department's Geophysical Fluid Dynamics Laboratory in Princeton, N.J. He is now helping Japan create a $500 million supercomputing center in Yokohama that is expected to dwarf all the other climate research efforts.

He explained that models incorporating everything from dust to vegetation looked more and more like the real world but that the error range associated with the addition of each new variable could result in nearly total uncertainty. Speaking of some climate models, he said, "They are more caught up in trying to show what a great gadget they have than in showing how profound their study is in understanding nature."

Of course, Dr. Manabe said, the models still play a vital role in earth science, providing practically the only means of looking into the future, albeit through a cloudy lens.

And there are still many ways to sharpen the picture, he and other climate experts said.

First, there is improving resolution and speed. Though climate modelers use the same machines that help nuclear weapons designers and astrophysicists, they still face a big trade-off between detail and time.

The most advanced models consist of several hundred thousand lines of computer code that divide the air, land and oceans into a grid of hundreds of interacting boxes. As conditions change in one box, the changes ripple through neighboring boxes.

Until now, modelers had been forced to dice the atmosphere into a grid where each box was about 185 miles on a side. The best ocean models right now are composed of cubes about 85 miles across. The Hadley Center is creating a new model that will take the ocean resolution to cubes about 20 miles on a side, which is detailed enough to capture the important eddies that shunt heat and carbon dioxide from the atmosphere into the depths.

But Dr. Geoff Jenkins, the director of climate prediction at the center, noted that the three-dimensional nature of the problem meant that each doubling of resolution required a 16- fold increase in computing. In tests, Dr. Jenkins said, the new model "completely clogged up" one of the center's supercomputers.

Many features of the earth that are critical to climate change remain much smaller than the model boxes so must still be approximated.

Dr. Blackmon, at the National Center for Atmospheric Research, said features as important as California's Central Valley and the mountain ranges around it remained invisible.

"We can't tell you anything about what's going to happen there," he said. To do so would require a grid of boxes 19 miles on a side, he said. To achieve that detail would require computers 1,000 times as powerful as those at the research center.

Dr. Manabe said the goal of the Japanese project, the Frontier Research System for Global Change, was to use vastly greater computer power to accelerate model runs, doing more work in less time and providing a much finer-scale view of what lies ahead.

The center will have 5,120 linked high-speed processors, able to perform 40 trillion calculations per second. The most powerful computers currently used for climate modeling have about 1,000 slower processors and crunch numbers at about a hundredth of that speed.

But more brute computing power is only part of the solution.

Ronald J. Stouffer, a senior meteorologist at the fluid dynamics laboratory in Princeton, said that the key to progress was to move ahead in three realms at once: in the models, in the basic research into the processes that are mathematically represented in models and in the measurements of environmental change that will allow the testing of models.

"It's a triangle," Mr. Stouffer said. "Observations, modeling and theory. Any one can lead the other two for a while but can't lead much before you get stuck." That leads the climate scientists inevitably back from their simulated worlds to the real one.

The modelers have been lobbying for more money, not just for their work but also for ongoing measurements of change in the oceans, atmosphere, polar ice and forests. The value of this work was illustrated this spring, many say, when 50 years of ocean temperature measurements showed warming that matched the models' projections.

Other large mysteries still confront the researchers when they look earthward. Within clouds, for example, the chemistry and physics of the particles like soot and sea salt that form droplets are only slowly being revealed, scientists say.

A small change in the way droplets form could have a large impact on the climate, said Dr. Jenkins, in Britain. He said that Dr. Anthony Slingo, another scientist there, found a decade ago that in theory, a decrease or an increase in the size of water droplets of just 10 or 20 percent "could either halve or double the amount of climate change you'd get."

Eventually, laboratory work and observations should narrow that range, many climate experts say, but uncertainty will always remain.

"The best we can do," said Dr. Manabe, in Yokohama, "is to see how global climate and the environment are changing, keep comparing that with predictions, adjust the models and gradually increase our confidence. Only that will distinguish our predictions from those of fortunetellers."



More information about the lbo-talk mailing list