Summary: The events following publication of a new paper about our climate illustrates how science works in our society. It shows the warts — and its self-correcting nature. The result should please everyone, both scientists and laypeople.
Science is and has always been a social process, and hence usually confused and often nasty. Now its a mainstream subject of great importance, unlike the days when Newton lectured to an almost-empty hall. But the hype about a new article is so often unrelated to its actual importance or quality. After the media blitz comes debate, as its data and methods are scrutinized. For too many years much climate science work escaped this process, but no longer. Thanks to the Internet we can watch the process of science at work.
Of special interest is the role of outsiders, non-credentialed people who have own their own developed expertise in an area of climate science. As usual with social systems, they act as disruptive elements — revealing flawed processes that insiders agree to ignore. That our society allows (if not encourages) such people is one of our strengths.
A new paper in Nature has received massive press attention (more adulation than journalism), as providing more proof of AGW: “Warming of the Antarctic ice-sheet surface since the 1957 International Geophysical Year”, Steig, E.J., D.P. Schneider, S.D. Rutherford, M.E. Mann, J.C. Comiso, and D.T. Shindell. Nature, January 2009, pp 459-462. (Abstract for Steig 2009 here; full text here).
Contents
- Initial reaction: Praise from the pews
- Surprising results from a first look at the data
- Initial results from the involvement of outsiders
- Some scientists were cautious about the study at its publication
- My conclusions
1. Initial reaction: Praise from the pews
On the FM site Oldskeptic has strongly praised it in two comments, mirroring the mainstream media applause.
“The latest, Antarctic results have come from a newer, and yes better, methodology, incorporating the only real data source that covers the whole World, satellites.” (source)
“Note that Nature is a premier, peer reviewed scientific journal, if they could not prove their work it would not have been published.” (source)
The latter assurance is a bit odd, since it appears that the code and data are not yet fully archived. In several past major climate science articles even the reviewers did not see this vital information. No word on the code, but the lead author (Steig) has said he will archive the data soon.
Update: Not so, only a few “legitimate” folks deserve to have access to “all our data, including the intermediate steps” (source). His definition of “legitimate” is very restrictive. To Steig only a very few get access to key information on which public policy will be made – policies that might determine the fate of the world (as Al Gore has told us). And a clarification: the data inputs to the study are all public, but replication is not possible with knowing how he arrived at his results (some combination of code and intermediate work results).
Update: It gets worse. Now an editor at RealClimate says “some of these data are proprietary (NASA), but will be made available in the near future”. See comment #15 below for the lead author’s statements otherwise.
Update: It appears that the folks at RealClimate have no consistent definition of “data.” The latest spin about Steig 2009 is “{T}he raw data are public; the processed data (i.e. cloud masking) are not yet, but will be in due course. so relax”. OK, you have your orders. Relax. The authors will release this vital information when they are good and ready to do so.
Quite a soap opera. Too bad that the issues are so serious.
2. Surprising results from a first look at the data
So far investigation has been made of only two stations in the networks used (42 occupied, 65 automatic stations). Steve McIntyre (of Climate Audit) discovered that it was seriously defective, and posted about the error at 4:41 pm EST, on February 1 (Super Bowl Sunday). A later comment noted another station with defective data.
This ignited a somewhat incoherent, contradictory, defensive, and vituperative series of replies on RealClimate. See here and here. I recommend skimming the comments. Nobody likes auditors!
Corrections followed by British Antarctic Survey (BAS) and Steig, with another post defending the study by Schmidt at RealClimate (here). It is fun to watch the messy process of science in action.
3. Initial results from the involvement of outsiders
(a) In 2 posts at Climate Audit (Sunday 1 February and 2 February), Steve McIntyre identified an anomaly in the data for the “Harry” automatic weather station (AWS). On Monday BAS posted a note about the error (although originally identified by McIntyre, Gavin Schmidt notified BAS on February 1 or 2). The British Antarctic Survey (BAS) corrected the error on Monday morning — fast work, but without crediting McIntyre for catching the error. (source; for a 3rd party’s view of this episode see this by Pielke Jr., Professor of environmental studies at U Colorado).
(b) A comment by Hu McCulloch at Climate Audit uncovered another anomaly in the BAS data (the “Racer Rock” site), at 1:22am on 4 February. Later that day BAS posted a note about the error.
(c) A serious problem in climate science is failure to publicly archive data, which makes replication difficult and results in multiple and often differing “grey versions” circulating for the same data. The BAS corrections would have followed this pattern, as they initially corrected the original file. Having no version control in the public data, the data actually used in the Nature paper would vanish. McIntyre alterted BAS to this problem, and they archived the original (incorrect) data.
(d) The lead author, Steig, has posted a brief note about the impact of these data corrections, concluding that the efffect is tiny. Since he has not archived his complete code, it is not possible to replicate his analysis.
There are several other lines of inquiry about this study. Here are two, as illustrations.
(i) Accuracy of the satellite data: The trend they find is quite small (o.1 degree C per year), probably much smaller then the uncertainty of the data. No error bars are given in the article. As a general reference (which might not apply to the Steig 2009 paper), NASA’s FAQ page on the Atmospheric Infrared Sounder satellite project says:
Land surface temperature is problematical, since the emissivity of bare earth will vary greatly over the 50 km diameter spot in which our retrieval is made. Our estimated uncertainty at present is 2->3 K. {bold emphasis added}
(ii) Accuracy of the surface temperature record: There are few few stations in antarctica, and few of them have long records. Also there is evidence suggests that the automatic weather stations are often buried in snow (an excellent insulator) for long periods. See this article for pictures of visits to these stations to dig them out.
No firm conclusions can be drawn yet, but finding such data problems in a study’s first cursory review is not a good indicator of the its quality. Only time will tell.
4. Some scientists were cautious about the study at its publication
Note that initial media coverage included a few cautionary notes, ignored by true believers.
“This looks like a pretty good analysis, but I have to say I remain somewhat skeptical,” Kevin Trenberth, climate analysis chief at the National Center for Atmospheric Research, said in an e-mail. “It is hard to make data where none exist. (AP; Trenberth has been a lead author for the UN’s IPCC)
“One must be very cautious with such results because they have no real way to be validated,” says atmospheric scientist John Christy of the University of Alabama-Huntsville, who was not part of the study. “In other words, we will never know what the temperature was over the very large missing areas that this technique attempts to fill in so that it can be tested back through time.” (USA Today)
Roger Pielke Sr also had some questions, posted at his website. He is emeritus professor of the Department of Atmospheric Science at Colorado State U. See his Wikipedia entry for more information. His concluding comment:
In terms of the significance of their paper, it overstates what they have obtained from their analysis. In the abstract they write, for example, “West Antarctic warming exceeds 0.1C per decade over the past 50 years”.
However, even a cursory view of Figure 2 shows that since the late 1990s, the region has been cooling in their analysis in this region. The paper would be more balanced if they presented this result, even if they cannot explain why.
5. My conclusions
(1) Scientists eventually will sort all these questions out. True believers (on both sides), who accept only science that confirms their views, will remain unaffected no matter what the result (see here for examples). It’s called confirmation bias (see Wikipedia for more on this).
For the pro-AGW fanatics, does this result from religious fervor, or self-training in doublethink. Doublethink is the ability simultaneously to have two contradictory beliefs. Such as…
Scientists are reliable guides and sources of information. Scientists who challenge my beliefs do not understand basic science; they are incompetent. (See here for examples)
The rest of us, who will pay for the efforts proposed to reduce or stop global warming, might choose to wait until some of these questions are resolved before supporting large new public policy changes.
(2) This re-enforces My “wish list” for the climate sciences in 2009, changes that would put the climate sciences on track to meet our public policy needs. Whatever the cost (although trivial compared, for instance, to global military spending), I believe these things must be done ASAP.
- Raise the standards when applying science research to public policy questions.
- Provide greater transparency of data and methods used in climate science research.
- Provide third party review of the data, analysis, and modeling is necessary.
- Rationally apply the precautionary principle.
Afterword
Please share your comments by posting below. Per the FM site’s Comment Policy, please make them brief (250 words max), civil, and relevant to this post. Or email me at fabmaximus at hotmail dot com (note the spam-protected spelling).
For information about this site see the About page, at the top of the right-side menu bar.
For more information
To read other articles about these things, see the FM reference page on the right side menu bar. Of esp relevance to this topic:
Some of the posts on the FM site about climate change:
- A look at the science and politics of global warming, 12 June 2008
- Global warming means more earthquakes!, 19 June 2008
- An article giving strong evidence of global warming, 30 June 2008
- More forecasts of a global cooling cycle, 15 July 2008
- Two valuable perspectives on global warming, 4 August 2008
- President Kennedy speaks to us about global warming and Climate Science, 7 August 2008
- Good news about global warming!, 21 October 2008
- One of the most interesting sources of news about science and nature!, 27 October 2008
- “Aliens cause global warming”: wise words from the late Michael Crichton, 15 November 2008