Summary: We havea rare moment when even laypeople can understand a major issue in a cutting edge scientific debate. Not only does this give a hint (but no more) about the strength of the two “sides” (global warming proponents vs skeptics), but also illustrates one of the most important effects of the Internet: opening previously closed social processes to public view, thereby stripping elites of their control of information — and thereby reducing their power. As Glen Reynolds (the Instapundit) says, we have created an “Army of Davids.”
Modern debates in the sciences are by their nature difficult for non-specialists to understand, let alone evaluate. Occasionally one facet will become so clear that even laypeople can judge for themselves. We have such a moment in one of the most important debates of our time — about climate trends.
The primary complaint of skeptics has been about the availability of data and methods used to produce the results on which advocates urge major public policy changes. Without this replication is impossible for scientists or interested “outsiders” to the establishment.
The response has been a grudging of data, often partial and poorly documented. The battle has been surprisingly difficult because most journals’ require documentation and release of this information at publication. Even more striking, much (most?) of this research is publicly funded — and hence keeping it secret usually violates regulations of the funding agencies.
We saw the battle lines reform with the publication — to a massive publicity campaign — of “Warming of the Antarctic ice-sheet surfacesince the 1957 International Geophysical Year”, Steig, E.J., D.P. Schneider, S.D. Rutherford, M.E. Mann, J.C. Comiso, and D.T. Shindell. Nature, January 2009, pp 459-462. (the abstract for Steig 2009 is here; you can see the full text here).
Steig 2009 was immediately greeted with requests for the supporting documentation. The replies tell us much about the state of climate science, and the foundation it provides for large public policy action. Read and decide for yourself; the tale is quite clear. Note the dates!
Contents
- Have the authors of Steig 2009 made their methods and data available to the public?
- Reminder: some scientists were skeptical after reading Steig 2009
- Significance of the debate over Steig 2009
- Nature’s policy about data availability
- Afterword
- Where to go for more information
1. Have the authors of Steig 2009 made their methods and data available to the public?
Nature, like most professional journals in the Sciences, has clear policies requiring supporting data to be available to the public. See section 4 below for details. How well have Steigandhis co-authors complied with these?
(1) Email from co-author Eric Steig to Steve McIntyre on January 23 (source):
“I have always intended to provide all the material on line; I wasn’t allowed to do this before the paper was published. I would have done it already but have been busy answering emails. I should have these up on line next week.”
(2) From Josefino Comiso (NASA Goddard Space Flight Center), in reply to request for data by McIntyre on January 23 (source):
“Thanks for your request for AVHRR {satellite} surface temperature IR data. I am actually planning to have the entire data set archived in the near future and as soon as I get the associated document that describes the data and discusses the errors and caveats completed. The data are indeed on a gridded monthly basis. I will let you know how to access them in the web as soon as they are archived and ready to be downloaded. Best Wishes”
*** The first two replies said that the data would soon be available. But then the story changes from “will be made available” to “the data was available.”
(3) From Steig’s website at the U of Washington (no date posted):
“All of the data used in the temperature reconstructions are from publically available data sources.”
(4) Steig replying to a comment at RealClimate on February 2:
[Response: ALL of the data that were used in the paper, and EXACTLY the code used in our paper have been available for a long time, indeed, long before we published our paper. This is totally transparent, and attempts to make it appear otherwise are disingenuous. This has always been clear to anyone that asked. If you wanted to do the work yourself, for legitimate reasons, you could do so. If the point is to “audit” our work, it makes no sense whatsoever to provide all the intermediate products used in our analysis. That would defeat the purpose of the supposed “audit”.–eric]
(5) Steig replying to a comment at RealClimate on February 2:
“… Excluding your study, would it be your opinion that code and data archiving practices are adequate in the climate science community, overall? This is a common complaint on sites like CA.”
[Response: Yes, I think we do a fine job. We could do better, but we do very very well. I’ve never had trouble getting data that I need from others. Indeed, our Nature study was based entirely on freely available data and code. –eric]
(6) Steig replying to a comment at RealClimate on February 3:
[Response: Nope I don’t think it is a bit strong. I released an electronic version of our data and links to all the original data and code almost as soon as our paper was published. Anyone paying attention would know that. … -eric]
*** Up to this point the replies on RealClimate have been clear and unequivocal: the data is available to the public. Now the story shifts again. Its’ available to some people, or its proprietary NASA data, or it will soon be available.
(7) Steig replying to a comment at RealClimate on February 3; bold emphasis added:
“… What I don’t understand is simply, why don’t you routinely publish those intermediate steps. Most people don’t have the scientific background, or the free hours, to truly dig into the data. However most people with real-world experience know transparency when they see it. …”
[Response: I do routinely make all our data available, as does everyone else that I know. In this particular case, anyone legitimate who has asked for all our data, including the intermediate steps, has received it. To continue with the analogy with financial auditing, let me very clear on what I mean by legitimate: In the business world, auditors
- don’t publicly accuse a company of withholding data prior to requesting said data;
- are not self-appointed;
- have to demonstrate integrity and competence;
- are regulated.
On this point, if you are suggesting that Steve McIntyre be regulated by an oversight committee, and have his auditor’s license revoked when he breaks ethical rules, then we may have something we can agree on.–eric]
(8) Gavin Schmidt (NASA Goddard Institute for Space Studies; see his bio) replying to comment #38 by by Nicolas Nierenberg on February 9, posted at RealClimate (post titled “On Replication”; bold emphasis added):
“Once again rather than the general, let’s be specific. I believe it removes the issues of how much extra work would be created. Dr. Steig has said that he is willing to provide the data to legitimate researchers. My response is to simply post what he would provide. I still haven’t heard from Dr. Schmidt what the objection is to that concept.
“Also as to the specifics in Dr. Steig’s paper. I believe that there is probably sufficient information on AWS trends. However I don’t think there is sufficient information to reproduce the gridded AVHRR {satellite} temperature results. They are quite dependent on corrections for clouds, and manipulation to produce temperature values as I understand it.”
[Response: Joey Comiso is apparently working on making that available with appropriate documentation – patience. – gavin]
(9) Reply to comment #63 by Nicolas Nierenberg at RealClimate on February 9:
“… Other than a couple of people the conversation seems to be converging on the fact that providing the code and data is preferable. Dr. Schmidt has said that this will be done in the case of the Steig paper.”
[editor note: this was done in the case of Steig et al withrespect to code though perhaps not with as much hand-holding as you seem to want. some of these data are proprietary (NASA), but will be made available in the near future]
(10) Reply to comment #136at RealClimate on February 11:
“If possible, please provide more information about the ‘proprietary NASA data’ used in Steig 2009. There have been several (at least 4) clear statements from Dr. Steig, both at his website and RealClimate, that his study used only ‘publically available data sources.’”
[reply: the raw data are public; the processed data (i.e. cloud masking) are not yet, but will be in due course. so relax]
(11) Gavin Schmidt replying to Comment #225by captdallas2 on February 16 at RealClimate:
“The satellite data was IR not RSS or MSU so the replicators could not gather the data needed to complete the replication. Do you know if that data is available online?”
[Response: Joey Comiso is apparently working on the data preparation along with sufficient explanation to make it usable. I have no particular insight into the timetable. – gavin]
Conclusion: You have your orders. Relax. The authors will release this vital information when they are good and ready to do so. Trust them, and rely on the fawning articles in the mainstream media.
2. Reminder: some scientists were skeptical after reading Steig 2009
Note that initial media coverage included a few cautionary notes, ignored by true believers.
“This looks like a pretty good analysis, but I have to say I remain somewhat skeptical,” Kevin Trenberth, climate analysis chief at the NationalCenter for Atmospheric Research, said in an e-mail. “It is hard to make data where none exist. (AP; Trenberth has been a lead author for the UN’s IPCC)
“One must be very cautious with such results because they have no real way to be validated,” says atmospheric scientist John Christy of the University of Alabama-Huntsville, who was not part of the study. “In other words, we will never know what the temperature was over the very large missing areas that this technique attempts to fill in so that it can be tested back through time.” (USA Today)
Roger Pielke Sr also had some questions, posted at his website. He is emeritus professor of the Department of Atmospheric Science at Colorado State U. See his Wikipedia entry for more information. His concluding comment:
In terms of the significance of their paper, it overstates what they have obtained from their analysis. In the abstract they write, for example, “West Antarctic warming exceeds 0.1C per decade over the past 50 years”.
However, even a cursory view of Figure 2 shows that since the late 1990s, the region has been cooling in their analysis in this region. The paper would be more balanced if they presented this result, even if they cannot explain why.
3. Significance of the debate over Steig 2009
First: the smokescreen has not stopped replication of Steig 2009
The smokescreen to conceal key aspects of has not stopped efforts to reverse-engineer it. Outside experts have made great progress. The initial skirmishes were reported in Science in action, a confused and often nasty debate that produces real progress (5 February 2009). Much progress has been made since then, but that is a subject for another time. You can see their work at many websites, such as Climate Audit and The Air Vent.
Here is the first result: “Steig’s Antarctic Heartburn“, by Jeff Id and Jeff C, posted at Anthony Watt’s Watts Up with That, 28 February 2009. It’s long; you might find the 16 page PDF easier to read.
Second, there is an important and long-term dynamic at work here.
The printing press unlocked knowledge, perhaps most important for the 15th century it made words of the Bible available to everyone. That blasted away the foundation to the power of the medieval Catholic Church’s priests . Today the Internet is doing the same thing to the guild of scientists.
Cloistered in a small number of institutions, they controlled the process of pure science by dominating key institutions. The major journals, academia (the gateway to the guild) and the great quasi-government research organizations (e.g., NASA, IPCC). The relevant knowledge was also widely available in the private sector, but most of these people were isolated outsiders — impotent to affect the institutional dynamics of the science establishment. The Internet changed this balance — linking them together and gave them voices. Now they are slowly realizing that they are powerful.
Although lacking the magic juice of corporate and government funding, modern communications and computer technology allows them to pool their resources. Their numbers, properly combined, allow them to challenge even the most entrenched scientific elites. Today that means the climate scientists.
SSSailor comments on another post that this is ..
“What Glen Reynolds (Instapundit) has described as an “Army of Davids” is in real terms, a Distributed Intelligence. The rate of change of information acquisition and the velocity of its dispersion is accelerating via the internet and other bandwidth modes. The Global Warming issue represents a clear example as: the consensus vs. science debate, in the digital domain.”
The reference is to Reynolds’ book An Army of Davids: How Markets and Technology Empower Ordinary People to Beat Big Media, Big Government, and Other Goliaths (2006),
Watch this conflict. The fun is just beginning.
4. Nature’s policy about availability of data and materials
From their website. It seems quite clear and unambiguous.
An inherent principle of publication is that others should be able to replicate and build upon the authors’ published claims. Therefore, a condition of publication in a Nature journal is that authors are required to make materials, data and associated protocols promptly available to readers without preconditions. Any restrictions on the availability of materials or information must be disclosed to the editors at the time of submission. Any restrictions must also be disclosed in the submitted manuscript, including details of how readers can obtain materials and information. If materials are to be distributed by a for-profit company, this should be stated in the paper.
Supporting data must be made available to editors and peer-reviewers at the time of submission for the purposes of evaluating the manuscript. Peer-reviewers may be asked to comment on the terms of access to materials, methods and/or data sets; Nature journals reserve the right to refuse publication in cases where authors do not provide adequate assurances that they can comply with the journal’s requirements for sharing materials.
After publication, readers who encounter refusal by the authors to comply with these policies should contact the chief editor of the journal (or the chief biology/chief physical sciences editors in the case of Nature). In cases where editors are unable to resolve a complaint, the journal may refer the matter to the authors’ funding institution and/or publish a formal statement of correction, attached online to the publication, stating that readers have been unable to obtain necessary materials to replicate the findings.
Details about how to share some specific materials, data and methods can be found in the sections below. The preferred way to share large data sets is via public repositories. Some of these repositories offer authors the option to host data associated with a manuscript confidentially, and provide anonymous access to peer-reviewers before public release. These repositories coordinate public release of the data with the journal’s publication date (advance online publication (AOP) or, if the manuscript is not published AOP, print/online publication). This option should be used when possible, but it is the authors’ responsibility to communicate with the repository to ensure that public release is made promptly on the journal’s AOP (or print/online) publication date.
Any supporting data sets for which there is no public repository must be made available as Supplementary Information files that will be freely accessible on nature.com upon publication. In cases where it is technically impossible for such files to be provided to the journal, the authors must make the data available to editors and peer-reviewers at submission, and directly upon request to any reader on and after the publication date, the author providing a URL or other unique identifier in the manuscript.
4. Afterword
Please share your comments by posting below. Per the FM site’s Comment Policy, please make them brief (250 words max), civil, and relevant to this post. Or email me at fabmaximus at hotmail dot com (note the spam-protected spelling).
For information about this site see the About page, at the top of the right-side menu bar.
5. For more information from the FM site
To read other articles about these things, see the FM reference page on the right side menu bar. Of esp relevance to this topic:
Posts on the FM site about the debate over climate science:
- More forecasts of a global cooling cycle, 15 July 2008
- Two valuable perspectives on global warming, 4 August 2008
- Good news about global warming!, 21 October 2008 – More evidence of cooling.
- Watching the world change before our eyes, 29 November 2008
- This week’s report on the news in climate science, 7 December 2008
- The Senate Minority report is out: “More Than 650 International Scientists Dissent Over Man-Made Global Warming Claims”, 12 December 2008
- Weekend reading recommenations about climate change, 13 December 2008
- An important new article about climate change, 29 December 2008
- My “wish list” for the climate sciences in 2009, 2 January 2009
- Important new climate science articles, 11 January 2009
- Climate science articles which you might enjoy reading!, 18 January 2009
- How warm is the Earth? How do we measure it?, 28 January 2009
- Science in action, a confused and often nasty debate among scientists, 5 February 2009
- Richard Feynmann, one of the 20th centuries greatest scientists, talks to us about climate science, 12 February 2009
