Science in action, a confused and often nasty debate that produces real progress

Summary:  The events following publication of a new paper about our climate illustrates how science works in our society.  It shows the warts — and its self-correcting nature.  The result should please everyone, both scientists and laypeople.

Science is and has always been a social process, and hence usually confused and often nasty.  Now its a mainstream subject of great importance, unlike the days when Newton lectured to an almost-empty hall.  But the hype about a new article is so often unrelated to its actual importance or quality.  After the media blitz comes debate, as its data and methods are scrutinized.  For too many years much climate science work escaped this process, but no longer.  Thanks to the Internet we can watch the process of science at work. 

Of special interest is the role of outsiders, non-credentialed people who have own their own developed expertise in an area of climate science.  As usual with social systems, they act as disruptive elements — revealing flawed processes that insiders agree to ignore.  That our society allows (if not encourages) such people is one of our strengths.

 A new paper in Nature has received massive press attention (more adulation than journalism), as providing more proof of AGW: “Warming of the Antarctic ice-sheet surface since the 1957 International Geophysical Year”, Steig, E.J., D.P. Schneider, S.D. Rutherford, M.E. Mann, J.C. Comiso, and D.T. Shindell. Nature, January 2009, pp 459-462. (Abstract for Steig 2009 here; full text here).

Contents

  1. Initial reaction:  Praise from the pews
  2. Surprising results from a first look at the data
  3. Initial results from the involvement of outsiders
  4. Some scientists were cautious about the study at its publication
  5. My conclusions

1.  Initial reaction: Praise from the pews

On the FM site Oldskeptic has strongly praised it in two comments, mirroring the mainstream media applause.

“The latest, Antarctic results have come from a newer, and yes better, methodology, incorporating the only real data source that covers the whole World, satellites.” (source)

“Note that Nature is a premier, peer reviewed scientific journal, if they could not prove their work it would not have been published.” (source)

The latter assurance is a bit odd, since it appears that the code and data are not yet fully archived. In several past major climate science articles even the reviewers did not see this vital information.  No word on the code, but the lead author (Steig) has said he will archive the data soon. 

Update:  Not so, only a few “legitimate” folks deserve to have access to “all our data, including the intermediate steps” (source).  His definition of “legitimate” is very restrictive.  To Steig only a very few get access to key information on which public policy will be made – policies that might determine the fate of the world (as Al Gore has told us).   And a clarification:  the data inputs to the study are all public, but replication is not possible with knowing how he arrived at his results (some combination of code and intermediate work results).

Update:  It gets worse.  Now an editor at RealClimate says “some of these data are proprietary (NASA), but will be made available in the near future”.   See comment #15 below for the lead author’s statements otherwise. 

Update:  It appears that the folks at RealClimate have no consistent definition of “data.”  The latest spin about Steig 2009 is “{T}he raw data are public; the processed data (i.e. cloud masking) are not yet, but will be in due course. so relax”.   OK, you have your orders.  Relax.  The authors will release this vital information when they are good and ready to do so.

Quite a soap opera.   Too bad that the issues are so serious.

2.  Surprising results from a first look at the data

So far investigation has been made of only two stations in the networks used (42 occupied, 65 automatic stations).  Steve McIntyre (of Climate Audit) discovered that it was seriously defective, and posted about the error at 4:41 pm EST, on February 1 (Super Bowl Sunday).  A later comment noted another station with defective data.

This ignited a somewhat incoherent, contradictory, defensive, and vituperative series of replies on RealClimate. See here and here.  I recommend skimming the comments.  Nobody likes auditors! 

Corrections followed by British Antarctic Survey (BAS) and Steig, with another post defending the study by Schmidt at RealClimate (here).  It is fun to watch the messy process of science in action.

3.  Initial results from the involvement of outsiders

(a)  In 2 posts at Climate Audit (Sunday 1 February and 2 February), Steve McIntyre identified an anomaly in the data for the “Harry” automatic weather station (AWS).   On Monday BAS posted a note about the error (although originally identified by McIntyre, Gavin Schmidt notified BAS on February 1 or 2).  The British Antarctic Survey (BAS) corrected the error on Monday morning — fast work, but without crediting McIntyre for catching the error. (source; for a 3rd party’s view of this episode see this by Pielke Jr., Professor of environmental studies at U Colorado).

(b)  A comment by Hu McCulloch at Climate Audit uncovered another anomaly in the BAS data (the “Racer Rock” site), at 1:22am on 4 February.  Later that day BAS posted a note about the error.

(c)  A serious problem in climate science is failure to publicly archive data, which makes replication difficult and results in multiple and often differing “grey versions” circulating for the same data.  The BAS corrections would have followed this pattern, as they initially corrected the original file.  Having no version control in the public data, the data actually used in the Nature paper would vanish.  McIntyre alterted BAS to this problem, and they archived the original (incorrect) data.

(d)  The lead author, Steig, has posted a brief note about the impact of these data corrections, concluding that the efffect is tiny.  Since he has not archived his complete code, it is not possible to replicate his analysis. 

There are several other lines of inquiry about this study. Here are two, as illustrations.

(i)  Accuracy of the satellite data:   The trend they find is quite small (o.1 degree C per year), probably much smaller then the uncertainty of the data.  No error bars are given in the article.  As a general reference (which might not apply to the Steig 2009 paper),  NASA’s FAQ page on the Atmospheric Infrared Sounder satellite project says:

Land surface temperature is problematical, since the emissivity of bare earth will vary greatly over the 50 km diameter spot in which our retrieval is made. Our estimated uncertainty at present is 2->3 K. {bold emphasis added}

(ii) Accuracy of the surface temperature record:  There are few few stations in antarctica, and few of them have long records.  Also there is evidence suggests that the automatic weather stations are often buried in snow (an excellent insulator) for long periods. See this article for pictures of visits to these stations to dig them out.

No firm conclusions can be drawn yet, but finding such data problems in a study’s first cursory review is not a good indicator of the its quality.  Only time will tell.

4.  Some scientists were cautious about the study at its publication

Note that initial media coverage included a few cautionary notes, ignored by true believers.

“This looks like a pretty good analysis, but I have to say I remain somewhat skeptical,” Kevin Trenberth, climate analysis chief at the National Center for Atmospheric Research, said in an e-mail. “It is hard to make data where none exist. (AP; Trenberth has been a lead author for the UN’s IPCC)

“One must be very cautious with such results because they have no real way to be validated,” says atmospheric scientist John Christy of the University of Alabama-Huntsville, who was not part of the study. “In other words, we will never know what the temperature was over the very large missing areas that this technique attempts to fill in so that it can be tested back through time.” (USA Today)

Roger Pielke Sr also had some questions, posted at his website. He is emeritus professor of the Department of Atmospheric Science at Colorado State U. See his Wikipedia entry for more information.  His concluding comment:

In terms of the significance of their paper, it overstates what they have obtained from their analysis. In the abstract they write, for example, “West Antarctic warming exceeds 0.1C per decade over the past 50 years”.

However, even a cursory view of Figure 2 shows that since the late 1990s, the region has been cooling in their analysis in this region. The paper would be more balanced if they presented this result, even if they cannot explain why.

5.  My conclusions

(1)  Scientists eventually will sort all these questions out.  True believers (on both sides), who accept only science that confirms their views, will remain unaffected no matter what the result (see here for examples).  It’s called confirmation bias (see Wikipedia for more on this).  

For the pro-AGW fanatics, does this  result from religious fervor, or self-training in doublethink.  Doublethink is the ability simultaneously to have two contradictory beliefs.  Such as…

Scientists are reliable guides and sources of information. Scientists who challenge my beliefs do not understand basic science; they are incompetent. (See here for examples)

The rest of us, who will pay for the efforts proposed to reduce or stop global warming, might choose to wait until some of these questions are resolved before supporting large new public policy changes.

(2)  This re-enforces My “wish list” for the climate sciences in 2009, changes that would put the climate sciences on track to meet our public policy needs.  Whatever the cost (although trivial compared, for instance, to global military spending), I believe these things must be done ASAP.

  1. Raise the standards when applying science research to public policy questions.
  2. Provide greater transparency of data and methods used in climate science research.
  3. Provide third party review of the data, analysis, and modeling is necessary.
  4. Rationally apply the precautionary principle.

Afterword

Please share your comments by posting below.  Per the FM site’s Comment Policy, please make them brief (250 words max), civil, and relevant to this post.  Or email me at fabmaximus at hotmail dot com (note the spam-protected spelling).

For information about this site see the About page, at the top of the right-side menu bar.

For more information

To read other articles about these things, see the FM reference page on the right side menu bar.  Of esp relevance to this topic:

Some of the posts on the FM site about climate change:

  1. A look at the science and politics of global warming, 12 June 2008
  2. Global warming means more earthquakes!, 19 June 2008
  3. An article giving strong evidence of global warming, 30 June 2008
  4. More forecasts of a global cooling cycle, 15 July 2008
  5. Two valuable perspectives on global warming, 4 August 2008
  6. President Kennedy speaks to us about global warming and Climate Science, 7 August 2008
  7. Good news about global warming!, 21 October 2008
  8. One of the most interesting sources of news about science and nature!, 27 October 2008
  9. “Aliens cause global warming”: wise words from the late Michael Crichton, 15 November 2008
Advertisements

16 thoughts on “Science in action, a confused and often nasty debate that produces real progress

  1. It is very disillusioning when you discover that the people you trusted and believed have shown themselves unreliable. Time must pass before these things can be reconciled with oneself.
    .
    .
    Fabius Maximus replies: I understand with the feeling, but this is just business as usual on Earth. That’s why one should always cut cards, get a second opinion before surgery, and insist that results be audited by an uninvolved third party (both in pharmaceutical research, climate science, and before investing).

    Like

  2. FM note: As usual with climate science, much of Oldskeptic’s comments are absurd. Sad how he repeats the same misinformation (or “disinformation”). Replies are interjected in the text. If Oldskeptic follows his past practice, he will not respond, but eventually will repeat these things.

    “Science is and has always been a social process”. Negative.

    Hard science is and always has been a system to focus talented, fallible, intelligent, flawed, creative, silly … people within a tight discipline towards increasing knowledge. And it has always been anti-social; “sorry folks the Universe does not revolve around you despite what some badly witten books say, you are a quantum bubble of virtual particles on an insignificant speck of dirt on an insignifcant speck of space, hanging around until picking nits off each other until your sun blows up, get used to it”.

    *** FM: We bow, we genuflect before the God of science! This is cracked, even funny. There is a large literature showing how science operates a social process. Thomas Kuhn and Karl Popper are two of the best know names in the field. Stephen Jay Gould’s books explain this for a general audience. What’s the alternative, to see scientists as priest-kings or gods? That they cease to have human frailities when they go into the lab?

    It is the only human activity that always improves. Normal behaviour, idiologies and ‘Social sciences’ do, as all human behavour does, flip/flop between competeting idiologies (basically arguments about the colour of the underwear of the angels dancing on the heads of pins), or go into ga-ga land .. anyone remember reading about men wearing tights as the peak of fashion?.

    *** FM, #1: That is one of the key points of this post. It is also a reason to be careful about the current consensus or paradigm — it changes as theory and data improve. Not to be confused with papal bulls.

    Your points in detail and I am assuming you mean climate science, otherwise this post will take 1 Billion words (just dealing with the dangers of Dihydrogen Monoxide would take hundreds of thousands):

    “Raise the standards when applying science research to public policy questions.”
    Done, the IPCC exists to pull together all the research and present to the public and policy makers a consensus view. Plus each independent centre supplies all results to their own public and
    Govt’s and then there is all the other research centres and Universities all adding to the process.

    *** FM, #2: Oldskeptic appears not to understand the meaning of “raise.” Nor does he cite any evidence that the IPCC raises standards — rather than summarize existing work. This is an esp bizarre comment to make right as we see the standards actually being raised by the work of non-credentialed scientists reviewing the antarctic temperature record (see here for some details). Also see here for another specific example of the poor state of the data.

    “Provide greater transparency of data and methods used in climate science research. ”
    Done. Download all the data from a variety of sources (Goddard, Hadley, CSIRO, etc,e tc).
    Plus download your own climate models and review the code and run them youreself.

    *** FM, #3: Despite how much evidence I present to show specific assertions (like this) are incorrect, he ignores them and trots the same nonesense out the next time. This is the sad thing about Oldkeptic’s comments in this area, which makes them impossible to take them seriously. Quite sad. Look at the tremendous effort being expended to replicate Steig 2009, made far more difficult by the authors’ refusal to adequately document their methods.

    “Provide third party review of the data, analysis, and modeling is necessary. ”
    Done. Each centre has all its work cross checked by others (inc Unis, etc) from all over the world (US, Austrlaia, Canada, UK, Japan, etc, etc). Plus independent reviewers and any papers submitted to scientific journals are indepedently peer reviewed.

    FM: Don’t just make stuff up; please give links to 3rd party review of one of the major climate models, done by a team of the appropriate experts (climate, statistics, software, etc). Such a problem is esp difficult as many of these models are large bundles of spaghetti code, such as the well-known GISS ModelE (go here for attempts to understand this model). Dan Hughes comments goes to the heart of the matter:

    “A nice little loophole has been created, whether implicitly or explicitly I don’t know. The scientific journals associated with the climate-change community will accept papers for publication the basis of which are calculations by computer software that has not been peer-reviewed. So, AOLGCM-based papers get peer-reviewed and published and then cited in the IPCC reports. This is not correct because it bypasses the independent verification and replication processes of the scientific method.” (source)

    “My primary conclusion, based on many attempts to engage GISS/NASA’s Real Climate on V&V and SQA topics are as follows. … Then usually I hear rationalizations that peer-review of papers for publication in Science Journals, comparisons of results from other software, and a couple of other softballs provide more than sufficient alternative assurance that there are no problems in my software. The very best one of all is the claim, which was written by a very respected Science Historian and colleagues, and has appeared in a peer-reviewed Science Journal, is that models/methods/codes/applications that involve Natural physical phenomena and processes can be neither verified or validated.

    “I do not know of any other areas of software applications (not a single one), the results of which might affect the health and safety of the public, for which this approach and these rationalizations are accepted.” (source)

    “Rationally apply the precautionary principle.”

    Done. The classic examaple was the UK Stern report working out the scientific and economic aspects (google it and read it everyone, it really is a superb piece of work).

    All a work in progress, as all science is, but constantly improving.

    Re Antarctic data, the important thing is that it is not cooling overall, as earlier data and so many (including me [yes, really FM]) hoped. There are at least some hotter spots, unfortunately not where we want them (inland), but closer to the coast.

    Fortunatly it is not (say) +1C, otherwise then I’d say, go to bed now and pull the blankets over your head and suck your thumb, mainly because it nice, plus it is easier on the back than bending over and KYAG. But it is a ‘warning’ …… a canary coughing a bit.

    Needs more work .. hey I know, lets take all the bonuses off all those Wall Street terrorists (they make BL look like an amateur) and put it into research.

    Like

  3. The interesting thing to me about the Antarctic Steig study is that the authors appeared to create data out of whole cloth. They produced warm temperatures from areas where no temperature sensing equipment exists. They indulged in computer fantasies, in other words. Computer fantasy is not science, it is hypothesis. Yet these curious men treat the output as if it were data! Most odd.
    .
    .
    Fabius Maximus replies: That is a somewhat exaggerated statement of Trenberth and Christy’s comments. Steig 2009 proposes to “extend” the satellite temp record back using the sketchy surface temperature record. It’s ingenious and creative. We will see how well it stands up to analysis.

    Like

  4. FM update: see comment #15 for more information; it appears that Id was right and I was wrong.

    Steig has claimed he already archived the code and data but this is not the case. See here.
    .
    .
    Fabius Maximus replies: That overstates the case a little.

    As I understand it (please correct me if I am wrong) the sources used in Steig 2009 are publicly available, but only in external datasets that change (with poor or no version control, explained here and in the following comment). Which means that over time there is no publicly available set of the specific numbers he used, making exact and authorative replication impossible. We see that with the correction to the BAS data. If McIntyre had not requested that they archive the incorrect data, people would be using different data than the Steig 2009 paper when attempting to replicate his work.

    As for the code, the last I saw was that Steig links to the statistical package (RegEm) he used. The implementation of this remains unclear (see here and here for details). As McIntyre explains:

    Everyone also needs to keep in mind the paleoclimate history in these discussions. One of the MM03 test was using updated data versions. Mann said that this was “WRONG”. An “auditor” had to do things EXACTLY the same way. There were unexplained steps in the methods which they refused to explain so we made reasonable assumptions. Mann said that these were “WRONG” because we had made some slight mis-step in re-tracing Mann’s steps – options that were by no means statistical necessities.

    My point in attention to precise detail is not because I think that there is any magic in Team methods (I don’t), but because I’ve had my fill of the Team fogging up a debate by screeching that some step in a plausible interpretation was “WRONG”. So I’d rather make sure that I know exactly what they did; benchmark my methods against theirs and THEN test variations.

    A comment to Id’s post by Chris H. gives a good summary:

    “BTW, for people not savvy with computer programming, it appears that Steig is basically saying “Write the code to perform the analysis yourself, using the standard Matlab functions that I linked to”. In other words “Please take a blind guess at how I did my analysis, because I won’t tell you”. This isn’t how (non-fake) science works.”

    I would correct the last line to read “how science should work, using best practices.” Replication is not possible for complex work like Steig 2009 without access to some combination of code and intermediate work products (see this for an example of what’s needed to replicate Steig’s processing of the satellite data)

    Like

  5. 2 Comments on Steig. 1 on his behalf; 1 to his detriment.

    1) Steig is attempting to make do with a limited set of data to explain a wide range of phenomena. This leaves him with a set of unknowns greater than his set of available parameters which in practical terms, means he can’t do his job if he leaves it at using what’s really there. Hence, he uses an old-school modeler’s trick called a synthetic anneal. He’s using some sort of interpolation techniques to fill inholes on a data grid with “scientifically feasible” synthetic data so that he has enough horsepower to set up a closed-form mathematical system. Theoretically, at least, he’s doing the right thing. This isn’t optimal science, but some days you do the best you can with what you have.

    2) What Steig seems to be missing (and why I’ll now turn my guns on him) is the logical set of syllogisms justifying how he set up his anneal criterion. (I’m low on words so I’ll elide details.) An anneal criterion states, “At x-distance on a continnuum between A and B, their exist physical conditions et cetera that allow me to make a strong objective guess at to what some point C will equal for some desired parameter.” He doesn’t do this well enough to pass the BS test, therefore MacIntyre started sniffing along his continnuum. So it goes.
    .
    .
    Fabius Maximus replies: Thanks for a great comment! I agree on all points.

    Like

  6. These two on FM’s wish list:

    Provide greater transparency of data and methods used in climate science research.
    Provide third party review of the data, analysis, and modeling is necessary.

    aren’t just minor, but what it means to be actual science. Results MUST be reproducable. The current failure of so many fields (not just climatology, but my field, computer science, and who knows what else) to make data, code, and models available to other researchers and general review has been a travesty.

    Like

  7. FM, thanks for the balanced commentary. Us humans are plagued by all kinds of flaws, including a reluctance to see (much less admit) mistakes and predilections to classify those who disagree with us as enemies while seeing ourselves as morally superior. Thus armed, we head into battle ….

    Like

  8. Update: More errors discovered in the Antarctica temperature record

    Improving the quality of the surface temperature record, of which these are tiny first steps, is imperative to better understand climatic trends.

    There are now 3 pages of correction to the British Antarctica Survey data. This flurry of corrections is largely attributed to non-credentialled scientists (not amateurs), who have promptly and courteously been acknowledged by BAS. The effort continues (e.g., see here).

    (1) “Corrections to AWS (Automatic Weather Stations) data” — Mentioned above, now listing 5 stations. Also they have revised the acknowlegement to read:

    “The above corrections have been made with the help from numberous individuals including Gavin Schmidt, Steve McIntype, Nicholas Lewis and other anonymous individuals on ClimateAudit.com. All their effors in identifyting any errors on the READER website are appreciated.”

    (2) “The following corrections have been made using data supplied by GISS” — Corrections to 6 stations, with an acknowledgement on the “Mean Monthly Surface Air Temperature Data” main page saying:

    “Notification was received from Steve McIntyre and Reto Ruedy of incorrect values for 1988 and 1989 in several of the tables below, these have now been updated and a list of the changes can be accessed here.”

    (3) “Corrections to surface station data” — A “list of errors currently being investigated.” Now containing 5, 2 of which are by Nicholas Lewis (see his comments at Climate Audit for more detail).

    Like

  9. Update: who gets access to information necessary to replicate climate science research?

    Per Eric Steig, the lead author of the new Nature study, only a few “legitimate” folks deserve to have access to “all our data, including the intermediate steps” (source). His definition of “legitimate” is very restrictive. To Steig only a very few should get access to key information on which public policy will be made – policies that might determine the fate of the world (as Al Gore has told us).

    Like

  10. A pure mathematician and an applied mathematician take a road trip together. As they drive along, the applied guy says, “Look, a field of flowers.” The pure guy nods in silent agreement. “Look, a field with many sheep.” says the applied guy. Again the pure guy nods. “The sheep have been shorn”, says the applied mathematician. After a long pause, the pure mathematician says,”We only know they are shorn on the side facing us.” This summarizes the debate over global warming.
    For balance…
    The dairy industry hires an applied mathematician to optimize milk production. He spends years studying the problem, but at last, he is ready to present his results. Before a huge audience he walks to the lectern, and puts up his first slide. “Assume a spherical cow…”, he begins.
    .
    .
    Fabius Maximus replies: Due to my well-known humor deficiency, I don’t get the relevance of either of these to the debate over climate science.

    The first is one of the basic lesson in general semantics (as in “the map is not the territory”), which IMO should be taught to schoolchildren. The second is not just the punchline of a classic joke, but also the title of an excellent book by John Harte about problem-solving.

    Like

  11. These jokes were told to me by my doctoral adviser, an applied mathematician. At the time, perturbation method, aka asymptotic stability theory, aka chaos theory, were “The next big thing”, but older physicists and mathematicians were highly skeptical of these methods. This is because the new methods used formally divergent power series expansions to “linearize” otherwise intractable physical problems. The result was that assigning error estimates to results which were bounded only by truncating the series to leading order was near impossible.

    The debate was not so much semantic, as it was philosophical. Could perturbation method be justified because it had demonstrated experimental predictive power even though as a formalism it was on shaky ground? You previously gave me a link to the “Fit an elephant” remark from Fermi to Freeman Dyson. There, I think Fermi was voicing his criticism of these emerging methods when he notes that he preferred well supported “formal” methods of setting up and solving physics problems. Failure to “Show their work” the climate guys may be attempting to avoil similar criticism of the methods they are using.
    .
    .
    Fabius Maximus replies: Thank you for the explanation; now I got it! The article you mentioned could be about climate science (it fits so well): Freeman Dyson’s classic “A meeting with Enrico Fermi“, Nature, 22 January 2004 – “How one intuitive physicist rescued a team from fruitless research.” — Excerpt:

    “There are two ways of doing calculations in theoretical physics”, he said. “One way, and this is the way I prefer, is to have a clear physical picture of the process that you are calculating. The other way is to have a precise and self-consistent mathematical formalism. You have neither.

    “… In desperation I asked Fermi whether he was not impressed by the agreement between our calculated numbers and his measured numbers. He replied, “How many arbitrary parameters did you use for your calculations?” I thought for a moment about our cut-off procedures and said, “Four.” He said, “I remember my friend Johnny von Neumann used to say, with four parameters I can fit an elephant, and with five I can make him wiggle his trunk.”

    Like

  12. Update: more evidence that we need large funding increases to improve the collection of temperature data

    About the data from the automatic weather stations (AWS) in Antarctica, by Guillaume Dargaud (software engineer with much experience working with instrumentation in Antarctica), from his website. (this links to his CV, in Word). Excerpt from his page about the AWS data:

    Another part of the problem is that the {AWS} data is very dirty: it relies on very old satellite protocols and errors creep into the messages by the truckload (about one message out of two has errors). The AMRC is supposed to clean the data but they seem to have stopped doing that a few years ago. It’s not like the AWS data is all that informative, it only contains wind speed, wind direction, temperature and pressure, but for the last 20 years, it’s the only recording of winter weather that we have, so it’s important to continue it so we can compare it with the new acquisition systems that will start with the opening of Concordia.

    Notes:
    * AMRC is the Antarctic Meteorological Research Center of the Space Science and Engineering Center at the University of Wisconsin-Madison
    * Concordia is the fourth and newest “permanent, all-year research station on the Antarctic Plateau”; see the Wikipedia entry for details.

    Like

  13. Update: was the data in Steig 2009 all from public sources, as claimed?

    It looks like Jeff Id was correct and I was wrong. Let’s update the evidence.

    Email from Steig to Steve McIntyre on January 23 (source):

    “I have always intended to provide all the material on line; I wasn’t allowed to do this before the paper was published. I would have done it already but have been busy answering emails. I should have these up on line next week.”

    From Steig’s website at the U of Washington:

    “All of the data used in the temperature reconstructions are from publically available data sources.”

    From Josefino Comiso (NASA Goddard Space Flight Center), in reply to request for data by McIntyre on January 23 (source):

    “Thanks for your request for AVHRR surface temperature IR data. I am actually planning to have the entire data set archived in the near future and as soon as I get the associated document that describes the data and discusses the errors and caveats completed. The data are indeed on a gridded monthly basis. I will let you know how to access them in the web as soon as they are archived and ready to be downloaded. Best Wishes”

    Steig replying to a comment at RealClimate on February 2:

    “ALL of the data that were used in the paper, and EXACTLY the code used in our paper have been available for a long time, indeed, long before we published our paper. This is totally transparent, and attempts to make it appear otherwise are disingenuous. This has always been clear to anyone that asked.”

    Steig replying to a comment at RealClimate on February 2:

    “… Excluding your study, would it be your opinion that code and data archiving practices are adequate in the climate science community, overall? This is a common complaint on sites like CA.”

    [Response: Yes, I think we do a fine job. We could do better, but we do very very well. I’ve never had trouble getting data that I need from others. Indeed, our Nature study was based entirely on freely available data and code. –eric]

    Steig replying to a comment at RealClimate on February 3:

    “I released an electronic version of our data and links to all the original data and code almost as soon as our paper was published. Anyone paying attention would know that.”

    Steig replying to a comment at RealClimate on February 3:

    “I do routinely make all our data available, as does everyone else that I know. In this particular case, anyone legitimate who has asked for all our data, including the intermediate steps, has received it. To continue with the analogy with financial auditing, let me very clear on what I mean by legitimate: In the business world, auditors
    (1) don’t publicly accuse a company of withholding data prior to requesting said data;
    (2) are not self-appointed;
    (3) have to demonstrate integrity and competence;
    (4) are regulated.
    On this point, if you are suggesting that Steve McIntyre be regulated by an oversight committee, and have his auditor’s license revoked when he breaks ethical rules, then we may have something we can agree on.”

    Gavin Schmidt (NASA Goddard Institute for Space Studies; see his bio) replying to comment #38 by by Nicolas Nierenberg on February 9, posted at RealClimate (post titled “On Replication”):

    “Once again rather than the general, let’s be specific. I believe it removes the issues of how much extra work would be created. Dr. Steig has said that he is willing to provide the data to legitimate researchers. My response is to simply post what he would provide. I still haven’t heard from Dr. Schmidt what the objection is to that concept.

    “Also as to the specifics in Dr. Steig’s paper. I believe that there is probably sufficient information on AWS trends. However I don’t think there is sufficient information to reproduce the gridded AVHRR temperature results. They are quite dependent on corrections for clouds, and manipulation to produce temperature values as I understand it.”

    [Response: Joey Comiso is apparently working on making that available with appropriate documentation – patience. – gavin]

    Reply to comment #63 by Nicolas Nierenberg at RealClimate on February 9:

    “… Other than a couple of people the conversation seems to be converging on the fact that providing the code and data is preferable. Dr. Schmidt has said that this will be done in the case of the Steig paper.”

    [editor note: this was done in the case of Steig et al with respect to code though perhaps not with as much hand-holding as you seem to want. some of these data are proprietary (NASA), but will be made available in the near future]

    Reply to <a href=”http://www.realclimate.org/index.php/archives/2009/02/on-replication/langswitch_lang/in#comment-112276″ rel=”nofollow”comment #136 at RealClimate on February 11:

    “If possible, please provide more information about the ‘proprietary NASA data’ used in Steig 2009. There have been several (at least 4) clear statements from Dr. Steig, both at his website and RealClimate, that his study used only ‘publically available data sources.’”

    [reply: the raw data are public; the processed data (i.e. cloud masking) are not yet, but will be in due course. so relax]

    Gavin Schmidt replying to Comment #225 by captdallas2 on February 16 at RealClimate:

    “The satellite data was IR not RSS or MSU so the replicators could not gather the data needed to complete the replication. Do you know if that data is available online?”

    [Response: Joey Comiso is apparently working on the data preparation along with sufficient explanation to make it usable. I have no particular insight into the timetable. – gavin]

    OK, you have your orders. Relax. The authors will release this vital information when they are good and ready to do so.

    Congratulations to Jeff Id, Steve McIntyre, and others for some great detective work — working to find the sources the data used in Steig 2009 so that the results can be replicated. Esp noteworth is their persevance. Needless to say, this is just one more step in the process, and it is far too early to draw conclusions.

    Like

  14. Four years later… my first visit to FM. Lookin’ good.

    I have a problem with Point (4) of your wish list, namely the precautionary principle. All right, we know that the greenhouse effect is good physics, so it’s not as if the AGW people were demanding we sacrifice our first born sons to ward off rabid unicorns.

    However, the action that the AGW people demand isn’t far short of sacrificing our first born sons. Billions of the poorest people around the world will have to bear the cost of the precautionary principle. The cost is out of all proportion to any possible downside.

    Like

Leave a comment & share your thoughts...

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s