Magical theories of science & how they influence us

Summary: Today we have an article by Professor Judith Curry that goes to the heart of the climate science debates, about epistemology (the philosophy of knowing) and the process of science. As we see here, we can do it better — and must in order to meet the challenges of the 21st century.

“Magical theories“

by Judith Curry, at Climate Etc.
13 February 2014.

Posted under her Creative Commons license.

[W]e have a field of sort-of-science in which hypotheses are treated as facts because they’re too hard or expensive to test, and there are so many hypotheses that what journalists like to call “leading authorities” disagree with one another daily.

– Gary Taubes, op-ed in the New York Times, 8 February 2014.

Why nutrition science is so confusing

Gary Taubes has an op-ed in the NY Times “Why nutrition is so confusing” {8 February 2014}, which has some interesting parallels with climate science. Excerpts:

Since the 1960s, nutrition science has been dominated by two conflicting observations. One is that we know how to eat healthy and maintain a healthy weight. The other is that the rapidly increasing rates of obesity and diabetes suggest that something about the conventional thinking is simply wrong.

In total, over 600,000 articles have been published purporting to convey some meaningful information on these conditions. It would be nice to think that this deluge of research has brought clarity to the issue. The trend data argue otherwise. The more we learn, the more we need to know.

Because the nutrition research community has failed to establish reliable, unambiguous knowledge about the environmental triggers of obesity and diabetes, it has opened the door to a diversity of opinions on the subject, of hypotheses about cause, cure and prevention, many of which cannot be refuted by the existing evidence. Everyone has a theory. The evidence doesn’t exist to say unequivocally who’s wrong.

In nutrition, the hypotheses are speculations about what foods or dietary patterns help or hinder our pursuit of a long and healthy life. The ingenious and severe attempts to refute the hypotheses are the experimental tests — the clinical trials and, to be specific, randomized controlled trials. Because the hypotheses are ultimately about what happens to us over decades, meaningful trials are prohibitively expensive and exceedingly difficult. And before any of this can even be attempted, someone’s got to pay for it.

Without such trials, though, we’re only guessing whether we know the truth. [A]dvice to restrict fat or avoid saturated fat has been based on suppositions about what would have happened had such trials been done, not on the studies themselves.

Nutritionists have adjusted to this reality by accepting a lower standard of evidence on what they’ll believe to be true. The associations that emerge from these studies used to be known as “hypothesis-generating data,” based on the fact that an association tells us only that two things changed together in time, not that one caused the other. So associations generate hypotheses of causality that then have to be tested. But this hypothesis-generating caveat has been dropped over the years as researchers studying nutrition have decided that this is the best they can do.

One lesson of science, though, is that if the best you can do isn’t good enough to establish reliable knowledge, first acknowledge it — relentless honesty about what can and cannot be extrapolated from data is another core principle of science — and then do more, or do something else.

Obesity and diabetes are epidemic, and yet the only relevant fact on which relatively unambiguous data exist to support a consensus is that most of us are surely eating too much of something. We’re going to have to stop believing we know the answer, and challenge ourselves to come up with trials that do a better job of testing our beliefs.

A truly magical theory

Melanie Phillips as a post at the Electric Media Blog entitled A truly magical theory {11 February 2014}, that provides some climate science examples of what Taubes discussed in the preceding article.

{Phillips discusses “Man-made warming to blame for storms, suggests Met Office“, The Times, 10 February 2014 — which is about the UK Met Office report “The Recent Storms and Floods in the UK“, February 2014 — “Has climate change been a contributing factor?”}.

Excerpts:

Dame Julia Slingo, Chief Scientist at the UK Met Office, says that while there is ‘not yet definitive proof’, nevertheless ‘all the evidence’ suggests that ‘climate change’ is a contributory factor to the storms and heavy rainfall now causing the devastating floods in southern England.

Just how does Dame Julia arrive at this ‘significant’ conclusion?

The Met Office paper also says that, for various reasons (UK weather notoriously volatile, appropriate computer modelling systems not yet in place to ‘prove’ what they already know to be unarguably true) attributing changes in rainfall, regional climate and weather extremes to ‘climate change’ is, uh, ‘challenging’.

By which Dame Julia et al mean these changes cannot be attributed to AGW. But does that dent their certainty that AGW is the cause? Of course not!

‘There is no evidence to counter the premise that a warmer world will lead to more intense daily and hourly heavy rain events.’

And so the evidence to support the theory that ‘climate change’ has caused the storms is… that there’s no evidence to falsify what is merely a supposition.

Truly, AGW is a magical theory that explains absolutely everything – including diametrically contradictory phenomena, lack of logic and absence of evidence – whenever people observe profoundly, ‘Something funny’s happening to the weather’.

I have another theory to explain the current deluge. It is Galileo, Newton and Einstein weeping uncontrollably from above.

Severe testing of hypotheses

Taubes’ article mentions ‘severe testing.’ Joel Katzav has a very interesting article “Severe testing of climate change hypotheses” published in History and Philosophy of Modern Physics, 4 November 2013. This is a very interesting paper; I provide here only some brief excerpts:

On the severe testing epistemology of science, scientific hypotheses’ truth should be assessed in light of how well they have withstood severe tests.

According to Mayo’s version of the severe testing epistemology,

an experimental result, e, counts as good evidence for an hypothesis h to the extent that the test that yielded e severely passes h with e.

h passes a severe test with e if, and to the extent that, h fits e and it would have been very unlikely that h would have fitted test results as well as it does had h been false.

Mayo’s definition makes, in accord with the severe testing approach, how good evidence is depend on the test procedure that produces the evidence. But she also wants to make sure that good evidence is strong indeed, and thus adopts a demanding version of the severe testing approach. On her view, we need a definition of ‘successful severe test’ that guarantees that (successful) severe tests are reliable indicators of the absence of error in tested hypotheses.

As to evidence that is less than good evidence for acclaim, Mayo allows it an important role in identifying aspects of the claim that remain to be tested. On her view, such evidence thus helps us to see how far the claim is from the truth and what alternatives to it remain to be explored. Attaching probabilities to claims is, supposedly, neither needed nor desired in science.

From the conclusions:

As to OUR FAULT, IPCC-AR4 does not appear to have severely tested it. From Mayo’s perspective, then, IPCC-AR4 does not appear to provide good evidence for OUR FAULT.

Mayo’s approach does indicate that the evidence may nevertheless provide guidance as to how far this claim may be from the truth and about what assumptions need further testing if it is to be well supported. But OUR FAULT is itself a broad estimate of where the truth may be with respect to a certain possible cause of global warming. There is no further error range estimate attached to it. Indeed, attaching such a further estimate would just amount to making another, weaker claim such as that anthropogenic greenhouse gas emissions are responsible for more than a third of the post-1950 warming.

JC note: This is a very rich paper, I have invited Katzav to do a guest post.

Professor Curry’s reflections

Nutrition science and climate science share some common challenges: complex system(s) and many confounding factors. Severe tests for nutrition science can in principle be done, but they are very expensive and take decades. Severe tests for climate science require better observational evidence, particularly in the past.

When there’s no evidence to falsify what is merely a supposition,we are left with ”magical theories that explains absolutely everything – including diametrically contradictory phenomena, lack of logic and absence of evidence.”

I agree with Mayo and Katzav that when evidence is inadequate for a severe test, it is important to identify aspects of the hypothesis that remain to be tested, and to provide an assessment of alternative hypotheses.

In climate science, the limitations of available evidence and weak reasoning behind the high confidence levels of the IPCC conclusions reflect acceptance of lower standards of evidence on what is believed to be true.

——————– End of Professor Curry’s article —————————–

About the author

Judith Curry is Professor and Chair of the School of Earth and Atmospheric Sciences at the Georgia Institute of Technology. She is also President and co-owner of Climate Forecast Applications Network (CFAN). Prior to joining the faculty at Georgia Tech, she served on the faculty of the University of Colorado, Penn State University and Purdue University.

She serves on the NASA Advisory Council Earth Science Subcommittee and the DOE Biological and Environmental Science Advisory Committee. She recently served on the National Academies Climate Research Committee and the Space Studies Board, and the NOAA Climate Working Group.

She is a Fellow of the American Meteorological Society, the American Association for the Advancement of Science, and the American Geophysical Union.

Her views on climate change are best summarized by her recent Congressional Testimony: