"Most scientific papers are probably wrong" - New Scientist

Summary: Yes, that is the conclusion from a remarkably large body of research. Bad news for those who consider science a religion (at least when it agrees with their beliefs).

One of the great oddities of the public (i.e., among lay people) discussion of global warming is the love-hate attitude of many warmists to science. Scientists are authorities, not be be questioned — except when they disagree with anthropogenic global warming. Then they’re fallible, often ignorant fools (a frequent trope in the FM comments section). This results from a near-total ignorance of the history and philosophy (e.g, Karl Popper, Thomas Kuhn) of science. And ignorance of the research by scientists about science.

This post looks at one example. It’s about medical research. But then medical research has tighter controls than found in most of the physical sciences. For a summary of their findings see “Most scientific papers are probably wrong“, Kurt Kleiner, New Scientist, 30 August 2005 — Opening:

Most published scientific research papers are wrong, according to a new analysis. Assuming that the new paper is itself correct, problems with experimental and statistical methods mean that there is less than a 50% chance that the results of any randomly chosen scientific paper are true.

John Ioannidis, an epidemiologist at the University of Ioannina School of Medicine in Greece, says that small sample sizes, poor study design, researcher bias, and selective reporting and other problems combine to make most research findings false. But even large, well-designed studies are not always right, meaning that scientists and the public have to be wary of reported findings. “We should accept that most research findings will be refuted. Some will be replicated and validated. The replication process is more important than the first discovery,” Ioannidis says.

In the paper, Ioannidis does not show that any particular findings are false. Instead, he shows statistically how the many obstacles to getting research findings right combine to make most published research wrong.

The article: “Why Most Published Research Findings Are False“, John P. A. Ioannidis, Public Library of Science Medicine, 30 August 2005 — Abstract:

There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when

the studies conducted in a field are smaller;

when effect sizes are smaller;

when there is a greater number and lesser preselection of tested relationships;

where there is greater flexibility in designs, definitions, outcomes, and analytical modes;

when there is greater financial and other interest and prejudice; and

when more teams are involved in a scientific field in chase of statistical significance.

Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.

Note these characteristics closely fit climate science.

Afterword and contact info