Summary: Today’s post continues our discussion about experts. Here are a few tips to help distinguish reliable and useful experts from those that dominate the news media, plus some warnings.
“For now we see through a glass, darkly.”
— 1 Corinthians 13:12
“Nature is objective, and nature is knowable, but we can only view her through a glass darkly — and many clouds upon our vision are of our own making: social and cultural biases, psychological preferences, and mental limitations (in universal modes of thought, not just individualized stupidity).”
— Stephen Jay Gould, Full House (1996)
We can only understand the world — even imperfectly — by seeing it through the eyes of experts. Journalists showcase experts, usually a selected coterie (note how the same few show up repeatedly in a newspapers’ article on each subject). Unfortunately, journalists’ criteria for choosing experts don’t well meet our needs. The catchy sound-bites they favor tend to come from the over-confident and arrogant, especially those that endorse the current narrative. Caveats, uncertainties, and long explanations — these are things seldom found in the news.
How can we find better sources to rely upon? How can we use them most effectively?
Evaluating experts
In my experience, one hallmark of a reliable expert is their recognition of uncertainty. The experts I trust recognize how quickly the world changes, its complexity, the severe limitation on the data we have about it, and the crude state of our theories. These traits distinguish headline-grabbing experts from economists like Nouriel Roubini, Brad Delong and Paul Krugman, physicists like Robert Hersch, climate scientists like Roger Pielke Sr and Judith Curry, and others.
How they grapple with uncertainty makes them more interesting to read, in contrast to the boring black and white certainties that dominate the news.
These experts have another useful characteristic distinguishing them from journalists’ favorites: they admit errors. Half of what we know is wrong, and top experts work to find which of their beliefs lie on each side of that line. From example, Brad DeLong (Prof Economics, Berkeley) runs posts about “smackdowns” of his work, which practice I copied in the Smackdowns page (top menu bar). Krugman often runs columns about his errors.
Learning is the ultimate competitive advantage, and that means awareness of one’s errors.
Limitations of even the greatest experts
First, about forecasts. Forecasts are the frontier in many fields, such as economics and climate science. Treat them skeptically. Barry Ritholz writes often about economic and financial predictions, and his observations are apt for other fields. Forecasts are opinion, not analysis. Forecasting is marketing.
Second, I’ve found that even experts who cautiously discuss their own field sometimes become over-confident (often wildly so) when discussing other subjects. Doctors are notorious for this (making them easy marks for salespeople and con men), but its a common phenomenon. Expertise is domain-specific. There might such a thing as general intelligence, but it does not provide the combination of experience and deep knowledge that produces expertise.
Reading the news
I read the news as a series of press releases by activists or enthusiasts. Some stories don’t fit that mold; most do. Reading stories critically requires classifying them by the writer’s narrative (or frame); seeing that allows better interpretation of its content. When I find a story where that’s difficult to do, I have a story that deserves close attention.
Experts are the folks that agree with me
The information highway has become the fast track to confirmation bias. Do all the experts you follow agree with you? Perhaps you have the process backwards, selecting experts to confirm your beliefs instead of informing them — or even challenging them.
For More Information
Where to find reliable information sources:
- Suggestions for your daily info diet. You are what you read!
- Economics can help understand events in America and the world. Here’s where to find those answers.
Other posts about experts:
- Today’s debate: a passionate defense of credentialism. State your view!
- Experts now run the world using their theories. What if they fail, and we lose confidence in them?
- Do we face a future without confidence in experts?
- Our confidence in science is crumbling. Why? How can we fix this?
- 2015 might bring an end to the great age of experts’ experiments on us.
I like your criteria for evaluating experts.
On the question of their limitations, I feel that no post on experts is complete without some Feynman quotes:
“I’ll never make that mistake again, reading the experts’ opinions.”
and of course “Science is the belief in the ignorance of experts”.
Paul,
The first quote — by Feynman about science — is brilliant, and nicely expresses how scientists see science. The first one is imo totally misinterpreted. The quote is from Surely You’re Joking, Mr. Feynman! (1985) — composed by Ralph Leighton from his interviews with Feynman. The passage:
How long would it take a person to read the newspaper every day, calculating everything for himself? Locate a few key experts, discuss the problem, “go through the whole work with” him, check their results. And that assumes that after reviewing the subject he has the competence to check experts in any meaningful sense.
The example Feynman gives is quark theory, a subject within his professional expertise. I doubt he meant this as a prescription for life, to apply to all subjects.
Historian Leo Treitler wrote an excellent article in 1984 called “What kind of story is history?” Treitler pointed out that humans are predisposed to favor histories which boast a recognizable narrative structure (beginning, middle, end) with a familiar Western dramatic curve (conflict, ultimate crisis, resolution, happy or tragic ending, with a convenient moral for the audience to draw) an familiar dramatic personae — heroes, villains, innocent dupes, conniving conspirators, etc.
Treitler notes that humans don’t do well when historical events fail to follow these trite cartoonish stereotypes. What if everyone does everything right and there are no obvious villains, but the policies still lead to disaster? (The change in American mental health policy following the introduction of psychotropic rugs like lithium, which has led to the current tragic epidemic of homeless mentally ill people living on the streets.) What if events degenerate into a muddle without a convenient moral at the end for us to learn from? (Korean War, desert Storm, the Cold War, etc.)
Humans like to think of history as a morality play with clear-cut heroes and villains. Sometimes this is the case: Munich, 1938, Iraq Invasion, 2003. Sometimes, though, history is just “one damn thing ater another” with no obvious moral or clear-cut villains — August 1914, the Great depression, the Great Society programs, the crack epidemic of the 1980s, the dramatic and inexplicable reduction in crime post-1990 in America. Americans, raised on cowboy movies with a culminant gunfight and a mustachio-twirling black-hatted villain, don’t seem well fitted for a world without black hats or showdowns at the OK corral.