Summary: Kip Hansen shows how journalists’ careless use of numbers and graphs creates powerful but misleading stories. More skepticism by us will force them to change.
By Kip Hasen at Watts Up With That.
Reposted with his generous permission.
In today’s digital and Internet-of-Things world, it is easy to transform information into images – graphs, charts and other visuals that are colorful and informative. Modern math and statistical software packages can do it all for you with a few clicks of your mouse or taps on the screen. These visuals can be very powerful in conveying your message to the public. That’s on the upside.
On the downside is the fact that these visuals can be very powerful in driving false or misleading messages into the public consciousness. These all-too-easy-to-create visuals are “a Blessing and a Curse”.
We have seen this in the last few days with the scandal of the Twit-o-verse banging on about fires in the Amazon promulgating photos that are more than a decade old (not of this time) or even of some other place. One of the results of the twitter-storm has been a special meeting at the G7 about Brazil’s fires and shock at the fact that the U.S. President didn’t attend that meeting. In the case of the fires in Brazil, it turns out that the numbers have become more important in the public mind than the what. See this from my comment to Les Johnson’s post “Amazon Fires are in…the Amazon.”
“Take a look at the Forest Fires Map at Global Forest Watch. Like the NY Times piece mentioned above, the map makes it looks like whole countries are on fire. This is an artifact of the size of the dots marking fires. ZOOM IN ON BRAZIL. Zoom in until you can see the Federal District of Brasilia clearly. See how the fires clump together in agricultural areas – sometimes, close enough, you can see that it is a series of burning local fields or pastures that it records as fires.”
“Now, zoom out and find the Dominican Republic – the island of Hispaniola – just to the west of Puerto Rico. Looks awful doesn’t it, island on fire. Zoom in and in and in and you will find that it is not prime cane harvest yet, only a few cane fields burning – all set intentionally as a necessary part of the cane harvest. Some may be rice paddies – where the rice stubble is burnt off after the harvest of the rice and rice straw.”
“All these alarming stories require local knowledge and local ground truthing….”
I recently saw an article that led me to chase up a graphic used about the number of natural disasters by type: from Our World in Data. Click to enlarge.
Our World In Data describes themselves as having “Research and data to make progress against the world’s largest problems. Scroll to all research: 2989 charts across 297 topics. All free: open access and open source.” Its website says that they are based at Oxford and are trusted in research and media and used in teaching.
Yet, there is something that does not seem right in that “Number of reported natural disasters” graphic. There is something that is a clue (and thus its saving grace): it is the word “reported”. The second clue is the gray text at the bottom giving the source of the data as EM-DAT, the international disaster database of the Centre for Research on the Epidemiology of Disasters of the School of Public Health at the Université catholique de Louvain. They research and maintain databases about emergencies and disasters around the world. They are respectable and respected. I wrote to EMDAT to check the data. The database manager at EMDAT replied: “You are right, it is an increase in the reporting.”
The importance of checking the data becomes shockingly clear. EM-DAT data and Our World In Data visuals are used and trusted by major media outlets, as shown in this graphic at the home page of Our World in Data.
When these outlets use Our World in Data graphics, or re-use the data underlying the graphics, it may well be that the journalists do not check the data itself. …They see a huge rise in natural disaster since the 1970s to the turn of the century. It is an entirely a false impression. …Of course, the actual data is correct – in its own way. Those are the numbers of reported natural disasters. Everything before the 1998 or so was due to spotty, incomplete reporting and the rise is solely “an increase in the reporting.” Once reporting infrastructure was set up properly by the late 1990s, we see the opposite: a decline in reported natural disasters.
Another example: the rising number of suicides.
From “Geography of loss: a global look at the uneven toll of suicide” by Meagan Weiland and Nirja Desai in Science, 23 August 2019 – “This paper is part of Science’s special series on unraveling suicide.”
“Suicide rates have fallen in countries shaded green and blue, with drops of up to nearly 48 deaths per 100,000 people since 1990. In other countries, colored yellow and red, the rate has jumped by as much as 10 deaths per 100,000. The 2017 rate is specified on some countries.”
How much trust should we invest in the data on the graphic? Is it factual that suicide rates are plummeting in China and Greenland and India, and rising in the US and Argentina? We needn’t look too far for the answer, starting with the graphic.
- Mauritius – “the only sub-Saharan country that consistently records and reports suicide rates.“
- India – “Even though its overall suicide rate has decreased, India accounts for more than 35% of female suicide deaths worldwide.”
- Greenland – “Although Greenland’s rate has declined dramatically, it remains the highest in the world.”
Does anyone believe that countries like the Democratic Republic of the Congo, which currently has a raging Ebola epidemic, are carefully recording each death of their citizens with ICD-10 (cause of death) codes for each death? And then reporting them to some international record-keeping organization? What about Venezuela – which is currently in total political and social disarray? Ridiculous ideas, of course, they are not recording and reporting suicide deaths because they cannot. From the paper …
“Suicide is a worldwide problem, but its effects are uneven. Although suicide rates – all rates noted here are annual deaths per 100,000 people – are rising in some countries, including the United States, most countries are seeing declines, for reasons that include restrictions on access to lethal means and improved mental health care. According to the World Health Organization (WHO), most countries do not collect detailed data on suicide; data for many countries here were drawn from rates estimated by organizations such as WHO and the Institute for Health Metrics and Evaluation’s Global Burden of Disease project.”
Only the more modern countries, with functional national health organizations and modern hospitals backed by medical bureaucracies, can even hope to accurately record and report suicides. In many nations, suicide is stigmatic, and coroners and other medical professionals have often erred on the side of compassion (for the families) and recorded suicides as “natural death” or “heart attack” — anything but suicide. As reported in this paper: “Comparative Analysis of Suicide, Accidental, and Undetermined Cause of Death Classification.”
“It is likely that suicide may be under reported due to both the social stigma associated with suicide as well as the reluctance of a medical examiner or coroner to make this classification if supporting data are uncertain (Timmermans, 2005).”
So, what of the suicide rate map from SCIENCE magazine and the Weiland and Desai paper? It appears most countries should have simply been marked “Not Enough Data”. No mention is made of any confounding factors such as “improved reporting” in the United States and other western countries. Guesses are not appropriate for the purpose of guiding International Policy decisions.
The effort put into the graphic for Science has possibly been wasted as it only serves to misinform readers about the rates of suicide in the various nations.
- Pretty graphics and fancy images do not mean that information/data is correct or dependable. They may not convey a factual visual impression of the data.
- Just because a fancy image or graph comes attached to the name of a respectable organization is no reason to accept the data as it has been presented to you. If it is important to you, check it.
- Pretty graphics can easily overcome or slip past readers’ critical thinking skills and thus misinform them.
- For my money? The prettier the picture, the closer I look at the underlying data.
It is not my intention to disparage any of the organizations mentioned in this essay. EM-DAT particularly has been civil and co-operative in sharing the facts about the reporting of natural disasters to their database and graphics others have made from it.
We see in the suicide paper how easily researchers can be fooled into accepting data that is not reliable in any scientific sense. How do WHO and the Institute for Health Metrics and Evaluation’s Global Burden of Disease project estimate suicide rates in countries that don’t even record cause of death?
However, I have worked with the World Health Organization and its regional Pan American Health Organization and out there, on the ground, where poverty and despair rule, they have dedicated people and they do great work!
As the news media spins further and further out of control, abandoning real journalism (often out of [seeming] necessity. There are too many stories and so few journalists and so little time in the 24-hour news race) and social media twists and transmogrifies every bit of news into some surreal monster, news consumers (you and I, dear reader) must supercharge our critical thinking skills and fact-checking routines.
Share your experiences and your best tips for ensuring that “we don’t get fooled again.”
This is an insightful essay. I disagree with one trivial aspect. Hansen says “There are too many stories and so few journalists.” The opposite is the case. See the horde of reporters at any event, often more than participants, all screaming similar questions. There is an oversupply of reporters and news media – and a shortage of profits. As a result, budgets are cut, and reporters can spend less time on each story. Press releases become stories. Clickbait becomes easy-to-write headlines.
About the author
Kip Hansen is a “lifelong science junkie, occasional science journalist.” See His posts at Watts Up With That, especially these.
- Darwin, We’ve Got a Problem – Protecting species when we don’t have a good definition of “species.”
- The Marvelous and Mysterious Monarch – The good news is that the bad news is wrong.
- Climate Disaster Confusion – A kind of journalist malpractice.
- Why You Shouldn’t Draw Trend Lines on Graphs – Done poorly, this is a serious cause of misinformation.
For More Information
Ideas! See my recommended books and films at Amazon.
- Politics in modern America: A users’ guide for journalists and reformers.
- We cannot agree on simple facts and so cannot reform America.
- Important advice: Learning skepticism, an essential skill for citizenship in 21st century America. About “extraordinary claims require extraordinary proof”.
- The missing but essential key to building a better America – Clear sight about our condition.
- Swear allegiance to the truth as a step to reforming America.
- We live in an age of ignorance, but can decide to fix this – today.
- Ways to deal with those guilty of causing the fake news epidemic.
- The secret source of fake news. Its discovery will change America.
- How journalists helped wreck the climate debate.
See the master’s books about using graphs.
As new sensors and computers provide more quantitative information, visual presentation of the results becomes the weak link in the process. Edward Tufte literally wrote the book about this increasingly important skill. I highly recommend these as both useful and fun to read.
This book should be taught to college freshmen: The Cognitive Style of PowerPoint: Pitching Out Corrupts Within. Powerpoint is on my list of most powerful and most mis-used information tools.