Summary: Kip Hansen shows how journalists’ careless use of numbers and graphs creates powerful but misleading stories. More skepticism by us will force them to change.
“Caution: Pretty Pictures Can Fool You“
By Kip Hasen at Watts Up With That.
Reposted with his generous permission.
In today’s digital and Internet-of-Things world, it is easy to transform information into images – graphs, charts and other visuals that are colorful and informative. Modern math and statistical software packages can do it all for you with a few clicks of your mouse or taps on the screen. These visuals can be very powerful in conveying your message to the public. That’s on the upside.
On the downside is the fact that these visuals can be very powerful in driving false or misleading messages into the public consciousness. These all-too-easy-to-create visuals are “a Blessing and a Curse”.
We have seen this in the last few days with the scandal of the Twit-o-verse banging on about fires in the Amazon promulgating photos that are more than a decade old (not of this time) or even of some other place. One of the results of the twitter-storm has been a special meeting at the G7 about Brazil’s fires and shock at the fact that the U.S. President didn’t attend that meeting. In the case of the fires in Brazil, it turns out that the numbers have become more important in the public mind than the what. See this from my comment to Les Johnson’s post “Amazon Fires are in…the Amazon.”
“Take a look at the Forest Fires Map at Global Forest Watch. Like the NY Times piece mentioned above, the map makes it looks like whole countries are on fire. This is an artifact of the size of the dots marking fires. ZOOM IN ON BRAZIL. Zoom in until you can see the Federal District of Brasilia clearly. See how the fires clump together in agricultural areas – sometimes, close enough, you can see that it is a series of burning local fields or pastures that it records as fires.”
“Now, zoom out and find the Dominican Republic – the island of Hispaniola – just to the west of Puerto Rico. Looks awful doesn’t it, island on fire. Zoom in and in and in and you will find that it is not prime cane harvest yet, only a few cane fields burning – all set intentionally as a necessary part of the cane harvest. Some may be rice paddies – where the rice stubble is burnt off after the harvest of the rice and rice straw.”
“All these alarming stories require local knowledge and local ground truthing….”
I recently saw an article that led me to chase up a graphic used about the number of natural disasters by type: from Our World in Data. Click to enlarge.
Our World In Data describes themselves as having “Research and data to make progress against the world’s largest problems. Scroll to all research: 2989 charts across 297 topics. All free: open access and open source.” Its website says that they are based at Oxford and are trusted in research and media and used in teaching.
Yet, there is something that does not seem right in that “Number of reported natural disasters” graphic. There is something that is a clue (and thus its saving grace): it is the word “reported”. The second clue is the gray text at the bottom giving the source of the data as EM-DAT, the international disaster database of the Centre for Research on the Epidemiology of Disasters of the School of Public Health at the Université catholique de Louvain. They research and maintain databases about emergencies and disasters around the world. They are respectable and respected. I wrote to EMDAT to check the data. The database manager at EMDAT replied: “You are right, it is an increase in the reporting.”
The importance of checking the data becomes shockingly clear. EM-DAT data and Our World In Data visuals are used and trusted by major media outlets, as shown in this graphic at the home page of Our World in Data.
When these outlets use Our World in Data graphics, or re-use the data underlying the graphics, it may well be that the journalists do not check the data itself. …They see a huge rise in natural disaster since the 1970s to the turn of the century. It is an entirely a false impression. …Of course, the actual data is correct – in its own way. Those are the numbers of reported natural disasters. Everything before the 1998 or so was due to spotty, incomplete reporting and the rise is solely “an increase in the reporting.” Once reporting infrastructure was set up properly by the late 1990s, we see the opposite: a decline in reported natural disasters.
Another example: the rising number of suicides.
From “Geography of loss: a global look at the uneven toll of suicide” by Meagan Weiland and Nirja Desai in Science, 23 August 2019 – “This paper is part of Science’s special series on unraveling suicide.”
“Suicide rates have fallen in countries shaded green and blue, with drops of up to nearly 48 deaths per 100,000 people since 1990. In other countries, colored yellow and red, the rate has jumped by as much as 10 deaths per 100,000. The 2017 rate is specified on some countries.”
How much trust should we invest in the data on the graphic? Is it factual that suicide rates are plummeting in China and Greenland and India, and rising in the US and Argentina? We needn’t look too far for the answer, starting with the graphic.
- Mauritius – “the only sub-Saharan country that consistently records and reports suicide rates.“
- India – “Even though its overall suicide rate has decreased, India accounts for more than 35% of female suicide deaths worldwide.”
- Greenland – “Although Greenland’s rate has declined dramatically, it remains the highest in the world.”
Does anyone believe that countries like the Democratic Republic of the Congo, which currently has a raging Ebola epidemic, are carefully recording each death of their citizens with ICD-10 (cause of death) codes for each death? And then reporting them to some international record-keeping organization? What about Venezuela – which is currently in total political and social disarray? Ridiculous ideas, of course, they are not recording and reporting suicide deaths because they cannot. From the paper …
“Suicide is a worldwide problem, but its effects are uneven. Although suicide rates – all rates noted here are annual deaths per 100,000 people – are rising in some countries, including the United States, most countries are seeing declines, for reasons that include restrictions on access to lethal means and improved mental health care. According to the World Health Organization (WHO), most countries do not collect detailed data on suicide; data for many countries here were drawn from rates estimated by organizations such as WHO and the Institute for Health Metrics and Evaluation’s Global Burden of Disease project.”
Only the more modern countries, with functional national health organizations and modern hospitals backed by medical bureaucracies, can even hope to accurately record and report suicides. In many nations, suicide is stigmatic, and coroners and other medical professionals have often erred on the side of compassion (for the families) and recorded suicides as “natural death” or “heart attack” — anything but suicide. As reported in this paper: “Comparative Analysis of Suicide, Accidental, and Undetermined Cause of Death Classification.”
“It is likely that suicide may be under reported due to both the social stigma associated with suicide as well as the reluctance of a medical examiner or coroner to make this classification if supporting data are uncertain (Timmermans, 2005).”
So, what of the suicide rate map from SCIENCE magazine and the Weiland and Desai paper? It appears most countries should have simply been marked “Not Enough Data”. No mention is made of any confounding factors such as “improved reporting” in the United States and other western countries. Guesses are not appropriate for the purpose of guiding International Policy decisions.
The effort put into the graphic for Science has possibly been wasted as it only serves to misinform readers about the rates of suicide in the various nations.
Conclusions.
- Pretty graphics and fancy images do not mean that information/data is correct or dependable. They may not convey a factual visual impression of the data.
- Just because a fancy image or graph comes attached to the name of a respectable organization is no reason to accept the data as it has been presented to you. If it is important to you, check it.
- Pretty graphics can easily overcome or slip past readers’ critical thinking skills and thus misinform them.
- For my money? The prettier the picture, the closer I look at the underlying data.
It is not my intention to disparage any of the organizations mentioned in this essay. EM-DAT particularly has been civil and co-operative in sharing the facts about the reporting of natural disasters to their database and graphics others have made from it.
We see in the suicide paper how easily researchers can be fooled into accepting data that is not reliable in any scientific sense. How do WHO and the Institute for Health Metrics and Evaluation’s Global Burden of Disease project estimate suicide rates in countries that don’t even record cause of death?
However, I have worked with the World Health Organization and its regional Pan American Health Organization and out there, on the ground, where poverty and despair rule, they have dedicated people and they do great work!
As the news media spins further and further out of control, abandoning real journalism (often out of [seeming] necessity. There are too many stories and so few journalists and so little time in the 24-hour news race) and social media twists and transmogrifies every bit of news into some surreal monster, news consumers (you and I, dear reader) must supercharge our critical thinking skills and fact-checking routines.
Share your experiences and your best tips for ensuring that “we don’t get fooled again.”
——————————————
Editor’s afterword
This is an insightful essay. I disagree with one trivial aspect. Hansen says “There are too many stories and so few journalists.” The opposite is the case. See the horde of reporters at any event, often more than participants, all screaming similar questions. There is an oversupply of reporters and news media – and a shortage of profits. As a result, budgets are cut, and reporters can spend less time on each story. Press releases become stories. Clickbait becomes easy-to-write headlines.
About the author
Kip Hansen is a “lifelong science junkie, occasional science journalist.” See His posts at Watts Up With That, especially these.
- Darwin, We’ve Got a Problem – Protecting species when we don’t have a good definition of “species.”
- The Marvelous and Mysterious Monarch – The good news is that the bad news is wrong.
- Climate Disaster Confusion – A kind of journalist malpractice.
- Why You Shouldn’t Draw Trend Lines on Graphs – Done poorly, this is a serious cause of misinformation.
For More Information
Ideas! See my recommended books and films at Amazon.
If you liked this post, like us on Facebook and follow us on Twitter. For more information see all posts about journalism, and especially these…
- Politics in modern America: A users’ guide for journalists and reformers.
- We cannot agree on simple facts and so cannot reform America.
- Important advice: Learning skepticism, an essential skill for citizenship in 21st century America. About “extraordinary claims require extraordinary proof”.
- The missing but essential key to building a better America – Clear sight about our condition.
- Swear allegiance to the truth as a step to reforming America.
- We live in an age of ignorance, but can decide to fix this – today.
- Ways to deal with those guilty of causing the fake news epidemic.
- The secret source of fake news. Its discovery will change America.
- How journalists helped wreck the climate debate.
See the master’s books about using graphs.
As new sensors and computers provide more quantitative information, visual presentation of the results becomes the weak link in the process. Edward Tufte literally wrote the book about this increasingly important skill. I highly recommend these as both useful and fun to read.
The Visual Display of Quantitative Information
This book should be taught to college freshmen: The Cognitive Style of PowerPoint: Pitching Out Corrupts Within
Thanks for calling out Our World in Data. I have twice contacted them about using unreliable sources for the excellently produced graphics, but to no avail.
OWID seems to lack all discrimination when choosing data and is more interested in pretty graphics. What a shame.
Don’t forget the pretty pictures they use in the news to fool us. https://news.globallandscapesforum.org/12424/indigenous-communities-slash-burn-changing-landscapes-amazon/
Indigenous people use slash and burn including the Amazon regions.
The first paragraph “In an indigenous village deep in the Amazon forest, a family clears a small plot in the forest, plants cassava or other crops for a couple of years, and then moves elsewhere, leaving the plot to be overtaken by forest.” Above this paragraph is a picture of “indigenous peoples” with women wearing modern cloth skirt with a hem, and a seated male with rubber calf high wading boots. Not sure if the front lady pictured is wearing an ancient Greek style bra using the extra of cloth, or a modern bra.
Ignore what “selling to upscale buyers in cities” means when you state “Baniwa people on the Upper Rio Negro in Brazil’s Amazonas state sell chili powder made from some of the dozens of varieties of hot peppers found in their territory, selling to upscale buyers in cities, he said.” It means an exchange economy is going on, and they could be doing for profit farming and not just subsistence farming.
Follow the mantra by writing “But global changes and climate threaten both that traditional lifestyle and forest people’s food security, Lars Løvold, special adviser for the Rainforest Foundation Norway, said at the Global Landscapes Forum in Lima.” Ignore all the growth and development by all persons in the area.
P.S. “Some tribes use shotguns for hunting…”
My point is that literally almost all humans have benefited from fossil fuel use. We are all guilty of being the progeny of survivors. We should only pick up the burdens that are actually ours, IMO. YMMV.
The use of misleading graphs is a big issue that many news agencies won’t address. The two main problems I see are the false zero and truncated data set.
For the former, the Y axis often starts at a value very close to the minimum value. This means the visual impression of the change is a lot greater than it really is. For example if the range is from say 18 down to 16, having the zero at 15 makes it look like a near 70% change, rather than just 11%. The more misleading issue is the truncated data set. The most glaring example of this is global temperatures. So many data files start in the 50s or even the 70s when the actual data goes back decades before. By showing the shortened set, the trend is exaggerated and the variability is lost. The current example of dodgy graphs is Amazon fires. Showing the complete data set gives a totally different picture to the one the media want to portray.
The big loss in all this is the credibility of the media. It just confirms the President’s claim of fake news.
Graphs, and statistics in general, should be viewed with skepticism and due diligence. Graphs are used a lot in finance, but any investor or analyst worth their salt knows graphs are worthless. And anyone that’s studied statistics knows they can be manipulated to show whatever you want to show if you’re clever enough. What is more troublesome is that “journalists,” of ten bloggers, don’t have the skillset to interpret and understand data to the degree they’re reporting it. They restate press releases and journal studies that are stuffed with the words “could,” “may,” might,” “possibly” and other obviously non-certain scenarios as fact. Which are immediately parsed and repeated as fact by the general public. Most often those with agendas. While the internet shoud be making people smarter and more informed, it’s making us more confused and flat-out wrong. And in many cases these days, to the point of believing false ideas where people take to the streets to literally beat it over people’s heads.
Musgrove,
“Graphs are used a lot in finance, but any investor or analyst worth their salt knows graphs are worthless.”
Grab that baby from the bath water before its lost! Like all powerful tool, graphs can be misused. But it’s nuts to say that they are worthless.
See the books cited at the end of this post for tips on using them properly.
“While the internet shoud be making people smarter and more informed, it’s making us more confused and flat-out wrong.”
It’s an article of faith in America that we’re little helpless puppies preyed upon by evil wolves. That nothing is our fault! If that’s the case, then just pick out who you want to serve as a peon and be done with it.
The posts cited in the For More Info section point to another path. We can learn skepticism and to think for ourselves. We can learn to punish, or at least ignore, those who lie to us. We can, like our predecessors, act like citizens.
“Grab that baby from the bath water before its lost! Like all powerful tool, graphs can be misused. But it’s nuts to say that they are worthless.”
OK, I’ll grant that they can be used for supportive purposes to those that aren’t mathematically inclined or haven’t studied statistics, finance and advanced data analysis. But not in lieu of thorough analysis, and definitely not as standalone proof of a theory, or purported as the “fact” of the matter. Knowing how to look at and manipulate and use the data, and determine if the data is even clean and worthwhile and most importantly, good data, is what’s needed. As in your excellent example, all too often graphs are generated that are deceptive in so many ways. And these days, often intentionally. It’s a fool’s paradise out there.
“It’s an article of faith in America that we’re little helpless puppies preyed upon by evil wolves. That nothing is our fault! If that’s the case, then just pick out who you want to serve as a peon and be done with it.”
This is a an overstatement of my point, which may be to make your own point, which is duly noted. And while few would admit this is the course in life they’ve chosen because of pride, (usually the ones who have adopted it most whole-heatedly) evidence easily shows it’s in fact the path many have chosen to take in America. Partly due to human nature and partly/largely our current culture if I had to guess the reason.
muysgrove,
“I’ll grant that they can be used for supportive purposes to those that aren’t mathematically inclined or haven’t studied statistics, finance and advanced data analysis”
I subscribe to Science and Nature, and read the occasional article from the Am Meterological Society journals. Most of those authors and readers are “mathematically inclined”, have studied statistics, and often “advanced data analysis.” These papers make ample use of graphics. I guess everybody just isn’t as smart as you are!