We woke up last Friday (in Oz) and Thursday (in the US). As usual, we scanned a selection of online newspapers, magazines and blogs: “Eurozone crisis will cost world’s poorest countries $238bn” blared the UK Guardian (once known as the Manchester Grauniad because of its typographic lapses). Really, not $237 billion or $239 billion? Perhaps this was just a dodgy headline. Sadly it was not — the article soberly reports that the cost will be exactly this figure, an impossible level of certainty for an economic forecast!
Many other such examples come to mind. On 18 Oct 2006, the venerable New York Times soberly declared that the 300 millionth American had arrived at 7:46am EST, according to the U.S. Census Bureau, with no mention of the well-known fact that millions of Americans are uncounted (or indeed the level of uncertainty in any such count) because they are poor or undocumented, and even ignoring such uncertainties the date and time could not possibly be specified. Similarly, a 31 Oct 2011 Times headline trumpeted the arrival of the seven billionth human on the planet. At least in this case, it noted that according to the best government censuses, there was a “window of uncertainty” of six months either way.
So are we are just finicky mathematicians? Does it really doesn’t matter? Sadly it does. These numbers often drive the news cycle. As a single example, on 27 Jan 2009, CNN alarmed U.S. readers with the headline “Bank bailout could cost $4 trillion.” But this accounting did not include expected paybacks. Now, according to a little-publicized April 2012 U.S. Treasury report, all but $60 billion of the original $700 billion has been repaid, and it is expected that eventually all will be repaid, returning a net profit to the taxpayer (see Math Drudge blog for additional details).
In a similar way, unemployment reports are volatile and are often dramatically revised after initial release—US figures are more frequently revised than Canadian and Australian figures which are released less quickly. For example, on June first 2012, the U.S. Labor Department reported that the U.S. economy added just 69,000 jobs, far fewer than the 165,000 predicted, and the same report slashed the April figure to 77,000, down from 115,000 in an earlier report.
[Added on 29 Sep 2012] Along this line, one of Romney’s key “talking points” against the Obama presidency is that there has been a net loss of jobs during the first Obama administration. However, the jobs statistics released by the U.S. Department of Labor on 27 Sep 2012 show that there has now been a slight net gain in employment.
Needless to say, employment reports are used by policy makers, investors and others worldwide to make very important decisions: deciding elections, driving countries into sovereign default, possibly making fortunes for a few while trimming the pensions of many. Indeed, recent employment figures in the U.S. could become a deciding factor in the upcoming presidential election.
Misleading, or at least less-than-fully explicit numerical reports are also rampant in reporting of health and environmental issues. The real numerical facts behind Mad cow disease, oil and natural gas reserves, earthquakes caused by fracking, and the health effects from the Fukushima accident are often far different than understood even by the moderately well educated public.
Some of this is inevitable, since ambiguous headlines, like uncertain witnesses, do not make cases. But even assuming pristine intentions, the discourse is nearly always hyperbolic and inimical to good policy making.
But perhaps we have been cherry picking here — recalling our favorite historical bloopers, and ignoring the much more numerous instances of responsible reporting? To test this hypothesis we decided to look online only for stories with a June 22 or Jun 23 byline. Here is what we saw.
- The LA Times reported that employment in Orange County’s hotels, restaurants, theme parks and other tourism-related business hit “181,500 jobs,” which exceeded the “181,400 jobs” figure in July 2008. Four significant digits?
- BBC told us that Bird flu ‘could mutate to cause deadly human pandemic’. Inside the article we read “The virus is also deadly to humans but can only be transmitted by close contact with infected birds. … It is for this reason that relatively few people have died of bird flu. Latest World Health Organization (WHO) figures indicate 332 people have died of the illness since 2003.” Many other articles reported the pandemic was only three mutations away, with no attempt to explain what that meant, but it ain’t good.
- The Chicago Sun-Times reported that 144,000 Medicare recipients in Illinois have received 50% discounts on brand-name drugs, for an average savings of “$667 per person.” Not $666 per person?
- The Dallas Morning News headlined “Middle-income family spends $235,000 to raise baby.” The article subsequently makes it clear that this is a current average figure, from a government report. But the article also points out, referencing a U.S. Department of Agriculture report, that the cost of raising a child was just over $25,000 in 1960. But it added that this figure “would be $191,720 today when adjusted for inflation.” Five significant figures!
- The Washington Post reports that the U.S. Army supports “99 bands” and intends to spend “$221.1 million” on them in the coming year. It is hard to believe that they have a perfect listing of all musical groups, much less a 4-significant-digit figure for the exact amount to be spent on them all in the coming budget year.
- The Toronto Star tells us that “Canada ranked 51’st in access to information list.” The article never describes how the rating was determined, being satisfied with noting that Canada has dropped by 12 places. It does tell us that “Serbia, India and Slovenia top the report’s ranking list, while Liechtenstein, Greece and Austria come last among the 89 countries with an access regime.” I guess we should all buy tickets to Serbia.
We have blogged earlier on the innumeracy crisis and its impact on the public’s assessment of risk. Let us finish here with a few suggestions for journalists and bloggers:
- Avoid bogus certainty: “may cost… over $200 billion Euro’s” is just as informative and lot more more honest than “will cost … $238bn.”
- Headlines should honestly reflect content. The Wall Street Journal’s headline Global Warming Seen Lifting California Sea Level a Foot distorts an otherwise good article which starts much more carefully:
Global warming may push sea levels as much as a foot higher in California in the next two decades, threatening airports, freeways, ports and houses, according to a report examining risks along the U.S. West Coast.
Increases are forecast to be greatest south of Cape Mendocino, with levels rising 1.5 inches to 12 inches (4 to 30 centimeters) by 2030.
- Talk about relative likelihood. Even with its cautious opening, the WSJ article made no attempt to quantify the probabilities involved or how much confidence even the researchers had in their analysis. This is now routine with opinion polls such as this Canadian poll, which ends “a randomly selected sample of 1,099 adult Canadians was interviewed online throughout the Ipsos online panel. The margin of error is 3.1 percentage points, 19 times out of 20,” although we wonder how much this means, even to educated readers. We also note that changes in polling methodology typically have much larger impacts that 3.1 percent.
- Resist the urge to over-quantify. Compare the following two headlines: Biomass Study Finds World’s Population 18.5 Million Tons Overweight (Voice of America), which headlines a silly article on a real problem, and a sensible discussion entitled Is Grexit good for the euro? (Economist), which refreshingly had no numbers!
Numbers are not holy water to sprinkle through each article. That kind of profligate number inflation does nothing but confuse and complicate. On the other hand, careful use of well-explained numerical data can make or break an issue. Next time you read any article will lots of numbers, you might ask yourself what they measure, how accurate they might be, and how they were determined.
[A version of this article also appeared in The Conversation.]