Numbers run through public life. They decide how money is shared among regions, which treatments doctors recommend, where new schools are built and how we judge whether a policy worked. A citizen who cannot read them is at the mercy of whoever presents them.
Counting is power
The United States Constitution requires a census every ten years, and the first was taken in 1790. Its original purpose was to divide seats in the House of Representatives among the states according to population. Today census figures also guide the distribution of large amounts of public money. Who is counted, and how carefully, has real consequences.
Statistics can also save lives. In the 1850s Florence Nightingale, famous as a nurse, was also a skilled statistician. Her analysis of British army records from the Crimean War showed that most soldiers' deaths came from disease rather than from battle wounds, much of it linked to filthy, overcrowded hospitals. Her clear diagrams helped persuade officials to reform military sanitation.
“Lies, damned lies, and statistics”
Numbers also have a reputation for deceit. In 1906 Mark Twain popularized the line “There are three kinds of lies: lies, damned lies, and statistics,” attributing it to the British prime minister Benjamin Disraeli. No one has found evidence that Disraeli said it. The joke lasts because numbers can feel objective while hiding many choices.
Every statistic involves choices
Behind every number are decisions about what to count, how to measure it and what to compare it with. Consider unemployment. Does someone who has given up looking for work count as unemployed? What about a person working five hours a week who wants full-time work? Different definitions give different numbers, all honest. Every statistic depends on choices about what to count, so when a definition changes, the number can change even if the world has not.
Common ways numbers mislead
- No denominator. “500 accidents at this intersection” sounds alarming, but without knowing out of how many, you cannot tell whether a number is large. Five hundred out of how many cars, over how many years?
- No comparison. Is the figure higher or lower than last year, than similar places, than what we would expect by chance?
- False precision. “The rally drew 23,417 people” may be a rough estimate dressed up as an exact count.
- Selective choices. Picking the starting year, the measure or the group that tells the story you want.
When measures become targets
The economist Charles Goodhart observed in the 1970s that statistical regularities tend to break down once they are used for control. The idea is now usually summarized as Goodhart's law: when a measure becomes a target, it stops being a good measure. If a police department is rewarded for lower recorded crime, it may record fewer crimes rather than prevent them. If schools are judged only on one test, teaching may narrow to that test. The numbers improve while the reality they were meant to track may not.
Two errors to avoid
There are two opposite mistakes. One is swallowing every number because it looks scientific. The other is dismissing every number because “you can prove anything with statistics.” Both leave you defenseless. In fact, it is much easier to lie without statistics than with them, and good statistics are often the only way to check a claim. This course aims at the middle path: asking a few calm, practical questions that reveal whether a number deserves trust.
Worked example: one headline, five questions
A local news site runs the headline: “Bike thefts soar by 200 in Riverton!” Last year there were 400 thefts; this year there were 600. Before reacting, ask a few questions.
- How big is the change in percentage terms? An increase of 200 on a base of 400 is 200 divided by 400, or 50 percent. That is a real rise, worth attention.
- Did the population change? Suppose Riverton grew from 50,000 to 60,000 residents after a new housing development. Last year's rate was 400 divided by 50,000, or 8 thefts per 1,000 residents. This year's is 600 divided by 60,000, or 10 per 1,000. Per person, thefts rose 25 percent, not 50.
- Did the definition change? Perhaps police began counting electric scooter thefts as bike thefts this year. If so, part of the rise is a change in counting, not in crime.
- Did reporting change? A new online form makes reporting easier. More reports could mean more thefts, or simply more people bothering to report.
- Compared to what? If the five-year average is 550, last year's 400 may have been unusually low, and this year looks less dramatic.
None of these questions dismisses the headline. Thefts may truly be rising, and the town may need better bike parking. But the honest summary might be: “Bike theft rate up about 25 percent per resident, partly due to new counting rules,” which is a different story from “soaring.” Asking calm questions turns a scary number into useful information.