You do not need advanced math to judge most statistics in the news. You need a short list of questions and the habit of asking them calmly. This level gathers the course into a practical checklist.

1. Who measured it, and why?

Find the original source: a statistics agency, a university study, a company report, an advocacy group. Sources differ in their methods, their transparency and their possible interests. A company selling a product, or a group campaigning for a cause, may be accurate, but it has reasons to present numbers in a favorable light. Good sources explain their methods and publish their data. If you cannot find where a number came from, treat it with caution.

2. What exactly was measured?

Headlines simplify. “Crime” might mean crimes reported to police, or crimes people describe in a survey, which can move in different directions. A study of “health” might have measured a blood marker, not illness itself. Definitions and proxies can differ from what the headline implies. Also ask who was studied: a study of 50 university students, or of mice, may not apply to everyone.

3. Compared to what?

A number alone is rarely meaningful. Ask whether there is a baseline, trend or comparison group that gives the number meaning. Is it higher than last year? Than the long-term average? Than similar places? Than what chance would produce? A drug that helps 60 percent of patients sounds impressive until you learn that 55 percent improve on a placebo.

4. How big is it?

Look for absolute numbers alongside percentages, rates per person alongside totals, and the size of an effect, not just its existence. Researchers call a result statistically significant when it would be unlikely to arise by chance alone if there were no real effect. That is not the same as important. With a big enough sample, a tiny difference can be significant. A result can be unlikely to be due to chance yet too small to matter in practice.

5. What is missing?

Ask what you are not being shown. Is there a margin of error or range reported with the estimate? Are failures or dropouts left out, as in survivorship bias? Is it the one study that found an effect among many that did not? One study is a clue, not a conclusion: it is wise to wait for consistent results from several independent studies before becoming confident.

A sanity check

A quick, rough calculation, called a Fermi estimate after the physicist Enrico Fermi, who was famous for them, can show whether a claim is even plausible. Suppose someone says a city of 500,000 people throws away 10 million plastic bottles a day. That is 10,000,000 divided by 500,000, or 20 bottles per person, every day, including babies. That seems far too many, so the claim deserves checking. A Fermi estimate is a rough calculation to see whether a claim is plausible, not a precise answer.

Questions to ask

• Where does this number come from, and can I see the original?
• What exactly does it count, and who is included?
• Compared to what: last year, other places, a control group?
• Is it a count, a rate or a percent? Relative or absolute?
• How uncertain is it?
• Could something else explain the pattern?
• Does a quick calculation make it plausible?

A citizen's attitude

The goal is not to distrust every number but to trust them in proportion to the evidence. Most official statistics are produced carefully by people trying to get them right, and they are among the best tools a democracy has for seeing itself clearly. Asking good questions respects that work, catches the misleading claims, and makes public debate better informed.

Worked example: the recess claim

A press release says: “Students at schools with longer recess score 15 percent higher on reading tests.” Run the checklist.

Who measured it? The release comes from a company that sells playground equipment. That does not make it false, but it gives the company a reason to highlight favorable findings. Is there a full report with methods?

What was measured? Which tests, which grades, which year? Was “longer recess” 5 extra minutes or 30?

Compared to what? Suppose the report shows an average score of 60 points at schools with short recess and 69 at schools with long recess. Fifteen percent of 60 is 9, so the arithmetic checks out. But were the schools similar? If schools with longer recess are mostly in wealthier districts, family income could be a confounder that raises both recess time and test scores.

How big? Nine points on a test scored out of 100 may be meaningful. But how many schools were studied? Ten schools would give a much shakier result than a thousand.

What is missing? No margin of error is given. Were schools that dropped out of the study, or that showed no difference, left out? Do other studies agree?

Sanity check. Could recess plausibly affect reading? Perhaps, through attention and well-being, so the idea is not absurd.

A fair conclusion: “An interesting correlation from an interested source; we would want a fuller report, comparisons of similar schools, and ideally a trial in which schools are randomly assigned longer recess.” Asking these questions takes a few minutes and protects you from both naive belief and reflexive dismissal.