At the center of the method is the hypothesis: a proposed explanation that can be checked. But what makes a hypothesis good? Some ideas can explain anything that happens, and that sounds like a strength. This level explains why it is actually a weakness, and why the best scientific ideas are the ones that take risks.

From hypothesis to prediction

A hypothesis is useful only when we can draw a prediction from it, usually in “if, then” form. If handwashing prevents the spread of infection, then wards where doctors wash their hands should have fewer infections than similar wards where they don't. The prediction is what we actually test. A hypothesis that predicts nothing in particular cannot be tested at all.

Popper's insight

The Austrian-British philosopher Karl Popper (1902 to 1994) noticed a basic asymmetry. No number of observations can prove a general claim true. You can see a thousand white swans, and the claim “all swans are white” can still be false, as Europeans learned when they encountered black swans in Australia. But a single clear counterexample can show a general claim is false. So Popper argued, in a book first published in 1934, that the mark of a scientific theory is falsifiability: it must forbid some possible observations. If nothing could ever count against it, it is not science.

Popper drew this lesson from the Vienna of his youth. He admired Einstein's theory of gravity because it made risky predictions that could have failed. By contrast, he felt that followers of Freud and Adler, and some Marxists, could explain any outcome at all after the fact. A theory that fits every possible result, he concluded, tells us nothing about which result to expect.

The 1919 eclipse

Einstein's general theory of relativity predicted that the Sun's gravity would bend starlight passing near it by about twice the amount Newton's physics suggested. The only time to see stars close to the Sun is during a total eclipse. On 29 May 1919, British expeditions led by Arthur Eddington and others photographed stars during an eclipse from the island of Príncipe and from Sobral in Brazil. The measurements agreed with Einstein, and newspapers around the world announced the result. The point for Popper was that the theory had stuck its neck out. If the stars had not shifted, relativity would have been in deep trouble.

Bold beats safe

Popper's view has a surprising consequence. A safe, vague hypothesis such as “something will happen to the economy next year” is almost certain to be true, but it is useless. A bold, precise hypothesis is more likely to be wrong, but if it survives tough tests it tells us much more. Good scientists therefore prefer claims that are precise enough to fail.

Escape hatches

When a prediction fails, it is tempting to rescue the hypothesis by adding a new excuse that was invented only for that purpose. Philosophers call this an ad hoc rescue. A psychic whose prediction fails might say the skeptics in the room disrupted the energy. Done repeatedly, this turns any idea into one that can never fail.

But there is a complication, pointed out by the physicist Pierre Duhem and the philosopher W. V. O. Quine. Every test depends on auxiliary assumptions: that the instruments work, that nothing else interfered, that the sample was what we thought. When a prediction fails, logic alone does not say whether the main hypothesis or one of these assumptions is to blame. Sometimes blaming an auxiliary assumption is exactly right, as Level 7 shows. The difference between honest adjustment and an ad hoc dodge is whether the new assumption can itself be tested independently.

Probabilistic claims

Many modern hypotheses are about chances, not certainties. “Smoking causes lung cancer” does not mean every smoker gets it. Such claims are still testable, but one counterexample does not refute them. They are judged by whether patterns across many cases fit the prediction, which is why careful comparison matters so much.

Worked example: testing a dowser

A dowser claims to find underground water by walking with a forked stick that dips over it. Is the claim scientific? It can be, if we turn it into a risky prediction.

• Make it precise: bury ten pipes under a field and run water through just one, chosen at random. The dowser must identify which.
• Agree in advance: before the test, the dowser and the testers agree that success means finding the right pipe far more often than one time in ten across many trials.
• Remove clues: no one present knows which pipe is flowing, so no one can give hints.
• Close the escape hatches: the dowser confirms beforehand that the conditions are fair.

If the dowser succeeds well above chance, the claim deserves serious attention. If not, the claim has been fairly tested and failed. Either way we learn something, which is the point. Controlled tests of this kind have generally found no better than chance performance. The method does not mock the dowser; it simply gives the claim a fair chance to be wrong.