Skip to main content

How do we know whether a country is backsliding? Journalists often cite a single score or label, but behind each lies a set of choices about what democracy means and how to observe it. Four major datasets dominate the field, and they do not always agree.

The main indices

Freedom House, an American organization founded in 1941, has published its Freedom in the World report since 1973. Analysts score each country on political rights (up to 40 points) and civil liberties (up to 60 points), for a total out of 100, and label it Free, Partly Free or Not Free. Its reports have recorded a net decline in global freedom every year since 2006.

Varieties of Democracy (V-Dem), based at the University of Gothenburg in Sweden, takes a different approach. V-Dem asks thousands of country experts to rate hundreds of specific indicators, such as whether journalists are harassed or whether courts act independently, and then combines their answers using statistical models that adjust for differences among raters. It measures several conceptions of democracy, including electoral and liberal democracy. V-Dem's recent reports estimate that about seven in ten people now live in autocracies of some kind, a figure heavily influenced by how it classifies India.

Polity, the oldest of the long-running datasets, scored regimes on a scale from minus 10 (fully autocratic) to plus 10 (fully democratic), focusing on how executives are chosen and constrained. Its last version covers events through 2018. The Economist Intelligence Unit's Democracy Index, begun in 2006, rates countries on 60 indicators and sorts them into full democracies, flawed democracies, hybrid regimes and authoritarian regimes.

Why they disagree

Indices differ because they define democracy differently. A thin, electoral view asks mainly whether rulers can lose elections. A thicker, liberal view also counts courts, press freedom, minority rights and the rule of law. A country can do well on the first and poorly on the second. Indices also differ in who codes them: a few staff analysts, thousands of outside experts, or published sources. And they weigh components differently, so the same facts can yield different labels.

The expert bias debate

Because most indices rely on expert judgment, critics ask whether experts' own views shape the scores. In work published in 2023 and 2024, Andrew Little and Anne Meng argued that objective indicators, such as whether incumbents lose elections or journalists are jailed, show much less global decline than expert-coded indices. They suggested experts may have grown more pessimistic, or more sensitive to problems, over time. Defenders of expert coding, including V-Dem researchers, replied that modern backsliding is designed to avoid crude, countable signals: a captured court or a self-censoring press does not show up in counts of jailed journalists or lost elections, so objective measures will miss it. The debate is unresolved and healthy. A sensible reader treats every index as a measurement with error, checks whether several indices agree, and looks at the specific indicators behind a headline number.

Reading a score wisely

Three habits help. First, ask what definition of democracy an index uses before comparing labels. Second, look at trends within one index rather than mixing scores across indices. Third, read the indicators: a fall driven by harassment of journalists means something different from a fall driven by a disputed election. Labels such as "electoral autocracy" are thresholds on a continuous scale, so a small change can flip a country across a line without much changing on the ground.

Worked example

India, the world's largest democracy by population, shows the stakes of measurement. In 2021 Freedom House downgraded India from Free to Partly Free, and V-Dem reclassified it as an electoral autocracy. The EIU kept it a flawed democracy. Critics and defenders can both point to evidence.

Those who see backsliding cite the use of sedition and antiterror laws against journalists, students and activists; tighter rules on foreign funding for nongovernmental groups; frequent internet shutdowns, especially in Kashmir; pressure on broadcasters; and laws such as the 2019 Citizenship Amendment Act, which critics say discriminates against Muslims. Those who reject the label point to huge, competitive elections run by an independent Election Commission, though opposition parties have questioned its neutrality; to opposition parties that win and govern many states; and to the 2024 general election, in which Prime Minister Narendra Modi's Bharatiya Janata Party lost its single-party majority and had to rely on coalition partners. India's government dismissed the downgrades as biased outsiders' judgments.

The measurement tools explain the split. On a thin, electoral view, elections that can and do change governments at the state level, and that cut the ruling party's majority, look democratic. On a liberal view that weighs press freedom and minority rights heavily, the trend is worrying. India also shows why global totals swing: with about 1.4 billion people, one country's label changes the share of humanity counted as living in autocracies. Remember too that Indian democracy has recovered before. In 1975 Prime Minister Indira Gandhi of the Congress Party declared a state of emergency, jailed opponents and censored the press; in 1977 she called an election and voters threw her out.

Quiz

5 questions · Practice: scored, not saved

1. How does V-Dem gather most of its data?
2. How did defenders of expert coding reply to the bias critique?
3. Before comparing labels from different indices, what should a reader ask?
4. What did Andrew Little and Anne Meng argue?
5. What scale did the Polity dataset use?

CivilSimian.com created by AxiomaticPanic, CivilSimian, Kalokagathia