Skip to main content

For most of the twentieth century, a photograph or a recording carried special weight. Images could be staged or doctored, but doing it convincingly took skill and time. That weight is now draining away. Software can generate a realistic face, voice or scene from a typed description, and editing tools can alter real footage in minutes. Seeing is no longer believing, and that changes both how we should check media and what the existence of fakes does to real evidence.

Cheapfakes come first

Before worrying about sophisticated fakes, notice how much damage simple tricks do. In May 2019 a video of the US House Speaker Nancy Pelosi, slowed down to make her speech sound slurred, spread widely and was viewed millions of times. Researchers call crude manipulation of this kind a cheapfake: media altered with simple tricks like slowing or relabeling, or by cutting a clip short. The most common cheapfake of all needs no editing: a real photo or video with a false caption, such as footage of a past disaster presented as today's. Most visual misinformation still looks like this.

Deepfakes arrive

The word deepfake comes from a Reddit user who posted face-swapped videos in 2017 under the name "deepfakes," using machine learning to paste one face onto another. Many of those early videos were sexual images of women made without their consent, and that abuse remains one of the most common and harmful uses of the technology.

Political uses followed. In March 2022, weeks after Russia's full-scale invasion, a crude deepfake appeared on a hacked Ukrainian news site showing President Volodymyr Zelensky telling Ukrainian soldiers to lay down their arms. It was quickly debunked, partly because Zelensky had warned of such fakes and posted a rebuttal. In May 2023 an AI-generated image of an explosion near the Pentagon spread from accounts that looked official, and US stock prices dipped briefly before officials confirmed that nothing had happened. In January 2024 voters in New Hampshire received robocalls that used an AI copy of Joe Biden's voice to discourage voting in the state's primary. The US Federal Communications Commission then ruled that AI-generated voices in robocalls fall under existing limits on artificial voices, and later fined the consultant behind the calls $6 million.

Fraud may be the fastest-growing use. In early 2024 police in Hong Kong reported that an employee of the engineering firm Arup had transferred about US$25 million after a video call in which the company's finance chief and several colleagues all seemed to be present. Every other face on the call was a deepfake. Voice-cloning scams, in which a caller sounds like a relative in trouble, have become common enough that many families now agree on a private code word.

The liar's dividend

The deeper danger may not be that people believe fakes. The legal scholars Robert Chesney and Danielle Citron pointed out in 2018 that as the public learns fakes are possible, people caught on real recordings can claim they are fake. They called this the liar's dividend. A society in which any inconvenient video can be waved away as "AI" loses one of its strongest tools for holding the powerful to account, from police misconduct to corruption. Information hygiene therefore cuts both ways: reflexive belief and reflexive dismissal are both failures.

Can fakes be detected?

Telltale signs such as odd hands, mismatched earrings or garbled lettering in the background were once fairly reliable, but each generation of tools fixes earlier flaws. Automated detectors exist, but they struggle with new kinds of fakes and can wrongly flag real images. Many researchers now put more hope in provenance: a record of where media came from and how it was edited. In 2021 Adobe, Microsoft, the BBC and others formed the Coalition for Content Provenance and Authenticity, which created a standard for attaching signed "content credentials" to images and video. Some AI companies also add invisible watermarks to what their tools generate. These help, but credentials can be stripped when a file is screenshotted or re-uploaded, and the absence of a label proves nothing either way.

Practical checks

For ordinary readers, the best tools remain the habits of the last level. Run a reverse image search to find earlier copies of a picture. Look for the original upload and its date. Check whether reliable outlets with people on the scene report the same event. Be most careful with media that shows exactly what you hoped or feared to see.

Worked example

A short clip shows a mayor apparently saying that one neighborhood "doesn't deserve" new schools. It has no source and is spreading fast. Two failures are possible here, so check for both. First, could it be fake or edited? A reverse image search on a frame finds a longer video of a council meeting. Second, does the clip mean what it seems? In the full video, the mayor is reading aloud from a critic's letter before rejecting it. The clip is real footage cut to reverse its meaning: a cheapfake. Now suppose instead that the full video had confirmed the remark, and the mayor's office replied that "this is obviously AI." Without provenance or any evidence, that denial would deserve the same scrutiny as the clip. Real evidence should not be dismissed just because fakes exist.

Quiz

5 questions · Practice: scored, not saved

1. What was unusual about the Arup video call in Hong Kong?
2. What did the New Hampshire robocalls of January 2024 do?
3. What did the March 2022 Zelensky deepfake show him doing?
4. What is provenance in the context of media?
5. What was done to the 2019 video of Nancy Pelosi?

CivilSimian.com created by AxiomaticPanic, CivilSimian, Kalokagathia