(Curiosity) The Real Threat of Deepfakes

This is a video of President Nixon giving a speech he never made. The speech itself is real. It was prepared in advance of the 1969 moon landing should something go wrong. The MIT Center for Advanced Virtuality created the clip and accompanying website to highlight the dangers of disinformation.

From Space.com:

The 7-minute video begins with real Apollo 11 footage, which team members edited deceptively to create the impression that the epic moon landing failed. Then comes Nixon’s speech. It was actually read by a voice actor, but you’d never know it. The team used deep-learning and artificial-intelligence tech to reproduce Nixon’s voice and facial movements.

“This alternative history shows how new technologies can obfuscate the truth around us, encouraging our audience to think carefully about the media they encounter daily,” project co-leader Francesca Panetta, XR Creative Director at MIT Virtuality, said in a statement.

“In Case of Moon Disaster” debuted last year as an art installation, which featured a reproduction of a 1960s-era living room. The project has been selected by several film festivals, and it received a Creative Media Award from the Mozilla Foundation.

“It’s our hope that this project will encourage the public to understand that manipulated media plays a significant role in our media landscape, and that with further understanding and diligence we can all reduce the likelihood of being unduly influenced by it,” project co-leader Halsey Burgund said in the same statement.

You can learn more about the project at http://moondisaster.org.

It’s hard to have a nuanced discussion about this technology. Projects like this have a novelty that is hard to separate from rational appreciation of its downstream effects. There’s a real sense in which this video is just cool, regardless of the rest.

Deepfake tech will revolutionize entertainment, particularly pornography, which will sell videos of actresses and other public figures getting fucked in the ass, among other things. It will dramatically expand the possibilities of satire — one can see Saturday Night Live using it — but of course libel and slander as well. It also raises ethical issues about using people’s likenesses without their permission, especially when they’re dead.

But I’m not sure it will be routinely used to mislead large numbers of people or feature prominently in national-scale propaganda anywhere except countries like China. And while that is a real threat, the problem is China’s political system, which retains a monopoly on narrative, and its use of technology broadly.

For one, while deepfakes can fool the human eye (and ear), they retain signs that they’re deepfakes. Even before we get to a detailed pixel analysis, there are the obvious questions like “Where did this video come from?” In the case of the moon landing disaster, such a video would had to have been stored on some kind of legacy media, which, if it were real, we should be able to examine.

We also have to be careful about predicting the future by drawing a straight line from the present. In the West, where narratives are hyper-competitive, large lies will be subject to scrutiny. And in fact, as these technologies proliferate, I expect people will become increasingly skeptical about fantastic claims, at least about anything they don’t already want to believe.

The MIT ended with a call to “Check Your Sources.” While that’s always good advice, it also seems surprisingly naive. It suggests this technology poses a direct threat, where bad actors will circulate standalone lies like this, which can be defeated with a little fact checking.

In such cases, the cost-benefit of the higher cognitive burden of critical evaluation is clearly positive. There is a net benefit to taking the time to critically analyze a video of a US president or other world leader. (Some people will be fooled, but some people are already fooled.)

In truth, this technology’s most pernicious effects will be felt locally, in places where motives are obscure and the cost-benefit of engaging in critical analysis is low, either because the benefit is low or the cost on the bearer is high (or both).

It will be used for all kinds of trolling behaviors, for example, like convincing a teen that her girlfriend is cheating or suggesting to the boss that you badmouthed him at the bar or getting the neighbor in trouble with law enforcement — situations where it’s not clear someone would bother to create a fake.

Political parties and intelligence services will also use deepfake tech of course, but nothing as ham-fisted as this. They have big budgets and will use it with sophistication and laser focus, creating small lies that link two real, verifiable but unrelated facts as part of a larger narrative warfare campaign, fed by big data, much of which will actually be true but which collectively creates a false version of reality.

The cognitive burden of parsing that campaign, which will never be obvious as a campaign, will be extremely high. Given that we won’t know what is fake and what is real, we will have to test all of it, which is costly and time consuming, leaving us victims of an enormous asymmetry. Corporations and governments have those resources ready to hand. Ordinary people do not.

In all these examples, “checking your sources” is either technically infeasible or hardly matters: there will be nothing you can do to prevent a pedophile from creating an X-rated video of your 11-year-old daughter, scraped from an innocent vacation video or surreptitious footage at the park.