When they're not lighthearted movie star cameos, the digital doppelgängers have scary disinformation potential. A deepfakes researcher hopes our wariness keeps up with the tech's quickening advances.

Transcript

MARY LOUISE KELLY, HOST:

Perhaps you have seen the viral TikTok where Tom Cruise purportedly performs a magic trick with a coin.

(SOUNDBITE OF TIKTOK VIDEO)

MILES FISHER: (As Tom Cruise) I'm going to show you some magic. It's the real thing (laughter).

KELLY: That laugh - unmistakably Tom Cruise, right? And it sure looks like Tom Cruise when you watch the video. It is not, though. This video is a deepfake, an image altered with artificial intelligence in a way that makes it difficult - really difficult - to tell that it is not real. Well, here to tell us how it works is University of California, Berkeley professor Hany Farid. Welcome.

HANY FARID: It's good to be here.

KELLY: What did you make of this Tom Cruise deepfake? The voice, the mannerisms, they are perfect. Did it fool you?

FARID: It's exceedingly well done. And it's been - it was interesting to see because part of the evolution of what we've been seeing since 2017, where every three to four months a video hits TikTok, YouTube, whatever, and it's just, wow, this is much, much better than before. And this is clearly a new category of deepfake that we have not seen before.

KELLY: Just explain, what are we actually seeing? Is this real video of Cruise, but it's been manipulated? Is this an actor? What's happening?

FARID: What you're seeing in these videos is not Tom Cruise. It is an actor who looks a little bit like Tom Cruise, clearly sounds like Tom Cruise. But on every frame of the video, at somewhere between 24 and 30 frames per second, the actor's frame was replaced with Tom Cruise's face. And that process is done digitally and with advances in machine learning and big data. And almost certainly, there was some post-production in this to sort of clean it up and get it really polished and high quality. And if you can replace somebody's face on every frame of a video, you can make it look like it's Tom Cruise or you or me or anybody else.

KELLY: Is this legal? I was looking - the account in question here is @deeptomcruise. That's the account posting this. Does TikTok have an obligation to take that account down once it has been established that this isn't real, that this is a deepfake?

FARID: Man, that's a great question. So I'm not the lawyer to ask this question to, but there is a really interesting question here around identity. So for example, many states have passed laws banning nonconsensual pornography, where one person's likeness is inserted into sexually explicit material. And you can see clearly why you would do that. It is harmful to that individual.

This one's a little bit different. It's not clear that it's harmful to Tom Cruise. Now, he may say, look, this is a copyright infringement because you're using my face and my likeness, at which point TikTok or YouTube or whomever would be obligated to take down the material.

But I think we are starting to tread into some interesting legal and ethical territory is, who owns that identity? And if that person is a person in sort of the public sphere - a president, an actor - do they have different rights than, say, a private individual like me or you? I don't think we've fully figured out how we're going to navigate that space.

KELLY: Yeah. And when you talk about the dangers of this, I'm thinking there's such a range. There's, you know, you touched on pornography, nonconsensual pornography - you know, a woman's photo being linked to something that she is not doing. I also read where you have talked about the potential of deepfakes to pose a national security risk. How so?

FARID: So here's a couple of scenarios you can imagine. Somebody creates a video of President Biden saying, I've launched nuclear weapons against Iran, and that goes viral online. How long does it take before somebody panics and pushes the button in return? And that's the danger here, is first of all, it's not just the content, but it's that we can deliver it online en masse to millions of people around the world, have it go viral and before anybody gets around to figuring out that it's fake, we have a global nuclear meltdown.

Here's another scenario. I create a video of Jeff Bezos quietly saying that Amazon's profits are down 20%. That video goes viral. How long does it take me to move the market to the tune of billions of dollars?

Now, are either of those scenarios highly likely? No. But are they possible? Yes. And the consequences should give us pause because we know that things can spread online incredibly fast. And before anybody circles around to figuring out what's what, you can imagine some very, very bad consequences from that material.

And frankly, that is outside of the deepfake phenomena. Why we have the misinformation apocalypse that is upon us now is because it's so easy to spread misinformation, and people are so willing and eager to spread it. And deepfakes is now throwing jet fuel onto that already burning fire.

KELLY: Professor Hany Farid of the University of California, Berkeley. Professor Farid, thanks.

FARID: It's very good to be with you. Thanks for talking.

(SOUNDBITE OF MUSIC) Transcript provided by NPR, Copyright NPR.