How Are Deepfakes Made?

Every deepfake you have seen was made one of three ways: a face swap, a lip-sync reenactment, or a fully AI-generated clip, usually with a cloned voice on top. This guide explains how each method works, what it actually takes to produce one, and the fingerprints every method leaves behind, because knowing how deepfakes are made is exactly how detectors catch them.

In early 2024, a finance worker at the engineering firm Arup joined a video call with his CFO and several colleagues. Every face and voice on that call was fake, and the company wired out $25.6 million before anyone realized (CNN, 2024). That is the technology this guide explains.

Direct answer: A deepfake is a video, image, or audio clip created or altered by artificial intelligence to make a real person appear to say or do something they never did. The name combines "deep learning" and "fake". Deepfakes range from harmless parodies to tools for fraud and disinformation.

This is the canonical guide. By the end you will know what a deepfake is, how the technology works, what it is used for, and how detection fights back. I build detection models for a living, so you also get the view from the other side of the arms race.

The Short Answer

A deepfake is produced by training or prompting a neural network on images, video, or audio of a real person, then using that model to generate media the person never recorded. In practice there are three production methods: swapping a face onto existing footage, puppeteering a real face to match new audio, or generating the entire clip from scratch with a text-to-video model. Voice cloning supplies the audio for all three. Consumer apps have collapsed the skill required from research-lab level to a phone screen, which is why deepfake volume keeps setting records.

Method 1: Face Swaps

The classic deepfake. An autoencoder or GAN is trained on two sets of faces: the person in the original footage and the person being inserted. The model learns to encode any face into a compact representation, then decode it as the target face, matched frame by frame for pose, expression, and lighting. Early face swaps needed thousands of photos of the target; modern one-shot systems can produce a passable swap from a handful of images, and the results are strongest on front-facing, well-lit footage such as interviews and video calls.

Where it breaks: profile angles, hands crossing the face, jewelry and glasses, hairline boundaries, and inconsistent lighting between the source scene and the training photos. Those seams are a primary target for video deepfake detection.

Method 2: Lip-Sync and Reenactment

Instead of replacing the face, reenactment keeps the real face and animates it. The attacker supplies new audio, and the model regenerates the mouth region, jaw, and often the head motion so the person appears to say the new script. This is the method behind most fake celebrity endorsements and executive announcements, because it starts from genuine, recognizable footage and only alters the parts that carry the message.

Where it breaks: the mouth interior. Teeth, tongue, and the timing between phonemes and lip shapes are hard to synthesize precisely, and the regenerated region often has subtly different texture and sharpness than the untouched rest of the frame.

Method 3: Fully AI-Generated Video

The newest method skips real footage entirely. Diffusion-based text-to-video models generate every pixel from a prompt, optionally conditioned on a reference photo of the person. Nothing is edited, so there is no editing seam to find; the entire clip shares one synthetic fingerprint. Quality has jumped generation over generation, and short vertical clips, the format of most social video, are already convincing at feed scrolling speed. Our guide to GAN versus diffusion deepfakes covers the architecture shift in depth.

Where it breaks: physics and persistence. Objects morph between frames, text and logos smear, backgrounds wobble, and lighting can disobey the scene. The artifacts are different from face-swap seams, which is why detectors are trained per method.

How AI Voices Are Cloned

Voice supplies the persuasion. Modern speech models clone a voice from seconds of reference audio by mapping its pitch, timbre, accent, and pacing into a voiceprint, then synthesizing arbitrary text in that voice. This is the engine behind grandparent scams, fake CEO calls, and the audio track of most video deepfakes. Cloned speech tends to be too clean: breath, room tone, and micro-hesitations are statistically flattened, signals an AI voice detector is built to catch even when human listeners are fooled.

What It Actually Takes

A decade ago a convincing deepfake took a research team, a GPU cluster, and weeks of training. Today the practical inputs are: publicly posted photos and clips of the target, a consumer app or hosted model that automates the pipeline, and minutes of compute. That collapse in cost is the entire story of the deepfake era. It also means the limiting factor is no longer skill but source material, and anyone with a public social profile has already supplied it. We deliberately do not publish tool walkthroughs; the point of understanding the pipeline is defense, and the harms are well documented.

Every Method Leaves Fingerprints

Generation is a statistical process, and statistics leave residue. Face swaps leave blending boundaries and identity flicker between frames. Reenactment leaves mouth-region texture mismatches and audio-visual timing drift. Fully generated video leaves globally consistent synthetic texture, impossible lighting, and temporal wobble. Cloned voices leave flattened spectral detail and missing room acoustics. Individually these are subtle; together they are measurable. That is the core of deepfake detection: you do not need to know what truth looks like, only what generation leaves behind.

How Detectors Catch Them

Detection models are trained on paired real and synthetic media until they separate the two distributions. In production, systems like ours run several signal families in parallel: pixel and frequency-domain forensics on images, frame-consistency and face-region analysis on video, and spectral analysis on voice, then return a verdict with a confidence score rather than a guess. No detector is perfect, and honest ones say so, but machines reliably beat unaided human judgment at this task. If you want the signs you can check with your own eyes first, start with how to spot a deepfake.

FAQ

How are deepfakes made?

By training or prompting AI models on images, video, or audio of a real person, then generating new media: face swaps replace a face in real footage, reenactment animates a real face to new audio, and text-to-video models generate entire clips from scratch.

Can anyone make a deepfake?

Increasingly yes. Consumer apps automate the full pipeline, and a public social profile usually provides enough source material. Skill is no longer the barrier.

How long does a deepfake take to make?

Minutes for app-based face swaps and voice clones; hours for higher-quality reenactment; a single prompt for fully AI-generated clips.

Is making a deepfake illegal?

It depends on use and jurisdiction. Fraud, defamation, election interference, and intimate imagery without consent are criminalized in a growing number of laws; parody and consented uses are generally not. See deepfake laws by state.

How are deepfakes detected?

Detection models look for the statistical residue each generation method leaves: blending seams, mouth-region mismatches, synthetic texture, temporal wobble, and flattened audio spectra, and return a confidence-scored verdict.

Related reading