How Do Deepfakes Work? A Plain-English Guide

A convincing deepfake does not appear out of nowhere. It is the output of a model that has learned what a particular face or voice looks and sounds like, then used that learning to generate new video, images or speech. This guide explains that process in plain English: what the model is doing, how the main types of deepfake are put together, why they are getting harder to spot, and how you can check a file you are unsure about. If you want the basics first — what a deepfake is, the main types and the risks — start with our guide to what a deepfake is.

The simple idea behind every deepfake

Deepfakes rely on artificial neural networks: computer systems that learn to recognize patterns in data. The U.S. Government Accountability Office (GAO) explains that building a deepfake photo or video typically involves feeding hundreds or thousands of images into a neural network and training it to identify and reconstruct patterns, usually faces (GAO, February 2020).

Once trained, the model can generate new frames of that face, place it onto another person's body, or make it move and speak in new ways. Voice deepfakes work on the same principle: a model learns the characteristics of a person's voice from recordings and then produces new speech in that voice.

The word itself points to this process. As the U.S. Department of Homeland Security explains, "deepfake" combines deep learning, the type of AI used to generate the content, with fake (DHS).

Two things follow from this. First, a deepfake is only as good as the examples the model has seen. People who appear often in public videos, interviews and social posts give these tools more to learn from, which is why executives, public figures and well-known brands are frequent targets. Second, not every edited clip is a deepfake. Cropping a video, adjusting colour or cutting it for length are ordinary edits. What makes something a deepfake is that AI has been used to fabricate a person's likeness or voice so that it is meant to pass as real.

The two kinds of model the GAO describes

The GAO notes that deepfakes use different underlying AI technologies, notably autoencoders and generative adversarial networks, or GANs.

Autoencoders

An autoencoder is a neural network trained to reconstruct its input from a simpler representation. In plain terms, it learns to squeeze a face down to a compact description — roughly, "this is a face looking slightly left, smiling, in this lighting" — and then rebuild a full face from that description.

A common way to swap faces uses this idea. The model learns a shared way to describe expressions and head position, and a separate way to rebuild each person's identity. At generation time, the expression and pose from one person are passed through the "rebuild" step for the other person. The result is person A's face, wearing person B's expression, ready to be placed back into B's video.

Generative adversarial networks

A GAN is made up of two competing neural networks, one trying to produce a fake and the other trying to detect it. That competition continues over many cycles, which, the GAO says, results in a more plausible rendering of, for example, faces in video. GANs generally produce more convincing deepfakes, but they are more difficult to use.

These are not the only methods in use today. Generative AI systems can also create new images, audio or video when given a text prompt, by learning patterns and relationships from large amounts of training data (GAO, June 2023). The important point for a viewer is the same in each case: the output is synthesized from patterns the model has learned, not captured by a camera or microphone in that moment.

How a face-swap video is put together

Face swapping is one of the two most common deepfake video techniques, according to the GAO. In outline, the process looks like this:

1. Collect examples. The maker gathers images or video of the person whose face will be inserted, and of the person (or footage) that will host that face. Public talks, interviews and social posts are a frequent source.
2. Learn the face. A model is trained to recognize and reconstruct that face from many angles and expressions. The GAO notes that realistic results have generally needed hundreds or thousands of training images, which is one reason well-known people are common subjects. Tools have continued to improve since that 2020 assessment, but plentiful public footage still makes the work easier.
3. Generate new frames. For each frame of the host video, the model produces a new version of the inserted face that follows the host's head position and expression.
4. Blend the face back in. The new face is placed onto the original frame and blended at the edges, so the join at the hairline, jaw and ears is less obvious.
5. Add audio, if needed. A cloned voice, or a separate lip-sync step, is often paired with the video so the person appears to say new words.

Each of those steps can leave traces — a seam, a colour mismatch, a mouth that is sharper than the rest of the face — which is why a careful look still helps, even as the tools improve. Our guide on how to spot a deepfake video walks through those checks.

How expression and lip-sync fakes work

Not every deepfake replaces the face. The GAO describes facial manipulation as the other common video technique: the model changes what an existing face does, rather than whose face it is. The expressions of one person are imitated by another, or the mouth is re-animated so the person appears to say new words.

This is often combined with a voice clone. The picture stays largely the same — same room, same clothes, same identity — which can make these fakes harder to catch than a full face swap, because there is no new face to blend in. The giveaways, when they appear, tend to sit around the mouth, teeth, eyes and the match between the voice and the lips.

Fully generated images and video

Some fakes are not edits of real footage at all. The underlying technology can also synthesize faces, as the GAO notes, and generative AI systems can produce new images or video from a prompt. In that case, nothing in the scene was photographed in that moment. Fake profile pictures, invented events and entirely generated clips of a recognizable person fall into this group.

Face-swap stills sit in between: they start from a real photo and replace the face. The background, body and clothes are real, which is why they can look more natural at a glance. For the practical checks that catch many of both kinds, see our guide on how to tell if an image is AI-generated.

How a voice clone is made

A voice clone is a model that has learned the sound of a specific person's speech — tone, pace, accent and the small habits that make a voice feel familiar — and can then produce new sentences that person never said.

The FBI has warned that criminals use AI-generated audio to impersonate public figures or relatives in order to request payments, and that cloned voice clips have been used in attempts to access bank accounts. In a December 2024 advisory, it noted that criminals generate short audio clips of a loved one's voice to impersonate a relative in a crisis (FBI IC3, December 2024). Public videos, podcasts, social clips and voicemail greetings can all be used as source material.

Because a good clone can sound nearly identical to the real person, you should not rely on the sound alone. Check the context, use a family code word, and hang up and call back on a number you already have. Our guide to voice cloning scams covers the warning signs and the steps that stop them.

Why deepfakes are getting harder to spot

The GAO's 2020 assessment already found that anyone with basic computer skills and a home computer could create a deepfake, that applications were openly available, and that the technology was advancing quickly alongside more powerful computing. In a later spotlight, it reported that current detection methods may not accurately identify deepfakes in real-world conditions — for example when lighting, facial expressions, or video or audio quality differ from the data used to train a detector, or when the fake was made with a different method (GAO, March 2024).

There is also a "cat and mouse" pattern: techniques used to identify deepfakes tend to lead to more sophisticated generation methods, so detectors have to be updated to keep pace. Future advances, the GAO notes, are expected to eliminate some older hallmarks, such as abnormal eye blinking. A clean visual check is useful, but it is not proof that a clip is real.

How detection tools fit in

Detection tools use machine learning, trained on known real and fake media, to flag a file without needing the original. The GAO describes methods that look for facial or vocal inconsistencies, evidence of the generation process, or colour abnormalities, and earlier work that analysed videos for digital artifacts or details deepfakes fail to imitate realistically, such as blinking or facial tics.

Results are probabilities, not proof. Accuracy can drop with heavy compression, low resolution, small or turned-away faces, screenshots and screen recordings, or when the fake was made in a way the detector has not seen. The GAO also cautions that even identifying a deepfake may not stop the harm, because disinformation can spread from the moment the file is viewed. Use a detector as one signal, alongside the source, the context and, where you can, a check with the person shown.

Authentication methods such as digital watermarks, secure metadata and Content Credentials can help when they are present, by recording how a file was made or whether it has been altered. Most images and clips online have no such credentials, and social platforms often strip metadata on upload, so their absence tells you very little.

How to check a file you are unsure about

Start with the context: who posted it, whether trusted sources show the same thing, and whether it is pushing you to act in a hurry. Then look or listen for the signs in our guides, and run a detection check on the best copy of the file you have.

DeepWall's free deepfake detector puts the image, video and audio checkers on one page, with no sign-up:

• Images: JPG, PNG or WEBP up to 4 MB, on the deepfake image detection page or the image tab. You get two scores, from 0 to 100%, for AI generation and for face swapping, plus the AI generator the image most likely came from when that is available.
• Video: MP4, MOV or WEBM up to 30 seconds and 50 MB, on the deepfake video detection page or the video tab.
• Audio: MP3, WAV, M4A, OGG or FLAC up to 25 MB, on the deepfake audio detection page or the audio tab.

Daily limits apply per visitor: 5 image checks and 2 video checks. Images are analysed in memory and not saved; video and audio files are held only while they are processed and then deleted. Treat the score as one piece of evidence, not a verdict.

Finding a fake you have not seen yet

A one-off check only helps with a file you already have. If the risk is a fake video of you or your leadership team appearing somewhere you are not looking, you need something that watches new uploads for you.

DeepWall's deepfake detection software is built for that job. You enroll a person with 3 to 5 photos and a 30-second voice sample. DeepWall then checks newly uploaded videos on YouTube, X, TikTok, Facebook, Instagram, Reddit and news sites with embedded video, matching faces frame by frame and checking the audio of those videos for a cloned voice. When a video crosses your match threshold, you get an alert, typically within minutes of detection. Professional and Executive plans add evidence export and takedown templates for reporting the video to the platform; removal decisions are made by each platform.

Plans start at $49 per month, and every plan includes a 30-day free trial with no credit card required. Start your free trial, or try the free detector first.

Frequently asked questions

How do deepfakes work, in one sentence?

An AI model learns the patterns of a real person's face or voice from examples, then generates new video, images or speech that look or sound like that person saying or doing something they never did.

Do you need thousands of photos to make a deepfake?

The GAO reported in 2020 that a somewhat realistic deepfake generally still needed hundreds or thousands of training images of the faces being swapped or manipulated. Tools have continued to improve, and the FBI has noted that short audio clips and public videos can be enough to start a voice clone. More public footage still makes the work easier.

What is the difference between an autoencoder and a GAN?

An autoencoder learns to compress a face or other input into a simpler representation and rebuild it. A GAN uses two networks that compete — one makes fakes, the other tries to spot them — which the GAO says generally produces more convincing results but is harder to use.

Can deepfakes always be detected?

No. The GAO reports that current detection methods may not be accurate in real-world conditions, and that generation methods keep adapting to detectors. Use a detector together with source checks, a close look at the file, and confirmation through a trusted channel.

How can I check whether a video, image or voice note is a deepfake?

Check the source first, then run the file through a detector. DeepWall's free deepfake detector accepts image, video and audio uploads with no sign-up. For step-by-step visual and audio checks, see our guides on spotting a deepfake video, telling if an image is AI-generated, and voice cloning scams.