1 of 10

Deepfake Detection

Team members:

Sayantani Bhattacharjee (sab301@pitt.edu)

2 of 10

Introduction and Goal

Deepfake techniques, which present realistic AI-generated videos of people doing and saying fictional things, have the potential to have a significant impact on how people determine the legitimacy of information presented online. These content generation and modification technologies may affect the quality of public discourse and the safeguarding of human rights—especially given that deepfakes may be used maliciously as a source of misinformation, manipulation, harassment, and persuasion.

The Goal:

Accurately classify videos as Real or Fake by detecting modifications in the video. We extract faces of the person/persons in the videos and detect the changes in them by training on the training data and its corresponding metadata.

3 of 10

Major Components

Face detection and Tracking
Frame extraction from videos
Extraction of Random frame from each video
Training data using Image classification and metadata information
Cross-validation
Testing the data

4 of 10

Input

Provided by Kaggle: https://www.kaggle.com/c/deepfake-detection-challenge/data

The dataset size is 10GB for training and 1.74 GB for cross validation

These are several small videos in mp4 format.

Give below are two examples of the types of videos we have our dataset

Real Video	Fake Video

5 of 10

Data Flow among components from input and output

Each video is sent through face detection and tracking. The frames of these face in each video is stored in folder that is of the same name of the video. This is done on the Pitt CRC cluster.
A random image is extracted from each of the above mentioned folders and stored with the same name as the original video name for easy corroboration with the metadata file.

Extracted face from real video:��
Extracted face from fake video:

6 of 10

Data Flow among components from input and output

The extracted faces are then trained against pretrained ResNet model (ResNet34) with BCELoss function and Adam optimizer.
This trained model is then used to classify the test data that is given. These are done on the local machine.

��

7 of 10

Model Design

8 of 10

Updated Model Design

9 of 10

Output

Best val Acc: 0.810000

Train

Validation

10 of 10

References