1 of 12

DeepFake Challenge

​

Simulated participation in Kaggle DeepFake Detection Challenge

​

W251 Deep Learning in the Cloud and at the Edge (Spring 2020) Final Project

​

Michael Zeng, Richard Ryu, Adam Sohn

​

2 of 12

What is DeepFake?

Amy Adams / Nicholas Cage

GAN w/o fake

Deepfake generating GAN

3 of 12

What is DeepFake Detection Challenge?

  • Training Set - (50) 10 Gb files in a Kaggle GCS bucket
  • Public Validation Set - 400 videos/ids for tests/Kaggle submission

​

4 of 12

EDA

Example Fake

Too dark to make out face

Solution: Augmentation to brighten

Strategy: By image - Normalize on same-image Max([R,G,B]) or use minimum color value threshold

Facial Keypoint Detection

5 of 12

Our Approach to Deepake Detection

GAN w/o fake

Deepfake generating GAN

LSTM Discriminator

Still images from video

Encoder

Latent Faces

REAL

FAKE

Decoder

6 of 12

Pipeline

7 of 12

Data

8 of 12

Model Architecture

9 of 12

Model Training/Validation

10 of 12

Results

One Face

Two Face

Three Face

# of faces

Test Data Points

Log Loss

Accuracy

Precision

Recall

1

~100k

0.32

0.75

0.75

0.94

2

~8.6k

0.60

0.58

0.58

0.89

3

292

0.62

0.49

0.59

0.86

Overall 0.34

Reference:

​

​

Accuracy = TP + TN

TP + TN + FP + FN

​

Precision = TP

TP + FP

​

Recall = TP

TP + FN

​

11 of 12

Kaggle Results

# of faces

Test Data Points

Log Loss

Accuracy

Precision

Recall

1

~100k

0.32

0.75

0.75

0.94

2

~8.6k

0.60

0.58

0.58

0.89

3

292

0.62

0.49

0.59

0.86

Overall 0.34

12 of 12

Future Improvement Paths

References

​

​

  • Sample additional frames (~300 frames available per video)

​

  • Attempt other model types & hyperparameter sets

​

  • Cultivating better training data set:
    • Augmentations to increase image count
    • Make similar distribution of fake vs. real training videos

​

  • Improve facial recognition
    • Filter out incorrect faces (ie. printed on shirt)
      • Torso + head logic

​

  • Tackle out-of-memory issues performing inference on Kaggle submission.
    • Reduce batch size
    • Increase cache size

https://github.com/zengm71/DeepFakeChallenge

https://www.kaggle.com/c/deepfake-detection-challenge/overview