Deepfake Detection System for Facial Evidence Verification in Criminal Justice and its Legal and Ethical Implications

Paper ID: 129

23^RD International Conference on Intelligent Systems Design and Applications

Olten, Switzerland on 11 December 2023

Ebrima Hydara, Masato Kikuchi, Tadachika Ozono

Contact Email: ebrima@ozlab.org

Nagoya Institute of Technology, Japan

1

Introduction

• Deepfake Technology Overview:
• Utilizes AI and ML algorithms, including generative adversarial networks (GANs) and deep neural networks (DNNs).
• Creates realistic manipulated media, such as videos and images, by synthesizing facial features onto existing footage or images.
• Importance of Facial Evidence in Criminal Justice:
• Facial evidence is crucial for criminal justice systems globally.
• Supports criminal investigations, legal proceedings, and determination of guilt or innocence.
• Used for identifying suspects through surveillance footage, eyewitness testimonies, and photographic evidence.

​

2

Introduction

• Objective:
• To addresses challenges posed by deepfakes in criminal investigations
• To introduce a deepfake facial evidence verification system for forensic analysts
• Identified Gaps in Current Research:
• Existing methods lack forensic analysis techniques relevant to forensic investigators
• Proposed system introduces confidence threshold, prediction timestamps, and prediction heatmaps for individual frames
• Lack of Deepfake Datasets specific to criminal justice
• Key Components of Proposed System:
• Confidence Threshold: Set to enhance confidence in prediction results.
• Prediction Timestamps: Display decision precision allocated to each frame per second in a video.
• Prediction Heatmaps: Indicate regions susceptible to manipulation, with red being more suspicious than yellow, green, and blue.

​

3

Related Work

• Current Facial Evidence Verification Techniques:
• Face Matching: most used technique for identifying crime suspects by comparing facial traits between evidence images and law enforcement image databases [1, 2]
• Facial Recognition Technology (FRT): Hill, D. et al. (2022) highlighted FRT as a key technology using face matching as one of its techniques [3]
• Automated Facial Recognition Systems (AFRS): Khan, Z. A. et al. (2021) reported AFRS involvement in suspect identification using facial traits comparison techniques [4]
• Limitations of Current Facial Evidence Verification Techniques with Regards to Deepfake Evidence:
• Current deepfakes are highly realistic in nature, posing a challenge to current evidence verification methods using face matching technique
• Lack of exposure of current methods to current deepfake datasets, limiting current facial evidence verification methods’ effectiveness to address the deepfake evidence treat [5]

4

Implementation Method

• Dataset Collection:
• 1000 sample videos from the combination of Celeb-DF [6], FaceForensics++ [7], and Deepfake Detection Challenge (DFDC) [8] datasets
• 15,000 frames extracted from the dataset where 50% are real and 50% are fake
• Data Standardization, Feature Extraction, and Representation:
• Model Choice: Vision Transformer (ViT) pretrained model which outperformed ResNet and EfficientNet on different datasets according to Dosovitskiy, A. et al. [9]
• Frames Augmentation, Resizing, and Transformation: all frames were resized to 256 pixels and transformed and augmented to conform with the ViT-specific transformer
• Training and Evaluation: pretrained ViT was finetuned and trained on 80% of the dataset for 10 epochs with a learning rate of 0.001 and was evaluated on 20% of the dataset

5

System Prototype

6

System’s video prediction at 80% confidence threshold

Experiment

• The system’s performance was evaluated based on accuracy, precision, recall, and F1 score. A total of 102 video frames of the Real (42 frames) and the Fake (60 frames) classes were evaluated at each instance of the following confidence thresholds: 50, 55, 60, 65, 70, 75, 80, 85, and 90.
• The record of prediction results at each confidence threshold was used to calculate the accuracy, precision, recall, and F1 score of the returned number of frames.

7

Results & Discussion

8

Legal Implications & Ethical Considerations of the Proposed System

• Legal Standards for Deepfake Detection Systems:
• Acceptance of such systems may vary across jurisdictions
• Users may doubt results due to perceived black-box nature of deep learning models
• Ethical System Usage:
• There are ethical concerns about complete reliance on the system without further investigations
• System is intended as a support tool and not a standalone solution

9

Conclusions

• The study aimed to tackle challenges posed by deepfake technology in criminal investigations, conducting a comprehensive exploration of deepfake evolution and existing detection methodologies. �
• It also discussed the significance of facial evidence in criminal justice and identified limitations in current verification systems.

​

• It proposed a deepfake detection system for facial evidence verification integrating forensic analysis techniques relevant to criminal justice. �
• The proposed system contributed to the field by providing a comprehensive deep learning-based solution with forensic-oriented techniques, confidence-boosting mechanisms, and explainability techniques.

10

References

1. Bacci, N. et al.: Forensic Facial Comparison: Current Status, Limitations, and Fu- ture Directions. Biology, vol. 10, no. 12, 1269, pp. 1–26 (2021).
2. Moreton, R.: Forensic face matching: Procedures and application. In M. Bindemann (Ed.), Forensic face matching: Research and practice, pp. 144–173 (2021).
3. Hill, D., Oconnor, C., Slane, A.: Police use of facial recognition technology: The po- tential for engaging the public through co-constructed policy-making. International Journal of Police Science and Management, vol. 24, no. 3, pp. 325–335 (2022).
4. Khan, Z. A., Rizvi, A.: AI Based Facial Recognition Technology and Criminal Jus- tice: Issues and Challenges. Turkish Journal of Computer and Mathematics Educa- tion, vol. 12, no. 14, pp. 3384–3392 (2021).
5. Hu, J., Wang, S., Li, X.: Improving the Generalization Ability of Deepfake Detec- tion via Disentangled Representation Learning. In: 2021 IEEE International Con- ference on Image Processing (ICIP), pp. 3577–3581 (2021).
6. Li, Y. et al.: Celeb-DF: a large-scale challenging dataset for deepfake forensics. IEEE Transactions on Multimedia, vol. 22, pp. 2981–2993 (2020).
7. Rossler,A.,Verdoliva,L.,Riess,C.,Thies,J.,Niesner,M.:FaceForensics++:learn- ing to detect manipulated facial images. In: 2019 IEEE/CVF International Confer- ence on Computer Vision (ICCV), pp. 1–11 (2019).
8. Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., Ferrer, C. C.: The deepfake detection challenge (DFDC) dataset. (2020).
9. Dosovitskiy, A. et al.: An Image is Worth 16x16 Words: Transformers for Im- age Recognition at Scale. In: International Conference on Learning Representations (2021).

​

​

​

​

11

Thank You

12

Q & A

• Question I was asked: How does your system classify a video as fake or as real?
• Answer Then: The system predicts a video’s veracity based on its individual video frames’ prediction results, where the video is assigned the class with the highest number of returned frames at a given confidence threshold.
• Best response I can think now: the answer provided above.

13