1 of 8

Towards Fair Face Verification: An In-depth Analysis of Demographic Biases

Ioannis Sarridis

​

Media Analysis, Verification and Retrieval group (MeVer)

Information Technologies Institute (ITI)�Centre for Research and Technology Hellas (CERTH)

Ioannis Sarridis, Christos Koutlis, Symeon Papadopoulos, Christos Diou

2 of 8

Bias in Face Recognition

https://www.aclu.org/blog/privacy-technology/surveillance-technologies/amazons-face-recognition-falsely-matched-28

3 of 8

Contributions

  • Broadening the scope of bias assessment in face verification systems to include race, gender, age, and their intersections.
  • Employing mAP, TPR, FPR, disparate impact, and disparate mistreatment metrics to assess bias.
  • Providing t-SNE visualizations and inter/intra-group similarity statistics across demographics.
  • Enriching the Racial Faces in the Wild (RFW) benchmark with annotations for gender and age.

4 of 8

Experimental setup

  • Training Data: BalancedFace-4 contains approximately 1.3 million images from 28,000 celebrities, with the images representing an approximate balance with respect to skin tones
  • Training Algorithm: ArcFace
  • Evaluation Benchmark: Racial Faces in the Wild (RFW) + gender and age annotations through the Amazon Rekognition service
  • Evaluation Metrics: accuracy, TPR, FPR, mAP, DFPR, DFNR, p-rule

​

5 of 8

Results

Overall and Race-Gender intersection performance on RFW

Reduced performance among female Asians.

Accuracy vs TPR:

TPR uncovers hidden biases

6 of 8

Results

Race-Age intersection performance on RFW

  • White Race:
    • Consistent performance
  • Other Races:
    • Large discrepancies
    • The youngest and oldest age groups often exhibit diminished performance

​

7 of 8

T-SNE visualization

Compact groups tend to demonstrate reduced performance

8 of 8

Thank you!

Ioannis Sarridis

gsarridis@iti.gr

https://mever.gr / @meverteam

​