Pasquale Lisena

EURECOM, Sophia Antipolis, France

pasquale.lisena@eurecom.fr

ANTRACT

FaceRec:

an AI-based System for Face Recognition in Video Archives

DataTV2020

How does FaceRec work?

CRAWL THE WEB

1

TRAIN A CLASSIFIER

2

RECOGNISE THE FACES

3

CLUSTER RECOGNITIONS

4

ON TRAINING IMAGES ON VIDEOS

https://github.com/hellock/icrawler

https://github.com/ipazc/mtcnn

icrawler

mtcnn

face detection and

alignment

​

download

CRAWL THE WEB

1

Facenet: https://arxiv.org/abs/1503.03832

facenet

embedding

[-0.24 0.31 0.45 … 0.60]

SVM

training

CLASSIFIER

TRAIN A CLASSIFIER

2

combination of N One-vs-All classifiers

CLASSIFIER

frame

mtcnn

face detection and

alignment

​

facenet

embedding

SORT

tracking

RECOGNISE THE FACES

3

CLASSIFIER

SORT

prediction

tracking id

For each tracking id:

• Selection of the most represented prediction (weighted by confidence)
• Discard uncertain trackings
• Merge consecutive trackings assigned to the same person
• Filter by average confidence

CLUSTER RECOGNITIONS

4

Some results

ANTRACT Ground Truth

• Realised in the context of ANTRACT project, with video material from 1950s
• 200 shots of different length, containing 0, 1 or more among 13 celebrities

Glasses or other occlusion

Side of the face

Small resolution

Correct matches: 93 (52.54%)

Unpredicted people: 83 (46.89%)

Wrong people: 1 (0.56%)

Common error cases

How to interact with Facerec?

http://facerec.eurecom.fr/

http://facerec.eurecom.fr/visualizer/

https://github.com/D2KLab/Face-Celebrity-Recognition