1 of 14

Sahil J, Sarthak K

2 of 14

Problems

  • Current techniques are too slow and an expensive process�
  • Hand-drawn sketches or animations are not representative of realistic faces�
  • Rely heavily on average witness’s imaginative abilities, making accuracy harder to achieve

3 of 14

Facial Composites

  • Traditionally drawn by sketch artists, it’s a graphical representation of one or more eyewitnesses' memory of a face�
  • Images are used to reconstruct the suspect's face in hope of identifying them.�
  • Modern systems use basic sketches to simulate this

4 of 14

immersive facial composite generation for suspect investigations -- uses deep learning to generate suspect images based on witness’s description

5 of 14

Some of the sketches imagined by Deep Dreamer:

Not real people -- fully generated by a generative adversarial network.

6 of 14

Architecture

  • StyleGAN: novel generative adversarial network�
  • Works by taking an arbitrary latent vector z, learns a fully connected mapping from z to an intermediary feature vector w�
  • Then learns a mapping from w to images of faces via convolutional layers in a synthesis network

7 of 14

Main Challenge

  • To map machine features to features humans can understand.�
  • Trained a Fully Connected Neural Network to learn a mapping from real features like race, ethnicity, age, hair color, to machine features (in the latent vector z)�
  • Took average of all faces’ latent vectors to generate the gradient to be added to allow witness to adjust features like masculinity, ethnicity and age

where Z is the latent representation

8 of 14

The Process

8

Show six randomized faces, and allow user to pick closest one or regenerate with a focused parameter like ethnicity, age, and accessories

​

​

Recursively generate variations of the selected face, until best face is chosen

9 of 14

Dataset used

​

9

  • Source: LFWA+ Dataset (fork of CelebA)
  • A Dataset of 73 image features with a total of 12,000 images

10 of 14

Results

See what our model produced

10

11 of 14

Python App

A deep generative network requires high CPU and GPU power, so we chose to build for PC

  • Integration with model on local level.
  • Ease of access, fast generation

11

12 of 14

Accomplishments

  • Allows investigators to create high resolutions and ultra-real images of suspects based on witness descriptions�
  • State of the art generative models indistinguishable from real faces�
  • Developed a detailed transformation from human features to machine features

13 of 14

Future Work

  • Integration with natural language processing models to record witness testimonials.

​

  • Reach out to police officials to attempt a trial of this system, extending its implications beyond the hackathon.

​

​

14 of 14

Thank You!

Questions?

14