1 of 14

IMPLICIT WARPING FOR ANIMATION WITH IMAGE SETS

  • Authors: Arun Mallya, Ting-Chun Wang, Ming-Yu Liu
  • Date: 2023/05/06

2 of 14

Main goal

Generate a video from a source image by using the motion from a driven video.

​

  • Faces
  • Pos / Gesture

​

​

​

​

​

​

​

​

From left to right: Driving video | FOMM| vid2vid | Their model

From left to right: Driving video | Their model(single)| FOMM | Their model (multiple images)

​

3 of 14

Previous works

 

From left to right: Driving video | Their model(single)| FOMM | Their model (multiple images)

​

4 of 14

Problem

 

5 of 14

Previous works (FOMM & Vid2Vid)

​

​

​

​

​

​

6 of 14

Main contribution

  • Multiple source image
  • Attention mechanism

​

7 of 14

Key point detector

  • Unsupervised

​

8 of 14

​

​

​

​

​

Spatial key point representation

9 of 14

​

​

​

​

​

Image to Feature & Feature to Image

10 of 14

Attention

11 of 14

Datasets

  • TalkingHead-1KH
  • VoxCeleb2
  • TED Talk

​

12 of 14

Train

 

13 of 14

References

[1] First Order Motion Model for Image Animation

[2] One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing

[3] Motion Representations for Articulated Animation

[4] Implicit Warping for Animation with Image Sets

​

​

​

​

14 of 14

THANK YOU