1 of 25

Neural Motion Compression�with Frequency-adaptive Fourier Feature Network

Kenji Tojo, Yifei Chen, Nobuyuki Umetani

The University of Tokyo

1

Reims 2022

2 of 25

Introduction

2

Video credit: “Make your first GAME CHARACTER in 1 day!” Game Dev Academy licensed under CC BY, https://www.youtube.com/watch?v=7LgQKFOLzlM

state machine

Stand

Jump

Run

A button

↑ button

↓button

B button

B button

Reims 2022

3 of 25

Introduction

3

Image credit: Cygon4 @ Unity Answers

Compression

….

Example of state machine

Reims 2022

4 of 25

Related Work: Motion Compression

☹ hand-crafted algorithms to exploit smoothness & redundancy

4

Bezier curves

Wavelets

Pattern indexing

[Beaudoin et al. 2007]

[Arikan 2006]

[Gu et al. 2009]

Reims 2022

5 of 25

Related Work: Implicit Neural Representation

  • NN learns a mapping from input coordinates to the data
  • ☺ High quality compression for general data

5

[Martel et al. 2021]

[Tancik et al. 2020]

[Mildenhall et al. 2020]

Reims 2022

6 of 25

Our Approach: Neural Motion Compression

6

Compression

Reims 2022

7 of 25

Pose Parameterization

7

 

 

 

 

 

 

 

Reims 2022

8 of 25

Implicit Motion Representation

8

 

Time

Pose

 

 

 

 

 

 

 

Reims 2022

9 of 25

Neural Motion Representation

9

 

 

 

 

 

 

 

 

Time

Pose

Reims 2022

10 of 25

Neural Motion Representation

10

 

# NN params

#

 

Reims 2022

11 of 25

Challenge

  • Hard to reconstruct complex motion from linear time input

11

Time

1D, linear

 

 

High-dimensional, non-linear

….

Head

Elbow

Hand

?

Reims 2022

12 of 25

Background: Fourier Feature (FF)

12

[Tancik et al. 2020]

 

 

 

2D coordinates

 

 

RGB image

Reims 2022

13 of 25

Background: Fourier Feature (FF)

13

 

[Tancik et al. 2020]

 

 

2D coordinates

 

 

RGB image

Fourier feature

 

 

 

 

 

….

….

….

encode

frequencies

Reims 2022

14 of 25

Motion Representation using FF

14

 

 

1D time

 

Animation

Fourier feature

 

 

 

 

 

….

….

encode

 

 

linear component

Reims 2022

15 of 25

Motion Representation using FF

15

 

 

 

 

 

 

NN

layer

 

 

Reims 2022

16 of 25

Frequency-adaptive FF

16

 

Stochastic sampling

[Tancik et al. 2020]

 

Adaptive selection

during training (Ours)

 

Reims 2022

17 of 25

Adaptive Frequency Selection

17

Discrete Cosine Transform

DCT

 

Trained

 

 

GT

 

Residual

 

 

 

 

 

 

 

 

 

 

 

 

 

 

….

 

 

 

 

 

 

 

Reims 2022

18 of 25

Compression Performance

  • Compressed 116 animation sequences from the CMU MoCap database

​

18

*Error is reported in maximum difference / character height

better

better

Reims 2022

19 of 25

Comparison with Baseline Methods

19

SVD:

 

Naïve frequencies:

 

better

better

Reims 2022

20 of 25

Visual Quality of Reconstruction

20

Red: original

Blue: 8x compressed

Reims 2022

21 of 25

Visual Quality of Reconstruction

21

Red: original

Blue: 8x compressed

Reims 2022

22 of 25

Summary

  • Motion compression using a neural implicit representation

​

  • Adaptive selection of Fourier feature frequencies based on iterative training

​

  • Compressed motion sequences with small error, and outperformed baseline methods, including naïve choice of frequencies

22

Reims 2022

23 of 25

Future Work

  • Rigorous comparison to existing motion compression approaches

​

  • Learning common motion styles (e.g., walking and running) from large dataset for better compression

​

​

​

23

Reims 2022

24 of 25

Code is Available

24

https://github.com/cgenglab/ffmc

Reims 2022

25 of 25

Thank you!

25

Reims 2022