Interactive Character Control with
Auto-Regressive Motion Diffusion Models
Yi Shi 1, 2 Jingbo Wang 2 Xuekun Jiang 2 Bingkun Lin 3
Bo Dai 2 Xue Bin Peng 1,4
Vi
1
2
4
3
© 2024 SIGGRAPH. ALL RIGHTS RESERVED.
Related Work
Human Motion Diffusion Model (MDM)
[Tevet et al. 2022]
Space-Time Models
Space-Time
Model
Noise
Related Work
Auto-Regressive Models
MotionVAE
[Hung et al. 2021]
Auto-Regressive Model
Auto-Regressive Motion Diffusion Model
Random Sampling
A-MDM:
Hierarchical Reinforcement Learning
Conditional Inpainting
Overview
Stage I : Base A-MDM Training
Stage II : Task-based Control
AMDM Base Model
A-MDM
AMDM Base Model: Training
Denoiser
Loss
Step
AMDM Base Model: Runtime
Denoiser
Step
T
x
A-MDM Denoising
A-MDM Denoising
MLP based Network Architecture
A-MDM
MLP
Denoising Steps
Denoising Steps
Random Synthesis - LAFAN1
Random Synthesis - LAFAN1
Random Synthesis - HumanML3D
Random Synthesis - HumanML3D
Overview
Stage I : Base A-MDM Training
Stage II : Task-based Control
Conditional Inpainting
Spatial Inpainting
A-MDM
Denoiser
Denoiser
Denoiser
User Command
Spatial Inpainting
Keyframe Motion In-betweening
Source Motion
Target Motion
Keyframe Motion In-betweening
Time
(Frames)
Denoising Step
Source Motion
Target Motion
Source Motion
Keyframe Motion In-betweening
Overview
Stage I : Base A-MDM Training
Hierarchical RL
Stage II : Task-base Control
Hierarchical Reinforcement Learning
Action
A-MDM
Controller Policy
Hierarchical Reinforcement Learning
A-MDM
Controller Policy
Action
Denoiser
Denoiser
Denoiser
Hierarchical Reinforcement Learning
Hierarchical Reinforcement Learning
Hierarchical Reinforcement Learning