1 of 72

Physically based animation

or “why I quit my job to play with Boxie for three years”

Danny Chapman (rowlhouse@gmail.com) - Oxford Video Games meetup - 31 Jan 2024

2 of 72

“Ragdoll physics” in games

3 of 72

NaturalMotion: Morpheme + Physics + Euphoria

2007-2017

4 of 72

Character physics in Unreal Engine

2020-

5 of 72

“Ragdoll physics” in games

“Ragdoll physics” in films/VFX

6 of 72

2017-2020

7 of 72

Here we have... Monte! He’s the star in a demo I wrote to prototype a completely different way to animate a physical character.

He’s fully simulated - just like a character simulated in game, or with WeightShift in a film.

He has muscles, and that’s how he moves. I’m throwing 100kg balls at him. This is a real-time capture (in a 13 year old PC!).

You might think this guy is programmed to avoid the ball? But he’s not. In fact he’s just rewarded for maintaining that standing pose, and everything else he does “emerges” from that.

Imagine you were offered £10 for every second you were able to just stand upright above a mark on the ground, as I throw things at you. You’d duck and jump, whatever you have to do to keep that money meter filling up!

To have the computer do this without any coding specific to standing, dodging etc - it felt like finding buried treasure... and I just knew I had to make something of this.

So I got it running on Saturday, after a week or two coding it up in my spare time...

8 of 72

9 of 72

Marcus Ottosson

10 of 72

WeightShift is born!

Danny Chapman

Physics Programmer

C++, Engine, Simulation

Marcus Ottosson

Technical animator

Tools, Python, Maya

Tim Daoust

Application developer

C++, Python, Tools

11 of 72

An animator working at a company like Pixar is expected to produce around one second of final quality animation per day.

Our mission is to double that.

12 of 72

13 of 72

14 of 72

15 of 72

16 of 72

17 of 72

Using WeightShift AI

18 of 72

19 of 72

How it works

20 of 72

WeightShift Solver

Scene description

Animation Input

Evaluators

Controls

Animation Output

21 of 72

WeightShift Solver

Scene description

Animation input

Evaluators

Controls

Animation Output & Optimal Control Plan

Control Plans

Simulation

22 of 72

What’s a Control Plan?

23 of 72

WeightShift Solver

Finds the optimal control plan. I.e. one that minimises the total cost.
The total cost is determined by the Evaluators working on the simulation output.

That’s a complicated “function”
We need to simulate the scene using a Control Plan to find out what its cost is

24 of 72

25 of 72

Canon controls are: angle a and speed s

Impact is distance d from the target
Cost or distance d = f(scene, a, s)

What should a and s be to minimise the cost?

26 of 72

Cost of missing the target

Speed (s)

Angle (a)

Low cost

High cost

27 of 72

Optimise by sampling: CMA-ES

Angle (a)

Speed (s)

28 of 72

Now on Boxie

29 of 72


Input scene
Animation	None
Controls	Speed and Angle
Evaluators	Distance to target	Distance to animation target
Output

Need: Input scene, animation, controls, evaluators

30 of 72

Input Scene

31 of 72

Input/Target Animation

N/A

32 of 72

Controls

33 of 72

Evaluators

Cost = d²

Cost per frame = d1 + d2 + c1 + ...

34 of 72

Output Animation and Control Plan

35 of 72

36 of 72

For implementers

37 of 72

Trajectory Optimisation

Trajectory Optimisation is the process of designing a trajectory that minimises (or maximises) some measure of performance while satisfying a set of constraints.

Not just cannon/projectile trajectories!

Trajectory means the “evolution of the whole system”

Many ways to solve this - different combinations of fast/slow, complex/simple, robust/sensitive etc.
Likely to be slow to process

38 of 72

Model Predictive Control

Model predictive control (MPC) is an optimal control technique in which the calculated control actions minimize a cost function for a constrained dynamical system over a finite, receding, horizon.

Constantly calculating a short term plan for the next few seconds
Suited to robotics, dynamic characters in games...
Likely to be real-time

39 of 72

Model Predictive Control

Model predictive control (MPC) is an optimal control technique in which the calculated control actions minimize a cost function for a constrained dynamical system over a finite, receding, horizon.

Constantly calculating a short term plan for the next few seconds
Suited to robotics, dynamic characters in games...
Likely to be real-time

40 of 72

WeightShift AI - Examples

41 of 72

Motion generation

42 of 72

Meet Boxie! Simple procedural motion.

43 of 72

Fully procedural motion generation

44 of 72

Direct and indirect control

45 of 72

Motion tracking - changed environment

46 of 72

Motion transfer - to a new environment

47 of 72

Motion tracking - changed character

48 of 72

Run with limp (cost for using the left leg)

49 of 72

Everything - backpack, terrain

50 of 72

Miscellaneous!

51 of 72

52 of 72

53 of 72

54 of 72

55 of 72

Summary

56 of 72

If you have:

a well defined system...
...that can be controlled
a goal that you can express as a reward/cost...
no idea how to achieve that goal directly

Trajectory Optimisation or Model Predictive Control

might be your friend!

Even if they’re not, have fun trying!

57 of 72

Thanks to Epic for letting me show you Boxie!

58 of 72

More WeightShift Dynamics examples

59 of 72

60 of 72

61 of 72

62 of 72

63 of 72

64 of 72

65 of 72

More WeightShift AI Examples

66 of 72

67 of 72

68 of 72

69 of 72

70 of 72

71 of 72

Properties of CMA-ES

Pro:

Exploration and refinement
Robust
Few configuration parameters
Parallel
Implementation is easy
Quality scales with iterations/samples
No training: can be used when you have no final quality animation

72 of 72

Properties of CMA-ES

Cons:

Slow to converge when there are many parameters

Not “inifinitely parallelisable”
1 or 2 hours for the most complex scenes

Doesn’t learn.

If anything changes in the scene/setup - needs to run again.