1 of 34

Trajectory Generation, Control, and Safety with �Denoising Diffusion Probabilistic Models

Nicolò Botteghi*, Federico Califano, Mannes Poel, and Christoph Brune

University of Twente – The Netherlands

SIAM UQ 27 February 2024

2 of 34

Learning and Controlling Dynamical Systems from Data

2

symmetries in nature

micro-macro

bio-inspired robots

human-machine interaction

3 of 34

Learning and Controlling Dynamical Systems from Data

Data are indirect, high-dimensional, and noisy measurements of the systems.

Challenges:

Uncertainty quantification

Imperfect sensing
Chaotic behavior

Dimensionality reduction

High-dimensional systems (e.g., in states or sensing)
Simulation accuracy vs computational complexity

Real-world learning

Bridging the gap between simulations and the real world

3

Learning the evolution of the systems from indirect measurement data however has several challenges! The two critical aspects we tackled in our research are high-dimensionality of the measurements over space and time and they are corrupted by noise.

In the context of learning dynamics from data, in general we need to three steps to properly learns models:

-First reduce the dimensionality of the data and infer the low dimensional set of latent state variables governing the evolution of the systems,

-At the same time properly quantify the uncertainties deriving from the measurement noise and the dynamics of the system itself (e.g. double pendulum).

- Learning using real-world data

The key message is that without properly quantify uncertainties for example whether they come from noise on the data or the dynamics of the systems itself, we cannot properly learn latent state variables and latent dynamical models from data.

In many scenarios, we do not only want to predict the autonomous evolution of the systems from measurements, but we also want to control it.

A way to do control from data is to use reinforcement learning…you can think of it as approximate optimal control

4 of 34

Offline and Safety-critical Control from Data

Our goals are to:

learn the dynamical system from data,
control the system toward desired targets, and
avoid unsafe regions of the state space

We can solve these three challenging problems with a single deep learning-based generative method:

the Denoising Diffusion Probabilistic Model (DDPM)

Botteghi, N., Califano, F., Poel, M. and Brune, C., 2023, July. Trajectory Generation, Control, and Safety with Denoising Diffusion Probabilistic Models. In ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems.

4

5 of 34

Generative Models

Generative models are models used to generate data 🡪 learning data distributions from samples

The main classes of generative models are:

Generative adversarial networks
Variational autoencoders
Flow-based models
Denoising diffusion probabilistic models

5

6 of 34

Denoising Diffusion Probabilistic Models

Ho, J., Jain, A. and Abbeel, P., 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33, pp.6840-6851.

Nichol, A.Q. and Dhariwal, P., 2021, July. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning (pp. 8162-8171). PMLR.

6

7 of 34

Denoising Diffusion Probabilistic Models

Nichol, A.Q. and Dhariwal, P., 2021, July. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning (pp. 8162-8171). PMLR.

7

simplification

8 of 34

Denoising Diffusion Probabilistic Models

Nichol, A.Q. and Dhariwal, P., 2021, July. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning (pp. 8162-8171). PMLR.

8

9 of 34

Unconditional Generation from Random Noise

9

K diffusion steps

10 of 34

Conditional Generation from Random Noise

10

K diffusion steps

11 of 34

Trajectory Generation, Control, and Safety with DDPMs

11

Botteghi, N., Califano, F., Poel, M. and Brune, C., 2023, July. Trajectory Generation, Control, and Safety with Denoising Diffusion Probabilistic Models. In ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems.

12 of 34

Trajectory Generation with DDPMs

12

Botteghi, N., Califano, F., Poel, M. and Brune, C., 2023, July. Trajectory Generation, Control, and Safety with Denoising Diffusion Probabilistic Models. In ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems.

13 of 34

Trajectory Generation

13

time

Data

14 of 34

Trajectory Generation

14

time

Data

15 of 34

Trajectory Generation

15

time

Data

16 of 34

Trajectory Generation

16

time

Data

After training, we can query the model multiple times to make prediction of the system’ evolution

17 of 34

Trajectory Generation

17

time

Data

After training, we can query the model multiple times to make prediction of the system’ evolution.

However, we accumulate more and more error for each prediction step!!!

18 of 34

Trajectory Generation with DDPMs

18

K diffusion steps

19 of 34

Trajectory Generation with DDPMs

19

True

Predicted

20 of 34

Optimal Control as Conditional Generation

20

21 of 34

Optimal Control as Conditional Generation

Janner, M., Du, Y., Tenenbaum, J.B. and Levine, S., 2022. Planning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205.09991.

21

22 of 34

Optimal Control as Conditional Generation

Janner, M., Du, Y., Tenenbaum, J.B. and Levine, S., 2022. Planning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205.09991.

22

23 of 34

Optimal Control as Conditional Generation

23

time

24 of 34

Optimal and Safe Control as Joint Conditional Generation

Botteghi, N., Califano, F., Poel, M. and Brune, C., 2023, July. Trajectory Generation, Control, and Safety with Denoising Diffusion Probabilistic Models. In ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems.

Ames, A.D., Coogan, S., Egerstedt, M., Notomista, G., Sreenath, K. and Tabuada, P., 2019, June. Control barrier functions: Theory and applications. In 2019 18th European control conference (ECC) (pp. 3420-3431). IEEE.

24

25 of 34

Optimal and Safe Control as Joint Conditional Generation

Botteghi, N., Califano, F., Poel, M. and Brune, C., 2023, July. Trajectory Generation, Control, and Safety with Denoising Diffusion Probabilistic Models. In ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems.

Ames, A.D., Coogan, S., Egerstedt, M., Notomista, G., Sreenath, K. and Tabuada, P., 2019, June. Control barrier functions: Theory and applications. In 2019 18th European control conference (ECC) (pp. 3420-3431). IEEE.

25

26 of 34

Optimal and Safe Control as Joint Conditional Generation

26

27 of 34

Optimal and Safe Control as Joint Conditional Generation

27

time

28 of 34

Optimal and Safe Control as Joint Conditional Generation

Botteghi, N., Califano, F., Poel, M. and Brune, C., 2023, July. Trajectory Generation, Control, and Safety with Denoising Diffusion Probabilistic Models. In ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems.

28

29 of 34

Conclusion

Our goals are to:

learn the dynamical system from data,
control the system toward desired targets, and
avoid unsafe regions of the state space

Pros:

Low computational cost compared to optimal control (e.g. Model Predictive Control) after training
Modular architecture allowing for changes of task or safety conditions without retraining all the models
Trajectory-based generation with flexible prediction horizon for long- or short-terms predictions

Cons:

Data-hungry architecture prone to overfitting
Physical consistency

29

30 of 34

Thank you for your attention!!!

31 of 34

Training and Sampling DDPMs

31

32 of 34

Control Barrier Functions

Given a system of the type:

33 of 34

Inpainting

Fix a priori certain values of the output

33

34 of 34

Inpainting

Fix a priori certain values of the output to enforce initial conditions, constraints, etc…

34