Physically-Motivated Machine Learning Models for Lagrangian Fluid Mechanics
Rene Winchenbach, Nils Thuerey
Technische Universität München
Physics Based Simulations Group (not with Hu)
Berlin, SPHERIC 2024
What do we want to do?
Machine Learning is a quickly growing field of research in many areas
Mostly areas without good existing solutions
PDEs have been well studied for a long time
We have good existing solutions
Machine Learning could:
But what is good enough and how do we get there?
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
2
Graph Neural Networks
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
3
Why Basis Functions?
Physics usually requires some constraints to be fullfilled
Solutions should also be smooth and symmetric
With Basis functions we can directly encode constraints:
Lower Parameters for equal results
Computationally expensive but efficient to implement
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
4
Training Task
Machine Learning already difficult enough
We chose a simple scenario:
Learning Task:
Purely based on positions over time
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
5
Problem #1: The Task
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
6
Problem #1: The Problem
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
7
Problem #1: The Solution
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
8
Problem # 2
In traditional SPH particle ordering has significant influence on the numerical accuracy
This is even more apparent for neural simulations
Generalization in general is difficult for networks
Assume a set of weakly compressible simulations
Only some particle-particle distributions are seen
Especially no short distance interactions!
If a network makes mistakes…
… then it has to generalize to unseen cases…
… which leads to more mistakes…
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
9
Particle Shifting Techniques
Particle Shifting is a well used approach in SPH
PST restores particle order from a disordered state
In classical simulations this improves accuracy
In neural simulations this avoids generalization
PST can be done implicitly to achieve good results even from random particle distributions
However, the error from frame to frame is low, similar to classic SPH, so explicit shifting suffices
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
10
Example Network Behavior
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
11
The First Solution
Why not simply add Shifting to networks?
Similar to classic SPH, integration and shifting are treated seperately!
No need to learn, or even include it in training
Simple to implement and gives good results
However, this is not very elegant
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
12
The First Solution
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
13
The Second Solution
Instead of excluding shifting during training, we can include it!
This requires a differentiable shifting operator and parameter gradients to propagate through shifting
Delta-Plus based shifting can work for this
However, training becomes more expensive
And now the network relies on shifting.
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
14
The Second Solution
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
15
The Second Solution
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
16
Generalization of the networks
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
17
Without PST
With PST
Conclusion (slides at spheric2024.fluids.dev)
Machine Learning is a black-box
This does not mean that there are not best practices!
ML best practices make the learning task simpler
Engineering best practices make the network better
Both can and should work in harmony
Enforced vs Learned behavior is an open question
Physics Based Machine Learning | SPHERIC Workshop 2024 | Berlin
18