1 of 22

Dissipative Hamiltonian Neural Networks

Andrew Sosanya

Sam Greydanus

Learning Dissipative and Conservative Dynamics Separately

2 of 22

Sam Greydanus

ML Collective, Oregon State

Formerly @GoogleBrain, @Dartmouth

Andrew Sosanya

Policy Analyst @ Day One Project

Formerly @Caltech & @Dartmouth

3 of 22

Conserved and Dissipative Quantities Are Everywhere

4 of 22

Physics + ML: A Burgeoning Field

5 of 22

6 of 22

Physics

+

Fluid Dynamics

+

ML

= Modeling Real Systems

7 of 22

Physics + Fluid Dynamics + ML = Modeling Real Systems

Helmholtz Decomposition

Hamiltonian Mechanics

8 of 22

Physics + Fluid Dynamics + ML = Modeling Real Systems

Helmholtz Decomposition

Hamiltonian Mechanics

9 of 22

Hamiltonian Mechanics

William Hamilton

Tenets of Hamiltonian Mechanics

Tool for modeling the time evolution of simple and complex dynamic systems
The Hamiltonian H is a scalar function such that Eq. 1 (below) is true.
Moving in the direction of symplectic gradient S_Hkeeps value of H constant.
Can only model conserved systems

Eq. 1

10 of 22

Hamiltonian Mechanics

William Hamilton

Tenets of Hamiltonian Mechanics

Tool for modeling the time evolution of simple and complex dynamic systems
The Hamiltonian H is a scalar function such that Eq. 1 (below) is true.
Moving in the direction of symplectic gradient S_Hkeeps value of H constant.
Can only model conserved systems

But real systems don’t truly conserve quantities...WTF?

11 of 22

Rayleigh Dissipation Function

Real dynamic systems often contain dissipative forces, like friction or radiation
Dissipation function allows for dynamic analysis

12 of 22

Helmholtz Decomposition

Hermann von Helmholtz

13 of 22

Physics + Fluid Dynamics + ML = Modeling Real Systems

Inputs

Dynamic system coordinates, like position and momentum

Outputs

Time derivatives of the system, which allows you to recover the full system

Helmholtz Decomposition

Hamiltonian Mechanics

14 of 22

Physics + Fluid Dynamics + ML = Modeling Real Systems

Helmholtz Decomposition

Hamiltonian Mechanics

15 of 22

Mass-Spring Experiment

16 of 22

Woah! Adaptive Friction Coefficients!

17 of 22

DHNNs vs HNNs & MLPs on a Real Damped Pendulum

18 of 22

Tested DHNNs on NASA Ocean Current Data

19 of 22

Tested it on Real Ocean Current Data

The ways of the ocean are com- plex. Oceanographers are typically interested in modeling ocean flow and identifying features such as eddy currents, upwelling and downwelling. (Zhang et al., 2019; Chelton et al., 2011; Petersen et al., 2013). Upwelling is a dissipative feature that occurs when cold, nutrient-rich waters rise to the surface of the ocean. Conversely, downwelling is an opposite process by which surface water descends. These features tend to occur in the centers of vortices, which form in order to preserve the geostrophic balance. Oceanogra- phers often describe current velocity fields by composing a divergence-free stream function ψ (conserved) and some curl-less velocity potential φ (dissipative) (Buhler et al., 2014; Caballero et al., 2020).

Our experiment showed that D-HNNs can separately model both ψ and φ—both unknown a priori—and uncover ocean flow features by performing an implicit Helmholtz decom- position across 69 time frames. From the bright and dark re- gions seen in the H’s learned scalar fields (Figure 5, we can identify eddies by locating the centers of vortices. To main- tain geostrophic balance in a vortex, water will move up- wards or downwards depending on the direction of the vor- tex. By identifying the centers of vortices, we can find where upwelling and downwelling features will be the strongest.

20 of 22

Summary Results

21 of 22

Open Questions

Are there better, more complex systems we can test this on? (needs good data)
What other physical priors can we embed to make the model smarter?
Do DHNNs work at scale? Does it work in higher dimensions? (theoretically yes)
Can we use DHNNs to upsample videos? (yes, but needs more experimenting)

Limitations & Challenges

Ocean dataset is not the best example to test it on (visually) since the magnitudes of the conservative and dissipative forces differ by magnitudes. Thus, we need more complex and exemplary systems to test it on.
We need to test our model against the new suite of HNN spin-offs that have emerged.
Getting the right real-world dataset is challenging because of time and space resolution.

22 of 22

Recap

There are conserved quantities in our world—mass, energy, momentum—that play an integral role in natural phenomena.
We can model these natural phenomena, or ‘systems’, with a neural network by embedding the NN with a physics prior.
However, previous work tends to focus on clean systems within the ‘ideal world’. But in the real world, data is often messy and noisy because dissipative quantities, like friction, are present in the real world.
Our DHNN model allows you to separate conservative quantities from the dissipative quantities ones. + other stuff

Extends Greydanus et al’s work on HNNs. Paper Link here!