1 of 1

Learning differentiable solvers for systems with hard constraints

Scientific Achievement

We develop a new method to enforce differential equation constraints into neural networks (NNs) with a high degree of accuracy. We show that our approach achieves significantly lower error compared to previous approaches, and converges much more quickly to the right solutions.

Significance and Impact

Partial differential equations (PDEs) are crucial for describing the complex phenomena of climate dynamics, and numerous other energy-related areas. NNs provide a way to approximate solutions to such systems much faster than numerical methods, but current approaches only enforce physical constraints approximately – we address this key problem through our method.

Technical Approach

We propose a method to enforce hard PDE constraints by creating a differentiable layer. We make the layer differentiable using implicit differentiation, thereby allowing us to train our model with gradient-based optimization methods. This layer allows us to find the optimal linear combination of functions in a learned basis, given the PDE constraint.
We provide empirical validation of our method on multiple PDEs. Compared to previous NN approaches, our approach takes fewer iterations to converge to the correct solution, and also requires less training time.

G. Negiar, M. W. Mahoney, A. S. Krishnapriyan, International Conference on Learning Representations (2023).

True solution

Different between ML prediction and true (our method)

Difference between ML prediction and true (previous best method)

Error compared to true solutions, our approach is more accurate and converges more quickly

TALKING POINTS:

PDEs are crucial for describing chemical, physical, and biological processes. However, it can be very slow to solve such systems.
Methods based on neural networks (NNs) have shown promise in recent years for physics-based problems
Current NN approaches try to enforce PDEs as “soft constraints” in the loss function, which have numerous optimization issues.
We introduce a new differentiable PDE-constrained layer that enforces the relevant physics much more precisely, and can be trained within a NN framework.
Our results show that incorporating hard constraints directly into the NN architecture achieves much lower test error when compared to training on an unconstrained objective (previous approaches).

METADATA:

Name of the associated awarded project: SciDac-5 Institutes RAPIDS2

PI name(s):

Name of the program manager:

CITATIONS:

G. Negiar, M. W. Mahoney, A. S. Krishnapriyan, International Conference on Learning Representations (2023).