Physics-informed Neural Networks

Potentials of Data-driven Approach

• Deep learning as a data-driven approach

​

​

2

Deep Learning

• Advantages of data-driven approaches
• If enough data is available, a legitimate level of prediction performance can be achieved without domain knowledge

​

​

​

​

​

3

Limitations of Pure Data-driven Approaches

• Interpretability: Physically interpretable output

​

4

Direction of Deep Learning in ME

• Artificial Intelligence + Mechanical Engineering

5

Data-driven AI

Direction of Deep Learning in ME

• Artificial Intelligence + Mechanical Engineering

6

Physics-informed AI

Physics-informed AI

7

Physics-informed AI

8

Physics-informed AI

9

Physics-informed AI

10

Physics-informed AI

11

Taxonomy of Informed Deep Learning

12

Sung Wook Kim, "Recent Advances of Artificial Intelligence in Manufacturing Industrial Sectors: A Review," IJPE

Differential Equation

Algebraic Equation

Knowledge Graph

Simulation Result

Human Feedback

Knowledge Representation

Knowledge Integration

Feature Engineering

Designing

Regularizing

Deep Neural Networks

ANN

CNN

RNN

GNN

Generative Model

…

…

…

Taxonomy of Informed Deep Learning

13

Differential Equation

Algebraic Equation

Knowledge Graph

Simulation Result

Human Feedback

Knowledge Representation

Knowledge Integration

Feature Engineering

Designing

Regularizing

Deep Neural Networks

ANN

CNN

RNN

GNN

Generative Model

Sung Wook Kim, "Recent Advances of Artificial Intelligence in Manufacturing Industrial Sectors: A Review," IJPE

…

…

…

Knowledge Integration

• Integrating external knowledge into DNNs is a critical step to ensure that models not only rely on large datasets but also leverage existing domain knowledge.

​

• Feature Engineering: Tailoring input features to better align with domain-specific knowledge

​

• Designing: Modifying model architectures to incorporate external knowledge sources

​

• Regularizing: Adding constraints, priors, or penalties to the loss function ensures that the model adheres to known properties of the domain. This prevents overfitting and encourages the model to find solutions consistent with real-world knowledge

​

14

Data-driven AI

• Models physical process near training data
• Fails to generalize beyond it
• Sole reliance on data raises the question of whether it truly "understood" the scientific problem

15

From Ben Moseley

Physics-informed AI

• Incorporating physics enhances AI
• Models learn from data and existing scientific knowledge
• Leads to more powerful, informed predictions

16

From Ben Moseley

Physics-informed Neural Networks

17

Multilayer Feedforward Networks are Universal Approximators

• The Universal Approximation Theorem
• Neural Networks are capable of approximating any Borel measurable function
• Neural Networks (1989)

18

Differential Equations

• Types of ODEs
• Linear

​

• Nonlinear

​

​

• Partial Differential Equation

​

​

​

• Types of ODE / PDE problems
• Initial Value Problem (IVP)
• Boundary Value Problem (BVP)

​

​

19

Journal of Computational Physics (2019)

• M. Raissi, P. Perdikaris, G.E. Karniadakis

20

Deep Learning as a Function Approximation

• Training a neural network on data approximates the underlying mapping from inputs to outputs

​

​

21

Architecture of Physics-informed Neural Networks (PINN)

• NN as an universal function approximator

​

• Given
• ODE or PDE
• Initial and boundary conditions

​

• Aim to approximate the solution of PDEs (target function) by training neural networks

​

​

​

​

​

​

​

22

[

Architecture of Physics-informed Neural Networks (PINN)

• NN as an universal function approximator

​

• Given
• ODE or PDE
• Initial and boundary conditions

​

• Aim to approximate the solution of PDEs (target function) by training neural networks

​

23

Architecture of Physics-informed Neural Networks (PINN)

• NN as an universal function approximator

​

• Given
• ODE or PDE
• Initial and boundary conditions

​

• Aim to approximate the solution of PDEs (target function) by training neural networks

​

24

Characteristics of Physics-informed Neural Networks (PINN)

• Loss function with multiple terms
• Adam and quasi-Newton L-BFGS optimizers are often used

​

• Mesh-free approach
• Collocation method
• Monte Carlo sampling method

​

25

Characteristics of Physics-informed Neural Networks (PINN)

• Loss function with multiple terms
• Adam and quasi-Newton L-BFGS optimizers are often used

​

• Mesh-free approach
• Collocation method
• Monte Carlo sampling method
• (optional) Uniform grid

​

26

Collocation Points

• Meshfree approach
• Collocation method
• Monte Carlo sampling method

​

• Time transient

27

PDE

• NN as an universal function approximator

​

• Given
• ODE or PDE
• Some measured data from initial and boundary conditions

​

​

​

​

​

​

​

​

​

​

28

PDE + Data

• NN as an universal function approximator

​

• Given
• ODE or PDE
• Some measured data from initial and boundary conditions

​

​

​

​

• Data

​

29

PDE + Data

• NN as an universal function approximator

​

• Given
• ODE or PDE
• Some measured data from initial and boundary conditions

​

​

​

​

• Data

​

30

PINN as Inverse Problem Solver

• Inverse problems involve determining unknown parameters, functions, or underlying physical laws

​

• Given the typically ill-posed nature of inverse problems, they often suffer from insufficient information, making it challenging to obtain unique and stable solutions

​

• To address this issue, a combination of observed data and physical laws is essential

31

Formulation of an Inverse Problem with PINNs

32

Summary of Physics-informed AI

• Leverage inductive bias and sparsity

33

Summary: Physics-informed Neural Networks (PINNs)

• PINN can be used as a physics solver to address physics problems
• PINN can operate without data by utilizing physics loss.
• Both physics and data can also be used to calculate the loss function

34

Only physics loss

Physics loss + Data loss

Lab 1: Euler Beam (Solid Mechanics)

35

Lab 1: Euler Beam (Solid Mechanics)

​

​

• Partial differential equations & boundary conditions

​

​

​

• One Dirichlet boundary condition on the left boundary:
• One Neumann boundary condition on the left boundary:
• Two boundary conditions on the right boundary:

​

• The exact solution for reference is

​

36

Collocation Points

• Collocation Points
• Uniformly distributed

​

​

​

​

​

​

​

​

​

​

​

• (also possible) Randomly distributed

​

37

Lab 1: Euler Beam (Solid Mechanics)

• PINN as a ODE solver
• Neural network and loss functions

38

Neural Network and Loss Functions

39

PDE

40

Boundary Conditions

41

Train

42

Result

43

Lab 2: Elastic Deformation for Thin Plate

44

Lab 2: Elastic Deformation for Thin Plate (2D Problem)

• Find displacement and stress distribution of thin plate
• Based on Kirchhoff-Love plate theory, three hypotheses were used
• Straight lines normal to the mid-surface remain straight after deformation
• Straight lines normal to the mid-surface remain normal to the mid-surface after deformation
• The thickness of the plate does not change during a deformation

​

• Only one quarter of the plate is considered since the geometry and in-plane forces are symmetric

45

Lab 2: Elastic Deformation for Thin Plate (2D Problem)

• Föppl–von Kármán equations for the isotropic elastic plate

​

​

• Dirichlet boundary conditions at ① and ②

​

​

​

• Free boundary conditions at ③

​

​

• Free boundary condition and in-plane force boundary condition at ④

46

Numerical Solution (= Exact Solution)

47

Collocation Points

• Uniformly distributed

​

​

​

​

​

​

​

​

​

​

​

​

• (also possible) Randomly distributed

​

48

Lab 2: Elastic Deformation for Thin Plate

49

Lab 2: Elastic Deformation for Thin Plate

50

Governing Equations

51

Boundary Conditions

52

Training

53

Results

54

FEM

PINN

55

PINN

PINN + Data

• Adding data constraints for regularization

56

Data

57

FEM

PINN

58

PINN

Lab 3: �(Inverse Problem) Unknown Parameter Estimation

59

Flow Around a Cylinder

• 2D Navier-Stokes equations

​

​

​

​

​

​

• Boundary conditions of flow around a cylinder

60

Inverse Problem: Unknown Parameter Estimation

• Solving 2D Naiver-Stokes equation with unknown density and viscosity variables

​

​

​

​

​

​

​

61

Physics + Data

62

Physics + Data

63

Collocation Points

64

PINN Network

65

Results

66

Lab 4�(Inverse Problem) Unknown Boundary Condition Estimation

67

Lab 4: Heat Transfer in 2D

• Solving 2D heat transfer equation with unknown boundary condition
• Laplace’s equation

​

​

​

​

​

​

​

68

Lab 4: Heat Transfer in 2D

• Solving 2D heat transfer equation with unknown boundary condition
• Laplace’s equation

​

​

​

​

​

​

​

69

Lab 4: Heat Transfer

70

Inverse Problem: Unknown Boundary Conditions

• PINN w/o data

​

​

​

​

​

​

• PINN + Data

71

+ Data and Prior Knowledge

• PINN + Data

​

​

​

​

​

​

• PINN + Data + Prior Knowledge (= constant temperature)

​

72

+ Data and Prior Knowledge

• Predicted temperature on top

​

​

​

​

​

​

​

​

73

Redistribute Collocation Points

• Mesh generation in CFD
• Fine mesh is required in areas with radical shifts.^[7]

74

Collocation points

Redistributed collocation points

Coarse mesh

Fine mesh