1 of 71

Deep Neural Operator

1

2 of 71

 

2

Burgers’ Equation

Korteveg-de Vries Equation

Kuramoto-Sivashinsky Equation

Navier-Stokes Equation

3 of 71

Limitation of PINN: Single Instance Solver

  • Single Instance Solver
    • PINN model is designed to solve a specific instance of a PDE problem
    • PINN models can accurately predict solutions for the given ICs, BCs, and governing equations during training

​

  • Need for Re-training
    • If ICs or BCs change, PINN model must be re-trained from scratch, making real-time or parametric predictions computationally expensive.

​

  • How can we enable real-time predictions for varying conditions without retraining the model for each new scenario?

4 of 71

Euler Beam

  •  

4

 

 

 

5 of 71

Euler Beam

  •  

5

 

 

 

6 of 71

Euler Beam

  •  

6

 

 

 

 

function

function

 

 

7 of 71

Transition to Deep Neural Operators

  • Deep Neural Operators (DNOs)
    • By learning solution mappings rather than solving individual instances of PDEs
    • DNOs learn function-to-function mappings between input conditions (ICs, BCs, PDE parameters) and their corresponding solutions

​

  • Shifting from instance-specific learning (as in PINNs) to operator learning

​

  • Deep Neural Operators unlock new possibilities for solving parametric PDEs, enabling rapid and flexible predictions across varying physical conditions
    • Boundary or initial conditions
    • Parameters in PDE
    • (Geometric) Domain

7

8 of 71

What is Operator?

  • An operator is generally a mapping between functions
    • Function: map numbers (vector) to numbers (vector)
    • Operator: map functions to functions

​

  • Examples of operator
    • Derivative

​

    • Integral

​

    • Fourier transform

​

    • Laplace transform

​

    • Projection

​

​

8

9 of 71

Can Neural Networks Learn Operators?

  •  

9

10 of 71

Neural Operators

  • Deep Operator Network (DeepONet)
    • Nature machine intelligence (2021)

10

  • Fourier Neural Operator (FNO)
    • https://arxiv.org/abs/2010.08895 (2021)

11 of 71

Neural Operators

11

DeepONet

Multiscale DeepONet

MIONet

PI-DeepONet

Fourier-DeepONet

FNO

AFNO

MFNO

FNO-Attention

GFNO

Fourier Neural Operator

Deep Operator Network

12 of 71

Deep Operator Network�(DeepONet)

12

13 of 71

Basic Concept of DeepONet (1/3)

  •  

13

14 of 71

Basic Concept of DeepONet (2/3)

  •  

14

15 of 71

Basic Concept of DeepONet (3/3)

  •  

15

16 of 71

Euler Beam

  •  

16

 

 

 

 

 

 

17 of 71

From Function to Operator

  • An operator is generally a mapping between functions
    • Function: map vectors to vectors
    • Operator: map functions to functions

​

17

 

 

 

 

 

 

 

 

 

 

 

 

 

Input vector

Output vector

18 of 71

From Function to Operator

  • An operator is generally a mapping between functions
    • Function: map vectors to vectors
    • Operator: map functions to functions

​

18

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

19 of 71

From Function to Operator

  • An operator is generally a mapping between functions
    • Function: map vectors to vectors
    • Operator: map functions to functions

​

19

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

20 of 71

From Function to Operator

  • An operator is generally a mapping between functions
    • Function: map vectors to vectors
    • Operator: map functions to functions

​

20

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

21 of 71

From Function to Operator

  • An operator is generally a mapping between functions
    • Function: map vectors to vectors
    • Operator: map functions to functions

​

21

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

22 of 71

From Function to Operator

  • An operator is generally a mapping between functions
    • Function: map vectors to vectors
    • Operator: map functions to functions

​

22

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

23 of 71

From Function to Operator

  • An operator is generally a mapping between functions
    • Function: map vectors to vectors
    • Operator: map functions to functions

​

23

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

24 of 71

DeepONet

  •  

24

25 of 71

DeepONet as Basis and Coefficients Representation

  • At first glance, the architecture of DeepONet may appear somewhat unexpected

​

  • Learns an operator representation by decomposing the solution function into a set of
    • learned basis functions and
    • their corresponding coefficients

25

26 of 71

DeepONet as Basis and Coefficients Representation

26

Slide from Prof. Taejin Kim (Jeonbuk National University)

27 of 71

DeepONet as Basis and Coefficients Representation

27

Slide from Prof. Taejin Kim (Jeonbuk National University)

28 of 71

DeepONet as Basis and Coefficients Representation

28

Slide from Prof. Taejin Kim (Jeonbuk National University)

29 of 71

DeepONet as Basis and Coefficients Representation

29

Slide from Prof. Taejin Kim (Jeonbuk National University)

30 of 71

DeepONet as Basis and Coefficients Representation

30

Slide from Prof. Taejin Kim (Jeonbuk National University)

31 of 71

DeepONet as Basis and Coefficients Representation

31

Slide from Prof. Taejin Kim (Jeonbuk National University)

32 of 71

DeepONet as Basis and Coefficients Representation

32

Slide from Prof. Taejin Kim (Jeonbuk National University)

33 of 71

DeepONet is Fundamentally Data-Driven

  • DeepONet is fundamentally a data-driven approach
    • Require solution samples obtained from either a numerical solver or real-world data to learn the operator

​

  • DeepONet is a supervised learning framework, designed to approximate operators in function spaces

​

​

33

34 of 71

Beyond Data-Driven: Physics-Informed DeepONet

  • Physics-informed DeepONet (PI-DeepONet)
    • Combines the strengths of DeepONet and PINNs
    • Integrates known physical laws into the training process to better approximate operators, even when training data is sparse or incomplete.

34

35 of 71

Physics-informed DeepONet (PI-DeepONet)

  • DeepONet is fundamentally a data-driven approach
    • Relies on solution samples from either a numerical solver or real-world data to learn the operator

​

​

​

​

​

​

​

​

​

​

​

35

36 of 71

Physics-informed DeepONet (PI-DeepONet)

  • DeepONet is fundamentally a data-driven approach
    • Relies on solution samples from either a numerical solver or real-world data to learn the operator

​

​

​

​

​

​

​

​

​

​

​

  • PI-DeepONet use only collocation points and governing equations without requiring any data
  • However, in practice, often recommended to utilize a combination of both observed data and physical laws

36

37 of 71

Scientific Machine Learning (1/3)

37

PINNs

Physics-informed DeepONet

  • Model architecture

​

​

​

​

  • Single instance and unsupervised method
    • For given ICs, BCs, and PDEs, the trained PINN models show effective performance
    • For untrained conditions, PINN models should be re-trained

​

​

Deep Operator Network

Inputs

Neural Network

Output

​

Single network

​

38 of 71

Scientific Machine Learning (2/3)

38

PINNs

Physics-informed DeepONet

Deep Operator Network

  • Model architecture

​

​

​

​

  • Multiple instance and supervised method
    • Predicts systems for untrained conditions
    • Physics loss is not directly reflected, but physics knowledge is embedded through training data

​

​

Inputs

Neural Network

Output

PDE conditions

Branch net

Basis coefficients

Domain

Trunk net

Basis functions

39 of 71

Scientific Machine Learning (3/3)

39

PINNs

Physics-informed DeepONet

Deep Operator Network

  • Model architecture

​

​

​

​

  • Multiple instance and unsupervised method
    • Identically to DeepONet, it predicts systems for untrained conditions
    • Embed physics knowledge with PDE loss

​

​

Inputs

Neural Network

Output

PDE conditions

Branch net

Basis coefficients

Domain

Trunk net

Basis functions

40 of 71

Lab 1: Heat Transfer in 1D

40

41 of 71

Heat Transfer in 1D

  •  

41

42 of 71

Note

  • The purpose of this example is to learn the code

​

  • Uses the simplest DeepONet that can be built:
    • One neuron of input to the branch network, representing the condition, and
    • One neuron of input to the trunk network, representing the evaluation location

​

  • This is a linear system with a closed-form solution

42

43 of 71

Heat Transfer in 1D

  •  

43

44 of 71

Heat Transfer in 1D

  •  

44

45 of 71

Heat Transfer in 1D

  •  

45

46 of 71

Heat Transfer in 1D

  •  

46

47 of 71

Heat Transfer in 1D

  •  

47

48 of 71

Heat Transfer in 1D

  •  

48

49 of 71

Training Data Generation

49

50 of 71

BranchNet and TruckNet

50

51 of 71

Result for Test

  •  

51

 

52 of 71

Lab 2: Euler Beam

52

53 of 71

Lab 2: Euler Beam

  •  

53

54 of 71

Input-Output Data Pairs

54

55 of 71

Result

  • For unseen loading conditions

55

56 of 71

Lab 3: Transient Heat Conduction

56

57 of 71

Problem Setup

  •  

57

58 of 71

Training Data Sets and DeepONet Structure

  • Operator learns

58

Three Initial Conditions

 

One Initial Condition Diffusing

 

59 of 71

Masking Function

  •  

59

60 of 71

Result

60

61 of 71

Result

61

62 of 71

Lab 4: A Vibrating String

62

63 of 71

Problem Setup

  •  

63

64 of 71

Training Data Sets and DeepONet Structure

  • Operator learns

64

One Initial Shape Vibrating

 

Three Initial Shapes

 

65 of 71

Masking Function and Fourier Features in Time

  •  

65

66 of 71

Result

66

67 of 71

Result

67

68 of 71

 Adding the Physics: PI-DeepONet

  •  

68

69 of 71

Result

69

70 of 71

PI-DeepONet

  • With the physics in the loss, the operator extrapolates beyond the record far better than with data alone

70

71 of 71

Summary

  • Instant Inference: Makes real-time predictions for varying conditions without retraining for each new scenario.

​

  • High Efficiency: Replaces time-consuming numerical solvers (FEM/FDM) once trained.

71