1 of 54

Fourier Neural Operator (FNO)

1

2 of 54

Response to a General Input (in Time)

  • Think about response to a "general input" in time

​

  • The solution is given

​

​

  • Can compute output response to any general input if an impulse response is given
    • Impulse response = LTI system

2

3 of 54

Convolution

  •  

3

4 of 54

Easier Way to Understand Continuous Time Signal

​

​

​

​

4

5 of 54

Impulse Response to LTI System

  • If a system is linear and time-invariant (LTI) then its output is the integral of weighted and shifted unit-impulse responses.

​

5

Time-invariant

Linear (scaling)

6 of 54

 

6

7 of 54

Kernel Integral Operator

  •  

7

 

 

8 of 54

Multi-layer Kernel Integral Operators

  •  

8

Operator

Operator

Operator

9 of 54

Check-up Points

  • Key Takeaways
    • Multi-Layer Stacking: Stacking multiple kernel integral operators allows the framework to achieve the expressiveness required to approximate general operators.
    • The Next Step: To build a data-driven solver, we aim to parameterize and replace the traditional kernel integral operator using neural networks.

​

  • Core Question
    • Can neural networks successfully approximate the kernel integral operator itself?

9

10 of 54

Bridge to Fourier Neural Operators (FNO)

  • The Challenge
    • Computational Complexity: Directly computing the kernel integral (or spatial convolution) at every point in continuous space is computationally expensive

​

  • Idea: Moving to the Frequency Domain
    • By applying the Fourier Transform, complex spatial convolutions are converted into simple element-wise matrix multiplications in the frequency domain
    • Standard neural networks (such as linear layers) are exceptionally efficient at learning these matrix multiplications

​

10

11 of 54

Fourier Transform

  •  

11

12 of 54

Response to General Input (in Frequency)

  •  

12

Multiplication is much easier than convolution !

13 of 54

Kernel Integral Operator

13

Integral Linear Operator

 

 

input

output

14 of 54

Kernel Integral Operator

14

Integral Linear Operator

 

 

input

output

Non-linearity

 

15 of 54

Kernel Integral Operator

15

Integral Linear Operator

 

 

input

output

Non-linearity

 

 

 

 

Convolution operator

(special case of integral operator)

16 of 54

Kernel Integral Operator

16

Integral Linear Operator

 

 

input

output

Non-linearity

 

 

 

 

Convolution operator

(special case of integral operator)

Solving convolution in Fourier domain

 

 

 

17 of 54

Learning R in Frequency Domain

  •  

17

 

 

 

 

 

 

 

 

 

 

 

 

 

frequency domain

 

18 of 54

Include W

​

​

​

​

​

​

​

​

​

​

​

18

 

 

 

 

 

 

 

 

  • The Global Path: Captures non-local, long-range dependencies and global periodic patterns in the frequency domain
  • The Local Path: A local linear transformation that preserves and passes local spatial information directly

19 of 54

Fourier Layer

​

​

​

​

​

  • Called Fourier Layer

19

Fourier Layer

 

 

20 of 54

Lifting and Projection Layers

  • Need to lift to a higher dimensional representation

20

 

 

Fourier Layer

21 of 54

Lifting and Projection Layers

  • Lifting layer: lift (map) to a higher dimensional space
  • Projection layer: project (map) back to the target dimensional space

​

21

 

 

 

 

 

Fourier Layer

 

A fully connected neural network

A fully connected neural network

input

output

22 of 54

Multiple Fourier Layers for Greater Expressiveness

  • Each layer:
    • Transforms to frequency domain
    • Applies learned weights
    • Transforms back

22

 

 

 

 

 

Fourier layers

 

A fully connected neural network

A fully connected neural network

Fourier layers

…

input

output

23 of 54

Lab 1: Euler Beam

23

24 of 54

Lab 1: Euler Beam

  •  

24

25 of 54

Lab 1: Euler Beam

  • Example of three training pairs

​

​

  • The loads are randomly generated using sinusoidal basis functions

​

​

​

​

​

​

​

​

​

  • The corresponding deflections are computed numerically

25

26 of 54

Operator

  • The operator maps the load to the deflection

​

​

​

  • FNO has one network and no place to ask for a point
    • Input is a field on the grid, an array with one or more numbers at every grid point, and
    • Output is another field on the same grid

​

​

​

    • The first channel is the condition, the load at that point
    • The second is the coordinate.

26

27 of 54

Result

  • For unseen loading conditions

27

28 of 54

Result

  • For unseen loading conditions

28

29 of 54

Lab 2: Transient Heat Conduction

29

30 of 54

Problem Setup

  •  

30

31 of 54

Training Data Sets and FNO Structure

  • Training data pair

​

​

​

​

​

​

​

​

  • Input

​

​

​

​

​

​

​

​

​

  • Output

31

Three Initial Conditions

 

One Initial Condition Diffusing

 

time at which the answer is wanted

temperature at that point at time t

32 of 54

Masking Function

  •  

32

33 of 54

Result

33

34 of 54

Result

34

35 of 54

Lab 3: A Vibrating String

35

36 of 54

Problem Setup

  •  

36

37 of 54

Training Data Sets and FNO Structure

  • Training data pair

37

One Initial Shape Vibrating

 

Three Initial Shapes

 

38 of 54

Masking Function and Fourier Features in Time

  •  

38

39 of 54

Result

39

40 of 54

Result

40

41 of 54

 Adding the Physics: PI-FNO

  •  

41

42 of 54

Result

42

43 of 54

Result

43

44 of 54

PI-FNO

  • With the physics in the loss, the operator extrapolates beyond the record far better than with data alone

44

45 of 54

Lab 4: Heat Conduction on a Square Plate

45

46 of 54

Problem Setup

  •  

46

47 of 54

Problem Setup

  •  

47

48 of 54

Training Data Set

  • Input
  • Output

48

49 of 54

Result

49

50 of 54

Result

50

51 of 54

Result

51

52 of 54

Result

52

53 of 54

Result through Time

53

54 of 54

Result through Time

54