1 of 62

Lab 1: Simply Supported Beam

1

2 of 62

Simply Supported Beam

2

3 of 62

Collocation Points

3

4 of 62

Physics Loss: PDE

4

5 of 62

Physics Loss: Boundary Conditions

5

6 of 62

Train

6

7 of 62

Result: Displacement

Displacement on 100 test points against the analytical solution

7

8 of 62

Result: Slope, Moment and Shear

8

9 of 62

Inverse Problem without Data

9

10 of 62

PDE with Unknown E

10

11 of 62

Result

11

12 of 62

Observed Data

12

13 of 62

Train

13

14 of 62

Training History (1/2)

14

15 of 62

Training History (2/2)

15

Loss Spike

16 of 62

Refine with L-BFGS

Adam ends on a loss spike
L-BFGS refines the network and E together

16

Why two stages. Adam is robust from a random initialisation but carries no curvature, so once the loss is small its fixed step overshoots the sharpest directions: the spikes on the previous slide, and E drifting past 1. L-BFGS uses curvature and a line search and converges fast near a minimum, but started far from one it can settle in the wrong valley. Warm-starting L-BFGS with Adam is the standard PINN recipe (Raissi et al. 2019; DeepXDE, Lu et al. 2021; Rathore et al. 2024).

The switch point is a budget, not a tuned number. Same seed and data, only the switch moved (CPU, 2026-09-06):

switch at 1000: Adam E 0.9629 -> L-BFGS 1.0109

switch at 2000: Adam E 1.0693 -> L-BFGS 1.0080

switch at 3000: Adam E 1.1388 -> L-BFGS 1.0108 (this deck)

switch at 5000: Adam E 1.2126 -> L-BFGS 1.0111

plateau-triggered (no 10% gain in 500 epochs, fired at 4717): Adam E 1.2041 -> L-BFGS 1.0108

Wherever Adam stops, L-BFGS lands in the same place.

Double precision: the strong-Wolfe line search compares loss values; near 1e-5 those differences are below float32 resolution, and the search stops at a step that differs between CPU and GPU (E 1.007 on CPU, 1.104 on GPU). In float64 both give 1.011. Details: docs/measured/2026-09-06_pinn-lbfgs-needs-float64.md.

17 of 62

L-BFGS

18 of 62

Why L-BFGS Converges Faster

Adam scales each coordinate on its own, so it cannot turn the direction
L-BFGS builds curvature from the last m steps and turns the whole vector

19 of 62

Training History with L-BFGS

19

20 of 62

Adam with a Longer Budget

The same budget spent on Adam alone leaves E drifting

21 of 62

Result

21

L-BFGS

22 of 62

Lab 2: Unsteady Drainage of a Tank

22

23 of 62

Unsteady Drainage of a Tank

23

24 of 62

Collocation Points

24

25 of 62

PINN

25

26 of 62

Physics Loss: ODE and Initial Condition

26

27 of 62

Train

27

28 of 62

Result

Relative L² error: 0.0142

28

29 of 62

Inverse Problem without Data

29

30 of 62

Network and Unknown Parameter

30

31 of 62

Result

31

32 of 62

Observed Data

32

33 of 62

Train

33

34 of 62

Refine with L-BFGS

Adam: one step size, fixed

Robust from a random start
A loss floor set by the learning rate

L-BFGS: curvature and a line search

Collocation points drawn once, then fixed
Double precision for the line search

Two stages: Adam first, L-BFGS after

35 of 62

Result

35

36 of 62

Lab 3: SIR Epidemic Model

36

37 of 62

SIR Epidemic Model

37

38 of 62

SIR Epidemic Model

38

39 of 62

Reference Solution

39

40 of 62

Observed Data

40

41 of 62

PINN with Three Outputs

41

42 of 62

Network and Unknown Parameters

42

43 of 62

Physics Loss: SIR Equations

43

44 of 62

Train

44

45 of 62

Training History with L-BFGS

45

46 of 62

Loss Converged, Parameter Not

Loss flat where Adam stops, the infection rate still climbing

47 of 62

Result

Susceptible and recovered fractions never observed, still reconstructed

47

48 of 62

Lab 4: �(Inverse Problem) Unknown Parameter Estimation

48

49 of 62

Flow Around a Cylinder

2D Navier-Stokes equations

Boundary conditions of flow around a cylinder

49

50 of 62

Inverse Problem: Unknown Parameter Estimation

Solving 2D Naiver-Stokes equation with unknown density and viscosity variables

50

51 of 62

Physics + Data

51

52 of 62

Physics + Data

52

53 of 62

Collocation Points

53

54 of 62

PINN Network

54

55 of 62

Results

55

56 of 62

Lab 5:�(Inverse Problem) Unknown Boundary Condition Estimation

56

57 of 62

Lab 4: Heat Transfer in 2D

Solving 2D heat transfer equation with unknown boundary condition
Laplace’s equation

57

58 of 62

Lab 4: Heat Transfer in 2D

Solving 2D heat transfer equation with unknown boundary condition
Laplace’s equation

58

sensors for temperature

59 of 62

Lab 4: Heat Transfer

59

60 of 62

Inverse Problem: Unknown Boundary Conditions

PINN w/o data

PINN + Data

60

61 of 62

+ Data and Prior Knowledge

PINN + Data

PINN + Data + Prior Knowledge (= constant temperature)

61

62 of 62

+ Data and Prior Knowledge

Predicted temperature on top

62