Effective State-Space Models Estimation for 2D Flows Using Convolutional Neural Networks in a SUBNET Approach
Gerben I. Beintemaa, Roland Totha,b , Maarten Schoukensa
aDepartment of Electrical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands
bSystems and Control Laboratory, Institute for Computer Science and Control, Budapest, Hungary.
emails: g.i.beintema@tue.nl, m.schoukens@tue.nl, r.toth@tue.nl
1
Spatial Systems
2
PDE simulation describe many systems accurately
Cameras are cheap
Analysis and Control of spatial systems
3
High dimensional/spatial system are challenging to analysis and control
Analysis/Design
Difficult due to high dimensional data
MPC
PDE simulations are often much slower than real time
Control synthesis
Applied up to
about 20 states
Strategy: Dimension Reduction
Model order reduction two approaches
4
System
Collect high Dim. data
High Dim.
IO data
Dim.
Reduction
Low Dim.
IO data
System Identification
Low Dim. model
Loss of information potentially important to system identification
PDE
System
Dim.
Reduction
Low Dim.
System
Approach 1:
Approach 2:
This presentation
5
High Dim.
System
Collect data
High Dim.
data
Dim.
Reduction
Low Dim.
data
System Identification
Low Dim. model
This presentation
SUBNET method for low dim. data
SUBNET
method for
high dim. data
Add some AI
Prediction/Simulation Error Approach
6
Short comings:
The Dataset:
The Model:
The Loss function:
Gradient and value Explosions
Local Minima
Non-smooth cost
Many nice theory properties and can provide great models!
SUBNET approach[1]
7
The Datasets:
The Model:
The Loss function:
Explosions Reduced
Local Minima Reduced
Smoothed cost
Advantages:
[1] Beintema, Gerben I., Maarten Schoukens, and Roland Tóth. "Deep subspace encoders for nonlinear system identification." Automatica 156 (2023): 111210.
Construction Short Datasets
8
Extract short datasets
from full dataset
Allow for overlap:
More sample efficient[1]
[1] Beintema, Gerben I., Maarten Schoukens, and Roland Tóth. "Deep subspace encoders for nonlinear system identification." Automatica 156 (2023): 111210.
SUBNET Initial states
9
Option 2:
State Encoder
SUBNET Approach for NL state-space
10
Excellent (benchmark) results using MLPs
Extend SUBNET to high dim. 2D Data
11
high 2D Dim.
data
SUBNET identification
Low dim. State-space model
Main idea: Progressive upscale!
12
Their model structure:
Take aways:
Karras, Tero, et al. "Progressive growing of GANS for improved quality, stability, and variation." arXiv preprint arXiv:1710.10196 (2017).
13
Stride = 2
Stride = 1
Downscaling Network
14
Upscaling Network
Computational Properties
15
Downscaling Network
Upscaling Network
Each upscale CNN:
Image W & H * 2
Feature size / 2
Each downscale CNN:
Image W & H / 2
Feature size * 1.5
Linear Scaling
Max image size:
About 250 by 250
Benchmark: Wake Flow Oscillating Cylinder
16
Decuyper, J., De Troyer, T., Tiels, K., Schoukens, J., & Runacres, M. C. (2020). A nonlinear model of vortex-induced forces on an oscillating cylinder in a fluid flow. Journal of Fluids and Structures, 96, 103029.
SUBNET and model structure
17
SUBNET parameters:
None of these are specify sensitive parameters
Others settings:
CNN Kernel size: 3
Activation: tanh
Normalization: Input and Output
MLPs: 2 hidden layers 64 nodes
Optimizer: Adam optimizer
Compute: 1 hour compute on GPU (1660)
Important: Good Early Stopping
18
Train SUBNET loss
Val. Sim. Error
Roll back to best validation error
Epochs
Results: Wake Flow Oscillating Cylinder
19
1.6M parameters
200K parameters
The Numbers
20
Model Inference: < 1 sec
Original PDE: > 1 day
Training: Few hours on a GPU
Errors:
About 8% NRMS
(excluding autonomous part)
Result: Controlled Thermal Convection
21
Gerben Beintema, Data–driven Learning of Nonlinear Dynamic Systems: A Deep Neural State–Space Approach, PhD Thesis 2024
Results: Camera on Unbalanced disk
22
Take aways
23
Our Aim:
We extended the SUBNET to high dim. 2D Data
Parameterization: Many small steps >
Few big steps
Progressive CNN up/downscaling
scales well!
Results:
Modelled both PDE data and camaras with great success