1 of 37

1

Nazanin Ahmadi�PhD Candidate Biomedical Engineering, Brown University

From PINNs to PIKANs: Physics-Informed AI for Systems Biology and Pharmacology

2 of 37

2

Papers

3 of 37

3

Inverse Problem

PINs: Physics-Informed Networks

Outline

PINNs

Optimizer

Schedular

Representation

(PIKANs)

4 of 37

4

Mathematical Equations

DATA

Partial Physics

Interactions

Neural Networks

How do we blend our understanding of physics and the interactions within the system into the workings of a neural network?

Gray-box discovery in Systems Biology and Pharmacology Modeling

5 of 37

5

PINs: Physics Informed Networks

6 of 37

6

AI-Aristotle Framework

7 of 37

7

J. Sturis, K. S. Polonsky, E. Mosekilde, and E. Van Cauter, “Computer model for mechanisms under- lying ultradian oscillations of insulin and glucose,” American Journal of Physiology-Endocrinology And Metabolism, vol. 260, no. 5, pp. E801–E809, 1991.

Ultradian Endocrine model

8 of 37

8

Ultradian Endocrine model

Where (tj, mj) = [(300, 60)(650, 40)(1100, 50)](min, g)

9 of 37

9

J. Sturis, K. S. Polonsky, E. Mosekilde, and E. Van Cauter, “Computer model for mechanisms under- lying ultradian oscillations of insulin and glucose,” American Journal of Physiology-Endocrinology And Metabolism, vol. 260, no. 5, pp. E801–E809, 1991.

Initial Condition

Ultradian Endocrine model

10 of 37

10

J. Sturis, K. S. Polonsky, E. Mosekilde, and E. Van Cauter, “Computer model for mechanisms under- lying ultradian oscillations of insulin and glucose,” American Journal of Physiology-Endocrinology And Metabolism, vol. 260, no. 5, pp. E801–E809, 1991.

Ultradian Endocrine model

f(t)

g(t)

11 of 37

11

Ahmadi Daryakenari N, De Florio M, Shukla K, Karniadakis GE. AI-Aristotle: A physics-informed framework for systems biology gray-box identification. PLOS Computational Biology. 2024 Mar 12;20(3):e1011916.

Ultradian Endocrine model – Schematic of the PINNs algorithm

12 of 37

12

Ultradian Endocrine model – Results

13 of 37

13

Pharmacokinetics Model

A single-dose compartmental pharmacokinetics model, which is described by the following system of ordinary differential equations:

This model captures the time-dependent concentration of a drug across three compartments over a 50-hour interval. Initially, the drug is introduced into the gastrointestinal (GI) tract (compartment G), where it dissolves and enters the bloodstream (compartment B). The drug is subsequently eliminated through metabolic and excretory processes involving the liver, kidneys, and urinary tract (compartment U )

14 of 37

14

Data Generation

The parameters k_g = 0.72 h−1 and k_b = 0.15 h−1 represent the rates at which the drug transitions from the GI tract to the bloodstream and is cleared from the bloodstream, respectively. For this study, the administered drug is modeled as 0.1 μg of tetracycline, an antibiotic.

The time span for simulation was set to 50 hours, with data sampled at 1-hour intervals.

15 of 37

15

Pharmacokinetics Model - Results

h(t)

16 of 37

16

Pharmacodynamics Model

A pharmacodynamics model used to describe the effect of a chemotherapy drug on cancer cell counts . The time evolution of the cell count is governed by the following differential equation:

where N (t) denotes the cell count at time t, k_p is the constant growth rate of the cells under untreated conditions, k_d(t, D) represents the rate of cell death, which is influenced by both the dosage D and the elapsed time t, and θ (D) is the carrying capacity dependent on the drug dosage, representing the upper limit of cell population that can be supported under a given dose.

17 of 37

17

Pharmacodynamics Model

A pharmacodynamics model used to describe the effect of a chemotherapy drug on cancer cell counts . The time evolution of the cell count is governed by the following differential equation:

18 of 37

18

Data Generation

For the simulations, we set the carrying capacity to θ(D) = 1, the intrinsic cell proliferation rate to kp = 0.0345, the maximum drug-induced death rate to kd,B = 0.03, and the drug efficacy decay rate to r(D) = 0.007.

The simulation was performed over a time span of 600 hours post treatment, with data sampled every 5 hours.

19 of 37

19

What limits the accuracy and efficiency of PINNs today?

Optimizer? Learning rate? Network Architecture?

20 of 37

20

Types of error in Physics-Informed Network

21 of 37

21

Kolmogorov-Arnold Networks Representation Model

22 of 37

22

Chebyshev Kolmogorov-Arnold Networks

23 of 37

23

Tanh-cKANs

We modified the KAN architecture to improve its stability, particularly for inverse problems. The modified version of cPIKAN, referred to as tanh-cPIKAN, is defined mathematically as follows.

24 of 37

24

cKAN vs Tanh-cKAN

cKAN

Tanh-cKAN

25 of 37

25

Network Design

We examined:

The effect of using two neural networks to capture potentially missing components or to decouple dynamics.
The influence of adaptive loss weighting compared to fixed weights.
The benefit of a two-step training strategy.

PINNs

Nc = 500-600
50,000 iterations
Example (PK): Arch: [50, 2]
#parameters: 2,752

PIKANs

Nc = 500-600
70,000 iterations
Example (PK): Arch: [50, 2, 3]
#parameters: 10,300

26 of 37

26

Comparison of Model Performance Across First-Order Optimizers and Learning Rates

Impact of first-order optimizers and initial learning rate choices on model performance in conjunction with a cosine learning rate scheduler.

Tanh-PIKANs

27 of 37

27

Comparison of Model Performance Across First-Order Optimizers and Learning Rates

Impact of first-order optimizers and initial learning rate choices on model performance in conjunction with a cosine learning rate scheduler.

PINNs

28 of 37

28

Comparison of Model Performance Across First-Order Optimizers and Learning Rates

Impact of first-order optimizers and initial learning rate choices on model performance in conjunction with a cosine learning rate scheduler.

29 of 37

29

Learning rate trajectories for different scheduling strategies

Starting from an initial learning rate of 0.001. The schedules were adjusted to ensure comparable learning rate ranges for a fair performance comparison between PINNs and PIKANs.

30 of 37

30

Comparison of Different Schedulers for RAdam Optimizer

Comparison of PINNs and tanh-cPIKANs accuracy in discovering the missing part of the pharmacokinetic model using different learning rate schedulers. All models were trained with the Radam optimizer and an initial learning rate of 0.001.

31 of 37

31

Precision Setting

PK model: Comparison of PINNs and tanh-cPIKANs performance with different optimizers for single precision.

In the course of our experimental investigations, we observed that first-order optimizers,

18

such as RAdam, exhibited relatively consistent performance across both single- and double-

precision settings. Their reliance on gradient-based updates and adaptive moment estimates

renders them less sensitive to minor numerical fluctuations introduced by reduced preci-

sion. In contrast, second-order and hybrid optimizers demonstrated significantly improved

performance under double precision. This enhancement is attributed to their reliance on

more precise curvature information, such as Hessian approximations and gradient norms,

during line search or trust-region updates. Second-order methods typically terminate when

the norm of the gradient falls below a given threshold; thus, the level of numerical precision

directly impacts their stopping criteria and convergence quality. In low-precision settings,

these fine changes may be obscured, potentially causing premature convergence or unstable

updates. These findings motivated our focused evaluation of hybrid and pure second-order

strategies (e.g., BFGS with backtracking line search or trustregion line search methods) in

single-precision mode.

32 of 37

32

Precision Setting

PK model: Comparison of PINNs and tanh-cPIKANs performance with different optimizers for double precision.

33 of 37

33

Precision Setting – double precission

Comparison of PINNs and tanh-cPIKANs using different optimizers (hybrid or second-order) in terms of MAE and computational time.

PK Model:

PD Model:

34 of 37

34

Robustness of PINNs and tanh-cPIKANs under noisy observations

PK model: Evaluation of the robustness of PINNs and tanh-cPIKANs using different optimizers (BFGS and SSBFGS) in terms of MAE under noisy observation. All experiments were performed in double precision, using a cosine learning rate scheduler for the RAdam optimizer with an initial learning rate of 0.001.

35 of 37

35

Summary

Frameworks Introduced

AI-Aristotle: A physics-informed framework for gray-box system identification

Models

Ultradian Endocrine
Pharmacokinetics
Pharmacodynamics for Chemotherapy

Outcomes

Tanh-cPIKANs demonstrated superior performance in gray-box discovery tasks
Strong robustness of tanh-cPIKANs under noisy observations
Training insights: Double precision + second-order optimizers yielded the most stable and accurate results

36 of 37

36

Acknowledgement

Dr. Mario De Florio

Prof. George Karniadakis

Dr. Khemraj Shukla

37 of 37

37

Thank you for listening

https://github.com/mariodeflorio/AI-Aristotle

Nazanin@Brown.edu

https://www.researchgate.net/profile/Nazanin-Ahmadi-Daryakenari