Differentiable Analog Circuit Simulation and Optimization

Xuliang Dong

11/30/2023

Motivation

• Spice simulators are not differentiable.
• Have to use Bayesian Optimization or other methods for optimal circuit parameters.
• Suffer from non-converging issues.

parameters

C

Spice Simulator

V

Bayesian Optimization

Motivation

• Differentiable simulators can:
• Support gradient-based optimization. dV/dC
• Help with non-convergence problems. dV/dt

Differentiable Simulator

parameters

C

V

Loss

Overview

1. DDSP: Differentiable Digital Signal Processing
2. Applications of DDSP to Virtual Analog (VA) Modeling

Discretize the ODE and train the NN or RNN; Circuit topology is fixed.

• White-box Virtual Analog (VA) Modeling
• Differentiable IIR Filters for Machine Learning Applications
• Analog circuit design with hypernetworks and differentiable simulator

Circuit topology is not fixed but it follows certain rules.

A space case when the output-input relationship is simply matrix multiplication…

• What if we don’t want to simulate all parameters for a new circuit using a non- differentiable simulator (like Spice)?

DDSP: Differentiable Digital Signal Processing

• The key idea is to formulate fundamental DSP components (such as oscillators) as differentiable functions. E.g. harmonic oscillator:

A: amplitude; 𝜙: instantaneous phase

• Enable direct integration of classical DSP elements. Modular and interpretable.

​

​

​

​

• Applications: audio synthesis; DDSP autoencoder; Virtual Analog (VA) modeling…

Engel, Jesse, Lamtharn Hantrakul, Chenjie Gu, and Adam Roberts. "DDSP: Differentiable digital signal processing." arXiv preprint arXiv:2001.04643 (2020).

DDSP Autoencoder

DDSP: Differentiable Digital Signal Processing (tensorflow.org)

Virtual Analog Modeling: reproduce/emulate analog effects

• White-box: requires knowledge of DSP as well as physics, circuit theory, etc.
• State-space formulation
• Modified nodal analysis (MNA)
• Port-Hamiltonian formulation
• Wave Digital Filters (WDF)

​

• Black-box: input-output mapping without circuit knowledge
• Convolution with Impulse Response (for linear systems)
• Volterra Series
• Weiner-Hammerstein Method
• Neural Networks

Differentiable White-Box Virtual Analog Modeling

1. Analyze the exact circuit in the state-space form

​

u: input; x: internal state; y: output

• Discretize using trapezoidal rule → recursion

​

• Training with DDSP and Pytorch

​

​

Esqueda, Fabián, Boris Kuznetsov, and Julian D. Parker. "Differentiable white-box virtual analog modeling." In 2021 24th International Conference on Digital Audio Effects (DAFx), pp. 41-48. IEEE, 2021.

Differentiable White-Box Virtual Analog Modeling - Linear Example

​

Differentiable White-Box Virtual Analog Modeling - Nonlinear Example

​

Ibanez TS-808 Overdrive Stage

(22) iterative root solver: Newton-Raphson

Differentiable White-Box Virtual Analog Modeling - Nonlinear Example

Differentiable IIR filters for machine learning applications

Formulate the IIR filter’s state-space form in the RNN framework.

x: input; h: hidden state; y: output

Kuznetsov, Boris, Julian D. Parker, and Fabián Esqueda. "Differentiable IIR filters for machine learning applications." In Proc. Int. Conf. Digital Audio Effects (eDAFx-20), pp. 297-303. 2020.

Summary

• When we have access to the measurements of input-output, we can discretize the ODE(s) and formulate them into a Neural Network to solve the problem.
• Limitation:
• Real-time cost
• Hand-derived functions
• Fixed circuit topology

Electric Analog Circuit Design With Herpernetworks and A Differential Simulator

An Inverse Problem

• Input: Desired characteristic functions of V and I
• Output: configuration S = {(a_i, c_i, v_i)}

E.g. {(P, C, 1), (S, R, 1), (P, L, 0.5)}

Rotman, Michael, and Lior Wolf. "Electric analog circuit design with hypernetworks and a differential simulator." In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4157-4161. IEEE, 2020.

Electric Analog Circuit Design With Herpernetworks and A Differential Simulator

Decoder: RNN GRU. d=26,380

Encoder: MultiScale Resnet. Wg = f(V, I)

Differentiable Simulator

Differentiable simulator: how does it help?

Change the loss function from:

To:

“An infinitesimal change to a component’s value should result only in a slight change in the characteristic function.”

Experiments and results

Comparing the configurations instead of waveforms.

New circuit: What if the simulator is non differentiable?

Given a system of ODEs, automatically derive a single governing ODE

Solve ODE with Neural Networks

Train a MLP y₁ to approximate v₁(t) when t ∊ [0, T₁].

• Input: t, C_0, L₀
• Output: y₁
• Initial condition: y₁(t=0, C₀=1, L₀=1)=0
• Loss function:

​

Credit:Solve Ordinary Differential Equation Using Neural Network - MATLAB & Simulink (mathworks.com)

Advantages and Limitations

Pros:

• Does not require measurements of simulation results.
• Automatic differential elimination.

​

Cons:

• Limited to a specific circuit topology.
• For transient analysis, there could be too many neural networks to train.

Summary

• By incorporating the circuit physics and formulating them into the auto-differentiation framework, we will have more control over the optimization/training process.
• How to deal with the governing ODE(s) results in various methods for circuit simulation.
• To build a differentiable analog circuit simulator, we can train a NN to solve the ODE.

Reference

1. Engel, Jesse, Lamtharn Hantrakul, Chenjie Gu, and Adam Roberts. "DDSP: Differentiable digital signal processing." arXiv preprint arXiv:2001.04643 (2020).
2. Dempwolf, Kristjan, Martin Holters, and Udo Zölzer. "Discretization of parametric analog circuits for real-time simulations." In Proceedings of the 13th international conference on digital audio effects (DAFx’10). 2010.
3. Esqueda, Fabián, Boris Kuznetsov, and Julian D. Parker. "Differentiable white-box virtual analog modeling." In 2021 24th International Conference on Digital Audio Effects (DAFx), pp. 41-48. IEEE, 2021.
4. ER, NIV. "Differentiable White Box Modeling of Moog VCF." (2022).
5. Kuznetsov, Boris, Julian D. Parker, and Fabián Esqueda. "Differentiable IIR filters for machine learning applications." In Proc. Int. Conf. Digital Audio Effects (eDAFx-20), pp. 297-303. 2020.
6. Rotman, Michael, and Lior Wolf. "Electric analog circuit design with hypernetworks and a differential simulator." In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4157-4161. IEEE, 2020.

​

​