1 of 33

AI Assisted Detector Design for EIC

Karthik Suresh, for AID(2)E Collaboration

Department of Data Science

College of William and Mary

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

2 of 33

Outline

Multi Objective Optimization
Need for AI in detector design – The AID(2)E project
What is this boot camp about

2

https://aid2e.github.io/boot-camp-2024/intro.html

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

3 of 33

Multi Objective Optimization

3

Design space spanned by ‘x’

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

4 of 33

Multi Objective Optimization

4

Objectives to optimize

Design space spanned by ‘x’

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

5 of 33

Multi Objective Optimization

5

Design space spanned by ‘x’

Constraints

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

6 of 33

Multi Objective Optimization

6

Design space spanned by ‘x’

Bounded Design Space

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

7 of 33

Multi Objective Optimization : Visual Intro

Multiple “objectives”

Momentum resolution
𝞱 resolution
KF efficiency
projected 𝞱 resolution @ PID

Goal : “Optimize” these Objectives
Map: “Design” space – “Objective” Space
Non-Feasible region to be avoided

7

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

8 of 33

Multi Objective Optimization : Visual Intro

What is “Optimal”?

Non-dominated (Pareto) Solutions

How to rank solutions?

“Fronts” of solutions

Methods of MOO

Evolutionary
Bayesian
Preferential Learning, etc.

8

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

9 of 33

Multi Objective Optimization through surrogate modelling

9

Surrogate Model – A model that will be able to successfully approximate the true function.
Acquisition Model – A quick evaluator to choose the next point to be computed

Based on Exploration and Exploitation in the search space.
Critically important, since, this is key in convergence.

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

10 of 33

Large Scale Experiments : An Ideal MOO problem

GEANT4 – computation intensive.

Curse of dimensionality due to multiple Objectives and multidimensional design space

Each Design point requires multiple physics studies and hence increased computational needs

Estimated simulation requirements based on observed performance in 2021.

https://arxiv.org/pdf/2205.08607.pdf

10

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

11 of 33

Workflow for AI Assisted detector design

11

Design Parameters

Objectives

Physics Events

Detector Simulations

Reconstructed Events

Desired kinematic range

Benefits from rapid turnaround time from simulations to analysis of high-level reconstructed observables

The EIC SW stack offers multiple features that facilitate AI-assisted design (e.g., modularity of simulation, reconstruction, analysis, easy access to design parameters, automated checks, etc.)

Leverages heterogeneous computing

Need to develop end to end pipeline

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

12 of 33

The AID(2)E Project

12

[2405.16279] AI-Assisted Detector Design for the EIC (AID(2)E) (2024)

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

13 of 33

A roadmap for scalable optimization

13

Monitoring — MLOps

Need for better visualizations

Beyond 3D Pareto visualizations

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

14 of 33

The AID(2)E Project

14

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

15 of 33

Project Workflow

15

AID(2)E Pipeline

Thrusts of development

Optimizer

eg. MOBO Algorithm

ePIC Software

Heavy simulations

EIC Analysis

Physics/Detector response

Detector Simulations

EIC-SIM-RECON

Total number of iterations to converge

Computations during an iteration

Distributed Computing

AID(2)E Wrapper

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

16 of 33

So what is AID2E and this boot camp all about

This project is a combination of software engineering and Data Science.
We will go through together a fairly in depth review on how efficient can we run computations when we access to tons of computing nodes (CPUs or GPUs)
We will also get into the theory of optimization and specifically, how does a multi objective optimization is defined in literature.
This is NOT a mathematical course work on optimization but an illustration of how optimization is being utilized and you will be able to better appreciate
The notes and exercises can be found in

16

https://aid2e.github.io/boot-camp-2024/intro.html

Feel free to ask questions….

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

17 of 33

Backups

17

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

18 of 33

Project workflow

18

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

19 of 33

Closure Test 1 – Stress testing SoTA MOBO

19

Acquisition function

qNEHVI – O (M(n + i)^M)^[2]

A “cheaper” function to evaluate as a proxy for the black box function
Identifies points of maximum improvements hence, the name
Scales nonlinearly with iteration, total points explored, design space and objective space.
Partially benefitted by GPU acceleration.

Gaussian Process

O(n³)

The PDF prior distribution, that describes the Design space to objective. This is the surrogate model.
SAAS^[1] priors have been proven to be successful upto 388 design dimensions.
Assumes several design variables has increased importance compared to others
Computational expensive as iteration increases
Benefit from GPU hardware acceleration

Bayesian Sampling from posteriors

NUTS – O (Md^5/4)^[NUTS]

Sample L points from the posterior distribution.
HMC is a popular algorithm
Mainly depends on the Number of objectives and design space dimensions
Has minimal dependence on iteration.
GPU acceleration through JAX backend.

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

20 of 33

Closure Test 1 – Stress testing MOBO

Stress test the SoTA algorithm used for optimization
MOBO stress-testing for problems with increasing complexity (design and objectives) and known Pareto

20

arXiv:2405.16279

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

21 of 33

Closure Test 2: PanDA/iDDS integration

Stress test scalability across distributed resources
Integrate PanDA/iDDS AI/ML service to support MOBO workflow for design optimization

21

PanDA: Production and Distributed Analysis System. Comput Softw Big Sci 8 , 4 (2024)

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

22 of 33

Current Detector Subsystems for optimization in ePIC

22

d-RICH detector at EIC

Design params: Mirror, sensor placement, gas, mirror material

Objectives: PID performance in bins of momentum, cost

Far Forward – B0 System

Design params: z positions of disks

Objectives: Momentum resolution, Acceptance

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

23 of 33

GP as a Surrogate Model

23

Question: What would be the next point to explore from this?

Choose a region?

Optimization problem:

In practice we do not know the True f.

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

24 of 33

GP as a Surrogate Model

24

The task: To minimize. So should we even care on regions which are not minimum?

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

25 of 33

The Acquisition function

25

Define a function that scans through the search space for values of f(x) using the built GP.
Much faster than evaluations.
Carefully choose the next point to evaluate*.
Model inaccurate in region out of interest

*Since evaluations are supposed to be very costly

Widely used Acquisition functions

Confidence Bound
Probability of Improvement
Expected Improvement

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

26 of 33

Confidence Bound

26

Exploitation

Exploration

Hyper parameter

Can now control where the search will happen in subsequent iterations.

Usually, bias is towards the mean. Since it is optimization

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

27 of 33

Probability of Improvement

Choose the point, that has the maximum probability of improvement.

Note: NO consideration of actual value.

27

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

28 of 33

Expected Improvement

Considers the Magnitude of improvement along with its probability^[1]

28

Now, With the suggested point, Start running iterations. Choose the first q points suggested by the Acquisition function.
Run for N iterations
Implement early stopping criterion if necessary

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

29 of 33

The Summary of MOGA Pipeline

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

Now since we discussed all the ingredients for the pipeline here I explain the software stack that was developed for this problem. We made a modular framework to be as flexible as possible.

We first initialise the population. Generate a N set of design points to start with.

We also modify a few of the design points to incorporate some of the reference design solutions. This could potentially accelerate our convergence.

Evaluate the generated design points. Go to next block.

Here we have 2 stages of parallelisation. One is to parallelise the whole design points itself. It runs as N processes. Then each of the design point1 basically runs the Geant4 simulation. This is further parallelised to M threads.

Once the evaluations are done and objectives are extracted. The MOO does its magic.

Sorts and ranks them uses GA to create new set of population to be explored again.

This continues and each time it is called a generation or a call.

Resorces for this project is shown here.

30 of 33

Multi Objective Evolutionary Algorithms

Inspired by Biological Systems.
Semi heuristic in nature.
Quite successful in solving MOO problems.
Embedding constraints relatively easier

30

Swarm Algorithms

Ant Colony optimization

Bees algorithm

Particle swarm optimization

Cuckoo search

Differential Evolution

Cellular Automata

Genetic Algorithms

Default Genetic Algorithm

NSGA

NSGA-II

U-NSGA-III

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

Now since we discussed all the ingredients for the pipeline here I explain the software stack that was developed for this problem. We made a modular framework to be as flexible as possible.

We first initialise the population. Generate a N set of design points to start with.

We also modify a few of the design points to incorporate some of the reference design solutions. This could potentially accelerate our convergence.

Evaluate the generated design points. Go to next block.

Here we have 2 stages of parallelisation. One is to parallelise the whole design points itself. It runs as N processes. Then each of the design point1 basically runs the Geant4 simulation. This is further parallelised to M threads.

Once the evaluations are done and objectives are extracted. The MOO does its magic.

Sorts and ranks them uses GA to create new set of population to be explored again.

This continues and each time it is called a generation or a call.

Resorces for this project is shown here.

31 of 33

31

Elitist Non-Dominated

Sorting Genetic (NSGA)

Population

@(t+1)

Population

@(t)

Offspring

Front

[1] Deb, K., et al. "A fast and elitist multiobjective genetic algorithm" IEEE transactions on evolutionary computation 6.2 (2002): 182-197.

This is one of the most popular approach

(>35k citations on google scholar), characterized by:

Use of an elitist principle
Explicit diversity preserving mechanism
Emphasis in non-dominated solutions

The population R_tis classified in non-dominated fronts.

Not all fronts can be accommodated in the N slots of available in the new population P_t+1. We use crowding distance to keep those points in the last front that contribute to the highest diversity.

The crowding distance d_i of point i is a measure of the objective space around i which is not occupied by any other solution in the population.

This is to illustrate

Binary Cross-over

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

Let me now briefly explain about the Elitist Non-Dominated sorting Genetic algorithm. In short this includes a GA which creates new set of design solutions from a population a sorting method to assist in choosing the parent population for mating.

Let us first look into GA. Consider a initial set of population, These could the set of population/ design solutions in the problem. 2 parents out of the efficient population is chosen for a crossover.

As in here we have the two parents from the healthy population. Blue is a parent which that one set of the design parameter. And the red has another set. Then there is a crossover that happens which creates a new offspring. And occasionally we have a mutation in the gene. Therefore, a final offspring population is born. This is the basic idea of GA

Now, In order to first select out healthy parents we need to have a sorting algorithm, Elitist Non Dominated Sorting is a method that performs non dominated sorting of the solutions.

In this picture there exist a population Pt at a time t. all these population are evaluated with a fit function. In our case we have the MOO so we have the evaluated objectives. We also have offsprings from previous calls, These are also evaluated for the objectives. Therefore we have a total of Rt parents. Now these points are sorted as fronts in the objective space. Because we are minimising we the first front corresponds to the one here, followed by f2 and f3. And this sorting happens until the population size is reached.

In order to preserve diversity a crowding distance metric here is used to choose the points from the same front.

******* NOTES FROM CRIS ****************

For the mutation, in the context of real-parameter optimization, a simple Gaussian probability distribution with a predefined variance can be used with its mean at the child variable value.
In other more recent studies, theoretical optimality conditions (such as the extent of satisfaction of Karush-Kuhn-Tucker (KKT) conditions) are used to determine the termination of a real-parameter EO algorithm. These conditions are first-derivative tests for a solution in nonlinear programming to be optimal.

32 of 33

MOEA or MOBO ?

32

Has been widely used for solving MOO problems
population /off spring — diversity —
Relatively easier to implement
Complexity relatively easy to compute
Ideal — Cost of computing “cheap”
Successful with large Design and Objective parameters
No Map : “Design” “Objectives”

Has been around for a while, gaining popularity
Sequential Strategy — global minimization
Relatively harder to implement
Complexity relatively easy to compute
Ideal — simulations can be heavily parallelized
Currently, Not recommended beyond 4-5 Objective parameters
Can Map : “Design” “Objectives” — Fast simulator can be built

MOEA

MOBO

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC

Here is how the optimisation workflow works. We use NSGA2 to suggest new design points. We simulate the new design point in the Geant4 based framework (Fun4All). Once that is done we perform an analysis on the result. So for the problem in hand we have 3 objectives namely the momentum resolution, theta resolution, Kalman Filtering inefficiency, This is a measure on the ability of tracking reconstruction. Without much math, these resolution measures are a ratio with respect to the baseline detector that is currently proposed in ECCE. The errors on these fits are also promptly carried over into the calculation.

As mentioned the problem also has Geometric constraints, The outermost barrel layer could have the radii less than 51cm the outer vertex layer should be less than 15 cms, the 4th barrel layer has to be less than 45 cms. The forward most Z disk has to be less than 125 cms, Minimum distance between any 2 layers of the detector has to be greater than or equal to 1 cm.

Finally once we have the optimal set of solutions then we can validate our solutions against other objectives such as the vertex resolution, reconstruction efficiency etc.

33 of 33

Far Forward Updates

33

Problem	Optimize the momentum resolution subject to the non-homogenous Magnetic field and to increase occupancy at B0 ECAL.
Objective Space = 2	Objective Parameter	Remarks
	Momentum resolution (𝑝_T)	Momentum range of 80 - 100 GeV/c is of interest and specifically proton tracks
	B0 ECAL acceptance	Ratio of number of tracks before 1st tracking disk to the number of showers detected by B0ECAL
Design Space = 4	Design Parameter	Range [cm]	Least count for variation [cm]
	Z₁	583.0 - 630.0	1.0
	𝛥Z₂, 𝛥Z₃, 𝛥Z₄	10.0 - 40.0	1.0
Constraints = 2	Z₁ + ∑_i=2,3,4 𝛥Z_i ≤ 685.5 cm
	\|Z_i+1 + Z_i\| ≥ 10.0 cm

AID2E Boot camp July 8 - 19 2024

AI Assisted Detector Design for EIC