1 of 21

Unfolding Data with Machine Learning

1

Vinicius Mikuni

vmikuni@lbl.gov

vinicius-mikuni

2 of 21

Unfolding

2

What we measure

What we want

3 of 21

Unfolding

3

Source: CMS-PAS-TOP-20-006

Traditional methods for unfolding use histograms

4 of 21

Unfolding

4

Source: MITHIG-MOD-21-001

Traditional methods for unfolding use histograms

ai = Rijbj

Rij is the response matrix: P(observed in bin i | true in bin j)

5 of 21

Unfolding

5

Source: MITHIG-MOD-21-001

Traditional methods for unfolding use histograms

ai = Rijbj

Rij is the response matrix: P(observed in bin i | true in bin j)

  • Traditional unfolding is all about inverting the matrix Rij

6 of 21

Unfolding

6

How to define the optimal binning?

  • Choice depends on the distribution and phase space
  • Need to compromise when combining results from different experiments

7 of 21

Unfolding

7

How to include multiple distributions?

  • Histograms are hard to scale: curse of dimensionality
  • Unfolding uncertainties can be reduced using additional observables

How to define the optimal binning?

  • Choice depends on the distribution and phase space
  • Need to compromise when combining results from different experiments

8 of 21

Unfolding

8

How to unfold distributions that are not defined for each event?

  • Moments of distributions
  • Energy Correlators

How to include multiple distributions?

  • Histograms are hard to scale: curse of dimensionality
  • Unfolding uncertainties can be reduced using additional observables

How to define the optimal binning?

  • Choice depends on the distribution and phase space
  • Need to compromise when combining results from different experiments

9 of 21

Going beyond histograms: Omnifold*

ML is used to define a method for unfolding that is unbinned and can use multiple distributions at a time

2 step iterative approach

  • Simulated events after detector interaction are reweighted to match the data
  • Create a “new simulation” by transforming weights to a proper function of the generated events

Machine learning is used to approximate 2 likelihood functions:

  • reco MC to Data reweighting
  • Previous and new Gen reweighting

9

* Andreassen et al. PRL 124, 182001 (2020)

For unfolding using invertible networks see:

  • SciPost Phys. 9 (2020) 074 e-Print: 2006.06685

10 of 21

Omnifold

10

Increasing adoption by experimental collaborations: ATLAS, CMS, H1, T2K, Aleph

11 of 21

Omnifold

11

Reco level

Generator level

MC

MC

Data

Data

12 of 21

Omnifold

12

Reco level

Generator level

MC

MC

Data

Data

Step 1:

  • Train a classifier to separate data from MC events
  • Reweight reco level MC with weights:

W(reco) = pData(reco)/pMC(reco)

Iteration 1

13 of 21

Omnifold

13

Reco level

Generator level

MC

MC

Data

Data

Step 2:

  • Pull weights from step 1 to generator level events
  • Train a classifier to separate initial MC at gen level from reweighted MC events
  • Define a new simulation with weights that are a proper function of gen level kinematics

MC reweighted

W(gen) = pweighted MC(gen)/pMC(gen)

Iteration 1

14 of 21

Omnifold

14

Reco level

Generator level

MC

MC

Data

Data

Start again from step 1 using the new simulation after pushing the weights from step 2

  • Guaranteed convergence to the maximum likelihood estimate of the generator-level distribution when number of iterations go to infinite
  • In practice, less than 10 iterations are enough to achieve convergence

​

Iteration 1

15 of 21

Omnifold

15

Reco level

Generator level

MC

MC

Data

Data

Start again from step 1 using the new simulation after pushing the weights from step 2

  • Guaranteed convergence to the maximum likelihood estimate of the generator-level distribution when number of iterations goes to infinite
  • In practice, less than 10 iterations are enough to achieve convergence

Iteration N

16 of 21

Part 2

Applications

16

17 of 21

OmniFold dataset

17

q

q

Only consider Z decaying to neutrinos: mostly a single jet per event.

We are going to unfold 6 jet substructure observables simultaneously using OmniFold

18 of 21

OmniFold dataset

18

q

q

Only consider Z decaying to neutrinos: mostly a single jet per event.

Phys. Rev. Lett. 124, 182001 (2020)

19 of 21

THANKS!

Any questions?

​

19

20 of 21

Backup

20

21 of 21

OmniFold dataset

21

q

q

Only consider Z decaying to neutrinos: mostly a single jet per event.

  • Observables:
    • Jet mass
    • Particle Multiplicity
    • 𝝉21 = 𝜏2 / 𝝉1 see Energ. Phys. 2012, 93 (2012).
    • Jet width (𝝉1)
    • log ρ = 2 log MSD/pT
    • Momentum fraction zg after using Soft Drop