1 of 14

High Energy Physics Statistics Serialization Standard (HS3)

Carsten Burgard, Jonas Rembser

Robin Pelkner, Cornelius Grunewald, Oliver Schulz, Jonas Eschle, Giordon Stark, Alexander Held, Matthew Feickert, Lukas Heinrich

2 of 14

A brief story (stolen from Lukas Heinrich)

3 of 14

But how can it be done?

  • ROOT has be storing statistical models (“the likelihood”) for decades
  • RooWorkspace is a success story, contains all required information more
    • Distributions (pdfs)
    • Observations (data)
    • Parametrizations
    • …

in a binary format (“.root”-files), fully forward compatible (in theory)

  • But the statistical universe is much larger than just ROOT

… and many more!

many high-level frameworks are built on top of RooFit, e.g. CMS combine

4 of 14

What do other people do?

  • pyhf has successfully established a JSON standard for HistFactory-style likelihoods
    • HistFactory: histogram template stacks
  • Very well received
    • dozens of models already published on HEPdata
    • used by many standalone publications
    • human-readable & -writeable
  • Only downside: restrained to HistFactory-like models
    • no generic analytic templates functions
    • no reparametrizations (EFT)

5 of 14

Can we generalize what pyhf has already achieved?

  • Obvious use-cases:
    • unbinned fits
    • analytic shapes
    • reparametrizations
    • unfolding
  • Simple recipe:
    • use a pyhf-like JSON text file (as similar as possible)
    • be feature complete with respect to RooWorkspace
    • get in touch with all framework developers
  • But is it really that simple? Obvious challenges:
    • infinite set of possible functions, can we hope to encompass them all?
    • unbinned data can become huge, can we keep the format readable?
    • there are many different frameworks, can we be generic enough?

6 of 14

How can we hope to encode all possible mathematical functions?

  • we do it all the time in mathematical formulae
  • as long as we can agree on function and distribution names, concrete implementations can be left to the framework developers
  • not every framework needs to support every function
  • need a standardized catalogue of function names and their meanings

no code or implementation here, let’s stick with the math!

7 of 14

Storing data

  • pyhf stores the observations alongside the model
    • not ideal, several datasets might go along with the same model
  • there are many different types of data to store
    • histograms in one or more dimension
    • unbinned list in one or more dimension
    • sparse or awkward arrays?
  • need to also allow for different options how to store datasets
    • can also allow storing data points in external files (JSON-LD?)
    • still left to be figured out!

8 of 14

Different frameworks

  • Actively getting all interested parties to the table
    • ROOT: HS3 draft 0.1 implementation exists since ROOT 6.26, HS3 draft 0.2 implementation scheduled for release in summer
    • pyhf: occasional meetings, have established roadmap
    • BAT.jl: HS3 draft 0.2 implementation ongoing
    • zfit: included in meetings, discussions ongoing
    • everyone else is invited to join the discussions!
  • Some informal meetings over the last weeks to discuss draft
    • future meetings will be announced via hep-statistics-serialization-standard mailing list

9 of 14

Draft 0.1

  • Feature complete w.r.t. Higgs workspaces
    • both current and historic
  • Using “computational graph” to structure JSON
    • functions and pdfs are nodes, �parameters are leaves
    • works for binned and unbinned
  • Currently implemented in ROOT
    • very closely aligned to ROOT in structure�and also in nomenclature
  • special Pdf type “HistFactory” close to pyhf JSON for HistFactory models
    • not identical, working on resolving differences

converter is available!

10 of 14

Draft 0.2

  • Robin Pelkner joined the team as a Master student

Project: Round-trip between ROOT and BAT for Higgs discovery workspaces

  • Will provide stress-test for current draft & generate new input
  • BAT uses Bayesian approach instead of frequentist (different concepts)
  • functional implementation instead of object-oriented

“data”

“ModelConfig”

NEW

11 of 14

A brief look into draft 0.2: Envisioned top-level keys

  • distributions (formerly “pdfs”)
  • functions: helper components for distributions
  • observations (formerly “data”)
  • likelihoods: map pdfs to data
  • domains: allowed ranges of parameters
  • estimates: snapshots of parameters, e.g. model predictions, good starting values for a fit, …
  • priors: used for bayesian inference
  • analyses: groups of likelihoods to be considered (RooSimultaneous), lists of POIs and NPs (ModelConfig)
  • metadata: arbitrary meta-info

12 of 14

Lists of functions and Pdfs

  • HistFactory channel
  • Gaussian
  • Poisson
  • Exponential
  • Polynomial
  • Bernstein Polynomial
  • …

With this list, we can already cover a significant chunk of interesting models!

13 of 14

Timeline and open items

  • Hope to iron out core parts of the standard in 2023
    • Fix remaining bugs in ROOT implementation, add versioning
    • First round-trip ROOT→BAT→ROOT
    • Full harmonization with pyhf

Open questions

  • What are good ways to store data?
  • What type of meta-info do we need to include?
  • Which other toolkits do we need to include in the development?
  • Is anything missing in order to make this attractive for end-users?
  • What would be requirements from the side of HEPdata?

14 of 14

Conclusions

  • HEP Statistics Serialization Standard wants to make statistical models
    • persistent
    • interchangeable
    • modifiable
    • readable
  • Things are moving
    • hope to get a first complete version by the end of 2023
  • Input from all sides is highly welcome
    • Now is the time to speak up & get in touch

Links