Group Update Spring 2020
Jonathan Nikoleyczik
1
PdfMaker
2
EFT Uncertainty
3
Previous Slides
4
Asymptotic Approximation to Discovery Significance
5
Fitting the asymptotic approximation
6
Cowan’s: ½ (δ+ꭓ2)
Floating Normalization:� a(δ+ꭓ2)
Floating Normalization and Relative Delta function:� a(bδ+ꭓ2)
EFT Uncertainty - Energy only discrimination
7
EFT Uncertainty limit curves
8
Adding PLR to EFT Uncertainty code
9
Energy only PLR of LZ
10
Some Cross Checks
11
Limit Curve
12
Reasons for discrepancies:
Different threshold esp important at low masses
Less discrimination btw ER and NR (dominates effects at high masses)
More automatic plots
13
Red: Fit to the S+B or B-only model
Blue: Box and whisker plot of all toy Unconditional fits to the S+B or B-only models
Contours are now actually 1, 2, and 3 σ
14
This was harder than I thought it would be... General multidimensional quantiles.
Note: here the levels are the 2D gaussian equivalent areas. Not 68%, 95%, 99.8%. Rather 39.3%, 86.4%, 98.8%
Updated Pull Distributions to be more useful
15
Solid: Unconditional fits
Dashed: Conditional fits
Blue: B-only
Red: S+B
Next Steps
Last few plots to be added:
Integration into LZStats:
Documentation...
16
Current Plot Suite
17
Model split out by component
2D fit results (generated values on x-axis and fit value on y-axis)
Current Plot Suite
18
Full model with X and Y projections
Pie Charts in any given dimension
3D models (S1, logS2, time)
19
Still working on the formatting (doesn’t work 100% of the time)
Planned Plots - Knut’s Thesis
20
Profile LL vs POI (easy) along with 90% limit vs POI (harder)
Profiled NP vs POI along with toy MC uncertainty
LZStats features cont’d
21
NMM workspace - Xe 124 counts issue
22
New LZStats features
23
-2 sigma issue - new problem
24
Old method
New method
Checks
25
Toys
26
A random S+B toy at high POI test value unconditional fit
A random B-only toy at high POI test value unconditional fit
Toys
27
A random S+B toy at high POI test value conditional fit
A random B-only toy at high POI test value conditional fit
Observed datasets
28
Unconditional fit
Conditional fit
Simplifying LZStats
29
30
POI
P-value
Pink and red: r_tot->GetLowerLimitDistribution()
Example plots
31
Yellin Method on real data
32
Implementing a fix to the -2σ problem
33
Lower Limits
Upper Limits
Median and ±1 sig
UL
Prepping LZStats pregeneration of plots for SR1
34
Yellin Max Gap method
35
Number of signal events
Red - Poisson Probability of getting 0 events in a gap
Black - Observed probability of signals inside the “maximum” gap
Maximum Probability
Max Patch Method
36
LZStats Updates
37
Goals for the semester
38
Recent PLR work
39
Run4 EFT workspaces in LZStats
40
P-value distributions
41
Combining p-values of two models
Remember:
Could try to do something like:
This is done on the right only median p-value shown on plot
Real problem with this method: the datasets are generated from different models!
42
The real solution to the WDNM case
43
LZStats Improvements
44
Combining p-values of different runs of LZStats
45
Benefits of adding time dependence to ER searches
46
Errors on limits based on small number of toys
47
Morphing PDFs
Varying electron lifetime as a function of time from 300 us to 900 us lifetime�Equal numbers of 40 GeV WIMPs and Rn222 for comparison
48
49
Increasing the initial concentration of Ar37
Results?
50
Time dependent workspaces
51
An example workspace
Only B8 and Rn222
52
Running Inside LZStats
53
Time dependent workspaces
54
How time dependence was done in LUX
55
Analytical Band Modeling - Method
56
Standard Deviation
Mean
Skew
Norm
Analytical Band Modeling - Results
57
Black line - “True” band
Histogram - model
Shape Varying NPs
58
Tritium results
59
LZLAMA -> LZStats
60
Pre-generation of toys
61
Tritium impact on LZ sensitivity
62
Tritium PDF
Shape Varying NPs
63
Ex: MDC3 ER band divided by Projected detector band
LZStats QoL improvements
64
Understanding Global Observables
65
Projected Sensitivity vs Livetime
66
Sensitivity study issues
67
If we have time - Moment Morphing of PDFs
68
LZStats Work
69
5/27/20
Test workspace results - Hybrid Calculator
70
Global observables
Nuisance parameters
5/27/20
Test workspace results - Frequentist Calculator
71
Global observables
Nuisance parameters
5/27/20
LZStats work
72
5/20/20
The Model
RooAddPdf::EventModel[ mu_sig * pdf_sig + mu_bg1 * pdf_bg1 + mu_bg2 * pdf_bg2 ]
BG2
BG1
SIG
BG2 overlaps significantly with sig but BG1 is well resolved
5/20/20
Constraint functions
RooGaussian::constraint_bg1[ x=mu_bg1 mean=a_bg1 sigma=3 ]
BEFORE LZSTATS: mu_bg1 = 10 L(-100 - 100)
AFTER LZSTATS: mu_bg1 = 42.9237 +/- 2.93583 L(13.5654 - 72.282)
a_bg1 = 50 C L(-INF - +INF) - NOTE:Unchanged by running LZStats
a_bg2 = 1 C L(-INF - +INF)
RooGaussian::constraint_bg2[ x=mu_bg2 mean=a_bg2 sigma=3 ]
BEFORE LZSTATS: mu_bg2 = 5 L(-100 - 100)
AFTER LZSTATS: mu_bg2 = 4.3437 +/- 1.53807 L(-11.037 - 19.7244)
a_bg2 = 1 C L(-INF - +INF) - NOTE:Unchanged by running LZStats
5/20/20
Best fit results
The mean here is way off of the constraint function. Should be 50! Somehow using observed as new mean?
5/20/20
Comparing bands between ALPACA and LZStats
76
5/13/20
Global Observables Cont’d
77
More LZStats improvements
78
5/6/20
Globals vs best fits
This just feels wrong...
79
Chosen global observable for a toy
Best fit rate for a toy
5/6/20
Global Observables
80
Gaussian constraint (red)
Poisson constrain (blue)
Mean value (green)
5/6/20
Bugfix in LZStats
81
PLR on MDC3 data
82
Fit values
Looking at the best fit values of nuisance parameters for many different toys and we see this
Where does the structure come from? Why do so many toys get best fits near very extreme values of this parameter?
Because it’s unconstrained? Deeper RooStats bug?
83
4/21/20
More PLR work
84
4/14/20
85
Unweighted
86
Weighted
PLR Updates (MDC3 focus)
87
4/8/20
MDC3 WS data + model (“Xenon1T” style)
Unconstrained fit to the data
Using hist2workspace in LZStats
88
4/1/20
Quantifying LZStats speed
89
4/1/20
| vanilla_wimp | hist2workspace | ||||
| Normal NLL function | | Normal NLL function | | ||
| Average Minimization time | Fraction with fit errors | Comment | Average Minimization time | Fraction with fit errors | Comment |
Minuit migrad | 3.20 ± 1.00 | 0.00% | | 45.4 ± 18.9 | 2% | |
Minuit migradimproved | 11.08 ± 3.50 | 100.00% | Status 1? | 42.5 ± 45.4 | 100% | All fits gave status 4000 (failed?) |
Minuit2 migrad | 2.24 ± 0.70 | 0.00% | | 11.0 ± 2.9 | 5% | |
Minuit2 simplex | 0.40 ± 0.12 | 100.00% | Status 1? | 1.6 ± 0.2 | 76% | Most have status 5 |
Minuit2 minimize | 2.23 ± 0.70 | 0.00% | | 11.3 ± 3.2 | 6% | |
GSLMultiMin bfgs | 3.54 ± 1.12 | 0.00% | | 15.8 ± 7.4 | 0% | |
GSLMultiMin bfgs2 | 2.42 ± 0.77 | 0.00% | | 11.4 ± 2.3 | 1% | |
GSLMultiMin steepestdescent | 7.24 ± 2.28 | 0.00% | | 35.5 ± 2.3 | 0% | |
flamedisx
90
4/1/20
LZStats Work
91
3/25/20
Foam Cutting, etc.
92
3/18/20
Foam
93
Foam cutting at PSL
94
3/11/20
95
Work On Site and LZStats
LZStats:
96
2/26/20
Underground Work
97
2/19/20
LZStats
98
PLR Work
99
2/12/20
On site work
100
2/12/20
Work on site
101
1/23/20
LZStats
102
1/23/20