PBD - Data Reduction

Colin Bischoff and Reijo Keskitalo

Data Quality

Data Quality performs “quick look” data analysis that will lag the real time data collection by ~24 hours, with the following goals:

• Identify problems with the operating instrument that are not noticed during real time operations.
• DAQ is responsible for the control / real time monitoring system. They will primarily monitor telescope mounts, housekeeping, etc. Will not do mass processing of bolometer data.
• Build database of data quality statistics to be used for cuts and other purposes (weather, functioning detectors, noise model).
• Make daily maps that are inputs to transient detection activity.
• Ideally serves as first stage of data reduction.

​

24 hour cadence requires Data Quality activities to run on site for South Pole. Expect that DQ activities for Chilean data will happen in North America.

​

2

Data Quality

Monitoring data quality will involve a combination of automated checks and human inspection of raw (or nearly raw) data. Draw on methods from current experiments for effective and efficient viewing of large datasets.

​

​

3

Example:

BICEP/Keck “reduc plots”

2500+ detectors for ~1 hour of observation

​

Could enhance viewer with access to DQ database

Experiment Characterization

The goal of this subsystem is to analyze calibration observations and CMB data in order to extract instrumental parameters that are necessary for the full CMB-S4 data reduction.

​

These include: detector noise model, pointing model, beam shapes, polarization angles, bandpasses, measured instrumental systematics (crosstalk, sidelobes, etc).

​

Experiment Characterization needs to develop a correspondingly wide range of analysis pipelines, but they will be built on a common foundation of data reduction tools that can be shared with the Data Quality and TOD-to-Maps subsystems.

​

There will be a large amount of Experiment Characterization activity near deployment, but it will continue throughout CMB-S4 operations.

​

​

4

TOD to maps

The TOD-to-maps pipeline consists of

​

1. Data selection
2. Noise-weighting and calibration
3. Filtering
4. Solving for sky map
5. Deriving statistical description of maps (signal and noise)

​

Steps 3 & 4 may be combined in certain mapmaking implementations. We propose to make a formal downselect of the mapmaking method by CD-2. Current simulations run with traditional filter-and-bin but minimally biased, memory and compute-intensive mapmaking may be better-suited for CHLAT data analysis.

5