LSSTC enabling science 2020 broker workshop

Minutes and summary document

Day 1

Session 1

Moderators: Gautham Narayan & Paula Sánchez-Sáez

15.00 - 15.10 UTC Introduction by Anais Möller (10 min)

First follow-up after the Seattle broker meeting in June 2019 and the broker session at PCW 2020.

First part: current status and requirements

Second part: Developments triggered by the first workshop and reflections from the different users and collaborations.

15.10 - 15.30 UTC LSST Data Management, Eric Bellm & Rubin Obs. Data Management (20 min)

2020 PCW Broker Presentations

Main document for alerts: LDM-612

Main place to ask for questions on alerts: the forum at community.lsst.org (see especially the okAlerts & Brokers topic: https://community.lsst.org/c/alertsbrokers/46 )

Simulated SSO available around November 15th, DIAObject timeseries features in progress.

Pre-operations (interim data facility) will be on a cloud provider. Exercise began. It is not clear where operations will happen.

Problem of cutout size: still investigating the different options to maximise the science output without impacting too much the bandwidth. Note that data right holders can retrieve cutouts by 24h of alert emission independently of the live stream.

Real bogus: how to detect efficiently artifacts? Likely to use the Zooniverse project to develop training samples.

Data right holders should also look at the Rubin Science Platform (RSP) that will provide access to the data. ‘Next-to-data’ analysis: jupyter notebook, API to access tabular and image data.

Can brokers interact with the RSP?

Information on derived data products: see RDO-013

Q: will the pre-operations cloud provider be announced before Dec 15th? A: We know it is Google. Press announcement soon.

Q: When broker-RSP integration will start? A: DM would be setting up API during the Data preview 1 (ComCam data). Note that Data preview 0 will be using DESC DC2 data, and Data preview 2 will be using LSSTCam data.

Q: Are there any plans to do transient/variable classifications apart from the brokers (i.e. acting on several years worth of data)? A: the LSST project will only characterize, rather than classify

15.30 - 15.43 UTC ZTF lessons learned, Matthew Graham (13 min)

To all brokers: contact us to listen to the alert stream!

ZTF is 10% LSST - scaling is not an issue. In addition, end-users do not request the full stream, but only a small fraction (will still be true for LSST).

ZTF is moving to a fully automatized classification system (without human in the loop, that brings latency, inefficiency and non-repeatability)

ZTF fields million queries/day. Important to attach history to alerts to reduce the need to retrieve data each time one runs an analysis (or have a strong server!). Though, there are too many ways to retrieve the data: need to reduce the number of repo/code/etc. to one single that everyone can use.

Concept of the one-stop-shop.

ZTF-II: Oct 2020 to Sep 2023.

Q: in ZTF, which request is more common/useful: 1) requesting specific transient type, or 2) requesting transients with generic features? (e.g., peak mag range, duration above detection, etc …) A: ??

Q: What will be the lookback time for the forced photometry service? A: lookback time will be available through forced photometry. All public data (or proprietary if you have the appropriate access rights)

Q: How can we fetch the forced photometry for interesting alerts?

Q: Will the format of DR light curve change? A lot of fields available through IRSA/API are missed there now.

Q: Are there differences to be expected between ZTF II and ZTF I alert stream? Does a new process need to be started to receive it?

Q: will ZTF-II still split the streams into public and private ones? A: Yes, but 50% of the observing time is now devoted to public surveys

Q: Looks like the ZTF alert schema has not changed in over a year.

https://github.com/ZwickyTransientFacility/ztf-avro-alert

Will there be schema changes when ZTF-II starts to include forced photometry? A: Yes, I believe so but hopefully minimally

15:43 - 15:48 UTC Requested talk: Hopskotch, Mario Juric (5 min)

Focus on multi-messenger astronomy (MMA)

Link to SCIMMA technologies. SCIMMA will provide an MMA archive.

Hopskotch: make reliable streams simple.

Q: Who is deciding on the MMA archive that SCIMMA will provide? Will it be a community discussion or it already exists? A: there are mo

15:48 - 15:53 UTC Short break (5 min)

Session 2.

Moderators: Federica Bianco & Nina Hernitschek

15:53 - 16.18 UTC Discussion: alert generation (25 min)

Notes for Breakout room #1

Main Q: “What types of xmatch info should be in the alert from LSST?” but also what the brokers does in terms of xmatch for the user

LSST only matches to known SS objects and the LSST DR catalog
the xmatches would be to provide LSST Object IDs and on-sky distances; information on the objects is then available, starting from the ID, via query to the Science Platform (e.g., to the API Aspect TAP service).
exactly how xmatch is done and “nearest” objects are defined is TBD

What types of xmatch are there?

types of xmatch: (1) to published catalogs like PanSTARRS and VLASS (ongoing VLA radio all sky survey, very likely that full catalog and image will be available before LSST starts), (2) to a users own favourite sources and (3) “dynamic xmatches” like TNS where new objects appear in real time.

What’s the bottleneck?

the quality of the input data: quality of astrometry, confusion limits
then, the algorithm that does the xmatch
the need to xmatch to large, low-z objects in addition

Are there issues with xmatch to proprietary data sets?

potentially; e.g., Lasair is envisioning “LSST users’ and “others” to enable access to the LSST DR datasets via Lasair

Alerts-related Q&A: https://community.lsst.org/c/sci/data/34

Alternative sets of notes - Breakout session #1 -- cross-match topic.

Fed Bianco moderator

Roy Williams - See notes above [I, Gizis, missed his comments ]

What is the bottleneck? Is it data access or technical?

Steven - quality of input catalog data is bottleneck. Astrometry, Star/Galaxy separation. Need ALGORITHM for cross-match. Confusion (galaxies), proper motion (stars) are issues.

Examples of huge galaxies like M31. Three nearest sources in PS don’t include.

Q: Is proprietary data problem solved?

A: For now assume summary information and as much detailed information should be available worldwide. Haven’t thought about it in detail, bulk will be public. Do envision two different types of users.

Q: (Jha): I am a little bit confused by the topic “included in the alerts” — I thought cross-matching would be a job for the broker? Is the idea that some of this would be pushed upstream to the LSST alert, or are we talking about broker cross-matching?

Thought of Project info to the brokers. Then this is a Project issue. If there is something missing that is scientifically interesting then inform the project soon. The Project is only matching its alerts to its own catalogs, like the deep coadd catalog that goes deeper than any one exposure. [and known Solar System objects]. Object IDs are included, data is subject to Data Rights.

Question about forced photometry? Project has not extensively analyzed that capacity but it is in the books.

Smartt: Initial alert will not have forced photometry. Within 24 hours the brokers can request forced photometry for a certain number of objects. [perhaps throttled.] The second alert will have forced photometry going back 30 days. Question is how long does forced photometry. Second alert might be next day, might be 7 days, depending on cadence. Forced photometry should be available within 12-24 hours.

Question to clarify. I thought forced photometry is for all alerts. Clarification: All alerts will have forced photometry in 2nd alert. If you want within 24 hours, the broker needs to request.

Cross-matches to radio, other wavelengths. ALLWISE in IR, X-Ray. Gives useful information. This is the responsibility of the broker.

Redshift of host galaxies. Photo-z of host galaxies is an LSST product. Will LSST include such information in the alert packet if cross-matched? A: Cross-match is object ID, so redshift can be looked up in catalog with data rights, same for host galaxy photometry. DLR information? Melissa Graham is currently working on a proposal to include host-galaxy size information in the cross-matching algorithm. Right now it is just based on on-sky distance.

Gregory: Will provide a few matches, not just one object.

Notes for Breakout room #2: Full stream vs lightweight only-metadata stream.

What metadata means? What to include? What are the options?

Lightweight streams: alert content is unchanged, but there will be a two-step delivery.

Q: what is the relevance of (relatively large) cutouts? Still on the early stage of it.

Q: what are the use-cases for having a lightweight stream?

A: Without the bottleneck of cutouts, history, comprehensive database records, many more users can receive the stream.
A: Many people are discarding alerts based on positional cross-match for example, not even seeing or using the full alert data.

The audience to get full packet content for a single high-demand packet could be a few hundred people, which is unlikely to be a technical hurdle

Lightweight stream in the meantime is breaking a few things (e.g. guarantee to have one-time delivery, ...).

The project (+SAC) should investigate how to feed all brokers, rather than cutting on the total number. Encouraging inter-broker collaboration rather than cutting the stream.

Q:what is the burden to maintain the ZTF alert system? What impact for LSST and having multiple streams? A: Several places for alerts: IPAC -> UW -> brokers. When no problem, fairly minimal to maintain (~0.5 FTE), but not trivial when problems start.

Science collaboration (SC) use cases I

Notes for Breakout room #3:

How many brokers will be supported for full-stream?

The requirement is minimum 5 but looking into supporting more. However, this possibility won't be known before full proposals are due for. Maybe for the next workshop?

16:18 - 16:30 UTC AGN, Paolo Coppi (12 min)

LSST will detect hundreds million AGN in the main survey area!

Q: Are these sources you expect to confidently detect and do science with

or do you think they will be in alerts but not confidently classified ? What are the timescales at which these might be classified? A:

16:30 - 16:42 UTC Dark Energy, Kara Ponder (12 min)

Using SN Ia for cosmology, but planning to follow-up any type of SNe.

No need for immediate information. Bulk alerts received within 24h, and processed by brokers is what is needed

DESC is running evaluation for brokers and metrics (with some broker teams to define inputs).

Q: What do you mean you want to evaluate CPU usage of brokers? A: it is more a question of reproducibility for testing purposes (example inside a light docker container).

Q: how many teams are thinking about 3rd party reproducibility (i.e., them being able to spin up a version of your broker, or providing such a service)? A: Mario, this is built into the ANTARES design (provenance tracking, containerized versions, etc.) ; also A: Fink’s design allows this reproducibility. All our framework is open source, with version control and can be deployed in any cloud.; also A: Thanks! Just to focus my question a bit -- will it be (or already is?) possible for someone to download (say) a container of your broker and run it by themselves (say, with a simulated stream to understand selection function)?; also A: Each ampel version can also be spin up at a later time. Expanding the question a bit, an as hard or harder question for reproducibility is whether all input streams will be available. All decisions in Ampel are logged, but to fully "rerun" a science season requires all input streams to be accessible, including e.g. follow-up decisions.; also A: That's in our development plan. We aren't making containers available yet, but will, for exactly the reasons you bring up.; also A: @mjuric to complement @Anais Möller answer, we have series of containers (to simulate alerts via Kafka, listen to and process alerts by Fink, and redistribute alerts to users). There is some documentation to use them, but it might need a good clean up at some point if there is someone else other than us using it. Just ping me whenever.

Find more information from DESC here: https://docs.google.com/document/d/13Kwxcl1DeUDwH94iWeqqYcEsEm1PsOAVE2R1nxFTMdI/edit?usp=sharing

16:42 - 16:55 UTC Informatics & Statistics, Ashish Mahabal (12 min)

Q: could ISSC help the broker community to design and test benchmarks? A: Yes + The ISSC is very interested in helping to coordinate relevant community efforts so anything involving statistics and machine learning .

Link to LSST forum: https://community.lsst.org/c/sci/statistics/41 linked

For Slack: #issc-ask-the-issc

Q: Can the sequence of classification updated be used for anomaly detection? Do brokers plan on archiving it and providing it? A: in Fink we will store all anomaly scores with time stamps and they will be accessible to the user.

The ISSC web site archives videos from past webinars. The last one was by Anais Moller on Bayesian neural networks. See the "Videos" link here: https://issc.science.lsst.org/

16.55 - 17.25 UTC Long break (30 min)

Session 3.

Moderators: Emille Ishida & Matthew Graham

Science collaboration (SC) use cases II

17:25 - 17:37 UTC Galaxies, Harry Ferguson (12 min)

More interested in long-term classification than real-time identification. Several epochs needed.

Interest in: Novae, Mira variables, Microlensing, emission-line sources, Supernovae, High-Redshift signposts, ...

Q: Will DCR (Differential chromatic reflexion) related changes trigger alerts (that will be sent out, not flagged as bogus)? A: Trying to build models. They will be flagged and minimised as the survey goes on.

Q: How will you know they are spurious in the early survey? E: depends what you mean by “know”. dipolar effects are by definition inconsistent with a subtraction that produces a clean point source--hence, a high spuriousness score. It is a separate question whether there is real 5-sigma variation + a DCR effect that produces the dipole--I think predicting that is essentially the same as correcting for the DCR effect in the first place. MWV: In theory it seems like it should be. I can imagine that in practice it takes more information to build a DCR-informed template than it does to identify that one might have concerns about an object. E: Yes. In the early survey I expect our ML models to simply be trained to recognize clean point sources. MJ: We'll have to be careful not to be _too_ good at it & accidentally remove moving object trails. MWV: But an overall takeaway for Brokers might be that that orientation of the dipole moment of a source with parallactic angle , combined with the dipolar strength itself might be of particular interest. E: @mjuric we won’t remove anything, just score it. (... other messages)

Q: Although brokers may not be critical for the Galaxies SC science cases, brokers could benefit from your input. Have you considered setting up a service, e.g. API, that brokers/all the community could query to get detailed galaxy properties? I'm sure brokers could help with that

17:37 - 17:49 UTC Stars, Milky Way & Local Volume, John Gizis (12 min)

Q: will you have a criteria for classification? like purity? A: (I missed it!)

Q: have you thought about the taxonomy required? A: (I missed it!)

Q: For microlensing, is there any plan for coordinating LSST alerts with specific networks for quick follow-up?

17:49 - 18:01 UTC Solar System, Mike Kelley (12 min)

(missed the broker wishlist...) See slides!

Q: will larger thumbnails be something useful for the cometary outbursts/impacts? A: yes.

Comment: there's a typo on your minimum requirements slide; the DIASources will be linked to SSObject records by LSST's pipelines -- I think you meant to say the brokers shouldn't drop that information.

18:01 - 18:13 UTC Strong Lensing, Timo Anguita (12 min)

Quasar microlensing variation: weeks to years.

LSST cadence only is probably not enough to do all string lensing science:

Q: how much does the inference on microlensing depend on static sky info about the star and would it break in very crowded fields where the diff image should still work well but static photometry would not? A: There's no "star" per se in quasar microlensing but the gravitational potential of many stars (think of 10000 star system galactic microlensing events.... 10000 unresolved micro-images). We see the effect in the uncorrelated brightness fluctuations of the "macro" lensed quasar images. To answer your question, as long as the lens is identified and the astrometry allows modeling the "macro" lens it is enough. In any case very accurate lens models will require high resolution imaging and spectroscopic redshifts. So "static" follow up of lensed quasars is necessary.

18:13 - 18-25 UTC Transients & Variable Stars, Michael Stroh (12 min)

Both Time-Critical and non-Time Critical science cases.

There is a broker task force inside TVS.

Q: What satellite orbits would be stationary in RA, Dec during 30 seconds? A: The point is what fields would be empty of satellites for 30 sec And what would be empty for 30 min to get 2 exposures. watch out survey burnout tho. Recommended paper: https://arxiv.org/abs/2003.01992

Q: Could we reuse the survey at some point? A: Yes, but (1) we need to think carefully to whom it should be sent to avoid the survey burnout and (2) If it is re-circulated I’d [EB] like to provide feedback before it is re-sent--many users say that they need data that is already included in the baseline alert packet (e.g., the cutouts, the lightcurve history, timeseries features…)

18:25 - 18:30 UTC Short break (5 min)

18:30 - 18:35 UTC Requested talk: PLAsTiCC 2, Mi Dai (5 min)

18.25 - 19.00 UTC Discussion: scientific use cases (25 min, breakout rooms)

README: You need Zoom 5.3+ to use breakout rooms. There will be a poll during the meeting to define the topics of the breakout sessions. Then move to the breakout rooms available. First action in each breakout room will be to elect a moderator.

Notes for the breakout session #2 room #2 (hierarchical broker)

In some sense, different brokers lead to different taxonomy as well.
Brokers won’t have enough infrastructure to complete all their goals. Are there any brokers that would need other brokers?
Fink: Two challenging tasks when linking to topical brokers: version control (both software and data) and computing resources to allocate to reprocess streams if needed.
ALeRCE: Using Kafka to communicate with topical brokers.
Will brokers release and maintain the full database of aggregated alerts, or just alerts at time t?
What happens if we have 5 brokers, and 5 of them emit their own labeled stream through Kafka - how do I accommodate for my science case? A: need to discuss with brokers to see if they are flexible enough to accommodate it (simple filter, or real processing module). Troy will give an example tomorrow. The path to interact should be easy at the very least.
There are programmatic tools to manage sub-brokering and multiple brokers (TOM).
Using plastic models to generate dummy alerts will help to design hierarchy. This is an ongoing task (e.g. in DESC).
What happens if I miss an alert? A: there will be a retention period (could be as long as 1 week to retrieve the alert) + brokers (and the LSST project) usually maintain their own database.
How to compare broker performances (in terms of metrics) so that users can choose the “best” one for their science case? DESC is putting in place such evaluations for example.

Notes for the breakout session #2 room #1 (taxonomy)

Typical taxonomy in Eyer & Mowlavi (10/2007) is shown, so we can discuss the taxonomies SC are interested in. Do we prefer physical or phenomenological taxonomies?
Brokers will not be able to do all the classifications. Should they have a very large filter, e.g. start by periodic?
How fast do we need the alerts? E.g. to differentiate between transient and stochastic - or flaring
We need to see what things are affordable and maybe do different groups that can handle different problems, as e.g. like PLAsTICC is doing. A lot of information will be needed for LSST
What is the division between a broker and a scientist? Who does what? Maybe brokers should do a broad classification and then subgroups could make more sophisticated sub-classification. AGN SC agree
What is the minimum from the brokers and what will be too much? Background information, i.e., already known sources. Flaring or brightening episodes for variable stars.
It is very important for brokers to know the opinion from the SCs in order to think on the taxonomies to be set up. Interesting cases will not be under the taxonomy given by the brokers, e.g. those AGN that show strange behavior
Warning that updating taxonomies will be difficult task
Hierarchical will make same with human classification, but for machine learning might now work, so think about this!

Notes for the breakout session #2 room #4 (Real time vs. delayed broker outputs)

Attendees: Harry Ferguson, Melissa Graham, Michael Kelly, Alex Kim, Ken Smith, Emille Ushida,

Michael Wood-Vasey

MLG Notes:

remains a bit unclear how Lasair will deal with SSObjects, yet they’re one of the main science drivers requiring immediate alerts (smaller, nearer → faster light changes)
Galaxies related science cases depend less on the alerts, more on what is in the full databases
one of the ‘delayed’ broker outputs will be useful for optimizing broker activities, and figuring out what does need to prioritized for rapid classification
‘delayed’ broker output that is outside the ‘urgency’ zone is good for experimentation
host galaxy association within the 60s limit for alert production must be (1) as scientifically useful as possible but also (2) be done within 60s. DM team is looking into options e.g., including 3 nearest galaxies where ‘near’ is in terms of ‘effective radii’, or using Object footprints (i.e., already-calculated object properties in the database)
science users of brokers might not very commonly work with the “streaming” aspect but instead “query” the broker’s (value-added) databases; this time window means there is a time-window for those delayed broker outputs and that they can be computationally-expensive --- so what should be done with that time?

e.g., xmatching on larger spatial scale which is slower
e.g., intra-night SSObject matching of newly detected SSObjects

MWV Notes (written after the fact):

There was some blurring of archival vs. historical data. The discussion focused on historical data not in the alert, rather that the more concrete distinction of products from the DRP vs. products from current Prompt Processing. Mostly I think because people didn't care about the distinction, or to the degree they did, having a specific DRP was more about provenance tracking -- there wasn't anything deeply important about having a reference fixed DRP. But perhaps this distinction is interesting in Year 3, but not important to think about in Year 1 or 2, when there isn't necessarily a DRP for that region of sky.

Solar System is inherently archival and covers archival space across RA, Dec. Every new detection is either linked to an existing object, in which case one wants to full history of where the object has been. Or it's not linked, but perhaps is clearly moving. Running back speculative upper limits is important

Solar System has many phenomena of very short time duration because the objects of interest are small, with negligible light-cross times. Events that last (are observable) for only 30 minutes or less are common. Fast follow-up and fast analysis of archival data can be key to new discovery space. In contrast, even "fast transients" from extra galactic science generally have timescales of at least hours and more often days to months.

Real-time classification will always be building on historical data. The training of classifiers will be backward looking, with deployment in the moment. Perhaps there is stability in training on a fixed set of data for provenance tracking and repeatability of classification.

Cross-matching is archival both LSST data and to other surveys. The Alerts from Rubin will come with identification of nearest 3 stars and 3 galaxies from the last data release (what is returned in Year One?). Real-time cross matching seems like it might be challenging. It depends on what's needed. If it's matching a new source to a catalog of sources with an association radius of arcseconds and even an arcminute, then cross-matching is easy with an appropriately indexed database. What's hard is when association radii become large. Computing effective radii is reasonable, but triggers overlap. E.g., is this event associated with a Messier object? MLG and MWV discussed plans for pre-computing "influence regions" across the sky based on effective radii stored in fast-lookup healpix-like schemes.

Day 2

Session 4. Moderators: Timo Anguita & Guillermo Cabrera-Vives

Brokers I

15:00 - 15:11 UTC SNAPS, David Trilling (11 min)

Real time outliers & feature outliers.

Period finding: GPU is 300 times faster! Below 10s to find it.

Web portal: pre-alpha version on Friday 30 October.

Q: in DES there was a lot of interest in finding Kuiper belt objects and beyond; for LSST, is there a plan to tag such objects in a broker ? A: <missed>

Q: Why does SNAPS need both LSST (or ZTF) & another broker stream (Antares in this case)? A: it is about flexibility. Also about optimizing the number of brokers that can connect directly to LSST stream.

Q: Are you able to keep up with the real-time alert stream considering you have so much Lomb–Scargle periodogram fitting to perform? A: That’s our GPU implementation, which was a big effort to make sure we can work at LSST scale. Paper on this GPU approach has been submitted. comment from Matthew Graham: We have GPU implementations of Lomb-Scargle, Conditional Entropy, and AoV period finding algorithms that should be released to the public soon

Q: sorry if I missed this. Will you only monitor known objects? or also find new objects? A: only known objects. The survey has to tell us “this is object xyz” and then we can handle it.

Q: Can you share a URL for Beta testers for Fridays Pre-Alpha release? A: sorry, I’m not ready to share a URL yet. We’re not ready to send to outside users. However, I can invite you when we get to a public alpha/beta version.

Q: what exactly do you use from the ANTARES stream? A: Do you mean what information? Object name, magnitudes in 8 and 18 pixels, filter, time — a few other parameters. Not much, in reality. I have a complete list that I can dig up if you are interested.

15:11 - 15:22 UTC Point of Interest, Nina Hernitschek (11 min)

Q: @Nina Hernitschek Are you planning on doing this closer to the MW or in the entire LSST footprint ? A: Specific regions chosen by users, and closer to MW.

Q: are you considering to include other classes of variable stars? A: Yes. We try to include as many classes as possible of variable stars.

Q: are you only focusing on RR Lyrae and Cepheids in PS1, or do you have classification of other types of variables from PS1 3pi data? This would be very useful for pre-LSST catalog A: We are not only looking at RR Lyrae and Cepheids. RR Lyrae and Cepheids were just what brought us to the idea of building such an alert broker. Users can define "points (regions) of interest" and get updates within them of a) anything that varies, b) anything that varies regarding a specific pattern, c) anything that was classified as a specific type of variable stars. Use cases are such as: "I want an updated light curve plus period/phase of the one RR Lyrae star I observe frequently", "I want to see all newly found RR Lyrae within Draco dSph", "give me anything that varies periodically within 1 arcmin of...".

Q: Are you thinking of expanding to non-regular variables? A: non-regular variables are a part of the "superset", so e.g. if the user's choice is to get "everything that varies" at a specific "point (region) of interest"

15:22 - 15:33 UTC Pitt-Google, Troy Raen (11 min)

Q: can you expand on the semantically compressed stream you mentioned? A: Very similar to the lightweight stream. Open to discussion and input with anyone with interest about what should be in that.

Q: will the redistributed full stream be free? or only the compressed version? A: We expect to have some baseline funding such that most user operations would be free and you would only incur cost if you wanted to do things like move large amounts of data out of the region or do compute heavy things like MCMC.

Q: When you say pub/sub stream, does that mean kafka stream?

Q: what is the database architecture you are using for the “Big Query” module ? A: Currently the full metadata from each alert simply gets its own row in the database. As we move forward we may consider doing something different, e.g. with the lightcurve history.

Q: is the user subscription free? Do you have an estimate of how many users you can support?

Q: for template classifier, is AV part if fit to allow objects that are redder/bluer than template ?

Q: So the users must be on google right ? If a user wants to add a single cloud function will they only be responsible for the compute running that ? A: Users could listen to Pub/Sub streams (similar to Kafka) and/or use the API independently from the Google Cloud Platform (GCP), so users do not necessarily need to be on Google. If you want to run a Cloud Function you would need to be on GCP and yes, you would only be responsible for the compute associated with your Cloud Function.

Q: what is the database architecture you are using for the “Big Query” module. A: Currently the full metadata from each alert simply gets its own row in the database. As we move forward we may consider doing something different, e.g. with the lightcurve history. Q: And what’s the database ? SQL-like or no-SQL like Cassandra ? A: Google proprietary SQL db.

15:33 - 15:44 UTC Fink, Anais Möller (11 min)

> 30 million ZTF alerts collected in one year

More people are welcomed to the collaboration!

Q: The portal, does it have classification and in such case, what are those? A: Models for SNe, random forest with SNe and other classifications

Q: What does it mean that the data will remain accessible for the full survey duration? Will we lose access immediately after. What about ZTF etc,? A: It means all the alerts+added values will be kept in a database, and made available to the public for 10y (i.e. you can query it, and download data). This represents more than 3PB.

15:44 - 15:55 UTC Lasair, Ken Smith (11 min)

Watchlist up to 1 million sources

Q: Are you planning to develop your own ML classifiers? A: Possibly, but we also want other people to integrate theirs.

15:55 - 16:00 UTC Short break (5 min)

Session 5. Moderators: Melissa Graham & David Young

Brokers II

16:00 - 16:11 UTC Babamul, Matthew Graham (11 min)

<3 min to process 240k alerts

Q: how would the community use the Google Edge TPUs in times of VRO/LSST? You would need to have access to the data locally? A: We demonstrated the streaming is sufficient (even using 4G). // You can deploy your own broker based on a RPi with a couple of hardware accelerators to do the heavy lifting (classifications, etc.) and then this feeds a cloud deployed Fritz instance to handle all the messy filtering, annotations, followup, etc.

16:11 - 16:22 UTC ANTARES, Tom Matheson (11 min)

Q: you are using kafka to both collect alerts, and send alerts to users. So what is the advantage (or ) of sending gzipped BSON to downstream users while the upstream alerts to Antares (at least from ZTF) are in the Avro format? A: Avro has heavier weight + annotation is easier with BSON.

16:22 - 16:33 UTC AMPEL, Jakob Nordin (11 min)

Idea of combining 3 operations (broker/supplement/join) in any orders and possibly many times to build system:

Alert ecosystem terminology needs to be better defined.

16:33 - 16:45 UTC ALeRCE, Francisco Förster (11 min)

Q: out of the 2000+ users, how many are repeat users? A: I don’t know the answer, the number is from Google Analytics (combination of IP address and device ID)

Q: What do you do with annotations? A: we store them in a different database than the main S3 storage.

16.45 - 16:50 UTC Requested talk: AEON, César Briceño (5 min)

Queueable observatory: SOAR (co-located with LSST), Gemini, BLANCO

Q: Is it possible to ask for target of opportunity observations? And in that case how should we ask for it; or is it only through asking for time? A: Yes, through proposals to the Noirlab.

16:50 - 17:20 UTC Long break (30 min)

Session 6.

17.20 - 17.50 UTC Discussion: brokers (30 min)

Notes for breakout room #1 inter-broker collaboration

Initial thoughts:

to make life easy for users: standardisation of the output streams and API
We won’t achieve 100% standardisation, but we should do as much as possible to make common tools.

Different brokers will be aiming to do quite different things, but some brokers will be working together (upstream/downstream) heavily.
Common xmatch needs
Common database issues
Common issues about classifiers -> exchange ideas, run evaluation -> there is also the risk that we all look the same after all.
Risk of circularity (??) (all brokers depending on all other brokers)
This topic should clearly be the next step to the 2nd workshop
What about MoU between brokers towards funding institutions, to enforce separate efforts while collaborating to maximise science output and support a diverse ecosystem?

What will the Rubin observatory be able to support for brokers? NSF? ERC in Europe?
Science should be the driver.
What would be the audience for this MoU? Not clear to all.

How do we register new candidates (like TNS)?
Brokers will compete, and we should have some agreement on how to treat the data, but unavoidably brokers will be competing.
In the case of ALeRCE we don't change the original names when submitting to TNS. We let TNS choose the official names for the reported objects.
First stage: We should agree on format, standard… How and when this will be decided?

2021 should be the year.

Discussions about technology: sharing tips, benchmarks, technical notes,

E.g. Service to check all brokers have complete data.

Brokers are designed to re-run as classification evolves?

Yes, most of them have this feature.

Notes for breakout room #2 public repo for training sets

Attendees (~13 attendees => add your name!): Franz Bauer, Gautham Narayan, Matthew Graham, Paula Sanchez-Saez, Paolo Coppi, Ashish Mahabal, Eric Bellm, Jakob Nordin, Vincenzo Petrecca, Alejandra Muñoz Arancibia ...

Hard to develop trusted training sets, especially outside of areas of expertise. Is there a good way to share?
Is “trust” equivalent to “classification probability”? … probably not. Probabilistic approach to training needed.
Some pruning human-made and machine-made classifications is often useful/necessary. How? Active learning to improve sources least confidently classified
ZTF as example => Matthew Graham mentioned that ZTF’s internal classifications from various groups (human-trained) and techniques (machine-learned) will be made public soonish. This should provide a large, relatively rich training set. At present, 1000 deg2, 20 pointings to sample different source densities.
Taxonomy definition is very important. some classes simply cannot be separated so well.
Does sharing training sets require a shared taxonomy?
E.g. for AGN we do not have models for their variability, do anyone have these types of models? There’s not a unique model but one can use the model they want, contact Gautham Narayan for that
Discussion of PLAsTiCC v2. add host galaxy info, add alert streams (as a function of epoch), new models. small team => hard to encapsulate all physics. Are there plans to use real data in PLAsTiCC? Yes as much as we can.
One suggestion/possibility is that the scientific community use something like Tensorflow hub to host their classifiers and models, AND provide open access to their training sets. => need community buy in => require documentation. Metadata discovery is very useful for Tensorflow. ZTF is doing this. ALerCE is also planning to make available all the models in the light curve classifier, Paula encourages every broker to do the same. In this way, one could pool and compare training sets.
peer reviewed catalogs, labels, etc. => ideally by MANY “referees”

Notes for breakout room #3 which features should be computed for brokers?

Attendees (fill your name if I forget): Kostya Malanchev, Nina Hernitschek, Massimo Dall’Ora, Ilsang Yoon, Ken Smith, Leanne Guy, Anais Moller

X-matching: some union catalog?
Period, color, amplitude change.
Template fitting for some sub-sample of faint variable stars or good SN candidates
Parametric fitting: early transient light curve
Color variation based features: AGNs. Remember about deredding. Which colors are important?
Rapid change of brightness or color: some rolling features?
Some marker of the object re-appearance.
Light-stream will keep the information except cutouts or will change the way it will provide information.

Notes for breakout room #4 Interaction with broker teams

Attendees: Melissa Graham, Troy Joseph Raen, Alex Kim

access to information and data; not clear in broker call any specification for how brokers communicate with the world/users… and in particular multiple brokers
common formats and standards in how brokers communicate with users to better enable people subscribing to multiple brokers
this goes for both winnowed streams but also value-added alerts
would alert id +value added products (i.e., not reproducing original packets) be useful or is it more useful to add to original packet?
the value-added data need not be packet-ed, but must be queryable
the querable aspect is important for the longer-delay value-added products too; i.e., updated classifications over following days, not something a broker to release a “alert” about
how does information get transferred between brokers e.g., classifications? does it get pushed around at all or is it all pulled?
e.g., for Google-Pitt broker, the modules are independent and so classifications can be done in parallel and wouldn’t hold up stream processing; and they do intend to send out notifications via pub/sub stream for e.g., new classifications
such modularity could be quite useful for users who don’t need the streaming aspect but do want to interact with the brokers’ components such as LC classifiers
It is expected that a cloud environment could facilitate such modularization?
would be useful if there was cross-broker consistency in how this functionality works
reproducibility/provenance aspect of this: some users will have a heavy need for this, so dockerized/containerized/portable software is very useful for users
versioning the catalogs used to create the value-added products is also needed for the reproducibility/provenance
rep/prov needed by brokers to inform broker devs to so perhaps would be happening
broker diagnostics: processing time, fraction of data lost, downtime, and other metadata regarding system health; so users could know in real time (and for provenance) if some parts of the sky are not covered, and have context for ‘minority reports’ when multiple broker outputs are being ingested by users
allows users to evaluate broker performance in deciding what to use (and potentially motivate grants proposal for science based on broker outputs, help to convince committees that what might be seen as ‘black box systems’ are truly working)

Open questions about whether most users just want queryable databases, or streams; want access to the individual modules within a broker and if so, which ones and what type of access.

Databases vs. streams: from a DESC perspective, most interest is in queryable database, but for kilonovae a stream might be needed (or a relatively quickly-updated database); beyond DESC it seems like queryable databases with short-ish latencies of updates are generally more broadly useful than an immediate stream-format access to processed alerts.

Modules: yes it would be nice to be able to go directly to the module outputs -- including information about calculations that might be done but not persisted, if such a thing happens, e.g., stuff that doesn’t become a ‘classification alert’ or a downstream published value-added product.

17:50 - 17:55 UTC Short break (5 min)

17.55 - 18.40 UTC Discussion: summary of discussion sessions (45 min)

Each of the moderators of each breakout session summarize the topics of the discussion. See notes above for a complete review (Days 1 & 2).

18.40 - 19.00 UTC Discussion: preparing for the next workshop (20 min)

There will be a second Broker workshop in 4-6 months.
What do the audience want? Is there something we haven’t mentioned in this first workshop? DESC SC is working together with brokers, encouraging other SC to work together with brokers.
Benchmarks: Work together and add new users, ISSC benchmarking for brokers. ISSC benchmarking came up in Ashish's ISSC presentation. It's a great idea in principle. An issue is that it's been hard to generate engagement in ISSC (especially from non-astronomers) without having significant resources (students, compensation). DESC is having its success in part because of the resources.
Poll