2020 Enabling Science Call for Proposals:

Alert brokers in the LSST era workshop

Five (5) pages maximum including itemized budget table. Submit as PDF with subject line 2020 Enabling Science Proposal (email to lsstc@lsst.org) no later than 11:59pm (PDT), Tuesday, May 26, 2020.

Proposal Title: Alert brokers and their user community in the era of LSST

Collaborator(s)/Co-Investigator(s), Institution(s), and Contact Email(s)

Emille E. O. Ishida, CNRS/LPC-Clermont / Fink, emille.ishida@clermont.in2p3.fr; Julien Peloton, CNRS/IJCLab / Fink, peloton@lal.in2p3.fr; Paula Sánchez Sáez, Millennium Inst. of Astrophysics / ALeRCE, pasanchezsaez@gmail.com; Eric Bellm, U. of Washington / LSST DM, ecbellm@uw.edu; Rahul Biswas, Stockholm U. / DESC, rbiswas4@gmail.com; Massimo Brescia, INAF - Capodimonte Astronomical Observatory, massimo.brescia@inaf.it; Matthew Graham, Caltech / Babamul / ZTF, mjg@cacr.caltech.edu; Melissa Graham, U. Washington / LSST DM, mlg3k@uw.edu; Nina Hernitschek, Vanderbilt University, nina.hernitschek@vanderbilt.edu; Gautham Narayan, U. of Illinois / ANTARES, gsn@illinois.edu; Jakob Nordin, Humboldt-Universität zu Berlin / AMPEL, jnordin@physik.hu-berlin.de; Chad Schafer, Carnegie Mellon University / ISSC, cschafer@stat.cmu.edu; Ken Smith, Queen’s University Belfast / LASAIR, k.w.smith@qub.ac.uk; Rachel Street, Las Cumbres Observatory / TVS, rstreet@lco.global; David Trilling, Northern Arizona University / SNAPS / SSSC, david.trilling@nau.edu; Roy Williams, Royal Observatory Edinburgh / LASAIR, roy@roe.ac.uk.

Abstract: We propose to organize two hands-on virtual workshops on alert brokers and their user community in the era of LSST. Each workshop will gather different broker teams aiming to become LSST community brokers, as well as their users, which includes the different LSST science collaborations, the LSST data management team, follow up initiatives, and other alert stream generating projects. In these workshops brokers will present their tools and services via hands-on tutorials, broker users will present their requirements and will connect with the different broker teams, and brokers will develop new collaborative efforts for the future ecosystem in the era of LSST to maximize the scientific outputs. The two workshops would be done remotely using high quality tools and will result in a legacy repository for LSST broker related topics.

Total Budget Request ($US) 15,000

1. PROJECT DESCRIPTION, MOTIVATION, AND RELEVANCE TO LSST

The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) will produce an alert stream that will be processed by an ecosystem of alert brokers (e.g. ALeRCE^[2], ANTARES^[3], Ampel^[4], Babamul, Fink^[5], GenESIS, MARS^[6], LASAIR^[7], SNAPS) which will ingest, aggregate, annotate, and classify it. They will become intermediaries between LSST and the follow-up infrastructure and will provide a consistent characterization and classification of the alert stream to enable new science. The data products and services provided by brokers will be relevant for a wide community of users, including the LSST science collaborations^[8], the LSST data management team^[9], the observatories, target observation managers^[10], other follow-up initiatives, and other survey telescopes providing alert streams such as the Zwicky Transient Facility^[11].

In June 2019, the LSST community broker workshop in Seattle^[12]  was the first interaction of potential LSST broker teams interested, the LSST project and LSST Science Collaborations. This workshop was highly successful and set up a timeline towards the full broker proposal by the interested teams. Since then, there has not been another meeting of this importance to discuss potential issues and exchange new ideas. Thus, we propose a set of workshops which are a natural continuation of interactions between all actors triggered by this first workshop, but driven by brokers teams and science challenges. Our proposal is also highly complementary to the potential broker session this year at the Rubin Observatory Project and Community Workshop (PCW). We plan not only to extend interactions between the different actors, but to establish baselines and legacy repositories with all content related to the LSST broker ecosystem. We will gather all the relevant communities at a dedicated event to define their needs, scientific questions, and to determine how to best add value to the alert stream for its scientific exploitation. The proposed workshops will be two hands-on, high quality remote workshops where these different players will be able to interact and build collaborations between them.

The first workshop will provide a first setting to establish baselines and communicate: (i) the status of all proposed brokers, their interfaces and current science modules; (ii) basic requirements of the science collaborations and expected outcomes (iii) LSST project and alert stream status. It will be a first step towards further development by brokers, build collaboration between them, the science collaborations and the LSST project. The first workshop is expected to last two-days with a maximum of 4-hours per day to be more time-zone friendly and engage the community.

The second workshop will be 4-6 months after this first workshop to assemble the developments triggered by the first workshop and reflections from the different users and collaborations. We foresee the second workshop to last 3 to 4 days (each day only 4-hours long) and to contain a set of invited speakers, talks by individual projects and a comprehensive set of tutorials. It will be the culmination of joint efforts between the different actors and provide the most comprehensive tutorial and content material for the legacy LSST broker archive.

Some of the activities which will take place during these workshops will be:

1. Broker teams will present their envisioned products and services, showing prototype versions in tutorial sessions that will allow broker users to experiment with them, imagine new use cases, and evaluate whether they are effective for their needs or if they need to be modified or extended.
2. Broker users will present their detailed scientific cases and will discuss how brokers can most effectively help them reach their goals. This will give the opportunity to broker teams to collaboratively develop the required products and services.
3. Broker teams will share ideas and will collaboratively design the basis for a future community where new standards, benchmarks, or initiatives to achieve more resilient operations may be required. It will be the place to enhance complementarity between broker teams.
4. A high quality repository of talks will be created for the benefit of the community with the contents of the workshops, including hands-on material.

One of the key questions for broker teams is to identify relevant scientific questions and technical requirements from the different science collaborations and to start interacting directly with Science Collaboration topical teams and members to create science-enabling features. For example, the Stars, Milky Way and Local Volume (SMWLV) science collaborations may be interested in a robust classification of pulsating stars to build reliable tridimensional maps of the Milky Way halo; the Solar System Science Collaboration (SSSC), in an early identification of unknown asteroid candidates, which may be possible using information from alert image cutouts, their spatial location, and their relation with other alerts; the Dark Energy Science Collaboration (DESC), in getting precise selection functions for Type Ia SN explosions to do precision cosmology; the Active Galactic Nuclei Science Collaboration (AGNSC), in identifying different populations of AGNs, QSOs and blazars; the Transients/Variable Stars (TVS) Science Collaboration, in classifying a large diversity of transients and variables, whose taxonomy may need to be discussed in detail; the Strong Lensing Science Collaboration (SLSC), in the spatio-temporal relation between alerts around strong lensing regions of the sky, as different light paths can connect the same transient event; and the Informatics and Statistics Science Collaboration (ISSC), in testing different classification algorithms with real data and in the LSST stream.

On the other hand, Science Collaboration teams and potential users need to identify existing broker capabilities, understand the interfaces and available features and to further define their science requirements in the broker environment. Furthermore, this will be an ideal setting to have discussions with LSST Project representatives that are defining the alert stream contents to identify possible new features or features that don’t require being in the stream to accelerate processing.

A different set of products will be associated with each of the points above at the end of these two workshops. We detail them in what follows:

1. The curation and publication of tutorial Jupyter notebooks explaining how users can interact with prototype versions of the different broker data products and services, indicating possible modifications to be included in the future, and links for additional references. Users will be able to post issues via github declaring their scientific needs, requesting new notebooks that would help their science, and the specification of their required tools and services in an organized fashion and in a unified repository.
2. A repository of documents detailing the different science cases, scientific questions, and technical requirements posed by broker users at the time of the two workshops. This will help broker users maintain an evolving documentation regarding their relation with brokers, complementing the already existing communication channels (LSSTC Slack channels).
3. A set of documents detailing the ideas shared between brokers regarding the future of the community. This includes the discussion of new standards and protocols, new APIs to allow the exchange of information between brokers, different benchmarks to test different database engines, classification challenges in the spirit of PLAsTiCC, user studies on the best visualization and user experience examples, the possible creation of redundancy measures that will add resilience to the community of brokers such as shared and independent data repositories, online integrity tests, or load distribution in case of infrastructure failure.
4. A centralized repository of talks on broker related topics. These workshops would generate legacy content for the integration of brokers with the LSST community.

2. BUDGET NARRATIVE

The cost of organizing the workshop can be divided into two categories:

1. The online services which will allow us to record the talks to be preserved as a legacy product and to improve the engagement in the virtual workshops, externalized to a professional company, SlidesLive^[13].
2. All the logistics for the organization of the virtual meeting, including hiring students to curate all the use case notebooks and assist participants, or the purchase of some cloud based services for computing and storage services.

We detail the previous items in what follows:

Recorded talks

We aim both to provide engaging virtual workshops as well as to produce legacy archives for the LSST and broker community. Thus, talks in the workshops must be interactive and a legacy repository must be created.  We have identified a professional conference recording agency that focuses on premium-quality customizable production, combining presentation slides and speakers on the same screen for a seamless experience for viewers. This type of formatting is now used in many professional conferences around the world.^[14]

We will use the services of SlidesLive to construct the repository and process the talks (slides and video).  We have chosen this company since they not only provide synchronised videos and slides but also allow interacting with the slides while playing the speakers video. We have not been able to identify other companies with similar services but we are open to changing providers if we find a cheaper provider for the same services.

For 15 to 30 pre-recorded talks (equivalent to a full-day of workshop) the company SlidesLive quotes €3.000 (~USD 3,300). We expect one-full day for the first workshop and 2 full days of recording for the second workshop (excluding tutorials which would be recorded using other public tools such as Zoom). These talks will be streamed and discussed in 4-hour sessions (1 session maximum per day of the workshops to simultaneously allow different time zones to participate).

This company also offers live-streaming and interactive tools such as live chat alongside recorded sessions which would be a possible addition to improve engagement during the workshop. For 4-hours with live-stream services, the costs are  €3.000 (~USD 3,300) for processed talks, chat and live-stream services. We propose 4-hours of livestream services in the second workshop to broadcast the invited speakers with Q&A sessions (using chat services and livestream).

We will adapt our number of recorded talks and services to our final budget. However, the goal remains the same, to provide a legacy repository with professional-quality talks and resources for the LSST community.

Logistics

We estimate about 100 person hours required for the curation of use case notebooks and the assistance to workshop attendees, which will be divided equally between the two organizing institutions. We will hire students to do most of the curation, which includes testing and modifying the notebooks, as well as assisting workshop participants to set up their own infrastructure or cloud based services before and during the conference. Assuming a cost of USD 20 per person hour this amounts to a total of USD 2,000.

To reduce other unforeseen administration costs, we will ask them to be covered by our institutions. We have identified possible public and private cloud service sponsors to host the tutorials and hands-on sessions.

Summary