1 of 14

What is Bluesky?

2025-09-21, Bluesky Workshop�ICALEPCS, Chicago

https://blueskyproject.io

doi:10.1080/08940886.2019.1608121

Synchrotron Radiation News 32.3 (2019) 19-22.

Thorough introduction here:�https://blueskyproject.io/bluesky-slides

2 of 14

But first, Why Bluesky?

It’s all about the scientists and their data

Scientists spend much more of their time processing their data than collecting it or building their instrumentation.

Each instrument has unique capabilities. Provide common tools to enable them.
Build software that enables collaboration and specialization. Focus on capabilities (scientists) over requirements (engineers).
Use solid software engineering practices (including version control, design patterns, CI/CD, adopt common standards for data & code, documentation) that encourage building on a shared core.

3 of 14

Bluesky is designed in service to data

When analyzing data we want....

To easily find our data.
Access to that data, not caring particularly where it's stored or in what format.
Well-structured data marked up with relevant context, to support easy and sometimes automated batch analysis.

Primary (scientific) data
Ancillary (engineering) data
Metadata

Seamless integration with popular tools for data processing, analysis, visualization, & management.

4 of 14

What is Bluesky?

Framework for scientific data acquisition, management, and analysis

Bluesky enables experimental science at the lab-bench or facility scale.

Web site: https://blueskyproject.io/�Bluesky is a collection of Python libraries that are co-developed but independently useful and may be adopted a la carte.
Collaboration: started at NSLS-II, spread through US-DOE, and beyond
Community: https://mattermost.hzdr.de/bluesky (requires login), weekly Zooms
Use Bluesky toolbox components from Python, IPython, Jupyter, GUIs, …
Python code on GitHub: https://github.com/bluesky
Installable via pip (some also published on conda-forge)

We are aware of an unrelated & newer social media web site also named bluesky. That’s a different community.

5 of 14

Collaborations

Bluesky use^* Spans 5 Continents

Asia

BSRF, HEPS / China
Pohang II / Korea
U Kerala / India

Australia

Europe

BESSY-II / Germany
DLS, ISIS / UK
PSI / Switzerland

North America

ALS, APS, NSLS-II, SLAC, UW Madison / US
CLS / Canada

South America

LNLS / Brazil

* as of 2024-04

Sources include: https://mattermost.hzdr.de/bluesky/channels/epics-2024-bluesky-tutorial��Institutions known to be using Bluesky:

· NSLS-II

· LCLS (widespread use) and SSRL (one or two instruments)

· APS (scaling up from a couple beamlines to a dozen or more)

· ALS (at least one beamline, also scaling up)

· Diamond Light Source (invested significant effort, adoption in progress)

· Australian Synchrotron (several beamlines)

· Canadian Light Source (at least one)

· PSI (evaluating, not yet committed to adoption)

· Pohang Light Source II

· BESSY II

· Various academic labs

�----

I would like to see HEPS and BSRF of IHEP, Beijing included in this list. See their most recent status report in SRN Volume 36, 2023 - Modern Beamline Controls Software Development

----

Is that marker accurate for India? I was remembering that from a PhD thesis work in Kerala.

6 of 14

Community Discussion on Mattermost

Visit https://blueskyproject.io/mattermost/ to register and get started

7 of 14

A few components in the Bluesky toolbox

Interoperability through carefully-defined interfaces

bluesky

Orchestrate the scan & publish data
Users compose their measurement plans
Also provides basic data visualization

ophyd

Hardware abstraction layer
Signals and Devices with human-friendly names that propagate into metadata

tiled (replaces databroker)

Rich search & access to saved data
Separates data from storage format

Libraries in the Bluesky Project interoperate through carefully-defined software interfaces. Any given piece may be separately used, extended, or replaced.

8 of 14

Advantages

Accessibility:

Based on Python (global support community)
Reduces learning curve of new users
Allows for inter-disciplinary teams due to common language

Integration

Automated workflows (data management, ESAF system)
Rich libraries of tools

Features

Lots of prebuilt devices (No need to reinvent the wheel)
Handles both multi-dimensional and asynchronous, �event-based data sources
Common metadata catalog

Expansion

Easy to add new features due to open-source nature
Integrate into any Python programming implementation

A major component of any modern scientific instrument

is its suite of support software

Credit: Jake vanderPlas, "The Unexpected Effectiveness of Python in Science", PyCon 2017

A bridge to the open-source ecosystem

9 of 14

How is Bluesky Integrated at the APS?

from 2006 APS Scientific Software Workshop

Critical tool to close the loop between measurement and analysis

Orchestrates scientific measurements

Beamline controls (EPICS)
Metadata: User, Proposal, Sample, …
User metadata

Starts the data pipeline

Data storage catalog
Processing workflows
Visualization
Modeling

React to results from processing

11 of 14

Bluesky Goals

Technical Goals

Generic across science domains
Lightweight
Put metadata in a predictable place
Handle asynchronous data streams
Support multi-modal: simultaneous, cross-beamline, cross-facility
Support streaming
Cloud friendly
Integrate with third-party (meta)data sources

Sociological Goals

Overcome "not-invented-here"-ism.
Make co-developed but separately useful components with well-defined boundaries which can be adopted piecemeal by other facilities.
As inspired by the numpy project, embrace protocols and interfaces for interoperability.

12 of 14

Bluesky is designed for the long term

Maintain a focus on the scientists and their data

Make it easy to keep file-reading and -writing code separate from scientific code.
Support streaming (live-updating) visualization and processing, and adaptive experiment steering.
Integrate well with web technologies and be cloud-friendly.
But also meet users where they are!