1 of 36

1

March 3, 2025

Intro to ARM Data/Accounts

Understanding and Accessing Data from the Atmospheric Radiation Measurement (ARM) User Facility

Michael Giansiracusa

ARM Data Center (ADC) DevOps Team Lead

Oak Ridge National Laboratory

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

2 of 36

What is the ARM User Facility?

A U.S. Department of Energy (DOE) Office of Science user facility.

Mission: To provide the climate research community with strategically located in situ and remote-sensing observatories.

Goal: To improve the understanding and representation of clouds and aerosols in climate and earth system models.

Data is free and available to everyone.

​

2

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

3 of 36

ARM Data Services: Introduction

  • Data flow operations and monitoring
  • Advanced data collection systems
  • High-performance computing (HPC)
  • Comprehensive data processing
  • Data interoperability
    • Advanced strategies for utilizing metadata
    • Data Discovery
    • Data Workbench
    • FAIR, Standards and protocols
  • User Management and Citations
  • AI-based approach in data management

3

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

4 of 36

The Value of ARM Data

Provides long-term, continuous, and high-quality atmospheric observations.

Supports advancements in climate modeling and prediction.

Enables research on cloud lifecycle, aerosol processes, and radiative transfer.

Critical for understanding atmospheric processes in diverse climatic regions.

​

4

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

5 of 36

A Global Network of Observational Capabilities

  • Fixed Sites: Long-term atmospheric observatories (e.g., Southern Great Plains, North Slope of Alaska, Eastern North Atlantic).
  • Mobile Facilities (AMF): Deployable observatories for targeted field campaigns (2-8 years).
  • Aerial Facilities (AAF): Instrumented aircraft for in-situ and remote sensing measurements.

5

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

6 of 36

Spotlight: The Southern Great Plains (SGP) Observatory

ARM's first and largest observatory, operating since 1992.

Located in Oklahoma and Kansas.

Extensive suite of instrumentation for comprehensive atmospheric profiling.

Ideal for studying continental convective cloud systems and land-atmosphere interactions.

​

6

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

7 of 36

ARM Data Products: Datastreams

  • Data collected directly from instruments, processed to various levels (e.g., .a0, .a1, .b1, .c1, .m1).
  • Naming Convention: sitefacilityinstrument.level (e.g., sgpmetC1.b1, enarsgpmplC1.c1).
  • Typically in NetCDF (Network Common Data Form) format.
  • Contains variables, dimensions, and metadata.
  • https://www.arm.gov/guidance/datause/formatting-and-file-naming-protocols
  • ​

7

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

8 of 36

ARM Data Products: Value-Added Products (VAPs)

  • Higher-level, research-oriented products.
  • Combine data from one or more instruments or apply advanced algorithms.
  • Examples: Cloud type classifications, aerosol optical depth, radiative fluxes.
  • Undergo extensive development and quality checks.
  • Also typically in NetCDF format.
  • https://www.arm.gov/data/science-data-products/vaps?sort=id.keyword%3Aasc
  • ​

8

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

9 of 36

Brief overview of data quality

9

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

10 of 36

The Standard for ARM Data: NetCDF

  • Self-describing: Includes data and metadata (units, variable descriptions, history).
  • Portable: Accessible across different operating systems.
  • Widely supported by analysis and visualization software (Python, MATLAB, IDL, Panoply).
  • https://www.arm.gov/publications/programdocs/doe-sc-arm-15-004.pdf
  • ​

10

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

11 of 36

Your Gateway to ARM Data: The Data Discovery Tool

Web-based portal for searching the entire ARM data archive.

URL: https://adc.arm.gov/discovery/

Search by:

  • Date range
  • Location (Site, Facility)
  • Measurement type or Instrument
  • Keywords, Variable names
  • Field Campaigns

​

11

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

12 of 36

Finding Data

12

March 3, 2025

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

13 of 36

Data Search

13

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

14 of 36

Using the Data Discovery Tool: Search and Filter

14

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

15 of 36

Finding Data

15

March 3, 2025

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

16 of 36

Finding Data

16

March 3, 2025

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

17 of 36

Placing a Data Order

  • Once data is found in Data Discovery, add selected datastreams to your "cart" or "order list."
  • Review your order: check dates, datastreams, and file counts.
  • Select data format (usually NetCDF) and any processing options if available.
  • Submit the order.
  • You will receive an email notification when your data is packaged and ready for download.
  • ​

17

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

18 of 36

Why Create an ARM Account?

  • Required for ordering most data from the ARM archive.
  • Allows you to track your data orders.
  • Enables notifications about data availability, updates, or potential issues with ordered data.
  • Provides access to some specialized tools and services.
  • Registration is free and straightforward.

​

18

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

19 of 36

Creating account

Navigate to the ARM website (www.arm.gov).

Look for "Login/Register" or "Create Account."

Fill out the registration form:

  • Name, email, institution.
  • Briefly state your research interest (helps ARM understand user needs).

Agree to the ARM Data Policy.

Verify your email address to activate the account.

19

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

20 of 36

Giving Credit: How to Cite ARM Data

  • ARM data is free, but proper acknowledgment and citation are required.
  • Each datastream and VAP is assigned a Digital Object Identifier (DOI).
  • Use the DOI for citation in publications, presentations, and reports.
  • Citation formats are provided with the data and on the ARM website.
  • Acknowledge "the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility."

20

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

21 of 36

Analyzing and Visualizing Data: Useful Tools

  • Python: Widely used with libraries like:
  • Other Software: IDL, MATLAB, NCL, Panoply (for quick viewing/plotting NetCDF).
  • ARM provides some example code and tutorials.

21

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

22 of 36

We're Here to Help! Resources and Support

  • ARM Help Desk: For questions regarding ARM data, please use the Data Questions form, or call 1-888-ARM-DATA (Monday though Friday, 8 a.m. to 4 p.m. Eastern time). Email arm at: https://www.arm.gov/data#contact-adc�ARM Website (www.arm.gov): Extensive documentation, FAQs, instrument details, VAP descriptions.
  • Instrument Mentors: Contact information often available for specific instruments.
  • Data Announcements & Newsletters: Stay updated on new data, campaigns, and tools.

22

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

23 of 36

Finding Data

23

March 3, 2025

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

24 of 36

https://jupyterhub.arm.gov

​

24

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

25 of 36

Jupyter Notebook

  • A Jupyter Notebook is an interactive software development environment used to create and share computational documents
  • Extensively used for Python-based development, but supports other languages as well
  • Easy to see and communicate how a program unfolds since the outcome of each cell/block is cached
  • Notebooks: Editable and shareable files, for example: markdown, .ipynb, .py and .txt files

25

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

26 of 36

JupyterLab

  • A web-based user interface that enables users to work with:
  • Jupyter notebooks
  • Markdown editors
  • Terminals
  • Custom components
  • File system navigation

​

  • Allows effective use of multiple notebooks together effectively

​

  • JupyterLab and Jupyter Notebooks can be installed on your machine

​

26

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

27 of 36

Introduction

The ARM Data Workbench (ADW) is a revolutionary new ecosystem, designed to fully utilize ARM’s capabilities to greatly enhance data access and interactivity for users.

​

Core Capabilities:

27

March 3, 2025

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

28 of 36

Dashboard

  • Central location for everything workbench related
  • Organized by action not by application
  • All items on the dashboard are exposed on the widget as well

​

28

March 3, 2025

​

​

​

​

​

https://workbench.arm.gov/

​

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

29 of 36

Finding Data

29

March 3, 2025

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

30 of 36

Data Studio Entering from Discovery

30

March 3, 2025

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

31 of 36

Data Studio Adding Criteria

31

March 3, 2025

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

32 of 36

Data Studio Retrieving Dates

32

March 3, 2025

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

33 of 36

Data Studio Visualize

33

March 3, 2025

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

34 of 36

Data Studio Get Data & Share

34

March 3, 2025

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

35 of 36

Widget Integration

  • Consistent across apps/services
  • Mini version of the dashboard
  • Easy to find

35

March 3, 2025

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0

36 of 36

Questions?

Thank you! If you have any questions or need more information, please let me know.

​

36

March 3, 2025

Contacts

Michael Giansiracusa

giansiracumt@ornl.gov

​

Giri Prakash

palanisamyg@ornl.gov

Primary Blue

#0070C1

ARM Blue

#194173

ARM Green

#00BD70

Gray

#828282

Black

#000000

White

#FFFFFF

Light Blue

#80B8E0