1 of 31

NASA Openscapes Lessons from Year 1:

openscapes.org

@openscapes

Erin Robinson • Julia Lowndes • Luis Lopez

and the NASA Openscapes Mentors

ESDSWG, April 21, 2022

All artwork by Allison Horst; @allison_horst

slides: https://nasa-openscapes.github.io/about

Supporting NASA Earth science research teams’ migration to the cloud

We believe Open Science can accelerate data- driven solutions and increase diversity, equity, inclusion, and belonging in research and beyond.

2 of 31

Open science as part of the climate movement

www.allwecansave.earth

www.ipcc.ch

Recurring themes

Create space & place
Focus on the common
Reuse existing
Contribute to fill gaps - and do so openly

“ ‘We’ speaks to the collective, to collaboration, to community, to the relational work at hand. Addressing the climate crisis...will take everyone.

‘We’ speaks to justice, to how we do the work that needs doing and whose contributions are valued. We cannot, we must not, go it alone”

Ayana Elizabeth Johnson & Katharine Wilkinson

3 of 31

4 of 31

Researcher-centered, focused on teams.

Practice and feel safe working openly with yourself and your team; then ease into more.

Create space & place to explore & learn. Cohort Calls, Seaside Chats, Co-Working; GitHub, R, Python, Quarto, Google Drive,

Slack; Efficiency Tips & Inclusion Tips.

Cultivate relationships & real connections.

Welcoming folks with diverse backgrounds; meeting where they are; skills to empower immediate work; kinder science.

Open culture: Learning, teaching, iterating.

Not a checklist - a continual practice. Imperfect, messy. Takes time.

Openscapes approach

https://openscapes.org/approach

5 of 31

NASA Openscapes Framework Design

Supporting NASA Earth science research teams’ migration to the cloud

https://nasa-openscapes.github.io

The overarching vision is to support scientific researcher teams using NASA EOSDIS data as they migrate their workflows to the cloud. We are doing this working with NASA Distributed Active Archive Centers (DAACs) over three years by:

Develop a cross-DAAC Mentor community of collaborative cloud data instructors, that co-create, curate, and use shared resources (“make once, use often”)
Empower science teams through the Champions program to migrate their download- intensive data analysis workflows to the cloud and open, kinder science
Scale the Openscapes Champions program with DAAC Mentors

to support more teams transforming their workflows

towards open, kinder science and the cloud

6 of 31

Focus on finding the common, teaching culture, contributing as part of their jobs

Carpentries Instructor Training
NASA-Openscapes GitHub Org
2i2c JupyterHub & AWS Credits
Quarto collections of notebooks

Support Mentors towards establishing a common set of NASA Earthdata tutorials that they can then build off their DAAC-specific and science examples

https://nasa-openscapes.github.io

Develop Mentors

Scale Open Science Leaders

Erin

2i2c

Work of the Mentor Community

How to grow it - working on real researchers in the science teams

Luis showing this real example of work the Mentor community has done

We are really grateful for our NASA Mentors, and to NASA who is supporting their time so that open science can be a paid part of their jobs.

We’re supporting Support Mentors towards establishing a common set of NASA Earthdata tutorials that they can then build off their DAAC-specific and science examples

Again, these recurring themes. Wanted to reuse existing so needed to identify what is unique to NASA earthdata and what’s not, tease that apart to reuse existing. Only contribute to combine and fill gaps rather than reinventing

We center researchers, and we focus on this feedback loop between tooling and people

---

7 of 31

“It was a really great week. The tutorials were AMAZING. Everyone did a great job, and everyone was very nice. I really appreciated welcoming environment. I don't have a strong python background. But i was supported in learning all around”

Blog summaries:

earthdata.nasa.gov/learn/articles/2021-cloud-hackathon

podaac.jpl.nasa.gov/announcements/2021-12-15-The-2021-Cloud-Hackathon

65 Openscapes 2i2c JupyterHub AWS instances

50 forks of the Cloud Hackathon GitHub repo

8 hack-team projects presented on Day 5

Empower Research Teams

2021 Cloud Hackathon

November 15-19

https://nasa-openscapes.github.io/2021-Cloud-Hackathon/

Teaching opportunity helped focus Mentor engagement and aligned with PO.DAAC’s priorities

8 of 31

9 of 31

“I still get kudos from those that attended…a participant shared what she learned and liked with another early adopter group she is in and the organizer is looking to implement or share those practices at their own workshops - yay for spreading open science”

- Catalina Oaida, PO.DAAC

Reusing, collaborating, & role-modeling Open Science

10+ talks & workshops led by Mentors, reusing tutorials

>> Including AGU, UWGs, SWOT Ocean, ECOSTRESS, Train, internal DAAC staff

Scale Open Science Leaders

Develop Mentors

Mentor support is external and internal facing: teaching researchers and DAAC staff, hands-on in the Hub

ECOSTRESS book

SWOT book

Julie slides 9-11

Preparing for the Hackathon really helped build this community of cross-DAAC Mentors that feel comfortable collaborating, giving feedback, co-teaching, cross-teaching, and picking up and helping each other out on a moment’s notice.

Through that process they’d also created content and process to reuse and role-model. The Mentors have reused this in talks and workshops. These screenshots are an example: When PO.DAAC had a SWOT Ocean workshop, Catalina Oaida started from the Cloud Hackathon materials, which the Mentors had already modified for a workshop at AGU. Several weeks later, Aaron Friesz at LP DAAC started from Catalina’s for the ECOSTRESS workshop. To develop and teach, they’re using the Hub. So we’re seeing this scaling and leadership, and in this first year something we didn’t expect is internal DAAC support - more PODAAC and GES DISC staff that have participated as learners. Internal and External facing.

ALso learning from communities like eScience Hackweeks

Created this content and reused
Created this Hub for them to collaborate, teach, adn do science. Many for the first time. This is the big wn for this group. How they’ve supported other DAAC staff to support on the cloud

ECOSTRESS built of PODAAC from

10 of 31

Pathway presentations next week! - sharing work in progress

10 Researcher teams planning their transition

DAAC Mentors assisting teams, co-working and Slack

https://nasa-openscapes.github.io/champions

Empower Research Teams

2022 Science Champions Program

March-April

https://nasa-openscapes.github.io/2022-nasa-champions

10 Cohorts since 2019, with NASA, NOAA, academic, non-profit, gov, and tribal groups

Openscapes Champions Program

Cohorts of 7 research teams meet virtually 5x over 2 months to explore open data science & apply to their own shared workflows (Pathway)

11 of 31

So many other wins to measure & share

I am championing open data science in faculty meetings • I said we’re doing our students a disservice • I’m asking to hire lecturer positions • I enabled labs to have seaside chats • we are developing onboarding plans • I support my students starting hacky hours • we now take lab meeting notes in google docs, openscapes style! • our lab tech will teach a new data science course • we have a code of conduct! • us too! • all this is something I’ve worried about a lot and feeling better about it • all our lab protocols are now on GitHub • I reorganized my teaching materials so it’s better next year • all my capstone students are using GitHub for their analyses • I'm now pushing GitHub on all my collaborators • we have a more open culture • I organized and put all my teaching materials online • I didn’t realize there would be so much to talk about • we have a shared lab Google Drive • we developed a way to offboard our undergrads • my lab has put all our scripts online • we are going to have a lab hackathon • thank you for saying the code of conduct on each call • I didn’t know about #rstats twitter • these efficiency tips save me so much time • I taught a course from R4DS, learning the night before • I want to do everything a bit better • I reran my collaborator’s code! • we improved our documentation • we improved our metadata • this saved me so much time • everyone in our lab now has a GitHub account • I led a discussion on data sharing practices for our lab • we are now officially an open science lab • I’m going to put all my teaching materials online • I am starting an RLadies chapter • I organized our lab’s code on GitHub • I feel a part of something bigger • I will start teaching R in my classes • I thought I was utterly incapable of learning GitHub • this happened just at the right time • this isn’t just about coding & GitHub, it’s about changing the way we do science

12 of 31

Skills & collaborative infrastructure:

Coding - to access and analyze
Notebooks - literate programming to combine prose/narration with code
GitHub - to move code to Cloud & collaborate
Reuse - comfort reusing existing code and packages
Asking for help - confidence, relationships
Developing NASA Earthdata vocabulary - granule, Level 2, s3 bucket, Earthdata Login, concept_id…

A lot required to “get to the science”

Through tooling and culture, how can we lower barriers and support scientists with empathy?

Transition to Cloud: alignment for DAACs & researchers they support

Erin & Julie. Things that stick out - exciting time. Alignment. An opportunity not to only work with researchers but the bigger DAAC community

Researchers we’re working with are excited and interested - Andy Mitchell’s comment about “we don’t need to do the elevator pitch anymore” - researchers are already on board, now they want to know how.

Where do people do this work - Openscapes has provided this 3rd place for DAACs to work together, with our NASA-Openscapes GH org and creating this workflow

Teaching skills is a key part of this support. DAAC mentors have really invested in learning how to teach, and practice. Can’t overemphasis of the dry runs, giving each other feedback, confidence reusing. When we think about how DAACs at scale are going to support researchers moving to the cloud these are the skills they’d need. Really happy to have ESDIS Buy-in.

Hoping that TOPS can leverage the capacity we’re building at these DAACs
Kinder science - offering up the support and meeting DAAC staff where they are, and the researchers where they are. There are a lot of assumptions with what people know. Building trust, confidence to speak up.
Making this transition required us paying for Cloud Cost. We’ve taken this conversation off the table with our sandbox, but what does this really look like for others - this isn’t a forever solution.

13 of 31

NASA Openscapes

Cloud Infrastructure

Luis López

Software Engineer @ NSIDC

14 of 31

A tale of two workflows � "This story is based on actual events. Characters and timelines have been changed for dramatic purposes."

Coffee
Open a data provider website

Only works on Chrome

Click here, click there, order data

Not very reproducible

Download and analyze data using GIS tools

If the data required is not that big (terabytes? Forget it!)
maybe compile some code
Install a library… oh wait what? A compiler error.

Coffee
Go to openscapes.2i2c.cloud

No need to install anything
Science!

15 of 31

Starting up my 2i2c JupyterHub instance…on live TV

16 of 31

2i2c cloud infrastructure

Right to replicate, Cloud Agnostic

My Cloud

17 of 31

But what is the experience for the user?

2 things we saw a need for, early on:

JupyterHub customization, shared environment
Streamlining NASA Earthdata Access

…Origin stories of corn environment and earthdata Python package

Observations from the Mentors Cohort

Observations working with researchers

Responsiveness of this group to user feedback: Eli Holmes (NOAA) had requests that we could meet -> increase how productive researchers can be

18 of 31

Openscapes environment

Scientists don’t need to install anything

https://github.com/NASA-Openscapes/corn

Github authentication
Session persistence
Deployed to us-west-2
Reproducible Conda environment
Extensibility
Multiple kernels are supported
Dask-kubernetes!

19 of 31

How to access NASA datasets and not die trying

`earthdata` Python package removes barriers for auth and access

CMR

CMR-STAC

EDL

Tokens

AWS credentials

One per DAAC

S3 Buckets

Boto3?

List goes on…

https://github.com/nsidc/earthdata

20 of 31

Show me

https://nasa-openscapes.github.io/earthdata-cloud-cookbook/examples/earthdata-access-demo

21 of 31

Observations (mine)

Ready to use cloud environments are very useful and valued by scientists -and developers-.

We need to learn from researchers how we can lower barriers to science - both tech and teaching

There is a need for open infrastructure on a permanent basis.�
There are tradeoffs, the cloud is not a silver bullet.

Costs, who pays? how?: https://www.cloudbank.org/
Complexity

22 of 31

https://nasa-openscapes.github.io�https://github.com/nasa-openscapes�https://github.com/nsidc/earthdata

THANKS!

Artwork by Allison Horst.

23 of 31

Cole Krehbiel

Aaron Friesz

Catalina Oaida

Jack McNelis

Makhan Virdi

Matt Tisdale

Vishal Bagadia

Amy Steiker

Luis Lopez

Andy Barrett

Christine Smit

Jennifer Adams

Alexis Hunzinger

Kumar Ramasubramanian

Shubhankar Ghalot

Iksha Gurung

Thank you to the

DAAC Mentors!

24 of 31

DAAC Mentors 2022!

Join us: Nominate DAAC colleagues!

Benefits of joining:

Aligns with DAAC priorities
Cross-DAAC community of expertise, tutorials, open science leadership
Carpentries instructor training
Access to 2i2c JupyterHub

2021 Mentors aren’t leaving, we’re growing the community. Welcoming Mentors from new and existing DAACs!

We’re looking for new mentors, anyone can answer questions.

Current mentors aren’t leaving -

We have these 3 objectives. One was to build this cross-DAAC mentor community. We’re going to try to intentionally reach out and include new people at the current DAACs and at new DAACs. That’s emerging: the current DAACs have more folks they want to include

These open science pathways are within a whole landscape of open data, tooling, and communities.

Our main focus is to empower research teams. We do this primarily through our Openscapes Champions Program, which is an open data science mentorship program for teams. We help researchers move from sad and lonely science like the animals on the bottom left to exploring open science with their teams cohort of their peers, exploring existing paths and assisting with new trailbuilding as needed. We are constantly learning and iterating, and focused on how to build trust and make it safe for more people to participate in open science.

25 of 31

Thank you!

We’re looking forward to working together!

Join us:

Twitter: @openscapes

Web: openscapes.org/contact

openscapes.org/events

nasa-openscapes.github.io

slides: https://nasa-openscapes.github.io/about

Openscapes artwork by Allison Horst

Julia Stewart Lowndes, PhD

Co-Director, Openscapes

National Center for Ecological Analysis & Synthesis (NCEAS), UC Santa Barbara (UCSB)

lowndes@nceas.ucsb.edu; @juliesquid

Erin Robinson

Co-Director, Openscapes

Metadata Game Changers

erin@metadatagamechangers.com

Luis Lopez

Software Engineer, NSIDC

luis.lopezespinosa@colorado.edu

26 of 31

Deliver Champions Program for a Cohort of research teams

Transform research teams workflows towards kinder, inclusive open science

Flywheel

Attract research teams interested in better practices for data-intensive science

Inspire broader scientific communities through visible examples and leaders – Open science shift

Invest in Champions Program curriculum

Develop Champions Mentor – professional development and leadership skills

Empower

research teams

Amplify leaders

Engage Mentors

Invest in Champions Program curriculum: NASA Earthdata cloud-specific materials

Empowering research teams through the champions program and helping them amplify as leaders as they transform their own work and inspire others is a big part of what we do. And we’ve been really taken by the idea of the Flywheel, which is a concept developed by Jim Collins in his book Good to Great. A flywheel is a complete picture of what a program does so that it can be sustainable, and get stronger with every revolution of the complete cycle.

Erin: (animate) So for us, we’re also now focused on engaging mentors who can contribute curriculum, help attract research teams, and deliver the champions program.

(animate) with NASA, we’re working with mentors from the Distributed Active Archive Centers and working on cloud-specific materials to support researchers transitioning their workflows to the cloud. Developing leaders at all levels

---

27 of 31

Action shot: collaboration

28 of 31

Slide from Mike Little - iterated since his GFSC, talk April 2022

NASA Openscapes Framework: Experience from Year 1

PI: Julia Stewart Lowndes, NCEAS/UCSB and Co-Lead: Erin Robinson, Metadata Game Changers

Collaborators: 2i2c, Carpentries and DAAC Mentors from: LPDAAC, NSIDC, PODAAC, GESDIS, ASDC, IMPACT

Key Milestones

Objective

To support scientific researchers using NASA Earthdata as they migrate their workflows to the cloud. We are doing this working with NASA Distributed Active Archive Centers (DAACs) over three years by:�

Developing a cross-DAAC Mentor community that supports growth into confident cloud data instructors, and create, curate and use shared resources and have a tutorial review process
Empowering science teams to experiment migrating their download-intensive data analysis workflows to the cloud through a partnership with Carpentries, hosted cloud environments like 2i2c & SMCE
Scaling the Openscapes Framework with DAAC Mentors to support science cohorts and amplify as many open science leaders as possible, transforming their workflows towards open, kinder science and the cloud

Benefits of Hosted Cloud Hub for Project:

Rapid Startup (minutes for users/a few weeks for initial hub setup)

Easy cross-DAAC collaboration in shared, common environment

Allowed for cross-DAAC creation of common, curated NASA EarthData cloud tutorial material

Low barrier to entry: Researchers can experience cloud environment and access NASA Earthdata in the cloud before having the cost discussion

Low maintenance for project

Multiple separate projects can operate independently

Easy to provide non-NASA colleagues and cross-DAAC colleagues with access for workshops and hackathons

Project kick-off at ESDSWG Feb/21
DAAC Mentors Accessed Jun/21
2i2c Jupyter Hub Launch Jun/21
1^st cross-DAAC Cloud Hackathon Nov/21
AGU cross-DAAC Cloud Workshop Dec/21
Multiple DAAC-specific workshops Mar/22
NASA Champions (10 teams) Kickoff Mar/22
Decision to experiment w/ SMCE Mar/22
First login to SMCE Apr/22