NASA Openscapes: �How we work supporting Earthdata science in the Cloud
First story of a scientist “fledging” from our JupyterHub!
NASA Openscapes Mentors and the Openscapes Community
Bri Lind • Catalina Taglialatela • Cassandra Nickles • Aaron Friesz • Michele Thornton • Daniel Kaufman • Alexis Hunzinger • Rupesh Shrestha • Mahsa Jami • Luis Lopez • Celia Ou • Alex Lewandowski • Chris Battisto • Jess Welch • Brianna Pagán • Sargent Shriver • Amy Steiker • Makhan Virdi • Brandon Bottomley • Stefanie Butland • Julia Stewart Lowndes • Erin Robinson & more!
ESDSWG @ Huntsville Alabama, March 5, 2024
What if we connected our skills & values as a daily practice, for climate?
data science + open science
mentorship + teamwork + community
“To address our climate emergency, we must rapidly, radically reshape society. We need every solution and every solver”. - Ayana Elizabeth Johnson
Ayana Elizabeth Johnson & Katharine Wilkinson, All We Can Save
2
NASA Openscapes
3
A mentor community across NASA Earth science data centers that helps support users though
“NASA Openscapes is a collaborative environment for DAAC staff to collectively support open science initiatives for NASA Earthdata users. We’ve developed awesome material to help Earthdata users (cheatsheets, a python package (earthaccess), NASA Earthdata Cloud Cookbook).
Perhaps just as important as what we’ve done however, are mindsets we’ve grown into along the way. It’s okay to share imperfect works in progress. Ideas are not too big or too small to share. We are better at dreaming and implementing the future together.” – Cassie Nickles (PO.DAAC)
Consistent feedback and iteration has shaped development of migration tools and support mechanisms
End-Users
NASA Openscapes Mentor Community Values
NASA Archive Center Staff
Open Science Community
How we work:
Openscapes Flywheel
An open source tool
for movement building
The Openscapes Flywheel:
A framework for managers to
facilitate and scale inclusive Open science practices
Leverage common workflows, skills, tools
Work Openly
Invest in learning and trust
Inspire
Flywheel
Create space and place
Welcome
Empower
Learning culture
Engage
A Future Us mindset
Amplify
Open leaders
Flywheel concept
transformations occur from consistently doing key activities that add up over time
(Collins)
Inspire
Leverage common workflows, skills, tools
Empower
Learning culture
Engage
A Future Us mindset
Amplify
Open leaders
Flywheel
Welcome
Work Openly
Create space and place
Invest in learning and trust
YEAR 1
Leverage common workflows, skills, tools
Inspire
Work Openly
Invest in learning and trust
Welcome
Empower
Learning culture
Engage
A Future Us mindset
Amplify
Open leaders
Flywheel
Create space and place
YEAR 1
Leverage common workflows, skills, tools
Inspire
Work Openly
Create space and place
Welcome
Empower
Learning culture
Engage
A Future Us mindset
Amplify
Open leaders
Flywheel
YEARS 2&3
15+ workshops & talks led by Mentors: reusing & extending tutorials with software & conceptual solutions & open community
Invest in learning and trust
Celebrate all this shared work!
earthaccess Python library
Cookbook: current tutorials & onboarding
Cheatsheets & guides
2i2c JupyterHub: Python, R, Matlab,
corn base image: built on Pangeo stack
Value of Hosted JupyterHubs White paper / RFI
Identifying & responding to user needs
Carpentries Instructors
15+ workshops & talks led by Mentors since: reusing & extending tutorials with software & conceptual solutions
Openscapes Flywheel
turning hundreds, thousands of times in ways big and small
The Openscapes Flywheel: A framework for managers to
facilitate and scale inclusive Open science practices
(Robinson & Lowndes 2022)
Shifting institutional culture to develop climate solutions
with Open Science (Lowndes et al 2023)
Welcoming more participation in open data science for the oceans (Fredston & Lowndes, Annual Review of Marine Science, 2024)
13
Slides from Posit conf: “Documenting things: Openly for Future Us https://openscapes.github.io/documenting-things (youtube)
Open documentation - using Quarto+GitHub (same tools we teach)
14
https://nasa-openscapes.github.io/news/2024-02-28-github-community-call, Bri Lind LP DAAC
Open project management - in the same place as code & docs
Developing a kinder, open science mindset
Mentorship is a skill we can all develop; let’s learn from & join existing efforts
15
“Perfection is a colonial conditioning.
Worrying that you’re going to get it wrong holds you back from trying, and that erases.”
- Ta7talíya Michelle Nahanee, Nahanee Creative
Territorial Acknowledgements Interactive Workshop
Psychological Safety
Growth Mindset
Generosity
Empathy
Asking questions
Listening
Not solutioneering
Meeting folks where they are
Reflecting
Connecting values to daily actions
Tooling
tools & practices
People
teams & community
Technical & social infrastructure together
3 new stories
“Daskifying” code: from serial to parallel
Mahsa Jami (LP DAAC) brings a GEDI notebook to our biweekly Mentor Calls. Screenshares with Luis López (NSIDC) while we all watch & learn.
Problem: processing is being done in a serial way. If we open files in region, we run into the credentials timing out. Luis talks through 2 approaches:
1) simplest: design for loop so only working on a file that’s recently opened. Processing would still be serial.
2) To make it faster, we can extract the `gedi_to_dataframe()` and make it parallel with Dask. Let’s do it!
PO.DAAC recently did this too →
Capacity in the DAACs - not just in the cloud but parallelization
`earthaccess` new features save time & egress $$
18
ESDS Tech Spotlight: earthaccess
Luis López, Matt Fisher, Aaron Friesz, Qiusheng Wu, Amy Steiker & community
Blog w/ slides, notebooks & video: https://nasa-openscapes.github.io/news/ 2024-03-04-earthaccess-tech-spotlight
|
|
earthaccess and the cloud: the force awakens notebook - Luis Lopez demo
Parallelized science in the Cloud - first “fledging” story from 2i2c JupyterHub!
“It’s like a super power”: I did the analysis on my AGU poster: 150 TB of L1 and L2, and I needed some tiny fraction that were of interest to me - in the old way I would need to find a way to download, don’t have a machine big enough. But with Coiled, I can subset. Now once I realize that, I realize there are bigger datasets that just seem unworkable before.”
“In Champions Cohort we got a teaser of how to work in the Cloud. With Coiled and I did some simple things and now I’ve moved into my own workflow to Cloud. I have an Amazon account now ‘paying my own way’.”
- Aronne Merelli, University of Wisconsin
Aronne will share his story with the 2024 Champions Cohort science teams in April!
Nominations open:
https://nasa-openscapes.github.io/champions!
What if we connected our skills & values as a daily practice, for climate?
data science + open science
mentorship + teamwork + community
“To address our climate emergency, we must rapidly, radically reshape society. We need every solution and every solver”. - Ayana Elizabeth Johnson
Ayana Elizabeth Johnson & Katharine Wilkinson, All We Can Save
20
Upcoming in 2024 - the Flywheel Keeps Turning
Champions Program - starts Spring 2024
Remote-by-design mentorship for environmental & Earth science research teams to explore open science. For NASA Openscapes, research teams will also spend time experimenting & planning what analytical workflows with NASA Earthdata are like in the Cloud. Complements workshops & hackweeks.
Our ask: personal invites to 2 DAAC UWG or Science Teams
Nominate your team by March 8! https://nasa-openscapes.github.io/champions
Calendar of events - join us (open science in action)
https://github.com/orgs/NASA-Openscapes/projects/7/views/3
Mentors Community - Growing NASA Openscapes
Current Mentors aren’t leaving, we’re growing the community. Welcoming Mentors from new and existing DAACs! Please email if you are interested in getting involved (julia@openscapes.org)
Join us:
Mastodon: @openscapes
openscapes.org�nasa-openscapes.github.io
Openscapes at the Water Boards
Learn, reuse, remix, support others, grow the movement
Thank you!
22
Julia Stewart Lowndes, Erin Robinson,
NASA Openscapes Mentors & Openscapes Community
Thanks to the people who made this possible!
Not pictured: more people!
Diverse, inclusive teams and communities are key
Slides: openscapes.org/media
Further resources (incomplete list):
NASA Openscapes: Supporting Open NASA Earth Science in the Cloud
NASA Openscapes: Lessons Learned supporting Cross-DAAC User Services to migrate to the Cloud (IN22C-0321). Aaron Friesz, Alexis Hunzinger, Amy Steiker, Catalina Oaida Taglialatela, Luis López, Cassandra Nickles, Bri Lind, Mahsa Jami, Celia Ou, Julia Stewart Lowndes, Erin Robinson, NASA Openscapes DAAC Mentors.
NASA Openscapes Mentor Community
Invest in learning and trust Everyone has something to learn, ask, teach; don’t need to be an expert in everything. Cultivate psychological safety, growth mindset. Slowing down to speed up.
Flywheel
Create space and place to connect and collaborate; remove barriers to participation (paid time, part of jobs)
Empower
Learning culture
Engage
A Future Us mindset
Amplify
Open leaders
Work Openly Put what you learn into practice quickly, role- modeling sharing imperfect work. Identifying common challenges & opportunities. Documentation. Openness is a spectrum; first Future You and soon Future Us.
Welcome
bright spots (be they mentors or researchers) –people who want to work better and collaborate
Leverage common workflows, skills, tools. This is where we speed up: Iterate, reuse, remix with each other and the broader community
Inspire broader research communities through celebrating people & small wins that accumulate – Open science shift
Inspire
Work Openly
Leverage common workflows, skills, tools
Create space and place
Invest in learning and trust
Welcome
Empower
Learning culture
Engage
A Future Us mindset
Amplify
Open leaders
Flywheel