Note: The Spring 2021 Users Committee meeting has been split into a series of short virtual meetings, taking place over several days:
Committee Attendees: Kevin Goebbert (Chair), Eric Bruning, Casey Davenport, Alex Davies, Alex Haberlie, Lena Heuscher, Kim Wood
Special Guest: Mike Zuranski, College of DuPage (2020 DeSouza Award winner)
Staff Attendees: Sean Arms, Drew Camron, Shay Carter, Julien Chastang, Nicole Corbin, Ethan Davis, Doug Dirks, Steve Emmerson, Ward Fisher, Yuan Ho, Mustapha Iles, Hailey Johnson, Ryan May, Tiffany Meyer, Jennifer Oxelson Ganter, Mohan Ramamurthy, Sheri Ruscetta, Jeff Weber, Tom Yoksas, Joshua Young
Alex Davies: A couple of exciting things happening at the US Naval Academy. We had a data server that worked for the last 5 years then it crashed several times. We were then able to get a contract in for a Dell server and we’ll be able to add a jupyterhub on to that server. We’re hoping to start to use some Metpy though in many ways we are still a Matlab shop. The second thing is that the Academy is trying to formalize our concept of data science and decide if it is a major, minor, or required concentration for an individual major. Working across departments (Math and Operations Management) to develop requirements for this program and communicating the importance of MetOceano in this space.
Eric Bruning: Thank you for highlighting the work to be done formatting data and all the work that has to be done with research data.
Kevin Goebbert: We have a young data science program at Valparaiso that was born from an interest in a new faculty member; however, it has floundered based on its relationship with existing departments and disciplines. We were only able to start to work with the Math Department productively when there were budget cuts.
Alex Davies: Is the course listed under Math?
Kevin Goebbert: It is housed under Math but they can take courses in the Meteorology or other departments.
Lena Heuscher: In the same building as my program, NASA has the Impact Program that does a lot more on the data science side: https://earthdata.nasa.gov/esds/impact
Eric Bruning: I was curious about the Valparaiso institutional credit system, when you teach a course in the Math department.
Kevin Goebbert: I think it is easier at Valparaiso because we are a smaller institution and those things seem to somehow work out. We have a fairly holistic view on teaching.
Lena Heuscher: One interesting aspect of COVID, I’ve just realized that schools are moving online for snow days. Here in Alabama, in-person school was cancelled Monday and Tuesday due to ice. So we just shifted to fully online with little notice.
Eric Bruning: I believe I mentioned this previously but I decided to do away with exams and instead did a group project that required them to grab a ticket on Metpy and do a deep dive and submit a solution. That was kind of an interesting curricular approach that would have never happened otherwise. I’m kind of the GLM person and I’ve been talking to Unidata about incorporating that more fully into AWIPS. Lots of students are engaged and making commits to xarray and the metpy stack.
Alex Haberlie: Things are going well besides the ice. No power outage but we have had classes cancelled. We cancelled both in-person and online classes. For the education side, in the fall Ryan was giving a demonstration of notebooks and I came across repl.it which allows you to do homework assignments that students then fork and submit. There is also some ability to automate grading. It has been really good thus far and I haven’t had any issues with it. Students seem to be getting it and learning more than they realize.
Kim Wood: I’ve been teaching fully in-person and then teaching fully online due to the ice. Lecture attendance is good but lab attendance is bad. I’ve been pre-generating all of the imagery for the lab ahead of time. This has always been an issue that we dealt with in the lab; however, now that it is all virtual I moved to downloading it myself. For graduate level, NWP courses I’ve been having them access the UCAR RDA so that they can do case studies. We’ve been working through pycharm and set up conda. I’m hoping that after this first exercise they’ll be more comfortable with the code to plot more fields. On the research side, I’m over stretched as seems to happen in spring. I’m working with undergrads on some python tools including glm tools. That student is looking at lightning in hurricanes. I was going to set up the new Unidata Equipment Award-provided server this week but hopefully that will happen next week. Fortunately, prices for everything went down from when I budgeted the proposal, so I was able to get an even better machine.
Mike Zuranski: I’m going to cover a lot of this in my presentation. College of DuPage has been remote since last March. The only classes onsite are the labs that must be in-person (e.g. automotive lab). I’m actually on the IT side of the house and not the academic side. I’m working with the meteorology faculty all of the time.
Kevin Goebbert: We decided when we had a snow day that everything was canceled both in-person and virtual. Two of my students are fully remote for the whole semester due to health reasons. It has been really helpful to work with Julien Chastang for a jupyterhub. I haven’t thought about research because I’m about a week behind on grading. Still doing a bit of Metpy work though at a slower pace than I often did in the past. There is a lot more chatter and use of Metpy which is great to see.
Kim Wood: I had the opportunity to give a colloquium at the University of Washington and I made it half-research and half-python and I got so many questions. http://arashi.geosci.msstate.edu/python/PythonResources.html When I did that talk I shared the tools I used for every step as part of my presentation.
Alex Davies: The AMS Short Course filled up before I could even click the link. Ryan and Drew are there any plans to do another training?
Ryan May: The quick answer is that we do have plans to do more training. We are also talking with Damien Irving and Scott Collis about doing this exact thing again. There is some discomfort doing this with AMS.
Kim Wood is interested in helping with planning the next Triennial. Others have been asked to consider their participation ahead of the next session (May).
Video available on the Unidata Seminars Page.
Committee Attendees: Kevin Goebbert (Chair), John Allen, Eric Bruning, Casey Davenport, Alex Davies, Alex Haberlie, Lena Heuscher, Kim Wood
NCEP Representative: Anny Myckow
Staff Attendees: Sean Arms, Drew Camron, Shay Carter, Julien Chastang, Nicole Corbin, Ethan Davis, Doug Dirks, Steve Emmerson, Ward Fisher, Yuan Ho, Mustapha Iles, Hailey Johnson, Ryan May, Tiffany Meyer, Jennifer Oxelson Ganter, Mohan Ramamurthy, Sheri Ruscetta, Jeff Weber, Tom Yoksas, Joshua Young
Date of next meeting: Tentative meeting date the week of May 3rd.
View Mohan’s Directors Report
Alex Davies: Any impact to Unidata’s ability to offer training based on the consolidated structure and one center focused on training.
Mohan Ramamurthy: Unidata and COMET have worked together for decades and the move to the Center structure doesn’t change that relationship.
Alex Davies: I think if you asked me this question two months ago my answer would have been different. We were in-person though now we are remote for one week. I feel like there is a building expectation that we will be in-person this fall. I think there is a lot of momentum towards the in-person normal.
Kim Wood: I feel very similar. The rate of guidance has really slowed down. I will say the week that we had snow and ice we were pushed to just go virtual and not cancel class due to the snow day. I think there will be a permanent change in that all faculty will be expected to participate and teach virtual classes. That said, I do think we need to have snow days. I also feel like my ability to use flipped classroom techniques has advanced. There are benefits to all of this shifting ground. I’m also a bit concerned about the expectations of normalcy while expecting sudden shifts.
Casey Davenport: We’ve already been told to expect to teach in-person next fall; however, we will all be expected to shift to online as needed or offer 500 level courses online.
Alex Davies: The IT department has become more flexible and open to supporting technology needs for virtual classes.
John Allen: Our president loves the phrase ‘new normal’ when it is really issues that already exists. In the Atmospheric science department, our students really hate being offsite. There are some students that like the flexibility but many students prefer in-person. Unfortunately, the university is only highlighting the students that like remote/hybrid learning. I think this online paradigm is something that must be dealt with and that we won’t be able to escape it. We also have budget constraints that we don’t know the resolution of until June. We have been told to expect budget reductions around 17%.
Kim Wood: My teaching assistant and I are pulling our hair out because we have students that have taken synoptic meteorology and can’t tell troughs vs ridges. I’ve started taking attendance to give students a bonus at the end of semester. While we can present the content online students need the opportunity to interact with the data and the products to understand them fully. At least many students do.
Casey Davenport: I’m trying to do some of this in an online synchronous format and it can work; however, it is harder.
Alex Davies: We have been trying to preserve labs as in-person. A lot of classes don’t necessarily have labs.
Casey Davenport: Unfortunately for us, labs count as contact hours so that it is difficult to have labs and meet the course load requirements.
Kevin Goebbert: I know at least for myself, I can’t sustain hybrid/online teaching in the long term. At the same time, I’ve been able to incorporate more flipped classroom techniques that I would like to maintain. What could be interesting is post-baccalaureate certificates that would allow us to do niche things that could pull from the full country. I think research is going to have a long tail to get back up to speed. I’m really looking forward to taking the whole summer off and I don’t mean taking the summer off teaching and doing research. I mean taking the full summer off. Fatigue is real and burn out is really real. The thing that is in front of you that has to be done is education or student research. However, any faculty research falls off.
Lena Heuscher: From the student perspective, at least at UAH I have no clue whether next semester will be virtual, hybrid, or in-person. For me it is not crucial because I’m not taking classes but it is an issue for others. There has been no communication from faculty, our chair, or our university. I do think there is something lost in the online education instead of in-person education. People often don’t ask questions in classes; however, at least in-person you can tell if folks are getting it more or less. That is much harder to do virtually.
Kevin Goebbert: Having guest lectures will be so much more robust than in the past since we know we can have someone zoom in and not have to fly them in.
Eric Bruning: I think one of the good things is that this has forced us to adopt technology that was present before but that we didn’t use. For example, I will probably continue to use an ipad for my notes and presentations. There are differences in the responsiveness of the technology that we use and students can tell the difference and either don’t enjoy it or are frustrated.
Kevin Goebbert: This was a really great conversation and perhaps we can have an update during our May meeting.
View Anne’s NCEP Update
Sean Arms: Is there any status update for adding the GFS output back onto CONDUIT?
Anne Myckow: I will look into that but I believe we were concerned that it would take too much capacity. I don’t have a definite update but I’ll look into it.
Steve Emerson: You should feel absolutely free to tell your people to contact me or Tom Yoksas if you have any questions. We are here to help.
Eric Bruning: Can you say anything on NOAA’s broader cloud strategy. I’ve heard things like the NDFD going up to AWS, can you comment on anything in that area?
Anne Myckow: All I can say about NDFD is that they are talking about that and trying to do it. However, I don’t know anything more than that. I will say that there is a lot of talk about trying to get more of our data on the cloud providers. All they are doing is pulling it from NOMADS and there aren’t any changes in how we are doing things.
Jeff Weber: I’ve been attending N-Wave this week and according to Michelle M NOAA is going to add more data to the cloud but there are no plans to exclusively making the data only available on the cloud. This is complementary to what is happening on SBN. The question for the committee in this session, is how should we present the available data to the community?
John Allen: I tend to agree with you on the necessary granularity. That said, I think an inventory of what data sets are available at what frequency (last week, last 2 weeks, etc).
Kevin Goebbert: Archive data is a problem. We are locked into a single source for archive data.
Lena Heuscher: It is really helpful to know if a dataset already has CAPE or if I need to calculate it. To me it would be really useful to know if a variable exists in a data set without deep diving into it for hours just to determine if I can use it.
Jeff Weber: It is always good to hear from everyone because you have different use cases. We are so close to these things that we forget what we know and what it is like not knowing these systems so closely.
Lena Heuscher: I think these issues apply to new graduate students especially or physics students that transfer into a new discipline as a graduate student.
John Allen: I will say that this can be really valuable for an undergraduate senior as well. That said, all of these data sets have unique calculations behind the variables and a naive user might combine two things with the same variable name that are fundamentally different based on how they are calculated.
Jeff Weber: I will say that unfortunately that verbose documentation that captures what is present in data is often not available.
Eric Bruning: There are lots of communities that are involved in data cataloging. Pangeo comes to mind. There might be an opportunity for Unidata to market itself to those channels as well. Otherwise how do you know you need to come to Unidata if you are responding to the current hot google search.
Sean Arms: We were in contact with the stack count catalogue efforts and they kind of ghosted us. I was pretty disheartened by that. I would welcome any kind of assistance in engaging that community post-ghosting.
Eric Bruning: It is interesting that you mentioned that because they just came up. From a user perspective, it is hard to know which product to use. I know the Undiata answer is THREDDS; however, there are competing technologies.
Ethan Davis: We believe that THREDDS is the right answer for somethings but not everything.
Alex Davies: I just wanted to second that for new users simply knowing that a variable is available can be an onramp for getting new users to use Unidata tools and products. A lot of our students end up in private engineering firms after their five years of Navy service. They are often looking for low hanging fruit that they can offer a client as a value add. My experience working with engineering firms is that they are trying to get it done very quickly.
Alex Haberlie: Trying to rewind to when I was trying to first look at the THREDDS catalogue. The first thing I wish I knew was that netcdf was available because the first thing I saw was grib2 and as a MS student I didn’t know how to access that on a Windows machine. Another big lesson was learning that I could coordinate systems with lat/long. Just being notified that that was an option would have really helped me.
John Allen: That would actually be really useful.
Lena Heuscher: Sharing that information in a Metpy Monday format would be really helpful for students.
Committee broadly supports the proposed message and distribution method.
Kevin Goebbert: Announcing on Gembud is a great starting point. I would encourage distribution on the broader Unidata lists.
10:00 - 10:05 Convene and outstanding items
10:05 - 10:25 Status Reports (Committee)
AWIPS/GEMPAK Great activity now that they are staffed up.
Cloud computing Mohan: We submitted a proposal for the renewal of our Jetstream allocation. It is possible that even by the fall semester we will have access to the Jetstream 2 resources
Allen: I finally took the deep plunge learning how to operate in the AWS environment for class. I’m definitely looking to expand what I have started doing and use it more for modeling classes. I’ve used these resources for research; however, I’m starting to bring it over to classes.
Community Services
Eric: I mentioned Unidata’s resources to Leah Wasser at CU Earth Resources Lab
Note from Eric: here's the Twitter thread with Leah Wasser I mentioned (tagged to my reply, but the whole thread worthwhile): https://twitter.com/deeplycloudy/status/1385300085139853313
ACTION: Jeff Weber speak to Leah Wasser about Unidata resources
Community Equipment Award
Allen: Did the ones that we encouraged to reapply last year.
Terry: We did have an applicant revise and resubmit.
Netcdf:
Kevin: How is Unidata handling the overhead cost of community contributions to software development?
Python
John A: One of the struggles is consistency within the meteo stack. There are issues that when you update one thing there is a conflict with something else that requires a different version. I’ve run into this issue and seen a lot of complaints on twitter as well.
Ryan: I’ve seen some of that but if you encounter these issues please drop me a line so that I know about it.
John A: I also want to acknowledge that you received a new award and congratulate you.
Kim W: Pipinstall addresses the numpy issue.
John A: Clean template install can help, I know there are workarounds but pipinstall scares me.
Ryan M: There are things we should address on the CondaForge side. It seems specific to some of our bespoke tools in meteorology. I’m thinking of GRIB.
Erik B: If I want to use the cloud and Dask do I need to move everything to zarr.
John A: I’ll add that in my experience there is a real specific sequence that things have to be completed. I’ve been moving things to zarr because it does make things work better. The workflow is there but the documentation on the proper sequence is probably not there.
Eric: I found a post that was six months on the best practices of using these tools and i wasn’t sure if I should follow it since it is 6 months old.
Ryan: I will say this will get better as we move forward given the charge of the new award. One question, what versions of python are you all running?
Committee: 3.6, 3.7, 3.8 some are testing 3.9
Ryan: Be aware that we are dropping 3.6 next release.
ACTION Item: Sean can do demos at the next Users Committee meeting
ACTION Item: Google document for capturing possible topics during the Triennial
10:25 - 10:30 Triennial Planning Committee (Josh Young)
10:30 - 10:45 AWIPS Update (Shay Carter/Tiffany Meyer)
10:45 - 10:55 AWIPS asynchronous training (Nicole Corbin/Committee)
10:55 - 11:05 Users Committee Community Survey (Josh Young/Committee)
Kim Wood and Alex Davies.
11:05 - 11:10 Unidata DEI Committee (DEI Committee Members)
11:10 - 11:20 DeSouza Nominations (Committee)
11:20 - 11:30 MSI Engagement Update (Jeff Weber)
ALex Davies: Campbell Scientific has been great for us to be able to run a small observing network. We’ve had great luck working with Jeff McWhorter on data and visualization.
Jeff Weber: These are all great suggestions and in line with what we were thinking.
John Allen: This is exactly what we were hoping to see.
11:40 - 11:55 IDV Update (Yuan Ho)
11:55 - 12:00 Close Meeting
ACTIONS
ACTION: Jeff Weber speak to Leah Wasser about Unidata resources
ACTION Item: Sean offer demos at the next Users Committee meeting
ACTION Item: Google document for capturing possible topics during the Triennial