Mohan Ramamurthy and Russ Schumacher
The Spring SAC meeting reviewed progress on the Unidata Strategic Plan, discussed the findings and recommendations of the Site Visit Team, and discussed possible future budgets.
Mohan Ramamurthy
Attended the Unidata python training workshop last fall, continue to use IDV in upper-level courses, rely heavily on College of DuPage website for lower level classes, mesoscale course is pretty small but interested in atmospheric rivers, so we’re using python to plot GFS.
Received an equipment grant last year to buy a data storage server, integrated into class with cloud computing. This didn't work as we wanted, students only use the cloud. This was used by 13 students in numerical methods class, running in AWS, with WRF in docker. Did a cost analysis, determined it cost $3 per student for two weeks of class (with education rates). Working on a report to community on the costs. Also working on a report detailing step-by-step (how to get edu credits, run wrf docker). Could do a talk at fall meeting. Exsede will be helpful to contain "leaky" AWS charges. For Big Weather Web, currently working on short courses. Most important to teach grad students these tools. POPPER framework end-to-end research workflow. Sharable blocks of analysis. Considering pursuing a TA training award from NSF, want to get NSF money as a collaborative grant. Need to find out if can be used to train undergrads as well as grads.
In the IOOS community, all the forecast models come from different academic institutions. Some cruft has accumulated at the older installations (discussing TDS). Excited about Docker, but finding that some server infrastructure is too old. Also some specialized Docker knowledge (how to keep root partition from filling up) is required to make effective use of these capabilities. Also, the IOOS catalog is making progress, and using ERDAPP.
Yield time
Regional workshop for python and AWIPS coming up with the UW Milwaukee and Western Illinois coming. Python: over half of grad students using python, undergrads still mostly using GEMPAK. May change after workshop. GOES-16 data is not readable in McIDAS. GOES 16 is turned on, pulled stuff from UPC notebooks and within 3 hours had visualizations up and running.
Big Weather Web already mentioned, one grad student really interested in machine learning and is working on applying that to heavy rainfall. The fear is that this is all a big black box but I have been convinced that there is some real insights to be gained by adding these techniques to meteorological questions. Interest in having UPC guest lecture at CSU. I also have a project with Brian Mapes on accessing NASA data and visualizing it. The current product is built around very specific data sets but the methodology could be applied to additional datasets. What Brian is building will launch a IDV bundle to display what you select.
UNCC has been adjusting their course work, the computer applications course changed by adding GRADS to Fortran. Brian Maggi is teaching a climate dynamics course that he hopes to build on base programming knowledge. Tried to incorporate GOES-16 but UNCC just doesn’t have the processing capabilities to handle it. I accepted a job at High Plains Climate Center and look forward to updating their workflow.
Attended the last day of the MU workshop, RU is all windows so I’m trying to get them to do dual boot or have an AWIPS windows version. Does the NOAA climate tool use netcdf-java? (Folks think so)
Transitioning to NIU this fall. Became active on github after the fall meeting. Will make a directory for the GOES-16 imagery. Experimented with full disc true color stuff, seeing some issues with calibration. Added save feature and it is being archived by Iowa State. Virtual 12 cores and 8G of memory. Archive should be up and running next meeting.
Started investigating Plotly, data comes from xmacis2 (midwestern climate center). Want to move from gempak to python but one of the limitations is producing calculations on grids. I have spent some of my sabbatical trying to do that through metpy. Also investigated Xarray to select a time. Potential vorticity is developed but not yet in MetPy. Created a function to work from siphon to xarray.
Gretchen - When you combine visualization with hard core calculations in python it will make assumptions that I don’t want. For example it will wrap the grid without being told to do so. Future help in this area would be valuable.
Available here
Julien Chastang (Slides), Jeff Weber
Julien provided an abridged version of his AMS talk on XSEDE, the synopsis of which is that everyone should apply for grants with the caveat that documentation is required and a bit cumbersome. Gretchen Mullendore noted that she has a template for an XSEDE application that she is happy to share and noted that an NSF grant is not required to receive XSEDE resources. Benefits of XSEDE include the absence of egress cost with the only cost being processing/storage that are handled via grants. Unidata has resources in XSEDE though at this time another account has not connected to Unidata’s holdings though it should be straightforward.
Easy to get free cycles on AWS (especially if you mention Unidata and Big Data Project in application). Started with NEXRAD L2 data since 1990, HRRR and GFS .25 degree available as 14 to 30 day archives. These are being put into AWS via LDM; however, for GOES-16 using ‘S3 copy’ instead. There was some discussion regarding how long these resources should be archived. Jeff Weber advocated for a full year; however, this is subject to Amazon support (and demand). There was a question whether S3 explorer existed for other data sets and the consensus was not at this time; however, Unidata is working with Amazon on expanding such functionality.
Rich Signell
ERDDAP was created by Bob Simons (NOAA Fisheries) using netcdf-Java and influenced by TDS. Rich recommends folks look at it and consider having conversations with Bob since he is interested in talking to other groups about the long term future for ERDDAP. This could proceed as a partnership with TDS. Available at github.com/BobSimons/erddap
Tom Yoksas
Session provided an overview of data and delivery methods from Unidata and input from Users Committee. Specifically, data will be distributed through ADDE and TDS servers. Unidata will use a LDM feed internally and examine the feasibility of providing external access. Once finalized, distribution methods should be widely announced to the community.
Topic started with a discussion of interest in continuing this workshop series. It was noted that the Workshop series is separately funded from the Unidata Program Center; however, the funds come from the same pot of money at NSF.
Committee members expressed appreciation for past events with the note that many attendees were either the usual suspects or local participants which resulted in a conversation on the ability to host this meeting in conjunction with another larger event. The last workshop proposal was late so moving forward this should be submitted much earlier allowing greater time for promotion. Funds support students, speakers, and committee members in attending the event. GOES-16 was discussed as a possible theme with the charge to identify outcomes as soon as a theme is finalized.
Josh Young/Mohan Ramamurthy
Brief to the committee on the current draft of the Strategic Plan. Clarification was provided on the intended use of ‘workflow’; consensus among committee members that workflow should be used less frequently in the document. The committee supported the emphasis on sharing tools, processes, and reproducibility.
The committee discussed the findings of the SVT (not publically available as it is the intellectual property of NSF).
The committee reached clear consensus on the next recipient to be notified by the Chair.
( NSharp is based on C it would be nice to not keep wrapping that)
Meeting Adjourned