Published using Google Docs
Python
Updated automatically every 5 minutes

Status Report: Python

October 2021 - May 2021

Ryan May, Drew Camron, Julien Chastang, Nicole Corbin

Areas for Committee Feedback

We are requesting your feedback on the following topics:

  1. Does the requirement of using units through Pint impede your teaching or use of MetPy?
  2. To you and your own community, what enhancements to documentation, tutorials, or examples would most benefit you, especially in the short-term?

Activities Since the Last Status Report

Python Training Efforts

Supported by engineers and Unidata’s instructional designer, Python education remains a focus for workforce development, community engagement, and user feedback. Synchronous and asynchronous virtual efforts continued throughout 2021-2022, and synchronous in-person activities are hoped to resume this year. Synchronous virtual workshops and tutorials have remained mostly bespoke, though we’ve learned more about the reusability and packaging of our materials, and plan to modularize these to better suit the needs of our community. We continue to refine our resources to be more accessible, with an eye towards supporting new institutions and communities.

To support this, we have refined the goals of our expanded Python Training resources and are close to having  a new draft website built on sustainable technologies in-line with our other Python projects. Lessons learned from our educational collaborations and the fast-moving Jupyter ecosystem are being leveraged to best fit the Python learning needs of our community.

Progress has been made on the following:

MetPy

Development continues to be driven by requirements for our dedicated awards (in addition to bug reports and pull requests from community members). MetPy 1.2 was released in January 2022 with a variety of fixes and enhancements including:

MetPy 1.3 (April 2022) and 1.3.1 (end of May 2022) were also released, dropping support for Python 3.7 and providing a variety of bug fixes.

The MetPy team has embarked on a plan to increase the release cadence for the project to every other month. This is aimed to avoid having releases slide to get “one more thing” in the release, and instead more readily get developments in the hands of the user–whenever they choose to upgrade.

 Moving forward, 1.4 is planned for release at the end of July. This will include the long-promised support for plotting fronts and analysis from the WPC. It will also finally include the full corrections of spherical terms for calculations involving spatial derivatives. More broadly, we will also be continuing the performance improvement work that is the focus of the CSSI award, as well as incorporate the “automated solver” from the previous award.

The 2022 MetPy User Survey was conducted from April to mid-May 2022. On this survey, 72.8% of users scored MetPy as a 4 or higher (on a 5-point scale) when asked “In your experience how easy is it to use MetPy for your various activities?”.  88.2% also rated the quality of MetPy’s documentation as either “Good” or “Excellent”. On the contribution side, while only 8.5% of respondents had submitted a Pull Request (PR) to MetPy, 13.6% had contributed PRs to other projects, while 37.3% had considered contributing to MetPy. The respondents to the survey were overwhelmingly from the University/Education sector (65.5%), with 84.7% indicating “Research” as one of their primary uses of MetPy.

Progress has been made on the following:

Siphon and Data Processing

Siphon continues to exist in a steady state–continued maintenance and use, but minimal feature advancement. Some of this is due to limited development resources being focused on MetPy’s needs; it is also due to limited pressing needs on the data access side. Largely, Siphon meets the needs we have identified for Python data access (that aren’t also already met by zarr, xarray, etc.). With that said, Siphon does remain an important part of the stack used by our training work, and by Unidata’s community of Python users in general. The most pressing developments we anticipate for Siphon are improvements to working with Siphon in interactive sessions, like the Jupyter notebook environment: improved catalog crawling interface, better string representations, and tab completion.The decision has been made to separate non-TDS functionality (e.g. Wyoming Upper Air archive access) out into a new remote-access toolset contained within MetPy, and we hope to begin this transition work soon.

We also continue to maintain the LDM Alchemy repository as a collection of LDM processing scripts in Python. Currently this includes the code powering the AWS NEXRAD archive as well as the program that reconstitutes NOAAPORT GOES-16/17 imagery. As we transition more of our internal data processing to Python, this repository will hold those scripts. We have seen several community questions regarding both the GOES and NEXRAD processing software.

External Participation

The Python team attends conferences as well as participates in other projects within the scientific Python ecosystem. This allows us to stay informed and to be able to advocate for our community, as well as keep our community updated on developments. As participants in a broader Open Source software ecosystem, the Python team regularly encounters issues in other projects relevant to our community’s needs. As such, we routinely engage these projects to address challenges and submit fixes. We also continue to host Jeff Whittaker’s netCDF4-python project repository; Jeff continues to be the active maintainer of the project. The overall involvement helps ensure that important portions of our community’s Python stack remain well-supported. Ryan May continues to serve as a core developer for CartoPy as well as a member of Matplotlib’s Steering Council and conda-forge’s core team.

Progress has been made on the following: 

Ongoing Activities

We plan to continue the following activities:

New Activities

Over the next three months, we plan to organize or take part in the following:

Over the next twelve months, we plan to organize or take part in the following:

Beyond a one-year timeframe, we plan to organize or take part in the following:

Relevant Metrics

MetPy

Siphon

Strategic Focus Areas

We support the following goals described in Unidata Strategic Plan:

  1. Providing Useful Tools
    Python has become a key tool in the atmospheric sciences, and the geosciences in general. MetPy leverages the rest of the scientific Python ecosystem to provide a suite of documented and tested domain-specific functionality, supporting greater use of Python by the community. Siphon serves to provide access to the growing collection of remote data sets. Together, MetPy and Siphon give the community a platform for scripted analysis of real-time and archived weather data. These tools are also readily used in the Jupyter Lab/Notebook environment, for ease of use in cloud and HPC computing environments, facilitating data-proximate analysis. We also participate in a variety of projects in the broader scientific Python ecosystem, to help ensure the ecosystem’s viability and that it continues to meet our community’s needs.
  2. Supporting People
    We provide a variety of online training resources to facilitate our community’s education and use of Python. We also regularly conduct training workshops to teach attendees how to use tools and apply them to their problems and challenges in research and education.

Prepared  May 2022