Published using Google Docs
Python
Updated automatically every 5 minutes

Status Report: Python

April 2016 - September 2016

Ryan May, Sean Arms, Julien Chastang, Ward Fisher

Activities Since the Last Status Report

Python Training Efforts

Progress has been made on the following:

Dependencies, challenges, problems, and risks include:

MetPy

The MetPy community continues to grow slowly. There have been several externally driven Pull Requests, both for bug fixes and new features; the MetPy twitter account has also reached 105 followers. Both of Unidata’s summer interns, Alex Haberlie and Kristen Pozsonyi, spent the majority of their time working on additions to MetPy. Kristen’s time was especially well-spent; as a beginner to Python, the challenges she experienced have helped inform some areas where MetPy’s user experience needs improvement.

We also continue to improve MetPy’s open development model. Ideas for further development are captured in the GitHub issue tracker; current plans are to focus on enhancing the abilities to read point data (both BUFR and raw METAR) to facilitate generating NetCDF data for hosting on THREDDS, as well as simplifications to MetPy for working with units. The MetPy project is investigating some development changes, such as greater use of milestones for planning, a time-based release schedule, and active solicitation of feedback on these milestones in consideration of additional staff hired to work on Python, and to facilitate more community feedback in priorities.

Progress has been made on the following:

Siphon

Siphon represents our official Python support for TDS. While development has been slow of late, this is largely because its current capabilities meet current needs (versus some needs in MetPy). We anticipate developing Siphon to ensure that it is easy as possible to download data from a TDS in Python (such as crawling a server looking for data), keeping pace with new features added on the Java side.

External Participation

The Python team attends conferences as well as participates in other projects within the scientific Python ecosystem. This allows us to stay informed and to be able to advocate for our community, as well as keep our community updated on developments.  Ryan May has also continued to be an active participant in the matplotlib community, reviewing some pull requests and contributing several others. We also continue to host Jeff Whittaker’s netCDF4-python project repository; Jeff continues to be the active maintainer of the project. Ryan May is also now a member of the planning committee for the Python Symposium at the AMS Annual Meeting; he will also be presenting the Core Science Keynote for Python at the annual meeting, as well as helping bring the future generation of students into fold by presenting on Python at the AMS Student Conference.

Progress has been made on the following:

Ongoing Activities

We plan to continue the following activities:

New Activities

Over the next three months, we plan to organize or take part in the following:

Over the next twelve months, we plan to organize or take part in the following:

Beyond a one-year timeframe, we plan to organize or take part in the following:

Areas for Committee Feedback

We are requesting your feedback on the following topics:

  1. In light of the success of the Madison Python workshop, should we be offering more regional Python workshops? Is anyone willing to help sponsor?
  2. Does offering our training at AMS (or other conferences) seem like a worthwhile effort in order to take more advantage of the opportunity presented by a gathering of our community? Are there other conferences in addition to AMS you would suggest?
  3. Are there any additions you’d like to make to MetPy’s or Siphon’s development roadmap?
  4. What are the biggest obstacles that you see to the use of Python with other Unidata technologies, or for use in meteorology in general?
  5. We continue to maintain the Unidata Python Workshop with fresh, relevant, and up-to-date content. However, we would welcome feedback from our committees  on topics we may not be covering in the workshop.

Relevant Metrics

Siphon

MetPy

Unidata Python Workshop

Unidata Online Python Training

Strategic Focus Areas

We support the following goals described in Unidata Strategic Plan:

  1. Enable widespread, efficient access to geoscience data
    Python can facilitate data-proximate computations and analyses through Jupyter Notebook technology. Jupyter Notebook web servers can be co-located to the data source for analysis and visualization through web browsers. This capability, in turn, reduces the amount of data that must travel across computing networks.

  1. Develop and provide open-source tools for effective use of geoscience data
    Our current and forthcoming efforts in the Python arena will facilitate analysis of geoscience data. This goal will be achieved by continuing to develop Python APIs tailored to Unidata technologies. Starting with the summer 2013 Unidata training workshop, we developed an API to facilitate data access from a THREDDS data server. This effort has been encapsulated with the new siphon project, which is an API for communicating with a THREDDS server. Moreover, Python technology coupled with the HTML5 Jupyter Notebook technology has the potential to address "very large datasets" problems. Jupyter Notebooks can be co-located to the data source and accessed via a web browser thereby allowing geoscience professionals to analyze data where the data reside without having to move large amounts of information across networks. This concept fits nicely with the "Unidata in the cloud" vision and the goals outlined Unidata 2018 Five-year plan. Lastly, as a general purpose programming language, Python has the capability to analyze and visualize diverse data in one environment through numerous, well-maintained open-source APIs. The additional development of MetPy fills the need for domain-specific analysis and visualization tools in Python.
  2. Provide cyberinfrastructure leadership in data discovery, access, and use
    The TDS catalog crawling capabilities found in siphon will facilitate access to data remotely served by the Unidata TDS, as well as other TDS instances around the world.
  3. Build, support, and advocate for the diverse geoscience community
    Based on interest from the geoscience community, Unidata, as part of its annual training workshop, now hosts a three day session to explore Python with Unidata technology. Also, to advance the use of NetCDF in Python, Unidata has promoted Jeff Whitaker’s NetCDF4-python project, including hosting its repository under Unidata’s GitHub account. Unidata is initiating a project to provide online Python training specifically targeting geoscience students. Unidata is also fostering some community development of meteorology-specific tools under the MetPy project.

Prepared  September 2016