Published using Google Docs
AWIPS
Updated automatically every 5 minutes

Status Report: AWIPS

November 2022- March 2023

Tiffany Meyer, Shay Carter

Executive Summary

Over the last year, the AWIPS team has been hard at work to put out its first major version release.   The current AWIPS team has been maintaining and adding updates to the last major version released, which was a variant of the NWS’ AWIPS v18.1.1.  We now have a beta version of AWIPS based off of NWS’ v20.3.2, available with key updates such as Python3 and Java11.

At the end of February 2023, we released our first beta of v20.3.2 (called 20.3.2-0.1) with two installation options for CAVE:

We also have a publically accessible beta EDEX available at edex-beta.unidata.ucar.edu.  Along with the release, we updated our documentation to have a dedicated page for installing the beta, and a new Google form specifically for reporting bugs/unexpected behavior with v20.  

As of early March, we have resolved significant challenges we were having with regards to building CAVE directly for MacOS and Windows.  We are hoping to put out a second release of the beta (v20.3.2-0.2) by the end of March that would include:

Aside from AWIPS development, the team is proud to have the first two Unidata online training modules in Learn AWIPS CAVE, and Learn Python-AWIPS.  We also just completed two consecutive years of releasing AWIPS Tips blogs every other week. The AWIPS team is also looking to host a workshop at AMS next year on CAVE.

Questions for Immediate Committee Feedback

As stated, the AWIPS team is looking to host CAVE training at AMS next year, what format would be more beneficial to the community - short course or student workshop?

Activities Since the Last Status Report

AWIPS

Our EDEX servers have been continuously running on the new Jetstream2 platform since our transition last summer, despite various hiccups with the new platform.   The AWIPS team was also able to experiment with a newly-offered “large” virtual machine instance during testing for the v20 beta.  We currently have both production v18 and beta v20 EDEX servers which continue to serve real-time weather and geographic data to CAVE clients and the python-awips data access framework API. 

The main challenge we investigated with the “large” VM instance, was whether one machine could handle all of the data we process and make accessible through our EDEX.  Unfortunately, with the given constraints of queuing architecture in EDEX, even the “large” instance was not capable of sustaining our entire real-time data flow.  However, through the use of ancillary EDEX machines we have been able to decouple certain datasets from the main EDEX instance. We still take advantage of distributing EDEX workload over three machines: a main EDEX, an ancillary radar EDEX, and an ancillary satellite EDEX.  These distributed architectural concepts of AWIPS allow us to scale EDEX in the cloud to account for the size of incoming data feeds.  

Texas A&M has also been using a similar distributed architecture since Summer of 2021.  We have worked closely with them since they first installed their system, as we’ve released new versions of AWIPS, to help them transition between versions and identify weaknesses in their EDEX server.

Since the last status report, we’ve put out a beta release for version 20 of AWIPS (20.3.2-0.1). A link to all of our AWIPS release notes can be found here.

This release is the first major version upgrade since the current AWIPS team has started.  Version 20 has some major upgrades in terms of dependent software as well, with the following significant upgrades:

For this beta release the AWIPS team has put out a direct install option for CentOS7 via new RPMs, and a Virtual Machine option that runs on both Windows or CentOS.  We have a public v20 EDEX available at edex-beta.unidata.ucar.edu.  Our beta release of CAVE includes all the latest NWS updates and features, as well as a few of our Unidata updates.  Any bugs or unexpected functionality can be reported through this brief form.

The AWIPS team is currently working on releasing direct install options for both Windows and MacOS.  Our next release will also include a few changes in response to issues that were identified in our beta reporting form, linked above.  The team intends to release updates to the beta as we are able to incorporate previous Unidata-added functionality into CAVE and EDEX.  We are hoping to have a production version of 20 by the end of the year.

Since last summer, the AWIPS team has been successfully maintaining and updating our ISatSS installation which allows us access to the NOAA-created GLM products.  This work has been assisted by Brian Rapp and Lee Byerle from NOAA.

A significant portion of our documentation both for CAVE, EDEX, and python-awips has been modified for easier understanding and comprehension.  We are continuing to update our python-awips example notebooks to follow our new template that contains a helpful table of contents, with consistent subsections across the various example topics.

Our blog series, AWIPS Tips, has successfully been running every other week for almost two years now.  As of March 22nd, 2023, we will have released 52 blog entries (two consecutive years worth)!   A current list and breakdown of all the entries is provided on our documentation website in the Educational Resources page. We plan to continue the blog series for the foreseeable future and have several more ideas already planned out for upcoming entries.  Announcements for new blog posts are shared through our mailing list (awips2-users@unidata.ucar.edu), and our social media accounts (Facebook, Twitter, LinkedIn, and YouTube when applicable).

Our asynchronous CAVE training course has been live since October 2021.  We have had over 100 users sign up and take our course.  We encourage those who have never used CAVE and those who have but still might be fairly new to the software to sign up and try out the course for themselves.  

Along the same lines, our second training module, Learn Python-AWIPS, has been live since September 2022.   Learn Python-AWIPS is tailored for users with entry level Python knowledge, and zero python-awips knowledge.  The course walks learners through the process of using python-awips to connect to an EDEX, interrogating the server to find what data is available, constructing and tailoring a refined data query, and displaying the results of the data in a user-friendly plot.  After completing the course, the learner will have a fully functioning Jupyter Notebook example from their work following along in the course.  The skills and knowledge gained from the course will hopefully allow the learners to then easily understand and alter our example notebooks to be able to produce customized, useful data plots.  We have had a handful of users take this course since we released it and are actively encouraging our University members to take the learning course themselves, and recommend it to their students as well.  We know Python is growing in popularity in the meteorologic field and would like to offer this learning experience as a tool for professors and students to take advantage of.

The AWIPS team still has an active support evaluation survey that is advertised in our support email signatures.  The majority of our feedback has been overwhelmingly positive, and the graphic below is a summary from all responses we’ve received regarding the quality of service we provide:

Forms response chart. Question title: Service Quality. Number of responses: .

Some of the latest open-ended feedback from the support evaluations includes the following:


“Your AWIPS Team is awesome.  I'm getting responses back usually in less than two to three hours.  More responsive than the NWS AWIPS Program Office!!”

“Tiffany was able to correctly answer and have the problem fixed in a timely fashion!”

“They are incredibly knowledgeable and approachable. They always go the extra mile to help. They are appreciated very much.”

At AMS 2023, the AWIPS team presented in a new Unidata session titled “Community Driven: Unidata Projects Enhancing Geoscience Teaching and Research”.  The talk, “Unidata AWIPS Hosted in the Cloud” covered an overview of AWIPS capabilities, the differences between our version and the NWS version, use of the Jetstream2 cloud platform, and our current development milestones and outlooks.  The team also had the chance to demonstrate CAVE and AWIPS, and interact with users at the Unidata booth during the AMS Student Career Fair. 

The AWIPS team is planning on submitting to host a workshop at AMS 2024.  At this time, we are still deciding if we’d like to host a “short course”, or a student workshop.  The idea is the workshop would focus primarily on CAVE, but could potentially touch on python-awips as well.

The team has also received several requests in the recent past inquiring about workshops focused on the EDEX side of AWIPS.  Most of these requests have come from either private companies or varying government agencies.  The team is considering planning an EDEX workshop for sometime in the future.

Software Releases

Since our last status report we have put out a beta for version 20 (20.3.2-0.1).  This update was made available for direct install on CentOS via new RPMs, and in a Virtual Machine which can be run on Linux or Windows.  We also have a publicly available version 20.3.2-0.1 beta EDEX server, hosted in our cloud allocation.

We are still actively developing the Unidata AWIPS beta release version 20 of AWIPS.  We are anticipating upcoming releases of the beta to support directly installing CAVE on both MacOS and Windows.  Additionally, we plan to make the EDEX installation available as we transition towards a production version of 20. Our plan is to continue releasing a few versions of the beta until we are comfortable supporting a production release, hopefully by the end of the year.

Activities Ongoing/In-Progress

AWIPS development activities are constantly ongoing.  Currently the following activities are in progress:

Future Activities

Future plans are constantly evolving to meet the needs of our users.  The AWIPS team is focused on developing and improving the current beta release (version 20).  This includes updating our python-awips package to work with our new beta version 20 of AWIPS. We are also looking forward to the opportunity of hosting an in-person workshop at AMS 2024.  The team is actively participating in conferences, workshops, and virtual message boards (blogs) to expand our user base.

Metrics

Downloads November 1, 2022 - February 28, 2023

AWIPS downloads: 1,666

Strategic Focus Areas

We support the following goals described in Unidata Strategic Plan:

  1. Managing Geoscience Data
    The cloud-based EDEX data server continues to see widespread use and growing adoption. More and more datasets continue to be added to the server as Unidata deploys more decode/ingest nodes.
  2. Providing Useful Tools
    All     AWIPS tools (EDEX, CAVE, and python-awips) are freely available, and also incorporate LDM/IDD technology for accessing geoscience data.
  3. Supporting People
    At this juncture, we are providing full technical support with regards to AWIPS for the community.

Prepared  March 2023