Status Report: AWIPS II and GEMPAK
September 2015 - April 2016
Michael James
AWIPS II Activities Since the Last Status Report
Python Data Access Framework (python-awips)
In March 2016 I published a standalone Python Data Access Framework (DAF) for retrieving data from an AWIPS II EDEX server. This package allows users to create a data request to be sent to a remote EDEX data server (examples use edex-cloud.unidata.ucar.edu). The interface will return Python data objects which are easily convertible to Numpy arrays, to be rendered with existing Python packages such as Matplotlib/Basemap, Cartopy, and MetPy.
- The DAF was released as its own Python package and is available on PyPI:
- Bundled in the AWIPS II Python Stack but runs on its own as well:
- yum install awips2-python-awips
AWIPS II User Guide on Github Pages
This new AWIPS II User Guide on Github Pages expands on the original Unidata AWIPS II workshop documents and provides data visualization examples for a growing set of CAVE plugins.
For the AWIPS II User Guide, I published a new framework for building documentation on Github Pages. Polymer Web Components, Jekyll, Vulcanizer, and Grunt are used to publish rich and dynamic Markdown-based documents.
AWIPS II Release Notes
15.1.1, Jan 21, 2016
- First OS X CAVE client
- Moved from Java 1.6 to 1.7
- OS X client required openGL bindings be refactored from JOGL 1 to JogAmp 2
- Added install scripts to manage EDEX and CAVE packages (config, install, update)
- Various RPM install/uninstall bugs
- Automatically reset purgejobs on startup (which was preventing EDEX from scouring data and thus filled up disks)
- MRMS grids support both from NOAAport and via direct feed from NSSL
- Various GFE client fixes (muted vcmodule to allow the GFE client to launch)
- Disabled all nwsauth, useradmin, and archive plugins
- Qpid updated to 0.32 to solve restart problems
- New colormaps for GOES sounder McIDAS product CAVE displays
- Muted all LDAD components
- Tested and confirmed archive plugins, muted for build (to prevent disks from filling)
- Image export now includes timestamp in the filename by default (to avoid repeated overwriting prompt for default filename)
- Incorporated a postgres radar table update into the execution of an RPM update to avoid users having to uninstall/reinstall their database.
15.1.2, released Feb 23, 2016
- OS X client now runs off its own bundled JRE
- Built libgempak and libnsharp for OS X distribution
- Single file sourced from /etc/profile.d/ rather than multiple files.
- ECMWF global grid compositing (replacing the previous N Hemisphere-only compositing)
- CAVE menubar buttons to open NCP/GEMPAK-like data displays from within D2D
- New satellite imagery in menus from UNIWISC feed
- New grid bundles for default models, ocean models, and mesoscale models.
- New isentropic analysis grid bundle
- Re-enabled custom colormaps
- GSD ensemble grid display added to OS X client
- Updated LDM to 6.12.14
- New bundles for ESTOFS and ETSS
- Extra Tropical Storm Surge
- Extra Tropical Combined Storm Surge and Tide
- Ocean Surface Elevation
15.1.3, released March 21, 2016
- Full global topographic map (finally!)
- Maps database and shapefile updates from the newly-created awips2-static repository.
- Maps database RPM now updates itself correctly (in the past it would skip update/install if the database already exists, now it overwrites).
- Expanded grib1->grib2 variable mapping to support FNMOC grids.
- New WaveWatch/WW3 bundles:
- Sig. Wave Height
- Wind Wave Height
- Swell Height & DIrection
- Peak Wave Period
- Sfc Wave Wind
- Primary Wave Period
- Secondary Wave Period
- Sfc Vorticity and Wind
- 4-panel displays of the above
- UKMET global grid compositing (replacing the previous N Hemisphere-only compositing)
- New LDM pattern actions available:
- DGEX Alaska 12km
- GFS Pacific 40 km
- GFS Pacific 20 km
- GFS CONUS 80 km
- GFS CONUS 20 km
- GFS Alaska 20 km
- NAM CONUS 80 km
- NAM CONUS 20 km
- NAM Puerto Rico Grid 237
- NAM Polar 90 km
- AK NamDNG 5km
- PR NamDNG 5km
- Hawaii NamDNG 5km
- AK NamDNG 3km
- Gridded Excessive Precipitation
- New grids supported via changes/additions to the baseline:
- HPCqpfNDFD
- MOSGuideExtended
- FNMOC NAVGEM
- FNMOC NCODA
- FNMOC FAROP
- FNMOC COAMPS Regional Grids
- MPE/QPE Puerto Rico
Ongoing Activities
Cloud Server Updates
- Now ingesting and serving 20+ gridded models due to some curious performance increases that happened on the Azure cloud.
- DGEX
- ECMWF
- GFS 0.5 Global
- GFS 40km
- NAM 12km
- NamDNG 2.5km
- NAVGEM
- NCWF
- RAP 13km
- RAP 40km
- RTMA 2.5km
- URMA 2.5km
- UKMET Global
- HRRR
- Global Wave Watch III
- ESTOFS
- ETSS
- FFG
- fnmocWave
- HFR
- HPC / RFC / QPF / QPE grids
16.1.4 release is being prepared
- Requires clients and servers be updated together because 15.x and 16.x are not cross-compatible.
- More map projections.
- Himawari-8 support.
- Better support for native projection McIDAS files.
- New maps database and shapefile imports for StormSurgeWW and NHAdomain
GEMPAK
GEMPAK v7.2.3 was released in early 2016, incorporating NCEP table and map updates as well as expanding support for Python access to remote EDEX data. Originally the build strictly required the AWIPS II Python packages be installed, but it now builds against system Python and only requires the python-awips (DAF) be installed for GEMPAK programs to accepts prepended aliases for gridded models, such as A2GFS, A2NAM, A2HRRR, etc.
New Activities
Over the next three months, we plan to organize or take part in the following:
- Investigate again the refactoring of pypies for h5py 2.6 compatibility.
- Collaborate with the Unidata Python team to integrate python-awips into workshop training materials and possibly into existing Unidata technologies.
- Move python-awips array manipulation to callable methods.
Relevant Metrics
Web server statistics?
Data download statistics?
Something else useful?
Strategic Focus Areas
We support the following goals described in Unidata Strategic Plan:
- Enable widespread, efficient access to geoscience data
Both AWIPS II and GEMPAK are freely available, and both incorporate LDM/IDD technology for efficiently accessing geoscience data. The cloud-based data server maintained by Unidata invites a number of users and we are seeing more adoption with time, as the software becomes more stable, more platforms are supported, more data is served conveniently and freely to the community.
- Develop and provide open-source tools for effective use of geoscience data
Both AWIPS II and GEMPAK are open-source, and while GEMPAK is now in maintenance mode, AWIPS II is continuously being developed.
- Provide cyberinfrastructure leadership in data discovery, access, and use
Unidata is the only known entity to provide a freely-available and non-operational version of the AWIPS II software package.
- Build, support, and advocate for the diverse geoscience community
Using LDM/IDD technology to provide access to real-time meteorological data; providing visualization tools for data analysis.
Prepared April 2016