Status Report: McIDAS
April 2019 - October 2019
Tom Yoksas
Areas for Committee Feedback
We are requesting your feedback on the following topics:
- Are there any features that users would like to be added to Unidata McIDAS-X and/or ldm-mcidas?
Activities Since the Last Status Report
Aside from routine updates/bugfixes to existing code and tables, the main area of activity was the creation of ADDE servers for NOAAPort-delivered GOES-16/17 satellite imagery and merging the resulting code into the McIDAS core at UW/SSEC.
Current Activities
- Unidata McIDAS version 2019 is currently being prepared for release
v2019 will include all SSEC versions up to and including the current McIDAS-X and -XCD releases, both of which are v2019.1.
- McIDAS-X is used to convert GOES-16 ABI imagery that is in netCDF4 format to McIDAS AREA format that is usable by all supported display and analysis packages except Python/MetPy for the Unidata-Wisconsin (UNIWISC aka MCIDAS) IDD feed.
The latest releases feature the following:
- Updated GOES-R Series ABI servers to store GOES-16 and -17 Temperature Data Quality Flag (TDQF) information and report the values in IMGLIST command’s FORM=AUX output, support ABI L2 image products that derive from NOAAPORT distributed image tiles, remove restrictions on names of directories containing the ABI files, and to skip past corrupted files so that IMGLIST and other commands will proceed to subsequent “good” files.
- Updated VIIR servers with improvements and enhancements. The changes include increasing the number of files allowed in a dataset from 5,000 to 20,000 and removing previous dataset descriptor naming requirements. The servers were also updated to use the SATANNOT file’s new sensor source numbers 320 (SNPP-SDR) and 321 (NOAA20-SDR).
- Added DAY= and TIME= keywords to IMGPROBE command to allow use with archive or other servers that require the DAY= keyword to be specified. The D command (Alt-D) was also updated to include the day value from the frame directory in its request so that it also works with those servers.
- Updated FRMLABEL and ZA commands to include new (DAYJ) placeholder that plots the date in format dd mon ccyyddd, e.g., 12 OCT 2019285. Also updated FRMLABEL to identify RGB frames (displayed using the RGBDISP command) and thus apply an appropriate image label with no overlapping characters.
Ongoing Activities
We plan to continue the following activities:
- SSEC McIDAS Advisory Committee (MAC)
The UPC (Yoksas, Ho) continues to participate as the Unidata representative to the McIDAS Advisory Committee (MAC) that is operated by SSEC.
The MAC was assembled by UW/SSEC to advise SSEC on McIDAS-X users needs/concerns/desires for development in the next generation McIDAS, McIDAS-V. The MAC was modeled after the Unidata IDV Steering Committee.
- Interest in McIDAS by non-core users
The UPC occasionally receives requests for McIDAS-X and help using McIDAS-X from international university users, U.S. government agencies and other non-traditional Unidata users (e.g., private businesses, etc.). Government agencies and non-traditional Unidata users are referred to UW/SSEC for access to McIDAS; international educational community user requests are granted on a case-by-case basis after they provide a clear statement of their acceptance of the terms of use provided by SSEC.
- Continued support of existing and new community members
New Activities
- Add support for new types of data when they become available, otherwise McIDAS-X support is in maintenance mode.
Relevant Metrics
- Data delivered by the Unidata McIDAS ADDE servers exceeds 1 TB/day. The great majority of the data being served is imagery from GOES-16 followed by imagery from GOES-17.
ldm-mcidas Decoders Activities
Development
ldm-mcidas releases are made when needed to support changes in software development and operating system environments. ldm-mcidas v2012 was released at the end of September, 2012.
The next major addition to this package is the development of a “decoder” for GRB delivered Geostationary Lightning Mapper (GLM) data. This development is aimed at greatly increasing the speed at which displays of the GLM data can be made in McIDAS-X, the IDV and McIDAS-V.
Geostationary Satellite Data Ingest and Data Serving
Unidata continues to ingest GOES-East and GOES-West imager data at the UCAR Foothills Lab and NCAR Mesa lab campuses in Boulder.
- Direct, programmatic access to real-time GOES-East (GOES-16) and GOES-West (GOES-17) data via McIDAS ADDE has been averaging over 40 TB/month since the since the late spring of this year
Planned Activities
Ongoing Activities
Continued ingest, distribution via the IDD and ADDE serving of GOES-East and GOES-West imagery from the GRB downlinks we installed in UCAR
Continued ingest and ADDE serving of GOES-15 and GOES-14 (when available) imagery. GOES-15 is scheduled to be decommissioned on December 31 of this year. GOES-14 will remain in its standby location (104W) and will be turned on for periodic testing as needed.
These efforts require maintenance of the satellite ingest and data serving equipment.
New Activities
Establish a testbed for generating Level 2 products from GOES-16/17 imagery and select model output. The intention is to be able to test vetted algorithms submitted by community members for a long enough period for the algorithms to be fully tested.
Strategic Focus Areas
We support the following goals described in Unidata Strategic Plan:
- Managing Geoscience Data
Remote, programmatic access to data provided by the Abstract Data Distribution Environment (ADDE) environment of McIDAS has been a model for the development of remote access methodologies since 1994. Concepts articulated in ADDE inspired the development of THREDDS (to address the lack of rich metadata available in ADDE) and RAMADDA. ADDE remains one of the most used data services in the Unidata suite. ADDE servers operated by Unidata are currently serving in excess of 1.4 TB/day.
- Providing Useful Tools
McIDAS remains the application of choice for the satellite meteorology community. The Abstract Data Distribution Environment (ADDE) component of McIDAS was the first application offered by Unidata to provide remote, programmatic access to a wide variety of data that is important to the atmospheric science community.
The fifth generation of McIDAS, McIDAS--V, unlike its predecessors, is a fully open source application that is in wide scale and growing use in the worldwide satellite meteorological community
McIDAS ADDE continues to evolve and provide access to a rapidly increasing volume of imagery and non-image data.
- Supporting People
McIDAS is sot for use by those interested in satellite meteorology worldwide.
Prepared September 2019