Published using Google Docs
IDV with RAMADDA
Updated automatically every 5 minutes

Status Report: IDV with RAMADDA

 October 2019 - March 2020

Yuan Ho, Julien Chastang

Areas for Committee Feedback

We have no questions at this time.

Activities Since the Last Status Report

IDV Release

IDV 5.7 has been released on February 5 of 2020.

IDV System Changes

 __Latest netCDF-Java Version__

The version of the netCDF-Java library currently distributed with the latest stable IDV version (5.6) is the 4.6.15-SNAPSHOT . The prior version of netCDF-Java to be distributed with the IDV was 4.6.12. There have been many improvements and bug fixes in that range.  The complete release notes for these versions can be found here.

 __IDV Certificates__

Java Webstart, Windows app and MacOS certificates have been renewed and will be valid until at least May 30, 2020 (MacOS certificate is valid until 2024). Moreover, as properly signing the IDV under these different environments can be an involved process, this information has thoroughly documented here.            

IDV Display Changes

 __Color Table__

New enhanced color tables have been added in this version, including AWIPS color tables and Satellite products customized color tables.

 __ NWS Hazard Warning__

NWS Tornado, Severe Thunderstorm, Flash Flood and Special Marine Warning KML files were supported in this release. A new menu item has been added to access the official NWS Warnings site directly under Data > Special data list.

 __CDM Profile data type__

Common Data Model profile feature data is a set of data points along a vertical line, including a collection of a time series of profile features at named locations. CDM Profile data can be loaded in the Data Choosers: RAOB > Soundings Local.

 __ABI and AHI Formulas__

Added a series of formulas specific to ABI (Advanced Baseline Imager), AHI (Advanced Himawari Imager) and MSG (Meteosat-8 through 11)to create displays of RGB and band subtraction products. These formulas can be found in the Field Selector under Formulas > Satellite. Also, derived fields for these products will be automatically added to the Field Selector when the multi bands ABI data source is loaded that includes bands required to create the product.

 __Grid Diagnostics Formulas__

Added a variety of formulas specific to gridded datasource to calculate Saturation Equivalent Potential Temperature, Heat Index, Helicity, Ellrod Index, LP Index, and Richardson number.       

 __Latest Version of VisAD__

The SSEC team at UW, Madison has made a number of improvements to support 3D trajectories.

 

IDV EarthCube Drilsdown Project

In collaboration with University of Miami Professor Brian Mapes, the project facilitates the use of data visualizations within curated, executable notebook documents, in the service of improving statistical algorithms for the data­rich geosciences. The project integrates Jupyter Notebooks as the medium for literate computing and documentation; Unidata's IDV for its powerful data integration, visualization, and algorithm design capabilities; and RAMADDA as a metadata-rich repository for the packages that are the objects of this workflows.

IDV Publication Highlights

Synoptic–Dynamic Meteorology in 3D: Introducing an IDV-Based Lab Manual by Gary Lackmann, B. Mapes and K. Tyle

A Google Scholar Search reveals a number of publications that cite use of the IDV (doi:10.5065/D6RN35XM).

IDV and RAMADDA Training, Conference Attendance and Presence

 __2019 AGU Fall Meeting__

 __2019/2020 IDV lectures at WRF tutorial workshop__

Ongoing Activities

We plan to continue the following activities:

 __NetCDF Java 5.0.0 integration__

The new version of netCDF-Java will be integrated with the IDV. There have been many improvements and bug fixes in this new netCDF java library.

 

 __Investigation of Java 3D Alternative__

Because of concerns about the long-term viability of the open-source Java 3D project, the IDV team has begun discussions with our University of Wisconsin, SSEC collaborators to replace Java 3D with a more viable alternative within the VisAD API. We have started investigating whether the Ardor 3D can meet that objective. Looking into alternatives to Java 3D was also a goal described in the Unidata 2018 Five-year plan.

New Activities

Over the past few months, we plan to organize or take part in the following:

We plan to finally migrate away from Oracle Java 8u51 and towards a more modern version of OPenJDK Java. This switch will necessitate  altering the IDV building and distribution workflow to work with OpenJDK.

Relevant Metrics

 __E-Support__

The IDV team continues to provide the geoscience community with high-quality support through e-support software and idv-users mail list. In the last half year the IDV team has closed ~20 e-support tickets.  Each individual ticket may and often does involve many back-and-forth messages. There is an especially large number of support requests coming from international users.

Top ten universities running IDV are: Millersville, Oklahoma, University of Utah, St Cloud state, Plymouth, NC State, West Kentucky, Lyndon State, University of Illinois, and San Francisco State.

 __GitHub Pull Requests__

In the area of greater collaborative development, since the migration of the IDV project to github, we have closed a total of 125 “pull requests” or code contributions from internal and external collaborators.

 __Youtube IDV Instructional Videos__

In the area of online IDV training, the Youtube IDV instructional videos have been viewed thousands of times.

Strategic Focus Areas

We support the following goals described in Unidata Strategic Plan:

  1. Managing Geoscience Data
    The IDV is a state of the art geoscience visualization application. It gives users the ability to view and analyze a rich set of geoscience data, including real time data, in a seamless and integrated fashion. This analysis is captured in IDV bundles. RAMADDA is a content management system and service specifically tailored towards the sharing and distribution of IDV bundles facilitating distribution of scientific data and analysis.
  2. Providing Useful Tools
    The IDV has been an open-source project for several years. The IDV is available on the github version control platform for greater open-source collaboration. The IDV provides users the unparalleled ability to analyze, integrate, and visualize heterogeneous geoscience data in two, three, and four dimensions. The IDV coupled with RAMADDA enables geoscience specialists the capability to share and collaborate their IDV analysis via social scientific networks.
  3. Supporting People
    Unidata offers yearly multi-day training and occasionally regional workshops for IDV and RAMADDA. The IDV coupled with RAMADDA enables our earth science community partners to distribute geoscience data and metadata through web-based technologies thereby fostering scientific collaborations. Moreover, the IDV's ability to share bundles through RAMADDA creates a scientific social and collaborative network for the geoscience community.


Prepared March 2020