Published using Google Docs
IDV with RAMADDA
Updated automatically every 5 minutes

Status Report: IDV with RAMADDA

 April 2024- September 2024

Yuan Ho, Julien Chastang

Executive Summary

We continue to support, update, and enhance the 3D  data visualization and analysis tool IDV for our community. Our current activities include:  coordinating with netCDF-Java group to add new data formats, collaborating with the SSEC developers to enhance the VisAD library, and working with our community to promote the usage of the IDV in research and education.

Questions for Immediate Committee Feedback

We have noticed that many advanced features of the IDV, such as formulas and trajectory displays, have not been widely used in the community and many data servers that the IDV can directly access are less well known to IDV users. We would like to provide help to classes, research groups and project teams to use these resources. Can committee members help to establish such connections?

Activities Since the Last Status Report

IDV Releases

The IDV 6.2u2 was  released in December 2023.

The release is currently on hold due to an issue with the renewal of the Windows code signing certificate.

IDV System Changes

 __IDV Certificates__

Java Windows app and MacOS certificates have been renewed and will be valid until at least May 30, MacOS certificate is valid until 2024). Moreover, as properly signing the IDV under these different environments can be an involved process, this information has been thoroughly documented here.            

 

 __Changes to nightly release that will eventually be incorporated into into stable version__

IDV Display Changes

 __Pre/Post process data for ML applications  __

The IDV offers a range of statistical analysis formulas, encompassing area averages, level averages, maximum, minimum, mean, percentiles, and summations. The results of these analyses can now be produced as non-geolocated data and exported in formats such as CSV or netCDF. This newly introduced feature empowers users to leverage the IDV's versatile access to multiple data servers, enabling them to preprocess data for applications including machine learning and other scientific uses.  

 __ More with Level II Radar Grid Displays __

With the newly developed Level 2 radar grid display feature, we've expanded its capabilities by incorporating derived formulas to calculate radar precipitation rates. These calculations are based on two key approaches: the Marshall-Palmer drop size distribution and dual polarization radar data. The Marshall-Palmer method provides a traditional estimate of precipitation rates using reflectivity, while the dual polarization approach enhances accuracy by factoring in both reflectivity and differential reflectivity. These advancements allow for more precise and varied precipitation rate calculations, improving radar data interpretation and weather analysis.         

 __Multi Variables Cross-Section Display__

After creating the first vertical cross-sections display, you can add a contour cross-section display for a second variable as well as a wind vector cross-section display for a derived variable. When working with multi variable cross-section displays, we recommend using a color-filled contour display or a color-shaded display, and contour displays for the second or the third variables. Multi-variable cross-section displays offer several advantages in data visualization and analysis, such as enhanced data comparisons, comprehensive analysis, and efficient use of screen space. Additionally, you can now switch the vertical coordinate scale from meters to pressure in hPa, providing greater flexibility in interpreting the data.

 

 __Two Variables Time Height Display__

A time-height display shows samples of a 3D parameter along a vertical profile from top to bottom of the available data, with time as the independent coordinate (x-axis). You can choose between contour, color-filled, and color-shaded time-height displays. After creating the time-height display for the first variable, you can add a contour time-height display for a second variable. This setup allows for a more detailed and layered analysis of vertical atmospheric data over time.

 __Zoom Enhancement__

We have updated the algorithm responsible for calculating the clip distance during zoom operations in both map and globe views. This enhancement ensures that users can now zoom in to street level without experiencing the disappearance of 3D objects, improving the overall user experience. The refined algorithm dynamically adjusts the clip distance based on zoom levels, allowing for seamless and detailed visualization at various scales. This update is particularly beneficial for users requiring precise close-up views in 3D environments, making the zoom functionality more reliable and effective.

IDV Community Support                                                           

With the tightening of computer system security, it has become more challenging for our community to host data and bundles on their own systems. As a result, UNIDATA RAMADDA is now hosting the IDV LMT Lab Manual, which is widely used in university weather teaching and laboratory settings.

https://ramadda.unidata.ucar.edu/repository/entry/show?entryid=fa7adc01-66a4-40ad-a89f-ec38be50e935

MSU IDV Project

 I have collaborated with professor Sun from MSU and submitted the proposal, "Scientific Visualization and Mathematical Modeling of Weather Data: An Interdisciplinary Approach to Learning with IDV (Interdisciplinary IDV)," to NSF. This joint endeavor seeks to weave together the realms of scientific visualization and mathematical modeling, using the Integrated Data Viewer (IDV) as a central tool. By capitalizing on the unique features of the IDV, we aim to provide a comprehensive platform for hands-on exploration of weather data, empowering learners to engage deeply with the intricacies of mathematical modeling in the context of atmospheric sciences.

This collaboration with MSU exemplifies a commitment to cross-disciplinary education and research, promising to contribute valuable insights to both meteorology and education. Together, we anticipate achieving impactful outcomes that advance the understanding and application of scientific principles in the dynamic field of weather data analysis.

IDV Publication Highlights

Synoptic–Dynamic Meteorology in 3D: Introducing an IDV-Based Lab Manual by Gary Lackmann, B. Mapes and K. Tyle

A Google Scholar Search reveals a number of publications that cite use of the IDV (doi:10.5065/D6RN35XM).

IDV and RAMADDA Training, Conference Attendance and Presence

  __2025 AMS Annual Meeting__

Ongoing Activities

We plan to continue the following activities:

  __Investigation of Java 3D Alternative__

Because of concerns about the long-term viability of the open-source Java 3D project, the IDV team has begun discussions with our University of Wisconsin, SSEC collaborators to replace Java 3D with a more viable alternative within the VisAD API. We have started investigating whether the Ardor 3D can meet that objective. Looking into alternatives to Java 3D was also a goal described in the Unidata 2018 Five-year plan.

New Activities

Over the past few months, we plan to organize or take part in the following:

We plan to upgrade the version of OPenJDK Java. This change will necessitate in depth testings and the IDV building and distribution workflow.

Relevant Metrics

 __E-Support__

The IDV team continues to provide the geoscience community with high-quality support through e-support software and idv-users mail list. In the last half year the IDV team has closed ~40 e-support tickets.  Each individual ticket may and often does involve many back-and-forth messages. There is an especially large number of support requests coming from international users.

Top ten universities running IDV are: Millersville, Oklahoma, University of Utah, St Cloud state, Plymouth, NC State, West Kentucky, Lyndon State, University of Illinois, and San Francisco State.

 __GitHub Pull Requests__

In the area of greater collaborative development, since the migration of the IDV project to github, we have closed a total of 125 “pull requests” or code contributions from internal and external collaborators.

 __Youtube IDV Instructional Videos__

In the area of online IDV training, the Youtube IDV instructional videos have been viewed thousands of times.


Prepared September 2024