Status Report: LDM

April - September 2015

Steve Emmerson, Tom Yoksas, Mike Schmidt

Strategic Focus Areas

We support the following goals described in Unidata Strategic Plan:

1. Enable widespread, efficient access to geoscience data
   By enabling researchers, teachers, and students to obtain a wide variety of meteorological and related data in near real time and at no cost via the Internet.
   
2. Provide cyberinfrastructure leadership in data discovery, access, and use
   By using the LDM to move data into the cloud.
   

Activities Since the Last Status Report

LDM

The LDM is the primary software package by which research and education institutions obtain near real time meteorological and related data.

Progress has been made on the following:

• Bugs dealing with the last successfully-received data-product and the last successfully-processed data-product were found and squashed.
• Static code analysis by Coverity Scan was added to the LDM development pipeline.
• The product-queue was hardened against simultaneous access by multiple threads.
• The build procedure was improved by making it more general and robust.
• Versions 6.12.8 through 6.12.14 were released.

Dependencies, challenges, problems, and risks include:

• Dealing with the ever-increasing volume of data.
• Very low bus factor (if Emmerson gets hit by a bus, the LDM’s in trouble).

Multicast LDM (alias LDM-7)

The multicast LDM project is separately funded by CISE in NSF. The goal is to reduce the outgoing bandwidth requirement of the LDM -- yet retain the current level of reliability -- by converting it into a hybrid system that combines use of the new, semi-reliable multicast protocol developed at the University of Virginia with the time-tested unicast capability of the current LDM.

Progress has been made on the following:

• Baseline statistics of LDM-6 behavior have been gathered.
• Testing of LDM-7 in a wide-area network has begun.

Dependencies, challenges, problems, and risks include:

• Time.
• Very low bus factor.

Ongoing Activities

We plan to continue the following activities:

• Support and maintenance of the LDM
• Deployment and testing of LDM-7

New Activities

Over the next three months, we plan to organize or take part in the following:

• Use of the LDM to bring data into the Cloud as part of the NOAA Big Data Project.

Over the next twelve months, we plan to organize or take part in the following:

• Use of the LDM to bring data into the Cloud as part of the NOAA Big data Project.

Relevant Metrics

The LDM powers the IDD, whose statistics can be found here.

Prepared  September 2015