Status Report: GOES-R/S
October 2017 - March 2018
Mike Schmidt, Tom Yoksas
Questions for Committee Members
- What new coverages and increased spatial and temporal resolutions should be considered for UNIWISC IDD feed imagery?
- Which GOES-16 bands (wavelength channels) and coverages should be converted from netCDF-4 to McIDAS AREA format so that GEMPAK users will continue to be able to use currently satellite imagery?
- What kind(s) of data access methods are most desired/usable for the community?
- Other questions?
Activities Since the Last Status Report
- Implemented IDD distribution of GOES-16 ABI imagery, Space Weather and Geostationary Lightning Mapper (GLM) products
NCAR/RAL was the first site to connect to this service to provide GOES-16 data for their science objectives. Other sites including two international sites are now receiving feeds of mostly ABI full disk imagery and GLM lightning data.
- Install robust data serving platforms to be used to provide programmatic access to GOES-16 ABI imagery, space weather products and the GLM
- atm.ucar.edu was the first machine to be replaced by the new equipment.
- Motherlode.unidata.ucar.edu will be the next machine to be upgraded.
- Implemented UW/SSEC’s “fanout server” (redistribution of the GRB-200 UDP unicast stream over TCP)
We are feeding from one of the SSEC fanout servers, and they are feeding from two that we are running. SSEC is currently experimenting with the creation of a composite GOES-16 datastream that will be composed of the best packets from all GOES-16 ingest systems that are participating in the experiment.
- Secured funding to install GOES-S ingest and data serving capabilities in UCAR
- Dealing with Terrestrial Interference (TI) on the GOES-16 satellite dish we installed at the NCAR Mesa Lab
In the fall of 2017 we began seeing TI in the signal being received by the GOES-16 dish at the NCAR Mesa Lab. We contacted Quorum Communications to get advice on how to identify the source of the TI and what, if anything, can be done to mitigate it.
Quorum sent an engineer to Boulder (no cost to us) to investigate the cause of the TI, and his findings were that the noise was in-band, so it can not be filtered out. His educated guess about the cause of the TI was that the source of the noise was close to the dish, so it may be from power lines down the hill south of the Mesa Lab or from somewhere in the Mesa Lab itself.
As part of the discussion we had, we described the GVAR GOES and NOAAPort ingest capabilities at the UCAR Foothills Lab 2 (FL-2) location. The engineer’s assessment was that we could probably ingest GOES-16 data on the 3.8 m solid dish located outside of the cafeteria at FL-2. He then offered to loan us one of their new GRB LNBs and GRB 200 receivers to use to test this hypothesis.
We installed the loaner LNB on the FL-2 cafeteria dish and installed the GRB 200 in the 2nd floor RAL computer room, and then realigned the dish to point at GOES-16. Ingest results (so far) have been very promising - we are receiving all products in the Left Hand Circularly Polarized (LHCP) channel of the GRB with very few errors. We are unable to ingest the Right Hand Circularly Polarized (RHCP) channel on the cafeteria dish since there is only one signal cable from it to the RAL computer room. We are now talking with UCAR facilities personnel about pulling an additional signal coax (LMR-400) from the cafeteria dish to the RAL computer so that we can also ingest the RHCP channel. In the interim, we have been feeding the RHCP channel from our Mesa Lab GOES-16 satellite dish (the RHCP channel has much fewer errors than the LHCP on the Mesa Lab dish) to the ingest machine for the FL-2 cafeteria dish. Combining the two GRB channels from two dishes has resulted in a very good GOES-16 ingest capability.
Ongoing Activities
We plan to continue the following activities:
- Ingest the GOES ReBroadcast (GRB) from GOES-16 in real-time using the 4.5 m satellite dish that we installed on the eastern pad at the NCAR Mesa Lab using the NOAA funded UW/SSEC/CIMSS Community Satellite Processing Package (CSPP) for Geostationary Data (GEO) package.
- If we can establish error free ingest for both GOES-16 channels on the FL-2 cafeteria dish, we will repoint the Mesa Lab dish to GVAR GOES-West to see if the TI problem disappears.
- Continue uploading all ABI imagery, Space Weather products and Geostationary Lightning Mapper (GLM) to an S3 bucket in Amazon Web Services (AWS)
- Transfer of ABI imagery and space weather products has been active since the first day of GRB broadcast using storage resources provided free-of-charge by Amazon, but there is a multi-month gap in the data record in the S3 bucket that was caused by a configuration error when we switched from ‘rsync’ing the data to moving products via an LDM pattern-acton file action.
- Continue working with NCAR/EOL to support their CSPP GEO installation at the NCAR Mesa Lab
- This installation will provide an in-house redundancy for GOES-16 ingest
- Canvas the community to learn more about their GOES-16 data needs
- Investigate additions to the IDD CONDUIT data stream that would be useful for creation of new GOES-16 based Level 2 products
Future Activities
- Serve GOES-16 Advanced Baseline Imager (ABI) imagery via McIDAS ADDE and the TDS
- Serving is already in progress
- We previously commented that the expected data volume of GOES-16 data would be too large to relay in the IDD. Our opinion was changed when the data volume was drastically reduced by the turning on of netCDF4 compression of the data being received in the GRB
- Investigate approaches that would insulate GRB ingestion from long standing twice per year power downs in the NCAR Mesa Lab facility
- Unidata-Wisconsin (UNIWISC) IDD imagery has been updated to incorporate GOES-16 imagery, but we envision that users would like to see more coverages added
Relevant Metrics
Strategic Focus Areas
We support the following goals described in Unidata Strategic Plan:
- Enable widespread, efficient access to geoscience data
Standing up ADDE and TDS data services for real-time GOES-R/S data will benefit the greater Unidata community.
- Develop and provide open-source tools for effective use of geoscience data
The IDD is powered by the Unidata LDM-6 which is made freely available to all. The Unidata NOAAPort ingest package is being used by a variety of university and non-university community members. Both the LDM and NOAAPort ingest packages are bundled in AWIPS.
- Provide cyberinfrastructure leadership in data discovery, access, and use
The community-driven IDDs provide push data services to users an ever increasing community of global educators and researchers.
- Build, support, and advocate for the diverse geoscience community
Providing access to data in real-time is a fundamental Unidata activity.
Prepared March 2018