Dr. Sreeja Nag

Senior Research Scientist,

NASA Ames Research Center/BAER Institute

​

Team Members: Vinay Ravindra (NASA ARC/BAER), Richard Levinson (NASA ARC/KBR), Mahta Moghaddam (USC), Daniel Selva (TAMU), and students (USC, TAMU)

June 2022

D-SHIELD: Distributed Spacecraft with Heuristic Intelligence �to Enable Logistical Decisions

1

Background: Motivation

• Multi-payload, multi-spacecraft constellation scheduling for spatio-temporally varying science observations
• Small Sat constellation + Full-body reorientation agility + scheduling autonomy �= More Coverage, for any given number of satellites in any given orbits
• Ground scheduling algorithm allows 2-sat, 1-imager constellation over 12 hours to observe 2.5x compared to the fixed pointing approach
• Onboard scheduling algorithm allows 24-sat, 1-rainradar constellation to observe ~7% more flood magnitude than ground scheduling

​

​

​

​

Published Use Cases:

1. Land coverage and coral tracking (ASR, 2018)
2. Cyclone tracking (IEEE TGRS 2020)
3. Urban Floods (J.Hydrology 2021)

2

Background and Architecture

3

Summary of Concept of Operations

Planning Ops using dynamic knowledge and forecast of soil moisture and precipitation,�that is sensitive to Season, vegetation, soil types are static, known constants

4

Summary of Science Advancements

For D-SHIELD: The scientific value of making a measurement of a ground point (GP) at a given time point (TP) as a function of the biased standard deviation (SD) of the soil-moisture prediction at that GP at TP.

We hypothesize that targeted, good quality observations of locations and times of higher uncertainty will serve as better inputs to prediction models and improve predictive output of augmented products.

We test this hypothesis using an example custom constellation that maximizes observation opportunity while minimizing cost, customized soil moisture specific instruments, an intelligent planner, and a science simulator in the loop to inform the planner.

5

Selected Constellation Architecture and Cost

The VASSAR (Value assessment of system architectures using rules) software suite was used to identify optimum heterogeneous constellations carrying the custom instrument suite.

​

The baseline radar constellation was selected from the Pareto front between maximum revisit time and percentage coverage of areas of interest for soil moisture.

*temporal gap between a satellite field of regard accessing any GP of interest

SELECTED: altitude = 502.5 km, Inclination 89 deg, with a 7 day repeat cycle, mass per satellite is 790 kg, 2.71-hour average revisit for any GP and 10.8-hour max revisit

6

Selected Constellation Architecture and Cost

Baseline 3-sat L+P band radar, radiometer, reflectometer constellation added with 16 CubeSats carrying radiometers and reflectometers

CubeSats with a non-radar microwave instruments were added to the constellation in simulation to assess impact of coverage for fractional increase in cost. Hundreds of heterogeneous CubeSat constellation options were considered to reduce maximum revisit time. Adding a constellation of 12 CubeSats (3 planes of 4 satellites each in a Walker delta constellation at Altitude: 504.19 km, inclination: 90 deg) containing radiometer and reflectometers reduces revisit time from ~7 hours to ~4 hours.

Gorr et al, IGARSS 2021

7

Selected Constellation Architecture and Fusion Capability

Thus, even a fully loaded constellation of small (16) and medium (3) sized satellites with 5 instrument types was found to be cheaper than SMAP.

​

To combine radar and radiometer measurements, we would like observations to be temporally close because the longer the delay between the two collections, the less physically correlated the observations are.

​

Therefore, Depending on the science assimilation constraints, a heterogeneous constellation provides the flexibility of joint inversion of data from various instruments in space (and even ground).

8

Science Simulator to inform the Planner Objective

Benefit of near real time retrieval, data assimilation, and prediction to inform the planner of the value of the next measurements.

​

When soil moisture changes rapidly (e.g. precipitation), higher frequency measurements are needed to keep the model error low, compared to times of slow changes.

Model error is the bias-corrected difference between soil moisture prediction standard deviation (SD) and retrieval measurement root mean square error (RMSE) => Planner Objective

9

End to End Simulation Experiment

• Experiment start date: 4 Jan 2020 1:30:00 UTC.
• 1 cycle = 24 hrs
• Target (Science) Value model produced prediction uncertainty for 24 hours period. Access data is also produced for the 24 hour period. Based on these two inputs, the planner produced plan targeting high value targets & high quality over the next 24 hours.
• Simulated soil-moisture observations (from the plan) were fed to the next run (Cycle 2) of the Science Value Model.
• The experiment ended with the production of the next cycle of outputs (5 Jan 1:30:00 UTC -> 24hrs) from the science value model. It was seen that this output issued prediction with lesser uncertainty as compared to the a separate parallel run of the science model without any assimilated observations.

*executed plan = plan in the experiment

10

Science Simulator: Soil Moisture Prediction Recap

Prediction SD per GP and time step is obtained using a convLSTM model that is a function of soil type, vegetation, season, solar conditions, precipitation, and soil saturation. For the 3-sat baseline SAR constellation, the planner selected GPs with an average SD that is ~2x the average global SD, i.e., it targeted regions with most uncertainty.

Output = SM and Variance as a function of space and future time

Kannan et al, IGARSS 2022

11

End to End Simulation Experiment

• Experiment start date: 4 Jan 2020 1:30:00 UTC.
• 1 cycle = 24 hrs
• Target (Science) Value model produced prediction uncertainty for 24 hours period. Access data is also produced for the 24 hour period. Based on these two inputs, the planner produced plan targeting high value targets & high quality over the next 24 hours.
• Simulated soil-moisture observations (from the plan) were fed to the next run (Cycle 2) of the Science Value Model.
• The experiment ended with the production of the next cycle of outputs (5 Jan 1:30:00 UTC -> 24hrs) from the science value model. It was seen that this output issued prediction with lesser uncertainty as compared to the a separate parallel run of the science model without any assimilated observations.

*executed plan = plan in the experiment

12

Science Simulator: Soil Moisture Retrieval Recap

Measurement error is a function of the instruments and their parameters used to make a/multiple measurements, and biome type expected (16 IGBP types into 5 major groups)

Land cover types

Instruments operation modes

Malebari et al, IGARSS 2022

13

a

Science Simulator: Soil Moisture Retrieval Updates

14

a

Science Simulator: Soil Moisture Retrieval Updates

15

a

• The multiple observations can be used to retrieve other geophysical parameters such as surface roughness and vegetation water content
• L-band radar expected to perform better than P-band radar in vegetation parameters retrievals

There is no significant performance improvement for using more than 2 observations in the retrieval

Retrievals using P-band radar have better or similar performance to retrievals using L-band radar

Science Simulator: Soil Moisture Retrieval Updates

Malebari et al, IGARSS 2022

16

End to End Simulation Experiment

• Experiment start date: 4 Jan 2020 1:30:00 UTC.
• 1 cycle = 24 hrs
• Target (Science) Value model produced prediction uncertainty for 24 hours period. Access data is also produced for the 24 hour period. Based on these two inputs, the planner produced plan targeting high value targets & high quality over the next 24 hours.
• Simulated soil-moisture observations (from the plan) were fed to the next run (Cycle 2) of the Science Value Model.
• The experiment ended with the production of the next cycle of outputs (5 Jan 1:30:00 UTC -> 24hrs) from the science value model. It was seen that this output issued prediction with lesser uncertainty as compared to the a separate parallel run of the science model without any assimilated observations.

*executed plan = plan in the experiment

17

Earth Observation Simulator

• Suite of Python packages: EOSim, OrbitPy, InstruPy, adcPy
• Produces mission data, with facility to accommodate heterogeneous, agile constellations (i.e. custom orbits, multiple & heterogenous set of instruments per instrument, maneuverability)
• Beta version avail in a public Github repo under a permissive open-source license (Apache 2.0) (from Jan 2021)
• Interface can be via the GUI or through Python API classes/function.

Ravindra and Nag, JSTARS 2019; Ravindra et al, IGARSS 2021, Sin et al, AIAA SciTech 2021

18

End to End Simulation Experiment

• Experiment start date: 4 Jan 2020 1:30:00 UTC.
• 1 cycle = 24 hrs
• Target (Science) Value model produced prediction uncertainty for 24 hours period. Access data is also produced for the 24 hour period. Based on these two inputs, the planner produced plan targeting high value targets & high quality over the next 24 hours.
• Simulated soil-moisture observations (from the plan) were fed to the next run (Cycle 2) of the Science Value Model.
• The experiment ended with the production of the next cycle of outputs (5 Jan 1:30:00 UTC -> 24hrs) from the science value model. It was seen that this output issued prediction with lesser uncertainty as compared to the a separate parallel run of the science model without any assimilated observations.

*executed plan = plan in the experiment

19

Planning and Scheduling

Dynamic Constraint Programming (DCP)

• Suboptimal but Fast
• Constraints enforced "on-demand"
• Variables dynamically eliminated �by constraint handlers

Mixed Integer Linear Programming (MILP)

• Optimal but Slow
• Less flexible constraint modeling
• Quantitative declarative constraints
• Scalability challenges

20

Planning and Scheduling

Algorithm: Constraint Satisfaction Programming (CP)

​

Objective: Reduction in soil moisture prediction uncertainty of GPs due to making measurements of those GPs at optimal times and with an optimized set of instrument parameters.

Results: Max GPs (30.9k) are observed with gpCount while the errReduction picks the most uncertain GPs and reduces total error the most. All planner autonomy strategies show superior science return compared to fixedPointing for minimum increase in cost. They also observe up to 3x more GPs compared to non-agile constellations.

Levinson et al, IWPSS 2021

21

Planning and Scheduling

Increasing constellation satellites or horizons planned: GP count as well as total error reduced (across all observed GPs) shows monotonic increase:

​

*Initial error = soil moisture prediction SD, final error = measurement RMSE of the observation, both shown per GP and both computed as unbiased parameters in m³/m³ volumetric soil moisture units

Nearly 87k GPs were observed by 3 satellites over a 24h period. D-SHIELD can smartly maneuver to maximize unique or fast-changing GPs and continue to increase coverage to ~175k GPs in 2 days and ~260k GPs in 3 days.

​

As battery+power duty cycle constraints tighten from {484Wh,266W} to {350Wh,260W} to {250Wh,266W} to {160Wh,253W}, the lowest depth of discharge the battery reaches drops from 77% to 71% to 63% to 47%.

Levinson et al, IWPSS 2021

22

Planner Algorithm Experiment

MILP is more Optimal and more Efficient…

6 test cases with increasing plan horizons and complexity: Cases 1 – 5: DCP achieves ~ 60 % optimal. Case 6: Unsolvable by MILP in 50 hour time limit, but DCP solves in 28 mins. MILP plans always has fewer commands (makespan): Less slewing & energy cost

​

23

Planner Algorithm Experiment

… but also is VERY computationally expensive

24

Planner Updates in Experiment

Total # observed GP in 4 horizons = 297,800 (~50% of SMAP)

25

9.6 % of initial horizon error is removed in Horizon 4

Planner Updates in Experiment

…Because of which nearly 20% improvement in prediction accuracy was observed!

26

End to End Simulation Experiment

• Experiment start date: 4 Jan 2020 1:30:00 UTC.
• 1 cycle = 24 hrs
• Target (Science) Value model produced prediction uncertainty for 24 hours period. Access data is also produced for the 24 hour period. Based on these two inputs, the planner produced plan targeting high value targets & high quality over the next 24 hours.
• Simulated soil-moisture observations (from the plan) were fed to the next run (Cycle 2) of the Science Value Model.
• The experiment ended with the production of the next cycle of outputs (5 Jan 1:30:00 UTC -> 24hrs) from the science value model. It was seen that this output issued prediction with lesser uncertainty as compared to the a separate parallel run of the science model without any assimilated observations.

*executed plan = plan in the experiment

27

Demo on D-SHIELD’s Visualization Tool

The screenshot below shows 18h coverage (on 2D Earth) of the smart 3-sat constellation. Observation value has been normalized globally, therefore biomes corresponding to high retrieval errors are penalized in red.

28

End to End Simulation Experiment

• Experiment start date: 4 Jan 2020 1:30:00 UTC.
• 1 cycle = 24 hrs
• Target (Science) Value model produced prediction uncertainty for 24 hours period. Access data is also produced for the 24 hour period. Based on these two inputs, the planner produced plan targeting high value targets & high quality over the next 24 hours.
• Simulated soil-moisture observations (from the plan) were fed to the next run (Cycle 2) of the Science Value Model.
• The experiment ended with the production of the next cycle of outputs (5 Jan 1:30:00 UTC -> 24hrs) from the science value model. It was seen that this output issued prediction with lesser uncertainty as compared to the a separate parallel run of the science model without any assimilated observations.

*executed plan = plan in the experiment

29

Science Simulator: Soil Moisture Prediction

Prediction SD per GP and time step is obtained using a convLSTM model that is a function of soil type, vegetation, season, solar conditions, precipitation, and soil saturation. For the 3-sat baseline SAR constellation, the planner selected GPs with an average SD that is ~2x the average global SD, i.e., it targeted regions with most uncertainty.

Predictions Run001

Planner

Assimilated images

ConvLSTM model

Predictions Run002

(Jan 4^th, 2020)

(Jan 5^th, 2020)

Output = SM and Variance as a function of space and future time

30

Prediction uncertainty Improvement due to Planner

Model run 001

• This model run is driven entirely based on SMAP L4 data.
• No observations are used during this run.

Model run 002

• This model run is based on predictions which are triggered by SMAP L4, the intermediate inputs are assimilated with satellite observed values.

31

Planner-Aware Constellation Architecture Optimization

The VASSAR (Value assessment of system architectures using rules) software suite was used to identify optimum heterogeneous constellations carrying the custom instrument suite. Previously shown baseline radar constellation was selected from the Pareto front between maximum revisit time and percentage coverage of areas of interest for soil moisture. Updates show joint optimization of satellite and instrument for an intelligent planner mission.

Gorr et al, 2022, in prep

32

Joint Constellation-Instrument Optimization Results

Access Area Coverage, Cost … and unique to D-SHIELD, instrument performance (SNEZ)

33

Joint Constellation-Instrument Optimization Results

… and unique to D-SHIELD, remaining soil moisture error, which is the total across all ground points of prediction uncertainty of unobserved points and measurement error of observed points by a historical planner. Used as a proxy in the absence of joint planner and constellation optimization

34

Open Source Released!

https://github.com/dshield-proj/

35

Thank you!��Questions?�Sreeja.Nag@nasa.gov

36

Publications

JOURNALS

1. R. Lammers, A.S. Li, V. Ravindra, S. Nag, "Prediction Models for Urban Flood Evolution for Satellite Remote Sensing", Journal of Hydrology,  603 (2021): 127175
2. V. Ravindra, S. Nag, A.S. Li, "Ensemble Guided Tropical Cyclone Track Forecasting for Optimal Satellite Remote Sensing", IEEE Transactions on Geoscience and Remote Sensing (TGRS),  59 (2020), no. 5, pp. 3607-3622
3. V. Ravindra, S. Nag, "Fast Methods of Coverage Evaluation for Tradespace Analysis of Constellations", IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 13 (2019), 89-101

�CONFERENCES (1/3)

1. R. Levinson, S. Niemoeller, S. Nag, V. Ravindra, "Planning Satellite Swarm Measurements for Earth Science Models: Comparing Constraint Processing and MILP methods", International Conference on Automated Planning and Scheduling, Applications Track, June 2022
2. A. Melebari, S. Nag, V. Ravindra, M. Moghaddam, "Soil Moisture Retrieval from Multi-Instrument and Multi-Frequency Simulated Measurements in Support of Future Earth Observing Systems", IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia, July 2022
3. A. Kannan, G. Tsagkatakis, R. Akbar, D. Selva, V. Ravindra, R. Levinson, S. Nag, M. Moghaddam, "Forecasting Global Soil Moisture using a Deep learning Model integrated with Passive Microwave Retrieval", IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia, July 2022

37

Publications

CONFERENCES (2/3)

4. A. Kannan et al, “Forecasting Global Geophysical States using a Deep Learning Model for Spacecraft Constellation Scheduling and Planning”, AGU Fall Meeting, December 2021
5. V Ravindra, R Ketzner, S Nag, “EO-Sim: An open-source library for design and evaluation of space observation systems; a discussion on the software design and development”, AGU Fall Meeting, December 2021
6. Levinson, S. Nag, V. Ravindra, "Agile Satellite Planning for Multi-Payload Observations to aid Earth Science", International Workshop on Planning and Scheduling for Space, Virtual Forum, July 2021
7. S. Nag, M. Moghaddam, D. Selva, J. Frank, V. Ravindra, R. Levinson, A. Azemati, B. Gorr, A. Li, R. Akbar, "Soil Moisture Monitoring using Autonomous and Distributed Spacecraft (D-SHIELD)", IEEE International Geoscience and Remote Sensing Symposium, Brussels Belgium, July 2021
8. V. Ravindra, R. Ketzner, S. Nag, "Earth Observation Simulator (EO-SIM): An Open-Source Software for Observation Systems Design", IEEE International Geoscience and Remote Sensing Symposium, Brussels Belgium, July 2021
9. B. Gorr, A. Aguilar, D. Selva, V. Ravindra, M. Moghaddam, S. Nag, "Heterogeneous Constellation Design for a Smart Soil Moisture Radar Mission", IEEE International Geoscience and Remote Sensing Symposium, Brussels Belgium, July 2021
10. E. Sin, M. Arcak, A. S. Li, V. Ravindra, S. Nag, "Autonomous Attitude Control for Responsive Remote Sensing by Satellite Constellations", AIAA Science and Technology Forum and Exposition (SciTech Forum), Nashville, January 2021 

38

Publications

​

​

CONFERENCES (2/3)

11. S. Nag, M. Moghaddam, D. Selva, J. Frank, V. Ravindra, R. Levinson, A. Azemati, A. Aguilar, A. Li, R. Akbar, "D-SHIELD: Distributed Spacecraft with Heuristic Intelligence to Enable Logistical Decisions", IEEE International Geoscience and Remote Sensing Symposium, Hawaii USA, July 2020
12. V. Ravindra, S. Nag, "Instrument Data Metrics Evaluator for Tradespace Analysis of Earth Observing Constellations", IEEE Aerospace Conference, Big Sky, Montana, March 202

​

DISSERTATION

1. E. Sin, “Trajectory Optimization and Control of Small Spacecraft Constellations”, PhD Thesis, University of California Berkeley, Spring 2021

​

KEYNOTE TALK

1. S. Nag, “Vehicular Robotics for Responsive Environmental Monitoring”, IEEE International Geoscience and Remote Sensing Symposium, Brussels Belgium, July 2021, Video URL: https://igarss2021.com/Keynotes.php (starting 53:40)

39