1 of 14

Heavy Artillery Damage Detection

FourthBrain MLOps Capstone

Sarah Majors

2 of 14

The Problems - background

Nearly 7 million people are displaced within Ukraine right now. Even more are refugees who have left the country

The current geopolitical status is vitriolic and it is hard, especially for those closest to it, to know the ‘truth’

3 of 14

The Problems

Displaced peoples need aid

Current solutions:

On the ground knowledge
Manual analyze satellite imagery for refugee camps - slow, waits for after

Misinformation - both intentional and unintentional

Current solutions:

On the ground knowledge
Some have satellite access - slow to analyze and harder than expected

There are two problems that I see technology helping with right now.

�The first is that so many people are in need of aid.

Currently organizations providing such aid rely on ‘on the ground knowledge’ which is limited.

Someone in one location can only know the devastation there, but not be able to compare the situation to other areas in need.

They also manually analyze satellite imagery for refugee camps, but that is slow in the sense that it waits for the displaced people to move and set up camp as well as slow in the sense that it take copious man hours to analyze.

Faster access to aid would help people in more urgent need as well as helping more vulnerable populations such as unaccompanied children and the disabled.

Unfortunately, atrocities compound and getting to minors before they are trafficked is a concern.

�The second issue is misinformation - both intentional and unintentional.

Small journalists, not associated with major media are trying very hard to gather information and inform people of the truth.

They rely on much the same information as the aid organizations, but it is expensive and slow.

Also, hard evidence can be very influential to sway people to see past propaganda.

4 of 14

The Solution

Utilize automated assessments of satellite imagery to detect building damage
Fast, efficient, accurate, actionable information

~ 2 sq km

5 of 14

The Users

Aid groups such as the United Nations

More efficiently get aid to people in need

Small journalists

Use facts to overcome the ‘fog of war’
Take one task off their to do list

6 of 14

The Data

xBD - General Building Damage from natural disasters

Labelled imagery: localization and classification, used for training
850,000 building polygons, 6 disaster types, 15 countries, 45,000 sq km
30 cm precision

UNOSAT damage datapoints

Excellent points, no imagery

PlanetLabs

5000 sq km / month
3 meter precision

Because I am using transfer learning, I needed a large body of data to train on so I used the

xBD dataset from the xView2 challenge - 850,000 building polygons, 6 types of natural disaster, 15 countries, 45,000 sq km

It has 30 centimeter precision and allows training for building identification and degree of damage to the building

�I have Unosat datapoints which have not yet been integrated. They do not have associated imagery and it is really hard to get access to satellite imagery of a war torn country during the conflict.

Data can become accessible to bad actors, and it could potentially put a vulnerable population at higher risk.

The imagery data available through Landsat is a problem due to an issue with the satellite itself and has compromised the clarity and usability of the images,

Copernicus is 10 meter precision and not super useful (resolution too low), some houses may only be represented by a single pixel, which doesn’t give enough information to the model at this time

PlanetLabs came through at the end, but too late for training, also, only 5,000 km per month so will slowly add to the dataset

7 of 14

The MVP - the model

8 of 14

The MVP - the deployment

9 of 14

Post-disaster png

Overlay png

10 of 14

The MVP - KPIs

Dev

True / False positives

Prod

Inference times

The KPIs I set out to hit were standards for true and false positives in dev due to the heavily imbalance nature of the data.

Then I wanted to watch inference times for now as I knew that in the future I would like to be able to process geotifs instead of the png that it takes now.

PNGs are only 3 bands - rgb, and have limited size and metadata.

Geotifs are a format that can include many more bands such as infrared or thermal data, contain metadata such as gis and crs information, so better for mapping, and are for much larger areas.

While in theory I could scale infinitely, I don’t have that kind of money, and I haven’t worked anywhere that allows that sort of leeway either.�

Did I hit those KPIs? No, I wish I had awesome graphs here to show you how awesome everything is doing.

The goals were ambitious.

But that does not mean that the project is not a success.

I have infrastructure in place to be able to watch the metrics and have hard numbers to support my improvements.

The infrastructure I have now will allow my iteration time to be much faster and the experiment tracking I have in place will allow me to collaborate with others much more easily. So this is still an exciting outcome for the capstone. That said, there are a lot of next steps

11 of 14

Project Next Steps

Dig into the data!!!
Fine tune both the localization and classification model to handle the lower resolution imagery with war damage
Use PySyft from OpenMined to demonstrate federated learning on satellite imagery - (keep an eye on my LinkedIn for updates)
Adjust the API to handle entire geotif instead of png - look at trends more than a specific building

12 of 14

Ops Next Steps

Automatic testing and linting on commit - avoid regression
Automatic restart of failed containers
Kubeflow
Staging and Production Environments
More monitoring - model drift and such in production
CI/CD
Kubernetes

13 of 14

Thank you all!!!

Ali Kadhim

Camilo Santa

Greg Land

Dr. Gary Zoppetti, Millersville University

14 of 14

Questions?

Contact me!

sfmajors373@gmail.com

https://www.linkedin.com/in/sarah-f-majors/

Source Links

https://github.com/sfmajors373/xView2_baseline

https://dagshub.com/sfmajors373/xView2_baseline