open source software for catalyzing conservation of �Southern Resident killer whales (in real-time)

Scott Veirs & Val Veirs on behalf of the Orcasound community

sveirs@ gmail.com | Orcasound Slack

orcasound.net/talks

​

Our orcas: the southern salmon seekers

Southern Resident Killer Whales (SRKWs) are:

• Southern = ranging from northern California to SE Alaska
• Resident = historically re-occurring within the Salish Sea (inland waters of WA and BC)
• Killer = apex predators, salmon specialists
• Whales = cultural icons, both historically & as modern “charismatic megafauna”

​

How can we collaborate across State and International boundaries to conserve these orcas?

Our orcas: recovery by a 1000 actions

Cumulative human impacts�NOAA recovery plan

• 3 key risks:
• Scarce salmon
• Persistent pollutants in blubber
• Vessel effects
• Vessel noise impacts:
• Ship noise masks communication & echolocation
• Boat interference lowers foraging efficiency

How can AI & human listeners work synergistically to conserve SRKWs (in real-time)?

How? cooperate as a hydrophone network

2002: Orcasound Lab (Val’s “back yard”)�2008-12 NOAA funding (expansion to 5 nodes)�2013-15 Philanthropy only (decline to 2 nodes)�2016-now Cooperative network (17 NGOs in 2021)

2017 crowd-funding ($20k Kickstarter)�2018 open-sourcing, crowd-sourcing, open data...�2020+ hackathons, philanthropy, Amazon+Microsoft cloud credits

$$

How? collaborate openly in soft/hard-ware

<2002-17 �-- humans listening via Shoutcast mp3 streams�-- Val building custom software, alone�-- Scott building web static web sites, Google sheets, manual Twitter/email notifications...

2017-18: software + hardware (Kickstarter for v1 web app), live.orcasound.net launched Nov ‘18

2019-20: v2 UI beta-tested in Nov ‘19, launched in May, 2020

2021: v3 UI + a proliferation of related projects (24 public Orcasound Github repositories)

• 3 cabled nearshore sites streaming 24/7 (2018-21)
• Streaming hardware cost: ~U$300 per node
• Hydrophone $300-1500
• PiSound HAT = 2 channel, �24 bit, 48/96/192 kHz
• Free open source code!

Community scientists detect orca & novel sounds in real-time via a web app -- live.orcasound.net

Easy to listen live. Scales inexpensively. Lets user tag data.

Orca sounds

orcanode code

orcasite code

How? open access raw data

aws --no-sign-request s3 sync streaming-orcasound-net

Data volume

• 3.2 TB (HLS audio only)
• 0.4 TB/yr/node
• Increasing soon...�...FLAC, 48/96 kHz

Data costs

• <$100/mo w/3 nodes
• ~30$/mo/node, but scalable
• Free thus far with cloud credits!
• AWS open data registry + Quilt

How? open labeled data & labeling tools

Labeled data (orcadata wiki)

• SRKW calls
• 2022: signals of other SRKW sounds, Bigg’s KW, and humpback

Labeling tools

• 2000 Orcasound human labels (~100/month, free text)
• Pod.Cast (2019) 10 “rounds”
• OrcaHello tags (50 hrs, 2020+) w/ D. Bain
• Orca Active Learning (2020+)
• Audacity (expert labeled test sets)
• 2022: HALLO annotation tool

Artificial Intelligence for orcas

AI for orcas (#ai4orcas) -- ai4orcas.net -- OrcaHello | Pod.Cast | OrcaAL

towards (more) open (marine) bioacoustic data science...

AI+human detection is optimal

For many end-users, expert validation of acoustic &/or visual detections is still important.

2021 results:

• 71 hr SRKW bouts
• 30 hr Bigg’s bouts
• 18 hr humpback bouts
• Of 41 known SRKW transits: AI detected 54%; humans 81%; and 97% combined
• AI outage in Oct missed 12 (31%)

​

Technical and scientific challenges

Where are they now?�Applied conservation problems��Can we automate reliable real-time notification of SRKW presence, or will humans remain in the loop?

1. Notify as soon as possible, or only when “sure” of SRKW?
2. What is end-user’s false positive tolerance?
3. The conservation challenge: how to inspire users to act for SRKWs when empathy is maximized during a live acoustic event?

What are they saying?�Basic biology problems��Bioacoustics topics our data can inform:

• marine mammal communication systems
• biosonar
• marine acoustic ecology
• soundscape analysis

Lots of visualization opportunities!

Sighting & listening together

Great synergies lie in integrating acoustic and visual real-time observations!

2022 goals:

• Data cooperative for sharing:
• vetted sightings
• verified human detections
• moderated OrcaHello detections
• Public API & Creative Commons license
• Notifications API
• Orca Network Facebook group

​

2022 challenge: Saving SRKWs & salmon

Will users label, heed a Conservation Call To Action, or both?

• 1800 subscribers
• Baseline survey via subscription form
• Conservation effect survey

Guiding principles:

• User-centered design
• Open roadmap (Trello)

Grand challenge: sharing solutions globally

There are lots of soniferous species that need a voice!

Expansion possibilities for our open source solutions:

• More SRKW nodes (U.S.)
• SRKW nodes in Canada
• Separate networks in other regions of the world (map) for other species? (belugas?!)

Explore our Google map of

NE Pacific hydrophones

Acknowledgements & links

Thanks to all our collaborators!

• The Orcasound open source community’s volunteer hackers
• The many NGOs & volunteers who maintain the hydrophones at each node
• Google Summer of Code
• Microsoft AI for Earth

More info:

• orcasound.net
• ai4orcas.net
• github.com/orcasound

Give orcas a voice!�live.orcasound.net

​

Extra slides...

From other Orcasound talks...

Extra discussion topics:�

Practical detection vs miss rates

From candidate table

False positive rate (goal: ~10 per day max)

• About right winter 2021
• Uncomfortable in spring (pigeon guillemots!)
• Too low in summer (ACI bugs, downtime)
• Still too low/slow in fall?
• Just right this winter?

​

User info

Adrian plots (e.g. subscriber growth?)

Geographic partitioning of live-listeners?

​