DETECTING AND TRACKING HARD-TO-DETECT BACTERIA IN DENSE POROUS BACKGROUNDS

MEDHA SAWHNEY¹, BHAS KARMARKAR², ARKA DAW¹, ERIC LEAMAN² KIRAN MARQUES², BAHAREH BEHKAM² AND ANUJ KARPATNE¹

¹DEPARTMENT OF COMPUTER SCIENCE, VIRGINIA POLYTECHNIC AND STATE UNIVERSITY, BLACKSBURG, VA

²DEPARTMENT OF MECHANICAL ENGINEERING, VIRGINIA POLYTECHNIC AND STATE UNIVERSITY, BLACKSBURG, VA

BACTERIA TRACKING

Goal: Track bacteria in complex environments such Agar or Collagen hydrogels.

​

Motivation: Understanding bacteria behavior. Impacts biomedical phenomenon.

Bacteria motility information is crucial to control biomedical phenomenon such as bacteria-based drug delivery methods targeting tumor environments.

BACTERIA TRACKING

Goal: Track bacteria in complex environments such Agar or Collagen hydrogels.

​

Motivation: Understanding bacteria behavior. Impacts biomedical phenomenon.

Previous Work:

1. Fluorescent images: High Contrast between bacteria and background

​

• Liquid Medium: Backgrounds are smoother with less textures

Presence of visually differentiable features between the background and bacteria make it easy to track bacteria using traditional Computer Vision methods.

Liquid Medium - Fluorescent

Liquid Medium – Bright Field

BACTERIA TRACKING

Goal: Track bacteria in complex environments such Agar or Collagen hydrogels.

​

Motivation: Understanding bacteria behavior. Impacts biomedical phenomenon.

Previous Work:

1. Fluorescent images: High Contrast between bacteria and background

​

• Liquid Medium: Backgrounds are smoother with less textures

Presence of visually differentiable features between the background and bacteria make it easy to track bacteria using traditional Computer Vision methods.

Liquid Medium - Fluorescent

Liquid Medium – Bright Field

WHY AGAR AND COLLAGEN?

• Represent more realistic habitats for bacteria

Agar is commonly used to culture bacteria.

Collagen is the most abundant extracellular matrix-protein in the body

• Fluorescent imaging is restrictive:
• Imaging at higher frame rates is not possible
• Fluorescent tagging impacts bacteria behavior

• Significantly unexplored

CHALLENGES

• Lack of visually distinguishable features between the background and bacteria

Agar

Collagen

Background

Background

Bacteria

Bacteria

People

Cars

CHALLENGES

• Lack of visually distinguishable features between the background and bacteria

• 2D imaging misses out-of-plane motion of bacteria, resulting in out of focus bacteria

Frame 18 (Easy)

Frame 22 (Hard)

Frame 28 (Very Hard)

• Easy: detected from a single frame
• Hard: need inspection of 2-3 consecutive frames
• Very Hard: need to go over multiple frames, back and forth

CHALLENGES

• Lack of visually distinguishable features between the background and bacteria

• 2D imaging misses out-of-plane motion of bacteria, resulting in out of focus bacteria

• Difficult to extract predictive motion features as bacteria exhibit “random walk”

PROPOSED APPROACH: MULTI-LEVEL MOTION ENHANCED TRACKER

RESULTS

• Precision significantly increases after the FPP module with a marginal drop in recall
• The Interpolated SORT module further boosts the recall with a tradeoff in Precision

• We compare our approach against two single model object detectors:
• No Motion Monolithic Tracker (NM-MT) - trained only on images
• Motion-based Monolithic Tracker (MMT) - trained on our motion enhanced features.

RESULTS

Agar

Collagen

CONCLUSION & FUTURE WORK

• Pipeline for detecting and tracking bacteria in complex environments: agar and collagen hydrogels
• Multi-level Motion Enhanced Tracker can predict even the hard-to-detect bacteria. Specifically, we achieve a precision of 0.82 and recall of 0.94 on Agar media

​

• Tracking in crowded scenarios. Bacteria moving close by are restricted by NMS to be predicted correctly
• Deep learning-based tracking methods for improved tracking
• Learning fine-grained labels of bacteria (easy/medium/hard) without additional supervision
• Trajectory forecasting algorithms that predict bacteria movement based on bacteria characteristics, environment textures, and time

​