1 of 21

TrichTrack: Multi-Object Tracking of Small-Scale Trichogramma Wasps

Vishal Pani^1,2, Martin Bernet³, Vincent Calcagno^3,4, Louise van Oudenhove^3,4, and François Bremond^1,4

¹INRIA Sophia Antipolis - Méditerranée, France

²Indian Institute of Information Technology, Allahabad, India

³INRAE, CNRS, Institut Sophia Agrobiotech, France

⁴Université Côte d’Azur, France

2 of 21

Context: Trichogramma as Endoparasitoids

Corn Pest: European corn borer

(Ostrinia nubilalis)

Trichogramma Deployment:

≈ 3,60,000 individuals / hectares.

Trichogramma: Egg Laying Cycle

3 of 21

Objectives

Objective 1	Objective 2	Objective 3
Detection: Identification and localization of trichogramma individuals.	ReID: Identifying different Trichogramma wasps that are indistinguishable to human eyes via ReID methods.	Tracking: Tracking each individual throughout the entire video, and maintaining correct tracking on cases like jumps and collisions.

Maintain consistent long duration tracking of each trichogramma individual.

4 of 21

Dataset:

4.6 cm

8.5 cm

Data Collection Setup

2 species: Cacoeciae, Brassicae

Cacoeciae (Lighter) Substrains: (i) ACJY144, (ii) PMBio1, (iii) PJ.
Brassicae (Darker) strains: (i) ISA3080, (ii) PR002,, (iii) E1.3.

Density Videos:

Videos contain only single strain.
Density of insects maybe high (~100 indv.) or low (~15 indv)
Total 78 videos with low density and 77 videos with high density.
Each video is of 8 mins and contains about 12000 frames.

Ground Truth present for videos (were provided later for benchmarking):

ISA3080_Low_13 (Reported)
E1.3_Low_7
ACJY_Low_14
PMbio1_Low_11 (Reported)
PJ_Low_8
PR002_Low_12 (Reported)

5 of 21

Dataset: Sample Video Frame (Strain: E1.3)

6 of 21

Existing Insect Trackers

CTRAX:

Developed by Caltech in 2009.
Uses position and orientation of insects to track.
Limitations:

Slow
Unable to handle jumps
High amount of false negatives detections

TREX:

Developed by MPI, Germany in 2020.
Uses novel tree-based method for tracking along with Hungarian algo.
Can segment insects too.
Aim is to provide fast inference.
Advantages:

Few false negatives
Fast for low density videos

Limitations:

Handles jumps sporadically

1. K. Branson, A. A. Robie, J. Bender, P. Perona, and M. H.Dickinson. High-throughput ethomics in large groups of drosophila. Nature Methods, 6(6):451–457, Jun 2009.

2. T. Walter and I. D. Couzin. Trex, a fast multi-animal tracking system with markerless identification, and 2d estimation of posture and visual fields.eLife, 10:e64000, feb 2021.

1.

2.

7 of 21

Methodology: Pipeline

8 of 21

Methodology: 2 Staged Tracker

1^st Stage

2^nd Stage

Cost matrix is calculated based on appearance matrix (from ReID model) and distance matrix.
Hungarian matching based on cost matrix.

9 of 21

Methodology: Training the Detector

1^st Iteration Preprocessing

2^nd Iteration Preprocessing

Initially, the dataset contained only CTRAX annotationswith lots of false negatives. Providing these false negatives would have hampered the learning of the detector. So wemasked them using OTSU masking algorithm [17]. Thisproduces masked inputs which only contain true positivesthat help the detector to learn properly. Since CTRAX annotations do not contain sides of thebounding boxes, the detector trained on the1stiterationcould only produce loose bounding boxes. Loose boundingboxes are not desirable as they hamper the feature vectorsproduced by the ReID Network. To fix this, we trained thedetector for a second time using annotations from the ini-tially trained detector instead of CTRAX. We follow a datapreprocessing scheme where we find out the contours of theinsects and use them to filter only the center-most insectusing OTSU algorithm to generate tight bounding box an-notations. This helps the detector to produce tighter boxes.Refer Fig. 4 for results of the detector after2nditeration oftraining

10 of 21

Methodology: Comparison of 1^st & 2^ndIteration Detection

1^st Iteration

2^nd Iteration

11 of 21

Methodology: Training the ReID Network

Sampling^[1]

Triplet Loss

1. S. Karthik, A. Prabhu, and V. Gandhi. Simple unsupervised multi-object tracking, 2020.

12 of 21

6. Results & Analysis

13 of 21

Results & Analysis: Tracking Performance

14 of 21

Results & Analysis: Saliency Maps by ReID Network

To plot the saliency maps generated by the ReID Networks,we take the channel-wise average of the output featuremap of the sixth intermediate block of ResNet-18. We usePMbio1Low11 for this experiment.On referring to Fig. 6, we observe that the activationsof the saliency maps produced by our ReID Network are more pronounced around the antenna, eye, thorax, and ab-domen of the Trichogramma individual when compared tothe results by the ImageNet pre-trained network. Thereby,verifying that our model attends to salient features of theindividual. This facilitates the production of discriminativefeature vectors of the individual. This result also indicatesa new avenue of research for insect ReID. Since now weknow the salient features of the Trichogramma individuals,future works can exploit it to develop better ReID networksfor insects with sparse features.

15 of 21

Results & Analysis: Detection Performance

16 of 21

Results & Analysis: Tracking Results

17 of 21

8. Demonstration

18 of 21

Demonstration: Tracking by TREX (Strain: ISA3080)

19 of 21

Demo: Tracking by TrichTrack (Strain: ISA3080)

20 of 21

Demo: Tracking by TrichTrack (High Density)(Strain: E1.3)

21 of 21

Thank You!

Contact:

panivishal17@gmail.com