I-MuPPET: Interactive Multi-Pigeon Pose Estimation and Tracking �Urs Waldmann, Hemal Naik, Máté Nagy, Fumihiro Kano,
Iain D. Couzin, Oliver Deussen, and Bastian Goldlücke�
Single Pigeon Data
Multi Pigeon Data
I-MuPPET
Evaluation
Comparison with DeepLabCut (DLC)
Comparison with 3D Bird Reconstruction (3DBR)
Quantitative results on pigeon data
RMSE: 3.2 px
PCK@0.05: 0.94
PCK@0.1: 0.97
Tracking performance
Inference speed
Inference speed up to 17 fps.
This work was supported by the DFG under Germany’s Excellence Strategy - EXC 2117 - 422037984.
Presented at the German Conference on Pattern Recognition (GCPR), Konstanz, September 2022.
References
[1] A. Mathis et al.: Deeplabcut: markerless pose estimation of user-defined body parts with deep learning. In Nat. Neurosci., 2018.
[2] M. Badger et al.: 3d bird reconstruction: A dataset, model, and shape recovery from a single view. In ECCV, 2020.
Pipeline
ID 1
ID 2
28 000 annotated RGB images
7 keypoints
Resolution of 1920 x 1080 x 3 pixels
Area of approx. 5x5 meters
2 video cameras
ID 1
ID 2
ID 3
Train with annotated single pigeon data.
Inference on multi-pigeon video sequences.
Model, iterations | RMSE [px] |
I-MuPPET, 200K iter. | 4.2 |
DLC, 200K iter. [1] | 3.6 ∓ 0.2 |
DLC, 350/600K iter. [1] | 3.2 ∓ 0.2 |
Model, epochs | PCK@0.05 | PCK@0.1 |
I-MuPPET, 45 epochs | 0.39 | 0.56 |
I-MuPPET, 60 epochs | 0.36 | 0.54 |
3DBR, 60 epochs [2] | 0.46 | 0.64 |
conf. score | HOTA | MOTA | MOTP | Rcll | Prcn | MT | ML | FPF | IDS | Frag | IDF1 |
0 | 0.53 | 0.48 | 0.61 | 0.83 | 0.70 | 0.64 | 0.01 | 0.99 | 24 | 292 | 0.75 |
0.5 | 0.57 | 0.65 | 0.61 | 0.83 | 0.83 | 0.64 | 0.01 | 0.49 | 8 | 278 | 0.82 |
0.75 | 0.57 | 0.67 | 0.61 | 0.83 | 0.84 | 0.64 | 0.01 | 0.44 | 11 | 280 | 0.83 |
0.9 | 0.56 | 0.68 | 0.61 | 0.82 | 0.85 | 0.64 | 0.01 | 0.39 | 14 | 277 | 0.83 |
7 900 frames
70 objects in total