1 of 23

vSLAM in Dynamic Environment

Sergei Linok

Center for Cognitive Modeling, MIPT

29 April 2021

2 of 23

DynaSLAM: Overview

2

Berta Bescos, Jos ́e M. F ́acil, Javier Civera and Jos ́e Neira

DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes, IROS 2018

https://github.com/BertaBescos/DynaSLAM

3 of 23

DynaSLAM: Multi-view Geometry

3

Berta Bescos, Jos ́e M. F ́acil, Javier Civera and Jos ́e Neira

DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes, IROS 2018

https://github.com/BertaBescos/DynaSLAM

4 of 23

DynaSLAM: Practice Results

4

Berta Bescos, Jos ́e M. F ́acil, Javier Civera and Jos ́e Neira

DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes, IROS 2018

https://github.com/BertaBescos/DynaSLAM

5 of 23

DynaSLAM: Background Inpainting

5

Berta Bescos, Jos ́e M. F ́acil, Javier Civera and Jos ́e Neira

DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes, IROS 2018

https://github.com/BertaBescos/DynaSLAM

6 of 23

DS-SLAM: Overview

6

Chao Yu, Zuxin Liu, Xin-Jun Liu, Fugui Xie, Yi Yang, Qi Wei, Qiao Fei

DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments, IROS 2018

https://github.com/ivipsourcecode/DS-SLAM

7 of 23

DS-SLAM: Practice Results

7

Chao Yu, Zuxin Liu, Xin-Jun Liu, Fugui Xie, Yi Yang, Qi Wei, Qiao Fei

DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments, IROS 2018

https://github.com/ivipsourcecode/DS-SLAM

Dense semantic 3D octo-tree map

8 of 23

DynaSlam vs DS-SLAM: TUM RGB-D Dataset

8

9 of 23

DOTMask for Visual SLAM: Overview

9

Jonathan Vincent, Mathieu Labbe, Jean-Samuel Lauzon, Franc¸ois Grondin

Dynamic Object Tracking and Masking for Visual SLAM, IROS 2020

https://github.com/introlab/dotmask

DOS - Dynamic Object State

MDI - Mask Depth Image

MOC - Moving Object Classification module

MO-MDI - Moving Object Masked Depth Image

10 of 23

DOTMask for Visual SLAM: Practice Results

10

Jonathan Vincent, Mathieu Labbe, Jean-Samuel Lauzon, Franc¸ois Grondin

Dynamic Object Tracking and Masking for Visual SLAM, IROS 2020

https://github.com/introlab/dotmask

11 of 23

DOTMask vs DynaSlam

11

12 of 23

The Need for speed: Overview

12

Mina Henein, Jun Zhang, Robert Mahony and Viorela Ila

Dynamic SLAM: The Need For Speed, ICRA 2020

https://github.com/halajun/VDO_SLAM

13 of 23

The Need for speed: Effect of front-end components (vKITTI)

13

Mina Henein, Jun Zhang, Robert Mahony and Viorela Ila

Dynamic SLAM: The Need For Speed, ICRA 2020

https://github.com/halajun/VDO_SLAM

RTE - Relative Translation Error

RRE - Relative Rotational Error

OMTE - Object Motion Translation Error

OMRE - Object Motion Rotation Error

OMSE - Object Motion Speed Error

14 of 23

More Dynamic SLAM + semantic

14

  • Linyan Cui, Chaowei Ma: SDF-SLAM, 2020 [ORB-SLAM2 + SegNet]
  • Shuo Wang, Xudong Lv , Junbao Li, Dong Ye: Coarse Semantic-Based Motion Removal for Robust Mapping in Dynamic Environments, 2020 [ORB-SLAM2 + CenterNet]
  • A. Pancham, D. Withey, G.Bright: Amore, 2020 [ORB-SLAM + Yolo]
  • Xinyang Zhao, Chanchong Wang, Marcelo H.: Real-Time VI Localization using Semantic Segmentation Towards Dynamic Environments, 2020 [VINS-Mono + Deeplab V3+]
  • Yingchun Fan, Qichi Zhang, Shaofeng Liu: Semantic SLAM With More Accurate Point Cloud Map in Dynamic Environments, 2020 [ORB-SLAM2 + BlitzNet]
  • Yubao Liu, Jun Miura: RDS-SLAM, 2021 [ORB-SLAM3 + Mask RCNN]
  • Karnik Ram, Chaitanya Kharyal, Sudarshan S.: RP-VIO, 2021 [VINS-Mono + Plane-Recover Model]

​

15 of 23

GMS: Overview

15

Handuo Zhang, Karunasekera Hasith, and Han Wang

GMC: Grid Based Motion Clustering in Dynamic Environment, 2019

16 of 23

GMS: Practice Results

16

Handuo Zhang, Karunasekera Hasith, and Han Wang

GMC: Grid Based Motion Clustering in Dynamic Environment, 2019

17 of 23

Visual Odometry in DE via Rigid Motion Model: Overview

17

Sangil Lee, Clark Youngdong Son, and H. Jin Kim

Robust Real-time RGB-D Visual Odometry in Dynamic Environments via Rigid Motion Model, IROS 2019

18 of 23

Visual Odometry in DE via Rigid Motion Model: Practice Results

18

Sangil Lee, Clark Youngdong Son, and H. Jin Kim

Robust Real-time RGB-D Visual Odometry in Dynamic Environments via Rigid Motion Model, IROS 2019

19 of 23

MID-Fusion: Overview

19

Binbin Xu, Wenbin Li, Dimos Tzoumanikas

MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM, ICRA 2019

20 of 23

MID-Fusion: Practice Results

20

Binbin Xu, Wenbin Li, Dimos Tzoumanikas

MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM, ICRA 2019

21 of 23

Subtotal

21

22 of 23

Object-Based Pose Graph: Overview

22

Clara Gomez , Alejandra C. Hernandez , Erik Derner

Object-Based Pose Graph for Dynamic Indoor Environments, IROS 2020

23 of 23

Dynamic SLAM: Current limitations

23

  • segmentation problems (outlier, limited classes)
  • tracking problems (breaks)
  • non rigid objects (tracking)
  • features extraction from static non structure objects
  • realtime