1 of 11

Cycle Safely

A collision prediction system��Simon Pointner

Supervisor: Stefan Ohrhallinger

Research Unit of Computer Graphics

Institute of Visual Computing & Human-Centered Technology

TU Wien, Austria

2 of 11

Goal

Develop a system for cyclists that predicts collisions with other road users.��

Sub-Problems:

  • Object Detection
  • Object Tracking
  • Trajectory Prediction

​

​

​

Cycle Safely - Simon Pointner

2

3 of 11

Input

​

​

​

​

Cycle Safely - Simon Pointner

3

https://www.arxiv-vanity.com/papers/1812.07179/

LiDAR

GPS

4 of 11

Related Work

Datasets: KITTI[1], Waymo Open Dataset[2], Argoverse[3], CARLA[4]

Object Detection: VoxelNet[5], PointRCNN[6], RTM3D[7]

Multiple Object Tracking: 3DMOT[8]

Trajectory Prediction:

  • HD Maps: DESIRE[10], Densetnt[11]�BEV semantic map: Mantra[12]�

Challenges:

  • Semantic information extraction
  • Limited FOV
  • Jitter
  • Realtime

Cycle Safely - Simon Pointner

4

5 of 11

Proposed Pipeline

Cycle Safely - Simon Pointner

5

Input

Object Detection

Trajectory Extraction

Trajectory Prediction

GPS

LIDAR

Trajectory prediction needs pre-segmented LIDAR data!

SFA3D

3D MOT

MANTRA

6 of 11

Trajectory Prediction

Problem: MANTRA requires pre-segmented LIDAR data�

Solution: Modify and train MANTRA�to use simulated data with LIDAR

​

Cycle Safely - Simon Pointner

6

Semantic Segmentation (RangeNet++[14])

7 of 11

Proposed Pipeline

Cycle Safely - Simon Pointner

7

Input

Object Detection

Trajectory Extraction

Trajectory Prediction

GPS

Input

Object Detection

Trajectory Extraction

Trajectory Prediction

LIDAR

Trajectory prediction needs pre-segmented LIDAR data!

SFA3D

3D MOT

MANTRA

RangeNet++

8 of 11

Dataset Generation

Cycle Safely - Simon Pointner

8

Result: Sequences consisting of LIDAR frames + Ground Truth of Trajectory of own vehicle and nearby vehicles + GPS

CARLA

LIDAR

GPS

Vehicle Trajectory GT

9 of 11

Contribution

  1. Design of the Pipeline (novel)
  2. Context aware future trajectory prediction for cyclists without pre-generated context

​

Methodology:

  1. Recombination of existing Methods
  2. Adapted training of MANTRA with a larger and adapted Dataset
  3. Validation of the prediction model trained with synthetic data used on real data (KITTI)

​

Cycle Safely - Simon Pointner

9

10 of 11

References

[1] Andreas Geiger and Philip Lenz and Raquel Urtasun, 2012, Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite, https://www.cvlibs.net/datasets/kitti/

[2] Waymo, 2019, Waymo Open Dataset, https://waymo.com/open/

[3] Benjamin Wilson et al., 2021, Argoverse 2: Next Generation Datasets for Self-driving Perception and Forecasting, https://www.argoverse.org/av2.html

[4] Alexey Dosovitskiy and German Ros and Felipe Codevilla and Antonio Lopez and Vladlen Koltun, 2017, CARLA - An Open Urban Driving Simulator, https://github.com/carla-simulator/carla

[5] Zhou, Yin, and Oncel Tuzel, 2018, Voxelnet: End-to-end learning for point cloud based 3d object detection

[6] Shi, Shaoshuai, Xiaogang Wang, and Hongsheng Li, 2019, Pointrcnn: 3d object proposal generation and detection from point cloud

[7] Li, Peixuan, et al., 2020, Rtm3d: Real-time monocular 3d detection from object keypoints for autonomous driving

[8] Wu, Hai, et al., 2021, 3d multi-object tracking in point clouds based on prediction confidence-guided data association, https://github.com/hailanyi/3D-Multi-Object-Tracker

[9] Gupta, Agrim, et al., 2018, Social gan: Socially acceptable trajectories with generative adversarial networks

[10] Lee, Namhoon, et al., 2017, Desire: Distant future prediction in dynamic scenes with interacting agents

[11] Gu, Junru, Chen Sun, and Hang Zhao., 2021, Densetnt: End-to-end trajectory prediction from dense goal sets

[12] Francesco Marchetti, 2020, Mantra: Memory augmented networks for multiple trajectory prediction, https://github.com/Marchetz/MANTRA-CVPR20

[13] Nguyen Mau Dzung, 2020, SFA3D, https://github.com/maudzung/SFA3D

[14] A. Milioto and I. Vizzo and J. Behley and C. Stachniss, 2019, RangeNet++: Fast and Accurate LiDAR Semantic Segmentation, https://github.com/PRBonn/lidar-bonnetal

​

​

Cycle Safely - Simon Pointner

10

11 of 11

Thank You for your Attention!

Cycle Safely - Simon Pointner

11