1 of 13

CSE 5539: �3D Object Detection

2 of 13

Data

  • LiDAR point clouds
    • A list of 3D (+ reflectance) points
    • 3D: (x, y, z) centered around the ego-car

​

  • Images (mainly as complementary information)

​

​

​

​

[Source: Graham Murdoch/Popular Science]

3 of 13

LiDAR points: 3D coordinates

3D object detection

3D instance segmentation

Identify objects’ 3D locations in (x, y, z), not 2D locations in image pixels!

4 of 13

LiDAR-based 3D perception

[Source: Graham Murdoch/Popular Science]

LiDAR:

  • Light Detection and Ranging sensor
  • accurate 3D point clouds of the environment

5 of 13

Two major ways to process LiDAR

  • Point-wise processing
    • PointNet
    • PointNet++
    • …

​

  • Voxel-based processing
    • Point-Pillar
    • Voxel-Net
    • PIXOR

6 of 13

Voxel-based processing + 3D object detectors

  • Occupation (PIXOR)

[Yang et al., PIXOR: Real-time 3D Object Detection from Point Clouds, 2019]

7 of 13

Voxel-based processing + 3D object detectors

  • VoxelNet (designed grid features)

[Zhou et al., VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection, 2017]

8 of 13

Voxel-based processing + 3D object detectors

  • PointPillars (designed grid features)

[Lang et al., PointPillars: Fast Encoders for Object Detection from Point Clouds, 2019]

9 of 13

Voxel-based processing + 3D object detectors

  • CentorPoint

[Ying et al., Center-based 3D Object Detection and Tracking, 2021]

10 of 13

Point-wise processing

  • PointNet

[Qi et al., PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, 2017]

11 of 13

Point-wise processing

  • PointNet++

[Qi et al., PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space, 2017]

12 of 13

Point-wise 3D object detectors

[Shi et al., PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud, 2019]

13 of 13

Point-wise 3D object detectors

[Shi et al., PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud, 2019]