1 of 21

Millimeter Wave Radar SLAM

Shuqin Xie, Dongfeng Yu

CMU Faculty: Michael Kaess

Sponsor: Amazon Lab126

1

1

1

1

2 of 21

Outline

  • Motivation / Scope
  • Past Progress
  • Current Progress
  • Next Steps

2

2

2

2

3 of 21

Motivation

  • Cheap
  • Robust
  • Long-range
  • Precise

3

Sheeny, Marcel, et al. "RADIATE: A Radar Dataset for Automotive Perception." arXiv preprint arXiv:2010.09076 (2020).

3

3

3

3

4 of 21

Motivation & Goal

4

This Semester

​

​

​

​

​

​

​

​

​

Sensor Reading and Preprocessing

Odometry (Frontend)

Loop Detection

Optimization (Backend)

Map Building

4

4

4

4

5 of 21

2D Outdoor

  • Dataset
    • Oxford Radar RobotCar
  • Task
    • Keypoint Detection
    • Keypoint Matching
  • Model
    • Input: 2D Frame
    • Output:
      • Coordinates
      • Features
    • Supervision: Relative Motion

5

5

5

5

6 of 21

Outline

  • Motivation
  • Past progress
  • Current progress
  • Next steps

6

6

6

6

7 of 21

ColoRadar

  • Scenes
    • Hallway, Courtyard, Cave
  • Format
    • 3D Intensity Map
    • Velocity
  • Synced Sensors
    • Lidar/IMU
  • Sparse / Noisy
  • Same Hardware

7

Kramer, Andrew, et al. "ColoRadar: The Direct 3D Millimeter Wave Radar Dataset." arXiv preprint arXiv:2103.04510 (2021).

7

7

7

7

8 of 21

Current progress

  1. Lidar-assisted radar keypoint detection

8

8

8

8

9 of 21

Lidar-assisted Radar keypoint detection

​

  • How do we deal with the noisy data?
    1. Signal Processing
    2. Extra supervision from time-synced Lidar sensors
  • Lidar-assisted radar keypoint detection
    • Training
      1. Keypoint Scores Automatically Learned From Relative Motion Supervision
      2. Keypoint Scores Constrained by Lidar
    • Inference
      • No Lidar Needed

9

9

9

9

10 of 21

Lidar-assisted Radar keypoint detection

From Volume-based to point-based

Sparse

Point cloud “Segmentation” problem:

  • For each radar point, predict 1 if close to a Lidar point, otherwise predict 0.

​

​

​

​

​

Data-preprocessing:

  • pick the top 6144 points based on intensity.
  • Convert intensity to log space and normalize.

​

Pointnet

10

10

10

10

11 of 21

Lidar-assisted Radar keypoint detection

​

11

11

11

11

11

12 of 21

Current progress

  • Lidar-assisted radar keypoint detection
  • Lidar-assisted radar keypoint matching

12

12

12

12

13 of 21

Lidar-assisted Radar Keypoint Matching

Goal: predict high scores and stable features for true landmarks

​

Pointnet

Pointnet

frame A:

frame B:

interpolation:

1. Warp to frame B,

2. Find

3. Compute

13

13

13

13

14 of 21

Lidar-assisted Radar Keypoint Matching

Goal: For true landmarks, return a high score and a stable feature

Point-cloud matching problem:

​

Pointnet

Pointnet

frame A:

frame B:

matching loss

matching loss:

1. loss for each point

2. Weighted by score

14

14

14

14

15 of 21

Lidar-assisted Radar Keypoint Matching

​

15

15

15

15

16 of 21

Discretization problem

​

​

16

16

16

16

17 of 21

Discretization problem

Odometry result

xy plane

z axis

17

17

17

17

18 of 21

Discretization problem

​

Attempts to fix:

offset A

offset B

1. Learning based: learn an offset to compensate for discretization

Sarlin et al, “Superglue: Learning Feature Matching with Graph Neural Networks”, CVPR 2020

18

18

18

18

19 of 21

Discretization problem

​

Attempts to fix:

Training time:

Test time:

1. Learning based: learn an offset to compensate for discretization

2. Optimization-based: reconstruct interpolation coefficients.

19

19

19

19

20 of 21

Feature discrimination ability

0.1

0.9

0.9

0.9

“Local confusion”:

Backbone considering locality should be helpful

20

20

20

20

21 of 21

Conclusion

Next steps

For discretization:

  • Use a better backbone and experiment with the optimization approach.
  • Volumn-based approach to avoid discretization.

For SLAM system:

  • Build a full SLAM system to compensate for odometry drift. (major work for next semester)
  • Managed to extract useful keypoints from Radar point cloud
  • Managed to associate keypoints in different frames to estimate motion.
  • identify two important problems: discretization and feature discrimination ability.

21

21

21

21