1 of 13

High Resolution Point Clouds from mmWave Radar

Akarsh Prabhakara

Tao Jin

Arnav Das

Gantavya Bhatt

Lilly Kumari

Elahe Soltanaghai

Jeff Bilmes

Swarun Kumar

Anthony Rowe

2 of 13

Firefighting robot navigating

thick, dense smoke

Infrared Lidar

Visible-light Cameras

Cameras and lidars suffer

in smoky environments

3 of 13

Single-chip Millimeter-Wave Radars for through-smoke perception

Through-smoke perception

Size

Weight

Power

Cost

SWaP-C

Without smoke

X

Y

X

Y

With smoke

IDENTICAL!

Radars

8 cm

6 cm

4 of 13

4

Camera with ~0.01° angular resolution

Problem with Single-chip Millimeter-Wave Radars

Single-chip mmWave radar with

~15° angular resolution

5 of 13

Past Approaches

Synthetic Aperture Radar Imaging

  • Robot can move arbitrarily
  • Robot can move slowly
  • Robot can choose to even remain static

Odometry Specific

Machine Learning

Radar

Point

Cloud

IMU

Odometry

Higher-level Application Specific Machine Learning

6 of 13

Our Approach

X

Y

Deep Learning

Super Resolution Model

Raw Radar Data

High Resolution

Interpretable &

General Purpose

High Resolution Ground Truth Label

E.g: Lidar

Odometry

Mapping

Object Detection

Object Classification

REUSE EXISTING LIDAR

PERCEPTION WORKFLOWS

7 of 13

Why is this hard?

Radar Data very different from Camera Data for Machine Learning

Single-chip mmWave radar

Low resolution camera image

8 of 13

Pre-Processing

Architecture Choices

Loss Functions

RadarHD: Our Overall Solution

Check out the paper for detailed design decisions!

9 of 13

RadarHD Hardware, Data and Implementation

mmWave Radar

Lidar

10 of 13

RadarHD Qualitative Result

Raw Single-Chip Radar

RadarHD: Our Solution

(also only using a

single-chip radar)

64 beam Mechanical Lidar

$

$

$$$$

Check the paper for quantitative results!

11 of 13

Perception on top of RadarHD

Running SLAM on Cartographer using RadarHD Output

0

25

0

20

17.5

0

0

15

X (m)

X (m)

Y (m)

Y (m)

RadarHD

Lidar Ground Truth

x

x

-8

2

X (m)

-6

6

Y (m)

0

10

-6

6

Y (m)

X (m)

RadarHD�GT�CFAR

12 of 13

RadarHD in smoky environments

No smoke

Low Smoke Density

High Smoke Density

Mean Error (meters)

13 of 13

  • Enabling quality perception in occluded scenes

​

  • Deep learning super resolution of single-chip radar to get lidar-like point clouds

​

  • A large raw radar-lidar indoor dataset

​

  • Use the generated high-res radar point clouds for perception tasks like odometry and mapping

High Resolution Point Clouds from mmWave Radar