1 of 37

Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation

​

​

​

​

Anthony Simeonov∗, Yilun Du∗, Andrea Tagliasacchi,�Joshua B. Tenenbaum, Alberto Rodriguez, Pulkit Agrawal†, Vincent Sitzmann†

​

2 of 37

Goal:

​

​

​

​

3 of 37

Goal:

​

​

​

​

4 of 37

Related Work:

​

​

​

​

:

​

​

​

​

5 of 37

Related Work:

​

​

​

​

6 of 37

Related Work:

​

​

​

​

7 of 37

8 of 37

Neural Descriptor Fields

​

​

​

​

9 of 37

Neural Descriptor Fields

​

​

​

​

10 of 37

Neural Descriptor Fields

​

​

​

​

11 of 37

Neural Descriptor Fields

​

​

​

​

12 of 37

Neural Descriptor Fields

​

​

​

​

13 of 37

Neural Descriptor Fields

​

​

​

​

14 of 37

Neural Descriptor Fields

​

​

​

​

15 of 37

Neural Descriptor Fields

​

​

​

​

  1. What region in space to correspond?

(What is the local coordinate frame)

ii) How to correspond?

(What is the representation)

16 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

17 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

18 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

19 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

20 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

21 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

Spherical CNNs

Spherical CNNs

22 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

23 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

24 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

25 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

26 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

27 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

28 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

29 of 37

Neural Descriptor Fields

​

​

​

​

How to correspond?

30 of 37

Neural Descriptor Fields

​

​

​

​

What region in space to correspond?

31 of 37

Neural Descriptor Fields

​

​

​

​

What region in space to correspond?

32 of 37

Neural Descriptor Fields

​

​

​

​

What region in space to correspond?

33 of 37

Neural Descriptor Fields

​

​

​

​

Results

34 of 37

Neural Descriptor Fields

​

​

​

​

Results

35 of 37

Neural Descriptor Fields

​

​

​

​

Results

36 of 37

Neural Descriptor Fields

​

​

​

​

What region in space to correspond?

37 of 37

Neural Descriptor Fields

​

​

​

​

What region in space to correspond?