1 of 15

LRM: Large Reconstruction Model for Single Image to 3D��Adobe Research Australian National University

Fengrui Tian

Oct 2, 2024

2 of 15

Single Image to 3D Object Task

2

3 of 15

Single Image to 3D Is Important yet Challenging

3

  • Important
    • Broad applications in industrial design, AR/VR, etc…
  • Challenging: lack a generic and efficient approach
    • Per-shape optimization (Diffusion + NeRF-based Optimization)
    • Category-level generation (pixel-NeRF)
  • How to instantly create a 3D shape from single image of an arbitrary object?
    • Learn strong generic 3D prior

4 of 15

How to Learn a Strong Generic 3D Prior?

4

  • Take the experience of the success of LLMs
    • Highly scalable and effective networks (Transformer)
    • Enormous Data (730,648 3D objects from Objaverse and 220,219 object videos from MVImgNet)
    • Self-supervised training objectives (Loss at novel views)

5 of 15

Large Reconstruction Model

5

Dino Features

6 of 15

Large Reconstruction Model

6

7 of 15

Large Reconstruction Model

7

8 of 15

Large Reconstruction Model

8

9 of 15

Implementation Details

9

  • Costly
    • 128 NVIDIA (40G) A100 GPUs
    • 3 Days

10 of 15

Qualitative Results

10

11 of 15

Qualitative Results

11

12 of 15

Qualitative Results

12

https://yiconghong.me/LRM/

13 of 15

Quantitative Results

13

  • Comparison with SOTA

14 of 15

Quantitative Results

14

  • Ablations on model parameters

15 of 15

Quantitative Results

15

  • Ablations on datasets