1 of 18

RetroSphere:

Self-contained Passive 3D Controller Tracking for Augmented Reality

Ananta Narayanan Balaji*, Clayton Kimber#, David Li^,

Shenghzi Wu#, Ruofei Du#, David Kim#

​

*National University of Singapore #Google Research ^University of Maryland, College Park

​

​

​

​

IMWUT 2022

2 of 18

RetroSphere - 6DoF Tracking for AR Glasses

Our proposed solution provides low-power 6 DoF tracking and a stylus that doesn't require any electronics or charging.

​

​

3 of 18

RetroSphere - 6DoF Tracking for AR Glasses

Enable spatial 3D interaction on portable devices with form-factor and power constraints.

​

​

​

​

4 of 18

RetroSphere

RetroSphere - 6DoF Tracking for AR Glasses

On AR glasses input is often limited to 1 or 2 dimensional on-device physical input, as hand tracking and other continuous camera-based tracking isn’t possible given the limited compute power.

​

VR controllers are not very portable for everyday AR use cases and a smaller controller or stylus may need frequent recharging.

​

​

​

5 of 18

RetroSphere

RetroSphere - 6DoF Tracking for AR Glasses

Our solution is self-contained and can augment existing AR glasses for prototyping purposes or built into future products.

6 of 18

RetroSphere - 3D Tracking Hardware

Stereo infrared blob tracking cameras

Colocated IR LED illuminators for tracking our retroreflective stylus

On-device 6DoF estimation pipeline for tracking our RetroReflective stylus

RetroSphere - 6DoF Tracking for AR Glasses

  • Always-on low-power 6DoF tracking hardware.

​

  • Low-cost and affordable hardware.

​

  • Easy integration into future AR glasses.

7 of 18

RetroSphere - Passive Retroreflective Stylus

​

​

8mm diameter

Passive stylus -

consists of 3 Retroreflectors

each of unique size

5mm diameter

10mm

diameter

Supports 3/5/6DoF tracking

It can be easily integrated into other form factors such as wristband, pencil, ring etc.

RetroSphere - 6DoF Tracking for AR Glasses

8 of 18

RetroSphere 6DoF Tracking Algorithm

​

​

​

​

Sparse and requires very low compute - Run on a ESP32

Offers 10X power saving over existing passive 3D tracking approach.

9 of 18

RetroSphere - Tracking Accuracy

​

​

​

​

​

​

RetroSphere - 6DoF Tracking for AR Glasses

10 of 18

RetroSphere - Drawing a Cube with the Stylus

​

​

​

​

​

​

​

​

​

​

RetroSphere - 6DoF Tracking for AR Glasses

11 of 18

RetroSphere Menu UI

​

​

​

​

​

​

​

​

RetroSphere - 6DoF Tracking for AR Glasses

12 of 18

RetroSphere - RetroPen

​

​

​

​

​

​

​

RetroSphere - 6DoF Tracking for AR Glasses

13 of 18

RetroPen - Working Mechanism

​

​

​

​

​

​

​

​

​

  • Writing
  • Erasing
  • Pressure sensitivity
  • Tilt sensitivity
  • Battery-free/Passive

RetroSphere - 6DoF Tracking for AR Glasses

14 of 18

RetroRing - Prototype

​

​

​

​

​

​

​

Retroreflective rings for thumb and index finger (each with a unique size).

​

Can be used to make gestures for 3D UI controls.

15 of 18

RetroRing for UI controls

​

​

​

​

​

​

​

​

​

RetroSphere - 6DoF Tracking for AR Glasses

16 of 18

RetroRing for 3D Object Manipulation

​

​

​

​

​

​

​

RetroSphere - 6DoF Tracking for AR Glasses

17 of 18

We hope that RetroSphere will allow researchers and practitioners to study and prototype spatial input on lightweight AR glasses more easily.

​

​

RetroSphere - 6DoF Tracking for AR Glasses

18 of 18

RetroSphere 6DoF Mid-air Drawings

​

​

​

​

​

​

​

​

RetroSphere - 6DoF Tracking for AR Glasses