1 of 16

Te-Lin Wu*, Yu Zhou*, Nanyun Peng

Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge

2 of 16

Motivation & Definitions

  • Active Objects:
    • Object Undergoing major state Changes (OUC)
    • Tools that facilitate such state changes

3 of 16

Ego4d SCOD Examples

3

4 of 16

Model Overview

4

5 of 16

Symbolic World Knowledge

5

6 of 16

GPT Pipeline

6

7 of 16

GPT Prompt Pipeline

7

8 of 16

GPT Pipeline Intrinsic Evaluation

8

9 of 16

GPT Error Analysis

9

10 of 16

Ego4D SCOD Results

10

11 of 16

Epic Kitchens TREK-150 Results

11

12 of 16

SCOD Qualitative Results

12

13 of 16

Tracking on Ego4d Detection Ablation

13

14 of 16

14

Q&A

15 of 16

15

16 of 16

16