“Data, Data, Data… Watson, I need Data!”

CS280: Computer Vision

1. Efros, UC Berkeley, Spring 2026

.

(much of) The Magic is in the Data

Importance of Past Experience

Claude Monet

Gare St.Lazare

Paris, 1877

Seeing more than meets the eye

Video by Antonio Torralba (starring Rob Fergus)

But actually…

Video by Antonio Torralba (starring Rob Fergus)

“Our perception relies

on memory as much as it does on incoming information, which blurs the border between perception and cognition.”

-- Moshe Bar

​

“Mind" is largely an emergent property of "data."

-- Lance Williams

​

“Our perception relies

on memory as much as it does on incoming information, which blurs the border between perception and cognition.”

-- Moshe Bar

​

“Mind" is largely an emergent property of "data."

-- Lance Williams

​

In old days, data got little respect

Algorithm

Features

Data

Face Detection: Early Success Story (late 1990s)

• Rowley, Baluja, and Kanade, 1998
• algorithm: neural network
• Schniderman & Kanade, 1999
• algorithm: naïve Bayes
• Viola & Jones, 2001
• algorithm: boosted cascade

Our Scientific Narcissism

​

​

All things being equal, we prefer to credit our own cleverness

“Unreasonable Effectiveness of Data”

• Parts of our world can be explained by elegant mathematics:
• physics, chemistry, astronomy, etc.
• But some cannot:
• psychology, genetics, economics, AI, etc.

​

• Enter: The Magic of Data

[Halevy, Norvig, Pereira 2009]

Decade before “The Bitter Lesson”

Evolution is messy

Navier-Stokes Equation

+ weather

+ location

+ …

Let’s Define a Tree

Brown trunk moving upward and branching with leaves

Are these trees?

• Hard to give a good definition of tree
• But any 3-year-old can tell a tree from a non-tree
• The 3-year-old has learned from data!

Slide from Makdon Ismail

With enough data, brain-dead lookup (aka Nearest Neighbor classifier) works surprisingly well

How many images does a 5-year-old see?

• “80 million tiny images: a large dataset for non-parametric object and scene recognition”, Antonio Torralba, Rob Fergus and William T. Freeman. PAMI 2008.

Where CIFAR-10/100 came from!

Tiny Image pack a punch!

Human Scene Recognition

32x32 turns out to be enough!

80,000,000 images

And after 1 year downloading images

A. Torralba, R. Fergus, W.T. Freeman. PAMI 2008

2 years before ImageNet!

Powers of 10

A. Torralba, R. Fergus, W.T.Freeman. PAMI 2008

Some images are unique

But not all image are so original

But not all image are so original

Lots

Of

Images

A. Torralba, R. Fergus, W.T.Freeman. PAMI 2008

Lots

Of

Images

A. Torralba, R. Fergus, W.T.Freeman. PAMI 2008

Lots

Of

Images

Automatic Colorization

Grayscale input High resolution

Colorization of input using average

A. Torralba, R. Fergus, W.T.Freeman. 2008

First Scaling Law!

[Hays & Efros, SIGGRAPH’07]

2 Million Flickr Images

Why does it work?

Nearest neighbors from a�collection of 20 thousand images

Nearest neighbors from a�collection of 2 million images

… 200 scene matches

im2GPS�(using 6 million GPS-tagged Flickr images)

Im2gps [Hays & Efros, CVPR’08]

6 Million Flickr Images

im2GPS�(using 6 million GPS-tagged Flickr images)

Im2gps [Hays & Efros, CVPR’08]

15 years later…

Algorithm vs. Data

PlaNet, 2016

im2gps, 2008

• Deep Net
• 91 million images

• Nearest Neighbors
• 6 million images

Algorithm vs. Data

Weyand et al, PlaNet, ECCV 2016

The Good News

​

​

Really stupid algorithms + Lots of Data

= “Unreasonable Effectiveness”

[Halevy, Norvig, Pereira 2009]

​

​

But can humans ever remember so much?

[Halevy, Norvig, Pereira 2009]

What’s the Capacity of Visual Long Term Memory?

http://olivalab.mit.edu/MM/

Aude Oliva, MIT

What’s the Capacity of Visual Long Term Memory?

“Basically, my recollection is that we just separated the pictures into distinct thematic categories: e.g. cars, animals, single-person, 2-people, plants, etc.) Only a few slides were selected which fell into each category, and they were visually distinct.”

According to Standing

Standing (1973)

10,000 images

83% Recognition

What was known…

What was not known…

Sparse Details

Dogs

Playing Cards

“Gist” Only

Highly Detailed

… people can remember thousands of images

… what people are remembering for each item?

Slide by Aude Oliva

Completely

different objects...

Different exemplars

of the same kind of object...

Different states of

the same object...

Massive Memory Experiment I

A stream of objects will be presented on the screen for

~ 3 second each.

Your primary task:

​

Remember them ALL!

afterwards you will be tested with…

Your other task:

​

Detect exact repeats anywhere in the stream

Massive Memory Experiment I

Ready?

(Seriously, get ready to clap. The images go by fast…)

<clap!>

<clap!>

10 Minutes Later...

<clap!>

<clap!>

30 Minutes Later...

1 Hour Later...

<clap!>

2 Hours Later...

<clap!>

4 Hours Later...

<clap!>

5:30 Hours Later...

Which one did you see?

(go ahead and shout out your answer)

-A-

-B-

-A-

-B-

-A-

-B-

Recognition Memory Results

Visual Cognition

Expert Predictions

92%

Recognition Memory Results

92%

88%

87%

Brady, et al. (2008), PNAS

So, do humans have �“photographic memory”?

“No meaning” textures matching 1^st & 2^nd order statistics of real scenes

(by Portila & Simmonceli)

Isola & Oliva,

unpublished

Ready?

Clap your hands when

you see an image repeat

<clap!>

<clap!>

<clap!>

<clap!>

<clap!>

<clap!>

<clap!>

Scene/object

texture

4

​

​

3

​

​

2

​

​

1

​

​

0

Isola & Oliva,

unpublished

chance

Humans do more than just memorize

Memorable

Hit rate: 67/70

False alarm rate: 4/80

Average

Hit rate: 59/81

False alarm rate: 7/92

Forgettable

Hit rate: 21/68

False alarm rate: 3/82

Memorability (Isola et al)