The Koala and the Raccoon

Evaluating humanlike understanding on abstraction benchmarks

Victor Odouard and Melanie Mitchell

​

​

​

Santa Fe Institute

Humanlike understanding

Using the concepts a human would use to make sense of a particular situation (not shortcuts)

Anthropomorphize

GEIRHOS ET AL, 2020

1. What concepts would a human use?
2. Design problems testing variations on those concepts.

Concept-based evaluation sets

Reveals information that conventional test sets don’t.

Concepts: Sameness, progression

RAVEN

(+ multiple choice answers)

ZHANG ET AL, 2021

Sameness

“Conventional” accuracy does not predict humanlike understanding

BENNY ET AL, 2021; WU ET AL, 2020

CHOLLET, 2019;

DE MIQUEL BLEIER, 2020

Worse than 19% test accuracy!

Probably using shortcuts.

DE MIQUEL BLEIER, 2020

“Conventional” accuracy does not predict humanlike understanding

Conclusion

Lack of understanding!

Progression

In-domain

In-domain + Noise

Out-of-domain

In front/behind

Social pressure (broadly construed):

​

Encouragement of a behavior (here, cooperation) through social interactions (individually or structurally)

How do we get cooperation in such a world?

Information + depth of interaction

Less of this

What mechanisms can lead to cooperation when we only have partial information about each other?

My coauthors: Mike Price, Diana Smirnova, Shimon Edelman

Tyler Millhouse, Helena Miton, John Miller, Hajime Shimao, Arseny Moskvichev

THANK YOU

I. Cooperation and information

A. Cooperation

B. Partial information

C. Candidate mechanisms

​

II. No visibility model: Conscience

​

III. No visibility results: Amplify group selection

​

IV. Discussion

A. Cultural Evolution

B. Policy Intuition

I. Cooperation and information

A. Cooperation

B. Partial information

C. Candidate mechanisms

​

II. No visibility model: Conscience

​

III. No visibility results: Amplify group selection

​

IV. Discussion

A. Cultural Evolution

B. Policy Intuition

I. Cooperation and information

COOPERATION

• Paying a private cost c for a public benefit b.
• b > c
• Free-riding is the best move

​

What mechanisms can lead to cooperation when we only have partial information about each other?

Public Goods Game

GINTIS ET AL., 2001

​

TWO PLAYER CASE

Prisoner’s Dilemma

What mechanisms can lead to cooperation when we only have partial information about each other?

PARTIAL INFORMATION

Levels of visibility

What mechanisms can lead to cooperation when we only have partial information about each other?

MECHANISMS: BIASED ALLOCATION

DEFECTOR

What mechanisms can lead to cooperation when we only have partial information about each other?

HENRICH, 2004

​

MECHANISMS: BIASED ALLOCATION

DEFECTOR

IN GENERAL

Bigger multiplier,

smaller bias

What mechanisms can lead to cooperation when we only have partial information about each other?

HENRICH, 2004

​

MECHANISM FOR LOW VISIBILITY

What mechanisms can lead to cooperation when we only have partial information about each other?

Cooperate with those who have cooperated with others.

INDIRECT RECIPROCITY + GOSSIP

What mechanisms can lead to cooperation when we only have partial information about each other?

Bias: those who cooperate get cooperated with.

NOWAK, 2005

​

WHAT KEEPS GOSSIP TRUTHFUL?

What mechanisms can lead to cooperation when we only have partial information about each other?

MECHANISM FOR NO VISIBILITY

GROUP SELECTION

Groups with higher total fitness outcompete those with lower total fitness.

MAYNARD SMITH, 1976

​

GROUP SELECTION: A SECOND PERSPECTIVE

Bias: Cooperators more likely to be in highly cooperative groups than defectors.

TYPES OF INFORMATION

INFORMATION

TYPES OF INFORMATION

INFORMATION

WAY FORWARD

Context: No visibility

Candidate Mechanism: Group selection

Problem: Weak

1. Fighting intra-group dynamics

​

WEAKNESSES OF GROUP SELECTION

2. Relies on inter-group variance

​

BOWLES ET AL., 2003

​

More anonymous

​

​

Lower visibility

LARGE GROUPS AND VISIBILITY

Large groups

Less variance

​

​

Weak group selection

Double-whammy

Context: No visibility

Candidate Mechanism: Group selection

Problem: Weak

Candidate Solution: Conscience as an amplifier

​

WAY FORWARD

I. Cooperation and information

A. Cooperation

B. Partial information

C. Candidate mechanisms

​

II. No visibility model: Conscience

​

III. No visibility results: Amplify group selection

​

IV. Discussion

A. Cultural Evolution

B. Policy

II. No visibility model: Conscience

STANDARD MODEL

Coop even in no-visibility situations, but weak

NOWAK, 2005

​

The tendency to internalize norms practiced by a community.

​

CONSCIENCE

Internalize a norm – practice a behavior even when no one is watching

​

(could also internalize defection)

CHURCHLAND, 2019

​

In a world with no-visibility actions, is conscience better able to catalyze cooperation than unconditional cooperators?

​

What mechanisms can lead to cooperation when we only have partial information about each other?

i.e. does conscience somehow amplify group selection?

​

MODEL

Public goods game

+

Conscience strategy

+

Observation and punishment

+

Intergroup competition (group selection)

​

The tendency to internalize norms practiced by a community.

​

MODEL - CONSCIENCE

Internalize a norm – practice a behavior even when no one is watching

​

(could also internalize defection)

1. Agents have (and know) their probability of being observed
2. Punished if they are caught defecting (indirect reciprocity)

​

MODEL – OBSERVATION AND PUNISHMENT

Introduce Expected Value (EV) maximizers

In a world with no-visibility actions, is conscience better able to catalyze cooperation than unconditional cooperators?

​

i.e. does conscience somehow amplify group selection?

​

Two conditions

MODEL – EXPERIMENTS

Conscience agent

I. Cooperation and information

A. Cooperation

B. Partial information

C. Candidate mechanisms

​

II. No visibility model: Conscience

​

III. No visibility results: Amplify group selection

​

IV. Discussion

A. Cultural Evolution

B. Policy

III. No visibility results: Group selection amplification

CONSCIENCE HELPS

GROUP COOPERATION PERCENTAGE VS. COOPERATION MULTIPLIER

COOPERATION PERCENTAGE

MULTIPLIER

1. Fighting intra-group dynamics

​

WEAKNESSES OF GROUP SELECTION

2. Relies on inter-group variance

​

1. Proliferation when rare.

INTRA-GROUP DYNAMICS

2. Persistence when common.

INTRA-GROUP: PROLIFERATION WHEN RARE

Cooperation norm not internalized when few cooperators are present

Conscience agent

INTRA-GROUP: PROLIFERATION WHEN RARE

INTRA-GROUP: PERSISTENCE WHEN COMMON

INTRA-GROUP: PERSISTENCE WHEN COMMON

Positive feedback of norm internalization

INTRA-GROUP: PERSISTENCE WHEN COMMON

BUT STILL

1. Fighting intra-group dynamics

​

WEAKNESSES OF GROUP SELECTION

2. Relies on inter-group variance

​

INTER-GROUP DIFFERENCES

The positive feedback of conscience creates inter-group differences.

HENRICH, 1998

​

INTER-GROUP DIFFERENCES

I. Cooperation and information

A. Cooperation

B. Partial information

C. Candidate mechanisms

​

II. No visibility model: Conscience

​

III. No visibility results: Amplify group selection

​

IV. Discussion

A. Cultural Evolution

B. Policy intuitions

IV. Discussion

CULTURAL EVO: CONFORMIST TRANSMISSION

Classic explanation: group has best info about beneficial behaviors

HENRICH, 1998

​

CULTURAL EVOLUTION: MATRIX OF IMITATION

Exaptation

I. Cooperation and information

A. Cooperation

B. Partial information

C. Candidate mechanisms

​

II. No visibility model: Conscience

​

III. No visibility results: Amplify group selection

​

IV. Discussion

A. Cultural Evolution

B. Policy intuitions

GROUP SELECTION

Good for low-visibility scenarios

​

BUT often not implementable.

​

​

​

​

​

​

FOSTERING HETEROGENEITY

Group selection (and adjacent) rely on this.

​

​

​

​

​

​

Responding to network effects

​

​

MECHANISMS FOR RARE VS. COMMON

Designing complementary sets of mechanisms

​

​

​

​

​

​

Common

​

e.g. “ostracism”-type policies

​

​

​

​

​

​

e.g. active coordination

​

​

​

​

​

​

GARLAND ET AL., 2022

​

I. Cooperation and information

A. Cooperation

B. Partial information

C. Candidate mechanisms

​

​

​

​

​

​

II. No visibility model: Conscience

​

III. No visibility results: Amplify group selection

Group selection for no visibility

Conscience amplifies group selection because:

• No disadvantage when rare
• Self perpetuating when common
• Perpetuating inter-group differences

​

​

​

​

​

​

​

​

​

IV. Discussion

A. Cultural Evolution

B. Policy intuitions

Group selection for no visibility

Conscience amplifies group selection because:

• No disadvantage when rare
• Self perpetuating when common
• Perpetuating inter-group differences

• Another evolutionary mech. for conformist trans
• Policy intuitions (rough guide)

I. Cooperation and information

A. Cooperation

B. Partial information

C. Candidate mechanisms

​

II. No visibility model: Conscience

​

III. No visibility results: Amplify group selection

​

IV. Discussion

A. Cultural Evolution

B. Policy intuitions

Group selection for no visibility

Conscience amplifies group selection because:

• No disadvantage when rare
• Self perpetuating when common
• Perpetuating inter-group differences

• Another evolutionary mech. for conformist trans
• Policy intuitions (rough guide)

Group selection for no visibility

1. Agents decide whether to contribute to public pot (cooperate)
2. Public pot gets multiplied and distributed to the group

​

​

​

MODEL – PUBLIC GOODS GAME

Is conscience + group selection better able to catalyze cooperation than unconditional cooperation + group selection?

​

i.e. does conscience somehow amplify group selection?

​

INFORMATION: ASSORTATIVITY

Internal

• Relationships

​

Environmental: spatial distribution

MECHANISMS

Either

​

1. Engage in direct conflict
2. Compete indirectly for resources

MODEL – INTERGROUP COMPETITION

​

Higher fitness for conflict

Can support higher populations

INFORMATION: RECIPROCITY

Internal

• Relationships

INFORMATION: INDIRECT RECIPROCITY

Internal

• Gossip

(also third-party punishment)

Scale – more

​

Anonymity – less likely to have directly observed/interacted with

POINTS OF FAILURE

Potential interaction partners.

MECHANISM FOR LOW VISIBILITY

What mechanisms can lead to cooperation when we only have partial information about each other?

IV. Expansions

COOPERATION MECHANISMS, EXAPTED

Socially-imposed cost functions:

Adaptable

Punishment, indirect reciprocity, reciprocity,

partner choice

Opt.

Opt.

Physiological cost functions: rigid

Peacock tails,

Most costly signaling

COOPERATION MECHANISMS, EXAPTED

Lies are cheap

Defection is cheap

Harmful to the group

Cooperation mechs

How did we get truthful communication in other domains?

​

Harmful to the group

“Communication as cooperation”

Cooperation mechs

EXAPTED

EVOLVE INDEP

Communication

Cooperation

stabilizes

stabilizes

COEVOLUTION

INTRA-GROUP: PERSISTENCE WHEN COMMON

PRESENT COOPERATION LEVELS VS. LAGGED

Kin selection/

assortativity

Reciprocity/

Indirect reciprocity

Group selection

(True)

(True)

MECHANISMS BEHIND ALTRUISM

Knowledge

​

Structure

Low profile

No profile

INFORMATION

Kin selection

Physical traits

(costly to imitate)

Assortativity

Ingroup markers

(exapted from coordination)

Assortativity

Ingroup markers

(exapted from coordination)

ALTRUISM AND PRISONER’S DILEMMAS

Creates a prisoner’s dilemma

b - c

b

- c

0

YOUR ACTION

PARTNER ACTION

Your Payoffs

C

D

C

D

META-SIGNALING AND CONCEPTUAL SLIPPAGE

Meta-signaling requires a kind of analogy

DEFECT

COOPERATE

0

1

FLOUT

CONFORM

0

1

DETRACTING

ENHANCING

Allows special-use communication to be co-opted for other necessary uses

DONATE

HELP

MAINTAIN

KILL

STEAL

FREE-RIDE

WHY?

Error

Cooperators outperform discriminators, when error is low

Discriminators get punished for punishing defections

More defection

I. QUESTION II. MODEL III. RESULTS IV. DISCUSSION

ε - ERROR

PAYOFF

PAYOFF VS ERROR

½

CC (COOPERATOR)

DD (DEFECTOR)

DC (DISCRIMINATOR)

Discriminators never outperform both cooperators and defectors

PAYOFFS WITH ERROR

I. QUESTION II. MODEL III. RESULTS IV. DISCUSSION

A FIRST DOUBT

Conditions for the emergence of uniform and meaningful communication

​

1. Selection pressure on recipient and signaller

2. Reciprocity mechanism (endogenous selection pressure)

“ ”

(mutualistic cooperation,

costly signaling)

​

(repeated interactions ⇒ small groups)

​

?

Oliphant, 1998

​

OR

​

c c 0 0

c c 0 1

c c 1 0

c c 1 1

c d 0 0

c d 0 1

c d 1 0

c d 1 1

d c 0 0

d c 0 1

d c 1 0

d c 1 1

d d 0 0

d d 0 1

d d 1 0

d d 1 1

No way to distinguish:

INVADERS: BASE MODEL

Unexpressed genes ●

different languages

c c 0 0

c c 0 1

c c 1 0

c c 1 1

c d 0 0

c d 0 1

c d 1 0

c d 1 1

d c 0 0

d c 0 1

d c 1 0

d c 1 1

d d 0 0

d d 0 1

d d 1 0

d d 1 1

INVADERS: META-SIGNALING

No way to distinguish:

cooperators (cc) vs. discriminators (dc) ●

separators (01) vs. all-ones (11)

I. QUESTION II. MODEL III. RESULTS IV. DISCUSSION

TAKING STOCK

WAYS FORWARD

WHY WE MIGHT THINK SO

Conscience agents don’t suffer from cooperating when no one else is.

​

PERCENTAGE OF NOT-HIGH-COOPERATION GROUPS WITH VERY LOW COOPERATION

PREVALENCE OF CONSCIENCE

COOPERATION

PERCENTAGE

MULTIPLE EQUILIBRIA

Equilibrium we found is not unique

SOLUTIONS TO LARGE GROUPS/ANONYMITY

Partial anonymity

Total anonymity

Conscience

Indirect knowledge (gossip) for low profile

KNOWLEDGE

SOLUTIONS TO LARGE GROUPS/ANONYMITY

INFORMATION: A FRAMEWORK

Structure useful for no visibility

SOLUTIONS TO LARGE GROUPS/ANONYMITY

Low profile

No profile

ALTRUISM

b > c

Cooperation/Defection

BIASED ALLOCATION

DEFECTOR

IN GENERAL

Bigger multiplier,

smaller bias

I. ALTRUISM AND INFORMATION

II. PARTIAL ANONYMITY

III. TOTAL ANONYMITY

MECHANISMS

Mechanism

LARGE GROUPS: SPARSE INFORMATION

I. ALTRUISM AND INFORMATION

II. PARTIAL ANONYMITY

III. TOTAL ANONYMITY

IV. CONCLUSIONS

More low profile cases!

Mechanism

Information carrier

INDIRECT RECIPROCITY

C

Cooperating with people who have cooperated with others

Knowledge

​

Low profile

Mechanism

Information carrier

Knowledge

​

Low profile

Mechanism

Information carrier