1 of 1

Result 4: Attention Patching shows evidence for each mechanism

We patch the first layer attention with 1) uniform attention 2) last token only.�This experiment show each mechanism explicitly.

Fig. 4-1: 1^st and 2^nd layer attention before the transition.

Fig. 4-2: 1^st and 2^nd layer attention after the transition.

Fig. 4-3: Uniform Attention patching.

Fig. 4-4: Last Token Only Attention patching.

Emergence of In-Context Learning�Beyond Bayesian retrieval: A mechanistic study

Core Francisco Park^{1, 2, 3}, Ekdeep Singh Lubana^{2, 3, 4}, Hidenori Tanaka^{2, 3}

1: Department of Physics, Harvard University, 2: CBS, Harvard University, 3: NTT Research, 4: University of Michigan

Result 1: Data Diversity and Compute are required for beyond Bayesian ICL�Models trained on enough data diversity (n>256 for k=10) and enough compute (>2k steps for d_embed=64) shows emergence of In-Context Learning beyond Bayesian retrieval of tasks within the training set.

Fig. 1-1: Next token KL(model|GT) depending on the data diversity of the train set (10¹⁵ FLOPs)

Fig. 1-2: Model implements the unseen chain and forgets the train set after n>256

Fig. 1-3: Next token KL(model|GT) depending on the number of steps (n=512, batch=128)

Poster

Result 5: MLP layers carry memory of the training distribution

We show evidence that MLP layers affect memorization by training attention only transformers and restricting the MLP hidden dimension

Fig. 5-1: KL for attention only transformers

Fig. 5-2: KL for different MLP hidden dimensions, smaller hidden dimension suppresses memorization.

Result 2: Test performance get worse with more context for the Bayesian phase

Past work (Lin 2024, Pan 2023) reported “early ascent” a few shot regime where more samples harm performance. We observe this phenomenon before the transition, and provide a simple intuition.

Fig. 2-1: KL of model and different solutions with varying context, n=64.

Fig. 2-2: Min. distance to a set of random vectors v.s. the distance to the distribution mean vector.

Result 3: The Bayesian phase implements a mixture of unigram and bigram likelihood

As hinted from 2, the Bayesian phase is a mixture of a unigram and bigram likelihood. We quantify the similarity of the inferred transition probabilities from these two methods and visualize the training dynamics in barycentric coordinates.

Fig. 3-1: Training dynamics of model strategy with compute for n=16

Fig. 3-2: Training dynamics of model strategy with compute for n=512

Setup: Probabilistic Mixture of n Markov chains with 10 states (k=10)

5-1

5-2

4-3

4-4