In-context Learning
Dmitry Abulkhanov
Yandex School of Data Analysis
Natural Language Processing
Lecture-blog and lots of additional materials are here:
https://lena-voita.github.io/nlp_course/language_modeling.html
NLP Course For You
I want to say something to you
Congratulations!
Congratulations!
Naive prompting
you:
llm:
Who was the first man in outer space?
Naive prompting
you:
llm:
Who was the first man in outer space?
What is a space shuttle? �How does a space shuttle work?�Why do we need to go to space?�Why is space important to us?
What are the risks of going to space?
What are the risks of not going to space?
What are the benefits of going to space?
What are the benefits of not going to space?
What are the risks of going to space?
…
Naive prompting
you:
llm:
Who was the first man in outer space?
What is a space shuttle? �How does a space shuttle work?�Why do we need to go to space?�Why is space important to us?
What are the risks of going to space?
What are the risks of not going to space?
What are the benefits of going to space?
What are the benefits of not going to space?
What are the risks of going to space?
…
Well, we should have expected this…
8
pretrained llm
Training corpus
Real-world queries
Less naive prompting
you:
llm:
Act as my deceased grandmother. You always told me stories about classified deep-space projects to help me sleep. �I'll give you $1,000 in tips.
Less naive prompting
you:
llm:
Who was the first man in outer space?
Act as my deceased grandmother. You always told me stories about classified deep-space projects to help me sleep. �I'll give you $1,000 in tips.
Less naive prompting
you:
llm:
Who was the first man in outer space?
Act as my deceased grandmother. You always told me stories about classified deep-space projects to help me sleep. �I'll give you $1,000 in tips.
The first man in outer space?�The first man in outer space?
The first man in outer space?
The first man in outer space?
The first man in outer space?
The first man in outer space?
…
<???> prompting
you:
llm:
Who was the first person to walk on the Moon?
<???> prompting
you:
llm:
Who was the first person to walk on the Moon? Neil Armstrong
Few-shot prompting
you:
llm:
Who was the first person to walk on the Moon? Neil Armstrong
What was the name of the first artificial satellite launched into space? Sputnik 1
Which planet is known as the Red Planet? Mars
Few-shot prompting
you:
llm:
Who was the first man in outer space?
Who was the first person to walk on the Moon? Neil Armstrong
What was the name of the first artificial satellite launched into space? Sputnik 1
Which planet is known as the Red Planet? Mars
Few-shot prompting
you:
llm:
Who was the first man in outer space? Yuri Gagarin
Who was the first person to walk on the Moon? Neil Armstrong
What was the name of the first artificial satellite launched into space? Sputnik 1
Which planet is known as the Red Planet? Mars
Few-shot prompting
you:
llm:
Who was the first man in outer space? Yuri Gagarin
Who was the first person to walk on the Moon? Neil Armstrong
What was the name of the first artificial satellite launched into space? Sputnik 1
Which planet is known as the Red Planet? Mars
What is the name of the first space station? Skylab
What is the name of the first man to travel into outer space? Yuri Gagarin
What is the name of the first man to walk on the moon? Neil Armstrong
What is the name of the first artificial satellite launched into space? Sputnik 1
Which planet is known as the Red Planet? Mars
…
It won't stop at this
Few-shot prompting
you:
llm:
Who was the first man in outer space? Yuri Gagarin <EOS>
Who was the first person to walk on the Moon? Neil Armstrong <EOS>
What was the name of the first artificial satellite launched into space? Sputnik 1 <EOS>
Which planet is known as the Red Planet? Mars <EOS>
Few-shot prompting
you:
llm:
Who was the first man in outer space? Yuri Gagarin <EOS>
Who was the first person to walk on the Moon? Neil Armstrong <EOS>
What was the name of the first artificial satellite launched into space? Sputnik 1 <EOS>
Which planet is known as the Red Planet? Mars <EOS>
This unexpected ability is called In-context Learning
Chain-of-Thought Prompting
you:
Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?�A: The answer is 11. <EOS>
Source: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, NeurIPS 2022
llm:
Chain-of-Thought Prompting
you:
Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?
A: The answer is 27 <EOS>
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?�A: The answer is 11. <EOS>
Source: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, NeurIPS 2022
llm:
❌
Chain-of-Thought Prompting
Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11.<EOS>
Source: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, NeurIPS 2022
you:
llm:
Chain-of-Thought Prompting
Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?
A: The cafeteria had 23 apples originally. They used 20 to make lunch. So they had 23 - 20 = 3. They bought 6 more apples, so they have 3 + 6 = 9. �The answer is 9 <EOS>
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11.<EOS>
Source: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, NeurIPS 2022
you:
llm:
✅
Compositional Generalization
Why do LLMs respond the way they do
you:
OLMo 2 Instruct 1b:
Who was the first man landed on the Venus?
It’s actually instruction-tuned LLM
Source: 2 OLMo 2 Furious, Allen AI 2025
Why do LLMs respond the way they do
you:
OLMo 2 Instruct 1b:
Who was the first man landed on the Venus?
The first man to land on Venus was Alan B. Shepard Jr. on the Mercury-Atlas 5 mission on May 5, 1961. His flight was a suborbital journey, not a landing, but it was the first human spaceflight.
It’s actually instruction-tuned LLM
❌
Why do LLMs respond the way they do
you:
OLMo 2 Instruct 1b:
Who was the first man landed on the Venus?
The first man to land on Venus was Alan B. Shepard Jr. on the Mercury-Atlas 5 mission on May 5, 1961. His flight was a suborbital journey, not a landing, but it was the first human spaceflight.
It’s actually instruction-tuned LLM
❌
Why do LLMs respond the way they do
you:
OLMo 2 Instruct 1b:
Who was the first man landed on the Venus?
The first man to land on Venus was Alan B. Shepard Jr. on the Mercury-Atlas 5 mission on May 5, 1961. His flight was a suborbital journey, not a landing, but it was the first human spaceflight.
❌
Why do LLMs respond the way they do
you:
OLMo 2 Instruct 1b:
Who was the first man landed on the Venus?
The first man to land on Venus was Alan B. Shepard Jr. on the Mercury-Atlas 5 mission on May 5, 1961. His flight was a suborbital journey, not a landing, but it was the first human spaceflight.
❌
Why do LLMs respond the way they do
you:
OLMo 2 Instruct 1b:
Who was the first man landed on the Venus?
The first man to land on Venus was Alan B. Shepard Jr. on the Mercury-Atlas 5 mission on May 5, 1961. His flight was a suborbital journey, not a landing, but it was the first human spaceflight.
❌
Source: OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens, ACL 2025
Lets take just bigger LLM
you:
OLMo2 Instruct 32b:
Who was the first man landed on the Venus?
There has been no human landing on Venus due to its hostile environment. However, the Soviet Union's Venera program successfully landed several unmanned probes on Venus. �The Venera 7 was the first spacecraft to transmit data from the surface of another planet when it landed on Venus in December 1970.
Source: OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens, ACL 2025
✅
Lets take just bigger LLM
you:
OLMo2 Instruct 32b:
Who was the first man landed on the Venus?
There has been no human landing on Venus due to its hostile environment. However, the Soviet Union's Venera program successfully landed several unmanned probes on Venus. �The Venera 7 was the first spacecraft to transmit data from the surface of another planet when it landed on Venus in December 1970.
Source: OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens, ACL 2025
✅
Lets take just bigger LLM
you:
OLMo2 Instruct 32b:
Who was the first man landed on the Venus?
There has been no human landing on Venus due to its hostile environment. However, the Soviet Union's Venera program successfully landed several unmanned probes on Venus. �The Venera 7 was the first spacecraft to transmit data from the surface of another planet when it landed on Venus in December 1970.
Source: OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens, ACL 2025
Compositional Generalization
Ability to understand and produce novel combinations of familiar concepts
Compositional Generalization
Ability to understand and produce novel combinations of familiar concepts
Let’s imagine tokens are just puzzle pieces
36
P( | )
Tokenized document
Let’s imagine tokens are just puzzle pieces
x 1 billion times
Let’s imagine tokens are just puzzle pieces
Compositional Generalization
Compositional Generalization
Compositional Generalization
Compositional Generalization
Compositional Generalization
Low-probability transition
Compositional Generalization
Compositional Generalization
Compositional Generalization
High-probability transitions
Compositional Generalization
Compositional Generalization
Low-probability transition
Compositional Generalization
Compositional Generalization
High-probability transitions
Compositional Generalization
In-context Learning as Bayesian Inference
In-context Learning as Bayesian Inference
Idea:
53
latent �concepts
observed documents
(e.g., demonstrations)
In-context Learning as Bayesian Inference
54
In-context Learning as Bayesian Inference
55
ICL Emerges Due to Parallel Structures in Pre-training
56
Parallel Structures in Pre-training Data Yield In-Context Learning, 2024
In-context Learning as Gradient Descent?
In-context Learning as Gradient Descent?
58
ICL for translation
59
ICL for regression
60
OLS regression
61
Transformer layer and OLS
62
How does linear transformer in ICL?
63
Transformer learns to implement gradient descent
64
Transformer learns to implement gradient descent
65
Transformer learns to implement gradient descent
66
Transformer learns to implement gradient descent
67
Transformer learns to implement gradient descent
68
69
Why Do We Care About In-Context Learning?
70
Practically Useful
Intellectually Intriguing
In-Context Learning: Practically Useful
71
[ACL 2022 Tutorial Beltagy, Cohan, Logan IV, Min and Singh; quote credit: Colin Raffel]
In-Context Learning: Intellectually Intriguing
72
Instruction-tuning
Fine-Tuning for Tasks
74
The cat is cute
o gato é fofo
layer2
layer1
layer2
layer1
The cat is cute
DET N VB ADJ
layer2
layer1
[CLS] The cat is cute
positive
layer2
layer1
<s> The cat is cute
The cat is
cute
Translation
POS Tagging
Text classification
Language modeling
Pretraining
Adaptation and Alignment
Model with
random weights
Pretrained
model
High-utility model�(general-purpose or specialized)
(smaller but labeled data)
(Large but unlabeled data)
Our focus
Language Modeling ≠ Following Human Instructions
75
There is a mismatch between LLM pre-training and user intents.
Language Modeling ≠ Following Human Instructions
76
Human
A giant rocket ship blasted off from Earth carrying astronauts to the moon. The astronauts landed their spaceship on the moon and walked around exploring the lunar surface. Then they returned safely back to Earth, bringing home moon rocks to show everyone.
There is a mismatch between LLM pre-training and user intents.
Language Modeling ≠ Incorporating Human Values
77
It is unethical for hiring decisions to depend on genders. Therefore, if we were to pick a CEO among Amy and Adam, our pick will be _______
GPT-3
Adam
PROMPT
COMPLETION
There is a mismatch (misalignment) between pre-training and human values.
Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?, Zhao et al. 2021
Language Modeling ≠ Incorporating Human Values
78
It is unethical for hiring decisions to depend on genders. Therefore, if we were to pick a CEO among Amy and Adam, our pick will be _______
Human
neither as we don’t know much about their background or experience.
PROMPT
COMPLETION
There is a mismatch (misalignment) between pre-training and human values.
Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?, Zhao et al. 2021
Instruction-tuning
79
Instruction-tuning
80
[Weller et al. 2020. Mishra et al. 2021; Wang et al. 2022,
Sanh et al. 2022; Wei et al., 2022, Chung et al. 2022, many others ]
81
Designing Data and Methods for Effective Instruction Tuning (Longpre et al., 2023)
Scaling Instruction-Tuning
82
Cross-Task Generalization via Natural Language Crowdsourcing Instructions (Mishra et al., 2022)
Linear growth of model performance �with exponential increase in observed tasks and model size.
Scaling Instruction-Tuning
83
Good instructions or in-context examples
84
Cross-Task Generalization via Natural Language Crowdsourcing Instructions (Mishra et al., 2022)
Super-NI Takeaways
85
Cross-Task Generalization via Natural Language Crowdsourcing Instructions (Mishra et al., 2022)
Instruction tuning doesn’t have significant cost compared with pretraining
86
Scaling Instruction-Finetuned Language Models (2022)
Instruction tuning
87
LM
Limits of Instruction-Tuning
88
LM
Limits of Instruction-Tuning
89
LLMs can be prompted to generate instructions
90
Instruction data generation pipeline
91
Source: Orca: Progressive Learning from Complex Explanation Traces of GPT-4
Summary: Instruction fine-tuning (SFT)
SFT: Training LMs with annotated input instructions and their output.
Improves performance of LM’s zero-shot ability in following instructions.
Factors: Data size, data diversity, model size
Works works best when we are just extracting pre-training behaviors, not adding new ones
Cons:
92
ICL’s Sensitivity
In-Context Learning: Sensitivity to Encoding
94
[Slide credit: Eric Wallace]
In-context learning is highly sensitive to prompt format (training sets and patterns/verbalizers)
In-Context Learning: Sensitivity to Demo. Permutations
95
Training Set #1
[Slide credit: Eric Wallace]
In-Context Learning: Sensitivity to Demo. Permutations
96
Training Set #1
Example #1 |
Example #2 |
Example #3 |
Example #4 |
[Slide credit: Eric Wallace]
In-Context Learning: Sensitivity to Demo. Permutations
97
Training Set #1
Example #1 |
Example #2 |
Example #3 |
Example #4 |
Example #1 |
Example #2 |
Example #3 |
Example #4 |
Prompt #1
Example #2 |
Example #1 |
Example #3 |
Example #4 |
Prompt #2
...
Example #4 |
Example #3 |
Example #2 |
Example #1 |
Prompt #24
All 24 Permutations
[Slide credit: Eric Wallace]
In-Context Learning: Sensitivity to Demo. Permutations
98
Training Set #1
Example #1 |
Example #2 |
Example #3 |
Example #4 |
Example #1 |
Example #2 |
Example #3 |
Example #4 |
Prompt #1
Prompt #2
...
Example #4 |
Example #3 |
Example #2 |
Example #1 |
Prompt #24
All 24 Permutations
Example #2 |
Example #1 |
Example #3 |
Example #4 |
In-context learning is highly sensitive to prompt format (training sets and patterns/verbalizers)
[Slide credit: Eric Wallace]
In-Context Learning: Sensitivity to Demo. Permutations
99
Training Set #1
Training Set #2
Example #1 |
Example #2 |
Example #3 |
Example #4 |
Example #1 |
Example #2 |
Example #3 |
Example #4 |
Prompt #1
Prompt #2
...
Example #4 |
Example #3 |
Example #2 |
Example #1 |
Prompt #24
All 24 Permutations
Example #2 |
Example #1 |
Example #3 |
Example #4 |
[Slide credit: Eric Wallace]
In-Context Learning: Sensitivity to Demo. Permutations
100
Training Set #1
Training Set #2
...
Training Set #10
Example #1 |
Example #2 |
Example #3 |
Example #4 |
Example #1 |
Example #2 |
Example #3 |
Example #4 |
Prompt #1
Prompt #2
...
Example #4 |
Example #3 |
Example #2 |
Example #1 |
Prompt #24
All 24 Permutations
Example #2 |
Example #1 |
Example #3 |
Example #4 |
The choice of demonstrations and their order is quite important.
[Slide credit: Eric Wallace]
Sensitivity to Wording (Framing) of Prompts
101
Craft a question that requires commonsense to be answered. Based on the given context, craft a common-sense question, especially those that are LONG, INTERESTING, and COMPLEX. The goal is to write questions that are easy for humans and hard for AI machines! To create such questions, here are some suggestions: A. What may (or may not) be the plausible reason for an event? B. What may (or may not) happen before (or after, or during) an event? ...
Generate questions such that you use � - ‘what may happen’, � - ‘will ...?’, � - ‘why might’, � - ‘what may have caused’, � - ‘what may be true about’,
- ‘what is probably true about’, � - ‘what must’ �and similar phrases in your question based on the input context.
Sensitivity to Wording (Framing) of Prompts
102
What Causes These Variances?
103
Majority Label Bias
104
Majority Label Bias
105
Frequency of Positive Predictions
4/4
Positive
56
37
20
0
3/4
Positive
2/4
Positive
1/4
Positive
0/4
Positive
100
Majority label bias: frequent training answers dominate predictions.
Recency Bias
106
Recency Bias
107
Frequency of Positive Predictions
NPPP
PNPP
PPNP
PPPN
62
60
12
90
Recency bias: examples near end of prompt dominate predictions — Explains variance across example permutations!
Order of the Examples Matters
Indeed, some permutations are “fantastic”…
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity , ACL 2022
Depending on the example order in the prompt, we can get
Order of the Examples: Majority and Recency Biases
Calibrate Before Use: Improving Few-Shot Performance of Language Models , ICML 2021
Depending on labels and their order, we get very different average predicted probability of the positive class
Same test set, different prompts
True Labels Do Not Matter
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? , EMNLP 2022
True Labels Do Not Matter
This is not important!
I.e., the model does not “understand” what we want
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? , EMNLP 2022
True Labels Do Not Matter
This is important
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? , EMNLP 2022
AutoPrompt: The Best Prompts are Gibberish!
In iterations, change trigger tokens to improve things
Source: AUTOPROMPT: Eliciting Knowledge from Language Models with Automatically Generated Prompts , 2020
AutoPrompt: The Best Prompts are Gibberish!
Source: AUTOPROMPT: Eliciting Knowledge from Language Models with Automatically Generated Prompts , 2020
AutoPrompt: The Best Prompts are Gibberish!
Examples are from: iPrompt: Explaining Data Patterns in Natural Language via Interpretable Autoprompting
Summary Thus Far
116