Prompt Engineering
John Berryman
Hi! I'm John Berryman
career
3
career
2
career
1
career
4
What is a Language Model?
How has this taking the world by storm?
What is a Large Language Model?
It's the same thing, just a lot more accurate.
Our model, called GPT-2 (a successor to GPT), was trained simply to predict the next word in 40GB of Internet text. Due to our concerns about malicious applications of the technology, we are not releasing the trained model. (ref)
What is a Large Language Model
(These are all from the Feb 2019 GPT-2 release article.)
What is a Large Language Model
(These are all from the Feb 2019 GPT-2 release article.)
Our model, called GPT-2 (a successor to GPT), was trained simply to predict the next word in 40GB of Internet text. Due to our concerns about malicious applications of the technology, we are not releasing the trained model. (ref)
Our model, called GPT-2 (a successor to GPT), was trained simply to predict the next word in 40GB of Internet text.
…And we figured out that now you can just ask it to do stuff and it will!
IT'S AMAZING
(But also it will help you make bombs, and drugs, and overthrow governments. So…)
Due to our concerns about malicious applications of the technology, we are not releasing the trained model.
Prompt Crafting
> How are you doing today?
< ¿Cómo estás hoy?�
> My name is John.�< Mi nombre es John.
�> Can I have fries with that?�< ¿Puedo tener papas fritas con eso?
technique #1: few-shot prompting
examples to set the pattern
the actual task
Prompt Crafting
Q: It takes one baker an hour to make a cake. How long does it take 3 bakers to make 3 cakes?
A: 3
technique #2: chain-of-thought reasoning
Prompt Crafting
Q: Jim is twice as old as Steve. Jim is 12 years how old is Steve.
A: In equation form: 12=2*a where a is Steve's age. Dividing both sides by 2 we see that a=6. Steve is 6 years old.
Q: It takes one baker an hour to make a cake. How long does it take 3 bakers to make 3 cakes?
A: The amount of time it takes to bake a cake is the same regardless of how many cakes are made and how many people work on them. Therefore the answer is still 1 hour.
technique #2: chain-of-thought reasoning
Prompt Crafting
technique #2: chain-of-thought reasoning
Q: It takes one baker an hour to make a cake. How long does it take 3 bakers to make 3 cakes?
A: Let's think step-by-step. The amount of time it takes to bake a cake is the same regardless of how many cakes are made and how many people work on them. Therefore the answer is still 1 hour.
Prompt Crafting
technique #3: document mimicry
# IT Support Assistant
The following is a transcript between an award winning IT support rep and a customer.
## Customer:
My cable is out! And I'm going to miss the Superbowl!
## Support Assistant:
What if you found this scrap of paper on the ground?
What do you think the rest of the paper would say?
Prompt Crafting
technique #3: document mimicry
# IT Support Assistant
The following is a transcript between an award winning IT support rep and a customer.
## Customer:
My cable is out! And I'm going to miss the Superbowl!
## Support Assistant:
Let's figure out how to diagnose your problem…
It uses markdown to establish structure
Document type is transcript
It tells a story to condition a particular response.
Prompt Crafting Intuition: LLMs are Dumb Mechanical Humans.
Building LLM Applications
The hard part!
Creating the Prompt
Creating the Prompt: Copilot Code Completion
// pkg/skills/search.go
// <consider this snippet from ../skill.go>
// type Skill interface {
// Execute(data []byte) (refs, error)
// }
// </end snippet>
package searchskill
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
)
type Skill struct {
█
}
type params struct {
file path
snippet from open tab
current document
cursor
The Introduction of Chat
# IT Support Assistant
The following is a transcript between an award winning IT support rep and a customer.
## Customer:
My cable is out! And I'm going to miss the Superbowl!
## Support Assistant:
Let's figure out how to diagnose your problem…
document
messages =
[{
"role": "system"
"content": "You are an award winning support staff representative that helps customers."
},
{"role": "user",
"content":"My cable is out! And I'm going to miss the Superbowl!"
}
]
API
benefits
<|im_start|> system
You are an award winning IT support rep. Help the user with their request.<|im_stop|>
<|im_start|> user
My cable is out! And I'm going to miss the Superbowl!<|im_stop|>
<|im_start|> assistant
Let's figure out how to diagnose your problem…
The Introduction of Tools
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state",
},
"unit": {
"type": "string",
"description": "degrees Fahrenheit� or Celsius"
"enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
},
}
Input:
{"role": "user",
"content": "What's the weather
like in Miami?"}
Function Call:
{"role": "assistant",
"function": {
"name": "get_weather",
"arguments": '{
"location": "Miami, FL"
}'}
Function Response:
{ "role": "tool",
"name": "get_weather",
"content": "78ºF"}
Assistant Response:
{"role": "assistant",
"content": "It's a balmy 78ºF"}
Building LLM Applications
Building LLM Applications:
Bag of Tools Agent
functions:
user: make it 2 degrees warmer in here
assistant: getTemp()
function: 70ºF
assistant: setTemp(72)
function: success
assistant: Done!
user: actually… put it back
assistant: setTemp(70)
function: success
assistant: Done again, you fickle pickle!
Creating the Prompt: Copilot Chat
Tips for Defining Tools
Questions?
P.S. I'm also available for LLM application consulting at jfberryman@gmail.com
Questions?