1

Outline

2

Outline

3

Outline

4

What we will learn

• The mathematical challenges of next word prediction

​

• The basic mathematical ideas behind Transformers

​

• How Transformers predict words and sentences

​

• To get comfortable at a high-level with how LLMs work at a mathematical level

​

• To lose your fear of the science of LLMs

5

What we will not learn

• How to train foundational models

​

• How to build transformer-based chatbots

​

• How to prompt

​

• Agentic AI

​

• How to make $$$Millions from AI

6

What I assume you know

• Basic probability, e.g. conditional probability

​

• Basic linear algebra, e.g. matrix multiplication as a transformation

​

• Basic machine learning, e.g. training a model by minimizing a loss function

7

Predicting the next word

What this enables, tokens and sequences

8

Predicting the next word

• What are we trying to achieve?

​

• The cat sat on the ….

​

9

Predicting the next word

• What are we trying to achieve?

​

• The cat sat on the mat

​

10

Predicting the next word

• What are we trying to achieve?

​

• The quick brown fox jumped over the …….

​

11

Predicting the next word

• What are we trying to achieve?

​

• The quick brown fox jumped over the lazy

​

• The quick brown fox jumped over the lazy …..

​

​

12

Predicting the next word

• What are we trying to achieve?

​

• The quick brown fox jumped over the lazy

​

• The quick brown fox jumped over the lazy dog

​

​

13

Predicting the next word

14

Predicting the next word

• What are we trying to achieve?

​

“I wandered lonely as a cloud that floats on high o'er vales and hills,

when all at once …….”

​

→ “I wandered lonely as a cloud that floats on high o'er vales and hills, when all at once I saw a crowd, a host of golden daffodils.”

​

15

William Wordsworth, 1807

Words as tokens

• Words are the sentence natural units of a sentence.

​

• We breaks down a sentence into these units:

the cat sat on the mat → [the, cat, sat, on, the mat]

​

• We call these units “tokens”

​

• Tokens don’t have to be complete words:

quick brown fox jumped over → [quick, brown, fox, jump, ed, over]

​

​

16

Sequences

• A natural language sentence is just a sequence of tokens

​

• Our modelling challenge is “predict the next token in given the preceding sequence of tokens”

​

• We find sequences of tokens everywhere, not just in language

​

• I will almost always be referring to words in a sentence

17

Simple approaches

Naïve models that are wrong and what we can learn from that

18

Simple approaches: Markov models

Example: “The more you know, the more you realize you don't know” Aristotle

19

From

To

Simple approaches: Markov models

Example: “The more you know, the more you realize you don't know” Aristotle

20

The more

Simple approaches: Markov models

Example: “The more you know, the more you realize you don't know” Aristotle

21

The more you

Simple approaches: Markov models

Example: “The more you know, the more you realize you don't know” Aristotle

22

The more you know

Simple approaches: Markov models

Example: “The more you know, the more you realize you don't know” Aristotle

23

The more you know the

Simple approaches: Markov models

Example: “The more you know, the more you realize you don't know” Aristotle

24

The more you know the more

Simple approaches: Markov models

Example: “The more you know, the more you realize you don't know” Aristotle

25

The more you know the more you

Simple approaches: Markov models

Example: “The more you know, the more you realize you don't know” Aristotle

26

The more you know the more you realize

Simple approaches: Markov models

Example: “The more you know, the more you realize you don't know” Aristotle

27

The more you know the more you realize you

Simple approaches: Markov models

Example: “The more you know, the more you realize you don't know” Aristotle

28

The more you know the more you realize you don’t

Simple approaches: Markov models

Example: “The more you know, the more you realize you don't know” Aristotle

29

The more you know the more you realize you don’t know

Simple approaches: Markov models

• Each separate word corresponds to a column in the transition matrix:

“know” = ( 0,0,0,1,0,0)

​

• Small fixed vocabulary ⇒ Moving six-dimensional discrete state-space

​

• Model is probabilistic:

P(“know” | “you”) = 0.3 , P(“realize” | “you”) = 0.3 , P(“don’t” | “you”) = 0.3

​

Could easily have generated,

‘The more you don’t the more you realize you don’t realize’

​

30

Simple approaches: Markov models

Problems:

1. Our vocabulary is large, e.g. 170000 – 700000 words

• Large transition matrix

​

2a. We only take into account the preceding word

→ “Many adults drink beer” , “Many newborn babies drink beer”

“beer” has the same probability of occurring after the word “drink” in this model

​

2b. Expanding to transitions between pairs of words increases state space

e.g. (many, adults) → (adults, drink) → (drink, beer)

31

Word embeddings

Correcting our mistakes

32

Word embeddings

• Reduce to a fixed lower dimensionality

33

Word embeddings

• Reduce to a fixed lower dimensionality

34

Word embeddings

35

Word2Vec model = “fasttext-wiki-news-subwords-300”

Code example

Working with word embeddings

36

Transformers

Creating context aware embeddings

37

Transforming embeddings

38

Input vector from preceding token into multi-class classifier

• Each token knows nothing about the broader context of the preceding words

​

• Need new vectors for each token that take into account the preceding context

​

• We will make a Transformer

​

• For now, create new vectors that take into account the context of the whole sentence

Attention

39

Input embeddings

Updated embeddings

Self-Attention

40

Self-Attention

41

Self-Attention

42

• We can add more flexibility to model.
• Different linear mappings to create the query vectors and to create the key vectors

Self-Attention

43

Attention: Masking

44

Attention Head

45

Attention Head

Input embedding vector for token

Output embedding vector for token

Multi-headed Self-Attention

46

• We solve this by having multiple (H) self-attention heads.
• We combine the output vectors of all the attention heads into a single vector
• Typically, we make output dimension of each attention head d/H

Neural Network Sub-Layer

47

Input embeddings

Output embeddings

• Attention process is column-wise (token-to-token)

​

• We haven’t allowed the different rows within a column to interact when producing the updated embedding

Neural Network Sub-Layer

48

Embedding vector

Neural Network

Output Embedding vector

Parameters of neural network are learnt as part of the overall model training

The Transformer Block

49

Input embeddings

Output embeddings

Transformer Block

Technical transformations

• We apply two main additional transformations to the data to improve efficiency and stability of the training process:

​

• Layer-wise normalization – apply a standardization (location and scale) transformation before each sub-layer

​

​

• We use the multi-headed attention and neural network transformations to model the residuals of the input-output relations

​

Output vector = Input vector + Transformation(Input vector)

50

The Transformer Block

51

Input embeddings

Output embeddings

The Transformer Model

52

Input embeddings

Output embeddings

Back to predicting the next word

Back to our goal

53

Predicting the next word from context embeddings

54

Input vector from preceding token into multi-class classifier

Training

55

Predicting the next word from context embeddings

56

Transformer Model

Positional encoding

57

• Let’s look at our formula for the attention probabilities

The input embeddings, and hence the context embeddings don’t take into account the position of the token in the context (other than through masking) !!

Positional encoding

58

• We solve by adding position dependent vectors to the input

+

=

Positional encoding

59

• We solve by adding position dependent vectors to the input

+

=

Positional encoding

60

• We solve by adding position dependent vectors to the input

+

=

Positional encoding

61

Positional encoding

62

• Putting it all together (finally!)

Input embeddings

Output embeddings

Position encodings

Different types of transformers

Encoders, Decoders, Cross-attention

63

Different types of transformers

• What we covered was embedding vector → token (word)

​

​

​

​

• We ‘decoded’ the embedding vector

​

• ChatGPT, Claude, etc are known as ‘decoder-only’ models

​

​

64

“mat”

Different types of transformers

• We also have encoders: token → context embedding vector

​

65

“mat”

• Encoders are useful where we have all the tokens (no prediction), we just need to map them to appropriate vectors

​

• E.g. “the cat is on the table” → “le chat est sur la table”

​

​

​

Different types of transformers

66

Different types of transformers

67

Encoder model

Different types of transformers

68

Decoder model

Different types of transformers

69

Code example

Using PyTorch to build transformer layers

70

Wrap up

71