Trikuti of LLMs
Pratyush Kumar, Sarvam AI��GenAI Meetup BLR, 29th July 2023
Three broad themes of the talk
The trikuti (the confluence of the three)
Pre-training
Fine-tuning
Prompting
LLM performance
The Full Stack of AI
Hot take: High-value work in AI will require building the full stack and hence require a foundational grasp
It begins with Fine-tuning
Fine-tuning
Given <x, y> tuples, �learn a neural network as an approximator for f(x) = y
It begins with Fine-tuning
Fine-tuning
Given <x, y> tuples, �learn a neural network as an approximator for f(x) = y
x = series of health indicators�y = efficacy of a treatment
It begins with Fine-tuning
Fine-tuning
Given <x, y> tuples, �learn a neural network as an approximator for f(x) = y
x = review of an Amazon product�y = sentiment
It begins with Fine-tuning
Fine-tuning
Given <x, y> tuples, �learn a neural network as an approximator for f(x) = y
x = sentence in Hindi�y = translation in English
It begins with Fine-tuning
Fine-tuning
Given <x, y> tuples, �learn a neural network as an approximator for f(x) = y
x = 20 page judgement�y = 1 page summary
It begins with Fine-tuning
Fine-tuning
Given <x, y> tuples, �learn a neural network as an approximator for f(x) = y
x = philosophy major question�y = sound answer
It begins with Fine-tuning
Fine-tuning
Given <x, y> tuples, �learn a neural network as an approximator for f(x) = y
x = topic�y = write a sonnet
Why? It is a hard problem
Hardness
How would you define the hardness of an optimization problem?
y = ax + b
y = ax2 + bx + c
One way to look at hardness is intrinsic dimensionality (ID) �or number of of free variables* in the optimizer
Then for function approximator of the following data the ID is ‘huge’
x = philosophy major question�y = sound answer
*minimal number of variables needed to be modified to find close to optimal solutions
Pre-training to the rescue
Pre-training
Train a model to predict the next token (sub-word) in a sequence of tokens
x = 1, 2, … n tokens�y = predict (n+1)th token
Pre-training to the rescue
Pre-training
Pre-train a neural network N on trillions of tokens
Fine-tuning
If we use pretrained N, then the task of fitting N to given <x, y> tuples has significantly lower intrinsic dimensionality
Pretraining + finetuning
Pre-training
Fine-tuning
With a pretrained neural network, we can now fine-tune for various <x, y> such as
x = 20 page judgement�y = 1 page summary
Pretraining + finetuning
Pre-training
Fine-tuning
With a pretrained neural network, we can now fine-tune for various <x, y> such as
x = topic�y = write a sonnet
An analogy
Pre-training
Fine-tuning
Fitting for an environment -> @ Evolution!
Teaching by instruction -> @ School!
Pretraining works only at billions of params?
Pre-training
Fitting for an environment -> @ Evolution!
Depends on what is the environment we are fitting for
Shows that models with tens of millions of params (not billions) can be pretrained to generate tiny stories
Shows that models trained on billions of tokens (not trillions) can effectively answer programming questions
Pretraining dataset is a design choice!
Pre-training
Fitting for an environment -> @ Evolution!
Opportunity to generate diverse, high quality, domain-specific subsets of pretraining data
Getting work done has another leg
Pre-training
Fine-tuning
Prompting
Even after evolutionary optimization, careful education at school, still we may not get the desired performance at the workplace !
Eliciting intelligence
Pretraining may have reduced to (close to) zero the intrinsic dimensionality of many many tasks
We have finetuned a pretrained model for several <x, y> pairs. �� - Are there other types of x where the model is already good?
- Are there specific ways of writing x so that I get the write y?
Important to appreciate that this is a highly unusual problem in machine learning
Shows that we can learn a task specific description that gives the best performance for that task
Shows that we can learn the entire prompt and that’s competitive at scale to finetuning
Prompt design / Prompt optimization
Pre-training
Fine-tuning
Prompting
The prompt can be learnt or optimized, and is definitely in the design space of getting good performance
So, what can we do in India, for India?
Time
A tech shift occurs
Quality
North American use-case
Indian use-case
Case in point:
Older gen AI such as translation and speech recognition
Our mission at
Bring parity in Indian language AI wrt English with data, models, and tools, �all in the open-source
About 2 years into the mission, we got to a reasonable state in a few axes
Translation
Speech recognition
But with LLMs, we decided …
that we need a lot more capital�we need strategic companies in India that can do full-stack AI
So, Sarvam is born, co-founded by me and Vivek Raghavan
We have a strong co-founding ML team of 14 people
Mission statement: Just as Jio changed the landscape in data consumption in the last decade, this decade we should do it with AI
Happy to chat offline and tell you more about what we are doing