Decoding the Giants

Unveiling the Mysteries of Large Language Models (LLM)

Anuj Dutt

GenAI @ Adobe

AI is Everywhere

Agenda

• LLM’s - A 10,000 ft view
• Evolution of LLM’s
• Applications/Demos
• Key Takeaways

GenerativeAI

LLM / LVM

GenAI - Models that Generate Content

Text

Video

Images

Audio

Others

LLM’s - A 10,000 ft View

Introduction to Language Models

Classic Natural Language Problems

LLM in Simple Terms

• Two Files
• Parameters - Model Weights
• Execution Code - run.c
• Example: LLAMA-2 70B parameter model by Meta AI Research takes 140GB space (2 Bytes, FP16 precision)

LLM Data Preprocessing Pipeline

LLM Model Pre-Training

Chunk of Internet,

~10 TB of Text

6000 GPUs for 12 days,

~$2M in Cost,

~1e24 FLOPS

~140 GB

parameters file

What is an LLM Really Learning?

Example Tasks

Roses are red,

Roses are red,

Violets are blue,

Sugar is sweet,

for(var i = 0

for(var i = 0; i <10; i++) {

Text Completion

Code Completion

ChatGPT’s Language Model

• Human Speech/Text is variable by nature
• So to make responses more human like, ChatGPT samples from the high probability words from it’s dataset
• As a result, the model will not predict the same word each time
• This adds more diversity and unpredictability to it’s responses

Next Word Prediction Task as Dataset Compression

• Goal of an LLM Model
• Given some text, predict the next word
• This task forces the model to learn a lot about the world
• Example:
• Imagine the model is trained on a diverse set of internet data, including books, articles, and websites.
• During inference, if we ask the model a question like, “What are the symptoms of the flu?” it can provide a detailed answer based on its training data.
• The model doesn’t need to search the internet in real time; instead, it recalls and compresses information from the training data.
• In short, the LLM has “compressed” knowledge from billions of words into its neural network, allowing it to generate responses that seem informed based on what it learned during training.

LLMs let us

prototype fast

Evolution of LLM’s

Understanding Transformers

Year: 2017 - The Birth of a New Era in Language Processing

• Key Components
• Self-Attention Mechanism - Allows the model to weigh the importance of different words in a sentence
• Example: In the sentence "The cat sat on the mat," it understands that "cat" is more related to "sat" than "mat."
• Positional Encoding - Adds information about the position of words in a sentence
• Example: Helps the model distinguish between "John loves Mary" and "Mary loves John."
• Process entire input all at once, unlike previous models that processed word by word
• Better at understanding context and relationships between words

Transformer Illustration for Translation Task

Breakthrough: BERT Model

• Bidirectional word context understanding both from left and right of a word in sentence
• Two Stages:
• Pre-Training - Masked Language Model (MLM) and Next Sentence Prediction (NSP)
• Fine-Tuning - Question Answering, Sentiment Analysis, etc.
• Imagine a sentence: "I accessed the bank account.”
• Previous Models - "bank" might be understood only as a financial institution
• BERT, through its context understanding, recognizes that "bank" in this sentence refers to a financial institution, not the land alongside a river

Breakthrough: GPT Model

Text Classification Task

Does a given premise

imply a hypothesis?

How similar two pieces

of text are?

Given a context, choose

The most appropriate

Answer from multiple

options.

GPT - 4

Sparks of Artificial General Intelligence…

LLama Model

• Emergence of smaller models like LLaMA 2/3 made the field more exciting.
• Achieving performance close to larger models like GPT-4.
• Faster inference times, ideal for real-time applications.
• More resource-efficient, allowing for broader deployments.
• Balancing model size with strong language capabilities.
• Suitable for scenarios requiring quick responses, on-device/in-browser deployment, such as chatbots etc.

Pre-training vs Fine-tuning

Applications/Demos

Understanding Self-Attention in Transformers

https://github.com/anujdutt9/Talks_and_Presentations/blob/main/Decoding_the_Giants/Demo_3_RAG_for_Website_PDF_QA.ipynb

Next Word Prediction using Pre-trained LLM

https://github.com/anujdutt9/Talks_and_Presentations/blob/main/Decoding_the_Giants/Demo_1_Pretrained_LLM_Next_Word_Prediction.ipynb

Pre-trained LLM Fine-tuning using LoRA

https://github.com/anujdutt9/Talks_and_Presentations/blob/main/Decoding_the_Giants/Demo_2_LLM_Fine_tuning.ipynb

Retrieval Augmented Generation using LLM’s

https://github.com/anujdutt9/Talks_and_Presentations/blob/main/Decoding_the_Giants/Demo_3_RAG_for_Website_PDF_QA.ipynb

Key Takeaways

• Transformers Revolutionized NLP
• The introduction of transformers marked a paradigm shift, enabling better context understanding through mechanisms like self-attention.
• LLMs as Compressed Knowledge Banks
• LLMs store vast amounts of knowledge from training, making them effective for a wide range of tasks without internet search.
• Challenges Require Innovation
• Addressing issues like bias, scalability, and high training costs is essential for sustainable development in the LLM space.
• Fine-Tuning for Specific Use-Cases
• Fine-tuning enables models to adapt to specific domains, offering flexibility and improved performance for targeted applications.
• Future Lies in Adaptability
• The focus is shifting towards models that can update knowledge efficiently and adapt to new information in real-time.
• Real-World Applications are Expanding
• From chatbots to content generation, LLMs are finding new applications in various industries, especially in regions with multilingual needs.

Questions?