The Magic of Language in the AIGC Era: Unlocking the Potential of GPT

Exploring How to build Large Language Models (LLMs)

Presented by Yuxiang (Kevin) Zheng

CRICOS 00026A TEQSA PRV12057

Yuxiang (Kevin) Zheng

Honour student

• Bachelor of Advanced Computing
• Computational Data Science

​

• Bachelor of Science
• Mathematics

​

• Dalyell Scholar

Researcher

• Trustworthy Machine Learning Lab

​

• Research Interests:
• AI for Biomedicine:
• Protein Design
• Molecule Generation
• Antibody Generation

​

• Robust Learning Algorithm:
• Noisy Label Learning
• Semi-supervised Learning

​

The University of Sydney

Acknowledgement of country

We recognise and pay respect to the Elders and communities – past, present, and emerging – of the lands that the University of Sydney's campuses stand on. For thousands of years they have shared and exchanged knowledges across innumerable generations for the benefit of all.

Overview

• Recaps
• Language Model
• Autoregressive Model
• Transformer
• Architecture
• Attention Mechanism
• Self-Attention
• Multi-Head Attention
• Tokenizer
• Position Encoding
• Data
• Reinforcement Learning from Human Feedback
• Downstream Tasks
• Future Insight

​

​

​

​

​

​

The University of Sydney

LMMs

• LLMs / chatbots took over the world

The University of Sydney

OpenAI, Introducing OpenAI o1, 2023

The University of Sydney

“Let’s assume we’re back in 2017, and you are the CTO of OpenAI. The board expects you to develop a groundbreaking natural language model, providing you with ample budget and computational resources.”

Scenario Setting

• Aim: develop a practical, efficient, and valuable chatbot.

​

​

​

• Questions:
• What model architecture to choose?
• How do we utilize data efficiently?
• How do we ensure model reliability?
• How to make the model adaptable to multiple tasks?

​

​

Most of Industry

Most of Academia

The University of Sydney

Language Model (LM)

The University of Sydney

Autoregressive (AR) Language Model

The University of Sydney

Loss of AR model

The University of Sydney

Autoregressive (AR) Language Model

Model

Linear Layer

OpenAI

Representation of context

The University of Sydney

What model architecture to choose?�

• Traditional Statistical Language Model
• n-gram Model
• Limitation: Can only capture local context (short-range dependencies)

​

• RNN based Language Model
• GRU/LSTM
• Limitation: There is a gradient vanishing problem when processing long texts.

​

• Transformer based Language Model
• A good understanding of long-range context

​

​

Transformer achieves the state-of-arts performance.

The University of Sydney

Transformer

Decoder

Encoder

The University of Sydney

Attention Mechanism�- psychology

• Animals need to efficiently focus on points of attention in complex environments.

​

• Psychological framework: Humans select points of attention based on endogenous and exogenous cues.

​

Fovea

Fovea

Macula

Macula

Exogenous Cues

Endogenous Cues

The University of Sydney

Attention Mechanism

• Convolutional, fully connected, and pooling layers only consider exogenous cues.
• The attention mechanism explicitly incorporates endogenous cues.
• Endogenous cues are represented as queries (Q).
• Each input is characterized by a value (V) and an associated exogenous cue, referred to as a key (K).
• The attention pooling layer selectively emphasizes certain inputs with a bias.

​

The University of Sydney

Self-Attention

The University of Sydney

Multi-Head Attention

The University of Sydney

Tokenizer

• Why?
• More general than words (e.g. typos).
• Shorter sequences than with characters.

​

• Idea: tokens as common subsequences.

​

• Whitespace Tokenization
• Subword Tokenization
• Lemmatization & Stemming Tokenization

Transformer revolutionizes natural language processing by learning contextual embeddings

Transformer revolutionizes natural language processing by learning contextual embeddings.

The University of Sydney

Tokenizer

• Why?
• More general than words (e.g. typos).
• Shorter sequences than with characters.

​

• Idea: tokens as common subsequences.

​

• Whitespace Tokenization
• Subword Tokenization
• Lemmatization & Steming Tokenization

Transform ##er revol ##ution ##ize natural language process ##ing by learn ##ing contextual embedding ##s

Transformer revolutionizes natural language processing by learning contextual embeddings.

The University of Sydney

Tokenizer

• Why?
• More general than words (e.g. typos).
• Shorter sequences than with characters.

​

• Idea: tokens as common subsequences.

​

• Whitespace Tokenization
• Subword Tokenization
• Lemmatization & Stemming Tokenization

Transform revolution nature language process by learn context embed

Transformer revolutionizes natural language processing by learning contextual embeddings.

The University of Sydney

Position Encoding

• Why?
• Unlike RNNs, Transformers process input in parallel, without a natural sense of sequence or order.
• Provides each word with unique position information, enabling the model to capture both local and long-range dependencies.

​

• No training required

​

Word Embedding + Positional Encoding → Transformer Input

The University of Sydney

How do we utilize data efficiently?

• Idea: use all of the clean internet
• Note: the internet is dirty & not representative of what we want. Practice:
• Download all of the internet. Common crawl: 250 billion pages, > 1PB (>1e6 GB)
• Text extraction from HTML (challenges: math, boiler plate)
• Filter undesirable content (e.g. NSFW, harmful content, PII)
• Deduplicates (url/document/line). E.g. all the headers/footers/menus in forums are always the same.
• Heuristic filtering. Remove low-quality documents (e.g. # words, word length, outlier tokens, dirty tokens)
• Model-based filtering. Predict if the page could be referenced by Wikipedia.

​

• Also: Ir annealing on high-quality data, continual pretraining with longer context
• GPT2 crawl all the links with at least 3 karma.

​

The University of Sydney

How do we ensure model reliability?

• Language models may generate toxic or biased statements.
• This can be particularly dangerous if the response is misleading but sounds convincing.

​

​

• Google spends billions of dollars every year managing public relations issues caused by inappropriate outputs from large models.

​

The University of Sydney

Reinforcement Learning from Human Feedback (RLHF)

The University of Sydney

Loss of RLHF

The University of Sydney

How to make the model adaptable to multiple tasks?�

• Question Answering (QA): A task where the model provides precise answers to user questions based on provided context or knowledge.

​

• Translation: The process of converting text from one language to another, preserving meaning and context.

​

• Summarization: Creating concise summaries of longer texts while retaining the core information and meaning.

​

• Classification: Assigning categories or labels to text based on its content, such as spam detection or sentiment analysis.

The University of Sydney

Multi-Task Learning

• Traditional fine-tuning for each task: The model is trained via repeated gradient updates using a large corpus of example tasks.

​

​

​

​

• Prompt engineering: In addition to the task description, the model sees a few examples of the task. No gradient updates are performed.

​

The University of Sydney

Future Insight

• Haven’t touched upon:
• Details of Transformer
• Context size
• Multimodality
• Legality of data collection

​

​

• Going further:
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• A Neural Corpus Indexer for Document Retrieval
• COMP5329: Deep Learning

​

The University of Sydney

End slide with university and social links

TML

Kevin Zheng

first.last@sydney.edu.au

​

yuxiang.zheng@sydney.edu.au

Thank you for your attention!