AGENTIC AI & LLMs TRACK
Agentic AI & LLMs
for Low-Resource NLP
Sub-1B Models
Fine-Tuning
Prompt Engineering
Agentic AI Frameworks
Joy Olusanya
AI/NLP Research Engineer
The Big Question
Can small language models work well
for low-resource languages?
The challenge is not simply making a model smaller. It is making a small model useful when the target language has limited data, weak representation, and fewer resources.
SUB-1B
< 1B
fewer than 1 billion parameters
CORE QUESTION
How can we get strong language- and task-specific performance without simply increasing model size?
AGENTIC AI & LOW-RESOURCE NLP
02
What Are Parameters?
Parameters are learnable numerical values inside a model. During training, the model adjusts these values to learn patterns and relationships in data.
500M
500 million parameters
1B
1 billion parameters
7B
7 billion parameters
Sub-1B models have fewer than 1 billion parameters.
IMPORTANT
Parameter count ≠ training-data size.
AGENTIC AI & LOW-RESOURCE NLP
03
The 3 Key Challenges
1. Limited Training Data
2. Poor Language Representation
A multilingual model may see much less of the target language during pre-training.
3. Limited Model Capacity
A sub-1B model has fewer parameters than larger models, so it has less capacity for representing complex patterns.
AGENTIC AI & LOW-RESOURCE NLP
04
CHALLENGE 1
Limited Training Data
THE PROBLEM
Low-resource languages often have small, noisy, fragmented or domain-limited datasets. A sub-1B model therefore has fewer examples from which to learn useful patterns.
SOLUTIONS
Data curation: remove duplicates, noise and poor-quality examples.
Data augmentation: create useful new examples from existing data, e.g. paraphrases or instruction variants.
Multilingual transfer: leverage knowledge learned from higher-resource languages.
Continual pre-training: continue training an existing multilingual model on target-language data.
KEY IDEA
With little data, quality and transfer matter as much as quantity.
AGENTIC AI & LOW-RESOURCE NLP
05
CHALLENGE 2
Poor Language Representation
THE PROBLEM
Multilingual pre-training is usually dominated by high-resource languages. A low-resource language may receive limited exposure, so the model may learn weak representations of its vocabulary, grammar and cultural patterns.
SOLUTIONS
Continual pre-training on target-language text.
Cross-lingual transfer from related or better-resourced languages.
Careful language/data balancing so target-language examples are not drowned out.
Use culturally relevant, native-speaker-validated data.
PIPELINE
Existing multilingual
model
→
Target-language
data
→
Continued
training
→
Better language
representation
AGENTIC AI & LOW-RESOURCE NLP
06
CHALLENGE 3
Limited Model Capacity
THE PROBLEM
A sub-1B model has fewer parameters and less capacity than a 7B, 13B or 70B model.
QUESTION
How can we adapt the model effectively without updating every parameter?
THE STRATEGY: PEFT
Parameter-Efficient Fine-Tuning trains only a small part of what is needed for adaptation while keeping most of the base model frozen.
PEFT METHODS
LoRA → learns small low-rank adapter matrices.
QLoRA → LoRA + a quantised base model.
AGENTIC AI & LOW-RESOURCE NLP
07
LoRA: Low-Rank Adaptation
LoRA is a Parameter-Efficient Fine-Tuning (PEFT) method.
INSTEAD OF
Updating every original model weight
LORA DOES THIS
Freeze the original model weights.
Add small trainable low-rank matrices to selected layers.
Train only these small adapter parameters.
CONCEPTUAL VIEW
Base model
+
LoRA adapters
→
Adapted model
WHY IT IS USEFUL
Fewer trainable parameters
Easier to save/share task-specific adapters
Useful when compute is limited
AGENTIC AI & LOW-RESOURCE NLP
10
QLoRA
QLoRA = Quantisation + LoRA.
QUANTISATION
Represent model weights using fewer bits.
FP16
16-bit representation
INT8
8-bit representation
INT4
4-bit representation
LoRA reduces how many parameters are trained.
Quantisation reduces how much memory is needed to store the base model.
TOGETHER
Quantised base model
+
Small LoRA adapters
↓
Memory-efficient fine-tuning
AGENTIC AI & LOW-RESOURCE NLP
11
Prompt Engineering
Prompt engineering = designing instructions that guide the model towards the desired behaviour.
ROLE
what the model should act as
TASK
exactly what it should do
CONTEXT
information it should use
CONSTRAINTS
what it must or must not do
OUTPUT FORMAT
how the answer should look
EXAMPLE
“You are a Yoruba NLP assistant. Translate this proverb into natural English while preserving its cultural meaning. Do not give a literal translation if it changes the intended meaning. Return: Translation + brief explanation.”
GOOD PROMPTS REDUCE AMBIGUITY.
AGENTIC AI & LOW-RESOURCE NLP
13
From LLMs to Agentic AI
TRADITIONAL LLM
User
→
Prompt
→
LLM
→
Answer
AGENTIC SYSTEM
Goal
→
Plan
→
Use tools
→
Observe
→
Decide
→
Act
→
Final answer
WHAT MAKES IT AGENTIC?
The LLM is part of a multi-step workflow. The system can decide when to retrieve information, call a tool, inspect the result, and continue the task.
An agent is therefore more than a single prompt-response interaction.
AGENTIC AI & LOW-RESOURCE NLP
14
Agentic AI Frameworks & Final Pipeline
COMMON FRAMEWORKS
LangChain
LangGraph
LlamaIndex
AutoGen
CrewAI
AGENTIC AI & LOW-RESOURCE NLP
16
THANK YOU
Let’s Connect On
linkedin.com/in/joy-olusanya-209340206
x.com/Naomi_Joy___
Joy Olusanya — AI/NLP Research Engineer