Workshop 4 RAG

​

​

Jordan Tian

January, 2026

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

IAP course structure

Mini-projects

Final Project

Tue

Wed

Thu

Fri

Mon

​

Intro

+�LLMs�

Fine Tuning�+�Agents

Front-end�(optional)

Backend

+

Deploy

Observability�+�RAG

Tue

Final Project Pitches

3-333

3-333

3-333

3-333

37-212!

37-212!

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Grading

• 6 Undergraduate credits
• Pass / Fail
• Grading:
• 20% – attendance of the lectures (mini-tests)
• 40% – exercises on the workshops� Due by the start of the next day lectures
• 40% – final project – solo� Start looking around for team members !
• Need 60% to pass

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Final Project?

• Automate a routine part of your work / your business
• Demo the project on Tuesday in 10 min
• Sign up for a pitch spot

​

• Come to Sundai to build the MVP!

https://partiful.com/e/BiGl8l8t5xyTdgzxVaVn?c=dIliEXxT

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

🔥

RAG 🔥

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Workshop 4 RAG

RAG allows LLM to selectively retrieve relevant information

• In our case, docs: Company background, product info
• and posts

Then with the information

• Create posts when docs update about the updates
• Respond to comments referencing docs

​

​

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Notebook Link

Colab Notebook

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Step 1: Setup SQLlite database

We want hybrid search

​

sqlite-vec – Add vector database functionality

​

FT25 – Add keyword search functionality

​

​

​

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Step 2 : Chunk Knowledge base

Split up knowledge base into semantically similar chunks

There’s a lot of way to split chunks. �By pages paragraphs, sentences, tokens.

250-500 tokens is a good size

​

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Step 3 : Create embeddings

Fastembed - library to create embeddings

​

​

from fastembed import TextEmbedding

​

# Initialize the embedding model (downloads on first use)

print("Loading MiniLM-L6-v2 embedding model (ONNX)...")

embedding_model = TextEmbedding(model_name="sentence-transformers/all-MiniLM-L6-v2")

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Step 4: Retrieval with Hybrid Search

Compare prompt to chunks using hybrid search

​

Semantic similarity range is 0 to 1

BM25 range is -infinity to 0

• Needs to be normalized 0 to 1!

Weigh each by 0.5 and get retrieve top scoring chunks

​

Image sourced from Salesforce engineering

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Step 5: Post Generation with RAG Context

Use context engineering to find use retrieved context and create post

Image sourced from promptingguide.ai

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Notebook Link

Colab Notebook: https://colab.research.google.com/drive/1EwxAr-Je46mK_jdGxhsa73CL-ObnInKl?usp=sharing

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Integration 1

Part 1: Replace local knowledge base with Notion api to get docs

Part 2: Chunk your docs and save to sqlite DB

Part 3: Add hybrid search RAG retrieval to your create_posts function

​

🔥

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Post Automation

Listeners vs Polling

Listeners send a signal when something changes

- less API calls, need streaming api or webhooks

Polling - check every interval for changes

​

Listeners are more complex but faster and use less API calls

Polling is more robust, watch out for API limits

​

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Integration 2

Part 4: Auto-create posts using Notion API listener

Part 5: Auto-reply to comments using Mastodon comments listener

​

🔥

RAG 🔥

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Integration 3

Part 6: Add posts to sqllite db storage for retrieval

🔥

RAG 🔥

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Post Chunking

From Eduardo about post chunking:

​

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Context Management

Be careful of all the retrieved data added to your LLM prompt

Response quality degrades from noise / distractors

• Best is less than 40% of max context window
• Nemotron-30B, 250k max tokens

Image source from Pinecone

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Context Management - Cost

Input tokens have a cost too!

• Top 10 responses of 250 tokens each -> 2500 tokens
• Top 200 responses of 1k tokens each -> 200k tokens

Image source from OpenAI

Image source from Pinecone

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Retrieval Techniques to Try

Query parsing - Use LLM or another method to parse the relevant parts of the comment first

Reranking - get more results (20-100) and let an LLM use reasoning to rerank to get top 5 or 10

Retrieval Threshold - A score that documents need to pass to be considered for retrieval

Retrieval failure - When there are no close comments let the context know this ( protect against hallucination)

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Notebook Link

Colab Notebook: https://colab.research.google.com/drive/1EwxAr-Je46mK_jdGxhsa73CL-ObnInKl?usp=sharing

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

How to submit?

Edit your card on�log.iap.sundai.club/

�Add to the description:�

1. Find the card for your project on log.iap.sundai.club from workshops 1/2/3.

​

2. Add to the description with the following info:

​

Workshop 4:

<Tell us what you did today, share screenshots>

​

[GitHub Link](Attach a link to the commit on GitHub where you integrated RAG into your codebase)

​

3. Put a link to the card here on Canvas

​

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

How to submit?

24

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club

Attendance for today

Sundai Club Team

January, 2025

Mini test for today's lecture.

Please fill in to get the attendance!��https://forms.gle/PAYNjtQUnm1Pbzxz8

​

MIT 6.S093 How to ship almost anything with AI

iap.sundai.club

sundai_club