1 of 31

Local Vector Database, Embedding Model and Generative Model RAG

Workshop

Scott Askinosie M.S. Ph.D.

Developer Educator & AI Engineer

2 of 31

Everything we do

(on the internet)

Starts with Search

Everything we do

(on the internet)

Starts with Search

v

3 of 31

v

4 of 31

Everything we do

(on the internet)

Starts with Search

What is the problem?

v

5 of 31

Everything we do

(on the internet)

Starts with Search

What is the problem?

v

6 of 31

The challenges of traditional search

Why do airplanes fly?

Why you should fly with ExpensiveAir’s airplanes!

…fly with ExpensiveAir’s modern fleet of airplanes to over 4 destinations. Why do you hesitate? Book now!

​

7 of 31

The challenges of traditional search

(the keywords match)

Why do airplanes fly?

Why you should fly with ExpensiveAir’s airplanes!

…fly with ExpensiveAir’s modern fleet of airplanes to over 4 destinations. Why do you hesitate? Book now!

​

8 of 31

Everything we do

(on the internet)

Starts with Search

In summary

v

9 of 31

Semantic Search to the rescue!

Why do airplanes fly?

Dynamics of Flight - NASA

Airplane wings are shaped to make air move faster over the top of the wing. When air moves faster, the pressure of the air decreases.

​

10 of 31

Everything we do

(on the internet)

Starts with Search

In summary

v

11 of 31

Everything we do

(on the internet)

Starts with Search

v

How does this work?

12 of 31

Everything we do

(on the internet)

Starts with Search

“You don’t need to understand it,

but it is pretty cool” 😉

v

How does this work?

13 of 31

Everything we do

(on the internet)

Starts with Search

“You don’t need to understand it,

but it is pretty cool” 😉

v

How does this work?

14 of 31

Everything we do

(on the internet)

Starts with Search

Machine Learning models

v

[ 0.23, 0.45, …, 0.84, 0.23 ]

ML�model

15 of 31

Everything we do

(on the internet)

Starts with Search

Vector Embeddings

v

[ 0.23, 0.45, …, 0.84, 0.23 ]

“Fly with ExpensiveAir’s modern fleet of airplanes to over 4 destinations. Why do you hesitate? Book now!”

ML�model

16 of 31

Everything we do

(on the internet)

Starts with Search

Vector Embeddings

v

ML�model

“Fly with ExpensiveAir’s modern fleet of airplanes to over 4 destinations. Why do you hesitate? Book now!”

Airplane wings are shaped to make air move faster over the top of the wing. When air moves faster, the pressure of the air decreases.

[ 0.26, 0.31, …, 0.12, 0.44 ]

[ 0.23, 0.45, …, 0.84, 0.23 ]

17 of 31

Everything we do

(on the internet)

Starts with Search

Vector Embeddings

v

ML�model

“Fly with ExpensiveAir’s modern fleet of airplanes to over 4 destinations. Why do you hesitate? Book now!”

Airplane wings are shaped to make air move faster over the top of the wing. When air moves faster, the pressure of the air decreases.

[ 0.26, 0.31, …, 0.12, 0.44 ]

[ 0.23, 0.45, …, 0.84, 0.23 ]

18 of 31

Everything we do

(on the internet)

Starts with Search

Vector Space

v

[ 0.26, 0.31, …, 0.12, 0.44 ]

[ 0.23, 0.45, …, 0.84, 0.23 ]

19 of 31

Everything we do

(on the internet)

Starts with Search

Vector Search

v

[ 0.26, 0.31, …, 0.12, 0.44 ]

[ 0.23, 0.45, …, 0.84, 0.23 ]

ML�model

Why do airplanes fly?

20 of 31

Everything we do

(on the internet)

Starts with Search

Vector Search

v

[ 0.26, 0.31, …, 0.12, 0.44 ]

[ 0.23, 0.45, …, 0.84, 0.23 ]

ML�model

Why do airplanes fly?

[ 0.24, 0.36, …, 0.16, 0.46 ]

21 of 31

Everything we do

(on the internet)

Starts with Search

Vector Search

v

[ 0.26, 0.31, …, 0.12, 0.44 ]

[ 0.23, 0.45, …, 0.84, 0.23 ]

ML�model

Why do airplanes fly?

[ 0.24, 0.36, …, 0.16, 0.46 ]

[ 0.26, 0.31, …, 0.12, 0.44 ] The NASA article

22 of 31

Large Language Model

Vector Database

One Key Difference

23 of 31

Large Language Model

(Stateless)

Vector Database

(Stateful)

24 of 31

Large Language Model

(Stateless)

Vector Database

(Stateful)

  • The knowledge base is trained into the model.
  • Insert data into the database, and generate vectors.

25 of 31

Large Language Model

(Stateless)

Vector Database

(Stateful)

  • The knowledge base is trained into the model.�
  • The model doesn’t learn on the fly.
  • Insert data into the database, and generate vectors.�
  • Data changes on the fly.

26 of 31

Large Language Model

(Stateless)

Vector Database

(Stateful)

  • The knowledge base is trained into the model.�
  • The model doesn’t learn on the fly.�
  • Requires fine-tuning to update its worldview.
  • Insert data into the database, and generate vectors.�
  • Data changes on the fly.�
  • CRUD operations to update�the state.

27 of 31

Everything we do

(on the internet)

Starts with Search

v

This is not a story of

VDBs vs LLMs

28 of 31

Everything we do

(on the internet)

Starts with Search

v

This is a story of

VDBs + LLMs

29 of 31

Everything we do

(on the internet)

Starts with Search

v

This is a story of

VDBs + LLMs

30 of 31

Everything we do

(on the internet)

Starts with Search

v

31 of 31