Hyperdemocracy
Coproducers:
HacDC
Singularity DC
Assemble Company
2023-06-23
Gabriel Altay
Forrest Murray
Calliope Youngblood
Itinerary
Extras:
Starting with NLP
How similar are these two words?
dog
cat
How similar are these two sentences?
The bank down the street is where I have a savings account.
The river bank is a nice place to have lunch on a sunny day.
How similar are these two vectors?
How similar are these two vectors?
What if you could treat words like vectors?
dog
cat
Embeddings help turn semantics into geometry
Embeddings let you do math with words
Embeddings have linear substructure
We have already seen that words with similar meanings will be close to each other in embedding space. In addition, the directions in the vector are related to certain concepts.
Analogies can be phrased as math
King is to Man as Queen is to Woman
becomes
King - Man + Woman = Queen
How do you create embeddings?
Training Samples (predicting words from context)
Embeddings are initialized randomly and improve as the model trains
Transformer architecture
From lookup tables to attention
LLM Application Stew
Generative Language Models
e.g. GPT, PaLM
Vector / Doc Stores
e.g. Pinecone, Chroma, ES
Langchain
LLM Abstractions
(agents, tools, prompts)
Embedding Models
e.g. sentence transformers, data2vec
Structured Indices
e.g. LlamaIndex
Data
Legislation text/xml/html, co-sponsor graphs,
Prompting is all you need:
LLMs for fun and profit…
Prompt Engineering
How do we use LLMs?
What do LLMs know?
The Good
The Bad
The Sorta
How do we fix the bad by using the good? 🤔
Retrieval Augmented Generation (RAG)
🤖✨Magic Prompt
Source: https://www.pinecone.io/learn/langchain-conversational-memory
ChatGPT – Just JSON? Maybe..
🔨🧰🪛Tools
🤖✨Magic Prompt pt. 2
Legal Reasoning