💎🔗 Langchain.rb

For: NYC.rb

By: Andrei Bondarev

Date: July 12, 2023

About Me

• 12 years of professional software development
• Built software for Spree Commerce, Department of HHS, Gannett/USA Today, WeddingWire, FiscalNote, NPR, and others.
• Running a software dev firm called Source Labs LLC since 2017
• Architect/Engineering Manager/Fractional CTO on client projects

Background

• Built a few (keyword) search applications
• Dove into vector search capabilities
• Discovered Langchain
• Started working on Langchain.rb

💎🔗 Langchain.rb

​

• github.com/andreibondarev/langchainrb
• 2 months old
• ⭐️ 400 stars
• 23 contributors
• 7k downloads
• Discord community
• Langchain.rb: an orchestration layer

Source: “Emerging Architectures for LLM Applications” (a16z blog)

Why Ruby?

• Ruby is not going anywhere. Is Ruby Dead?
• Pivot to AI Engineering
• Python == Ruby
• Pragmatism
• SOLID / OOP principles
• Monoliths are back in fashion

Use-cases

1. Semantic search/Vector search
2. Q&A over documents
3. Chat bots
4. *Agents (experimental)

Tools

• Prompt management
• Output parsing
• Context length validator
• Data chunking
• File imports
• Chat conversation memory
• Agent Tools
• ActiveRecord integration

Vector search

Definition: Vector search is a way to find related objects that have similar characteristics using machine learning models that detect semantic relationships between objects in an index.

Word embedding: a vector (array) of float numbers (coordinates) that represents properties of the word.

Vector size == Dimension.

Example: [0.15, 0.67, 0.54, 0.21, … ]

Vector search

​

Sentence embeddings: similar to word embeddings but converts the semantic meaning to a vector of numbers for the whole sentence

Vector search X LLM matrix

Benefits

​

• Common DSL/APIs
• Interoperability
• Reduced vendor lock-in
• Optionality
• Best practices

Vector search

ActiveRecord integration

Vector search

Q&A on top of imported corporate “benefits brochure” (2014)

Q&A over data

Chat bots

• Interface into a service and/or data
• Use-cases:
• Improve UI
• Increase the engagement
• Guide user through information retrieval/collection
• Business process automation

alchaplinsky/aiconvo

Project: alchaplinsky/aiconvo

Agents (experimental)

• Autonomous general purpose LLM-powered programs
• Tons of emergent research: Chain of Thought (CoT) , Tree of Thought (ToT), Reasoning+Act (ReAct)
• Can be used to automate workflows or execute multi-step tasks
• Work better with powerful LLMs
• Driven by carefully engineered prompts
• Need lots of error handling and constraints around them (for resiliency)
• Can use “Tools” (API wrappers, similar to OpenAI Plugins)

Tools

• Calculator
• Wikipedia
• Google Search
• Weather
• Ruby Code Interpreter
• Database

Prompt used in Langchain::Agent::ReActAgent

Langchain::Agent::ReActAgent

Langchain::Agent::SQLQueryAgent

What’s next?!

• Collecting feedback
• Sharpening up the full end-to-end flows
• Features:
• Observability
• Analytics
• Deeper integration with Rails
• Expanding into adjacent areas
• Caching layer

References

• Langchain.rb
• The Rise of the AI Engineer
• What is vector search?
• What Are Word and Sentence Embeddings?
• Iceland monolith on Unsplash
• Re-implementing LangChain in 100 lines of code
• Emerging Architectures for LLM Applications

Questions?

• Open to contributors/ideas/constructive criticism!
• Contact:
• twitter.com/@rushing_andrei
• linkedin.com/in/andreibondarev
• andrei@sourcelabs.io