Accel AI Webinar

21st April, 2023

prayank@accel.com

https://www.linkedin.com/in/prayank/

Today’s focus

• Kickstarting Accel AI community

Today’s focus

• Kickstarting Accel AI community
• Appetizing your AI diet - LLMs + Stable Diffusion Demo + LLMOps

​

Lots of AI appetizers

Today’s focus

• Kickstarting Accel AI community
• Appetizing your AI diet - LLMs + Stable Diffusion Demo + LLMOps
• Group discussion - Fine-tuning an LLM discussion

​

Understanding where our community is in our AI journey

Survey of webinar participants - 1/4

Survey of webinar participants - 2/4

Survey of webinar participants - 3/4

Survey of webinar participants - 4/4

Lots to learn !!

AI Landscape

The AI Techstack

https://a16z.com/2023/01/19/who-owns-the-generative-ai-platform/

AI landscape is buzzing

https://base10.vc/post/generative-ai-mission-critical/

There are a lot of GenAI models

https://huggingface.co/models

​

Major AI models ruling the world

The world is coalescing around the big models

https://base10.vc/post/generative-ai-mission-critical/

But not everything is ready for prime time …

Source

Lot of issues need to be solved …

http://review.insignia.vc/

LLMs

What does an LLM do?

https://jalammar.github.io/applying-large-language-models-cohere/

The kind of tasks LLMs can perform

https://txt.cohere.com/generative-ai-part-2/

LLM Usecases

https://platform.openai.com/examples/�https://towardsdatascience.com/choosing-the-right-language-model-for-your-nlp-use-case-1288ef3c4929

https://vectara.com/large-language-models-use-cases/

Not all LLMs are made equal

Multiple LLMs in the market

https://www.cerebras.net/blog/cerebras-gpt-a-family-of-open-compute-efficient-large-language-models/

https://arxiv.org/abs/2303.18223

https://github.com/RUCAIBox/LLMSurvey

https://towardsdatascience.com/choosing-the-right-language-model-for-your-nlp-use-case-1288ef3c4929

Slight detour …

​

​

Slight detour … looking at what data LLMs are trained on

BloombergGPT - 708B tokens

https://arxiv.org/abs/2303.17564

The Pile - 825GB

https://arxiv.org/abs/2101.00027

https://arxiv.org/abs/2201.07311

Ratios of various data sources in pre-training data for existing LLMS

In summary … its a wild west of LLMs out there !

Resources for training LLM from scratch

• https://blog.replit.com/llm-training
• https://www.anyscale.com/blog/ray-common-production-challenges-for-generative-ai-infrastructure
• https://huyenchip.com/2023/04/11/llm-engineering.html
• https://home.mlops.community/home/content

Stable Diffusion Demo

oil on matte canvas, sharp details, the expanse scifi spacescape ceres colony, intricate, highly detailed, digital painting, rich color, smooth, sharp focus, illustration, Unreal Engine 5, 8K, art by artgerm and greg rutkowski and alphonse mucha

Stable Diffusion Demo

oil on matte canvas, sharp details, the expanse scifi spacescape ceres colony, intricate, highly detailed, digital painting, rich color, smooth, sharp focus, illustration, Unreal Engine 5, 8K, art by artgerm and greg rutkowski and alphonse mucha

(knollingcase:1.2), (symmetry:1.1) , Vintage car, pink and gold and opal color scheme, beautiful intricate filegrid facepaint, intricate, high-resolution OLED GUI interface display, micro-details, octane render, photorealism, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, labelled, overlays, oled display, annotated, technical, knolling diagram, technical drawing, display case, dramatic lighting, glow, dof, reflections, refractions

(knollingcase:1.2), (symmetry:1.1) (floral:1.05) woman as a beautiful goddess, pink and gold and opal color scheme, beautiful intricate filegrid facepaint, intricate, elegant, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, labelled, overlays, oled display, annotated, technical, knolling diagram, technical drawing, display case, dramatic lighting, glow, dof, reflections, refractions

LLM Ops

Taking an LLM to production

We are going to talk about this

https://foundationcapital.com/foundation-model-ops-powering-the-next-wave-of-generative-ai-apps/

Prompt Engineering & Management

​

Prompt Engineering, templates, market places, management

Data & Embedding Management

​

Bring external data into your AI Applications

Fine-Tuning

​

Further training your generalized models to a specific use case

Deploy, Optimize & Monitor

​

Manage, Manage and Optimize your production AI apps

Foundational Model Programming Frameworks

​

Orchestrate multiple parts of the app workflow

Adapt

LLM Ops

https://www.youtube.com/watch?v=bA5z4PQmM9M&t=541s

Designing Prompts

https://medium.com/@thebabar/the-art-and-science-of-crafting-effective-prompts-for-llms-e04447e8f96a

Prompt Templates & Marketplaces

Templates: ��Ready-made templates with placeholders for input variables that automatically suggest starting points and improvements

Marketplace: ��Users can share, discover, buy, and sell prompts for a wide range of use cases.

Prompt Management

Because prompt design is an iterative, experimental process, builders need management tools that help them organize, track, and collaborate on prompts, along with optimization tools that enable them to A/B test iterations, feed them to multiple foundation models, and measure their performance against industry-standard ML benchmarks. �

PromptLayer Demo (start at 2:00)

​

Data & Embeddings Management

• Add memory to a prompt
• Context can be inserted directly into your LLM

Connecting LLMs to external data - 1/3

https://blog.langchain.dev/langchain-chat/

Ingestion

• Take a set of proprietary documents
• Split them up into smaller chunks
• Create an embedding for each document

Connecting LLMs to external data - 2/3

Query

• Create an embedding for the query
• Find the most similar documents in the embedding space
• Pass those documents, along with the original query, into a language model to generate an answer

Connecting LLMs to external data - 3/3

How to make this into a chatbot setting?

• We need the ability to be able to answer follow-up questions
• Given a conversation history and a new question, create a single, standalone question
• Use that question in a normal Vector Database Question Answer Chain.

This adds context (memory)

LLM Programming Frameworks

Prompt templates, loader integrations, embedding models, third-party APIs, agents, coordinating other apps

​

Langchain - why you need it?

LangChain is a framework for developing applications powered by language models.

• Be data-aware
• Be agentic
• Be LLM agnostic

Langchain - why you need it?

Usecases

​

• Personal assistants
• Question answering over docs
• Chatbots
• Querying Tabular Data
• Interacting with APIs
• Extraction
• Evaluation
• Summarization

🦜️🔗 LangChain

Components

​

• Schema
• Models
• Prompts
• Indexes
• Document Loaders (link)
• Vector Stores (link)
• Retrievers (link)
• Memory (link)
• Chains
• Agents

Group discussion on fine-tuning

Jacob Joseph

VP, Data Science

CleverTap

Naveen Aiathurai

Principal Product Engineer

Oslash

Pointers for today’s discussion

• Deciding on usecases
• Design choices:
• Fine-tune or train a new model?
• Which LLM?
• LLM chaining?
• Which Vector DB? (link)
• Challenges:
• Data
• Data pipelines
• Compute cost
• Measuring performance (link)
• LLM security, LLM governance

How to use LLMs effectively

https://twitter.com/transitive_bs/status/1642974419520741377

Fine-tune or create your own LLM?

https://arxiv.org/abs/2302.08091 - Do we still need clinical language models ?

Typical data-preprocessing pipeline for pre-training LLMs

https://arxiv.org/abs/2303.18223

Measuring performance of LLMs

https://crfm.stanford.edu/helm/latest/

Some resources for fine-tuning LLMs

• https://www.wandb.courses/courses/training-fine-tuning-LLMs

​

Thanks!

prayank@accel.com