Session 2
|
|
|
|
|
| |
| |
| |
| |
| |
| |
|
|
|
|
|
| |
| |
| |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Building a Gen AI application
Open Challenges
Getting started with Gen AI
Under the hood
Building a Gen AI application
Open Challenges
Getting started with Gen AI
Under the hood
Example: Multilingual Marketing Content Generation
मुंह में पिघल जाने वाली इन विशेष कैडबरी चॉकलेटों के साथ अपने प्रियजनों का दिवाली यादगार बनाएं।
Setting: Christmas
Language: English
Celebrate Christmas with your loved ones with this exquisite melt-in-the-mouth chocolates from the House of Cadburys
Setting: Diwali
Language: Hindi
Objective: Create contextualized marketing content + images for a target setting and language
5
Prompting Basics
I am trying to build a demo to automatically create new marketing content.
Context
Instructions
Can you provide me python code for a streamlit demo with the following functionality?
Data
Examples
< Input instruction, Output code>
Step 1: Create an upload button to take in a product image.
Step 2: ….
Segment each component
Provide context or persona for LLM to adopt
Break up a complex task into simpler ones
Provide input-output examples
Be precise on expected output
Advanced strategies: Chain/Tree of thought prompting, many more …
Prompt: EM Marketing Content Generation
I am trying to build a demo to automatically create new marketing content and images. Can you provide me python code for a streamlit demo with the following functionality?
Step 1: Create an upload button to take in an image of a product.
Step 2: Create text inputs to take in (a) 2-3 lines marketing content, (b) target setting, (c) product, and (d) target language
Step 3: Create a drop down or radio button to specify target languages from a set of popular languages including Indian ones
Step 4: Use GPT-3.5 api to generate a new version of the specified marketing content customised to the target setting and also to translate the new custom message into the target language
Step 5: Use GPT 3.5 api to generate a suitable prompt for creating a image of product in the specified target setting
Step 6: Use DallE to generate a new image of the uploaded product image for the target setting
Step 7: Display the content in English and the target language in an output textbox
Step 8: Display the new image
8
Demo vs. Production System
There is a world of difference !
Quality metrics, compute costs, robustness, scalability ….
Building a Gen AI application
Open Challenges
Getting started with Gen AI
Under the hood
Evaluation Metrics for Generative AI
Question: Mt Everest kitna tall hai ? Kisne measure kiya ?
Factual Answer: Mt Everest is 8848 m tall. It was first measured by an Indian mathematician Radhanath Sikdar
Answer 1: How can you be so dumb? Everyone knows Mt. Everest hight is 30000 ft. I went there last summer
There can be multiple “correct” responses !
Evaluation of generated complex artifacts (e.g., natural language , images, videos) is non-trivial even when there is a ”ground truth” response.
Answer 2: Mt. Everest ka height 29032 ft hai. Ek Indian mathematician Radhanath Sikdar ne usko sabse pehla measure kiya
Evaluation Metrics for Generative AI
Question: Mt Everest kitna tall hai ? Kisne measure kiya ?
Factual Answer: Mt Everest is 8848 m tall. It was first measured by an Indian mathematician Radhanath Sikdar
Answer 1: How can you be so dumb? Everyone knows Mt. Everest hight is 30000 ft. I went there last summer
Answer 2: Mt. Everest ka height 29032 ft hai. Ek Indian mathematician Radhanath Sikdar ne usko sabse pehla measure kiya
Factual Accuracy
Completeness
Specificity
Toxicity
Style/Language
Stated facts are accurate; might entail semantic understanding
e.g., tall -> height,
ft-m conversion
Response should be relevant to the query and not contain unnecessary additional info, e.g. summer visit
Response should address all parts of the task. e.g., both height and who measured it
Response should be polite and in general aligned to the intended application’s policy
e.g., no rudeness
Response style should be as desired
e.g., succinct, reciprocate formality level, language patterns
Many more criteria …. ; LLM/Gen AI models can themselves be used for evaluation as well !
Prompting Strategies - Summary
Zero-shot vs. Few-shot Prompting
Zero Shot Prompting
Few Shot Prompting
(In-context Learning ICL)
Chain of Thought (COT) Prompting
Thought Exploration Frameworks
Prompt Chaining
Chain of Verification (CoVE)
Special case of Prompt Chaining
Prompting Strategies Summary
Practical prompting tips for ChatGPT/Claude/Bard
Fluent in English, coding, bookish knowledge of Indian languages
Knowledgeable about general topics, e.g., programming
Can sometimes be wrong on detailed facts, e.g., RAM of a laptop
Often wrong on recent events and dynamic information, e.g. Laptop price
Better performance with advanced prompting strategies (e.g., ICL, COT, chaining)
Consult model-specific prompt engineering guide
Response
Generator
Auto
Evaluator
Prompt Template
Generator
Prompt Policy
Optimizer
Input
Queries
Prompt Template(s)
Output
Responses
Eval. Metric
(Reward)
Prompt Policy Vector
Final Prompt Template
Ground Truth (optional)
Automated Prompt Tuning
LLM-Agent Architectures
LLM-Agent Architectures
Dialog Planner
Customer Profile Engine
LLM-based Generator
Agent UI
Customer
UI
Any mobile with discounts? BTW, what does no-cost EMI mean?
Internal APIs
Tool Registry
Cust. Profile: Price conscious,
Brand: Samsung , Discount: >50% ASINs 1) XX1, 2) XX2
You will love this! We have a 50% discount on these Samsung phones: 1) XX1, 2) XX2…
Glad you asked about NCEMI …. It is ….
Other related questions
NL Utterance + Tap Input
NL + Tappable Response
Plan
Pre-conversation State
Prior Activity
Retrieved
Evidence
1
2
3
4
Next Suggestions
(topics, questions, refinements)
Transfer to agent if no appropriate response is identified
Retriever
No-cost EMI or NCEMI means …..
Safeguards
Tiered
Know
-ledge Store
Prompt
Generator
LLM
invocation
LLM
invocation
Inside a Chatbot
Building a Gen AI application
Open Challenges
Getting started with Gen AI
Under the hood
Open Research Challenges
Building a Gen AI application
Open Challenges
Getting started with Gen AI
Under the hood
Session 2: Key Takeaways
Building LLM-based Applications
Smart prompting can go a long way (mix code, text, multi-media generation)
Evaluation requires application-specific considerations
Prompting strategies
Few-shot, COT, SC, ToT, Chaining, CoVE, and many more
Mimic patterns of human problem solving
Agent architectures
LLM-based Planning + Tools – powerful paradigm
Start your Gen AI journey
Focus on fundamentals, try out things, keep learning
THANK YOU !
|
|
|
|
|
| |
| |
| |
| |
| |
| |
|
|
|
|
|
| |
| |
| |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|