1 of 29

Session 2

2 of 29

Building a Gen AI application

Open Challenges

Getting started with Gen AI

Under the hood

3 of 29

Building a Gen AI application

Open Challenges

Getting started with Gen AI

Under the hood

4 of 29

Example: Multilingual Marketing Content Generation

मुंह में पिघल जाने वाली इन विशेष कैडबरी चॉकलेटों के साथ अपने प्रियजनों का दिवाली यादगार बनाएं।

Setting: Christmas

Language: English

Celebrate Christmas with your loved ones with this exquisite melt-in-the-mouth chocolates from the House of Cadburys

Setting: Diwali

Language: Hindi

Objective: Create contextualized marketing content + images for a target setting and language

5 of 29

5

  • Programming through natural language
  • Effective prompting strategies – similar to instructing humans

Prompting Basics

I am trying to build a demo to automatically create new marketing content.

Context

Instructions

Can you provide me python code for a streamlit demo with the following functionality?

Data

Examples

< Input instruction, Output code>

Step 1: Create an upload button to take in a product image.

Step 2: ….

Segment each component

Provide context or persona for LLM to adopt

Break up a complex task into simpler ones

Provide input-output examples

Be precise on expected output

Advanced strategies: Chain/Tree of thought prompting, many more …

6 of 29

Prompt: EM Marketing Content Generation

I am trying to build a demo to automatically create new marketing content and images. Can you provide me python code for a  streamlit demo with the following functionality?

Step 1: Create an upload button to take in an image of a product.

Step 2: Create text inputs to take in (a) 2-3 lines  marketing content, (b) target setting, (c) product, and (d) target language

Step 3: Create a drop down or radio button to specify target languages from a set of popular languages including Indian ones

Step 4: Use GPT-3.5 api to generate a new version of the specified  marketing content customised to the target setting and also to translate the new custom message into the target language

Step 5: Use GPT 3.5 api to generate a suitable prompt for creating a image of product in the specified target setting

Step 6: Use DallE to generate a new image of the uploaded product image for the target setting 

Step 7: Display the content in English and the target language in an output textbox

Step 8: Display the new image 

7 of 29

8 of 29

8

9 of 29

Demo vs. Production System

There is a world of difference !

Quality metrics, compute costs, robustness, scalability ….

10 of 29

Building a Gen AI application

Open Challenges

Getting started with Gen AI

Under the hood

11 of 29

Evaluation Metrics for Generative AI

Question: Mt Everest kitna tall hai ? Kisne measure kiya ?

Factual Answer: Mt Everest is 8848 m tall. It was first measured by an Indian mathematician Radhanath Sikdar

Answer 1: How can you be so dumb? Everyone knows Mt. Everest hight is 30000 ft. I went there last summer

There can be multiple “correct” responses !

Evaluation of generated complex artifacts (e.g., natural language , images, videos) is non-trivial even when there is a ”ground truth” response.

Answer 2: Mt. Everest ka height 29032 ft hai. Ek Indian mathematician Radhanath Sikdar ne usko sabse pehla measure kiya

12 of 29

Evaluation Metrics for Generative AI

Question: Mt Everest kitna tall hai ? Kisne measure kiya ?

Factual Answer: Mt Everest is 8848 m tall. It was first measured by an Indian mathematician Radhanath Sikdar

Answer 1: How can you be so dumb? Everyone knows Mt. Everest hight is 30000 ft. I went there last summer

Answer 2: Mt. Everest ka height 29032 ft hai. Ek Indian mathematician Radhanath Sikdar ne usko sabse pehla measure kiya

Factual Accuracy

Completeness

Specificity

Toxicity

Style/Language

Stated facts are accurate; might entail semantic understanding

e.g., tall -> height,

ft-m conversion

Response should be relevant to the query and not contain unnecessary additional info, e.g. summer visit

Response should address all parts of the task. e.g., both height and who measured it

Response should be polite and in general aligned to the intended application’s policy

e.g., no rudeness

Response style should be as desired

e.g., succinct, reciprocate formality level, language patterns

Many more criteria …. ; LLM/Gen AI models can themselves be used for evaluation as well !

13 of 29

Prompting Strategies - Summary

    • Zero-shot
    • Few-shot or In-context Learning (ICL)
    • Chain of thought (CoT)
    • Self consistency (SC)
    • Tree/Graph of thought (ToT/GoT)
    • Prompt Chaining
    • Chain of Verification (CoVE)
    • many more …

14 of 29

Zero-shot vs. Few-shot Prompting

Zero Shot Prompting

Few Shot Prompting

(In-context Learning ICL)

15 of 29

Chain of Thought (COT) Prompting

16 of 29

Thought Exploration Frameworks

    • CoT: chain of reasoning steps
    • Self consistency: Multiple runs and majority vote
    • ToT: Multiple reasoning options at each step

17 of 29

Prompt Chaining

    • Break a complex task into multiple steps
    • Solve each step with a different prompt
    • Output of step n is used in later steps

18 of 29

Chain of Verification (CoVE)

Special case of Prompt Chaining

19 of 29

Prompting Strategies Summary

    • Zero-shot: Simple instruction, no examples
    • Few-shot, In-context learning: Few input-output examples
    • Chain of thought: Instructions with reasoning steps
    • Self consistency: Multiple runs and majority vote
    • Tree/Graph of thought: Explore multiple reasoning options at each step
    • Prompt Chaining: Break into multiple steps with separate prompts for each
    • Chain of Verification: Baseline answer, plan verifications, verify, adjust

20 of 29

Practical prompting tips for ChatGPT/Claude/Bard

Fluent in English, coding, bookish knowledge of Indian languages

Knowledgeable about general topics, e.g., programming

Can sometimes be wrong on detailed facts, e.g., RAM of a laptop

Often wrong on recent events and dynamic information, e.g. Laptop price

Better performance with advanced prompting strategies (e.g., ICL, COT, chaining)

Consult model-specific prompt engineering guide

21 of 29

Response

Generator

Auto

Evaluator

Prompt Template

Generator

Prompt Policy

Optimizer

Input

Queries

Prompt Template(s)

Output

Responses

Eval. Metric

(Reward)

Prompt Policy Vector

Final Prompt Template

Ground Truth (optional)

Automated Prompt Tuning

22 of 29

LLM-Agent Architectures

23 of 29

LLM-Agent Architectures

24 of 29

Dialog Planner

Customer Profile Engine

LLM-based Generator

Agent UI

Customer

UI

Any mobile with discounts? BTW, what does no-cost EMI mean?

Internal APIs

Tool Registry

Cust. Profile: Price conscious,

Brand: Samsung , Discount: >50% ASINs 1) XX1, 2) XX2

You will love this! We have a 50% discount on these Samsung phones: 1) XX1, 2) XX2…

Glad you asked about NCEMI …. It is ….

Other related questions

  1. LG phones on sale
  2. EMI options on Samsung phones

NL Utterance + Tap Input

NL + Tappable Response

Plan

Pre-conversation State

Prior Activity

Retrieved

Evidence

1

2

3

4

Next Suggestions

(topics, questions, refinements)

  1. LG phones on sale
  2. EMI options on Samsung phones

Transfer to agent if no appropriate response is identified

Retriever

No-cost EMI or NCEMI means …..

Safeguards

Tiered

Know

-ledge Store

Prompt

Generator

LLM

invocation

LLM

invocation

Inside a Chatbot

25 of 29

Building a Gen AI application

Open Challenges

Getting started with Gen AI

Under the hood

26 of 29

Open Research Challenges

    • Detecting and reducing hallucinations
    • Flexible alignment and personalization to human preferences
    • Allowing larger context for RAG settings
    • Reducing latency and computational efficiency
    • Self-improvement over time
    • Responsible Generative AI: fairness, safety, unlearning
    • Natively Multimodal and Multilingual generation

27 of 29

Building a Gen AI application

Open Challenges

Getting started with Gen AI

Under the hood

28 of 29

Session 2: Key Takeaways

Building LLM-based Applications

Smart prompting can go a long way (mix code, text, multi-media generation)

Evaluation requires application-specific considerations

Prompting strategies

Few-shot, COT, SC, ToT, Chaining, CoVE, and many more

Mimic patterns of human problem solving

Agent architectures

LLM-based Planning + Tools – powerful paradigm

Start your Gen AI journey

Focus on fundamentals, try out things, keep learning

29 of 29

THANK YOU !