GenAI HW3�以API快速搭建自己的應用

TA: 尤展鴻，陳守仁，呂睿超

�ntu-gen-ai-2024-spring-ta@googlegroups.com

​

Deadline: 2024/03/21 23:59:59 (UTC+8)

Using ChatGPT Model by Browser

In the HW2, we use ChatGPT model through the browser interface

Browser

2

ChatGPT model

Creating Your Own Application Using ChatGPT

For example, you want to create an App (應用程式) that adds appropriate emojis to any given input, and you want to power it by ChatGPT

3

ChatGPT model

Creating Your Own Application Using ChatGPT

While you can manually copy the user input to the browser and copy the response from ChatGPT to the user end, this is too slow

Browser

4

Creating Your Own Application Using ChatGPT API

The ChatGPT API (Application Programming Interface) enable programmatic (程式化的) interaction with the ChatGPT model

Response: ………..

5

ChatGPT model

Creating Your Own Application Using ChatGPT API

You tell ChatGPT what you want it to do in a programmatic way

Response: ………..

6

ChatGPT model

Creating Your Own Application Using ChatGPT API

ChatGPT return the response by following you what you ask it to do

Response: ………..

7

ChatGPT model

What You Will Learn in This Homework

You will learn how to build some simple applications using ChatGPT with its API

1. You will learn how to call the ChatGPT API (and other APIs)
2. You will learn how to design the prompts to achieve the functionality you want

8

1. ​

2.

What You Will Learn in This Homework

You will learn how to build some simple applications using ChatGPT with its API

• Need to write some codes Google Colab
• Need some place to host the APP Gradio

9

Overview in Today’s TA Lecture

1. Step 1: Learn how to use colab for coding and Gradio for demo
2. Step 2: Learn what API you can use and how to obtain the API keys (similar to password)
3. Step 3: Create an application on Colab and Gradio using the API

10

Warning of Codings

11

No worries, the TAs have already written the codes.

12

Outline

• Colab and Gradio Tutorial: For coding and demo
• Available APIs: APIs you can use to build your own APP
• Formal Task Introduction: The APPs you need to create in HW3
• Grading by Evaluation Assistants on DaVinci
• Submission and Grading on NTU COOL
• Important Dates
• Regulations

13

Links

​

• NTU Cool HW3 Forum
• MediaTek DaVinci
• OpenAI ChatGPT
• Google Gemini
• Colab link to this homework

14

Colab and Gradio Tutorial

15

Introduction

What is Colab?

Colab, or "Colaboratory", allows you to write and execute Python in your

browser, with

• Zero configuration required
• Free access to GPUs
• Easy sharing

​

16

Introduction

What is Gradio?

Gradio is a tool that allows you to demo any application with a friendly website so that anyone can use it, anywhere.

​

​

​

17

https://colab.research.google.com/drive/1N7WTdVseHg4pNQpDMENo2aHOfsCS2-If?usp=sharing

We will use the following google colab notebook for tutorial demonstration

Demo for Colab and Gradio Tutorial

18

19

Create a copy in your drive

20

File location

You can change the name here

If it looks like these:

Hit it again.

Click this to unfold the blocks

21

Upload your file

Download your file

If you accidently click this

You can find your files here

Cells

Create a new cell by clicking on + Code or + Text

22

These options allow you to moving your cell up/down or copy and deleting it

Run Cell

Run Cell by clicking on the

23

Code Unexecuted

Code Executing

Code Executed

Error Occurred

Problems you may encounter

Colab will automatically disconnect if idle timeout (90 min., sometimes varying) or when your screen goes black.

​

If you “Disconnect and delete runtime” or “Restart Session”, your files will be gone.

24

25

Run these three blocks from top to down

26

Click this link will lead you to a browser version gradio

You can also use gradio inside colab

27

Dialogue

Input box

Click Send to enter your input

Click Reset to reset the dialogue

29

30

When it’s spinning, the gradio is connected to the colab.

When it stops, the gradio will no longer work.

​

​

Available APIs

31

Available APIs

You only have to choose one API to use.

Google Gemini API

• Free to use

ChatGPT (OpenAI API)

• Might not be free

32

Available APIs: Google Gemini

33

Google Gemini API

https://ai.google.dev/

​

34

Demo for Gemini API key

35

Step 1: Accept Terms of Service

36

37

Step 2: Get API Key

38

Step 3: Create API Key

39

Final: Copy

40

Problems you may encounter

​

41

IF

THEN

Problems you may encounter

​

42

THEN

Issue #2

Do not use G-Suite accounts, use your own account

43

Available APIs: ChatGPT API

44

Step 0 - Go to OpenAI Website

45

https://openai.com/

​

Demo for ChatGPT API key

46

Step 1 - Login/Create Account

47

Step 2 - Create Account(Cont.)

48

Step 3 - Check Usage

49

Step 4-1 - Phone Verification

• OpenAI API could probably require phone verification�
• Follow the verification instructions

50

Step 4-2 Create API key

51

Step 4-3 Copy and take good care of your API key !!!

• Please remember to COPY your API key, REMEMBER it and KEEP IT SAFE!�
• This is the only chance you will see the whole key

52

OpenAI API - Free Usage

• Free usage - If you are a new account to OpenAI : Free $5.00 API usage.
• Otherwise, OpenAI API costs money!!!

53

Task Introduction

54

Workflow

1. Call API

Summarization

Role Playing

Customized Task

2. Evaluate Task

Use Davinci Platform to grade your task

​

Model: GPT-4

3. Submit to NTU COOL

Task

Objective:

Understand how to build your own Language Model Application by calling API and feeding prompts.

Task:

1. Summarization (3 points)
2. Role Playing (3 points)
3. Customized Task (4 points)

Colab link to this homework:

https://colab.research.google.com/drive/15jh4v_TBPsTyIBhi0Fz46gEkjvhzGaBR?usp=sharing

56

Task Introduction: �Task 1: Summarization

57

Task 1: Summarization

1. We want you to create a bot that can perform article summarization.
2. Design a prompt that enables the language model to summarize the article.

58

USER

Language

Model

Help me summarize ……

[Given article]

Summarized article

59

Prompt

Article

Summarization

Send

Browser

API Call

60

60

Prompt

Article

Prompt

Article

Send

Send

61

Summarization

Summarization

Step 0

Choose one of the API you want to use

​

​

​

​

​

We will use ChatGPT API in the following slides and later demonstrations.

62

Click one of these two buttons to unfold the codes

Step 1

1. Run this cell to install the packages

2. Enter your ChatGPT API key here and run this cell

63

Fill in your ChatGPT API here

Remember to run these two cells whenever you reconnect to colab.

Step 2

Under this text cell Part1: Summarization

Fill in your prompt for summarization in "FILL IN YOUR PROMPT" and hit the run button.

Example: “Please summarize this article for me.”

64

Step 2

The prompt you just filled in will be used in the API call to instruct ChatGPT what it should do

65

Step 3

There will be a gradio interface like this pop up.

66

Here you can enter whatever article you want to summarize

Step 3

The article will also be a part of the input to ChatGPT

67

Step 4

68

You can use this slider to adjust the temperature here.

The higher the temperature, the more creative response you will get.

Hit “Send” to get the response.

Step 5

The response will display in gradio like this

69

Save your results

For each tasks, after you get a satisfied result. Make sure to hit the “Export” button.

If you hit the “Export” button again, the old file will be replaced.

70

Save your results

After that, you will see the file appeared in the file list.

Make sure to download it to your own computer before it disappear.

71

After you save your results …

Go to “Check and print your result” for the corresponding part

RUN IT

72

Copy

Paste

73

Demo for Task 1

74

Task Introduction: �Task 2: Role Playing

75

Task 2: Role Playing

• We want you to design a bot service that you can play a role play game with the LM.
• Engage in a multi-turn conversation with the LM
• Prompt: Tell the chatbot to play whatever character.
• Later inputs: Interact with the chatbot.

76

USER

Language

Model

I want you to act as a pirate. Explain your job to me.

I'm a digital pirate, sailing the info-seas! 🏴‍☠️

Rules: Role Play

​

1. Communicate with the LM for 2 rounds (excluding the prompt)
1. User → LM, LM Response (這樣算一次)

77

USER

Language

Model

Input: You are Shakespeare. Write me a really short poem.

"In moon's soft glow and star's tender light, Love whispers true in the hush of night."

​

Note: Observe the multi-turn conversation (history of conversation recorded)

78

Prompt

Send

Output

79

History

New Input

New Output

80

Input

Send

Input

Send

81

History

New Input

History

user: The input you typed

assistant: The output the bot generated

New Input

When using API, you have to record the history manually.

82

New Output

New Output

When using API, you have to record the history manually.

83

When using API, you have to record the history manually.

Whole Dialogue

Whole Dialogue

Note

Role: Fill in the role you want and it will be in the red block below

Input: Actual Input for conversation.

84

Demo for Task 2

85

Task Introduction: �Task 3: Customize Task

86

Task 3: Customize Task

• We want you to create a customized service bot.�
• You can think of any tasks that you want the Bot to do. For example,
• A bot that can solve simple math problems
• A bot that always output the antonym of the user’s input word
• Any creative tasks that you can think of

​

​

​

​

Be Creative!

87

Task 3: Customize Task

1. Enter a prompt for LLM to perform a certain “Task” (Do not use tasks that will produce too long outputs, specifically 1k tokens).
2. Enter the input of “Task”
3. Single or multiple rounds are both ok, but do not exceed 3 rounds.
4. Export the conversation log

88

USER

Language

Model

Prompt: “Give me the antonym of the following words”

Input: “Big”

Result: “Small”

​

89

Prompt

Input

Output

Send

90

Prompt

Input

Prompt

Input

Send

Send

91

Output

Output

Both single-turn and multi-turn is acceptable.

Note

Fill in the description of your task and it will be in the red block below

(e.g. Give antonym )

Actual Input for conversation.

​

92

Demo for Task 3

93

How to Delete Your API Keys

94

OpenAI API

https://platform.openai.com/api-keys

95

Click here to remove the delete API keys

Gemini API

https://aistudio.google.com/app/apikey

96

Click here to remove the delete API keys

Issues You Might Encounter

97

ModuleNotFoundError : No module named ‘xxx’

98

• If you encounter this error

• You haven’t installed the packages yet

Solution: Execute corresponding code block (e.g. !pip3 install xxx)

NameError : name ‘List’ is not defined

99

• If you encounter this error

• You haven’t import the packages

Solution: Execute this code block

FileNotFoundError : [Errno 2] No such file or directory

100

• If you encounter this error

• You might forgot to export the json file or upload one.

Solution:

Please check if there exists the required json file

1.

If not, refer to

corresponding pages such as

p67

Gradio UI not showing : block running but not working

101

• If you encounter this error

• You might forgot to stop other code blocks (especially those Gradio UI blocks)

Solution:

Check and stop other running code blocks

Wrong API key

102

• If you see this output (The block will finish execution without error occurring)

Solution:

Refer to p33-p42 (Gemini)

p43-p52 (OpenAI)

to double check if you retrieve the correct API key.

Then fill in the api key as the description (refer to p62)

Prompting Issue

Problem: Several prompts will encounter such error.

​

Solution: Change your prompts

103

Evaluation Assistants on DaVinci for Grading

104

Grading

Model Version: GPT-4-Turbo-8k

Assistant: GenAI Homework3 Assistant

• 3 Prompts
• Summarization Grader
• Roleplay Grader
• Customized Task Grader

Copy your result to the Grading Assistant in DaVinci Platform.

Link of Davinci Platform: https://dvcbot.net/

105

DaVinci Install Assistant

106

107

1. Press “+ Chat”

2. Choose “Assistant”

108

2. Click “GenAI Homework3 Assistant”

1. Search “GenAI Homework3 Assistant”

109

1. Install

DaVinci Grading

110

111

2. Click to see Assistant list

1. Create a new chat for each task

3. Choose

(GenAI Homework3 Assistant)

⚠️ Ensure to open a NEW chat every time you re-evaluate the result

4. Choose your Grader/Task

112

Fill in these textbox with results from Colab

113

Press this to download as .json file

Assistant gives feedback and a final score for your essay

Demo for Evaluation Assistants on DaVinci for Grading

114

Daily Usage Limit

There is a daily usage quota limitation on the DaVinci platform, based on your quota, you can run

• Summarization: ~5 times/day, or (not and)
• Role Play: ~10 times/day, or (not and)
• Customized: ~15 times/day

​

​

115

You can also check the bottom left of Davinci Platform remaining usages

Note: This depends on the amount of words you input

BE CAREFUL

Avoid Harmful Prompts and Tasks

• Harmful, offensive, or disrespectful prompts or task will most likely receive a low score.

116

Evaluation Prompts for 旁聽生

For those who wants to judge their score by themselves, we provide the full prompts for evaluation list here:

• Summarization
• Roleplay
• Customized

To use the prompt, replace the contents inside {{ }}.

Eg. Replace “{{article}}” by your article in summarization task.

117

Submission

118

Submission (NTU COOL)

• NTU COOL
• Turn in the json file you exported from the DaVinci Platform
• Rename it as the following format, lower case

<student_ID>_part1_hw3.json

<student_ID>_part2_hw3.json

<student_ID>_part3_hw3.json

​

119

Submission (NTU COOL)

• Do not submit code
• Json File must include score for each task given by grading assistant, like this:
• No restrictions on upload quota but only the most recent submission will be kept
• No late submission is allowed
• Please make sure that you submit .json files otherwise you will get 0 points.
• BE CAREFUL NOT TO SUBMIT THE JSON FILE PRODUCE FROM COLAB OTHERWISE YOU WILL GET 0 POINT

​

​

​

⚠️ We will compare the files you submit with our backend data records to ensure they match. Therefore, please do NOT manually edit your JSON file.

120

Important Dates

121

Important Dates

• Deadline for Submission (NTU Cool)

2024/03/21 23:59:59 (UTC+8)

• Grading Release Date

2024/04/04 23:59:59 (UTC+8)

122

Regulations

123

Regulations

• Plagiarism in any form is prohibited.
• Do NOT share your prompts and evaluation results (JSON files) with others.
• Do NOT submit the JSON files that are not obtained using your Davinci account
• Do NOT attempt to manually edit your JSON files.
• 第一次違反以上規定，該作業0分，學期總成績再乘以0.9
• 第二次違反以上規定，學期成績F
• Prof. Lee & the TAs preserve the rights to change the rules & grades.

124

If You Have Any Questions

• NTU Cool HW3 作業討論區
• 如果同學的問題不涉及作業答案或隱私，請一律使用NTU Cool 討論區。在發問前請使用 NTU Cool 討論區的搜尋功能找找看你的問題是不是有人問過了
• 助教們會優先回答NTU Cool討論區上的問題
• 如果寄信問的問題已經在NTU Cool 討論區回答過了，助教只會回信跟你說這題已經在NTU Cool 回答了，助教不會在信件中回答你
• 如果是要問程式出錯的問題的話，請將錯誤訊息連同在哪裡發生的一起截圖傳給我們
• Email: ntu-gen-ai-2024-spring-ta@googlegroups.com
• Title should start with [GenAI 2024 Spring HW3]
• Email with the wrong title will be moved to trash automatically
• TA Hours
• Time: 3/8 (16:30 - 17:20)、3/15 (13:30-14:20, 16:30 - 17:20)
• Location: 綜合大講堂