1 of 33

Grounding & Research

July 6, 2023

bit.ly/mui-asdrp

ETHICS

2 of 33

AGENDA

Logistics

Prompting (review)

Retrieval Augmented Grounding

Research Proposals

Next week

3 of 33

Logistics

  • Github: github.com/philmui/asdrp2023
  • Website: bit.ly/mui-asdrp
  • Slack Workspace: “ASDRP 2022-2023 Algorithms for Good”
  • Weekly research all-hands meeting
  • Intra-week : individual group research meeting (starting in 2 weeks)
  • Group research timeline
  • July 8 (S) : submit 1-pager research proposal
  • July 29 (S) : submit initial data analysis
  • July 30-Aug 13: group presentation of results
  • mid-August : draft research paper
  • mid-August : ASDRP symposium presentation
  • September : finalize paper for publication

​

4 of 33

Milestones

  • July 8 (S) : submit 1-pager research proposal
  • July 29 (S) : submit initial data analysis
  • July 30-Aug 13: group presentation of results
  • mid-August : draft research paper
  • mid-August : ASDRP symposium presentation

​

5 of 33

Proposal Spreadsheet

6 of 33

AGENDA

Logistics

Prompting (review)

Retrieval Augmented Grounding

Research Proposals

Next week

7 of 33

Stochastic Parrots

“The ersatz fluency and coherence of LMs raises several risks, precisely because humans are prepared to interpret strings belonging to languages they speak as meaningful and corresponding to the communicative intent of some individual or group of individuals who have accountability for what is said.”

8 of 33

Problem Statement : An Alignment Problem

“Large language models (LMs) can be ‘prompted’ to perform a range of natural language processing (NLP) tasks, given some examples of the task as input. However, these models often express unintended behaviors such as making up facts, generating biased or toxic text, or simply not following user instructions (Bender et al., 2021; Bommasani et al., 2021; Kenton et al., 2021; Weidinger et al., 2021; Tamkin et al., 2021; Gehman et al., 2020).”

​

(OpenAI) Ouyang, et al, 2022

9 of 33

Instruction Tuning + Human Feedback : InstructGPT (March 2022)

10 of 33

Zero-Shot Prompting

11 of 33

Few-Shot Prompting

12 of 33

Chain-of-Thought (CoT)

13 of 33

Zero-shot Chain-of-Thought (CoT)

14 of 33

Automatic Chain-of-Thought (Auto-CoT)

15 of 33

Self-Consistency

16 of 33

Tree of Thoughts (ToT)

17 of 33

Tree of Thoughts (ToT)

18 of 33

Tree of Thoughts (ToT)

19 of 33

Tree of Thoughts (ToT)

20 of 33

Single Prompt ToT (2023)

21 of 33

AGENDA

Logistics

Prompting (review)

Retrieval Augmented Grounding

Research Proposals

Next week

22 of 33

Chatbot

LLM

LLM

LLM

response

What are the main factors for a nation to win a war?

23 of 33

Contextualized prompt with the top-k documents

Chatbot

LLM

LLM

LLM

response

Embedding

What are the main factors for a nation to win a war?

24 of 33

Chunk 1

Chunk 2

Chunk 3

Chunk 4

Chunk N

…

Embedding 1

Embedding 2

Embedding 3

Embedding 4

Embedding N

…

Vector database

Contextualized prompt with the top-k documents

Chatbot

LLM

LLM

LLM

response

Embedding

What are the main factors for a nation to win a war?

25 of 33

Retrieval Augmented Generation (Grounding) RAG

Chunk 1

Chunk 2

Chunk 3

Chunk 4

Chunk N

…

Embedding 1

Embedding 2

Embedding 3

Embedding 4

Embedding N

…

Top “2” semantically related document chunks

Vector database

Contextualized prompt with the top-k documents

Chatbot

LLM

LLM

LLM

response

Embedding

What are the main factors for a nation to win a war?

26 of 33

AGENDA

Logistics

Prompting (review)

Retrieval Augmented Grounding

Research Proposals

Next week

27 of 33

Proposal Spreadsheet

28 of 33

AGENDA

Logistics

Prompting (review)

Retrieval Augmented Grounding

Research Proposals

Next week

29 of 33

Milestones

  • July 8 (S) : submit 1-pager research proposal
  • July 29 (S) : submit initial data analysis
  • July 30-Aug 13: group presentation of results
  • mid-August : draft research paper
  • mid-August : ASDRP symposium presentation

​

30 of 33

Proposal Guidance

  • Research Title: <title>
  • Team members: <who>
  • Problem Statement: <what>
  • Why is this an interesting problem? <why>
  • What key techniques do you plan to use to approach this problem? <how>
  • Data Sources (that you can find so far):
  • Relevant references:

31 of 33

“Can you assign me to a research group?”

“You're not in Kansas anymore.”

32 of 33

This Sunday

  • We will go over various prompting techniques
  • Read papers from our website

​

33 of 33

Research Projects

  • Existing Projects
    1. GPT Biases (RishiS, HaydenF, …)
    2. Political Polarity across US Colleges (NLP) (TheodoreM, AngelaL, …)
    3. Housing Discrimination (AditiG, AlexanderS, …)
    4. TriageTrial: Hospital Triaging (PatrickT, AayaanS, HireshP, …)�
  • New Projects
    • FairFake: Biases in Facial Emotion Detection (MihikaD, AadrijU)
    • Molecular Visualization with AR
    • (many more to be created by you)