1 of 19

Agents

From ReAct to {insert most recent paper}

2 of 19

Agenda

  • What are agents?
  • Why use agents?
  • Typical implementation
  • ReAct
  • Challenges with agents
  • Memory
  • Recent papers/projects

3 of 19

What are agents?

  • Use LLM as the “reasoning engine”
  • Non-deterministic sequence of actions

4 of 19

Why use agents?

  • Connect LLM to outside sources of data/computation
    • Search
    • APIs
    • Database
    • Calculators
    • Run code
  • Recover from errors, handle multi-hop tasks

​

Example: SQL query

  • Chain: Natural Language -> SQl Query -> Result -> Natural Language
  • What if error in SQL Query?
  • What if multiple queries need to be made?

5 of 19

Typical Implementation

Pseudocode:

  • Choose tool to use
  • Observe output of the tool
  • Repeat until stopping condition met

​

Stopping condition:

  • LLM determined
  • Hardcoded rules

6 of 19

ReAct

7 of 19

Challenges with agents

  • Getting them to use tools in appropriate scenarios
  • Getting them to not use tools when not needed
  • Parsing LLM Output to tool invocation
  • Getting them to “remember” previous steps taken
  • Getting them to incorporate long observations
  • Getting them to stay focused
  • Evaluation

8 of 19

Challenge #1: Getting them to use tools in appropriate scenarios

  • Instructions
  • Tool descriptions in prompts
  • Tool retrieval before hand
  • Few shot examples
  • Fine tuned model (toolformer)

9 of 19

Challenge #2: Getting them to not use tools when not needed

  • Instructions
  • “Human Input” tool

10 of 19

Challenge #3: Parsing LLM Output to tool invocation

  • JSON/Typescript schema
  • Retrying/fixing logic
  • Output Parsers!

11 of 19

Challenge #4: Getting them to “remember” previous steps taken

  • List of steps in memory
  • Fetch previous steps with retrieval

12 of 19

Challenge #5: Getting them to incorporate long observations

  • Parsing the long output
  • Storing long output and doing retrieval

13 of 19

Challenge #6: Getting them to stay focused

  • Reiterate the objective
  • Separate planning/execution step

14 of 19

Challenge #7: Evaluation

  • Evaluate end result
  • Evaluate intermediate steps
    • Correct action
    • Correct action input
    • Correct sequence of steps
    • Most efficient sequence of steps

15 of 19

Memory

  • Memory of User <> AI interactions
  • Memory of AI <> Tool interactions
  • Used to achieve personalization

16 of 19

AutoGPT

  • Long term memory of Agent <> Tool interactions

​

https://github.com/Significant-Gravitas/Auto-GPT

17 of 19

BabyAGI

  • Long term memory of Agent <> Tool interactions
  • Separate planning / execution steps

​

https://github.com/yoheinakajima/babyagi

18 of 19

CAMEL

  • Simulation environment

​

https://www.camel-ai.org/

19 of 19

Generative Agents

  • Memory
    • Time weighted
    • Importance weighted
    • Relevancy weighted
  • Reflection step
  • Simulation environment

​

https://arxiv.org/abs/2304.03442