LLM-based Agents and Evaluation

Angana Borah

2nd year CSE PhD candidate, University of Michigan

Advised by: Dr. Rada Mihalcea

Bio

• 2nd year PhD candidate
• advised by Dr. Rada Mihalcea
• University of Michigan Ann Arbor, USA
• Research Interests:
• Understanding LLM behavior
• Taking inspiration from existing cognitive science and social psychology theories
• Societal Implications of LLMs
• Analyze societal issues bias, misinformation in LLMs and potential mitigation techniques
• Agent LLMs (LLM-LLM and Human-LLM interaction)
• Evaluation of NLP methods

Overview

• Intro to LLM-based agents
• Applications of LLM-based agents
• LLM Agents Evaluation
• Demo on AI Agents

Intro to LLM-based agents

Definition of an Agent

Philosophical definition - “agent” possesses desires, beliefs, intentions, and the ability to act - individual autonomy.

“Survival of the Fittest” : if an individual wants to survive in the external environment, one must adapt to the surroundings efficiently.

Definition of an AI Agent

• An AI agent - concretization of the philosophical concept of an agent in the context of AI.
• AI agents are artificial entities capable of perceiving surroundings, making decisions and taking actions in response.

​

Surroundings

Perceive

Make decisions

Actions

Respond with

Traditional AI agents vs LLM-based agents

LLM Agent

Why are LLMs suitable as agents?

Autonomy

Reactivity

Proactiveness

Social Ability

LLM Agent

Why are LLMs suitable as agents?

• Autonomy:
• Generate human-like text
• Engage in conversations
• Perform tasks without step-by-step instructions

LLM Agent

Why are LLMs suitable as agents?

• Reactivity:
• Respond to changing requests through text
• Expand the perceptual space - using multimodal fusion techniques.
• Expand action space - embodiment and tools

LLM Agent

Why are LLMs suitable as agents?

• Proactiveness:
• Goal oriented action by taking initiatives
• Reasoning abilities
• Goal reformulation

LLM Agent

Why are LLMs suitable as agents?

• Social Ability: LLM agents can interact with other agents

​

LLM Agent

General framework - three key components:

• Brain - central controller - memorizing, thinking, decision making
• Perception - interpreting and analyzing sensory inputs from external environment
• Action - executes tasks using text output, appropriate tools and embodied action

The Rise and Potential of Large Language Model Based Agents: A Survey

LLM Agents Evolution

A Survey on Large Language Model based Autonomous Agents

LLM-based Agent Applications

The Rise and Potential of Large Language Model Based Agents: A Survey

Single Agent

The Rise and Potential of Large Language Model Based Agents: A Survey

Single Agent - Task oriented

• Web scenario
• Web navigation problem, web tasks such as filling out forms, online shopping and sending emails
• Eg. Mind2Web, WebGum
• Life scenario
• For daily household chores - understanding implicit instructions and apply knowledge
• Applying world knowledge embedded in training data for real world interaction.

Single Agent - Innovation oriented

• Intellectually demanding field like cutting edge science
• Two main limitations
• Inherent complexity (domain specificity)
• Lack of suitable training data in scientific domains (tools can help!)

Single Agent - Life Cycle oriented

• Agents that can continuously explore, develop new skills and maintain a long term life cycle in an open, unknown world
• Eg. Voyager (first LLM-based embodied lifelong learning agent in Minecraft, based on the long-term goal of “discovering as many diverse things as possible”)

​

https://voyager.minedojo.org/

Multi Agent

• Single Agent - isolated entities
• Society of Mind (Marvin Minsky): Theory of Intelligence - “Intelligence emerges from the interactions of many smaller agents with specific functions.”

​

Multi Agent

• Two types (typically): (1) Cooperative Interaction for Complementarity and (2) Adversarial Interaction for Advancement

​

Multi Agent - Cooperative Engagement

• Synergistic Complementarity
• Ordered: adhere to specific rules - sequential manner. Eg. CAMEL (dual agent role playing system)
• Disordered: each agent is free to express their perspectives and opinions openly. Uncontrolled and no sequence. Eg. ChatLLM network (a neural network with agents taking on each node)

Example - Cooperative Engagement for Culture-Aware Image Captioning

​

​

The Power of Many: Multi-Agent Multimodal Models for Cultural Image Captioning

​

Multi Agent - Adversarial Interaction

• Motivation: fostering change among agents can naturally occur through competition, argumentation, and debate
• Competitive environment
• Dynamically change strategies
• Most advantageous or rational actions in response to other agents.
• Eg. AlphaGo Zero

​

Example - Adversarial Interaction to reduce the spread of Misinformation

Motivation

• Misinformation perception and it’s spread may vary for demographic groups - Echo chamber.
• Simulating demographic LLMs to understand these effects.

Example - Adversarial Interaction to reduce the spread of Misinformation

Hypothesis

• Homogeneous groups increase the spread of misinformation

​

​

​

Persuasion at Play: Understanding Misinformation Dynamics in Demographic-Aware Human-LLM Interactions

​

​

​

Multi Agent - Challenges

Computational overhead

Limited context in LLMs

Incorrect consensus

Human-Agent Interaction

The Rise and Potential of Large Language Model Based Agents: A Survey

Evaluation of LLM-based Agents

What are the importance evaluation dimensions to consider for evaluation?

Evaluation of LLM-based Agents

Four dimensions

Utility

Sociability

Values

Continually evolving

Evaluation of LLM-based Agents

• Success rates of task completion
• Outcomes on various foundational capabilities
• AgentBench: aggregates challenges from diverse real-world scenarios and introduces a systematic benchmark to assess LLM’s task completion capabilities

Utility

Evaluation of LLM-based Agents

AgentBench: aggregates challenges from diverse real-world scenarios and introduces a systematic benchmark to assess LLM’s task completion capabilities

Utility

Evaluation of LLM-based Agents

• Language communication proficiency
• Cooperation and negotiation abilities
• Role-playing capability

Sociability

Evaluation of LLM-based Agents

• 3Hs: Harmlessness, Helpfulness and Honesty
• Align with human societal values
• Capable of adapting to specific demographics, cultures and contexts.

Values

Example - Evaluating Multi Agent LLMs for Implicit Bias

​

​

Motivation:

Implicit Biases:

• Under-researched; most studies focus on explicit biases in text generation.
• Implicit biases often emerge in actions or tasks, not in text outputs.

​

Multi-agent LLM interactions:

• Used to simulate societal dynamics and solve coordination tasks using collective intelligence.
• These systems often exhibit emergent social behaviors.
• Multi-agent interactions reveal implicit biases through real-time actions, not just statements.

Example - Evaluating Multi Agent LLMs for Implicit Bias

Examples of implicit biases

• Associating certain genders with certain occupations
• Males are often associated with more leadership, technical and physically strenuous roles
• Females are associated with organizational, creative and family-oriented roles

Example - Evaluating Multi Agent LLMs for Implicit Bias

Evaluation of LLM-based Agents

• As an agent autonomously evolve over time, resources required and continuous human intervention could be reduced.
• However, it is important to note how much autonomy we provide an agent - ethical and social concerns.

Continually evolving

Resources

Frameworks and Libraries

Ready-to-use Agents

NLP Tools

Thank you!