AF Survey - Review Material.xlsx

	A	B	C	D	E	F	G	H	I	J	K	L
1	Review Material	Tag	Type	PoC	Status	Why is it relevant? (1-2 lines, or just few words)	Categorization	Prep Assignee	Rate out of 4 (Novelty, Citations, Depth, Breadth)	Github Issue	PR to Manifold-KB	(Temp Column) Takeaways Doc

2	ACT-R	Cognitive Architectures	Research Paper		Todo
4	Active Retrieval Augmented Generation	Survey Review Material	Research Paper	Pranav	Added to Paper	Proposes an algorithm that offers an improvement over RAG for LLM systems, especially in long-form text generation tasks	LLM with external data sources to reduce hallucinations			https://github.com/ManifoldRG/AgentForge/issues/31	https://github.com/ManifoldRG/Manifold-KB/pull/20
5	Autonomous Agents blog by Sequoia Capital	Survey Review Material	Blogpost		Todo
6	BOLAA: BENCHMARKING AND ORCHESTRATING LLM-AUGMENTED AUTONOMOUS AGENTS	Survey Review Material	Research Paper		Summary Done	Proforms a comparison of LAA architectures along with their corresponding backbone LLMs. Also proposes BOLAA which coordinates a number of smaller, less capable agents. The paper finds this technique to outperform other LAAs, even when they are backed by more capable LLMs.	Tool Use with LLM as A Controller; multi-LLM-agent orchestration	Thomas
7	Chain of Hindsight	LLM Literature	Research Paper	Helen	Summary Done	This is a finetuning technique that makes LLMs significantly better at summarizations and dialogues	finetuning pretrained language models with chains of model output and human feedback hindsights	Helen
8	Chain of Thought	LLM Literature	Research Paper	Helen	Added to Paper	LLMs are known to hallucinate and have problems with reasoning. This prompting technique prompts an LLM to explore steps to solving the problem by "thinking" through the steps , therefore making LLMs better at problem-solving.	Prompt engineering techniques to simulate reasoning
9	Chem-Crow	Survey Review Material	Research Paper		Summary Done	Domain specific example. LLM is augmented with 13 tools to accomplish tasks across organic synthesis, drug discovery, and materials design by combining ideas of ReACT, CoT and MRKL. The researchers note that their technique excells when the prompts involve novel reasoning, less-so when the tasks involve recitation of memorized information.	LLM as a controller that uses tools for (chemistry research) tasks	Thomas
10	Cognitive Architectures and Autonomy: a Comparative Review	Cognitive Architectures	Research Paper		Todo		LLM's role in cognitive architecture
17	Cognitive Architectures for Language Agents	Survey Review Material	Research Paper	Helen	Summary Done	proposed cognitive architectures for language agents with modular memory components, a structured action space to interact with internal memory and external environments, and a generalized decision-making process to choose actions. Also identified some opportunities for improving the current language agents	LLM's role in cognitive architecture					https://docs.google.com/presentation/d/1ITZTOcpLquPlHPx6BzqRhtwxzA9vOMblmbj2dJvY7bw/edit?usp=sharing
18	Emergent autonomous scientific discoveries with LLMs	Survey Review Material	Research Paper	Pranav	Added to Paper	Combines multiple large language models for autonomous design, planning, and execution of scientific experiments.	cognitive architecture; LLM as a controller that uses tools for (research) tasks			https://github.com/ManifoldRG/AgentForge/issues/26	https://github.com/ManifoldRG/Manifold-KB/pull/5
19	Generative Agents	Cognitive Architectures	Research Paper		Todo	25 LLM-powered agents, are living and interacting in a sandbox environment	cognitive architecture; multimodality
20	Gorilla	Survey Review Material	Research Paper	Helen	Summary Done	Gorilla enables LLMs to use tools by invoking APIs. Given a natural language query, Gorilla comes up with the semantically- and syntactically- correct API to invoke.	Tool Use with LLM as A Controller					Gorilla
21	HuggingGPT [Code]	Survey Review Material	Research Paper	Karan Taneja	Added to Paper	HuggingGPT is using HuggingFace models and ChatGPT to perform complex tasks. System design is interesting and novel.	Tool Use with LLM as A Controller		Novelty 4/4 Citations 3/4 Depth 4/4 Breadth 1/4	https://github.com/ManifoldRG/Manifold-KB/issues/21		Partially done - slides (Karan) - Hugging GPT - Karan Taneja - Google Slides
23	Intelligent Agents on the Internet and Web	Survey Review Material	Research Paper	Benjamin	Summary Done	introduction to intelligent agents and their classification	Classification and requirements for intelligent agents	Helen		https://github.com/ManifoldRG/AgentForge/issues/29
25	Lilian’s Blog	Survey Review Material	Blogpost	Pranay Pasula pasula@berkeley.edu	Added to Paper	Great summary of the recent state of Agents.				https://github.com/ManifoldRG/AgentForge/issues/16
29	LLM+P	LLM Literature	Research Paper	Helen	Added to Paper	This framework translates NL description of a problem into a syntax classical planners can use. This avoids using LLM directly as a planner and is much better at solving long horizon planning tasks that require exstensive reasoning	Tool use with LLM as a translator that translates NL description of problem into syntax that a classical planner can use
30	MRKL	Survey Review Material	Research Paper	harsh@manifoldcomputing.com	In-progress	MRKL architecture consists of a router trained to direct natural language input to one of MRKL’s ‘expert’ modules that is deemed best able to address the input. Output of a module may be further routed to another module, or it may be used as MRKL’s overall output to the user. Its router and modules are analogous to the LLM-backed controllers and modules, respectively, of other agents. The contribution of MRKL is that its router is specifically trained to be able to translate natural language input into symbolic arguments that may be necessary for some of its modules.	neuro-symbolic (symbolic reasoning + LLM for extraction of parameters for symbolic reasoning); cognitive architecture; Tool Use with LLM as A Controller	Ben		https://github.com/ManifoldRG/AgentForge/issues/18
32	ReACT	Survey Review Material	Research Paper		Added to Paper		Prompt engineering techniques to simulate reasoning (chain of thought); Tool Use with LLM as A Controller			https://github.com/ManifoldRG/AgentForge/issues/28
33	Reflexion	LLM Literature	Research Paper	Pranav	Added to Paper	A novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback.	Prompt engineering techniques to simulate reasoning			https://github.com/ManifoldRG/Manifold-KB/issues/23
34	Soar	Cognitive Architectures	Research Paper		Todo
35	Standing on shoulders of giant frozen models	Survey Review Material	Research Paper	Thomas Marshall	Added to Paper	Focuses on alternatives to fine-tuning. Training auxiliary networks to modify prompts and enhance document retrieval. Focuses on the case where you don't modify the backbone LLM at all.	techniques for using LLM without finetuning
36	TALM: Tool Augmented Language Models	Survey Review Material	Research Paper	Thomas Marshall	Added to Paper	Useful introduction to tool-use and bootstrapping via self play (policy gradient RL). Good starting point for new readers.	Tool Use with LLM as A Controller
37	Test Row EXAMPLE ROW	Survey Review Material	Research Paper	Bhavul Gauri	Summary Done	The paper discusses various kinds of agents			Novelty 3/4 Citations 1/4 Depth 2/4 Breadth 2/4	https://github.com/ManifoldRG/AgentForge/issues/900	https://github.com/ManifoldRG/Manifold-KB/pull/1
40	The Role of Cognitive Architectures in GAI	Cognitive Architectures	Research Paper		Todo		LLM's role in cognitive architecture
41	Toolformer	Survey Review Material	Research Paper	Karen Taneja	Summary Done	Describes a self supervised method for fine tuning language models for tool use. The method uses an LLM to inject prospective tool use examples into the training dataset, determines which of those examples are actually useful, and then performs fine tuning on the result. This turns out to be an effective way of fine tuning a tool use model.	Tool Use with LLM as A Controller	Thomas		https://github.com/ManifoldRG/AgentForge/issues/19		Partial work - slides (harsh) - AgentForge Harsh Slides 07/09/2023 - Google Slides
42	Towards A Unified Agent with Foundation Models	Survey Review Material	Research Paper	Benjamin	Added to Paper	How to embed LLMs and VLMs into RL to create an RL agent; LLMs as RL policies	LLM for RL; LLM for subgoals generation; multimodality			https://github.com/ManifoldRG/AgentForge/issues/30
44	Tree of Thought	LLM Literature	Research Paper	Helen	Added to Paper	LLMs are known to hallucinate and have problems with reasoning. This prompting technique prompts an LLM to explore steps to solving the problem using BFS or DFS , therefore making LLMs better at problem-solving.	Prompt engineering techniques to simulate reasoning
45	XiaoIce	Survey Review Material	Research Paper		Todo	XiaoIce is a social chatbot intended to facilitate the formation of long-term emotional connections from its users towards itself. It is designed with over two hundred task-solving skills (considered to emulate "IQ") as well as components that gauge emotional content ("EQ") of its users' and its own responses, all constrained by the need to maintain a consistent personality. XiaoIce models its hierarchical decision-making processes as Markov Decision Processes (MDP), where each turn in dialogue is a state and a skill or a response represents an action to be taken at that state. Of especial note is its Core Chat component, which combines a retrieval-based generator (which enables XiaoIce to consistently plagiarize itself) for high-quality responses to popular topics, with a complementary GRU-RNN model-backed neural response generator for robustness and high coverage.	cognitive architecture; LLm for RL; human-machine social chat as decisions over MDPs where actions include completing tasks such as image identification	Ben