Building AI Agents

In some interviews with tech gurus in Silicon Valley, they often mention that AI Agents are "the road to AGI" (The road to AGI).

Earlier, I had created a presentation deck about AI Agents. On the 20th, Anthropic published an article titled "Building Effective Agents" (https://www.anthropic.com/research/building-effective-agents). Today, I will combine my previous knowledge with this article to outline a guide for constructing AI Agents.

Concept explanation

The image below is a screenshot from a paper by Li Feifei and the Microsoft Lab team this year, which describes the basic concepts and framework of AI Agents. The following is the reference link: https://www.microsoft.com/en-us/research/project/agent-ai/

Comparison: humans vs. LLM-based intelligent agents

Source:https://arxiv.org/pdf/2309.07864

, but architecturally distinguishes the two importantly:

• Workflows：

• coordinates the execution of LLMs (Large Language Models) and tools.
• Its behavior is predictable and constrained by established logic.

• Intelligent agents (Agents)：

• and decide how to use tools to complete tasks.
• Intelligent agents possess greater flexibility and adaptability, with higher control over the way tasks are executed.

When to use Agents

When building applications using LLMs, it is recommended to opt for the simplest solution whenever possible, adding complexity only when necessary. This means that sometimes it may not even be necessary to build an intelligent agent system. Intelligent agent systems typically trade off higher latency and cost for better task performance, so these trade-offs need to be balanced according to actual requirements.

1. Prioritize simple solutions

• For most applications, optimizing a single LLM call, combining it with retrieval mechanisms or contextual examples, is usually sufficient.

• When tasks are clear and rules are fixed, workflows (Workflow) provide higher predictability and consistency.

• is a better choice.

Implementation method

The following content is organized based on a figure titled "A Basic AI Agent" that I saw last year, source: https://lilianweng.github.io/posts/2023-06-23-agent/

1. Agent (Intelligent Agent)

The core part is a large language model (LLM), combined with the following key capabilities:

• : Develop action strategies based on current goals and the environment.
• : Continuously adjust strategies and improve behavior based on execution results.
• ：Accomplish specific tasks with the help of various tools.

2. Memory (Memory Ability)

Memory helps intelligent agents work smarter and achieve recursive optimization of tasks.

• Short-term Memory:
• Store the current context, usually depending on the token window size of LLMs.
• Long-term Memory:
• : Extract relevant information from the knowledge base through a retrieval mechanism.
• : Record long-term interaction logs to provide references for future planning or reflection.

The function of memory:

1. : Optimize the current strategy based on past experiences.
2. : Identify mistakes and summarize successful experiences.

3. Planning (Planning Module)

Planning capabilities are divided into the following sub-functions:

• : Check the rationality of goals and execution strategies.
• : Actively identify issues in actions or plans.
• : Solve complex problems through step-by-step reasoning.
• : Break down complex tasks into executable subtasks.

WithAI Werewolfas an example (previously shared:Using LLM's Agent to play Werewolf), through continuous optimization of various modules in Planning, LLM can better understand the game logic and strategies, and perform more intelligently.
4. Tools (Tools)

The tool extends the capabilities of intelligent agents, enabling them to solve problems beyond the inherent abilities of LLMs:

• : Perform mathematical operations.
• : Analyze and run code, handling complex programming tasks.
• : Obtain real-time information or verify existing knowledge.
• : Obtain results by calling external services.

：Enable intelligent agents to act more efficiently in the real world.

5. Action（Action）

The ways intelligent agents act include:

1. ：Execute strategies according to a plan.
2. : Tools serve as an "extension" of intelligent agents, helping to accomplish complex or technical tasks.

Reflection

Reflection is a crucial part of intelligent agents. By reflecting on and adjusting executed behaviors, the system's intelligence level and adaptability can be improved. The following content is explained in conjunction with https://lilianweng.github.io/posts/2023-06-23-agent/.

Core structure and functions

1. Self-reflection (Reflexive Module, LM)

• Self-checking: Analyzing the accuracy and efficiency of its own execution.
• Metacognition: Improving execution strategies and long-term decision-making through "reflection."
• By analyzing external and internal feedback, generating reflective text to provide adjustment references for subsequent behaviors.
• The reflexive ability allows intelligent agents to evaluate their own behaviors and decision-making logic.
• Function:
• Characteristics:

• Rapidly adapt to dynamic changes in the environment.
• Provides contextual data during action execution.
• Stores recent observation (Obs) and reward (Reward) trajectories, providing real-time references for the evaluator and behavior modules.
• Function:
• Characteristics:

• Supports dynamically adjustable strategies.
• Interacts with the reflex module, providing data support for long-term improvements.
• Analyzes trajectories in short-term memory and generates internal feedback by combining external feedback.
• Ensures that actions align with objectives, identifying potential issues and areas for optimization.
• Function:
• Characteristics:

• Dynamic behavioral adjustment, capable of adapting to environmental feedback.
• Directly impacts the environment (Environment).
• Executes specific actions based on plans and feedback.
• Function:
• Characteristics:

• Supports the accumulation of knowledge and generalization capabilities across tasks.
• Store past experiences and reflective texts to provide historical references for future decision-making and actions.
• Function:
• Characteristics:

System Characteristics

1. Self-examination Capability

• Dynamic optimization of behavior through reflection and evaluation modules.
• Able to discover issues in real-time during execution.

• Adjust the strategies and actions of behavior modules based on internal and external feedback.

• Can adjust plans and actions according to environmental changes, demonstrating strong flexibility.

• Reflexive modules and evaluator support system self-debugging, reducing maintenance costs.

Tools

Let's also elaborate on the tools, which mainly draw from the experience of Anthropic. Regardless of what kind of intelligent agent system is being built, tools are often a critical component of the agent. Tools enable Claude to interact with external services and APIs, defining its structure and functionality through API calls. When Claude decides to invoke a tool, it will include a tool call block (Tool Use Block) in the API response. Similar to overall prompt engineering, the definition and specification of tools also require careful design.

Suggestions for designing tool formats

When specifying tools, there are often multiple ways to accomplish the same operation. For example:

• files or rewriting the entire file.
• ：You can choose to nest the code in Markdown or JSON.

Although from a software engineering perspective, these differences are merely superficial and can be converted losslessly, for LLMs, the difficulty of writing in different formats varies significantly. For example:

• files requires pre-calculating the number of lines that need to be changed before adding new code.
• Writing code in JSON format requires handling additional escape characters (such as newline characters and quotes).

Here are some suggestions for selecting tool formats:

1. Reserve sufficient Token space

• Ensure the model has enough "thinking" space before generating code to avoid getting stuck in logical dead ends.

• Prioritize formats that the model commonly encounters on the internet to increase its familiarity.

• Prevent the model from requiring extra computation or processing complex formats, such as large-scale row counting or string escaping.

Enhance the best practices in tool design

Just as human-computer interaction (HCI) requires significant design investment, the interaction interface between intelligent agents and tools (ACI) also demands equal attention. The following are some specific suggestions:

1. Think from the model's perspective

• Are the descriptions and parameters of the tool intuitive? Does the model require significant effort to understand?
• Provide clear usage examples, boundary cases, input format requirements, and explicit distinctions between this tool and others.

2. Optimize parameter names and descriptions

• Design parameter names and descriptions to be more intuitive, as if writing excellent documentation comments (docstrings) for a new developer on your team.
• This is especially important when using multiple similar tools.

3. Testing and Iteration

• Run multiple example inputs in the workbench, observe the errors the model makes when using tools, and continuously improve the design.

4. Error-Proofing Design (Poka-Yoke)

• Adjust the parameters and interfaces of the tool to make it harder for usage errors to occur.
• Example: In Anthropic's SWE-bench agent, the model was prone to errors when using relative paths after leaving the root directory. To address this issue, Anthropic designed the tool to only accept absolute paths, resulting in perfect performance by the model when using this method.

By optimizing tool design, intelligent agents can better accomplish complex tasks. For example, in the implementation of SWE-bench, Anthropic spent more time optimizing the tools rather than the overall prompt. This investment not only improved the reliability of the tools but also enhanced the usability and accuracy of the entire system.

Develop Agents

Optional Frameworks

There are many frameworks that can help developers more easily implement intelligent agent systems, including:

1. Providing a modular toolchain that supports the combination of functions for language models, a series of articles have been written about this previously:

2. Build and deploy intelligent agents through a unified interface.

3. A drag-and-drop GUI LLM workflow builder.

4. A GUI tool that supports building and testing complex workflows.

These frameworks simplify standardized low-level tasks, such as calling LLMs, defining and parsing tools, chaining calls, etc., allowing developers to get started quickly. However, they also introduce an additional layer of abstraction that may obscure the underlying prompt and response logic, increasing the difficulty of debugging. Moreover, these frameworks may lead developers to add unnecessary complexity when a simple setup is often sufficient.

Development Suggestions

1. Start with directly using the LLM API

• Many patterns can be implemented directly with a small amount of code without relying on a framework.

• If you use a framework, make sure you understand its underlying code and how it works.
• Incorrect assumptions are a common source of many customer issues.

Building Block: Augmented LLM

is the building block of intelligent agent systems. By integrating enhancements such as retrieval, tools, and memory, LLMs can proactively generate search queries, select appropriate tools, and decide what information needs to be retained.

1. Retrieval

• Integrate knowledge bases or real-time data sources to provide the most up-to-date and relevant contextual information for tasks.
• Example: Enhance the accuracy of answering complex questions through a retrieval mechanism.

• Expand the capability range of LLMs, enabling them to accomplish complex tasks.
• Example: Calling a calculator for mathematical calculations or using an API to retrieve real-time information.

• Supports short-term and long-term memory, which is used to optimize interactions and task execution.
• Example: Saving task context to assist with contextual understanding in multi-round conversations.

Focus on the following two aspects when implementing enhanced LLMs:

1. Customization capability

• Adjust and enhance features according to specific application scenarios, making them more aligned with business needs.
• Ensure that the system design meets task objectives while avoiding unnecessary complexity.

• Provide clear and user-friendly interfaces for LLMs so that developers can quickly understand and utilize the enhanced features.
• Ensure that all functions are documented in detail to facilitate debugging and maintenance.

The enhanced LLM lays a solid foundation for the intelligent agent system, allowing developers to flexibly extend these capabilities according to task requirements, thereby achieving efficient applications in a wider range of scenarios.

Compositional Workflows

Below are some common workflows in LLM applications that help choose the appropriate implementation based on task requirements.

1. Prompt Chaining (Prompt Chain)

Break down the task into multiple steps, where each LLM call processes the output of the previous step, and checkpoints (Gates) can be set at intermediate steps to ensure the process is correct.

Applicable scenarios

• The task can be clearly broken down into fixed subtasks.
• Prioritize improving accuracy over achieving the lowest latency.

Examples

• : Generate marketing copy and then translate it into another language.
• : Write an outline first, check it for approval, and then write the content based on the outline.

2. Routing (路由)

By classifying the input,分流 tasks to different subsequent processing paths or tools, suitable for task categories that require specialized processing.

Applicable scenarios

• Task categories are clear, and the classification results have high accuracy.
• Each type of task requires a unique processing method.

Example

• : Categorize the handling of common questions, refund requests, and technical support.
• : Assign simple questions to small models and complex questions to advanced models.

3. Parallelization

Run multiple tasks simultaneously and aggregate the results at the end.

Divided into two methods:

• : Break down tasks into independent subtasks for parallel processing.
• ：Run the same task multiple times to generate diverse outputs.

Applicable scenarios

• Subtasks can be processed in parallel to save time.
• Multiple perspectives are needed to improve the accuracy of results.

Example

• : One model handles user queries, while the other is responsible for filtering inappropriate content.
• : Repeatedly review code vulnerabilities to ensure higher detection reliability.

4. Orchestrator-Workers (Coordinator-Workers)

A central LLM dynamically decomposes tasks, assigns them to multiple sub-LLMs for execution, and integrates the results. Unlike parallelization, subtasks are dynamically determined by the input.

Applicable scenarios

• Complex tasks where subtasks cannot be predefined.
• Tasks that require dynamic adjustment of processes.

Examples

• : Dynamically modify the contents of multiple files based on requirements.
• : Integrate information from multiple sources and analyze relevant content.

5. Evaluator-Optimizer

One LLM generates results, another LLM evaluates and provides feedback, and the cycle continues until satisfaction is achieved.

Applicable scenarios

• Tasks have clear evaluation criteria, and iterative optimization leads to significant improvements.
• LLM can generate valuable feedback and make improvements.

Examples

• : After translating the LLM output, the evaluator provides suggestions for improvement.
• : Multi-round search and analysis to ensure comprehensive information.

The above workflow provides structured solutions for different scenarios, helping to find a balance between task complexity, accuracy, and performance.

Autonomous Agents

are beginning to emerge in production environments.

Way of working

1. Task Initiation

• The agent's task begins with a user command or interactive discussion with the user.
• Once the task is clear, the agent independently plans and executes it, while possibly returning to the user for more information or judgment when needed.

• During the task execution, the agent needs to obtain "real data" (Ground Truth) from the environment through tool call results, code execution feedback, etc., to evaluate the progress of the task.
• The agent can pause at critical points or when encountering obstacles, and request feedback from the user.

• Tasks generally end after completion, but stop conditions (e.g., maximum number of iterations) can also be set to maintain control.

Characteristics and implementation

• The agent can handle complex tasks, but the implementation is usually relatively simple, mainly consisting of LLM calling tools according to the environment in a feedback loop.

• Design focus

• Clear design of the toolkit and its documentation is critical, ensuring that the agent can correctly understand and use the tools.
• See best practices in the section above on tools (“Prompt Engineering your Tools”).

Application scenarios for Autonomous Agents

Applicable scenarios

• When facing open-ended questions, it is difficult to predict the required steps, and a fixed path cannot be hard-coded.
• When the agent needs to perform multiple rounds of operations, and there is a certain level of trust in its decision-making.
• Autonomous agents are particularly suitable for scaling large-scale tasks when executing tasks in a trusted environment.

Precautions

• The high cost of autonomy and the risk of error accumulation.
• It is strongly recommended to conduct extensive testing in a sandbox environment and set up appropriate guardrails.

For example 🌰

Below are two examples from Anthropic's actual implementation, demonstrating practical use cases for intelligent agents:

1. Coding Agent

• : Solve SWE-bench tasks. These tasks require modifying multiple files based on the task descriptions.
• : The intelligent agent analyzes the codebase according to the task description, plans the necessary file changes, executes the tasks step by step, and dynamically adjusts strategies to meet complex code editing requirements.

• : Let Claude use a computer to complete tasks.
• : Through means such as tool invocation and environmental interaction, the agent can operate in real computational environments, such as executing commands, retrieving data, or completing complex computational tasks.

The agent demonstrates strong flexibility and scalability through autonomous planning and feedback mechanisms, making it an important tool for handling complex tasks. However, it also requires careful design and rigorous testing to ensure its stability and reliability.

History of Agents Development

The concept and technology of AI Agents have evolved over many years, gradually transitioning from theory to practical application. The following content provides a brief review based on https://arxiv.org/pdf/2308.11432:

AI Agent Applications

Below are some cases and demonstration content: https://x.com/omooretweets/status/1740774601876177375. I have also shared this Landscape before (

As of 2023, the applications of AI Agents had already covered multiple fields, showcasing their strong generalization capabilities and wide applicability. In 2024, with the maturation of technology, the application scenarios of AI Agents became even more diversified, which I will organize later.

Recently, Anthropic also provided examples for this year:

Through collaboration with clients, Anthropic has identified two categories of use cases that are particularly suitable for AI Agents. These scenarios demonstrate the practical value intelligent agents can create in tasks that combine dialogue and action, have clear success criteria, enable feedback loops, and incorporate effective human oversight.

A. Customer Support (Customer Support)

I heard this before in a Sequoia Capital sharing session, where they also mentioned that customer service and legal fields are the most suitable scenarios (

Customer support combines the familiar chatbot interface with tool integration enhancements, providing a natural fit for more open-ended intelligent agent applications:

1. Natural conversation flow

• Interactions typically follow a conversational flow but also require access to external information and the ability to take real-world actions.

• Callable tools for retrieving customer data, order history, and knowledge base articles.

• Perform tasks such as refunds via programming, updating work orders, etc.

• Success is measured by user-defined resolution criteria (such as issue resolution or task completion).

Several companies have validated the feasibility of this approach through usage-based pricing models, charging only for successfully resolved issues, demonstrating confidence in the effectiveness of their agents.

B. Coding Agents (代码代理)

The field of software development has demonstrated the enormous potential of LLM capabilities, which have evolved from code completion to autonomous problem-solving. Coding agents excel in the following areas:

1. Verifiable Solutions

• Code solutions can verify correctness through automated testing.

• The agent can iteratively improve the code based on the test results.

• The problem space in software development is usually structured and clearly defined.

• The output quality can be objectively evaluated through functional testing.

In Anthropic's implementation, the agent has been able to address actual GitHub issues from the SWE-bench Verified benchmark based on pull request descriptions. While automated tests can verify functionality, human review remains critical to ensuring that solutions meet broader system requirements.