Decision Making & AI Agent Architecture

Decision Making & AI Agent Architecture

Asking a chatbot like ChatGPT a question and getting a text response is easy. Instructing an AI Agent to perform a simple task like executing a function is also easy. But what if the chatbot needs more information? Or what if the agent gets an instruction that the function can’t handle? All these situations require flexible decision-making capabilities at runtime. Being able to make decisions is one of the key capabilities that an AI Agent needs to handle complex workflows.

In this segment, you’ll learn how to give your LangGraph agent the ability to make decisions. You’ll also learn about a few different architectures an agent can take.

Decision Making

You dealt with the issue of decision-making in Lesson 1 before you ever learned about LangGraph. Programming languages make decisions with if statements:

if response == "do_something":
  do_something()
else:
  raise ValueError("error")

The power of LLMs is that you can prompt them to return a structured output such as "do_something" in response to certain conditions in the user input. You also learned that an AI Agent is essentially an LLM with the power to call a function. Having the power to take an action but not being forced to implies the ability to make a decision.

LangGraph also has these features baked in. In this section, you’ll learn how LangGraph allows agents to make decisions, and in the next segment, you’ll learn how LangGraph allows function calling through the use of what it calls “tools”.

You’ll recall that LangGraph uses nodes as a unit to represent some tasks in the workflow. The nodes are usually just wrappers for Python functions. When one node finishes its task, it must know what to do next. That’s where edges come in. In LangGraph, an edge connects two nodes to indicate what happens after one task is finished. With normal edges, there’s only one route to leave a node:

Decision-making, on the other hand, implies that there are at least two routes to go after a task finishes. In this case, a single node has two edges leaving from it:

These are known as conditional edges in LangGraph. The add_conditional_edges method points to two or more nodes and gives a function to tell LangGraph how to choose. You can see this in the following code block:

graph.add_conditional_edges(
  "node_1", 
  my_routing_function,
  {
    True: "node_2", 
    False: "node_3"
  }
)

In the code snippet above, node_1 has conditional edges to both node_2 and node_3. The output of node_1 is passed to the routing function. If my_routing_function returns True, this is mapped to node_2, so node_2 will execute next. Otherwise, node_3 will execute.

Note: This function’s path_map parameter (third parameter) is optional. You can leave off the dictionary if your routing function returns a node name. However, specifying it makes the routes more clear.

Looping

Another control-flow concept related to decision-making is looping. Imagine a situation where you create an essay-writing agent. You might have one node write the first draft. Then, the output is passed to a checker node. If the checker node approves the content, the workflow is finished. But if not, the checker sends feedback to a reviser node that revises the content. When the reviser is finished, the output goes back to the checker. This continues in a loop until the checker finally decides to pass it. The following diagram shows that architecture:

In code, you’d construct the graph like this:

graph.add_edge(START, "writer")
graph.add_edge("writer", "checker")
graph.add_edge("reviser", "checker")

graph.add_conditional_edges(
  "checker", 
  check,
  {
    "fail": "reviser", 
    "pass": END
  }
)

The checker node provides feedback, whereas the check function serves as the router pointing to the next node.

AI Agent Architectures

Once you can branch and loop, the sky is the limit for how you set up your agent architecture. The following sections describe a few architectures that others have proposed. This is certainly not an exhaustive list. Use them to inspire your own architectural designs when building AI agent systems.

Reflection

The writer-reviser example above is an example of basic reflection. One node generates a result, and another reflects on its quality, sending feedback to the generating node, which then regenerates another response based on that feedback. This continues x number of times or until a certain quality level is achieved.

There are additional variations on this theme including Reflexion, Language Agent Tree Search, and Self-Discover Agent.

Planning

Planning agents take a complex task and break it down into smaller subtasks that are easier to solve. Once you have the subtasks, another agent can solve each one at a time.

An example prompt taken from the LangGraph documentation will help you understand how you’d prepare the planning agent to generate tasks:

For the given objective, come up with a simple step by step plan.
This plan should involve individual tasks, 
that if executed correctly will yield the correct answer.
Do not add any superfluous steps.
The result of the final step should be the final answer. 
Make sure that each step has all the information needed - do not skip steps.

Variations on the architecture include Reasoning Without Observation (ReWOO), where the Single-Task Agent completes all the tasks before the evaluation phase, and Plan-and-Execute, where the task list is updated every time the Single-Task Agent finishes a task.

Multi-Agent Systems

Several different architectures involve more than one agent. You’ll find a few notable ones below.

Collaboration

In a Collaboration architecture, you have different agents that are experts at different things. For example, if the overall task is building a data analysis pipeline, you could have expert agents in data cleaning, statistical analysis, and data visualization.

Another multi-agent architecture is where one agent is a supervisor that directs the activities of other agents:

A variation on the supervisor agent architecture is to further divide the work hierarchically. This is useful when a subtask is still too complex to be carried out by a single agent. For this architecture, you have a single top-level supervisor that manages mid-level supervisors that in turn oversee still more specialized agents.