Making Improvements to AI Agents

Once you know your performance metrics and how to monitor them, you’ll be in a good position to optimize your AI agent.

Optimizing Prompts

If you aren’t getting the desired output from an LLM, one solution might be to improve your prompt. Study different prompt engineering techniques to guide the LLM toward better answers. Some techniques you should be familiar with are:

• Clarity: Be specific and provide all the relevant details and context.
• Chain-of-Thought: Tell the LLM to break the task down and solve it step by step.
• Few-shot: Provide the LLM with several examples of the output you want for a given input.

Prompt writing isn’t an exact science. You should iteratively modify your prompt to discover what produces the best results. And if you change your LLM, you’ll need to re-evaluate your prompts.

Optimizing Efficiency

You can do many things to improve the efficiency of your AI agents in terms of time and resources.

Time

One way to speed up an agent is to perform tasks in parallel rather than sequentially. For example, if two nodes both need to make an LLM call and neither depends on the result of the other, then this is a good candidate for running them in parallel.

LangGraph supports both sequential and parallel execution. It just depends on how you build your graph. In the image below, the graph is set up to execute sequentially.

However, the graph in this next diagram shows nodes B and C running in parallel.

Branching isn’t limited to conditional edges. You can also set multiple normal edges that “fan out” from a given node. Then, they can “fan in” to a single node where the values are combined according to a reducer function. You can read more about this in the LangGraph branching documentation.

Another trick to get an agent to respond faster is to stream the tokens from the LLM rather than waiting for a request to complete before presenting it to the user. LangGraph supports this with astream_events. Read more about this in the streaming tokens documentation.

Resources

When you think about optimizing resources, consider how you can reduce token usage. Do you need to send the entire message conversation on every request? Probably not. You noticed in the Lesson 3 and 4 demos that when the screenshot image was converted to Base64, it was a massive text string. The tutorial didn’t have you send the entire message list to the LLM because you wouldn’t upload all those tokens on every request. You only needed the screenshot image when you were generating the contextual comments. After you had those, you no longer needed the image.

Even if you do need to retain a memory of the chat history, there are tricks to cut down on token usage. For example, when the message list grows over a certain length, you can ask the LLM to summarize the chat history. Then, in future requests, you can drop the old messages and just include the summary. You’ll find this example in the persistence documentation.

In addition to decreasing the number of tokens you use, you can also experiment with different models. The less powerful models are cheaper but are still quite good at producing natural-sounding text and answering basic questions. Because of this, you may be able to maintain the quality of your agent while decreasing its cost by using a more powerful model for complex reasoning tasks while using a cheaper model for simple tasks.

Note: While optimization and minimizing cost are important, don’t worry if your agent consumes a lot of tokens. As mentioned previously, the cost of LLMs is on a downward trend. Things that are expensive today may be affordable tomorrow. And even if you continuously streamed tokens from an LLM provider, you’d probably still pay less per hour than you would for a human.

Optimizing UX

Step back occasionally and ask yourself what would make the entire experience better for the end user. Perhaps you need to re-architect how the application works. Perhaps you need to use a more powerful model or a better text-to-speech engine. Maybe you need to work on decreasing latency. Don’t be afraid to make big changes or even start over from scratch if your current implementation isn’t working.

You also need to accept the limitations of the technology and the current models. LLMs still haven’t reached the level of humans, so part of optimizing your agentic workflow might be to add more human-in-the-loop interactions.