Structured Output

Structured Output

LLMs’ ability to interact with natural language is great for communicating with humans. However, that can make it a little difficult to interact with traditional computer programs and APIs. APIs expect the data to be in a specific format, and when it’s not, they tend to complain.

The good news is that with a little coaxing, LLMs can be prompted to generate data in a standard format. The output is usually fairly reliable. LangChain is also there to help by providing the with_structured_output method on supported LLMs.

First, you provide either a Pydantic model, TypedDict class or JSON schema in the structure that you want the data to follow. Some advantages of choosing a Pydantic model are its support for data validation and JSON serialization. Here’s an example:

from langchain_core.pydantic_v1 import BaseModel, Field

class Person(BaseModel):
  """Profile of a human."""

  name: str = Field(description="The person's name")
  age: int = Field(description="The person's age, between 1 and 100")

Once you have the data structure, you prompt the LLM to follow it like so:

structured_llm = llm.with_structured_output(Person)
structured_llm.invoke("Create a random character for a story")

The output is a Pydantic object that might look something like this:

Person(name='Gandalf', age=100)

The process for creating TypedDicts and JSON schemas is similar. Use good descriptions, and that’ll go a long way in helping the model out.