Introducing Guided Generation

Introducing Guided Generation

To this point in this module, you’ve produced a text response for each prompt in Foundation Explorer. Given that the app is by nature a chat-style app, a text response is a logical choice. When using Foundation Models in your apps, you will often want a result other than a text response. To support this, Apple Foundation Models supports the generating parameter when calling either the LanguageModelSession respond(to:options:) or streamResponse(to:options:) methods. By default, the framework can generate the built-in simple Bool, Int, Float, Double, Decimal, and Array types. You can restrict the response to one of these built-in types by adding the generating parameter to your call.

Open the starter project for this lesson. You’ll see a few changes from the final project at the end of lesson two. The most noticeable change is the addition of a new menu to the leading side of the top toolbar. If you tap it, you will see two entries. The first presents a new view that shows the transcript of the current session. You’ll use this as a learning and troubleshooting aid during this lesson. You will look at the Dining Menu option later in this lesson.

Run the starter app and enter the following prompt.

How many millimeters are in an inch?

This should yield the correct response of 25.4 millimeters in one inch.

While that is useful for us, any app using that prompt likely only cares about the number. To get just the response, you could change the call to produce a Float instead of text. The code to do this would resemble:

let prompt = "How many millimeters are in an inch?"
let response = try await session.respond(to: prompt, generating: Float.self)
let mmInInch = response.content

Note that response.content and therefore mmInInch will be a Float since you passed Float.self to the generating parameter. If you try to do this with responses that do not result in a numeric answer, then the response will vary from humorous to disastrous, depending on your use case.

While there are places where producing a single built-in type can be helpful, the true power of this guided generation comes when you define your data structure and provide Foundation Models guidance on generating it. While generating data with LLMs has always been possible using the correct prompts, this has typically required careful tuning to return a format such as JSON and meticulous text parsing. The native inclusion of this ability may be the most important feature of Foundation Models compared to general LLM solutions.

In the next section, you will learn how to produce more complex custom data structures using guided generation.