Apple Foundation Models

Streaming Guided Generation

Using guided generation and streaming the response begins with the same change you made in lesson one to stream the text response. Replace the current call to respond(to:) in generateLunchMenu() with:

let streamedResponse = session.streamResponse(to: prompt, generating: RestaurantMenu.self)

This changed code will stream the model’s response. To handle the stream, replace the line of code after comment five with:

do {
  for try await partialResponse in streamedResponse {
    menu = partialResponse.content
  }
} catch {
  print(error.localizedDescription)
}

You will see an error after this change: “Cannot assign value of type ‘RestaurantMenu.PartiallyGenerated’ to type ‘RestaurantMenu’. Xcode produces this error because a streamed response is not of the same type as the full response generated by respond(to:). When streaming the response, every property must be optional because the model may not have generated it yet. This requires a few changes to the code to handle these optionals. First, change the type of the menu property to:

@State var menu: RestaurantMenu.PartiallyGenerated?

The @Generable macro produces this PartiallyGenerated type, which matches the original type, RestaurantMenu in this case, except that every property will now be optional. Since all the properties of RestaurantMenu.PartiallyGenerated are now optional, you must change any uses of the PartiallyGenerated values to unwrap or otherwise handle the optional type. Change the if let code you added earlier to:

if let menu = menu {
  if let menuItems = menu.menu {
    ScrollView {
      ForEach(menuItems, id: \.name) { item in
        MenuItemView(menuItem: item)
        Divider()
      }
    }
  }
}

You now must attempt to unwrap the menu property before looping through it. Next you must update the MenuItemView view to also manage these optional types. Open MenuItemView.swift and change the declaration of the menuItem property to:

var menuItem: MenuItem.PartiallyGenerated

This will let you pass it the MenuItem.PartiallyGenerated. Now change the body of the view to:

HStack {
  Text(menuItem.name ?? "")
  Spacer()
  if let cost = menuItem.cost {
    Text(cost, format: .currency(code: "USD"))
  }
}
.font(.title)
Text(menuItem.description ?? "")
  .frame(maxWidth: .infinity, alignment: .leading)
  .padding(.leading, 15.0)
  .font(.headline)
if let ingredients = menuItem.ingredients {
  Text(ingredients.joined(separator: " • "))
    .font(.subheadline)
}

For the string properties name and description, you use the nil-coalescing operator to provide an empty string in the case where the property doesn’t exist. For the cost and ingredients properties, you attempt to unwrap them and only display information when the unwrapping succeeds.

One final change. In the preview, change the call to the view to:

MenuItemView(
  menuItem: item.asPartiallyGenerated()
)

The asPartiallyGenerated() method converts any Generable type to its PartiallyGenerated equivalent.

Run the new app, tap on Dining Menu in the menu, and tap Generate Lunch Menu. You will now see that, instead of the final menu appearing all at once, it will appear in pieces as it is generated. While watching this, you should observe the importance of ordering properties, as properties defined first appear before those defined later in the struct.

As discussed in lesson one, showing information as soon as the model generates it improves the user’s perception of the response time. It provides immediate feedback, making the process feel shorter. No longer do you wait for a menu. You watch the menu assemble.

In the next section, you’ll learn how to use guided generation when you don’t know the data structure until runtime.