8. Using Foundation Models for Voice Notes

Written by Bill Morefield

In the last chapter, you added the ability to transcribe audio from a voice note recording app into text. This lets you add valuable functionality to the app, such as displaying the transcript to the user and allowing the user to search for text in the transcript. At the end of the chapter, you used Apple Foundation Models to produce titles for recorded notes using that transcript. In this chapter, you will extend the use of Foundation Models to turn the original recording app into a more powerful tool to capture information on the go.

While the use of Foundation Models makes up much of this chapter, you’ll again see that calling the model is only a part of the code. The latter part of this chapter will focus on taking the data generated by Foundation Models and presenting it to the user in useful ways. The goal is to use artificial intelligence to provide value in your app.

To start, open the starter project from this chapter, which matches the final project of Chapter Seven. Repeat the steps to add the NSSpeechRecognitionUsageDescription key to your Target:

1. Go to the Project for the app in Xcode and select the VoiceNotes target.
2. Go to the Info tab, and you will see the existing list of properties.
3. Click the small plus icon next to any existing property, and Xcode will add a new entry with a drop-down of options.
4. Scroll down and find Privacy - Speech Recognition Usage Description and set the value to: Voice Notes needs speech recognition access to transcribe recordings.

Producing Data Structures for Analysis

In Chapter Five, you learned about perhaps the most powerful feature of Foundation Models, guided generation. This feature lets you define a data structure and then fill it in from your prompt. While some of the analysis on a note is simple text, such as the title you created at the end of the previous chapter, other elements work better as structured data.

Create a new Swift file under Models named VoiceNoteAnalysis.swift. Replace the contents of the file with:

import Foundation
import FoundationModels

@Generable(description: "A concise analysis of a transcribed voice note.")
struct NoteAnalysis {
  @Guide(description: "A concise title of a few words that summarizes the note contents.")
  let title: String

  @Guide(description: "A two to three sentence summary of the voice note.")
  let summary: String

  @Guide(description: "Up to five short lowercase topic tags.", .count(1...5))
  let tags: [String]

  @Guide(description: "People referenced in the note.")
  let people: [String]

  @Guide(description: "Specific action items or tasks mentioned in the note.")
  let actionItems: [GeneratedNoteActionItem]
}

This struct uses the @Generable macro, which allows the model to respond to prompts by creating an instance of the type. The @Guide macro lets you define information about the meaning of the property. Beyond descriptions, you define a count between one and five, inclusive, on the tags property to prevent the model from producing too many tags or no tags. This struct defines the analysis you can do on the note. You’ll create the title, a summary, and tags for the note. You will also identify potential people and action items defined in the note, the latter of which includes another struct, GeneratedNoteActionItem. Define GeneratedNoteActionItem used in the actionItems property of the NoteAnalysis struct by adding the following code after NoteAnalysis:

@Generable(description: "An actionable item or task extracted from the voice note.")
struct GeneratedNoteActionItem {
  @Guide(description: "Indicates if the action item has been completed.")
  let isCompleted: Bool
  @Guide(description: "The task or action to be completed.")
  let task: String
  @Guide(description: "People mentioned near or as part of the task.")
  let people: [String]
}

This struct defines the three items the model generates for an actionable item: its completion status, the task, and any people referenced in the task. Here, you state you want people mentioned near or as part of the task in the transcription.

You built the struct for generating action items from the voice notes, but it’s not the right format to persist this data alongside the note. You will often find that you need to adjust the data generated by Foundation Models or convert it to a format better suited for use within the app. In this case, a vital missing element is a unique id for the action item. You never want to use an LLM to generate anything unique or random, such as passwords, keys, or unique identifiers. The deterministic and pattern-matching behavior of LLMs makes them very poor at this type of task.

First, find VoiceNote.swift under the Models folder and add the following new code after the VoiceNote struct:

struct NoteActionItem: Identifiable, Codable, Equatable {
  let id: UUID
  var isCompleted: Bool
  let task: String
  let people: [String]

  init(from generatedItem: GeneratedNoteActionItem) {
    id = UUID()
    isCompleted = generatedItem.isCompleted
    task = generatedItem.task
    people = generatedItem.people
  }
}

This new struct implements several protocols to allow the existing store to persist the NoteActionItem. Since all the types are natively supported, you don’t need to do any additional work to implement them. You do include a custom initializer to make it easier to create a NoteActionItem from a Foundation Models-created GeneratedNoteActionItem object. Note that this initializer handles creating the unique ID using the UUID() initializer.

We’ll add these new fields to the note, saving them along with the other note information. Go to the VoiceNote struct and add these new properties after var transcript:

var summary: String?
var tags = [String]()
var people = [String]()
var actionItems = [NoteActionItem]()

You define the summary as an optional string since it will not exist until the analysis runs. You create the others as empty arrays, which is their initial state. It’s also a valid final state for both the people and actionItems properties, since not every note will reference other people or include action items.

Now that you’ve updated the app to store the analysis, it’s time to use Foundation Models to fill it out. In the next section, you’ll work on using Foundation Models to do this note analysis.

Using Foundation Models to Analyze a Note

Open NoteAnalysisService.swift. You’ll begin by producing some error states. Add the following code to the top of the file after the imports:

enum NoteAnalysisError: LocalizedError {
  case missingTranscript
  case transcriptTooLarge

  var errorDescription: String? {
    switch self {
    case .missingTranscript:
      "No transcript is available to analyze."
    case .transcriptTooLarge:
      "This transcript is too long for the current model."
    }
  }
}

This defines two errors for the cases where no transcript exists or if the transcript is too long for the current model to process. Since you’re going to add a more robust analysis to the app, delete the determineTitle(transcript:) method you created at the end of Chapter Seven. Replace it with:

private func fitsInContext(_ prompt: String) async -> Bool {
  let tokenLength: Int
  // 1
  let promptRatio = 0.33

  // 2
  if #available(iOS 26.4, *) {
    // 3
    let promptCalc = try? await
      SystemLanguageModel.default.tokenCount(for: prompt)
    if let promptCalc = promptCalc {
      tokenLength = promptCalc
    } else {
      tokenLength = prompt.count * 3 / 4
    }
  } else {
    // 4
    tokenLength = prompt.count * 3 / 4
  }
  
  // 5
  return tokenLength < Int(Double(SystemLanguageModel.default.contextSize) * promptRatio)
}

Here’s how this code checks to ensure the prompt can handle the transcript:

1. This constant is the maximum size the prompt can occupy and leave space for the analysis, expressed as a fraction. In this case, you are allowing the prompt to be up to 1,333 tokens or about 1,000 words. This will accommodate voice notes over six minutes long for average speakers.
2. The methods to directly calculate token lengths are only available on 26.4 or later versions of Apple operating systems. We verify this using the if #available check.
3. If the app is running on a version that supports token counts, it gets the token size of the prompt. If anything goes wrong, it falls back on the same estimating method used in step three.
4. If the token calculations are not available, then the method falls back on the rule of thumb that a token is generally four characters. In this case, the code divides the string length by four to estimate the token count.
5. The method then compares this calculated token count of the prompt to the contextSize, doing the necessary Double to Int conversions.

The value chosen for promptRatio was determined through experimentation. You’ll see the code used for that in a moment.

Now add the following new method to perform the analysis:

func analyze(transcript: String) async throws -> NoteAnalysis {
  // 1
  guard !transcript.isEmpty else {
    throw NoteAnalysisError.missingTranscript
  }

  // 2
  let session = LanguageModelSession()
  let prompt = """
    Analyze the following voice note transcription.

    Create:
    - A concise title of a few words
    - A two to three-sentence summary focused on the overall topic and
      key points.
    - Up to five short lowercase tags, each one up to three words.
    - Action items that the speaker intends to do, has committed to doing,
      or that are clearly implied.
    - A list of people mentioned in the note

    Do not invent details, deadlines, assignees, or people.
    If no action items are present, return an empty actionItems array.
    If no people are mentioned, return an empty people array.

    Transcription: \(transcript)
    """

  // 3
  guard await fitsInContext(prompt) else {
    throw NoteAnalysisError.transcriptTooLarge
  }

  // 4
  let response = try await session.respond(to: prompt, generating: NoteAnalysis.self)
  // 5
  #if DEBUG
  if #available(iOS 26.4, *) {
    let promptSize = (try? await
      SystemLanguageModel.default.tokenCount(for: prompt)) ?? 0
    let responseSize = SystemLanguageModel.default.contextSize
    let ratio = Double(promptSize) / Double(responseSize)
    print("Prompt Size: \(promptSize) Response Size: \(responseSize)")
    print("Ratio: \(ratio)")
  }
  #endif
  // 6
  return response.content
}

The method uses the same Foundation Models pattern you’ve used throughout this book. The prompt reflects the lessons on prompt creation from Chapter Three. Using bullet points removes ambiguity on where one instruction ends and another begins. It is specific by defining the sizes of a few words for the title and “two or three” sentences for the summary. Negative reinforcement helps reduce an LLM’s tendency to fill in gaps in vague information. Placing the transcript at the end helps the instructions frame the desired actions before the model reaches the transcript.

1. First, the method ensures there is a transcript and throws the appropriate error if not.
2. The method creates a LanguageModelSession and appends the transcript at the end of the prompt.
3. Before calling the model, you use the previous method to see if the prompt, which includes the transcript, will likely fit in the model. If not, you throw the appropriate error.
4. If all looks good, you call respond(to:generating:includeSchemaInPrompt:options:) on the session, using the generating parameter to have Foundation Models return the results as the NoteAnalysis struct.
5. The #if/endif block includes the enclosed code only in debugging builds of the app. This code does a simple calculation of the prompt token size and the transcript token size, and then prints them and the ratio to the debugging console. You can use this to see the ratio in your own testing of the analysis and find if it corresponds to the 0.33 value used in fitsInContext(_:).
6. Whether the debugging block exists or not, the method returns the generated NoteAnalysis struct to the calling method.

You notice we do not use streamed generation here. Ultimately, analyses like this are background processes. It will take place in the background after the user records a note and the transcription completes. From Chapter Two, you learned that this circumstance is the perfect place to use the less interactive respond(to:generating:includeSchemaInPrompt:options:) on a session. The complexity of supporting a streamed response here isn’t worth the additional code and work since the user will rarely view it.

You’ll now add code to update the title from the default to the one created from the analysis. Go back to VoiceNoteStore.swift and find the stopRecording() method. Find the Task at the end of the method and replace the two lines of the closure that set title and call updateTitle(_:for:) with:

let analysis = try await noteAnalysis.analyze(transcript: noteTranscript)
updateTitle(analysis.title, for: note.id)

This runs the full analysis prompt and then updates the title as before.

Run the app and record a new note. As a reminder, simulators don’t support SpeechTranscriber. If you do not have a device to run the app on for this chapter, you can take advantage of the Designed for iPad option to run the iPad version of the app on your Mac, which does provide SpeechTranscriber support. Do this by selecting the My Mac (Designed for iPad) option as the device to run the app on.

You should see, as before, that the note gets a title after a slightly longer pause since the transcription and analysis both need to be completed.

Of course, that’s only one part of the new information available. In the next section, you’ll update the app to process and show all this information.

Updating Notes with Analysis

Open VoiceNoteStore.swift and begin by adding a reference to the analysis service after transcriptionService:

private let analysisService = NoteAnalysisService()

With this available, add the following new method after updateTitle(_:for:):

private func updateAnalysis(_ analysis: NoteAnalysis, for noteID: VoiceNote.ID) {
  guard let index = notes.firstIndex(
    where: { $0.id == noteID }
  ) else { return }
  notes[index].title = analysis.title
  notes[index].summary = analysis.summary
  notes[index].tags = analysis.tags
  notes[index].actionItems = analysis.actionItems.map {
    NoteActionItem(from: $0)
  }
  notes[index].people = analysis.people
  saveNotes()
}

This code takes the analysis passed into the method and the note id, updates all the properties to match the analysis, and then saves these changes.

Now add the following new method to the end of the struct after defaultTitle(for:):

func performAnalysis(_ transcript: String, for noteId: VoiceNote.ID) async {
  guard !transcript.isEmpty else { return }

  do {
    let analysis = try await analysisService.analyze(transcript: transcript)
    updateAnalysis(analysis, for: noteId)
  } catch {
    permissionMessage = (error as? LocalizedError)?.errorDescription
      ?? "This voice note could not be analyzed."
  }
}

You first ensure the transcript is not empty before using the analysisService to perform the analysis. You then update the note with the information. Go back to stopRecording() and change the closure of the Task to:

let transcript = await transcribeRecording(note)
guard let transcript = transcript else { return }
await performAnalysis(transcript, for: note.id)

This changes the method to attempt to create the transcript, then proceed to analysis if the transcript exists. Run the app and record a note to see if the method still works as before.

While the analysis will run automatically after a user records a note, you’ll also add an analysis button to the toolbar. This will allow the user to force a re-run of the analysis or run the analysis on one of the sample notes. Open VoiceNoteDetailView.swift and add the following code after the isShowingDeleteConfirmation view property:

@State private var analyzingNote = false

This will provide a flag to update the interface during the analysis. With the flag in place, find the toolbar under the VoiceNoteDetailView view and add the following code between the two existing Buttons:

if let transcript = note.transcript {
  Button {
    Task {
      analyzingNote = true
      await store.performAnalysis(
        transcript,
        for: note.id
      )
      analyzingNote = false
    }
  } label: {
    Image(systemName: "sparkles.2")
      .accessibilityLabel("Perform Analysis")
  }
  .disabled(analyzingNote)
}

This button only appears when a transcript exists and allows the user to re-run the analysis. Due to the non-deterministic nature of LLMs, this will produce different results. When a user taps the button, it sets the analyzingNote property on the view to true, then calls the performAnalysis(_:for:) method on the store. Once the analysis completes, you set analyzingNote back to false. You disable the button during analysis so the user cannot queue up multiple analyses of the same note.

So far in this chapter, you’ve completed the core work of the Foundation Models for the app. You’re using the model to analyze the note’s transcription and extract several pieces of useful information, including a summary, people mentioned in the note, and action items. You use the model to provide useful information to the user.

Now that you have this information, the next step is to add it to the note’s detail view. You’ll do that in the next section.

Showing Note Analysis

With the analysis in place, the next step is to present this information to the user. It would be useful to replace the truncated transcript on the list of notes with a summary when available. To do this, open VoiceNoteRow.swift and find the private TranscriptSummary view. Find the line that reads } else if let transcript = note.transcript, !transcript.isEmpty {. Insert the following code before that line:

} else if let summary = note.summary {
  Text(summary)
    .font(.subheadline)
    .foregroundStyle(.primary)
    .lineLimit(2)
    .padding(.top, 4)

This code checks whether the note contains a summary and, if so, displays it. This takes place after the check to see if the note is being transcribed and before checking for the presence of a transcript. This will mean the view will transition from showing the transcript when one is available to showing the summary after analysis completes.

Run the app, and you’ll see the notes now showing these short summaries.

For most of the analysis data, you’ll show it in the view that shows the details for a note. Create a new SwiftUI view named VoiceNoteTextSection.swift. Replace the contents of the file with:

import SwiftUI

struct VoiceNoteTextSection: View {
  let text: String?
  let title: String
  
  var body: some View {
    VStack(alignment: .leading, spacing: 12) {
      Text(title)
        .font(.headline)
      if let text = text {
        Text(text)
          .font(.body)
          .textSelection(.enabled)
          .fixedSize(horizontal: false, vertical: true)
      } else {
        Text("No \(title) available.")
          .font(.body.italic())
      }
    }
    .frame(maxWidth: .infinity, alignment: .leading)
    .padding(16)
  }
}

#Preview {
  VoiceNoteTextSection(
    text: "Sample Text",
    title: "Summary"
  )
}

This view takes in a section title, which it renders in the headline font along with text as an optional String. It handles the case where text is nil by showing an italicized message that no text exists.

Now open VoiceNoteDetailView.swift. Inside VoiceNoteDetailView, look for the VoiceNoteTranscriptSection(note: note) line. Add the following code after it:

VoiceNoteTextSection(text: note.summary, title: "Summary")

Run the app and view any analyzed note. You will now see the analysis on it.

Now to show the people and tags, find VoiceNoteTagsSection.swift and add the following new view to the top above FlowLayout:

struct VoiceNoteTagsSection: View {
  var tags: [String]
  var title: String

  var body: some View {
    if !tags.isEmpty {
      VStack(alignment: .leading, spacing: 12) {
        Text(title)
          .font(.headline)
        FlowLayout(spacing: 8) {
          ForEach(tags, id: \.self) { tag in
            Text(tag)
              .font(.subheadline.weight(.medium))
              .foregroundStyle(.secondary)
              .padding(.horizontal, 12)
              .padding(.vertical, 6)
              .background(Color(.systemGray6), in: Capsule())
          }
        }
      }
      .padding(16)
    }
  }
}

This view expects a string array along with a title. If the array is empty, then the view will not show. If there are array elements, the view uses the FlowLayout already defined in the file to lay out the text views containing each string in the array. The result will be a series of Capsule shapes, each containing one element of the array. Now add a preview for this new view at the bottom of the file:

#Preview {
  VoiceNoteTagsSection(
    tags: ["running", "marathon", "city park"],
    title: "Tags"
  )
}

To use this new view, go back to VoiceNoteDetailView.swift and find the VoiceNoteDetailView view. Add the following after VoiceNoteTextSection(text: note.summary, title: "Summary"):

VoiceNoteTagsSection(tags: note.tags, title: "Tags")
VoiceNoteTagsSection(tags: note.people, title: "People")

These will add a list of the tags and people to the view. Run the app to see this new information on any analyzed note.

Notice that the prompt provided accountant and kids as people referenced in the note. If you wanted only proper names, you could adjust the prompt and @Guide for the property to reflect that.

One last item to show, the action items the analysis found in the note. Since these are action items, you’ll first add a method to allow the user to mark them as completed or unmark them. Open VoiceNoteStore.swift and add the following new method after updateAnalysis(_:for:):

func updateTaskCompletion(_ status: Bool, for actionItem: UUID) {
  for noteIndex in notes.indices {
    let taskIndex = notes[noteIndex].actionItems.firstIndex(
      where: { $0.id == actionItem }
    )
    if let taskIndex = taskIndex {
      notes[noteIndex].actionItems[taskIndex].isCompleted = status
    }
  }
  saveNotes()
}

This method takes the new completion state for the action item along with the id of the item. It searches through the notes and within each note for any action item with that id. If found, the code sets the isCompleted property of the action item to the status passed to the method. It saves the notes afterward.

Create a new SwiftUI view named VoiceNoteTasksSection.swift. Replace the contents with:

import SwiftUI

struct VoiceNoteTasksSection: View {
  var actionItems: [NoteActionItem]
  
  var body: some View {
    if !actionItems.isEmpty {
      VStack(alignment: .leading, spacing: 12) {
        Text("Action Items")
          .font(.headline)
        VStack(spacing: 8) {
          ForEach(actionItems) { task in
            ActionItemRow(task: task)
          }
        }
      }
      .frame(maxWidth: .infinity, alignment: .leading)
      .padding(16)
    }
  }
}

Like the others, this view will show a title and then loop through the action items identified by the model, showing each using an ActionItemRow. Add this code to the file after VoiceNoteTasksSection:

struct ActionItemRow: View {
  @EnvironmentObject private var store: VoiceNoteStore
  let task: NoteActionItem
  
  var body: some View {
    HStack(alignment: .center, spacing: 12) {
      Button {
        withAnimation {
          store.updateTaskCompletion(!task.isCompleted, for: task.id)
        }
      } label: {
        Image(systemName: task.isCompleted ?
              "checkmark.circle.fill" : "circle"
        )
        .font(.body)
        .foregroundStyle(.secondary)
        .frame(width: 20, height: 20)
        .padding(.top, 1)
      }
      VStack(alignment: .leading, spacing: 4) {
        Text(task.task)
          .font(.body)
          .strikethrough(task.isCompleted)
          .textSelection(.enabled)
          .fixedSize(horizontal: false, vertical: true)
        if !task.people.isEmpty {
          ForEach(task.people, id: \.self) { person in
            Label(person, systemImage: "person")
              .font(.caption)
              .strikethrough(task.isCompleted)
              .foregroundStyle(.secondary)
          }
        }
      }
    }
    .frame(maxWidth: .infinity, alignment: .leading)
    .padding(12)
    .background(
      Color(.secondarySystemGroupedBackground),
      in: RoundedRectangle(cornerRadius: 10)
    )
  }
}

This view will show each action item as a horizontal row with an empty circle followed by the task name and a list of involved people.

Now go back to VoiceNoteDetailView.swift and add the following after VoiceNoteTagsSection(tags: note.people, title: "People"):

VoiceNoteTasksSection(actionItems: note.actionItems)

Run the app, and you should see the last element from the analysis on display.

This completes showing the information Foundation Models added to the note’s detailed information view. But there is far more value you can provide the user with this analysis. In the next section, you’ll start leveraging this analysis to allow the user to search the content.

Searching Note Analysis

After implementing transcripts for the voice notes, you added the ability to search for text within the transcripts. Now you’ll allow the user to search within these new fields. Open ContentView.swift and add a new enum to the top of the file:

enum SearchScope {
  case all
  case transcript
  case actionItems
  case people
}

While being able to search everything is useful, providing filters can help the user find only the information they seek. This enumeration provides the search contexts for this app: all data, just the transcript, only the actionable items, and just people. Now add a new state property after searchText to the view:

@State private var searchScope: SearchScope = .all

This property will hold the search context.

Now find the searchable(text:placement:prompt:) modifier on the view and add the following code after it:

.searchScopes($searchScope) {
  Text("All").tag(SearchScope.all)
  Text("Transcript").tag(SearchScope.transcript)
  Text("Action Items").tag(SearchScope.actionItems)
  Text("People").tag(SearchScope.people)
}

This modifier adds the search scopes to the search bar using a picker. This picker lets the user choose one after they enter search text. You could now update the computed property to handle the search actions. Instead, you’ll move that search ability into the VoiceNote itself. Doing so both keeps the view cleaner and makes it easier to update in the future.

Open VoiceNote.swift and add the following code to the end of the VoiceNote struct after the init(id:title:createdAt:duration:filename:transcript:) method:

func matchesPeople(_ text: String) -> Bool {
  // 1
  people.contains { $0.localizedStandardContains(text) } ||
  // 2
  actionItems.contains {
    $0.people.contains { $0.localizedStandardContains(text) }
  }
}

This method takes advantage of the implicit return for single-expression functions. Since a logical OR operator connects the two lines, it remains a single-expression. Here’s how it finds people in a voice note:

1. First, it searches the people property on the note. The contains instance method returns a Bool indicating whether the sequence contains an element in the closure. We evaluate each element in the sequence using the predicate, referencing the current element as $0. This uses the same localizedStandardContains(_:) on the string used in the transcript search in Chapter Seven to do a broad comparison of the text in question. If any element within the collection fulfills this condition, the result will be true.
2. This step builds on the search process in step one by nesting a contains method inside another contains method. One property of actionItems contains the people referenced in the action item. For each element in that collection, another contains method then determines if the text of the person matches the text search string. If any element of the people property of the actionItems property of the note matches, then the result will be true.

Now add the following method:

func matchesActionItems(_ text: String) -> Bool {
  actionItems.contains {
    $0.task.localizedStandardContains(text) ||
    $0.people.contains { $0.localizedStandardContains(text) }
  }
}

This method uses the same nested contains pattern as matchesPeople(_:), checking the action item’s task text and its associated people against the search string.

You will move the transcript search from Chapter Seven into a method by adding this code:

func matchesTranscript(_ text: String) -> Bool {
  transcript?.localizedStandardContains(text) == true
}

This method implements your earlier check. You use optional chaining on the transcript property when calling localizedStandardContains(_:). This means when transcript is nil, the left side of the equals sign will be nil. Comparing nil to true is false, so the comparison will be false when transcript is nil.

Finally, add the following method that will search all applicable fields in the note for the text:

func anyFieldMatches(_ text: String) -> Bool {
  title.localizedStandardContains(text) ||
  matchesTranscript(text) ||
  summary?.localizedStandardContains(text) == true ||
  matchesPeople(text) ||
  matchesActionItems(text)
}

This method uses the other four methods to compare all parts of the note and adds a check for the title. With these in place, go back to ContentView.swift and change the visibleNotes computed property to:

var visibleNotes: [VoiceNote] {
  if searchText.isEmpty {
    return store.notes
  }
  
  return store.notes.filter { note in
    switch searchScope {
    case .all:
      note.anyFieldMatches(searchText)
    case .transcript:
      note.matchesTranscript(searchText)
    case .actionItems:
      note.matchesActionItems(searchText)
    case .people:
      note.matchesPeople(searchText)
    }
  }
}

The computed property uses the methods added to VoiceNote to perform the different searches without cluttering the computed property, as would happen if we tried to include all the search logic here.

Run the app and make sure the analysis has run on all the notes. Experiment with entering different text and seeing how the results match.

You can see how allowing the user to search the additional data generated by Foundation Models adds value to the app. No longer do you need to hope the title contains Pharmacy. You can search for action items that mention pharmacy. Or find all notes related to a single person. In the next section, you will add one more aspect to the app, a concise view of the action items contained in the voice notes.

Showing Action Items

Create a new SwiftUI view named ActionItemsListView.swift. This view will show all action items contained in any voice note. Replace the contents of the view with:

import SwiftUI

struct ActionItemsListView: View {
  @EnvironmentObject private var store: VoiceNoteStore
  @State private var showCompleted = true

  var body: some View {
    VStack {
      // 1
      Toggle("Show Completed", isOn: $showCompleted)
        .padding()
      // 2
      ForEach(store.notes) { note in
        Text(note.title)
          .font(.headline)
        // 3
        ForEach(note.actionItems) { item in
          if !item.isCompleted || showCompleted {
            ActionItemRow(task: item)
          }
        }
        Divider()
      }
    }
    .navigationTitle("Action Items")
  }
}

#Preview {
  NavigationView {
    ActionItemsListView()
      .environmentObject(VoiceNoteStore.mock)
  }
}

Here’s how this view shows the information:

1. You provide the user a toggle tied to the showCompleted state property that determines if the view will show completed tasks.
2. This loop goes through each note in the store and shows the title using the .headline font.
3. The inner loop will go through each action item in the note and do a check. If the action item is not completed or the user has chosen to show completed items, then you show it using the ActionItemRow view.

To use this, return to ContentView.swift and add the following to the end of the List right after the Section for the notes:

Section("Action Items") {
  ActionItemsListView()
}

This places the action items list after the notes list. As the app expands, you might want to move this into a new navigation structure or place, but for now, it’s not too much information to overwhelm the user.

Run the app and ensure you’ve analyzed your notes so there are action items to work on. Scroll down past the notes to see the action items listed.

Conclusion

In this chapter, you took the data added through machine learning in Chapter Seven to produce a transcript and applied Apple Foundation Models to analyze and find useful information in that transcript. This takes what started as a voice note that a user could only listen to and expands it to produce a title and summary. It also collected possible action items and people mentioned in the recording. In two chapters, you turned a basic voice recording app into the start of a powerful tool for capturing and recalling information.

This book has taken you from the simplest implementation of Apple Foundation Models to exploring all aspects of the model’s first release. You concluded by exploring the use of Foundation Models in a voice note app. You did not add AI as a checklist item, but examined where an LLM could take the app’s purpose and improve the user experience. This final app demonstrates the importance of preparing high-quality data and presenting model output to the user in ways that are informative and clearly accessible. Always keep in mind that the goal of using the lessons from this book is to make your app better.

Though this is the end of the book, continue using the voice recording app and look for other places where you can present the information you already gathered or new information. Some ideas:

• Presenting the people mentioned in notes to the user, along with the context where they are mentioned
• Gather more information about the voice notes, such as emotional tone, locations, contact information, dates, and deadlines for action items, etc.
• Other ways to link and combine multiple notes that might reference the same event or topic
• Extending the app to allow creation of reminders or calendar events from the information in voice notes.

Key Points

• When using Foundation Models, you can produce simple text responses or more specialized data structures using the @Generable and @Guide macros, which avoid parsing unstructured text responses. And you can mix and match as it is appropriate for your app.
• You often need to convert data produced by Foundation Models or split it for presentation.
• Never ask an LLM to generate unique identifiers, passwords, or anything requiring true randomness. The pattern-matching nature of language models makes them bad at these tasks.
• It’s good to check the context length for prompts. You can determine this through experimentation. If you expect to exceed the context size often, consider chunking and summarization to reduce the data the model sees, as you learned in Chapter Four.
• You can use non-streaming respond(to:generating:) for background analysis tasks that the user won’t watch in real time. Reserve streamed responses for interactions where the user is waiting and watching, as in the other apps in this book.
• Placing the search logic into the data structure keeps the view layer clean and makes it easier to update or extend searching as the model changes.