App Intents with Siri

Transcript

Intro

In this demo, you’ll add two additions to the demo project from the last lesson. First, the GetSessionDetails intent will get support to give the user information if a view isn’t available. Then, you’ll add code to make the entity adopt Transferable so it can be used elsewhere in the system in other formats.

Enhancing the Intent

Updating GetSessionDetails

The perform function for intents is pretty flexible regarding what it can return. Originally, it only returned the session. Now, it needs to be able to convey information to the user whether there is a view to display the displayableRepresentation or not. Start by adding additional protocol adoption to the return type of perform:

func perform() async throws -> some IntentResult & ReturnsValue<SessionEntity> & ProvidesDialog & ShowsSnippetView {

The ProvidesDialog and ShowsSnippetView protocols tell the compiler that a dialog and a view will be part of what the function returns.

In the body of the perform function, make an instance of SessionSiriDetailsView that takes in the session data that matches the provided id:

let snippet = SessionSiriDetailView(session: sessionData)

The function can return this view. The function can also return a dialog, which can either be presented to the user or have its information read to the user. Add the following IntentDialog:

let dialog = IntentDialog(
  full: """
  The runtime reported for \(sessionToGet.name) is \(sessionToGet.sessionLength ?? "no runtime reported")
  and has the following description: \(sessionToGet.sessionDescription ?? "no description provided").
  """,
  supporting: "Here's the information on the requested session."
)

The full parameter reads information about this session to the user. The string in the supporting parameter will be displayed at the top of the dialog.

To return the session, view, and dialog, use one of the other versions of .result:

return .result(value: sessionToGet, dialog: dialog, view: snippet)

Now, the system can display information about the session using whatever is available.

Demoing the Intent View

Let’s see how all this looks in the simulator. Run the app once, just in case this is the first time. This allows Spotlight to index all of the session information. Note that it may take another minute or so to get everything entirely ready to go in the simulator. Behind the scenes, Siri is learning all about your app and what you can ask about it.

Now, return to the Home Screen. Activate Siri by choosing the Device -> Siri menu item or pressing shift-option-command-H. When using the keyboard command, make sure that the simulator has the focus.

Once activated, speak the phrase defined in SessionShortcuts. This can be performed with whatever is the active microphone on your device, including your AirPods.

Siri parses that phrase, and since a session was not stated, it will then ask what session to use.

Now, with a session to match against the known sessions, Siri displays the IntentDialog on screen, with the supporting text above it. Siri then speaks out the string in the full parameter from the IntentDialog.

Adding Transferable Support

Getting your entities ready for Transferable requires two code additions. Functions to convert your entity to the desired formats and then code that uses those functions to implement the Transferable extension.

Updating the Entity

Start by adding some required imports and an extension block to SessionEntity, where code related to Transferable will go, but don’t adopt that protocol just yet:

import CoreTransferable
import UIKit

extension SessionEntity {

Then, add a toString() function that will be used later.

func toString() -> String {
  return "\(self.name) (\(self.sessionLength ?? "No Length"): \(self.sessionDescription ?? "No Description"))"
}

For this demo, SessionEntity can be transformed into JSON and JPEG. In the extension, add a new function called sessionAsJSON() that returns an optional Data instance.

func sessionAsJSON() async -> Data? {
  var json: [String: String] = [:]
  json["id"] = "\(self.id)"
  json["name"] = self.name
  json["imageName"] = self.imageName
  json["sessionDescription"] = self.sessionDescription
  json["sessionLength"] = self.sessionLength
  json["url"] = self.url?.absoluteString
  do {
    return try JSONSerialization.data(withJSONObject: json, options: JSONSerialization.WritingOptions.prettyPrinted)
  } catch let myJSONError {
    print(myJSONError)
  }
  return nil
}

This code stores all of the information about the session in a dictionary and then uses JSONSerialization to convert that dictionary to a data instance.

To convert the session to a JPG, add the following code to set up an image renderer and then draw the output of the toString() function to the image’s frame.

func sessionAsJPEG() async -> SentTransferredFile? {
  var transferredFile: SentTransferredFile?
  let size = CGSize(width: 300, height: 500)
  let frame = CGRect(origin: .zero, size: size)
  let image = UIGraphicsImageRenderer(size: size).image { rendererContext in
    UIColor.systemGray.setFill()
    rendererContext.fill(CGRect(origin: .zero, size: size))
    toString().draw(in: frame)
  }
  ...

To complete the sessionAsJPEG() function, convert the image to a data instance and then write that to disk. Finally, create a SentTransferredFile so this file can be sent back to the caller of the intent.

  ...
  if let data = image.jpegData(compressionQuality: 0.4) {
    let path = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!
    let file = path.appendingPathComponent("session.jpg")
    try? data.write(to: file)
    transferredFile = SentTransferredFile(file)
  }
  return transferredFile
}

Adding the Transferable Extension

Now that the helper methods are defined, the extension can adopt Transferable and add the necessary protocol property.

Start by updating the extension to adopt Transferable.

extension SessionEntity: Transferable {

Next, define the static transferRepresentation property required by the protocol.

static var transferRepresentation: some TransferRepresentation {

This property can represent several different TransferRepresentations. Start by defining a DataRepresentation for the .json exported content type, and use the helper defined earlier.

  DataRepresentation(exportedContentType: .json) { sessionEntity in
    guard let sessionData = await sessionEntity.sessionAsJSON() else { throw TransferableError.invalidJSON
  }
  return sessionData
}

sessionAsJSON() can return nil, so a guard let checks for that and throws a TransferableError when the nil is encountered.

The compiler is complaining about the missing error type, so add that with another case for when a file can’t be found.

enum TransferableError: Error {
  case noFileFound
  case invalidJSON
}

Add another representation, this time a FileRepresentation for the .jpeg export type, and again, use the helper method from earlier.

      FileRepresentation(exportedContentType: .jpeg) { sessionEntity in
        guard let sessionJPG = await sessionEntity.sessionAsJPEG() else { throw TransferableError.noFileFound
      }
      return sessionJPG
    }
  }
}

A similar guard let check is used in case there is an error when creating the file, and it returns nil.

Demoing Sending the Entity to Notes via the Shortcuts App

Build and run the app so the updated code is sent to the simulator. To check the entity’s conversion to JSON and JPEG, make some shortcuts. Start by opening shortcuts and starting a new shortcut.

Scroll down to the SessionTracker app and choose the Get Session Details shortcut.

Select the First Session as the one to get details for.

Open the disclosure button and deselect “Show when run” to keep the dialog from opening.

Then, due to the simulator’s limited capabilities—it doesn’t have the Notes app—send the content of the shortcut to the clipboard by choosing Device -> Copy to clipboard from the actions menu. Right after that, choose Device -> Get clipboard. This will display the text in JSON form.

To get a look at the JPEG capability, make a new shortcut, once again get the Get Session Details action, and choose the First Session

Then, add a “Make PDF” action and run the shortcut

The information from the toString() function is shown on a grey background—that’s the JPEG image!

Now, with these examples in place, you can send your session information to all sorts of apps, thanks to Transferable.