Instruction

Exporting the Model from Create ML

To export your trained model from Create ML, navigate to the Output section, where you’ll find your model ready for export. Simply click the Get button, which will prompt you to choose a location to save the model file. Ensure you select a location that’s easy to access for the following steps.

The exported model will have a .mlmodel extension, which is the Core ML format. This format is directly compatible with iOS applications, so there’s no need to convert the model to another format.

Integrating the Model Into a SwiftUI App

To integrate your custom image classification model into a SwiftUI app, you’ll follow a process that involves creating an instance of your model, setting up the image-classification request, and updating the UI with the results. Here’s how you can achieve this:

1. Creating an Image Classifier Instance

Start by creating an instance of your Core ML model. This should be done when the app launches, ensuring that you have a single instance of the model available for efficient performance throughout the app.

// 1. Initialize the model
private let model: VNCoreMLModel

init() {
  // 2. Load the Core ML model
  guard let model = try? VNCoreMLModel(for: EmotionsImageClassifier().model) else {
    fatalError("Failed to load Core ML model.")
  }
  self.model = model
}

Here’s a breakdown of the code above:

1. Initialize the model: This line declares a property to hold the Core ML model instance.
2. Load the Core ML model: This line attempts to create a VNCoreMLModel instance from your Core ML model. If it fails, it triggers a fatal error, ensuring you’re notified if something goes wrong.

2. Creating an Image-Classification Request

To classify an image, you must create a VNCoreMLRequest using your model. This request will process the image and provide classification results.

func classifyImage(_ image: UIImage) {
  // 1. Create a VNCoreMLRequest with the model
  let request = VNCoreMLRequest(model: model) { (request, error) in
    // 2. Handle the classification results
    guard let results = request.results as? [VNClassificationObservation],
          let firstResult = results.first else {
      return
    }
    print("Classification: \(firstResult.identifier), Confidence: \(firstResult.confidence)")
  }

  // 3. Configure the request to crop and scale images
  request.imageCropAndScaleOption = .centerCrop
}

Here’s a breakdown of the code above:

1. Create a VNCoreMLRequest with the model: This line creates a new image-classification request using the model you initialized. It includes a completion handler to process the results.
2. Handle the classification results: Inside the completion handler, this code checks if the results can be cast to an array of VNClassificationObservation and then processes the first result.
3. Configure the request to crop and scale images: This line sets the image crop and scale option to .centerCrop, ensuring that images are properly adjusted for the model’s input requirements.

3. Creating a Request Handler

You use the VNImageRequestHandler to handle the image and perform the request. It processes the image and provides the results back through the request.

func performClassification(for image: UIImage) {
  guard let cgImage = image.cgImage else {
    return
  }

  // 1. Create a VNImageRequestHandler with the image
  let handler = VNImageRequestHandler(cgImage: cgImage, options: [:])

  // 2. Perform the classification request
  let request = VNCoreMLRequest(model: model) { (request, error) in
    // Handle the results in the completion handler
  }
  do {
    try handler.perform([request])
  } catch {
    print("Failed to perform classification request: \(error)")
  }
}

Here’s a breakdown of the code above:

1. Create a VNImageRequestHandler: This line initializes a request handler with the provided image. The image must be converted to a CGImage format.
2. Perform the classification request: This line performs the classification request using the handler and the request created earlier. It includes error handling to catch and report any issues that arise during the process.

4. Handling and Extracting High-Confidence Results

Once you receive the classification results from the Core ML model, the next step is to handle these results and identify the most accurate classification based on confidence scores. This process involves checking for valid results and selecting the one with the highest confidence to ensure that you present the most reliable classification to the user.

// 1. Handle the classification results
guard let results = request.results as? [VNClassificationObservation] else {
  print("No results found")
  completion(nil, nil)
  return
}

// 2. Find the top result based on confidence
let topResult = results.max(by: { a, b in a.confidence < b.confidence })
guard let bestResult = topResult else {
  print("No top result found")
  completion(nil, nil)
  return
}

Here’s a breakdown of the code above:

1. Handle the classification results: In this part, the code checks whether request.results can be cast to an array of VNClassificationObservation. This step ensures that the results are valid and contain the expected classification observations. If the cast fails, indicating that no results are found, an error message is printed and the completion handler is called with nil values.
2. Find the top result based on confidence: This section finds the classification observation with the highest confidence score. The results.max(by:) method iterates through the VNClassificationObservation array and compares each observation’s confidence score. The observation with the highest confidence is selected as topResult. If no result is found, an error message is printed and the completion handler is called with nil values. If a top result is successfully identified, it’s used for the final classification output.

By focusing on the classification with the highest confidence, you ensure that the most accurate and reliable result is presented to the user, enhancing the effectiveness of your app’s image classification feature.

5. Updating the UI with Classification Results

After receiving the classification results, it’s essential to update the UI to present these results to the user in a clear and meaningful way. This step involves converting the raw prediction data into a user-friendly format and ensuring the UI elements reflect the updated information. Typically, this means updating labels, text fields, or other UI components with the classification results. It’s crucial to perform these updates on the main thread to ensure smooth and responsive user interactions.

Tips to Optimize the Model for Real-Time Performance

Optimize Predictions on Background Threads

Run your model’s predictions off the main thread to keep the UI responsive.

DispatchQueue.global(qos: .userInitiated).async {
  classifyImage(inputImage)
}

Batch Processing

For tasks requiring multiple classifications in a short period, consider batching your requests. This method minimizes the overhead of individual requests.

func classifyBatchImages(_ images: [UIImage]) {
  let requests = images.map { image in
    VNCoreMLRequest(model: EmotionClassifier.shared.model)
  }
  let handler = VNImageRequestHandler(cgImage: images[0].cgImage!, options: [:])
  try? handler.perform(requests)
}

Reduce Image Size

Before passing images to the model, resize them to match the input size your model expects (e.g., 224x224 pixels). This reduces the computational load.

func resizeImage(_ image: UIImage) -> UIImage? {
  UIGraphicsBeginImageContext(CGSize(width: 224, height: 224))
  image.draw(in: CGRect(x: 0, y: 0, width: 224, height: 224))
  let resizedImage = UIGraphicsGetImageFromCurrentImageContext()
  UIGraphicsEndImageContext()
  return resizedImage
}

Profile Performance

Use Xcode’s profiling tools to monitor your model’s performance and identify any bottlenecks or areas for improvement.