Core Image Tutorial for iOS: Custom Filters

Applying the Color Kernel Filter

Open ImageProcessor.swift. Add the following method to ImageProcessor:

private func applyColorKernel(input: CIImage) {
  let filter = ColorFilter()
  filter.inputImage = input
  if let outputImage = filter.outputImage,
    let renderImage = renderAsUIImage(outputImage) {
    output = renderImage
  }
}

Here, you declare applyColorKernel(input:). This takes a CIImage as input. You create the custom filter by creating an instance of ColorFilter.

The filter’s outputImage has the color kernel applied. You then create an instance of UIImage using renderAsUIImage(_:) and set this as the output.

Next, handle .colorKernel in process(painting:effect:) as shown below. Add this new case above default:

case .colorKernel:
  applyColorKernel(input: input)

Here, you call applyColorKernel(input:) to apply your custom color kernel filter.

Finally, open PaintingWall.swift. Add the following in the switch statement right below case 0 in the Button‘s action closure:

case 1:
  effect = .colorKernel

This sets the effect to .colorKernel for the second painting.

Build and run. Now tap the second painting, “The Last Supper”. You’ll see the color kernel filter applied and the RGBA values swapped in the image.

Great job! Next, you’ll create a cool warp effect on da Vinci’s mysterious “Salvator Mundi.”

Creating a Warp Kernel

Similar to the color kernel, you’ll start by adding a Metal source file. Create a new Metal file in the Filters group named WarpFilterKernel.ci.metal. Open the file and add:

#include <CoreImage/CoreImage.h>
//1
extern "C" {
  namespace coreimage {
    //2
    float2 warpFilter(destination dest) {
      float y = dest.coord().y + tan(dest.coord().y / 10) * 20;
      float x = dest.coord().x + tan(dest.coord().x/ 10) * 20;
      return float2(x,y);
    }
  }
}

Here’s what you added:

You access the x and y coordinates of the destination pixel using coord(). Then, you apply simple math to transform the coordinates and return them as source pixel coordinates to create an interesting tile effect.

1. Like in the color kernel Metal source, you include the Core Image header and enclose the method in an extern "C" enclosure. Then you specify the coreimage namespace.
2. Next, you declare warpFilter(_:) with an input parameter of type destination, allowing access to the position of the pixel you’re currently computing. It returns the position in the input image coordinates you can then use as a source.

   You access the x and y coordinates of the destination pixel using coord(). Then, you apply simple math to transform the coordinates and return them as source pixel coordinates to create an interesting tile effect.

   Note: Try replacing tan with sin in warpFilter(_:) and you’ll get an interesting distortion effect! :]

Loading the Warp Kernel

Similar to the filter you created for the color kernel, you’ll create a custom filter to load and initialize the warp kernel.

Create a new Swift file in the Filters group. Name it WarpFilter.swift and add:

import CoreImage

// 1
class WarpFilter: CIFilter {
  var inputImage: CIImage?
  // 2
  static var kernel: CIWarpKernel = { () -> CIWarpKernel in
    guard let url = Bundle.main.url(
      forResource: "WarpFilterKernel.ci",
      withExtension: "metallib"),
    let data = try? Data(contentsOf: url) else {
      fatalError("Unable to load metallib")
    }

    guard let kernel = try? CIWarpKernel(
      functionName: "warpFilter",
      fromMetalLibraryData: data) else {
      fatalError("Unable to create warp kernel")
    }

    return kernel
  }()

  // 3
  override var outputImage: CIImage? {
    guard let inputImage = inputImage else { return .none }

    return WarpFilter.kernel.apply(
      extent: inputImage.extent,
      roiCallback: { _, rect in
        return rect
      },
      image: inputImage,
      arguments: [])
  }
}

Here, you:

1. Created WarpFilter as a subclass of CIFilter with inputImage as the input parameter.
2. Next, you declare the static property kernel to load the contents of WarpFilterKernel.ci.metallib. You then create an instance of CIWarpKernel using the contents of .metallib.
3. Finally, you provide the output by overriding outputImage. Within override, you apply the kernel to inputImage using apply(extent:roiCallback:arguments:) and return the result.

Applying the Warp Kernel Filter

Open ImageProcessor.swift. Add the following to ImageProcessor:

private func applyWarpKernel(input: CIImage) {
  let filter = WarpFilter()
  filter.inputImage = input
  if let outputImage = filter.outputImage,
    let renderImage = renderAsUIImage(outputImage) {
    output = renderImage
  }
}

Here, you declare applyColorKernel(input:), which takes CIImage as input. You then create an instance of WarpFilter and set inputImage.

The filter’s outputImage has the warp kernel applied. You then create an instance of UIImage using renderAsUIImage(_:) and save it to output.

Next, add the following case to process(painting:effect:), below case .colorKernel:

case .warpKernel:
  applyWarpKernel(input: input)

Here, you handle the case for .warpKernel and call applyWarpKernel(input:) to apply the warp kernel filter.

Finally, open PaintingWall.swift. Add the following case in the switch statement right below case 1 in action:

case 2:
  effect = .warpKernel

This sets the effect to .warpKernel for the third painting.

Build and run. Tap the painting of Salvator Mundi. You’ll see an interesting warp-based tile effect applied.

Congrats! You applied your own touch to a masterpiece! ;]

Challenge: Implementing a Blend Kernel

The CIBlendKernel is optimized for blending two images. As a fun challenge, implement a custom filter for CIBlendKernel. Some hints:

1. Create a subclass of CIFilter that takes in two images: an input image and a background image.
2. Use the built-in available CIBlendKernel kernels. For this challenge, use the built-in multiply blend kernel.
3. Create a method in ImageProcessor that applies the blend kernel filter to the image and sets the result as the output. You can use the multi_color image provided in the project assets as the background image for the filter. In addition, handle the case for .blendKernel.
4. Apply this filter to the fourth image in PaintingWall.swift.

You’ll find the solution implemented in the final project available in the downloaded materials. Good luck!

Debugging Core Image Issues

Knowing how Core Image renders an image can help you debug when the image doesn’t appear the way you expected. The easiest way is using Core Image Quick Look when debugging.

Using Core Image Quick Look

Open ImageProcessor.swift. Put a breakpoint on the line where you set output in applyColorKernel(input:). Build and run. Tap “The Last Supper”.

When you hit the breakpoint, hover over outputImage. You’ll see a small popover that shows the address.

Click the eye symbol. A window will appear that shows the graph that makes the image. Pretty cool, huh?