Chapters
Chapters

Chapters

Hide chapters

Metal by Tutorials

Third Edition · macOS 12 · iOS 15 · Swift 5.5 · Xcode 13

Before You Begin

Section 0: 6 chapters
Show chapters Hide chapters

Section I: Beginning Metal

Section 1: 10 chapters
Show chapters Hide chapters

Section II: Intermediate Metal

Section 2: 8 chapters
Show chapters Hide chapters

Section III: Advanced Metal

Section 3: 8 chapters
Show chapters Hide chapters

Section IV: Ray Tracing

Section 4: 7 chapters
Show chapters Hide chapters

30. Metal Performance Shaders
Written by Caroline Begbie & Marius Horga

Heads up... You're reading this book for free, with parts of this chapter shown beyond this point as scrambled text.

You can unlock the rest of this book, and our entire catalogue of books and videos, with a Kodeco Personal Plan.

Unlock now

In Chapter 19, “Tessellation & Terrains”, you had a brief taste of using the Metal Performance Shaders (MPS) framework. MPS consists of low-level, fine-tuned, high-performance kernels that run off the shelf with minimal configuration. In this chapter, you’ll dive a bit deeper into the world of MPS.

Overview

The MPS kernels make use of data-parallel primitives that are written in such a way that they can take advantage of each GPU family’s characteristics. The developer doesn’t have to care about which GPU the code needs to run on, because the MPS kernels have multiple versions of the same kernel written for every GPU you might use. Think of MPS kernels as convenient black boxes that work efficiently and seamlessly with your command buffer. Simply give it the desired effect, a source and destination resource (buffer or texture), and then encode GPU commands on the fly!

The Sobel Filter

The Sobel filter is a great way to detect edges in an image.

The Sobel filter
Xti Janiy gihvej

let shader = MPSImageSobel(device: device)
shader.encode(
  commandBuffer: commandBuffer,
  sourceTexture: inputImage,
  destinationTexture: drawable.texture)

Image Processing

There are a few dozen MPS image filters, among the most common being:

A Gaussian blur matrix
I Jaiyteoz mson cuhqay

Convolution
Zusjafehuel

(6 * 1  +  7 * 2  +  3 * 1  +
 4 * 2  +  9 * 4  +  8 * 2  +
 9 * 1  +  2 * 2  +  3 * 1) / 16 = 6
Convolution applied to border pixels
Medxahizuaz attvaen ci gilrin mogomm

(0 * 1  +  0 * 2  +  0 * 1  +
 0 * 2  +  6 * 4  +  7 * 2  +
 0 * 1  +  4 * 2  +  9 * 1) / 9 = 6

Bloom

The bloom effect is quite a spectacular one. It amplifies the brightness of objects in the scene and makes them look luminous as if they’re emitting light themselves.

The bloom effect
Cce vxuej ogjurc

The Starter Project

➤ In Xcode, open the starter project for this chapter and build and run the app.

Setting Up the Textures

➤ In the Post Processing group, open Bloom.swift, and import the MPS framework:

import MetalPerformanceShaders

var outputTexture: MTLTexture!
var finalTexture: MTLTexture!

outputTexture = TextureController.makeTexture(
  size: size,
  pixelFormat: view.colorPixelFormat,
  label: "Output Texture",
  usage: [.shaderRead, .shaderWrite])
finalTexture = TextureController.makeTexture(
  size: size,
  pixelFormat: view.colorPixelFormat,
  label: "Final Texture",
  usage: [.shaderRead, .shaderWrite])

Image Threshold to Zero

The Metal Performance Shader MPSImageThresholdToZero is a filter that returns either the original value for each pixel having a value greater than a specified brightness threshold or 0. It uses the following test:

destinationColor =
  sourceColor > thresholdValue ? sourceColor : 0

guard
  let drawableTexture =
    view.currentDrawable?.texture else { return }
let brightness = MPSImageThresholdToZero(
  device: Renderer.device,
  thresholdValue: 0.5,
  linearGrayColorTransform: nil)
brightness.label = "MPS brightness"
brightness.encode(
  commandBuffer: commandBuffer,
  sourceTexture: drawableTexture,
  destinationTexture: outputTexture)

metalView.framebufferOnly = false

The Blit Command Encoder

➤ Open Bloom.swift, and add this to the end of postProcess(view:commandBuffer:):

finalTexture = outputTexture
guard let blitEncoder = commandBuffer.makeBlitCommandEncoder()
  else { return }
let origin = MTLOrigin(x: 0, y: 0, z: 0)
let size = MTLSize(
  width: drawableTexture.width,
  height: drawableTexture.height,
  depth: 1)
blitEncoder.copy(
  from: finalTexture,
  sourceSlice: 0,
  sourceLevel: 0,
  sourceOrigin: origin,
  sourceSize: size,
  to: drawableTexture,
  destinationSlice: 0,
  destinationLevel: 0,
  destinationOrigin: origin)
blitEncoder.endEncoding()
Brightness threshold
Frugqksusv ftquvyiqc

Gaussian Blur

MPSImageGaussianBlur is a filter that convolves an image with a Gaussian blur with a given sigma value (the amount of blur) in both the X and Y directions.

let blur = MPSImageGaussianBlur(
  device: Renderer.device,
  sigma: 9.0)
blur.label = "MPS blur"
blur.encode(
  commandBuffer: commandBuffer,
  inPlaceTexture: &outputTexture,
  fallbackCopyAllocator: nil)
Brightness and blur
Kgojydyuls awt vkup

Image Add

The final part of creating the bloom effect is to add the pixels of this blurred image to the pixels of the original render.

let add = MPSImageAdd(device: Renderer.device)
add.encode(
  commandBuffer: commandBuffer,
  primaryTexture: drawableTexture,
  secondaryTexture: outputTexture,
  destinationTexture: finalTexture)
Brightness, blur and add
Fwawjgpiqj, pvuy urv ufm

let brightness = MPSImageThresholdToZero(
  device: Renderer.device,
  thresholdValue: 0.8,
  linearGrayColorTransform: nil)
Glowing skeletons
Mhebodz yxinujawx

Matrix / Vector Mathematics

You learned in the previous section how you could quickly apply a series of MPS filters that are provided by the framework. But what if you wanted to make your own filters?

import MetalPerformanceShaders

guard let device = MTLCreateSystemDefaultDevice(),
      let commandQueue = device.makeCommandQueue()
else { fatalError() }

let size = 4
let count = size * size

guard let commandBuffer = commandQueue.makeCommandBuffer()
else { fatalError() }

commandBuffer.commit()
commandBuffer.waitUntilCompleted()

func createMPSMatrix(withRepeatingValue: Float) -> MPSMatrix {
  // 1
  let rowBytes = MPSMatrixDescriptor.rowBytes(
    forColumns: size,
    dataType: .float32)
  // 2
  let array = [Float](
    repeating: withRepeatingValue,
    count: count)
  // 3
  guard let buffer = device.makeBuffer(
    bytes: array,
    length: size * rowBytes,
    options: [])
  else { fatalError() }
  // 4
  let matrixDescriptor = MPSMatrixDescriptor(
    rows: size,
    columns: size,
    rowBytes: rowBytes,
    dataType: .float32)

  return MPSMatrix(buffer: buffer, descriptor: matrixDescriptor)
}

let A = createMPSMatrix(withRepeatingValue: 3)
let B = createMPSMatrix(withRepeatingValue: 2)
let C = createMPSMatrix(withRepeatingValue: 1)

let multiplicationKernel = MPSMatrixMultiplication(
  device: device,
  transposeLeft: false,
  transposeRight: false,
  resultRows: size,
  resultColumns: size,
  interiorColumns: size,
  alpha: 1.0,
  beta: 0.0)

multiplicationKernel.encode(
  commandBuffer: commandBuffer,
  leftMatrix: A,
  rightMatrix: B,
  resultMatrix: C)

// 1
let contents = C.data.contents()
let pointer = contents.bindMemory(
  to: Float.self,
  capacity: count)
// 2
(0..<count).map {
  pointer.advanced(by: $0).pointee
}
Show result
Kpes yitatp

Value history
Cuzeo wetjaty

Challenge

You may have noticed that in the app where you did the bloom post processing, the Outline option does nothing. Your challenge is to fill out Outline.swift so that you have an outline render:

Outline
Uelpala

Key Points

  • Metal Performance Shaders are compute kernels that are performant and easy to use.
  • The framework has filters for image processing, implementations for neural networks, can solve systems of equations with matrix multiplication, and has optimized intersection testing for ray tracing.
  • Convolution takes a small matrix and applies it to a larger matrix. When applied to an image, you can blur or sharpen or distort the image.
  • Bloom adds a glow effect to an image, replicating real world camera artifacts that show up in bright light.
  • The threshold filter can filter out pixels under a given brightness threshold.
29. Advanced Lighting
31. Performance Optimization
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2024 Kodeco Inc.

You're reading for free, with parts of this chapter shown as scrambled text. Unlock this book, and our entire catalogue of books and videos, with a Kodeco Personal Plan.

Unlock now