24. Performance Optimization

Written by Marius Horga

In the previous chapter, you took a first stab at optimizing your app by profiling your shaders and using Instruments to find even more bottlenecks to get rid of. In this chapter, you’ll look at:

1. CPU-GPU Synchronization
2. Multithreading
3. GPU Families
4. Memory Management
5. Best Practices

CPU-GPU synchronization

Always aim to minimize the idle time between frames.

Managing dynamic data can be a little tricky. Take the case of Uniforms. You’re changing them usually once per frame on the CPU. That means that the GPU has to wait until the CPU has finished writing the buffer before it can read the buffer. Instead, you can simply have a pool of reusable buffers.

Triple buffering is a well-known technique in the realm of synchronization. The idea is to use three buffers at a time. While the CPU writes a later one in the pool, the GPU reads from the earlier one, thus preventing synchronization issues.

You might ask, why three and not just two or a dozen? With only two buffers, there’s a high risk that the CPU will try to write the first buffer again before the GPU finished reading it even once. With too many buffers, there’s a high risk of performance issues.

Before you implement the triple buffering, use Instruments to run a Metal System Trace (MST) session and get a baseline level of the CPU activity:

Notice that most tasks peak at about 10% and this is fine, assuming that the GPU has enough work to do on its own without waiting for more work from the CPU.

All right, time to implement that triple buffering pool like a champ!

Open the starter project that comes with this chapter. In Scene.swift, replace this line:

var uniforms = Uniforms()

With this code:

static let buffersInFlight = 3
var uniforms = [Uniforms](repeating: Uniforms(), 
                          count: buffersInFlight)
var currentUniformIndex = 0

Here, you replaced the uniforms variable with an array of three buffers and defined an index to keep track of the current buffer in use.

In update(deltaTime:), replace this code:

uniforms.projectionMatrix = camera.projectionMatrix
uniforms.viewMatrix = camera.viewMatrix

With this:

uniforms[currentUniformIndex].projectionMatrix = 
    camera.projectionMatrix
uniforms[currentUniformIndex].viewMatrix = camera.viewMatrix
currentUniformIndex = 
    (currentUniformIndex + 1) % Scene.buffersInFlight

Here, you adapted the update method to include the new uniforms array and created a way to have the index loop around always taking the values 0, 1 and 2.

Back in Renderer.swift, add this line to draw(in:), before the renderables loop:

let uniforms = scene.uniforms[scene.currentUniformIndex]

Replace scene.uniforms with uniforms in the two places Xcode complains about.

Build and run the project. It’ll show the same scene as before. Run another MST session and notice that now the CPU activity has increased.

This is both good news and bad news. It’s good news because that means the GPU is not getting more work to do. The bad news is that now the CPU and the GPU will spar over using the same resources.

This is known as resource contention and involves conflicts, called race conditions, over accessing shared resources by both the CPU and GPU. They’re trying to read/write the same uniform, causing unexpected results.

In the image below, the CPU is ready to start writing the third buffer again. However, that would require the GPU to have finished reading it, which is not the case here.

What you need here is a way to delay the CPU writing until the GPU has finished reading it.

In Chapter 8, “Character Animation,” you solved this synchronization issue in a naive way by using waitUntilCompleted() on your command buffer. A more performant way, however, is the use of a synchronization primitive called a semaphore, which is a convenient way of keeping count of the available resources — your triple buffer in this case.

Here’s how a a semaphore works:

• Initialize it to a maximum value that represents the number of resources in your pool (3 buffers here).
• Inside the draw call the thread tells the CPU to wait until a resource is available and if one is, it takes it and decrements the semaphore value by one.
• If there are no more available resources, the current thread is blocked until the semaphore has at least one resource available.
• When a thread finishes using the resource, it’ll signal the semaphore by increasing its value and by releasing the hold on the resource.

Time to put this theory into practice.

At the top of Renderer, add this new property:

var semaphore: DispatchSemaphore

In init(metalView:), add this line before super.init():

semaphore = DispatchSemaphore(value: Scene.buffersInFlight)

Add this line at the top of draw(in:):

_ = semaphore.wait(timeout: .distantFuture)

At the end of draw(in:), but before committing the command buffer, add this:

commandBuffer.addCompletedHandler { _ in
  self.semaphore.signal()
}

At the end of draw(in:), remove:

commandBuffer.waitUntilCompleted()

Build and run the project again, making sure everything still renders fine as before.

Run another MST session and compare the performance metrics with the previous ones.

If you look at the GFX bar under your specific graphics processor, the gaps are all narrower now because the GPU is not sitting idle as much as it was sitting before. You can intensify the rendering workload by increasing the number of trees, rocks or grass blades, and then the gaps might be completely gone. Those “Thread blocked waiting for next drawable” messages are also gone.

Notice an old issue you did not fix yet. Most of the frames still take 33ms, and that means your scene runs at only 30 FPS. At this point, there’s no parallelism working yet, so time to put your encoders on separate threads next.

Multithreading

Build all known pipelines up front and asynchronously.

When balancing workloads you can find yourself in one of these two extremes: the CPU is giving the GPU too much work to do (your app is GPU-bound), or the CPU is working too much and the GPU is sitting idle (your app is CPU-bound). You will work on managing the CPU workload next.

The biggest performance gain will come from running different command buffers on different threads. You can even split one encoder into new smaller encoders and run them on multiple threads using MTLParallelRenderEncoder.

Assume that in your project you have grass that takes 10ms to render; you have trees that take another 7ms to render; and you have rocks + skybox + ground that all take 5ms to render.

Instead of having an encoder that takes 22ms to finish, you could split the encoder into three smaller new encoders and run them in parallel on an MTLParallelRenderEncoder that would have all three threads finish by the time the longer running thread finishes (10ms).

You don’t have to type the code for building a parallel render command encoder, but it would look similar to this:

let commandBuffer = Renderer.commandQueue.makeCommandBuffer()
let descriptor = MTLRenderPassDescriptor()
let parallelEncoder = commandBuffer.makeParallelRenderCommandEncoder(
                                    descriptor: descriptor)
let encoder1 = parallelEncoder.makeRenderCommandEncoder()
// ... encoder1.draw() ...
encoder1.endEncoding()
let encoder2 = parallelEncoder.makeRenderCommandEncoder()
// ... encoder2.draw() ...
encoder2.endEncoding()
parallelEncoder.endEncoding()
commandBuffer.commit()

You’re going to implement multithreaded command buffers where each command buffer has its own encoder, and they’re running on separate threads:

At the top of Renderer, add this new property:

let dispatchQueue = DispatchQueue(label: "Queue", 
                                  attributes: .concurrent)

In draw(in:), locate // compute debugging, and create a second command buffer for the compute command encoder by replacing this line:

guard let computeEncoder = 
    commandBuffer.makeComputeCommandEncoder() 
else {

With this code:

guard let computeCommandBuffer = 
        Renderer.commandQueue.makeCommandBuffer(),
      let computeEncoder = 
        computeCommandBuffer.makeComputeCommandEncoder() else {

At the end of draw(in:), replace this code:

commandBuffer.addCompletedHandler { _ in
  self.semaphore.signal()
}
commandBuffer.commit()

With this:

// 1
commandBuffer.enqueue()
computeCommandBuffer.enqueue()
// 2
dispatchQueue.async(execute: commandBuffer.commit)
weak var sem = semaphore
dispatchQueue.async {
  computeCommandBuffer.addCompletedHandler { _ in
    sem?.signal()
  }
  computeCommandBuffer.commit()
}
// 3
__dispatch_barrier_sync(dispatchQueue) {}

Going through everything:

1. Use the enqueue() function, which is the explicit way to guarantee buffer execution order.
2. Dispatch each buffer in parallel, asynchronously, and then commit the work for each buffer.
3. Use a barrier to block the threads until all of them finish.

Run a new MST session, and compare the metrics with previous traces.

Notice on the bottom bar that some of the frames are taking 16.7ms to render again. This is a great start, but your work is not done yet. You should lower instanceCount to see if that helps get your project back to a stable 60 FPS status.

Also, notice the command encoders are now rendered on separate threads which is what you set out to achieve in this part of the chapter.

You should be proud of what you’ve achieved in this chapter so far. Keep tweaking your project like a pro until you’re pleased with the performance.

GPU families

GPU families are classes of GPUs categorized by device and/or build target type. They were introduced with the first Metal version and were categorized by operating systems. At WWDC 2019 Apple repurposed and renamed them as follows:

1. Apple Families: Refers to GPUs manufactured by Apple for iOS and tvOS devices:

• Apple 1 - the A7 GPU.
• Apple 2 - the A8 GPU.
• Apple 3 - the A9 and A10 GPUs.
• Apple 4 - the A11 GPU.
• Apple 5 - the A12 GPU.
• Apple 6 - the A13 GPU.

2. Mac Families: Refers to GPUs manufactured by Intel, Nvidia and AMD for Macs:

• Mac 1 - the Intel HD Graphics 4000, Intel Iris, Intel Iris Pro, Intel Iris Graphics 6100, and all the Nvidia GeForce GT GPUs.
• Mac 2 - the Intel HD Graphics 5xx, Intel Iris Plus Graphics 6xx, AMD Radeon, AMD Radeon Pro, and AMD FirePro GPUs.

3. Common Families: Refers to Metal features that are available to all GPU families.

• Common 1 - all the universally supported features.
• Common 2 - Indirect Draw/Dispatch, Counting Occlusion Queries, Tessellation, Read/Write Buffer Arguments, Arrays of Textures/Samplers, Compressed Volume Textures, Metal Performance Shaders, and more.
• Common 3 - Stencil Feedback, MSAA Depth/Stencil Resolve, Programmable Sample Positions, Invariant Vertex Position, Indirect Stage-In, Indirect Command Buffers, Quad-scoped Shuffle/Broadcast, Cube Texture Arrays, Read/Write Texture Arguments, Attachment-less Render Passes, Layered Rendering, Multi-Viewport Rendering, Argument Buffers, Pipelined Compute, Indirect Tessellation, Heap Placement, Texture Swizzle, and more.

4. iOSMac Families: Refers to Metal features for Mac Family GPUs running iPad apps ported to macOS via Catalyst.

• iOSMac 1 - features from the Common 2 family, plus BC Pixel Formats, Managed Textures, Cube Texture Arrays, Read/Write Textures Tier 1, Layered Rendering, Multiple Viewports/Scissors and Indirect Tessellation.
• iOSMac 2 - features from the Common 3 family, plus BC Pixel Formats and Managed Textures.

Note: A GPU can be a member of more than one family so it supports one of the Common families and one or more of the other families. For a complete list of supported features, consult Apple’s webpage at https://developer.apple.com/metal/Metal-Feature-Set-Tables.pdf

You can test what GPU Families your devices have by using an #available clause. Add this code at the end of init(metalView:) in Renderer.swift:

let devices = MTLCopyAllDevices()
for device in devices {
  if #available(macOS 10.15, *) {
    if device.supportsFamily(.mac2) {
      print("\(device.name) is a Mac 2 family gpu running on macOS Catalina.")
    }
    else {
      print("\(device.name) is a Mac 1 family gpu running on macOS Catalina.")
    }
  }
  else {
    if device.supportsFeatureSet(.macOS_GPUFamily2_v1) {
      print("You are using a recent GPU with an older version of macOS.")
    }
    else {
      print("You are using an older GPU with an older version of macOS.")
    }
  }
}

Build and run. The output in the debug console will look something like this:

AMD Radeon RX Vega 64 is a Mac 2 family gpu running on macOS Catalina.
Intel(R) HD Graphics 530 is a Mac 2 family gpu running on macOS Catalina.
AMD Radeon Pro 450 is a Mac 2 family gpu running on macOS Catalina.

Memory management

Whenever you create a buffer or a texture, you should consider how to configure it for fast memory access and driver performance optimizations. Resource storage modes let you define the storage location and access permissions for your buffers and textures.

All iOS and tvOS devices support a unified memory model where both the CPU and the GPU share the system memory, while macOS devices support a discrete memory model where the GPU has its own memory. In iOS and tvOS, the Shared mode (MTLStorageModeShared) defines system memory accessible to both CPU and GPU, while Private mode (MTLStorageModePrivate) defines system memory accessible only to the GPU. The Shared mode is the default storage mode on all three operating systems.

macOS also has a Managed mode (MTLStorageModeManaged) that defines a synchronized memory pair for a resource, with one copy in system memory and another in video memory for faster CPU and GPU local accesses.

There are basically only four main rules to keep in mind, one for each of the storage modes:

• Shared: Default on macOS buffers, iOS/tvOS resources; not available on macOS textures.
• Private: Mostly use when data is only accessed by GPU.
• Memoryless: Only for iOS/tvOS on-chip temporary render targets (textures).
• Managed: Default mode for macOS textures; not available on iOS/tvOS resources.

For a better big picture, here is the full cheat sheet in case you might find it easier to use than remembering the rules above:

The most complicated case is when working with macOS buffers and when the data needs to be accessed by both the CPU and the GPU. You should choose the storage mode based on whether one or more of the following conditions are true:

• Private: For large-sized data that changes at most once, so it is not “dirty” at all. Create a source buffer with a Shared mode and then blit its data into a destination buffer with a Private mode. Resource coherency is not necessary in this case as the data is only accessed by the GPU. This operation is the least expensive (a one-time cost).
• Managed: or medium-sized data that changes infrequently (every few frames), so it is partially “dirty”. One copy of the data is stored in system memory for the CPU and another copy is stored in GPU memory. Resource coherency is explicitly managed by synchronizing the two copies.
• Shared: For small-sized data that is updated every frame, so it is fully dirty. Data resides in the system memory and is visible and modifiable by both the CPU and the GPU. Resource coherency is only guaranteed within command buffer boundaries.

How do you make sure coherency is guaranteed? First, make sure that all the modifications done by the CPU are finished before the command buffer is committed (check if the command buffer status property is MTLCommandBufferStatusCommitted). After the GPU finishes executing the command buffer, the CPU should only start making modifications again only after the GPU is signaling the CPU that the command buffer finished executing (check if the command buffer status property is MTLCommandBufferStatusCompleted).

Finally, how is synchronization done for macOS resources?

• For buffers: After a CPU write, use didModifyRange to inform the GPU of the changes so Metal can update that data region only; after a GPU write use synchronize(resource:) within a blit operation, to refresh the caches so the CPU can access the updated data.
• For textures: After a CPU write, use one of the two replace region functions to inform the GPU of the changes so Metal can update that data region only; after a GPU write use one of the two synchronize functions within a blit operation to allow Metal to update the system memory copy after the GPU finished modifying the data.

Now, look at what happens on the GPU when you send it data buffers. Here is a typical vertex shader example:

vertex Vertices vertex_func(
  const device Vertices *vertices [[buffer(0)]], 
  constant Uniforms &uniforms [[buffer(1)]], 
  uint vid [[vertex_id]]) {}

The Metal Shading Language implements address space qualifiers to specify the region of memory where a function variable or argument is allocated:

• device: Refers to buffer memory objects allocated from the device memory pool that are both readable and writeable unless the keyword const precedes it in which case the objects are only readable.
• constant: Refers to buffer memory objects allocated from the device memory pool but that are read-only. Variables in program scope must be declared in the constant address space and initialized during the declaration statement. The constant address space is optimized for multiple instances executing a graphics or kernel function accessing the same location in the buffer.
• threadgroup: Used to allocate variables used by kernel functions only and they are allocated for each threadgroup executing the kernel, are shared by all threads in a threadgroup and exist only for the lifetime of the threadgroup that is executing the kernel.
• thread: Refers to the per-thread memory address space. Variables allocated in this address space are not visible to other threads. Variables declared inside a graphics or kernel function are allocated in the thread address space.

Starting with macOS Catalina some Mac systems directly connect GPUs to each other (they are said to be in the same peer group), allowing you to quickly transfer data between them. These connections are not only faster, but they also avoid using the memory bus between the CPU and GPUs, leaving it available for other tasks. If your app uses multiple GPUs, test to see if they’re connected (if device.peerGroupID returns a non-zero value), and when they are, you can use a blit command encoder to transfer data. You can read more on Apple’s webpage at https://developer.apple.com/documentation/metal/transferring_data_between_connected_gpus

Best practices

When you are after squeezing the very last ounce of performance from your app, you should always remember to follow a golden set of best practices. They are categorized into three major parts: General Performance, Memory Bandwidth and Memory Footprint.

General performance best practices

The next five best practices are general and apply to the entire pipeline.

1. Choose the right resolution.

The game or app UI should be at native or close to native resolution so that the UI will always look crisp no matter the display size. Also, it is recommended (albeit, not mandatory) that all resources have the same resolution. You can check the resolutions in the GPU Debugger, on the Dependency Viewer. Below is the multi-pass render from Chapter 14, “Multipass & Deferred Rendering”:

Notice here that the G-Buffer pass uses render targets that have a different resolution than the shadow and composition passes. You should consider the performance trade-offs of each image resolution and carefully choose the scenario that best fits your app needs.

2. Minimize non-opaque overdraw.

Ideally, you’ll want to only draw each pixel once. That means you will want only one fragment shader process per pixel. You can check the status of that in the Metal Frame Debugger, by clicking the Counters gauge. In the right side pane at the bottom there is a filter bar. In there type FS Invocations followed by pressing the Enter key and then type again Pixels Stored followed by pressing the Enter key again:

Overdraw would be if the number of shader invocations would be much larger than the number of pixels stored. In your case, they seem to all match which means it is an opaque scene. The best practice is to render opaque meshes first followed by translucent meshes. If they are fully transparent that means they are invisible so they should never be rendered.

3. Submit GPU work early.

You can reduce latency and improve the responsiveness of your renderer by making sure all the off-screen GPU work is done early and is not waiting for the on-screen part to start.

You can do that by using two or more command buffers per frame:

create off-screen command buffer
encode work for the GPU
commit off-screen command buffer
...
get the drawable
create on-screen command buffer
encode work for the GPU
present the drawable
commit on-screen command buffer

Create the off-screen command buffer(s) and commit the work to the GPU as early as possible. Get the drawable as late as possible in the frame and then have a final command buffer that only contains the on-screen work.

4. Stream resources efficiently.

All resources should be allocated at launch time if they are available because that will take time and will prevent render stalls later. If you need to allocate resources at runtime because the renderer streams them, you should make sure you do that from a dedicated thread. You can see the resource allocations in the Metal System Trace, under the Allocation track:

You can see here that there are a few allocations, but all at lunch time. If there were allocations at runtime you would notice them later on that track and identify potential stalls because of them.

5. Design for sustained performance.

You should test your renderer under serious thermal state. This can improve the overall thermals of the device as well as the stability and responsiveness of your renderer.

Xcode now lets you see and change the thermal state in the Devices window from Window ▸ Devices and Simulators:

You can also use Xcode’s Energy Gauge to verify the thermal state that the device is running at:

Memory Bandwidth best practices

Since memory transfers for render targets and textures are costly, the next six best practices are targeted to memory bandwidth and how to use shared and tiled memory more efficiently.

6. Compress texture assets.

Compressing textures is very important because sampling large textures may be inefficient. For that reason, you should generate mipmaps for textures that can be minified. You should also compress large textures to accommodate the memory bandwidth needs. There are various compression formats available. For example, for older devices you could use PVRTC and for newer devices you could use ASTC. Review Chapter 6, “Textures,” for how to create mipmaps and change texture formats in the asset catalog.

With the frame captured, you can use the Metal Memory Viewer to verify compression format, mipmap status and size. You can change which columns are displayed, by right clicking the column heading:

Some textures, such as render targets, cannot be compressed ahead of time so you will have to do it at runtime instead. The good news is, the A12 GPU and newer supports lossless texture compression which allows the GPU to compress textures for faster access.

7. Optimize for faster GPU access.

You should configure your textures correctly to use the appropriate storage mode depending on the use case. Use the private storage mode so only the GPU has access to the texture data, allowing optimization of the contents:

textureDescriptor.storageMode = .private 
textureDescriptor.usage = [ .shaderRead, .renderTarget ]
let texture = device.makeTexture(descriptor: textureDescriptor)

You shouldn’t set any unnecessary usage flags such as unknown, shaderWrite or pixelView, since they may disable compression.

Shared textures that can be accessed by the CPU as well as the GPU, should explicitly be optimized after any CPU update on their data:

textureDescriptor.storageMode = .shared 
textureDescriptor.usage = .shaderRead
let texture = device.makeTexture(descriptor: textureDescriptor)
// update texture data
texture.replace(region: region, mipmapLevel: 0, 
                withBytes: bytes, 
                bytesPerRow: bytesPerRow)
let blitCommandEncoder = commandBuffer.makeBlitCommandEncoder()
blitCommandEncoder.optimizeContentsForGPUAccess(
                       texture: texture) 
blitCommandEncoder.endEncoding()

Again, the Metal Memory Viewer will show you the storage mode and usage flag for all textures, along with noticing which ones are compressed textures already, as in the previous image.

8. Choose the right pixel format.

Choosing the correct pixel format is crucial. Not only will larger pixel formats use more bandwidth, but the sampling rate also depends on the pixel format. You should try to avoid using pixel formats with unnecessary channels and also try to lower precision whenever possible. You’ve generally been using the RGBA8Unorm pixel format in this book, however, when you needed greater accuracy for the G-Buffer in Chapter 14, “Multipass & Deferred Rendering,” you used a 16-bit pixel format. Again, you can use the Metal Memory Viewer to see the pixel formats for textures.

9. Optimize load and store actions.

Load and store actions for render targets can also affect bandwidth. If you have a suboptimal configuration of your pipelines caused by unnecessary load/store actions, you might create false dependencies. An example of optimized configuration would be this:

renderPassDescriptor.colorAttachments[0].loadAction = .clear 
renderPassDescriptor.colorAttachments[0].storeAction = .dontCare

In this case, you’re configuring a color attachment to be transient which means you do not want to load or store anything from it. You can verify the current actions set on render targets in the Dependency Viewer.

As you can see, there is an exclamation point that suggests that you should not store the last render target.

10. Optimize multi-sampled textures.

iOS devices have very fast multi-sampled render targets (MSAA) because they resolve from Tile Memory so it is best practice to consider MSAA over native resolution. Also, make sure not to load or store the MSAA texture and set its storage mode to memoryless:

textureDescriptor.textureType = .type2DMultisample 
textureDescriptor.sampleCount = 4 
textureDescriptor.storageMode = .memoryless
let msaaTexture = 
    device.makeTexture(descriptor: textureDescriptor)
renderPassDesc.colorAttachments[0].texture = msaaTexture 
renderPassDesc.colorAttachments[0].loadAction = .clear 
renderPassDesc.colorAttachments[0].storeAction = .
    multisampleResolve

Dependency Viewer will again help you see the current status set for load/store actions.

11. Leverage tile memory.

Metal provides access to Tile Memory for several features such as programmable blending, image blocks and tile shaders. Deferred shading requires storing the G-Buffer in a first pass, and then sampling from its textures in the second lighting pass where the final color accumulates into a render target. This is very bandwidth-heavy.

iOS allows fragment shaders to access pixel data directly from Tile Memory in order to leverage programmable blending. This means that you can store the G-Buffer data on Tile Memory and all the light accumulation shaders can access it within the same render pass. The four G-Buffer attachments are fully transient and only the final color and depth are stored, so it’s very efficient.

Memory Footprint best practices

12. Use memoryless render targets.

As mentioned previously in best practices 9 and 10, you should be using memoryless storage mode for all transient render targets which do not need a memory allocation, that is, are not loaded from or stored to memory:

textureDescriptor.storageMode = .memoryless 
textureDescriptor.usage = [ .shaderRead, .renderTarget ]
// for each G-Buffer texture
textureDescriptor.pixelFormat = gBufferPixelFormats[i] 
gBufferTextures[i] = 
    device.makeTexture(descriptor: textureDescriptor)
renderPassDescriptor.colorAttachments[i].texture = 
    gBufferTextures[i] 
renderPassDescriptor.colorAttachments[i].loadAction = .clear 
renderPassDescriptor.colorAttachments[i].storeAction = .dontCare

You’ll be able to see the change immediately in the Dependency Viewer.

13. Avoid loading unused assets.

Loading all the assets into memory will increase the memory footprint so you should consider the memory and performance trade-off and only load all the assets that you know will be used. The GPU frame capture Memory Viewer will show you any unused resources:

Fortunately, your app correctly uses all its textures.

14. Use smaller assets.

You should only make the assets as large as necessary and consider again the image quality and memory trade-off of your asset sizes. Make sure that both textures and meshes are compressed. You may want to only load the smaller mipmap levels of your textures, or use lower level of detail meshes for distant objects.

15. Simplify memory-intensive effects.

Some effects may require large off-screen buffers, such as Shadow Maps and Screen Space Ambient Occlusion so you should consider the image quality and memory trade-off of all of those effects, potentially lower the resolution of all these large off-screen buffers and even disable the memory-intensive effects altogether when you are memory constrained.

16. Use Metal resource heaps.

Rendering a frame may require a lot of intermediate memory especially if your game becomes more complex in the post-process pipeline so it is very important to use Metal Resource Heaps for those effects and alias as much of that memory as possible. For example, you may want to reutilize the memory for resources which have no dependencies such as those for Depth of Field or Screen Space Ambient Occlusion.

Another advanced concept is that of purgeable memory. Purgeable memory has three states: non-volatile (when data should not be discarded), volatile (data can be discarded even when the resource may be needed) and empty (data has been discarded). Volatile and empty allocations do not count towards the application memory footprint because the system can either reclaim that memory at some point or has already reclaimed it in the past.

17. Mark resources as volatile.

Temporary resources may become a large part of the memory footprint and Metal will allow you to set the purgeable state of all the resources explicitly. You will want to focus on your caches that hold mostly idle memory and carefully manage their purgeable state, like in this example:

// for each texture in the cache
texturePool[i].setPurgeableState(.volatile)
// later on...
if (texturePool[i].setPurgeableState(.nonVolatile) == .empty) {
  // regenerate texture
}

18. Manage the Metal PSOs.

Pipeline State Objects (PSOs) encapsulate most of the Metal render state. You create them using a descriptor which contains vertex and fragment functions as well as other state descriptors. All of these will get compiled into the final Metal PSO.

Metal allows your application to load most of the rendering state up front, improving the performance over OpenGL. However, if you have limited memory make sure to not hold on to PSO references that you don’t need anymore. Also don’t hold on to Metal function references after you have created the PSO cache because they are not needed to render, they are only needed to create new PSOs.

Note: Apple have written a Metal Best Practices guide that provides great advice for optimizing your app: https://developer.apple.com/library/archive/documentation/3DDrawing/Conceptual/MTLBestPracticesGuide/index.html.

Where to go from here?

Getting the last ounce of performance out of your app is paramount. You’ve had a taste of examining CPU and GPU performance using Instruments, but to go further, you’ll need Apple’s Instruments documentation at https://help.apple.com/instruments/mac/10.0/.

Over the years, at every WWDC since Metal was introduced, Apple have produced some excellent WWDC videos describing Metal best practices and optimization techniques. Go to https://developer.apple.com/videos/graphics-and-games/metal/ and watch as many as you can, as often as you can.

Congratulations on completing the book! The world of Computer Graphics is vast and as complex as you want to make it. But now that you have the basics of Metal learned, even though current internet resources are few, you should be able to learn techniques described with other APIs such as OpenGL, Vulkan and DirectX. If you’re keen to learn more, look at the books suggested in references.markdown.