31. Best Practices

Written by Marius Horga & Caroline Begbie

When you want to squeeze the very last ounce of performance from your app, you should always remember to follow a golden set of best practices. These rules are categorized into three major parts: general performance, memory bandwidth and memory footprint. This chapter will guide you through all three.

General Performance Best Practices

The next five best practices are general and apply to the entire pipeline.

Choose the Right Resolution

The game or app UI should be at native or close to native resolution so the UI will always look crisp no matter the display size. Also, it is recommended (albeit not mandatory) that all resources have the same resolution. You can check the resolutions in the GPU Debugger on the dependency graph. Below is a partial view of the dependency graph from the multi-pass render in Chapter 14, “Deferred Rendering”:

Notice the size of the shadow pass render target. For crisper shadows, you should have a large texture, but you should consider the performance trade-offs of each image resolution and carefully choose the scenario that best fits your app needs.

Optimize Shader Pipelines

Group draw calls by shader to minimize changing your pipeline states. Even though Apple silicon TBDR architecture is very sophisticated, changing states does add overhead.

When creating your shaders, use function specialization like you did for the skeleton in Chapter 24, “Character Animation”. The fragment shaders currently have conditionals in them where a texture may be missing. This is an opportunity for improvement by changing to function specialization and removing the conditionals.

Submit GPU Work Early

You can reduce latency and improve the responsiveness of your renderer by making sure all of the off-screen GPU work is done early and is not waiting for the on-screen part to start. You can do that by using two or more command buffers per frame:

create off-screen command buffer
encode work for the GPU
commit off-screen command buffer
...
get the drawable
create on-screen command buffer
encode work for the GPU
present the drawable
commit on-screen command buffer

Create the off-screen command buffer(s) and commit the work to the GPU as early as possible. Get the drawable as late as possible in the frame, and then have a final command buffer that only contains the on-screen work.

Stream Resources Efficiently

All resources should be allocated at launch time — if they’re available — because that will take time and prevent render stalls later. If you need to allocate resources at runtime because the renderer streams them, you should make sure you do that from a dedicated thread.

You can see resource allocations in Instruments, in a Metal System Trace, under the GPU ➤ Allocation track:

You can see here that there are a few allocations, but all at launch time. If there were allocations at runtime, you would notice them later on that track and identify potential stalls because of them.

Design for Sustained Performance

You should test your renderer under a serious thermal state. This can improve the overall thermals of the device, as well as the stability and responsiveness of your renderer.

Xcode now lets you see and change the thermal state in the Devices window from Window ▸ Devices and Simulators.

You can also use Xcode’s Energy Impact gauge to verify the thermal state that the device is running at:

Memory Bandwidth Best Practices

Since memory transfers for render targets and textures are costly, the next five best practices are targeted to memory bandwidth and how to use shared and tiled memory more efficiently.

Compress Texture Assets

Compressing textures is very important because sampling large textures may be inefficient. For that reason, you should generate mipmaps for textures that can be minified. You should also compress large textures to accommodate the memory bandwidth needs. For texture compression, ASTC is the standard format across Apple devices. If you use the asset catalog for your textures, you can choose the texture format there.

With the frame captured, you can use the Metal Memory Viewer to verify compression format, mipmap status and size. You can change which columns are displayed by right-clicking the column heading:

Optimize for Faster GPU Access

You should configure your textures correctly to use the appropriate storage mode depending on the use case. Use the private storage mode so only the GPU has access to the texture data, allowing optimization of the contents:

textureDescriptor.storageMode = .private
textureDescriptor.usage = [ .shaderRead, .renderTarget ]
let texture = device.makeTexture(descriptor: textureDescriptor)

Use shared storage mode for textures only when the CPU needs to access them as well as the GPU.

You shouldn’t set any unnecessary usage flags. Only use shaderWrite when you actually need to write to the texture, such as during a compute shader.

Shared textures that can be accessed by the CPU as well as the GPU should explicitly be optimized after any CPU update on their data:

textureDescriptor.storageMode = .shared
textureDescriptor.usage = .shaderRead
let texture = device.makeTexture(descriptor: textureDescriptor)
// update texture data
texture.replace(
  region: region,
  mipmapLevel: 0,
  withBytes: bytes,
  bytesPerRow: bytesPerRow)
let blitCommandEncoder = commandBuffer.makeBlitCommandEncoder()
blitCommandEncoder
  .optimizeContentsForGPUAccess(texture: texture)
blitCommandEncoder.endEncoding()

Again, the Metal Memory Viewer shows you the storage mode and usage flag for all textures, along with noticing which ones are compressed textures already, as in the previous image.

Choose the Right Pixel Format

Choosing the correct pixel format is crucial. Not only will larger pixel formats use more bandwidth, but the sampling rate also depends on the pixel format. You should try to avoid using pixel formats with unnecessary channels and also try to lower precision whenever possible. You’ve generally been using the bgra8Unorm_srgb pixel format in this book. However, when you needed greater accuracy for the G-Buffer in Chapter 14, “Deferred Rendering”, you used a 16-bit pixel format. Again, you can use the Metal Memory Viewer to see the pixel formats for textures.

Optimize Load and Store Actions

Load and store actions for render targets can also affect bandwidth. If you have a suboptimal configuration of your pipelines caused by unnecessary load/store actions, you might create false dependencies. An example of optimized configuration would be as follows:

renderPassDescriptor.colorAttachments[0].loadAction = .clear
renderPassDescriptor.colorAttachments[0].storeAction = .dontCare

In this case, you’re configuring a color attachment to be transient, which means you do not want to load or store anything from it. You can verify the current actions set on render targets in the Dependency Viewer.

You can see here the GPU wrongly stores the depth texture, even though it isn’t passed to a following render pass.

Optimize Multi-Sampled Textures

Apple’s TBDR architecture handles MSAA efficiently in tile memory. When implementing MSAA, make sure not to load or store the MSAA texture and set its storage mode to memoryless:

textureDescriptor.textureType = .type2DMultisample
textureDescriptor.sampleCount = 4
textureDescriptor.storageMode = .memoryless
let msaaTexture = device.makeTexture(descriptor: textureDescriptor)
renderPassDesc.colorAttachments[0].texture = msaaTexture
renderPassDesc.colorAttachments[0].loadAction = .clear
renderPassDesc.colorAttachments[0].storeAction = .multisampleResolve

MSAA still increases GPU workload, so first evaluate whether the visual improvement is worth it.

Memory Footprint Best Practices

Use Memoryless Render Targets

As mentioned previously, you should be using memoryless storage mode for all transient render targets that do not need a memory allocation, that is, are not loaded from or stored to memory:

textureDescriptor.storageMode = .memoryless
textureDescriptor.usage = [.shaderRead, .renderTarget]
// for each G-Buffer texture
textureDescriptor.pixelFormat = gBufferPixelFormats[i]
gBufferTextures[i] =
  device.makeTexture(descriptor: textureDescriptor)
renderPassDescriptor.colorAttachments[i].texture =
  gBufferTextures[i]
renderPassDescriptor.colorAttachments[i].loadAction = .clear
renderPassDescriptor.colorAttachments[i].storeAction = .dontCare

You’ll be able to see the change immediately in the dependency graph.

Avoid Loading Unused Assets

Loading all the assets into memory will increase the memory footprint, so consider the memory and performance trade-off, and only load all the assets that you know will be used. The GPU frame capture Memory Viewer will show you any unused resources.

Use Smaller Assets

You should only make the assets as large as necessary and consider the image quality and memory trade-off of your asset sizes. Make sure that both textures and meshes are compressed. You may want to only load the smaller mipmap levels of your textures or use lower level of detail meshes for distant objects.

Simplify memory-intensive effects

Some effects may require large off-screen buffers, such as Shadow Maps and Screen Space Ambient Occlusion, so you should consider the image quality and memory trade-off of all of those effects, potentially lower the resolution of all these large off-screen buffers and even disable the memory-intensive effects altogether when you are memory constrained.

Use Metal Resource Heaps

Rendering a frame may require a lot of intermediate memory, especially if your game becomes more complex in the post-process pipeline, so consider using Metal Resource Heaps for those effects and alias as much of that memory as possible. For example, you may want to reutilize the memory for resources that have no dependencies, such as those for Depth of Field or Screen-Space Ambient Occlusion.

Another advanced concept is that of purgeable memory. Purgeable memory has three states: non-volatile (when data should not be discarded), volatile (data can be discarded even when the resource may be needed) and empty (data has been discarded). Volatile and empty allocations do not count towards the application’s memory footprint because the system can either reclaim that memory at some point or has already reclaimed it in the past.

Mark Resources as Volatile

Temporary resources may become a large part of the memory footprint and Metal will allow you to set the purgeable state of all the resources explicitly. You will want to focus on your caches that hold mostly idle memory and carefully manage their purgeable state, like in this example:

// for each texture in the cache
texturePool[i].setPurgeableState(.volatile)
// later on...
if (texturePool[i].setPurgeableState(.nonVolatile) == .empty) {
  // regenerate texture
}

Manage the Metal PSOs

Pipeline State Objects (PSOs) encapsulate most of the Metal render state. You create them using a descriptor that contains vertex and fragment functions as well as other state descriptors. All of these will get compiled into the final Metal PSO.

Metal allows your application to load most of the rendering state upfront. However, if you have limited memory, make sure not to hold on to PSO references that you don’t need anymore. Also, don’t hold on to Metal function references after you have created the PSO cache because they are not needed to render; they are only needed to create new PSOs.

Note: Apple has written a Metal Best Practices guide that provides great advice for optimizing your app.

Where to Go From Here?

Getting the last ounce of performance out of your app is paramount. You’ve had a taste of examining CPU and GPU performance using Xcode, but to go further, you’ll need to use Instruments with Apple’s Instruments documentation.

Over the years, at every WWDC since Metal was introduced, Apple has produced some excellent WWDC videos describing Metal best practices and optimization techniques. Go to https://developer.apple.com/videos/graphics-and-games/metal/ and watch as many as you can, as often as you can.

Congratulations on completing the book! The world of Computer Graphics is vast and as complex as you want to make it. But now that you know the basics of Metal, even though current internet resources are few, you should be able to learn techniques described with other APIs, such as OpenGL, Vulkan and DirectX. If you’re keen to learn more, check out the great books in the resources folder for this chapter.