30. Profiling

Written by Marius Horga & Caroline Begbie

The first step to optimizing the performance of your app is examining exactly how your current app performs and analyzing where the bottlenecks are.

Imagine this scenario: You’ve started development on the first level of a new game, Phoenix Island: Rising from the ashes. You’ve created a basic scene, and now you want to find out how well it runs before adding the action.

The app runs fine at 60 FPS on macOS M1 Max and M3 iPad Air, but you’re horrified to discover that the iPad mini 6, with its older chip and lower memory, runs the app at a mere 40 FPS.

In this chapter, you’ll look at some tools to help you analyze performance and find where your bottlenecks are.

Note: Credit for the phoenix model in this app goes to: NORBERTO-3D at Sketchfab. All the other models and HDRI sky were created by the folks at Poly Haven

The Starter App

➤ In Xcode, review the project for this chapter. There are a number of interesting features.

Assets

First, there are two Assets folders. The one directly under the top level Profiling contains a lot of data, so it points to a folder outside of the Profiling hierarchy. If the content names are red, select both Assets and game-scene.usda, and in the File inspector, click the folder icon. Then, locate and select the assets folder to reconnect the files. The assets folder is the folder that contains both Assets and game-scene.usda.

The USD Scene

assets/game-scene.usda is an editable text file that describes the scene. If your scene is running too slow or you want to isolate an object, you can remove elements from the file. For example, to remove the landscape, delete the following lines:

def Mesh "Landscape" (
    prepend references ...
  )
{
    token visibility = "inherited"
    matrix4d xformOp:transform ...
    uniform token[] xformOpOrder = ["xformOp:transform"]
}

USD is a fantastic file format for composing scenes. Unlike the usdz extension, which includes all the textures and meshes for a single object, the usd and usda format points to mesh and texture files in their own separate locations. This means you can split up work on a scene across different parts of your team. All of the def objects in game-scene.usda point to other files in the Assets folder.

The other Assets folder contains single, separately placed objects in usdz and obj formats.

The Render Passes

In Renderer.swift, you can see the usual render passes, along with these new ones:

• NatureRenderPass: Nature renders the rocks with one draw call. You’ll see this render pass later in the chapter.
• WaterRenderPass: Water renders the ocean. Unlike the reflection and refraction you learned about in Chapter 22, “Reflection & Refraction”, here, the “reflection” comes from sampling mip map level 5 in the lower half of the skybox texture. There is no refraction, and the depth of the water is calculated using the distance from the centre of the land mass.
• ParticlesRenderPass: Improving on the particles from Chapter 17, “Particle Systems”, particles are now in 3D, rendering the fire in the fire pit.
• UpscalePass: You’ll take a brief look at MetalFX upscaling later.
• Brighten is a post-processing pass that runs the final render target through tone mapping to improve the color.

➤ Build and run the starter app for this chapter.

This is a top-down aerial view of the island. You can use your WASD and arrow keys to move around and view the scene. Scroll to go up and down on the Y axis. The buttons at the bottom right will take you to different views of the island, where you’ll see the phoenix circling above.

You’re in the early stages of developing Phoenix Island, so the project is not as efficient as it could be, and some of the asset artwork needs upgrading too. As well as the changes you’ll make throughout this chapter, you could upgrade the app to render the scene using GPU-driven indirect encoding commands, and also coalesce some of the render passes.

Profiling

There are a few ways to monitor and tweak your app’s performance. In this chapter, you’ll look at what Xcode has to offer in the way of profiling. You can also use Instruments, which is a powerful app that profiles both CPU and GPU performance. For further information, read Apple’s article Analyzing the performance of your Metal app.

Metal Performance HUD

A great place to start is looking at information about how your app is running is the Metal Performance HUD.

➤ Select the Profiling Scheme at the top of the Xcode window, and choose Edit Scheme.

➤ Under Run, choose the Diagnostics tab, and check Show Graphics Overview. This setting engages the HUD.

➤ Close the scheme and build and run your app.

You can immediately see that although the app runs at 60 FPS on the M1 Max and M3 iPad Air, the older chips on iPad mini 6 and M1 iPad Pro struggle to keep up. The app uses about 1.2GB memory, which is almost at the limit for older devices.

Culling Back Faces

You can achieve a quick performance win by not rendering so many vertices. Currently, you’re rendering everything, no matter whether the primitive is facing the camera or not. Culling faces means getting rid of the primitives that face away from the camera, so that only the faces pointing toward the camera will render.

➤ In the Renderer folder, open Renderer.swift, and change let cullFaces = false to:

let cullFaces = true

This change will trigger the culling that’s already implemented in your starter app, which applies to all render passes.

➤ Build and run the app, and compare the HUD result.

On the iPad mini 6, you can achieve at least a half millisecond performance gain.

You might think that you always want to cull faces, but you do have to be a bit selective. For example, the tower in your scene is a one-sided mesh. If you go to the Tower view in your app and press W to go forward and scroll to go down, your tower will disappear while you’re inside it because you can’t see the back faces.

To find out more about the Metal Performance HUD, watch Apple’s Tech Talk Discover Metal Performance HUD.

The GPU Report

➤ With your app running, and in Xcode on the Debug navigator, click FPS.

The GPU report shows in the central pane and contains two major GPU metrics.

The first GPU report metric is Frames Per Second, which represents the current frame rate of your app. Your target should always be 60 FPS or better. The screenshot shows an app running on iPad mini 6. There’s a lot of work ahead to get it to run at 60 FPS.

The second GPU report metric is Frame Time. This report represents the actual time spent processing the current frame on the CPU and the GPU. What’s most important here is that the frame does not take longer than 16.6ms which corresponds to 60 FPS.

You know from the Metal Performance HUD that your GPU time is about 20ms, and it’s the GPU time that is pushing the frame time to 25ms. The HUD will be more accurate than the frame time in the Debug navigator.

The general rule is this: If your GPU time is low, but your frame time is high, then you’re CPU-bound. But here, your GPU time is high, so you’re GPU-bound. This means that your command encoding and animation calculations, which happen on the CPU, is of less concern than what’s happening on the GPU.

GPU Workload Capture

In previous chapters, you captured the GPU workload to inspect textures, buffers and render passes. The GPU capture is always the first point of call for debugging. Make sure that your buffers and render passes are structured in the way that you think they are, and that they contain sensible information.

Next, you’ll look at what else GPU capture can show you.

Summary

➤ With your app running, capture the GPU workload, and in the Debug navigator, click Summary.

Note: If the GPU capture fails, check that you have enough free space on your device, and disengage the Metal Performance HUD.

You’ll see an overview of your frame. The Insights section often contains useful insights when you might bind resources on the CPU, but not use them in your shaders. The previous image, under Memory, shows a number of bound unused resources, most noticeably, the Tangent Buffers.

Note: To take full advantage of the GPU capture, you should add a label to all your buffers, so that you can easily track down issues. name Tangent Buffer is a label added in Mesh.swift.

This insight highlights an error in your app. The app should be using the tangent buffer.

➤ In the Shaders folder, open Vertex.metal, and locate the assignment to worldTangent and worldBitangent.

These assignments are currently set to 0, when they should be using the tangent values.

➤ Change the assignments to:

.worldTangent = model->normalMatrix * in.tangent,
.worldBitangent = model->normalMatrix * in.bitangent,

➤ Build and run the app again.

You’ll notice the scene is much better lit now, with proper specular values.

➤ Capture the GPU workload and check the Insights section.

Sometimes the GPU capture is not completely reliable. Although the Memory insight still reports a bound unused resource, the Rocks vertex buffer is very much needed by the Nature render pass.

Under the Bandwidth insight, the unused textures are in the Bloom post-processing effect. It’s possible that the MPS encoders are either not as efficient as they could be, or reporting incorrectly.

As to coalescing the encoders, you should review your render passes. The render passes in this app are all separate so that you can understand what’s going on, but they are not efficient. You should set up a render pass system where you can combine some into a single render command encoder.

➤ Click the API Usage insight.

There are a lot of redundant bindings. Most of these are the result of not sorting the submeshes by pipeline state at the start of the app. Submeshes use different pipeline states depending on submesh transparency and the model having a skeleton for animation. Sorting by pipeline state will prevent a lot of GPU switching and unnecessary binding.

Checking Insights is a great place to start optimizing your app, as it could pick up a few simple errors.

Encoded Command Performance

The next place to look at profiling your app is in the Debug navigator, which details the performance of render passes and pipeline states.

➤ In the Debug navigator, switch to Group by Pipeline State. You can now see how much relative time each pipeline took during the frame. The total frame time is under Performance.

Here, you can see that the Forward PSO takes a large percentage of the render time. This is to be expected because you render the USD scene here. However, drilling down into the draws, there’s one draw that takes up a whopping 11% of the render time. You might have found one of the major reasons your app frame rate isn’t buttery smooth.

When you select the draw call, you can see that the rendered object is the Landscape.

The speed of the draw largely depends on the bandwidth speed of the buffers and textures. In the case of the Landscape, the vertex buffers aren’t very large, but the base color texture is 21.33MB. Another object that takes 6% of the draw call is the Dutch Ship. This object’s vertex buffer takes up 14MB, which an artist could very likely optimize.

Memory

Inefficient use of memory can do a lot of damage to performance.

➤ In the Debug navigator, click the Memory tool (below Performance) to see how the various resources are allocated in memory:

Clicking on the Allocated Size column shows the buffers and textures in order of size. The heap of textures containing the PBR textures for each object is by far the largest resource.

You should be judicious about the size of textures. The color map is the most important, followed by the normal map. When I first exported the scene from Unreal Engine to USD, all the texture maps were the same size, 2048x2048. In app, they all took up 21.33MB of memory each. Reducing the sizes of metallic, roughness and ambient occlusion maps to 1024x1024, they take up 5.33MB each, and to 512x512, only 1.33MB each. This difference allowed the app to run on the iPad mini 6, with no obvious loss of quality.

If you use the asset catalog for your textures, you can more easily set texture variations for different device capabilities. However, this means taking more responsibility for how you store your resources, rather than simply loading a usdz file.

Loss of quality can affect the look of your scene.

Compare the low quality on the left with the huge 8192x8192 texture on the right. This large texture takes up 341MB in app memory. Instead of shipping with a huge texture, it would be better to tile the underwater part with a small texture, and only put detailed textures where it matters.

You should handle terrain on its own. If you’re going to cover it up with grass, you don’t need a good texture, but if it’s a highlight of your app, you can investigate Apple’s sample Streaming large images with Metal sparse textures.

GPU Timeline

The GPU timeline tool gives you an overview of how your vertex, fragment and compute functions perform, broken down by render pass.

➤ Build and run the app, and capture the GPU workload with a frame count of 3.

➤ In the Debug navigator, change Group by Pipeline State to Group by API call, and click on Command Buffer.

You’ll see the dependency graph of the frame, with each render pass texture showing how it’s passed on from the previous render pass.

With a few of the render passes removed, this is an overview of the dependency graph:

This graph highlights the linear nature of your render passes. Each of them, except for the Particle Compute Pass writes to the view’s drawable, resulting in each pass having to wait for the previous pass to finish. For better efficiency, you could parallelize some of the passes by writing to separate render targets, as you did in Chapter 14, “Deferred Rendering”. For example, the Water render pass is quite expensive, so you could do the water calculations in a separate pass, and then combine these into the view’s render target later.

➤ In the Debug navigator, click on Performance.

You’ll see a track for each of your vertex, fragment and compute shaders, so you can visualize where in the timeline your shaders are performed, and how long they take to perform. The longest encoders are the Forward Render Pass and the Coalesced 4 Encoders that are the post processing Metal Performance Shaders producing bloom effect. Looks like you’ve found another greedy culprit.

➤ Open Renderer.swift, and in draw(scene:in:), comment out:

bloom.postProcess(
  view: view,
  commandBuffer: commandBuffer,
  inputTexture: descriptor.colorAttachments[0].texture)

➤ Build and run again. Notice how the Metal Performance HUD is now showing that you’re running at 60FPS, with a frame time of 14.5ms (iPad mini 6 statistics).

➤ Capture three GPU frames again, and see the difference in the GPU timeline.

Where you don’t have a dependency, vertex, fragment, and compute shaders can run in parallel. For example, the GPU can perform your Particles compute shader in parallel with the Shadow render pass, whereas the Brighten post-processing effect depends upon the final render target. Unfortunately, when you look at your render passes, each render is dependent on the previous drawable, so there are a few gaps in the timeline.

Instancing

Reducing the number of draw calls is one of the best ways of improving performance. Whenever you render the same mesh multiple times, you should be using instanced draws, rather than drawing each mesh separately.

As an example of an instanced system, the app includes a procedural nature system. Rocks.swift creates a rock pile of 20 rocks with three random shapes, and three random textures.

The Procedural Nature System

Using homeomorphic models, you can choose different shapes for each model. Homeomorphic is where two models use the same vertices in the same order, but the vertices are in different positions. A famous example of this is Spot the cow by Keenan Crane.

Spot is modeled from a sphere by moving vertices, rather than adding them. Because the vertices are in the same order as the sphere, the uv coordinates don’t change either.

The random shapes of the rocks are modeled in a similar fashion, using the same basic shape, then readjusting the vertices for each shape. Each adjusted shape is called a morph target.

For the rocks, Nature loads the three vertex meshes into one buffer, and each rock, when initialized, is allocated a random number between 0 and 2. It’s then simple to extract the correct mesh from the buffer in the vertex function.

The most important feature of the nature system is that, depending on how powerful your device is, it can render numerous instances with one draw call:

 encoder.drawIndexedPrimitives(
   type: .triangle,
   indexCount: submesh.indexCount,
   indexType: submesh.indexType,
   indexBuffer: submesh.indexBuffer.buffer,
   indexBufferOffset: submesh.indexBuffer.offset,
   instanceCount: instanceCount)

In Nature.metal, vertex_nature uses the instance_id attribute to extract the transform information for the current instance. With the morph target, the vertex function renders a random shape. With the texture ID, the fragment function renders a random texture.

The files involved in the nature system are:

• Common.h: Contains a NatureInstance structure which holds a random texture and shape ID as well as the model and normal matrix.

• Nature.swift: This is in the Geometry folder and is a cut-down version of Model. It loads up the mesh and creates a buffer that contains an array of NatureInstance, one element for each instance.

• Nature.metal: Contains the vertex and fragment functions.

• NatureRenderPass.swift: Renders the scene’s nature array, in the same way as ForwardRenderPass.

➤ Examine these files to see how the nature system works.

Inspecting Shaders

It’s easy to debug Swift code by using break points and printing out values. But how do you find out what your Metal Shading Language code is doing? The Shader editor has you covered. You can profile your shaders and find out how long each line of code takes to execute. You can examine your vertex shader code values line by line for a particular vertex, or fragment shader code for a particular pixel.

Maybe you want to change the color of the ocean, but re-running and returning to the same app state every time is a pain. Or perhaps you want to understand how the foam on the wave works.

➤ Build and run your app. Go to the Tower view, and rotate it until you have a better view of the ocean.

➤ Capture one frame of the GPU workload and click the Command Buffer. Locate the Water Render Pass, and double-click the render target until you see the full texture.

➤ Click the Debug Shader icon, which looks like a bug, in the toolbar above the Debug console.

Here, you can can either debug the geometry in the vertex or pixels in the fragment shader.

➤ Click on a pixel that you want to examine, and choose Fragment Shader and click the bug icon in the right corner of the window.

The fragment function will then run for the selected pixel and the code with calculated values will show.

The Debug navigator lists each function command. If you call other functions, you can click the disclosure triangle to navigate to the code there.

On the right, you’ll see each calculated value.

➤ Change float3 nearColor = farColor * 0.1; to:

float3 nearColor = float3(1, 0, 0);

➤ Click the Reload Shaders arrow in the toolbar above the Debug console.

You should have your render target side by side with your shader code, and you’ll see the result of the shader being re-run.

Wait for Xcode to finish profiling the code after you change it. When it’s done, you can experiment with how each command affects the render target. You can return values early, such as checking out the fresnel value.

When you’ve finished making your changes to the shader, make sure you copy the code. When you stop the GPU frame capture, Xcode should ask you whether you want to save changes, but often, it doesn’t.

➤ Click the blue Shader Debugger bug icon to exit the Shader Debugger.

The Shader Profiler

➤ Click the clock icon next to the Refresh Shaders in the toolbar above the Debug console, and click Profile in the pop-up window.

Xcode will now time each operation of your shader code.

You can run your cursor over the circles to see a breakdown of operations. Analyze the percentages for each GPU activity. A high number might indicate an opportunity for performance optimization.

Here’s an opportunity for optimization using shader profiling. Processing floats takes more time than processing other types. As you might know, a half is, well, half the size of a float, so you can optimize this one spot.

➤ With the profiler still running, check the percentage for the value returned from fragment_water. This will vary slightly each time you profile. On my M1 Max MacBook Pro, it’s 5.69%.

➤ Change the fragment header so that you return a half4 instead of a float4 and change the return value to:

return half4(half3(color), alpha);

➤ Click Reload Shaders and wait for the profiling to finish.

The percentage on my computer lowered to 4.63%. The cost of processing halfs over floats is less, and it’s an easy change to make to your shader functions. Do ensure that you don’t need the extra precision of the float value though. Other ALU optimizations you can do include replacing ints with shorts, simplifying complex instructions such as trigonometry functions (sin, cos, etc.) and other arithmetic calculations.

CPU-GPU Synchronization

Measuring GPU performance is important, but you should also consider interaction between CPU and GPU. Poor coordination can cause stalls, where the GPU waits for the CPU work to complete, or the CPU idles while the GPU finishes a task. Synchronization issues can also cause frame stutters.

Managing dynamic data can be a little tricky. Take the case of Uniforms, which is now stored in an MTLBuffer, rather than a simple structure, to help you understand synchronization. Uniforms contains only camera and shadow data, so you update it usually once per frame on the CPU. That means that the GPU should wait until the CPU has finished writing the buffer before it can read the buffer.

Instead of halting the GPU’s processing, you can have a pool of reusable buffers.

Triple Buffering

Triple buffering is a well-known technique in the realm of synchronization. The idea is to use three buffers at a time. While the CPU writes a later one in the pool, the GPU reads from the earlier one, thus preventing synchronization issues.

You might ask, why three and not just two or a dozen? With only two buffers, there’s a high risk that the CPU will try to write the first buffer again before the GPU finished reading it even once. With too many buffers, there’s a high risk of performance issues.

➤ In the Renderer folder, open Renderer.swift.

At the top of the file, you’ll see a global variable which determines the number of frames in flight. Frames in flight is a graphics term for how many frames you can write to at once. Renderer.currentFrameIndex keeps track of the current frame.

➤ Change let maxFramesInFlight = 1 to:

let maxFramesInFlight = 3

When the app creates the initial uniforms buffer array, it will now create an array of three of them.

➤ Locate Renderer.updateUniforms(scene:) and examine the code. In previous chapters you were updating Uniforms, a structure. Now you update the contents of the Metal buffer for the current frame.

➤ At the start of draw(scene:in:), after guard, add this code to update the current frame:

Self.currentFrameIndex =
  (Self.currentFrameIndex + 1) % maxFramesInFlight

Here, you make the index loop around always taking the values 0, 1 and 2.

➤ Build and run the app.

Your app shows the same scene as before. However, in this example you probably won’t notice any difference.

There is some bad news. The CPU can write to uniforms at any time and the GPU can read from it. There’s no synchronization to ensure the correct uniform buffer is being read. When there is little action going on in your app, that’s not so important, but when you want to keep track of all sorts of camera and animation information, you need to ensure that the GPU is getting correct and timely data.

The problem is known as resource contention and involves conflicts, known as race conditions, over accessing shared resources by both the CPU and GPU. This can cause unexpected results, such as animation glitches.

In the image below, the CPU is ready to start writing the first buffer again. However, that would require the GPU to have finished reading it, which is not the case here.

The following example shows two uniform buffers available:

What you need here is a way to delay the CPU writing until the GPU has finished reading it.

A naive approach is to block the CPU until the command buffer has finished executing.

➤ Still in Renderer.swift, add this to the end of draw(scene:in:):

commandBuffer.waitUntilCompleted()

➤ Build and run the app.

You’re now sure that the CPU thread is successfully being blocked, so the CPU and GPU are not fighting over uniforms. However, causing the CPU to wait until the GPU has finished may affect the frame rate.

Semaphores

A more performant way, is the use of a synchronization primitive known as a semaphore, which is a convenient way of keeping count of the available resources. In this case, your triple buffer.

Here’s how a semaphore works:

• Initialize it to a maximum value that represents the number of resources in your pool (3 buffers here).
• Inside the draw call the thread tells the CPU to wait until a resource is available and if one is, it takes it and decrements the semaphore value by one.
• If there are no more available resources, the current thread is blocked until the semaphore has at least one resource available.
• When a thread finishes using the resource, it’ll signal the semaphore by increasing its value and by releasing the hold on the resource.

Time to put this theory into practice.

➤ At the top of Renderer, add this new property:

var semaphore: DispatchSemaphore

➤ In init(metalView:options:), add this before super.init():

semaphore = DispatchSemaphore(value: maxFramesInFlight)

➤ Add this code to draw(scene:in:), before updateUniforms(scene:):

_ = semaphore.wait(timeout: .distantFuture)

Note: If you add the semaphore wait before if doUpscaling, then, if there is a return from that conditional, the scene will never render. The CPU will be forever waiting for the semaphore to signal completion.

➤ At the end of draw(scene:in:), but before committing the command buffer, add this:

commandBuffer.addCompletedHandler { _ in
  self.semaphore.signal()
}

➤ At the end of draw(scene:in:), remove:

commandBuffer.waitUntilCompleted()

➤ Build and run the app again, making sure everything still renders as it did before.

Your frame rate may have reduced slightly. But now the frame renders more accurately without fighting over resources.

MetalFX Upscaling

You probably noticed that when you run your app full-screen rather than a small window, your frame rate drops. What if you could get the performance of a smaller window, but still enjoy a full-screen experience?

That’s what MetalFX gives you. There are two ways to upscale your renders:

• Spatial upscaling enlarges a render texture using advanced graphics techniques.
• Temporal antialiased upscaling takes samples from previous frames and integrates them with the current frame render.

➤ In the Render Passes folder, open UpscalePass.swift.

This pass is ready for you to use, showing how simple the technique is to implement. You create an render texture at a small size, and a final texture for the upscaled result. You also update a render pass descriptor each frame with the current textures, then encode the textures with the MTLFXSpatialScalar.

Renderer updates the view’s drawableSize to match the down-scaled render texture.

➤ First, build and run with the Metal Performance HUD engaged. Take a note of the GPU time, and also the quality of the render.

➤ Open Renderer.swift, and at the top of the file, change let doUpscaling = false to:

let doUpscaling = true

Changing this flag will cause Renderer to reduce the view’s drawable size by kUpscaleAmount, currently set to 1.25.

➤ Build and run the app again, and compare the difference. With an upscaling of 1.25, the frame rate improves very slightly, and the quality is still acceptable. An upscaling of 2 gives a great frame rate, but makes the render very muddy.

As always, check what is acceptable to you and your current renders. In some situations, upscaling may even make your frame times slower due to the encoding overhead.

Visibility Culling

The fastest geometry to render is geometry that you don’t have to render because it’s not in the frame. Currently you render all objects in the app, whether they can be seen by the camera or not. You process the fire particles even though they might not be on screen. Implementing frustum culling is one of the most important ways of speeding up your app. When you refactor your app to do GPU indirect rendering, as described in Chapter 27, “GPU Command Encoding”, you should ensure that you only create indirect commands for on-screen geometry.

Key Points

• The Metal Performance HUD is the easiest way to profile your app.
• Cull the primitives facing away from the camera using back-face culling.
• Capture the GPU workload for insight into what’s happening on the GPU. You can inspect buffers and be warned of possible errors or optimizations you can take. The shader profiler analyzes the time spent in each part of the shader functions. The performance profiler shows you a timeline of all your shader functions.
• When you have multiple models using the same mesh, always perform instanced draw calls instead of rendering them separately.
• Textures can have a huge effect on performance. Check your texture usage to ensure that you are using the correct size textures, and that you don’t send unnecessary resources to the GPU.

Where to go From Here

The resources for this chapter contain a list of the Apple articles and videos on profiling. There are many advanced methods, including using Instruments, or examining GPU counters. The Apple documentation and videos are very good on this topic. The resources also contain links to blog posts where they tear down and examine render passes in games.