14. Multipass & Deferred Rendering

Written by Marius Horga

Up to this point, you’ve been running projects and playgrounds that only had one render pass. In other words, you used a single command encoder to submit all of your draw calls to the GPU.

For more complex apps, you may need multiple render passes before presenting the texture to the screen, letting you use the result of one pass in the next one. You may even need to render content offscreen for later use.

With multiple render passes, you can render a scene with multiple lights and shadows, like this:

Take note, because in this chapter, you’ll be creating that scene. Along the way, you’ll learn a few key concepts, such as:

• Shadow maps.
• Multipass rendering.
• Deferred rendering with a G-buffer.
• The blit command encoder.

You’ll start with shadows first.

Shadow maps

A shadow represents the absence of light on a surface. A shadow is present on an object when another surface or object obscures it from the light. Having shadows in a project makes your scene look more realistic and provides a feeling of depth.

Shadow maps are nothing more than textures containing shadow information about the scene. When a light shines on an object, anything that is behind that object gets a shadow cast on it.

Typically you render the scene from the location of your camera, but to build a shadow map, you need to render your scene from the location of the light source - in this case the sun.

The image on the left shows a render from the position of the camera with the directional light pointing down. The image on the right shows a render from the position of the directional light.

The eye shows where the camera was positioned in the first image.

You’ll do two render passes:

• First pass: Using a separate view matrix holding the sun’s position, you’ll render from the point of view of the light. Because you’re not interested in color at this stage, only the depth of objects that the sun can see, you’ll only render a depth texture in this pass. This is a grayscale texture, with the gray value indicating depth. Black is close to the light and white is far away.

• Second pass: You’ll render using the camera as usual, but you’ll compare the camera fragment with each depth map fragment. If the fragment’s depth is lighter in color than the depth map at that position, it means the fragment is in the shadow. The light can “see” the blue x in the above image, so it is not in shadow.

Shadows and deferred rendering are complex subjects, so there’s a starter project available for this chapter. Open it in Xcode and take a look around.

The code is similar to what’s available at the end of Chapter 5, “Lighting Fundamentals”.

For simplicity, you’ll be working on the diffuse color only; specularity and ambient lighting are not included with this project.

Build and run the project, and you’ll see a train and a tree model, both on top of a plane:

Add these properties in Renderer.swift, at the top of Renderer:

var shadowTexture: MTLTexture!
let shadowRenderPassDescriptor = MTLRenderPassDescriptor()

Later, when you create the render command encoder for drawing the shadow, you’ll use this render pass descriptor. Each render pass descriptor can have up to eight color textures attached to it, plus a depth texture and a stencil texture. The shadowRenderPassDescriptor points to shadowTexture as a depth attachment.

You’ll need several textures through the course of the chapter, so create a helper method for building them.

Add this new method to Renderer:

func buildTexture(pixelFormat: MTLPixelFormat, 
                  size: CGSize, 
				  label: String) -> MTLTexture {
  let descriptor = MTLTextureDescriptor.texture2DDescriptor(
                              pixelFormat: pixelFormat,
                              width: Int(size.width),
                              height: Int(size.height),
                              mipmapped: false)
  descriptor.usage = [.shaderRead, .renderTarget]
  descriptor.storageMode = .private
  guard let texture = 
    Renderer.device.makeTexture(descriptor: descriptor) else {
    fatalError() 
  }
  texture.label = "\(label) texture"
  return texture
}

In this method, you configure a texture descriptor and create a texture using that descriptor. Textures used by render pass descriptors have to be configured as render targets. Render targets are memory buffers or textures that allow offscreen rendering for cases where the rendered pixels don’t need to end up in the framebuffer. The storage mode is private, meaning the texture is stored in memory in a place that only the GPU can access.

Next, add the following to the bottom of the file:

private extension MTLRenderPassDescriptor {
  func setUpDepthAttachment(texture: MTLTexture) {
    depthAttachment.texture = texture
    depthAttachment.loadAction = .clear
    depthAttachment.storeAction = .store
    depthAttachment.clearDepth = 1
  }
}

This creates a new extension on MTLRenderPassDescriptor with a new method that sets up the depth attachment of a render pass descriptor and configures it to store the provided texture. This is where you’ll attach shadowTexture to shadowRenderPassDescriptor.

You’re creating a separate method because you’ll have other render pass descriptors later in the chapter. The load and store actions describe what action the attachment should take at the start and end of the render pass. In this case, you clear the texture at the beginning of the pass and store the texture at the end of the pass.

Now, add the following to the Renderer class:

func buildShadowTexture(size: CGSize) {
  shadowTexture = buildTexture(pixelFormat: .depth32Float,
                               size: size, label: "Shadow")
  shadowRenderPassDescriptor.setUpDepthAttachment(
                               texture: shadowTexture)
}

This builds the depth texture by calling the two helper methods you just created. Next, call this method at the end of init(metalView:):

buildShadowTexture(size: metalView.drawableSize)

Also, call it at the end of mtkView(_:drawableSizeWillChange:) so that when the user resizes the window, you can rebuild the textures with the correct size:

buildShadowTexture(size: size)

Build and run the project to make sure everything works. You won’t see any visual changes yet; you’re just verifying things are error-free before moving onto the next task.

Multipass rendering

A render pass consists of sending commands to a command encoder. The pass ends when you end encoding on that command encoder. Multipass rendering uses multiple command encoders and facilitates rendering content in one render pass and using the output of this pass as the input of the next render pass.

Why would you need two passes here? Because in this case, you’ll render the shadow from the light’s position, not from the camera’s position. You’ll then save the output to a shadow texture and give it to the next render pass which combines the shadow with the rest of the scene to make up a final image.

Since you already have code that renders the scene, you can easily refactor this code into a new function that you can reuse for shadows as well.

The shadow pass

During the shadow pass, you’ll be rendering from the point of view of the sun, so you’ll need a new view matrix.

In Common.h, add this to the Uniforms struct:

matrix_float4x4 shadowMatrix;

You’ll also need a new pipeline state to hold the different vertex function you’ll be calling.

Add this property to Renderer:

var shadowPipelineState: MTLRenderPipelineState!

Now add the following to the end of Renderer:

func renderShadowPass(renderEncoder: MTLRenderCommandEncoder) {
  renderEncoder.pushDebugGroup("Shadow pass")
  renderEncoder.label = "Shadow encoder"
  renderEncoder.setCullMode(.none)
  renderEncoder.setDepthStencilState(depthStencilState)
  // 1
  renderEncoder.setDepthBias(0.01, slopeScale: 1.0, clamp: 0.01)
  // 2
  uniforms.projectionMatrix = float4x4(orthoLeft: -8, right: 8, 
                                       bottom: -8, top: 8, 
                                       near: 0.1, far: 16)
  let position: float3 = [sunlight.position.x, 
                          sunlight.position.y,
                          sunlight.position.z]
  let center: float3 = [0, 0, 0]
  let lookAt = float4x4(eye: position, center: center, 
                        up: [0,1,0])
  uniforms.viewMatrix = lookAt
  uniforms.shadowMatrix = 
       uniforms.projectionMatrix * uniforms.viewMatrix
  
  renderEncoder.setRenderPipelineState(shadowPipelineState)
  for model in models {
    draw(renderEncoder: renderEncoder, model: model)
  }
  renderEncoder.endEncoding()
  renderEncoder.popDebugGroup()
}

Going through this code:

1. This adjusts the depth values in the pipeline by a scaling factor and a scaling bias, clamping the bias to a maximum amount.
2. Here, you create an orthographic projection matrix. You previously used an orthographic matrix in Chapter 9, “The Scene Graph”, to render the scene from above. Because sunlight is a directional light, this is the correct projection type for shadows caused by sunlight. If instead you used a spotlight, for example, you would use the perspective projection matrix. You also use a lookAt matrix to set the direction vector of the sun vector.

Now you need to call this method for every frame.

Add the following code in draw(in:), right below // shadow pass:

guard let shadowEncoder = commandBuffer.makeRenderCommandEncoder(
                descriptor: shadowRenderPassDescriptor) else {
  return
}
renderShadowPass(renderEncoder: shadowEncoder)

Next, create a new method after init(metalView:):

func buildShadowPipelineState() {
  let pipelineDescriptor = MTLRenderPipelineDescriptor()
  pipelineDescriptor.vertexFunction = 
       Renderer.library.makeFunction(name: "vertex_depth")
  pipelineDescriptor.fragmentFunction = nil
  pipelineDescriptor.colorAttachments[0].pixelFormat = .invalid
  pipelineDescriptor.vertexDescriptor =
      MTKMetalVertexDescriptorFromModelIO(
                Model.defaultVertexDescriptor)
  pipelineDescriptor.depthAttachmentPixelFormat = .depth32Float
  do {
    shadowPipelineState = 
       try Renderer.device.makeRenderPipelineState(
                     descriptor: pipelineDescriptor)
  } catch let error {
    fatalError(error.localizedDescription)
  }
}

This creates a pipeline state without a color attachment or fragment function. Remember, at this point, you’re only interested in depth information, not color information.

Call this method at the end of init(metalView:):

buildShadowPipelineState()

As you may have noticed, you’re referencing a shader function named vertex_depth which does not exist yet.

In the Shaders group, using the Metal File template, create a new file named Shadow.metal. Make sure to check both macOS and iOS targets. Then, add this code to the newly created file:

#import "../Utility/Common.h"

struct VertexIn {
  float4 position [[ attribute(0) ]];
};

vertex float4 
       vertex_depth(const VertexIn vertexIn [[ stage_in ]],
                    constant Uniforms &uniforms [[buffer(1)]]) {
  matrix_float4x4 mvp = 
        uniforms.projectionMatrix * uniforms.viewMatrix
        * uniforms.modelMatrix;
  float4 position = mvp * vertexIn.position;
  return position;
}

Note that you have to put the correct relative path to Common.h.

This simple code receives a vertex position, and returns the transformed position.

Build and run the project.

Whoa! What happened to the scene?

Don’t worry. This happened because you used the orthographic projection matrix for the shadow pass; to correct things, the main pass needs to use the perspective projection matrix again, so add this line in draw(in:) before the for loop where you process models:

uniforms.projectionMatrix = camera.projectionMatrix

Build and run the project, and you should see the starter scene rendered again.

Ok, that’s nice! But where’s the shadow?

Click the Capture GPU Frame button on the debug bar (circled in red below):

In the Debug navigator, click on the Shadow pass group:

Excellent, the app window shows the shadow map. Yay! :D

This is the scene rendered from the light’s position. You used the shadow pipeline state, which you configured not to have a fragment shader, so the color information is not processed here at all — it’s purely depth. Lighter colors are further away, and darker colors are closer.

The main pass

Now that you have the shadow map saved to a texture, all you need to do is send it to the next pass — the main pass — so you can use the texture in lighting calculations in the fragment function.

In draw(in:), before the for loop of the main pass, add this line:

renderEncoder.setFragmentTexture(shadowTexture, index: 0)

You’re passing the shadow texture to the main pass’s fragment function.

In Main.metal, add a new member to the VertexOut struct:

float4 shadowPosition;

You’ll hold two transformed positions for each vertex. One transformed within the scene from the camera’s point of view, and the other from the sun’s point of view. You’ll be able to compare them in the fragment function.

Add this line in vertex_main, before return;:

out.shadowPosition = 
     uniforms.shadowMatrix * uniforms.modelMatrix 
     * vertexIn.position;

The rest of the work happens in fragment_main. First, add one more function parameter after &material:

depth2d<float> shadowTexture [[texture(0)]]

Unlike the textures you’ve used in the past which have a type of texture2d, the texture type of a depth texture is depth2d.

Then, add this code before the return statement:

// 1
float2 xy = in.shadowPosition.xy;
xy = xy * 0.5 + 0.5;
xy.y = 1 - xy.y;
// 2
constexpr sampler s(coord::normalized, filter::linear,
                    address::clamp_to_edge, 
                    compare_func:: less);
float shadow_sample = shadowTexture.sample(s, xy);
float current_sample = 
     in.shadowPosition.z / in.shadowPosition.w;
// 3
if (current_sample > shadow_sample ) {
  diffuseColor *= 0.5;
}

Going through this code, you:

1. Determine a coordinate pair from the shadow position that will serve as a screen space pixel locator on the shadow texture. Then, you normalize the coordinates from [-1, 1] to [0, 1]. Finally, you reverse the Y coordinate since it’s upside down.

2. Create a sampler to use with the shadow texture, and sample the texture at the coordinates you just created. Get the depth value for the currently processed pixel.

3. Compare the current depth value to the shadow depth value, and you set a darker grey for pixels that have the depth greater than the shadow value stored in the texture.

Build and run the project, and you’ll finally see models with shadows! :]

Note: If you’re not entirely pleased about the quality of your shadows, there are some common techniques you can use to improve shadow maps at: https://msdn.microsoft.com/en-us/library/windows/desktop/ee416324(v=vs.85).aspx.

Deferred rendering

Before this chapter you’ve only been using forward rendering but assume you have a hundred models (or instances) and a hundred lights in the scene. Suppose it’s a metropolitan downtown where the number of buildings and street lights could easily amount to the number of the objects in this scene.

With forward rendering, you render all of the models and process all of the lights in the fragment shader, for every single fragment, even though the final render may exclude a particular fragment. This can easily become a quadratic runtime problem that seriously decreases the performance of your app.

Deferred rendering, on the other hand, does two things:

• It collects information such as material, normals and positions from the models and stores them in a special buffer — traditionally named the G-buffer where G is for Geometry — for later processing in the fragment shader; by that time, the GPU only keeps visible fragments, so unnecessary calculation does not occur.
• It processes all of the lights in a fragment shader, but only on the final visible fragments.

This approach takes the quadratic runtime down to linear runtime since the lights’ processing loop is only performed once, and not for each model.

Here’s a breakdown of the steps:

• A first pass renders the shadow map. You’ve already done this.
• A second pass reads the framebuffer attachments and constructs G-buffer textures containing these values: material color (or albedo), world space normals and positions and shadow information.
• A third and final pass using a full-screen quad combines all of this information into one final composited texture.

The G-buffer pass

All right, time to build that G-buffer up! First, create four new textures. Add this code at the top of Renderer:

var albedoTexture: MTLTexture!
var normalTexture: MTLTexture!
var positionTexture: MTLTexture!  
var depthTexture: MTLTexture!

Define the G-buffer pass descriptor and pipeline state:

var gBufferPipelineState: MTLRenderPipelineState!
var gBufferRenderPassDescriptor: MTLRenderPassDescriptor!

Create a new method after init(metalView:) that builds the four textures using the convenience method you created earlier in the shadow pass:

func buildGbufferTextures(size: CGSize) {
  albedoTexture = buildTexture(pixelFormat: .bgra8Unorm, 
                          size: size, label: "Albedo texture")
  normalTexture = buildTexture(pixelFormat: .rgba16Float, 
                          size: size, label: "Normal texture")
  positionTexture = buildTexture(pixelFormat: .rgba16Float, 
                          size: size, label: "Position texture")
  depthTexture = buildTexture(pixelFormat: .depth32Float, 
                          size: size, label: "Depth texture")
}

Create another method within private extension MTLRenderPassDescriptor that will allow you to attach a texture to a render pass descriptor color attachment:

func setUpColorAttachment(position: Int, texture: MTLTexture) {
  let attachment: MTLRenderPassColorAttachmentDescriptor = 
    colorAttachments[position]
  attachment.texture = texture
  attachment.loadAction = .clear
  attachment.storeAction = .store
  attachment.clearColor = MTLClearColorMake(0.73, 0.92, 1, 1)
}

This is similar to the depth attachment you created earlier. This time, you’re dealing with a color attachment, so you can set the clear color. The scene depicts a sunny day with sharp shadows, so you’re setting the color to sky blue.

Create another function in Renderer, that configures the G-buffer render pass descriptor using all of the convenience functions you’ve created:

func buildGBufferRenderPassDescriptor(size: CGSize) {
  gBufferRenderPassDescriptor = MTLRenderPassDescriptor()
  buildGbufferTextures(size: size)
  let textures: [MTLTexture] = [albedoTexture, 
                                normalTexture, 
                                positionTexture]
  for (position, texture) in textures.enumerated() {
    gBufferRenderPassDescriptor.setUpColorAttachment(
          position: position, texture: texture)
  }
  gBufferRenderPassDescriptor.setUpDepthAttachment(
          texture: depthTexture)
}

Here, you attach three color textures and a depth texture to the render pass descriptor. Call this method at the end of mtkView(_:drawableSizeWillChange:):

buildGBufferRenderPassDescriptor(size: size)

When the user resizes the window, all of the render pass descriptor attachment textures will now get resized too. Next, in Renderer, create a method to build the G-buffer pipeline state:

func buildGbufferPipelineState() {
  let descriptor = MTLRenderPipelineDescriptor()
  descriptor.colorAttachments[0].pixelFormat = .bgra8Unorm
  descriptor.colorAttachments[1].pixelFormat = .rgba16Float
  descriptor.colorAttachments[2].pixelFormat = .rgba16Float
  descriptor.depthAttachmentPixelFormat = .depth32Float
  descriptor.label = "GBuffer state"

  descriptor.vertexFunction = 
    Renderer.library.makeFunction(name: "vertex_main")
  descriptor.fragmentFunction = 
    Renderer.library.makeFunction(name: "gBufferFragment")
  descriptor.vertexDescriptor = 
    MTKMetalVertexDescriptorFromModelIO(
               Model.defaultVertexDescriptor)
  do {
    gBufferPipelineState = try 
	  Renderer.device.makeRenderPipelineState(
               descriptor: descriptor)
  } catch let error {
    fatalError(error.localizedDescription)
  }
}

Previously, you only configured colorAttachment[0].pixelFormat and depthAttachmentPixelFormat.

Now that you’re storing an extra two attachments on the render pass descriptor, you’ll specify their pixel formats in the pipeline state. bgra8Unorm has the format of four 8-bit unsigned components, which is all that’s necessary to hold color values between 0 and 255.

However, you’ll need to store the position and normal values in higher precision than the color values by using rgba16Float.

You can reuse vertex_main from the main render pass, as all this does is transform the positions and normals. However, you’ll need a new fragment function that stores the position and normal data into textures and doesn’t process the lighting.

At the end of init(metalView:), call this function:

buildGbufferPipelineState()

At the end of Renderer, add this function to perform the G-buffer pass:

func renderGbufferPass(renderEncoder: MTLRenderCommandEncoder) {
  renderEncoder.pushDebugGroup("Gbuffer pass")
  renderEncoder.label = "Gbuffer encoder"

  renderEncoder.setRenderPipelineState(gBufferPipelineState)
  renderEncoder.setDepthStencilState(depthStencilState)

  uniforms.viewMatrix = camera.viewMatrix
  uniforms.projectionMatrix = camera.projectionMatrix
  fragmentUniforms.cameraPosition = camera.position
  renderEncoder.setFragmentTexture(shadowTexture, index: 0)
  renderEncoder.setFragmentBytes(&fragmentUniforms, 
                  length: MemoryLayout<FragmentUniforms>.stride, 
                  index: 3)
  for model in models {
    draw(renderEncoder: renderEncoder, model: model)
  }
  renderEncoder.endEncoding()
  renderEncoder.popDebugGroup()
}

This pass is similar to the old main render pass, which you’ll be removing shortly. You set the G-buffer’s pipeline state, passing all of the fragment shader information the main pass used, except for the lights. Lighting will take place in the composition pass.

To perform the G-buffer pass, add this code in draw(in:), under // g-buffer pass:

guard let gBufferEncoder = commandBuffer.makeRenderCommandEncoder(
                        descriptor: gBufferRenderPassDescriptor) else {
  return
}
renderGbufferPass(renderEncoder: gBufferEncoder)

Now, to create the G-buffer fragment shader!

In the Shaders group, create a new file named Gbuffer.metal using the Metal File template. As usual, make sure to check both macOS and iOS targets. Then, add this code to the newly created file:

#import "../Utility/Common.h"

struct VertexOut {
  float4 position [[position]];
  float3 worldPosition;
  float3 worldNormal;
  float4 shadowPosition;
};

struct GbufferOut {
  float4 albedo [[color(0)]];
  float4 normal [[color(1)]];
  float4 position [[color(2)]];
};

VertexOut is the same as in Main.metal.

GbufferOut is a return from the fragment function. Instead of returning a float4 containing the color of the fragment, you’ll return a color for each of the render pass descriptor attachments. That’s what the [[color(i)]] attribute indicates.

Add the fragment function:

fragment GbufferOut gBufferFragment(VertexOut in [[stage_in]],
             depth2d<float> shadow_texture [[texture(0)]],
             constant Material &material [[buffer(1)]]) {
  GbufferOut out;
  // 1
  out.albedo = float4(material.baseColor, 1.0);
  out.albedo.a = 0;
  out.normal = float4(normalize(in.worldNormal), 1.0);
  out.position = float4(in.worldPosition, 1.0);

  // 2
  // copy from fragment_main
  float2 xy = in.shadowPosition.xy;
  xy = xy * 0.5 + 0.5;
  xy.y = 1 - xy.y;
  constexpr sampler s(coord::normalized, filter::linear, 
                      address::clamp_to_edge, 
                      compare_func:: less);
  float shadow_sample = shadow_texture.sample(s, xy);
  float current_sample = 
         in.shadowPosition.z / in.shadowPosition.w;

  // 3
  if (current_sample > shadow_sample ) {
    out.albedo.a = 1;
  }
  return out;
}

Going through this code:

1. You populate the G-buffer struct for each fragment with the provided information. You’re currently only using RGB values for the albedo, so you can use the alpha channel to save shadow information.
2. This is the same shadow code from fragment_main in Main.metal.
3. This sets the alpha channel for the albedo struct member to 1 for pixels that have the depth value greater than the shadow value stored in the texture. Only the pixels not supposed to be in the shadow will retain the value 0.

Build and run the project. You should see the same scene you rendered after the main pass. That’s because you rendered to multiple render targets with gBufferEncoder and not to the framebuffer directly.

That said, look in the GPU Debugger and click the Gbuffer encoder group, then the Gbuffer pass group; this lets you see the four textures to which you just rendered.

If you don’t see the textures, click the Navigate to Related items icon at the top left of the center pane and choose Automatic ▸ Attachments.

You can render these textures directly to the app window using a type of encoder called the blit command encoder.

The Blit Command Encoder

To blit means to copy from one part of memory to another. You use a blit command encoder on resources such as textures and buffers. It’s generally used for image processing, but you can (and will) also use it to copy image data that is rendered offscreen.

In Renderer.swift, add this code in draw(in:) under // blit:

guard let blitEncoder = commandBuffer.makeBlitCommandEncoder() else {
  return
}
blitEncoder.pushDebugGroup("Blit")
blitEncoder.label = "Blit encoder"
let origin = MTLOriginMake(0, 0, 0)
let size = MTLSizeMake(Int(view.drawableSize.width), Int(view.drawableSize.height), 1)
blitEncoder.copy(from: albedoTexture, sourceSlice: 0, 
                 sourceLevel: 0, 
                 sourceOrigin: origin, sourceSize: size, 
                 to: drawable.texture, destinationSlice: 0, 
                 destinationLevel: 0, destinationOrigin: origin)
blitEncoder.endEncoding()
blitEncoder.popDebugGroup()

Here, you create a blit command encoder and copy from the albedo texture to the view’s current drawable.

In init(metalView:), add this line to allow the view to render blitted textures:

metalView.framebufferOnly = false

Build and run the project, and you should see the albedo texture rendered to the window this time:

You can see how fast the blit is — it’s happening every frame without a glitch. However, the holy grail of Deferred Rendering is having multiple lights in the scene, so time to work on lights next!

The Lighting pass

Up to this point, you’ve rendered the scene color attachments to multiple render targets, saving them for later use in the fragment shader. This assured that only the visible fragments get processed, thus reducing the amount of calculation that you would have otherwise done for all the geometry in the models in the scene.

Computing hundreds or thousands of lights would not be possible in a forward renderer while also preserving performance.

By rendering a full-screen quad, you’ll render to every fragment on the screen. This allows you to process each fragment from your three textures and calculate lighting for each fragment. The results of this composition pass will end up in the view’s drawable.

At the top of Renderer, declare a new render pipeline state, two quad buffers and create two arrays to hold the coordinates for the quad vertices and its texture:

var compositionPipelineState: MTLRenderPipelineState!

var quadVerticesBuffer: MTLBuffer!
var quadTexCoordsBuffer: MTLBuffer!

let quadVertices: [Float] = [
  -1.0,  1.0,
   1.0, -1.0,
  -1.0, -1.0,
  -1.0,  1.0,
   1.0,  1.0,
   1.0, -1.0
]

let quadTexCoords: [Float] = [
  0.0, 0.0,
  1.0, 1.0,
  0.0, 1.0,
  0.0, 0.0,
  1.0, 0.0,
  1.0, 1.0
]

At the end of init(metalView:), create the two quad buffers you declared above:

quadVerticesBuffer = 
    Renderer.device.makeBuffer(bytes: quadVertices, 
      length: MemoryLayout<Float>.size * quadVertices.count, 
      options: [])
quadVerticesBuffer.label = "Quad vertices"
quadTexCoordsBuffer = 
    Renderer.device.makeBuffer(bytes: quadTexCoords, 
      length: MemoryLayout<Float>.size * quadTexCoords.count, 
      options: [])
quadTexCoordsBuffer.label = "Quad texCoords"

At the end of Renderer, create a new function for the composition pass that will combine lights with the G-buffer information:

func renderCompositionPass(
             renderEncoder: MTLRenderCommandEncoder) {
  renderEncoder.pushDebugGroup("Composition pass")
  renderEncoder.label = "Composition encoder"
  renderEncoder.setRenderPipelineState(compositionPipelineState)
  renderEncoder.setDepthStencilState(depthStencilState)
  // 1
  renderEncoder.setVertexBuffer(quadVerticesBuffer, 
                                offset: 0, index: 0)
  renderEncoder.setVertexBuffer(quadTexCoordsBuffer, 
                                offset: 0, index: 1)
  // 2
  renderEncoder.setFragmentTexture(albedoTexture, index: 0)
  renderEncoder.setFragmentTexture(normalTexture, index: 1)
  renderEncoder.setFragmentTexture(positionTexture, index: 2)
  renderEncoder.setFragmentBytes(&lights, 
    length: MemoryLayout<Light>.stride * lights.count, 
    index: 2)
  renderEncoder.setFragmentBytes(&fragmentUniforms, 
    length: MemoryLayout<FragmentUniforms>.stride, 
	index: 3)
  // 3
  renderEncoder.drawPrimitives(type: .triangle, 
                               vertexStart: 0, 
                               vertexCount: quadVertices.count)
  renderEncoder.endEncoding()
  renderEncoder.popDebugGroup()
}

Going through this code:

1. Send the quad information to the vertex shader.
2. Send the G-buffer textures and lights array to the fragment shader.
3. Draw the quad. Notice that you’re not looping through scene models anymore! :]

In draw(in:), comment out the entire block of code for both the main pass and the blit encoder (but not the g-buffer pass).

Then, add this code under // composition pass:

guard let compositionEncoder = 
    commandBuffer.makeRenderCommandEncoder(
                        descriptor: descriptor) else {
  return
}
renderCompositionPass(renderEncoder: compositionEncoder)

Add the following method to the end of Renderer:

func buildCompositionPipelineState() {
  let descriptor = MTLRenderPipelineDescriptor()
  descriptor.colorAttachments[0].pixelFormat = 
      Renderer.colorPixelFormat
  descriptor.depthAttachmentPixelFormat = .depth32Float
  descriptor.label = "Composition state"
  descriptor.vertexFunction = Renderer.library.makeFunction(
    name: "compositionVert")
  descriptor.fragmentFunction = Renderer.library.makeFunction(
    name: "compositionFrag")
  do {
    compositionPipelineState = 
      try Renderer.device.makeRenderPipelineState(
          descriptor: descriptor)
  } catch let error {
    fatalError(error.localizedDescription)
  }
}

This create the composition pipeline state and is similar to the ones you created for the other two pipeline states. The main thing to note is the two shaders that you’re going to create next.

But first, call this method at the end of init(metalView:):

buildCompositionPipelineState()

Finally, you now need to create the composition shaders.

In the Shaders group, create a new file from the Metal File template named Composition.metal and ensure you check both iOS and macOS targets.

In your new file, create a struct to hold the processed vertices:

#import "../Utility/Common.h"

struct VertexOut {
  float4 position [[position]];
  float2 texCoords;
};

Next, create the vertex shader which assigns the proper position and texture coordinate to each vertex:

vertex VertexOut compositionVert(
  constant float2 *quadVertices [[buffer(0)]],
  constant float2 *quadTexCoords [[buffer(1)]],
  uint id [[vertex_id]]) {
  VertexOut out;
  out.position = float4(quadVertices[id], 0.0, 1.0);
  out.texCoords = quadTexCoords[id];
  return out;
}

Finally, create the fragment shader.

First copy the entire lighting function diffuseLighting from Main.metal to Composition.metal, and rename it compositeLighting.

Note: In a real-world project, this would be a common function, but duplicate it here for convenience.

In Composition.metal, create a new fragment function:

fragment float4 compositionFrag(VertexOut in [[stage_in]],
     constant FragmentUniforms &fragmentUniforms [[buffer(3)]],
     constant Light *lightsBuffer [[buffer(2)]],
     texture2d<float> albedoTexture [[texture(0)]],
     texture2d<float> normalTexture [[texture(1)]],
     texture2d<float> positionTexture [[texture(2)]],
     depth2d<float> shadowTexture [[texture(4)]]) {
  // 1
  constexpr sampler s(min_filter::linear, mag_filter::linear);
  float4 albedo = albedoTexture.sample(s, in.texCoords);
  float3 normal = normalTexture.sample(s, in.texCoords).xyz;
  float3 position = positionTexture.sample(s, in.texCoords).xyz;
  float3 baseColor = albedo.rgb;
  // 2
  float3 diffuseColor = compositeLighting(normal, position, 
                          fragmentUniforms, 
                          lightsBuffer, baseColor);
  // 3
  float shadow = albedo.a;
  if (shadow > 0) {
    diffuseColor *= 0.5;
  }
  return float4(diffuseColor, 1);
}

Going through this code, you:

1. Create a sampler and read in the values from the passed-in textures.
2. Call the same lighting function as you did in the main pass. This time you’re sending values from the textures as parameters.
3. Look at the value of the albedo’s alpha channel where the shadow information is, and if this value is non-negative, make the diffuse color darker.

Build and run the project, and you should see the same scene as you started with.

Well, that’s not very exciting! But don’t worry, this is where the fun begins. Because you’re now doing deferred rendering, you can load up your scene with models and lights, and they’ll all be processed more efficiently than with forward rendering. Don’t believe it? Read on…

There’s a pre-defined method in RenderExtension.swift to create a random number of point lights, so you’ll start by adding thirty point lights to the scene.

In Renderer.swift, in init(metalView:), after this line:

lights.append(sunlight)

Add the following line to create thirty lights in the scene, moderately packed near the center of the scene:

createPointLights(count: 30, min: [-3, 0.3, -3], max: [1, 2, 2])

This creates lots of extra light in the scene, so you can reduce the intensity of the sun. Find where you set up the property sunlight, and change light.intensity to 0.8.

In draw(in:), right before the shadow pass, add this line to rotate the train and show off those new lights and shadows:

models[0].rotation.y += 0.01

Build and run, and you’ll see thirty point lights in your scene rendering — without a glitch!

So far so good, but what if you wanted to render hundreds of light instead of just a few dozen?

To render more lights, you first need to replace setFragmentBytes(_:length:index:) with setFragmentBuffer(_:offset:index:) in renderCompositionPass(renderEncoder:) because you’re only allowed to send up to 4KB in an extemporary buffer.

Add a new buffer for lights at the top of Renderer:

var lightsBuffer: MTLBuffer!

At the end of init(metalView:), initialize the buffer:

lightsBuffer = Renderer.device.makeBuffer(bytes: lights, 
  length: MemoryLayout<Light>.stride * lights.count, 
  options: [])

In renderCompositionPass(renderEncoder:), replace this line:

renderEncoder.setFragmentBytes(&lights,
  length: MemoryLayout<Light>.stride * lights.count,
  index: 2)

With:

renderEncoder.setFragmentBuffer(lightsBuffer, 
                                offset: 0, index: 2)

In init(metalView:), update the call to createPointLights(count:min:max:) with the following:

createPointLights(count: 300, min: [-10, 0.3, -10], 
                  max: [10, 2, 20])

This will render 300 lights instead of just 30. Build and run the project, and you’ll see a lot more lights now:

What a fantastic journey through multipass rendering!

You’ve seen, tried and learned so much. Let’s recap:

• You started with a simple shadow pass where you learned that rendering is not always from a camera’s position but also possible from the light’s position.
• Then, you learned about multipass rendering and how you can use a shadow map that was saved in a first pass during a second pass.
• Next, you learned what a G-buffer is and how deferred rendering works.
• You also learned how to improve your app’s performance by processing lights based on the visibility of the fragments affected.
• Finally, you learned there’s another type of command encoder other than the rendering one — the blit command encoder.

Where to go from here?

If you’re trying to improve your app performance, you can try a few approaches. One is to render the lights as light volumes and use stencil tests to select only lights that are affecting the fragments and only render those lights instead of all.

In the next chapter, you’re in for some more advanced GPU topics!