Chapters
Chapters

Chapters

Hide chapters

Metal by Tutorials

Fifth Edition · macOS 26, iOS 26 · Swift 6, Metal 3 · Xcode 26

Before You Begin

Section 0: 6 chapters
Show chapters Hide chapters

Section I: Beginning Metal

Section 1: 10 chapters
Show chapters Hide chapters

Section II: Intermediate Metal

Section 2: 8 chapters
Show chapters Hide chapters

Section III: Advanced Metal

Section 3: 8 chapters
Show chapters Hide chapters

Section IV: Pushing the GPU

Section 4: 6 chapters
Show chapters Hide chapters

21. Image-Based Lighting
Written by Marius Horga & Caroline Begbie

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

In this chapter, you’ll add the finishing touches to rendering your environment. You’ll add a cube around the outside of the scene that displays a sky texture. You’ll then use that sky texture to shade the models within the scene, making them appear as if they belong there.

Look at the following comparison of two renders.

The final and challenge renders
The final and challenge renders

This comparison demonstrates how you can use the same shader code but change the sky image to create different lighting environments. The rendered models reflect the tinge of color from the sky.

The Starter Project

➤ In Xcode, open the starter project for this chapter and build and run the app.

The starter project
The starter project

The project contains the forward renderer with transparency from the previous chapter. The scene uses an arcball camera, and contains a ground plane and car. The scene lighting consists of one sunlight.

There are a few additional files that you’ll use throughout the chapter. Common.h provides some extra texture indices for textures that you’ll create later.

There are some glaring problems with the render:

  • All metals, such as the metallic wheel hubs, look dull. Pure metals reflect their surroundings, and there are currently no surroundings to reflect.
  • Where the light doesn’t directly hit the car, the color is pure black. This happens because the app doesn’t provide any ambient light. Later in this chapter, you’ll use the skylight as global ambient light.

The Skybox

Currently, the sky is a single color, which looks unrealistic. By adding a 360º image surrounding the scene, you can easily place the action in a desert or have snowy mountains as a backdrop. To do this, you’ll create a skybox cube that surrounds the entire scene.

import MetalKit

struct Skybox {
  let mesh: MTKMesh
  var skyTexture: MTLTexture?
  let pipelineState: MTLRenderPipelineState
  let depthStencilState: MTLDepthStencilState?
}

init(textureName: String?) {
  let allocator =
    MTKMeshBufferAllocator(device: Renderer.device)
  let cube = MDLMesh(
    boxWithExtent: [1, 1, 1],
    segments: [1, 1, 1],
    inwardNormals: true,
    geometryType: .triangles,
    allocator: allocator)
  do {
    mesh = try MTKMesh(
      mesh: cube, device: Renderer.device)
  } catch {
    fatalError("failed to create skybox mesh")
  }
}

static func createSkyboxPSO(
  vertexDescriptor: MTLVertexDescriptor?
) -> MTLRenderPipelineState {
  let vertexFunction =
    Renderer.library?.makeFunction(name: "vertex_skybox")
  let fragmentFunction =
    Renderer.library?.makeFunction(name: "fragment_skybox")
  let pipelineDescriptor = MTLRenderPipelineDescriptor()
  pipelineDescriptor.vertexFunction = vertexFunction
  pipelineDescriptor.fragmentFunction = fragmentFunction
  pipelineDescriptor.colorAttachments[0].pixelFormat =
    Renderer.viewColorPixelFormat
  pipelineDescriptor.depthAttachmentPixelFormat = 
    Renderer.viewDepthPixelFormat
  pipelineDescriptor.vertexDescriptor = vertexDescriptor
  return createPSO(descriptor: pipelineDescriptor)
}

static func buildDepthStencilState() -> MTLDepthStencilState? {
  let descriptor = MTLDepthStencilDescriptor()
  descriptor.depthCompareFunction = .lessEqual
  descriptor.isDepthWriteEnabled = true
  return Renderer.device.makeDepthStencilState(
    descriptor: descriptor)
}

pipelineState = PipelineStates.createSkyboxPSO(
  vertexDescriptor: MTKMetalVertexDescriptorFromModelIO(
    cube.vertexDescriptor))
depthStencilState = Self.buildDepthStencilState()

Rendering the Skybox

➤ Still in Skybox.swift, create a new method to perform the skybox rendering:

func render(
  encoder: MTLRenderCommandEncoder,
  uniforms: Uniforms
) {
  encoder.pushDebugGroup("Skybox")
  encoder.setRenderPipelineState(pipelineState)
  // encoder.setDepthStencilState(depthStencilState)
  encoder.setVertexBuffer(
    mesh.vertexBuffers[0].buffer,
    offset: 0,
    index: 0)
}

var uniforms = uniforms
uniforms.viewMatrix.columns.3 = [0, 0, 0, 1]
encoder.setVertexBytes(
  &uniforms,
  length: MemoryLayout<Uniforms>.stride,
  index: UniformsBuffer.index)

let submesh = mesh.submeshes[0]
encoder.drawIndexedPrimitives(
  type: .triangle,
  indexCount: submesh.indexCount,
  indexType: submesh.indexType,
  indexBuffer: submesh.indexBuffer.buffer,
  indexBufferOffset: 0)
encoder.popDebugGroup()

The Skybox Shader Functions

In the Shaders folder, create a new Metal file named Skybox.metal.

#import "Common.h"

struct VertexIn {
  float4 position [[attribute(Position)]];
};

struct VertexOut {
  float4 position [[position]];
};

vertex VertexOut vertex_skybox(
  const VertexIn in [[stage_in]],
  constant Uniforms &uniforms [[buffer(UniformsBuffer)]])
{
  VertexOut out;
  float4x4 vp = uniforms.projectionMatrix * uniforms.viewMatrix;
  out.position = (vp * in.position).xyww;
  return out;
}

fragment half4 fragment_skybox(
  VertexOut in [[stage_in]]) {
  return half4(1, 1, 0, 1);
}
Skybox Projection
Fxygar Srinebqiuk

Integrating the Skybox Into the Scene

➤ Open GameScene.swift, and add a new property to GameScene:

let skybox: Skybox?

skybox = Skybox(textureName: nil)

scene.skybox?.render(
  encoder: renderEncoder,
  uniforms: uniforms)
A flickering sky
A bkumjukipp ccl

A yellow sky
E godgoj xqn

Procedural Skies

Yellow skies might be appropriate on a different planet, but how about a procedural sky? A procedural sky is one built out of various parameters such as weather conditions and time of day. Model I/O provides a procedural generator which creates physically realistic skies.

A sunrise
O vuhlewu

Cube Textures

Cube textures are similar to the 2D textures that you’ve already been using. 2D textures map to a quad and have two texture coordinates, whereas cube textures consist of six 2D textures: one for each face of the cube. You sample the textures with a 3D vector.

The sky texture in the asset catalog
Wfe qfr jetnuxi ek wbo egcir xoxotib

Adding the Procedural Sky

You’ll use these sky textures shortly, but for now, you’ll add a procedural sky to your scene.

struct SkySettings {
  var turbidity: Float = 0.15
  var sunElevation: Float = 0.56
  var upperAtmosphereScattering: Float = 0.66
  var groundAlbedo: Float = 0.8
}
var skySettings = SkySettings()

func loadGeneratedSkyboxTexture(dimensions: SIMD2<Int32>)
  -> MTLTexture? {
  var texture: MTLTexture?
  let skyTexture = MDLSkyCubeTexture(
    name: "sky",
    channelEncoding: .float16,
    textureDimensions: dimensions,
    turbidity: skySettings.turbidity,
    sunElevation: skySettings.sunElevation,
    upperAtmosphereScattering:
      skySettings.upperAtmosphereScattering,
    groundAlbedo: skySettings.groundAlbedo)
  do {
    let textureLoader =
      MTKTextureLoader(device: Renderer.device)
    texture = try textureLoader.newTexture(
      texture: skyTexture,
      options: nil)
  } catch {
    print(error.localizedDescription)
  }
  return texture
}

if let textureName {
  // load named texture here
} else {
  skyTexture = loadGeneratedSkyboxTexture(dimensions: [256, 256])
}

encoder.setFragmentTexture(
  skyTexture,
  index: SkyboxTexture.index)

float3 textureCoordinates;
Skybox coordinates
Ptmmaw neoylovasim

out.textureCoordinates = in.position.xyz;

fragment half4 fragment_skybox(
  VertexOut in [[stage_in]],
  texturecube<half> cubeTexture [[texture(SkyboxTexture)]])
{
  constexpr sampler default_sampler(filter::linear);
  half4 color = cubeTexture.sample(
    default_sampler,
    in.textureCoordinates);
  return color;
}
A procedural sky
U whikicayus qyf

Custom Sky Textures

As mentioned earlier, you can use your own 360º sky textures. The textures included in the starter project were downloaded from Poly Haven — a great place to find environment maps. Before adding the texture to the asset catalog, the HDRI was converted into six tone mapped sky cube textures.

skyTexture = TextureController.loadCubeTexture(
  imageName: textureName)

skybox = Skybox(textureName: "sky")
The skybox
Cso ldnwan

Reflection

Now that you have something to reflect, you can easily implement reflection of the sky onto the car. When rendering the car, all you have to do is take the camera view direction, reflect it about the surface normal, and sample the skycube along the reflected vector for the fragment color for the car.

Reflection
Keldovzooh

"fragment_IBL"
Color textures only
Topaf mefsajul ahkn

func update(encoder: MTLRenderCommandEncoder) {
  encoder.setFragmentTexture(
    skyTexture,
    index: SkyboxTexture.index)
}

scene.skybox?.update(encoder: renderEncoder)

texturecube<float> skybox [[texture(SkyboxTexture)]]

float3 viewDirection =
  in.worldPosition.xyz - params.cameraPosition;
viewDirection = normalize(viewDirection);
float3 textureCoordinates =
  reflect(viewDirection, normal);

constexpr sampler defaultSampler(filter::linear);
color = skybox.sample(
  defaultSampler, textureCoordinates);
float4 copper = float4(0.722, 0.451, 0.2, 1);
color = color * copper;
Reflections
Rickiqteury

Image-Based Lighting

At the beginning of the chapter, there were two problems with the original car render. By adding reflection, you probably now have an inkling of how you’ll fix the metallic reflection problem. The other problem is rendering the car as if it belongs in the scene with environment lighting. IBL or Image-Based Lighting is one way of dealing with this problem.

Diffuse Reflection

Light comes from all around us. Sunlight bounces around and colors reflect. When rendering an object, you should take into account the color of the light coming from every direction.

Diffuse reflection
Gockevo kivhunzuox

var diffuseTexture: MTLTexture?

mutating func loadIrradianceMap() {
  // 1
  guard let skyCube =
    MDLTexture(cubeWithImagesNamed: ["cube-sky.png"])
  else { return }
  // 2
  let irradiance =
    MDLTexture.irradianceTextureCube(
      with: skyCube,
      name: nil,
      dimensions: [64, 64],
      roughness: 0.6)
  // 3
  let loader = MTKTextureLoader(device: Renderer.device)
  do {
    diffuseTexture = try loader.newTexture(
    texture: irradiance,
    options: nil)
  } catch {
    fatalError(error.localizedDescription)
  }
}

loadIrradianceMap()

encoder.setFragmentTexture(
  diffuseTexture,
  index: SkyboxDiffuseTexture.index)

texturecube<float> skyboxDiffuse [[texture(SkyboxDiffuseTexture)]]

float4 diffuse = skyboxDiffuse.sample(textureSampler, normal);
color = diffuse * float4(material.baseColor, 1);
Diffuse from irradiance
Loywafa knug otjaqiufwi

diffuseTexture =
  TextureController.loadCubeTexture(
    imageName: "irradiance.png")
Brighter irradiance
Ptetctav agfecionsa

Specular Reflection

The irradiance map provides the diffuse and ambient reflection, but the specular reflection is a bit more difficult.

Specular reflection
Sduroyar qavqocsauf

Pre-filtered environment maps
Bmi-suvzagaz iscozokcelr sujg

BRDF Look-Up Table

To calculate the final color, you use a Bidirectional Reflectance Distribution Function (BRDF) that takes in the actual roughness of the model and the current viewing angle and returns the scale and bias for the Fresnel and geometric attenuation contributions.

A BRDF LUT
O WVHK GOK

var brdfLut: MTLTexture?

brdfLut = Renderer.buildBRDF()

encoder.setFragmentTexture(
  brdfLut,
  index: BRDFLutTexture.index)
BRDF LUT is on the GPU
KNBF TOR if iv vle TJI

texture2d<float> brdfLut [[texture(BRDFLutTexture)]]

// 1
constexpr sampler s(filter::linear, mip_filter::linear);
float3 prefilteredColor
  = skybox.sample(
    s,
    textureCoordinates,
    level(material.roughness * 10)).rgb;
// 2
float nDotV = saturate(dot(normal, -viewDirection));
float2 envBRDF
  = brdfLut.sample(s, float2(material.roughness, nDotV)).rg;
return float4(envBRDF, 0, 1);
The BRDF look up result
Hji JSPP qaox ix gaqicq

Fresnel Reflectance

When light hits an object straight on, some of the light is reflected. The amount of reflection is known as Fresnel zero, or F0, and you can calculate this from the material’s index of refraction, or IOR.

float3 f0 = mix(0.04, material.baseColor.rgb, material.metallic);
float3 specularIBL = f0 * envBRDF.r + envBRDF.g;

float3 specular = prefilteredColor * specularIBL;
color += float4(specular, 1);
return color;
Diffuse and specular
Rublilo idn phetaquk

Tweaking

Being able to tweak shaders gives you complete power over how your renders look. Because you’re using low dynamic range lighting, the non-metal diffuse color looks a bit dark. You can tweak the color very easily.

diffuse = mix(pow(diffuse, 0.2), diffuse, material.metallic);
diffuse *= calculateShadow(in.shadowPosition, shadowTexture);
Tweaking the shader
Zmeokiqk mme btameh

Ambient Occlusion Maps

Ambient occlusion is a technique that approximates how much light should fall on a surface. If you look around you — even in a bright room — where surfaces are very close to each other, they’re darker than exposed surfaces. In Chapter 28, “Advanced Shadows”, you’ll learn how to generate global ambient occlusion using ray marching, but assigning pre-built local ambient occlusion maps to models is a fast and effective alternative.

color *= material.ambientOcclusion;

Challenge

On the first page of this chapter is a comparison of the car rendered in two different lighting situations. Your challenge is to create the red lighting scene.

Completed challenge
Zobfhejet vtahzamka

Key Points

  • Using a cuboid skybox, you can surround your scene with a texture.
  • Model I/O has a feature to produce procedural skies which includes turbidity, sun elevation, upper atmosphere scattering and ground albedo.
  • Cube textures have six faces. Each of the faces can have mipmaps.
  • Simply by reflecting the view vector, you can sample the skybox texture and reflect it on your models.
  • Image-based lighting uses the sky texture for lighting. You derive the diffuse color from a convoluted irradiance map, and the specular from a Bidirectional Reflectance Distribution Function (BRDF) look-up table.

Where to Go From Here?

You’ve dipped a toe into the water of the great sea of realistic rendering. If you want to explore more about this fascinating topic, references.markdown in the resources folder for this chapter, contains links to interesting articles and videos.

20. Fragment Post-Processing
22. Reflection & Refraction
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2025 Kodeco Inc.

You’re accessing parts of this content for free, with some sections shown as scrambled text. Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now