Chapters
Chapters

Chapters

Hide chapters

Metal by Tutorials

Second Edition · iOS 13 · Swift 5.1 · Xcode 11

Before You Begin

Section 0: 3 chapters
Show chapters Hide chapters

Section I: The Player

Section 1: 8 chapters
Show chapters Hide chapters

Section II: The Scene

Section 2: 7 chapters
Show chapters Hide chapters

Section III: The Effects

Section 3: 10 chapters
Show chapters Hide chapters

11. Tessellation & Terrains
Written by Caroline Begbie

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

So far you’ve used normal map trickery in the fragment function to show the fine details of your low poly models. To achieve a similar level of detail without using normal maps requires a change of model geometry by adding more vertices. The problem with adding more vertices is that when you send them to the GPU, it chokes up the pipeline. A hardware tessellator in the GPU can create vertices on the fly, adding a greater level of detail and thereby using fewer resources.

In this chapter, you’re going to create a detailed terrain using a small number of points. You’ll send a flat ground plane with a grayscale texture describing the height, and the tessellator will create as many vertices as needed. The vertex function will then read the texture and displace these new vertices vertically; in other words, move them upwards.

Notice how many vertices the tessellator creates. In this example, the number of vertices depends on how close the control points are to the camera. You’ll learn more about that later.

Tessellation

Instead of sending vertices to the GPU, you send patches. These patches are made up of control points — a minimum of three for a triangle patch, or four for a quad patch. The tessellator can convert each quad patch into a certain number of triangles: up to 4,096 triangles on a recent iMac and 256 triangles on an iPhone that’s capable of tessellation.

Note: Tessellation is available on all Macs since 2012 and on iOS 10 GPU Family 3 and up; this includes the iPhone 6s and newer devices. Tessellation is not, however, available on the iOS simulator.

With tessellation, you can:

  • Send less data to the GPU. Because the GPU doesn’t store tessellated vertices in graphics memory, it’s more efficient on resources.
  • Make low poly objects look less low poly by curving patches.
  • Displace vertices for fine detail instead of using normal maps to fake it.
  • Decide on the level of detail based on the distance from the camera. The closer an object is to the camera, the more vertices it contains.

The starter project

Before creating a tessellated terrain, you’ll tessellate a single four-point patch. Open and run the starter project for this chapter. The code in this project is the minimum needed for a simple render of six vertices to create a quad.

Tessellation patches

A patch consists of a certain number of control points, generally:

Tessellation factors

For each patch, you need to specify inside edge factors and outside edge factors. The four-point patch in the following image shows different edge factors for each edge — specified as [2, 4, 8, 16] — and two different inside factors — specified as [8, 16], for horizontal and vertical respectively.

let patches = (horizontal: 1, vertical: 1)
var patchCount: Int {
  return patches.horizontal * patches.vertical
}

var edgeFactors: [Float] = [4]
var insideFactors: [Float] = [4]

lazy var tessellationFactorsBuffer: MTLBuffer? = {
  // 1
  let count = patchCount * (4 + 2)
  // 2
  let size = count * MemoryLayout<Float>.size / 2
  return Renderer.device.makeBuffer(length: size, 
                              options: .storageModePrivate)
}()

Set up patch data

Instead of an array of six vertices, you’ll create a four-point patch with control points at the corners.

var controlPointsBuffer: MTLBuffer?

let controlPoints = createControlPoints(patches: patches,
                                        size: (2, 2))
controlPointsBuffer = 
  Renderer.device.makeBuffer(bytes: controlPoints,
    length: MemoryLayout<float3>.stride * controlPoints.count)

Set up the render pipeline state

You can configure the tessellator by changing the pipeline state properties. Until now, you’ve been processing vertices, however, you’ll modify the vertex descriptor so it processes patches instead.

vertexDescriptor.layouts[0].stepFunction = .perPatchControlPoint

Kernel (compute) functions

You’re accustomed to setting up a render pipeline state for rendering, but compute allows you to do different tasks on different threads.

static func buildComputePipelineState() 
                 -> MTLComputePipelineState {
  guard let kernelFunction =
    Renderer.library?.makeFunction(name: "tessellation_main") else {
      fatalError("Tessellation shader function not found")
  }
  return try! 
      Renderer.device.makeComputePipelineState(
                         function: kernelFunction)
}

var tessellationPipelineState: MTLComputePipelineState

tessellationPipelineState = Renderer.buildComputePipelineState()

Compute pass

You’ve set up a compute pipeline state and an MTLBuffer containing the patch data. You also created an empty buffer which the tessellation kernel will fill with the edge and inside factors. Next, you need to create a compute command encoder. This encoder is similar to a render command encoder in that you add commands to the encoder and then send it off to the GPU.

let computeEncoder = commandBuffer.makeComputeCommandEncoder()!
computeEncoder.setComputePipelineState(
                       tessellationPipelineState)
computeEncoder.setBytes(&edgeFactors, 
    length: MemoryLayout<Float>.size * edgeFactors.count, 
    index: 0)
computeEncoder.setBytes(&insideFactors, 
    length: MemoryLayout<Float>.size * insideFactors.count, 
    index: 1)
computeEncoder.setBuffer(tessellationFactorsBuffer, offset: 0, 
                         index: 2)

let width = min(patchCount, 
                tessellationPipelineState.threadExecutionWidth)
computeEncoder.dispatchThreadgroups(MTLSizeMake(patchCount, 
                                                1, 1),
    threadsPerThreadgroup: MTLSizeMake(width, 1, 1))
computeEncoder.endEncoding()

The tessellation kernel

Open Shaders.metal. Currently, this file contains simple vertex and fragment functions. You’ll add the tessellation kernel here.

kernel void 
  tessellation_main(constant float* edge_factors [[buffer(0)]],
               constant float* inside_factors [[buffer(1)]],
               device MTLQuadTessellationFactorsHalf* 
                              factors [[buffer(2)]],
               uint pid [[thread_position_in_grid]]) {
}

factors[pid].edgeTessellationFactor[0] = edge_factors[0];
factors[pid].edgeTessellationFactor[1] = edge_factors[0];
factors[pid].edgeTessellationFactor[2] = edge_factors[0];
factors[pid].edgeTessellationFactor[3] = edge_factors[0];
  
factors[pid].insideTessellationFactor[0] = inside_factors[0];
factors[pid].insideTessellationFactor[1] = inside_factors[0];

Render pass

Back in Renderer.swift, before doing the render, you need to tell the render encoder about the tessellation factors buffer that you updated during the compute pass.

renderEncoder.setTessellationFactorBuffer(
                    tessellationFactorsBuffer,
                    offset: 0, instanceStride: 0)

renderEncoder.setVertexBuffer(vertexBuffer, offset: 0, index: 0)

renderEncoder.setVertexBuffer(controlPointsBuffer, 
                              offset: 0, index: 0)

renderEncoder.drawPatches(numberOfPatchControlPoints: 4,
                          patchStart: 0, patchCount: patchCount,
                          patchIndexBuffer: nil, 
                          patchIndexBufferOffset: 0,
                          instanceCount: 1, baseInstance: 0)

The post-tessellation vertex function

Open Shaders.metal again. The vertex function is called after the tessellator has done its job of creating the vertices and will operate on each one of these new vertices. In the vertex function, you’ll tell each vertex what its position in the rendered quad should be.

// 1
[[patch(quad, 4)]]
// 2
vertex VertexOut 
// 3
  vertex_main(patch_control_point<ControlPoint> 
    control_points [[stage_in]],
// 4          
              constant float4x4 &mvp [[buffer(1)]],
// 5                             
              float2 patch_coord [[position_in_patch]]) {              
}

float u = patch_coord.x;
float v = patch_coord.y;
  
VertexOut out;
out.position = float4(u, v, 0, 1);
out.color = float4(u, v, 0, 1);
return out;


float2 top = mix(control_points[0].position.xz,
                 control_points[1].position.xz, u);
float2 bottom = mix(control_points[3].position.xz,
                    control_points[2].position.xz, u);


out.position = float4(u, v, 0, 1);

float2 interpolated = mix(top, bottom, v);
float4 position = float4(interpolated.x, 0.0, 
                         interpolated.y, 1.0);
out.position = mvp * position;

Multiple patches

Now that you know how to tessellate one patch, you can tile the patches and choose edge factors that depend on dynamic factors, like distance.

let patches = (horizontal: 2, vertical: 2)


uint patchID [[patch_id]],

out.color = float4(0);
if (patchID == 0) {
  out.color = float4(1, 0, 0, 1);
}

Tessellation by distance

In this section, you’re going to create a terrain with patches that are tessellated according to the distance from the camera. When you’re close to a mountain, you need to see more detail; when you’re farther away, less. Having the ability to dial in the level of detail is where tessellation comes into its own. By setting the level of detail, you save on how many vertices the GPU has to process in any given situation.

typedef struct {
  vector_float2 size;
  float height; 
  uint maxTessellation;
} Terrain;

static let maxTessellation = 16

var terrain = Terrain(size: [2, 2], height: 1, 
          maxTessellation: UInt32(Renderer.maxTessellation))

let controlPoints = createControlPoints(patches: patches, 
                          size: (width: terrain.size.x, 
                                 height: terrain.size.y))

var cameraPosition = viewMatrix.columns.3
computeEncoder.setBytes(&cameraPosition, 
                        length: MemoryLayout<float4>.stride, 
                        index: 3)
var matrix = modelMatrix
computeEncoder.setBytes(&matrix, 
                        length: MemoryLayout<float4x4>.stride, 
                        index: 4)
computeEncoder.setBuffer(controlPointsBuffer, 
                         offset: 0, index: 5)
computeEncoder.setBytes(&terrain,
                        length: MemoryLayout<Terrain>.stride,
                        index: 6)

constant float4 &camera_position [[buffer(3)]],
constant float4x4 &modelMatrix   [[buffer(4)]],
constant float3* control_points  [[buffer(5)]],
constant Terrain &terrain        [[buffer(6)]],


float calc_distance(float3 pointA, float3 pointB, 
                    float3 camera_position, 
                    float4x4 modelMatrix) {
  float3 positionA = (modelMatrix * float4(pointA, 1)).xyz;
  float3 positionB = (modelMatrix * float4(pointB, 1)).xyz;
  float3 midpoint = (positionA + positionB) * 0.5;
  
  float camera_distance = distance(camera_position, midpoint);
  return camera_distance;
}

uint index = pid * 4;   

float totalTessellation = 0;

for (int i = 0; i < 4; i++) {
  int pointAIndex = i;
  int pointBIndex = i + 1;
  if (pointAIndex == 3) {
    pointBIndex = 0;
  }
  int edgeIndex = pointBIndex;
}

float cameraDistance = 
       calc_distance(control_points[pointAIndex + index],
                     control_points[pointBIndex + index],
                     camera_position.xyz,
                     modelMatrix);

float tessellation = 
    max(4.0, terrain.maxTessellation / cameraDistance);
factors[pid].edgeTessellationFactor[edgeIndex] = tessellation;
totalTessellation += tessellation;

factors[pid].insideTessellationFactor[0] = 
      totalTessellation * 0.25;
factors[pid].insideTessellationFactor[1] = 
      totalTessellation * 0.25;

// 1
descriptor.tessellationFactorStepFunction = .perPatch
// 2
descriptor.maxTessellationFactor = Renderer.maxTessellation
// 3
descriptor.tessellationPartitionMode = .fractionalEven

Displacement

You’ve used textures for various purposes in earlier chapters. Now you’ll use a height map to change the height of each vertex.

let heightMap: MTLTexture

do {
  heightMap = try Renderer.loadTexture(imageName: "mountain")
} catch {
  fatalError(error.localizedDescription)
}

renderEncoder.setVertexTexture(heightMap, index: 0)
renderEncoder.setVertexBytes(&terrain,
                      length: MemoryLayout<Terrain>.stride,
                      index: 6)

texture2d<float> heightMap [[texture(0)]],
constant Terrain &terrain [[buffer(6)]],

// 1
float2 xy = (position.xz + terrain.size / 2.0) / terrain.size;
// 2
constexpr sampler sample;
float4 color = heightMap.sample(sample, xy);
out.color = float4(color.r);

// 3
float height = (color.r * 2 - 1) * terrain.height;
position.y = height;

out.color = float4(0);
if (patchID == 0) {
  out.color = float4(1, 0, 0, 1);
}

var rotation = float3(Float(-20).degreesToRadians, 0, 0)


static let maxTessellation: Int = {
  #if os(macOS)
  return 64
  #else
  return 16
  #endif
} ()

let patches = (horizontal: 6, vertical: 6)
var terrain = Terrain(size: [8, 8], height: 1, 
                maxTessellation: UInt32(Renderer.maxTessellation))

Shading by height

In the last section, you sampled the height map in the vertex function, and the colors are interpolated when sent to the fragment function. For maximum color detail, you need to sample from textures per fragment, not per vertex.

let cliffTexture: MTLTexture
let snowTexture: MTLTexture
let grassTexture: MTLTexture

cliffTexture = 
    try Renderer.loadTexture(imageName: "cliff-color")
snowTexture = try Renderer.loadTexture(imageName: "snow-color")
grassTexture = 
    try Renderer.loadTexture(imageName: "grass-color")

renderEncoder.setFragmentTexture(cliffTexture, index: 1)
renderEncoder.setFragmentTexture(snowTexture, index: 2)
renderEncoder.setFragmentTexture(grassTexture, index: 3)

float height;
float2 uv;

out.uv = xy;
out.height = height;

texture2d<float> cliffTexture [[texture(1)]],
texture2d<float> snowTexture  [[texture(2)]],
texture2d<float> grassTexture [[texture(3)]]

constexpr sampler sample(filter::linear, address::repeat);
float tiling = 16.0;
float4 color;
if (in.height < -0.5) {
  color = grassTexture.sample(sample, in.uv * tiling);
} else if (in.height < 0.3) {
  color = cliffTexture.sample(sample, in.uv * tiling);
} else {
  color = snowTexture.sample(sample, in.uv * tiling);
}
return color;


descriptor.tessellationPartitionMode = .pow2

Shading by slope

The snow line in your previous render is unrealistic. By checking the slope of the mountain, you can show the snow texture in flatter areas, and show the cliff texture where the slope is steep.

Metal Performance Shaders

The Metal Performance Shaders framework contains many useful, highly optimized shaders for image processing, matrix multiplication, machine learning and raytracing. You’ll read more about them in Chapter 21, “Metal Performance Shaders.”

import MetalPerformanceShaders

static func heightToSlope(source: MTLTexture) -> MTLTexture {
}

let descriptor = 
  MTLTextureDescriptor.texture2DDescriptor(
         pixelFormat: source.pixelFormat,
         width: source.width,
         height: source.height,
         mipmapped: false)
descriptor.usage = [.shaderWrite, .shaderRead]

guard let destination = 
      Renderer.device.makeTexture(descriptor: descriptor),
  let commandBuffer = Renderer.commandQueue.makeCommandBuffer() 
      else { fatalError() }

let shader = MPSImageSobel(device: Renderer.device)
shader.encode(commandBuffer: commandBuffer,
              sourceTexture: source,
              destinationTexture: destination)
commandBuffer.commit()
return destination

let terrainSlope: MTLTexture

terrainSlope = Renderer.heightToSlope(source: heightMap)

Challenge

Your challenge for this chapter is to use the slope texture from the Sobel filter to place snow on the mountain on the parts that aren’t steep. Because you don’t need pixel perfect accuracy, you can read the slope image in the vertex function and send that value to the fragment function.

Where to go from here?

With very steep displacement, there can be lots of texture stretching between vertices. There are various algorithms to overcome this, and you can find one in Apple’s excellent sample code: Dynamic Terrain with Argument Buffers at https://developer.apple.com/documentation/metal/fundamental_components/gpu_resources/dynamic_terrain_with_argument_buffers. This is a complex project that showcases argument buffers, but the dynamic terrain portion is interesting.

10. Fragment Post-Processing
12. Environment
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2024 Kodeco Inc.

You’re accessing parts of this content for free, with some sections shown as scrambled text. Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now