13. Instancing & Procedural Generation

Written by Caroline Begbie

Now that you have an environment with a sky that you can populate with models, you’ll want to add some randomness to your scene. Trees and grass make the scene more natural, but they can take up a lot of valuable resources without adding any player action to your game.

In this chapter, you’ll first find out how to efficiently render many trees and blades of grass using instancing. You’ll then render instanced rocks, and you’ll use morphing for different shapes. Finally, you’ll create a procedural house system that will create a row of houses whose size and style will change every time you run the app.

As well as all that, you’ll improve your versatility and proficiency in handling GPU resources which will enable you to access any data in MTLBuffers with confidence.

The starter project

Open the starter project for this chapter. This is almost the same project as the previous chapter, with a few skybox tweaks, but it includes a new GameScene that renders 100 trees in random places. Each tree consists of 9,119 vertices, with a color texture of size 8 MB and 2048×2048 pixels.

Note: You generally wouldn’t spend 9,119 vertices just on a tree, unless you wanted some fine detail. Low-poly trees are much more efficient. However, this example will show you just how important instancing is.

The project also contains two other scenes with supporting files for later sections in which you’ll create a rock system and perform procedural house generation.

Build and run the app. If your device can’t handle 100 trees, you might need to reduce instanceCount in GameScene.

In Xcode, check the Debug navigator. This is the result on a 2015 iMac:

Your aim in the first part of this chapter is to reduce that huge memory footprint, and maybe even address that terrible frame rate.

Note: If you’re using faster hardware and are seeing 60 FPS, you might consider increasing the number of trees to 200 or even 300 for the purpose of this exercise – set instanceCount in GameScene accordingly.

Instancing

Note: Instance drawing for iOS is available for GPU Family 3 and up - that’s a minimum hardware device of the iPhone 6s.

In your scene, there’s a lot of duplicated data going to the GPU. For each tree render, you’re sending a texture and 9,119 vertices.

This texture and geometry are the same for every tree, and by using instancing, the GPU can use one texture and geometry for multiple models. In this scene, it will mean 24 fewer texture and geometry resources transferred to the GPU.

Each instance of the model will have unique data. The unique data for each tree is its position, rotation and scale, which you set in GameScene. Instead of rendering many Models, you’ll render one Model many times using the unique instance data.

The following diagram shows Model with its existing properties of geometry (the vertex buffer), textures and position and rotation. You’ll add a new array of Transforms containing the position and rotation for all the instances.

Open Transform.swift. This holds a single struct that contains position, rotation and scale information. There are two computed properties that return matrices. You’ll hold an array of Transforms in Model - one for each instance.

Open Model.swift in the Nodes group. Add properties for the array of transforms and the instance count:

private var transforms: [Transform]
let instanceCount: Int

transforms will hold the transform data for each tree, and instanceCount the total number of trees. Your app won’t compile until you’ve initialized these new properties.

Add an extra optional parameter to init to accept an instance count, so that the function parameters are:

init(name: String,
     vertexFunctionName: String = "vertex_main",
     fragmentFunctionName: String = "fragment_IBL",
     instanceCount: Int = 1) {

If you don’t specify a count when creating a Model, the default number of instances is 1.

Create a new type method to initialize transforms:

static func buildTransforms(instanceCount: Int) -> [Transform] {
  return [Transform](repeatElement(Transform(), 
                     count: instanceCount))
}

This method creates an array for the number of instances required.

In init, before super.init(), initialize the new properties:

self.instanceCount = instanceCount
transforms = Model.buildTransforms(instanceCount: instanceCount)

Your project will now compile.

Create the instance MTLBuffer

The GPU only requires matrices, not the position and rotation data, so instead of sending all the transform data to the GPU, you’ll only send a buffer of matrices.

When you update the position or rotation of an instance, at the same time you’ll update the model matrix and the normal matrix in this new buffer.

In Model.swift, create a new property to hold the matrices:

var instanceBuffer: MTLBuffer

In Common.h, in the Metal Shaders group, define a struct that will format the data for both Swift and your shader functions:

struct Instances {
  matrix_float4x4 modelMatrix;
  matrix_float3x3 normalMatrix;
};

Back in Model.swift, initialize the instances buffer with this new type method:

static func buildInstanceBuffer(transforms: [Transform]) -> MTLBuffer {
// 1
  let instances = transforms.map {
      Instances(modelMatrix: $0.modelMatrix,
          normalMatrix: float3x3(normalFrom4x4: $0.modelMatrix))
  }
// 2
  guard let instanceBuffer = 
      Renderer.device.makeBuffer(bytes: instances, 
        length: MemoryLayout<Instances>.stride 
                     * instances.count) else { 
    fatalError("Failed to create instance buffer")
  }
  return instanceBuffer
}

With this code, you:

1. Convert the position and rotation data in transforms into an array of Instances.
2. Create the instances buffer and initialize it with this new array.

In init, before super.init(), initialize instanceBuffer:

instanceBuffer = 
     Model.buildInstanceBuffer(transforms: transforms)

Your project will now compile again.

Accessing MTLBuffer data

An MTLBuffer contains bytes, which can be of any data type. Swift is a strongly typed language, meaning that Swift can only access the data in the buffer if you tell Swift what type the data is. You do this by binding the data to a type. In this case, the type is Instances.

When you bind the memory to the MTLBuffer contents, you’re given a pointer of type UnsafeMutablePointer<Instances>, which points to the first instance in the buffer.

Unsafe means that the buffer is unsafe to read from unless it’s correctly typed. You can bind to the data with any type you want, and Swift has no way of knowing whether the formatting is correct. You could bind instanceBuffer.contents() to Int.self, for example, which is incorrect in this case, but Swift will still allow you to access the buffer as if it is made up of Ints. Take care to bind the buffer to the correct type!

In Model.swift, create a new method to update a single transform:

func updateBuffer(instance: Int, transform: Transform) {
  transforms[instance] = transform
}

At the end of this method, add the following:

var pointer = 
    instanceBuffer.contents().bindMemory(to: Instances.self,
                                 capacity: transforms.count)

This binds instanceBuffer to a pointer, and formats the data so that Swift knows that the data is of type Instances, and what the specific number of Instances is. Using the pointer’s pointee property, you can access the data in the buffer directly.

Continue adding to updateBuffer(instance:transform:):

pointer = pointer.advanced(by: instance)
pointer.pointee.modelMatrix = transforms[instance].modelMatrix
pointer.pointee.normalMatrix = transforms[instance].normalMatrix

First, you advance the pointer to the correct instance. You then place the matrix information directly into instanceBuffer.

You also need to send instanceBuffer to the GPU. Add this to render(renderEncoder:uniforms:fragmentUniforms:) at the top of the method, just after setting uniforms:

renderEncoder.setVertexBuffer(instanceBuffer, offset: 0,
                   index: Int(BufferIndexInstances.rawValue))

The index was already set up for you in the starter project.

Here’s the hardware magic where you tell the GPU how many instances to render. In render(renderEncoder:submesh:), change the draw call to the following:

renderEncoder.drawIndexedPrimitives(type: .triangle,
        indexCount: mtkSubmesh.indexCount,
        indexType: mtkSubmesh.indexType,
        indexBuffer: mtkSubmesh.indexBuffer.buffer,
        indexBufferOffset: mtkSubmesh.indexBuffer.offset,
        instanceCount: instanceCount)

This includes the number of instances. instanceCount alerts the GPU that it has to call the vertex shader indexCount times for each instanceCount.

GPU instances

So you can access the instance array on the GPU side, you’ll change the vertex shader.

In Shaders.metal, add these two parameters to vertex_main:

constant Instances *instances [[buffer(BufferIndexInstances)]],
uint instanceID [[instance_id]]

The first parameter is the array of matrices, and the second — with the [[instance_id]] Metal Shading Language attribute — holds the index into the instances array.

Note: If you’re rendering a grid of grass plants, for example, you could also use [[instance_id]] to calculate the position in the grid instead of sending position data in a buffer.

Change the initialization of out to include the matrices for the particular instance:

Instances instance = instances[instanceID];
VertexOut out {
  .position = uniforms.projectionMatrix * uniforms.viewMatrix 
      * uniforms.modelMatrix * instance.modelMatrix * position,
  .worldPosition = (uniforms.modelMatrix * 
            instance.modelMatrix * position).xyz,
  .worldNormal = uniforms.normalMatrix * 
            instance.normalMatrix * normal.xyz,
  .worldTangent = uniforms.normalMatrix * 
            instance.normalMatrix * tangent.xyz,
  .worldBitangent = uniforms.normalMatrix * 
            instance.normalMatrix * bitangent.xyz,
  .uv = vertexIn.uv
};

You include the matrices for each instance in the position and normal calculations. With this separation of Model modelMatrix and instance modelMatrix, you can treat the Model as a group.

By changing the transform of the Model, it will affect all the instances, so that you can, for example, move an entire patch of grass or a forest at once. Each instance is then offset by its own modelMatrix.

You’ve completely set up an instancing system, and you can now use it in your scene. In GameScene.swift, change the for loop to:

let tree = Model(name: "tree.obj", instanceCount: instanceCount)
add(node: tree)
for i in 0..<instanceCount {
  var transform = Transform()
  transform.position.x = .random(in: -10..<10)
  transform.position.z = .random(in: -10..<10)
  let rotationY: Float = .random(in: -.pi..<Float.pi)
  transform.rotation = [0, rotationY, 0]
  tree.updateBuffer(instance: i, transform: transform)
}

Here, you create one Model for the group of trees and set transform data for each instance of a tree.

Build and run, and you’ll see almost the same result as your starter project. Your tree positions are randomized, so they won’t be in the same place.

Check the Debug navigator. Notice the memory footprint has gone way down. That’s because you’re now only using one texture and one set of vertices for all the trees. You should also be rendering at 60 frames per second again.

Using the Capture GPU frame tool, you can check that only one tree model, instead of twenty-five, is in the list of render encoder commands:

Instancing is a powerful and easy way of improving performance. Whenever you render more than one of a particular model, consider rendering them all as instances.

Morphing

You rendered multiple instances of the same high poly tree, but your scene will look boring if you render the same model with the same textures all over it. In this section, you’ll render a rock with one of three random textures and one of three random shapes, or morph targets. You’ll hold the vertex information for these three different shapes in a single buffer: an array of vertex buffers. You’ll also learn how to render vertices that you’ve read in using Model I/O without using the stage_in attribute.

Using homeomorphic models, you can choose different shapes for each model. Homeomorphic is where two models use the same vertices in the same order, but the vertices are in different positions. A famous example of this is “Spot the cow” by Keenan Crane:

Spot uses the same number and order of vertices as a sphere. The uv coordinates don’t change either.

Morph targets are commonly used for human figures. The 3D content supplier Daz 3D has a figure named Genesis, a generic human model, but you can purchase morph targets to change how the model looks.

You can turn the base model from a toon to a muscled super-hero just by switching the morph target mesh. You can also use morph targets for animating expressions from a frown to a smile, for example.

The only prerequisite for a morph target is that it has been built from a base mesh by rearranging the vertices, not by adding or removing any.

For this next section, you’ll use three differently shaped rocks that were modeled from a sphere. You’ll find a Blender file with the three rocks and a UV-mapped sphere in the resources for this chapter. You could experiment with making your own rock shapes from the sphere.

Note: The base shape, although it looks like a sphere, is actually a subdivided cube. It’s much easier to UV map a cube than a sphere, so before the artist subdivided the cube, she made the UV maps. Rocks are simple shapes, so small imperfections don’t show too much.

Because you’ll build up a fairly complex class, the starter project contains a Nature class, which is a cut-down version of the instanced Model that you just built. Open and examine Nature.swift. You’ll initialize the class with an array of texture names and an array of OBJ file names for the morph targets, but nothing here should be new to you.

Common.h contains a struct named NatureInstance that will describe each rock instance. Each rock instance has a morph target ID and a texture ID that will tell the GPU which morph target and texture to use. The class Nature has an instance buffer for each rock, and currently only renders a single base color texture and single shape for all the rocks.

An important thing to note is that to make things simpler, Nature assumes that all of the OBJ files have just one material submesh and that the vertex descriptor has only position, normal and uv data. This class does not hold tangent information for normal map usage.

Open ViewController.swift, and change the initialization of scene from GameScene to RocksScene:

let scene = RocksScene(sceneSize: metalView.bounds.size)

Open RocksScene.swift. It’s almost an exact copy of GameScene.swift, but it’s using Nature instead of Model. The scene lists three morph target OBJ files and three textures, but it currently only uses one.

Build and run the app to see instanced rocks:

All of the rocks are the same color and shape, but you’ll soon fix that!

Vertex descriptors and stage_in

So far, to render OBJ models, you’ve been using the [[stage_in]] attribute to describe vertex buffers in the vertex shader function.

The following image shows what Model I/O reads into the vertex MTLBuffer:

There are two float3s and a float2 for each vertex. The MTLVertexDescriptor describes this layout and assigns the position, normal and texture coordinate fields. The vertex descriptor stride describes the number of bytes between the start of one vertex and the start of the next.

When you use a parameter with a stage_in attribute, the vertex shader uses the layout from the vertex descriptor to read the buffer. These don’t have to be the same lengths as the data in the buffer, as the shader will automatically convert. Notice that, whereas the position is a float3 in the buffer, the vertex shader can map it to the Position attribute and read it in as a float4.

You’re going to be using one of three different vertex buffers for the three differently shaped rocks, and the stage_in conversion can only be used on one buffer. Because of this, you’ll read the raw buffer bytes in the vertex shader. You’ll have to match the actual format of the buffer data in the shader, not the struct format in VertexIn that the shader currently uses.

In Nature.metal, change VertexIn to:

struct VertexIn {
  float3 position;
  float3 normal;
  float2 uv;
};

If you’re familiar with reading MTLBuffers in shaders, you may be aware that this particular struct will cause you problems. Unfortunately, the format of this struct is a common mistake, so you’ll deliberately make this mistake to see what happens; you’ll correct it later.

Change the first parameter of vertex_nature to:

constant VertexIn *in [[buffer(0)]],
uint vertexID [[vertex_id]],

You’ve changed the [[stage_in]] parameter to a pointer to an array of VertexIns. The [[vertex_id]] attribute gives you the current vertex so that you can retrieve the position from the array.

Add this at the top of the vertex function:

VertexIn vertexIn = in[vertexID];

This gives you the current vertex.

You’ll get a compile error because position is now a float3. Change the assignment of position to:

float4 position = float4(vertexIn.position, 1);

Build and run, and you’ll get one of those interesting and generally frustrating renders where the vertices aren’t in their proper places:

This usually means that the struct format in the vertex shader doesn’t match the contents of the MTLBuffer, which you’ll fix now.

Packed floats

In Apple’s Metal Shading Language Specification, available at https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf, you can see the size and alignment of vector data types.

A float3 has an alignment of 16 bytes; each field of the struct must start at the alignment offset, and the compiler inserts padding where needed. This will change the size of the stride of VertexIn, so the vertex shader doesn’t read the data correctly. Your VertexIn struct currently contains float types, and you’ll need to change these to packed_float types.

The packed_float3 type has an alignment of only 4 bytes with a size of 12 bytes, so the stride will be consistent with the actual data in the buffer.

This is one problem; however, there is another problem: The vertex descriptor currently has a stride of 40 bytes, as you may have noticed printing out in the debug console. The actual stride in the buffer is 32 bytes: 12 + 12 + 8.

In Nature.swift, where you define the mdlVertexDescriptor property, after:

var offset = 0

Add this:

let packedFloat3Size = MemoryLayout<Float>.stride * 3

After both Position and Normal, change:

offset += MemoryLayout<float3>.stride

To:

offset += packedFloat3Size

Instead of using the size of a float3, which is 16 bytes, you’re now using the size of three Floats which is 12 bytes. You don’t need to change the offset for float2, as the size is 8 bytes, whether it’s packed or not.

In Nature.metal, change VertexIn to:

struct VertexIn {
  packed_float3 position;
  packed_float3 normal;
  float2 uv;
};

Build and run, and you’ll see in the debug console that the vertex descriptor stride is now 32 bytes, and your rocks will render correctly.

MTLBuffer array

You’re now able to render a single rock shape. In Swift, if you were going to have several rock shapes, you’d probably consider putting each shape into an array. However C++ does not allow variable length arrays, and you want to have the ability to add a variable number of morph targets to your Nature system.

You’ll append each morph target into a single MTLBuffer. Because the different rock models all have the same number of vertices, you can easily calculate the offset to the correct rock shape for the current instance. Each instance will have a random morph target ID from 0 to 2.

Note: You can track the contents of the buffer that you send to the GPU by using the Capture GPU Frame icon. On the Debug navigator, choose the Rocks command folder and view the contents of Vertex Buffer 0. You’ll see there are currently 1649 rows - one for the position, normal and uv of each vertex.

In Nature.swift, in Nature, create a new property to hold the number of vertices in each morph target:

var vertexCount: Int

This needs to be a var, not a constant, as you’ll be sending this to the GPU in a buffer later.

In init, locate where you load up the first morph target into a buffer. This is where you’ll concatenate all of the morph target buffers into the single MTLBuffer vertexBuffer. Change:

vertexBuffer = mesh.vertexBuffers[0].buffer

To:

let bufferLength = mesh.vertexBuffers[0].buffer.length
vertexBuffer = Renderer.device.makeBuffer(length: bufferLength 
  * morphTargetNames.count)!

You’ve already loaded the first OBJ and saved the submesh index data from it. All the morph targets will have the same index data, so you only need to save this once.

You can use the loaded buffer to find out the length of the morph target data for each element. Similar to the submesh index data, all of the morph targets will have the same number of vertices, so each vertex buffer will be the same length for each morph target.

Following on from the previous code, calculate the number of vertices:

let layout = mesh.vertexDescriptor.layouts[0] 
                 as! MDLVertexBufferLayout
vertexCount = bufferLength / layout.stride

To extract the vertex count from the mesh, you divide the length of the vertex buffer by the layout stride taken from the vertex descriptor.

The Blit Command Encoder

You’ll copy each morph target vertex buffer into the single MTLBuffer using a blit operation. You were introduced to a render command encoder in Chapter 1, “Introduction to Metal,” and then briefly to a compute encoder in Chapter 11, “Tessellation”. You are now learning about yet another type of encoder. A blit command encoder does a fast copy between resources such as textures and buffers. Just like with an MTLRenderCommandEncoder, you create an MTLBlitCommandEncoder and issue commands to the command buffer.

After the previous code, add this to create the command buffer and blit encoder:

let commandBuffer = Renderer.commandQueue.makeCommandBuffer()
let blitEncoder = commandBuffer?.makeBlitCommandEncoder()

Process all of the OBJ files and blit each vertex buffer into the single MTLBuffer. Continue by adding this code:

for i in 0..<morphTargetNames.count {
  guard let mesh = Nature.loadMesh(name: morphTargetNames[i]) else {
    fatalError("morph target not loaded")
  }
  let buffer = mesh.vertexBuffers[0].buffer
  blitEncoder?.copy(from: buffer, sourceOffset: 0,
                    to: vertexBuffer, 
                    destinationOffset: buffer.length * i,
                    size: buffer.length)
}

The offset of each morph target vertex buffer is the length of the original morph target buffer multiplied by the current morph target array index.

Finish off the blit operation just as you would a render pass:

blitEncoder?.endEncoding()
commandBuffer?.commit()

Note: If you build and run now, click the Capture GPU Frame icon and inspect the vertex buffer in the Rocks folder, you’ll see that the buffer now contains 4947 entries, which is 3 times 1649 vertices.

Random rock shape

In Nature.swift, change updateBuffer(instance:transform:) to receive a random number for both the morph target and the texture:

func updateBuffer(instance: Int, transform: Transform, 
                  textureID: Int, morphTargetID: Int) 

Add a check at the top of the method to make sure that the morph target or texture id is not out of range:

guard textureID < textureCount 
        && morphTargetID < morphTargetCount else {
  fatalError("ID is too high")
}

Add this at the end of the method to update the instance data with the index ids. You’ll randomly generate these in the scene:

pointer.pointee.textureID = UInt32(textureID)
pointer.pointee.morphTargetID = UInt32(morphTargetID)

Now, correct the compile errors. You call updateBuffer in two places. At the end of init(name:instanceCount:textureNames:morphTargetNames:), add the two extra parameters:

updateBuffer(instance: 0, transform: Transform(),
             textureID: 0, morphTargetID: 0)

In RocksScene.swift, in setupScene(), also add the two extra parameters:

let textureID = Int.random(in: 0..<textureNames.count)
let morphTargetID = Int.random(in: 0..<morphTargetNames.count)
rocks.updateBuffer(instance: i, transform: transform,
                   textureID: textureID, 
                   morphTargetID: morphTargetID)

As well as adding the two parameters, you create a random index to access the data in the texture and morph target arrays.

On the GPU side, in Nature.metal, you’re currently using vertexID to access the currently rendered vertex. To access the next morph target in the buffer, you’ll have to add the number of vertices for each morph target to vertexID.

In the below example, where each morph target has 1649 vertices, the first vertex for the second morph target should be vertex ID 1649.

In Nature.swift, in render(renderEncoder:uniforms:fragmentUniforms:), add the following code below // set vertex buffer:

renderEncoder.setVertexBytes(&vertexCount, 
                             length: MemoryLayout<Int>.stride, 
                             index: 1)

Here, you send the GPU the vertex count so that the vertex shader will be able to calculate the morph target offset.

Now, open Nature.metal to receive this buffer length variable. Add this extra parameter to vertex_nature:

constant int &vertexCount [[buffer(1)]],

Still in vertex_nature, change:

VertexIn vertexIn = in[vertexID];
NatureInstance instance = instances[instanceID];

To:

NatureInstance instance = instances[instanceID];
uint offset = instance.morphTargetID * vertexCount;
VertexIn vertexIn = in[vertexID + offset];

This calculates the offset of each morph using the morph target ID and the vertex count.

Build and run, and you now get randomly positioned and shaped rocks:

Texture arrays

Accessing a random texture is slightly easier than a random morph target since you can load the textures into an MTLTexture with a textureType of type2DArray. All of the textures are held in one MTLTexture, with each element of the array being called a slice.

You need to create a method to load up the textures into a temporary array of MTLTextures. This is a useful method, so you’ll add it to the Texturable protocol so that you can use it with any class that conforms to Texturable. In Texturable.swift, add this:

static func loadTextureArray(textureNames: [String]) -> MTLTexture? {
}

Add this to the new method:

var textures: [MTLTexture] = []
for textureName in textureNames {
  do {
    if let texture = 
         try Nature.loadTexture(imageName: textureName) {
      textures.append(texture)
    }
  }
  catch {
    fatalError(error.localizedDescription)
  }
}
guard textures.count > 0 else { return nil }

Here, you load up an array of textures.

Following on from that code, create a new texture array:

let descriptor = MTLTextureDescriptor()
descriptor.textureType = .type2DArray
descriptor.pixelFormat = textures[0].pixelFormat
descriptor.width = textures[0].width
descriptor.height = textures[0].height
descriptor.arrayLength = textures.count
let arrayTexture = 
     Renderer.device.makeTexture(descriptor: descriptor)!

You use the first texture in the array for the pixel format, width and height. textureType indicates that this texture will have slices. These are equivalent to array elements: each texture will go into a slice.

Continue with this code to blit the textures into arrayTexture:

let commandBuffer = Renderer.commandQueue.makeCommandBuffer()!
let blitEncoder = commandBuffer.makeBlitCommandEncoder()!
let origin = MTLOrigin(x: 0, y: 0, z: 0)
let size = MTLSize(width: arrayTexture.width, 
                   height: arrayTexture.height, depth: 1)
for (index, texture) in textures.enumerated() {
  blitEncoder.copy(from: texture, 
                   sourceSlice: 0, sourceLevel: 0,
                   sourceOrigin: origin, sourceSize: size,
                   to: arrayTexture, destinationSlice: index, 
                   destinationLevel: 0, 
                   destinationOrigin: origin)
}
blitEncoder.endEncoding()
commandBuffer.commit()
return arrayTexture

You create the command buffer and blit command encoder as before. For each texture, you copy it to the correct slice of arrayTexture. The copy command has a large number of parameters, which gives you total control over the copy. You can copy a corner of one texture to another corner of a second texture, for example. In this case, you want to copy the whole texture, so you use a zero origin with the full texture size.

sourceLevel is the mipmap level. Previously, you have filled mipmaps using the asset catalog, but this is how you can set the mipmap texture for each level in code.

This method now takes in a list of strings, loads up single textures, and then combines them all into one 2D texture array and returns this texture.

Back in Nature.swift, in init(name:instanceCount:textureNames:morphTargetNames:), locate // load the texture and replace:

do {
  baseColorTexture = 
       try Nature.loadTexture(imageName: textureNames[0])
} catch {
  fatalError(error.localizedDescription)
}

With:

baseColorTexture = 
   Nature.loadTextureArray(textureNames: textureNames)

You’re now loading all of the three rock textures into baseColorTexture.

In Nature.metal, in fragment_nature, replace the baseColorTexture parameter with:

texture2d_array<float> baseColorTexture [[texture(0)]]

The fragment function is now set up to receive the texture array. The compile error in the fragment function means that you’re not indicating which slice of the texture array to use.

Previously, you were using a single 2D texture which doesn’t have slices, but now when you sample the texture array, you have to specify the slice. You’ll pass the texture ID from the vertex shader to the fragment shader.

Add this variable to VertexOut:

uint textureID [[flat]];

The [[flat]] Metal Shading Language attribute prevents the rasterizer from interpolating the value between vertices so that the fragment shader will receive the value exactly as the vertex shader set it.

At the end of vertex_nature, before the return, load up the variable with the instance’s random texture:

out.textureID = instance.textureID;

In fragment_nature replace the baseColor assignment with:

float4 baseColor = baseColorTexture.sample(s, in.uv, 
                                           in.textureID);

You give textureID as the index into the texture array to get the base color. Your project should now compile.

Build and run to see your new rock system. Each of your rocks has one of three shapes and one of three textures.

Procedural systems

With the techniques you’ve learned so far, you have the power to load random objects into your scene and make each scene unique. However, you’re currently limited to having a single mesh size. In games such as No Man’s Sky, each planet and all of the animals that you meet on the planets are procedurally generated using many different meshes.

Procedural generation essentially means that you send a system some random parameters, such as integers or even noise, and the system generates a new and unique model or game level.

In this section, you’ll use a similar technique to procedural animal generation in No Man’s Sky and create a house system consisting of a row of houses.

Each house will have a random number of floors, and each floor will use a random model. However, even though the system might use a floor model or a roof model several times in different houses, for better efficiency, there will only be one instance of the model geometry.

Rules

The secret to procedural systems is having a set of rules. For example, if you’re procedurally generating animals out of multiple parts, you don’t want to be attaching heads to leg joints; this isn’t The Island of Doctor Moreau after all.

So, in this case, you’d have a rule that heads only attach to head joints.

Your houses will consist of a first (ground) floor, middle floors and an optional roof. The rules for the house system are:

• First floors are a ground model.
• Roofs are only on the top floor.
• Maximum number of floors.
• Minimum and maximum gap between the houses.
• Maximum number of houses.

These are the models for each floor type, created by Kenney at https://kenney.nl:

The Houses scene

Add HousesScene.swift, Houses.swift and Houses.metal to the targets by Cmd-Selecting all three files, and on the File inspector, check Target Membership for both macOS and iOS targets.

Note: These files were not part of the original targets because Houses has a reference to house.instanceBuffer, which you only added at the beginning of this chapter.

Open ViewController.swift, and change the initialization of scene from RocksScene to HousesScene:

let scene = HousesScene(sceneSize: metalView.bounds.size)

Look at Houses.swift. Houses is a subclass of Node and contains an array of Models. render(renderEncoder:uniforms:fragmentUniforms:) renders this array with currently only one Model.

In Houses.metal, the vertex and fragment shaders are very simple. vertex_house extracts the current instance from the Model’s instance array just as you set it up in the earlier part of this chapter. fragment_house uses the material colors from the Model’s submesh with just a single sunlight for lighting.

HousesScene.swift simply loads the Houses system. Build and run the app to see the lone house:

You’ll now create rules to add houses to your Houses system.

Determine the rules

In Houses.swift, add an enum to Houses to lay down the rules with constants:

enum Rule {
  // gap between houses
  static let minGap: Float = 0.3
  static let maxGap: Float = 1.0

  // number of OBJ files for each type
  static let numberOfGroundFloors = 4
  static let numberOfUpperFloors = 4
  static let numberOfRoofs = 2

  // maximum houses
  static let maxHouses: Int = 5
    
  // maximum number of floors in a single house
  static let maxFloors: Int = 6
}

The number of OBJ files reflects the models held in the group Models ▸ Houses. These are carefully named for easy loading.

Add the following instance variable to Houses:

var floorsRoof: Set<Int> = []

This is a set of integers holding the indices of the roof OBJs in houses. If the system selects a floor with an element index in this set, then it is a roof, and no further building of that house can take place.

Load the OBJ files

Create a new method to read all the house OBJ files and place them into a Swift array.

func loadOBJs() -> [Model] {
  var houses: [Model] = []
  func loadHouse(name: String) {
    houses.append(Model(name: name + ".obj",
                        vertexFunctionName: "vertex_house",
                        fragmentFunctionName: "fragment_house"))
  }
  for i in 1...Rule.numberOfGroundFloors {
    loadHouse(name: String(format: "houseGround%d", i))
  }
  for i in 1...Rule.numberOfUpperFloors {
    loadHouse(name: String(format: "houseFloor%d", i))
  }
  for i in 1...Rule.numberOfRoofs {
    loadHouse(name: String(format: "houseRoof%d", i))
    floorsRoof.insert(houses.count-1)
  }
  return houses
}

Here, you use the constant values to load each of the OBJ types into the array of Models. If you were to print out the index and names of the Models in the array, this is what you’d see:

For the two roofs, you insert the indices into floorsRoof so that you can determine later whether a particular OBJ is a roof.

In init(), replace:

houses.append(Model(name: "houseGround1.obj",
                    vertexFunctionName: "vertex_house",
                    fragmentFunctionName: "fragment_house"))

With:

houses = loadOBJs()

All of the house OBJ files are now loaded and ready for use. You’ll generate arrays of integers to index to the correct house.

Create the first (ground) floors

Create two new properties in Houses:

var remainingHouses: Set<Int> = []
var housefloors: [[Int]] = []

When you create the first (ground) floors, you’ll add each house to the remainingHouses set. When the house is complete, you’ll remove it from the set. When the set is empty, then your procedural job is done, and all of the houses will be complete.

housefloors is a two-dimensional array. The first dimension is for each house and the second for the floors within each house.

Add this code at the end of init():

let numberOfHouses = 5
for _ in 0..<numberOfHouses {
  let random = Int.random(in: 0..<Rule.numberOfGroundFloors)
  housefloors.append([random])
  let lastIndex = housefloors.count - 1
  remainingHouses.insert(lastIndex)
}

You set numberOfHouses to 5, but you could, of course, randomize this number. For each house choose a random number between 0 and 3 (or the number of ground floor OBJ files you have). This number will index into houses. Remember that you loaded the ground floor OBJ files first, so they are elements 0 to 3. You insert the floor into remainingHouses to keep track of whether the house is complete.

Iterate through the houses and add floors while remainingHouses still contains elements. Add this after the previous code:

while remainingHouses.count > 0 {
  for i in 0..<housefloors.count {
    // 1
    if remainingHouses.contains(i) {
      let offset = Rule.numberOfGroundFloors
      let upperBound = 
          offset + Rule.numberOfUpperFloors + Rule.numberOfRoofs
      let random = Int.random(in: offset..<upperBound)
      housefloors[i].append(random)

      // 2
      if floorsRoof.contains(random) ||
        housefloors[i].count >= Rule.maxFloors ||
        Int.random(in: 0...3) == 0 {
        // 3
        remainingHouses.remove(i)
      }
    }
  }
}
// 4
print(housefloors)

Going through this code:

1. If the house index is still in remainingHouses, then it doesn’t yet have a roof, so is not yet complete. Add a random index number that will index into houses for a new floor. This index number has to be greater than the first few elements that contain only the first (ground) floors.
2. Finish up the house if the floor is a roof, or if the house has reached the maximum number of floors, or, for more randomness, a 1 in 4 chance.
3. Remove the house from remainingHouses if it is complete.
4. Temporarily print out housefloors to examine its contents.

Build and run the app to examine your progress.

This is a render of five first floors all in one place.

You’ll see something like this in the debug console (yours will be different as it’s all random):

This is the contents of the multidimensional array housefloors. Using these indices, you’ll be able to access the correct model in houses for each entry. In this example (which will change every time, as it’s random), there are 5 houses in the array which contain several floors each. House 0 has a first floor index of 2, meaning use the Model houseGround3. House 0’s upper floor is 5, which uses houseFloor2. House 2 has four floors, and house 4 has 3 floors with houseRoof2 on top.

You’ll now create a final one-dimensional array: floors, which will contain a list of all the floors. In Houses, create a new property for the array:

struct Floor {
  var houseIndex: Int = 0
  var transform = Transform()
}
var floors: [Floor] = []

Each floor indexes to the correct Model in houses. You’ll also calculate the correct position for each floor.

Replace the print statement at the end of init() with:

var width: Float = 0
var height: Float = 0
var depth: Float = 0
for house in housefloors {
  var houseHeight: Float = 0
  
  // add inner for loop here to process all the floors 
  
  let house = houses[house[0]]
  width += house.size.x
  height = max(houseHeight, height)
  depth = max(house.size.z, depth)
  boundingBox.maxBounds = [width, height, depth]
  width += Float.random(in: Rule.minGap...Rule.maxGap)
}

You set up variables for the bounding box of the entire house system and process each house. houseHeight will keep the height of the current house so that you know where to locate the next floor. After processing all of the floors, you’ll update the width and depth from each first (ground) floor, and the height from all the floors. You also add to the total width of the system a random gap between each house.

Add the inner for loop — where you commented in the previous code — to process each floor of the current house:

for floor in house {
  var transform = Transform()
  transform.position.x = width
  transform.position.y = houseHeight
  floors.append(Floor(houseIndex: floor, transform: transform))
  houseHeight += houses[floor].size.y
}

You assign a transform and the house index to each floor and update houseHeight for the next floor.

Now, you’ll update the render loop. In render(renderEncoder:uniforms:fragmentUniforms:), replace:

for house in houses {

With:

for floor in floors {
  let house = houses[floor.houseIndex]

You’ll render each floor instance using the floor’s transform. Replace uniforms.modelMatrix and uniforms.normalMatrix assignments with:

uniforms.modelMatrix = modelMatrix * floor.transform.modelMatrix
uniforms.normalMatrix = 
  float3x3(normalFrom4x4: modelMatrix * floor.transform.modelMatrix)

Build and run. You now have a procedural house system:

The arrangement of houses and floors will be different every time you run the app. Notice that a roof is always the top story, the first floor OBJ models are always on the ground, and the height of the houses doesn’t exceed the maximum number of floors.

This is a simplified procedural system, but you can explore procgen (procedural generation) much further with the links in references.markdown for this chapter.

Challenge

Using the Nature system that you set up for the rocks, you can easily create a field full of blades of grass.

Your challenge is to use the grass OBJ files in the Models ▸ Grass group and the grass textures in Textures.xcassets. The grass OBJ files all have the same number and order of vertices, so you can use them as morph targets.

This will be a similar exercise to creating the rocks system earlier. You’ll create a new scene and a Nature system with the textures and morph targets.

The project in the challenge folder shows lush grass as far as the eye can see, but still running at 60fps on a 2015 iMac.

For the record, it doesn’t perform as well on an iPhone 6s, or even a 2018 iPad Pro.

The app achieves this with a bit of trickery. GrassScene creates several grass systems with reduced amounts of grass the further you go back into the scene. By tracking where the camera goes, you can place grass with higher level of detail closer to the camera, and sparse low detail grass further away.

Where to go from here?

Procedural systems are fun to create. Later in this book, you’ll encounter Perlin noise, and if you use this noise for your randomness, you can generate infinite terrains, or even dynamic wind or animation.

You’re currently generating your systems on the CPU, but in a few chapters, you’ll learn how to use compute kernels, and you’ll be able to create more massive systems in parallel. A particle system is an excellent example of a procedural system, and in just a few short chapters you’ll also master particles.

One of the sections in references.markdown is about Lindenmayer systems. Using L-systems for creating plants is interesting and approachable. Now that you know how to manipulate vertices more comfortably, you might want to try your hand at generating 3D plants.

The next chapter will take you further into lighting techniques. You’ll find out how to render to a texture without immediately putting it onto the screen.