Chapters

Hide chapters

Metal by Tutorials

Third Edition · macOS 12 · iOS 15 · Swift 5.5 · Xcode 13

Section I: Beginning Metal

Section 1: 10 chapters
Show chapters Hide chapters

Section II: Intermediate Metal

Section 2: 8 chapters
Show chapters Hide chapters

Section III: Advanced Metal

Section 3: 8 chapters
Show chapters Hide chapters

24. Character Animation
Written by Marius Horga & Caroline Begbie

In the previous chapter, you learned how to move objects over time using keyframes. Imagine how long it would take to create a walk cycle for a human figure by typing out keyframes. This is the reason why you generally use a 3D app, like Blender or Maya, to create your models and animations. You then export those animations to your game or rendering engine of choice.

Skeletal Animation

Rarely, will you move the entire character when you’re animating it. Instead, you’ll move parts of the mesh, such as an arm, rather than the whole thing. Using a 3D app, the rigger creates a skeleton — in Blender, this is known as an armature. She assigns bones and other controls to parts of the mesh so that the animator can transform the bones and record the movement into an animation clip.

You’ll use Blender 3.0 to examine an animated model and understand the principles and concepts behind 3D animation.

Note: If you haven’t installed Blender 3.0 yet, it’s free, and you can download it from https://www.blender.org.

➤ Go to the resources folder for this chapter, and open skeleton.blend in Blender 3.0.

You’ll see something like this:

The skeleton model in Blender 3.0
The skeleton model in Blender 3.0

➤ Before examining the bones further, left-click on the skeleton’s head to select the skeleton object. Press the Tab key to switch to Edit Mode:

The skeleton mesh
The skeleton mesh

Here, you can see all of the skeleton’s vertices. This is the original model, which you can export as a static .obj file. The skeleton has its arms stretched out in what’s known as the bind pose. This is a standard pose for figures as it makes it easier to add animation bones to the figure.

➤ Press the Tab key to go back to Object Mode.

To animate the figure, you need to control groups of vertices. For example, to rotate the head, you’d rotate all of the head’s vertices.

Rigging a figure in Blender means creating an armature with a hierarchy of joints. Joints and bones are generally used synonymously, but a bone is simply a visual cue to see which joint affects which vertices.

The general process of creating a figure for animation goes like this:

  1. Create the model.
  2. Create an armature with a hierarchy of joints.
  3. Apply the armature to the model with automatic weights.
  4. Use weight painting to change which vertices go with each joint.

Just as in the song Dem Bones, “The toe bone’s connected to the foot bone,” this is how a typical rigged figure’s joint hierarchy might look:

A joint hierarchy
A joint hierarchy

In character animation, it’s (usually) all about rotation — your bones don’t translate unless you have some kind of disjointing skeleton. With this hierarchy of joints, when you rotate one joint, all the child joints follow. Try bending your elbow without moving your wrist. Because your wrist is lower in the hierarchy, even though you haven’t actively changed the wrist’s position and rotation, it still follows the movement of your elbow. This type of movement is known as forward kinematics and is what you’ll be using in this chapter. It’s a fancy name for making all child joints follow.

Note: Inverse kinematics allows the animator to make actions, such as walk cycles, more easily. Place your hand on a table or in a fixed position. Now, rotate your elbow and shoulder joint with your hand fixed. The hierarchical chain no longer moves your hand as in forward kinematics. As opposed to forward kinematics, the mathematics of inverse kinematics is quite complicated.

The skeleton model that you’re looking at in Blender has a limited rig for simplicity. It has four bones: the body, left upper arm, left forearm and left hand. Each of these joints controls a group of vertices.

Weight Painting in Blender

➤ Left-click the skeleton’s head.

➤ At the bottom of the Blender window, click on the drop-down that currently reads Object Mode, and change it to Weight Paint.

Weight Paint Dropdown
Weight Paint Dropdown

The weight painting editor shows you how each bone affects the vertices. Currently the body vertex group is selected, which is attached to the body bone. All vertices affected by the body bone are shown in red. The arm mesh has its own bones and are shown in blue.

The skeleton's body bone weights
The skeleton's body bone weights

The process of weight painting and binding each bone to the vertices is called skinning. Unlike human arms, the skeleton’s arm bones here have space between them, so, in this case, all mesh is assigned to only one bone. However, if you’re rigging a human arm, you would typically weight the vertices to multiple bones.

Here’s a typically weighted arm with the forearm selected to show gradual blending of weights at the elbow and the wrist.

A weighted arm
A weighted arm

This is a side-by-side example of blended and non-blended weights at the elbow joint with the forearm selected:

Blended and non-blended weights
Blended and non-blended weights

The blue area indicates no weighting, whereas the red area indicates total weighting. You can see in the right image, the forearm vertices dig uncomfortably into the upper arm vertices, but in the left image, the vertices move more evenly around the elbow joint.

At the elbow, where the vertices are green, the vertex weighting would be 50% to the upper arm, and 50% to the forearm. When the forearm rotates, the green vertices will rotate at 50% of the forearm’s rotation. By blending the weights gradually, you can achieve an even deformation of vertices over the joint.

Animation in Blender

➤ Select the drop-down at the bottom of the window that currently reads Weight Paint, and go back into Object Mode.

➤ Press the space bar to start an animation.

Your skeleton will now get friendly and wave at you. This wave animation is a 60 frame looping animation clip.

➤ At the top of Blender’s window, click the Animation tab to show the Animation workspace.

You can now see the animation keys at the top left in the Dope Sheet. The dope sheet is a summary of the keyframes in the scene. It lists the joints on the left, and each circle in the dope sheet means there’s a keyframe at that frame.

The dope sheet
The dope sheet

Note: Although animated transformations are generally rotations, the keyframe can be a translation or a scale. You can click the arrow on the left of the joint name to see the specific channel the key is set on.

➤ Press space bar to stop the animation if it’s still going. Scrub through the animation by dragging the playhead at the top of the pane (the blue rectangle with 26 in it in the above image). Pause the playhead at each set of keyframes. Notice the position of the arm. At each keyframe, the arm is in an extreme position. Blender interpolates all the frames between the extremes.

Now that you’ve had a whirlwind tour of how to create a rigged figure and animate it in Blender, you’ll move on to learning how to render it in your rendering engine.

Note: You’ve only skimmed the surface of creating animated models. If you’re interested in creating your own, you’ll find some additional resources in references.markdown.

The Starter App

➤ In Xcode, open the starter project. The scene is a skeleton character rendered with the forward renderer, using the PBR shader, without shadows.

➤ Build and run the app.

You see a skeleton model, which was exported from Blender in .fbx format and converted to .usda format. You can open skeletonWave.usda, in the Models / Skeleton group, with a text editor to see what’s inside. The file contains an animation, but the skeleton won’t wave until you’ve implemented the chapter code.

Implementing Skeletal Animation

Importing a skeletal animation into your app is a bit more difficult than importing a simple .obj file or a USDZ file with transform animation, because you have to deal with the joint hierarchy and joint weighting. You’ll read in the data from the USD file and restructure it to fit your rendering code. This is how the objects will fit together in your app:

The code architecture
The code architecture

Each model could have a number of animation clips, such as walk and wave. Each animation clip has a list of animations for a particular joint. Each mesh can have a skeleton that holds a list of joint names, and using the joint name as a key, you’ll be able to access the correct animation for that joint.

TransformComponent that you created in the previous chapter still remains for any transform animation. The starter project has several extra helper files in the Animation group to aid with importing the skeleton and animations.

  • Skeleton.swift: To create the Mesh‘s skeleton, you’ll use the MDLAnimationBindComponent from the mdlMesh, if there is one. Skeleton holds the joint names in an array, and also the joints’ parent indices in another array.

  • AnimationComponent.swift: To load the animations for the asset, load(animation:) iterates through the joints and loads up Animations for each joint. These are all combined into an AnimationClip.

  • AnimationClip.swift: AnimationClip is a collection of Animations. Model will hold a dictionary of these AnimationClips keyed on the animation’s name.

  • Animation.swift: You created Animation in the previous chapter for rotations and translations. The loading code now includes scale transformations.

➤ In the Geometry group, open Model.swift, and add a new property to Model to hold the model’s animation clips:

let animations: [String: AnimationClip]

➤ At the end of init(name:), add the following to load the animations:

// animations
let assetAnimations = asset.animations.objects.compactMap {
  $0 as? MDLPackedJointAnimation
}
let animations
  = Dictionary(uniqueKeysWithValues: assetAnimations.map {
  ($0.name, AnimationComponent.load(animation: $0))
  })
self.animations = animations

Here, you extract all the MDLPackedJointAnimation objects from the asset and load them using the provided loading code. The result will be a dictionary of animation clips keyed by animation name, held in animations.

➤ After the previous code, add this:

animations.forEach {
  print("Animation:", $0.key)
}

A list of the available animations will print out in the debug console so that you can use the name later.

➤ Build and run the app.

The debug console
The debug console

The skeleton is still in his bind pose, but you’ll see the message in the debug console to show that the animation has loaded.

➤ Remove the previous print closure.

You’ve just loaded a set of animations listing translation, rotation and scaling on all joints. Now to set up the meshes’ skeletons.

➤ In the Geometry group, open Mesh.swift, and add a new property to Mesh:

let skeleton: Skeleton?

➤ At the top of init(mdlMesh:mtkMesh:), initialize the skeleton using mdlMesh’s MDLAnimationBindComponent:

let skeleton =
  Skeleton(animationBindComponent:
    (mdlMesh.componentConforming(to: MDLComponent.self)
    as? MDLAnimationBindComponent))
self.skeleton = skeleton

You’ve now loaded up a skeleton with joints. When rendering the skeleton, you’ll be able to access the model’s current animation and apply it to the mesh’s skeleton joints.

➤ Add a debug print statement after the previous code to show the joints:

skeleton?.jointPaths.forEach {
  print($0)
}

➤ Build and run the app, and in the debug console, you’ll see a listing of the skeleton model’s four joints:

These joints correspond to the bones that that you previously saw in Blender.

➤ Remove the forEach print debug closure.

Loading the Animation

To update the skeleton’s pose every frame, you’ll create a method that takes the animation clip and iterates through the joints to update each joint’s position for the frame.

First, you’ll create a method on AnimationClip that gets the pose for a joint at a particular time. This will use the interpolation methods that you’ve already created in Animation. The main difference is that these poses will be in joint space. For example, in this animation, the forearm swings by 45º. All the other joints’ rotations and translations will be 0.

➤ In the Animations group, open AnimationClip.swift, and add a new method to AnimationClip:

func getPose(at time: Float, jointPath: String) -> float4x4? {
  guard let jointAnimation = jointAnimation[jointPath],
    let jointAnimation = jointAnimation
    else { return nil }
  let rotation =
    jointAnimation.getRotation(at: time) ?? simd_quatf()
  let translation =
    jointAnimation.getTranslation(at: time) ?? float3(repeating: 0)
  let scale =
    jointAnimation.getScale(at: time) ?? float3(repeating: 0)
  let pose = float4x4(translation: translation) * float4x4(rotation)
    * float4x4(scaling: scale)
  return pose
}

Here, you retrieve the interpolated transformation, made up of rotation, translation and scale, for a given joint at a given time. You then create a transformation matrix for the joint and return it as the pose. This is much the same code as you used earlier for retrieving a transform at a particular time.

The Joint Matrix Palette

You’re now able to get the pose of a joint. However, each vertex is weighted to up to four joints. You saw this in the earlier elbow example, where some vertices belonging to the lower arm joint would get 50% of the upper arm joint’s rotation. Soon, you’ll change the default vertex descriptor to load vertex buffers with four joints and four weights for each vertex. This set of joints and weights is known as the joint matrix palette.

The vertex function will sample from each of these joint matrices and, using the weights, will apply the transformation matrix to each vertex. The following image shows a vertex that is assigned 50% to joint 2 and 50% to joint 3. The other two joint indices are unused.

After multiplying the vertex by the projection, view and model matrices, the vertex function will multiply the vertex by a weighting of each of the joint transforms. Using the example in the image above, the weighting will be 50% of Bone 2’s joint matrix and 50% of Bone 3’s joint matrix.

➤ Open Skeleton.swift, and create a new method in Skeleton:

func updatePose(
  animationClip: AnimationClip?,
  at time: Float
) {
}

This method, when you’ve completed it, will iterate through the joints and fill a joint matrix palette buffer with the current pose for each joint.

You’ll send this buffer to the GPU’s vertex function, so that each vertex will be able to access all of the joint matrices that it is weighted to.

➤ Add the following to updatePose(animationClip:at:):

guard let paletteBuffer = jointMatrixPaletteBuffer
  else { return }
var palettePointer = paletteBuffer.contents().bindMemory(
  to: float4x4.self,
  capacity: jointPaths.count)
guard let animationClip = animationClip else {
  palettePointer.initialize(
    repeating: .identity,
    count: jointPaths.count)
  return
}

You initialize the buffer pointer and bind the contents of the buffer to an array of 4x4 matrices. If an animation clip is not loaded, initialize the buffer with identity matrices and return without updating the joints.

➤ Now, to iterate through the skeleton’s joints, add the following:

var poses =
  [float4x4](repeatElement(.identity, count: jointPaths.count))
for (jointIndex, jointPath) in jointPaths.enumerated() {
  // 1
  let pose = animationClip.getPose(
    at: time * animationClip.speed,
    jointPath: jointPath) ?? restTransforms[jointIndex]
  // 2
  let parentPose: float4x4
  if let parentIndex = parentIndices[jointIndex] {
    parentPose = poses[parentIndex]
  } else {
    parentPose = .identity
  }
  poses[jointIndex] = parentPose * pose
}

Going through this code:

  1. You retrieve the transformation pose, if there is one, for the joint for this frame. restTransform gives a default pose for the joint.
  2. The poses array is in flattened hierarchical order, so you can be sure that the parent of any joint has already had its pose updated. You retrieve the current joint’s parent pose, concatenate the pose with the current joint’s pose and save it in the poses array.

The Inverse Bind Matrix

➤ Examine the properties held on Skeleton.

When you first instantiate the skeleton, you load these properties from the data loaded by Model I/O.

One of the properties on Skeleton is bindTransforms. This is an array of matrices, one element for each joint, that transforms vertices into the local joint space.

The bind pose
The bind pose

When all the joint transforms are set to identity, that’s when you’ll get the bind pose. If you apply the inverse bind matrix to each joint, it will move to the origin. The following image shows the skeleton’s joints all multiplied by the inverse bind transform matrix.

The inverse bind matrix applied to all joints
The inverse bind matrix applied to all joints

Why is this useful? Each joint should rotate around its base. To rotate an object around a particular point, you first need to translate the point to the origin, then do the rotation, then translate back again. (Review Chapter 5, “3D Transformations” if you’re unsure of this rotation sequence.)

In the following image, the vertex is located at (4, 1) and bound 100% to Bone 2. With rotations 10º on Bone 1 and 40º on Bone 2, the vertex should end up at about (3.2, 0) as shown in the right-hand image.

When you currently render your vertices, you multiply each vertex position by the projection, view and model matrices in the vertex function. To get this example vertex in the correct position for the right-hand image, you’ll also have to multiply the vertex position by both Bone 1’s transform and Bone 2’s transform.

➤ Add the following code to the end of the previous for loop, after setting poses[jointIndex]:

palettePointer.pointee =
  poses[jointIndex] * bindTransforms[jointIndex].inverse
palettePointer = palettePointer.advanced(by: 1)

You translate the pose back to the origin with the the inverse bind transform, and combine it with the current pose into the final joint palette matrix.

With all of the frame data set up, you can now set the pose.

➤ Open Model.swift, and in update(deltaTime:), replace the existing animation code:

for i in 0..<meshes.count {
  meshes[i].transform?.getCurrentTransform(at: currentTime)
}

➤ With:

for i in 0..<meshes.count {
  var mesh = meshes[i]
  if let animationClip = animations.first?.value {
    mesh.skeleton?.updatePose(
      animationClip: animationClip,
      at: currentTime)
  }
  mesh.transform?.getCurrentTransform(at: currentTime)
  meshes[i] = mesh
}

You take the first animation in the list of animations and, if there is an animation, update the pose for the current time. Update the current transform as well, as you were doing before.

Note: Currently USDZ files only hold one animation. With Blender not yet exporting skeletal animation as of version 3.0, it’s difficult to get multiple animations into one USD file without hand editing a .usda file. Apple suggests, with some judicious coding, you could load multiple USDZ files, one with the geometry and skeleton, and others with solely the animation. As time goes on, and Blender improves, there will likely be better alternatives.

All of the meshes are now in position and ready to render.

➤ In render(encoder:uniforms:params:), at the top of the loop for mesh in meshes, add this:

if let paletteBuffer = mesh.skeleton?.jointMatrixPaletteBuffer {
  encoder.setVertexBuffer(
    paletteBuffer,
    offset: 0,
    index: JointBuffer.index)
}

You set up the joint matrix palette buffer so that the GPU can read it. The vertex shader function will take in this palette and apply the matrices to the vertices.

➤ In the Shaders group, open Vertex.h, and add two attributes to VertexIn:

ushort4 joints [[attribute(Joints)]];
float4 weights [[attribute(Weights)]];

The attribute constants Joints and Weights were set up for you in the starter project in Common.h in Attributes.

To match VertexIn, you’ll need to update Model’s vertex descriptor.

➤ Open VertexDescriptor.swift, and uncomment the two extra vertex attributes for joints and weights. Model I/O, when loading the file, will now load joint index and joint weight information to the model’s vertex buffers.

Your vertex buffer layout will now look like this:

Vertex Buffer 0
Vertex Buffer 0

➤ Build and run the app to ensure that everything still works:

No obvious changes yet
No obvious changes yet

Updating the Vertex Shader

➤ In the Shaders group, open Shaders.metal, and add a new parameter to vertex_main:

constant float4x4 *jointMatrices [[buffer(JointBuffer)]]

➤ At the top of vertex_main, replace the float4 position assignment with:

bool hasSkeleton = true;
float4 position = in.position;
float4 normal = float4(in.normal, 0);

Some models will have skeletons and joint matrices, but others, such as the ground plane won’t. You’ll have to set up a conditional to determine which type of model you are rendering. For the moment you assume that all models have a joint matrix palette.

➤ After the code you just added, add the following code to combine the joint matrix and weight data with the position and normal:

if (hasSkeleton) {
  float4 weights = in.weights;
  ushort4 joints = in.joints;
  position =
      weights.x * (jointMatrices[joints.x] * position) +
      weights.y * (jointMatrices[joints.y] * position) +
      weights.z * (jointMatrices[joints.z] * position) +
      weights.w * (jointMatrices[joints.w] * position);
  normal =
      weights.x * (jointMatrices[joints.x] * normal) +
      weights.y * (jointMatrices[joints.y] * normal) +
      weights.z * (jointMatrices[joints.z] * normal) +
      weights.w * (jointMatrices[joints.w] * normal);
}

You take each joint to which the vertex is bound, calculate the final position and normal, and then take the weighted part of that calculation.

If the function constant hasSkeleton is false, you’ll just use the original position and normal.

➤ Change the VertexOut out assignment to:

VertexOut out {
  .position = uniforms.projectionMatrix * uniforms.viewMatrix
                * uniforms.modelMatrix * position,
  .uv = in.uv,
  .color = in.color,
  .worldPosition = (uniforms.modelMatrix * position).xyz,
  .worldNormal = uniforms.normalMatrix * normal.xyz,
  .worldTangent = 0,
  .worldBitangent = 0,
  .shadowPosition =
    uniforms.shadowProjectionMatrix * uniforms.shadowViewMatrix
    * uniforms.modelMatrix * position
};

You use position and normal instead of in.position and in.normal. You should also pre-multiply the tangent and bitangent properties too, but for brevity, you set worldTangent and worldBitangent properties to zero.

➤ Build and run the app.

You’ll get a run time error: failed assertion Draw Errors Validation Vertex Function(vertex_main): missing buffer binding at index 15 for jointMatrices[0].

This is because you’re rendering the ground, which doesn’t have any joint matrices.

➤ Open GameScene.swift, and in init(), change models = [ground, skeleton] to:

models = [skeleton]

➤ Build and run the app, and your animated skeleton will now wave at you.

Skeleton waving
Skeleton waving

Of course you’ll want to render the ground, so you’ll need to tell the GPU pipeline that it has to conditionally prepare two different vertex functions, depending on whether the mesh has a skeleton or not.

➤ Undo the previous change to models.

Function Specialization

Over the years there has been much discussion about how to render conditionally. For example, in your fragment shaders when rendering textures, you use the Metal Shading Language function is_null_texture(textureName) to determine whether to use the value from the material or a texture.

To test whether or not you have a joint matrix, you don’t have a convenient MSL function.

Should you create separate short fragment shaders for different conditionals? Or should you have one long “uber” shader with all of the possibilities listed conditionally? Function specialization deals with this problem, and allows you to create one shader that the compiler turns into separate shaders.

When you create the model’s pipeline state, you set the Metal functions in the Metal Shading library, and the compiler packages them up. At this stage, you can create properties, and assign them index numbers to deal with conditional states. You can then pass these properties to the Metal library when you create the shader functions. The compiler will examine the functions and generate specialized versions of them.

In the shader file, you reference the properties by their index numbers.

Function constants
Function constants

➤ In the Render Passes group, open Pipelines.swift.

You’ll first create a set of function constant values that will indicate whether to render with animation.

➤ Add this new method to PipelineStates:

static func makeFunctionConstants(hasSkeleton: Bool)
-> MTLFunctionConstantValues {
  let functionConstants = MTLFunctionConstantValues()
  var property = hasSkeleton
  functionConstants.setConstantValue(
    &property,
    type: .bool,
    index: 0)
  return functionConstants
}

MTLFunctionConstantValues is a set that contains a Boolean value depending on whether a skeleton exists. You defined a Boolean value here, but values can be any type specified by MTLDataType. On the GPU side, you’ll soon create a Boolean constant using the same index value. In functions that use these constants, you can conditionally perform tasks.

➤ Change the signature of createForwardPSO() to:

static func createForwardPSO(hasSkeleton: Bool = false)
-> MTLRenderPipelineState {

➤ At the top of createForwardPSO(hasSkeleton:), change the assignment to vertexFunction to:

let functionConstants =
  makeFunctionConstants(hasSkeleton: hasSkeleton)
let vertexFunction = try? Renderer.library?.makeFunction(
  name: "vertex_main",
  constantValues: functionConstants)

Here, you tell the compiler to create a library of functions using the function constants set. The compiler creates multiple shader functions and optimizes any conditionals in the functions.

Repeat this for createForwardTransparentPSO().

➤ Change the signature of createForwardTransparentPSO() to:

static func createForwardTransparentPSO(hasSkeleton: Bool = false)
-> MTLRenderPipelineState {

➤ At the top of createForwardTransparentPSO(hasSkeleton:), change the assignment to vertexFunction:

let functionConstants =
  makeFunctionConstants(hasSkeleton: hasSkeleton)
let vertexFunction = try? Renderer.library?.makeFunction(
  name: "vertex_main",
  constantValues: functionConstants)

Currently, you set the same pipeline for all models. You create a standard pipeline state for rendering models, one for rendering with transparency and one for rendering shadows. Generally, when you have any complexity, you’ll have to work out a system appropriate for your app to manage all your various pipeline states. You could use function specialization where possible, or create different vertex and fragment functions.

In this app, the time when you know whether your model has a skeleton or not, is when you load Mesh.

➤ Open Mesh.swift, and add a new property to Mesh:

var pipelineState: MTLRenderPipelineState

➤ Add the following code to the end of init(mdlMesh:mtkMesh:):

let hasSkeleton = skeleton?.jointMatrixPaletteBuffer != nil
pipelineState =
  PipelineStates.createForwardPSO(hasSkeleton: hasSkeleton)

When you load the mesh, you’ll create a pipeline state with the appropriate vertex function.

➤ Open Model.swift, and in render(encoder:uniforms:params:), at the top of the for mesh in meshes loop, add this:

encoder.setRenderPipelineState(mesh.pipelineState)

Each time you render a model, you’ll load the appropriate pipeline state object. As long as you do the creation of the pipeline states at the start of your app, they are lightweight to swap in and out.

Now for the GPU side!

➤ Open Shaders.metal, and add this code after the import statements:

constant bool hasSkeleton [[function_constant(0)]];

The function constant index matches the constant you just created in the MTLFunctionConstantValues set.

➤ In the vertex_main header, change the jointMatrices parameter to:

constant float4x4 *jointMatrices [[
  buffer(JointBuffer),
  function_constant(hasSkeleton)]]

You’ll have two different vertex_mains in your Metal shader library, one for each condition of hasSkeleton. One vertex_main will have jointMatrices as a parameter, if hasSkeleton is true, and the other won’t have that parameter at all.

➤ In vertex_main, remove:

bool hasSkeleton = true;

You use the pipeline constant in place of the local one.

Your animation may glitch because of synchronization issues. You’ll find out how to optimize your CPU / GPU synchronization in Chapter 26, “GPU-Driven Rendering”.

➤ For the moment, open Renderer.swift, and at the end of draw(scene:in:), add this:

commandBuffer.waitUntilCompleted()

The thread will be blocked until the command buffer has finished executing all its commands.

➤ Build and run the app, and you’ll see your full scene of animated skeleton and static ground.

Key Points

  • Character animation differs from transform animation. With transform animation, you deform the mesh directly. When animating characters, you use a skeleton with joints. The geometry mesh is attached to these joints and deforms when you rotate a joint.
  • The skeleton consists of a hierarchy of joints. When you rotate one joint, all the child joints move appropriately.
  • You attach the mesh to joints by weight painting in a 3D app. Up to four joints can influence each vertex (this is a limitation in your app, but generally weighting four joints is ample).
  • Animation clips contain transformation data for keyframes. The app interpolates the transformations between keyframes.
  • Each joint has an inverse bind matrix, which, when applied, moves the joint to the origin.
  • When your shaders have different requirements depending on different situations, you can use function specialization. You indicate the different requirements in the pipeline state, and the compiler creates multiple versions of the shader function.

Where to Go From Here?

This chapter took you through the basics of character animation. But don’t stop there! There are so many different topics that you can investigate. For instance, you can:

  • Learn how to animate your own characters in Blender and import them into your renderer. Start off with a simple robot arm, and work upward from there.
  • Download models from http://sketchfab.com, convert them to USD and see what works and what doesn’t.
  • Watch Disney and Pixar movies… call it research. No, seriously! Animation is a skill all of its own. Watch how people move; good animators can capture personality in a simple walk cycle.
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2026 Kodeco Inc.