8. Textures

Written by Marius Horga & Caroline Begbie

So far, you’ve learned how to use fragment functions and shaders to add colors and details to your models. Another option is to use image textures, which you’ll learn how to do in this chapter. More specifically, you’ll learn about:

• UV coordinates: How to unwrap a mesh so that you can apply a texture to it.
• Texturing a model: How to read the texture in a fragment shader.
• Samplers: Different ways you can read (sample) a texture.
• Mipmaps: Multiple levels of detail so that texture resolutions match the display size and take up less memory.
• Asset catalog: How to organize your textures.

Textures and UV Maps

The following image shows a house model with twelve vertices. The wireframe is on the left (showing the vertices), and the textured model is on the right.

Note: If you want a closer look at this model, you’ll find the Blender and .obj files in the resources/LowPolyHouse folder for this chapter.

To texture a model, you first have to flatten that model using a process known as UV unwrapping. UV unwrapping creates a UV map by unfolding the model. To unfold the model, you mark and cut seams using a modeling app. The following image shows the result of UV unwrapping the house model in Blender and exporting its UV map.

Notice that the roof and walls have marked seams. Seams are what make it possible for this model to lie flat. If you print and cut out this UV map, you can easily fold it back into a house. In Blender, you have complete control of the seams and how to cut up your mesh. Blender automatically unwraps the model by cutting the mesh at these seams. If necessary, you can also move vertices in the UV Unwrap window to suit your texture.

Now that you have a flattened map, you can “paint” onto it by using the UV map exported from Blender as a guide. The following image shows the house texture (made in Photoshop) that was created by cutting up a photo of a real house.

Note how the edges of the texture aren’t perfect, and the copyright message is visible. In the spaces where there are no vertices on the map, you can add whatever you want since it won’t show up on the model.

Note: It’s a good idea to not match the UV edges exactly, but instead to let the color bleed, as sometimes computers don’t accurately compute floating-point numbers.

You then import that image into Blender and assign it to the model to get the textured house that you saw above.

When you export a UV mapped model to an .obj file, Blender adds the UV coordinates to the file. Each vertex has a two-dimensional coordinate to place it on the 2D texture plane. The top-left is (0, 1) and the bottom-right is (1, 0).

The following diagram indicates some of the house vertices, with the matching coordinates from the .obj file. You can look at the contents of the .obj file using TextEdit.

One of the advantages of mapping from 0 to 1 is that you can swap in lower or higher resolution textures. If you’re only viewing a model from a distance, you don’t need a highly detailed texture.

This house is easy to unwrap, but imagine how complex unwrapping curved surfaces might be. The following image shows the UV map of the train (which is still a simple model):

Photoshop, naturally, is not the only solution for texturing a model. You can use any image editor for painting on a flat texture. In the last few years, several other apps that allow painting directly on the model have become mainstream:

• Blender (free)
• Substance Designer and Substance Painter by Adobe ($$): In Designer, you can create complex materials procedurally. Using Substance Painter, you can paint these materials on the model.
• 3DCoat by 3Dcoat.com ($$)
• Mudbox by Autodesk ($$)
• Mari by Foundry ($$$)

In addition to texturing, using Blender, 3DCoat or Mudbox, you can sculpt models in a similar fashion to ZBrush and create low poly models from the high poly sculpt. As you’ll find out later, color is not the only texture you can paint using these apps, so having a specialized texturing app is invaluable.

The Starter App

➤ Open the starter project for this chapter, and build and run the app.

The scene contains the low poly house. The fragment shader code is the same code from the challenge in the previous chapter, with hemispheric lighting added and a different background color. The vertex and fragment shaders are combined in Shaders.metal.

The other major changes are:

• Mesh.swift and Submesh.swift extract the Model I/O and MetalKit mesh buffers into custom vertex buffers and submesh groups. Model now contains an array of Meshs in place of an MTKMesh. Abstracting away from the Metal API allows for greater flexibility when generating models that don’t use Model I/O and MetalKit. Remember, it’s your engine, so you can choose how to hold the mesh data.
• VertexDescriptor.swift contains a UV attribute. Model loads UVs in the same way as you loaded normals in the previous chapter. Notice how the UVs will go into a separate buffer from the position and normal. This isn’t necessary, but it makes the layout more flexible for use with custom-generated models.
• Renderer.swift passes uniforms and params to Model to perform the rendering code.

➤ Open Shaders.metal.

VertexIn and VertexOut contain the uv property. The vertex function passes the interpolated UV to the fragment function. This process is the same as adding the normal in the previous chapter.

In this chapter, you’ll replace the sky and earth colors in the fragment function with colors from the texture. Initially, you’ll use lowpoly-house-color.png located in the group Models ▸ LowPolyHouse. To read the texture in the fragment function, you’ll take the following steps:

1. Load and store the image texture centrally.
2. Pass the loaded texture to the fragment function before drawing the model.
3. Change the fragment function to read the appropriate pixel from the texture.

1. Loading the Texture

A model typically has several submeshes that reference one or more textures. Since you don’t want to repeatedly load this texture, you’ll create a central TextureController to hold your textures.

➤ Create a new Swift file named TextureController.swift. Be sure to include the new file in both the macOS and iOS targets. Replace the code with:

import MetalKit

enum TextureController {
  static var textures: [String: MTLTexture] = [:]
}

TextureController will grab the textures used by your models and hold them in this dictionary.

➤ Add a new method to TextureController:

static func loadTexture(filename: String) throws -> MTLTexture? {
  // 1
  let textureLoader = MTKTextureLoader(device: Renderer.device)
  // 2
  let textureLoaderOptions: [MTKTextureLoader.Option: Any] =
    [.origin: MTKTextureLoader.Origin.bottomLeft]
  // 3
  let fileExtension =
    URL(fileURLWithPath: filename).pathExtension.isEmpty ?
      "png" : nil
  // 4
  guard let url = Bundle.main.url(
    forResource: filename,
    withExtension: fileExtension)
    else {
      print("Failed to load \(filename)")
      return nil
  }
  let texture = try textureLoader.newTexture(
    URL: url,
    options: textureLoaderOptions)
  print("loaded texture: \(url.lastPathComponent)")
  return texture
}

Going through the code:

1. Create a texture loader using MetalKit’s MTKTextureLoader.

2. Change the texture’s origin option to ensure that the texture loads with its origin at the bottom-left, which is what you need for lowpoly-house-color.png.

3. Provide a default extension for the image name.

4. Create a new MTLTexture using the provided image name and loader options. Also, return the newly created texture, and for debugging purposes, print the name of the loaded texture.

Note: Loading textures can get complicated. When Metal was first released, you had to specify everything about the image — like pixel format, dimensions and usage — using MTLTextureDescriptor. However, with MetalKit’s MTKTextureLoader, you can use the provided default values and optionally change them as needed.

➤ Add a new method to TextureController:

static func texture(filename: String) -> MTLTexture? {
  if let texture = textures[filename] {
    return texture
  }
  let texture = try? loadTexture(filename: filename)
  if texture != nil {
    textures[filename] = texture
  }
  return texture
}

Here, you return a reference to the texture, and if the filename is new, you save the new texture to the central texture dictionary.

Loading the Submesh Texture

Each submesh of a model’s mesh has a different material characteristic, such as roughness, base color and metallic content. For now, you’ll focus only on the base color texture. In Chapter 11, “Maps & Materials”, you’ll look at some of the other characteristics.

Conveniently, Model I/O loads a model complete with all the materials.

➤ Open lowpoly-house.mtl in the group Models ▸ LowPolyHouse. To see the text in the file, you may have to right-click the file and choose Open with External Editor from the menu.

The Kd value holds the diffuse material color — in this case, a light gray. At the very bottom of the file, you’ll see map_Kd lowpoly-house-color.png. This gives Model I/O the diffuse color map file name.

➤ Open Submesh.swift, and inside Submesh, create a structure and a property to hold the textures:

struct Textures {
  let baseColor: MTLTexture?
}

let textures: Textures

Don’t worry about compile errors; your project won’t compile until you’ve initialized textures.

MDLSubmesh holds each submesh’s material information in an MDLMaterial property. You provide the material with a semantic to retrieve the value for the relevant material. For example, the semantic for base color is MDLMaterialSemantic.baseColor.

➤ At the end of Submesh.swift, add an initializer for Textures:

private extension Submesh.Textures {
  init(material: MDLMaterial?) {
    func property(with semantic: MDLMaterialSemantic)
      -> MTLTexture? {
      guard let property = material?.property(with: semantic),
        property.type == .string,
        let filename = property.stringValue,
        let texture =
          TextureController.texture(filename: filename)
      else { return nil }
      return texture
    }
    baseColor = property(with: MDLMaterialSemantic.baseColor)
  }
}

property(with:) looks up the provided property in the submesh’s material, finds the filename string value of the property and returns a texture if there is one. Remember, there was another material property in the file marked Kd. That was the base color using floats. Material properties can also be float values where there is no texture available for the submesh.

This loads the base color texture with the submesh’s material. Here, Base color means the same as diffuse. Later, you’ll load other textures for the submesh in the same way.

➤ At the bottom of init(mdlSubmesh:mtkSubmesh) add:

textures = Textures(material: mdlSubmesh.material)

This code completes the initialization and removes the compiler warning.

➤ Build and run your app to check that everything’s working. Your model will look the same as in the initial screenshot. However, you’ll get a message in the console:

The texture loader has successfully loaded lowpoly-house-color.png.

2. Passing the Loaded Texture to the Fragment Function

In a later chapter, you’ll learn about several other texture types and how to send them to the fragment function using different indices.

➤ Open Common.h, and add a new enumeration to keep track of these texture buffer index numbers:

typedef enum {
  BaseColor = 0
} TextureIndices;

➤ Open VertexDescriptor.swift, and add this code to the end of the file:

extension TextureIndices {
  var index: Int {
    return Int(self.rawValue)
  }
}

This code allows you to use BaseColor.index instead of Int(BaseColor.rawValue)). A small touch, but it makes your code easier to read.

➤ Open Model.swift.

In render(encoder:uniforms:params:) where you process the submeshes, add the following code below the comment // set the fragment texture here:

encoder.setFragmentTexture(
  submesh.textures.baseColor,
  index: BaseColor.index)

You’re now passing the texture to the fragment function in texture buffer 0.

Note: Buffers, textures and sampler states are held in argument tables. As you’ve seen, you access these things by index numbers. On iOS, you can hold at least 31 buffers and textures, and 16 sampler states in the argument table; the number of textures on macOS increases to 128. You can find out feature availability for your device in Apple’s Metal Feature Set Tables.

3. Updating the Fragment Function

➤ Open Shaders.metal, and add the following new argument to fragment_main, immediately after VertexOut in [[stage_in]],:

texture2d<float> baseColorTexture [[texture(BaseColor)]]

You’re now able to access the texture on the GPU.

➤ Replace all the code in fragment_main with:

constexpr sampler textureSampler;

When you read or sample the texture, you may not land precisely on a particular pixel. In texture space, the units that you sample are known as texels, and you can decide how each texel is processed using a sampler. You’ll learn more about samplers shortly.

➤ Next, add this:

float3 baseColor = baseColorTexture.sample(
  textureSampler,
  in.uv).rgb;
return float4(baseColor, 1);

Here, you sample the texture using the interpolated UV coordinates sent from the vertex function, and you retrieve the RGB values. In Metal Shading Language, you can use rgb to address the float elements as an equivalent of xyz. You then return the texture color from the fragment function.

➤ Build and run the app to see your textured house.

sRGB Color Space

You’ll notice that the rendered texture looks much darker than the original image. This change in color happens because lowpoly-house-color.png is an sRGB texture. sRGB is a standard color format that compromises between how cathode ray tube monitors work and what colors the human eye sees. As you can see in the following example of grayscale values from 0 to 1, sRGB colors are not linear. Humans are more able to discern between lighter values than darker ones.

Unfortunately, it’s not easy to do the math on colors in a non-linear space. If you multiply a color by 0.5 to darken it, the difference in sRGB will vary along the scale.

You’re currently loading the texture as sRGB pixel data and rendering it into a linear color space. So when you’re sampling a value of, say 0.2, which in sRGB space is mid-gray, the linear space will read that as dark-gray.

To approximately convert the color, you can use the inverse of gamma 2.2:

sRGBcolor = pow(linearColor, 1.0/2.2);

If you use this formula on baseColor before returning from the fragment function, your house texture will look about the same as the original sRGB texture. However, a better way of dealing with this problem is not to load the texture as sRGB at all.

➤ Open TextureController.swift, and in loadTexture(filename:), locate:

let textureLoaderOptions: [MTKTextureLoader.Option: Any] =
  [.origin: MTKTextureLoader.Origin.bottomLeft]

➤ Change it to:

let textureLoaderOptions: [MTKTextureLoader.Option: Any] = [
  .origin: MTKTextureLoader.Origin.bottomLeft,
  .SRGB: false
]

➤ Build and run the app, and the texture now loads with the linear color pixel format bgra8Unorm.

Note: An alternative to loading the textures with SRGB as false is to change the MTKView‘s colorPixelFormat to bgra8Unorm_srgb. This change will affect the view’s color space, and the clear color background will also change. You’ll find further reading on chromaticity and color in references.markdown in the resources folder for this chapter.

Capture GPU Workload

There’s an easy way to find out what format your texture is in on the GPU, and also to look at all the other Metal buffers currently residing there: the Capture GPU workload tool (also called the GPU Debugger).

➤ Run your app, and at the bottom of the Xcode window (or above the debug console if you have it open), click the M Metal icon, change the number of frames to count to 1, and click Capture in the pop-up window:

This button captures the current GPU frame. On the left in the Debug navigator, you’ll see the GPU trace:

Note: To open or close all items in a hierarchy, you can Option-click the arrow.

You can see all the commands that you’ve given to the render command encoder, such as setFragmentBytes and setRenderPipelineState. Later, when you have several command encoders, you’ll see each one of them listed, and you can select them to see what actions or textures they have produced from their encoding.

When you select drawIndexedPrimitives, the Vertex and Fragment resources show.

➤ Double-click each vertex resource to see what’s in the buffer:

• MDL_OBJ-Indices: The vertex indices.
• Buffer 0: The vertex position and normal data, matching the attributes of your VertexIn struct and the vertex descriptor.
• Buffer 1: The UV texture coordinate data.
• Vertex Bytes: The uniform matrices.
• Vertex Attributes: The incoming data from VertexIn, and the VertexOut return data from the vertex function.
• vertex_main: The vertex function. When you have multiple vertex functions, this is very useful to make sure that you set the correct pipeline state.

Going through the fragment resources:

• lowpoly-house-color.png: The house texture in texture slot 0.
• Fragment Bytes: The width and height screen parameters in params.
• fragment_main: The fragment function.

The attachments:

• CAMetalLayer Drawable: The result of the encoding in color attachment 0. In this case, this is the view’s current drawable. Later, you’ll use multiple color attachments.
• MTKView Depth: The depth buffer. Black is closer. White is farther. The rasterizer uses the depth map.

You can see from this list that the GPU is holding the lowpoly-house-color.png texture as BGRA8Unorm. If you reverse the previous section’s texture loading options and comment out .SRGB: false, you’ll be able to see that the texture is now BGRA8Unorm_sRGB. (Make sure you restore the option .SRGB: false before continuing.)

If you’re ever uncertain as to what is happening in your app, capturing the GPU frame might give you the heads-up because you can examine every render encoder command and every buffer. It’s a good idea to use this strategy throughout this book to examine what’s happening on the GPU.

Samplers

When sampling your texture in the fragment function, you use a default sampler. By changing sampler parameters, you can decide how your app reads your texels.

You’ll now add a ground plane to your scene to see how you can control the appearance of the ground texture.

➤ Open Renderer.swift, and add a new property:

lazy var ground: Model = {
  Model(name: "plane.obj")
}()

In draw(in:) after rendering the house and before renderEncoder.endEncoding(), add:

ground.scale = 40
ground.rotation.y = sin(timer)
ground.render(
  encoder: renderEncoder,
  uniforms: uniforms,
  params: params)

This code adds a ground plane and scales it up.

➤ Build and run the app.

The ground texture stretches to fit the ground plane, and each pixel in the texture may be used by several rendered fragments, giving it a pixellated look. By changing one of the sampler parameters, you can tell Metal how to process the texel where it’s smaller than the assigned fragments.

➤ Open Shaders.metal. In fragment_main, change the textureSampler definition to:

constexpr sampler textureSampler(filter::linear);

This code instructs the sampler to smooth the texture.

➤ Build and run the app.

The ground texture — although still stretched — is now smooth. There will be times, such as when you make a retro game of Frogger, that you’ll want to keep the pixelation. In that case, use nearest filtering.

In this particular case, however, you want to tile the texture. That’s easy with sampling.

➤ Change the sampler definition and the baseColor assignment to:

constexpr sampler textureSampler(
  filter::linear,
  address::repeat);
float3 baseColor = baseColorTexture.sample(
  textureSampler,
  in.uv * 16).rgb;

This code multiplies the UV coordinates by 16 and accesses the texture outside of the allowable limits of 0 to 1. address::repeat changes the sampler’s addressing mode, so it’ll repeat the texture 16 times across the plane.

The following image illustrates the other address sampling options shown with a tiling value of 3. You can use s_address or t_address to change only the width or height coordinates, respectively.

➤ Build and run your app.

The ground looks great! The house… not so much. The shader has tiled the house texture as well. To overcome this problem, you’ll create a tiling property on the model and send it to the fragment function with params.

➤ In Common.h, add this to Params:

uint tiling;

➤ In Model.swift, create a new property in Model:

var tiling: UInt32 = 1

➤ In render(encoder:uniforms:params:), just after var params = fragment, add this:

params.tiling = tiling

➤ In Renderer.swift, replace the declaration of ground with:

lazy var ground: Model = {
  var ground = Model(name: "plane.obj")
  ground.tiling = 16
  return ground
}()

You’re now sending the model’s tiling factor to the fragment function.

➤ Open Shaders.metal. In fragment_main, replace the declaration of baseColor with:

float3 baseColor = baseColorTexture.sample(
  textureSampler,
  in.uv * params.tiling).rgb;

➤ Build and run the app, and you’ll see that both the ground and house now tile correctly.

Note: Creating a sampler in the shader is not the only option. You can create an MTLSamplerState, hold it with the model and send the sampler state to the fragment function with the [[sampler(n)]] attribute.

As the scene rotates, you’ll notice some distracting noise. You’ve seen what happens on the grass when you oversample a texture. But, when you undersample a texture, you can get a rendering artifact known as moiré, which is occurring on the roof of the house.

In addition, the noise at the horizon almost looks as if the grass is sparkling. You can solve these artifact issues by sampling correctly using resized textures called mipmaps.

Mipmaps

Check out the relative sizes of the roof texture and how it appears on the screen.

The pattern occurs because you’re sampling more texels than you have pixels. The ideal would be to have the same number of texels to pixels, meaning that you’d require smaller and smaller textures the further away an object is. The solution is to use mipmaps. Mipmaps let the GPU compare the fragment on its depth texture and sample the texture at a suitable size.

MIP stands for multum in parvo — a Latin phrase meaning “much in small”.

Mipmaps are texture maps resized down by a power of 2 for each level, all the way down to 1 pixel in size. If you have a texture of 64 pixels by 64 pixels, then a complete mipmap set would consist of:

Level 0: 64 x 64, 1: 32 x 32, 2: 16 x 16, 3: 8 x 8, 4: 4 x 4, 5: 2 x 2, 6: 1 x 1.

In the following image, the top checkered texture has no mipmaps. But in the bottom image, every fragment is sampled from the appropriate MIP level. As the checkers recede, there’s much less noise, and the image is cleaner. At the horizon, you can see the solid color smaller gray mipmaps.

You can easily and automatically generate these mipmaps when first loading the texture.

➤ Open TextureController.swift. In loadTexture(filename:), change the texture loading options to:

let textureLoaderOptions: [MTKTextureLoader.Option: Any] = [
  .origin: MTKTextureLoader.Origin.bottomLeft,
  .SRGB: false,
  .generateMipmaps: NSNumber(value: true)
]

This code will create mipmaps all the way down to the smallest pixel.

There’s one more thing to change: the sampler.

➤ Open Shaders.metal, and add the following code to the construction of textureSampler:

mip_filter::linear

The default for mip_filter is none. However, if you provide either .linear or .nearest, then the GPU will sample the correct mipmap.

➤ Build and run the app.

The noise from both the building and the ground is gone.

Using the Capture GPU workload tool, you can inspect the mipmaps. Choose the draw call, and double-click a texture. At the bottom-left, you can choose the MIP level. This is MIP level 4 on the house texture:

Anisotropy

Your rendered ground is looking a bit muddy and blurred in the background. This is due to anisotropy. Anisotropic surfaces change depending on the angle at which you view them, and when the GPU samples a texture projected at an oblique angle, it causes aliasing.

➤ In Shaders.metal, add this to the construction of textureSampler:

max_anisotropy(8)

Metal will now take eight samples from the texel to construct the fragment. You can specify up to 16 samples to improve quality. Use as few as you can to obtain the quality you need because the sampling can slow down rendering.

Note: As mentioned before, you can hold an MTLSamplerState on Model. If you increase anisotropy sampling, you may not want it on all models, and this might be a good reason for creating the sampler state outside the fragment shader.

➤ Build and run, and your render should be artifact-free.

When you write your full game, you’re likely to have many textures for the different models. Some models are likely to have several textures. Organizing these textures and working out which ones need mipmaps can become labor-intensive. Plus, you’ll also want to compress images where you can and send textures of varying sizes and color gamuts to different devices. The asset catalog is where you’ll turn.

The Asset Catalog

As its name suggests, the asset catalog can hold all of your assets, whether they be data, images, textures or even colors. You’ve probably used the catalog for app icons and images. Textures differ from images in that the GPU uses them, and thus they have different attributes in the catalog. To create textures, you add a new texture set to the asset catalog.

You’ll now replace the textures for the low poly house and ground and use textures from a catalog.

➤ Create a new file using the Asset Catalog template (found in the Resource section), and name it Textures. Remember to check both the iOS and macOS targets.

➤ With Textures.xcassets open, choose Editor ▸ Add New Asset ▸ AR and Textures ▸ Texture Set (or click the + at the bottom of the panel and choose AR and Textures ▸ Texture Set).

➤ Double-click the Texture name and rename it to grass.

➤ Open the Models ▸ Textures group and drag barn-ground.png to the Universal slot in your catalog. With the Attributes inspector open, click on the grass to see all of the texture options.

Here, you can see that by default, all mipmaps are created automatically. If you change Mipmap Levels to Fixed, you can choose how many levels to make. If you don’t like the automatic mipmaps, you can replace them with your own custom mipmaps by dragging them to the correct slot.

Asset catalogs give you complete control of your textures without having to write cumbersome code, although you can still write the code using the MTLTextureDescriptor API if you want.

Now that you’re using named textures from the asset catalog instead of .png files, you’ll need to change your texture loader.

➤ Open TextureController.swift, and at the top of loadTexture(filename:), after defining textureLoader, add this:

 if let texture = try? textureLoader.newTexture(
   name: filename,
   scaleFactor: 1.0,
   bundle: Bundle.main,
   options: nil) {
   print("loaded texture: \(filename)")
   return texture
 }

This now searches the bundle for the named texture and loads it if there is one. When loading from the asset catalog, the options that you set in the Attributes inspector take the place of most of the texture loading options, so these options are now nil.

The last thing to do is to make sure the model points to the new texture.

➤ Open plane.mtl, located in Models ▸ Ground. If your file is not text-editable, you can right-click the file and choose Open in External Editor.

➤ Replace:

map_Kd ground.png

➤ With:

#map_Kd ground.png
map_Kd grass

Here, you commented out the old texture and added the new one. The grass texture will now load from the asset catalog in place of the old one.

➤ Repeat this for the low poly house to change it into a barn:

1. Create a new texture set in the asset catalog and rename it barn.
2. Drag lowpoly-barn-color.png into the texture set from the Models ▸ Textures group.
3. Change the name of the diffuse texture in Models ▸ LowPolyHouse ▸ lowpoly-house.mtl to barn.

Note: Be careful to drop the images on the texture’s Universal slot. If you drag the images into the asset catalog, they are, by default, images and not textures. And you won’t be able to make mipmaps on images or change the pixel format.

➤ Build and run your app to see your new textures.

You can see that the textures have reverted to the sRGB space because you’re now loading them in their original format. You can confirm this using the Capture GPU workload tool.

➤ Open Textures.xcassets, click on the barn texture, and in the Attributes inspector, change the Interpretation to Data:

When your app loads the sRGB texture to a non-sRGB buffer, it automatically converts from sRGB space to linear space. (See Apple’s Metal Shading Language document for the conversion rule.) By accessing as data instead of colors, your shader can treat the color data as linear.

You’ll also notice in the above image that the origin — unlike loading the .png texture manually — is Top Left. The asset catalog loads textures differently.

➤ Repeat for the grass texture.

➤ Build and run, and your colors should now be correct.

The Right Texture for the Right Job

Using asset catalogs gives you complete control over how to deliver your textures. Currently, you only have two color textures. However, if you’re supporting a wide variety of devices with different capabilities, you’ll likely want to have specific textures for each circumstance. On devices with less RAM, you’d want smaller graphics.

For example, here is a list of individual textures you can assign by checking the different options in the Attributes inspector, for the Apple Watch, devices with 3GB and 4GB memory, and sRGB and P3 displays.

Texture Compression

In recent years, people have put much effort into compressing textures to save both CPU and GPU memory. There are various formats you can use, such as ETC and PVRTC. Apple has embraced ASTC as being the most high-quality compressed format. ASTC is available on the A8 chip and newer.

Using texture sets within the asset catalog allows your app to determine for itself which is the best format to use.

With your app running on macOS, take a look at how much memory it’s consuming.

➤ Click on the Debug navigator and select Memory.

This is the usage after 45 seconds — your app’s memory consumption will increase for about five minutes and then stabilize.

If you capture the frame with the Capture GPU Workload button, you’ll see that the texture format on the GPU is RGBA8Unorm. When you use asset catalogs, Apple will automatically determine the most appropriate format for your texture.

➤ In Textures.xcassets, select each of your textures, and in the Attributes inspector, change the Pixel Format from Automatic to ASTC 8×8 Compressed - Red Green Blue Alpha. This is a highly compressed format.

➤ Build and run your app, and check the memory usage again.

You’ll see that the memory footprint is slightly reduced. However, so is the quality of the render. For distant textures, this quality might be fine, but you have to balance memory usage with render quality.

Note: You may have to test the app on an iOS device to see the change in texture format in the GPU Debugger. On iOS, the automatic format will be ASTC 4×4, which is indistinguishable from the png render.

Key Points

• UVs, also known as texture coordinates, match vertices to the location in a texture.
• During the modeling process, you flatten the model by marking seams. You can then paint on a texture that matches the flattened model map.
• You can load textures using either the MTKTextureLoader or the asset catalog.
• A model may be split into groups of vertices known as submeshes. Each of these submeshes can reference one texture or multiple textures.
• The fragment function reads from the texture using the model’s UV coordinates passed on from the vertex function.
• The sRGB color space is the default color gamut. Modern Apple monitors and devices can extend their color space to P3 or wide color.
• Capture GPU workload is a useful debugging tool. Use it regularly to inspect what’s happening on the GPU.
• Mipmaps are resized textures that match the fragment sampling. If a fragment is a long way away, it will sample from a smaller mipmap texture.
• The asset catalog is a great place to store all of your textures. Later, you’ll have multiple textures per model, and it’s better to keep them all in one place. Customization for different devices is easy using the asset catalog.
• Topics such as color and compression are huge. In the resources folder for this chapter, in references.markdown, you’ll find some recommended articles to read further.