vi. Introduction

Written by Marius Horga & Caroline Begbie

Welcome to Metal by Tutorials!

Metal is a unified, low-level, low-overhead application programming interface (API) for the graphics processing unit, or GPU. It’s unified because it applies to both 3D graphics and data-parallel computation paradigms. Metal is a low-level API because it provides programmers near-direct access to the GPU. Finally, Metal is a low-overhead API because it reduces the runtime cost by multi-threading and pre-compiling of resources.

But beyond the technical definition, Metal is the most appropriate way to use the parallel processing power of Apple GPUs to visualize data or solve numerical challenges. It’s also tailored to be used for machine learning, image/video processing or, as this book describes, graphics rendering.

About This Book

This book introduces you to low-level graphics programming in Metal — Apple’s framework for programming on the graphics processing unit (GPU). As you progress through this book, you’ll learn many of the fundamentals that go into making a game engine and gradually put together your own engine. Once your game engine is complete, you’ll be able to put together 3D scenes and program your own simple 3D games. Because you’ll have built your 3D game engine from scratch, you’ll be able to customize every aspect of what you see on your screen.

How Did Metal Come to Life?

Historically, you had two choices to take advantage of the power of the GPU: OpenGL and the Windows-only DirectX. In 2013, the GPU vendor AMD announced the Mantle project in an effort to revamp GPU APIs and come up with an alternative to Direct3D (which is part of DirectX) and OpenGL. AMD were the first to create a true low-overhead API for low-level access to the GPU. Mantle promised to be able to generate up to 9 times more draw calls (the number of objects drawn to the screen) than similar APIs and also introduced asynchronous command queues so that graphics and compute workloads could be run in parallel. Unfortunately, the project was terminated before it could become a mainstream API.

Metal was announced at the Worldwide Developers Conference (WWDC) on June 2, 2014 and was initially made available only on A7 or newer GPUs. Apple created a new language to program the GPU directly via shader functions. This is the Metal Shading Language (MSL) based on the C++11 specification. A year later at WWDC 2015, Apple announced two Metal sub-frameworks: MetalKit and Metal Performance Shaders (MPS). In 2018, MPS made a spectacular debut as a Ray Tracing accelerator.

The API continued its evolution to work with the exciting features of the new Apple GPUS designed in-house by Apple. Metal 2 added support for Virtual Reality (VR), Augmented Reality (AR) and accelerated machine learning (ML), among many new features, including image blocks, tile shading and threadgroup sharing. Metal Shading Language is now based on the C++14 specification.

In 2022, Apple introduced Metal 3 with a new framework MetalFX for upscaling lower resolutions. Metal 3 features also include fast resource loading of textures directly from disk, mesh shaders for adding or reducing geometry and using C/C++ for Metal.

In 2025, Apple introduced Metal 4. This latest version improves command encoding and resource management. It supports machine learning and tensors. New API commands allow you to adopt Metal 4 gradually. This book doesn’t cover Metal 4, but when you have learned the vocabulary and concepts of Metal and Computer Graphics, you will be able to watch Apple’s introductory videos and understand what the changes are.

Why Would You Use Metal?

Metal is a top-notch graphics API. That means Metal can empower graphics pipelines and, more specifically, game engines such as the following:

• Unity and Unreal Engine: The two leading cross-platform game engines today are ideal for game programmers who target a range of console, desktop and mobile devices. However, these engines haven’t always kept pace with new features in Metal. For example, tessellation was long delayed in Unity, and mesh shaders are still not fully supported. If you want to use cutting-edge Metal developments, and use the power of Apple silicon, you can’t always depend on third-party engines.

• Divinity - Original Sin 2: Larian Studios worked closely with Apple to bring their amazing AAA game to iPad, taking advantage of Metal and the Apple GPU hardware. It truly is a stunning visual experience.

• Cyberpunk 2077: This game, when the new version is released, will take full advantage of the power of Metal 4.

• Many Others: From notable game titles such as Hitman, BioShock, Deus Ex, Mafia, Starcraft, World of Warcraft, Fortnite, Unreal Tournament, Batman and even the beloved Minecraft.

But Metal isn’t limited to the world of gaming. There are many apps that benefit from GPU acceleration for image and video processing:

• Procreate: An app for sketching, painting and illustrating. Since converting to Metal, it runs four times faster than it did before.
• Pixelmator Pro: A Metal-based app that provides image distortion tools. In fact, they were able to implement a new painting engine and dynamic paint blending technology powered by Metal 2.
• Affinity Photo: Available on the iPad. According to the developer Serif, “Using Metal allows users to work easily on large, super high-resolution photographs, or complex compositions with potentially thousands of layers.”
• Blender: Apple developers contributed Metal code to the open-source Blender, supporting GPU rendering using Metal.

When Should You Use Metal?

GPUs belong to a special class of computation that Flynn’s taxonomy terms Single Instruction Multiple Data (SIMD). Simply, GPUs are processors that are optimized for throughput (how much data can be processed in one unit of time), while CPUs are optimized for latency (how much time it takes a single unit of data to be processed). Most programs execute serially: they receive input, process it, provide output and then the cycle repeats.

Those cycles sometimes perform computationally-intensive tasks, such as large matrix multiplication, which would take CPUs a lot of time process serially, even in a multithreaded manner on a handful of cores.

In contrast, GPUs have hundreds or even thousands of cores which are smaller and have less memory than CPU cores, but perform fast parallel mathematical calculations.

Choose Metal when:

• You want to render 3D models as efficiently as possible.
• You want your game to have its own unique style, perhaps with custom lighting and shading.
• You will be performing intensive data processes, such as calculating and changing the color of each pixel on the screen every frame, as you would when processing images and video.
• You have large numerical problems, such as scientific simulations, that you can partition into independent sub-problems to be processed in parallel.
• You need to process multiple large datasets in parallel, such as when you train models for deep learning.

Who This Book Is For?

This book is for intermediate Swift developers interested in learning 3D graphics or gaining a deeper understanding of how game engines work. If you don’t know Swift, you can still follow along, as all the code instructions are included in the book. You’ll gain general graphics knowledge, but it would be less confusing if you cover Swift basics first. We recommend the Swift Apprentice: Fundamentals book, available in our catalog.

A smattering of C++ knowledge would be useful too. The Metal Shader Language that you’ll use when writing GPU shader functions is based on C++. But, again, all the code you’ll need is included in the book.

How to Read This Book?

If you’re a beginner to iOS/macOS development or Metal, you should read this book from cover to cover.

If you’re an advanced developer, or already have experience with Metal, you can skip from chapter to chapter or use this book as a reference.