Instruction

You’ll can pick up the app you’ve been building from the previous lesson, or you can download the starter and start to fill out the second tab with a Volume view. Before you dig into that, look at some key concepts and tools.

While Windows provide a spatial experience for relatively flat 2D content, Volumes provide more depth on the z-axis and move toward and away from the viewer. The new RealityView, a special SwiftUI View, adds the capability to display, add gestures, and functionality to 3D elements asynchronously in your view. The content is then presented in your app’s 3D space as a RealityViewContent content type.

A RealityView conforms to RealityViewContentProtocol, which includes make and update closures. The async make closure loads the content when the view is created, and the update closure deals with changes in the view’s state. You can also use the optional attachments closure to implement ViewBuilder attachment views to a scene.

You may recall that you had a Model3D view in the starter app. It’s a simpler SwiftUI view for asynchronously loading a model with a .usdz or .reality file. During loading, SwiftUI displays a spinner until it becomes a ResolvedModel3D. Model3D isn’t as robust as RealityView. RealityView also supports optional anchors for attaching your models, useful for plane tracking, hand tracking, and image tracking.

Pro Tip: If you’re familiar with SceneKit’s ARSCNView, you might find RealityView similar.

Universal Scene Description

A zipped Universal Scene Description or, more commonly, USDz, is often referred to in discussions of visionOS and Reality Composer Pro. The file format was initially developed by Pixar and maintained by Pixar, Apple, AudioDesk, and Nvidia.

USD is an extensible 3D scene description of 3D content, sound, color, and textures. A Scene.usda is used as an Entity in visionOS and is created and managed in Reality Composer Pro - more on that in a bit.

The A, in USDa, stands for ASCII encoded, and if you look at the source code in the RealityKitContent folder in Xcode, you’ll see it’s a readable XML file.

A USD Crate or USDc is a binary encoded file format. All USD files can contain material, color, textures, shaders, and geometry for 3D elements, as well as design, robotics, and CAD data.

USDz files can be unzipped to show that they are made up of crate files. USDz can also contain USD, USDc, USDa, PNG, JPEG, M4A, MP3, and WAV files. For visionOS and Reality Composer Pro, there should only be one Root element. Reality Composer Pro will tell you if the file you’re trying to use won’t work or is unsupported.

Reality Composer Pro

If you select the Scene.usda or the Package.realitycomposerpro Swift Package file in Xcode 15, you’ll see a button on the top right that says Open in Reality Composer Pro. Tap that button to launch the Reality Composer Pro app and open the file.

The leftmost pane is the Navigator and shows each Scene and its hierarchy. The default has a Root element. Below that, you may find a 3D primitive or model, as well as a Material, audio and more. These are files that define colors, textures, and shaders. A material can be applied to any element. For example, you can apply a gold color with a shiny metal texture to make your model appear to be made of gold.

In the middle pane or ViewPort, you’ll see a 3D space with a grid where you can place models. The circle at the center is where the x and y axis meet and represents the scene’s (0,0,0) coordinates. When you add a model, you drag it in from the Project Browser under the main editor screen.

Tap a model in the scene to see and edit the properties in the Inspector pane on the right. The element’s name is at the top. Next is the Transform editor, where you can adjust the position, rotation, and scale.

Below are References and Material Bindings for the model where you can attach shaders such as the gold properties. You can also create and define your own Shader Graphs and attach spacial, ambient, and channel audio as well as Particle Emitters. You’ll learn shader graphs and Particle Emitters in a later lesson.

Reality Composer Pro also contains a Content Library under the + on the top of the window. It contains 3D primitives, pre-shaded models, materials, and audio files. You can use these in your app’s scenes or as references to create your own. Drag any item from the library into the center Viewport editor or Navigator hierarchy, and it’ll be added to the scene.

As you change the Scene, add elements, and even make additional named scenes in Reality Composer Pro, you’ll see the additional elements and edits in the RealityKitContent Swift package in Xcode.

Automate With Timings

At WWDC 2024, Apple added Timings to Reality Composer Pro 2.0. When you tap Create Timeline you add a Timeline. You can add additional timelines with the + under the list of Timelines on the left. This gives you the option to automate several Entities individually in your scene.

On the right side, you can drag an Action onto the timeline. With actions you can add force, animate, play audio, or add an orbit to a chosen entity. Since timings are time based, you can start and end them where you like on the timeline, and you can drag the right side to set the duration. Typically you would set the Target entity and adjust the settings in the Inspector pane.

For example, you could have one entity orbit another. Choose your target, and then choose another entity for the first to pivot around, as well as set the axises, x, y, z for the direction of the orbit.

Timings was added to Reality Composer Pro after this course was created. We won’t cover them beyond this introduction. You can explore Timings on your own. Later in this course, you’ll create an orbit with RealityKit, however that could also be done with Timings.

Adding a Scene to the App

Back in Xcode, you can add Scenes or elements in the hierarchy inside a RealityView using the make: closure with loadEntity(), or load a scene by referencing the realityKitContentBundle. You can also simply load a model from the main Bundle or the realityKitContentBundle with Model3D. A RealityView is a more robust way to add 3D content.

In the next Demo video, you’ll learn how to create a Volume view and use Reality Composer Pro 2.0 to set up a Scene with 3D elements. Then, you’ll add a 3D model to it, load the model in the app using RealityKit, and enable the animations in the model with the click of a button.