RealityKit & Reality Composer Pro

Transcript

Building to Vision Pro

You can start with the app in the Starter folder or continue the build from lesson 1.

If you plan run the code on Vision Pro hardware, follow these instructions. Otherwise, skip to Updating the Code.

In order to build to a device, you’ll also need to modify the build settings. Before that, you’ll enable Developer Mode on the Vision Pro.

On the Vision Pro, open the Settings app. Under the General section, scroll down to Remote Devices. Select your Mac name and click the Pair button. If you don’t see your Mac, check that your Mac and Vision Pro are on the same Wi-Fi network. Enter the pairing code on your Mac.

*Note: The video from the Vision Pro uses foveated imaging. The device tracks the user’s eyes and only sharpens the image were the user is looking. As a result, screenshots and videos may appear blurry.

Back on the Vision Pro, go to Privacy and Security in the Settings app. After pairing to a Mac, the Developer Mode option will appear at the bottom. Click the toggle to enable it, and enter your Vision Pro’s pin code. Your Vision Pro will reboot.

After rebooting, you’ll see an alert asking if you want to enable Developer Mode.

Note: You can disable Developer Mode with the same toggle switch. The Vision Pro will need to reboot whenever you enable or disable Developer Mode.

Now that Developer Mode is enabled, you’ll need to select the project in the Project Navigator. Select your target and go the Signing & Capabilities. Choose your developer team from the selector. Enter your company’s reverse domain name to form the Bundle ID. Run a Build from the Product Menu, or press Command-B, to check for errors.

Updating the Code

You may be familiar with the wooden labyrinth toys, where you roll a metal ball by tilting the game. In this lesson, you’ll extend the code to put the ball on a surface and tilt to move left and right.

Generally apps on Vision Pro use the SF Symbol mountain.2 to indicate that the user is going into Immersion. Open up ContentView.swift, add a systemImage to the toggle.

Toggle("Show ImmersiveSpace", systemImage: "mountain.2", isOn: $showImmersiveSpace)

Now open ImmersiveView.swift, remove the metal ball created in Reality Composer Pro by deleting the line where you added the scene to the content:

//content.add(scene)

Since you aren’t using the scene, you can replace scene in the if let with an underscore where you loaded the Entity, like so:

if let _ = try? await Entity(named: "Immersive", in: realityKitContentBundle) { //.. }

Above the body variable, create a state variable for the maze piece:

@State var mazeA = Entity()

Set up some local constants below where ball is added to the scene:

// dimensions
let mazeX: Float = 1.0
let mazeY: Float = 0.1
let mazeZ: Float = 0.1

Use the RealityKit’s ModelEntity function to create an elongated box, using SimpleMaterial:

mazeA = ModelEntity(mesh: .generateBox(width: mazeX, height: mazeY, depth: mazeZ), materials: [SimpleMaterial()])

Place the box code below the ball and add it to the content:

mazeA.position.y = 0.9
mazeA.position.z = -1.5025
content.add(mazeA)

Build and run, then toggle the Immersive Space.

*Pro Tip: If you’re running on the hardware, the entities might block the Immersive Space toggle. When in the Immersive Space the window controls are available. The entities stay anchored. You can move around them, or you can grab the window’s bar and move it since they can be moved independently.

As before, the ball falls through the box. Add a physics and collision component to the box before the maze is added to the scene:

mazeA.components.set(CollisionComponent(shapes: [.generateBox(width: mazeX, height: mazeY, depth: mazeZ)]))
mazeA.components[PhysicsBodyComponent.self] = .init(
  PhysicsBodyComponent(
    massProperties: .default,
    material: .generate(
      staticFriction: 0.8,
      dynamicFriction: 0.0,
      restitution: 0.0
    ),
    mode: .kinematic
  )
)

For the ball’s stability, update the values for mass and friction. Also make the ball smaller by setting it at 5cm radius like so:

// change ball constructor:
let ball = ModelEntity(
  mesh: .generateSphere(radius: 0.05),
  materials: [SimpleMaterial(color: .white, isMetallic: true)]
)

// change PhysicsBodyComponent massProperties:
massProperties: .init(mass: 50.0),

// change the material .generate:
staticFriction: 10.0,
dynamicFriction: 50.0,
restitution: 0.0

// add a CollisionComponent
ball.components.set(CollisionComponent(shapes: [.generateSphere(radius: 0.05)]))

Now at the top of the View add a rotation state variable to capture the current rotation of the maze:

@State var rotationA: Angle = .zero

Refactor the DragGesture to only apply to the box. Make it tilt by applying a rotation. Remove the computed variable dragGesture, which you used to drag the balls in the previous lesson.

Replace the .gesture modifier on the RealityView.:

.gesture(DragGesture()
  .targetedToAnyEntity()
)

Add an .onChange observer to the gesture, and inside create a State variable rotation for the bar you added:

.onChanged { value in
  rotationA.degrees = value.translation.height / 20
  mazeA.transform = Transform(roll: Float(rotationA.radians))
  // Keep starting distance between models
  mazeA.position.y = 0.9
  mazeA.position.z = -1.5
}

Add an InputTargetComponent to the box where you added the physics and collision to the bar:

mazeA.components.set(InputTargetComponent())

Build and run. Drag the bar to tilt it. Oops! It rolls right off!

Let’s add some occluded blocks on the ends on the box as child entities. They’ll inherit the rotation.

At the bottom of the view, add a function to create the occluded boxes using DRY (don’t repeat yourself).

func occludedBlock(width: Float, height: Float, depth: Float, posX: Float, posY: Float ) -> Entity {
  let entity = ModelEntity(mesh: .generateBox(width: width, height: height, depth: depth))
  entity.components.set(CollisionComponent(shapes: [.generateBox(width: width, height: height, depth: depth)]))
  entity.components[PhysicsBodyComponent.self] = .init(mode: .static)
  entity.position.x = posX
  entity.position.y = posY
  return entity
}

Back in the RealityView, add the blocks as children of the box after adding the maze to the scene. For the position values, use mazeA‘s values and calculate their offsets with half of the box’s width:

let blockRight = occludedBlock(
  width: mazeY,
  height: mazeY * 2,
  depth: mazeY * 2,
  posX: mazeX / 2 + mazeY / 2,
  posY: mazeY
)
mazeA.addChild(blockRight)

let blockLeft = occludedBlock(
  width: mazeY,
  height: mazeY * 2,
  depth: mazeY * 2,
  posX: -(
    mazeX / 2 + mazeY / 2
  ),
  posY: mazeY
)
mazeA.addChild(blockLeft)

Build and run. Notice that the boxes appear pink and purple because there’s no material applied. If you tilt the bar, the ball rolls and stays on the bar.

Inside occludedBlock apply a material with zero opacity to hide the blocks:

entity.components[OpacityComponent.self] = .init(opacity: 0.0)

Add an attachment to display a hint to the player.

At the top of the RealityView declaration, add accessory output.

RealityView { content, attachments in

Before the gesture modifier, add the attachments closure.

} attachments: {
}

Inside the attachment closure, add a title — “Maze” — and instructions, some padding, and a frame. Decorate it for visionOS with a glass background.

Attachment(id: "maze-attach") {
  VStack {
    Text("Maze")
      .font(.largeTitle)
    Text("Drag to tilt the maze.")
      .font(.title)
  }
  .padding(.all, 20)
  .frame(maxWidth: 250, maxHeight: 250)
  .glassBackgroundEffect()
}

Next add the attachment as a child of the box, below where you added the blocks. It’s an optional so you use an if let to unwrap it.

if let mazeAttachment = attachments.entity(for: "maze-attach") {
  mazeAttachment.position = [mazeX / 2, 0, 2 * mazeZ ]
  mazeA.addChild(mazeAttachment)
}

Add a cone as a fulcrum to complete the picture.

let fulcrum = ModelEntity(mesh: .generateCone(height: 0.2, radius: 0.1), materials: [SimpleMaterial()])
fulcrum.position.y = 0.75
fulcrum.position.z = -1.5
content.add(fulcrum)

Build and run. You’ll notice that the simple game is now complete!

In this lesson you learned how to create a scene entirely in code with RealityKit. You made plenty of use of Entities and Components here. You used physics and collision components to give the game a realistic feel which is just what you want for a visionOS app since visionOS apps run inside the user’s real world environment in front of them.