16. Focus Nodes & Billboards

Written by Chris Language

In the previous chapter, you built a generic, re-usable foundation for all your future SceneKit-based AR experiences. The app operates in a few basic states and, as an added bonus, it also conforms to a standard onboarding process thanks to Apple’s AR Coaching Overlay view. This will make your users feel right at home when they pick up your app and play with the AR experiences you create.

In this chapter, you’ll continue to add more components to the reusable foundation. You’ll learn how to create and manage a focus node that helps the user know where content will be placed. You’ll also get to build the entire AR experience with some basic interaction.

Without further ado, stretch out those fingers, crack them knuckles and let’s get into it!

Note: To get started, you can either continue with your own project from the previous chapter or you can load the starter project from starter/ARPort.

Importing 3D Assets

In the previous chapter, you learned about the SceneKit Asset Catalog, which is a folder that your entire team of artists and developers can share. It keeps the graphics component of your app completely separate from the code. This allows you and your team to merge any graphical changes and additions into the app with minimal disruption.

Your first step to get started is to add the ready-made asset catalog to the project.

With your project open in Xcode on one side and Finder open on the other side, find art.scnassets inside starter/resources.

Drag and drop the art.scnassets folder into Xcode, placing it just above Assets.xcassets.

Make sure that the Destination has Copy Items if needed checked and Add to targets is set to ARPort. Select Finish to complete the process.

Excellent, you’ve successfully imported all of the 3D assets you’ll need to complete the AR experience.

Focus Nodes

The app already detects horizontal surfaces, so your goal now is to show the user exactly where on that surface they’re pointing. This is where a focus node comes in handy.

What is a Focus Node?

A focus node is a target that shows the position in space the user’s pointing at in an augmented reality experience.

To determine where to place the focus node, you use ray casting. A ray shoots from a focus point at the center position of the screen into augmented space. The app will place the focus node wherever the ray intersects with a previously detected surface.

Creating a Focus Point

Before you can create a focus point, you need to define the onscreen position to use for the ray cast. For this particular app, you’ll use the center point of the screen.

You’ll need to create a property to hold this center position so start by adding the following to the Properties section:

var focusPoint:CGPoint!

Then add the following function to the Focus Node Management section:

func initFocusNode() {
  focusPoint = CGPoint(x: view.center.x, 
    y: view.center.y + view.center.y * 0.1)
}

This is where you’ll initialize the focus node along with everything required to help manage it. You initialize the focus point at the center of the screen, with a slight 10% offset on the y-axis for a more natural feel. The ray will shoot from this screen position.

That’s simple enough, but what if the user changes the screen orientation? You’ll handle that issue next.

Handling Orientation Changes

To solve the problem, you need to update the focus point every time the user changes screen orientation.

Add the following helper function under the Focus Node Management section:

@objc
func orientationChanged() {
  focusPoint = CGPoint(x: view.center.x,
    y: view.center.y  + view.center.y * 0.1)
}

This essentially just readjusts the focus point. Note that the function has an @objc attribute. This makes the function available to the NotificationCenter, which is part of the Objective-C runtime.

Now, you need to call the helper function when the orientation changes.

Add the following code to the bottom of initFocusNode():

NotificationCenter.default.addObserver(self,
  selector: #selector(ViewController.orientationChanged),
  name: UIDevice.orientationDidChangeNotification,
  object: nil)

This notifies the app every time the orientation changes. It also calls the helper function, which updates the focus point to the correct position on the screen.

Creating a Focus Node

To create a new focus node, you first need to create a new SceneKit scene.

Right-click art.scnassets/Scenes and select New File.

This creates a new empty scene. Rename it to FocusScene.scn.

With FocusScene.scn still open, select and delete the camera node under Scene Graph.

Select the + symbol at the bottom-left to add an empty node, then name it Focus.

Add a new object from the Object Library, then search for Plane. Drag and drop a new Plane from the Object Library into the Scene Graph as a child of the Focus node.

Open art.scnassets/Textures/Focus, then drag and drop Footprint_DIFFUSE.png on top of the Footprint Plane within the scene.

Follow the same process as before and add another Plane called Shadow, also as a child of the Focus node.

With the Shadow node selected, open the Attributes inspector and set the Plane Size to (width: 0.15, height: 0.15).

Open the Node inspector and set the Transform Position to (x:0, y:0.001, z:0) so the shadow is slightly above the Footprint, preventing z-fighting. Then drag and drop the Shadow_DIFFUSE.png texture onto it.

To lighten up the shadow a bit, open the Material inspector and set the Settings Transparency Value to 0.75.

Again, follow the same process as before and drag yet another Plane as a child node of the Focus node, this time naming it Icon.

Set the Plane Size to (width:0.1, height: 0.1) and the Transform Position to (x:0, y:0.05, z:0).

Finish it off by assigning the Focus_DIFFUSE.png texture to it. Under the Materials inspector, set its Material Properties Shading to Constant, which prevents it from reacting to light.

Here’s how the result will look:

Adding Billboard Constraints

It would look really cool if the focus node always faced the user. To achieve that effect, you can use a billboard constraint. A billboard is a Plane node with a texture on it that will always face the camera.

With FocusNode.scn still open, select the Focus node, then open the Node inspector. Find the Constraints section, then select the + button to add a constraint. Select Billboard from the list.

Finally, you don’t want the billboard constraint to affect all of the axes, only the y-axis.

Under Constraints Settings Axes, uncheck both the x-axis and the z-axis.

Excellent, now the focus node will always face the same direction as the user.

Loading the Focus Node

Now that the Focus scene is ready to go, you need to load it and add it to the main scene.

Start by creating a variable to hold the focus node. Add the following variable to the Properties section:

var focusNode: SCNNode!

Add the following code to the top of initFocusNode():

// 1
let focusScene = SCNScene(
  named: "art.scnassets/Scenes/FocusScene.scn")!
// 2
focusNode = focusScene.rootNode.childNode(
  withName: "Focus", recursively: false)!
// 3
focusNode.isHidden = true
sceneView.scene.rootNode.addChildNode(focusNode)

Take a look at what’s happening here:

1. This loads the entire FocusScene.scn into a local variable.
2. Now that you have a freshly loaded scene, you’re only interested in the focus node. This code searches all the child nodes within the Focus scene for a node named Focus. Once found, it stores the node in focusNode.
3. Finally, you set the focus node’s default state to hidden before adding it to the main scene.

With the function in place, don’t forget to initialize the focus node when the app starts. To do this, add the following function call to the bottom of viewDidLoad():

self.initFocusNode()

Excellent, you’ve now initialized the focus node.

Updating the Focus Node

Now that the focus node is ready to go, you need some code to manage the node’s visibility.

Add the following function to the Focus Node Management section:

func updateFocusNode() {
  // 1   
  guard appState != .Started else { 
    focusNode.isHidden = true
    return
  }
  // 2
  if let query = self.sceneView.raycastQuery(
    from: self.focusPoint,
    allowing: .estimatedPlane,
    alignment: .horizontal) {
    // 3
    let results = self.sceneView.session.raycast(query)
    if results.count == 1 {
      if let match = results.first {
        // 4
        let t = match.worldTransform
        // 5
        self.focusNode.position = SCNVector3(
          x: t.columns.3.x, y: t.columns.3.y, z: t.columns.3.z)
        self.appState = .TapToStart
        focusNode.isHidden = false
      }
    } else {
      // 6
      self.appState = .PointAtSurface
      focusNode.isHidden = true
    }
  }
}

Quite a bit is happening in this update function:

1. For starters, the app should only update the focus node while the app’s in a Started state. If not, the focus node state should be invisible at all times.
2. This performs a ray-cast test that shoots a virtual ray from the focus point outward into augmented space. It’s also important to note that the ray-cast test will only consider intersections with estimated planes that have been identified as horizontal surfaces.
3. Once the test finishes, you’re only interested in the first hit result.
4. You use the hit result’s worldTransform, a transform that contains position, orientation and scale information.
5. Here, you update the focus node’s position based on the hit result transform. You can find the positional information in the third column of the transform matrix. At this point, you can make the focus node visible and change the app state to TapToStart.
6. Ultimately, if the ray-cast test had no hit results, the app should continue to instruct the user to point at a valid surface and the focus node should be kept in a hidden state.

Now, to keep the focus node updated at all times, add the following function call in renderer(:updateAtTime:):

self.updateFocusNode()

This ensures that the focus node updates once every frame.

To test the focus node, build and run.

After the onboarding process, pointing toward a horizontal surface makes the focus node appear and the app switches to TapToStart. When you point away from the surface, the focus node will hide and the app will switch to PointToSurface. Excellent!

Creating the Scene

Now that you know where you want to place your virtual content, it’s time to create some cool content to actually place. :]

Building the Scene

Create a new blank scene named ARPortScene.scn by right-clicking on the art.scnassets/Scenes folder and selecting New File. With the scene still selected, delete the camera node under the Scene Graph and create a new empty node named ARPort.

The ARPort node will act as the root node for the entire scene. You’ll add all the elements as children of this node.

Drag and drop art.scnassets/Models/Base.scn into the empty space of the Scene graph.

This adds the Base.scn scene as a reference node to the ARPort scene in a default position.

Finally, drag the Base reference node on top of the ARPort node, making it a child node.

With the Base node selected, open the Node inspector and set the Transform Position to (x: 0, y: 0.49, z:0) to place the Base node on top of the ground plane.

Now, follow the same process as you just did for the Base node and drag and drop the remaining nodes from art.scnassets/Prefabs into the ARPortScene.scn as children of the ARPort node.

Start with Buildings.scn, positioned at (x: 0, y: 0.1, z:0).

Now, you see the main airport terminal and the control tower with its big radar.

Add Planes.scn next, positioned at (x: 0, y: 0.1, z:0).

This adds four airplanes, one parked at each of the available terminals.

Do the same for SolarFarm.scn, FuelDepots.scn and Trees.scn, all positioned at (x: 0, y: 0.1, z:0).

Fantastic, your airport is coming along great. But hang on, isn’t something crucial still missing? Oh, of course, the runway, whoops! :]

Just like before, add Runway.scn and position it at (x: 0, y: 0.1, z:0).

Nicely done, what a slick-looking runway; there’s even a plane ready to take off.

But… you might notice the trees look a bit flat. That’s because the scene still needs lighting.

Adding Lights & Shadows

Create a new empty node as a child of ARPort and name it Lights & Shadows. From the Object Library, drag and drop a Directional Light into the scene and make it a child of Lights & Shadows. Rename it to DirectionalLight, too.

Position the light at (x: 0, y: 1, z:0) and set the Euler Rotation to (x: -75, y: 0, z:-40).

The trees should no longer appear flat. Instead, they’re nicely lit from above, making them bright at the top and dark at the bottom. However, there’s no shadow drop yet. You’ll fix that next.

With DirectionalLight still selected, open the Attributes inspector. Find the Shadow section and check Enable shadows to set the directional light to cast shadows. Also, set the Shadow Color to a 75% Transparency so the shadow isn’t a hard black color.

Adding a Shadow Catcher

To push the realism factor of your AR experience a bit, it would look amazing if the tall control tower would drop a shadow on top of the ground surface below it.

You can achieve this effect with something known as a shadow catcher. You’ll add one to the scene next.

From the Object Library, name a Plane ShadowCatcher and drag and drop it into the scene as a child of Lights & Shadows. Under the Attributes inspector, set the Plane Size to (width: 5, height: 5).

This creates a nice big, white plane that catches all of the shadows in the scene. One problem though: The plane is white, which will spoil the entire experience.

You’re only interested in the shadows — the rest of the plane should be transparent. SceneKit has a special shader for just such an occasion.

With ShadowCatcher still selected, open the Material inspector and change Properties Shading to Shadow Only.

The final result will look like this:

The big white plane is now transparent, but it’s catching the all-important shadows.

Loading the Scene

With the scene built, you now need to do two things: First, load the scene and then, when the user taps to start the AR experience, place the ARPort at the focus node’s location.

Add the following variable to the Properties section:

var arPortNode: SCNNode!

This creates a variable that will hold the loaded ARPort node.

Add the following block of code to the bottom of initScene():

// 1
let arPortScene = SCNScene(
  named: "art.scnassets/Scenes/ARPortScene.scn")!
// 2
arPortNode = arPortScene.rootNode.childNode(
  withName: "ARPort", recursively: false)!
// 3
arPortNode.isHidden = true
sceneView.scene.rootNode.addChildNode(arPortNode)

Well, that was simple and should look familiar. It’s pretty much the same process you used to load the focus node.

Nevertheless, here’s a closer look:

1. Start by loading the entire ARPortScene.
2. Then, search for the child node named ARPort within the scene.
3. Set its default state to hidden, then add the node as a child to the main scene.

Presenting the Scene

With the ARPort node ready and waiting to display, there’s one thing left to do: Display the node when the user taps the screen.

Add the following code to tapGestureHandler(_:):

// 1
guard appState == .TapToStart else { return }
// 2
self.arPortNode.isHidden = false
self.focusNode.isHidden = true
// 3
self.arPortNode.position = self.focusNode.position
// 4
appState = .Started

Here’s what this does:

1. The app has to be in a TapToStart state.
2. You then set the focus node to a hidden state and the Airport node to a visible state.
3. Next, you set the Airport node’s position to be the same as the focus node’s.
4. Finally, progress the app state to Started.

What time is it? It’s time to build and run that project!

Find a space big enough, then point at the floor and let the focus node guide you. Tap to start the AR experience and stand back and be amazed. Surprise, did someone forget to mention that the scene is animated? :]

Just look at that shiny plane coming in for a landing. Don’t neglect to notice its pretty shadow falling on the floor tile. Awesome!

Adding Interaction

Your app is shaping up nicely, and you’re almost done. But first, you’ll make it a little more useful by giving the user some elements to interact with.

When the user taps on certain elements, like the runway, for example, a billboard will pop up showing the user some fake departure and arrival information.

As you learned earlier in this chapter, a billboard is a Plane node with a texture on it that will always face the camera. The effect is achieved simply by adding a billboard constraint to the node, similar to what you did for the focus node earlier.

Adding the Billboards

To speed things up, there’s a ready-made scene for you to use.

With ARPortScene.scn open, drag and drop art.scnassets/Prfabs/Interaction.scn into the Scene graph, then make the node a child of ARPort.

You might wonder why you’re not seeing anything. That’s because everything is invisible. To see what the node and its elements look like, open art.scnassets/Prfabs/Interaction.scn.

In the Scene graph, under the Interaction node, you’ll see a whole bunch of Touch nodes. Select the first one and open the Material inspector. Then change Properties Diffuse to 50% transparency.

This will reveal primitive boxes and spheres that act as the interaction points. When the user touches any of these Touch nodes, you’ll unhide the child Billboard node.

Note: Don’t forget to go back and set Material Properties Diffuse back to 0% transparency when you finish testing.

Handling Touch Input

In ViewController.swift, add the following code under the Scene Management section:

override func touchesBegan(_ touches: Set<UITouch>, 
  with event: UIEvent?) {
  DispatchQueue.main.async {
    // 1
    if let touchLocation = touches.first?.location(
      in: self.sceneView) {
      if let hit = self.sceneView.hitTest(touchLocation,
        options: nil).first {
        // 2
        if hit.node.name == "Touch" {
          // 3
          let billboardNode = hit.node.childNode(
            withName: "Billboard", recursively: false)
          billboardNode?.isHidden = false
        }
    // 4
        if hit.node.name == "Billboard" {
          hit.node.isHidden = true
        }
      }
    }
  }
}

Here’s what’s happening:

1. This takes the first onscreen touch location, then performs a hit test to determine if any node has been touched in AR space. You’re only interested in the first node.
2. You’re only interested in nodes named Touch.
3. This locates the child node named Billboard and sets its state to visible.
4. If the node wasn’t a Touch node but it was a Billboard node, it means the user touched a visible Billboard and wants to dismiss it. You then simply set the Billboard node back to a hidden state.

Enabling Statistics & Debugging (Optional)

When dealing with problems, it’s extremely helpful to enable the scene statistics and debugging information.

Note: This step is an optional step you can use to debug ARKit and SceneKit scenes. Don’t forget to turn it off again when you’re done testing.

Add the following to the bottom of initScene():

// 1
sceneView.showsStatistics = true
// 2
sceneView.debugOptions = [
  ARSCNDebugOptions.showFeaturePoints,
  ARSCNDebugOptions.showCreases,
  ARSCNDebugOptions.showWorldOrigin,
  ARSCNDebugOptions.showBoundingBoxes,
  ARSCNDebugOptions.showWireframe]

Here’s what’s happening:

1. To enable statistics, you simply set showStatistics to true.
2. To debug a particular scene, just provide the list of debugging options as an array.

Do a quick build and run to test it out. You should notice a bar at the bottom of the screen with a little + symbol. Press it to open the SceneKit statistics panel.

Here’s what you see:

• At the top-left, a display of the current rendering technology shows Mt, short for Metal. Remember SceneKit was built on top of Metal, so that’s perfect.
• Next to that, you see the current frame rate. A frame rate of 60fps means that SceneKit is currently rendering the scene 60 times in a single second. If this number drops below 30fps, you should probably optimize the elements in your scene.
• The ◆ shows the total number of draw calls per frame.
• The ▲ shows the total polygons per frame.
• The big circle at the bottom-left shows the current frame time with a color legend of each component and their total time.

Adding the Final Touches

You’re basically done, there are just a few tiny housecleaning issues that need to be done to ensure you handle every situation correctly.

Add the following lines to startApp():

self.arPortNode.isHidden = true
self.focusNode.isHidden = true

This will ensure that both the focus node and the airport node start in a hidden state.

Add the following lines of code to resetApp():

self.arPortNode.isHidden = true

This checks that the airport node returns to a hidden state when the AR experience restarts.

There’s one final issue you need to resolve: The AR experience is simply way too big. You need to scale it down a bit so it will fit into the predicted footprint, as indicated by the focus node.

Open art.scnassets/Scenes/ARPortScene.scn and select ARPort in the Scene graph. Open the Node inspector and set the Transforms Scale to (x: 0.75, y: 0.75, z: 0.75).

This scales the entire AR experience down to 75% of its previous size. Now, it should fit on your dining room table!

Bam! Just like that, you’re all done. Do one final build and run to reap the rewards of your hard work.

You’ll notice that the AR experience is slightly smaller than before. Now, tapping the runway will show a pop-up with departure and arrival information. Tap the pop-up to dismiss it. Fantastic!

Key Points

Congratulations, you’ve reached the end of this chapter and section, and you’ve created a super cool AR experience using SceneKit with ARKit.

Before signing off, take a look at some final key points:

• SceneKit Asset Catalogs: It’s super easy to import 3D content into your SceneKit-based projects with an Asset Catalog. Best of all, the Asset Catalog is just a folder that can be shared, which keeps the code separate from the graphics.
• Focus Node: With basic ray casting, you can easily add a focus node to your AR experience, showing the user exactly what they’re interacting with.
• Billboards: Adding billboard constraints to nodes is child’s play. Now those nodes always face the camera.
• SceneKit Scenes: Scenes are simple to create and build. You can drag and drop primitive shapes from the Object Library, or you can reference other scenes with custom objects in them.
• Lights & Shadows: Adding lights to a scene brings that scene to life. Lights are especially important if you want the objects to cast shadows.
• Shadow Catchers: You can catch an object’s shadow with a basic Plane node that uses a special Shadows Only shader as a material.
• Presenting Scenes: SceneKit makes it easy to load a scene from the Asset Catalog. Displaying that scene is a simple as adding it to the main scene as a child node.
• Interaction: With the power of hit testing, you can quickly add scene interaction to any AR experience.
• Statistics & Debugging: When things don’t make sense, adding statistics and debugging information makes all the difference in finding weird bugs.

Go show your friends your awesome AR airport, but don’t forget to come back for the next and final project. This time around you’ll get to learn all about how to create collaborative AR experiences. See you there!