8. Object Anchors

Written by Chris Language

In this chapter, you’ll learn about object anchors. Object anchors are like 2D image anchors with an added third dimension. To understand how to use object anchors, you’ll augment a physical toy truck with interactive buttons that will reveal some fun facts about its real-world counterpart.

Crafting a toy truck

For this project, you’ll need a physical object that you can use to create an object model. Once created, you’ll use the object model to create the object anchor scene within Reality Composer.

As it turns out, physically attaching a toy truck to this book wasn’t possible. But don’t worry, we found a solution.

FoldUpToys offers a massive collection of foldable paper toys, and you’ll use one of them for this tutorial. Follow this link and see for yourself: https://bit.ly/395UELW

Download the Tesla Cybertruck project and print the blueprint.

With your printout in hand, grab your scissors and glue — it’s time to build yourself a paper truck!

Depending on the level of your crafting skills, your paper truck will look something like this:

What a beauty! Ready for a test drive, so to speak?

What are object anchors?

Reality Composer offers another anchor type known as an object anchor. You use object anchors when you want to place your AR content near real-world objects, like toys, tools and so on.

First, you need to create a 3D scan of the real-world object. You then store that scanned data within an .arobject file, which you use as the input for the Reality Composer project.

Reality Composer automatically detects real-world objects based on the scanned data you provide. Once it detects a suitable object, it anchors your AR content to it.

Installing the 3D object scanner

Before you can do anything, you need to install ARKit Scanner on your iPhone. You’ll use this app to create 3D scans of physical objects.

Unfortunately, ARKit Scanner isn’t available on the App Store, so you’ll need to download the source code, compile it, and then install it onto your iPhone.

Start by downloading the source code from https://apple.co/32uF8qF.

In addition to the source code, you’ll find information about Scanning and Detecting 3D Objects with ARKit. For now, you’re only interested in the download.

You’ll see the following page:

Click the Download button to download the source code for ARKit Scanner.

A zip file named ScanningAndDetecting3DObjects.zip will download. Once it’s done, extract the contents to a destination of your choice.

You’ll see a folder named ScanningAndDetecting3DObjects. Within that folder, you’ll find an Xcode project named ScanningApp.xcodeproj.

Open ScanningApp.xcodeproj in Xcode.

In the Project navigator, select the root ScanningApp, then update the Signing & Capabilities with your team’s information. Connect your iPhone to your Mac and build the app to your iPhone.

You now have ARKit Scanner installed on your device, and you’re ready to scan.

Preparing for 3D object scanning

For the data to represent the target object well, you need a good scan. To get the best scan possible, follow these guidelines for preparing your physical environment:

• Lighting Intensity: You should light the object with an illuminance of 250 to 400 lux.

• Lighting Direction: Light the object from all directions, casting as few shadows as possible.

• Lighting Temperature: Provide a light temperature similar to that of daylight, which is around 6,500 kelvin. Avoid warm or colored lights.

• Background: For the best results, place the object in front of a textureless, matte, middle-gray background.

To prepare your environment, put four large pieces of white paper on a table, then place the toy truck in the middle. This forms a neutral, textureless background for your target object from all viewable angles. Make sure the room is brightly lit and that the target object is lit from multiple directions.

The resulting environment should look similar to this:

You’re ready to scan the object!

Scanning 3D objects

At this point, you’ve installed the ARKit Scanner app on your iPhone, the environment is set, and the toy truck is ready for you to scan.

Getting started

Start ARKit Scanner on your iPhone and follow the step-by-step instructions provided on-screen.

Here’s an overview of the process, going from left to right.

The 1st Image:

• The app shows a Ready to scan status at the top. This means that tracking is good, and the app is ready to scan the object.

• Start by pointing the phone’s camera at the toy truck.

The 2nd Image:

• Move the camera to the left side of the toy truck. Yellow dots, known as feature points, will appear as you move. This means the scanner’s starting to detect the object and collect trackable feature points. When the Next button turns blue, ignore it for now. You want to collect as many initial feature points as possible.

The 3rd Image:

• Move the camera to the right side of the toy truck. The scanner will continue to collect feature points.

• The scanner is now ready to create an approximate bounding box around the scanned object based on the feature points it collected.

Refining the bounding box

You’re ready to place the approximated bounding box. Tap Next to continue.

Continue the process, moving from left to right.

The 1st Image:

• The app shows a Define bounding box status at the top. It created an estimated bounding box around the toy truck based on the initial feature points.

• Rotate and align the bounding box with the toy truck. You can use basic gestures. For example, you can rotate the bounding box by twisting two fingers, and you can move it using one finger.

The 2nd Image:

• With one finger, long-press on the front of the bounding box. Four rays will appear to indicate that you can now adjust the front side.

• Still holding your finger on the front side, slide it forward or backward to refine the bounding box so that it covers the entire toy truck.

The 3rd Image:

• Follow the same process as before and adjust all sides of the bounding box. Make sure the toy truck is fully covered by the bounding box — from back to front, from top to bottom, and from side to side. Also, make sure it’s nicely centered.

Note: While adjusting the sides of the bounding box, you’ll see a pop-up at the top showing its exact width, height and length.

Scanning all the sides

Once you’re sure that the bounding box fully covers the toy truck, tap Scan to continue.

Continue the scanning process, going from left to right.

The 1st Image:

• The app shows a Scan (%) status at the top. This indicates the current completed scanned percentage, which will increase as you scan all sides.

• Start with the top and move the camera over it until that part of the box turns yellow. This indicates you’ve completed that area of the scan. Notice that the scan percentage increases after each completed area.

• Continue to scan the entire top side. The whole side will turn light yellow when you’ve completed the side.

The 2nd Image:

• Move to the right side and follow the same process, making sure you move the camera until the side turns yellow.
• Continue to scan the entire right side. It will turn light yellow once done.

The 3rd Image:

• Move to the front side and follow the process again until the side turns yellow.

• Continue to scan the left and back sides until you’ve completed 100% of the scan.

Adjusting the origin point

Once you’ve scanned 100% of the object, the scanner will automatically finish and place an estimated origin point on the object.

You now need to adjust the origin point until it’s grounded and in the center of the scanned object. Again, you’ll start with the left image and move right.

The 1st Image:

• Move to the left side of the toy truck, then move the origin point until it looks like it’s in the center of the object. You can push and pull each axis by pulling on the ball-points. Green is the Y-Axis, Red is the X-Axis and Blue is the Z-Axis.

The 2nd Image:

• Move over the front side and continue to adjust the origin point until it’s positioned in the middle.

The 3rd Image:

• Move over the right side and continue to adjust origin until it feels perfect.

• Repeat this process while moving around the toy truck from all possible viewing angles.

Once you’re happy with the origin point, tap Test to continue.

Note: If you have a virtual model of the object you’re scanning in USDZ format, you can load the model and preview it by tapping Load Model. You won’t do that in this tutorial, however.

Testing and improving 3D scans

Now that the scanning process is complete, the only thing left to do is to test the object model.

You should test the detection from different angles. Move the truck to different environments and see if ARKit Scanner detects it there, too.

The 1st Image:

• First, do a basic test from one angle. A notification will appear once ARKit Scanner successfully detects the object. Detected after indicates how long it took to recognize the object. Anything less than one second is acceptable.

The 2nd Image:

• Now, move to a different angle. You should see another notification that ARKit Scanner detected the object successfully. Great!

The 3rd Image:

• Turn the light on with the toggle button at the bottom-left. Continue testing from various angles to make sure shadows didn’t cause any issues with the scan.

Ok, great, you’ve scanned the object and tested the results. If the detection took too long, or the recognition wasn’t consistent, you should re-scan the object.

When you’re done, store the scanned data by tapping Merge Scans… at the top-left.

When asked to Merge another scan, select Merge New Scan…. This will store your current scanned data and start a new scanning process. Simply follow the previous instructions to create another scan.

Once you complete the new scan, it will automatically merge with the previous one.

You’ll see an on-screen notification that the merge was successful. Multiple scans improve the overall quality of the object model. When testing, you’ll see an improved detection time.

Exporting AR objects

At last, you’ve scanned your object, tested the scan and even improved the model by merging multiple scans. Your next step is to save the scanned data as an ARObject.

Tap the Share button at the bottom, then AirDrop the .arobject file to your Mac.

Once the file finishes copying to your Mac, rename it TeslaCybertruck.arobject.

You now have an ARObject reference file ready to go. The only thing left to do now is to create the AR experience.

Creating object anchors

You should be familiar with this part by now, so instead of boring you with the details, we’ve supplied a ready-made AR experience. All you need to do is add your freshly-created ARObject file.

Open starter/TeslaCybertruck.rcproject in RealityComposer.

It’s a very basic AR experience with four interactive spheres that show some interesting facts about the toy truck. Spend a few minutes playtesting the experience within RealityComposer.

Currently, the experience uses a Horizontal Anchor. Change it to use an Object Anchor instead.

Once that’s done, click Choose next to the AR Object Asset label and import the AR object file from starter/resources/TeslaCybertruck.arobject. Click Import to complete the process.

You’ve now imported and set your scanned AR object file. You’ll see a yellow cube with an image of the scanned object on it.

You’ve successfully created an Object Anchored AR experience.

Export the AR experience by selecting File ▸ Export ▸ Export Project…. Rename the experience TeslaCybertruck.reality, then click Export to complete the process.

AirDrop the TeslaCybertruck.reality file to your iPhone and test it.

The scene recognizes the toy truck and displays four spheres around it. Tapping a sphere reveals some interesting facts about the toy truck.

Key points

Congratulations, you’ve finished this chapter, and you’ve successfully created an Object Anchor AR Experience.

Here are some of the key things you achieved in this chapter:

• Object Anchors: You learned about Object Anchors, which allow you to anchor AR content to physical objects in the real world.

• ARKit Scanner: You compiled and installed ARKit Scanner onto your iPhone so you can scan your own physical objects.

• Scanning Preparation: You learned how to prepare your environment so that you can scan physical objects. You now know the importance of lighting intensity, direction and temperature.

• Scanning Objects: You went through the process of scanning a 3D object. First, you defined a basic bounding box, then you scanned the object, and finally, you adjusted the origin point.

• Improving Scans: You improved a scan by merging multiple scanned data to create an even better AR object.

• AR Object files: You exported an AR object file from ARKit Scanner, then imported it as reference for an Object Anchor AR experience.

Take a well deserved breather. See you in the next chapter where you get to learn more about creating your own AR apps with RealityKit.