4. USDZ Files & USD Python Tools

Written by Chris Language

In the previous chapters, you explored the power of AR Quick Look. You also learned how to create your own AR experiences with Reality Composer using models from the built-in content library. But what about creating AR experiences with your own custom content?

The key technology behind AR Quick Look and Reality Composer is a file format known as USDZ.

In this chapter, you’ll learn about USD and the USDZ file format. You’ll also learn how to convert your own virtual content into this universal format using Python-based command-line tools. This knowledge will empower you to create your own USDZ content for both AR Quick Look and Reality Composer.

What is USD?

Before getting to USDZ, you need to understand the technology behind it: Universal Scene Description (USD). This is a universal format to exchange 3D content. Pixar developed it to improve the graphics and animation workflow pipeline of large-scale animation productions.

USD has been under development since 2016. It’s one of the core technologies behind stunning animation blockbusters like “Finding Dory” and “Cars”.

Since Pixar open-sourced the technology, it’s become an industry standard. Many of the big players within the 3D graphics and animation industry now back it.

USD features

USD comes with an onslaught of great features. Here’s a shortened list of just some of its key features:

• Robust schemas for the interchange of geometry, shading and skeletal deformation.
• High-performance data retrieval, object rendering and instancing.
• Ability to natively package user-selectable content variations.
• Future-friendly, flexible architecture designed to easily adapt to changing requirements.
• Powerful composition engine built with a focus on speed, scalability and collaboration.
• Supports Live Composition, Scalable to Complex scenes and Scene Graph.

USD is essentially a universal 3D file format. It’s a C++ library that can read and write USD files with Python bindings.

There are three file extensions mainly associated with USD:

• USDA: Plain text file designed to be human-readable and easy to understand.
• USDC: Binary version of the USDA file, designed to be as efficient as possible.
• USD: Can be either a text file or a binary file.

Note: You can easily convert USDA into USDC and vice-versa.

What is USDZ?

At WWDC 2018, Apple introduced AR Quick Look along with the new USDZ file format. As you know by now, USD stands for Universal Scene Description; the added Z merely indicates that it’s a ZIP archive.

USDZ features

A number of critical features make USDZ an excellent choice for 3D content delivery:

• USDZ is essentially a distribution format for USD.
• USDZ contains all of the related files for a specific USD scene, packaged nicely in a single ZIP archive.
• USDZ is optimized specifically for sharing. It forms the basis behind AR Quick Look, which is supported on iOS, macOS and tvOS.
• USDZ supports Scalable to Complex scenes and Scene Graph.
• USDZ is an uncompressed archive that has 64-byte boundary-aligned files. This is the most efficient format for high-performance memory mapping.

A USDZ archive contains two types of files:

• Scene Description Files: These can be USD, USDA, USDC or even USDZ files.
• Texture Files: These can be PNG or JPEG image files.

USDZ tools

At WWDC 2019, Apple announced a new set of tools to help developers create their own USDZ files. These tools are known as USDZ Tools or the USD Python Tools.

USDZ Tools is a pre-compiled Python library containing binaries of Pixar’s USD library for macOS. The tools allow you to convert, validate, generate and inspect USDZ files.

In the next section, you’ll learn how to convert a 3D model into the USDZ file format.

Installing USDZ tools

Your first step is to download the USDZ Tools: https://apple.co/36TN9WJ

Note: To download these tools, you need to log into the Apple Developer site using your Apple Developer login credentials.

When you open the above link, you’ll see the following download page:

Expand the latest available version, then select the download link on the right:

Note: At the time of writing, the latest available version is USDPython 0.64, released on April 24th, 2020.

Once the ZIP file downloads, extract it. After it’s extracted you should see the USDPython-0.64.pkg installer. Double-click the installer and follow the installation instructions to get the tools installed.

Once done, find and open Applications/usdpython in Finder. You’ll see the following contents.

Here’s what this is all about:

• LICENSE: The legal information about the tools.
• README.md: A detailed description of all the files contained in the archive. It’s a good idea to read it.
• samples: A set of sample Python scripts that demonstrate how to use the USD Python library.
• USD: The precompiled Python library plus a suite of command-line tools from Pixar. You can use these tools to validate, check and inspect your USDZ files.
• USD.command: A command file that will set up the required environment variables for you.
• usdconvert: Contains the USDZ Converter tool and another special tool for fixing opacity issues.

Right-click the USD.command file and select Open. If you see the following security message, click Open.

A Terminal window opens with a prompt like:

That’s it, you’re ready to go!

Reviewing the project

For this example, you’ll use a cool AR drum set Reality Composer project. Open starter/ARDrumSet.rcproject in Reality Composer, then press the Play button to test it.

Click on any of the objects to hear how they sound. Nice!

Hey, wait a minute! There’s a crucial piece missing: The kick drum is missing, which brings you to your next section.

Exporting 3D models

The first thing to do is to export the kick drum from the original 3D composition in a format that USDZ Tools supports.

Note: For this example, you’ll use Blender 2.8x to demonstrate a few important steps required to export your 3D models. You can apply this knowledge to any of the other modern 3D authoring tools. If you don’t have Blender installed, feel free to skip this section.

Start Blender 2.8x and load the 3D model located at starter/ARDrumSet.blend.

Select the kick drum by left-clicking it. Press the Backslash key to isolate the drum from the rest of the composition.

Look at the object transform information. Note that the drum has both a rotation and a scale applied to it.

It’s a good idea to export your 3D objects with a zero rotation and scale of 1 transform because they are generally easier to work with at this size and with no rotation.

To fix this, make sure you’ve selected the drum, apply the current rotation and scale by pressing Control-A, then select Apply ▸ Rotation & Scale.

You’ve set the rotation to 0 and the scale to 1.

Perfect! You now need to fix the origin point of the object. You’ll use the 3D Cursor to set it, so first make sure you’ve positioned the 3D Cursor at the World Origin point.

Press Shift-S to bring up the Snap pie menu, then select Snap ▸ Cursor to World Origin.

Done! Press 3 to switch into side orthographic view.

Press G to grab the object, then hold Control while moving the drum to the right until it centers with the World Origin.

Nicely done! Right-click the object to display the Object Context Menu. Select Set Origin ▸ Origin to 3D Cursor to move the object origin to the same location as the 3D Cursor.

Excellent, your object is now ready to export!

With the drum still selected, go to File ▸ Export ▸ glTF 2.0 (.glb/.gltf) to bring up the export options.

The following export options will appear:

Set the export location to Desktop. Then go through the export options and check the following settings:

• Include ▸ Selected Objects. Now, only the selected model will export.

• Transform ▸ +Y Up. Converts Blender +Z Up to +Y Up, which is what Reality Composer uses.

• Geometry ▸ Apply Modifiers. Applies all necessary modifiers to the object geometry.

• Geometry ▸ UVs. Includes UV mapping information.

• Geometry ▸ Normals. Includes Normal information.

• Geometry ▸ Vertex Colors. Includes Vertex color information.

• Geometry ▸ Materials. Includes Material information.

Set the file name to Kick, then select Export glTF 2.0 to finalize the export.

You’ve just exported the kick drum. You now have a file named Kick.glb on your Desktop. You can preview the file in Finder.

Fantastic!

Note: Find the resulting exported file under starter/Kick.glb. For the sake of simplicity, this book assumes you’ve copied the Kick.glb file to your Desktop. Please make sure to adjust the paths accordingly if you’re using a different path.

Converting 3D models to USDZ

It’s time to convert Kick.glb into a USDZ file. To do this, jump back to the open Terminal window from the first section and execute the following command-line instructions:

cd /
cd Users/<YourUserName>/Desktop

This sets your active working directory to the same location as the Kick.glb file on your Desktop. Now, you don’t have to worry about dealing with paths when providing parameters. You’ll simply work out of the currently-active directory.

Using usdzconvert

For your next step, you’ll use the usdzconvert tool. To find out more about the available options, execute the following command-line instructions to request detailed help information:

usdzconvert -h

This gives you a detailed list of options you can provide as input for usdzconvert.

Execute the following command-line instructions:

usdzconvert Kick.glb -v Kick.usdz

This tells the converter that you’re providing Kick.glb as input and to produce Kick.usdz as output. -v instructs the converter to show verbose output.

The resulting output looks like this:

The converter tool receives an input file named Kick.glb containing a single mesh called Kick. It then produces an output file named Kick.usdz. It finally checks the output file for any issues with usdARKitChecker. It found no issues, so the output file passed with flying colors.

That’s it, you’re done! You just created your first USDZ file.

Converting USDZ to USDA

To see what’s inside the USDZ file, you need to convert it to a USDA file, which is the plain text representation of the USDZ.

Using usdcat

To do this, you’ll use usdcat. This tool converts USDZ into USDA files.

To find out more about it, execute the following command-line instruction to request detailed help information:

usdcat -h

This gives you a detailed list of options that you can provide as input for the usdcat tool.

Execute the following command-line instructions:

usdcat Kick.usdz -o Kick.usda

The first parameter specifies the input file name as Kick.usdz. The -o specifies the output file as Kick.usda.

That’s it, you’ve converted Kick.usdz to Kick.usda. You can now open the USDZ file with a normal text editor to see what the code looks like.

Inspecting and validating USDZ

Now that you’ve converted your 3D model into USDZ, you can inspect and validate the USDZ file.

Using usdtree

To get a high-level overview of the model hierarchy, you’ll use the usdtree tool.

To find out more about this tool, execute the following command-line instruction to request detailed help information:

usdtree -h

This gives you a detailed list of options that you can provide as input for the usdtree tool.

Execute the following command-line instructions:

usdtree Kick.usdz

The Kick.usdz parameter specifies the input file name.

The resulting output looks like this:

This is a quick way to get an overview of your model hierarchy.

Using usdchecker

Finally, to validate the generated USDZ and to make sure the file is compliant, you’ll use the usdchecker tool.

To find out more about this tool, execute the following command-line instructions to request detailed help information:

usdchecker -h

This gives you a detailed list of options that you can provide as input for the usdchecker tool.

Execute the following command-line instructions:

usdchecker Kick.usdz -v

The Kick.usdz parameter specifies the input file name. Finally, the -v parameter specifies verbose output.

The resulting output looks like this:

The file is validated and checked. Excellent, your USDZ file passed the check with flying colors!

Importing USDZ assets

Now that your USDZ file is ready to go, open starter/ARDrumSet.rcproject again with Reality Composer.

Yes, you’ve already established that the kick drum is missing. You’re about to fix that! :]

To import a new USDZ asset, select the Objects (+) button to access the built-in asset library. Then select the Import button at the top-right to add custom assets.

Find and select Kick.usdz on your Desktop, then select Import to complete the process.

This imports the kick drum into the active scene, but to the right of the other drums. To position it properly, make sure you still have the kick drum selected, then open the Properties panel. Set the Position to (X: 0cm, Y: 0,24cm, Z: 0cm) and set the Rotations to (X: -180°, Y: 0, Z: 180°).

The kick drum is now present within the scene.

Your next step is to make it play like all the other drums. Select Behaviors at the top-right to open the Behaviors panel.

This is where you add interactions, animations and sound effects to all the virtual content in the scene.

Hang on, did you notice all those Alert signs?

The alerts tell you that there’s something wrong with the behavior. In this case, it’s because you haven’t connected the behavior to the kick drum yet.

To connect the Trigger Tap event, select Choose, then select the kick drum. The Choose button will change into a Done button. Select it to complete the connection.

Now that you’ve connected the event to the kick drum, the alert indicator disappears. Follow the same process for all the other alerts to connect them all to the kick drum.

There’s one final step: You need to reset all of the Position Actions so they don’t move the kick drum when the user taps it. Select the Reset Position button – the curly back arrow – to clear any positional information.

Now, when the user triggers the action sequence, it won’t affect the kick drum’s position.

Excellent, you’re all done! You can now Play the scene to test your awesome AR Drum Set. Feel free to export the scene as a Reality file, then send it to your iPhone to test it in augmented reality.

Key points

Well done, you’ve just reached the end of this chapter.

In this chapter, you learned:

• All about USD and USDZ.

• That it’s easy to get started with the new USDZ Tools, also known as the USD Python Tools, created by Pixar.

• How to prepare your 3D content for export by setting the model’s origin point, rotation and scale.

• All about converting 3D content into the USDZ format with the usdzconvert tool.

• How to convert USDZ into USDA with the usdcat tool.

• How to inspect USDZ files with usdtree and test them with usdchecker.

Where to go from here?

Here are a few recommended links you can explore to advance your knowledge on this topic:

• WWDC 2018 & 2019 Videos about USD: https://apple.co/2YmYX3B

• Pixar – Introduction to USD: https://bit.ly/2CeiYPK

• Pixar – USDZ File Format Specification: https://bit.ly/2wDDK5j

It’s super-easy to convert your own content into USDZ. The only thing left to do is to go forth and augment your world with your very own content!