Chapters
Chapters

Chapters

Hide chapters

Apple Augmented Reality by Tutorials

Second Edition · iOS 15 · Swift 5.4 · Xcode 13

Before You Begin

Section 0: 5 chapters
Show chapters Hide chapters

Section I: Reality Composer

Section 1: 5 chapters
Show chapters Hide chapters

Section II: Reality Composer & Image Tracking

Section 2: 2 chapters
Show chapters Hide chapters

Section III: Reality Composer & Object Tracking

Section 3: 1 chapter
Show chapters Hide chapters

Section IV: RealityKit & Face Tracking

Section 4: 3 chapters
Show chapters Hide chapters

Section V: ARKit & SpriteKit

Section 5: 3 chapters
Show chapters Hide chapters

Section VI: ARKit & SceneKit

Section 6: 2 chapters
Show chapters Hide chapters

Section VII: ECS & Collaborative Experiences (Bonus Section)

Section 7: 1 chapter
Show chapters Hide chapters

Section VIII: Creating 3D Models With Object Capture (Bonus Section))

Section 8: 2 chapters
Show chapters Hide chapters

18. Creating 3D Models With Object Capture
Written by Chris Language

Heads up... You're reading this book for free, with parts of this chapter shown beyond this point as scrambled text.

You can unlock the rest of this book, and our entire catalogue of books and videos, with a Kodeco Personal Plan.

Unlock now

Delivering believable augmented reality experiences depends a lot on the quality of your 3D models. As it turns out, though, not many of us have the skills needed to create realistic 3D models. Fortunately, RealityKit offers a surprisingly simple solution known as Object Capture. Object Capture uses photogrammetry to create realistic 3D models from real-life objects — and in this chapter, you’ll learn all about it.

What is Photogrammetry?

Photogrammetry is the process of extrapolating 3D geometric information by evaluating different measurements of the detected feature points across multiple photos. Wow, that’s a lot to process! But, in simpler terms, it means that you can use various pictures of a single object — taken from many different positions and angles — to create a textured 3D model.

Preparing for Object Capture

Before you start snapping pics and converting real-life objects into their digital counter-parts, you’ll first need to build yourself a photo booth. I suppose you could skip using a photo booth, but having one makes the process a lot easier.

Building a Photo Booth

To build a photo booth on a budget, you need only a few things:

Taking Photos

Although RealityKit’s Photogrammetry framework makes it possible to convert real-world objects into 3D models, taking pictures of those objects requires patience — and can sometimes get tricky. Here are a few things you can do to help:

Creating a Photogrammetry App

Let’s look at a practical example. An excellent place to start is with Apple’s very own photogrammetry example: Creating a Photogrammetry Command-Line App

Checking Supported Hardware

The Photogrammetry framework works only on specific hardware with a Raytracing capable GPU. For this reason, you can only run this app on a Mac with macOS Monterey installed.

private func supportsObjectReconstruction() -> Bool {
  for device in MTLCopyAllDevices() where
    !device.isLowPower && 
	device.areBarycentricCoordsSupported &&
    device.recommendedMaxWorkingSetSize >= UInt64(4e9) {
    return true
  }
  return false
}

private func supportsRayTracing() -> Bool {
  for device in MTLCopyAllDevices() where 
    device.supportsRaytracing {
    return true
  }
return false
}

PhotogrammetrySession Configuration

The magic component that does all the work is the PhotogrammetrySession. Before you can start a request, you first need to configure the session:

typealias Configuration = PhotogrammetrySession.Configuration

PhotogrammetrySession Request

With the configuration in place, you can now create the photogrammetry request:

typealias Request = PhotogrammetrySession.Request

var maybeSession: PhotogrammetrySession? = nil
do {
  maybeSession = try PhotogrammetrySession(
    input: inputFolderUrl,
    configuration: configuration)
      logger.log("Successfully created session.")
} catch { ... }

Monitoring Session Output

While the session is running, you can monitor the session.outputs for any information:

for try await output in session.outputs {
  switch output { ... }
}

Testing the Photogrammetry App

All right, enough about the code; let’s take this app for a spin!

HelloPhotogrammetry.main([
  "/Users/<YourName>/Desktop/FruitCakeSlice",
  "/Users/<YourName>/Desktop/FruitCakeSlice.usdz",
  "--detail", "preview",
  "--feature-sensitivity", "normal",
  "--sample-ordering", "sequential"
])

[HelloPhotogrammetry] Processing is complete!

Introducing PhotoCatch

Not a fan of command-line apps? No problem! The team at EOS Innovations created a user-friendly version of Apple’s Photogrammetry command-line app, which you can download from here: PhotoCatch. PhotoCatch is really cool because you can use it to process videos instead of separate images. Let’s try it out.

Key Points

Congratulations, you just became a master artist capable of creating hyper-realistic 3D models in just a few basic steps.

17. ECS & Collaborative Experiences
19. Conclusion
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2024 Kodeco Inc.

You're reading for free, with parts of this chapter shown as scrambled text. Unlock this book, and our entire catalogue of books and videos, with a Kodeco Personal Plan.

Unlock now