Designing for visionOS & Accessibility

Transcript

Demo: SwiftUI Accessibility API

In this demo, you’ll see a few ways to use the SwiftUI Accessibility API to customize labels and values, sort priorities, and hide or combine elements. You’ll use Xcode’s Accessibility inspector to examine iOS projects because, at the time of writing, the Accessibility inspector doesn’t work for visionOS apps.

Open the RGBullsEye app in the Starter folder.

Change the target’s Bundle Identifier, and set a Team.

If necessary, adjust the project’s iOS Deployment Target.

Connect your iOS device to your Mac and select it as the run destination.

Then, build and run the app on your device.

Turn on VoiceOver.

Swipe up with two fingers

VoiceOver says “R 3 question marks G 3 question marks B 3 question marks. R 127 grams 127 B 127”, then says the same thing for each of the three sliders: “0. 50 percent, adjustable. 255” before “Hit me. Button”.

Tap a slider

to hear “50% adjustable. Swipe up or down with one finger to adjust the value.” The swipe up/down slider increments are too large to get a high score. To control the slider more accurately, tap a slider to select it, then use the standard action gesture: Double-tap and hold the slider thumb until you hear three rising tones, then drag the slider in the usual way.

This app is an RGB version of our original BullsEye app: You adjust the three sliders to match the target color. Admittedly, it’s not the most obvious app for a vision-impaired user. However, it’s a good example for applying accessibility fixes because the default labels and element ordering don’t make the app understandable, interactable, and navigable.

Now, activate Hit me to show the alert.

Swipe right to hear your score:

Then swipe right again to move to the button.

There are two problems with this alert:

1. You must swipe right to hear your score, which is the most important information, then again to select the OK button.

2. The target Text view now shows the target’s color values, but there’s no way to get VoiceOver to read them.

In Xcode, stop the app and change the run destination to a simulator.

Refresh the ContentView preview,

set the preview to Selectable,

then select the ZStack

and show the Accessibility Inspector.

Here are all the labels you heard VoiceOver read out.

Final Version

To see some accessibility fixes, open the RGBullsEye app in the Final folder.

Leave the starter app open to compare it with the final version.

Build and run this final app on your device so you can listen to VoiceOver while you check out the fixes.

In Xcode, stop the app and change the run destination to a simulator.

In the final app’s ContentView, refresh the preview.

And set mode to Selectable.

Now, select the first BevelText at line 55,

and open the Accessibility Inspector.

I’ve arranged the project windows so you can see the starter app’s Accessibility inspector too.

In the starter, also select the first BevelText at line 55:

And here are the two versions side by side.

The final app BevelText has an accessibilityLabel that overrides the default text label, translating “???” to something that makes sense.

// BevelText(text: "R ??? G ??? B ???", ...)
.accessibilityLabel("Target red, green, blue, values you must guess")

The comma after “blue” isn’t grammatically correct, but it makes VoiceOver pause before saying “values”.

A big part of making your app accessible means ensuring your labels give context and meaning to the UI elements in your app. You can fix many problems by replacing the default label with a custom label.

Now, compare the BevelText below the second ColorCircle — line 69 in the final app:

and line 68 in the starter app:

The guess color needs a few computed variables to enable VoiceOver to say “Red”, “Green”, and “Blue” instead of “R”, “G” (or “grams”), and “B”.

This happens in Model/RGB, where intString becomes:

var rInt: Int {
  Int(red * 255.0)
}
var gInt: Int {
  Int(green * 255.0)
}
var bInt: Int {
  Int(blue * 255.0)
}

/// A String representing the integer values of an RGB instance.
var intString: String {
  "R \(rInt) G \(gInt) B \(bInt)"
}

var accString: String {
  "Red \(rInt), Green \(gInt), Blue \(bInt)."
}

Computed variables for the red, green and blue integer values

appear in ContentView, in the intString displayed on screen,

and are spoken in the accessibility label’s accString.

BevelText(
  text: guess.intString,
  width: proxy.size.width * labelWidth,
  height: proxy.size.height * labelHeight)
  .accessibilityLabel("Your guess: " + guess.accString)
  .accessibilitySortPriority(2)

The new Sort Priority attribute comes from the accessibilitySortPriority modifier. By default, when the app launches, VoiceOver starts reading from the top of the screen. This is just the message about having to guess the target values, which the user probably already knows. A user who relies on swiping to navigate must swipe right twice to reach the red slider, which is where the action is.

For someone playing this game, a more useful navigation order is to start with the sliders,

then move to the guess string,

and then to the button.

The guess string has sort priority “2”, which is a low priority,

so VoiceOver will read this element after reading any elements with sort priority greater than 2. The color sliders now have sort priority 5, 4, and 3.

Compare the first ColorSlider — line 75 in the final app

and line 72 in the starter.

In the final version, this element has sort priority 5.

This is the highest priority in ContentView, so VoiceOver will read it first. In addition, the “0” and “255” labels are now hidden as they don’t add any useful information.

And finally, the slider value has been translated from a value between 0 and 1 — a percentage —

into an integer between 0 and 255 and identified with the color of the slider.

This magic happens down in the ColorSlider code:

Text("0")
  .accessibilityHidden(true)
Slider(value: $value)
  .accentColor(trackColor)
  .accessibilityValue(
      String(describing: trackColor) + " " +
      String(Int(value * 255)))
Text("255")
  .accessibilityHidden(true)

Color conforms to the CustomStringConvertible protocol, so String(describing: trackColor) is “red”, “green”, or “blue”.

The trackColor parameter

is passed to ColorSlider from ContentView:

ColorSlider(value: $guess.red, trackColor: .red)

Now, what happens when you activate Hit me?

A quick fix would be to include the score in the alert’s title:

title: Text("You scored \(game.scoreRound)"),

and present the target color values in message.

message: Text("Target values: " + game.target.intString),

In the final version, you would use accString instead of intString so VoiceOver could read “Red”, “Green”, and “Blue” instead of “R”, “G”, and “B”. But VoiceOver still can’t access the now-revealed target color values.

Maybe you could combine the three parts of the alert into a single accessibility label, then add the target color values as an accessibility value or hint? Unfortunately, you can’t use accessibility modifiers with the SwiftUI Alert view. UIAlertController can set its view.accessibilityLabel and view.accessibilityValue, so one solution would be to use this instead of Alert. That would require integrating UIKit into this SwiftUI app.

But here’s a situation where you can change the UI to benefit all your users.

It turns out, the original Figma design for RGBullsEye actually has a full-screen SuccessView modal sheet instead of the Alert. I didn’t implement it in the original RGBullsEye because it would’ve covered the guess and target color values, and the design didn’t include this information in the modal. But, if the alert can include the target color values, so can SuccessView. And it can also show the user’s guess color values.

SuccessView displays the target and guess color values on the backgrounds of those colors.

Thanks to the nifty computed variable accessibleFontColor from an older version of Apple’s Scrumdinger app, the text colors are black or white, depending on the background colors.

This code is in Model/ColorExtension.

So, back in ContentView, SuccessView replaces the alert:

.sheet(isPresented: $showScore) {
  SuccessView(
    game: $game,
    score: game.scoreRound,
    target: game.target,
    guess: $guess)
}

Now, the information includes the target color values.

One last thing: The advantage of using a modal sheet instead of an Alert is that Text views can now be combined into a single readout.

The outer VStack has this modifier:

.accessibilityElement(children: .combine)

All of the text elements are useful, so this combines them to make VoiceOver read them without stopping after each one. And each component can have accessibility modifiers.

The step-by-step instructions for these accessibility fixes are in SwiftUI by Tutorials “Chapter 12: Accessibility”.