5. Perceivable — Time-Based Media & Cues

Written by Tori Gonda

Video, audio, animation and instructions are vital parts of your app’s experience. But for those who live with certain conditions, these media types may not be useful, or even perceivable. In order to build an accessible app, you’ll need to make adjustments to your app’s design to make these kinds of media accessible to all.

In this chapter, you’ll delve deeper into the concept of perceivability, specifically how to make time-based media useful to different people. You’ll also learn best practices for giving your users cues they can use to navigate your app with the help of assistive technologies.

Displaying time-based media

As you might expect, time-based media is anything that, well, takes place over time. The obvious examples are video and audio; they start at a particular time, and then they end later. Animations also fall into this category. There is only one way to consume these media types.

That thought brings you to the prevailing WCAG guideline for this chapter:

Guideline 1.2 Time-based Media Provide alternatives for time-based media.

There are many ways you can provide these types of media. For prerecorded audio, you can have on-screen captions. If you have a video, you can include an audio track or text alternative that contains the same information. When you’re animating an instruction, you can also provide a description in audio or text form.

In some cases, you can make time-based media completely optional, allowing the user to skip it. Be careful with optional settings though — you don’t want to keep people from accessing content that might be valuable.

Consider the guideline’s success criteria, which specify a heuristic that these elements need equivalent alternatives, for example, text, captions, or other form factors.

Consider this criterion:

Success Criterion 1.2.3 Audio Description or Media Alternative (Prerecorded): An alternative for time-based media or audio description of the prerecorded video content is provided for synchronized media, except when the media is a media alternative for text and is clearly labeled as such.

Level A

Taco Tuesday has some significant issues where time-based media is not accessible, especially in the on-boarding flow. Once again, you’ll improve the app so that you can learn.

Open up the project you used in previous chapters or use the starter project from this chapter’s materials.

Improving the on-boarding flow

Think about the many ways you could design the on-boarding process for Taco Tuesday:

1. You could have a video that shows the different actions you can take. But without an equivalent audio element to this video, your on-boarding flow will be inaccessible to people who cannot see the video. To address this, you can add an audio track to the video that describes the instructions or add text for the user to read.
2. You could have an audio track that follows some fun animations. But if the animations don’t convey the same information that the audio does, then your process will be inaccessible to those who can’t hear it. To make it accessible, you need captions or other text.
3. You could already have text to accompany your time-based media, but the media advances and the text vanishes faster than some people can read it. To make the flow more accessible, you could add controls so that the user can manage the pace.

While there are other options, Taco Tuesday currently suffers from the third issue. It has instructional text and pages of animated, self-advancing instructions that run regardless of whether the user is ready.

Build and run.

Exploring the on-boarding

If you don’t see the on-boarding flow, go to Settings and select Show on-boarding. Then close and reopen the app. You can do this anytime you want to see on-boarding again.

Another option is to comment out several lines in onCreate() under MainActivity.kt to ensure onboarding always runs:

//  val sharedPref = PreferenceManager.getDefaultSharedPreferences(this)
//  val showOnboarding = sharedPref.getBoolean("onboarding", true)
//  if (showOnboarding) {
   OnboardingActivity.startActivity(this)
//   finish()
//  }

With that, you’ve enabled the on-boarding flow to run every time. Remember to uncomment it when you’re done with this chapter, so you don’t find yourself annoyed by the constant on-boarding flow!

The on-boarding flow uses a pager where each page contains an image and some text. It auto-advances after five seconds, and there are no buttons to control it.

So if you didn’t read the description on time, you’re out of luck. No tacos for you! You might be clever enough to discover that you can swipe to go back. But consider that you must be physically able and curious enough to find and perform the swipe gesture — that’s not a great user experience, even without regard for accessibility.

Removing auto-advance

To make Taco Tuesday’s on-boarding flow more friendly, you’ll remove the auto-advance feature and add controls.

For this step, you’ll get to delete some code! Since the plan is to put the user in control of when on-boarding advances to the next page, you’ll remove the logic that makes it advance.

Open OnboardingActivity.kt. In onCreate() look for a coroutines block that starts with lifecycleScope.launch.

Delete the entire block of code:

lifecycleScope.launch(Dispatchers.IO) {
 val options = resources.getStringArray(R.array.pop_up_options)
 while (isActive) {
  delay(5000) // 5 seconds

  withContext(Dispatchers.Main) {
   if (binding.onboardingPager.currentItem == NUM_PAGES - 1){
    MainActivity.startActivity(this@OnboardingActivity)
    this.cancel()
   } else {
    binding.onboardingPager.currentItem++
   }
  }
 }
}

This code would wait five seconds then advance to the next page. If it was already on the next page, it would open the MainActivity. But now it is gone.

Build and run. Use the settings so that you can view the on-boarding flow. Note how the auto-advancing feature is gone, and you have as much time as you need to view the instructions.

Now that you’ve had fun deleting somebody else’s code, it’s your turn to add code.

Adding controls

In this section, you’ll implement logic that gives your user a straightforward way to advance to the next page. First, you’ll add the layout for a Next button.

Open activity_onboarding.xml. Add the following view to the bottom of the constraint layout:

<com.google.android.material.button.MaterialButton
  android:id="@+id/onboarding_next_button"
  style="@style/Widget.MaterialComponents.Button.TextButton"
  android:layout_width="wrap_content"
  android:layout_height="wrap_content"
  android:padding="@dimen/space_normal"
  android:text="@string/onboarding_next"
  android:textColor="?colorOnPrimary"
  app:layout_constraintBottom_toBottomOf="parent"
  app:layout_constraintEnd_toEndOf="parent" />

This code adds a Next button to the bottom of the screen.

But now the pager might overlap the button. So you need to replace this attribute on the ViewPager2:

app:layout_constraintBottom_toBottomOf="parent"

With this one:

app:layout_constraintBottom_toTopOf="@id/onboarding_next_button"

This constrains the pager to the top of the Next button. Build and run, or just look at the Design view to see what it looks like.

Now you need a click listener for this button.

Go back to the OnboardingActivity. At the bottom of onCreate(), add this listener:

binding.onboardingNextButton.setOnClickListener {
 if (binding.onboardingPager.currentItem == NUM_PAGES - 1) {
  MainActivity.startActivity(this)
 } else {
  binding.onboardingPager.currentItem =
    binding.onboardingPager.currentItem + 1
 }
}

With this code, you’re making it so that the pager will advance when you click Next. If you’re already on the next page, it will go to the MainActivity. Build and run to see that everything is working as expected.

While you could stop here, you’d be depriving people of the ability to go back in the on-boarding flow. You’ll add a Back button to improve the experience.

Return to activity_onboarding.xml. Add the following code:

<com.google.android.material.button.MaterialButton
  android:id="@+id/onboarding_back_button"
  style="@style/Widget.MaterialComponents.Button.TextButton"
  android:layout_width="wrap_content"
  android:layout_height="wrap_content"
  android:padding="@dimen/space_normal"
  android:text="@string/onboarding_back"
  android:textColor="?colorOnPrimary"
  app:layout_constraintBottom_toBottomOf="parent"
  app:layout_constraintStart_toStartOf="parent" />

This adds a Back button to the layout.

Now, you need to connect a listener.

Add the following to the bottom of onCreate() in the OnboardingActivity:

binding.onboardingBackButton.setOnClickListener {
 binding.onboardingPager.currentItem =
   binding.onboardingPager.currentItem - 1
}

Now the Back button will return to the previous page.

But what if you’re on the first page? Your new Back button would probably crash the pager because there is no previous page. You should hide this button on the first page.

And when you’re on the last page, the Next button doesn’t make sense. It should have text that describes what it actually does.

You can address both issues with a listener that hides the Back button when you’re on the first page and changes the text for the Next button on the last page.

In onCreate(), add the following alongside your other listeners:

binding.onboardingPager.registerOnPageChangeCallback(object :
  ViewPager2.OnPageChangeCallback() {
 override fun onPageSelected(position: Int) {
  binding.onboardingNextButton.text =
    if (position == NUM_PAGES - 1) {
     getString(R.string.onboarding_done)
    } else {
     getString(R.string.onboarding_next)
    }
  binding.onboardingBackButton.visibility =
    if (position == 0) {
     View.GONE
    } else {
     View.VISIBLE
    }
   }
 }
)

This new listener registers an OnPageChangeCallback to monitor for page changes. When it does change, it sets the Next button text if the user is on the last page, and it sets the Done button visibility depending on if it’s the first page. Lastly, when the user is on the first page, it hides the Back button.

Build and run. Open up the on-boarding flow and notice how the buttons change as you flip back and forth between pages.

You just played around with some buttons, and now Taco Tuesday on-boarding is far more perceivable than it was. Good work!

Next up, you’ll learn about other improvements you can make with cues.

Giving cues

Another important part of on-boarding is what you’re saying. How do you make sure your instructions are meaningful? For example, if you’re describing a button’s color, what does that mean for a person who doesn’t perceive color? This brings you to the second criterion you’ll explore in this chapter:

Success Criterion 1.3.3 Sensory Characteristics: Instructions provided for understanding and operating content do not rely solely on sensory characteristics of components such as shape, color, size, visual location, orientation, or sound.

Level A

While attributes such as colors and shapes can be used, you should not rely on them to draft quality cues.

Imagine if the buttons on the discover screen to thumbs-up or thumbs-down a recipe were green and red circles, respectively. The screen reader says, “Tap the green button to save this recipe”. But you can’t see color well, so now you’re terrified to tap any button because you really want those tacos but don’t know how to save the recipe.

You might be thinking that giving the buttons a distinct shape would address the issue. Then the reader would say, “Tap the green triangle to save this recipe”. Unfortunately, calling out a shape won’t help people with severe vision impairments.

Now you’re starting to understand why this criterion exists!

You may have noticed the ambiguous directions in the pager: “Orange to the right, tacos in sight”!

Orange what? What if you can’t see orange? What if you also can’t see the image to give you a hint about what this means?

Clearly, Taco Tuesday’s on-boarding is not meeting success criterion 1.3.3. That’s fortunate for you, because you’ll get to make it better and learn a few things along the way.

Improving cues in on-boarding

There are a number of things you’ll do to improve the onboarding flow’s cues. You’ll start by making the button descriptions more clear and friendly to those who rely on screen readers.

Clarifying the instructions

The instructions are defined in strings.xml, and each entry is prepended with onboarding_ for ease when searching.

You’ll rewrite the first three instructions.

The first is “Orange to the right, tacos in sight”! This instruction means to say that you should swipe the orange card to the right to save it. You need the description to be more specific and less poetic.

Replace the onboarding_try_it value with: Swipe the orange recipe card to the right to save it to try later.

You can follow the same logic to replace the instruction “Orange to the left, move on to the rest”.

Replace the string named onboarding_discard with: Swipe the orange recipe card to the left to say \"no thank you!\" to a recipe.

Finally, the third instruction reads: “Green is keen to lead the way”. This gives little information at all!

Replace the value of onboarding_view_list to be: Then, you can view the list of recipes you want to try.

Your new instructions are much more descriptive, but you’re going to improve them further!

Remember that some people are not able to swipe. And the instructions don’t inform the reader they can save a recipe from the detail screen.

Run the app and notice where the button shown below is and what it looks like. Look for it in the details view — you can get there by clicking the view icon from the discover screen or the list of saved recipes.

How would you describe this button? Your description can’t rely on color or shape, nor can it use a visual location. It doesn’t have text, so you can’t use that either. A combination of these would be an improvement but not a full fix.

Adding text to the button

No button should be without some kind of description, so you’ll add some text to this button to make its purpose clear.

In the fragment_recipe_detail.xml layout, find the ImageView with the id recipe_detail_try_discard_button. To enable support for text, replace ImageView with the following class:

com.google.android.material.button.MaterialButton

Now you’re working with a proper button where you can use a compound image with text.

Remove the image source:

android:src="@drawable/ic_baseline_thumb_up_24"

And replace it with these attributes:

android:text="@string/shared_try_it"
app:icon="@drawable/ic_baseline_thumb_up_24"

When you look at the Design view, this button now has some descriptive text along with the thumbs-up icon.

Modifying the button state

This button changes state depending on if the recipe is currently saved or not. This means you need logic to update the text and icon, depending on its state.

Open RecipeDetailFragment.kt and find showEditableFields() and hideEditableFields(). These are the methods that control the view depending on the state.

In showEditableFields(), delete this statement:

recipeDetailTryDiscardButton.setImageDrawable(
 ResourcesCompat.getDrawable(resources,
  R.drawable.ic_baseline_thumb_down_24,
  requireContext().theme))

And replace it with these statements:

recipeDetailTryDiscardButton.text =
  getString(R.string.shared_discard)
recipeDetailTryDiscardButton.icon =
  ResourcesCompat.getDrawable(resources,
    R.drawable.ic_baseline_thumb_down_24,
    requireContext().theme)

This makes it so that when the recipe is already saved, the button gives the user the option to discard it.

Next, for the reverse case, delete this statement from hideEditableFields():

recipeDetailTryDiscardButton.setImageDrawable(
 ResourcesCompat.getDrawable(resources,
  R.drawable.ic_baseline_thumb_up_24,
  requireContext().theme))

And replace it with this code:

recipeDetailTryDiscardButton.text =
  getString(R.string.shared_try_it)
recipeDetailTryDiscardButton.icon =
  ResourcesCompat.getDrawable(resources,
   R.drawable.ic_baseline_thumb_up_24,
   requireContext().theme)

Now, when a recipe is not saved, the button allows the user to save it to try later.

Build and run to see your changes.

Adding state-specific instructions

Now you can add instructions for this button to your on-boarding. You’ll need to add a new page just for these instructions.

First, you need to write and add the message. In strings.xml, add the following line:

<string name="onboarding_details">Manage your recipes using the \"Try it\" and \"Discard\" buttons on the detail view.</string>

Then, add the new image resource to the project. Look for it in this chapter’s materials ▸ assets ▸ onboarding_details.xml. Add this new image to the drawable folder of this project.

You also need a content description for your image if you chose to add it during the challenge in Chapter 2, “Hello, Accessibility”.

Add the following to strings.xml:

<string name="onboarding_details_description">Thumbs-up icon</string>

Now, you can show your new on-boarding page.

Go to OnboardingActivity.kt and find the companion object with the pages list. Add the following to the list of pages right after the onboarding_discard item:

OnboardingItem(
  R.drawable.onboarding_details,
  R.string.onboarding_details,
  R.string.onboarding_details_description
),

Finally, update NUM_PAGES to use a computed getter, so it always has the correct number of pages:

private val NUM_PAGES: Int
 get() = pages.size

Build and run to see your new on-boarding page. And enjoy how much more accessible your on-boarding is!

Key points

• Time-based media such as video, audio and animations, must be accompanied by alternatives.
• Alternatives to visual media can be text or audio, and alternatives to audio can be equivalent visuals.
• Users must be able to control media that’s important for them to understand. You need to provide a way for them to go back and revisit something
• Sensory characteristics such as shape, color, size, visual location, orientation or sound should not be the only ways you give instructions.

Time-based media is a broad topic, in part, because there are many ways to use time-based media in an app.

When you’re unsure how to handle something, remember to lean on the WCAG, which offers ideas in addition to guidance: https://www.w3.org/TR/WCAG21/#time-based-media

When you’re ready to move on, you’ll finish up this three-chapter series about perceivability in the next chapter by digging into colors.