4.
The Testing Pyramid
Written by Fernando Sproviero
Heads up... You're reading this book for free, with parts of this chapter shown beyond this point as text.
Traditionally, software testing was done manually. It consisted of deploying the application to a test environment similar to the real environment but with fake (test) data. The Quality Assurance (QA) members would perform black-box testing on it — without knowing the internals — and raise bug tickets. Then, the developers went back and fixed the bugs.
Even nowadays, without any kind of automation, this is happening on the Android ecosystem. Testing usually consists of compiling the release candidate application, installing it on a physical device or emulator, and passing it to the QA team. The QA members would then follow a test plan, executing their use cases manually to find bugs.
You’ll find that automating these repetitive use cases is the way to go. For each use case, you can write one or more automated tests. However, first, you need to understand that there are different kind of tests and how to classify them.
Tests are typically broken into three different kinds:
This is the testing pyramid, a concept originally explained by Mike Cohn in his book Succeeding with Agile. The testing pyramid gives you a way to group different types of tests and gives an understanding of how many tests you should consider on each layer.
You should have lots of small unit tests, some integration and fewer UI tests.
You’ll now go through each of the layers.
Unit tests
Unit tests are the quickest, easiest to write and cheapest to run. They generally test one outcome of one method at a time. They are independent of the Android framework.
The System Under Test (SUT) is one class and you focus only on it. All dependencies are considered to be working correctly — and ideally have their own unit tests — so they are mocked or stubbed. This way, you have complete control of how the dependencies behave during the test.
These tests are the fastest and least expensive tests you can write because they don’t require a device or emulator to run. They are also called small tests. To give an example of an unit test, consider a game app.
The Game class is one of the main classes.
A common use case is to increment the score using a function like incrementScore(). Whenever the score increments and exceeds the highscore, it should also increment the highscore. A simple and incomplete definition of the Game class can look like this:
class Game() {
var score = 0
private set
var highScore = 0
private set
fun incrementScore() {
// Increment score and highscore when needed
}
}
Therefore, a test that checks this could be as follows:
fun shouldIncrementHighScore_whenIncrementingScore() {
val game = Game()
game.incrementScore()
if (game.highScore == 1) {
print("Success")
} else {
throw AssertionError("Score and HighScore don't match")
}
}
If you run this test, you’ll see the test doesn’t pass. We now have our failing (red) test. You can then fix this to get our passing (green) test by writing the actual method for the Game class:
fun incrementScore() {
score++
if (score > highScore) {
highScore++
}
}
Some common libraries for unit testing are JUnit and Mockito. You’ll explore both of these in later chapters.
Google, in its testing fundamentals documentation, also suggests Robolectric for local unit tests. Robolectric simulates the Android runtime, it allows you to test code that depends on the framework without using a device or emulator. This means that these tests run fast because they run using just the regular JVM of your computer, just like any other test that uses JUnit and Mockito. However, some may consider Robolectric as an integration testing tool, because it helps you test integrating with the Android framework.
Integration tests
Integration tests move beyond isolated elements and begin testing how things work together. You write these type of tests when you need to check how your code interacts with other parts of the Android framework or external libraries. Usually, you’ll need to integrate with a database, filesystem, network calls, device sensors, etc. These tests may or may not require a device or emulator to run; they are a bit slower than unit tests. They are also called medium tests.
fun shouldLaunchLogin_whenAddingFavorite() {
// 1
val user: User = null
val detailActivity = createProductDetailActivity(user)
detailActivity.findViewById(R.id.addFavorite).performClick()
// 2
val expectedIntent = Intent(detailActivity,
LoginActivity::class.java);
// 3
val actualIntent = getNextStartedActivity()
if (expectedIntent == actualIntent) {
print("Success")
} else {
throw AssertionError("LoginActivity wasn't launched")
}
}
UI tests
Finally, every Android app has a User Interface (UI) with related testing. The tests on this layer check if the UI of your application works correctly. They usually test if the data is shown correctly to the user, if the UI reacts correctly when the user inputs something, etc. They are also called large tests.
fun shouldWelcomeUser_whenLogin() {
onView(withId(R.id.username)).perform(typeText("Fernando"))
onView(withId(R.id.password)).perform(typeText("password"))
onView(withId(R.id.login_button)).perform(click())
onView(withText("Hello Fernando!"))
.check(matches(isDisplayed()))
}
Distributing the tests
A typical rule of thumb is to have the following ratio among the categories:
Key points
- Testing is commonly organized into the testing pyramid.
- There are three kinds of tests in this pyramid: unit, integration and UI tests. These are also called small, medium and large tests, respectively.
- On Android, you can also distinguish between local tests, which run on the JVM and instrumentation tests, which require a device or emulator. Local tests run faster than instrumented tests.
- You’ll write tests of different granularity for different purposes.
- The further down you get, the more focused and the more tests you need to write, be mindful of how expensive the test is to perform.
Where to go from here?
In the following chapters, you’ll start doing TDD by writing each kind of test with the appropriate tools and libraries.