8. Transforming Operators in Practice

Written by Alex Sullivan & Marin Todorov

In the previous chapter, you learned about the real workhorses behind reactive programming with RxJava: the map and flatMap dynamic duo. Of course, those aren’t the only two operators you can use to transform Observables, but a program can rarely do without using those two at least few times. The more experience you gain with these two, the better (and shorter) your code will be.

You’ve already gotten to play around with transforming operators in the safety of an Kotlin project, so hopefully you’re ready to take on a real-life project. Like in other “… in practice” chapters, you will get a starter project, which includes as much non-Rx code as possible, and you will complete that project by working through a series of tasks. In the process, you will learn more about map and flatMap, and in which situations you should use them in your code.

Note: In this chapter, you will need to understand the basics of transforming operators in RxJava. If you haven’t worked through Chapter 7, “Transforming Operators,” do that first and then come back to this chapter.

Without further ado, it’s time to get this show started!

Getting started with GitFeed

I wonder what the latest activity is on the RxKotlin repository? In this chapter, you’ll build a project to tell you this exact thing.

The project you are going to work on in this chapter, named GitFeed, displays the activity of a GitHub repository, such as all the latest likes, forks or comments. To get started with GitFeed, open the starter project for this chapter.

In the chapter, you’ll use Retrofit, a networking library, and Gson, a JSON serialization library. Retrofit has a several nifty utilities that allow it to work particularly well with RxJava.

If you’re not familiar with Retrofit, it’s a simple networking library that allows you to declare your API in an interface and instantiate that API using Retrofits magical annotation processor. You’ll see more about it in Chapter 18, “Retrofit”.

Run the app. You’ll see the following blank screen:

Start off by opening the app module build.gradle file and looking at the Retrofit and Gson dependencies:

def retrofit_version = "2.9.0"
implementation "com.squareup.retrofit2:retrofit:$retrofit_version"
implementation "com.squareup.retrofit2:adapter-rxjava3:$retrofit_version"
implementation "com.squareup.retrofit2:converter-gson:$retrofit_version"
implementation "com.squareup.okhttp3:logging-interceptor:4.3.1"

There are four dependencies to note:

1. The actual Retrofit dependency.
2. An adapter that Retrofit provides that makes working with RxJava seamless.
3. A converter that allows you to use Gson.
4. An interceptor from the OkHttp library (on which Retrofit is built) that allows you to easily log all network output.

Fetching data from the web

Open GithubService.kt. All that’s there now is a companion object create method that builds up an instance of the GitHubApi Retrofit interface. There’s no actual networking code in here—yet.

Add the following method in the interface. Make sure the method is defined outside the companion object.

@GET("repos/ReactiveX/{repo}/events") // 1
fun fetchEvents(@Path("repo") repo: String) // 2
  : Observable<Response<List<AnyDict>>> // 3

Take care to import the Observable from the ReactiveX package and the Response class from Retrofit.

If you’re not familiar with Retrofit, the above code can be intimidating. Here’s a breakdown:

1. Retrofit allows you to use HTTP method type annotations on your methods to specify what type of HTTP action should be taken (POST, PUT, GET, etc). You also specify the path to the endpoint in this header. You can even add variables in the path, which is what {repo} is doing in this example.
2. After the annotation you create the actual method. In the method annotation you accept the name of the repo you want to fetch events for. By using the @Path annotation on the repo parameter, you’re telling Retrofit that the passed in value should replace the {repo} variable you specified in the method annotation.
3. Retrofit integrates very nicely with RxJava. By specifying the return type as an Observable, Retrofit will create your network call in a way that allows the result to be propagated out via an Observable. Pretty nifty! The Response type is a Retrofit type that contains all the information about your network call, like the status code and any errors. The last interesting thing about this code is the AnyDict type. AnyDict is a simple typealias for a Map<String, Any>. In typical usage you’d specify a concrete modal class here instead of a map, but for this app a simple map is fine.

Note: Normally it makes more sense for a API function to use the Single type instead of Observable. This is because REST API requests can only respond once. However this complicates the code later in this chapter, so here we use Observable for simplicity.

Now, navigate to MainViewModel and add the following line in the empty fetchEvents method:

val apiResponse = gitHubApi.fetchEvents(repo)

You’re using the fetchEvents method you defined earlier in GitHubService to get an Observable<Response<List<AnyDict>>>, that is, an Observable of Response objects, each of which contains a List<AnyDict> representing the list of things that happened on the provided repo.

Transforming the response

It’s time to start doing some transformations! And you’ll mix in some filtering operators too.

Add the following below the apiResponse declaration:

apiResponse.filter { response ->
  (200..300).contains(response.code())
}

You’re using the filter operator to filter out any response whose status code isn’t in the 200 to 300 range — i.e., any response that isn’t successful. In a production app you’d want to also handle any error that might occur, but for our purposes ignoring error codes is fine.

Note: You can read more about HTTP response codes in this Wikipedia article, List of HTTP status codes https://en.wikipedia.org/wiki/List_of_HTTP_status_codes.

Now continue the chain by adding the following map call:

.map { response ->
  response.body()!!
}

Remember that map transforms the item emitted by your Observable. In the above code, you’re transforming the Response object into a List<AnyDict>? by using the body method on Response. It’s safe to use !! here but take care that there are times when the body of a request is null.

Continue the chain with the following filter call:

.filter { objects ->
  objects.isNotEmpty()
}

You’re filtering out any response that provides an empty list of GitHub actions. Last but not least, add the following map call:

.map { objects ->
  objects.mapNotNull { Event.fromAnyDict(it) }
}

In this block, you’re converting the AnyDict objects into Events, which are model objects that represents GitHub API events. The companion object fromAnyDict method on Event just converts from an AnyDict into an Event. That code isn’t particularly interesting so it’s already been written in the starter project.

Note that you’re using the mapNotNull method on List that the Kotlin Standard library provides because AnyDict could contain null values if the API returns something we’re not expecting.

Note: In a production app, you’d be able to skip a few of these steps by having Retrofit directly convert the API response into an Event object. However, leaving that out helps demonstrate using the map operator and makes the Rx chain more substantial, which is a great way to learn about these operators.

Processing the response

Finish this chain up with the following code:

// 1
.subscribeOn(Schedulers.io())
// 2
.observeOn(AndroidSchedulers.mainThread())
.subscribeBy(
  // 3
  onNext = { events -> processEvents(events) },
  // 4
  onError = { error ->
    println("Events Error ::: ${error.message}") }
)
// 5
.addTo(disposables)

There’s a bit of magic in this code, so let’s see what it does:

1. You use subscribeOn to make sure the networking code happens off the main thread.

2. You use observeOn to get the results in subscribeBy on the main thread.

3. In the onNext lambda you’re passing the events over to the processEvents method. processEvents takes the first 50 events provided by the API and sends them over to MainActivity via the eventLiveData object. Just like in the Combinestagram app from earlier chapters, the MainActivity class observes the eventLiveData object and updates its list adapter when it gets new items.

4. In the onError block, you’re simply printing out the error. Again, in a production app you’d want to handle this error in a smart handy way, but for now this will do.

5. Finally, you’re adding the disposable that is created by calling subscribeBy into a handy CompositeDisposable object that’s already defined. You’re using the RxKotlin extension function addTo that lets you add the disposable to the composite as part of the operator chain.

Build and run the app. You should see a healthy list of GitHub actions for the RxKotlin repo.

Persisting objects to disk

It’d be great to be able to persist these GitHub actions to app storage, so you can view them without a network connection. Ideally, the app should first load events up from the local database, then show those saved events in the app RecyclerView. In parallel, the app can fetch new events, show them, and finally save them off to be loaded next time the user opens the app.

Add the following line to save the actions at the bottom of the processEvents method in MainViewModel:

EventsStore.saveEvents(events)

EventStore.saveEvents is a simple method that uses Gson to convert a list of Event instances into JSON and then saves those events to a file. Again, that code is straightforward enough that it’s not worth spending time on it.

Now that you’re saving events in the processEvents method, you can read the events and send them off to the Activity before the network provides a fresh set of events. Add the following to the top of fetchEvents:

eventLiveData.value = EventsStore.readEvents()

EventsStore.readEvents predictably pulls any saved events from the device storage and returns them.

You’re now serving up a list of events saved to the device. To test this new feature, first uninstall the app. Then build and run the app again. After the items are pulled down from the server the events will be saved to the disk. Run the app once more, and you should see events instantly loaded on screen.

You may not see any new events, since what was saved could be whatever the API has to offer at this point in time, but the events should be loaded nice and quick.

Adding a last-modified header

GitFeed is looking pretty good, but there’s still a few issues to iron out. One issue is that the app is being very wasteful when it comes to using a user’s network data. Even if the app already has events saved, it requests all of the events every time it makes a network request.

That’s about to change. You’re going to update the app to only download events that it hasn’t yet seen. And in the process you’re going to see flatMap used in a real app. Riveting, right?!

First, head over to GithubService and replace the fetchEvents method with the following:

@GET("repos/ReactiveX/{repo}/events")
fun fetchEvents(
  @Path("repo") repo: String,
  @Header("If-Modified-Since") lastModified: String
): Observable<Response<List<AnyDict>>>

Notice that fetchEvents now takes a new parameter, a String representing the last date that the app accessed the API. Instead of being a Path parameter like repo, lastModified will be added as a header parameter. Specifically, the GitHub API utilizes the If-Modified-Since header to specify the last time the client tried to access that resource. Retrofit exposes the @Header annotation to specify that an argument should be added as a header value. Retrofit truly is an amazing library!

Open up MainViewModel. You should now notice that the line declaring the apiResponse variable has an error in it. That’s because it’s not passing in the lastModified value. Add the following code, replacing the apiResponse line:

val lastModified = EventsStore.readLastModified()

val apiResponse = gitHubApi.fetchEvents(repo, lastModified?.trim() ?: "")

You’re fetching the lastModified value and passing it through to the fetchEvents method, making sure to trim any whitespace and defaulting to an empty string if there is no last-modified value. EventsStore.readLastModified hides some boilerplate around reading a last modified value from a text file you’ll write to next.

Next up you’ll want to reach into the Response object you get when you make the fetchEvents call and save the last modified value, which exists as a header object.

You have a few options, here. You could add a doOnNext operator to the Rx chain in fetchEvents and try to save off the last-modified value there. But that adds mutation into the Rx chain and muddies the purpose of that individual Observable.

Alternatively, you could make another call to the API and create a new Rx chain to get that last-modified value. That feels a bit better, but making a whole new API call is incredibly wasteful.

You may now be wondering: “Why not just use the share operator?”

THAT’S A GREAT IDEA!

Update the apiResponse value one more time, this time utilizing the share operator to share the API response:

val apiResponse =
  gitHubApi.fetchEvents(repo, lastModified?.trim() ?: "")
    .share()

Now that you have a shared Observable, you can start building up a new Rx chain to get and save that last modified value.

Add the following code below the previous Rx chain:

apiResponse
  .filter { response ->
    (200 until 300).contains(response.code())
  }

You’re again filtering out any failed calls.

Now you want to pull the last-modified value out of the Response object. Response exposes its headers, and the GitHub API utilizes the Last-Modified header to send down the last-modified date. Unfortunately, it’s nullable and RxJava doesn’t allow you to emit null values.

Again, there are a few options for how to handle this situation. One would be to create a wrapper class that itself either contains null or the last-modified value, and then use the map operator to map from Response to that new wrapper class. There’s an Optional type in Java just for this kind of situation. But that’s a lot of boilerplate just to get around a possibly nullable value.

Instead, you can use flatMap!

Add the following to the new chain:

.flatMap { response ->
  // 1
  val value = response.headers().get("Last-Modified")
  if (value == null) {
    // 2
    Observable.empty()
  } else {
    // 3
    Observable.just(value)
  }
}

Since flatMap is so tricky, here’s a breakdown of the above code:

1. Pull the last-modified value out of the Response objects headers. This value could be null.

2. If the value is null, return an empty Observable. If there’s no last-modified value then there’s nothing left for this Observable to do, so returning an empty Observable will just finish the chain.

3. If the value is present, return a new Observable that contains the last-modified value. This new Observable will now emit that last-modified value and finish. Perfect!

   

You’ve now got an Observable that’s emitting the last modified value from the API. All that’s left is to subscribe to it and save off that value. Add the following code to finish off the chain:

.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribeBy(
  onNext = { EventsStore.saveLastModified(it) },
  onError = { error ->
    println("Last Modified Error ::: ${error.message}") }
)
.addTo(disposables)

You’re again using a background scheduler to do the actual work of making the API call and the main thread scheduler to run the subscribeBy code.

In the onNext lambda, you’re saving off the last modified value. In the onError lambda you’re simply logging an error.

Challenge

Challenge: Fetch top repos and spice up the feed

In this challenge, you will go through one more map/flatMap exercise. You will spice up GitFeed a little bit: instead of always fetching the latest activity for a given repo like RxKotlin, you will find the top trending Kotlin repositories and display their combined activity in the app.

At first sight, this might look like a lot of work, but in the end you’ll find it’s only about a dozen lines of code.

Here’s the general structure of the flatMap that you’ll use:

apiResponse
  .flatMap { response: TopResponse ->
    if (response.items == null) {
      Observable.empty()
    } else {
      Observable.fromIterable(
        response.items.map { it["full_name"] as String })
    }
  }

To get you started, here’s the Retrofit code you’ll need to add to GithubService to fetch the top Kotlin repos and their associated activities:

@GET("repos/{repo}/events")
fun fetchEvents(@Path("repo", encoded = true) repo: String)
    : Observable<Response<List<AnyDict>>>

@GET("search/repositories?q=language:kotlin&per_page=5")
fun fetchTopKotlinRepos(): Observable<TopResponse>

You’ll also want to create a new TopResponse class to handle the top Kotlin repositories. It should look like this:

class TopResponse(val items: List<AnyDict>?)

You’ll use another flatMap to convert the JSON items value you get in the Response into a list of repo names using the full_name property of each repo. You’ll want to check that items is not null, or else return an empty Observable — just as you’ve done before.

If you’d like to play around some more, you can sort the combined list of events by date and other interesting ways. What other types of sorting or filtering can you come up with?

When you’ve completed the challenge, your results will look something like this:

If you wrapped up this challenge successfully, you can consider yourself a transformation pro! Oh… if you could only use a map in real life to turn lead into gold, that would really be something! But data transformation with RxJava comes a close second — and that’s great, too.

Note: The GitHub JSON API is a great tool to play with. You can grab a bunch of very interesting data such as trending repositories, public activity, and more. If you are interested to learn more, visit the API homepage at https://developer.github.com/v3/.

Key points

• GitHub has a nice API to play with. It’s a good place to experiment with transforming operators and Rx in general.
• Retrofit and Gson are a great networking duo for Android. The fact that Retrofit can return Observables and Singles makes it a good choice for learning Rx.
• Transforming operators can be chained in a flexible way. Experiment without fear! Sometimes, there’s a better way of chaining them to get the result you want.
• Always handle errors in network requests to prevent crashes. There can be a number of errors that are out of control. Don’t forget to use the onError case to prevent the app from crashing with an exception.
• You can easily filter out HTTP Status codes with Rx. Success codes are in the 2xx range, others status codes are mostly errors.
• Network requests in Android must be subscribed to on a background thread and observed on the main thread.
• map and flatMap let you transform the data in a server response to something that the app understands.

Where to go from here?

You’ve now seen filtering and transforming operators in action in an Android app. There’s one more type of operator that we’ll consider in detail: combining operators. So, back to IntelliJ in the next chapter to begin your look at how to use combining operators in RxJava.