Paging Library for Android With Kotlin: Creating Infinite Lists

Breaking Free of Network Dependency

At this point, the app is firing on all cylinders. But there’s one problem — it’s dependent on the network to run. If you have no internet access, the app is doomed!

Next up, you are going to use the Room database library for persisting/caching the list so it is functional even when offline.

Navigate to database/dao package and create a new interface name RedditPostsDao.kt with the following code:

import androidx.paging.PagingSource
import androidx.room.Dao
import androidx.room.Insert
import androidx.room.OnConflictStrategy.REPLACE
import androidx.room.Query
import com.raywenderlich.android.redditclone.models.RedditPost

@Dao
interface RedditPostsDao {

    @Insert(onConflict = REPLACE)
    suspend fun savePosts(redditPosts: List<RedditPost>)

    @Query("SELECT * FROM redditPosts")
    fun getPosts(): PagingSource<Int, RedditPost>
}

For the most part, this is a standard Dao object in Room. But there’s one unique thing here: The return type for getPosts is a PagingSource. That’s right, Room can actually create the PagingSource for you! Room will generate one that uses an Int key to pull RedditPost objects from the database as you scroll.

Now, navigate to database/RedditDatabase.kt. Add the following code to the class:

abstract fun redditPostsDao(): RedditPostsDao

Here, you’re adding an abstract method to return RedditPostsDao.

At this point, you have methods for inserting and fetching posts. Now it’s time to add the logic to fetch posts from Reddit API and insert them into your Room database.

Creating a RemoteMediator

A RemoteMediator is like a PagingSource class. But RemoteMediator does not display data to a RecyclerView. Instead, it uses the database as a single source of truth. You fetch data from the network and save it to the database.

Navigate to repositories/RedditRemoteMediator.kt. You’ll see the following code:

@OptIn(ExperimentalPagingApi::class)
class RedditRemoteMediator(
    private val redditService: RedditService,
    private val redditDatabase: RedditDatabase
) : RemoteMediator<Int, RedditPost>() {
    override suspend fun load(
        // 1
        loadType: LoadType,
        // 2
        state: PagingState<Int, RedditPost>
    ): MediatorResult {
        TODO("not implemented")
    }
}

This class extends RemoteMediator and overrides the load(). You’re going to add the logic to fetch data from the Reddit API and save it to a database. The arguments for this method are different from those in PagingSource:

1. LoadType, is an enum that represents the loading type. It can have any of these values:
   • REFRESH indicates it’s a new fetch.
   • PREPEND indicates that content is being added at the start of the PagingData.
   • APPEND indicates that content is being added at the end of the PagingData.
2. PagingState. This takes a key-value pair, where the key has type Int and the value has type RedditPost.

Replace the TODO() with the following code:

return try {
    // 1
    val response = redditService.fetchPosts(loadSize = state.config.pageSize)

    // 2
    val listing = response.body()?.data
    val redditPosts = listing?.children?.map { it.data }

    // 3
    if (redditPosts != null) {
        redditDatabase.redditPostsDao().savePosts(redditPosts)
    }

    // 4
    MediatorResult.Success(endOfPaginationReached = listing?.after == null)

    // 5
} catch (exception: IOException) {
    MediatorResult.Error(exception)
} catch (exception: HttpException) {
    MediatorResult.Error(exception)
}

Here is what is happening in above code block:

1. This is a call to the Reddit API where you use the pageSize from the state parameter to fetch the data from the network.
2. Get the list of posts from the response body
3. If there are posts returned from the API, you save the list in the database.
4. If the network response is successful, you set the return type of the method to MediatorResult.Success. You also pass endOfPaginationReached, a Boolean variable that indicates when you are at the end of the list. In this case, the list ends when after is null. Notice that RemoteMediator uses the sealed class MediatorResult to represent the state of the data fetch.
5. Finally, you handle any exceptions that may occur during the loading operation and pass them to the Paging library.

Notice that RemoteMediator uses the sealed class MediatorResult to represent the state of the data fetch.

Next, you’ll modify your repository class to use RedditRemoteMediator in order to start using Room for persisting the list of posts.

Adding RemoteMediator to Your Repository

Navigate to repositories/RedditRepo.kt. Add the following code to the class body:

private val redditDatabase = RedditDatabase.create(context)

This creates an instance of the Room database using the create(). You’ll use it with RemoteMediator.

Next, replace the complete fetchPosts() with the following code:

@OptIn(ExperimentalPagingApi::class)
fun fetchPosts(): Flow<PagingData<RedditPost>> {
    return Pager(
        PagingConfig(
              pageSize = 40, 
              enablePlaceholders = false, 
              // 1
              prefetchDistance = 3),

        // 2
        remoteMediator = RedditRemoteMediator(redditService, redditDatabase),

        // 3
        pagingSourceFactory = { redditDatabase.redditPostsDao().getPosts() }
    ).flow
}

Here’s a breakdown of code above:

1. As part of your paging configuration, you add prefetchDistance to PagingConfig. This parameter defines when to trigger the load of the next items within the loaded list.
2. You set RedditRemoteMediator, which you created earlier. RedditRemoteMediator fetches the data from the network and saves it to the database.
3. Finally, you set pagingSourceFactory, in which you call the Dao to get your posts. Now your database serves as a single source of truth for the posts you display, whether or not you have a network connection.

You don’t have to modify the ViewModel or the activity layer, since nothing has changed there! That’s the benefit of choosing a good architecture. You can swap the data source implementations without modifying other layers in your app.

Now, build and run. You'll see the following screen:

As you scroll, you'll notice that your app only fetches one page. Why went wrong? Well, no worries! In the next section, you're going to fix it.

Fetching Previous and Next Pages With RemoteMediator

In PagingSource, you passed the before and after keys to LoadResult. But you're not doing this in RedditRemoteMediator. That's why the app currently fetches only one page.

To scroll continuously, you have to tell Room how to fetch the next and previous pages from the network when it reaches to the start or end of the current page. But how do you do this, when you're not passing the keys to MediatorResult?

To achieve this, you're going to save the keys in the Room database after every network fetch. Navigate to database/dao and create a new interface named RedditKeysDao.kt with below code:

@Dao
interface RedditKeysDao {

    @Insert(onConflict = REPLACE)
    suspend fun saveRedditKeys(redditKey: RedditKeys)

    @Query("SELECT * FROM redditKeys ORDER BY id DESC")
    suspend fun getRedditKeys(): List<RedditKeys>

}

This is a standard Dao object in Room with two methods for saving and retrieving the keys.

Next, go back to database/RedditDatabase.kt. In your RedditDatabase class, append the below line of code:

abstract fun redditKeysDao(): RedditKeysDao

redditKeysDao() is used to get access to the RedditKeysDao so that you can access the keys in the database easily.

Now navigate to repositories/RedditRemoteMediator.kt. Replace the body of load() with the following code:

 return try {
     // 1
     val loadKey = when (loadType) {
         LoadType.REFRESH -> null
         LoadType.PREPEND -> return MediatorResult.Success(endOfPaginationReached = true)
         LoadType.APPEND -> {
             state.lastItemOrNull()
                 ?: return MediatorResult.Success(endOfPaginationReached = true)
             getRedditKeys()
         }
     }

     // 2
     val response = redditService.fetchPosts(
         loadSize = state.config.pageSize,
         after = loadKey?.after,
         before = loadKey?.before
     )
     val listing = response.body()?.data
     val redditPosts = listing?.children?.map { it.data }
     if (redditPosts != null) {
         // 3
         redditDatabase.withTransaction {
             redditDatabase.redditKeysDao()
                 .saveRedditKeys(RedditKeys(0, listing.after, listing.before))
             redditDatabase.redditPostsDao().savePosts(redditPosts)
         }
     }
     MediatorResult.Success(endOfPaginationReached = listing?.after == null)
 } catch (exception: IOException) {
     MediatorResult.Error(exception)
 } catch (exception: HttpException) {
     MediatorResult.Error(exception)
 }

Here is what is happening in above code block:

1. You fetch the Reddit keys from the database when the LoadType is APPEND.
2. You set the after and before keys in fetchPosts.
3. Finally, you create a database transaction to save the keys and the posts you retrieved is there response returned a lits of posts

Next, add a new getRedditKeys() to the RedditRemoteMediator.kt class body:

private suspend fun getRedditKeys(): RedditKeys? {
    return redditDatabase.redditKeysDao().getRedditKeys().firstOrNull()
}

This is a suspend method that fetches RedditKeys from the database. Notice that you're using firstOrNull(). This will return the first items in the list. If there are no items in the database, it returns null.

Build and run, and voila! Your endless scrolling list is back.

You now have an endlessly scrolling app that works whether or not you have a network connection!