Build an API with Kotlin on Google Cloud Platform

Deploying to Google Cloud

Now that you have something non-trivial, you can take it to the next level. You can deploy it to an actual server in the cloud! :]

This is where all the hard work you did setting Google Cloud pays off. Run the below command in the terminal of the IDE.

./gradlew appengineDeploy

When it starts building, you should see something similar to this:

The output tells you where the SDK deploys the app. So, once it’s finished building, visit https://[your_appengine_project_name].appspot.com/talks and see that it shows the same list with one entry on your browser. Remember to replace the project name with yours.

Next, go to MainActivity.kt and replace [your_appengine_project_name] with the name that you got from the deployment. It doesn't change on each deployment, so this is the only time that you’ll have to change it.

Now the Android app is ready to start consuming the service. Run the app by selecting the app configuration and clicking the play button next to it:

The app runs and displays just one event:

Adding Dynamic Data with Google Drive

Now, it's time to load some data from Google Sheets.

Reading Data From a Spreadsheet

Create a new Kotlin class named SpreadSheetDataSource, right next to RayConfApp.kt. This class is going to be in charge of loading data from a Google Sheet.

In this file, add the below code:

class SpreadSheetDataSource {

  // 1
  private val transport = GoogleNetHttpTransport.newTrustedTransport()
  // 2
  private val jacksonFactory = JacksonFactory.getDefaultInstance()
  // 3
  private val sheets = Sheets.Builder(transport, jacksonFactory, AppCredentials.local).build()                        
  // 4
  private val getAll = sheets.spreadsheets()
    .values().get("10mOp9WSGCZbMmlreLwy0GJiMBeTN_t05TaXXmFmBnVI", "A:G")                           
  // 5
  fun listAll() = getAll.execute().getValues()                                                       

}

1. transport provides the Google API the means to generate and execute network calls.
2. jacksonFactory provides the Google API the means to deserialize entities.
3. Create a new Sheets Builder with required arguments such as the credentials that you included in the resources as credential.json.
4. Using sheetsinstance to define the getAll query. It fetches all the available rows for the first six columns: A:G. The first parameter is the ID of the Spreadsheet.
5. listAll() method executes the query when asked and returns the values.

You may have noticed that the return type for that last function is of type:

(Mutable)List<(Mutable)List<Any!>!>!

That scary sausage of code means that it returns a list containing lists of objects of Any type. Not extremely useful. So, you’ll have to convert it into something to present to the user.

Start by creating a function inside SpreadSheetDataSource.kt to convert the raw columns to a list of maps:

private fun List<List<Any>>.toRawEntries(): List<Map<String, Any>> {
  val headers = first().map { it as String }           // 1
  val rows = drop(1)                                   // 2
  return rows.map { row -> headers.zip(row).toMap() }  // 3
}

The toRawEntries function doesn't change the state of the data source so you can define it in the same file, as a top-level function.

In this function, you are:

1. Extracting the first list and casting the values to Strings.
2. Then, you drop the first row to get the data rows.
3. Here you are converting each row to a map. The zip function creates a
   List<Pair<String, Any>>. zip pairs the elements with the same index. Finally, there is a convenience function called toMap() that converts a list of pairs into a Map.

Next, you need a function to convert a Map into a AgendaEntry.

private fun Map<String, Any>.toAgendaEntry() = AgendaEntry(
    id = longFor("Id"),
    title = stringFor("Title"),
    date = stringFor("Date"),
    startTime = stringFor("Start Time"),
    endTime = stringFor("End Time"),
    description = stringFor("Description"),
    speaker = stringFor("Speaker")
)

private fun Map<String, Any>.stringFor(key: String) = (this[key] as? String) ?: ""
private fun Map<String, Any>.longFor(key: String) = stringFor(key).toLongOrNull() ?: -1

stringFor(key) looks up a value in the receiver Map and tries to cast it to a String using as?. This conditional cast operator returns null if the cast is not successful, in which case you return an empty String.

longFor(key) uses the previous function to look up a value, and after toLongOrNull(), tries to parse the string into a Long. A null result means it can’t parse it into a Long, so you return a default value of -1.

toAgendaEntry() uses these two functions to build a AgendaEntry matching the headers from the following spreadsheet:

• https://docs.google.com/spreadsheets/d/10mOp9WSGCZbMmlreLwy0GJiMBeTN_t05TaXXmFmBnVI

Now, update listAll to return a list of AgendaEntry entities:

fun listAll(): List<AgendaEntry> =
    getAll.execute().getValues()
        .toRawEntries()
        .map { it.toAgendaEntry() }

The return value now is a list of AgendaEntry.

Now, go to the main() function inside RayConfApp.kt file and create a property named entries with the data source:

val entries = SpreadSheetDataSource()

Next replace talks.toJsonString() with entries.listAll().toJsonString() inside get("talks") block:

get("talks") {
   call.respondText(contentType = ContentType.Application.Json) {
     entries.listAll().toJsonString() // <--- replaced here
   }
 }

Run the server locally using the command ./gradlew appengineRun in the terminal. In the browser, the result from http://localhost:8080/talks now has all the entries from the Spreadsheet.

Now, deploy the server to the cloud using ./gradlew appengineDeploy. Once deployed, all the events are also visible on the Android app when relaunched/refreshed.

1. Create a copy you can edit.
2. Make the copy public.
3. Change the ID of the spreadsheet inside SpreadSheetDataSource to use the ID from your copy. You can find the ID on the URL of the document, as the segment after /d/

Caching Response

One thing you may have noticed is that the request takes a long time to return, anywhere between 300ms and 2s. Such a delay is understandable since the server is talking to Google Drive each time. Google Cloud provides several ways to persist data, making it more readily available. For this API, you are going to use Data Storage which provides basic key-value storage. You can see other storage options here.

Before writing a Local data source, you need a couple of mapper functions to convert from and to Entity instances. So, create a file named LocalDataSource.kt under the server module and add the following code:

import com.google.appengine.api.datastore.*
import com.raywenderlich.common.AgendaEntry

private const val AGENDA_ENTRY_NAME = "AgendaEntry"

private fun AgendaEntry.toEntity() = Entity(AGENDA_ENTRY_NAME, id).apply {
  setProperty("id", id)
  setProperty("title", title)
  setProperty("date", date)
  setProperty("startTime", startTime)
  setProperty("endTime", endTime)
  setProperty("description", description)
  setProperty("speaker", speaker)
  setProperty("votes", votes)
  setProperty("updated", updated)
}

private fun Entity.toEntry() = AgendaEntry(
    id = getProperty("id") as Long,
    title = getProperty("title") as String,
    date = getProperty("date") as String,
    startTime = getProperty("startTime") as String,
    endTime = getProperty("endTime") as String,
    description = getProperty("description") as String,
    speaker = getProperty("speaker") as String,
    votes = getProperty("votes") as Long,
    updated = getProperty("updated") as Long
)

Entity is a type from DataStore. It is similar to a Map. You can read and write values with the following two methods:

• getProperty(key: String): Any? tries to find a value for the provided key or returns null if not found.
• setProperty(key: String, value: Any?) creates or updates a value for a given key.

For the toEntity() mapper, you are creating a new Entity with "AgendaEntry" as the name and the AgendaEntry's id as the ID for the Entity. The ID is used to find and/or update entities. Then, you set the properties with the relevant keys.

In toEntry(), you do the reverse. Using the same keys used for the previous function, you read the relevant value and cast it to either a String or a Long.

Now that you can transform your application models into DataStore Entities, and vice versa, you can start building a local data source. Add the class LocalDataSource inside the LocalDataSource.kt file:

class LocalDataSource {

  private val dataStore = DatastoreServiceFactory.getDatastoreService()
  private val agendaEntryQuery = dataStore.prepare(Query(AGENDA_ENTRY_NAME))

  fun listAll(): List<AgendaEntry> = 
      agendaEntryQuery.asList(FetchOptions.Builder.withDefaults()).map { it.toEntry() }

  fun save(entries: List<AgendaEntry>): List<Key> {
      entries.map { it.toEntity() }.also { return dataStore.put(it) }
  }

}

The two properties in this component are a DataStore instance and a query to get all values for entities with the name "AgendaEntry". There are also two entry points:

• listAll() is similar to what you had in SpreadSheetDataSource, but this time you are executing agendaEntryQuery on dataStore as a list and mapping it to Entries using toEntry.
• save(entries) lets you put a list of entries into the dataStore. The Data Store overrides any entities which have the same ID.

Now you have a new data source to read and write instances of AgendaEntry in the local cache. So now you need to combine it with your other data store and do some "cache balancing".

Create a file named AgendaEntryRepository.kt under the server module and add the following code:

import com.raywenderlich.common.AgendaEntry

private const val HOUR = 3_600_000L

class AgendaEntryRepository {

  private val spreadSheetDataSource = SpreadSheetDataSource()
  private val cacheDataSource = LocalDataSource()
  private val cachePeriod: Long = HOUR

  fun listAll(): List<AgendaEntry> = cacheDataSource.listAll().let { cached ->
    cached.takeUnless { it.requiresUpdate(cachePeriod) }
        ?: spreadSheetDataSource.listAll()
            .also { entries -> cacheDataSource.save(entries) }
  }

}

private fun List<AgendaEntry>.requiresUpdate(cachePeriod: Long) =
    isEmpty() || any { System.currentTimeMillis() - it.updated > cachePeriod }

In this component, you have three properties: two for the different data sources and a third one for how long the entities are kept in memory. listAll() first tries to retrieve the data from the LocalDataSource. Then, you return the cached entities unless they need to be reloaded from the spreadsheet. The decision to load data from the spreadsheet comes down to whether the cache has no entries, or whether the last time they were loaded was over an hour ago.

If an update is required, you handle it using the Elvis operator. There, you fetch the values from the spreadsheet and tell the cacheDataSource to save the values for next time.

Go to the main() function under the RayConfApp.kt file and instead of using SpreadSheetDataSource, use AgendaEntryRepository.

val entries = AgendaEntryRepository()

Deploy the server again by executing ./gradlew appengineDeploy in the terminal, and you will see that the first request takes about 2-3 seconds, but any consecutive calls take around 10-20ms. That's a 10x improvement!