17. Securing Data in Transit

Written by Kolin Stürt

Network security is an integral part of development. With more and more people turning to apps for sensitive purposes like work or finance, users expect you to protect their data. Almost every app communicates over a network. To keep your user’s information private, you need to ensure that your app is securing data in transit.

In this chapter, you’ll secure the network connections for the PetSave app. During the process, you’ll learn the following best practices:

• Using HTTPS for network calls.
• Trusting a connection with certificate pinning.
• Verifying the integrity of transmitted data.

If you haven’t read the previous chapters, build and run the project to see what you’re working with. Browse through the selection of pets and try tapping the report tab, which lets you send anonymous concerns:

In the previous chapter, you secured that data at rest. Now, your job is to ensure the data is secure when it leaves the app.

Understanding HTTPS

URLs that start with http:// transmit unprotected data that anyone can view — and many popular tools are available to monitor that data. Some examples are:

• Wireshark: https://www.wireshark.org
• mitmproxy: https://mitmproxy.org
• Charles: https://www.charlesproxy.com

Because pets tend to be fussy about their privacy, the requests in this app use HTTPS. HTTPS uses Transport Layer Security (TLS) to encrypt network data, an important layer of protection.

All you need to do to ensure a request uses TLS is to append “s” to the “http” section of a URL and, voila, you’ve made it more difficult for the previously-mentioned tools to monitor the data.

However, this doesn’t provide perfect protection.

Using Perfect Forward Secrecy

While encrypted traffic is unreadable, IT companies can still store it. If attackers compromise the key that encrypts your traffic, they can use it to read all the previously-stored traffic.

To prevent this vulnerability, Perfect Forward Secrecy (PFS) generates a unique session key for each communication session. If an attacker compromises the key for a specific session, it won’t affect data from other sessions.

Android 5.0+ implements PFS by default and prohibits TLS ciphers that don’t support it. As of Android N, you enforce this by using Network Security Configuration: https://developer.android.com/training/articles/security-config. You’ll add this to your app now.

Enforcing TLS with Network Security Configuration

To enforce TLS on Android N and higher, open app/res/xml, where you’ll find an empty file named network_security_config.xml. In this file, add the following code:

<?xml version="1.0" encoding="utf-8"?> 
<network-security-config> 
  <domain-config cleartextTrafficPermitted="false"> 
    <domain includeSubdomains="true">petfinder.com</domain> 
  </domain-config> 
</network-security-config>

Here, you set cleartextTrafficPermitted to false. It blocks network requests that don’t use TLS for specified domains. You then add petfinder.com as a domain and set its includeSubdomains attribute to true. This enforces TLS for subdomains like api.petfinder.com.

Next, you need to tell the Android system to use that file. In AndroidManifest.xml, add the android:networkSecurityConfig attibute to <application/> like in the following code:

<?xml version="1.0" encoding="utf-8"?>
<manifest package="com.raywenderlich.android.petsave"
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools">
  <!-- // ... -->
  <application
      android:networkSecurityConfig="@xml/network_security_config" 
  >
    <!-- // ... -->
  </application>
</manifest>

To test that it works, replace the BASE_ENDPOINT value in ApiConstants.kt with this:

const val BASE_ENDPOINT = "http://api.petfinder.com/v2/" 

Here, you changed the URL to use HTTP to test what happens when you send data without encryption.

Build and debug the project in an emulator or device running Android N or newer. You’ll see an error message in Debug that says CLEARTEXT communication to api.petfinder.com not permitted, as shown below:

That’s because Android blocked the calls so it won’t retrieve unencrypted data. Because you’ve previously launched the app, you might still get some pre-cached pet data.

Undo that change so the code is back to this:

const val BASE_ENDPOINT = "https://api.petfinder.com/v2/" 

Build and debug the app. The app displays the data again, but this time without the error — so you know it enforced TLS.

Don’t stop now! There are a few more simple changes that will make your app more secure.

Updating security providers

Often, when security researchers find vulnerabilities in software, the software company releases a patch. It’s a good idea to make sure you’ve patched the security provider for TLS. If you see an error such as, SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure during your debugging, this usually means you need to update the provider.

For more information about this procedure, see Android’s Update Your Security Provider page: https://developer.android.com/training/articles/security-gms-provider#patching.

Understanding certificate and public key pinning

Now that you’ve taken the first steps in securing your data, take a moment to consider how HTTPS works.

When you start an HTTPS connection, the server presents a certificate that verifies it’s the real entity. This is possible because a trusted certificate authority (CA) signed the certificate.

An intermediate authority might also have signed an intermediate certificate — there can be more than one signature. The connection is secure as long as a root certificate authority that Android trusts signed the first certificate. The Android system evaluates that certificate chain and, if a certificate isn’t valid, it closes the connection.

That sounds good, but it’s far from foolproof. There are many weaknesses that can make Android trust an attacker’s certificate instead of one that’s legitimately signed. For example, a company might have a work device configured to accept its own certificate. Or hackers can manually instruct Android to accept their installed certificate.

This is called a man-in-the-middle attack — it allows the entity in possession of the certificate to decrypt, read and modify the traffic.

Certificate pinning comes to the rescue by preventing connections when these scenarios occur. It works by checking the server’s certificate against a copy of the expected certificate.

Implementing certificate pinning

Certificate pinning is easy to implement on Android N+. Instead of comparing the entire certificate, it compares the hash (more on this later) of the public key, often called a pin:

To get the pin for the host you’re talking to, head to SSL Lab’s website: https://www.ssllabs.com/ssltest/analyze.html. Type api.petfinder.com for the Hostname field and click Submit:

On the next page, select one of the servers from the list:

You’ll see there are two certificates listed; the second one is a backup. Each entry has a Pin SHA256 value:

These values may change over time, so be sure to look them up before using them. They’re the hashes of the public keys that you’ll add to the app.

Return to network_security_config.xml and add them right after the domain tag for petfinder.com like this:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
  <domain-config cleartextTrafficPermitted="false">
    <domain includeSubdomains="true">petfinder.com</domain>
    <!-- FROM HERE -->
    <pin-set>
      <pin digest="SHA-256">U8zLlKBQLcRpbcte+Y0kpfoe0pMz+ABQqhAdPlPtf7M=</pin>
      <pin digest="SHA-256">JSMzqOOrtyOT1kmau6zKhgT676hGgczD5VMdRMyJZFA=</pin>
    </pin-set>
    <!-- TO HERE -->
  </domain-config>
</network-security-config>

Note: There are many ways to get the public key hash. One alternative is to download the certificate directly from the website and run OpenSSL commands on it. Or, if you’re developing an app for a company, you can bug IT for one. :]

Build and run, and you won’t see any changes. To test that everything works, change any character other than = for each of the pin digest entries. Here’s an example:

<pin digest="SHA-256">U8zLlT56PmiT3SR0WdFOR3dghwJrQ8yXx6JLSqTIRpk=</pin> 
<pin digest="SHA-256">JSMzq7xBsOVe1PQRwOsHsw3bsGT2VzIqz5K+59sNQws=</pin> 

Build and run now and you’ll see an error that says something like javax.net.ssl.SSLHandshakeException: Pin verification failed:

Don’t forget to undo those changes! With that, you’ve added certificate pinning support for Android N and higher… but what if your app needs to support versions under N? You’ll handle this case next.

Implementing pinning for early Android versions

In this app, you’re using OKHttp as the network library. Fortunately, this library lets you add pinning manually.

Head to APIModule.kt and add this to provideOkHttpClient, where it reads TODO: Add pinning for versions lower than M:

val hostname = "**.petfinder.com" //Double-asterisk matches any number of subdomains.
val certificatePinner = CertificatePinner.Builder()
    .add(hostname, "sha256/U8zLlKBQLcRpbcte+Y0kpfoe0pMz+ABQqhAdPlPtf7M=")
    .add(hostname, "sha256/JSMzqOOrtyOT1kmau6zKhgT676hGgczD5VMdRMyJZFA=")
    .build()

This tells OKHttp to enable certificate pinning with the pins for petfinder.com. For the hostname, one asterisk before the domain enables it for a single subdomain only. A double asterisk enables it for any number of subdomains.

Next, add this right after the line that reads return OkHttpClient.Builder():

.certificatePinner(certificatePinner)

This tells the OkHttpClient builder to involve the interceptor when making a connection. That will make sure the certificates match before completing the connection.

There are some other solutions for different network libraries:

• TrustKit is a third party library that uses the same format in network_security_config.xml to add support for versions under Android N. You can find it here: https://github.com/datatheorem/TrustKit-Android.
• For implementations for other libraries or more information about certificate pinning in general, see the OWASP documentation: https://www.owasp.org/index.php/Certificate_and_Public_Key_Pinning.

While pinning is popular, some companies don’t like having to update their apps from time to time with new pins as the old certificates expire. That’s a problem that Certificate Transparency solves.

Using Certificate Transparency

Certificate Transparency is a new standard that audits the presented certificates when you set up an HTTPS connection without requiring hard-coded values in the app.

When a CA issues a certificate, it must submit it to a number of append-only certificate logs. Certificate Transparency has nearly real-time monitoring to determine if someone has compromised the CA or if the CA issued the certificate maliciously. The owner of the domain can scrutinize the entries, and your app cross-checks the logs. The certificate is only valid if it exists in at least two logs.

When an entity revokes a certificate, you want to know about it immediately. You can use Certificate Transparency on top of pinning for greater security, so you’ll add it to your app next.

Implementing Certificate Transparency

In the app module build.gradle, add the following to the list of dependencies and sync Gradle:

implementation 'com.babylon.certificatetransparency:certificatetransparency-android:0.3.0' 

Go back to provideOkHttpClient in APIModule.kt and add this to the top of the imports:

import com.babylon.certificatetransparency.certificateTransparencyInterceptor

Find the line that reads TODO: Add certificate transparency here and add the following right under that line:

val ctInterceptor = certificateTransparencyInterceptor {
  // Enable for the provided hosts
  +"*.petfinder.com" //1 For subdomains
  +"petfinder.com" //2 asterisk does not cover base domain
  //+"*.*" - this will add all hosts
  //-"legacy.petfinder.com" //3 Exclude specific hosts
}

Here, you:

1. Enabled Certificate Transparency for the subdomains of petfinder.com.
2. Since an asterisk doesn’t cover the base domain in this case, you added it explicitly.
3. Added a comment to exclude specific domains using -. That example would allow all petfinder.com domains except the one starting with legacy.

Then, add this to the OkHttpClient builder, after the line you previously added that reads .certificatePinner(certificatePinner):

.addNetworkInterceptor(ctInterceptor)

Now the OkHttpClient builder will invoke both certificate pinning and certificate transparency. You’ll be able to build and run the app without any issue.

Next, you’ll learn about a few more options that affect certificate checking.

Preventing information leaks with OCSP stapling

The traditional way to determine if an entity revoked a certificate is to check a Certificate Revocation List (CRL). To do this, your app must contact a third party to confirm the validity of the certificate, which adds network overhead. It also leaks private information about the sites you want to connect with to the third party.

Online Certificate Status Protocol (OCSP) stapling comes to the rescue. When you start an HTTPS request to the server using this method, the validity of the server’s certificate is already stapled to the response.

OCSP stapling is enabled by default, but you can disable it or customize the behavior of certificate revocation using PKIXRevocationChecker.Option. You can look at the commented code inside ReportManager.kt’s init block for sample code, or visit the documentation for PKIXRevocationChecker here: https://developer.android.com/reference/kotlin/java/security/cert/PKIXRevocationChecker.Option.

With OCSP stapling, the server you’re connecting to can’t forge this info. That’s because the CA signs that info ahead of time, and it’s why it doesn’t know which site you want to access.

So what is signing? It’s a way to verify the data’s integrity. Even though your data is encrypted, how do you know was authentic in the first place? Signing and authentication help ensure the integrity of the information you send and receive over the network.

Understanding authentication

During World War II, German bombers used Lorenz radio beams to navigate and to find targets in Britain. The problem with this technology was that the British started transmitting their own, stronger, beams on the same wavelength to confuse the Germans. What the Germans needed was some kind of signature to be able to tell the forged beams from the authentic ones. Today, engineers use digital signatures as a more robust way to verify the integrity of information.

Digital signatures ensure that you’re the one accessing your health data, starting a chat or logging into a bank. They also ensure no one has altered the data.

At the heart of a digital signature is a hash function. A hash function takes a variable amount of data and outputs a signature of a fixed length. It’s a one-way function, also known in math as a trap-door function. Given the resulting output, there’s no computationally-feasible way to reverse it to reveal what the original input was.

The output of a hash function is always the same if the input is the same. The output is drastically different if you change even one byte or character. That makes it the perfect way to verify that a large amount of data isn’t corrupted — you simply hash the data and compare that hash with the expected one.

To authenticate that data is untampered, you’ll use Secure Hash Algorithm (SHA), which is a well-known standard that refers to a group of hash functions.

NOTE: SHA1 hash functions are unsafe and should never be used, but anything from the SHA-2 family, such as SHA-512, is recommended. For more information about SHA, go here: https://en.wikipedia.org/wiki/Secure_Hash_Algorithms.

Authenticating with Public-Key Cryptography

In many cases, when an API sends data over a network, the data also contains a hash. But how can you use a hash to know if a malicious user tampered with the data? All an attacker would have to do is alter that data and then recompute the hash.

What you need is to add some secret information to the mix when you hash the data. Developers call this kind of hash a signature. The attacker cannot recompute the signature without knowing the secret. But how do both parties let each other know what the secret is without someone intercepting it? That’s where Public-Key Cryptography comes into the picture.

Public-Key Cryptography works by creating a set of keys, one public and one private. The private key creates the signature, while the public key verifies it.

Given a public key, it’s not computationally feasible to derive the private key. Even if malicious users know the public key, all they can do is to verify the integrity of the original message. Attackers can’t alter a message because they don’t have the private key to reconstruct the signature. The most modern way to do this is through Elliptic-Curve Cryptography (ECC):

Verifying integrity with Elliptic-Curve Cryptography

ECC is a new set of algorithms based on elliptic curves over finite fields. While the math is out of scope for this chapter, you can read more about it here: https://en.wikipedia.org/wiki/Elliptic-curve_cryptography.

You can use ECC for encryption, but in this chapter, you’ll use it for authentication, known as Elliptic Curve Digital Signature Algorithm (ECDSA).

To start using ECDSA, open Authenticator.kt. This is a template that imports the necessary key and factory classes that you can use to create your public and private key pair.

Adding public and private keys

Add a public key and private key just after the Authenticator class definition:

class Authenticator {

  private val publicKey: PublicKey
  private val privateKey: PrivateKey
  // ...
}

You need to initialize these keys, so right after the variables, add the init block:

class Authenticator {
  // ...
  init {
    val keyPairGenerator = KeyPairGenerator.getInstance("EC") // 1
    keyPairGenerator.initialize(256) // 2
    val keyPair = keyPairGenerator.genKeyPair() // 3

    // 4
    publicKey = keyPair.public
    privateKey = keyPair.private
  }
  // ...
}

Here’s what you did in this code:

1. Created a KeyPairGenerator instance for the Elliptic Curve (EC) type.
2. Initialized the object with the recommended key size of 256 bits.
3. Generated a key pair, which contains both the public and private key.
4. Set the publicKey and privateKey variables of your class to those newly-generated keys.

Adding the sign and verify methods

To complete this class, update the sign and verify methods. Replace the contents of sign() with this:

class Authenticator {
  // ...
  fun sign(data: ByteArray): ByteArray {
    val signature = Signature.getInstance("SHA512withECDSA") // 1
    signature.initSign(privateKey) // 2
    signature.update(data) // 3
    return signature.sign() // 4
  }
  // ...
}

This method takes in a ByteArray and:

1. Gets an ECDSA instance using the recommended hash type of SHA-512.
2. Initializes Signature with the private key for signing.
3. Adds the ByteArray data.
4. Returns a ByteArray signature.

Next, you’ll need a way to verify data given a public key you receive. Replace the last verify() in the class with the following:

class Authenticator {
  // ...
  fun verify(signature: ByteArray, data: ByteArray, publicKeyString: String): Boolean {
    val verifySignature = Signature.getInstance("SHA512withECDSA")
// 1
    val bytes = android.util.Base64.decode(publicKeyString,
        android.util.Base64.NO_WRAP)
    val publicKey =
        KeyFactory.getInstance("EC").generatePublic(X509EncodedKeySpec(bytes))
    verifySignature.initVerify(publicKey) // 2
    verifySignature.update(data) // 3
    return verifySignature.verify(signature) // 4
  }
  // ...
}

This code:

1. Converts a Base64 public key string into a PublicKey object.
2. Initializes the Signature with the public key for verification.
3. Updates the Signature with your data.
4. Performs the verification. The method returns true if the verification succeeds.

Base64 is a format that allows you to pass raw data bytes over the network as a string. You can read more about it here: https://en.wikipedia.org/wiki/Base64.

Update the helper function to convert the key object into a String by replacing publicKey() with the following:

class Authenticator {
  // ...
  fun publicKey(): String {
    return android.util.Base64.encodeToString(publicKey.encoded, android.util.Base64.NO_WRAP)
  }
}

Now that you have an Authenticator, you’ll use it to sign requests to the report server.

Why you sign a request

The PetSave app uses test code to simulate connecting to the pet report server via a back-end API. Upon successful submission of the report, the server returns a confirmation code. For your privacy, the test code doesn’t really send your data anywhere. It’s just a simulation. :]

In the previous chapter, you created an app login session that authenticates your credentials by using a fingerprint or a device passcode. This ensured that only you could access the app’s data stored on the device. It created a unique token, protected by the device’s keystore, that’s only accessible upon authenticating on your device.

Now, you’ll use that token to log in to the Pet Reporter server. The app will send your token and public key to the server before you can access the report endpoints.

Once the server knows who you are, the app needs to sign its requests to the Send Report endpoints to use them successfully. That way, the server authenticates that only you are accessing the endpoints.

How to build the signature

Open MainActivity.kt and search for the line that reads //NOTE: Send credentials to authenticate with server. Here, you’ve logged in to the server with your token and public key. Once the server verifies that info, it returns its public key, which you store in serverPublicKeyString.

When signing a request, it’s common to take selected parts of the request — such as HTTP Headers, GET or POST parameters — and the URL and join them into a string. You use that string to create the signature. On the back end, the server repeats the process of joining the strings and creating a signature. If the signatures match, it proves that the user must have possession of the private key. No one can impersonate the user because they don’t have that private key.

Since specific parameters of the request are part of the string, it also guarantees the integrity of the request by preventing attackers from altering the request parameters. For example, a bank wouldn’t be happy if attackers could alter the destination account number for a money transfer or alter the mailing address to receive the victim’s credit card statements in the mail.

For your next step, you’ll create a signature for the request to send the report.

Creating the signature

Back in ReportDetailFragment.kt, add the following code to sendReportPressed(), just under the line that reads //TODO: Add Signature here:

val stringToSign = "$REPORT_APP_ID+$reportID+$reportString" // 1
val bytesToSign = stringToSign.toByteArray(Charsets.UTF_8) // 2
val signedData = mainActivity.clientAuthenticator.sign(bytesToSign) // 3
requestSignature = Base64.encodeToString(signedData, Base64.NO_WRAP) // 4

Here’s what this code does:

1. Concatenates the parameters for the request string.
2. Converts the string into a ByteArray.
3. Signs the bytes using your private key and returns the signature bytes.
4. Turns the signature bytes into a Base64 string that you can easily send over the network.

Now that you’ve created a signature, you’ll verify that it worked.

Verifying the signature

To verify that your signature is correct, head to ReportManager.kt and look at sendReport(). You’ll find simulated server code that calls serverAuthenticator.verify.

Debug and run to check that it worked. Set a breakpoint on the if (success) { line to check that success is true:

To test what happens when there are problems, alter the data the server receives. Add the following right after val bytesToVerify = stringToVerify.toByteArray(Charsets.UTF_8):

bytesToVerify[bytesToVerify.size - 1] = 0 

The above line of code replaces the last byte of the data with 0.

Debug and run again. This time success is false:

You just secured your data with a signature. Don’t forget to remove that test line you just added!

Authenticating the response

Now that the server has authenticated the report, you also want to authenticate the response so you know the confirmation code, or any other communication from the server, is legitimate. Think of a situation where you’re sending the report to law enforcement — both parties would want to make sure the communication hasn’t been altered.

Just as you provided your public key when you registered with the reporting service, the reporting service passed its public key back. A chat app might use the same setup, for example, where each user might exchange public keys upon initiating a chat session.

In this case, however, you’ll use the server’s public key to verify the report data that the server returned. Back in ReportDetailFragment.kt, replace success = true right after the line that reads TODO: Verify signature here in sendReportPressed():

// 1
val serverSignature = it["signature"] as String
val signatureBytes = Base64.decode(serverSignature, Base64.NO_WRAP)

// 2
val confirmationCode = it["confirmation_code"] as String
val confirmationBytes = confirmationCode.toByteArray(Charsets.UTF_8)

// 3
success = mainActivity.clientAuthenticator.verify(signatureBytes,
    confirmationBytes, mainActivity.serverPublicKeyString)

Here’s what you did:

1. Retrieved the signature string and converted it to bytes.
2. Obtained the result data — the confirmation code.
3. Verified the result data with the signature from the server.

Testing your authentication

To test that it worked, set a breakpoint on the if (success) { line inside onReportReceived(). Build and debug to see the result in the Debug tab.

Alter the request data to see what happens. Add the following code right before calling clientAuthenticator.verify() in sendReport() in the ReportManager.kt file:

confirmationBytes[confirmationBytes.size - 1] = 0 

Build and run. This time, success is false in the Debug tab:

Congratulations! You’ve secured both sides of the communication. Don’t forget to remove the test code that makes it fail. You should also be aware of a few other standards when it comes to authentication:

• RSA is a popular and accepted standard. Its key sizes must be much larger, such as 4096 bits, and key generation is slower. You might use it if the rest of your team is already familiar with or using this standard.
• HMAC is another popular solution that, instead of using public-key cryptography, relies on a single shared key. You must exchange the secret key securely. Developers use HMAC when speed considerations are very important.
• OAuth is a standard to delegate access so users can grant a service access to their information without revealing their password. You used it in previous chapters to perform basic authentication with the petfinder.com API. PetFinder uses this to control its API use by weeding out botting and abuse by spammers. Read more about it here: https://developer.android.com/training/id-auth/authenticate.html.
• The Account Manager is a centralized helper for user account credentials so your app doesn’t have to deal with user passwords directly. Read more about it here: https://developer.android.com/reference/android/accounts/AccountManager.html.

End-to-end encryption

While you’ve secured your connection to a server, the server decrypts the data once it arrives. Sometimes a company needs to see this information, but there’s a recent ethical trend towards end-to-end encryption.

An example of end-to-end encryption is a chat app where each user begins by exchanging their public key. Then when a user, Alice, wants to send a message to Bob, she encrypts the message using Bob’s public key, which she received. Bob then decrypts the message using his private key. Only the sender and receiver have the private keys to decrypt each others’ messages.

The chat service never receives the private keys; it has no way of knowing what the content is. This is a proactive way to avoid liability during a server-side data breach or compromise.

To learn more about implementing this approach, a good place to start is the open-source Signal App GitHub repo: https://github.com/signalapp.

Key points

In this chapter, you discovered that you should:

• Always use HTTPS instead of HTTP.
• Enable certificate transparency, certificate pinning or both for maximum security.
• Authenticate your network requests.

Where to go from here?

Here are some other points about network safety:

• Google has a network security testing tool to help you spot cleartext traffic or other connection vulnerabilities in your app. Visit nogotofail for more info: https://github.com/google/nogotofail.
• For more security tools, check out the SafetyNet API, which includes safe browsing, integrity and reCAPTCHA to protect your app from spammers, phishing URLs and other malicious traffic. Find it here: https://developer.android.com/training/safetynet/attestation.

You’ve been securing and verifying the integrity of the data, but that’s not a replacement for regular data validation checks like type and bounds checking.

For example, if you expect a string of 256 characters or less in the network response, you should still check for that. If the server expects a parameter with only numbers, you’d want to sanitize that output.

This is called app hardening, and it’s what the next chapter is all about!