35. Microservices, Part 1

Written by Tim Condon

Note: This update is an early-access release. This chapter has not yet been updated to Vapor 4.

In previous chapters, you’ve built a single Vapor application to run your server code. For large applications, the single monolith becomes difficult to maintain and scale. In this chapter, you’ll learn how to leverage microservices to split up your code into different applications. You’ll learn the benefits and the downsides of microservices and how to interact with them. Finally, you’ll learn how authentication and relationships work in a microservices architecture.

Microservices

Microservices are a design pattern that’s become popular in recent years. The aim of microservices is to provide small, independent modules that interact with one another. This is different to a large monolithic application. This makes the individual services easier to develop and test as they are smaller. Because they are independent, you can develop them individually. This removes the need to use and build all the dependencies for the entire application.

Microservices also allow you to scale your application better. In a monolithic application, you must scale the entire application when under heavy load. This includes parts of the application that receive low traffic. In microservices, you scale only the services that are busy.

Finally, microservices make building and deploying your applications easier. Deploying very large applications is complex and prone to errors. In large applications, you must coordinate with every development team to ensure the application is ready to deploy. Breaking a monolithic application up into smaller services makes deploying each service easier.

Each microservice should be a fully contained application. Each service has its own database, its own cache and, if necessary, its own front end. The only shared part should be the public API to allow other services to interact with that microservice. Typically, they provide an HTTP REST API, although you can use other techniques such as protobuf or remote procedural calls (RPC). Since each microservice interacts with other services only via a public API, each can use different technology stacks. For instance, you could use PostgreSQL for one service that required it, but use MySQL for the main user service. You can even mix languages. This allows different teams to use the languages they prefer.

Swift is an excellent choice for microservices. Swift applications have low memory footprints and can handle large numbers of connections. This allows Swift microservices to fit easily into existing applications without the need for lots of resources.

The TIL microservices

In the first few sections of this book, you developed a single TIL application. You could have used a microservices architecture instead. For instance, you could have one service that deals with users, another that deals with categories and another for acronyms. Throughout this chapter, you’ll start to see how to do this.

Download and open the starter project for this chapter. There are two Vapor applications in there:

• TILAppUsers: a microservice for users running on port 8081. This services uses a Postgres database to persist the users’ information.
• TILAppAcronyms: a microservice for the acronyms running on port 8082. This service uses a MySQL database to store the acronyms.

The user microservice

Navigate to the TILAppUsers directory in Terminal. Enter the following the start the database:

docker run --name postgres -e POSTGRES_DB=vapor \
  -e POSTGRES_USER=vapor -e POSTGRES_PASSWORD=password \
  -p 5432:5432 -d postgres

Here’s what this does:

• Run a new container named postgres.
• Specify the database name, username and password through environment variables.
• Allow applications to connect to the PostgreSQL server on its default port: 5432.
• Run the server in the background as a daemon.
• Use the Docker image named postgres for this container. If the image isn’t present on your machine, Docker automatically downloads it.

Next generate and open the project in Xcode:

vapor xcode -y

First, open User.swift. The User model for this service is a simplified version from the main TIL application.

Next, open UsersController.swift. Again, like the TIL application, this contains routes to create a user, retrieve a user and retrieve all users.

Build and run the application and launch RESTed. Configure a request as follows:

• URL: http://localhost:8081/users
• method: POST
• Parameter encoding: JSON-encoded

Add three parameters with names and values:

• username: a username of your choice
• name: a name of your choice
• password: a password of your choice

Click Send Request. This creates a user in the application.

Configure a new request as follows:

• URL: http://localhost:8081/users
• method: GET

Click Send Request. You’ll see the user you created:

For now, that’s as complicated as the user service needs to be!

The acronym microservice

Keep the user service running and navigate to the TILAppAcronyms directory in Terminal. Enter the following the start the database:

docker run --name mysql -e MYSQL_USER=vapor \
  -e MYSQL_PASSWORD=password -e MYSQL_DATABASE=vapor \
  -p 3306:3306 -d mysql/mysql-server:5.7

Here’s what this does:

• Run a new container named mysql.
• Specify the database name, username and password through environment variables.
• Allow applications to connect to the MySQL server on its default port: 3306.
• Run the server in the background as a daemon.
• Use the Docker image named mysql/mysql-server for this container. If the image isn’t present on your machine, Docker automatically downloads it. This also specifies the image tagged with version 5.7, the version compatible with Fluent.

Next enter the following in Terminal:

vapor xcode -y

Again, this generates and opens the Xcode project. This service contains the exact same Acronym model as the main TIL application. Open AcronymsController.swift. You’ll see routes for CRUD operations on Acronym. Build and run the service and configure a new request in RESTed as follows:

• URL: http://localhost:8082/
• method: POST
• Parameter encoding: JSON-encoded

Add three parameters with names and values:

• short: OMG
• long: Oh My God
• userID: The ID of the user created earlier

Click Send Request. This creates an acronym in the service:

Configure a new request as follows:

• URL: http://localhost:8082/
• method: GET

Click Send Request. You’ll see the acronym you created:

Dealing with relationships

At this point you can create both users and acronyms in their respective microservices. However, dealing with relationships between different services is more complicated. In Section 1 of this book, you learned how to use Fluent to enable you to query for different relationships between models. With microservices, since the models are in different databases, you must do this manually.

Getting a user’s acronyms

In the TILAppAcronyms Xcode project, open AcronymsController.swift. Below updateHandler(_:) add a new route handler to get the acronyms for a particular user:

func getUsersAcronyms(_ req: Request)
  throws -> Future<[Acronym]> {
    // 1
    let userID = try req.parameters.next(UUID.self)
    // 2
    return Acronym.query(on: req)
      .filter(\.userID == userID)
      .all()
}

Here’s what the route handler does:

1. Get the user’s ID as a UUID from the request’s parameters. You can’t use User.parameter as this microservice doesn’t have access to the user database.
2. Perform a query on the Acronym table to get all acronyms with a userID that matches the ID passed in.

Since the Acronym table contains the user ID, you don’t need to request any external information to perform the query. Register the route in boot(router:) under router.put(Acronym.parameter, use: updateHandler):

router.get("user", UUID.parameter, use: getUsersAcronyms)

This routes GET requests to /user/<USER_ID> to getUsersAcronyms(_:). Build and run the TILAppAcronyms service and configure a new request in RESTed as follows:

• URL: http://localhost:8082/user/<ID_OF_THE_USER_CREATED_EARLIER>
• method: GET

Click Send request and you’ll see all acronyms created by that user:

Getting an acronym’s user

You can already get an acronym’s user with the current projects. You make a request to get the acronym, extract the user’s ID from it, then make a request to get the user from the user service. Chapter 37, “Microservices, Part 2” discusses how to simplify this for clients.

Authentication in Microservices

Currently a user can create, edit and delete acronyms with no authentication. Like the TIL app, you should add authentication to microservices as necessary. For this chapter, you’ll add authentication to the TILAppAcronyms microservice. However, you’ll delegate this authentication to the TILAppUsers microservice.

In practice, it works like this:

• A user logs in to the TILAppUsers microservice and obtains a token.
• When creating an acronym, the user provides the token to the TILAppAcronyms service.
• The TILAppAcronyms service validates the token with the TILAppUsers service.
• If the token is valid, the TILAppAcronyms proceeds with the request, otherwise it rejects the request.

Logging in

Open the TILAppUsers project in Xcode. The starter project already contains a Token type and an empty AuthContoller. You could store the tokens in the same database as the user. Since every validation request requires a lookup and you have multiple services, you want this to be as quick as possible. One solution is to store them in memory. However, if you want to scale your microservice, this doesn’t work. You need to use something like Redis. Redis is a fast, key-value database, which is ideal for storing session tokens. You can share the database across different servers which allows you to scale without any performance penalties.

In Terminal, type the following to start a Redis database server:

docker run --name redis -p 6379:6379 -d redis

Here’s what this does:

• Run a new container named redis.
• Allow applications to connect to the Redis server on its default port: 6379.
• Run the server in the background as a daemon.
• Use the Docker image named redis for this container. If the image isn’t present on your machine, Docker automatically downloads it.

Back in Xcode, open configure.swift for the TILAppUsers project. At the top of the file, add the following underneath import Authentication:

import Redis

This allows you to use Redis in your application. The project already has Redis configured as a dependency. Next, below databases.add(database: postgres, as: .psql) add the following:

// 1
var redisConfig = RedisClientConfig()
// 2
if let redisHostname = Environment.get("REDIS_HOSTNAME") {
  redisConfig.hostname = redisHostname
}
// 3
let redis = try RedisDatabase(config: redisConfig)
// 4
databases.add(database: redis, as: .redis)

Here’s what the code does:

1. Create a RedisClientConfig type using default values.
2. Use the REDIS_HOSTNAME environment variable for the Redis server hostname, if it’s set.
3. Create an instance of RedisDatabase using the configuration.
4. Add the redis database to the DatabasesConfig.

You’ve now configured the TILAppUsers project to use Redis. Notice the project now uses two databases — PostgreSQL and Redis. Next, open AuthContoller.swift and create a new route handler below boot(router:) to handle a user logging in:

func loginHandler(_ req: Request) throws -> Future<Token> {
  // 1
  let user = try req.requireAuthenticated(User.self)
  // 2
  let token = try Token.generate(for: user)
  // 3
  return req.withPooledConnection(to: .redis) { redis in
    // 4
    redis.jsonSet(token.tokenString, to: token)
      .transform(to: token)
  }
}

Here’s what the new code does:

1. Get the authenticated user from the request. The route will use Basic HTTP Authentication to retrieve the user.
2. Generate a Token for the user.
3. Get a database connection to Redis from the pool of connections held by the application.
4. Save the token in Redis as a JSON string for the value, using the token string as the key. Return the Token as the response.

Finally, register the route in boot(router:):

// 1
let authGroup = router.grouped("auth")
// 2
let basicAuthMiddleware =
  User.basicAuthMiddleware(using: BCryptDigest())
// 3
let basicAuthGroup = authGroup.grouped(basicAuthMiddleware)
// 4
basicAuthGroup.post("login", use: loginHandler)

Here’s what the routing code does:

1. Create a new route group under /auth for handling all authentication routes.
2. Create the HTTP Basic Authentication middleware from User using BCryptDigest.
3. Create a new route group using the middleware.
4. Route POST requests to /auth/login to loginHandler(_:).

For more information on HTTP Basic Authentication, see Chapter 18, “API Authentication, Part 1.”

Authenticating tokens

First, create a new type to represent the data sent in token validation requests. At the bottom of AuthController.swift add the following:

struct AuthenticateData: Content {
  let token: String
}

The request only needs the token to validate the request. Next, create a new route below loginHandler(_:) to handle the requests with this data from other microservices:

func authenticate(_ req: Request, data: AuthenticateData)
  throws -> Future<User.Public> {
    // 1
    return req.withPooledConnection(to: .redis) { redis in
      // 2
      return redis.jsonGet(data.token, as: Token.self)
        .flatMap(to: User.Public.self) { token in
          // 3
          guard let token = token else {
            throw Abort(.unauthorized)
          }
          // 4
          return User.query(on: req)
            .filter(\.id == token.userID)
            .first()
            .unwrap(or: Abort(.internalServerError))
            .convertToPublic()
        }
    }
}

Here’s what the route handler does:

1. Get a Redis connection from the pool of connections held by the application.
2. Retrieve the data in Redis using the token sent in the request as the key. Decode the data to Token.
3. Ensure the token exists, otherwise return a 401 Unauthorized response.
4. Query the user database to get the user with the ID from the Token. Ensure the user exists, otherwise throw an internal server error. The application should never store a token in the database with a user ID of a user that does not exist. Return the public representation of the user to avoid sending the user’s password in the response.

Finally, register the route in boot(router:) below basicAuthGroup.post("login", use: loginHandler):

authGroup.post(
  AuthenticateData.self,
  at: "authenticate",
  use: authenticate)

This routes a POST request to /auth/authenticate to authenticate(_:data:), decoding the request body to AuthenticateData. Build and run the application and configure a new request in RESTed as follows:

• URL: http://localhost:8081/auth/login
• method: POST

Click the Authorization button and set Username and Password to the values for the user you created earlier. Ensure you check Present Before Authentication Challenge and click OK. Click Send Request and you’ll see the token returned in the response:

Click Authorization again and uncheck the checkbox. This ensures the HTTP Basic Authentication header isn’t sent with the next request. Configure a new request as follows:

• URL: http://localhost:8081/auth/authenticate
• method: POST

Add a single parameter with the name token and value of the token returned in the previous request. Click Send request and you’ll see the user returned in the response:

Authenticating with other microservices

Go back to the TILAppAcronyms project in Xcode and stop the app and close Xcode. Navigate to the project directory in Terminal and enter the following commands:

touch Sources/App/Middlewares/UserAuthMiddleware.swift
vapor xcode -y

This creates a new file for the middleware you’ll use to talk to the other microservice. The final command regenerates and opens the Xcode project. First, open User.swift and import the Authentication module below import Vapor:

import Authentication

Next, add the following at the bottom of the file, below extension User: Content {}:

extension User: Authenticatable {}

This allows you to add authenticated users to requests, using Vapor’s authentication package. Then, open UserAuthMiddleware.swift and insert the following:

import Vapor

struct AuthenticateData: Content {
  let token: String
}

This represents the data sent to the TILAppUsers microservice to validate tokens. Notice this is the exact same code as used in that microservice. Next, above AuthenticateData, add the middleware to authenticate tokens with the TILAppUsers microservice:

final class UserAuthMiddleware: Middleware {
  // 1
  func respond(to request: Request, chainingTo next: Responder)
    throws -> Future<Response> {
      // 2
      guard let token =
        request.http.headers.bearerAuthorization else {
          throw Abort(.unauthorized)
        }

      // 3
      return try request
        .client()
        .post("http://localhost:8081/auth/authenticate") {
        authRequest in
          // 4
          try authRequest.content
            .encode(AuthenticateData(token: token.token))
          }.flatMap(to: Response.self) { response in
            // 5
            guard response.http.status == .ok else {
              if response.http.status == .unauthorized {
                throw Abort(.unauthorized)
              } else {
                throw Abort(.internalServerError)
              }
            }
            // 6
            let user =
              try response.content.syncDecode(User.self)
            // 7
            try request.authenticate(user)
            // 8
            return try next.respond(to: request)
          }
    }
}

Here’s what the new middleware does:

1. Implement respond(to:chainingTo:) as required by Middleware.
2. Ensure the request contains a bearer token in the Authorization header. Otherwise, return a 401 Unauthorized response.
3. Send a request to the TILAppUsers microservice to validate the token.
4. Encode the token into the request string using the beforeSend parameter of post(_:headers:beforeSend).
5. Ensure the response code is 200 OK. If not, return a 401 Unauthorized if the service returned that status, otherwise return a 500 Internal Server Error.
6. Decode the response body into a User.
7. Authenticate the request with the user returned from the TILAppUsers service.
8. Call the next middleware in the chain.

For more information on middleware, see Chapter 28, “Middleware”.

Use the new middleware to protect the routes that mutate the database. Open AcronymsController.swift and in boot(router:), add the following below router.get("user", UUID.parameter, use: getUsersAcronyms):

let authGroup = router.grouped(UserAuthMiddleware())
authGroup.post(use: createHandler)
authGroup.delete(Acronym.parameter, use: deleteHandler)
authGroup.put(Acronym.parameter, use: updateHandler)

This creates a new route group using UserAuthMiddleware and protects the create, update and delete routes. Delete the following routes that are now duplicated:

router.post(use: createHandler)
router.delete(Acronym.parameter, use: deleteHandler)
router.put(Acronym.parameter, use: updateHandler)

Now those routes contain an authenticated user, change the route handlers to use that user instead. At the bottom of the file, add a new type for the data required to create an acronym:

struct AcronymData: Content {
  let short: String
  let long: String
}

Since the user comes from the request, you only need the short and long properties. Next, replace the body of createHandler(_:) with the following:

// 1
let data = try req.content.syncDecode(AcronymData.self)
// 2
let user = try req.requireAuthenticated(User.self)
// 3
let acronym = Acronym(
  short: data.short,
  long: data.long,
  userID: user.id)
return acronym.save(on: req)

Here’s what the new code does:

1. Get the acronym data from the request body, using the new type created above.
2. Get the authenticated user from the request.
3. Create an Acronym from the user and data and save it.

Next, in updateHandler(_:), replace the type decoded from the request:

return try flatMap(
  to: Acronym.self,
  req.parameters.next(Acronym.self),
  req.content.decode(AcronymData.self)) { acronym, updateData in

This uses AcronymData instead of Acronym. Finally, replace acronym.userID = updateData.userID with the following:

let user = try req.requireAuthenticated(User.self)
acronym.userID = user.id

This uses the ID of the request’s authenticated user. Build and run the app and configure a new request in RESTed as follows:

• URL: http://localhost:8082/
• method: POST
• Parameter encoding: JSON-encoded

Add two parameters with names and values:

• short: IKR
• long: I Know Right

Add a header for Authorization with the value Bearer . Click Send Request. You’ll see the new acronym returned in the response:

There are a number of options for authenticating requests across microservices. For large applications, you could split the authentication out into another microservice. You may also want authentication between microservices, even if the original request from the user doesn’t need it. Finally, another option is to use JWT (JSON Web Tokens). These are JSON tokens that contain information encoded in them and a signature. They are useful because the signature ensures you can trust the token without needing a request to another microservice.

Where to go from here?

In this chapter, you learned how to split the TIL app into different microservices for users and acronyms. You’ve seen how to handle authentication and relationships across different services.

In the next chapter, you’ll build another microservice that acts as a gateway for clients to access the different services. You’ll also learn how to build and run the different services together easily on Linux using Docker.