26. Database/API Versioning & Migration

Written by Jonas Schwartz

Note: This update is an early-access release. This chapter has not yet been updated to Vapor 4.

In the first three sections of the book, whenever you made a change to your model, you had to delete your database and start over. That’s no problem when you don’t have any data. Once you have data, or move your project to the production stage, you can no longer delete your database. What you want to do instead is modify your database, which in Vapor, is done using migrations.

Note: This chapter requires that you have set up and configured PostgreSQL. Follow the steps in Chapter 6, “Configuring a Database”, to set up PostgreSQL in Docker and configure the Vapor application.

In this chapter, you’ll make two modifications to the TILApp using migrations. First, you’ll add a new field to User to contain a Twitter handle. Second, you’ll ensure that categories are unique. Finally, you’re going to modify the app so it creates the admin user only when your app runs in development or testing mode.

Note: The version of TILApp provided for this chapter’s sample files is not the complete version from the end of Section 3. Instead, it’s a simplified, earlier iteration. You can integrate these changes in your working copy of the project, if you wish.

Modifying tables

Modifying an existing database is always a risky business. You already have data you don’t want to lose, so deleting the whole database is not a viable solution. At the same time, you can’t simply add or remove a property in an existing table since all the data is entangled in one big web of connections and relations.

Instead, you introduce your modifications using Vapor’s Migration protocol. This allows you to cautiously introduce your modifications while still having a revert option should they not work as expected.

Modifying your production database is always a delicate procedure. You must make sure to test any modifications properly before rolling them out in production. If you have a lot of important data, it’s a good idea to take a backup before modifying your database.

To keep your code clean and make it easy to view the changes in chronological order, you should create a directory containing all your migrations. Each migration should have its own file. For file names, use a consistent and helpful naming scheme, for example: YY-MM-DD-FriendlyName.swift. This allows you to see the versions of your database at a glance.

Writing migrations

A Migration is generally written as a struct when it’s used to update an existing model. This struct must, of course, conform to Migration. Migration requires you to provide three things:

typealias Database: Fluent.Database

static func prepare(
  on connection: Database.Connection) -> Future<Void>

static func revert(
  on connection: Database.Connection) -> Future<Void>

Typealias Database

First, you must specify what type of database the migration can run on. Migrations require a database connection to work correctly as they must be able to query the MigrationLog model. If the MigrationLog is not accessible, the migration will fail and, in the worst case, break your application.

Prepare method

prepare(on:) contains the migration’s changes to the database. It’s usually one of two options:

• Creating a new table
• Modifying an existing table by adding a new property.

Here’s an example that adds a new model to the database:

static func prepare(
  on connection: PostgreSQLConnection) -> Future<Void> {
  // 1
  return Database.create(
    NewTestUser.self,
    on: connection) { builder in
      // 2
      builder.field(for: \.id, isIdentifier: true)
  }
}

1. You specify the action to perform and the model to use. If you’re adding a new Model type to the database, you use create(_:on:closure:). If you’re adding a field to an existing Model type, you use update(_:on:closure:). This example uses create(_:on:closure:) to create a new model with the field id.
2. Next, you specify a closure that accepts a SchemaBuilder for your model and performs the actual modifications. You call field(for:isIdentifier:) on the builder to describe each field you’re adding to your model. Normally, you don’t need to include the type of the field as Fluent can infer the best one to use.

Revert method

revert(on:) is the opposite of prepare(on:). Its job is to undo whatever prepare(on:) did. If you use create(_:on:closure:) in prepare(on:), you use delete(_:on:) in revert(on:). If you use update(_:on:closure:) to add a field, you also use it in revert(on:) to remove the field with deleteField(for:).

Here’s an example that pairs with the prepare(on:) you saw earlier:

static func revert(
  on connection: PostgreSQLConnection) -> Future<Void> {
    return Database.delete(NewTestUser.self,
                           on: connection)
}

Again, you specify the action to perform and the model to revert. Since you used create(_:on:closure:) to add the model, you use delete(_:on:) here.

This method executes when you boot your app with the --revert option.

Adding users’ Twitter handles

To demonstrate the migration process for an existing database, you’re going to add support for collecting and storing users’ Twitter handles. First, you need to create a new folder to hold all your migrations and a new file to hold the AddTwitterToUser migration. In Terminal, navigate to the directory which holds your TILApp project and enter:

# 1
mkdir Sources/App/Migrations
# 2
touch Sources/App/Migrations/18-06-05-AddTwitterToUser.swift
# 3
vapor xcode -y

Here’s what this does:

1. Create a new directory, Migrations, in the App module.
2. Create a new file, 18-06-05-AddTwitterToUser.swift, in the Migrations directory you just created.
3. Regenerate the Xcode project to add the new file to the App target.

Next, open User.swift in Xcode and add the following property to User below var password: String:

var twitterURL: String?

This adds the property of type String? to the model. You declare it as an optional string since your existing users don’t have the property and future users don’t necessarily have a Twitter account.

Next, replace the initializer with the following to account for the new property:

init(name: String,
     username: String,
     password: String,
     twitterURL: String? = nil) {
  self.name = name
  self.username = username
  self.password = password
  self.twitterURL = twitterURL
}

Creating the migration

When you use a migration to add a new property to an existing model, it’s important you modify the initial migration so that it adds only the original fields. By default, prepare(on:) adds every property it finds in the model. If, for some reason — running your test suite, for example — you revert your entire database, allowing it to continue to add all fields in the initial migration will cause your new migration to fail.

Find the existing prepare(on:) in the User: Migration extension and replace try addProperties(to: builder) with the following:

builder.field(for: \.id, isIdentifier: true)
builder.field(for: \.name)
builder.field(for: \.username)
builder.field(for: \.password)

This manually adds the existing properties — excluding the new twitterURL — to the database.

Next, open 18-06-05-AddTwitterToUser.swift and add the following to create a migration that adds the new twitterURL field to the model:

import FluentPostgreSQL
import Vapor

// 1
struct AddTwitterURLToUser: Migration {

  // 2  
  typealias Database = PostgreSQLDatabase

  // 3  
  static func prepare(
  	on connection: PostgreSQLConnection
  ) -> Future<Void> {
    // 4
    return Database.update(
      User.self, 
      on: connection
    ) { builder in
      // 5
      builder.field(for: \.twitterURL)
    }
  }

  // 6  
  static func revert(
  	on connection: PostgreSQLConnection
  ) -> Future<Void> {
    // 7
    return Database.update(
      User.self, 
      on: connection
    ) { builder in
      // 8
      builder.deleteField(for: \.twitterURL)
    }
  }
}

Here’s what this does:

1. Define a new type, AddTwitterURLToUser, that conforms to Migration.
2. As required by Migration, define your database type with a typealias.
3. Define the required prepare(on:).
4. Since User already exists in your database, use update(_:on:closure:) to modify the database.
5. Inside the closure, use field(for:) to add a new field corresponding to the key path \.twitterURL.
6. Define the required revert(on:).
7. Since you’re modifying an existing Model, you again use update(_:on:closure:) to remove the new field.
8. Inside the closure, use deleteField(for:) to remove the field corresponding to the key path \.twitterURL.

Now open configure.swift and register AddTwitterURLToUser as one of the migrations.

Since migrations are performed in order, it must be after the existing migrations in the list. Add the following immediately before services.register(migrations):

migrations.add(
  migration: AddTwitterURLToUser.self,
  database: .psql)

The next time you launch the app, the new property is added to User. As with AdminUser, you should use the add(migration:database:) to register the migration since it isn’t a full model. Build and run your application; you should be able to see the new property in your table.

On your development machine, you can see the table’s properties by entering the following in Terminal:

docker exec -it postgres psql -U vapor
\d "User"
\q

Versioning the API

You’ve changed the model to include the user’s Twitter handle but you haven’t altered the existing API. While you could simply update the API to include the Twitter handle, this might break existing consumers of your API. Instead, you can create a new API version to return users with their Twitter handles.

To do this, first open User.swift and add following definition after Public:

final class PublicV2: Codable {
  var id: UUID?
  var name: String
  var username: String
  var twitterURL: String?

  init(id: UUID?, 
       name: String, 
       username: String, 
       twitterURL: String? = nil) {
    self.id = id
    self.name = name
    self.username = username
    self.twitterURL = twitterURL
  }
}

This creates a new PublicV2 class that includes the twitterURL. Next, add the following to the end of the file to conform this new class to Content:

extension User.PublicV2: Content {}

Next, create the two convert function for the version 2 API. Add the following to the extension for User after convertToPublic():

func convertToPublicV2() -> User.PublicV2 {
  return User.PublicV2(
  	id: id, 
  	name: name, 
  	username: username, 
  	twitterURL: twitterURL)
}

Now, add the following to the extension for Future after convertToPublic():

func convertToPublicV2() -> Future<User.PublicV2> {
  return self.map(to: User.PublicV2.self) { user in
    return user.convertToPublicV2()
  }
}

Finally, open UsersController.swift and add the following after getHandler(_:):

 // 1
func getV2Handler(_ req: Request) throws 
    -> Future<User.PublicV2> {
  // 2
  return try req.parameters.next(User.self).convertToPublicV2()
}

This method is just like getHandler(_:) with two changes:

1. Return a User.PublicV2.
2. Call convertToPublicV2() to produce the correct return item.

Now, add the following at the end of boot(router:):

// API Version 2 Routes
 // 1
let usersV2Route = router.grouped("api", "v2", "users")
 // 2
usersV2Route.get(User.parameter, use: getV2Handler)

Here’s what this does:

1. Add a new API group that will resolve on /api/v2/users.
2. Connect GET requests to getV2Handler().

Now you have a new endpoint to get a user, with a v2 in the API, that returns the twitterURL.

Note: For a more complicated API revision, you should create new controllers to handle the new API version. This will simplify how you reason about the code and make it easier to maintain.

Updating the web site

Your app now has all it needs to store a user’s Twitter handle and the API is complete. You need to update the web site to allow a new user to provide a Twitter address during the registration process.

Open register.leaf and add the following after the form group for name:

<div class="form-group">
  <label for="twitterURL">Twitter handle</label>
  <input type="text" name="twitterURL" class="form-control"
   id="twitterURL"/>
</div>

This adds a field for the Twitter handle on the registration form. Next, open user.leaf and replace <h2>#(user.username)</h2> with the following:

<h2>#(user.username)
#if(user.twitterURL) {
 - #(user.twitterURL)
}
</h2>

This shows the Twitter handle, if it exists, on the user information page. Finally, open WebsiteController.swift and add the following to the end of RegisterData:

let twitterURL: String?

This allows your form handler to access the Twitter information sent from the browser. In registerPostHandler(_:data:), replace

let user = User(
  name: data.name,
  username: data.username,
  password: password)

With:

var twitterURL: String?
if 
  let twitter = data.twitterURL,
  !twitter.isEmpty {
    twitterURL = twitter
}
let user = User(
  name: data.name,
  username: data.username,
  password: password,
  twitterURL: twitterURL)

If the user doesn’t provide a Twitter handle, you want to store nil rather than an empty string in the database.

Build and run. Visit http://localhost:8080/ in your browser and register a new user, providing a Twitter handle. Visit the user’s information page to see the results of your handiwork!

Making categories unique

Just as you’ve required usernames to be unique, you really want category names to be unique as well. Everything you’ve done so far to implement categories has made it impossible to create duplicates but you’d like that enforced in the database as well. It’s time to create a Migration that guarantees duplicate category names can’t be inserted in the database.

First, create a new file inside the Migrations directory. In Terminal, enter:

touch Sources/App/Migrations/18-06-05-MakeCategoriesUnique.swift
vapor xcode -y

This creates a new file to contain the new Migration and regenerates your Xcode project.

In Xcode, open 18-06-05-MakeCategoriesUnique.swift and enter the following:

import FluentPostgreSQL
import Vapor
// 1
struct MakeCategoriesUnique: Migration {
  // 2  
  typealias Database = PostgreSQLDatabase
  // 3  
  static func prepare(
    on connection: PostgreSQLConnection
  ) -> Future<Void> {
    // 4
    return Database.update(
      Category.self,
      on: connection
    ) { builder in
      // 5
      builder.unique(on: \.name)
    }
  }
  // 6  
  static func revert(
    on connection: PostgreSQLConnection
  ) -> Future<Void> {
    // 7
    return Database.update(
      Category.self,
      on: connection
    ) { builder in
      // 8
      builder.deleteUnique(from: \.name)
    }
  }
}

1. Define a new type, MakeCategoriesUnique, that conforms to Migration.
2. As required by Migration, define your database type with a typealias.
3. Define the required prepare(on:).
4. Since Category already exists in your database, use update(_:on:closure:) to modify the database.
5. Inside the closure, use unique(on:) to add a new unique index corresponding to the key path \.name.
6. Define the required revert(on:).
7. Since you’re modifying an existing Model, you again use update(_:on:closure:) to remove the new index.
8. Inside the closure, use deleteUnique(from:) to remove the index corresponding to the key path \.name.

Finally, open configure.swift and register MakeCategoriesUnique as one of the migrations. Add the following immediately before services.register(migrations):

migrations.add(
  migration: MakeCategoriesUnique.self,
  database: .psql)

Build and run; observe the new migration in the console.

Seeding based on environment

In Chapter 18, “API Authentication, Part 1,” you seeded an admin user in your database. As mentioned there, you should never use “password” as your admin password. But, it’s easier when you’re still developing and just need a dummy account for testing locally. One way to ensure you don’t add this user in production is to detect your environment before adding the migration. In configure.swift replace:

migrations.add(migration: AdminUSer.self, database: .psql)

With the following:

switch env {
case .development, .testing:
  migrations.add(migration: AdminUser.self, database: .psql)
default:
  break
}

Now the AdminUser is only added to the migrations if the application is in either the development (the default) or testing environment. If the environment is production, the migration won’t happen. Of course, you still want to have an admin in your production environment that has a random password. In that case you can switch on the environment inside AdminUser or you can create two versions, one for development and one for production.

Where to go from here?

In this chapter, you learned how to modify your database after your app enters production using migrations. You saw how to add an extra property — twitterUrl — to User, how to revert this update, and how to enforce uniqueness of category names. Finally, you saw how to switch on your environment in configure.swift, allowing you to exclude migrations from the production environment.

You can learn more about migrations in the Vapor documentation at https://docs.vapor.codes/3.0/fluent/migrations/.