7. Context Switch & Dispatching

Written by Luka Kordić

Right about now, you’ve amassed a good amount of knowledge about coroutines, suspendable functions and the Kotlin’s Coroutines API. But you haven’t learned much about how you can deal with threading, and which threading solutions exist within the API itself. However, you did learn what the CoroutineContext is, and what it’s used for. Having the ability to combine multiple CoroutineContexts, and different context types, to produce powerful coroutine mechanisms makes coroutines really extensible and versatile.

In fact, the CoroutineContext is a fundamental part of something called context switching, and the process of dispatching, which in turn revolves around threading.

Work Scheduling

Organizing work at a system level is the bread and butter of all things related to multi-threading and parallel computing. Back in the day, when systems had a single core processor and could only utilize a single thread, it was extremely important to write optimized organizing algorithms so that the system didn’t freeze up and so that actions didn’t take forever to complete.

The process of figuring out the order, severity and resource usage for units of work the system needs to complete is called scheduling. Just like with a regular schedule, which holds all your meetings and chores, it serves to best organize which event should happen before others. It also deals with where the work lives in memory and when it starts or ends — the lifecycle and how it behaves when things break.

So when the system receives, let’s say, five events it needs to process, it first looks at the computational power it has available. If it’s already under 70% load, then it cannot take on a task which would require 40% of the total load. It then tries to fill in the available computational power by dividing the resources between other tasks which won’t overload the system. But there’s a caveat here, if you keep trying to fill in the workload with smaller tasks, you may never get to free up enough computational power to finally process the bigger unit of work.

In an operating system, all of these responsibilities belong to a construct called a scheduler. Schedulers decide when and how they should assign the computer’s resources to which tasks. They also take care of the lifecycle of the work you give them, since the events won’t start until a scheduler gives the system a green light, nor will they finish until they are completely processed. If any of the events breaks, an exception occurs, the scheduler is notified, and the system kills the process.

In terms of coroutines and modern-day systems, scheduling usually comes down to the distribution and organization of work between threads in thread pools. They allow the system to abstract away all of the responsibilities in one seemingly simple object.

Swimming in a Pool of Threads

A thread pool is a number of threads pooled together and distributed between work events that the system receives in its queue. Today’s hardware supports doing multiple things at the same time and effectively handling quite a few times the amount of work than before due to multiple cores. Combining that with the fact that coroutines can be completed one piece at a time, instead of running the entire operation, it can make coroutines extremely performant. This allows you to run several coroutines at once and schedule threads in such a way that each of the threads does a bit of work on each of the coroutines until all of the work is done, while all of the threads are constantly being reassigned.

Internally, this is where thread pools kick in. You can tell a thread pool to complete five coroutines, like the example above. The thread pool will then assign threads to each of the coroutines, effectively switching them out if needed and suspending coroutines if some work that’s higher priority comes up — like an important system call that is triggered from outside of your application. Once the thread is free again, it returns to the thread pool, and the system once again decides if there’s work to be done or if it should hold on and wait for more work events.

It’s also important to know how the system handles the state of each of the threads.

Context Switching

In the Coroutines API, you don’t have to worry about creating your own threads or thread pools, or about scheduling how multiple coroutines are executed, and their lifecycle. The Coroutines API has a specific way of communicating all of this information — via ContinuationInterceptors, which you provide through Dispatchers, which you’ll learn about later in this chapter.

To fully understand how these Dispatchers work, it’s important to understand the underlying pattern of communication of the process and thread state, which the system prepares for you. This pattern is called context switching. However, the definition varies from single to multi-tasking systems. But we’ll focus on multi-tasking in this chapter.

Essentially, when the system switches the context, it means that it’s moving from one task to another, saving the state of the previous task, so it can be resumed later on. This sounds familiar? It’s very similar to what the Continuation does, internally, when it comes to suspension points in a suspendable function. Well, this is also why all the dispatchers actually implement ContinuationInterceptor, because through the process of intercepting Continuations, and their execution flow, can the system suspend and resume - switch the context, of the current task, or function, at hand.

But pausing and resuming tasks is not everything, the system should also be able to switch between the threads in a single task. When you think about it, these two concepts stand toe-to-toe in Kotlin Coroutines.

If you need to do something in the background, and then switch to the main thread, posting a value or some result of an operation, you ultimately create another coroutine, push it to the main thread, and then switch to that coroutine from within. This is basically context switching, with the addition that it switched between threads. So it’s no coincidence that the most important part of every coroutine is called the CoroutineContext.

Now that you understand a bit how the system can handle coroutines’ context switching, it’s time to move onto dispatching! :]

Explaining ContinuationInterceptors

Even though this chapter mentions ContinuationInterceptors, it may still be a bit unclear on how they work. If you remember from the diagram of what happens with functions in the call stack and when suspendable functions are called:

When you had multiple functions in the stack, and multiple continuations, you learned that you can return all the way down to the main Continuation, by propagating the value, or an exception, back down the stack.

ContinuationInterceptors work with that function execution and threading. Every time you launch a coroutine, or call a suspend function with a Dispatcher, you give the interceptor the ability to pause and resume the continuation of the coroutine - the execution flow. It can intercept value propagation at one point, and redirect it to another coroutine or task.

Because of that, if you create one coroutine using Dispatchers.Default to get some value, and then within it, you launch a new coroutine with Dispatchers.Main to push it on the main thread, you’ll effectively intercept the first coroutine’s execution, continue on with the second coroutine passing in the context and values so you can do some work on the main thread, and then you finish both coroutines when you’re done. If anything goes wrong in the second coroutine, the interceptor will propagate the exception all the way up, to the parent coroutine, cancelling both coroutines.

This type of behavior is achieved through the process of wrapping continuations. Every time you switch the context with a ContinuationInterceptor, it creates a new Continuation, by wrapping the previous one, using interceptContinuation(). The signature of the function is the following:

abstract fun <T> interceptContinuation(
    continuation: Continuation<T>
): Continuation<T>

It is very simple, but also very powerful. Any time the system signals a function’s Continuation, with a new value or an exception, the ContinuationInterceptor can take that Continuation, do some work with it, and finally resume execution. In case of Dispatchers the work ContinuationInterceptors do is generally context switching, by shifting from one Thread pool to another.

Coroutine Dispatcher Types

Kotlin provides a concise way of communicating threading options in coroutines using Dispatchers. They are a CoroutineContext.Element implementation, forming one part of the puzzle that handles how coroutines behave when executed. In general computing, a dispatcher is a module that gives control of the CPU to whichever process the scheduling mechanism selected for execution. So a scheduler decides which process is next in line for a bit of CPU power, but it passes the process down to a dispatcher to allow the process to use up the actual resources. Together, these two modules or mechanisms control processes in an operating system.

A similar thing happens with CoroutineDispatchers. They decide how coroutines use up available resources by delegating threads or thread pools to them. Once you attach a certain dispatcher to a coroutine, it is assigned to a thread or thread pool the dispatcher knows about.

Since they deal with threads, dispatchers in coroutines can be confined and unconfined. Confined dispatchers always rely on predefined system contexts — like the Dispatchers.Main. No matter how many times you use the Main dispatcher, it will always make the coroutine work on the main thread. Unconfined dispatchers, on the other hand, don’t have a specific context they operate in, nor do they follow any strict rules. They either create new threads to run coroutines in or push the work to the thread in which the code was called, making them unpredictable.

There’s only a finite number of pre-defined dispatchers, and these are:

• Dispatchers.Default: The default threading strategy for starting coroutines, confined to the parent’s context, usually a thread pool of workers.
• Dispatchers.IO: Similar to Default, it’s based on the JVM, and is backed by a thread pool to offload IO-related tasks.
• Dispatchers.Main: The main thread dispatcher, connected to the thread, which operates with UI objects.
• Dispatchers.Unconfined: The name states it, it’s unconfined, and it will run on whichever thread is currently using it.

Let’s go over each of them individually and see what you can use them for.

Default Dispatcher

The default dispatcher’s name pretty much gives it away. It’s used in the foundation of coroutines and is used whenever you don’t specify a dispatcher. It’s convenient to use because it’s backed by a worker thread pool, and the number of tasks the Default dispatcher can process is always equal to the number of cores the system has, and is at least two. Because the entire threading mechanism and the thread pool is pre-built, you can rely on it for your day-to-day work related to coroutines and operations you want to off-load from the main thread.

IO Dispatcher

Again, the name says a lot. Whenever you’re trying to process something with input and output, like uploading or decrypting/encrypting files, you can use this dispatcher to make things easier for you. That being said, it’s bound to the JVM, so if you’re using Kotlin/JavaScript or Kotlin/Native projects, you won’t be able to use it.

Main Dispatcher

This dispatcher is tied to systems that have some form of user interface, such as Android or visual Java applications. And as mentioned, it dispatches work to the thread that handles UI objects. You cannot use this without a UI, and if you try to call Dispatchers.Main in a project that doesn’t use Swing, JavaFX or isn’t an Android app, your code will crash.

It’s best used within another coroutine after you fetch the data you need, and then handle all the logic before displaying it. You simply post the data back to the main thread, and have it render. Or better yet, you can run the coroutine on the main thread, bridging to the background, using withContext or async/await, ultimately pulling the result back to the main thread for rendering.

Using Dispatchers

Now that you know which dispatchers are out there, it’s time to learn how to utilize them. Import this chapter’s starter project, using IntelliJ, and selecting Import Project, and navigating to the context-switch-and-dispatching/projects/starter folder, selecting the context-switch-and-dispatching project. Open Main.kt and have a look at the code:

fun main() {
  GlobalScope.launch {
    println("This is a coroutine")
  }
  Thread.sleep(50)
}

You can’t tell anything about the threading or scheduling, which happens behind the scenes. Let’s review the launch function’s signature:

public fun CoroutineScope.launch(
  context: CoroutineContext = EmptyCoroutineContext,
  start: CoroutineStart = CoroutineStart.DEFAULT,
  block: suspend CoroutineScope.() -> Unit
): Job

The first part is important, here. It uses EmptyCoroutineContext by default, unless you specify a different one. You’ve learned that the context defines how the coroutine is started and where, how it handles errors, and what its lifecycle is. An EmptyCoroutineContext doesn’t have any error-handling defined, it has no parent context, it uses the default lifecycle management, and it doesn’t have a CoroutineInterceptor, so it uses Dispatchers.Default.

From what you’ve learned about the dispatchers, this means the coroutine will use a predefined pool of threads to do its work. If you want to use any other dispatcher, simply pass it in to replace the EmptyCoroutineContext default argument. Once you do that, the dispatcher will be used as the context of the coroutine, and it will dictate all-things-threading.

Next, change the example a little. Instead of printing some dummy text, make the coroutine print the thread it’s located in, replacing the previous code with:

fun main() {
  GlobalScope.launch { println(Thread.currentThread().name) }

  Thread.sleep(50)
}

If you run this, it will print out something similar to this:

DefaultDispatcher-worker-1

This supports what you know about the default dispatcher. You get the same result with the following:

fun main() {
  GlobalScope.launch(context = Dispatchers.Default) { 
    println(Thread.currentThread().name) 
  }

  Thread.sleep(50)
}

If you were to pass in a different dispatcher, you’d get different results. An example would be the Dispatchers.Unconfined instance. So if you had the following code:

fun main() {
  GlobalScope.launch(context = Dispatchers.Unconfined) { 
    println(Thread.currentThread().name) 
  }

  Thread.sleep(50)
}

It will print out main as it’s thread. This is because the unconfined dispatcher just takes in whichever thread the code is run in, and attaches the coroutine to that thread. But what if you don’t want to confine your code to a certain set of constrains, which the Coroutines API provides for you? What if you need tasks which require a separate thread?

Creating a Work-Stealing Executor

With the standard Coroutines API, you also have the ability to create new threads or thread pools for coroutines. This is done by creating a new Executor. Executors are objects that execute given tasks. They are usually tied with Runnables, since they wrap the task in a runnable, which needs executing. Creating a work-stealing executors for example means that it will use all available resources at its disposal, in order to achieve a certain level of parallelism, which you can define. To use the work-stealing-executor, replace the previous code with the following:

fun main() {
  val executorDispatcher = Executors
      .newWorkStealingPool()
      .asCoroutineDispatcher()

  GlobalScope.launch(context = executorDispatcher) {
    println(Thread.currentThread().name)
  }

  Thread.sleep(50)
}

If you run this code, it will print out something similar to this:

ForkJoinPool-1-worker-9

The executor uses all available resources here, like the ForkJoinPool thread pool, to finish your task. Once it’s done with the task, it can re-allocate the taken resources to the rest of the application. If you were to pass that executor the parallelism level of four, and you used the executor in many coroutines, it would distribute resources to achieve four parallel executions as long as there’s work to be distributed.

If you want to check out this final example in the final project, import this chapter’s final project, using IntelliJ, and selecting Import Project, and navigating to the context-switch-and-dispatching/projects/final folder, selecting the context-switch-and-dispatching project.

Key Points

• One of the most important concepts in computing, when using execution algorithms, is scheduling and context switching.
• Scheduling takes care of resource management by coordinating threading and the lifecycle of processes.
• To communicate thread and process states in computing and task execution, the system uses context switching and dispatching.
• Context switching helps the system store thread and process state, so that it can switch between tasks which need execution.
• Dispatching handles which tasks get resources at which point in time.
• ContinuationInterceptors, which take care of the input/output of threading, and the main and background threads are provided through the Dispatchers class.
• Dispatchers can be confined and unconfined, where being confined or not relates to using a fixed threading system.
• There are four main dispatchers: Default, IO, Main and Unconfined.
• Using the Executors class you can create new thread pools to use for your coroutine work.
• Simply put, when lauching a new coroutine, with a different Dispatcher, you’re switching the context of coroutines, as you’re switching between two tasks and, in turn, threads.