Parallel Map in Kotlin
Ever wonder how to run map
in parallel using coroutines? This is how you do it.
Confused? Let’s unpack it.
First we have the function signature which is pretty similar to the actual map
extension function signature on Iterable
. The only thing we added were the suspend
keywords. One for our function and another one on the parameter.
Then we have the coroutineScope
that marks the scope in which the async
calls are going to be executed. This way if something goes wrong and an exception is thrown, all coroutines that were launched in this scope are cancelled. That’s the main benefit of structured concurrency.
Finally we have the actual execution which is divided in 2 steps: The first step launches a new coroutine for each function application using async
. This effectively wraps the type of each element with Deferred
. In the second step we wait for all function applications to complete and unwrap the result with await
.1
How to use it
Easy! Just like you use map
:
(Psst! I’m using Kotlin Playground so you can actually run this code!)
Prove that it’s running in parallel
Ok so let’s resort to the good old delay
to prove that this is actually running in parallel. We are going to add a delay of 1 second on each multiplication and measure the time it takes to run.
Running over 100 elements the result should be: close to 1,000 milliseconds if it’s running in parallel and close to 100,000 milliseconds if it’s running sequentially.
Why not runBlocking
?
A previous iteration of this article proposed the following solution:
And the description said:
Then we have the
runBlocking
which lets us bridge the blocking code with the coroutine world. As the name suggests this will block the current thread until everything inside the block finishes executing. Which is exactly what we want.
Fortunately Gildor commented on why this a bad idea. First, we were not using Structured Concurrency, so an invocation of f
could fail and all other executions would continue unfazed. And also we were not playing nice with the rest of the code. By using runBlocking
we were forcefully blocking the thread until the whole execution of pmap
finishes, instead of letting the caller decide how the execution should go.
-
Since I’m not explicitly passing any
CoroutineContext
theDefaultDispatcher
will be used. ↩