Parallel Map in Kotlin
Ever wonder how to run
map in parallel using coroutines? This is how you do it.
Confused? Let’s unpack it.
First we have the function signature which is pretty similar to the actual
map extension function signature on
Iterable. The only thing we added were the
suspend keywords. One for our function and another one on the parameter.
Then we have the
coroutineScope that marks the scope in which the
async calls are going to be executed. This way if something goes wrong and an exception is thrown, all coroutines that were launched in this scope are cancelled. That’s the main benefit of structured concurrency.
Finally we have the actual execution which is divided in 2 steps: The first step launches a new coroutine for each function application using
async. This effectively wraps the type of each element with
Deferred. In the second step we wait for all function applications to complete and unwrap the result with
How to use it
Easy! Just like you use
(Psst! I’m using Kotlin Playground so you can actually run this code!)
Prove that it’s running in parallel
Ok so let’s resort to the good old
delay to prove that this is actually running in parallel. We are going to add a delay of 1 second on each multiplication and measure the time it takes to run.
Running over 100 elements the result should be: close to 1,000 milliseconds if it’s running in parallel and close to 100,000 milliseconds if it’s running sequentially.
A previous iteration of this article proposed the following solution:
And the description said:
Then we have the
runBlockingwhich lets us bridge the blocking code with the coroutine world. As the name suggests this will block the current thread until everything inside the block finishes executing. Which is exactly what we want.
Fortunately Gildor commented on why this a bad idea. First, we were not using Structured Concurrency, so an invocation of
f could fail and all other executions would continue unfazed. And also we were not playing nice with the rest of the code. By using
runBlocking we were forcefully blocking the thread until the whole execution of
pmap finishes, instead of letting the caller decide how the execution should go.
Since I’m not explicitly passing any
DefaultDispatcherwill be used. ↩