Java: Executors, Futures and CompletableFuture

Description

Creating a new thread for every piece of work is like hiring a new cashier for every customer: slow to set up, and with a sudden crowd you'd have thousands of cashiers and no room for them. An executor is the better arrangement: a fixed team of threads (a thread pool) and a waiting line of tasks. You hand it work; it decides which thread runs it and when.

Almost all concurrent Java code runs on executors: web servers, message consumers, scheduled jobs, parallel calls to other services. Their defaults are also behind some classic outages, like a service that ran out of memory because work piled up in a queue nobody could see. This skill covers how to use executors, how a pool decides to grow, and how to chain asynchronous work with CompletableFuture.

Here's the path, from easiest to most technical:

1. Submitting work and getting results back with a Future.
2. Shutting a pool down properly.
3. How a pool grows, where work waits, and what happens when it's full.
4. How many threads a pool should have.
5. CompletableFuture: describing what happens when a result arrives.

Submitting work and reading results

ExecutorService pool = Executors.newFixedThreadPool(4);         // 4 worker threads

Future<Quote> future = pool.submit(() -> fetchQuote("ACME"));   // runs on a pool thread
Quote quote = future.get(2, TimeUnit.SECONDS);                  // wait for the result, at most 2 s

A Future is a receipt for work that's still running. A few things to know about it:

• Exceptions are kept inside the Future. If the task throws, nothing is printed; the exception waits there until someone calls get(), which throws an ExecutionException with the original as its cause. A future nobody reads is a failure nobody sees.
• get(timeout) stops waiting, not the task. It throws a TimeoutException, but the task keeps running. To stop it, call future.cancel(true), which interrupts the thread (and only helps if the task responds to interruption).
• Periodic tasks stop on the first exception. A task scheduled with scheduleAtFixedRate that throws once is never run again, silently. Catch exceptions inside periodic tasks.

Shutting down

A pool's threads keep the program alive until you shut it down:

pool.shutdown();                                   // accept no new tasks; finish the queued ones
if (!pool.awaitTermination(30, TimeUnit.SECONDS)) {
    pool.shutdownNow();                            // interrupt running tasks, drop the queued ones
}

shutdown() returns immediately; it doesn't wait. awaitTermination does the waiting. Since Java 19 you can also write try (ExecutorService pool = ...) { ... }, which shuts the pool down and waits at the end of the block.

How a pool grows

Every pool from the Executors factory methods is a ThreadPoolExecutor with five settings: a core size, a maximum size, a queue, a keep-alive time for extra threads, and a rejection policy. When a task arrives, the pool decides like this:

The surprise is the order: the queue fills up before any extra threads start. A pool with core 2, max 10 and a queue of 100 runs ten slow tasks on just two threads, with eight waiting.

That explains the two factory methods to be careful with:

• newFixedThreadPool(n) has an unbounded queue. It never grows and never rejects; when work arrives faster than it finishes, the queue just grows in memory. When a slow downstream service made tasks take seconds instead of milliseconds, that queue is how a service quietly ran out of memory.
• newCachedThreadPool() has no queue capacity and no thread limit, so a burst of slow tasks can create thousands of threads.

The safer setup is a pool with a bounded queue and a deliberate rejection policy:

How many threads?

It depends on what the tasks spend their time doing:

• CPU-bound work (calculations, parsing, compression) can't go faster with more threads than cores: Runtime.getRuntime().availableProcessors().
• I/O-bound work (database queries, HTTP calls) spends most of its time waiting, so more threads help. A classic estimate is cores × (1 + wait time ÷ compute time): a task that waits 90 ms for every 10 ms of CPU needs about 10 threads per core.

Measure rather than guess, and give separate dependencies separate pools, so one slow service can't tie up every thread (the bulkhead pattern). For lots of blocking I/O, Java 21's virtual threads remove the sizing problem; the Java 21 module covers them.

CompletableFuture

future.get() makes you wait. A CompletableFuture lets you say what should happen when the result arrives, as a chain of steps:

CompletableFuture<Receipt> receipt =
    CompletableFuture.supplyAsync(() -> loadCart(userId), ioPool)   // start: run on ioPool
        .thenApply(cart -> price(cart))                             // then: turn a value into a value
        .thenCompose(total -> payments.charge(userId, total))       // then: start more async work
        .exceptionally(e -> Receipt.failed(e));                     // if anything above failed

The steps you'll use most:

Three rules prevent most mistakes:

1. Use thenCompose for a step that returns a future. With thenApply, you'd get a CompletableFuture<CompletableFuture<Receipt>>.
2. A failure skips the remaining steps until it reaches an exceptionally or handle. If loadCart throws, price and charge never run.
3. Pass your own executor for anything that blocks. Without one, supplyAsync runs on the JVM-wide common pool, which has only about one thread per CPU core and is shared with parallel streams and other libraries. Blocking HTTP or database calls there slow down everything else in the program.

To get the final value, join() waits and throws an unchecked CompletionException on failure; get() throws a checked ExecutionException. Both wrap the original exception.

Common mistakes

Check yourself

Try to answer before opening each one. Recalling beats re-reading.

1. A pool has core 2, max 10 and a queue of 100. Ten slow tasks arrive. How many threads run them?

2. A task submitted with submit throws an exception. Where does the exception go?

3. What's the difference between thenApply and thenCompose?

4. Where does CompletableFuture.supplyAsync(() -> httpCall()) run, and why does it matter?

Key takeaways

• Use executors, not raw threads, and always read your Futures (or catch inside the task).
• A pool fills core threads, then the queue, then extra threads, then rejects: bound the queue and choose a rejection policy.
• Size pools by workload: about the core count for CPU work, more for I/O.
• shutdown() plus awaitTermination (or try-with-resources) to stop cleanly.
• CompletableFuture: thenCompose to chain futures, exceptionally or handle to recover, and your own executor for blocking steps.