executors and thread pools

the picture

a restaurant kitchen. there are a few cooks on permanent staff (the core threads), a ticket rail where orders wait (the queue), and a phone number for temp cooks (threads up to maximumPoolSize). a new order goes to a free permanent cook if the kitchen is still hiring permanent staff. otherwise it goes on the rail. only when the rail is completely full does the manager call a temp, who takes the newest order straight away while older tickets keep waiting. and when the rail is full and every temp is busy, the door gets a "sorry, we're full" sign (the rejection policy). temps who stand around idle for too long get sent home (keepAliveTime).

why not just new Thread()

platform threads are expensive to create and each one reserves a stack. starting one per request means thread creation on every request and no upper bound when traffic spikes. a pool fixes both: threads are reused, and the number of them is capped.

the API is three interfaces deep:

Executor e = command -> new Thread(command).start();   // just "run this, somehow"
 
ExecutorService pool = Executors.newFixedThreadPool(4); // + lifecycle and results
Future<Integer> f = pool.submit(() -> 6 * 7);           // Callable in, Future out
int answer = f.get();                                    // blocks until it's done
pool.close();                                            // java 19+: shutdown, then wait

Executor just runs Runnables. ExecutorService adds submit (returns a Future), invokeAll/invokeAny, and shutdown. ScheduledExecutorService adds delays and repeats. ThreadPoolExecutor is the class behind almost all of them.

the execute() decision

every task handed to a ThreadPoolExecutor goes through exactly this, in this order:

here it is for real: core 2, max 4, a queue of 2, and seven tasks that all block, so nothing finishes while we watch:

import java.util.concurrent.*;
 
void main() throws InterruptedException {
    CountDownLatch release = new CountDownLatch(1);    // keeps every task busy
    ThreadPoolExecutor pool = new ThreadPoolExecutor(
            2, 4,                                       // core 2, max 4
            30, TimeUnit.SECONDS,                       // extra threads idle out after 30 s
            new ArrayBlockingQueue<>(2));               // the queue holds 2
    for (int task = 1; task <= 7; task++) {
        try {
            pool.execute(() -> {
                try { release.await(); } catch (InterruptedException e) { }
            });
            IO.println("task " + task + ": threads=" + pool.getPoolSize()
                    + " queued=" + pool.getQueue().size());
        } catch (RejectedExecutionException e) {
            IO.println("task " + task + ": rejected");
        }
    }
    release.countDown();
    pool.shutdown();
}

task 1: threads=1 queued=0
task 2: threads=2 queued=0
task 3: threads=2 queued=1
task 4: threads=2 queued=2
task 5: threads=3 queued=2
task 6: threads=4 queued=2
task 7: rejected

tasks 1 and 2 get core threads. 3 and 4 wait in the queue. 5 and 6 get brand-new threads and start running before 3 and 4, which arrived earlier. task 7 finds a full queue and a maxed-out pool, and the default AbortPolicy throws RejectedExecutionException.

play with a pool

the Executors factories

Executors has static factories for the common shapes. they're convenient, and most of them hide a decision you should make yourself:

when the pool is full: rejection policies

CallerRunsPolicy is the underrated one: instead of failing, it slows the producer down to the pool's speed. try it in the simulator with a high arrival rate: the caller gets stuck running tasks and arrivals stop. the two discard policies lose work without telling anyone, so only use them when that's genuinely fine (sampling, metrics).

futures, and where exceptions go

execute(runnable) and submit(task) look interchangeable. they handle failure very differently:

import java.util.concurrent.*;
 
void main() throws InterruptedException {
    try (ExecutorService pool = Executors.newFixedThreadPool(2)) {
        pool.execute(() -> { throw new IllegalStateException("from execute"); });
        Future<?> f = pool.submit(() -> { throw new IllegalStateException("from submit"); });
        try {
            f.get();
        } catch (ExecutionException e) {
            IO.println("get() threw: " + e.getCause());
        }
    }
    IO.println("pool closed");
}

get() threw: java.lang.IllegalStateException: from submit
pool closed

the execute one kills its worker thread with a stack trace on stderr (Exception in thread "pool-1-thread-1" java.lang.IllegalStateException: from execute), and the pool quietly starts a replacement. the submit one is captured inside the Future and only comes out, wrapped in ExecutionException, when someone calls get(). if nobody ever calls get(), that exception is gone. no log, nothing.

shutting down

pool threads are non-daemon, so a pool you forget to shut down keeps the JVM alive after main returns. the program just never exits.

• shutdown(): stop accepting new tasks, finish everything already submitted (queued ones too), then let the threads die.
• shutdownNow(): stop accepting, interrupt running tasks, and return the queued ones as a list without running them. tasks that ignore interrupts keep running.
• awaitTermination(timeout): block until everything has finished, or the timeout passes.
• close() java 19+: shutdown() then wait until done. it's why ExecutorService works in try-with-resources, as above.

how many threads?

• CPU-bound work (compression, hashing, number crunching): about one thread per core. more just adds context switching. Runtime.getRuntime().availableProcessors() tells you how many.
• I/O-bound work (calling databases, HTTP): more threads than cores, roughly cores × (1 + wait time / compute time), since most threads are waiting at any moment.
• lots of blocking I/O: skip the sizing math and use virtual threads, which make "one thread per task" cheap again. pools still make sense for limiting how many tasks hit a slow downstream at once, but a Semaphore does that more directly.