The thread pool

1 min read
On trails:CSASYNCstep 2/11NETTHREADSstep 2/5

Starting an OS thread is expensive: it allocates a stack and a kernel object, and many threads on few cores waste time on context switches. A runtime avoids that with a pool of worker threads that are created once and reused. Work goes into a queue, and a free worker takes the next item.

Work items wait in a queue. Each of four pool threads runs one item, and a free thread takes the next from the queue. QUEUE job 8 job 7 job 6 job 5 next thread 1 runs job 1 thread 2 runs job 2 thread 3 runs job 3 thread 4 runs job 4
Jobs wait in a queue. A thread that finishes takes the next one.

What runs on it

.NET’s ThreadPool runs Task work items, such as Task.Run, and the continuations that resume a method after an await. Parallel loops and PLINQ use it too.

Sizing

The pool starts with a minimum of about one thread per core. When work is queued and every thread is busy, it adds threads, but slowly: roughly one every 500 ms once the minimum is reached. The maximum is very large, about 32,767 by default. SetMinThreads and SetMaxThreads change the limits, and the defaults are usually right.

Starvation

A pool thread blocked by Thread.Sleep or .Result is out of service while the queue keeps growing. New threads arrive slowly, so the queued work waits. This is thread pool starvationLivelock and starvationThreads that keep running without progress, and threads that never get a turn.Opens in a new tab, and it shows up as sudden latency under load (see Thread.Sleep versus Task.DelayThread.Sleep versus Task.DelaySleep blocks the thread. Delay frees it.Opens in a new tab).

Where it matters

A small pool can serve a large number of requests as long as threads are not held while waiting, which async I/O ensures (see how many threads handle one requestHow many threads handle one request?One at a time, but not always the same one.Opens in a new tab).