Async, Concurrency, and Parallelism

Axum uses Tokio, but the layers divide responsibility carefully.

Where tasks actually appear

serve runs an accept loop. For each accepted connection it uses an Executor; the default implementation delegates to tokio::spawn. Hyper then drives that connection and may ask the executor to run internal work such as HTTP/2 connection management.

Axum does not manually spawn every handler. A handler produces a future, which becomes part of the request’s Tower service future and is driven by the connection machinery.

Tokio worker pool
├── accept-loop future
├── connection A future
│   ├── request A1 future
│   └── request A2 future when the protocol permits multiplexing
└── connection B future
    └── request B1 future

Concurrency is not parallelism

While one handler awaits a database response, another future can advance on the same OS thread. That is concurrency. Tokio’s multi-threaded scheduler may also poll ready tasks simultaneously on different workers; that is parallelism. Axum permits both, but neither makes CPU-heavy synchronous work non-blocking.

CPU-heavy parsing or computation must be moved to an appropriate bounded worker facility such as spawn_blocking, or designed as separate parallel work. A plain expensive loop inside a handler occupies a runtime worker.

`Send + 'static` explains many compiler errors

Spawned connection futures may move between runtime threads and outlive the stack frame that created them. Consequently serve, Executor, services, bodies, and handler futures carry Send and often 'static bounds. A handler that holds a non-Send guard across .await cannot satisfy this contract.

Shared state is explicit

Router is cheap to clone because its inner routing table is stored in an Arc. Application state must also be Clone + Send + Sync + 'static. Often the outer state contains Arc handles to pools or services rather than placing the entire application behind one mutex.

The key question is not “which mutex works with async?” but “what must actually have shared mutable ownership?” An immutable config can use Arc<Config>. A database pool already manages its own concurrency. Small synchronous state may use a standard mutex if its guard never crosses .await.

Concurrency limits are policy, not a default

The router reports itself ready. It does not guess the capacity of every downstream operation. Apply a ConcurrencyLimitLayer, timeout, queue, or load shed at the boundary whose scarce resource you understand.

limit a costly inference endpoint separately from cheap health checks;
let a database pool bound database connections;
time out the whole request if downstream work shares one deadline;
shed load before a queue if stale work has little value.

Scheduling creates concurrency; middleware turns it into explicit resource policy.

Keyboard shortcuts