Async, Concurrency, and Parallelism

Vector is a Tokio application whose concurrency structure follows the dataflow graph rather than a single request tree.

Components are long-lived tasks

Sources, stateful transforms, and sinks run concurrently as independent tasks. Bounded channels let one component suspend without occupying a worker thread. Unlike an Axum request, these tasks may live for the lifetime of the process and process millions of values.

One configured source can create multiple tasks: the source server, an output pump supervisor, and one pump per named output. A network source may also spawn per-connection work internally. The source decides the natural unit of concurrency because a file tailer and socket listener have different shapes.

Async pipelining overlaps stages

At one instant the source can read batch C while the transform handles batch B and the sink sends batch A. This is pipeline concurrency. Each stage still owns its local mutable state and communicates through moved values.

time →
source      [A read][B read][C read]
transform          [A map ][B map ][C map ]
sink                      [A send][B send]

Stateless work may run in parallel

SyncTransform is cloneable. For selected compute-heavy transforms, the runner spawns bounded work tasks. Tokio’s multi-threaded scheduler may poll those tasks simultaneously on different cores. FuturesOrdered releases completed output in input order even if later computation finishes first.

This deliberately couples three policies:

• cloneable/stateless work is safe to distribute;
• an in-flight limit bounds memory and scheduling overhead;
• ordered completion preserves observable event order.

Stateful TaskTransforms instead own a stream and remain coordination points.

Sink concurrency is layered

A network sink can overlap work at several distinct boundaries:

• collect events into batches;
• encode/compress several batches with a bounded concurrent mapper;
• wait for Tower service readiness;
• keep multiple network response futures in flight;
• retry requests inside the service stack.

These limits should not be conflated. Encoding is CPU work; service readiness usually represents destination capacity; the input buffer represents tolerated pipeline backlog.

select! expresses service policy

The sink driver simultaneously watches completed requests, service readiness, and new input. Its biased ordering checks completions first. This is not a minor optimization: finalizing completed requests allows acknowledgements and buffer deletion to make forward progress under sustained traffic.

Cancellation must preserve accounting

Source sends may be cancelled while waiting on backpressure. The sender keeps an UnsentEventCount guard so dropped futures still report events that never crossed the boundary. Throughout the pipeline, finalizers use Drop to ensure abandoned event ownership contributes a terminal status rather than silently stranding a source waiter.