Why Is It Designed This Way?

Vector’s choices preserve operational properties under continuous load.

Why compile configuration into a DAG?

Acyclic flow gives the runtime a direction for type validation, startup, backpressure, reload, and shutdown. Cycles would require explicit feedback buffering and termination semantics. Rejecting them keeps component contracts local and draining understandable.

Why one task per stateful component?

The task owns mutable transform or sink state and processes an ordered stream. This avoids sharing that state behind locks. Independent components still run concurrently, so serialization exists only where the configuration names a stateful boundary.

Why give sources their own concurrency model?

A universal worker count would ignore the shape of the input. Files naturally partition by tailed file; sockets by connection; Kafka by partition. The source knows where independent progress is safe and where ordering or protocol state must remain coordinated.

Why separate the source task from output pumps?

Sources can expose multiple named outputs, while each output needs independent fanout control and pressure accounting. Pumps keep topology wiring out of integration code and let reload control messages progress even when no events arrive.

Why does fanout wait for every blocking branch?

If one branch could be skipped merely because it was slow, a configured output would silently stop receiving data. Waiting preserves reliable broadcast and propagates the slowest branch’s pressure. Operators who prefer availability over completeness must choose a drop or buffer policy explicitly.

Why parallelize only selected stateless transforms?

Cloning a stateful transform could split counters, windows, deduplication sets, or ordering state into inconsistent islands. Stateless cloneable work is safe to spread across cores. Even then, bounded in-flight work and ordered release prevent memory growth and surprising reorder.

Why put configurable buffers at sinks?

External destinations are the common long-latency and outage boundary. A sink buffer names exactly which destination owns the backlog and lets its durability and full behavior be chosen independently. Giant buffers between every pair of transforms would hide local bottlenecks and multiply memory planning.

Why use Tower services inside sinks?

Once events are encoded as requests, delivery has the same shape as other request systems: readiness, call, response, timeout, rate limit, retry, and concurrency. Tower composes these mechanisms while sink-specific code supplies request building and response classification.

Why prioritize completed service calls in select!?

Starting more requests while completed responses wait would retain finalizers and buffer records unnecessarily. Handling completions first releases delivery obligations and improves forward progress without forbidding new work.

Why implement acknowledgement through finalizers and Drop?

Events can be cloned, split, merged, filtered, buffered, retried, and dropped. A central acknowledgement task would need a parallel protocol for every one of those operations. Ownership already observes when the last branch disappears. Finalizers attach delivery accounting to the value whose lifecycle matters.

Why shut sources down first?

Stopping sinks first would make upstream components fail while still producing. Stopping admission and letting channel closure move downstream preserves the same direction as dataflow and gives queued work a chance to drain.

Why use the same machinery for startup and reload?

Reload is the harder path: resources must be reclaimed, fanouts paused or replaced, persistent buffers retained, and rollback possible. Exercising that machinery at initial startup reduces the chance that a rarely used secondary path has different invariants.