Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

The Standard Library as Architecture

StdlibOwnership made visibleRevision 9fdf144
  • Arc<dyn ExecutionPlan> makes a plan node shareable by optimizers, parents, metrics, and partition streams without pretending there is one unique owner.
  • Vec<Arc<dyn ExecutionPlan>> represents heterogeneous tree children; ordinary recursion and iterator transforms implement plan rewrites.
  • Option, Result, and ? make missing statistics, optional metrics, and fallible planning explicit instead of relying on sentinel values.
  • HashMap and HashSet hold catalogs, function registries, grouping state, and optimizer bookkeeping where identity rather than sequence matters.
  • atomics cheaply account for shared pool usage and metrics; mutexes protect compound policies such as fair sharing among spillable consumers.
  • Drop on a MemoryReservation shrinks the pool and eventually unregisters its consumer. Resource accounting follows ownership on success, error, and cancellation paths.
  • Pin<Box<dyn Stream<Item = Result<RecordBatch>> + Send>> gives the executor a stable, uniform asynchronous output type while each stream keeps its own concrete state machine.

The architecture is not “Tokio all the way down.” Standard ownership types describe durable structure; async types describe when progress can pause.