One Structured Pipeline, Fully Traced

Consider open people.json | where active | get name | first 10.

1. Parsing produces calls and typed shapes

The parser resolves declarations from a working set, checks command signatures, and produces a block. Spans connect each expression and error to original source. Parser/type errors can accumulate before evaluation begins.

2. Evaluation separates global and local state

EngineState holds declarations, blocks, files, configuration, signals, and long-lived services. Stack holds variables, environment changes, redirection, and call-local state. eval_call borrows the engine, mutably borrows the stack, and moves pipeline input.

3. Commands receive one closed data enum

Command::run accepts PipelineData and returns Result<PipelineData, ShellError>. Each stage can preserve streaming, intentionally collect, or reject an incompatible input.

4. Iterator adapters keep rows lazy

ListStream owns a boxed Iterator<Item = Value> + Send. where, get, and first can wrap it with filter/map/take behavior. Upstream produces only when downstream calls next; first 10 naturally prevents parsing the remainder.

5. The sink chooses materialization

Display consumes values incrementally; assignment or a command needing a whole list calls into_value and collects. PipelineMetadata and spans travel beside the data so rendering and errors retain provenance.