Build a Smaller Nushell
Build a structured shell whose lazy values interoperate with child-process byte streams and whose parallel map makes ordering and bounds explicit.
1. Define values and spans
Support int, string, record, list, nothing, and error. Give every value a source span. Implement field access and display without parsing source strings again.
2. Parse a concrete pipeline
Parse command names, flags, literals, and | into calls. Give commands
signatures with accepted input/output types and report multiple spanned parse
errors before execution.
3. Add PipelineData
Implement empty, value, list stream, and byte stream. Back list streams with
Box<dyn Iterator<Item = Value> + Send>. Write lazy where, get, and first,
plus an explicit collect.
4. Build the command registry
Define a cloneable object-safe Command with signature and run. Separate
immutable EngineState from mutable Stack. Move input and output so a stream
has one clear consumer.
5. Run external processes
Pipe a byte stream directly into stdin. Encode structured rows on a helper thread when necessary. Return a child-backed byte stream that owns stdout, stderr, an exit-status receiver, and cleanup.
6. Prevent pipe deadlocks
Capture large stdout and stderr simultaneously. Drain them concurrently and wait for exit only after both make progress. Turn nonzero status into a spanned shell error.
7. Add cooperative Ctrl-C
Share an atomic signal. Wrap list iterators and check it per item; check inside long loops; terminate foreground children. Verify an infinite source stops.
8. Add par-each
Use a dedicated fixed Rayon pool and bounded result channel. Stream unordered
results first. Add --keep-order with indices and documented collection cost.
Test nested parallel calls with a private inner pool.
9. Compare with production
Map to Value, PipelineData, ListStream, ByteStream, ChildProcess,
Signals, Command, EngineState, Stack, evaluator, plugin streams, and
par-each. Nushell adds a rich language, IR, job control, custom values,
plugins, formats, table rendering, completion, and cross-platform process
behavior. Preserve its center: structured values are repeatable; streams and
processes are owned, lazy lifecycles.