ripgrep: Orientation

Ripgrep is a recursive text-search application. A useful first model is not “grep, but in Rust.” It is a pipeline that discovers eligible files, searches several of them in parallel, and commits each file’s completed output without tearing it together with output from another worker.

The architectural center

The traversal is not merely a producer feeding a separate search pool. Each walk worker owns a callback, and that callback owns a cloned SearchWorker. The same operating-system thread discovers a searchable file and searches it.

Design thesis

Ripgrep parallelizes at the file boundary because files are independently searchable work units, while output becomes visible only through a short, serialized commit.

• Worker-local searchers keep hot mutable state out of locks.
• Purpose-built work stealing balances irregular directory trees.
• Synchronous threads match filesystem and CPU-heavy work better than async I/O.
• Per-file buffering trades global ordering for untorn, readable output.

The important crates

• The root ripgrep package and crates/core assemble CLI configuration and choose sequential or parallel execution.
• ignore walks directory trees, applies ignore and type rules, and owns the work-stealing scheduler.
• grep-searcher chooses an mmap, whole-file, or incremental line-buffer strategy and drives a Sink.
• grep-matcher defines the matching contract independently of a regex engine.
• grep-printer turns search events into standard, summary, or JSON output.

This is not Tokio or Rayon

The normal search path has no async runtime. Filesystem reads and searches are synchronous operations performed by scoped OS threads. Ripgrep also does not use Rayon for traversal: ignore builds its own scheduler from crossbeam_deque::Worker and Stealer.

That choice is the reason this case study belongs here. We are studying how an application chooses concurrency boundaries around real work—not how a general runtime implements them.

Scope

We will follow rg pattern directory through parallel traversal, ignore filtering, file searching, output serialization, errors, and early shutdown. Regex-engine construction, every flag, PCRE2, archive decompression, and the experimental index are secondary paths.