Build a Smaller Tokio

Do not begin with work stealing, timers, networking, and lock-free task state at once. Preserve the architectural center in layers and keep each invariant observable.

What are we preserving?

• a future runs only when polled;
• Pending stores a route back to its task;
• wake makes an idle task runnable without polling every dormant future;
• one task is never polled concurrently;
• readiness and scheduling are separate concerns;
• cancellation drops suspended state;
• blocking work does not occupy the executor thread.

Stage 1: a single-thread executor

Store Pin<Box<dyn Future<Output = ()>>> tasks in a map and runnable task IDs in VecDeque. Poll until the runnable queue is empty. At this stage, accept the allocation and trait object: clarity is the goal.

Stage 2: implement a task waker

Give each task an Arc handle back to a synchronized runnable queue. Implement Wake so it inserts the task ID only if it is not already queued. Create a future that returns Pending, saves the waker, and can be completed from another thread.

Verify that an idle task is not repeatedly polled and ten wake calls before the next poll produce one runnable entry.

Stage 3: add a typed join result

Wrap each submitted typed future in an erased Future<Output = ()> that sends T through a oneshot. Return JoinHandle<T> containing the receiver. This teaches the same “erase work, preserve results” boundary without Tokio’s raw vtable.

Stage 4: integrate one readiness source

Use a nonblocking Unix stream or mio::Poll. A Readable future registers its waker by token and returns Pending. The driver thread waits for OS events and wakes only matching tasks. The executor still decides when polling occurs.

Stage 5: park instead of spin

When no runnable task exists, block on the driver. Add an executor unpark token so spawning from another thread interrupts the OS wait.

Stage 6: add timers

Start with a binary heap ordered by deadline. Make the driver wait until the earlier of I/O or the next timer. Expired timers wake their tasks. Only after this works should you compare the heap with Tokio’s multi-level timing wheel.

Stage 7: cancellation and shutdown

Give a task explicit Idle, Queued, Running, Complete, and Cancelled states under a mutex first. Abort marks cancellation and queues the task so the executor can exclusively drop its future. Shutdown closes spawn admission, cancels owned tasks, and drains runnable references.

Stage 8: a blocking bridge

Send FnOnce() jobs to one dedicated blocking thread and return typed results by oneshot. Demonstrate that a one-second synchronous sleep no longer delays a ready async timer on the executor.

Only then: parallel workers

Add one local queue per worker, a synchronized injection queue, and stealing. Write the invariant that one task can have only one queued notification and only one poll owner. Use Loom before replacing mutex state with packed atomics.

Failure exercises

• Lose a stored waker and observe permanent Pending.
• Wake during a poll and verify the task is scheduled again afterward.
• Abort while idle and while running.
• Drop a typed join handle before completion.
• Spawn concurrently with shutdown.
• Block an executor worker and measure unrelated timer delay.
• Overflow one local queue and preserve every runnable task.

Compare with production Tokio

Tokio adds optimized raw task allocation and type erasure, atomic refcount and lifecycle state, current-thread/local/multi-thread schedulers, work stealing, cooperative budgets, Mio resource registration, a hierarchical timing wheel, async synchronization primitives, platform networking and processes, blocking pool management, feature gating, metrics, tracing hooks, panic isolation, careful shutdown, and extensive Loom coverage.

Your smaller executor is successful when you can narrate one task’s exact ownership and runnable state from spawn to Pending, wake, completion, join, cancellation, and shutdown.