Build a Smaller Apalis

Rebuild the architectural center, not the spelling of the builder API.

What are we preserving?

• an ordinary async handler;
• an owned task payload plus typed shared data;
• a backend independent from execution;
• readiness before dequeue;
• bounded unordered execution;
• result-aware acknowledgement;
• retry as composable policy;
• shutdown that stops admission and drains ownership.

Stage 1: one task and one handler

Define Task<A, Id> and adapt FnMut(A) -> Future into a tiny service with ready and call. Move A into the future. Do not add persistence yet.

Stage 2: a backend contract

Define associated types for payload, compact form, ID, error, and codec. Add poll_ready, poll_next, and poll_close. Implement an in-memory backend with VecDeque and a waker.

Stage 3: the admission loop

Use FuturesUnordered. Poll completed jobs, then backend readiness, then service readiness, and only then dequeue one task. Add a test proving the backend is not polled while the service has no permit.

Stage 4: typed extraction

Add a task extension map keyed by TypeId, a Data<T> extractor, and a two-argument function adapter. Observe why extraction returns owned values rather than references held across .await.

Stage 5: failure and acknowledgement

Return Result from handlers. Wrap execution with an acknowledgement service that records success or failure only after the inner future resolves. Add an abort error and a retry policy that must clone the task.

Stage 6: bounded concurrency and executor injection

Implement readiness with a semaphore permit. Separately accept a generic spawn function. Demonstrate that admission limits and task scheduling solve different problems.

Stage 7: graceful drain

Signal shutdown, make readiness pending for new jobs, drain in-flight handlers and acknowledgements, then close the backend. Add a cancellation test proving a dropped tracked future decrements the count.

Compare with production Apalis

Your reconstruction now contains the same architectural argument. Production Apalis adds many backends and codecs, generated extractor arities, Tower interoperability, richer retry/backoff, worker events and restart policies, status metadata, observability, panic containment, feature gating, and backend maintenance capabilities.

The final exercise is diagnostic: replace the in-memory backend without editing the worker loop, then add middleware without editing the handler. If both work, you preserved the boundaries rather than merely imitating the surface syntax.