Build a Smaller Linkerd2-proxy

Build a target-specialized Tower proxy with dynamic endpoint discovery, bounded readiness, balancing, replay-safe retries, and graceful drain.

1. Proxy one concrete HTTP service

Accept TCP, serve HTTP, rewrite the destination, and forward with Hyper. Record connection and request ownership before adding generic layers.

2. Define NewService<T> and target parameters

Create Target { logical, original_dst, labels }, a service factory, and Param<P>. Construct a service once per target instead of consulting global maps in every call.

3. Build middleware layers

Implement timeout, request counting, response classification, and authorization as Tower services/layers. Keep their errors concrete until an outer response mapping boundary.

4. Add discovery and a cached router

Use a watch/change channel to add and remove endpoints. Cache one router per logical destination and evict it after idle time. Preserve its endpoint state across requests.

5. Implement readiness and bounded pressure

Give endpoints capacity flags, poll only ready endpoints, and place a bounded queue with a failfast deadline before the balancer. Demonstrate queue, reject, and recovery under a paused endpoint.

6. Balance by load

Choose two ready endpoints randomly and send to the lower observed load. Hold a load guard until response body completion/drop, not merely response headers.

7. Add safe retries

Buffer request bodies up to a cap, classify retryable responses, enforce a token budget and overall deadline, and prove a non-replayable streaming body is never retried.

8. Add TLS and live credentials

Distribute Arc<ClientConfig> through a watch channel. New connections use the latest config while established connections retain the config that negotiated them.

9. Drain

Broadcast shutdown, stop accepts, reject new routing work, and wait for active bodies/connections under a deadline. Test a stream that finishes and one that never does.

10. Compare with production

Map to inbound/outbound stacks, NewService, Param, Tower Service, queues, failfast, idle caches, routers, endpoint discovery, P2C pools, retry replay, classifiers, mTLS, metrics, and drain. The real proxy adds mesh protocols, policies, opaque TCP, gateways, tap, observability, and extensive control-plane recovery. Preserve its center: capacity-aware typed services whose dynamic state follows the target hierarchy.