perf(queue): cut redis round-trips out of the worker hot path

[LouisHaftmann]

A CPU profile of a concurrency-50 drain showed the worker 69% idle — round-trip bound, not CPU bound. So this is not micro-optimisation; every change deletes a Redis round-trip from a hot path. Adds an e2e benchmark first so the numbers are measured rather than asserted.

Results

Adjacent runs on the same (loaded) machine, best of three:

scenario	before	after
drain 1 worker c=1	2 543	3 303 ops/s (+30%)
drain 1 worker c=50	8 986	15 677 ops/s (+74%)
drain c=50, 3 steps	5 536	8 319 ops/s (+50%)
drain c=50, 10 groups	8 716	14 066 ops/s (+61%)
run → wait round trip	0.85 ms	0.46 ms (-46%)
enqueue	8 315 / 37 263	7 966 / 31 872

The enqueue path is untouched, so it scores identically — that was the control. Any run where it moved was a run where the machine, not the code, had changed.

What changed

One reserve per wake, claiming a batch. It used to take four commands to get round the loop: a schedule poll, a ms-to-next-schedule poll, one reserve per job claimed, and a stalled scan. Now reserve claims a batch atomically and returns both timers and the due-schedule list with it. The schedule polls were pure overhead for the common case of a workflow with no cron schedules at all.

A saturated worker parks locally. It was doing a network round-trip to be told that a slot it owns had freed — and that hop sat directly between one job finishing and the next being claimed. It now waits on an in-process signal. When slots are free it still BRPOPs, because then the interesting event genuinely is remote.

Step memo ships with the claim. Every step.do began with an HGET that, on a first attempt, is a guaranteed miss. The claim makes the worker the sole writer of that hash, so HGETALL inside the reserve script is an exact snapshot and step reads now cost nothing. Writes still persist before a step counts as done.

The done publish carries the result. It was a content-free doorbell: the waiter got woken, then had to go ask what happened. wait() goes from four blocking round-trips to two.

Local throttle on the stalled scan, which Redis was rejecting via its gate anyway — the worker was paying a round-trip to find out.

Notes for review

• RESERVE_BATCH_CAP = 64 bounds the batch. Redis runs Lua single-threaded, so an unbounded concurrency would turn one wake into thousands of serialized commands and pause every client on the instance. Filling a larger concurrency just takes more passes.
• resolvePublished's non-record fallback is load-bearing, not defensive coding. During a rolling deploy a peer on the previous build publishes "1"; parsing that as a result would silently hand the caller an empty string.
• Queue.getStepData is deleted. Not a breaking change — src/index.ts doesn't export Queue.
• Namespace and workflow caps are client-side config, passed into the Lua on every reserve rather than stored in Redis. Pre-existing, not from this PR, but it means a process configured with a looser cap silently widens it for the whole fleet. Surfaced while writing the multi-process tests.
• keepFailed's retention score is millisecond-granular, so same-ms failures evict in uuid order rather than finish order. Always true; removing a round-trip from wait() just made it reachable, and one existing test was relying on the latency. Test premise fixed, library behaviour unchanged.

Tests

75 passing. New coverage targets what the batch and the local park could break silently: group and namespace caps under batch claiming, distinct claim tokens within one batch, close() racing an in-flight batch, a worker held off by a cap a dead peer owns, a saturated worker still firing cron, a retry replaying steps on a different worker, the legacy publish shape, and four real OS processes draining a backlog exactly once.