Repository navigation
Flaky test: sandbox-orphan-reaping "bounds a hung removal" wall-clock bound (< 6500 ms) exceeded under load #3879
Description
packages/cli/src/utils/sandbox-orphan-reaping.bun.test.ts, case "sandbox orphan recovery startup > bounds a hung removal and continues processing other orphans", fails intermittently on macOS (Apple Silicon, Bun 1.3.14). The assertion at line 807 checks elapsed wall-clock time:
expect(received).toBeLessThan(expected)
Expected: < 6500
Received: 6930
How often it fails:
- It has failed repeatedly in full CLI-suite runs on the Eliminate ambient runtime global state: explicit dependency injection, delete getCliRuntimeServices and the singleton maze #2616 integration branch, at load averages of roughly 40 to 150.
- It also failed once when run alone, with 22 of 23 tests passing; that log was not captured.
- The other 7 standalone runs passed 23 of 23.
- The integration branch does not modify this test or the orphan-reaping code.
The test measures how long recovery takes when one container removal hangs, and checks the total against a fixed wall-clock limit. On a loaded machine, the bounded removal timeout plus the processing of the remaining orphans can exceed 6.5 s even when the hung removal is cut off correctly.
Suggested fix: assert on the recovery sequence rather than elapsed real time. For example, check that the hung removal was abandoned at its bound, that the remaining orphans were processed afterwards, and that startup continued. Alternatively, drive the timeout from an injected clock, so the bound is verified without depending on scheduler latency.
Activity
acoliver commentedon Oct 16, 2026
A second failure mode in the same file. In 8 standalone runs (
bun test --timeout 60000 ./src/utils/sandbox-orphan-reaping.bun.test.ts, machine load average about 35-45), 7 passed 23/23. The other failed in "records and retains matching owners under a non-UTC host timezone" after 30.5 s:rerunInNonUtcTimezonere-runs the test file in a childbun testprocess under a differentTZ, usingspawnSyncwith a 30 s timeout. A cold Bun start, plus that file's module graph and the case itself, can take longer than 30 s on a loaded machine. When it does, the parent fails with ETIMEDOUT, whatever the timezone behaviour is.A possible fix is to run the timezone-sensitive code in-process with an injected timezone or clock. If the child process stays, have it import only the code under test, and give it a budget derived from the parent test timeout rather than a separate fixed 30 s.
github-actions commentedon Oct 16, 2026
LLxprt Issue Planner — infrastructure failure
The autonomous planner failed with exit code 1. Please inspect the workflow logs (Run planner agent section) and re-run once resolved.
acoliver commentedon Oct 16, 2026
A third failure mode in the same file, which depends on test order rather than on a fixed wall-clock bound.
The case is "sandbox orphan recovery startup > applies the same dead-owner recovery through Podman" (line 655). The dead owner's row is kept in the Podman state file instead of being removed:
What I measured on 2026-10-16, at a machine load average of about 40 to 46:
-t), 2 runs-t), 2 runsf55fc2e passed the whole CLI suite, including this file, in a full run about 90 minutes before these measurements. The two commits do not differ in the sandbox files.
So the case fails only after earlier cases in the same file have run, and only in the machine state at the time. The failing runs take 5.7 to 7.3 s, close to the "bounds a hung engine listing" case (about 5.1 s). That suggests the Podman listing in this case hits its 5 s bound, possibly because a hung-listing fixture or child process from an earlier case is still running, and recovery then keeps the row.
Suggested fix: make each case fully reap the fixture processes it starts, and give each case its own engine state and fixture directories, so that hung-listing and hung-removal cases cannot affect later ones. Logs are in the integration worktree under
tmp/verify-int/full78/rerun-orphan*.logand in the scratch worktree undertmp/orphan/.