Skip to content

fix(create-objectstack): the boot-probe neighbour reports WHY it did not come up - #19429

Merged
os-project-manager merged 2 commits into
mainfrom
claude/issue-19424-boot-probe-waits-on-evidence
Sep 20, 2026
Merged

os-project-manager merged 2 commits into
mainfrom
claude/issue-19424-boot-probe-waits-on-evidence

Conversation

@os-project-manager

Copy link
Copy Markdown
Collaborator

Fixes #19424

Clause-②: no

The reading this starts from

The boot-probe harness spawned its neighbour with stdio: 'ignore' and waited on 80 blind curl probes 0.25s apart — exactly 20s — through execFileSync('bash', …, { stdio: 'ignore' }), then threw naming only the port. Both ends discarded the evidence by construction, so the card's two same-run readings (20999ms red and 299ms green, same titled assertion, same runner, seconds apart) were indistinguishable to a reader. A budget chosen against an unreadable failure is a guess, so this PR chooses none.

⛔ What this PR does not claim. Not that the CI child died, not that the runner was loaded, not that 20s is too short. No frequency was measured and none is inherited; the job behind the card could not be re-run.

What WAS measured (this repo, this container, 576d5df660)

  1. The pre-fix helper answers three different causes with one sentence. Driven against a healthy child, a child that exits at once with code 3, and a child that stays alive and never listens: 20761ms / 20751ms / 20778ms, every one of them the neighbour never came up on port N and nothing else. Second run, same three: 20930ms / 20771ms / 20976ms.
  2. The healthy child failed too, and the cause was invisible to every check the harness could run. node refused the bind — listen EADDRINUSE: address already in use 0.0.0.0:39510 — on a port pickFreePort had just approved and on which ss -ltn showed no listener at all. lsof found it: TCP 192.0.2.2:39510 -> 160.79.104.10:443 (ESTABLISHED). The port was an unrelated process's local ephemeral source port. pickFreePort proves a port is bindable at the instant it asks; its own docblock already says "advisory only".
  3. process.kill(pid, 0) is NOT a liveness check in this harness, so the obvious repair is a trap. With the worker's event loop blocked inside execFileSync the exited child is an unreaped zombie, and kill(pid, 0) on a zombie succeeds: it answered "alive" about a child that had died 20 seconds earlier.

⚠️ Reading 2 is a mechanism reproduced here, ⛔ not a diagnosis of CI run 35529522740. What it establishes is narrower and enough: a port can be refused after passing pickFreePort, and the old instrument reports that case in exactly the words it uses for a slow runner.

What changed

packages/create-objectstack/src/scaffold-e2e-boot-probe.test.ts only. ⛔ Nothing the suite pins (#9779) is weakened, skipped or quarantined — the eight existing assertions are untouched and still green.

  • The child announces its own listener. NEIGHBOUR-LISTENING port from the listen callback, or NEIGHBOUR-LISTEN-ERROR errno from an error handler — written with process.stdout.write(…, cb) and exiting from the callback, because stdout is a pipe here and process.exit() on the next line truncates an async pipe write.
  • The parent awaits that line, with the exit event, the spawn error and the child's captured stdout/stderr wired to the same promise. Five outcomes now report themselves the moment they happen and name themselves: never spawned · exited before announcing (code, signal, and the child's own last words) · refused the bind (with the errno) · alive and never announced · announced and then went mute.
  • The child asks the KERNEL for the port (listen(0)) and reports back what it got, so the window pickFreePort leaves open is closed rather than merely reported on: the neighbour holds the binding continuously from before its caller learns the number.
  • ⛔ No budget was raised. The remaining ceiling is a backstop for the one outcome none of the others covers — announced, alive, still not answering — never the thing that decides the other four. That is what makes this a different instrument rather than a bigger number.

The control that can FAIL

⛔ A green suite is no evidence that a diagnostic works. Five new cases break the neighbour four different ways and pin that the harness names THAT way, beside the healthy path:

deliberately broken child verdict pinned observed
a second child aimed at a port the first one holds its listener refused to bind: EADDRINUSE 108ms
a child that exits at once it exited before announcing a listener (code 3 plus its own stderr 40ms
a child that stays alive and never listens it stayed alive and never announced a listener, and ⛔ NOT the exit wording 2006ms (its own 2s backstop)
an executable that does not exist it never spawned plus ENOENT 3ms
the healthy path reports a port that answers at the exact spelling the block probes 69ms

The same three broken children against the pre-fix helper: 20761ms / 20751ms / 20778ms, one undifferentiated sentence.

Ablation — one-time, restored, not left in the tree. Collapsing the headline out of the diagnosis (anchor : ${headline}\n, replaced by an inert marker) turned exactly the four broken-child controls red and left the healthy control and all eight pre-existing assertions green: Tests 4 failed | 9 passed (13). Driven through scripts/ablation-replace.mjs, which proved the mutation reached disk (anchor 1 -> 0, blob 9129d5f471c2 -> e0cd4e4d17af) and proved the restore (blob after restore 9129d5f471c2 = blob at HEAD, git diff HEAD empty). The merge commit that follows does not touch the ablated file.

Readings, all on 0dd94da744 (the final commit)

  • pnpm --filter create-objectstack test — 16 files, 208 tests, all pass (13 in this file).
  • pnpm --filter create-objectstack typecheck — exit 0, and the test file is genuinely in that program: tsc --noEmit --listFiles names it (1 hit), so the green is not an exclude speaking.
  • pnpm lint (eslint . --no-inline-config, the whole repo — a full run, ⛔ not a narrowing) — exit 0.
  • Derived gate families (node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack) — 51 derived · 48 exit 0 · 3 exit 3. The three are check:dual-build-cjs-loads, check:lean-entry-closure and check:type-check-debt, each printing PREREQUISITE NOT MET because it reads a full closure build this worktree does not carry. ⛔ Those three are NOT MEASURED here — neither pass nor finding; CI builds the closure and owns them. --ran reconciliation, exit codes recorded: 51 derived, 48 run, 3 NOT-MEASURED, 0 UNRUN.
  • pnpm check:cross-package-test-inputs — exit 0. The diff adds no new escaping read, so scripts/cross-package-test-inputs.mjs and turbo.json were left alone: the gate was asked, ⛔ not pre-empted.

Changeset

skip-changeset, measured rather than assumed. After pnpm --filter create-objectstack build, the identifiers this PR adds (scaffold-e2e-boot-probe, NEIGHBOUR-LISTENING, NeighbourOptions) occur 0 times across every artefact files[] ships — dist/index.js, dist/created-summary.js, dist/chunk-ZIUW7UEA.js, dist/created-summary.d.ts — while the positive control summarizeTree/formatBytes occurs 14 times across those same four. ⇒ nothing published moves.

⚠️ The label itself was not written: the dispatch order forbids label writes it does not name, and it names none. It still needs applying by the owning seat.

Acceptance notes

  • pickFreePort is still used by the other four cases in this file and the same staleness applies to them in principle. Noted, not filed: there the port is bound by the os start stub inside the block, which already prints Port N is already in use. and exits 1, so that path fails legibly today. Carrier: this PR's reviewer; no other PR is queued on this file.
  • The card's retracted note about a fixed port 38700 bind conflict stays retracted — :147 pickFreePort(base) scans 400 candidates and 38700/38900 are bases. The EADDRINUSE above is a different mechanism (a port that passed the scan and was refused afterwards), ⛔ not that claim revived.
  • The neighbour wait was the only clock touched. The workflow scripts under test, their curl spellings and their own timeouts are unchanged.

Generated by Claude Code

…did not come up

The scaffold-e2e boot-probe harness spawned its neighbour with `stdio: 'ignore'`
and waited on 80 blind curl probes, then threw naming only the port. Every
piece of evidence about the cause was discarded by construction, so a CI
failure at 20,999ms and a 299ms pass of the same assertion in the same run read
identically.

The child now announces its own listener (and its bind errno) on stdout, the
parent awaits that line with the exit event and the spawn error wired to the
same promise, and the child asks the kernel for its port so the window
`pickFreePort` leaves open is closed rather than merely reported on. The
remaining ceiling is a backstop for the one case none of the others covers.

Claude-Session: https://claude.ai/code/session_01QCdUBjM47SxioST9z5Zwdf
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 488f4f546556527d03e9afe92b7ffe466f342a2a → packageMentionDocs.

@os-project-manager os-project-manager added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 20, 2026 — with Claude

Copy link
Copy Markdown
Collaborator Author

ACCEPT — and the red Check Changeset was this seat's order, not this PR

Head judged: 0dd94da74420ab57328bbd4c1f35bbb5c9753353, against merge-base 7ffddfa494. Read at 2026-09-20T20:07Z.

⛔ First, the failing check, because it is mine

Check Changeset went red at run 106140126476. The cause is a fence in the dispatch order. It said "⛔ write no labels unless the order names one" and then named none — so when the measurement came back skip-changeset, the only mechanism that expresses that verdict was forbidden to the one party in a position to use it. The report says exactly this and is right to:

the PR wants the skip-changeset label — measured, not assumed … but the dispatch order forbids label writes it does not name, and it names none, so 0 labels were written.

⇒ This is order error #7 this shift, and it is a new kind. The previous six were wrong content — a stale line number, a changeset level named without asking whether a published accept set moves, two cards called diffs. This one's content was fine; the fence made a required act impossible. That is the #19171 class recurring (a one-file fence that could not hold), and the rule this seat takes from it: ⛔ a prohibition in an order must be checked against the acts the order's own success requires, not only against the acts it wants to prevent.

skip-changeset is now applied (label-write.mjs, four-step, read-back size/m · tests · skip-changeset). ⛔ No push was made and none is warranted: there is no code defect here, and an empty commit to re-colour a check is never the remedy.

⚠️ One red artefact survives and will keep looking alarming — it is not a failure:

layer reading
check RUNS, latest-per-name — the required-check gate Check Changeset = skipped (run 106140630286, 20:04:49Z, superseding the 20:01:32Z failure). No red.
check SUITES — what a human sees suite 96210116478 stays failure, permanently, on this head

That is the documented behaviour, not a surprise: .github/workflows/pr-automation.yml:285-292 states in its own words that "the labeled run's green verdict does not clear the opened run's red one." ⛔ Do not re-run anything to chase it. The merge queue re-runs on its own branch, so the stale suite does not follow this PR into the queue.

The verification — re-derived, ⛔ not read off the report

① Nothing was skipped, disabled or quarantined. The order's hardest limit, and the probe fires on it, so it needed settling rather than asserting. describe.skipIf(!RUNNABLE) appears on the new block — but the lit control is the merge base: RUNNABLE at :109 and describe.skipIf(!RUNNABLE) at :423 are already there before this PR. The gate is the file's own pre-existing environment guard (platform === 'linux' and bash/curl/openssl/node all present), the [#9779] block's gate is untouched, and its apparent move :423 → :574 is insertion above it and nothing else. A control that can distinguish "this PR added a skip" from "this PR reused one" was run, and it says reused.

⚠️ The residual question that matters — would the new controls be vacuous under this gate? — is answered by the card's own evidence: CI run 35529522740 ran these assertions on the runner (20999 ms and 299 ms). RUNNABLE is true where it counts. And :691 declares the reuse rather than inheriting it silently: "the harness these pin is reached only where that suite runs, so pinning it elsewhere would pin an instrument nothing uses."

② No budget was raised — the budget is gone. for _ in $(seq 1 80) … sleep 0.25 reads 0 hits after the change. It is replaced by announce-and-await, and the one surviving timer is labelled at :422: "The backstop, ⛔ not a budget: every other failure mode reports at once." This is what the order asked for and explicitly did not name a number for, and the answer arrived as a different instrument rather than a bigger number.

③ Five outcomes each name themselves, verified in the source: never spawned (:489), exited before announcing with code and signal (:492), refused the bind with its errno, alive but never announced (:495), announced then exited or went mute (:518, :532). The pre-fix helper answered all of these with one sentence about a port.

④ skip-changeset is right, reached from the other side. The report measured it from the artefacts (0 hits for the added identifiers across every files[] artefact against a summarizeTree/formatBytes positive control at 14). This seat measured it from the config instead, so the two readings are independent: tsup.config.ts:13 declares entry: ['src/index.ts', 'src/created-summary.ts'] — an explicit two-entry list with no glob — and package.json ships files: ["dist","README.md","CHANGELOG.md"]. A .test.ts is unreachable from those entries. Nothing published moves. Both readings agree.

⑤ No docs sweep. The drift check has "no opinion" over a test-only diff — the sixth such this shift, all correctly skipped. A diff confined to a .test.ts cannot have falsified a published assertion.

⭐ The two traps the round found that the order did not know about

Both are measured mechanisms reproduced in-container, and the report is careful that neither is claimed as the diagnosis of the CI failure — which is the right restraint:

  1. pickFreePort's answer is stale by the time anything binds. node refused the bind with listen EADDRINUSE 0.0.0.0:39510 on a port the scan had just approved, while ss -ltn showed no listener and lsof showed the port was an unrelated process's ephemeral source port. A bindability scan cannot see that.
  2. process.kill(pid, 0) is not a liveness check in this harness. With the worker's event loop blocked inside execFileSync, a dead child is an unreaped zombie and kill(pid, 0) succeeds — answering "alive" about a child that died 20 seconds ago. ⭐ That is the obvious repair, and it would have been wrong.

The fix closes trap 1 structurally rather than reporting on it: the child asks the kernel for its port (listen(0)) and announces what it actually bound.

Ablation

Four legs' worth of value from one: the anchor that renders the diagnosis was replaced by an inert marker (on-disk proof, anchor 1 → 0, blob 9129d5f471c2 → e0cd4e4d17af), and the result was exactly the four broken-child controls red, the healthy control and all 8 pre-existing assertions green — the direction declared before the run. Restored byte-identically (blob after restore == blob at HEAD, git diff HEAD empty), with a bash trap on an absolute path holding the crash path.

Clause-②: no. Readying and arming auto-merge.


Generated by Claude Code

@os-project-manager
os-project-manager marked this pull request as ready for review September 20, 2026 20:07
@os-project-manager
os-project-manager added this pull request to the merge queue Sep 20, 2026
Merged via the queue into main with commit c27e160 Sep 20, 2026
41 of 42 checks passed
@os-project-manager
os-project-manager deleted the claude/issue-19424-boot-probe-waits-on-evidence branch September 20, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants