Skip to content

test(cli): pin the union fold by the finding's pedigree, not by its exit code - #19369

Merged
os-project-manager merged 2 commits into
mainfrom
claude/issue-18897-echo-satisfied-controls-sweep
Sep 20, 2026
Merged

os-project-manager merged 2 commits into
mainfrom
claude/issue-18897-echo-satisfied-controls-sweep

Conversation

@os-project-manager

Copy link
Copy Markdown
Collaborator

Fixes #18897

Clause-②: no

The sweep, and what it found

The card's discriminant: a control is satisfied by an impostor whenever some mechanism other than the one under test can supply the asserted value in that fixture. The method is ablation — delete the mechanism a control claims to pin, re-run, and read the colour. A control that stays green was never pinning it.

21 packages/cli fixtures were enumerated by the property (a control that asserts a survivor or a resolution exists), not by the three files the card body tabulates, and every one of them was ablated. Baseline first: 21 files / 225 tests, all green at the dispatch base 13d52947d8.

One control family was impostor-satisfied, and its impostor is a THIRD mechanism — neither the echo nor the id tail.

test/union-fold-command-parity.test.ts claimed to pin authoringRuleUnionStack, the fold that makes the author-time rule table's INPUT non-empty on an option-B (packages[]-only) stack. It could not. Since #18677 (os validate) and #18778 (os lint) all three doors ALSO run runPerPackageAuthoringRules, which judges each package body as its own stack — and on this fixture that pass raises the SAME rule at the SAME path and refuses with the same exit code.

ablation union-fold-command-parity.test.ts
authoringRuleUnionStack never folds 6/6 GREEN
runPerPackageAuthoringRules yields no findings 6/6 GREEN
BOTH of the above 3 RED (the three refusal cases)

Either mechanism alone kept every case green. Only deleting both turned the refusal cases red. The control was correct when it was written (#17069) and became impostor-satisfied when a sibling pass was added to the same doors a year of cards later — which is the generalisation this PR contributes back to the card.

The repair

The two mechanisms are told apart by the finding's pedigree. runPerPackageAuthoringRules prefixes the where of every finding it raises with package 'ID' — ; the union run renders where bare. Measured on that fixture, with only the fold's presence moving:

fold intact    • object "ob_order" · field "ghost": …
fold ablated   • package 'com.example.ob' — object "ob_order" · field "ghost": …

So each refusal case now asserts that prefix is ABSENT, and a new positive pedigree control asserts the same suite can make it PRESENT — on a fixture whose only finding really is per-package-only (core declares ob_account.industry, orders owns the view that displays it: the falsifier shape PR #18878 landed). Without that pair the negative assertion would be satisfied by a prefix this suite never produces at all.

Verified in both directions, from the committed state:

run result
repaired file at HEAD 9/9 GREEN
union fold ablated 3 RED — the three refusal cases (was 6/6 green before this change)
per-package pass ablated 3 RED — the three pedigree controls; refusal cases correctly stay green

One half of the prescribed remedy is not available here, and the header says so

The card's remedy shape is a real falsifier AND a pedigree assertion. The falsifier half cannot be built for this rule class, and that was measured rather than assumed: the per-package pass is a SUPERSET of the union run for reference rules (utils/artifact-packages.ts: "the per-package run is STRICTER"), because each body is judged with the artifact's own packages[] handed in as resolution context. A name no package provides therefore dangles in BOTH views, under every arrangement of packages. Two candidate union-only fixtures were built and probed; neither produced a finding the per-package pass could not also raise. The pedigree is the whole discriminant here, and the positive control is what makes it falsifiable.

The full ablation table

Every fixture is reported, including the ones that needed no change — a sweep that lists only what it changed cannot be checked for coverage.

fixture mechanism deleted red? verdict
test/union-fold-command-parity.test.ts authoringRuleUnionStack fold NO — 6/6 green IMPOSTOR (third family) — repaired here
test/union-fold-command-parity.test.ts per-package pass findings NO — 6/6 green same control, other mechanism
test/union-fold-command-parity.test.ts both of the above YES — 3 confirms neither alone is load-bearing
test/validate-per-package-authoring-parity.test.ts per-package pass findings YES — 1/4 correct (only the lit control; the parity pins go vacuous, as the card predicts)
test/validate-per-package-authoring-parity.test.ts the fixture's falsifier view YES — 1/4 correct; expected 0 to be greater than 0
test/build-text-face-advisory-count.test.ts per-package pass findings YES — 3/5 correct
test/build-text-face-advisory-count.test.ts the fixture's falsifier view YES — 1/5 correct
test/lint-per-package-authoring-parity.test.ts per-package pass findings YES — 2/5 correct (documented already-correct; verified, not inherited)
test/lint-per-package-authoring-parity.test.ts the fixture's falsifier view YES — 1/5 correct
test/lint-per-package-authoring-seam.test.ts per-package pass findings YES — 2/6 correct
test/lint-per-package-authoring-seam.test.ts the fixture's falsifier view YES — 1/6 correct
test/validate-per-package-authoring-seam.test.ts per-package pass findings YES — 2/6 correct for its narrower claim (it de-duplicates against an EMPTY union by construction, and its header says so)
test/per-package-dedup-positional-echo.test.ts per-package pass findings YES — 5/6 correct by construction
src/utils/collect-docs.package-docs.test.ts docsPackageRefs id-TAIL branch YES — 12/32 correct
src/utils/collect-docs.package-docs.test.ts docsPackageRefs FULL-ID branch YES — 2/32 correct — and the 30 staying green is the #18962 reading reproduced
src/utils/artifact-packages.test.ts packageBodyAsStack resolution context YES — 2/4 correct; it already carries its own contextless CONTROL leg
test/lint-handwritten-checks-package-fold.test.ts authoringRuleUnionStack fold YES — 5/7 correct
src/utils/format.metadata-stats-package-fold.test.ts authoringRuleUnionStack fold YES — 3/8 correct (the top-level-only and empty-stack rows rightly stay green)
src/utils/stack-collections.test.ts authoringRuleUnionStack fold YES — 3/16 correct
src/utils/stack-collections.test.ts resolveStackCollection packages leg YES — 2/16 correct
test/info-detail-package-fold.test.ts resolveStackCollection packages leg YES — 7/7 correct
src/utils/nav-contribution-groups.test.ts artifactPackages reports no packages YES — 6/9 correct
src/utils/nav-contribution-groups.package-id.test.ts artifactPackages reports no packages YES — 4/4 correct
src/utils/permission-set-name-collisions.test.ts artifactPackages reports no packages YES — 5/10 correct
test/build-multi-package-artifact.e2e.test.ts packageBodyAsStack resolution context YES — 1/7 correct
test/build-multi-package-artifact.e2e.test.ts compile drops packages from the artifact YES — 2/7 correct
test/build-package-docs-attachment.e2e.test.ts docsPackageRefs id-TAIL branch YES — 2/3 correct
test/build-package-docs-attachment.e2e.test.ts docsPackageRefs FULL-ID branch NO — 3/3 green correct, NOT an impostor: its fixture directory is an id tail by construction and its header says so. It never claims the full-id spelling. Recorded as a coverage boundary, not a defect
test/compile-artifact-packages.e2e.test.ts artifactPackages reports no packages NO — 4/4 green correct, NOT an impostor: it reads the written artifact, which is not produced through artifactPackages
test/compile-artifact-packages.e2e.test.ts compile drops packages from the artifact YES — 1/4 correct; this IS its mechanism
test/validate-build-gate-parity.test.ts authoringRuleUnionStack fold body NO — green correct, NOT an impostor: it is a source-text WIRING pin with its own positive control (the export must exist) and fabricated negative controls. Ablating the body leaves the wiring, which is what it asserts
test/init.test.ts not ablated n/a EXCLUDED, with evidence: its packages: occurrences are the pnpm-workspace key, and no control in it asserts a per-package survivor or resolution. It appears in the card's keyword tally as a false positive

Every ablation was performed through scripts/ablation-replace.mjs, so each mutation carries its own on-disk proof (anchor count moved, blob hash changed) and each restore is proven by blob equality against HEAD plus an empty git diff HEAD. No ablation is left on disk.

Fences observed

Verification

At e2b7d13d0f:

  • pnpm --filter @objectstack/cli exec vitest run --project unit — 220 files / 3114 tests, all pass
  • pnpm --filter @objectstack/cli exec vitest run --project integration — 51 files / 430 tests, all pass, run in two halves for the foreground cap
  • pnpm --filter @objectstack/cli typecheck — pass
  • node scripts/pm/dispatch-gates.mjs --commands derived 46 gate families; all 46 run, all exit 0, reconciled with --ran carrying each exit code: "46 derived families accounted for — 46 run, 0 NOT-MEASURED (a DERIVED zero)"
  • pnpm lint (repo-wide, eslint . --no-inline-config) — exit 0, run in full rather than narrowed
  • pnpm --filter '@objectstack/cli^...' build and the full turbo run build over the package farm — both exit 0

check:dual-build-cjs-loads and check:type-check-debt first answered PREREQUISITE NOT MET (exit 3, which both scripts state is neither a pass nor a finding) because only the CLI closure had been built. Both were re-run after the prescribed full build and both exit 0.

Acceptance notes

Noted, not filed — observations from the sweep that are not reproducible defects, contract violations or authoring traps:

  • test/build-package-docs-attachment.e2e.test.ts cannot distinguish docsPackageRefs' full-id branch from its id-tail branch. That is deliberate in its fixture and stated in its own comments, so it is a coverage boundary rather than a defect. Next toucher: whoever changes docsPackageRefs' spelling set — the unit pins next door already cover both branches, and they go red.
  • test/validate-per-package-authoring-seam.test.ts and the non-vacuity case of test/lint-per-package-authoring-seam.test.ts de-duplicate against an EMPTY union by construction, so their survivors are not asserted to be union-invisible. Both headers say so explicitly; the stronger claim lives in the parity files. Next toucher: none currently in flight.
  • The card's keyword tally lists test/init.test.ts as an unexamined multi-package fixture. It is not one; the match is the pnpm-workspace packages: key.

Generated by Claude Code

…xit code

`union-fold-command-parity.test.ts` claimed to pin `authoringRuleUnionStack`,
the fold that makes the author-time rule table's INPUT non-empty on an option-B
(`packages[]`-only) stack. It could not: since #18677 and #18778 all three doors
also run `runPerPackageAuthoringRules`, which judges each package body as its
own stack and raises the SAME rule at the SAME path on this fixture. Measured by
ablation at 13d5294 — the fold deleted: 6/6 green; the per-package pass
deleted: 6/6 green; BOTH deleted: 3 red.

The two mechanisms are told apart by the finding's pedigree: the per-package
pass prefixes `where` with `package '<id>' — ` and the union run renders it
bare. Each refusal case now asserts that prefix is ABSENT, and a new positive
control on a per-package-only fixture asserts the same suite can make it
PRESENT, so the absence is a measurement rather than a regex that never matches.

A structural falsifier is not available for this rule class and the header says
so with the reason: the per-package pass is a superset of the union run for
reference rules, so a dangling name dangles in both views.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCdUBjM47SxioST9z5Zwdf
@os-project-manager os-project-manager added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 20, 2026 — with Claude
@github-actions github-actions Bot added the tests label Sep 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json c334ba0f3a6dc2a107ffca06bc28fb27775cba28 → packageMentionDocs.

Copy link
Copy Markdown
Collaborator Author

Seat grading — ACCEPT. The sweep found a third impostor mechanism, and proved the remedy shape is load-bearing.

What the seat verified independently, ⛔ not taken from the report

claim check result
scope GET /pulls/19369/files 1 file, +172/−0
serial fence diff scanned for collect-docs* and anything outside packages/cli/test/ clean — nothing outside; collect-docs.ts (then held by PR #19352 in the queue) untouched
the repair is paired, not one-sided read the added assertions PER_PACKAGE_WHERE → toBe(false) on each refusal case, plus a positive control on a per-package-only fixture asserting toBe(true)
changeset skip-changeset on a test-only diff correct
CI both layers 0 failed suites, 0 failed runs

⭐ The repair is additive for the right reason, and I checked that rather than assuming it

+172/−0 removes nothing. That could have meant a good control bolted beside a dead one — it does not. The existing assertions were never wrong, only non-discriminating, so they stay and a pedigree assertion is added that only the union run can satisfy.

⭐ And the pedigree assertion is paired. The file says so itself: "a toBe(false) on a regex is only a measurement" if the regex is shown capable of matching — so a positive control proves PER_PACKAGE_WHERE fires on a fixture whose only finding is the per-package pass's own. ⇒ the negative half cannot pass by silence. That is the same discipline this lane spent the shift on, applied inside the test file without being told to.

⛔ And the regex is not this file's guess at a format: it is derived from the pass that owns the prefix, with three sibling files pinning the same shape.

The finding

union-fold-command-parity.test.ts claimed to pin authoringRuleUnionStack. Since #18677 and #18778, all three doors also run runPerPackageAuthoringRules, which raises the same rule at the same path on this one-package fixture.

ablation result
union fold alone 6/6 green
per-package pass alone 6/6 green
both 3 refusal cases RED

⇒ neither mechanism alone was load-bearing — the definition of the defect, and a third impostor: neither the echo (#18878) nor the id tail (#18962 round 3).

⭐ The leg that makes this round worth more than its diff

The historical impostor reconstruction — falsifier removed and findingKey reverted to the positional form — turned the three parity pins red with expected 2 to be 1.

⚠️ The survivor count was still above zero. ⇒ the card's central claim ("'the count is above zero' is not the property these controls mean to assert") is now measured, not argued. A count provably could not have caught it; only the pedigree assertion did.

Coverage, and why the other 20 are reported rather than silent

21 fixtures were enumerated by the property — a control asserting that a survivor or resolution exists — ⛔ not by the three files the card body tabulates. 30 ablation legs across 9 mechanisms, every mutation and restore proven on disk.

17 went red under their own claimed mechanism. 3 stayed green and are each argued, which is the part a sweep usually omits: build-package-docs-attachment.e2e cannot separate the full-id branch from the id-tail branch because its directory is an id tail by construction and its own comment says so; the two authoring-seam files de-duplicate against an empty union by construction, so they make a narrower claim rather than a false one.

⭐ ⛔ A sweep that lists only what it changed cannot be checked for coverage. This one can.

The generalisation — filed as #19371

The dev recorded it and declined to file, on the ground that a repo-wide sweep is a scope proposal rather than a reproducible defect. ⭐ That reasoning is right, and the insight was still too load-bearing to leave in a report:

a control can be correct when written and decay into an impostor-satisfied one with no edit to it or its fixture, because a sibling mechanism reaches the same doors later.

⛔ No commit to union-fold-command-parity.test.ts ever weakened it, and #18677/#18778 were both correct work. The decay is invisible to review of the diff that causes it and invisible to CI, because everything stays green — which is the whole problem. #19371 records the trigger (when a PR adds a mechanism to a door, ask which controls assert something about that door's output) and explicitly argues against a repo-wide test audit: 21 fixtures yielded exactly one instance, so the base rate does not pay for a blanket sweep.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

2 participants