Skip to content

Commit 805811e

Browse files
fix(devx): classify the tree-measurement claims under scripts/ instead of sweeping them (#19220)
Fixes #17797 The deliverable is a **classifier**, not a sweep. Three kinds of sentence live under one phrase and take different repairs, and only one of them is a defect. ## ⭐ The card's own headline number had decayed — which is the card's best self-evidence The card states that `scripts/pm/dispatch-gates.mjs` "alone carries **40** sentences containing `measured on this tree`". Re-measured at `0046a41b4`, 2026-09-20T00:30Z, with the card's own case-sensitive probe: **14**, of which **3** carry a revision by the card's own rev probe. A card about bare magnitudes decaying in silence had its own bare magnitude decay by 65% before anyone read it. ⛔ The `40` is not carried anywhere in this PR. ## 1. The declared population — which files, which phrase forms All readings below are mine, taken at `0046a41b4` (this branch's merge base), 2026-09-20T00:30Z. ⚠️ Every one of them decays; the tool added here re-derives them. | probe | occurrences | files | |---|---:|---:| | the card's probe — `measured on this tree`, case-sensitive, over `scripts/` | 46 | 25 | | all three spellings that actually occur, case-insensitive — `measured on this tree`, `re-measured on this tree`, `measured against this tree` | 121 | 47 | | the same, excluding the two files scoped out below | 74 | 45 | ⚠️ **That is an upper bound on a population, ⛔ not a count of defects.** The card says so in its own words and this PR does not contradict it. Three findings about the population itself, each of which says a phrase probe is not a population: - **The card's probe is case-folded shut.** Lower-case-only reaches 46 of the 121 sentences that carry the phrase in some spelling. The gap is not rounding. - **A line-based probe cannot see a sentence that wraps.** The phrase itself straddles a line break in five files, so no `grep` of it lists them at all: `scripts/check-examples-live-imports.mjs`, `scripts/check-parse-guard.mjs`, `scripts/check-self-test-wired.mjs`, `scripts/eslint-fatal-guard.mjs`, `scripts/i18n-bundle-surface.mjs`. The tool reads sentences, so it holds them. - **⭐ And the card's own named instance carries no phrase at all.** It is a bare corpus size in the present tense. A population defined by the phrase would have reported the tree clean at the one site everybody already agreed was broken — so the population is declared in two forms, FORM A (the phrase) and FORM B (declared sites the phrase cannot reach, currently the hand-written-corpus size in `scripts/docs-audit/README.md`). ## 2. The classifier `scripts/pm/measurement-claim-triage.mjs`: ``` node scripts/pm/measurement-claim-triage.mjs the population + every verdict node scripts/pm/measurement-claim-triage.mjs --review the kind-3 residue alone node scripts/pm/measurement-claim-triage.mjs --self-test the controls, both directions ``` It records **no** number about its own population — the scope line it prints is the number, re-derived on the tree it runs against. That is the durable half of the fix: the reason the card's `40` went stale is that it was written down instead of re-derivable. Four tests, applied in that order. Each can only move a hit **out** of the residue, and each prints the cue that moved it, so every verdict is auditable against the sentence. - **T1 FRAME** — is the magnitude scoped to a moment that has passed, or to a counterfactual condition? (`before this gate was written`, `at the commit that took it`, `an earlier revision`, `with the reservation removed`, `was auditing`) ⇒ **KIND 1**. - **T2 ANCHOR** — does the sentence carry a revision or a date? ⇒ **KIND 1**. ⛔ A bare card number is deliberately **not** an anchor: `#NNNN` occurs in every kind of sentence here, and admitting it would clear the residue by matching everything. - **T3 LOUDNESS** — would the drift announce itself? Three ways: a named failure direction (a zero whose falsifier is spelled out is the commonest), a pointer at the instrument that reprints the value, or a live assertion that reds on drift. ⇒ **KIND 2**. - **T0 MAGNITUDE** — last, and weakest: nothing decays if nothing is counted. A ZERO counts, because kind 2 is defined on one. - Everything else is **REVIEW**, a *candidate* bucket. ⛔ Reading its size as a finding count is the error the card exists to prevent. **T4 — level or relation — is a judgement and no regex decides it.** Does a reader act on the magnitude's **level**, or on a relation (a ratio, an ordering, an existence claim, a set equality) that the level's drift survives? A level is kind 3; a relation is an ILLUSTRATION, and rewriting it buys nothing. T4 is recorded per site in the tool's `TRIAGE` table with its reason, and a REVIEW hit with no row prints UNTRIAGED and reds the self-test. ⛔ **Every `TRIAGE` row is keyed by an excerpt, never by a line number** — the card cited its own known instance at `:493` and it was at `:555` by the time anyone read it. ### The window, and a bug worth recording The first build read a fixed ±6-line block and **cleared** the one site everybody had agreed was broken: six lines above it an unrelated sentence cites a revision, and a block window handed that anchor to a claim that has none. T2 and T3 now read the **sentence**; T1 also sees the sentence before (a narrative frame leads), T3 also the sentence after (a named failure direction trails — the card's own kind-2 exemplar spells the zero in one sentence and names its falsifier in the next); ⛔ T2 sees neither. That control is pinned in the self-test. ## 3. ⭐ Controls, both directions `node scripts/pm/measurement-claim-triage.mjs --self-test` → **green**, six controls: | control | site | expects | |---|---|---| | kind 1 — a citation carrying its revision, **left alone** | `scripts/check-tier-file-adoption.mjs` ("Measured on this tree at `d03c3c96d6` …") | `KIND-1` | | kind 2 — a zero whose failure direction is named, **left alone** | `scripts/docs-audit/affected-docs.mjs` ("zero commands declare `static topic` …" + "The failure direction if that ever changes …") | `KIND-2` | | kind 2 — a refusal a live assertion holds, **left alone** | `scripts/check-skill-compatibility-version.mjs` ("the refusal is pinned in the self-test …") | `KIND-2` | | **kind 3 — the card's named instance, CAUGHT** | `scripts/docs-audit/README.md` (pre-repair text) | `REVIEW` | | kind 3 repaired — the same site now points at the gate | `scripts/docs-audit/README.md` (post-repair text) | `KIND-2` | | ⛔ the one-sentence anchor rule — a neighbour's revision does **not** clear a bare magnitude | synthetic | `REVIEW` | Each control carries a liveness key that must still be present in the file it was quoted from (and the pre-repair one carries an `absent` key instead — if that sentence comes back, so has the defect). ⛔ A control quoting a sentence the tree no longer holds passes in silence, which is the failure this whole PR is about. The self-test also reds on a **dead cue**: every cue must still match something in the population or the controls — nine that matched nothing were deleted rather than left as decoration. ## 4. The kind-3 hits and their repairs Five, judged out of a pre-repair review residue of 27. After the repairs the tool reports `87 claim(s) over 48 file(s) — KIND-1 35 · KIND-2 11 · NO-FIGURE 19 · REVIEW 22`, and all 22 survivors are recorded ILLUSTRATION. ⛔ No fresh bare number is written anywhere in this diff. **Repaired by pointing at the live instrument** (#16200's preferred shape): - `scripts/docs-audit/README.md` — "the anchor derivation reads the same **178**-page corpus the old one did". Present tense, and **wrong today**: `pnpm check:docs-audit-scope` printed `195 hand-written doc(s)` at 2026-09-20T00:11:30Z, my own reading. Repaired to name the corpus without sizing it and to send the reader to that gate, and it says out loud that the figure is **deliberately absent**. **Repaired by pinning the reading to the commit that recorded it** — the historical figure does work a live reading cannot (it justifies a decision taken at that moment) and no gate reprints it. Each carries a deliberately-absent note, so the next author does not helpfully restore a present-tense figure: - `scripts/check-pnpm-filter-targets.mjs` — the `scripts/**` declaration-honesty ratio, now `Measured at 52a41b7 (2026-08-23)`. The counts move with every file added under `scripts/`; the SHARE is what the paragraph argues. - `scripts/check-published-files.mjs` — the same shape over the whole publishable workspace, now `Measured at 52a41b7 (2026-08-23)`. - `scripts/check-skill-compatibility-version.mjs` — the four precision ratios deciding which roots are declared, now `measured at f29e897 (2026-08-22)`, with a pointer to `scripts/pm/bare-root-worklist.mjs`, which carries this gate's package-root rows with their own dates. - `scripts/check-slot-lookup-ratchet.mjs` — `46s`, the worst-ageing kind of figure: a wall-clock reading is a reading of one **box** as much as of one tree, so it decays without the tree moving at all. Now `Measured at 46s when this was recorded (99ca662, 2026-08-19)`, with the order (seconds against a CI round) left as the load-bearing claim. Each repair's excerpt is recorded `REPAIRED` in `TRIAGE`, and the self-test reds if it returns to the population. ### ⛔ What the classifier deliberately did NOT touch The 22 remaining REVIEW hits are recorded `ILLUSTRATION` with a reason each. Two worth naming, because they are the shape a sweep would have destroyed: - `scripts/docs-audit/README.md` lines carrying `178` in the historical narrative — the card predicted these were kind 1 and judging each one agrees. Two more, at the `--all` backstop and the cost note, already point at `check-audit-scope.mjs` **in the same sentence** and classify `KIND-2` mechanically: already repaired in #16200's shape, so ⛔ this PR leaves them exactly as they are. - `scripts/check-dispatcher-error-vocabulary.mjs`'s per-glob member counts: kind 2, not kind 3 — the docblock names what would falsify them and `PUBLISHED_SOURCE_FACE_FLOOR` plus the per-glob presence pin red in that direction. ## ⛔ Two files are outside the file surface, for two independent reasons **`scripts/pm/dispatch-gates.mjs` and `scripts/pm/check-widening-tells.mjs` are not edited here**, and both reasons are carried as data in the tool's `EXCLUDED` table so every run prints them rather than leaving them in prose: 1. **Live conflict.** `dispatch-gates.mjs` is being edited by open PRs **#19162** and **#19024**; `check-widening-tells.mjs` by **#19153** and **#19024**. Editing underneath them is a conflict, not duplicated work. 2. **The card's own verification constraint**: `scripts/pm/dispatch-gates.mjs` is enormous and its `--self-test` exceeds the agent container's foreground cap (recorded on #17765), so the card required whoever took this to say how they verified a change to that file **or scope it out and say so**. This is the saying-so. ⇒ The residue those two hold is **un-swept**, and the tool's scope line says so on every run instead of reading as complete. Also untouched, and reported rather than edited: `.claude/**`, `skills/**`, `docs/adr/**`, `AGENTS.md`, `CLAUDE.md` (governed surfaces) and #16200's three carriers, settled by #17795. ## Verification Derived with `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` at `60f9b648a` — 6 paths, 722 changed lines. **35 derived families, 35 run, all exit 0**; reconciled with `--ran`: `35 derived famil(ies) accounted for — 35 run, 0 UNRUN`. - `node scripts/pm/measurement-claim-triage.mjs --self-test` → `✓ controls hold in BOTH directions (6 controls, 27 recorded judgements)`. - `pnpm check:pm-dispatch-gates` → `✓ dispatch-gates self-test: 1866 cases pass`, **593.2s on this box**, `VERDICT command-exit 0`, run detached under `scripts/pm/os-verify-lock.sh` (held 594s, waited 0s), 2026-09-20T00:23:16Z–00:33:10Z. ⚠️ The reference line at `content/docs/qa/platform-readings.md:432` records 430–450s; this container reads well above that band, and this run is one more point in it. - ESLint, narrowed and the narrowing measured: `npx eslint --no-inline-config --format json` over the five changed `.mjs` files at `60f9b648a` → **5 files linted, 0 errors, 0 warnings** (counts read from the JSON reporter). The narrowing excludes nothing: `eslint.config.mjs` enables type-aware linting for **no** configuration in this repo (no `parserOptions.project`, no typed rules), so this diff cannot move the verdict on a file it does not touch. The repo-wide `eslint .` run is CI's. - ⊘ **NOT MEASURED** — `pnpm check:published-readme-exports` and `pnpm check:dts-closure` both exited **3, PREREQUISITE NOT MET** (46 packages' `dist/` not built). Neither is in the derived family; both are artifact-roster families whose roster sits under `scripts/`, run here only because silence there is not evidence. This diff changes no package source, so neither could be moved by it. ⛔ Recorded as not measured, not as a pass. - No changeset: measured, not assumed. The root manifest is `private: true` and no package's `files[]` names the repo-root `scripts/` tree, so nothing published moves. Labelled `skip-changeset`. ## Acceptance notes (out of scope, noted and not filed) - `scripts/pm/dispatch-gates.mjs` holds 42 of the 121 phrase occurrences (all spellings) and is un-swept here for the two reasons above. **Carrier: the PR that next touches that file — #19162 or #19024.** Worth a card of its own once they land; the tool classifies it the moment its `EXCLUDED` row is removed. - `scripts/pm/check-widening-tells.mjs` holds 5, same disposition. **Carrier: #19153 / #19024.** - The tool is **not CI-wired**: a `package.json` script and a workflow step both sit outside this card's file surface, so `--self-test` is run by hand and its controls watch nothing on their own. The docblock says so out loud. ⛔ Do not read a green run here as CI coverage. **Carrier: none today** — it needs a seat that owns the root manifest and the lint workflow. Raised as an open question in the report rather than filed, because wiring it is a decision about CI cost, not a defect. --- _Generated by [Claude Code](https://claude.ai/code/session_017ef78bLdybu3AffehKkhfk)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent 7e4ecc5 commit 805811e

6 files changed

Lines changed: 705 additions & 17 deletions

‎scripts/check-pnpm-filter-targets.mjs‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -127,10 +127,16 @@ const SCANNED_EXTENSIONS = [...JS_EXTENSIONS, ...HASH_COMMENT_EXTENSIONS, '.json
127127
* ── Why `scripts/**` is honest here, with the measurement ───────────────────
128128
*
129129
* This is the `subtree` case: the walk descends the whole of scripts/ and every
130-
* file carrying a scanned extension is judged. Measured on this tree, the
131-
* declaration names 235 tracked files under scripts/ and this gate reads 228 of
132-
* them — 97.0%. The 7 it skips are the non-code files the extension filter
133-
* drops, not a subtree it never opens.
130+
* file carrying a scanned extension is judged. Measured at `52a41b72e`
131+
* (2026-08-23): the declaration named 235 tracked files under scripts/ and this
132+
* gate read 228 of them — 97.0%, the 7 it skipped being the non-code files the
133+
* extension filter drops rather than a subtree it never opens.
134+
*
135+
* ⛔ That reading is deliberately pinned to its commit and NOT refreshed here.
136+
* Both counts move with every file added under scripts/ and nothing reprints
137+
* them, so a figure restored in the present tense would read as current and go
138+
* false in silence. What this paragraph argues is the SHARE, and the share is
139+
* what survives the churn; for today's counts, run the walk below.
134140
*
135141
* ── Why the workspace manifests stay UNDECLARED ─────────────────────────────
136142
*

‎scripts/check-published-files.mjs‎

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -388,11 +388,17 @@ function matcher(pattern) {
388388
* This is the `subtree` case, not the `filtered` one check-examples-live-imports
389389
* refuses. `walk()` below enumerates EVERY non-build file of every publishable
390390
* member and MINIMAL judges each of them against FORBIDDEN, so the declaration
391-
* names files this gate really opens. Measured on this tree: the declaration
392-
* names 5263 tracked files and the gate judges 4803 of them — 91.3%. The 460 it
393-
* does not judge are the members whose OWN manifests this gate read in order to
394-
* exclude them (`private`), which is itself a read of the declared subtree, so
395-
* a manifest card there is a true lead rather than a fabricated one.
391+
* names files this gate really opens. Measured at `52a41b72e` (2026-08-23):
392+
* the declaration named 5263 tracked files and the gate judged 4803 of them —
393+
* 91.3%. The 460 it did not judge are the members whose OWN manifests this gate
394+
* reads in order to exclude them (`private`), which is itself a read of the
395+
* declared subtree, so a manifest card there is a true lead rather than a
396+
* fabricated one.
397+
*
398+
* ⛔ Those counts are deliberately pinned to that commit and NOT refreshed. They
399+
* move with every package added to or removed from the workspace, no gate
400+
* reprints them, and the claim this paragraph rests on is the SHARE — that the
401+
* declaration names what the gate really opens — never the level.
396402
*
397403
* The contrast that sets the boundary is in check-published-readme-exports.mjs,
398404
* which enumerates the same members and scores 2.8% — its refusal docblock

‎scripts/check-skill-compatibility-version.mjs‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -97,14 +97,22 @@ const PACKAGE_ROOTS = ['packages', 'apps', 'examples'];
9797
* The instrument can only express a SUBTREE: `hintCovers` collapses globs, so a
9898
* declared hint names every tracked file beneath it and there is no way to spell
9999
* "the package manifests under this root". That makes the two sides of this gate
100-
* completely different trades, and both were measured on this tree:
100+
* completely different trades, and both were measured at `f29e89717`
101+
* (2026-08-22):
101102
*
102-
* skills/** 49 of 50 tracked files are skill directories this gate
103+
* skills/** 49 of 50 tracked files were skill directories this gate
103104
* reads — 98% precision over a 50-file subtree.
104105
* packages/** 73 package.json files out of 4903 tracked files — 1.5%,
105106
* pasted into every packages/** dispatch prompt in the repo.
106107
* apps/** 1 of 35 (2.9%) · examples/** 4 of 238 (1.7%).
107108
*
109+
* ⛔ Those four readings are deliberately pinned to that commit and NOT
110+
* refreshed in place. Every term moves with the tree, nothing reprints them,
111+
* and what decides the trade is the ORDER of magnitude between the two sides —
112+
* which is why a figure restored in the present tense would be a new decaying
113+
* claim rather than a better one. `node scripts/pm/bare-root-worklist.mjs`
114+
* carries this gate's rows for the three package roots, with their own dates.
115+
*
108116
* The three package roots are therefore the +139084 fabrication one level up —
109117
* the very measurement in `hintCovers`' docblock, which prices accepting bare
110118
* top-level directory words at that many fabricated (gate, file) pairs because

‎scripts/check-slot-lookup-ratchet.mjs‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -147,9 +147,17 @@ const BASELINE_PATH = 'scripts/slot-lookup-baseline.json';
147147
* a lead it does not need. That trade is the one `CHANGE_KIND_GATES` already
148148
* decided for the three structurally identical whole-tree ratchets in
149149
* `dispatch-gates.mjs`, and it is decided the same way here — by what the gate
150-
* costs to run needlessly. Measured on this tree: 46s, no build required, and a
151-
* failure names the offending file and line. A seat that runs it needlessly
152-
* loses seconds; a seat that is never prompted loses a CI round.
150+
* costs to run needlessly. Measured at 46s when this was recorded
151+
* (`99ca6623f`, 2026-08-19), with no build required, and a failure names the
152+
* offending file and line. A seat that runs it needlessly loses seconds; a seat
153+
* that is never prompted loses a CI round.
154+
*
155+
* ⛔ The duration is deliberately left at that commit and NOT refreshed in
156+
* place. A wall-clock reading is a reading of one BOX as much as of one tree —
157+
* this repo's own agent containers read several times the figures recorded for
158+
* them — so a fresh number written here would decay without the tree moving at
159+
* all. Time it on the box you are deciding for; what the trade needs is the
160+
* ORDER (seconds against a CI round), and that is what is stated.
153161
*
154162
* Two things a seat prompted by this declaration needs to know, and neither is
155163
* visible from a green `pnpm lint`:

‎scripts/docs-audit/README.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -552,9 +552,12 @@ How often it renders, re-derived over the 40 first-parent commits ending at `e43
552552
3 of the 17 package-touching runs (18%) — `20a452e664`, `f213793ddb`, `dd4113ec0b` — so it
553553
is a rare notice rather than a per-PR banner, which is what keeps it readable.
554554

555-
**Cost** (the card's open question): the anchor derivation reads the same 178-page corpus
556-
the old one did, plus the 18 route-source/ledger files (~875 KB) and one `git show`
557-
per changed file per side. Measured end-to-end on the ten PRs above, `node affected-docs.mjs`
555+
**Cost** (the card's open question): the anchor derivation reads the same hand-written
556+
corpus the old one did, plus the 18 route-source/ledger files (~875 KB) and one `git show`
557+
per changed file per side. ⛔ The corpus SIZE is deliberately absent from that sentence —
558+
run `check-audit-scope.mjs` (`pnpm check:docs-audit-scope`) for today's page count. A size
559+
written down in the present tense decays with nothing going red, which is exactly what the
560+
`178` that used to stand here did. Measured end-to-end on the ten PRs above, `node affected-docs.mjs`
558561
went from 85-195 ms to 114-582 ms. The heaviest case is the widest diff; every case stays
559562
well under a second, against a job that already spends seconds checking out the repo and
560563
setting up Node. It is the right default for every PR.

0 commit comments

Comments
 (0)