Skip to content
This repository was archived by the owner on Oct 26, 2026. It is now read-only.

fix(ci): the affected-tests fast path runs: the shard step compares the output value, not the selector's key=value line (objectui#13350) - #13358

Merged
objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-13350-affected-fast-path
Oct 24, 2026
Merged

objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-13350-affected-fast-path

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Requested by Sales · project thread

Fixes #13350
Clause-②: no
Metadata-protocol: none

维护者速读(草稿)

改了什么:ci.yml 测试分片步骤里的一行比较,从 'mode=affected' 改成 'affected';另加一个钉测,把"选择步骤写出的输出"和"分片步骤读到并比较的值"按 ci.yml 原样真跑一遍对上。

为什么改:两速 CI(#12214)设计成 PR 和合并队列只跑受影响的测试,但这行比较永远为假,快速通道从没生效,每个 PR 和每次队列构建都跑全部 16 片(约 150 分片分钟)。

风险与代价(含回滚):快速通道第一次真正生效。它漏掉的测试由 main 上每次 push 和每小时的全量运行兜底,这是 #12214 已接受的取舍;选择器出错仍按全量跑。回滚就是 revert 这一行。

席位意见:

你要做的:看过后在本 PR 上点 Approve(os-zhuang 或 hotlong)。

What was wrong

The test job's select step (Select the tests this change affects, id select) appends the selector's verdict line to $GITHUB_OUTPUT: the literal mode=affected or mode=full. The runner splits that line at its first =, so the step output is mode with the value affected or full. The shard step (Run tests (shard N/16)) reads steps.select.outputs.mode into $MODE and compared it with 'mode=affected', which is never true. Every pull request and merge-group build therefore took the else branch and ran the full suite on all 16 legs. The type-check scope step in the same file already uses the right shape (it writes scope=affected and compares against 'affected').

The fix

One line: the shard step compares $MODE with 'affected'. A comment above the step says what $MODE holds and names the pin. No other workflow logic changes.

The pin: scripts/__tests__/select-affected-tests-wiring.test.ts

It restates neither end. It extracts both steps' env: and run: blocks from ci.yml (textually, as markdown-test-inputs.test.ts does), runs the select step with bash -e against this checkout (the real selector, based on HEAD), parses what it appended the way the runner does, gives the shard step whatever output its env: names, and reads which command the shard step chose off a pnpm stand-in that records its argv.

  • pull_request and merge_group: vitest gets exactly the filters the selector wrote, and the leg prints Affected speed: N vitest filter(s), shard 1 of 16.
  • push, and a pull request with no base sha (fail-open): the full pnpm test command.
  • Controls: the comparison reverted to the whole line, the output written under another key, or another value under the same key each put a pull request on the full suite.
  • It also asserts that both steps sit in one job, the select step first, and that the shard step binds exactly one select output.

The selector runs once (about 3 s) and is replayed for the other legs. The file runs in about 3.5 s.

Verification

  • New pin: 8 passed. Ablations on the committed fix, each restored from HEAD afterwards: the comparison reverted to 'mode=affected' fails 2 tests (pull_request, merge_group); the select step alone writing speed=... instead of mode=... fails 3.
  • Every test file that reads ci.yml (git grep over **/__tests__/** and *.test.*, plus shard-headroom.test.ts): 62 files, 2064 tests passed, run under the verify lock.
  • One real affected leg, run locally exactly as the step spells it (pnpm exec vitest run --shard=1/16 plus this branch's 669 filters): 36 files, 576 tests passed. The sequencer printed shard 1/16 runs 36 of 669 files, 118.5 s of 1895.2 s by the per-file timings.
  • The planner over this branch's selection gives all 16 legs between 36 and 43 files, about 118 s of measured weight each.
  • pnpm run type-check:scripts exit 0; eslint on the new file exit 0; pnpm check:new-line-citations 0 new; check:control-bytes, check:shell-escape-residue, check:test-path-roots, check:bash32-floor and check:unreferenced-sources exit 0.
  • node scripts/check-changeset-presence.mjs: no changeset owed (no released package source changed), so none is added.
  • node scripts/check-governed-queue-guard.mjs --test on both paths: GOVERNED (Tier H, .github/workflows/**).

Acceptance (read on this PR's own CI)

This PR is the scripts-only class: the selector on this branch answers mode=affected with 669 of 5721 test files (the always-on set plus the new test). So this PR's Test (shard K/16) legs should each print Affected speed: 669 vitest filter(s), shard K of 16., and the Test aggregator's headroom notice is the total-shard-time reading the card asks for.

For the other direction, the selector replayed on b911ebb (10 files, packages/react and packages/app-shell) answers mode=affected with 5056 of 5721 files, which is near-full as the card expects.

Acceptance notes

  • The headroom step (The longest test shard's headroom against the cap) is if: always() and continue-on-error: true, so it cannot red Test. On affected runs its suggested width will read low. That is the sibling dynamic-shard-width card's subject, not this one.
  • An affected leg passes filters, so the root config's passWithNoTests is false and a leg handed zero files would red. The planner puts a file on every leg whenever the selection holds at least 16 files. Today the always-on set alone is 668 files, and the selector already refuses an empty selection (exit 2, which the step turns into a full run). No guard is added, because this card adds no behaviour.
  • Affected legs do not write per-file timings. Only push and schedule upload them, unchanged.
  • Sibling objectui#13349 (weekly timings refresh) touches test-shard-timings-refresh.yml and the timings file, which is disjoint from this diff.

Governed surface

.github/workflows/ci.yml is Tier H. This PR stays draft until an authorized approval (os-zhuang / hotlong) is on record. ⛔ No seat approves it. The maintainer's direct-dispatch word is quoted on the card (comment 6357584416).

  • Adds no product-specific widget, dialog, hook or string (see AGENTS.md › Product-neutral renderer)

Implemented by the ui lane card thread, session https://claude.ai/code/session_01LyeJkFWzSqWc3SDQPAqpsn, branch claude/issue-13350-affected-fast-path. The claim is on the card, comment 6364039593.


Generated by Claude Code

🤖 Generated with Claude Code

https://claude.ai/code/session_01LyeJkFWzSqWc3SDQPAqpsn

… the output value, not the selector's key=value line

The select step appends the selector's verdict line (`mode=affected`) to
$GITHUB_OUTPUT, which the runner stores as output `mode` = `affected`.
The shard step compared $MODE with the whole line, so the affected
speed never ran and every pull request and merge-group build ran the
full sixteen-shard suite. Compare against `affected`, the shape the
type-check scope step already uses.

scripts/__tests__/select-affected-tests-wiring.test.ts runs both steps
as ci.yml writes them (the real selector, the runner's output parsing,
a pnpm stand-in recording argv) and holds the two ends together.

Claude-Session: https://claude.ai/code/session_01LyeJkFWzSqWc3SDQPAqpsn
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: b684b30ff4631b757d1bf0622c5eed5769bb2974
Local-runs: none
Scope: .github/workflows/ci.yml + scripts/tests/select-affected-tests-wiring.test.ts, scripts/select-affected-tests.mjs, scripts/tests/select-affected-tests.test.ts, vitest.config.mts, scripts/shard-headroom.mjs, scripts/check-governed-queue-guard.mjs

Face: .github/workflows/ci.yml is GOVERNED_SURFACES id workflows (scripts/check-governed-queue-guard.mjs:326, glob .github/workflows/**) — the rules face. The PR's other file is the pin that enforces the rule, read as one change with it.

Inputs read: card #13350 body and its three comments (6357584416 filing, 6364039593 claim, 6364342801 os-dev-report); PR #13358 body, two-file list, and the net diff against main at merge-base 2c4e5f5 (one commit, +311 −1); the check-runs on the head, polled until every one completed. Not read, by rule: the dispatch order and the dispatching seat's conclusions.

Check-runs on the head: 33 of 33 completed — 28 success, 3 skipped by design (Test (coverage) and Test (coverage shard …/8) are the push lane; dependabot is not a Dependabot PR), 0 failure. The Test aggregate gate (job 120401845643, run 39949632539): success — its shard-results step, dist-pins step and headroom step all success. Governed Surface Queue Guard success on the PR leg, as designed for a draft governed PR. Lint and Repo Gates, still in progress at brief time, completed success; last poll 2026-10-23T12:04Z.

① Derived judgments

  1. The one-line fix is right. The select step (ci.yml 1244–1288 on the head) appends the selector's last stdout line, mode=affected or mode=full, to $GITHUB_OUTPUT; the runner splits each line at its first =, so steps.select.outputs.mode is affected or full. The shard step (1311–1324) binds MODE: ${{ steps.select.outputs.mode }} and now compares with 'affected' — the shape the type-check lane already uses (756 writes scope=affected, 781 reads steps.tc-scope.outputs.scope, 784 compares with 'affected'). The old literal 'mode=affected' could never match. steps.select.outputs has exactly one reader in the file (1314), so the change has no second consumer.

  2. Accept-set on pull_request and merge_group: the affected speed now runs, which is exactly what objectui#12214 accepted ("affected-only tests on PR and merge-queue runs, the full suite and coverage on main's schedule"). Read on this head (run 39949632539, the scripts-only class): every shard's Select the tests this change affects step took 1–3 s, every Run tests (shard N/16) step 59–122 s, 21.3 min of test work across the 16 legs and 33.5 min of job wall, against the ≈ 150–156 shard-min the card measured for the same class; all 16 legs success. Upload this shard's per-file timings was skipped on all 16, as its if: (push or schedule only) requires — an affected slice cannot feed the weekly timings refresh. Right.

  3. The full suite is still forced wherever it must be — nothing narrowed. (a) push, schedule and every other event: the select step's *) arm writes mode=full and exits 0 before the selector runs, so $MODE is full and the else branch runs pnpm test --shard=N/16 --reporter=default --reporter=github-actions --reporter=./scripts/test-file-timings.mjs, byte-for-byte unchanged. (b) The selector's own mode=full verdict on a PR or queue run (a package.json, vitest/vite config, tsconfig, root non-doc file, patches/, config-reach or broad-setup package in the change): else. (c) A selector non-zero exit, no verdict line, or a missing base sha: verdict='mode=full' with a ::warning::, then else. (d) An empty selection: the selector refuses with exit 2 (select-affected-tests.mjs 697–700), which is (c). (e) steps.relevant.outputs.should_run false: both steps are skipped together, unchanged. The else text, the matrix width and the test-aggregate gate (check-test-shard-results.mjs --shards 16, which refuses a partial set) are untouched. Right.

  4. The pin executes the steps as written, so a regression turns it red. select-affected-tests-wiring.test.ts lifts both steps' env: and run: text out of ci.yml (refusing a duplicate header or a missing run: |), resolves ${{ }} through a stand-in table that throws on any expression it does not know, runs the select step with bash -e against the checkout (the real selector, --base HEAD: an empty change, the case the existing CLI pin in select-affected-tests.test.ts already holds to mode=affected plus the always-on set), parses $GITHUB_OUTPUT as the runner does (first =), binds the shard step's env from the parsed outputs, and reads the chosen command off a pnpm stand-in that records argv. Legs: pull_request and merge_group hand vitest exactly the selector's filters and print Affected speed: N vitest filter(s), shard 1 of 16.; push and a PR without a base sha hand it pnpm test. Controls: the comparison reverted to the key=value line, the outputs under another key, another value under the same key — each alone lands a PR on the full suite. Traced through the code, each end edited alone fails a leg: a key rename in the select step reaches the shard env as an unwritten output (empty string), a value rename in the selector or a literal change in the shard step fails the equality, and only a consistent rename of both passes — the card's "fails if either side of the comparison is edited alone" holds. The file is in the root unit project (scripts/**/*.test.ts, vitest.config.mts 398–404, not in sharedExclude) and in the selector's always-on set (outside every workspace package), so it ran on one of this head's 16 green legs. Right.

  5. Behaviour reachable for the first time, read and accepted — not a breach. The affected command carries no --reporter=github-actions and no timings reporter, so affected legs report through vitest's default reporter only (no inline annotations on a red). That is the command objectui#12214 accepted and the existing pin holds (keeps the matrix width and the full-suite command); if the lane wants annotations on the fast path it is a sibling, not this card. With filters on argv, passWithNoTests is false (vitest.config.mts 472), so a leg handed zero files reds rather than passing silently — the right direction for a gate; all 16 legs were non-empty here (669 filters, 36–43 files per leg per the planner).

  6. Public surface: none. No package exports, no packages/spec/src/**, no schema and no .d.ts are touched; scripts/__tests__/** is not shipped. The only face is the governed .github/workflows/ci.yml. The Tier H human approval (os-zhuang / hotlong) is still owed and the PR is correctly draft; this record does not stand in for it.

  7. Card acceptance reading, for the lane seat's ACCEPT (not a contract clause): the headroom notice measures job wall (shard-headroom.mjs 31 and 81: completed_at − started_at per job). On this head that total is 33.5 min over 16 shards, of which ≈ 12 min is the fixed per-leg checkout, node and install the card itself prices at 35–60 s per leg, and 21.3 min is test work — against ≈ 156 before. The card's "≤ 30 min" is met on test work and missed by 3.5 min on job wall; that remainder is a width question for the sibling dynamic-shard-width card, not this one.

② Semver level

None owed, none declared — right. The diff touches .github/workflows/ci.yml and one test under scripts/__tests__/; no released package source changes, so check-changeset-presence owes nothing and the diff adds no changeset. Changeset Declaration, Changeset Fixed Group Check and Changeset Claim Re-read are success on the head. Clause-②: no on the PR body (line 5) and on the claim (6364039593) matches the diff: no published accept set is narrowed; the CI accept-set change is the one objectui#12214 already accepted and the card restores. Metadata-protocol: none matches a diff that writes no metadata.

③ Boundary flags

  • os-dev-report (6364342801) open_questions: empty — nothing to answer.
  • out_of_scope_finding 1, the headroom suggested width reads low on affected runs: confirmed report-only — if: always() and continue-on-error: true (ci.yml 1587–1592); on this head the step completed success in 1 s and Test is success. The card's item 3 ("make sure it does not red") is answered by the head's check-run. Carrier: the sibling dynamic-shard-width card. Not escalated.
  • out_of_scope_finding 2, a zero-file leg would red: confirmed (passWithNoTests: !cliHasTestFilters(process.argv)); the selector refuses an empty selection (exit 2 ⇒ full), the always-on set alone is 668 files, and every leg was non-empty here. The residual (a 1–15 file selection) is unreachable while scripts/** tests exist and belongs to the same sibling if the width ever becomes dynamic. Not escalated.
  • Dev-flagged governed face: Tier H, the PR stays draft until an authorized approval — a human gate outside this record; the queue guard on the head is success on the PR leg as designed.
  • Two check-runs in progress at brief time (Lint, Repo Gates): both completed success before this verdict.
  • Job logs were not readable through this container's proxy (the blob-storage redirect is refused), so the Affected speed: line itself is not quoted here; the step-level timings of all 16 legs and the skipped timings upload on every leg are the reading in its place.

Implemented-by: claude/issue-13350-affected-fast-path
Reviewed-by: session_01WJozybDmRH8z2c5SD7vJkj

VERDICT: PASS


Generated by Claude Code

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

维护者速读

  • 改了什么: ci.yml 改一行,外加一个新钉测文件。测试分片步骤原来拿选择器的输出和错误的字符串比,结果永远不等,所以「只跑受影响测试」的快速通道从来没生效过,每个 PR 都跑全量 16 片。现在改成与选择器实际输出的 affected 比较。新钉测按 ci.yml 原文真跑这两个步骤,以后任一端改坏都会变红。
  • 为什么改: objectui#12214 设计的两档 CI 一直没生效。以本 PR 自己的 CI 为例,16 片总耗时从约 156 分钟降到 33.5 分钟,最长一片 3 分 2 秒,约 60% 的 PR 能少等 6–9 分钟。
  • 风险与代价: 该跑全量的场景一律还跑全量:push 到 main、定时任务、选择器报错、找不到 base、选出空集,都会回落到全量。改动只决定 PR 和合并队列跑哪些测试,Test 汇总门本身不变。回滚就是 revert 这一行。卡面「总分片时间 ≤ 30 分钟」这次读数是 33.5 分钟,多出来的主要是每片约 1 分钟的固定安装开销,要靠兄弟卡的动态分片宽度来压,本卡不越界。
  • 席位意见: 同意合并。fable 档隔离复核 PASS(comment 6364543703),CI 33 项全绿或按设计跳过,钉测确认在 CI 上实际跑到。
  • 你要做的: 在 PR fix(ci): the affected-tests fast path runs: the shard step compares the output value, not the selector's key=value line (objectui#13350) #13358 上点 Approve。批准后由席位转 ready、进合并队列。

席位核对

Seat: domain:ui#1 · Clause-②: no · head b684b30ff4631b757d1bf0622c5eed5769bb2974 · 2026-10-23T12:10Z

项 读数
形态 2 个文件:.github/workflows/ci.yml(+8/−1)、scripts/__tests__/select-affected-tests-wiring.test.ts(新增)。ci.yml 属受治理面,Tier H,走终局四件套。没有 changeset,因为不碰已发布源码;changeset 三项检查都绿。
前提实测 选择器往 GITHUB_OUTPUT 写的是 mode=affected / mode=full。分片步骤读 steps.select.outputs.mode,现在与 'affected' 比较,写法与同文件 tc-scope 步骤一致。整个文件里只有这一处读 steps.select.outputs。
门禁 check-runs 33/33 完成:28 success,3 按设计跳过(覆盖率只在 push 跑;dependabot),0 失败。Test 汇总门 success(job 120401845643,run 39949632539)。
契约复核 CONTRACT_REVIEW_TIER 隔离复核 PASS,comment 6364543703,Local-runs: none。
接受的偏差 卡面「总分片时间 ≤ 30 分钟」的读数是 33.5 分钟:其中实际测试约 21.3 分钟,约 12 分钟是每片固定开销。余量归兄弟卡(动态分片宽度)。快速通道的命令没有 --reporter=github-actions,变红时不出行内标注;这是 #12214 当时接受的命令,现有钉测也守着它。
范围外去向 headroom 建议宽度读数偏低,仅作报告(continue-on-error),不会变红。零文件分片不可达:选择器会拒绝空集,常驻集有 668 个文件。
落地条款 本 PR 同笔挂 needs-user-decision,保持草稿,等授权批准人(os-zhuang / hotlong)批准。获批后席位 pr_ready + automerge_enable 进合并队列。⛔ 席位不批准。

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 24, 2026 02:12
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 24, 2026 02:12
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 24, 2026
Merged via the queue into main with commit 861fdff Oct 24, 2026
34 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-13350-affected-fast-path branch October 24, 2026 02:21
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

3 participants