Repository navigation
ci(test): the coverage lane runs eight shards, then both test matrices shard by measured duration (objectui#12208, objectui#12256) - #12263
Conversation
… the measured walls (objectui#12256, objectui#12208) The `test-coverage` matrix goes from 4 to 8. At four, a coverage shard was cancelled at its 40-minute ceiling on 7 of the 25 newest `push` runs on `main` (objectui#12256), and every shard of the runs that finished sat one to three minutes under it. The four shards summed to about 91-153 minutes on the finished runs; at a 20-minute target per shard that is eight. Every spelling of the width moves with it: the job name, the step name and `--shard=N/8`, and `coverage-report`'s blob count, which now reads one job variable, `COVERAGE_SHARDS`, for both the blob glob and the count it refuses to go below. `scripts/__tests__/coverage-shard-width.test.ts` holds each spelling to the matrix. The Dependabot gate classifies the eight new shard names, and the red-cause census reads a coverage shard at any width. Not changed: `timeout-minutes: 40`, any test, and `coverage.thresholds`, which are still enforced once, by the merge of every shard's blob. Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB Co-authored-by: Claude <noreply@anthropic.com>
…sure it on the full battery, print each aggregator's headroom (objectui#12208) PR-2 proper of objectui#12208 (ruling 6121816480, letter B). - `scripts/test-shard-sequencer.mjs` is the root vitest config's `sequence.sequencer`. Its `shard()` bin-packs the files heaviest first (`scripts/test-shard-partition.mjs`, pure) from the committed per-file timings, `scripts/test-shard-timings.json`. A file the timings do not name is weighed at their median. While the timings hold no file, the split stays vitest's own path-hash split, byte for byte; this commit ships them empty. - `scripts/test-file-timings.mjs` is three things in one file. As a vitest reporter, the full-battery `test` legs run it to record each file's whole time on its worker (the built-in json reporter's span leaves out import, setup and environment). As `select`, it picks the eligible runs on `main`. As `merge`, it writes the dataset from their artifacts. - `.github/workflows/test-shard-timings-refresh.yml` regenerates the dataset weekly and opens a pull request. - `scripts/shard-headroom.mjs`: `Test` and `Test (coverage)` print the longest shard against the cap, and the width the run's measured total implies at the target per shard. The reading is report-only: `continue-on-error`, and exit 0 whatever it reads. TEMPORARY in this commit only: a `test` step that prints each leg's per-file timings as notices on this pull request's run. A dev container cannot download an artifact, so this carries out the first measurement. The next commit removes the step. Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB Co-authored-by: Claude <noreply@anthropic.com>
Brings in objectui#12222's type-check change (PR #12251), which edits the `type-check` job of ci.yml and turbo.json. The CI doc's job table conflicted on adjacent rows. The resolution takes main's `type-check` row and this branch's `test` row: each side changed only its own row. Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB Co-authored-by: Claude <noreply@anthropic.com>
…me (objectui#12208) The runner keeps at most ten notices per step. On run 38268236293 each leg held about 415 files, which is twelve notice parts at 3,000 characters, and parts 11 and 12 were dropped. The parts now go out in three steps of nine at most. The next commit removes all three steps. Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB Co-authored-by: Claude <noreply@anthropic.com>
…p steps go (objectui#12208) `scripts/test-shard-timings.json` now holds 4,979 files, so both matrices split by measured duration from this commit on. The numbers come from this pull request's full run 38269820609 (head 5874484, the twelve legs all green). Each leg's writer report was carried out as notice annotations and rebuilt with its file count and millisecond sum checked against the leg's own. `scripts/test-file-timings.mjs merge` then wrote the file. No timing was edited by hand. The run is named under `measured.runs`, with the channel it came through. From the first refresh on, the source is the artifacts of main's full-battery runs. On the real spec list (4,979 specifications from vitest's own resolution), the committed sequencer gives every leg its bin with nothing missing and nothing twice, at 12 legs and at 8. The generator's `generated_by` text now points at `measured.runs` rather than claiming artifacts. The three TEMPORARY notice steps are removed. ci.yml's `test` job is back to the commit-2 shape. Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB Co-authored-by: Claude <noreply@anthropic.com>
… them from both hash-split runs (objectui#12208) The first full run on the path-keyed dataset (run 38271639400) went red in one leg. objectui#8800's retired-spelling pin sweeps the tracked tree with `git grep` and excludes only its own file. A dataset that lists every test path carries that pin's own name, which IS the spelling it retires. Any present or future pin of that shape would meet the same collision on the next refresh. The dataset is now keyed by `pathKey()`: 16 hex digits of the sha1 of the path (schema `/2`). The reader refuses a key that is a path. A new pin asserts that no tracked test file's name occurs in the dataset text. The refresh summary still names files, from the runs' own reports. The dataset is regenerated from BOTH hash-split pull request runs, with the per-file median of 38268236293 (partial: the runner kept 10 notices per step, so about 380 of each leg's 415 files) and 38269820609 (complete). A single run carries its legs' runner speed into every file it measured. On the real spec list, the plan still places 4,979 specifications exactly once at 12 legs and at 8. Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB Co-authored-by: Claude <noreply@anthropic.com>
维护者速读(终稿)· PR #12263 · 席位已对照 diff 逐行校正,2026-10-12T16:28Z改了什么:两件事在同一个 PR 里。第一(首个提交,也是 main-red #12256 的止血):main 上的覆盖率测试从 4 路改成 8 路,宽度在 为什么改:最近 25 次合并到 main 有 7 次覆盖率分片跑满 40 分钟被取消,那几次的覆盖率门禁根本没执行。普通测试线同一次运行里最慢分片比最快的慢六到九成,原因是按路径哈希分、不看耗时。 风险与代价(含回滚):覆盖率线每次全量多用 4 台 runner(公共仓库不计费)。按耗时分片会改变哪些文件同组跑,文件间的隐性相互影响可能暴露;本 PR 自己的两次全量运行已经暴露并修掉一处(第一份数据集按路径做键,被 #8800 的「退役拼写」钉子撞上,改成哈希键)。两点实测提醒:① 两次对比运行看不出均衡收益——main 上同一分片序号的耗时本身每次波动约 100 秒,和哈希分配的不均衡一样大,真正的读数要等落地后十次全量运行;② 覆盖率 8 路的耗时在 PR 上测不到(只在 push/schedule 跑),落地后第一次全量运行才知道。周刷新 PR 用 Actions 自带 token 开,GitHub 不会自动给它跑 CI,需要席位推一次(objectui#5397 已裁不引入 PAT)。回滚:把 席位意见:建议批准、先落地。另有一个要你定的数:普通测试线的路数。裁决定的是 12,PR 也保持 12;但按裁决自己的算法(总耗时 ÷ 每路约 10 分钟)main 中位数算出来是 13(最大值 14),两次均衡运行最慢分片 753 s / 731 s 都超过卡上 720 s 的验收线,12 路很可能过不了十次验收。dev 建议现在就改 13(一个机械提交,多一台 runner)。我的建议:先按 12 落地止血,落地后 main 每次全量运行都会打印「应该几路」,连续几次都说 13(或 14)就补一个机械提交——数据驱动的改动,不另立卡。要现在就改 13 也行,代价是多跑一轮 PR CI(约 15 分钟)再落地。 你要做的:在 PR 上 Approve(并告诉我 12 还是 13);席位随后翻正式、入队,落地后读第一次全量运行的 8 路覆盖率耗时和 headroom 读数,再读十次验收。 |
…eck-root-dist One conflict, content/docs/guide/ci-cd-pipeline.md, two adjacent table rows: the `type-check` row is this branch's, the `test` row is main's. Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB Co-authored-by: Claude <noreply@anthropic.com>
Fixes #12208
Fixes #12256
Clause-②: no
PR-2 of objectui#12208 (ruling 6121816480, letter B, staged), with objectui#12256 (the
Test (coverage)main-red) folded in by the director's note 6126024274. Dispatched by the director seat; implemented in sessionsession_01LqNgdPZB2Jnr5HpGbaqLeB.First commit: the coverage lane goes from 4 to 8 shards
This is the p1 main-red fix, and it is mechanical. At four shards a coverage shard was cancelled at the 40-minute ceiling on 7 of the 25 newest
pushruns onmain(objectui#12256), and every shard of the runs that finished sat one to three minutes under it, so on those 7 commits the coverage gate never ran. The four shards summed to about 91-153 minutes on the finished runs; at a 20-minute target per shard that is eight.ci.ymltest-coverage: the matrix is[1..8], and the job name, the step name and--shard=N/8all say 8.ci.ymlcoverage-report: the width is said once, as the job variableCOVERAGE_SHARDS: '8'. The blob count step reads it for both the glob (blob-*-${COVERAGE_SHARDS}.json) and the count it refuses to go below. The verdict's wording reads it too.scripts/__tests__/coverage-shard-width.test.ts(new) holds every spelling of the width to the matrix: the job name, the--shardflag, the step name,COVERAGE_SHARDS, no literal width left in the count step, and a merge step that overrides no threshold.scripts/dependabot-merge-gate.mjsclassifies the eight new shard names. Its partition test expands ci.yml's matrix and fails on any name the list lacks.scripts/coverage-red-cause-census.mjsreads a coverage shard at any width.content/docs/guide/ci-cd-pipeline.md: the two coverage rows and theneeds:paragraph.Not changed:
timeout-minutes: 40, any test (nothing skipped or quarantined), andcoverage.thresholds. The thresholds are still enforced once, bycoverage-report, over the merge of every shard's blob.Shown locally: the shard legs wrote 8 blobs from ten test files, one per
--shard=k/8, all exit 0.--merge-reportsread all 8 back (Test Files 10 passed (10)). Over that merged map, a lines threshold of 70 exits 0 and 71 exits 1 withERROR: Coverage for lines (70.57%) does not meet global threshold (71%). Ablations on the committed tree: withCOVERAGE_SHARDSset back to'4', the width pin goes red (1 failed); with the matrix set back to[1, 2, 3, 4]it goes red on 3 of 5 tests. Both restores were verified (git diff HEADempty). The coverage walls at eight shards are NOT MEASURED here, because this lane runs onpushandscheduleonly. The seat reads them on the first full-battery run after landing.The commits that follow, in this same pull request
These are planned. The report comment on objectui#12208 states what landed and what was measured.
shard()bin-packs test files by measured per-file duration (greedy, longest first), for both matrices. Vitest 4.1.10 callssequence.sequencer'sshard()whenever--shardis set.testshards. The built-in json reporter's per-file span covers only the tests, not import, setup or environment.main's full-battery runs by a workflow that opens a pull request. The timings are never hand-edited.TestandTest (coverage)aggregators: the longest shard against the cap, and the width the measured total implies at the target per shard. It is not a gate.Acceptance notes
mainwith the longest shard under the target plus margin, and the coverage reds stopping at the first full-battery run after this lands. The seat reads both after landing..github/workflows/**(Tier H). It stays a draft until aGOVERNED_APPROVERSmember approves. It is never marked ready or queued by the dev.AGENTS.mdstill gives--shard=1/4as its example.changeset-release.ymlstill describes the coverage lane as four shards in two comments.维护者速读(草稿)
改了什么:main 上的覆盖率测试由 4 路并行改为 8 路。本 PR 后续提交会在同一个 PR 里继续三件事。第一,两条测试线都按每个测试文件的实测耗时分配分片,不再按文件路径哈希。第二,耗时数据每周从 main 的全量运行自动刷新,并开 PR。第三,两个测试汇总检查打印「最慢分片离上限还有多少」,只报告,不拦截。
为什么改:最近 25 次合并到 main,有 7 次的覆盖率分片跑满 40 分钟被取消,那几次提交的覆盖率门禁根本没有执行。普通测试线也一样:同一次运行里,最慢的分片比最快的慢六到九成(两次全量运行的读数)。原因是分片按路径哈希分,不看耗时。
风险与代价(含回滚):覆盖率线每次全量运行多用 4 台 runner,每台多约半分钟准备时间。按耗时分片会改变哪些测试文件一起跑;以前没暴露的文件间相互影响,可能在新组合下暴露,而且会在本 PR 自己的全量运行里先暴露。回滚很简单:把
ci.yml的宽度改回去,或删掉vitest.config.mts里的 sequencer 一行,就回到原来的哈希分配。席位意见:
你要做的:本 PR 改了
.github/workflows,需要你在 PR 上批准,之后由席位落地。另外,每周的耗时刷新 PR 由 Actions 自带的 token 打开,GitHub 不会自动给它跑 CI,需要席位推一次才会触发(与 release PR 同一限制,objectui#5397 已裁不引入 PAT)。Generated by Claude Code