Skip to content

ci: retire Spec Coverage and Engine Split Metric; run the showcase smoke in the Nightly Tiers live tier - #26529

Merged
os-zhuang merged 7 commits into
mainfrom
claude/project-thread-q9cfh6
Oct 27, 2026
Merged

os-zhuang merged 7 commits into
mainfrom
claude/project-thread-q9cfh6

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Requested by Steve · project thread
Fixes #26445

Clause-②: no
Metadata-protocol: none
Ruling-ref: 6433804956

维护者速读

  • 改了什么:删掉两条没人读的定时工作流(Spec Coverage 每天、Engine Split Metric 每周);Showcase Smoke 不再是独立工作流,它的整套 Playwright 冒烟改由夜跑 Nightly Tiers 里的一个 *.live.test.ts 运行。
  • 为什么改:裁决 B(6433804956)。三条工作流红了都不开卡、没人看;并进夜跑后,showcase 冒烟红了会由夜跑的 report 步骤开卡。
  • 风险与回滚:夜跑分片 1 多约 8 分钟(构建 Console + 装 Chromium + 冒烟本身),仍在 45 分钟上限内;回滚即 revert 本 PR,三条工作流原样回来。
  • 先看:.github/workflows/test-nightly-tiers.yml(新增准备步骤、诊断上传、分片腿顺序)、examples/app-showcase/test/showcase-smoke.live.test.ts、examples/app-showcase/vitest.config.ts。
  • 两分钟验证:本 PR 改了 test-nightly-tiers.yml,所以它自己会在 PR 上跑一次 Nightly Tiers(分支上不写看板);看 Nightly tiers (1/2) 里 showcase 那一腿的 53 passed。
  • 需要你做的:这是 Tier H(.github/workflows/**、scripts/pm/**),请批准后由席位落地。

What changes

Retired (deleted):

  • .github/workflows/coverage-nightly.yml (Spec Coverage, nightly). Its only product was a coverage-report artifact; nothing reads it. packages/spec's test:coverage script stays as an on-demand developer command (it is not a check:* gate and no workflow calls it after this PR).
  • .github/workflows/engine-split-metric.yml (Engine Split Metric, weekly, report-only). The ADR-0076 D7 metric script stays in scripts/, unwired from any schedule: scripts/check-engine-split-ratio.mjs runs on demand (its header now says so), and its --self-test stays wired in lint.yml, so the instrument stays correct while unrun. Keeping it means ADR-0076 still names a script that exists; its sentence naming the deleted workflow needs a one-line note, which is a separate governed PR (this one does not touch docs/adr/**).
  • .github/workflows/showcase-smoke.yml (Showcase Smoke, nightly).

Showcase smoke moved under Nightly Tiers:

  • New examples/app-showcase/test/showcase-smoke.live.test.ts. The *.live.test.* name puts it in the nightly live tier (scripts/nightly-tiers.mjs), so no pull request or merge-queue run collects it. It does not restate any assertion: it runs playwright test with the package's own playwright.config.ts (all three specs under e2e/: the 49 nav surfaces, the ADR-0085 detail shapes, the bulk capability gate), streams the list reporter to stdout for the stall guard, and passes only when Playwright's JSON report has no failing test, no run-level error, and at least one test ran. A red names each failing spec (file › title) and, for a whole-run red, says to read it as environment first.
  • examples/app-showcase/vitest.config.ts reads OS_TEST_TIERS through scripts/nightly-tiers.mjs (readTierMode, isNightlyTierFile, imported, not re-spelled): under queue the tier files are excluded by exact path and the default include is unchanged; under nightly they are the include. turbo.json's @objectstack/example-showcase#test already hashed OS_TEST_TIERS; it now also lists scripts/nightly-tiers.mjs (+ .d.mts) as inputs, as @objectstack/cli#test does.
  • .github/workflows/test-nightly-tiers.yml:
    • new step "Prepare the showcase browser smoke": on the shard carrying @objectstack/example-showcase only, pnpm console:build and playwright install --with-deps chromium (the two prerequisites the retired workflow had). continue-on-error, so a failed Console build cannot take the shard's CLI slice down with it; the smoke then fails as a whole-run red and the night still files a card.
    • new step "Publish the showcase smoke's Playwright diagnostics" (examples/app-showcase/test-results/, always(), if-no-files-found: ignore), replacing the retired workflow's two diagnostics uploads.
    • the leg loop runs the file-level slices first and the whole-package leg last (for LEG in $SLICES __whole__), with a comment on why it differs from ci.yml here: the loop stops at the first red leg, and the cheap whole package (the smoke) must not stop the shard's slice of the CLI e2e tier.
    • header comments that named the two retired workflows are reworded.

Pins and readers updated:

  • scripts/check-stall-guard-budget.mjs: self-test battery 10 registers one case per real (workflow, job) with a guarded step; coverage-nightly's job was one, so its floor goes 13 → 12 with a comment saying why.
  • scripts/pm/dispatch-gates.self-test.mjs: the live-tree case that required scripts/check-engine-split-ratio.mjs --days 90 in workflow text drops that key (same treatment as check-prerelease-pin-watch.mjs --verbose when its workflow left); the fixture row quoting it stays.
  • docs/qa/platform-checklist/areas/platform-core.json: the showcase item's source names the live test and test-nightly-tiers.yml instead of the deleted workflow.
  • Comment pointers in half-state-patrol.yml, shard-timings-refresh.yml and examples/app-showcase/playwright.config.ts.

Readers grepped (file names, workflow names, job names, artifact names, test:smoke, test:coverage): dependabot.yml, the required-context set, the aggregator roster, check:self-test-wired and scripts/ci/scheduled-full-run.mjs hold none of the three. scheduled-full-run.mjs uses 'Spec Coverage' only as an example of a name outside its map, which stays true.

Tests

Local, on this branch (worktree objectstack-issue-26445), closure built with pnpm turbo run build --filter=@objectstack/example-showcase^... and pnpm console:build:

  • Live tier, real stack (OS_TEST_TIERS=nightly vitest run in examples/app-showcase, Chromium from /opt/pw-browsers): 53 passed (2.3m) from Playwright, Test Files 1 passed (1), Tests 1 passed (1), at 0d08883a.
  • Negative control: scripts/ablation-replace.mjs injected a failure into the Task Board surface only (anchor 1 → 0, blob cf29f07379cd → 26b53aa6a54e). Result: Playwright 52 passed, Task Board failed twice (with its retry), and the vitest test failed with 1 of 53 showcase smoke test(s) failed, 0 run-level error(s) naming showcase-smoke.spec.ts › surface renders cleanly: Task Board. Restored: blob equals HEAD, git diff HEAD empty.
  • Two reds on the way, both fixed in this PR and kept in the test's comments: vitest's worker sets NODE_ENV=test, which disarms the dev-admin seed, so global setup's sign-in got 401 (the wrapper now drops vitest's worker env); a randomly picked port that was busy made serve --dev shift one port up while Playwright waited on the original until its 180 s timeout (the wrapper now takes a port from the OS).
  • Queue tier: pnpm --filter @objectstack/example-showcase test → Test Files 39 passed (39), Tests 484 passed (484) (the live file is not collected). pnpm --filter @objectstack/example-showcase typecheck exit 0.
  • node scripts/nightly-tiers.mjs --check: @objectstack/example-showcase owns 1 nightly-tier file(s); vitest collects 39 under queue, 1 under nightly; CLI unchanged (512 / 85). OS_TEST_TIERS=bogus is refused by the config.
  • Partition of the nightly document (--packages → partition-test-shards.mjs --shard k/2): shard 1 = @objectstack/cli 1/2 + @objectstack/example-showcase (kept whole, "not the run's floor"), shard 2 = @objectstack/cli 2/2. nightly-tiers.mjs --self-test and partition-test-shards.mjs --self-test exit 0.

Acceptance notes

  • Measured on a runner by this PR itself: test-nightly-tiers.yml is in its own pull_request paths filter, so this PR runs Nightly Tiers on a hosted runner (branch runs never write to the board). That run is the first reading of the prepare step and the smoke under turbo's strict env.
  • Cost: shard 1 took ~24.6 min on the last scheduled night (run 40275583120). The retired workflow spent ~4.6 min on the Console build and ~1.4 min on the smoke (run 40286891669), so shard 1 should land near 33 min against its 45-minute limit. The partitioner weighs the showcase by its queue-tier timing (48 s) and does not know the live run's cost; if shard 1 gets tight, the lever is the matrix, not this file.
  • Not changed, read-only for this card: .github/workflows/ci.yml L810 still says "coverage-nightly workflow" in a comment. Left as dated readings: docs-deploy.yml (cron-hour survey "on the day this landed"), lint.yml L1495 (a 2026-08-21 measurement), scripts/docs-audit/README.md, check-pnpm-acquisition.mjs, check-node-version.mjs, check-stall-guard-budget.mjs's "measured when this landed" line, and dispatch-gates.mjs comments that cite the old workflows as history.
  • Follow-up owed (separate governed PR): ADR-0076's D7 line names .github/workflows/engine-split-metric.yml; a one-line note that the script now runs on demand.
  • Root test:smoke (turbo) and the package's test:smoke stay: pnpm test:smoke is still how a person runs the suite by hand, and the QA checklist items cite it.

Verification

Gate list derived with node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack at the final head: 86 commands at 57f8caa3, each run with its exit code captured before any pipe: 86 exit 0. dispatch-gates --ran: "86 derived famil(ies) accounted for — 86 run, 0 NOT-MEASURED (a DERIVED zero)". Two were red on the way and are fixed in this PR: check:stall-guard-budget (battery 10 floor, see above) and check:pm-dispatch-gates (1 of 1935 fast-tier cases, the engine split live key). The dispatch-gates slow tier (node scripts/pm/check-dispatch-gates.mjs --slow, push/schedule only in CI) also ran: ✓ dispatch-gates self-test: 2200 cases pass, exit 0. After merging origin/main (2094edeb, which brought the shard-matrix reader into partition-test-shards.mjs) the derivation is unchanged (86) and the overlapping gates were re-run: partition-test-shards.mjs --self-test, nightly-tiers.mjs --self-test, nightly-tiers.mjs --check all exit 0, and the nightly partition is unchanged. Families CI runs (Lint & Repo Gates, TypeScript Type Check, Test Core, Build Core, the UI lanes, Nightly Tiers on this PR) are read from this PR's check-runs.

Changeset: none. skip-changeset — workflows, root scripts, turbo.json and the private @objectstack/example-showcase publish nothing.

Human Minimum

  • 1. Schemas start as Zod; packages/spec holds no runtime logic. (not touched)
  • 2. Every name I added uses the casing for its kind.
  • 3. No formatter ran; pnpm lint passes (CI).
  • 4. One issue, one branch off an up-to-date main, one pull request.
  • 5. Nothing publishes; skip-changeset.
  • 6. packages/spec not touched.
  • 7. No hand edits to content/docs/references/, packages/*/CHANGELOG.md or content/docs/releases/.
  • 8. Title, body and comments are in English (the maintainer brief is Chinese by convention).
  • The Before Submitting list in CONTRIBUTING.md is done.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ADHkmTHHuvZqTM7XKWfv8Q


Generated by Claude Code

…oke in the nightly live tier

Spec Coverage (coverage-nightly.yml) and Engine Split Metric
(engine-split-metric.yml) are deleted: nothing read their artifact or run
summary. The engine split ratio script stays in scripts/ and runs on demand;
its self-test stays wired in lint.yml.

Showcase Smoke (showcase-smoke.yml) is retired as a workflow. Its Playwright
suite now runs from examples/app-showcase/test/showcase-smoke.live.test.ts in
the nightly live tier, so a red night files a card through the Nightly Tiers
report job. The showcase vitest config reads OS_TEST_TIERS through
scripts/nightly-tiers.mjs. Nightly Tiers builds the Console SPA and installs
Chromium on the shard that carries the showcase, and runs its file-level
slices before the whole-package leg so a red smoke cannot stop the CLI slice.

Claude-Session: https://claude.ai/code/session_01ADHkmTHHuvZqTM7XKWfv8Q
Co-authored-by: Claude <noreply@anthropic.com>
…ker env and reports run-level errors first

Claude-Session: https://claude.ai/code/session_01ADHkmTHHuvZqTM7XKWfv8Q
Co-authored-by: Claude <noreply@anthropic.com>
…the retired coverage workflow

Claude-Session: https://claude.ai/code/session_01ADHkmTHHuvZqTM7XKWfv8Q
Co-authored-by: Claude <noreply@anthropic.com>
…lit invocation its workflow took with it

Claude-Session: https://claude.ai/code/session_01ADHkmTHHuvZqTM7XKWfv8Q
Co-authored-by: Claude <noreply@anthropic.com>
@os-zhuang
os-zhuang marked this pull request as ready for review October 27, 2026 07:23
@os-zhuang
os-zhuang enabled auto-merge October 27, 2026 07:23
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

席位核对 · ACCEPT · devx seat 2 · 2026-10-27T07:22Z

Seat domain:devx#2 (#20163) · session_01Q4MDXRbEJCNdPAP4SiXwp9 · reviewed head 2094edeb · card #26445 (claim 6434447902, File surface amended to this PR's 15 files), ruling B. Thread-read: none.

  • What it does: it retires coverage-nightly.yml and engine-split-metric.yml, which no one read. It also moves showcase-smoke.yml into test-nightly-tiers.yml, so the showcase browser smoke runs as a nightly-tier file of @objectstack/example-showcase. vitest.config.ts includes it under the nightly tier and excludes it under the queue tier (isNightlyTierFile / readTierMode from scripts/nightly-tiers.mjs). turbo.json adds those two files to showcase#test inputs, so a tier change busts the cache.
  • The new prepare step: pnpm console:build and playwright install --with-deps chromium run only on the shard holding the showcase, and the step is continue-on-error. If it fails, the smoke itself reds and the diagnostics upload (showcase-smoke-diagnostics-<shard>) carries the trace.
  • First runner reading: Nightly Tiers run 40462712355 on this head, shard 1/2, job 122050818700. The showcase leg ran the live smoke: Playwright reported 53 passed (1.1m), including showcase-smoke.spec.ts "surface renders cleanly" across every surface, under a vitest wrapper of 1 file / 1 test. Both shards and the verdict job are success, and every step ran.
  • Gate edits follow the deletions: check-stall-guard-budget battery 10 floor 13 → 12, the engine-split literal case dropped from dispatch-gates.self-test, check-engine-split-ratio.mjs no longer names the deleted workflow, platform-core.json source path, and two comment pointers in half-state-patrol.yml / shard-timings-refresh.yml.
  • CI: 72 check-runs on the head, 61 success and 11 skipped, none red.

Landing: this PR edits .github/workflows/**, so it is Tier H. It stays a draft with auto-merge off until a GOVERNED_APPROVERS approval. +323/−201, under the 3000-line human-merge threshold.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd size/l skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

3 participants