Skip to content

ci(#4472): docs-only PRs skip the Rust-only fat sections; required checks still report - #4717

Closed
noahgift wants to merge 3 commits into
mainfrom
ci/4472-docs-only-segmentation
Closed

noahgift wants to merge 3 commits into
mainfrom
ci/4472-docs-only-segmentation

Conversation

@noahgift

@noahgift noahgift commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Refs #4472. This PR does not close the issue; the live docs-only proof lands after it.

What changes

A new change-class job runs scripts/ci_change_class.sh once per run, after the classifier's own case table (--self-test, 14 rows) passes.

The docs class requires every path in the diff to be:

  • added or modified (A or M), and
  • a *.md file, and
  • outside .github/, and
  • not compiled into any crate. A file counts as compiled in when a .rs file names it in include_str!, include! or include_bytes! (resolved against that file's own directory, with multi-line includes handled), or a build.rs names it. An env!/concat! literal is matched by basename.

Everything else is full. That covers any other diff, any non-PR event, and a classifier that fails. Every consumer tests == 'docs', so a failure always runs the full set.

class=docs full (unchanged)
x86-main vendored-schemas, guard-tree, guard-cargo, pr-review-shadow, pr-review-sign all sections
determinism (ARM64) skipped runs
mac-check skipped runs
workspace-test shards + Σ unchanged: they run, and their tier still runs the tree readers (readme_contract …) unchanged
ci / gate runs, and requires x86-main plus guard-tree and guard-cargo = success. Prints sov.gate: not-triggered: docs-only change (<reason>) unchanged sov.gate rule
gate runs, and requires WT, guard-tree and guard-cargo = success. Prints sov.gate and determinism-compare as not-triggered: docs-only change, by name unchanged

All three required checks (ci / gate, workspace-test, gate) still run and still report on every PR. A section is absent only when the classifier said docs, and the verdict log names it.

Why the skipped sections can be skipped. sov.* (test, lint, coverage, bench, security, provenance), mutants, determinism and provable-ladder all compile or run Rust. A Markdown file that no crate compiles in cannot change their result. The include rule exists because crates/aprender-review-experiment/src/prereg.rs include_str!s and sha-pins docs/specifications/review-experiment-protocol.md and docs/audits/rex-001/analysis-plan.md. Those two files classify full on the real tree.

Red/green proof

  1. Classifier case table: 14/14. A mutation (removing the no-match fix for build.rs) turns it red at 2/14.

  2. Merge-ref emulation on this commit (--event pull_request, with HEAD^1..HEAD as on a PR's merge ref):

    • README.md + book/src/introduction.md → class=docs
    • README.md + src/lib.rs → class=full (src/lib.rs: not a Markdown file)
  3. Verdict-branch guard, wired into guard-tree-steps: scripts/check_ci_gate_docs_class_rule.sh runs the real gate and ci / gate step scripts from ci.yml over 18 rows. Docs is accepted only from a change-class job that succeeded.

    • docs with guard-tree failed → RED
    • docs with workspace-test failed → RED
    • docs with empty results → RED
    • full with docs-shaped results → RED
    • empty class (the classifier failed) → RED
    • a docs output from a failed change-class job → RED
    • docs with everything ok → GREEN
    • full with everything ok → GREEN

    Its --self-test plants three wrong rules, and each one turns the table RED:

    • trusting the output without checking the job result;
    • excusing every section on a docs run;
    • ci / gate trusting x86-main's job result alone.
  4. This PR's own run is the docs+code arm. It touches ci.yml and a .sh, so it must classify full and run every section.

  5. Live docs-only arm: after merge, a docs-only probe PR's run IDs will be posted on CI has no smart segmentation for docs-only PRs — full fat job + mandatory pr-review quorum run regardless of change type #4472. ci.yml only triggers for PRs against main, so it cannot run before then.

Guards

All of these are green:

  • guard_tree_job_test 8/8
  • check_ci_gate_mutants_{rule,table_rule,cuda_rule}
  • check_receipt_gate_base_owned
  • check_workflow_env_defined
  • check_ci_fat_secrets_plumbed
  • check_guards_are_wired
  • check_no_hosted_runners
  • check_guard_steps_isolated
  • ci_gpu_touched_test 27/27
  • fat_driver self-test 60/60
  • actionlint (no new findings)

guard_tree_job_test.sh row 5 ("x86-main dropped from gate.needs") was already RED on main. Its sed stopped matching once gate.needs grew past determinism], so the mutant was never applied. The anchor is fixed here.

Notes

Agent: aprender-w4472

🤖 Generated with Claude Code

…ecks still report

A new `change-class` job runs scripts/ci_change_class.sh once. class=docs means
every path is an added/modified *.md outside .github/ that no Rust source
compiles in (include_str!/include!/build.rs, resolved per file; env!/concat!
literals matched by basename). Everything else, any non-PR event, and a failed
decision mean the full set: every consumer tests == 'docs'.

class=docs: x86-main runs only the doc-content sections (vendored-schemas,
guard-tree, guard-cargo, pr-review-shadow, pr-review-sign); determinism and
mac-check skip; workspace-test is unchanged. `gate` and `ci / gate` still run
and print sov.gate / determinism-compare as "not-triggered: docs-only (<reason>)"
by name; every other required section must still succeed.

Also fixes guard_tree_job_test.sh row 5: its sed stopped matching gate.needs
once that list grew past `determinism]`, so the mutant was never applied (RED
on main too).

Refs #4472
Agent: aprender-w4472

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…gets a mutation-tested guard

`gate` and `ci / gate` now excuse sov.gate / determinism-compare only when
needs.change-class.result == success AND class == docs; a docs output from a
failed classifier job judges every section as before.

scripts/check_ci_gate_docs_class_rule.sh extracts both verdict steps from ci.yml
and executes them over an 18-row table (docs passes only with guard-tree,
guard-cargo, workspace-test / x86-main green; empty, failed or full class never
excuses a section). --self-test plants three wrong rules (trust the output
without the job result, excuse every section on docs, ci / gate trusting
x86-main alone): each turns the table RED; a workflow with no verdict steps is
ENV rc=2. Wired into guard-tree-steps beside the mutants-rule checker.

Adopted from the review of #4639 (its changes-job SUCCEEDED requirement and its
executed gate-block checker), in #4717's own terms; no commit of #4639 is moved.

Refs #4472
Agent: aprender-w4472

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noahgift

noahgift commented Oct 4, 2026

Copy link
Copy Markdown
Contributor Author

Auto-merge disarmed by the cop: this changes what ci / gate accepts (docs-only skip), and a gate change waits for the operator's sign-off (C297). It is a row in the 0.70.2 sign-off batch. It re-arms on a signed yes.

@noahgift

noahgift commented Oct 4, 2026

Copy link
Copy Markdown
Contributor Author

OPERATOR, verbatim: "yes, sign off #4717". Gate change signed off; auto-merge re-armed by the cop.

@noahgift
noahgift enabled auto-merge October 4, 2026 09:42
@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=4717 head=d7a6b3caadfef35a145e4bc1d55b1be9804c7d52 verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

… now run over a real history

The independent review of 78551b0 blocked on B3: five planted mutants of the
two guards survived, and so did one more found while fixing them.

- ci / gate dropping guard-cargo from the sections a docs run still needs:
  a ci/gate row with guard-cargo missing, and a `cigc` self-test mutant.
- gate ending the whole check at the first excused section instead of skipping
  it: the excused pairs now come first in the loop, so an `exit` there would skip
  the guards and the case table sees it; an `exit` self-test mutant plants it.
- ci_change_class.sh's pull_request and merge_group paths were never executed
  by the self-test. Six rows now build a real history (main gains Rust, the PR
  adds a doc and renames one) and check: the PR alone on pull_request, the
  --base range on merge_group (main's Rust in and out of range), a rename read
  as a delete on both events, and merge_group without --base as ENV.

Dropping the BASE guard is an equivalent mutant: an empty BASE gives an empty
diff, which is already ENV. bashrs findings on both scripts equal the base.

Refs #4472
Agent: aprender-w4472

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noahgift
noahgift disabled auto-merge October 4, 2026 16:32
noahgift added a commit that referenced this pull request Oct 4, 2026
…ry step

Refs #4678

Round 3 left 13 mutants alive. Each edited a line that no row reads: a
`|| true` or a dropped flag on the tree nextest, the docker SHARD env, the quick
Σ shard test, the upload path and if-no-files-found, the keep step's target, the
staging TIER, a continue-on-error on the quick Σ step or the upload, and the
fan-in's name check. The job-level keys (if, needs, env, continue-on-error,
matrix: the #4717 m18-m21 class) were outside every step.

scripts/check_quick_tier_shard_sigma.sh now extracts four job blocks with awk,
each scoped to its top-level parent:
- ci.yml jobs.workspace-test-shard and jobs.workspace-test;
- sections.yml matrix-pins.workspace-test-shard and jobs.workspace-test-shard.

It compares their sha256 against ci/goldens/quick-tier-shard-sigma.sha256. Any
change is RED until --update-golden re-pins it in the same diff. A missing job
or golden is ENV rc=2. Table row 1 is the golden, so the table now has 26 rows.

--self-test, run against this tree:
- the 24 planted rules from rounds 1-3 are RED under a golden re-pinned to each
  mutant, so the semantic rows still kill them on their own;
- 19 golden mutants are RED against the stale golden, each by the golden row:
  13 round-3 survivors and 6 job-level (workspace-test if/needs/env, shard job
  continue-on-error, matrix [1, 2], a matrix-pins entry deleted);
- the unchanged tree is GREEN; a pinned edit with a stale golden is RED; the
  same edit with the golden re-pinned is GREEN; an edit outside the pinned
  jobs is GREEN; a missing golden is rc=2.

The tools are awk, sha256sum, comm and sort, all already on the gate path.
bashrs reports 0 errors. Wiring is unchanged: guard-tree runs the table and
--self-test report-only (L31).

Agent: aprender-w4472
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noahgift

noahgift commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

Superseded to clear the open-PR queue; the branch is kept, and aprender-w4472 reopens this when main's T43 change-class table allows the docs-only rule.

@noahgift noahgift closed this Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant