Skip to content

[PMAT-254] README derived + enforced by pv contract and SHACL shape - #256

Merged
noahgift merged 2 commits into
mainfrom
fix/readme-shacl-pv
Sep 25, 2026
Merged

noahgift merged 2 commits into
mainfrom
fix/readme-shacl-pv

Conversation

@noahgift

@noahgift noahgift commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Refs #254. Operator: "fix README.md via SHACL/pv enforce it is shambolic".

What changed

  • README.md rewritten (117 lines). Every version, MSRV, example, command and link in it is extracted into contracts/ruchy-readme-claims.json. The following wrong claims are removed:
    • ruchy wasm compile|run: these subcommands do not exist.
    • cargo install ruchy was presented as installing 5.x; it installs 4.2.1.
    • The hand-typed 16,102 test count.
    • An async/http example that did not run.
    • MCP described as if on by default; it needs --features mcp.
  • Examples now live in examples/readme/*.ruchy, each with a .expected output file. The README fences are byte-identical to those files.
  • contracts/ruchy-readme-v1.yaml: pv contract with an ONT-001 SHACL shape (closed; pattern/in/min/max) over the claims JSON. pv validate passes with 0 errors and 0 warnings. pv lint contracts --gate shapes returns Pass with 0 violations.
  • src/readme_contract.rs (lib, 7 tests). This is the half that runs in sovereign-ci. It checks:
    • the claims JSON is fresh against the README;
    • version and MSRV equal Cargo's values;
    • each example matches its file and evaluates;
    • links resolve;
    • claims conform to the same YAML shape, graded in-process;
    • the README has no hand-typed counts;
    • a positive control: the shape rejects planted claims.
  • tests/readme_contract.rs (binary, 3 tests) runs against the ruchy built from HEAD:
    • each example's stdout equals its .expected;
    • every Quick start line exits 0, and the compiled binary prints the expected output;
    • every Commands-table entry is listed in ruchy --help, and <cmd> --help succeeds;
    • bare ruchy starts the REPL.
  • CI: a new readme-contract job. gate now needs both ci and readme-contract.
  • rust-version.workspace = true on the ruchy package. Without it CARGO_PKG_RUST_VERSION was empty, so the MSRV claim had no source to check against.

Defects found (filed, not hidden)

Mutation proof (restore trapped first)

10 plants (the commit message for a14f517 says 11; it is 10). Exit codes: 0 green, non-zero RED, - means that gate was not run for the plant. Script and table: docs/audits/readme-mutation-PMAT-254.{sh,txt}.

M0   baseline (no plant)                          lib=0 bin=0 pv=0
M1a  wrong MSRV number, claims NOT re-blessed     lib=101 bin=- pv=-
M1b  wrong MSRV number, claims re-blessed         lib=101 bin=- pv=0
M1c  hand-typed test count                        lib=101 bin=- pv=-
M2a  broken example path in Quick start           lib=0 bin=101 pv=-
M2b  broken example path on an Examples block     lib=101 bin=- pv=-
M3   example output drift (.expected)             lib=- bin=101 pv=-
M4   example no longer runs (README+file)         lib=101 bin=101 pv=-
M5   required section removed                     lib=101 bin=- pv=1
M6   Commands table names a missing subcommand    lib=0 bin=101 pv=0
M7   dead relative link                           lib=101 bin=- pv=0
M8   baseline after restore                       lib=0 bin=0 pv=0

The first run left M4 green on the lib gate. The cause: Interpreter::eval_string parses only the first expression. a14f517 changed the gate to parse the whole file, and M4 is now RED on both gates. M2a and M6 are caught only by the binary gate, and the gate job requires readme-contract.

Quorum

  • Round 1 on d738779: claude-sonnet-5, agy gemini-3.1-pro-high and claude-haiku-4-5 all PASS (docs/audits/quorum-PMAT-254.json).
  • Delta round on a14f517: 3/3 PASS.
  • The author is claude-opus-5-5 and sat on neither round. Two of the three lanes are Claude-family, so the rounds are marked degraded: same-family.

🤖 Generated with Claude Code

…shape (Refs #254)

- README.md rewritten short; examples are files under examples/readme/ with
  .expected output; version and MSRV derive from Cargo.toml.
- Removed wrong claims: `ruchy wasm compile`/`wasm run` (no such commands),
  hand-typed 16,102-test count and badge, the async/http example that does not
  run, `cargo install ruchy` as the way to get 5.0.0-beta.2.
- contracts/ruchy-readme-v1.yaml: pv contract (pv validate clean) whose SHACL
  shape grades contracts/ruchy-readme-claims.json (pv lint --gate shapes: Pass).
- src/readme_contract.rs (lib test, runs in CI): claims fresh, version/MSRV,
  examples == files and evaluate, links resolve, shape graded in-process,
  no hand-typed counts, positive control.
- tests/readme_contract.rs + readme-contract CI job (gate needs it): examples'
  stdout == .expected, Quick start commands exit 0, compiled binary output,
  Commands-table entries exist in `ruchy --help`.
- ruchy package inherits rust-version (only [workspace.package] declared it).

Closes: PMAT-254

Pmat-Ticket: PMAT-254
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): three PASS — agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-254",
 "head": "d7387791ad0e2c8834e37e1e0d0035bf92f103fd",
 "width": 1,
 "executor": "agy",
 "agreed": true,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "PASS",
   "findings": 0
  }
 ]
}

…oof (Refs #254)

- src/readme_contract.rs: evaluate() parses the whole file with
  Parser::parse. Interpreter::eval_string uses parse_expr, which stops
  after the first expression, so plant M4 (a syntax error on the last
  line of an example) stayed green on the lib gate. It is now RED on
  both gates.
- docs/audits/readme-mutation-PMAT-254.{sh,txt}: 11 plants each turn a
  gate RED; baseline M0 and M8 (after the trapped restore) are green.
- docs/audits/quorum-PMAT-254.json: sonnet-5 + agy gemini-3.1-pro-high
  + haiku-4-5 all PASS on d738779.

Pmat-Ticket: PMAT-254
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

Delta quorum on a14f517 (diff d738779..a14f517): claude-sonnet-5 PASS, agy gemini-3.1-pro-high PASS (model measured from the agy log), claude-haiku-4-5 PASS. Sonnet noted that the commit message says 11 plants when there are 10. That is corrected in the PR body. Author claude-opus-5-5 is excluded. Marked degraded: same-family.

@noahgift
noahgift enabled auto-merge (squash) September 25, 2026 12:06
@noahgift
noahgift merged commit f9ff174 into main Sep 25, 2026
10 checks passed
@noahgift
noahgift deleted the fix/readme-shacl-pv branch September 25, 2026 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant