Repository navigation
Conversation
Both example workflows still pinned talongate/move-test-gen@v1.6.0's SHA while package.json and README.md had already moved to v1.6.1. The identical drift was fixed by hand one release earlier and came straight back, because nothing checks it. Adds scripts/check-pins.mjs: reads package.json's version, resolves the matching git tag to a commit, derives every reference to the action from git ls-files (never a hard-coded file list, which is exactly what let this drift ship twice), and classifies each one -- a SHA-pinned reference must match the release commit (and its trailing # vX.Y.Z comment, if present, must agree too); a tag-pinned reference must match the release version; anything else is reported as unpinned/unrecognized. Prints one line per reference and a summary, exits nonzero on any mismatch. Fails loud if the release tag itself does not resolve, rather than passing or skipping silently. Re-pins both example workflows to the current release (f2efe33, v1.6.1). README.md was already correct and is untouched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Checks out full history (fetch-depth: 0) so the release tag actually resolves, runs scripts/check-pins.mjs, then a negative self-test: corrupt one pin with sed, assert the checker exits nonzero and names the mismatch, restore the file. Without the negative step this workflow is green forever with nothing proving it can still go red. No eval/gate-selftest/ case for this -- that suite pins the scorer's own behaviour; this is repo-hygiene CI, the same class the checks already inside action-selftest.yml carry no case for either. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Review of the pin checker found it could be green while blind, and red for the wrong reason during the exact release window it exists to police. Six fixes, each changes what the checker CALLS a failure, so each is verified in both directions: the shape it now catches, and a shape that must stay unaffected. Zero references found used to exit 0 -- a vacuous pass, and this repository has already lost this exact scan once before, by address change (mehvetero/move-test-gen -> talongate/move-test-gen, PR talongate#61). Now exits 3 ("no verdict was reached"), named as such in the file header alongside the cost: a legitimately reference-free tree also goes red, accepted because README.md always carries one reference. A broken precondition -- package.json declaring a version with no matching tag yet, which RELEASE.md's own ordering makes a real state between "bump the version" and "cut the tag" -- used to crash with a raw Node stack trace at exit 1, the code this repo's own exit-code table reserves for "a pin drifted." Wrapped main() now prints one human line and exits 2, matching the same contract scripts/lint.mjs already carries a comment about having fixed once. This also closes the release-window false-red: the message now routes the reader correctly instead of reading as a second, unrelated drift. The reference regex's `\S+` ran past the reference into ordinary prose punctuation -- a sentence-ending period, a comma, a closing paren, a backtick or bold-close, none of which are part of a real SHA or tag -- turning correct documentation into a reported mismatch. Stripped before classifying. Also: the capture now stops at `#` as well as whitespace (closing a same-line-no-space trailing-comment false positive), a SemVer pre-release/build suffix is recognized as a tag SHAPE so such a reference gets a precise "does not match" message instead of falling into the unrecognized bucket, and an optional `/<subpath>` is accepted between the repo name and `@` so a reusable action living in a subdirectory of this repo is no longer invisible to the scan. One self-caught defect while proving the SemVer widening in both directions: comparing only the captured X.Y.Z core would have treated `v1.6.1-beta.1` as equal to `v1.6.1` -- a different git ref that can point at a different commit. Fixed to compare the whole token against the release tag; the pre-release SHAPE is still recognized for a precise mismatch message, never treated as equivalent to the release. The trailing `# vX.Y.Z` comment was checked only inside the SHA branch; a stale comment beside a genuinely correct tag reference passed silently. Hoisted above the SHA/tag split so both branches see it. The CI workflow's negative self-test proves ONE reference is still seen after a corruption; it never proved ALL of them still are. A regression that narrowed the scan from three references to one would have passed both the green run and the negative step. Added a glob-derived assertion (`examples/workflows/*.yml`, never a hard-coded file count) that every known example workflow is named in the checker's own output. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…assertion firing on compliant examples Follow-up to the previous commit. Each item below undoes value an earlier change in this branch had bought, so each was verified in both directions. The workflow's first step runs under `bash -e`, so a nonzero exit from check-pins aborted the step before the checker's own output was printed: the exit 2 and exit 3 diagnostics never reached the job log. The step now uses the same `set +e ... rc=$? ... set -e` shape as the negative step below it. The "every example workflow is named in the output" assertion looped over every file in examples/workflows/, so a legitimate example with no reference to the action (one that calls the standalone CLI through npx, for instance) failed CI with a message that was false: the scan had not narrowed. The loop now covers only example workflows whose `uses:` line points at this action, with the address built from two halves as the checker does. The cost is stated in a comment: an example that loses its `uses:` line for the action is no longer caught there; that is a broken example, not a pin drift. The exit 2 message asserted one cause for a catch-all failure, and the stdio change that removed git's stray stderr also removed the evidence that used to contradict it (outside a git repo the old text still said the tag did not exist yet). The explanation is now phrased as the usual causes, and git's own first stderr line is appended, so the message stays one line and carries both. The `#` stop added to the reference pattern made `...@# v1.6.1` (an empty ref) match nothing, so a line the previous version flagged became invisible. The ref capture may now be empty and is reported as unrecognized. Cost: a bare address followed by `@` and nothing else, in prose, is now flagged too. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
examples/workflows/pr-gate.yml:14andexamples/workflows/nightly-mutation.yml:27were still pinned to3afcb1a731e058835c622475b6203b42a4b52d84 # v1.6.0— a full release behindpackage.json's1.6.1(tagv1.6.1=f2efe3338348e5bd2238a752bdf00b1c3800dfd8).README.md's own CI snippet was already correct.Same two files, third time:
7daae46aligned them once ("fixes #4"),babc3fa(PR #84) aligned them again two releases later ("two releases behind"), and now this — neither fix was re-checked. This one ships with a check for the class instead of a fourth manual re-pin.What it checks — version agreement, not SHA-pinning
Reads
package.json'sversion, resolves it to the matching tag's commit, then derives everytalongate/move-test-gen[/<subpath>]@<ref>reference in the tracked tree fromgit ls-files— never a hard-coded file list. Each must agree with that version: a 40-hex SHA must equal the release SHA; avX.Y.Ztag (SemVer pre-release/build suffix included) must equal the release tag exactly; a trailing# vX.Y.Zcomment, if present, is checked against both forms. A branch,@main, a short SHA, or a version that doesn't match — fails. Trailing prose punctuation (.,)backticks) is stripped before classifying, which also forgives a malformeduses: …@v1.6.1)— an accepted trade against flagging correct docs.It does not enforce pinning.
README.md:247's own@v1.6.1is a mutable tag, reportedOKtoday — the checker has no opinion on SHA-vs-tag, only on whether a reference names the current release. That tag goes red at the next bump, which is the real value here.Exit codes match
README.md:156–170's own contract0clean ·1a reference disagrees ·2the release tag doesn't resolve ·3nothing found to check (empty isn't a pass here).1and2print differently on purpose.RELEASE.mdbumps the version before cutting the tag, so a window exists where the tag doesn't resolve. The check exits 2 there withcheck-pins: cannot determine the release to check against -- …plus git's own first stderr line (a tag-less checkout fails the same way), not a crash. We considered comparing against the newest existing tag to close that window, and rejected it — it would drop the package.json-vs-tag agreement check, which is half of what caught this drift.Red-first, before either pin was touched
EXIT=1. Both files re-pinned in this branch; the checker now reports3/3 references OK, exit0..github/workflows/pin-sync.ymlTwo steps. First runs the checker (its own message prints before the step fails, so a red run shows why) and asserts every example workflow that references the action is named in its output (a narrowed scan would otherwise still exit
0). Second corrupts a pinned SHA inpr-gate.yml, asserts the checker exits nonzero withMISMATCH, then restores the file.fetch-depth: 0, since tag resolution needs tags a shallow checkout doesn't have.Why no
gate-selftestcaseThat suite pins the test-generation scorer, not repo hygiene.
gate-selftest: 24/24 green, 1 skipped for nosuiCLI.One question, not a decision
package.json'sfilesallowlist publishesscripts/, so this ships to npm consumers, where it's meaningless — no tag, no matching references, in a stranger's tree. Your call whether a repo-hygiene script belongs there; happy to exclude it.