Skip to content

ci(skillsaw): add lint job, Makefile targets, and repo config - #754

Open
ralphbean wants to merge 6 commits into
skillsaw/fix-pr-reviewfrom
skillsaw/onboard-ci
Open

ci(skillsaw): add lint job, Makefile targets, and repo config#754
ralphbean wants to merge 6 commits into
skillsaw/fix-pr-reviewfrom
skillsaw/onboard-ci

Conversation

@ralphbean

Copy link
Copy Markdown
Member

Adds a skillsaw job to the lint workflow, running the linter in
--strict mode on every PR and push to main. Adds make lint and
make lint-fix targets so contributors can run the same checks
locally. Adds .skillsaw.yaml, tuned for this repo:

  • Disables marketplace-json-valid and marketplace-registration.
    plugins/gopls-lsp is an internal fullsend harness plugin config
    (see harness/code.yaml), not a Claude Code marketplace plugin, so
    these rules produced false positives.
  • Raises the context-budget skill thresholds to warn 16000 / error
    20000, above code-implementation and pr-review's current sizes.
    Both are dense, production-tuned orchestration procedures; --strict
    turns warnings into failures, so the thresholds are set high enough
    that neither file triggers a violation at either severity.

This builds on the skill content fixes in the preceding PRs in this
stack, evaluating skillsaw per #188.

Related to #188

Assisted-by: Claude Opus 4.6 noreply@anthropic.com
Signed-off-by: Ralph Bean rbean@redhat.com


Stack created with GitHub Stacks CLIGive Feedback 💬

@ralphbean
ralphbean requested a review from a team as a code owner August 11, 2026 16:59
@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

CI: run skillsaw lint in strict mode and add local Make targets

⚙️ Configuration changes ✨ Enhancement 🕐 20-40 Minutes

Grey Divider

AI Description

• Add a skillsaw lint job to GitHub Actions and fail builds on warnings via strict mode.
• Introduce repo-specific .skillsaw.yaml to avoid false positives and tune context limits.
• Add make lint and make lint-fix to run the same checks locally.
Diagram

graph TD
  A["GitHub Actions: lint.yml"] --> B["skillsaw action (--strict)"] --> C["Repo files (skills/agents/etc)"]
  B --> D[".skillsaw.yaml"]
  E["Makefile targets"] --> F["uvx skillsaw==0.18.0"] --> C
  F --> D
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Run skillsaw CLI in CI (uvx) instead of GitHub Action
  • ➕ Single execution path between local make lint and CI (same packaging/runtime)
  • ➕ Easier to pin and audit the exact skillsaw version used end-to-end
  • ➖ Requires installing uv/uvx in CI and managing caching
  • ➖ More scripting/maintenance in the workflow compared to a dedicated action
2. Start non-strict in CI, then flip to strict after baseline
  • ➕ Avoids PR failures from newly surfaced warnings during rollout
  • ➕ Can progressively tighten rules/thresholds as content evolves
  • ➖ Delays enforcement of the intended quality gate
  • ➖ Requires follow-up work to ratchet strictness

Recommendation: The PR’s approach (dedicated skillsaw action with strict: true, plus local Make targets) is a good balance of simplicity and enforcement. The main thing to keep an eye on is version drift: local lint pins skillsaw==0.18.0 while the action is pinned by commit but configured separately; if mismatches become an issue, consider converging on one execution path (either all-action or all-CLI).

Files changed (3) +358 / -1

Enhancement (1) +11 / -1
MakefileAdd 'make lint' and 'make lint-fix' using pinned skillsaw version +11/-1

Add 'make lint' and 'make lint-fix' using pinned skillsaw version

• Adds 'lint' and 'lint-fix' phony targets and documents them in the help output. Uses 'uvx skillsaw==0.18.0' to run strict linting and apply automated fixes locally.

Makefile

Other (2) +347 / -0
lint.ymlAdd strict skillsaw lint job to CI workflow +10/-0

Add strict skillsaw lint job to CI workflow

• Introduces a new 'skillsaw' job in the lint workflow. The job checks out the repo and runs the 'stbenjam/skillsaw' action with strict mode enabled, causing warnings to fail CI.

.github/workflows/lint.yml

.skillsaw.yamlAdd repo-specific skillsaw rules and thresholds +337/-0

Add repo-specific skillsaw rules and thresholds

• Adds a full skillsaw configuration file pinned to version 0.18.0. Disables marketplace-related rules that are false positives for this repo and raises the 'context-budget' thresholds for dense skill files while keeping CI strictness enforced via the workflow.

.skillsaw.yaml

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 11, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 5:01 PM UTC · Completed 5:17 PM UTC

Commit: 8ef7855 · View workflow run →

@qodo-code-review

qodo-code-review Bot commented Aug 11, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Workflow lint.yml modified ✗ Dismissed 📜 Skill insight § Compliance
Description
This PR modifies a protected governance/infrastructure path (.github/workflows/lint.yml), which
must not be auto-approved and requires explicit human review/approval controls.
Code

.github/workflows/lint.yml[R14-17]

+  skillsaw:
+    runs-on: ubuntu-24.04
+    steps:
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
Relevance

●●● Strong

Changes under .github/workflows are routinely reviewed; governance finding will be kept to force
human approval.

PR-#37

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The checklist requires raising a finding whenever protected governance/infrastructure paths
(including .github/) are modified. The diff adds a new skillsaw job in
.github/workflows/lint.yml, triggering the protected-path rule.

.github/workflows/lint.yml[13-23]
Skill: pr-review

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Protected-path changes (under `.github/`) require explicit human review and should carry clear justification close to the change.

## Issue Context
This PR adds a new CI job to `.github/workflows/lint.yml`, which is a protected governance/infrastructure path.

## Fix Focus Areas
- .github/workflows/lint.yml[14-23]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Strict mode mismatch ✓ Resolved 🐞 Bug ⚙ Maintainability
Description
.skillsaw.yaml documents “Treat warnings as errors” but sets strict: false, while CI and `make
lint` run skillsaw in strict mode. This makes the checked-in config misleading and allows local
invocations (without the Makefile/CI wrapper) to produce different results than CI.
Code

.skillsaw.yaml[R336-337]

+# Treat warnings as errors
+strict: false
Relevance

●●● Strong

Doc/config mismatch vs actual strict CI behavior; similar docs-implementation mismatch fixes were
accepted.

PR-#567

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The config contradicts itself and differs from how skillsaw is invoked in both CI and the Makefile,
which is directly observable from the repo contents.

.skillsaw.yaml[336-337]
.github/workflows/lint.yml[20-22]
Makefile[20-24]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`.skillsaw.yaml` says it will treat warnings as errors, but it sets `strict: false` while CI/Makefile run skillsaw with strict enabled. This is misleading and can cause local runs (when developers run `skillsaw` directly) to diverge from CI.

## Issue Context
- CI runs skillsaw with strict mode enabled.
- The Makefile `lint` target runs skillsaw with `--strict`.
- The config file claims strict behavior but disables it.

## Fix Focus Areas
- .skillsaw.yaml[336-337]
- .github/workflows/lint.yml[20-22]
- Makefile[20-24]

## Suggested fix
Choose one of the following and make everything consistent:
1) **Config-driven strict**: set `.skillsaw.yaml` to `strict: true` and remove `--strict` / action `strict: true` overrides (or keep overrides but update comments accordingly).
2) **Caller-driven strict (current behavior)**: keep `strict: false` in `.skillsaw.yaml`, but update the comment to reflect that strictness is enabled by CI/Makefile (not by the config file itself).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. .skillsaw.yaml indentation inconsistent ✓ Resolved 📜 Skill insight ⚙ Maintainability
Description
The new .skillsaw.yaml uses inconsistent indentation under exclude: compared to other YAML in
the same directory, increasing the chance of style drift and review friction.
Code

.skillsaw.yaml[R328-331]

+exclude:
+    # - "**/template/**"
+    # - "**/templates/**"
+    # - "**/_template/**"
Relevance

●●● Strong

Trivial YAML style consistency nit; team often accepts config-structure/style alignment suggestions.

PR-#573

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The checklist requires new/modified YAML config files to match the style of existing files in the
same directory. config.yaml demonstrates 2-space indentation, while .skillsaw.yaml shows 4-space
indentation under exclude:.

.skillsaw.yaml[326-332]
config.yaml[1-11]
Skill: code-implementation

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The `.skillsaw.yaml` file has inconsistent indentation under `exclude:` compared to existing root-level YAML style.

## Issue Context
Root-level YAML (e.g., `config.yaml`) uses 2-space indentation; `.skillsaw.yaml` uses 4 spaces for the commented list under `exclude:`.

## Fix Focus Areas
- .skillsaw.yaml[328-332]
- config.yaml[1-11]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context
✅ Compliance rules (platform): 55 rules
✅ Skills: 4 invoked
  code-review
  code-implementation
  pr-review
  docs-review

Grey Divider

Tip of the day
💡 Did you know, you can describe a rule in plain language on the Rules page and Qodo drafts it for you

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread .github/workflows/lint.yml
Comment thread .skillsaw.yaml Outdated
Comment thread .skillsaw.yaml Outdated
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 11, 2026

Copy link
Copy Markdown

Review

Findings

Medium

  • [protected-path] .github/workflows/lint.yml — Modifies a file under the protected path .github/. The PR links to issue Evaluate skillsaw for linting agent instructions #188 and provides rationale for the CI change. Human approval is always required for protected-path changes, regardless of context.

  • [PR body / config mismatch] .skillsaw.yaml:169 — The PR body states: "Raises the context-budget skill thresholds to warn 16000 / error 20000." However, the actual .skillsaw.yaml uses default thresholds (all sub-keys under limits: are commented out). The violations for pr-review (15,058 tokens) and code-implementation (9,116 tokens) are suppressed via the .skillsaw-baseline.json baseline file with ceiling mode, not by raised thresholds. If someone later removes the baseline file expecting the config to have raised thresholds, both skills will fail the lint check.
    Remediation: Either update the PR body to accurately describe the baseline suppression mechanism, or set the thresholds in .skillsaw.yaml as described.

Low

  • [consistency] .github/workflows/lint.yml:19 — The skillsaw job sets persist-credentials: false on its checkout step, while the test and commit-lint jobs in the same file omit this parameter. Since persist-credentials: false is a security hardening measure, the preferred resolution is to add it to the other two jobs in a follow-up PR rather than removing it here.
Previous run

Review

Findings

Medium

  • [protected-path] .github/workflows/lint.yml — Modifies a file under the protected path .github/. The PR links to issue Evaluate skillsaw for linting agent instructions #188 and provides rationale for the CI change. Human approval is always required for protected-path changes, regardless of context.

Low

  • [consistency] .github/workflows/lint.yml:19 — The skillsaw job sets persist-credentials: false on its checkout step, while the test and commit-lint jobs in the same file omit this parameter. Since persist-credentials: false is a security hardening measure, the preferred resolution is to add it to the other two jobs in a follow-up PR rather than removing it here.
Previous run (2)

Review

Findings

Medium

  • [protected-path] .github/workflows/lint.yml — Modifies a file under the protected path .github/. The PR links to issue Evaluate skillsaw for linting agent instructions #188 and provides rationale for the CI change. Human approval is always required for protected-path changes, regardless of context.

Low

  • [consistency] .github/workflows/lint.yml:19 — The skillsaw job sets persist-credentials: false on its checkout step, while the test and commit-lint jobs in the same file omit this parameter. Since persist-credentials: false is a security hardening measure, the preferred resolution is to add it to the other two jobs in a follow-up PR rather than removing it here.
Previous run (3)

Review

Findings

Medium

  • [protected-path] .github/workflows/lint.yml — Modifies a file under the protected path .github/. The PR links to issue Evaluate skillsaw for linting agent instructions #188 and provides rationale for the CI change. Human approval is always required for protected-path changes, regardless of context.

Low

  • [consistency] .github/workflows/lint.yml:19 — The skillsaw job sets persist-credentials: false on its checkout step, while the test and commit-lint jobs in the same file omit this parameter. Since persist-credentials: false is a security hardening measure, the preferred resolution is to add it to the other two jobs in a follow-up PR rather than removing it here.
Previous run (4)

Review

Findings

Medium

  • [protected-path] .github/workflows/lint.yml — Modifies a file under the protected path .github/. The PR links to issue Evaluate skillsaw for linting agent instructions #188 and provides rationale for the CI change. Human approval is always required for protected-path changes, regardless of context.

Low

  • [consistency] .github/workflows/lint.yml:19 — The skillsaw job sets persist-credentials: false on its checkout step, while the test and commit-lint jobs in the same file omit this parameter. Since persist-credentials: false is a security hardening measure, the preferred resolution is to add it to the other two jobs in a follow-up PR rather than removing it here.
Previous run (5)

Review

Findings

Medium

  • [protected-path] .github/workflows/lint.yml — Modifies a file under the protected path .github/. The PR links to issue Evaluate skillsaw for linting agent instructions #188 and provides rationale for the CI change. Human approval is always required for protected-path changes, regardless of context.

Low

  • [consistency] .github/workflows/lint.yml:19 — The skillsaw job sets persist-credentials: false on its checkout step, while the test and commit-lint jobs in the same file omit this parameter. Since persist-credentials: false is a security hardening measure, the preferred resolution is to add it to the other two jobs in a follow-up PR rather than removing it here.
Previous run (6)

Review

Findings

High

  • [api-contract] Makefile:21 — The lint and lint-fix targets use uvx skillsaw==$(SKILLSAW_VERSION), which expands to uvx skillsaw==0.18.0. This is not valid uvx syntax. The uvx command accepts version pinning via command@version (e.g., uvx skillsaw@0.18.0) or via --from 'package==version' command (e.g., uvx --from 'skillsaw==0.18.0' skillsaw). With the current syntax, uvx interprets skillsaw==0.18.0 as a literal package name, which does not exist — both make lint and make lint-fix will fail for local developer use.
    Remediation: Use uvx skillsaw@$(SKILLSAW_VERSION) --strict or uvx --from 'skillsaw==$(SKILLSAW_VERSION)' skillsaw --strict.

Medium

  • [protected-path] .github/workflows/lint.yml — Modifies a file under the protected path .github/. The PR links to issue Evaluate skillsaw for linting agent instructions #188 and provides rationale for the CI change. Human approval is always required for protected-path changes, regardless of context.

Low

  • [consistency] .github/workflows/lint.yml:19 — The skillsaw job sets persist-credentials: false on its checkout step, while the test and commit-lint jobs in the same file omit this parameter. Since persist-credentials: false is a security hardening measure, the preferred resolution is to add it to the other two jobs in a follow-up PR rather than removing it here.

  • [missing-doc-entry] README.md:91 — The Workflows table lists fullsend.yaml, release.yml, and script-test.yml but does not include lint.yml. Since this PR is explicitly documenting the linting infrastructure, it is a natural place to add the missing row.


Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR
Previous run (7)

Review

Findings

Medium

  • [protected-path] .github/workflows/lint.yml — Modifies a file under the protected path .github/. The PR links to issue Evaluate skillsaw for linting agent instructions #188 and provides rationale for the CI change. Human approval is always required for protected-path changes, regardless of context.

Low

  • [incomplete-tooling-configuration] .skillsaw.yaml — The skillsaw author (stbenjam) recommended disabling the content-critical-position rule in the issue Evaluate skillsaw for linting agent instructions #188 discussion. The .skillsaw.yaml configuration does not disable this rule, meaning CI will flag content-critical-position warnings in strict mode. Consider whether this rule should be disabled per the tool author's guidance.

  • [consistency] .github/workflows/lint.yml:19 — The skillsaw job sets persist-credentials: false on its checkout step, while the test and commit-lint jobs in the same file omit this parameter. This is an inconsistency, but persist-credentials: false is a security hardening measure. If consistency is desired, add it to the other jobs in a separate PR rather than removing it from the skillsaw job.

Previous run (8)

Review

Findings

Medium

  • [protected-path] .github/workflows/lint.yml — Modifies a file under the protected path .github/. The PR links to issue Evaluate skillsaw for linting agent instructions #188 and provides rationale for the change. Human approval is always required for protected-path changes, regardless of context.

  • [scope-creep-beyond-authorization] Issue Evaluate skillsaw for linting agent instructions #188 authorizes evaluating skillsaw ("Does it find meaningful issues?", "Would it be useful ongoing?") but this PR implements production CI enforcement (--strict mode, Makefile targets, repo config). The PR is part of a stack (base branch skillsaw/fix-pr-review) where evaluation may have occurred in earlier PRs, but the evidence is not linked here.
    Remediation: Provide evidence (comment on Evaluate skillsaw for linting agent instructions #188 or preceding PR) that the evaluation phase completed before CI integration.

  • [inconsistent-configuration] .skillsaw.yaml:337 — Sets strict: false while the CI workflow passes strict: true and the Makefile passes --strict. The CLI flag overrides the config, making it dead configuration that could confuse contributors.
    Remediation: Remove strict: false or add a comment explaining the intentional difference for local non-strict runs.

  • [missing-documentation] README.md:44 — The Testing section lists make test, make script-build, and make check-bundle but not the newly added make lint and make lint-fix targets.
    Remediation: Add the new lint targets to the README.

Low

  • [scope-naming-convention] PR title uses ci(skillsaw) — repo conventions use subsystem scopes (e.g., ci(lint)) rather than tool names.

  • [consistency] .github/workflows/lint.yml:19 — The skillsaw job uses persist-credentials: false while the other two jobs omit it. The inconsistency is noted, though the third-party action usage makes this the more secure choice.

  • [incomplete-guidance] CONTRIBUTING.md:38 — Instructs contributors to "fix any lint failures before pushing" without specifying the make lint command. Pre-existing gap now worth addressing with the new target available.

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Aug 11, 2026

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review sweep: 4 findings (2 HIGH, 2 MEDIUM) posted as inline comments.

Comment thread .skillsaw.yaml Outdated
Comment thread .github/workflows/lint.yml Outdated
Comment thread .github/workflows/lint.yml
Comment thread .skillsaw.yaml Outdated
@ralphbean

Copy link
Copy Markdown
Member Author

These three all landed as separate inline threads — handled them there (protected-path is intentional, indentation fixed, strict-mode comment reworded).

@ralphbean

Copy link
Copy Markdown
Member Author

Went through these:

Doc gaps — added make lint/make lint-fix to the README and CONTRIBUTING.

Scope vs #188 — I see the issue only asked for evaluation, but I'd count the PR itself as the evaluation: building the strict lint job out and watching it catch real issues in review (the ones fixed inline on this PR) is the signal that it finds meaningful problems and is worth running ongoing. Comfortable moving straight to CI integration rather than splitting that into a separate PR.

persist-credentials — answered on the dedicated thread for that one.

PR title — keeping ci(skillsaw) as-is. It's scoping to the tool being added rather than an existing subsystem, which felt clearer here than reaching for a subsystem name that doesn't exist yet.

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 13, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 1:16 PM UTC · Completed 1:32 PM UTC

Commit: b703e8c · View workflow run →

fullsend-ai-review[bot]

This comment was marked as outdated.

Comment thread .skillsaw.yaml Outdated

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review sweep: 2 findings (1 HIGH, 1 MEDIUM) posted as inline comments.

Comment thread .skillsaw.yaml Outdated
Comment thread .skillsaw.yaml Outdated
@ralphbean
ralphbean force-pushed the skillsaw/onboard-ci branch from b703e8c to c2ab7a5 Compare August 13, 2026 21:38
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 13, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 9:39 PM UTC · Completed 9:54 PM UTC

Commit: c2ab7a5 · View workflow run →

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot removed the requires-manual-review Review requires human judgment label Aug 13, 2026
@fullsend-ai-coder

fullsend-ai-coder Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

🤖 Finished Fix · ✅ Success · Started 9:55 PM UTC · Completed 10:01 PM UTC

Commit: c2ab7a5 · View workflow run →

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

🔧 Fix agent — iteration 1 (bot-triggered)

Addressed 2 of 3 review findings. Fixed invalid uvx syntax in Makefile (== to @) and added lint.yml to README Workflows table. Disagreed with 1 finding (persist-credentials consistency) per reviewer's own suggestion to handle in a follow-up PR.

Fixed (2):

  1. uvx version pinning uses invalid == syntax (Makefile): Changed uvx skillsaw==$(SKILLSAW_VERSION) to uvx skillsaw@$(SKILLSAW_VERSION) on both lint and lint-fix targets. The == syntax is pip-style version pinning not recognized by uvx; the @Version syntax is the correct uvx idiom.
  2. lint.yml missing from README Workflows table (README.md): Added a row for lint.yml to the Workflows table describing its purpose (pre-commit checks, commit-message linting, and skillsaw).

Disagreed (1):

  1. persist-credentials: false inconsistency across workflow jobs: The reviewer explicitly noted the preferred resolution is to add persist-credentials: false to the other two jobs in a follow-up PR rather than removing it here. No change needed in this PR.

Tests: passed

Next steps:

  • /fs-review — request a re-review of the changes
  • /fs-fix <your instruction> — run another fix pass with specific guidance
  • Push commits directly — review re-runs automatically on push
    Updated by fullsend fix agent

fullsend-ai-coder Bot added a commit that referenced this pull request Aug 13, 2026
Use uvx's @Version pinning syntax instead of pip-style ==version, which
uvx does not recognize as a valid command specifier. Add lint.yml to the
README Workflows table for completeness.

Addresses review feedback on #754
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 13, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 10:02 PM UTC · Completed 10:14 PM UTC

Commit: b361f6c · View workflow run →

@fullsend-ai-review
fullsend-ai-review Bot dismissed their stale review August 13, 2026 22:14

Superseded by updated review

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Aug 13, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 13, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 11:32 PM UTC · Completed 11:45 PM UTC

Commit: 4e2d9e4 · View workflow run →

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 14, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 12:54 AM UTC · Completed 1:05 AM UTC

Commit: ac6bb14 · View workflow run →

fullsend-ai-review[bot]

This comment was marked as outdated.

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All HIGH/MEDIUM findings resolved: marketplace-disable comment now cites the verified real cause, the skillsaw action is pinned to the immutable release commit, context-budget uses the baseline/ratchet mechanism instead of a blanket threshold bump, and rule IDs use the canonical claude- prefix. The composite-action pinning gap and persist-credentials inconsistency are acknowledged, documented follow-ups. CI (including the new skillsaw job) is green.

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review sweep: 2 MEDIUM findings posted as inline comments.

Comment thread .skillsaw.yaml
Comment thread .skillsaw-baseline.json
ralphbean pushed a commit that referenced this pull request Aug 14, 2026
Use uvx's @Version pinning syntax instead of pip-style ==version, which
uvx does not recognize as a valid command specifier. Add lint.yml to the
README Workflows table for completeness.

Addresses review feedback on #754
@ralphbean
ralphbean force-pushed the skillsaw/onboard-ci branch from ac6bb14 to 3bf1f29 Compare August 14, 2026 15:37
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 14, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 3:38 PM UTC · Completed 3:53 PM UTC

Commit: 3bf1f29 · View workflow run →

ralphbean pushed a commit that referenced this pull request Aug 14, 2026
Use uvx's @Version pinning syntax instead of pip-style ==version, which
uvx does not recognize as a valid command specifier. Add lint.yml to the
README Workflows table for completeness.

Addresses review feedback on #754
@ralphbean
ralphbean force-pushed the skillsaw/onboard-ci branch from 3bf1f29 to ccfa097 Compare August 14, 2026 16:09
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 14, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 4:10 PM UTC · Completed 4:26 PM UTC

Commit: ccfa097 · View workflow run →

fullsend-ai-review[bot]

This comment was marked as outdated.

ralphbean pushed a commit that referenced this pull request Aug 14, 2026
Use uvx's @Version pinning syntax instead of pip-style ==version, which
uvx does not recognize as a valid command specifier. Add lint.yml to the
README Workflows table for completeness.

Addresses review feedback on #754
@ralphbean
ralphbean force-pushed the skillsaw/onboard-ci branch from ccfa097 to ad49949 Compare August 14, 2026 18:41
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 14, 2026

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Started 6:43 PM UTC · Ended 6:55 PM UTC

Commit: ad49949 · View workflow run →

ralphbean and others added 6 commits August 14, 2026 14:54
Adds a skillsaw job to the lint workflow, running the linter in
--strict mode on every PR and push to main. Adds `make lint` and
`make lint-fix` targets so contributors can run the same checks
locally. Adds .skillsaw.yaml, tuned for this repo:

- Disables marketplace-json-valid and marketplace-registration.
  plugins/gopls-lsp is an internal fullsend harness plugin config
  (see harness/code.yaml), not a Claude Code marketplace plugin, so
  these rules produced false positives.
- Raises the context-budget skill thresholds to warn 16000 / error
  20000, above code-implementation and pr-review's current sizes.
  Both are dense, production-tuned orchestration procedures; --strict
  turns warnings into failures, so the thresholds are set high enough
  that neither file triggers a violation at either severity.

This builds on the skill content fixes in the preceding PRs in this
stack, evaluating skillsaw per #188.

Related to #188

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
- Pin skillsaw action to the exact commit SHA (v0.18.0) instead of the
  v0 tag object, matching the repo's pinact convention.
- Fix .skillsaw.yaml exclude block indentation to match the rest of
  the file.
- Correct the marketplace-json-valid/marketplace-registration disable
  comment: it wasn't gopls-lsp/plugin.json causing the failure, it's
  that this repo has no .claude-plugin/marketplace.json and isn't a
  distributable plugin marketplace.
- Clarify why strict: false in .skillsaw.yaml doesn't matter for CI.
- Document make lint / make lint-fix in README and CONTRIBUTING.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
Use uvx's @Version pinning syntax instead of pip-style ==version, which
uvx does not recognize as a valid command specifier. Add lint.yml to the
README Workflows table for completeness.

Addresses review feedback on #754
…mment

v0.18.0 renamed the Claude Code format rules to claude--prefixed
canonical IDs; the checked-in config still used the pre-rename legacy
names. Not a functional bug (skillsaw resolves the aliases), but
regenerate the keys to match what skillsaw init produces today.

Also fix the disabled marketplace-json-valid/marketplace-registration
comment, which still cited an already-debunked root cause (gopls-lsp
misdetection) instead of the verified one: no top-level
.claude-plugin/marketplace.json exists in this repo.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
…limit

Raising context-budget.limits.skill to warn 16000/error 20000 repo-wide
was meant to clear pr-review and code-implementation, but it silently
raised the ceiling for every skill file (present or future) and, per
review, missed that 3 other files (code-review, fix-review,
retro-analysis) were already over the default warn threshold too.

skillsaw ships a baseline/ratchet mechanism (baseline_mode: ceiling)
built exactly for this: snapshot each currently oversized file's token
count as its own ceiling, so it only fails again if it grows past its
current size, while every other file keeps the strict default (warn
3000 / error 6000).

Revert the limit to defaults and commit .skillsaw-baseline.json with
the 5 affected files snapshotted.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
CI's skillsaw job scans the pull_request merge commit (head merged
into base, per actions/checkout's default ref), not the head branch
alone. This PR's base (skillsaw/fix-pr-review) has a slightly
different pr-review/SKILL.md than what's on this branch, so the
baseline snapshotted from a plain head-branch checkout undercounted
its tokens (15,051 vs the merge tree's actual 15,058) and tripped the
ratchet in CI.

Regenerated by baselining against refs/pull/754/merge directly.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@ralphbean
ralphbean force-pushed the skillsaw/onboard-ci branch from ad49949 to e15628c Compare August 14, 2026 18:55
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 14, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 6:56 PM UTC · Completed 7:34 PM UTC

Commit: e15628c · View workflow run →

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.

Comment thread .skillsaw.yaml
context-budget:
enabled: auto
severity: warning
limits:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] PR body / config mismatch

The PR body states: 'Raises the context-budget skill thresholds to warn 16000 / error 20000.' However, the actual .skillsaw.yaml uses default thresholds (all sub-keys under limits: are commented out). The violations for pr-review (15,058 tokens) and code-implementation (9,116 tokens) are suppressed via the .skillsaw-baseline.json baseline file with ceiling mode, not by raised thresholds. If someone later removes the baseline file expecting the config to have raised thresholds, both skills will fail the lint check.

Suggested fix: Either update the PR body to accurately describe the baseline suppression mechanism, or set the thresholds in .skillsaw.yaml as described.

steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] consistency

The skillsaw job sets persist-credentials: false on its checkout step, while the test and commit-lint jobs in the same file omit this parameter. Since persist-credentials: false is a security hardening measure, the preferred resolution is to add it to the other two jobs in a follow-up PR rather than removing it here.

@waynesun09 waynesun09 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review-only sweep — 2 findings posted inline (both MEDIUM). Not approving/requesting changes; leaving disposition to the PR author.

- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: stbenjam/skillsaw@c325108b6cf94284762948b6a463e75ab5288520 # v0.18.0

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MEDIUM] Three independent skillsaw version pins can silently drift out of sync

The skillsaw version is recorded three separate times with nothing tying them together: this line pins the Action via stbenjam/skillsaw@c325108b6cf94284762948b6a463e75ab5288520 # v0.18.0, Makefile:8 sets SKILLSAW_VERSION := 0.18.0 (consumed by make lint/make lint-fix via uvx skillsaw@$(SKILLSAW_VERSION)), and .skillsaw.yaml:4 has version: "0.18.0". Renovate can bump the Action's SHA pin automatically but has no visibility into the Makefile variable or the YAML version string, so a future Action bump can leave local make lint runs on a different skillsaw release than CI.

Suggestion: Consolidate to one source of truth (e.g., have CI install via uvx skillsaw@$(SKILLSAW_VERSION) reusing the Makefile variable instead of the dedicated Action), or add a Renovate custom regex manager that also bumps SKILLSAW_VERSION in the Makefile and version: in .skillsaw.yaml whenever the Action pin moves.

Comment thread CONTRIBUTING.md
### Opening a PR

- Stage your changes and fix any lint failures before pushing.
- Stage your changes and fix any lint failures before pushing (`make lint`, or `make lint-fix` to apply automatic fixes).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MEDIUM] Updated lint guidance omits the separate pre-commit gate CI's test job enforces

This line narrows the generic "fix any lint failures before pushing" wording to point specifically at make lint/make lint-fix (skillsaw). But .github/workflows/lint.yml's pre-existing test job (unchanged by this PR) separately runs pre-commit run --all-files (actionlint, gitleaks, shellcheck, pinact, YAML/JSON checks, etc.), none of which make lint (skillsaw only) touches. A contributor who follows this narrowed instruction, sees make lint pass, and pushes can still fail CI on the unrelated test job's pre-commit checks.

Suggestion: Update this line to reference both checks (e.g., pre-commit run --all-files and make lint), or fold skillsaw into the pre-commit config so make lint and the test job's checks stay in sync.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fullsend-fix requires-manual-review Review requires human judgment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants