release: SkillSpector 2.12.0 - #550
Conversation
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
|
Codex on behalf of Mohit Gupta — independent qualification update, 16 September 2026. The freshly built 2.12.0 wheel from Fresh results after the AS3 update:
Both frozen corpora retain exactly the previous failure sets. Their behavioral expectations and fixture bytes were unchanged; only candidate identity/version provenance was refreshed. The consumer policy/limit assertions were retained. An initial provider-enabled consumer harness attempt was terminated and excluded; the reported counts come from the subsequent run with the established sanitized environment. Draft #563 separately fixes unsupported-primary and pure/mixed newline completeness gaps. At Current-head hosted CI/review and the remaining production/provider, Windows, published-artifact and adoption gates are still required. These results do not constitute release sign-off; no release or production pin was changed. |
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Reviewed current head e6bcf1044b781d9a28ae537cd8a56147edc9afa4, including the complete version/changelog/release-note diff, candidate history, existing review thread, version/lock consistency, component source identity, exact-head checks, and the stated release gates. The package and lock versions agree and all five hosted checks pass.
Changes are requested because two security guarantees in the release notes are not true of this exact candidate. Its letter-spacing implementation is byte-identical to the currently reviewed #470 implementation and still permits alternating-width spacing to evade P3/P4 and AE6. Its companion-classification analyzer blobs are identical to the unresolved #547 implementation, where four contextual fail-open paths remain. Either correct and requalify those implementations before publication or describe the precise limitations without claiming the risky cases remain covered. Deployment/provider, Windows, published-artifact, downstream-adoption, and independent release sign-off also remain merge gates.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed current head ff10467cd93832bc5a9a0ee8c07a2bcd3b91e01f after the automatic synchronization with main. This merge has the previously reviewed head e6bcf1044b781d9a28ae537cd8a56147edc9afa4 and current main e19ec01cc85b9891b2976a1157fce492504cfc13 as its exact parents. The only intervening tree changes are #558's three batch-scan files; none overlaps the four release files, whose reviewed contents are unchanged.
The two existing non-outdated release-note findings therefore remain current and unresolved. The candidate still claims irregular letter spacing fails closed although alternating-width short runs can evade P3/P4 and AE6, and it still makes an unconditional companion-context guarantee despite #547's unresolved PE3 and RA1 paths. Correct and requalify those implementations or accurately document the limitations. I have not duplicated the inline findings.
Exact-head CI run 35136927518 concluded action_required without starting jobs, leaving no check results. The security-release corrections, unresolved threads, active change request, missing exact-head checks, GitHub BLOCKED state, and previously identified release qualification/sign-off gates prevent merge.
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed the complete four-file release diff, version/lockfile consistency, prior review, and commit ancestry through the current head. The formerly missing #562/#526/#421/#491/#517/#571/#575 entries are now documented.
The same release-baseline requirement is not yet satisfied for this newer candidate. docs/release/skillspector-2.12.0.md still says the catalog covers 47 PRs through a345778, while the actual head includes #576, #556, #551, #580, #383, #584, #600, #560, #561, #542, #546, #541, and now #544 beyond that baseline. In particular, Known Limitations still says SARIF lacks occurrence columns although #584 is included. Update CHANGELOG.md, the release catalog/migration notes, and stale limitations to reflect the actual shipping commit; record validation against the refreshed candidate instead of referring to older counts/runs as current. Existing candidate/pending language should remain until owner release gates are satisfied. No duplicate inline thread added for this continuing baseline issue.
The version bump itself is consistent between pyproject.toml and uv.lock. Exact-head checks are currently unavailable; that separately blocks merging, not the reason for the documentation change request. Contributor code/tests were not executed locally.
Reviewed head: 9319c497cbed529cd0c12bd0e24c3c2a177c305a.
Priority: P0 — Release owners need an accurate shipping inventory and candidate-specific validation.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Incremental re-review after the latest automatic main synchronization. The delta only imports the already-reviewed #539/#588/#591/#607/#597; none of this PR's four release files changed. Therefore the outstanding release-inventory/validation request in review #550 (review) remains open; the newly included PRs also belong in the final shipping inventory. I have not repeated the finding inline. No exact-head checks are attached. The candidate remains unmerged.
Static review only; no contributor code/tests executed.
Reviewed head: 4a282fff4b3d9d3feaabbf9e65c742fd9dbd54a7.
Priority: P0 — Release sign-off requires an accurate candidate inventory and validation record.
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
rng1995
left a comment
There was a problem hiding this comment.
Release-candidate sign-off — SkillSpector 2.12.0
Approved at commit 41fca42e980d955d355b717a93a30831c26b284c against main baseline 224ba292d91b9e5fc59398fc38ab8971f9ab01ec.
The P0 inventory/validation request in review 5286455426, continuing review 5286353393, is addressed:
- All 65 merged PRs since v2.11.2 are cataloged, including the latest 18 merges. The changelog, release notes, migration guidance, and PR description match the current candidate. Fixed occurrence-column and recursive Markdown-output limitations are no longer described as open.
- The complete diff against main contains only the changelog, versioned release notes, and matching 2.12.0 package/lockfile versions. Third-party dependency versions are unchanged. The minor version remains appropriate for the added capabilities.
- CI run 35820158876 completed successfully for this head: all six checks passed. Its test-merge
340f81bc9ff36cde656893de3941edd5d2436334has the documented main/head parents and the identical release source treefc50629557ef61b9821b941576202796d3857a24. - Hosted and local Python validation each recorded 6,843 passed, 14 skipped, 134 deselected, 4 expected failures, and 90% coverage. The integration/live-provider exclusions remain explicit. The recorded candidate validation also includes 95 selected static integration tests, 65 extension tests, locked installation, lint/format, version/release-helper checks, wheel/sdist builds, Twine validation, and whitespace checks.
- Prior inline threads are resolved. Letter-spacing, companion-classifier, and other documented product limitations remain accurately qualified; unmerged #514/#553 are not included.
This is sign-off and approval of the current release-preparation PR, based on the candidate-specific evidence above. It is not a claim that documented scanner limitations have been fixed or that live-provider/deployment, Windows, published-artifact, or downstream-adoption qualification has passed. Those separate owner release gates remain recorded and are not waived by this review. Approval applies only to this commit; later merges require refreshed inventory and validation. No merge, tag, publication, or deployment is performed by this approval.
|
514 and 563 are pending for merge. 553 is merged. |
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
rng1995
left a comment
There was a problem hiding this comment.
Release-preparation sign-off for SkillSpector 2.12.0 at 29f7acb52866a634c5ad84b1c37d5023e64b47c9, synchronized through main 05119f4b868aafc7a6347043d3c3b8f9ac548091.
The stale-inventory/validation P0 is addressed for this candidate. The changelog, versioned release notes, and PR description now account for all 69 merged PRs since v2.11.2, including the latest #553, #615, #563, and #514. The Markdown-reference and runtime-command fixes are no longer described as excluded. The inventory was independently checked against git history, and the new behavior, migration effects, bounded-analysis caveats, and remaining limitations were reviewed against the implementation.
Version 2.12.0 remains appropriate: this unpublished release includes new user-facing capabilities and integrations in addition to fixes. The diff from main contains only the changelog, versioned release notes, and matching project-version changes in pyproject.toml and uv.lock; third-party dependency versions are unchanged.
Validation for this exact candidate:
- Local Python 3.12.13 suite with coverage: 7,859 passed, 14 skipped, 134 deselected, 4 expected failures; 90% coverage in 46m 34s. Integration and live-provider tests were excluded from this run. The four warnings were three unregistered timeout markers and the intentional duplicate-ZIP-member fixture.
- 65 OpenCode/Pi extension tests passed.
- 95 selected static integration tests passed (8 deselected), with no live providers.
- Installed-wheel smoke: 8 fixtures / 16 CLI and MCP-core scans passed, with the installed package path and version verified. This does not claim MCP stdio-transport coverage.
- Locked dependency installation, Ruff lint/format, CLI version, release-helper dry run, wheel/sdist builds, Twine checks, and whitespace checks passed.
- All six hosted checks passed in CI run 35919725480: changes, lint, Python tests, OpenCode/Pi tests, DCO, and Docker smoke.
- Hosted Python results matched the local counts: 7,859 passed, 14 skipped, 134 deselected, 4 expected failures; 90% coverage in 1h 22m 54s.
- The hosted test-merge tree
e2c5b5d610cd203a2cbb67437ef644f26dfae499matches the reviewed release head exactly and has the recorded main baseline and release head as parents.
This approves the release-preparation PR at the recorded commit. It does not certify that disclosed scanner limitations are fixed or that live-provider, Windows, deployment, published-artifact, downstream-adoption, or separate release gates have passed. Other reviewers' outstanding requests remain theirs to resolve. No merge, tag, release publication, or deployment is included in this sign-off.
Signed-off-by: Narendran Raghavan nraghavan@nvidia.com
Prepare SkillSpector 2.12.0 from v2.11.2, with the package version, lockfile, changelog, and release notes covering 69 merged PRs (43 categorized as fixes) through
05119f4b868aafc7a6347043d3c3b8f9ac548091.The release remains 2.12.0 because it adds CLI controls, integrations, LLM provenance, dependency-source analysis, GitHub subdirectory inputs, and other capabilities alongside security and correctness fixes. The latest four merges are #553, #615, #563, and #514; the previously excluded Markdown-reference and runtime-command fixes are now included. Migration requirements and changes to findings, completeness, and installation decisions are detailed below.
Included changes
Capabilities and providers
metadata.llm_provenanceschema 1 with configured/resolved/effective provider identity and requested-versus-observed sampling/reasoning controls; this does not guarantee deterministic provider behavior (feat(report): add LLM analysis provenance #556).SKILL.md/README.mdshell fences (Detect dependency source redirection #383)./tree/directory selection with longest advertised branch/tag resolution, including slash-containing refs, bounded ingestion, and path-escape rejection (feat(input): support GitHub tree subdirectories #561).SKILLSPECTOR_COMPACT_PROMPTS=1|true|yes; defaults remain unchanged (feat(llm): add compact prompt line numbering #542).--fail-on-findingsacross single-skill, recursive, and MCP registry scans (feat(cli): add fail-on-findings option #469), and a configurable per-artifact static-analysis allowance (feat: make static analysis runtime configurable #522).opencode_clisemantic provider, now requiring exactly OpenCode 1.18.31, with authentication and deny-all policy checks (feat(providers): add opencode_cli semantic-scan provider #536, feat(providers): verify OpenCode 1.18.31 for the deny-all policy #575)./skillspectorcommand andskillspector_scantool with static analysis by default and semantic analysis opt-in (feat(analyzer): add OpenCode-native SkillSpector invocation skill and tool #537).gemini-3.5-flashmetadata through the existing OpenAI-compatible provider (Feature/gemini support and cli ux #7).Retry-Afterhandling within the workflow deadline; retain sanitized incomplete-analysis diagnostics when recovery fails (fix(llm): retry transient provider failures #555).Security and completeness
printf/executable wrappers, boundedevaland shell-c, PowerShell-embedded shell commands, unsupported brace expansion, cross-window commands, and parser/deadline uncertainty. Successful semantic analysis cannot override these coverage gaps (fix: keep runtime-selected printf reconstruction incomplete #514).SKILL.md/skill.md, including nested or renamed ZIP members, with fatalunsupported_primary_content, failed execution, CLI exit 2, and MCP installation blocking. Preserve raw bytes and remove rejected primary text from provider input; supported ZIP and incidental-asset policies remain (fix: preserve incomplete scans for unsupported input and multiline prompts #563).obfuscated_instruction_textpartial evidence at original source lines, with interruptible workflow-bounded matching rather than a confirmed semantic P3/P4 claim (fix: preserve incomplete scans for unsupported input and multiline prompts #563).analyzer_load_errorcoverage gaps rather than silently complete scans; execution success remains distinct from completeness (fix(analyzers): surface analyzer modules dropped at registry load time #591).reference_missing) from ambiguous references (reference_unresolved). MCPsafe_to_installcan allow missing-reference-only caveats if every discovered file was inspected and all other gates pass; ambiguous references and other coverage failures still block (fix(mcp): stop an unresolved reference from blocking safe_to_install #526).__dict__/vars(module)subscripts andget/setdefault/pop, emitting AST7 for dynamic keys and AST9 for dangerous literal names ( fix: flag module __dict__ subscript as reflective attribute access #517)..venv,venv, and.toxunder existing traversal limits (fix(sc8): inspect bytecode in virtual environments #571).opaque_contentcoverage for unmodeled active-hook payloads (fix(hooks): fail closed on unmodeled payload analysis #573).Accuracy, output, and performance
.gitentries only during active clones, followed by strict final ingestion measurement; permission, checkout, and resource-limit failures remain failures (fix: keep runtime-selected printf reconstruction incomplete #514).--output(fix(cli): make recursive scans fail closed #576).MODERATEseverity toMEDIUM(fix(supply-chain): normalize GHSA "MODERATE" severity to MEDIUM #588), and avoid Rustdrop(&mut self)false positives while preserving YARAROP(elf)controls (fix(yara): require word boundary on exploit_framework $rop_chain #607)./blob/URLs (fix(input): download the raw file for GitHub and GitLab /blob/ URLs #566).Compatibility and migration
safe_to_install=false. Default CLI exit 0 is not a completeness guarantee; AE6 reports unresolved interpretation.Incomplete referenced artifact analysispattern andtarget_path,target_disposition, boundedreasons, andreasons_truncatedevidence. Proven Perl print literals can lose false AE1 findings without exempting their payloads from security checks.analyzer_load_errorandtransitive_child_scan_failed, preserve aggregate failure/omission evidence, and distinguish JSON zero-based columns from SARIF one-based Unicode-codepoint columns.SKILLSPECTOR_BUILD_REVISION; unavailable revision data stays unknown, and provenance is not a reproducibility guarantee..opencode/in a checkout and is not installed by the wheel..venvor setSKILLSPECTOR_BINto an existing absolute executable path. AmbientPATHlookup is removed, and reports must be written within the current workspace. These restrictions do not change the standalone CLI or OpenCode tool's output-path contract.reference_missingseparately fromreference_unresolved. MCPsafe_to_installcan be true for a missing-reference-only caveat even though the report remains incomplete/CAUTION; partially inspected files, ambiguity, and requested semantic failures still block.SAFE./blob/scans now analyze raw file bytes.--fail-on-findingsis opt-in. Valid model-registry overrides now affect CLI-provider token budgets. The static allowance defaults to 300 seconds per artifact within the existing workflow deadline; setSKILLSPECTOR_MAX_STATIC_ANALYSIS_SECONDS_PER_ARTIFACT=30to retain the earlier allowance.--no-llmfor intentional static-only scans; unavailable or incomplete requested semantic work remains incomplete.Validation
05119f4b868aafc7a6347043d3c3b8f9ac548091(fix: keep runtime-selected printf reconstruction incomplete #514); all 69 merged PRs since v2.11.2 are cataloged.29f7acb52866a634c5ad84b1c37d5023e64b47c9.timeoutmarkers and the intentional duplicate-ZIP-member fixture.e2c5b5d610cd203a2cbb67437ef644f26dfae499is identical to the release head's tree, with the recorded main baseline and release head as parents.Validation applies to the commit recorded here. Previous candidate test counts and workflow runs do not certify subsequent main merges.
Known limitations and release status
TypeErrorwhen an optional end column is absent, although normal parsed files pass.No 2.12.0 release has been published.