fix(analyzer): stop value-only parameter expansions from marking files partial - #686
Conversation
A file that documents shell defaults such as `${VAR:-default}`, uses
`"${ARR[@]}"` as a command, or assigns `NAME=$(printf ...)` was marked
partial with static_parse_limit, and every SKILL.md link to it became a
HIGH AE1 finding.
The printf reconstruction check labelled every braced expansion other
than `${NAME}` as a dynamic word, so a backtick or `$(...)` span that
started with one looked like a runtime-built printf. Expansions whose
operator words contain no command substitution, arithmetic, indirection,
transformation or zsh flag now get the same runtime-parameter treatment
as `${NAME}`. Prefix assignment values no longer count as command-name
reconstruction; their nested command bodies are still scanned.
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Self-review of 7662059 after #634 merged to main (7822cb1). The core change holds up: #686 on top of the new main merges cleanly, the VSS skill from NVIDIA-AI-Blueprints/video-search-and-summarization#2386 scans with 0 AE1 (5 partial files, none referenced from SKILL.md), and both PRs' test files pass together (76 tests).
Corpus check. 6,073 files from NVIDIA/skills, the VSS develop skills and anthropics/skills, new main vs this branch merged with main:
has_bounded_parse_exhaustion: 0 newly exhausted, 14 newly complete.static_patterns_tool_misuse.analyzefindings (TM1–TM4): identical in every file.
Findings
- [Process] CI predates #634. Every check passed, but against the old
main. Mergemainso CI covers the combined parser. - [Tests] Pin the "a value never names the command" argument. A later use of the assigned value as a command must stay partial:
RESULT=$(printf 'r%s' m); $RESULT -rf /, the quoted"$RESULT" -rf /form, andexport TOOL=... && $TOOL -rf /. All three are partial today, but no test protects them (inline). - [Comment accuracy] Quoted-value branch of
_is_assignment_word(inline). #634 now skips the printf and operand checks for quote-after-=candidates, so this branch no longer changes an outcome. Across the corpus it was reached 4,787 times, always from those candidates. Keep it so_parse_shell_command_wordstays consistent for any caller and skips a wasted printf evaluation, but the comment should say that. - [Docs] PR description. The "Relationship to open PRs" section still describes #634 as open and uses pre-merge numbers.
I'll push the fixes for all four.
Skipping the printf reconstruction check on `NAME=$(printf ...)` is safe because a later `$NAME` command word is still checked. Pin that with controls for `RESULT=$(printf 'r%s' m); $RESULT -rf /`, the quoted `"$RESULT" -rf /` form and an exported value run after `&&`. All three stay partial. Explain why `_is_assignment_word` still recognizes a quoted value now that the exhaustion sweep skips quote-after-`=` candidates itself: it keeps the parser consistent for any caller and avoids a discarded printf evaluation. Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
All four findings from the self-review are addressed at fce4c43:
- Branch includes #634. The branch contains the
mainmerge77d2df4with #634 (7822cb1). The full non-integration suite on this head passes:8482 passed, 14 skipped, 4 xfailed. Ruff lint and format pass. - Later-use controls added.
$RESULT -rf /,"$RESULT" -rf /andexport TOOL=... && $TOOL -rf /after aprintfassignment all assert partial. - Comment updated. The comment on the quoted-value branch of
_is_assignment_wordnow says the exhaustion sweep already skips those candidates and why the branch is kept. - PR description updated. It now reflects #634 being merged, with current numbers.
Validation on main + this PR
- VSS
skills/vss-build-vision-ai(NVIDIA-AI-Blueprints/video-search-and-summarization#2386,--no-llm):mainhas 8 partial files and 4AE1. This PR brings that to 5 partial files and 0AE1. - 6,073-file corpus: 0 files newly partial, 14 newly complete, identical TM1–TM4 findings.
Decision: Ready to approve (reviewed head fce4c4304ba29a8f5b128e702fd60f73bf5e0da3). This account opened the PR, so GitHub does not allow it to approve. A maintainer approval is needed to merge.
Summary
static_patterns_tool_misusemarks a filepartial(static_parse_limit) when a backtick or$(...)span begins with an ordinary parameter expansion that uses an operator (${VAR:-default},${VAR#prefix},"${ARR[@]}"), or when a prefix assignment's value contains$(printf ...). EverySKILL.mdlink to such a file then becomes a HIGHAE1finding, which blocks signing gates even though the content is harmless documentation or a harmless script.This PR gives value-only expansions and assignment values the same treatment
${NAME}already gets. Expansions that can evaluate code stay fail-closed.Customer impact
Reported internally against SkillSpector 2.11.2 (SkillEvaluator 1.5.6) for
skills/vss-build-vision-aiin NVIDIA-AI-Blueprints/video-search-and-summarization#2386. The gate reported 11 HIGHAE1findings, which also block the catalog sync in NVIDIA/skills#587. The skill documents shell defaults in prose and comments; the content does not need to change.The issue still reproduces on
main(8831219):skillspector scan skills/vss-build-vision-ai --no-llmat the PR head (fbf0232) reports 16 of 61 files partial (73.8% coverage) and 35AE1findings.Root cause
_invocation_expansion_markerclassifies a braced expansion as either a runtime parameter or a dynamic word. Only the bare forms matched by_SIMPLE_BRACED_PARAMETER_RE(${NAME},${1},${@}, ...) counted as runtime parameters; everything else was labelled dynamic because it "may contain nested command substitutions". When a dynamic word is the first word of a backtick or$(...)span,_consume_printf_invocationreports a possible runtime-builtprintf,_parse_shell_command_wordsetslimited, andhas_bounded_parse_exhaustionreturnsTrue.# A `${VSS_CONTAINER_TAG:-...}` fallback ...(Python comment)Defaults may nest (`${A:-${B}/x:${C}}`)(docstring)A `${...}` or `$NAME` that survived expansionDEPLOYMENT=$("${VSS[@]}" configure show)ES_URL=$(printf '%s' "${DEPLOYMENT}" | jq ...)The last row only fires when the variable name starts with one of the command-word candidate letters (
r R d D e E):RESULT=$(printf '%s' "$X")was partial,MY_URL=$(printf '%s' "$X")was not. The shell recognizesNAME=valuebefore expansion and never runs the value as the command name.Fix
In
src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py:_is_value_parameter_expansionreplaces_SIMPLE_BRACED_PARAMETER_RE. It accepts an expansion only when it and every nested${...}match a value-only head (optional#length prefix; a name with an optional[@],[*]or numeric subscript, or a numeric/special parameter; then}or a value operator:- - := = :? ? :+ + # ## % %% / // /# /% ^ ^^ , ,,). The expansion must also contain no backtick,$(,$[,<(or>(. The documentation placeholder${...}is accepted because every shell rejects it as a bad substitution. Accepted forms get_RUNTIME_SHELL_PARAMETER_SENTINEL, so they still need the same printf invocation evidence as$NAME._is_assignment_wordrecognizes a prefix assignment (NAME=/NAME+=after a clause boundary, a reserved word such asthen/do/if, orexport/local/declare/readonly/typeset), including a candidate at the value's opening quote._parse_shell_command_wordskips only the printf reconstruction check for such a word. Nested command bodies remain independent candidates and are still scanned.What stays fail-closed
Each of these is a regression test and still reports
static_parse_limit:${X@P},${!X},${ARR[i]},${X:offset}, zsh${(e)X},${ cmd; }, and any expansion containing$(, backticks or$[(${X:-$(id)},${X:-${Y@P}}).$(printf ${FORMAT:-%s}) -rf /,$(${TOOL:-printf} 'r%s' m) -rf /,$("${VSS[@]}" configure show) -rf /.alias rmall="$(printf ...)",echo ES_URL=$(printf ...),a/ES_URL=$(printf ...), a quoted command after an assignment, andeval "RESULT=$(printf ...)".test_runtime_printf_arguments_and_nested_reconstruction_stay_partialcases are unchanged.Validation
tests/nodes/analyzers/test_parameter_expansion_reconstruction.py(48 tests): hook-level clean and fail-closed cases, ledger outcomes, assignment recognition, and two CLI end-to-end scans (shell-defaults bundle is complete with noAE1; runtime-command control keepsAE1). On the pre-fix(analyzer): avoid false parse limits on quoted shell assignments #634main, the 20 bug-case tests fail and all control tests pass. Controls include a later$RESULT -rf /use of an assigned value, which stays partial.main(with fix(analyzer): avoid false parse limits on quoted shell assignments #634) plus this PR:8482 passed, 14 skipped, 4 xfailed.ruff checkandruff format --checkpass onsrcandtests.echoin bash, dash and zsh.Relationship to #634 and remaining work
The VSS skill hits two independent parser defects. #634 (merged as
7822cb1) fixed quote ownership. This PR fixes the operator/printf family above. Measured with--no-llmon the VSS skill:AE18831219)mainwith #634 (7822cb1)main+ this PRAfter #634, quote-after-
=candidates skip the printf and operand checks in_has_shell_command_word_exhaustion. The quoted-value branch of_is_assignment_wordis kept so_parse_shell_command_wordstays consistent for any caller, as the code comment now explains.Corpus check against
main(7822cb1), 6,073 files fromNVIDIA/skills, the VSSdevelopskills andanthropics/skills: 0 files newly partial, 14 newly complete, and identical TM1–TM4 findings in every file.The 5 remaining partial files come from prose apostrophes and non-shell quoting in YAML/logstash comments and Python string literals. They lower coverage but are not referenced from
SKILL.md, so they produce noAE1.🤖 Generated with Claude Code