fix(analyzers): drop P2 BOM, RA2 plist-substring and EA3 OFL false positives - #645
Conversation
…sitives - P2: a U+FEFF at offset 0 is a byte-order mark, not hidden text. Skip it only at the start of the file; mid-file U+FEFF and other zero-width characters are still reported. - RA2: require that `plist` is not preceded by a letter, so identifiers such as `relationshipList` or `CT_GradientStopList` no longer match. `.plist` paths and `write_plist(...)` are still detected. - EA3: add the SIL Open Font License disclaimer to the canonical license ranges and accept `OFL.txt` / `<Font>-OFL.txt` basenames. Only the exact canonical lines are suppressed, as for the existing ranges. Refs NVIDIA#644. Claude-Session: https://claude.ai/code/session_01HgEFtvq6aWpN5jC1MRCpEB Signed-off-by: Wen <jwcastillo@gmail.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Hi @jwcastillo, thank you for your contribution to SkillSpector — we really appreciate it!
This PR fixes three narrow false-positive sources from #644:
- P2: a U+FEFF at offset 0 of the scanned text is skipped in the zero-width branch of
_p2_pattern_matches. - RA2:
plistmust not follow a letter. - EA3: the SIL OFL disclaimer is added as a canonical license range, and
OFL/<Font>-OFLbasenames are recognized.
For each fix I checked for the false negatives it could introduce, read the diff against the surrounding code and current main, and evaluated the changed regexes on sample inputs.
What I verified
-
P2 / BOM. The skip applies only to a zero-width candidate that starts at offset 0 and is U+FEFF, and scanning then continues on the same line. These still report P2, and the new tests cover all three:
- a BOM followed by U+200B, on the same line or a later one
- a second U+FEFF
- any mid-file U+FEFF
Offset 0 of a later raw window is at least 8 KiB into the previous window, which owns that offset, so windowing cannot hide a mid-file U+FEFF. HTML-comment, bidi, tag-block and data-URI detection are unchanged. The change is consistent with #471 on main (a leading BOM is stripped for frontmatter detection) and composes with #473 (the bidi loop does not use the zero-width branch).
-
RA2. Under
re.IGNORECASE,(?<![a-z])excludes any ASCII letter. These still match:.plistpaths including~/Library/LaunchAgents/*.plist,PlistBuddy,plistlib,write_plist(...),defaults write, andlaunchctl load. Only camelCase identifiers lose the match, which the PR acknowledges. I found no other LaunchAgent/plist persistence signal in the static rules or YARA, so this alternative carries that coverage alone (see Finding 1). -
EA3.
_is_license_basenameonly gates suppression of EA3 on exact canonical lines. Broadening the basenames toOFL*/*-OFL*therefore cannot hide non-boilerplate text, and a test pins an attacker line in an OFL-named file. The canonical lines match the OFL 1.1 disclaimer after casefold and whitespace normalization, and the existing range-parametrized tests pick up the new range automatically.
Findings
- [Non-blocking, optional]
src/skillspector/nodes/analyzers/static_patterns_rogue_agent.py:174: you can recover camelCase coverage without re-admitting the…pListidentifiers by adding a case-sensitive alternative,(?-i:[a-z]Plist). On sample strings it matchessavePlist,agentPlistandwriteLaunchAgentPlist, but notrelationshipList,CT_GradientStopList,IPListorshopList. If you add it, include positives for those names. - [Non-blocking, optional] The window-overlap argument in the description is correct but not pinned by a test. A regression would guard the invariant against future windowing changes: put U+FEFF exactly at the second raw-window start (
static_runner._RAW_WINDOW_OWNED_CHARS - static_runner._WINDOW_OVERLAP_CHARS, i.e. offset 231,424) in a markdown file longer than 256K characters and assert P2. - [Note] This PR overlaps with #622 in
static_patterns_prompt_injection.py, but the code paths are disjoint (zero-width iteration here, comment carve-out there). Applying #622 and then this PR onto current main merges cleanly, and there is no semantic interaction. The[//]: #spanning issue noted in #644 (B1) is still open and out of scope here.
Tests/CI
All checks pass at this head, and GitHub reports it as mergeable with current main. Each new false-positive test targets behavior that fails on main, and the true-positive guards (a mid-file or later U+FEFF, LaunchAgent .plist and write_plist, and an OFL-named file with non-boilerplate text) are meaningful. No docs or CHANGELOG change is needed; the CHANGELOG is updated at release time.
Decision: Approved (reviewed head 0ac5d2a6215b9d9dc01d7dea165cf104f689acc5)
Problem
Scanning public third-party skills with
--no-llmproduced false positives from three narrow pattern bugs (issue #644, items A1 to A3):.xsdfiles that start withEF BB BF <?xml.plistanywhere, case-insensitively, including inside identifiers such asrelationshipList(JS) andCT_GradientStopList(XSD). One schema set produced 11 hits and one HTML template produced 30.OFL.txt/<Font>-OFL.txt. The fix(analyzer): filter license boilerplate from EA3 static findings (#312) #328 filter only covers Apache/MIT/BSD ranges andLICENSE/COPYING/NOTICEnames.Reproduction
A synthetic skill with a BOM-prefixed XSD, a JS file containing
relationshipList, and aSomeFont-OFL.txtdisclaimer, scanned withskillspector scan <dir> --no-llm --format json:main@ 89e9087Fix
(?<![a-z])plist..plistpaths,PlistBuddyandwrite_plist(...)still match. The known limit is that camelCasesavePlistno longer matches, butdefaults write,launchctl loadand the.plistfile path of a LaunchAgent do.ofl, optionally prefixed (<Font>-OFL.txt), as a license basename. As with the existing ranges, only the exact canonical lines are suppressed.Tests
Each new FP test fails on
mainand passes on this branch (verified by revertingsrc/only):test_p2_leading_byte_order_mark_no_false_positive[SKILL.md|schemas/types.xsd]TestRogueAgent::test_ra2_plist_substring_inside_identifier_not_detected[camel_case_identifier|xsd_type_name]TestLicenseFiles::test_ofl_font_license_disclaimer_suppresses_ea3[OFL.txt|assets/SomeFont-OFL.txt]TestLicenseFiles::test_helper_boundaries[OFL.txt-True|fonts/SomeFont-OFL.txt-True]True-positive guards:
test_p2_zero_width_after_byte_order_mark_still_produces_finding: covers a mid-file U+FEFF, a BOM followed by U+200B on a later line, and a BOM followed by U+200B on the same line.test_ra2_detected[launch_agent_plist|snake_case_plist]test_ofl_named_file_with_non_boilerplate_content_reports_ea3, andtest_helper_boundaries[profl.txt-False|ofl.py-False].test_each_canonical_range_suppresses_only_ea3andtest_attacker_line_after_canonical_range_reports_ea3now also cover the new range.Local results:
make lintandmake format-checkare clean.make test-unitgave 7926 passed, 14 skipped, 4 xfailed.make test-integration(no provider keys) gave 127 passed, 4 skipped.Out of scope
Everything else is in #644 and is not changed here:
[//]: #pattern has the same DOTALL spanning problem.static_parse_limit, RP1 in tests and docs, E2 on env pass-through (E2 fires HIGH on subprocess env pass-through (env={**os.environ, …}) with the same severity and confidence as a real harvester — precision cost of the #329 fix #441 / fix(e2): exempt child-process env pass-through from harvesting #492), TM1/TM2/TM3, PE5/TM4 context, and a suggested rule for predictable temp paths.Refs #644.
https://claude.ai/code/session_01HgEFtvq6aWpN5jC1MRCpEB