fix(baseline): preserve exact findings and protect baseline output

.github/workflows/ci.yml

@@ -91,6 +91,36 @@ jobs:
       - run: uv run skillspector --version
       - run: uv run make test-ci
 
+  test-baseline-integration:
+    name: Baseline CLI and MCP Integration Tests
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    env:
+      # These selected suites use static analysis and require no provider credentials.
+      SKILLSPECTOR_PROVIDER: offline-integration-tests
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - name: Set up uv
+        uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5
+        with:
+          version: ${{ env.UV_VERSION }}
+          enable-cache: true
+          cache-dependency-glob: uv.lock
+          python-version: ${{ env.PYTHON_VERSION }}
+      - run: make install-dev
+      - name: Run credential-free baseline and real MCP transport regressions
+        run: >-
+          uv run pytest -m integration
+          tests/integration/test_baseline_coverage.py
+          tests/integration/test_baseline_mcp.py
+          --junitxml=baseline-integration-results.xml
+      - if: always()
+        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
+        with:
+          name: baseline-integration-results
+          path: baseline-integration-results.xml
+          if-no-files-found: ignore
+
   test-opencode:
     name: OpenCode TypeScript Tests
     runs-on: ubuntu-latest

docs/SUPPRESSION.md

@@ -39,6 +39,16 @@ skillspector scan ./my-skill/ --baseline .skillspector-baseline.yaml --show-supp
 | `skillspector scan <path> --baseline FILE --show-suppressed` | Also list the suppressed findings (they still don't affect the score). |
 
 A missing, malformed, or unsupported baseline file exits with code 2.
+Generation also exits with code 2 without creating or replacing the output when
+scan execution fails, including failed semantic analysis or unreadable primary
+content. Resolve the failure and rerun the scan before accepting its findings.
+Use `--no-llm` when you intentionally want to accept only static findings.
+
+A successful but incomplete scan can generate a baseline for its observed
+findings, with a warning on stderr. This does not accept uninspected content or
+clear coverage gaps: subsequent reports retain their incomplete status and
+recommendation even if every observed finding is suppressed.
+
 When a selected baseline or baseline output is stored inside the scan target,
 SkillSpector treats that exact file as an explicit scope exclusion. This
 prevents sensitive rule text from creating a finding against itself or entering
@@ -103,6 +113,26 @@ Generated by `skillspector baseline`, it is intentionally exact:
 editing the source or upgrading SkillSpector keeps the finding active until it
 is reviewed and the baseline is regenerated.
 
+LLM and meta-analysis can change explanations, remediation, confidence, or other
+evidence between scans of unchanged files. Those results remain active when
+their exact fingerprints differ. For repeatable static acceptance, use
+`--no-llm` for both baseline generation and subsequent scans.
+
+Generation fingerprints each original finding before report compaction, including
+repeated matches on different lines and identical matches in different files.
+Only identical exact fingerprints share an entry. A baseline generated by an
+affected version may omit these occurrences; regenerate it to include them.
+If the complete baseline exceeds the loader's size or record limits, generation
+fails before replacing the output file instead of writing a partial baseline.
+After validation, generation writes a complete temporary file beside the
+destination and atomically replaces the output. A failed write or replacement
+preserves the existing baseline. Existing destinations must be writable regular
+files; symlinks and special files are rejected. Replacement requires a writable
+parent directory. On POSIX, generated files grant access only to their owner;
+regeneration removes group/other access instead of risking wider
+access when replacing a file with platform-specific ACLs. Reapply shared access
+explicitly after generation if your workflow requires it.
+
 Every v2 entry must be a mapping with a 64-hex-character `sha256:` hash and a
 non-empty `reason`. `rule_id` and `file` are informational fields for reviewers.
 If source content is unavailable or `scanner_version` does not match, exact

src/skillspector/cli.py

@@ -3181,7 +3181,28 @@ def baseline(
         state = _scan_state(input_path, FormatChoice.json, no_llm)
         state["baseline_path"] = os.path.abspath(output.expanduser())
         result = graph.invoke(state)
-        findings = effective_findings(result)
+        completeness_value = result.get("analysis_completeness")
+        completeness = completeness_value if isinstance(completeness_value, dict) else {}
+        if (
+            result.get("execution_successful") is False
+            or completeness.get("execution_successful") is False
+            or completeness.get("status") == "failed"
+        ):
+            raise ValueError(
+                "Cannot generate baseline because scan execution failed. "
+                "Run 'skillspector scan' to inspect analysis completeness, "
+                "resolve the failures, and retry."
+            )
+        if completeness.get("is_complete") is False or completeness.get("status") == "partial":
+            err_console.print(
+                "[yellow]Warning:[/yellow] Scan analysis is incomplete; the baseline "
+                "accepts only observed findings and coverage gaps remain."
+            )
+        # Report compaction retains locations but not each occurrence's exact
+        # context/confidence. Fingerprint the same originals suppression sees.
+        findings = result.get("baseline_findings")
+        if not isinstance(findings, list):
+            findings = effective_findings(result)
         data = build_baseline_dict(
             findings,
             reason=reason,

src/skillspector/nodes/report.py

@@ -1839,6 +1839,7 @@ def report(state: SkillspectorState) -> dict[str, object]:
         "risk_recommendation": risk_recommendation,
         "report_body": report_body,
         "filtered_findings": reported_findings,
+        "baseline_findings": active_findings,
         "suppressed_findings": suppressed,
         "execution_successful": execution_successful,
         "analysis_completeness": dict(analysis_completeness),

src/skillspector/state.py

@@ -307,6 +307,9 @@ class SkillspectorState(TypedDict, total=False):
     baseline_path: str | None
     show_suppressed: bool
     suppressed_findings: list[object]
+    # Sanitized active findings before report compaction/output limits. Exact
+    # baselines need each original occurrence's location and evidence fields.
+    baseline_findings: list[Finding]
 
     # Model IDs per LLM-using node: e.g. {"default": "...", "meta_analyzer": "..."}
     model_config: dict[str, str]

src/skillspector/suppression.py

@@ -55,16 +55,19 @@
 
 from __future__ import annotations
 
+import errno
 import fnmatch
 import hashlib
 import json
 import os
 import posixpath
 import re
+import sys
+import tempfile
 from collections.abc import Mapping
 from dataclasses import dataclass, field
 from pathlib import Path
-from stat import S_ISREG
+from stat import S_IMODE, S_ISREG
 from typing import Any
 
 import yaml
@@ -642,14 +645,115 @@ def build_baseline_dict(
     }
 
 
+def _restrict_baseline_temporary(descriptor: int) -> None:
+    """Remove inherited access before a temporary file receives baseline data."""
+    if os.name == "posix":
+        # Also masks named-user/group ACL grants on POSIX ACL implementations.
+        os.fchmod(descriptor, 0o600)
+    if sys.platform != "darwin":
+        return
+
+    # macOS extended ACL grants are independent of permission bits. Use the
+    # already-open descriptor so clearing them cannot follow a swapped path.
+    import ctypes
+
+    try:
+        libc = ctypes.CDLL(None, use_errno=True)
+        init_acl = libc.acl_init
+        set_acl = libc.acl_set_fd_np
+        free_acl = libc.acl_free
+    except AttributeError as error:
+        raise OSError(errno.ENOTSUP, "Cannot clear inherited baseline ACLs") from error
+    init_acl.argtypes = [ctypes.c_int]
+    init_acl.restype = ctypes.c_void_p
+    set_acl.argtypes = [ctypes.c_int, ctypes.c_void_p, ctypes.c_int]
+    set_acl.restype = ctypes.c_int
+    free_acl.argtypes = [ctypes.c_void_p]
+    free_acl.restype = ctypes.c_int
+    empty_acl = init_acl(0)
+    if not empty_acl:
+        raise OSError(ctypes.get_errno(), "Could not initialize baseline ACL")
+    try:
+        if set_acl(descriptor, empty_acl, 0x100) != 0:  # ACL_TYPE_EXTENDED
+            raise OSError(ctypes.get_errno(), "Could not clear inherited baseline ACLs")
+    finally:
+        free_acl(empty_acl)
+
+
 def dump_baseline(data: dict[str, object], path: str | Path) -> None:
-    """Write a baseline mapping to *path* as YAML (``.json`` extension -> JSON)."""
+    """Validate and atomically replace a regular baseline (``.json`` -> JSON).
+
+    On POSIX, new files have owner-only permissions. Replacements preserve
+    ownership and existing owner read/write bits, clearing group/other bits;
+    the old file must be writable. Symlinks and special files are rejected.
+    Concurrent writers publish complete documents; the last replacement wins.
+    """
+    baseline_from_dict(data)
     p = Path(path)
     if p.suffix.lower() == ".json":
-        p.write_text(json.dumps(data, indent=2), encoding="utf-8")
+        # PyYAML does not combine JSON's escaped UTF-16 surrogate pairs. Emit
+        # astral characters directly, escaping only genuine lone surrogates.
+        content = (
+            json.dumps(data, indent=2, ensure_ascii=False)
+            .encode("utf-8", errors="backslashreplace")
+            .decode("utf-8")
+        )
     else:
         header = (
             "# SkillSpector baseline — findings listed here are suppressed on future scans.\n"
             "# Edit 'reason' fields and add glob 'rules' as needed. See docs/SUPPRESSION.md.\n"
         )
-        p.write_text(header + yaml.safe_dump(data, sort_keys=False), encoding="utf-8")
+        content = header + yaml.safe_dump(data, sort_keys=False)
+    # A complete population can exceed the loader's limits even when a compact
+    # report fits. Reject it before overwriting an existing, usable baseline.
+    encoded = content.encode("utf-8")
+    if len(encoded) > MAX_BASELINE_BYTES:
+        raise ValueError(f"Baseline file exceeds byte limit ({MAX_BASELINE_BYTES}): {p}")
+    yaml.load(content, Loader=_BoundedBaselineLoader)
+
+    destination = None
+    try:
+        destination = p.lstat()
+    except FileNotFoundError:
+        pass
+    else:
+        if not S_ISREG(destination.st_mode):
+            raise ValueError(f"Baseline output must be a regular file: {p}")
+        if not destination.st_mode & 0o222:
+            raise PermissionError(errno.EACCES, "Baseline output is not writable", str(p))
+        # Atomic replacement needs directory permissions, but must not bypass an
+        # existing file's write restrictions (including ACLs). Never truncate it.
+        flags = os.O_WRONLY | getattr(os, "O_NONBLOCK", 0) | getattr(os, "O_NOFOLLOW", 0)
+        descriptor = os.open(p, flags)
+        try:
+            destination = os.fstat(descriptor)
+            if not S_ISREG(destination.st_mode):
+                raise ValueError(f"Baseline output must be a regular file: {p}")
+        finally:
+            os.close(descriptor)
+
+    temporary_path = None
+    try:
+        with tempfile.NamedTemporaryFile(
+            mode="wb", dir=p.parent, prefix=".skillspector-baseline.", suffix=".tmp", delete=False
+        ) as temporary:
+            temporary_path = Path(temporary.name)
+            _restrict_baseline_temporary(temporary.fileno())
+            temporary.write(encoded)
+            temporary.flush()
+            if destination is not None:
+                current = os.fstat(temporary.fileno())
+                if (current.st_uid, current.st_gid) != (destination.st_uid, destination.st_gid):
+                    os.fchown(temporary.fileno(), destination.st_uid, destination.st_gid)
+                # Do not broaden group/other access when replacing a file whose
+                # ACL metadata may be more restrictive than its mode bits.
+                mode = S_IMODE(destination.st_mode) & 0o600
+                if os.name == "posix":
+                    os.fchmod(temporary.fileno(), mode)
+                else:
+                    os.chmod(temporary_path, mode)
+            os.fsync(temporary.fileno())
+        os.replace(temporary_path, p)
+    finally:
+        if temporary_path is not None:
+            temporary_path.unlink(missing_ok=True)