Skip to content

Protect Test Files From Unintended AI Edits #142

Description

@yeus

Problem

Our tests are spread across the repo (Rust unit tests, integration tests,
wasm/vscode smoke tests).
When AI agents make changes, it is hard to notice test modifications in
review.
Tests are a key safety net, so accidental test edits are risky.

Goal

Add simple protections so test changes are always deliberate and human-
reviewed.

Proposal

  1. Add CI check: fail if test files changed without an explicit opt-in.
  2. Add CODEOWNERS for all test paths so human review is required.
  3. Document the rule in CONTRIBUTING.md:
    • “AI-generated PRs should not modify tests unless explicitly
      requested.”
    • “If tests must change, explain why in the PR description.”

Scope (test paths to protect)

  • crates//tests/
  • crates/**/*tests.rs
  • crates//src//tests/**
  • editors//tests/

Acceptance Criteria

  • PRs that change protected test files fail CI by default.
  • There is a clear, manual override path (label or env flag) for
    intentional test updates.
  • At least one human reviewer is required for test-file changes.
  • Contributor docs explain the policy clearly.

Why this helps

  • Reduces accidental weakening or drifting of tests.
  • Keeps reviews focused on intentional behavior changes.
  • Works with current Rust-style distributed test layout (no major test
    reorganization needed).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions