Retro agent: add configurable gating skill to control issue filing decisions

[fullsend-ai-triage]

Note

This issue was automatically generated from meeting notes by the scribe agent.
Please review, edit, and add any missing context before prioritizing.

Problem

The retro agent files issues for all findings without evaluating whether each finding warrants a separate issue. This creates noise in target repositories, especially when multiple concurrent retro runs identify similar or low-value findings. Existing mitigations (post-script dedup guards, retraction logic) only filter after the agent has already decided to file. A gating mechanism within the agent itself is needed to make filing decisions based on configurable rules.

Options considered

• Gating skill within the agent (selected): Add a skill that evaluates each finding against configurable criteria before filing. This runs before the post-script and gives the agent judgment about what to file. The default configuration preserves the current "file all" behavior for backward compatibility.
• Post-script-only filtering: Continue improving post-script dedup guards (Add post-script dedup guard for concurrent retro proposals #5571, Retro post-script should deduplicate proposals against recently-filed bot issues before creating new ones #5570) and retraction logic (Retro post-script should retract issues a subsequent run identifies as incorrectly filed #5592). Considered insufficient because post-scripts run after the agent has already committed to filing, wasting compute and creating race conditions.
• Hardcoded skip rules: Embed fixed heuristics (e.g., always skip cosmetic findings). Rejected in favor of configurable rules that repo owners can tune.

Acceptance criteria

• New gating skill added to the retro agent skill set
• Configurable rules determine whether a finding should be filed as an issue
• Default configuration maintains current "file all" behavior for backward compatibility
• Gating evaluation runs before issue creation, not as a post-script filter
• Skill produces a clear accept/reject decision with reasoning for observability
• Integration with existing retro analysis pipeline does not break current behavior

Related

• Builds on noise reduction work in Broaden retro skip criteria to all unmerged PRs with no agent interaction #5619, Retro post-script should retract issues a subsequent run identifies as incorrectly filed #5592, Add post-script dedup guard for concurrent retro proposals #5571, Retro post-script should deduplicate proposals against recently-filed bot issues before creating new ones #5570, Retro post-script should add evidence as comments on parent issues instead of filing new issues #5171
• In progress: PR feat(retro): add flapping detection to retro analysis skill (#5512) #5513 (flapping detection in retro analysis)
• Source: Meeting notes

[fullsend-ai-prioritize]

RICE Priority Score: 1.4

Score breakdown

Dimension	Score	Reasoning
Reach	2	The retro agent runs across multiple target repositories and organizations. The noise problem is platform-wide, as evidenced by five related issues (#5619, #5592, #5571, #5570, #5171) tracking noise reduction. Most active users of the retro agent are affected by unnecessary issue filings cluttering their repos.
Impact	1.5	Noisy issue filings degrade the signal-to-noise ratio in target repositories, but workarounds exist (post-script dedup guards, retraction logic). The gating skill would be a significant efficiency gain by preventing wasted compute and race conditions from filing-then-retracting, but it does not unblock core functionality.
Confidence	0.7	The issue is well-structured with clear acceptance criteria, options considered with rationale, and multiple related issues providing context. However, it was auto-generated from meeting notes and has no comments or labels, leaving some uncertainty about exact scope and priority alignment with the team.
Effort	1.5	Requires designing a new configurable gating skill, integrating it into the existing retro analysis pipeline, implementing rule evaluation with accept/reject reasoning, and ensuring backward compatibility with the current 'file all' behavior. This touches multiple components and involves design work for the configuration system, but the scope is well-bounded.

Formula: (2 x 1.5 x 0.7) / 1.5 = 1.4