Repository navigation
test(loop): exercise production functions in unit suites - #380
Conversation
Cross-family review — opencode (zai/glm), post-PR pass on 538372dThe Review: PR #380 — exercise production functions in unit suitesContract check (criterion 1): All three files load the real functions via Criterion 3 (no copied bodies): All inline copies ( Test 26 fixture change (5→4 indicators): Justified, not gaming. The real function's ≥5 breaker force-exits regardless of EXIT_SIGNAL, so the old fixture would legitimately fail; 4 indicators still exercises the test's intent (≥2 heuristics + explicit Findings
Vacuous-pass audit: No test becomes vacuous. Tests 36–41 switching from the synthetic Verdict: APPROVE Maintainer triage
|
…tion tests (frankbria#374) With the real should_exit_gracefully under test, a 5->500 threshold mutant of its safety breaker still passed (opencode review). Add the positive case next to the explicit-false boundary test, and record the real-function rule in CLAUDE.md's Test Suite section.
Demo verification (Showboat,
|
| Real function | Mutation | Suite | this PR | main |
|---|---|---|---|---|
should_exit_gracefully |
test-saturation -ge → -gt |
test_exit_detection | caught (1) | survived |
should_exit_gracefully |
safety breaker 5 → 500 | test_exit_detection | caught (1)* | survived |
_count_blocking_unchecked |
optional-section skip removed | test_exit_detection | caught (5) | survived |
can_make_call |
invocation limit -ge → -gt |
test_rate_limiting | caught (2) | survived |
increment_call_counter |
+1 → +2 |
test_rate_limiting | caught (5) | survived |
extract_token_usage |
output tokens dropped | test_rate_limiting | caught (3) | survived |
reset_session |
exit-signal reset removed | test_session_continuity | caught (2) | survived |
* caught by the test added in 29be2ae, after the opencode review.
| Criterion | Evidence | Status |
|---|---|---|
| Each file loads the real function | for function_name in … sed-extract + eval in setup(): exit detection (3 fns), rate limiting (4), session (2); setup fails on an empty body or an undefined function |
VERIFIED |
| A change to each real function fails a test in its file | table above: 7/7 caught on this branch, 0/7 on main | VERIFIED |
| No copied function bodies | the only test functions sharing a ralph_loop.sh name are log_status test doubles (a : stub, and a 3-line echo double in session_continuity), not copies of the 20-line production function |
VERIFIED |
Local: npm test 1355/1355.
|
Thanks @widechaos, merged as ff9a900. Your extraction approach held up under every mutant we threw at it: all 7 caught on this branch, none on main. I pushed one follow-up commit (29be2ae). It adds a positive test for the 5-indicator safety breaker, which your test-26 fix correctly stepped around but nothing else pinned, and a CLAUDE.md note so future tests load real functions too. |
|
Post-merge disposition: filed #384 ([P8.5] skip the GLM review job on fork PRs, which have no secrets; it failed on both pushes here). Dropped as already handled: the mis-scoped-extraction suggestion (the |
The exit-detection, rate-limit, and session-reset suites tested inline copies of loop functions, so production regressions could leave these tests green. This loads the functions directly from
ralph_loop.shusing the existing extraction pattern and removes the copies, including the permission-denial variant. Production behavior is unchanged.The explicit-false exit-signal fixture now stays below the independent five-indicator safety breaker, so it tests its intended condition against the real implementation.
Closes #374.
Validation: