Skip to content

fix(jobs): keep the blind-turn guard's rc=7 out of bg job exit codes; map devin's non-interactive reject to permission-blocked - #37

Merged
alexgreensh merged 3 commits into
alexgreensh:mainfrom
danikdanik:fix/job-rc-guard-leak
Oct 10, 2026
Merged

alexgreensh merged 3 commits into
alexgreensh:mainfrom
danikdanik:fix/job-rc-guard-leak

Conversation

@danikdanik

@danikdanik danikdanik commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Hey Alex, this one had me going for a while. Two devin jobs came back "failed" while their work was actually fine, and it turned out devin wasn't the cause.

What

  • A bg job's child process no longer runs the blind-turn guard, so the job's exit code is the delegate's own again.
  • After exit, _supervise recognizes devin's newer "rejected a tool call that requires confirmation ... non-interactive mode" warning and records permission-blocked (exit 3, reason permission-blocked:noninteractive-reject), not done?. The check is devin lane only, tail-anchored, exit-0 only, and skipped if the delegate ended on OSRC::DONE.
  • New suite test_job_rc_guard_leak.sh, with scrubbed replays of the two jobs as fixtures, registered in conformance.sh. There's also a short note in references/jobs-and-safety.md.

Why

On 0.13.4 two devin swe-2 bg jobs ended as status=failed, exit=7, reason=exit-nonzero:rc=7:

  1. bg run -m swe-2, a read-only review. result returned a complete review (a verdict plus 8 findings), and classify said REUSE-OUTPUT false-stall:deliverable.
  2. bg edit -m swe-2. The edits landed correctly. The delegate stopped when it tried to run the tests, and the log ends with devin's non-interactive warning.

Devin exited 0 both times. The 7 comes from the blind-turn guard: run_job runs the job by re-entering outsourcerer.sh <verb> ... under _supervise, and main() runs the guard at the end of that child. If anything in the fleet looked like it needed attention, the guard returned 7 and _supervise recorded it as the delegate's failure. Both out.logs end with the guard notice, printed from inside the job. (What the guard flagged in my case was my own session, fixed separately in #36. The leak holds for any flagged item, though.)

On the second part: I ran devin 3000.11.1 directly. In accept-edits, a command it wants confirmed is rejected, the warning goes to stderr, and devin exits 0. It doesn't hang like the older chisel::repl::handler: Print mode: ... case that _printmode_needle matches. So with only the leak fixed, the edit job would have read done?, with nothing saying the test step never ran. references/jobs-and-safety.md already names permission-blocked as the state for exactly this wall.

How

  • main(): return $_cmd_rc before the guard when the child carries --osrc-job-child-internal, a private argv sentinel run_job puts first in the child's command line and main() consumes next to --osrc-preflight-internal. Keying on OSRC_JOB_DIR would not work: the variable is functional state the child legitimately reads (capture dirs) and it is inheritable -- run_job exports it into the child, so a delegate that runs outsourcerer itself would have its guard silently disabled, and delegate_codex's own error text tells users to export it. An argv sentinel cannot leak through the environment.
  • _noninteractive_reject_needle is assembled at runtime, like _printmode_needle, so reading the script can't trip it. The fixture keeps a @@DEVIN_REJECT@@ placeholder that the test fills from a rot13-encoded recorded line after asserting the line equals the needle, and the doc paraphrases it, so no file in the repo carries it verbatim and a wrong needle cannot fill its own fixture.
  • The check runs post-exit, after the empty-output and BLOCKED-marker checks and before the exit-code checks. It only maps a clean exit 0 -- a nonzero devin exit keeps its real code on the exit-nonzero path. Lane comes from meta.json, falling back to the OUTSOURCERER_PROVIDER env run_job exports (meta.json is jq-written; without jq it never exists). Respects OSRC_NO_PRINTMODE_ABORT=1.
  • classify is untouched. The edit job's landed work still classifies as REUSE-OUTPUT.

How to reproduce

The leak:

  1. Get any fleet item into a state the guard flags (MAYBE STUCK or WAITING ON YOU).
  2. outsourcerer.sh bg run -m swe-2 "<any short read-only question>", then look at the job dir: status is failed, exit is 7, reason is exit-nonzero:rc=7, and out.log ends with >>> [outsourcerer] blind-turn guard: ....

The devin exit code:

mkdir -p /tmp/p/sub && cd /tmp/p
devin --model swe-2-high --permission-mode accept-edits --respect-workspace-trust false \
  -p "Run exactly this single shell command and report its output: cd sub && rm -f nothing.txt && npm --version"
echo "rc=$?"   # prints the reject warning on stderr, then rc=0

Without devin: bash plugins/outsourcerer/skills/outsourcerer/scripts/tests/test_job_rc_guard_leak.sh. Run against the previous revision's script it fails 8 checks (an inherited OSRC_JOB_DIR disabled the guard, the missing-meta case read done?, a nonzero exit was rewritten to 3, and the sentinel was absent from run_job's argv); on this branch it's 19/19.

Tests

  • test_job_rc_guard_leak.sh: 19/19. It covers the control (the guard still returns 7 at orchestrator level), the inherited-env non-exemption, both recorded jobs, the warning replayed on stderr, a mid-log line pushed out of the tail, OSRC::DONE winning, a non-devin lane quoting the line, the opt-out, provider-only and missing meta.json, a non-devin provider env, a nonzero exit with the reject in the tail, the needle not appearing verbatim in the script, and run_job's child argv carrying the sentinel.
  • Also green standalone: test_blind_turn_guard, test_preflight_env_isolation, test_preflight_guard_exempt, test_job_lifecycle, test_supervise_pgroup_kill, test_autodetach, test_hardening, test_selfcontained_hardening.
  • Full conformance.sh: the only failures are the ones main already has on my machine (test_autodetach, test_cline_lane, test_devin_plan_quota, test_lane_plan_limit, test_failover_pick; test_heartbeat_ownership is flaky on main too).

run_job re-enters the script under _supervise with OSRC_JOB_DIR set, and
main() ran the blind-turn guard at the end of that child, so a delegate
that exited 0 was recorded as failed / exit-nonzero:rc=7 whenever unrelated
fleet state needed attention. Skip the guard inside a supervised job child.

Also map devin's non-interactive tool-call reject (devin 3000.11 prints a
warning and exits 0 instead of hanging) to permission-blocked after exit,
devin lane only, tail-anchored, unless the delegate ended on OSRC::DONE.
Scrubbed replays of the two affected jobs are the test fixtures.
OSRC_JOB_DIR is functional state the child legitimately reads (capture
dirs) and it is inheritable: run_job exports it into the child, so a
delegate that runs outsourcerer itself would have its blind-turn guard
silently disabled, and delegate_codex's own error text tells users to
export it. Move the exemption to --osrc-job-child-internal, consumed in
main() the same way --osrc-preflight-internal is.

Also: lane detection falls back to OUTSOURCERER_PROVIDER when meta.json
is absent (it is jq-written, so no jq means no file and the mapping
never fired), and the devin non-interactive reject maps to
permission-blocked only on a clean exit 0; a nonzero exit keeps its
real code on the exit-nonzero path.

test_job_rc_guard_leak: fill @@DEVIN_REJECT@@ from a rot13-encoded
recorded line the needle is pinned against (a wrong needle can no
longer fill its own fixture); new cases pin that an inherited
OSRC_JOB_DIR does not exempt, provider-only and missing meta.json
still map, a nonzero exit with the reject in the tail keeps its real
code, and run_job's child argv carries the sentinel. Unset OSRC_JOB_DIR
and OSRC_STREAM so the suite is valid inside a job too.
Unset OUTSOURCERER_PROVIDER too (a suite run inside a devin job would
inherit it and feed the provider-env fallback), carry OSRC_STREAM in
the inherited-env case since a real nested delegate gets both, and pin
that a non-devin provider env does not map the reject.
@alexgreensh
alexgreensh merged commit 3ed96f2 into alexgreensh:main Oct 10, 2026
2 of 3 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 10, 2026
@alexgreensh

Copy link
Copy Markdown
Owner

Thank you, Dani. Merged in 0.13.5. The argv sentinel instead of OSRC_JOB_DIR was a great call, and the scrubbed replays made this easy to verify. One follow-up landed with it: a job that already ended permission-blocked now shows as stopped in the fleet view, so the guard doesn't ask you to answer a pane for a process that has exited.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants