Skip to content

fix: handle exhausted provider funding and surface consolidation failures - #239

Open
FocalFactotum wants to merge 3 commits into
kunchenguid:mainfrom
FocalFactotum:fm/upstream-bp-230
Open

FocalFactotum wants to merge 3 commits into
kunchenguid:mainfrom
FocalFactotum:fm/upstream-bp-230

Conversation

@FocalFactotum

@FocalFactotum FocalFactotum commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Closes #230

What Changed

  • Fall through to another agent on explicit billing, quota, or credit exhaustion, caching the refusal as model-unavailable while excluding transient rate limits.
  • Preserve gap consolidation failures in proposal summary notes when consolidation degrades to lexical identity.
  • Document both behaviors and add regression coverage.

Risk Assessment

✅ Low: The change is bounded, handles the reported billing refusal through existing fallthrough and cache behavior, and preserves consolidation degradation in proposal notes.

Testing

Four focused regressions passed, and a manual builder-to-renderer check captured the named degradation in terminal output. No full suite or static checks ran. Live CLI/provider scenarios remain unverified because required tools are absent and provisioning them is forbidden. No screenshots were generated, as required.

  • Live validation: ⚠️ inconclusive - 0 of 3 scenarios driven live against the product
Scenario Result Live Evidence
Analysis and synthesis recover from exhausted funds and cache the unavailable candidate ⏸️ untested no PATH checks found neither acpx nor opencode. The regression exercises simulated paid-call failures, not real providers. Downloads are forbidden by the intent; live validation requires pre-provisioned…
Ordinary upstream errors and transient rate limits do not trigger candidate fallthrough ⏸️ untested no Classifier checks exercised the adversarial error shapes, but real upstream calls require unavailable acpx and opencode tools and disposable provider credentials. Installing those tools is forbidden b…
A degraded consolidation appears in the proposal summary ⏸️ untested no Executed the real proposal builder and terminal renderer against disposable staged memory and observed the degradation note. A complete CLI run could not be driven without acpx and an isolated authent…
Evidence: Rendered proposal summary with consolidation degradation note

proposal · backpass-stage-SwzSd5 · AGENTS.md · 0 edit(s) from 2 session(s)
  budget [................................] 8 -> 8 / 5,000 tok
  evidence: 0 positive · 0 negative · 0 gap clusters

  no edits proposed - the evidence did not clear the thresholds this run
  note: gap consolidation degraded to lexical gap identity: no parseable merge list

Review and apply with `backpass apply` (nothing has been written).
- Outcome: ⚠️ 1 warning across 1 run (1m16s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

⏭️ **Rebase** - skipped

Step was skipped.

✅ **Review** - passed

✅ No issues found.

⚠️ **Test** - 1 warning
  • ⚠️ live validation verdict: inconclusive (0 of 3 scenarios were driven live against the product); untested: Analysis and synthesis recover from exhausted funds and cache the unavailable candidate, Ordinary upstream errors and transient rate limits do not trigger candidate fallthrough, A degraded consolidation appears in the proposal summary
  • Live validation: ⚠️ inconclusive - 0 of 3 scenarios driven live against the product
Scenario Result Live Evidence
Analysis and synthesis recover from exhausted funds and cache the unavailable candidate ⏸️ untested no PATH checks found neither acpx nor opencode. The regression exercises simulated paid-call failures, not real providers. Downloads are forbidden by the intent; live validation requires pre-provisioned…
Ordinary upstream errors and transient rate limits do not trigger candidate fallthrough ⏸️ untested no Classifier checks exercised the adversarial error shapes, but real upstream calls require unavailable acpx and opencode tools and disposable provider credentials. Installing those tools is forbidden b…
A degraded consolidation appears in the proposal summary ⏸️ untested no Executed the real proposal builder and terminal renderer against disposable staged memory and observed the degradation note. A complete CLI run could not be driven without acpx and an isolated authent…
  • command -v node acpx opencode codex
  • node --import ./test/helpers/hermetic-config.js --test --test-name-pattern='falls through a successful probe when the paid call exhausts funds|billing classification excludes|a degraded consolidation is retained' test/agents.test.js test/proposal.test.js
  • Executed buildProposal and printProposal with disposable staged memory and a consolidation failure; captured terminal output.
  • Removed disposable workspace data and confirmed no worktree changes.
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@greptile-apps

greptile-apps Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge.

Reviews (1) · Last reviewed commit: "no-mistakes(document): Clarify billing r..." · Reviewed by Greptile

@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate: thanks @FocalFactotum for taking on #230. I read the diff on c927a2f and the approach looks right: the new BILLING_REJECTED pattern catches the real "Insufficient account funds" shape from the report and other explicit funding refusals, a bare 429 or rate limit still fails rather than skipping a candidate, the refusal reuses the existing model-unavailable fallthrough and negative cache, and a failed consolidation now shows up in the proposal summary notes instead of only a warning. CI is green.

This is waiting on you for live evidence. Your PR body shows 0 of 3 scenarios driven live. Could you run it against a real setup and post the output here? The main one is a ladder where the first candidate is logged in but out of funds and a later one works: the run should name the refusal, fall through, and the summary should show the consolidation note if it degraded. A quick check that an ordinary upstream error still fails rather than falling through would round it out. No rebase needed from you.

VISION per-rule:

  • Evidence is the only currency: aligns (consolidation runs on the funded candidate instead of degrading to lexical identity).
  • The human owns the weights: aligns (no apply or write change).
  • Nothing the model says is taken on faith: aligns (classification is on the harness's stderr text, not model narration).
  • The budget is the constraint: aligns (no memory-file growth).
  • It reads what you already have and owns nothing: aligns (no key or service; uses the user's own candidates).
  • Failure is loud and named: aligns (the refusal is named and the degraded consolidation is in the run summary).
  • Scope: aligns (agent ladder classification only).

Contract-class: restore. Falling through when a candidate cannot serve the call is the existing specified ladder behavior, and the README already promised fallthrough for an out-of-credits account; this makes the explicit refusal shape take that same path. Once live evidence is posted this can merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent ladder does not fall through on billing/credit exhaustion (opencode Zen 'Insufficient account funds')

2 participants