Skip to content

fix(proposal): drop only a still-ambiguous edit instead of the whole proposal - #225

Open
jadhavgaurav wants to merge 2 commits into
kunchenguid:mainfrom
jadhavgaurav:fix/drop-ambiguous-anchor-200
Open

jadhavgaurav wants to merge 2 commits into
kunchenguid:mainfrom
jadhavgaurav:fix/drop-ambiguous-anchor-200

Conversation

@jadhavgaurav

@jadhavgaurav jadhavgaurav commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Intent

Fix issue #200: one non-unique hunk anchor ("find" text appears N times; must be unique) discards the whole synthesis proposal after the re-prompts are spent. Tell the re-prompt where each occurrence is and what repeats it, and when the re-prompts are exhausted drop only the still-ambiguous edit, naming why in a warning and the proposal notes, keeping every other edit that cleared its gates. Other violations still fail the run loudly. Closes #200.

What Changed

  • src/proposal.js: replaceOnce now raises an AMBIGUOUS_FIND-coded error that reports the 1-based line of each occurrence of a non-unique find anchor (capped, with a "+N more" tail), so a re-prompt can point at exactly what repeats. buildProposal gains a dropAmbiguous option: when set, an edit whose anchor is still ambiguous is spliced out of accepted and recorded in a new dropped list (with id, title, and reason) instead of failing the whole proposal; its target files are staged separately so a dropped edit leaves no partially-applied file behind. Other violations are unaffected and still fail the run.
  • src/synthesize.js: in annotateLoop, once the re-prompt budget (ANNOTATE_TURNS) is exhausted and violations remain, the loop rebuilds the proposal with dropAmbiguous: true; if that drops anything, the retried proposal is used and a warning is emitted per dropped edit via warn(...).
  • README.md: documents that an edit whose anchor is still ambiguous after re-prompts is dropped alone, named in a warning and in the proposal notes, while the rest of the proposal proceeds to review.
  • Adds coverage in test/proposal.test.js and test/synthesize.test.js for the new drop-only-the-ambiguous-edit behavior.
  • Review follow-up: mergeNestedProposals (src/nested.js) gives each nested edit an id no edit already holds. A dropped middle edit leaves ids like [e1, e3], and numbering by count gave the first nested edit a second e3; review and apply key decisions by edit.id, so a rejected change could have been applied.
  • Review follow-up: test/nested.test.js runs a drop together with a nested merge and applies decisions by id; test/proposal.test.js adds a multi-target extract where the first target applies and a later one is ambiguous.

Risk Assessment

✅ Low: The change is well-bounded to the stated fix (issue #200), gated precisely to fire only after re-prompts are exhausted, correctly excludes dropped edits from all downstream accounting (edits list, budget, token deltas), leaves non-ambiguous violations fully blocking, and is backed by behavioral tests with an independent oracle (hand-computed line numbers) that verify both the proposal-level and synthesis-loop-level behavior; it also incidentally fixes a pre-existing partial-apply leak into the running map for multi-target edits.

Testing

  • pnpm run check (eslint, prettier, tsc, 958 tests) passes on 1e46faf.
  • test/nested.test.js: a root with a dropped middle edit (e1, e3) merged with a nested edit gets unique ids (e1, e3, e4), and accepting e1 and e4 while rejecting e3 writes the nested file and not the rejected root edit. With only src/nested.js reverted (saved patch, git apply -R) it fails with edit ids must be unique: e1, e3, e3.
  • test/proposal.test.js: a dropped extract that spans AGENTS.md, an existing skill and a created skill leaves the surviving proposal with the budget delta at 0, no skill extraction counted, no created skill, and apply changes none of the three files for it. With the per-edit next staging changed to write running after each target (a temporary edit to src/proposal.js, reverted), this test fails and nothing else does.

Pipeline

Updates from git push no-mistakes

✅ **Intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • Live validation: ✅ go - 6 of 6 scenarios driven live against the product
Scenario Result Live Evidence
Drop a middle root edit then merge a nested proposal: nested edit gets an id no existing edit holds ✅ pass live test/nested.test.js: "a root edit dropped from the middle never leaves its id to a nested edit" — asserts ids=[e1,e3,e4], unique, and nested.edits=[e4]
Apply decisions by edit id after drop+merge: rejecting the surviving root edit (e3) does not also reject the nested edit (e4) that took its old id ✅ pass live Same test: applyDecisions({e1: accepted, e3: rejected, e4: accepted}) → accepted=2, rejected=1; AGENTS.md keeps e1's change, drops e3's, and the API file gets e4's nested edit, read back from real wri…
Multi-target extract where the first target applies and a later target is ambiguous: neither file changes, and the dropped edit's token cost and created skill are excluded ✅ pass live test/proposal.test.js: "a dropped extract that spans several files leaves none of them changed and costs nothing" — budget.delta=0, stats.skillExtractions=0, no skills listed, targetFiles excludes the…
Full synthesis pipeline (re-prompt loop) drops only the still-ambiguous edit after retries, keeping the other edits and naming the drop in notes/re-prompt text ✅ pass live test/synthesize.test.js: "one still-ambiguous edit is dropped and named after the re-prompts, and the other edits survive" — proposal.edits titles=[copy,tail], notes contain "dropped edit ... (rewrite…
Adversarial: dropping the ambiguous edit must not mask a separate, real gate violation (e.g. missing evidence) on another edit ✅ pass live test/synthesize.test.js: "dropping an ambiguous edit never hides another gate violation" — run() rejects with ProposalViolation citing the no-verbatim-evidence violation, and confirms the ambiguous-fi…
Drop behavior itself is unchanged: ambiguous-find detection still reports every occurrence by line and still gates normally outside dropAmbiguous mode ✅ pass live test/proposal.test.js: "an ambiguous find names every occurrence by line..." and "a non-unique find after an earlier edit lands is a violation..." both pass against real buildProposal/applyEdit calls
  • node --test test/nested.test.js test/proposal.test.js test/synthesize.test.js (163 pass, 0 fail)
  • git status --porcelain (clean worktree, no transient artifacts)
✅ **Document** - passed

✅ No issues found.

⚠️ **Lint** - 1 info
  • ℹ️ package.json - Lint/format/typecheck could not be run: node_modules is not installed in this worktree and no eslint/prettier binaries are on PATH or available locally; installing dependencies is outside this phase's scope.
✅ **Push** - passed

✅ No issues found.

Issues

Closes #200

…proposal

A hunk's find text is unique in the file as read, but an earlier edit can
repeat it once applied. The resulting 'must be unique' violation survived
every re-prompt and discarded every other measured edit with it.

Name each occurrence by line and say an earlier change repeats it, so the
re-prompt can act on it. When the re-prompts are spent, drop only the edit
that is still ambiguous, warn and record why in the proposal notes, and keep
the rest. Other gate violations still fail the run loudly. Stage each edit's
effect on the running text so a dropped edit leaves nothing half applied.

Closes kunchenguid#200

Signed-off-by: Gaurav Jadhav <gaurav.vjadhav01@gmail.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T10:00:37.322776Z 97833f7 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@greptile-apps

greptile-apps Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no blocking issues remain.

Reviews (2) · Last reviewed commit: "fix(nested): never reuse a live edit id ..." · Reviewed by Greptile

Comment thread test/proposal.test.js

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 97833f7adc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/proposal.js
const ambiguous = err.code === AMBIGUOUS_FIND;
if (ambiguous && dropAmbiguous) {
// Only the still-ambiguous edit goes; every other edit already cleared its gates.
accepted.splice(accepted.indexOf(edit), 1);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep surviving IDs unique across nested merges

When a root run drops a non-final edit (for example e2 from [e1, e2, e3]) and also has a nested-memory proposal, this splice leaves the root IDs as [e1, e3]; mergeNestedProposals then assigns its first nested edit e${edits.length + 1} at src/nested.js:398, producing a second e3. The review/apply paths key decisions by edit.id (src/apply/writer.js:305-306), so those two edits cannot be accepted or rejected independently and a file change the reviewer rejected can be applied; renumber survivors or allocate nested IDs from an actually unused ID.

Useful? React with 👍 / 👎.

@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate: thanks @jadhavgaurav for picking up #200. Putting line numbers in the re-prompt is a clear win, and staging each edit's targets in next so a dropped edit leaves nothing half-applied is a nice touch. I reviewed the diff and approved CI for this fork, so the checks are running now.

This is waiting on you first. Two review findings on 97833f7 need a fix or a reply:

  1. Codex P1, duplicate edit ids. Dropping a middle edit (say e2 of e1..e3) leaves [e1, e3]. mergeNestedProposals (src/nested.js) then gives the first nested edit e${edits.length + 1}, which is a second e3. Review and apply key decisions by edit.id (src/apply/writer.js), so a change the reviewer rejected could still be applied. That breaks per-edit review, so it has to be fixed, with a test that runs a drop together with a nested merge.
  2. Greptile P2, multi-target drop test. Please add an extract edit where the first target applies and a later target is ambiguous. Assert that neither file changes and that the dropped edit's token cost and created skill are excluded.

CI and the no-mistakes check also need to be green.

Heads-up so nothing surprises you later: once those are fixed, this goes to Kun for a decision before it merges. On main, the README says that when the re-prompts are spent, backpass fails loudly and keeps the rejected proposal. This PR replaces that documented default with a partial proposal (one edit dropped and named) that goes on to review. That may well be the right call, but it changes the default path, so it's Kun's decision.

contract-class: new-default. The unconfigured run now hands a partial proposal to review where the specified path failed loudly. A bugfix motive doesn't make a replaced default into a restore. (Our earlier issue triage called this restore; on reading the tip against the README, I'm correcting that.)

VISION per-rule (main 2125624):

  • Evidence is the only currency: aligns (surviving edits already cleared the evidence gates)
  • The human owns the weights: does not align until finding 1 is fixed (duplicate ids can break per-edit accept and reject); otherwise it aligns, since nothing is written without review
  • Nothing the model says is taken on faith: aligns (the drop comes from a measured apply failure)
  • The budget is the constraint, not a setting: aligns (dropped edits are excluded from the deltas)
  • It reads what you already have and owns nothing: aligns
  • Failure is loud and named: cannot tell yet. The drop is named in a warning and the notes, so it isn't silent truncation, but whether a named partial beats a terminal failure is the captain's call.
  • Scope: aligns

Dropping a middle edit (e2 of e1..e3) leaves [e1, e3]; the first nested
edit then took e${edits.length + 1}, a second e3. Review and apply key
decisions by edit.id, so a rejected change could be applied. Allocate
nested ids that no edit already holds.

Also pin that a dropped edit spanning several files leaves none of them
changed and its token cost and created skill out of the proposal.

Closes kunchenguid#200

Signed-off-by: Gaurav Jadhav <gaurav.vjadhav01@gmail.com>
@jadhavgaurav jadhavgaurav changed the title fix(proposal): drop only a still-ambiguous edit instead of the whole proposal fix(nested): never reuse a live edit id when merging nested proposals Oct 23, 2026
@jadhavgaurav jadhavgaurav changed the title fix(nested): never reuse a live edit id when merging nested proposals fix(proposal): drop only a still-ambiguous edit instead of the whole proposal Oct 23, 2026
@jadhavgaurav

Copy link
Copy Markdown
Contributor Author

Thanks. Both findings are addressed in 1e46faf.

  1. Duplicate ids: mergeNestedProposals now gives each nested edit an id that no edit already holds, so a root left with e1, e3 after a drop gets e4 for its first nested edit instead of a second e3. I left the ids the drop leaves behind as they are, so the dropped edit's id in the warning and notes still points at nothing else. The new test in test/nested.test.js drops the middle of three root edits, merges a nested edit, then accepts e1 and e4 and rejects e3: the nested file is written and the rejected root edit is not. With only src/nested.js reverted it fails with edit ids must be unique: e1, e3, e3.
  2. Multi-target test: test/proposal.test.js now has an extract over AGENTS.md, an existing skill and a created skill, whose skill anchor becomes ambiguous after an earlier edit copies the line. The proposal keeps only the earlier edit, the budget delta stays 0, no extraction or created skill is counted, and applying leaves the dropped extract's parts out of all three files. Changing the staging so each target is written to the running text as it applies makes that test fail.

pnpm run check passes locally (958 tests). Not run: GitHub Actions on this head (still waiting for approval when I looked), and nothing here ran against a real harness, only the fake acpx.

@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate: thanks @jadhavgaurav, both findings are handled well in 1e46faf. Nested edits now never reuse a live id, and the new test proves a rejected root edit isn't applied after a drop plus a nested merge. The multi-target extract test covers the half-applied case, the budget, and the created skill. I reviewed the diff and approved CI for the new head, and CI on ubuntu and macOS, the generated-files guard, and no-mistakes are all green.

This is waiting on you for one last thing: live evidence. Everything so far ran against the fake acpx. Could you run a real backpass synthesis with a real harness on a repo where one edit's anchor is ambiguous after the re-prompts (for example, a memory file with a repeated line the edit targets) and post the output? It's most useful to show the warning naming the dropped edit and the apply review listing the rest with unique ids. If you can't get a real run to hit an ambiguous anchor, just say so.

After that, it goes to Kun for a decision, not back to you. As noted before, the README says that once the re-prompts are spent, backpass fails loudly and keeps the rejected proposal, and this PR replaces that documented default with a partial proposal.

contract-class: new-default. The unconfigured run now produces a partial proposal where the documented default fails loudly.

VISION per-rule (main 3882b86, head 1e46faf):

  • Evidence is the only currency: aligns (no change to the evidence bar)
  • The human owns the weights: aligns (every surviving edit still goes through per-edit review, and ids are now unique so decisions land on the right edit)
  • Nothing the model says is taken on faith: aligns (an ambiguous edit is still refused, not guessed)
  • The budget is the constraint, not a setting: aligns (a dropped edit's tokens and created skill are excluded)
  • It reads what you already have and owns nothing: aligns
  • Failure is loud and named: cannot tell yet, and that's Kun's call: the drop is named in a warning, but it replaces the documented loud failure
  • Scope: aligns

@jadhavgaurav

Copy link
Copy Markdown
Contributor Author

Live runs against a real harness, on head 1e46faf. Short answer: I could not get a real run to hit an ambiguous anchor, so there is no warning naming a dropped edit and no apply listing after a drop to show. Details of what I tried and what the runs did show are below.

Setup

  • macOS 27.0.1, Node 24.14.0, Claude Code 2.1.309, acpx@0.20.0 (installed to a scratch prefix and put on PATH for the runs only), backpass 0.1.38 from this branch (node bin/backpass.js).
  • A separate Claude Code sign-in on the same account (own CLAUDE_CONFIG_DIR), so these are real model calls, not the fake acpx. ladders in the scratch .backpassrc.json is restricted to claude (claude-sonnet-5 for analysis, claude-opus-5 for synthesis, effort high by default).
  • Target is a scratch git repo with an AGENTS.md and two small skills under .agents/skills/. Input is ten of my own Claude Code transcripts from unrelated projects, copied with their cwd rewritten to the scratch repo so they associate with it. XDG_CACHE_HOME and XDG_CONFIG_HOME point at scratch dirs. I did not run backpass apply; nothing was written anywhere.
  • Same command for all three runs (analysis reruns each time because the memory file changes):
node bin/backpass.js --harness claude --analysis-agent claude --jobs 4 --since all --limit 10

What I tried to provoke

The ambiguity needs a hunk whose find is unique in the original but appears twice after an earlier change. The shapes I tried, each against a freshly edited AGENTS.md:

  1. A 15 bullet file with the same bullet (Keep answers short and concrete.) in two sections, and the evidenced rules last.
  2. The same duplicate twice in a row at the bottom, with the evidenced rules filed under a late Release section and an early Before you finish section, to invite a move upward.
  3. No duplicate; the three evidenced rules filed last under Release, again inviting a move upward.

Run 1 (duplicate bullet in two sections)

warn synthesis edited the staging copy again; re-measuring and re-annotating (annotation attempt 1 of 3 is still unspent)
warn synthesis violated 1 gate(s); re-prompting with the exact violations
warn synthesis edited the staging copy again; re-measuring and re-annotating (annotation attempt 2 of 3 is still unspent)

proposal · bpdemo-r2 · AGENTS.md · 3 edit(s) from 8 session(s) · interactive 8 · non-interactive 0
  e1 REWRITE  Sharpen "Keep answers short and concrete" to name the failure shape it must rule out (+30 tok, 2 transcript(s))

(trimmed: edits e2 and e3 were two more REWRITEs, notes omitted.) The one violation was not an ambiguous anchor. The re-prompt text saved in .backpass/prompts/synthesis-annotate-3.md says:

- edit e1 ("Dedupe the two copies of "Keep answers short and concrete" into one sharpened rule") deletes [AG-010] "- Keep answers short and concrete." backed by 0 session(s) of harm-class negative evidence; removing an instruction needs 2, and non-compliance never counts - revert the deletion

So the model tried to merge the two copies, hit the harm-evidence gate, and rewrote one copy instead. Each copy is widened with its surrounding context, so no anchor stayed ambiguous.

Run 2 (duplicate twice at the bottom)

warn synthesis violated 1 gate(s); re-prompting with the exact violations
warn synthesis edited the staging copy again; re-measuring and re-annotating (annotation attempt 2 of 3 is still unspent)

proposal · bpdemo-r2 · AGENTS.md · 2 edit(s) from 8 session(s) · interactive 8 · non-interactive 0
  e1 MOVE     Reposition "Keep answers short and concrete." from Release to Before you finish (0 tok, 3 transcript(s))
  e2 ADD      Add a Shell rule: use absolute paths because the working directory persists (+34 tok, 3 transcript(s))

Again the violation was the harm-evidence gate on a dedupe attempt (synthesis-annotate-2.md: deletes [AG-014] "- Keep answers short and concrete." backed by 0 session(s) of harm-class negative evidence ... revert the deletion (2 unit(s) affected)), not an ambiguous anchor. The surviving proposal has unique ids e1, e2.

Run 3 (no duplicate, rules filed last)

· synthesizing with claude (claude-opus-5) effort=high

proposal · bpdemo-r2 · AGENTS.md · 2 edit(s) from 8 session(s) · interactive 8 · non-interactive 0
  e1 MOVE     Reposition the test-suite rule under "Before you finish" and lift "Keep answers short and concrete" to the top of the file (0 tok, 2 transcript(s))
  e2 ADD      Add a Shell rule: use absolute paths instead of a relative `cd` (+31 tok, 3 transcript(s))

No warnings. The model did make the move I was hoping for, but proposal.json shows why it did not become ambiguous: the two adjacent bullets it moved were deleted as one two-line hunk, and re-added as two separate single lines in other places, so the deletion's find (two lines) never appeared in the text an earlier hunk added:

H1 find:    "## Before you finish\n- Run the linter before reporting a change as done.\n"
H1 replace: "- Keep answers short and concrete.\n\n## Before you finish\n- Run the linter before reporting a change as done.\n- Run the full test suite before claiming a fix works.\n"
H3 find:    "- Run the full test suite before claiming a fix works.\n- Keep answers short and concrete.\n"
H3 replace: ""

A single unique line moved upward, or a contiguous block moved intact, would be the shape that trips it, and the model did not choose that shape in any of the three runs.

What this does and does not show

  • Three full real synthesis runs on this head completed end to end with this change in place: proposals produced, edit ids unique, and the re-prompt loop behaved as before for a gate violation that is not an ambiguous anchor.
  • The new drop path (dropping edit ... : ... appears N times ...) was not exercised live, so I have no real warning line or apply listing to post for it.
  • I stopped after three full attempts to bound token use. This is macOS only, one model (opus-5), and the memory file was written by me to invite the shape, so it says nothing about how often real memory files hit it. The One non-unique hunk anchor discards the whole synthesis proposal #200 reporter saw it with a different synthesis model on long skill bodies.

@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate: thanks @jadhavgaurav, three real synthesis runs on 1e46faf are really useful. They show the change sits safely on the normal path (proposals produced, edit ids unique, the gate re-prompt loop unchanged), and you were clear that no real run hit an ambiguous anchor, so the new drop path is covered by the tests rather than live. That's a fair place to stop, and there's nothing more we need from you.

This is now with Kun for a decision, not waiting on you. As flagged before, the README says that once the re-prompts are spent backpass fails loudly and keeps the rejected proposal, and this PR replaces that documented default with a partial proposal (the ambiguous edit dropped and named) that goes on to review. CI is green on ubuntu, macOS, the generated-files guard, and no-mistakes, and it merges cleanly onto main 1a1a175. If anything changes we'll say so here.

contract-class: new-default. The unconfigured run now produces a partial proposal where the documented default fails loudly.

VISION per-rule (main 1a1a175, head 1e46faf):

  • Evidence is the only currency: aligns (no change to the evidence bar)
  • The human owns the weights: aligns (every surviving edit still goes through per-edit review with unique ids)
  • Nothing the model says is taken on faith: aligns (an ambiguous edit is refused, not guessed)
  • The budget is the constraint, not a setting: aligns (a dropped edit's tokens and created skill are excluded)
  • It reads what you already have and owns nothing: aligns
  • Failure is loud and named: cannot tell, Kun's call: the drop is named in a warning and the notes, but it replaces the documented loud failure
  • Scope: aligns

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

One non-unique hunk anchor discards the whole synthesis proposal

2 participants