fix(cycle): track bracket depth in extract-atoms trailing-prose recovery - #5064
Open
Masashi-Ono0611 wants to merge 1 commit into
Open
Masashi-Ono0611 wants to merge 1 commit into
Masashi-Ono0611 wants to merge 1 commit into
Conversation
parseArrayAtOffset's trailing-prose recovery used slice.lastIndexOf(']')
to find the array's closing bracket when the whole-slice JSON.parse
failed. A valid atoms array followed by a bracketed citation (e.g.
"...]\nSee [Source: X]." -- a shape gbrain's own house style encourages)
has that citation's ']' sort after the array's real terminator, so the
naive scan trimmed back to the wrong bracket and included the dangling
citation text in the re-parse, which then failed as "unparseable JSON
array" even though the array itself was well-formed.
Replace the naive scan with findArrayCloseIndex(), which tracks bracket
depth from the array's opening '[' and skips brackets inside JSON string
literals (honoring \" escapes), so it lands on the array's true closing
bracket regardless of what brackets appear in trailing prose.
Scope is limited to parseArrayAtOffset's internal recovery logic; the
function signature and callers (parseAtomsOutcomeInner, the anchor scan)
are unchanged. Bumped extract-atoms.ts's module-size-limits.tsv ceiling
(1742 -> 1780) for the added helper, per the module-size ratchet rule.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kthq1tdcmzggtc8if3jHtW
Masashi-Ono0611
marked this pull request as ready for review
September 13, 2026 13:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
parseArrayAtOffset(the atoms-extraction JSON parser insrc/core/cycle/extract-atoms.ts) recovers a valid atoms array from a response that has trailing prose after it by trimming back toslice.lastIndexOf(']')when a whole-sliceJSON.parsefails. If the trailing prose itself contains a bracketed citation (e.g.[Source: X]— a shape this brain's own house style encourages, perdocs/architecture/...extraction prompts),lastIndexOf(']')finds the citation's closing bracket instead of the array's own, so the recovery slice spans past the real array into the dangling citation text and fails to parse. A well-formed atoms array is then reported asunparseable JSON array, which counts as a deterministic failure toward a page's tombstone threshold.Example:
Before this fix:
unparseable JSON array(the citation's]hijacks the trim boundary). After: parses correctly, 1 atom recovered.Fix
Added
findArrayCloseIndex(), which walks the string from the array's own opening[, tracking bracket depth and skipping[/]characters that appear inside JSON string literals (honoring\"escapes). This finds the array's own true closing bracket regardless of what brackets appear afterward in trailing prose.parseArrayAtOffsetnow uses this instead of the naivelastIndexOf(']').Scope is limited to this one recovery path inside
parseArrayAtOffset: no exported signature changed, no new CLI/config surface, no other parser touched.One side effect worth calling out explicitly (see the NOTE comment added on
parseArrayAtOffset): because this now finds each candidate's own array boundary correctly instead of occasionally over-running into later text, a response embedding two complete top-level atoms arrays (e.g. a "draft, then corrected answer" pattern — not a documented or prompted output shape) will now have the FIRST array win, where before it sometimes (accidentally, via the same bug) fell through to the last one. This is the anchor scan's existing, already-documented "first candidate that parses to >=1 atom wins" policy applying correctly, not a new precedence rule. A test pins this explicitly (two complete top-level arrays: the FIRST one wins...).Testing
test/extract-atoms-array-anchor-scan.test.ts(created for a prior related anchor-scan fix): the citation-hijack repro, an unmatched-trailing-bracket variant, the two-full-arrays precedence pin, and a test exercising escaped quotes / bracket-like substrings inside strings / a nested array on the recovery path together.test/extract-atoms-array-anchor-scan.test.ts,test/llm-json-reasoning-ladder.test.ts,test/cycle/extract-atoms-synthesize-concepts.test.ts,test/extract-atoms-failure-classes.test.ts) still pass unmodified in behavior — 101 tests pass across those 4 files.bun run typecheckclean.bun run verify(54 checks) passes.scripts/module-size-limits.tsvceiling for this file bumped 1742 -> 1788 (mechanical, same-commit, per the repo's module-size ratchet rule) to cover the added helper + doc comments.Discrimination test: reverted src/core/cycle/extract-atoms.ts to merge-base, ran test/extract-atoms-array-anchor-scan.test.ts → 19 pass / 4 fail. Restored → all pass.
Not verified
test/extract-atoms-array-anchor-scan.test.tsfile already uses for its other cases).