core: `search_chunks`' 3× hit pool buys a torn-read-only defense at a steady-state ranking cost

[samkeen]

Follow-up to #137, shipped in #140. The change is defensible either way; it was made without the cost being measured, so it deserves an explicit ruling rather than inheritance.

What shipped

#137's scope note asked that Vault::search_chunks get the same drop-don't-under-fill treatment as search::resolve_hits. It did: both façade reads now retrieve vault::hit_pool(limit) = limit * 3 and stop at limit, so a hit they drop is backfilled from the next candidate.

For Vault::search the 3× is load-bearing and pre-dates #137: several top chunks routinely share one note, so the pool must over-pull for note dedup to still fill limit distinct notes. Nothing about that changed.

For Vault::search_chunks there is no dedup — it is deliberately the un-deduped, passage-level view. The only reason it drops a hit is a resolve_b2id_to_path / chunk_detail lookup that missed because the row vanished mid-query: the C1 concurrent-reindex window (invariants.md C1, index-engine.md §3). Its doc-comment previously said, in as many words, that under-filling in that window was acceptable.

The cost that wasn't priced

retrieve(query, 3 * limit) reaches pool_size(3 * limit) = 15 * limit, so the BM25 LIMIT and the in-process vector scan both widen 3× — for limit = 10, from 50 candidates per signal to 150.

That is not just more work, it is different answers. RRF scores Σ 1/(k + rank + 1) with k = 60, so a deeper pool admits candidates that displace shallower ones: a chunk at rank ~60 in both lists scores 2/121 ≈ 0.0165, above a chunk at rank 0 in one list only (1/61 ≈ 0.0164).

Measured on fixtures/test-vault (200 notes, 700+ chunks) with the fake embedder: 5 of 7 probe queries returned a different top-10, differing in membership, not only order. The fake embedder's vector ranking is blake3-derived and therefore uncorrelated with BM25, which exaggerates the effect — real bge rankings correlate, so the true shift is smaller — but the mechanism is real and it bites on any vault with more than ~50 matching chunks.

The committed eval corpus (26 chunks) cannot detect this — see #141.

The decision

1. Keep 3×, and document it as a quality change rather than a plumbing one. More candidates into RRF is defensible retrieval on the merits, and search_chunks is the eval's passage-rank view, so a real measured lift would justify the scan cost outright.
2. Give search_chunks a small fixed headroom instead — limit + 2, say — so a torn-read drop still backfills without materially changing the candidate set, and leave search's dedup-driven 3× alone. This restores the pre-Hidden means hidden: dot-prefixed names are never vault material #140 ranking while keeping everything core: search can under-fill limit when a ranked chunk no longer resolves — take(limit) runs before resolution #137 asked for.

(2) is the conservative reading of #137, whose subject was under-filling during a torn read, not fusion width. (1) needs evidence, and that evidence is exactly what the eval currently cannot produce — so this likely sequences after #141.

Either way vault::hit_pool's doc-comment should state which case each caller's headroom is for; today it explains both reasons in one place for two callers that have different ones.

[samkeen]

Ruled: (2) — a fixed torn-read headroom for search_chunks, search keeps its 3×

Shipped on claude/issue-142-k0q8cd.

The measurement

#141's probe is the instrument this ruling needed, so it ran first. On fixtures/test-vault (200 notes / 780 chunks, fake embedder), across exactly the 60→150 candidates/signal step the 3× introduced:

  ask  4→10: 10/10 probes changed their top-4 chunks (60→150 candidates/signal), 9/10 their top-4 notes
  ask 10→30:  1/10 probes changed their top-4 chunks (150→450),                   1/10 their top-4 notes

So the issue's estimate holds and is if anything understated: 10 of 10, on membership, not just order. The eval corpus prints bit-identical numbers across the same change (26 chunks < either pool).

That settles different. It does not settle better, and (1) needs better — which the labelled corpus still cannot produce, since #141 shipped the probe, not a bigger corpus. So width goes back to the conservative setting.

What changed

• hit_pool → two functions, each stating its own caller's reason:
  • note_hit_pool(limit) = limit * 3 — dedup-driven, and dedup scales with the ask.
  • chunk_hit_pool(limit) = limit + TORN_READ_HEADROOM (2) — the C1 torn-read window only, which is bounded and does not scale with the ask.
• Everything core: search can under-fill limit when a ranked chunk no longer resolves — take(limit) runs before resolution #137 asked for is intact: both loops still walk a pool wider than limit and backfill a dropped hit instead of charging it a slot. search_chunks_still_fills_limit_when_a_ranked_chunk_is_dead passes unchanged.
• vault::candidate_pool split into note_candidate_pool / chunk_candidate_pool — one number can no longer describe both views, and a measurement that picks the wrong one lies. The eval now judges blindness on the narrower pool (blindness is a claim about every number it prints) and records both depths in results.jsonl; the stability table labels each column with its own view's widths.
• The committed stability baseline is re-blessed: the note half is byte-identical (that 3× never moved), the chunk half is the intended change on all 10 probes.
• index-engine.md §4 now carries the rule as spec — candidate width is a per-view quality knob, and the 5× in pool_size is why façade headroom is never just plumbing — and §5 records that the gate has now ruled once.

The honest counter-argument, for the record

The pre-change probe shows the passage view converging under widening at 150 (150→450 moved only 1/10) while 60 is not converged (60→160 moves 10/10 post-change). Someone could read that as "150 is past the knee, 60 is not." I don't think it carries: convergence under widening says the tail stopped displacing the head, not that the head is more relevant — and with the fake embedder's uncorrelated vector list, some of that convergence is an artifact. But it's the strongest thing that could reopen (1), and it points at what would actually settle it: a labelled corpus larger than 150 chunks. Worth its own issue if passage retrieval quality becomes the thing being tuned.

---

Generated by Claude Code

[samkeen]

Fixed by #144