Skip to content

item 9: the corpus — every candidate program, its trace, and the verdict - #28

Merged
compusophy merged 3 commits into
mainfrom
agent/corpus-capture
Aug 26, 2026
Merged

item 9: the corpus — every candidate program, its trace, and the verdict#28
compusophy merged 3 commits into
mainfrom
agent/corpus-capture

Conversation

@compusophy

Copy link
Copy Markdown
Owner

Plan §9 says the opcode-model horizon must not start "until the corpus collection exists, or there is nothing to train on". It exists now.

what it records

src/cartridges/corpus.rs. Every candidate policy that goes through a swap — from the UI or from the agent's swap_cartridge tool — becomes a Sample: rustlite source, opcode trace, Origin (Human | Agent { intent }), and the runtime's Verdict. Persisted to vanish-cartridges/corpus.json, bounded at 200, keyed by an FNV-1a fingerprint of the source so re-trying a program updates its verdict in place rather than growing the log.

Captured inside Cognition::swap_policy, which now returns (Sample, Result<…>). That signature is the design: every attempt yields a sample; whether it also yields a swap is the verdict.

three decisions worth defending

  • Refusals are kept. A corpus of only successes teaches nothing about the boundary, and the boundary is where a generated program actually fails. A program that emitted and then trapped keeps its trace — a refused opcode sequence with the rehearsal's own words attached is the most useful negative available. A program that never compiled has an empty trace, and the sample says so rather than inventing a label.
  • The trace drops operands. Emission is deterministic, so emit_module(parse(source)) rebuilds the module exactly: the source is the record, and the trace is the same program in the shape a model would emit it. Storing operands would double the corpus to hold what it already holds.
  • The histogram counts verified programs only. A rejected opcode sequence is not evidence about what good code looks like, even though it is excellent evidence about what fails.

The intent argument on swap_cartridge is now required — one line on what the model meant the program to do. It is the only prompt-shaped thing the swap path ever sees, and it is stored even when the attempt is refused.

what this still does not buy

Whether a policy helped. The corpus records what was tried and whether it ran. "Did the shaped prompt produce a better answer" needs an outcome signal the loop does not collect, and no amount of corpus makes that question answerable. §9 now says this in those words rather than letting the item imply otherwise.

evidence

9 new evals (31 in tests/cognitive_wiring.rs), including: the trace naming its exported functions and ending every body on end; v3 emitting a demonstrably richer trace than v1 (br_if, i32.load8_u) so the measurement distinguishes programs; refusals kept with and without traces; fingerprint dedup updating a verdict in place; the bound dropping the oldest; a round-trip plus every decode refusal named; and an oversized source truncated on a character boundary — a naive String::truncate mid-character panics, and the corpus must not be the thing that takes the worker down.

bash ci/run_tests.sh → unit tests + 20 suites + clippy; cargo check --lib --bins --target wasm32-unknown-unknown clean.

smoke-preview will be red for the usual Vercel Deployment Protection reason.

🤖 Generated with Claude Code

plan section 9 says the opcode-model horizon must not start "until the
corpus collection exists, or there is nothing to train on". it exists now.

src/cartridges/corpus.rs: every candidate policy that goes through a swap —
from the ui or from the agent's swap_cartridge tool — becomes a Sample:
rustlite source, opcode trace, Origin (Human | Agent{intent}), and the
runtime's Verdict. persisted to vanish-cartridges/corpus.json, bounded at
200, keyed by an FNV-1a fingerprint of the source so re-trying a program
updates its verdict in place rather than growing the log.

captured inside Cognition::swap_policy, which now returns (Sample,
Result<…>). that signature is the design: every attempt yields a sample,
and whether it also yields a swap is the verdict.

three decisions worth defending:

- refusals are kept. a corpus of only successes teaches nothing about the
  boundary, and the boundary is where a generated program actually fails. a
  program that emitted and then trapped keeps its trace; one that never
  compiled has an empty one and says so rather than inventing a label.
- the trace drops operands. emission is deterministic, so the source
  rebuilds the module exactly — storing operands would double the corpus to
  hold what it already holds.
- the histogram counts verified programs only. a rejected sequence is not
  evidence about what good code looks like, though it is excellent evidence
  about what fails.

the intent argument on swap_cartridge is now required: one line on what the
model meant the program to do, stored even when the attempt is refused. it
is the only prompt-shaped thing the swap path ever sees.

9 new evals. the feed and the tool result both carry corpus stats and the
most-emitted opcodes.

what this still does not buy, and section 9 now says so: whether a policy
HELPED. the corpus records what was tried and whether it ran; "did the
shaped prompt produce a better answer" needs an outcome signal the loop
does not collect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Aug 26, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
vanish Ready Ready Preview Aug 26, 2026 11:07pm

Request Review

the sample was recorded on the refusal path but the feed only mentioned the
corpus on success, so "refusals are kept" was a claim with nothing behind it
on screen. apply_policy_swap now hands the stats back on BOTH paths (None
only when there was no candidate at all), and both doors — the ui command
and the agent's tool — emit the corpus note after a refusal as well as after
a swap. a run that produced no swap still produced training data, and that
is worth seeing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
on the preview of 1a2490c: swapping to v3 gives "corpus: 1 program(s), 1
verified, 0 refused — most-emitted: local.get×87, local.set×42,
i32.const×38, i32.add×25, i64.const×24" — section 9's opcode vocabulary,
measured rather than assumed. a module that compiles and then traps, and one
that does not compile at all, are both refused AND both counted (2 then 3
programs, 1 verified). the histogram does not move across either refusal,
which is the visible confirmation that it counts verified programs only. a
reload comes back as "corpus restored: 3 program(s), 1 verified".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@compusophy

Copy link
Copy Markdown
Owner Author

live on the preview of 1a2490c

Corpus watched filling, refusals included, surviving a reload:

🧪 rehearsal passed: "rehearsal: a prompt" → "rehearsal: a prompt … (policy: to carry one line …)"
🔁 cartridge 'reasoner' hot-swapped
📚 corpus: 1 program(s), 1 verified, 0 refused — most-emitted: local.get×87, local.set×42, i32.const×38, i32.add×25, i64.const×24

hot-swap refused: it failed on a prompt: cartridge 'reasoner': cartridge trapped: memory access out of bounds: 4 byte(s) at 2000000000 exceed the 1048576-byte memory
📚 corpus: 2 program(s), 1 verified, 1 refused — most-emitted: local.get×87, …

hot-swap refused: rustlite `cart_handle`: call to unknown function 'nope'
📚 corpus: 3 program(s), 1 verified, 2 refused — most-emitted: local.get×87, …

Then a full page reload:

📚 corpus restored: 3 program(s), 1 verified

Three things this shows that the evals cannot: the histogram is a real measurement of rustlite's instruction selection, not an assumption; both flavours of refusal (emitted-then-trapped, and never-compiled) are counted; and the histogram does not move across either refusal — visible confirmation that it counts verified programs only.

The last commit also closes a gap I found while setting this up: the sample was recorded on the refusal path but the feed only mentioned the corpus on success, so "refusals are kept" was a claim with nothing behind it on screen. Both doors now emit the corpus note after a refusal too.

@compusophy
compusophy merged commit 57b7972 into main Aug 26, 2026
4 of 5 checks passed
@compusophy
compusophy deleted the agent/corpus-capture branch August 26, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant