feat(store): one-time legacy v11 store import into the control database

[Bnjoroge1]

One-time legacy v11 store import into the control database

Base: Bnjoroge/lock-refactor · Head: Bnjoroge/legacy-db-import · Draft — superseded by the integration branch Bnjoroge/control-import-e2e (stacks on #372, adds the missing read paths, then gets the end-to-end test on a real database). This PR is pushed for visibility; review the follow-up for the merge vehicle.

What it does

preloop store import-legacy --source <old preloop.db> --target <new control.db> [--state-dir DIR --key FILE --active refuse|requeue|cancel --json] performs the one-time, explicit, offline move from a released v11 durable-state store (the pre-ControlBackend schema) to a fresh control SQLite database:

• Source is opened read-only, sha256-hashed before and after, data_version re-checked; a changed source aborts.
• Everything is decoded/audited before a byte is written: wrong key, unknown user_version/table set, unknown meta keys, duplicate unique-index keys, unreadable blobs all refuse with a report.
• Target is built in <target>.importing in one transaction, verified (PRAGMA foreign_key_check, row counts, schema check), then published atomically and never overwriting (hard_link, AlreadyExists on a racing target); a failure removes staging + staged sidecars, so retry is safe.
• In-flight work (claimed attempts, session bindings, live concurrency gates) refuses by default and is listed; --active=requeue|cancel resolves it explicitly. Never a silent rerun.
• Mapped: runs/jobs/specs/needs/messages/attempts/leases/steps, logs → live-log segments, run-tier secrets (sealed, values never enter the control DB, post-strip canary scan), counters/check ids/outputs/annotations, runners/sessions, webhooks, timelines, assignments/cancellations/token requests, OIDC grants, control_events → outbox (ids/order/payload preserved), per-attempt frames verified or reconstructing terminal-job templates, finalized artifact registries (meta + artifact_v2_registry.json).
• Fixture SQL for tests is the released v11 migration chain copied verbatim with a commit reference and per-migration sha256 pins.

Known gaps (being completed in the follow-up branch)

• v1 artifact GET/list and runs/:id/events still read node-local in-memory maps, so imported artifacts/events are durable but not yet served; the follow-up wires the backend catalog + outbox-backed event snapshot, plus public-endpoint smokes.
• target.rs still initializes via the old lite schema; the follow-up swaps to the embedded refinery migrations (needed for the boot ledger guard).

Verification

No build/test has been run on this commit (delegated work, gates owned by the integration branch). Tests exist for the full import path (tests/legacy_import.rs + module tests) and will be executed on Bnjoroge/control-import-e2e; CI is the backstop.

---

Summary by cubic

Adds preloop store import-legacy, a one-time offline command that moves a released v11/v12 preloop.db (the durable-state store the control backend replaced) into a fresh control SQLite database. The source is opened read-only and hashed before and after; the target is built in <target>.importing, verified, and atomically published, so a failure leaves no target and retry is safe.

Key behaviors

• Everything is decoded and audited before any write: wrong key, unknown source version, unreadable blobs, and duplicate unique keys abort with a report.
• In-flight work (claimed attempts, session bindings, concurrency gates) refuses by default and is listed; --active=requeue|cancel drains it explicitly.
• Secrets move to the SecretProvider run tier (sealed, never written to control SQL) and stored messages are reduced to secret-free templates.
• The legacy event log is carried into the outbox with ids and order preserved.
• v12 sources are handled too: message payloads use the row-bound AAD, a terminal status overrides a stale recorded conclusion, and settled attempts drop their lease.
• Verified end to end against a real 37 MB v12 database (264 runs / 1736 attempts / 754 steps) with the source digest unchanged; tests use a fixture built from the released migration chain.
• The import lands on the refactored lock base; the merge repaired the legacy writer for the base's type and borrow drift without changing behavior.

^{Written for commit 92bb3d0. Summary will update on new commits.}

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Bnjoroge1]

Superseded for merge by #376 (Bnjoroge/control-import-e2e), which stacks this import on the migration runner, fixes the compile/test fallout, adds the artifact/event read paths, and is verified end-to-end on a real 260-run v12 database.

[Bnjoroge1]

Closing as superseded by #376, which is the merge vehicle for the import work (fixes the compile errors in this PR).