Skip to content

feat: on-disk DB format-version guard (txpool + blockchain) - #4

Open
orrinfrazier wants to merge 1 commit into
mainfrom
feat/s04-1-db-versioning
Open

feat: on-disk DB format-version guard (txpool + blockchain)#4
orrinfrazier wants to merge 1 commit into
mainfrom
feat/s04-1-db-versioning

Conversation

@orrinfrazier

@orrinfrazier orrinfrazier commented May 24, 2026

Copy link
Copy Markdown
Owner

Closes #14

Summary

Adds an on-disk schema/format-version marker to both fjall databases (txpool + blockchain) and a guard that refuses to start on mismatch — closing the S04-1 beta blocker. Without a persisted version, any future tapes/keyspace schema change on a shipped v0.1.0 database could not be detected, and a newer binary would silently misread or panic instead of refusing/migrating.

What changed

A format_version value (u64 little-endian) is persisted under b"format_version" in each DB's metadata keyspace at creation, and verified on every open:

  • absent → fresh/pre-versioning DB: stamp the current version and proceed
  • present & equal → proceed
  • present & mismatched, or malformed (len ≠ 8) → refuse with DbFormatVersionMismatch { expected, found } (returned, never panics), leaving room for a future migration step

Per piece:

  • txpool (storage/txpool/src/txpool.rs, error.rs): activate the previously-unused metadata keyspace (dropped #[expect(dead_code)] // TODO: version?), add DATABASE_FORMAT_VERSION, add the error variant, and change open_with_databaseResult<Self, TxPoolError> (sole caller init_with_pool already returns TxPoolError, so no caller change).
  • blockchain (storage/blockchain/src/database.rs, error.rs): reuse the existing constants::DATABASE_VERSION; check/stamp it before Tapes::open so an incompatible DB is refused without creating any tape files. Added the error variant + tempfile dev-dep.

Open question (resolved)

Fjall 3.0.4's internal FormatVersion (V1/V2/V3) guards only fjall's own LSM disk format per major release — not the application schema (keyspace layout, value encodings, tapes layout). So an app-level marker is still required.

How to test

cargo test -p cuprate-txpool -p cuprate-blockchain
cargo clippy -p cuprate-txpool -p cuprate-blockchain --all-targets

10 new tests cover all branches in both crates: fresh-stamp, reopen-with-match, mismatch → error, and malformed-value → error (no panic), plus a db_version() sanity check.

Notes

  • Pre-existing repo-wide issues NOT touched by this PR: cuprate-types dead-code warnings (PoolTxInfo, default_zero) and cargo audit advisories in transitive deps (rsa, rustls-webpki, lz4_flex, bincode, paste).
  • Reviewed via 3-way cross-family consensus (Claude + Gemini + Codex); the one finding (fail-fast placement on the blockchain check) was resolved before this PR.

Persist a `format_version` marker (u64 little-endian under `b"format_version"`)
in the `metadata` keyspace of both the txpool and blockchain fjall databases,
and verify it on open: a fresh DB is stamped, a matching version proceeds, and
a mismatched or malformed version is refused with `DbFormatVersionMismatch`
(never panics), leaving room for a future migration step.

- txpool: activate the previously-unused `metadata` keyspace (drop the
  `#[expect(dead_code)] // TODO: version?`), add `DATABASE_FORMAT_VERSION`,
  and change `open_with_database` to return `Result<Self, TxPoolError>`.
- blockchain: check/stamp the existing `DATABASE_VERSION` before `Tapes::open`
  so an incompatible database is refused without creating any tape files.

Resolves the S04-1 beta blocker: without this marker, a future tapes/keyspace
schema change on a shipped v0.1.0 database could not be detected and would
silently misread or panic instead of refusing.
@orrinfrazier
orrinfrazier force-pushed the feat/s04-1-db-versioning branch from fcd7ddc to bb466c5 Compare July 28, 2026 23:17
@github-actions github-actions Bot added A-storage Area: Related to storage. A-workspace Area: Changes to a root workspace file or general repo file. A-dependency labels Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-dependency A-storage Area: Related to storage. A-workspace Area: Changes to a root workspace file or general repo file.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

storage: add on-disk DB schema versioning and migration guard

1 participant