Skip to content

Port SlateDB fork to v0.16 - #12

Merged
matthewsanetra merged 67 commits into
mainfrom
port-v0.16
Sep 2, 2026
Merged

Port SlateDB fork to v0.16#12
matthewsanetra merged 67 commits into
mainfrom
port-v0.16

Conversation

@matthewsanetra

@matthewsanetra matthewsanetra commented Sep 1, 2026

Copy link
Copy Markdown

Summary

  • Rebase the HelixDB SlateDB fork feature series onto upstream v0.16.0.
  • Preserve fork APIs for readers and snapshots, multi-get, query metrics, cache snapshots, typed database-missing errors, and commutative transaction merges.
  • Integrate v0.16 custom WAL readers and streaming, ManifestV2, read tracing, explicit close flushing, and safe metrics callbacks.
  • Record current main through a content-neutral ancestry merge.

Validation

  • Rust feature-matrix checks and warning-denied Clippy.
  • All-feature nextest: 2,322 passed, 1 configured skip.
  • Documentation tests, DST, macOS cross-tests, FlatBuffers validation, and both reader benchmark compile targets.
  • Python: 47 tests; Node: 49 tests; Go and Java 22 checks.
  • v0.15 upgrade fixtures covering flushed, WAL-only, merged, and checkpointed databases.
  • HelixDB db full suite, strict Clippy, and targeted topology and bitmap concurrency tests.

agavra and others added 30 commits July 29, 2026 10:02
@agavra
add distributed compaction blog post (slatedb#1929)
480bd76
@rodesai
[rfc-30 2/N]: add traits for WAL iteration and update replay paths (s…
3428d06 
…latedb#1976)
@rodesai
disallow clone with projection when WAL must be copied (slatedb#1984)
f730735
@criccomini
Clarify TTL millisecond units (slatedb#1989)
b37f69b
@rodesai
remove notifiers from WalBuffer (slatedb#1987)
38df98a
@criccomini
Remove await_durable from WriteOptions (slatedb#1985)
8f673c0
@bowenli86
test(dst): vary scan options (slatedb#1952)
287a36e
@sorenbs
Add cooperative yield points to SST build and compaction merge loops (s…
2caad87 
…latedb#1964)
@criccomini
Increase default GC interval to 10 minutes (slatedb#1993)
1b527bd
@nomiero
Remove unnecessary SR clones in scan APIs (slatedb#1994)
c7e9caf
@criccomini
Disable descending scan checks in DST (slatedb#1996)
8cab540
@criccomini
Deflake object store cache metrics test (slatedb#1997)
b25a5b8
@Aetherance
Allow closing a DB without flushing the active memtable (slatedb#1948)
1bc2008
@criccomini
site: add Prisma to adopter listings (slatedb#1999)
03b18f6
@criccomini
Add consume_budget in hot loops to help Tokio (slatedb#1998)
90117dc
@rodesai
[rfc-30 3/N]: add trait and native implementation for wal gc (slatedb…
e67d7d4 
…#2000)
@criccomini
fix(compactor): prevent terminal compaction resurrection (slatedb#2002)
d0c3d63
@bit2swaz
Add cleanup_db admin fn and cli command for clone teardown (slatedb#2001
aa46b9b 
)
@criccomini
Run nightly pprof on a Warp ARM runner (slatedb#2008)
0717cc1
@1996fanrui
[1972] Expose manifest version count as a metric (slatedb#1974)
889d405
@rodesai
[rfc-30 4/N]: move max wal file flush check to wal trait (slatedb#2007)
0f946cb
@Ten0 @criccomini
Actually skip the WAL at DbReader open when skip_wal_replay is set (h…
9e82d15 
…elps slatedb#2003) (slatedb#2005)

Co-authored-by: Chris <criccomini@users.noreply.github.com>
@rodesai
make close flush mode more explicit by using explicit flush type (sla…
a494d0a 
…tedb#2011)
@criccomini
Replace nightly bencher work with benchmark.slatedb.io (slatedb#2009)
d12891f
@criccomini
Run nightly microbenchmarks on Warp ARM (slatedb#2016)
5b2a905
@criccomini
Select previous release tag dynamically (slatedb#2017)
bbb09be
@rodesai
[rfc-30 5/N]: add WalAdmin trait for administrative wal operations (s…
47040a3 
…latedb#2014)
@1996fanrui
[2004] Do not invoke user metrics callbacks under the db.state write …
bc34de6 
…lock (slatedb#2006)
@FiV0
Expose SstFile::index as an opaque view instead of copying data. (sla…
8aa2c10 
…tedb#2019)
@rodesai
[rfc-30 6/N]: wire custom WAL into builders (slatedb#2015)
e9a14cd
bit2swaz and others added 29 commits August 24, 2026 12:29
@bit2swaz
perf(wal): read a whole WAL SST per request during replay (slatedb#2003…
163359d 
…) (slatedb#2042)
@mdwaud
Remove SQLync from Adopters (slatedb#2045)
84775a8
@Efrat19
Tolerate manifests GCed during admin.list_manifests (slatedb#2047)
dd5a227
@samuelstroschein
add lixray as adopter (slatedb#2050)
a6f0522
@cadonna
Propose tracing instrumentation for read path (slatedb#2035)
5d0da8d
@cadonna
add tracing options to read path (slatedb#2046)
31656fe
@criccomini
fix: preserve sequence tracker interval after deserialization (slated…
ae07acd 
…b#2053)
@rkhachatryan
manifest: write ManifestV2 universally (slatedb#2055) (slatedb#2056)
c325738
@rodesai
move wal read/write paths to their own types under wal module (slated…
ea75d12 
…b#2052)
@vaibhav8a
docs: fix typo wihtout -> without (slatedb#2057)
4919857
@criccomini
Bump version to 0.16.0
3fb9e8a
@xav-db @matthewsanetra
Add request-scoped query storage metrics
20e4165
@matthewsanetra
Add snapshot isolation to DbReader
5c2ad29
@matthewsanetra
Harden reader snapshot WAL replay test
7660fb6
@xav-db @matthewsanetra
Add multi-get functionality across database components
8b7c263 
- Introduced `multi_get` and `multi_get_with_options` methods in `Db`, `DbReader`, `DbSnapshot`, and `DbTransaction` to retrieve multiple values efficiently.
- Implemented a new `WriteBatchLookup` enum to facilitate lookup operations in `WriteBatch`.
- Enhanced the `lookup_latest_for_key` method to support the new multi-get functionality.
- Added tests to ensure correctness of multi-get operations, preserving input order and handling duplicates.

This update improves the performance and usability of batch retrieval operations in the database.
@matthewsanetra
ci: drop Windows test runner
53ab758
@xav-db @matthewsanetra
Add SlateDB cache usage snapshots
539f1e8
@xav-db @matthewsanetra
Update SlateDB dependency lockfile
1ec9fc2
@xav-db @matthewsanetra
Keep unbounded cache accounting synchronous
fd0a6cb
@xav-db @matthewsanetra
Add typed database-missing reader error
3d2e4af
@xav-db @matthewsanetra
Update missing database binding tests
36b3465
@matthewsanetra
Finish v0.16 reader API adaptation
0fbf6c0
@matthewsanetra
Add conflict-compatible transaction merges
c37a70e
@matthewsanetra
Restore HelixDB cache and reader compatibility
822c97b 
Keep the fork-facing object-store cache switch while routing it through v0.15's split flush and compaction admission policy. Accept the legacy optional checkpoint argument at DbReader::open through DbReaderMode conversion without weakening the v0.15 builder API.
@matthewsanetra
Adapt fork changes to SlateDB v0.16
b418694
@matthewsanetra
Finish v0.16 CI and consumer compatibility
2b8f979
@matthewsanetra
Merge current fork main after v0.16 port
6c88089
@matthewsanetra
Merge latest main into v0.16 port
51ed0cb
@matthewsanetra
fix reader options in generated bindings
41ce040
@matthewsanetra
matthewsanetra merged commit 50c0633 into main Sep 2, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.