Skip to content

chore(deps): bump rocksdb to v11.0.4 - #3431

Closed
aleksraiden wants to merge 4 commits into
unstablefrom
aleksraiden-patch-1
Closed

aleksraiden wants to merge 4 commits into
unstablefrom
aleksraiden-patch-1

Conversation

@aleksraiden

Copy link
Copy Markdown
Contributor

Bump rocksdb to v11.0.4 (see: https://github.com/facebook/rocksdb/releases/tag/v11.0.4)

This is a first public release at 11.х line.

Key changes:

  • Add a new table option separate_key_value_in_data_block. When set to true keys and values will be stored separately in the data block, which can result in higher cpu cache hit rate and better compression. Works best with data blocks with sufficient restart intervals and large values. Previous versions of RocksDB will reject files written using this option
  • Change the default value of CompactionOptionsUniversal::reduce_file_locking from false to true to improve write stall and reduce read regression
  • Added new virtual methods AbortAllCompactions() and ResumeAllCompactions() to the DB class
  • Added support for storing wide-column entity column values in blob files
  • Added CompactionOptionsFIFO::max_data_files_size to support FIFO compaction trimming based on combined SST and blob file sizes
  • Include interpolation search as an alternative to binary search, which typically performs better when keys are uniformly distributed. This is exposed as a new table option index_block_search_type. The default is binary_search.
  • Remove deprecated options and clean-up codebase
  • Bug fixes

@aleksraiden

aleksraiden commented Apr 8, 2026 •

Copy link
Copy Markdown
Contributor Author

As I see, need a code patch to use this release:
/home/runner/work/kvrocks/kvrocks/src/common/db_util.h:78:7: error: address of overloaded function 'Open' cannot be static_cast to type 'rocksdb::Status (*)(const rocksdb::Options &, const std::string &, rocksdb::DB **)' (aka 'rocksdb::Status (*)(const rocksdb::Options &, const basic_string<char> &, rocksdb::DB **)') [clang-diagnostic-error] 78 | static_cast<rocksdb::Status (*)(const rocksdb::Options&, const std::string&, rocksdb::DB**)>(rocksdb::DB::Open),

@torwig

torwig commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

I believe the public API was changed (breaking change) in version 11 (that's why it's incremented from 10 to 11):

/Users/runner/work/kvrocks/kvrocks/src/common/db_util.h:63:10: error: no matching function for call to 'OpenForReadOnly'
  return rocksdb::DB::OpenForReadOnly(db_options, dbname, column_families, handles, dbptr);
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/runner/work/kvrocks/kvrocks/build/_deps/rocksdb-src/include/rocksdb/db.h:186:17: note: candidate function not viable: no known conversion from 'rocksdb::DB **' to 'std::unique_ptr<DB> *' for 5th argument
  static Status OpenForReadOnly(
                ^
/Users/runner/work/kvrocks/kvrocks/build/_deps/rocksdb-src/include/rocksdb/db.h:175:17: note: candidate function not viable: requires at most 4 arguments, but 5 were provided
  static Status OpenForReadOnly(const Options& options, const std::string& name,
                ^
In file included from /Users/runner/work/kvrocks/kvrocks/src/cluster/slot_migrate.cc:26:
/Users/runner/work/kvrocks/kvrocks/src/common/db_util.h:70:10: error: no matching function for call to 'OpenAsSecondary'
  return rocksdb::DB::OpenAsSecondary(db_options, dbname, secondary_path, column_families, handles, dbptr);
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/runner/work/kvrocks/kvrocks/build/_deps/rocksdb-src/include/rocksdb/db.h:256:17: note: candidate function not viable: no known conversion from 'rocksdb::DB **' to 'std::unique_ptr<DB> *' for 6th argument
  static Status OpenAsSecondary(
                ^
/Users/runner/work/kvrocks/kvrocks/build/_deps/rocksdb-src/include/rocksdb/db.h:221:17: note: candidate function not viable: requires 4 arguments, but 6 were provided
  static Status OpenAsSecondary(const Options& options, const std::string& name,

@torwig

torwig commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

Remove deprecated raw DB* variants of DB::Open and related functions. Some other minor public APIs were updated as a result

@PragmaTwice

Copy link
Copy Markdown
Member

@copilot Could you fix the build issue introduced by the new rocksdb version?

@PragmaTwice

Copy link
Copy Markdown
Member

Hmm the ASF org doesn't enable copilot agent.

@PragmaTwice

Copy link
Copy Markdown
Member

refer to #3468.

@PragmaTwice PragmaTwice closed this May 2, 2026
@PragmaTwice
PragmaTwice deleted the aleksraiden-patch-1 branch May 2, 2026 04:50
PragmaTwice added a commit that referenced this pull request May 2, 2026
Bump rocksdb to v11.1.1 (see:
https://github.com/facebook/rocksdb/releases/tag/v11.1.1)

**This PR need to merge** #3431 

**Key changes**

- Fix a bug in round-robin compaction that missed selecting input files
that are needed to guarantee data correctness and cause crashing in
debug builds or silent data corruption in release builds for Get()
- Fix a memory accounting leak in IODispatcher
- Add a new option open_files_async
- Added BlockBasedTableOptions::kAuto index block search type that
automatically selects between binary and interpolation search on a
per-index-block basis
- Add memtable_batch_lookup_optimization option to use batch lookup
optimization for memtable MultiGet
- Added new mutable DB option verify_manifest_content_on_close (default:
false)
-

---------

Co-authored-by: PragmaTwice <twice@apache.org>
Co-authored-by: Twice <twice.mliu@gmail.com>
nkroker pushed a commit to nkroker/kvrocks that referenced this pull request May 29, 2026
Bump rocksdb to v11.1.1 (see:
https://github.com/facebook/rocksdb/releases/tag/v11.1.1)

**This PR need to merge** apache#3431 

**Key changes**

- Fix a bug in round-robin compaction that missed selecting input files
that are needed to guarantee data correctness and cause crashing in
debug builds or silent data corruption in release builds for Get()
- Fix a memory accounting leak in IODispatcher
- Add a new option open_files_async
- Added BlockBasedTableOptions::kAuto index block search type that
automatically selects between binary and interpolation search on a
per-index-block basis
- Add memtable_batch_lookup_optimization option to use batch lookup
optimization for memtable MultiGet
- Added new mutable DB option verify_manifest_content_on_close (default:
false)
-

---------

Co-authored-by: PragmaTwice <twice@apache.org>
Co-authored-by: Twice <twice.mliu@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants