Skip to content

[codex] batch leaf log split appends - #1496

Draft
snissn wants to merge 1 commit into
mainfrom
codex/leaf-log-batch-locking
Draft

[codex] batch leaf log split appends#1496
snissn wants to merge 1 commit into
mainfrom
codex/leaf-log-batch-locking

Conversation

@snissn

@snissn snissn commented May 13, 2026

Copy link
Copy Markdown
Owner

Summary

Shortens the leaf-log append critical path for split sibling leaves without changing leaf-generation liveness semantics.

Changes:

  • adds a LeafPageBatchLog path in the zipper
  • implements AppendLeafPages for the caching leaf-page log so sibling leaf pages can be compacted before entering the value-log append path and persisted through one append call
  • switches the two-leaf split path to persist both sibling leaves together
  • forces outer-leaf value-log records to stay one leaf per frame, preserving record-granular leaf-generation accounting

Correctness note

I intentionally did not batch bulk-build leaf generation in this PR. A broader batching attempt caused TestLeafGenerationPack_RewritesCollectionInternalRootsBeforeGC to fail because dead-byte accounting no longer saw the expected dead leaf records. This PR keeps the safe split-sibling batching only and keeps outer-leaf frames at one leaf record per frame.

Performance evidence

Correctness guard that specifically protects leaf-generation accounting:

GOWORK=off go test ./TreeDB/db -run '^TestLeafGenerationPack_RewritesCollectionInternalRootsBeforeGC$'

Result:

ok  github.com/snissn/gomap/TreeDB/db  (cached)

Focused package tests:

GOWORK=off go test ./TreeDB/caching ./TreeDB/zipper ./TreeDB/db \
  -run 'TestCachingLeafPageLog|TestZipper|TestLeafPageLog|TestLeafGenerationPack_RewritesCollectionInternalRootsBeforeGC'

Result:

ok  github.com/snissn/gomap/TreeDB/caching 0.050s
ok  github.com/snissn/gomap/TreeDB/zipper 0.004s
ok  github.com/snissn/gomap/TreeDB/db 0.143s

Example update benchmark run on this branch:

GOWORK=off go test ./cmd/mongo_gateway_bench \
  -run '^$' \
  -bench 'BenchmarkDirectCollectionConcurrentUpdateBSONIndexes2$' \
  -benchmem \
  -benchtime=1s \
  -count=1

Selected result line:

BenchmarkDirectCollectionConcurrentUpdateBSONIndexes2-12  167571  6299 ns/op  158747 docs/sec  1455 B/op  3 allocs/op

Leaf-log/root-apply metrics from the same run:

publish_delta_group_root_apply_ns/doc=476.2
publish_delta_group_root_apply_leaf_log_pages_written/doc=0.09181
publish_delta_group_root_apply_leaf_log_record_hint_write_bytes/doc=61.12
publish_delta_group_lock_hold_ns/doc=3.792
update_buffer_lock_hold_ns/doc=525.1

Full local benchmark output was saved under /tmp/gomap_leaflog_pr_validation_* on the development host.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant