Skip to content

fix(mito): backport skip-WAL close flush to release/5e02efa9 - #9625

Merged
v0y4g3r merged 2 commits into
GreptimeTeam:release/5e02efa9from
v0y4g3r:backport/flush-skip-wal-on-close
Oct 16, 2026
Merged

v0y4g3r merged 2 commits into
GreptimeTeam:release/5e02efa9from
v0y4g3r:backport/flush-skip-wal-on-close

Conversation

@v0y4g3r

@v0y4g3r v0y4g3r commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

I hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

Backport of #9440 to release/5e02efa9.

What's changed and what's your intention?

Preserve request-level skip-WAL rows during graceful region close by flushing memtables that may contain them. Logged-only writes do not cause an extra close-time flush.

  • Track skip-WAL state on mutable memtables, carry it into immutable memtables during rotation, and clear the immutable state once its data is flushed.
  • Distinguish incoming requests from WAL replay, so replay does not mark recovered rows as skip-WAL writes.
  • Share the installation path for ordinary and bulk writes, retaining existing sequence-publication ordering.
  • Preserve WAL-only recovery coverage by stopping workers without issuing region Close, and retain close/reopen and in-flight-flush regression coverage.

Backport adaptations

Updated after #9627 (skip-WAL prerequisites) and #9624 merged into release/5e02efa9. The latest base was merged without rewriting existing commits.

  • Honor request-level bulk skip_wal in memtable tracking, matching the original fix(mito): flush memtables containing skip-WAL writes on region close #9440 behavior; remove the earlier region-only adaptation.
  • Restore the original bulk source/policy, close/reopen, and WAL-only recovery coverage for Mito and metric-engine. WAL-only recovery deliberately bypasses Close; graceful close retains skipped rows.
  • Preserve the release branch's physical bulk partition-column regression, and exercise close/recovery with retained and stripped partition columns.
  • Do not introduce the absent sst/primary_key.rs module just to port its test-constructor change. Keep the release's test imports and existing write-retry behavior.

Crashes and forced termination still provide no durability guarantee for skip-WAL writes. Follower flush restrictions and shutdown error handling are unchanged. No public API, wire-format, WAL-format, or persisted-metadata changes.

Validation

Validated after merging the prerequisite base and restoring bulk close coverage:

  • cargo nextest run --locked -p mito2 -p metric-engine -p tests-integration --no-fail-fast -E 'not package(tests-integration) | test(skip_wal)': 1481 passed (1338 Mito, 108 metric-engine, 35 skip-WAL integration), 818 excluded.
  • cargo clippy --locked -p mito2 -p metric-engine -p tests-integration --all-targets --no-deps -- -D warnings: passed.
  • cargo sqlness bare -t '(session_skip_wal|copy_skip_wal)': 3 cases passed (session in standalone/distributed and COPY in standalone); result files unchanged.
  • cargo fmt -p mito2 -p metric-engine -- --check and diff checks against the updated release base: passed.

Full workspace tests, the full sqlness suite, and service-backed Kafka runs were not executed. Tests that return early without external-service configuration do not establish external-service coverage.

PR Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR requires documentation updates.
  • API changes are backward compatible.
  • Schema or data changes are backward compatible.
  • This PR needs to be backported to release branches, and I have added the backport-<target> labels (e.g. backport-v1.3 targets release/v1.3).

Track skip-WAL writes across mutable and immutable memtables and flush them on graceful region close. Distinguish incoming requests from WAL replay, and preserve WAL-only recovery tests by bypassing Close.

Adapt bulk tracking to the release branch's existing region-level WAL policy without adding request-level bulk skip-WAL support. Keep release-only tests and omit changes to the absent SST primary-key module.

Signed-off-by: WenyXu <wenymedia@gmail.com>

(cherry picked from commit a026b3b)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Copilot AI balanced review requested due to automatic review settings October 15, 2026 16:12
@v0y4g3r v0y4g3r added the docs-not-required This change does not impact docs. label Oct 15, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@v0y4g3r

v0y4g3r commented Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

Merge #9627 first: it backports the missing skip-WAL integration harness (#9093) and bulk request support (#9110). After that prerequisite lands, this close-flush backport should be updated to retain the original bulk skip-WAL coverage instead of the current older-interface adaptation.

Merge release/5e02efa9 after GreptimeTeam#9627 and GreptimeTeam#9624. Resolve the worker test conflict using the shared memtable installation path, honor request-level bulk skip-WAL flags, and restore bulk close/reopen and WAL-only recovery coverage while preserving release-specific partition-column tests.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
@github-actions github-actions Bot added size/L and removed size/M labels Oct 16, 2026
@v0y4g3r

v0y4g3r commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Resolved against the latest release base (including #9627 and #9624) in 467d27a, without force-pushing. Restored request-level bulk skip-WAL tracking and bulk close/WAL-only recovery coverage. Validation: 1481 tests, 3 SQLness cases, and targeted all-target Clippy passed. The prerequisite ordering note above is now satisfied.

@v0y4g3r
v0y4g3r merged commit 8282f72 into GreptimeTeam:release/5e02efa9 Oct 16, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-not-required This change does not impact docs. size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants