Skip to content

[Feature][PPL] Apply Calcite post-processing to pushed aggregation snapshots #5800

Description

@penghuo

Parent: #5765

Goal

Return current results when an eligible pushed aggregation has Calcite operators above the aggregation leaf.

Scope

  • Detect the eligible pushed aggregation leaf from the optimized physical plan.
  • Store aggregation-boundary rows with REPLACE semantics.
  • Replace the aggregation leaf with a StateStoreScan for current-result evaluation.
  • Compile the residual Calcite plan once and reuse it for subsequent POST timeout and GET reads.
  • Do not reparse, reoptimize, or restart the primary query during polling.

Acceptance criteria

  • Queries such as pushed stats followed by partial-safe eval or fields can return meaningful current results.
  • Every read evaluates the cached residual plan against one coherent StateStore view.
  • The primary query still executes once and remains the authoritative final path.
  • Final output exactly matches synchronous Calcite PPL output.
  • Ambiguous or unsupported physical shapes fail classification safely and do not publish semantically invalid rows.
  • Concurrent StateStore replacement, preview evaluation, cancellation, and close are covered by tests.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions