Follow-up to #5765.
Problem
The first asynchronous PPL API implementation limits retained jobs by count and relies on plugins.query.size_limit for row count. Neither limit bounds retained bytes. A successful retained job keeps its result object graph reachable until DELETE, expiration, or shutdown, so a small number of wide rows or large string values can consume substantial heap. plugins.query.memory_limit protects execution-time heap pressure but does not account for retained results.
Scope
- Define a per-node retained-result byte budget and setting.
- Define the byte measurement used for admission (for example, serialized result size versus an explicit heap estimate).
- Define terminal job behavior when a completed result cannot be retained within the budget.
- Reserve and release capacity exactly once across completion, DELETE, expiration, startup abort, owner shutdown, and races between them.
- Add metrics and concurrency tests for retained-byte usage and rejected retention.
- Preserve the existing
AsyncQueryExecution contract; execution producers should not depend on lifecycle retention policy.
Out of scope
Result paging or delta delivery remains separate work.
Follow-up to #5765.
Problem
The first asynchronous PPL API implementation limits retained jobs by count and relies on
plugins.query.size_limitfor row count. Neither limit bounds retained bytes. A successful retained job keeps its result object graph reachable until DELETE, expiration, or shutdown, so a small number of wide rows or large string values can consume substantial heap.plugins.query.memory_limitprotects execution-time heap pressure but does not account for retained results.Scope
AsyncQueryExecutioncontract; execution producers should not depend on lifecycle retention policy.Out of scope
Result paging or delta delivery remains separate work.