Skip to content

feat(model-info): add per-turn cache diagnostics - #372

Merged
tt-a1i merged 5 commits into
openpi-dev:mainfrom
testikun:codex/issue-156-cache-diagnostics
Sep 5, 2026
Merged

feat(model-info): add per-turn cache diagnostics#372
tt-a1i merged 5 commits into
openpi-dev:mainfrom
testikun:codex/issue-156-cache-diagnostics

Conversation

@testikun

@testikun testikun commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Problem

Issue #156 identified that OpenPI only exposed a cumulative cache percentage. A warm prefix becoming cold could not be located to a turn, while treating every cacheRead=0 as an invalidation would create false claims for first turns, tiny prefixes, and best-effort providers.

Value

Extensions can now consume a bounded, provider-aware observation for each assistant turn and correlate cache changes with local lifecycle boundaries without changing the model context or the existing footer metric.

Approach

  • Add a pure cache diagnostics tracker with explicit first-turn, cold, warm, partial-hit, miss-after-warm-prefix, and unknown outcomes.
  • Require a prior 2,048-token warm prefix and an explicit-prefix provider contract before reporting a miss; OpenAI/Google best-effort and unknown providers remain unknown.
  • Fingerprint the actual system prompt and selected tool surface at Pi's before_agent_start seam.
  • Record model, thinking, tool surface, system prompt, compaction, and branch boundaries only as correlations. Every record keeps verifiedCause: null.
  • Skip failed and cancelled assistant responses so placeholder zero usage cannot create a false cache miss, replace the valid baseline, or consume pending lifecycle correlations.
  • Emit observations on a dedicated extension event channel; preserve cumulative cachePercent, TUI state, and model-visible context unchanged.

Related to #156. Does not close the Issue. Remaining #156 work: a provider usage-field matrix; TTL expiry as its own kind only with evidence; at least one real OpenPI / Bare Pi / OMP Session trace replay before any opt-in marker.

Validation

  • bun run check — passed.
  • bun run test — passed: Node 1248 passed / 1 skipped; Vitest 30 passed.
  • Focused cache diagnostics tests — 14 passed, including failed/cancelled response regressions.
  • Updated-main validation: bun run check and bun run test passed with Node 22.19.0, including Vitest.
  • git diff --check — passed.

No UI change was made, so a UI screenshot is not applicable.

Impact

  • User-visible behavior: none; no marker is enabled by default.
  • Model-visible context/tools: none.
  • Runtime/lifecycle: one bounded observation is emitted after each assistant turn; tracker state resets per Session.
  • Persisted config/data: none.
  • Compatibility/risk: additive event surface. Provider classification is intentionally conservative and unsupported providers remain unknown.

A late turn_end after /new, /reload, or quit could reuse a stale warm baseline and emit a false observation. Clear the tracker and pending identity at the same lifecycle boundary as session metrics.
tt-a1i
tt-a1i previously approved these changes Sep 5, 2026

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at exact head 7e283300f61f1e3bd5aac4c934e2eef95912c4c4. I independently reviewed the scoped implementation and boundary conditions, verified the current diff against main, and found no unresolved P0/P1 issues. Required CI checks are green at this head.

@tt-a1i
tt-a1i merged commit 24d7407 into openpi-dev:main Sep 5, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants