Skip to content

fix: stabilize remote compaction streams#699

Merged
chenyme merged 1 commit into
mainfrom
compaction
Jul 18, 2026
Merged

fix: stabilize remote compaction streams#699
chenyme merged 1 commit into
mainfrom
compaction

Conversation

@chenyme

@chenyme chenyme commented Jul 18, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes intermittent Codex session interruptions that appeared in request audits as successful 200 / Stream / 0 Tokens requests.

These requests are Codex Remote Compaction V2 calls sent through POST /v1/responses with a terminal compaction_trigger. Codex requires exactly one compaction output item followed by response.completed; a stream ending before that point must not be treated as successful.

Root cause

The gateway previously had several gaps around compaction and streaming responses:

  • compaction_trigger requests were classified as ordinary Responses requests.
  • Route selection did not explicitly restrict compaction to Providers that support it.
  • EOF was treated as successful stream completion even when the required terminal event was missing.
  • Old, mismatched, or cross-instance gateway compaction blobs could return a hard 400 and block the session.
  • Partial upstream usage could produce an incomplete synthetic response.completed payload.
  • Request audits displayed compaction calls as generic 200 / Stream / 0 Tokens.

Changes

Codex compaction

  • Detect compaction_trigger items in /v1/responses.
  • Classify these requests as the new compaction audit operation.
  • Route compaction only to Providers that explicitly declare Compact support, currently Grok Build.
  • Preserve the existing adapter-level validation that the trigger appears exactly once and as the final input item.
  • Normalize partial compaction usage so standard token fields remain structurally valid.
  • Keep usage absent when the upstream omitted it entirely instead of fabricating token counts.
  • Replace undecodable or session-mismatched gateway blobs with a compatibility boundary message, retaining the remaining conversation history instead of returning a hard 400.

Stream integrity

Added protocol-aware terminal-event validation:

  • Responses: response.completed
  • Chat Completions: [DONE]
  • Anthropic Messages: message_stop
  • Image streaming: image_generation.completed

Streams are now classified as:

  • upstream_stream_incomplete
  • upstream_stream_interrupted
  • upstream_stream_error
  • stream_interrupted for downstream/client write failures

Transport-level incomplete or interrupted upstream streams temporarily cool the selected account so the client can retry against another account. Semantic failure events do not cool the account because they may be request-specific.

The gateway does not replay a stream after partial output, avoiding duplicated text or tool calls.

Audit and migration

  • Add compaction to the audit domain and frontend API contract.
  • Upgrade the request_audits.operation CHECK constraint for SQLite and PostgreSQL.
  • Keep the migration idempotent for existing databases.
  • Display compaction as “上下文压缩 / Context compaction”.
  • Display “上游未返回用量” when compaction usage is genuinely unavailable.
  • Show HTTP 200 streams carrying an internal stream error with warning styling instead of success styling.

Compatibility

  • No configuration changes are required.
  • Normal Responses, Chat Completions, Anthropic Messages, and image streaming behavior remains unchanged when their terminal events are valid.
  • Stored-response ownership is not created for compaction responses.
  • No stream is retried after downstream output has started.
  • Existing audit data and account state are preserved during migration.

Test plan

  • go test ./... -count=1
  • go vet ./...
  • go build ./cmd/grok2api
  • pnpm lint
  • pnpm build
  • git diff --check
  • Race tests:
go test -race \
  ./internal/transport/http/inference \
  ./internal/infra/provider/cli \
  ./internal/application/gateway \
  ./internal/infra/persistence/relational

@chenyme
chenyme merged commit d6ed4ff into main Jul 18, 2026
9 checks passed
@chenyme
chenyme deleted the compaction branch July 18, 2026 17:42
@chenyme
chenyme restored the compaction branch July 18, 2026 17:42
@chenyme
chenyme deleted the compaction branch July 18, 2026 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant