feat(glm53): continue from the cache instead of re-prefilling

[enitimeago]

Summary

Related to #1748.

Stopping a reply and clicking continue can make the glm53 engine process the entire prompt again, even though it has already cached the work.

The existing reuse path requires at least one new prompt token, hence an exact match misses, and so does a shorter prompt after the client trims trailing whitespace tokens to satisfy the continuation API. (For example, "# Hummingbird\n\n" cannot be continued by sending "# Hummingbird", whilst "# Hummingbird\n\n" is rejected by the gateway.)

This change supports preserving that work for two continuation cases:

• Exact match (equal): retain the final logits row, which predicts the next token at the cached position. The next request can use that row even though it has no new prompt tokens to process.
• Trimmed whitespace (shorter): save the KDA recurrent state, convolution windows, and logits before feeding the first whitespace-only token of a generated run. If the next prompt ends at that saved position and matches the cached prefix, restore it. This handles a client trimming the reply before continuation; the gateway rejects trailing whitespace.

Rules for both resume points:

• They are valid only for the turn right after the one that made them. Each turn decides, clears them, and writes them again at the end.
• Token IDs are checked against the slot's history before any pin restore.
• Requests with image embeddings or logprobs use the existing path, and image turns leave no resume points: matching placeholder IDs cannot establish that the image embeddings match.
• Whitespace snapshots recognize only ASCII whitespace-only tokens. A shorter prompt must match the saved boundary exactly; other cases use the existing prefill and pin logic.

This change does not provide arbitrary cache rewind.

The trimmed whitespace case is opt-in (GLM53_REWIND=0, documented in docs/ENVIRONMENT.md). It requires a buffer the size of the KDA state, about 149 MiB per slot, plus a logits row, on first use; the buffer is retained across slot resets and reused. The state is copied once at the start of each eligible generated whitespace run, including during ordinary generation.

Exact-match resumption stays on, since it costs one logits row.

Validation

• make -C c check
• CUDA changes were tested with make -C c cuda-test (if applicable) (N/A; no CUDA changes)
• Performance claims include hardware, commands, and repeatable measurements (N/A; no timing or throughput claims)
• Performance claims include a validated experiment manifest with raw evidence (N/A; no timing or throughput claims)

Serve harness checks (glm53-oracle CI job and vision serve harness passed locally):

• Continue after a mid-turn CANCEL reuses the exact cached prompt (REUSE 20 3 3 3 3 equal) and matches a fresh engine's output.
• A generated run of two whitespace tokens is tested at two stopping points. Ending the budget at the second whitespace token leaves only the first cached. Allowing one more token of budget feeds both whitespace tokens, exercising the snapshot guard on the second as well. Continuing from the prefix before the run must report shorter, reuse that prefix, and match a fresh engine's output in both cases. These are deterministic cache-state tests using token budgets; the existing CANCEL case covers cancellation.
• The first whitespace case reuses zero tokens with GLM53_REWIND=0.
• The vision serve harness ends an image turn after one token, then sends the same prompt without IMAGE. It must reuse nothing; without the image rule it resumed all 22 tokens.

Compatibility

• The default CPU build remains dependency-free
• No model files, generated binaries, or benchmark artifacts are included

[JustVugg]

Thanks, this works: the oracles and serve harnesses pass, also under ASan and UBSan, including the new equal and shorter cases compared against a fresh engine.

One change before merging, about the default. The whitespace snapshot costs about 149 MiB per slot outside the GLM53_EXPERT_GB budget, and a full state copy at every generated whitespace run. Every reply has whitespace runs, so with the default on, every user pays both the memory and the copies during ordinary generation, including users who never continue a reply. On the machines GLM-5.3 targets that margin is small. Please make GLM53_REWIND default to 0 (opt-in) and update docs/ENVIRONMENT.md; exact-match resumption stays on, since it costs one logits row.

About #1792 (image-keyed KV prefix): whichever of the two lands second has to fold in the other. The points are:

• slot_shared and slot_remember work on the keys;
• keys[total] = next stays in the reworked decode loop;
• slot_pin_restore(..., common) sits under resume ? 0 :;
• vision_new, n_new are passed to forward_prefill;
• the "no resume point after an image turn" rule should test image_here rather than n_vision == 0, since after glm53: keep the KV prefix across image requests by keying the picture #1792 a turn whose picture was fully reused also has n_vision == 0.

[enitimeago]

@JustVugg Done in 1ba5b09.

GLM53_REWIND now defaults to 0 and is enabled with GLM53_REWIND=1; docs/ENVIRONMENT.md is updated and notes the snapshot sits outside GLM53_EXPERT_GB. Exact-match resumption is unchanged. The serve harness turns rewind on for the shorter cases and now checks that a trimmed continue re-prefills under the default. Harness and tiny oracles pass.

Noted on the image-keyed KV prefix PR, including image_here instead of n_vision == 0 for the no-resume-after-image rule. I'll fold it in if this lands second.