feat(runtime): cross-request prefix seeding via a content-addressed seed store - #90

Closed

igorls wants to merge 62 commits into

Neroued:masterfrom

igorls:feat/prefix-seed-store

Commits on Aug 25, 2026

fix(ops): size persistent grids from the active device's SM count
igorls
committed
f52e209
feat(runtime): cross-request prefix seeding via a content-addressed seed store
igorls
committed
bf3fad9
fix(runtime): budget the seed-capture chunk split and skip trivial seeds
igorls
committed
6b5294b
fix(serve): accept content parts in tool messages
igorls
committed
9dda665
feat(serve): accept vLLM enable_thinking dialect
igorls
committed
2605c5d
merge(serve): accept vLLM enable_thinking dialect (issue #2, upstream #50)
igorls
committed
352a49c
perf(ops): prefetch next projection weights from moe d4 tail

ARVLab
authored and
igorls
committed
74c52a8
perf(ops): prefetch shared expert down weights behind moe router

ARVLab
authored and
igorls
committed
5064a2d
perf(ops): fuse sigmoid gate into attention reduce epilogue

ARVLab
authored and
igorls
committed
4570fde
perf(ops): fuse q/k rmsnorm into one decode kernel

ARVLab
authored and
igorls
committed
375cff9
perf(ops): fold the moe router selection into the last D1 block

MichaelDementii
authored and
igorls
committed
4672e52
docs: add decode micro-opt A/B GPU runbook
igorls
committed
b728ee7
fix(serve): name the exception that terminates the process
igorls
committed
3c66e2b
fix(serve): fail fast on warmup failure and clarify auto kv capacity bounds
igorls
committed
9b6ff3c
feat(platform): native Windows (MSVC + CUDA) build for ninfer-serve

devan-carlin
authored and
igorls
committed
f6fbd86
fix(platform): Windows vision build fixes (FFMPEG/curl + I/O)

devan-carlin
authored and
igorls
committed
8f52f87
docs: add native Windows vs WSL2-tax A/B runbook
igorls
committed
6c10b7f
merge(platform): native Windows MSVC+CUDA build via upstream #84 (issue #6, phase 1)
igorls
committed
b79b5d2
feat(serve): recover drifted Qwen tool calls with tolerant tool calling mode
igorls
committed
31bf1b5
feat(serve): reconcile content parts with allowed_types support
igorls
committed
c09eb18
docs(serve): add live model runbook procedures for tool-calling robustness
igorls
committed
adee82d
fix(ops): fence last-block MoE D1 score loads
igorls
committed
e2511e9
feat(serve): report prefix-cache hits in usage
igorls
committed
2ebef26
merge(serve): prefix-cache observability in usage + models context (issue #3)
igorls
committed
d3ee0d3
fix(serve): add pre-listen boot watchdog and clean runbook
igorls
committed
326a5ff
merge(serve): fail fast on warmup failure + boot watchdog (issue #4, upstream #79+#54)
igorls
committed
ee6de59
fix(serve): keep the boot watchdog armed through exception unwind
igorls
committed
811e7b5
fix(serve): faithfully adapt upstream #10 tolerant tool call parser and add algorithm include
igorls
committed
6aa77e9
merge(serve): tool-call recovery (upstream #10) + content-parts reconciliation (issue #5)
igorls
committed
ba72762
docs(serve): drop stale near-miss wording from the parser header
igorls
committed
e6f15d1
merge(ops): decode micro-opts from upstream #69+#67, validated +4.4% (issue #1)
igorls
committed
db7a122
fix(runtime): include prefix-seed arena in device reservation
igorls
committed
e6c261a
merge(runtime): include prefix-seed arena in device reservation (issue #8)
igorls
committed
72a274c
fix(serve): type tool-call args by schema and accept JSON payloads
igorls
committed
c29e538

Commits on Aug 26, 2026

Commits on Aug 27, 2026