Repository navigation
fp16 decoder tags, the model lifecycle, and numbers from results/ (0.14.0) - #82
Merged
Merged
Conversation
… number from results/ - Split instead of SplitToSequence in every Qwen3.5 graph: the same parity numbers, 1.5 to 3 times faster fp32 (ADR 0008). - Six -fp16 tags (kev:4b, kev:9b, nimble:9b, clef:flash, pplx-decider:flash, strands-decider:2b), each through its own gate; kev:0.8b-fp16 passed but is not shipped, its fp32 graph being as fast. - docs/model-lifecycle.md and ADR 0009: a rolling edition, bench_plan decides what a release measures; parity_run, bench_plan, bench_run, bench_edition --update and scripts/bump-version.sh. - Pages, docs and the README take their numbers from results/; Ollama 0.40.6's runs are result files. - Edition 2026.10 updated by 0.14.0: strands-decider:2b, pplx-decider:flash, nimble:9b-v3, the fp16 tags and the new graphs' latency. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fp16 tags for the Qwen3.5 decoders, a written model lifecycle, and every site number from
results/.Runtime and models (ADR 0008)
Splitinstead ofSplitToSequence, which ONNX Runtime ran on the CPU. The values are unchanged: all 14 fp32 tags re-ran parity with the same numbers (results/runs/2026-10-19-parity-rtx4090-cuda.json).-fp16tags:kev:4b-fp16,kev:9b-fp16,nimble:9b-fp16,clef:flash-fp16,pplx-decider:flash-fp16andstrands-decider:2b-fp16. Each has its own gate: the encodings exact and at least 99.5 % of decisions the same as the fp32 reference. All pass, at 99.58–100 %.kev:0.8b-fp16passed too, but it is not shipped because the fp32 graph is as fast.ollaya run --askasks your own questions from the command line.decision's upstream org was renamed tovllm-sr.Process (ADR 0009, docs/model-lifecycle.md)
bench_plandecides what that is from the manifests and the parity records.parity_run,bench_plan,bench_run(resumable),bench_edition --updateandscripts/bump-version.sh.results/({{result <tag> <path> <digits>}},result(), generated README blocks). The build fails on a missing result or a stale README./resultshas a table with both servers.Measured (edition 2026.10, updated by 0.14.0)
Measured for 0.14.0 (
results/runs/2026-10-19-*): parity on the RTX 4090; typed-decisions, the median five-questionrequest over HTTP and the GPU memory on the RTX 5090 of Ollaya Bench 2026.10. The fp32 typed-decisions numbers are the
edition's; the fp32 graphs' outputs did not change.
kev:4b-fp16kev:9b-fp16nimble:9b-fp16clef:flash-fp16pplx-decider:flash-fp16strands-decider:2b-fp16kev:0.8b-fp16(not shipped)Typed-decisions accuracy is within 0.002 of the fp32 tag's for every fp16 tag.
kev:0.8b-fp16passed its gate but is notshipped: once
SplitreplacedSplitToSequence, the fp32 graph of the 0.8B model was as fast.Splitalone made every fp32 graph faster, with the same parity numbers (median five-question request on the RTX 5090):decider:0.8bdecider:2bdecider:4bdecision:eoskev:0.8bkev:4bkev:9bclef:flashjeeves:9bnimble:9bChecks
cargo fmt --check,clippy -D warningsandcargo test --workspacepass on Linux.python3 -m unittest discover -s convert/testspasses.npm run buildpasses.The registry manifests are not in this PR: they go in as "Registry: …" commits at release time.
🤖 Generated with Claude Code