Skip to content

fix(qwen35): per-arch e2e logits budget from measurement + sm_121 nightly lane (Closes #4523) - #4532

Closed
noahgift wants to merge 9 commits into
mainfrom
fix/4523-sm121-logits-budget
Closed

noahgift wants to merge 9 commits into
mainfrom
fix/4523-sm121-logits-budget

Conversation

@noahgift

Copy link
Copy Markdown
Contributor

Closes #4523

What

The qwen35 e2e CUDA-vs-CPU parity test measured 7.105e-2 relative L-inf on gx10 (sm_121/aarch64), against a 7e-2 budget. This is not an sm_121 defect:

comparison relative L-inf
GPU sm_89 (4090) vs GPU sm_121 (GB10) logits 6.466e-7
CPU ref x86_64 (AVX2) vs CPU ref aarch64 (NEON) 2.602e-2

The GPUs agree with each other. The difference comes from the CPU reference (Q8_K activation quantization, SIMD path per arch). So the budget belongs to the host CPU arch. It is derived from measurement (the worst reading rounded up to one significant figure), not widened by hand:

  • x86_64: worst 6.721e-2 → 7e-2 (unchanged)
  • aarch64: worst 7.105e-2 → 8e-2

The falsifier: the old 7e-2 budget fails 5/5 on gx10. With the fix, gx10 runs the module 17/17 with 0 skips, and lambda passes at 7e-2.

Also

  • QWEN35_E2E_DUMP=<dir> is a measurement mode that writes per-position GPU/CPU logits. It always fails, so it can never read as a pass.
  • .github/workflows/cuda-nightly.yml gets a new step in the gx10 job that runs the qwen35 CUDA parity module on sm_121. The step is RED on a non-zero rc, on any SKIP/absent line, or on 0 passed. The PR-time cuda-unit lane has no models, so the module SKIPs there. This is a workflow edit, with no runner-host or secret changes.
  • Findings: docs/findings/4523-sm121-logits.jsonl (PREREG, M1–M3). Roadmap: PMAT-4523.
  • Quorum: docs/audits/quorum-PMAT-4523.json. AGREED 3/3 PASS (gemini-3.1-pro-high ×2 and claude-haiku-4-5; the author is claude-opus-5-5).

🤖 Generated with Claude Code

noahgift and others added 6 commits September 27, 2026 11:14
…vs sm_121 GPU)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the first over-budget one

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… sm_121 GPU

GPU sm_89 vs sm_121 pos0 logits agree to 6.5e-7; x86 AVX2 vs aarch64 NEON CPU
references differ by 2.6e-2. aarch64 budget = measured worst 7.105e-2 rounded up
(8e-2); x86_64 stays 7e-2. QWEN35_E2E_DUMP now fails after printing, never passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y gx10) + findings

yoga-eph has no models, so the module SKIPped at PR time and the whole-forward
budgets were only ever read on sm_89/x86_64. The gx10 nightly holds the model;
the step refuses a skip or an empty selection. Verified: 17/17 on GB10.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noahgift
noahgift enabled auto-merge September 27, 2026 10:31
@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=4532 head=ef44857b9f00c496daafc6617b0e32683e7d34fd verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

@noahgift noahgift added the owner:aprender-41 owning session (cop inbox claims) label Sep 27, 2026
@noahgift

Copy link
Copy Markdown
Contributor Author

Moved into #4607 (PRCAP fold, cop order 14:02Z). Fold = MOVE: head ef44857b9f00c496daafc6617b0e32683e7d34fd is an ancestor of the pushed fold head, so no work is lost. The branch is kept. Agent: aprender-59

@noahgift noahgift closed this Sep 28, 2026
auto-merge was automatically disabled September 28, 2026 14:24

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

owner:aprender-41 owning session (cop inbox claims)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

qwen35 CUDA-vs-CPU e2e logits exceed LOGITS_BUDGET 7e-2 on GB10 sm_121 (7.105e-2, 2/2 deterministic; budget measured on 4090 only)

1 participant