Skip to content

fix(cuda): pin speculative expert kernel family - #1234

Merged
JustVugg merged 2 commits into
JustVugg:devfrom
ZacharyZcR:fix/cuda-spec-kernel-pin
Aug 28, 2026
Merged

JustVugg merged 2 commits into
JustVugg:devfrom
ZacharyZcR:fix/cuda-spec-kernel-pin

Conversation

@ZacharyZcR

Copy link
Copy Markdown
Contributor

Summary

  • extend SPEC_PIN to the synchronous CUDA grouped-expert path so speculative verification cannot switch into row-count-gated W4A4 or W4A16 Tensor Core kernels
  • keep the shared-expert W4A16 path off during a pinned speculation window
  • add an optional DLL symbol; older CUDA DLLs fail closed to the CPU fallback when pinning is required

This addresses the uncovered CUDA half of #689: CPU dispatch was already pinned, while CUDA grouped experts could still change numeric families with verify-row depth. The existing non-speculative path is unchanged.

Verification

  • make -C c test-c
  • make -C c glm
  • make -C c glm CUDA_DLL=1 CC=x86_64-w64-mingw32-gcc
  • git diff --check

test_backend_cuda now checks that a pinned group remains byte-identical to the small-batch grouped path even when W4A4 or W4A16 Tensor Core dispatch is enabled. This host has no NVIDIA GPU, so H100/H200 deep-draft token identity and acceptance-rate reproduction remains a hardware validation gate rather than a claimed result.

@ZacharyZcR
ZacharyZcR force-pushed the fix/cuda-spec-kernel-pin branch from 26b6d05 to 9f4a8d1 Compare August 28, 2026 02:06
@JustVugg
JustVugg merged commit f2e4849 into JustVugg:dev Aug 28, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants