Repository navigation

feat(cpu-moe): serve block-fp8 experts on the cpu and hybrid backends - #662

Merged

jason-fxz merged 12 commits into

mainfrom

feat/cpu-moe-fp8-block

Oct 12, 2026

Merged

feat(cpu-moe): serve block-fp8 experts on the cpu and hybrid backends#662
jason-fxz merged 12 commits into
mainfrom
feat/cpu-moe-fp8-block

Commits on Oct 11, 2026
- feat(cpu-moe): block-FP8 expert GEMV (W8A16) for the cpu/hybrid backends
  
  gdevenyi
  authored and
  jason-fxz
  committed
  5becbb9
- fix(cpu-moe): honor padded fp8 scale strides
  
  ChenyuHeee
  authored and
  jason-fxz
  committed
  b853ff3
- test(cpu-moe): add float64 parity reference and an avx512f fp8 tier
  
  ChenyuHeee
  authored and
  jason-fxz
  committed
  f575f6f
- test(cpu-moe): cover fp8 dispatch and cpu-only regressions
  
  ChenyuHeee
  authored and
  jason-fxz
  committed
  20fbd74
- review: force each ISA tier in the fp8_block test, and stop the docstring overclaiming
  
  gdevenyi
  authored and
  jason-fxz
  committed
  5131361
- fix(moe): let fp8_block experts decode on the cpu executor
  jason-fxz
  committed
  798efff
- fix(cpu-moe): build the bf16 fp8 tier only when the compiler has avx512bf16
  jason-fxz
  committed
  69ce9b1
- fix(bench): version the bw profile and ignore older ones
  jason-fxz
  committed
  7347a0e
- perf(cpu-moe): read fp8 weights as fp16 bits in the cpu dot kernels
  jason-fxz
  committed
  b564477
- refactor(cpu-moe): write the fp8_block bank checks like the other formats
  jason-fxz
  committed
  b042d6e
Commits on Oct 12, 2026
- refactor(cpu-moe): name the fp8 cpu symbols fp8_block
  jason-fxz
  committed
  8dd36da
- docs(server): list fp8_block in the --moe-cpu-layers help
  jason-fxz
  committed
  194fbca