feat(gguf): serve MiniMax-M2 and MiniMax-M3 GGUF checkpoints

[jason-fxz]

Summary

Serves llama.cpp minimax-m2 GGUF files (MiniMax-M2, M2.1, M2.5, M2.7 and their REAP prunes) on the existing minimax_m2 model classes, and minimax-m3 files (MiniMax-M3, text only) on the existing minimax_m3 classes. The tokenizer fixes MiniMax needs (control tokens with spaces, the phantom <|endoftext|> at id 200064) came with #722.

Changes

• models/minimax_m2/gguf.py (new):
  • Builds the config from the GGUF metadata: partial rope (64 of 128 dims), per-layer q/k norms, sigmoid routing with the selection bias, fewer experts for REAP prunes.
  • Maps checkpoint names to GGUF tensors. The converter only stacks the experts, and both sides run NeoX rope and plain RMSNorm, so nothing is undone.
  • Reads the single attn_qkv matrix of files converted with --fuse-qkv as qkv_proj.
  • Refuses gating other than sigmoid, rope scaling, prism files, per-tensor scales, and old conversions that name the router bias exp_probs_b without .bias (llama.cpp refuses them too).
• models/minimax_m3/gguf.py (new):
  • Builds the config from the GGUF metadata, with the values llama.cpp hard-codes for this arch: swiglu-oai alpha 1.702 / limit 7, max block score, sparse attention on every layer after the leading dense ones.
  • The converter adds 1 to every RMSNorm weight so llama.cpp can run plain RMSNorm. The loader subtracts it for this family's (1 + w) norms.
  • Refuses older conversions without the sparse-attention indexer (llama.cpp refuses them too), gating other than sigmoid, routing without renormalized weights, and rope scaling.
• models/minimax_m3/attention.py, mlp.py: the fused qkv_proj and gate_up_proj pass output_sizes, so the GGUF linear runs one GEMM per ggml type. The HF path does not change: nvidia/MiniMax-M3-NVFP4, MiniMaxAI/MiniMax-M3 and nvidia/MiniMax-M2.5-NVFP4 get the same method, kernel and scheme on every quantized layer before and after.
• model.py of both families: embed_tokens takes the quant config, so a quantized token_embd is served packed.
• moe/expert_banks.py: GGUF expert banks start at blk.<first_k_dense_replace>. Before, a GGUF with leading dense layers (M3 has 3) looked for experts in blk.0.
• register.py: the GGUF specs list the experts as (w1, w3, w2), the (gate, up, down) order in which the GGUF scheme reads their types. Most MiniMax files store ffn_down_exps in another type than gate/up (e.g. Q3_K vs Q2_K).
• gguf/config.py, register.py, kernel/aot_models.py: register the two archs; the existing MiniMax-M2.5 and MiniMax-M3 AOT entries claim them.

Tested models

RTX PRO 6000 Blackwell (sm_120), ft serve --model-path <file> --max-running-requests 16. AIME25 problems 1–8 × 4 samples with each model's generation_config sampling (M2: temperature 1.0, top_p 0.95, top_k 40; M3: temperature 1.0, top_p 0.95). llama.cpp f2cc728 ran with the same sampling and min_p 0. Also 48 staggered requests at concurrency 16.

Model	Quant	bs=1 tok/s	AIME FreeToken / llama.cpp f2cc728
bartowski/cerebras_MiniMax-M2-REAP-139B-A10B-GGUF	Q2_K, down Q3_K in 30 layers	126	32/32 / 30/32
unsloth/MiniMax-M2-GGUF	UD-TQ1_0, IQ1_S–IQ4_XS experts	135	–
bartowski/MiniMax-M3-GGUF	IQ2_XXS, offload (cache rate 0.5)	60	26/32 / –

M3's 6 misses (problems 2 and 7) all hit the 32768-token limit. llama.cpp did not run M3's AIME: on this card it keeps 26 of the 57 expert layers on the CPU, about 7 tok/s per slot.

Staggered requests: no corrupted output on any file.

Results

• Weights: after dequantization, against cerebras/MiniMax-M2-REAP-139B-A10B and MiniMaxAI/MiniMax-M3 at the first and last layers: every F32 tensor is exact (norms, router, selection bias, M3 indexer), M3's norms after the 1 is taken off. Q8_0 tensors reach cos 0.99998, Q6_K 0.9998, Q4_K 0.997. Per expert, gate / up / down match w1 / w3 / w2 at cos 0.93–0.99 (Q2_K–IQ2_XXS), while gate vs w3 is 0.00–0.38. Dry load against the meta-built model matches every param: 623 (REAP Q2_K), 611 (UD-TQ1_0), 798 (M3 IQ2_XXS).
• Tokenizer: the GGUF tokenizer equals HF tokenizer.json of MiniMax-M2, M2.5, M2.7 and M3 on 46 strings (control tokens with spaces, <think>, tool-call markup, rendered chat templates with tools) for bartowski and unsloth files, and every added token has HF's id and special flag. HF also NFC-normalizes, which llama.cpp does not.
• vs llama.cpp f2cc728: same file, greedy, 6 prompts × 128 tokens, identical prompt tokens. Identical outputs: REAP 1/6, M3 2/6, UD-TQ1_0 0/6. At every divergence FreeToken's pick is in llama.cpp's own top 3, mostly a near-tie (e.g. 0.425 vs 0.414; widest 0.325 vs 0.171 on M3). On UD-TQ1_0, when llama-server scores the same prefix again, FreeToken's pick is its top token in 5 of the 6 divergences.
• Coverage: header check (dry load, kernel coverage, tokenizer, chat template) of 1489 files from 180 HF repos: 1309 pass (minimax-m2 1237/1337, minimax-m3 72/114). The rest: 58 IQ1_M, 38 old M3 conversions without the indexer (refused), 33 without a chat template, 20 draft models, 18 in ggml types llama.cpp lacks, 6 old M2 conversions with exp_probs_b (refused), 6 whose gate and up experts differ in type, 1 with TQ1_0 linears.
• Unit tests: tests/models, tests/moe and tests/kernels/test_ggml.py pass, except failures that also fail on main: 5 test_quant_config cases and test_offload.py::test_graph_capture_reuses_warm_offload_cache_before_capture.

Not covered

• M3 image input (the mmproj files).
• IQ1_M weights (no MMQ kernel), and files whose gate and up experts differ in ggml type.
• Conversions from before the current llama.cpp layout (refused).
• TP > 1.