Conversation
|
Thanks for splitting this out. The hardening is right: We checked every expected count against the forward pass on dev, including the GLM MTP norms (
One thing blocks it. Since #1758, dev generates header dependencies with tests/test_colibri_read_trust$(EXE): tests/test_colibri_read_trust.c colibri.c $(VK_OBJ)
tests/test_inkling_read_trust$(EXE): tests/test_inkling_read_trust.c inkling.c $(INK_CUDA_OBJ) $(METAL_OBJ)
tests/test_olmoe_read_trust$(EXE): tests/test_olmoe_read_trust.c olmoe.c(keep whatever recipe line each rule already has). A small note on the description: the 35B MoE runs exercise qwen36, which this PR does not touch, so the tiny oracles above are the relevant evidence for colibri, inkling and olmoe. Your CI has not run yet because this is a first contribution; we will approve it after the update and merge when it is green. |
Splits out the loader-hardening part of #1564, as requested. Rebuilt from scratch on current
dev(4521832a) so it carries no unrelated changes.Problem
load_t()sized tensors by reading whatever the file contained. A truncated or mismatched container could therefore be read as a config-sized tensor, and the resulting out-of-bounds access happened during inference rather than at load time, where it can be refused cleanly.Change
Pass the expected element count to the loader at every call site in
colibri.c,inkling.candolmoe.c, and refuse a mismatch before allocation or read. Upstream'sembed_norm_name()handling ininkling.cis preserved.The guard runs before allocation, so no correctly shaped container is rejected and no short one is ever read at config width.
Verification
Rebuilt onto current
dev; no upstream behaviour is removed (the only deletions are loader call sites rewritten to pass the expected cardinality).make colibri inkling olmoeand thebuild/segment/{inkling,olmoe}.oobjects — cleanmake test-c— full suite passestest_colibri_read_trust,test_inkling_read_trust,test_olmoe_read_trust— all passIdentical to pristine
devon the same containers.Notes
This replaces the loader-hardening portion of #1564. The rest of that PR is still being split out separately; this one stands alone and can be reviewed and merged on its own.