Files
zeekay d7af2339a8 test(glm-dsa): CPU oracle proving GLM-5.2 forward matches transformers token-for-token
Colocated CPU integration test loads a tiny random-weight glm-dsa GGUF (split-MLA
+ DSA lightning indexer + sigmoid/noaux MoE + shared expert) on Device::Cpu and
asserts the forward matches a transformers reference token-for-token: teacher
forcing over 32 tokens (argmax positions 11..=30) and 20-step greedy decode. With
index_topk=4096 >> seq and no attn_indexer_* tensors, DSA degenerates to dense
attention, validating the whole forward without the sparse path. Fixtures
(glm_tiny_f32.gguf + ref_glm.json) committed; converter scripts/tiny_glm_to_gguf.py.

Fix: fused-MoE forward assumed quantized (QTensor) expert banks; an all-F32 GGUF
dequantizes the bank to a plain Tensor, which moe_gate_up/indexed_moe_forward do
not cover -> panic. Add a dense per-expert twin (same gather->matmul->scatter
contract) that activates only for unquantized banks, leaving the quantized path
untouched. Token-exact F32 is required for argmax parity with the f32 oracle.
2026-07-13 12:10:25 -07:00
..