Files
engine/hanzo-engine/tests/fixtures/ref_glm.json
zeekay d7af2339a8 test(glm-dsa): CPU oracle proving GLM-5.2 forward matches transformers token-for-token
Colocated CPU integration test loads a tiny random-weight glm-dsa GGUF (split-MLA
+ DSA lightning indexer + sigmoid/noaux MoE + shared expert) on Device::Cpu and
asserts the forward matches a transformers reference token-for-token: teacher
forcing over 32 tokens (argmax positions 11..=30) and 20-step greedy decode. With
index_topk=4096 >> seq and no attn_indexer_* tensors, DSA degenerates to dense
attention, validating the whole forward without the sparse path. Fixtures
(glm_tiny_f32.gguf + ref_glm.json) committed; converter scripts/tiny_glm_to_gguf.py.

Fix: fused-MoE forward assumed quantized (QTensor) expert banks; an all-F32 GGUF
dequantizes the bank to a plain Tensor, which moe_gate_up/indexed_moe_forward do
not cover -> panic. Add a dense per-expert twin (same gather->matmul->scatter
contract) that activates only for unquantized banks, leaving the quantized path
untouched. Token-exact F32 is required for argmax parity with the f32 oracle.
2026-07-13 12:10:25 -07:00

1 line
391 B
JSON

{"prompt_ids": [3, 14, 159, 26, 53, 58, 200, 11, 77, 240, 5, 99], "full_ids": [3, 14, 159, 26, 53, 58, 200, 11, 77, 240, 5, 99, 207, 187, 119, 103, 103, 103, 103, 103, 119, 34, 103, 103, 103, 103, 103, 136, 112, 7, 119, 34], "tf_pred": [139, 197, 123, 34, 197, 34, 197, 197, 197, 193, 193, 207, 187, 119, 103, 103, 103, 103, 103, 119, 34, 103, 103, 103, 103, 103, 136, 112, 7, 119, 34, 103]}