mirror of
https://github.com/hanzoai/engine.git
synced 2026-11-21 02:36:58 +00:00
Colocated CPU integration test loads a tiny random-weight glm-dsa GGUF (split-MLA + DSA lightning indexer + sigmoid/noaux MoE + shared expert) on Device::Cpu and asserts the forward matches a transformers reference token-for-token: teacher forcing over 32 tokens (argmax positions 11..=30) and 20-step greedy decode. With index_topk=4096 >> seq and no attn_indexer_* tensors, DSA degenerates to dense attention, validating the whole forward without the sparse path. Fixtures (glm_tiny_f32.gguf + ref_glm.json) committed; converter scripts/tiny_glm_to_gguf.py. Fix: fused-MoE forward assumed quantized (QTensor) expert banks; an all-F32 GGUF dequantizes the bank to a plain Tensor, which moe_gate_up/indexed_moe_forward do not cover -> panic. Add a dense per-expert twin (same gather->matmul->scatter contract) that activates only for unquantized banks, leaving the quantized path untouched. Token-exact F32 is required for argmax parity with the f32 oracle.
1 line
391 B
JSON
1 line
391 B
JSON
{"prompt_ids": [3, 14, 159, 26, 53, 58, 200, 11, 77, 240, 5, 99], "full_ids": [3, 14, 159, 26, 53, 58, 200, 11, 77, 240, 5, 99, 207, 187, 119, 103, 103, 103, 103, 103, 119, 34, 103, 103, 103, 103, 103, 136, 112, 7, 119, 34], "tf_pred": [139, 197, 123, 34, 197, 34, 197, 197, 197, 193, 193, 207, 187, 119, 103, 103, 103, 103, 103, 119, 34, 103, 103, 103, 103, 103, 136, 112, 7, 119, 34, 103]} |