mirror of
https://github.com/hanzoai/engine.git
synced 2026-11-20 14:17:52 +00:00
The packed varlen arm defaulted a missing FlashParams to "the mask says", so a quantized model's later chunk (no FlashParams, no mask) ran the fused kernel bidirectional. None means a causal decoder, as run_attention documents. The mask helper now keys on whether a fused varlen kernel will run, not on the device, so CUDA with flash off or an F32 model keeps its mask too. Co-authored-by: Hanzo Dev <[email protected]>
hanzo-engine
Core crate of hanzo including the models and associated executors.
Documentation: https://docs.rs/hanzo-engine/