mirror of
https://github.com/hanzoai/engine.git
synced 2026-11-20 14:17:52 +00:00
On ROCm the quantized GDN layer (Qwen3.5/3.6, zen6) now runs: - its causal conv as hanzo-ml's gdn_conv_rocm, for prefill and decode alike, with the conv state advanced in place in the pool; - everything from the conv output to the gated norm's output as gdn_mixer_rocm: l2 norms, gates, the delta-rule recurrence and the output norm, in one launch. A forward that keeps a trail for speculative rewind still steps the recurrence per position. sigmoid() calls hanzo_nn's op everywhere. hanzo-ml's ROCm backend runs it as one kernel, which retires the five-launch composition kept for ROCm. RmsNormGated::gates_with_sigmoid and ::eps exist for the fused mixer alone, so they build with `--features rocm`. Measured on evo (gfx1151), Qwen3.8-Flash-Next UD-IQ4_XS, engine bench: - pp512: 153 -> 344 t/s - pp2048: 147 -> 387 t/s - tg64: 16.5 -> 20.0 t/s Co-authored-by: Hanzo Dev <[email protected]>
hanzo-engine
Core crate of hanzo including the models and associated executors.
Documentation: https://docs.rs/hanzo-engine/