Files
zeekayandHanzo Dev 688112e67c perf(rocm): GGUF gated-delta-net layers run the fused conv and mixer
On ROCm the quantized GDN layer (Qwen3.5/3.6, zen6) now runs:
- its causal conv as hanzo-ml's gdn_conv_rocm, for prefill and decode alike, with
  the conv state advanced in place in the pool;
- everything from the conv output to the gated norm's output as gdn_mixer_rocm:
  l2 norms, gates, the delta-rule recurrence and the output norm, in one launch.

A forward that keeps a trail for speculative rewind still steps the recurrence per
position.

sigmoid() calls hanzo_nn's op everywhere. hanzo-ml's ROCm backend runs it as one
kernel, which retires the five-launch composition kept for ROCm.
RmsNormGated::gates_with_sigmoid and ::eps exist for the fused mixer alone, so they
build with `--features rocm`.

Measured on evo (gfx1151), Qwen3.8-Flash-Next UD-IQ4_XS, engine bench:
- pp512: 153 -> 344 t/s
- pp2048: 147 -> 387 t/s
- tg64: 16.5 -> 20.0 t/s

Co-authored-by: Hanzo Dev <[email protected]>
2026-10-17 19:33:35 -07:00
..

hanzo-engine

Core crate of hanzo including the models and associated executors.

Documentation: https://docs.rs/hanzo-engine/