Files
zeekayandHanzo Dev 7ca92acd41 ROCm GPU inference: rotary dispatch, paged-attn gating + native-stack changes
ROCm:
- layers.rs: on-device HIP rope dispatch (rocm_rope_positions_qk) + per-token
  positions expansion in the gather fallback
- utils/mod.rs: exclude ROCm from paged_attn_supported() (HIP PagedAttention v1
  kernel hangs at runtime on gfx1151) so ROCm defaults to working non-paged attn

Plus this session's native-stack tree changes (quant fast_mmq/mmvq, gemv, moe,
diffusion/speech/multimodal, zen3-serving, dub e2e tests).

ROCm GGUF Q8_0 on gfx1151: ~112 tok/s decode, ~96x CPU.

Co-authored-by: Hanzo Dev <[email protected]>
2026-06-11 12:41:24 -07:00
..