Files
zeekayandHanzo Dev dd284c479f paged-attn: one default per device, for the server and the bench alike
The server turned PagedAttention on for ROCm. hanzo-bench kept its own copy of
that decision and turned it off there, to match llama-bench's contiguous KV. A
paged cache is what lets the decode step replay as a captured graph, so the
bench timed an eager path the server never runs: on a 27B hybrid on gfx1151,
0.92 tok/s where the served path replays a token in 105 ms.

paged_attn_default(device) in the engine is now the one answer. The server's
five per-device constants are gone, and the bench's flags only override it.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-17 11:40:33 -07:00
..