zeekay f041a6c1ca hanzo-engine: portable single-binary LLM inference measured vs llama.cpp across 4 backends ...

Systems paper + numbers-as-data results. Honest board (Qwen3-1.7B-Q4_K_M, N=7, 95% CI):
prefill at parity (CUDA WIN 1.09x at pp4096); decode behind on all four backends
(CUDA 0.84, Metal 0.72, ROCm 0.72, Vulkan 0.52), decomposed into APU bandwidth-utilization
(measured 213 GB/s peak; llama at 93%, us at 48-72%) and discrete fixed-overhead that a
4B control amortizes. Root-causes the Vulkan pp4096 cliff (not VK_WORK_CAP -- refuted).
Overturns several internal claims against a properly-built baseline. latexmk-clean.

2026-07-21 18:42:58 -07:00

..

architecture.tex

hanzo-engine: portable single-binary LLM inference measured vs llama.cpp across 4 backends

2026-07-21 18:42:58 -07:00