mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-29 07:34:31 +00:00
zeekay
f041a6c1ca
Systems paper + numbers-as-data results. Honest board (Qwen3-1.7B-Q4_K_M, N=7, 95% CI): prefill at parity (CUDA WIN 1.09x at pp4096); decode behind on all four backends (CUDA 0.84, Metal 0.72, ROCm 0.72, Vulkan 0.52), decomposed into APU bandwidth-utilization (measured 213 GB/s peak; llama at 93%, us at 48-72%) and discrete fixed-overhead that a 4B control amortizes. Root-causes the Vulkan pp4096 cliff (not VK_WORK_CAP -- refuted). Overturns several internal claims against a properly-built baseline. latexmk-clean.
29 lines
1.5 KiB
TeX
29 lines
1.5 KiB
TeX
% Architecture: one request path, one kernel source, four backends.
|
|
\begin{figure}[t]\centering
|
|
\begin{tikzpicture}[
|
|
font=\scriptsize,
|
|
box/.style={draw=hzslate,rounded corners=2pt,align=center,inner sep=3pt,minimum height=6mm},
|
|
api/.style={box,fill=hzblue!12,draw=hzblue},
|
|
sched/.style={box,fill=hzslate!10},
|
|
dsl/.style={box,fill=hzgreen!14,draw=hzgreen},
|
|
be/.style={box,fill=hzslate!8,minimum width=13mm},
|
|
ar/.style={-{Latex[length=1.6mm]},draw=hzslate},
|
|
node distance=3.2mm and 5mm]
|
|
\node[api] (req) {\texttt{/v1} request};
|
|
\node[sched,below=of req] (sch) {scheduler\\{\tiny paged KV $\cdot$ Orca batching $\cdot$ spec-decode}};
|
|
\node[sched,below=of sch] (pipe) {pipeline\\{\tiny GQA flash-SDPA $\cdot$ block-MoE $\cdot$ fused router}};
|
|
\node[dsl,below=of pipe] (dsl) {hanzo-fusion DSL \;{\tiny(auto-fusion, hanzo-cl)}};
|
|
\node[be,below left=6mm and 8mm of dsl] (cuda) {CUDA};
|
|
\node[be,right=of cuda] (metal) {Metal};
|
|
\node[be,right=of metal] (hip) {HIP/ROCm};
|
|
\node[be,right=of hip] (vk) {Vulkan};
|
|
\draw[ar] (req)--(sch); \draw[ar] (sch)--(pipe); \draw[ar] (pipe)--(dsl);
|
|
\draw[ar] (dsl.south)--(cuda.north); \draw[ar] (dsl.south)--(metal.north);
|
|
\draw[ar] (dsl.south)--(hip.north); \draw[ar] (dsl.south)--(vk.north);
|
|
\end{tikzpicture}
|
|
\caption{One binary. A single kernel source in the hanzo-fusion DSL lowers to CUDA,
|
|
Metal, HIP, and SPIR-V; the scheduler and pipeline (paged KV, Orca continuous batching,
|
|
prompt-lookup speculation, fused GQA/MoE) are backend-independent above the DSL cut.}
|
|
\label{fig:arch}
|
|
\end{figure}
|