mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-29 07:34:31 +00:00
zeekay
0d1d121451
Fix shared/ include paths to ../shared/ for 55 subdirectory papers so
that hanzocover.sty and lstlang.tex resolve when building from each
paper's own directory (the build pattern used by latexmk per paper).
Build fixes:
- hanzo-operative/sections/03-screen-understanding-pipeline.tex: move
\COMMENT from after \FOR{...} onto the \STATE line so the algorithmic
env doesn't mis-parse.
- hanzo-network-architecture (abstract + sections 04, 12, 13): escape
unicode mu (U+03BC) as $\mu$ for pdflatex.
- ntt-transform/sections/08-benchmark-results.tex: same mu escape.
All 55 Tier-2/3 papers now build clean with latexmk -pdf.
74 lines
2.8 KiB
TeX
74 lines
2.8 KiB
TeX
% Hanzo GPU Inference: GPU-Accelerated AI Inference on Hanzo Cloud
|
|
\documentclass[11pt]{article}
|
|
\usepackage{../shared/hanzocover}
|
|
\usepackage{amsmath, amssymb, amsthm}
|
|
\usepackage{mathtools}
|
|
\usepackage{bm}
|
|
\usepackage{graphicx}
|
|
\usepackage{booktabs}
|
|
\usepackage{hyperref}
|
|
\usepackage{enumitem}
|
|
\usepackage{algorithm}
|
|
\usepackage{algpseudocode}
|
|
\usepackage{xcolor}
|
|
\usepackage{float}
|
|
\usepackage{multirow}
|
|
|
|
\hypersetup{colorlinks=true,linkcolor=black,citecolor=blue,urlcolor=blue}
|
|
|
|
\newtheorem{theorem}{Theorem}
|
|
\newtheorem{lemma}[theorem]{Lemma}
|
|
\newtheorem{proposition}[theorem]{Proposition}
|
|
\newtheorem{corollary}[theorem]{Corollary}
|
|
\newtheorem{definition}{Definition}
|
|
\newtheorem{remark}{Remark}
|
|
|
|
\title{HANZO GPU INFERENCE\\[0.3em]
|
|
{\large GPU-Accelerated AI Inference\\on Hanzo Cloud}}
|
|
\author{
|
|
Zach Kelling \\
|
|
\textit{Hanzo Industries Inc (Techstars '17)} \\
|
|
\texttt{[email protected]}
|
|
}
|
|
\date{March 2026}
|
|
|
|
\begin{document}
|
|
|
|
\hanzocoverpage
|
|
|
|
% ============================================================================
|
|
% ABSTRACT
|
|
% ============================================================================
|
|
\begin{abstract}
|
|
We present the GPU inference architecture of Hanzo Cloud, a production system
|
|
serving over 47 million inference requests per month across heterogeneous GPU
|
|
clusters. The system addresses four challenges in large-scale model serving:
|
|
(i)~batch inference with adaptive batching that maximizes GPU utilization while
|
|
meeting latency SLAs, (ii)~multi-model GPU sharing via spatial and temporal
|
|
multiplexing on a single device, (iii)~GPU-aware scheduling that accounts for
|
|
memory bandwidth, compute capacity, and thermal constraints, and (iv)~latency
|
|
optimization through speculative decoding, KV-cache management, and continuous
|
|
batching. We describe the architecture deployed on Hanzo ACI nodes, where GPU
|
|
operators stake HANZO tokens and serve Zen foundation models (Qwen3+). Production
|
|
measurements on NVIDIA H100 and A100 clusters demonstrate 3.2$\times$ throughput
|
|
improvement over naive serving, p99 latency under 180ms for 7B-parameter models,
|
|
and 94\% average GPU utilization across the fleet. The system integrates with
|
|
Hanzo Gateway for authentication and routing, Hanzo IAM for tenant isolation,
|
|
and the Candle runtime for Rust-native inference kernels.
|
|
\end{abstract}
|
|
|
|
% ============================================================================
|
|
|
|
\input{sections/01-introduction}
|
|
\input{sections/02-system-architecture}
|
|
\input{sections/03-adaptive-continuous-batching}
|
|
\input{sections/04-multi-model-gpu-sharing}
|
|
\input{sections/05-gpu-aware-cluster-scheduling}
|
|
\input{sections/06-latency-optimization}
|
|
\input{sections/07-production-results}
|
|
\input{sections/08-integration-with-hanzo-infrastructure}
|
|
\input{sections/09-related-work}
|
|
\input{sections/99-conclusion}
|
|
|
|
\end{document}
|