mirror of
https://github.com/hanzoai/papers.git
synced 2026-11-03 02:27:30 +00:00
zeekay
0d1d121451
Fix shared/ include paths to ../shared/ for 55 subdirectory papers so
that hanzocover.sty and lstlang.tex resolve when building from each
paper's own directory (the build pattern used by latexmk per paper).
Build fixes:
- hanzo-operative/sections/03-screen-understanding-pipeline.tex: move
\COMMENT from after \FOR{...} onto the \STATE line so the algorithmic
env doesn't mis-parse.
- hanzo-network-architecture (abstract + sections 04, 12, 13): escape
unicode mu (U+03BC) as $\mu$ for pdflatex.
- ntt-transform/sections/08-benchmark-results.tex: same mu escape.
All 55 Tier-2/3 papers now build clean with latexmk -pdf.
79 lines
3.2 KiB
TeX
79 lines
3.2 KiB
TeX
\documentclass[11pt,twocolumn]{article}
|
|
\usepackage{../shared/hanzocover}
|
|
\usepackage[utf8]{inputenc}
|
|
\usepackage{amsmath,amssymb,amsthm}
|
|
\usepackage{graphicx}
|
|
\usepackage{booktabs}
|
|
\usepackage{algorithm}
|
|
\usepackage{algorithmic}
|
|
\usepackage{hyperref}
|
|
\usepackage{xcolor}
|
|
\usepackage{listings}
|
|
\input{../shared/lstlang}
|
|
\usepackage{tikz}
|
|
\usetikzlibrary{shapes,arrows,positioning,fit,backgrounds,calc}
|
|
\usepackage{enumitem}
|
|
\usepackage{tabularx}
|
|
\usepackage{multirow}
|
|
\usepackage{subcaption}
|
|
|
|
\definecolor{codegreen}{rgb}{0,0.6,0}
|
|
\definecolor{codegray}{rgb}{0.5,0.5,0.5}
|
|
\definecolor{codepurple}{rgb}{0.58,0,0.82}
|
|
\definecolor{backcolour}{rgb}{0.95,0.95,0.92}
|
|
|
|
\lstdefinestyle{mystyle}{
|
|
backgroundcolor=\color{backcolour},
|
|
commentstyle=\color{codegreen},
|
|
keywordstyle=\color{codepurple},
|
|
numberstyle=\tiny\color{codegray},
|
|
stringstyle=\color{codegreen},
|
|
basicstyle=\ttfamily\footnotesize,
|
|
breakatwhitespace=false,
|
|
breaklines=true,
|
|
captionpos=b,
|
|
keepspaces=true,
|
|
numbers=left,
|
|
numbersep=5pt,
|
|
showspaces=false,
|
|
showstringspaces=false,
|
|
showtabs=false,
|
|
tabsize=2
|
|
}
|
|
\lstset{style=mystyle}
|
|
|
|
\newtheorem{definition}{Definition}
|
|
\newtheorem{theorem}{Theorem}
|
|
\newtheorem{lemma}{Lemma}
|
|
\newtheorem{proposition}{Proposition}
|
|
|
|
\title{Hanzo LLM Gateway: Unified Proxy Architecture\\for 100+ AI Provider Integration}
|
|
\author{Zach Kelling \\
|
|
Hanzo AI Research\\
|
|
\texttt{[email protected]}\\
|
|
\textit{Hanzo AI \quad San Francisco, CA}}
|
|
\date{October 2025}
|
|
|
|
\begin{document}
|
|
|
|
\hanzocoverpage
|
|
|
|
\begin{abstract}
|
|
The proliferation of large language model (LLM) providers---each with incompatible APIs, pricing models, rate limits, and capability profiles---creates a fragmentation problem for production AI systems. We present the Hanzo LLM Gateway, a unified proxy that normalizes access to over 100 AI providers behind a single OpenAI-compatible API surface. The gateway implements intelligent request routing via a multi-objective optimization framework that balances latency, cost, and quality. Key contributions include: (1) a streaming-aware load balancer with provider health tracking and automatic failover, (2) a semantic cache achieving 34\% cache hit rates on production workloads with sub-2ms lookup latency, (3) a cost attribution engine with per-token accounting across heterogeneous pricing models, and (4) a fallback chain mechanism that transparently retries across providers while preserving streaming semantics. In production at Hanzo AI serving 2.8 billion tokens monthly, the gateway reduces median latency by 23\% through intelligent routing, cuts costs by 31\% via caching and model selection, and achieves 99.97\% availability through multi-provider redundancy. We release the gateway as open-source software under the Apache 2.0 license.
|
|
\end{abstract}
|
|
|
|
\input{sections/01-introduction}
|
|
\input{sections/02-background-and-related-work}
|
|
\input{sections/03-system-architecture}
|
|
\input{sections/04-load-balancing}
|
|
\input{sections/05-cost-tracking-and-attribution}
|
|
\input{sections/06-streaming-architecture}
|
|
\input{sections/07-observability}
|
|
\input{sections/08-security}
|
|
\input{sections/09-implementation}
|
|
\input{sections/10-evaluation}
|
|
\input{sections/11-discussion}
|
|
\input{sections/99-conclusion}
|
|
|
|
\end{document}
|