mirror of
https://github.com/hanzoai/papers.git
synced 2026-11-03 02:27:30 +00:00
zeekay
f9db4f5737
Bylines, paper authorship, correspondence addresses and the founder blurb under each post named an individual. They name the organisation that published the work instead: the brand on posts and pages, the research byline on papers, and research@ in place of a personal address. Both are what these repositories already used elsewhere — nothing new was coined. Co-authors are untouched. Where a paper named someone else alongside, they stay named; only the one byline changed. Bibliography entries, running heads, PDF metadata and affiliations move with it, so a citation still resolves to whoever the cited paper names. Sample and fixture data uses the placeholder name these tests already use elsewhere, so no assertion changed meaning.
79 lines
3.2 KiB
TeX
79 lines
3.2 KiB
TeX
\documentclass[11pt,twocolumn]{article}
|
|
\usepackage{../shared/hanzocover}
|
|
\usepackage[utf8]{inputenc}
|
|
\usepackage{amsmath,amssymb,amsthm}
|
|
\usepackage{graphicx}
|
|
\usepackage{booktabs}
|
|
\usepackage{algorithm}
|
|
\usepackage{algorithmic}
|
|
\usepackage{hyperref}
|
|
\usepackage{xcolor}
|
|
\usepackage{listings}
|
|
\input{../shared/lstlang}
|
|
\usepackage{tikz}
|
|
\usetikzlibrary{shapes,arrows,positioning,fit,backgrounds,calc}
|
|
\usepackage{enumitem}
|
|
\usepackage{tabularx}
|
|
\usepackage{multirow}
|
|
\usepackage{subcaption}
|
|
|
|
\definecolor{codegreen}{rgb}{0,0.6,0}
|
|
\definecolor{codegray}{rgb}{0.5,0.5,0.5}
|
|
\definecolor{codepurple}{rgb}{0.58,0,0.82}
|
|
\definecolor{backcolour}{rgb}{0.95,0.95,0.92}
|
|
|
|
\lstdefinestyle{mystyle}{
|
|
backgroundcolor=\color{backcolour},
|
|
commentstyle=\color{codegreen},
|
|
keywordstyle=\color{codepurple},
|
|
numberstyle=\tiny\color{codegray},
|
|
stringstyle=\color{codegreen},
|
|
basicstyle=\ttfamily\footnotesize,
|
|
breakatwhitespace=false,
|
|
breaklines=true,
|
|
captionpos=b,
|
|
keepspaces=true,
|
|
numbers=left,
|
|
numbersep=5pt,
|
|
showspaces=false,
|
|
showstringspaces=false,
|
|
showtabs=false,
|
|
tabsize=2
|
|
}
|
|
\lstset{style=mystyle}
|
|
|
|
\newtheorem{definition}{Definition}
|
|
\newtheorem{theorem}{Theorem}
|
|
\newtheorem{lemma}{Lemma}
|
|
\newtheorem{proposition}{Proposition}
|
|
|
|
\title{Hanzo LLM Gateway: Unified Proxy Architecture\\for 100+ AI Provider Integration}
|
|
\author{Hanzo AI Research \\
|
|
Hanzo AI Research\\
|
|
\texttt{[email protected]}\\
|
|
\textit{Hanzo AI \quad San Francisco, CA}}
|
|
\date{October 2025}
|
|
|
|
\begin{document}
|
|
|
|
\hanzocoverpage
|
|
|
|
\begin{abstract}
|
|
The proliferation of large language model (LLM) providers---each with incompatible APIs, pricing models, rate limits, and capability profiles---creates a fragmentation problem for production AI systems. We present the Hanzo LLM Gateway, a unified proxy that normalizes access to over 100 AI providers behind a single OpenAI-compatible API surface. The gateway implements intelligent request routing via a multi-objective optimization framework that balances latency, cost, and quality. Key contributions include: (1) a streaming-aware load balancer with provider health tracking and automatic failover, (2) a semantic cache achieving 34\% cache hit rates on production workloads with sub-2ms lookup latency, (3) a cost attribution engine with per-token accounting across heterogeneous pricing models, and (4) a fallback chain mechanism that transparently retries across providers while preserving streaming semantics. In production at Hanzo AI serving 2.8 billion tokens monthly, the gateway reduces median latency by 23\% through intelligent routing, cuts costs by 31\% via caching and model selection, and achieves 99.97\% availability through multi-provider redundancy. We release the gateway as open-source software under the Apache 2.0 license.
|
|
\end{abstract}
|
|
|
|
\input{sections/01-introduction}
|
|
\input{sections/02-background-and-related-work}
|
|
\input{sections/03-system-architecture}
|
|
\input{sections/04-load-balancing}
|
|
\input{sections/05-cost-tracking-and-attribution}
|
|
\input{sections/06-streaming-architecture}
|
|
\input{sections/07-observability}
|
|
\input{sections/08-security}
|
|
\input{sections/09-implementation}
|
|
\input{sections/10-evaluation}
|
|
\input{sections/11-discussion}
|
|
\input{sections/99-conclusion}
|
|
|
|
\end{document}
|