papers/hanzo-ai-cloud/sections/05-model-serving.tex

\section{Model Serving}

\label{sec:serving}
% ============================================================================

Hanzo AI Cloud serves models from a registry anchored to the Lux A-Chain
(Attestation Chain). Every model version, every inference result, and every
provider claim is recorded on-chain with cryptographic receipts.

\subsection{Model Registry}

The model registry stores model metadata on the A-Chain:

\begin{table}[H]
\centering
\small
\begin{tabular}{lp{6cm}}
\toprule
\textbf{Field} & \textbf{Description} \\
\midrule
\texttt{model\_id} & Content-addressed hash of model weights \\
\texttt{format} & ONNX, GGUF, SafeTensors, Candle \\
\texttt{architecture} & Transformer, CNN, Diffusion, MoE \\
\texttt{parameters} & Total parameter count \\
\texttt{quantization} & FP16, INT8, INT4, or none \\
\texttt{benchmarks} & Standardized evaluation scores \\
\texttt{publisher} & I-Chain DID of the publishing entity \\
\texttt{license} & Machine-readable license identifier \\
\bottomrule
\end{tabular}
\caption{Model registry fields. The \texttt{model\_id} is a SHA3-256 hash of
the model weights, ensuring immutable identification.}
\label{tab:registry}
\end{table}

Model weights are stored off-chain (on Hanzo Storage, S3-compatible) with the
hash anchored on-chain. This ensures that a model ID always refers to the same
weights---the provider cannot silently swap a model.

\subsection{Verifiable Inference Receipts}

Every inference call produces a receipt:

\begin{equation}
\text{receipt} = \text{sign}_{k_{\text{provider}}}\bigl(
\text{model\_id} \| \text{input\_hash} \| \text{output\_hash} \|
\text{hw\_id} \| \text{timestamp}
\bigr)
\end{equation}

The receipt is submitted to the A-Chain. A client can verify: (i) which model
produced the output, (ii) on what hardware, (iii) at what time, and (iv) that
the provider's stake is intact (not slashed for prior misbehavior). Receipts
enable auditability for regulated industries where ``an AI said so'' is not
sufficient---the specific model version and provider must be traceable.

\subsection{GPU Scheduling}

The Hanzo scheduler assigns inference requests to providers based on:

\begin{itemize}[leftmargin=1.1em]
    \item \textbf{Model availability}: Does the provider have the requested
    model loaded in GPU memory?
    \item \textbf{Latency estimate}: Predicted latency based on provider
    location, current load, and historical performance.
    \item \textbf{Stake weight}: Providers with higher HANZO stake receive
    priority. Stake is a credible commitment to quality.
    \item \textbf{Cost}: The provider's advertised price per token/image/second.
\end{itemize}

The scheduler uses a modified weighted round-robin that favors loaded models
(avoiding cold-load latency) while maintaining fairness across providers.

% ============================================================================