papers/zoo-network-architecture/sections/09-gpu-compute-nodes.tex

\section{GPU Compute Nodes}


\subsection{Node Architecture}

Each GPU node runs:
\begin{itemize}
\item \textbf{Base Model Loader}: Downloads frozen model weights from IPFS/HuggingFace.
\item \textbf{Experience Injector}: Fetches experience library, retrieves relevant experiences, constructs prompt.
\item \textbf{Inference Engine}: HuggingFace Transformers, vLLM \cite{kwon2023vllm}, TGI \cite{huggingface2023tgi}.
\item \textbf{Proof Generator}: Optional zkSNARK prover.
\item \textbf{RPC Interface}: Listens for queries from Inference Router.
\end{itemize}

\subsection{Supported Models}

\begin{itemize}
\item \textbf{Qwen Series}: Qwen3-4B/7B/14B/32B/72B \cite{qwen2024qwen3}
\item \textbf{LLaMA Series}: LLaMA 2/3/3.1/3.2/3.3 (all sizes) \cite{touvron2023llama}
\item \textbf{DeepSeek}: V2, V2.5, V3 \cite{bi2024deepseek}
\item \textbf{Mistral/Mixtral}: Including MoE variants \cite{jiang2023mistral}
\item \textbf{Multimodal}: Qwen3-Omni (vision + audio) \cite{chu2024qwen2audio}
\end{itemize}

\subsection{Resource Requirements}

\begin{center}
\begin{tabular}{lrrr}
\toprule
\textbf{Model} & \textbf{Parameters} & \textbf{VRAM (FP16)} & \textbf{VRAM (4-bit)} \\
\midrule
Qwen3-4B & 4B & 8 GB & 3 GB \\
Qwen3-7B & 7B & 14 GB & 5 GB \\
Qwen3-14B & 14B & 28 GB & 9 GB \\
Qwen3-32B & 32B & 64 GB & 18 GB \\
Qwen3-72B & 72B & 144 GB & 38 GB \\
DeepSeek-V3 & 671B & 1342 GB & 350 GB \\
\bottomrule
\end{tabular}
\end{center}

\subsection{Multi-GPU Inference}

For large models ($>$ 70B parameters), Zoo supports:
\begin{itemize}
\item \textbf{Tensor Parallelism}: Split individual layers across GPUs (via Megatron-LM \cite{shoeybi2019megatron}).
\item \textbf{Pipeline Parallelism}: Split model depth across GPUs (via DeepSpeed \cite{rasley2020deepspeed}).
\item \textbf{Sequence Parallelism}: Split sequence length (for very long contexts).
\end{itemize}