papers/hanzo-ai-cloud/sections/07-jin-architecture.tex

\section{Jin Architecture}

\label{sec:jin}
% ============================================================================

Jin is Hanzo's unified multimodal AI architecture~\cite{hanzojin}. Where most
AI systems treat vision, language, and audio as separate pipelines with
separate models, Jin unifies them in a single framework with shared
representations.

\subsection{Architecture}

Jin consists of three components:

\begin{enumerate}[leftmargin=1.4em]
    \item \textbf{Modality encoders}: Separate encoders for text (tokenizer +
    embedding), vision (ViT patch encoder), and audio (mel-spectrogram +
    convolutional encoder). Each encoder projects its modality into a shared
    latent space of dimension $d = 4096$.
    \item \textbf{Fusion transformer}: A standard transformer decoder that
    operates on the concatenated encoded representations. Cross-modal
    attention enables the model to attend across modalities (e.g., attending
    to an image region when generating text about it).
    \item \textbf{Modality decoders}: Separate decoders for text (token
    generation), vision (diffusion decoder for image generation), and audio
    (vocoder for speech synthesis).
\end{enumerate}

The key property: a single Jin model handles text-to-text, text-to-image,
image-to-text, audio-to-text, and any other modality combination without
separate model deployments. On Hanzo AI Cloud, a Jin model occupies one GPU
slot but serves all modality combinations.

\subsection{Mixture of Experts}

Jin uses Mixture of Experts (MoE) routing in the fusion transformer. Each
transformer block routes tokens to $k = 2$ of $E = 16$ experts, based on a
learned gating function:

\begin{equation}
\text{output} = \sum_{i \in \text{top-}k(g(\bm{x}))} g_i(\bm{x}) \cdot
\text{Expert}_i(\bm{x})
\end{equation}

MoE enables Jin to scale to 128B total parameters while activating only 16B
per token, keeping inference cost comparable to a dense 16B model. Different
experts specialize in different modality combinations, learned during training
without explicit supervision.

% ============================================================================