papers/hanzo-jin/sections/01-introduction.tex

\section{Introduction}


Human cognition seamlessly integrates information across sensory modalities---we understand speech by combining audio with visual lip movements, interpret scenes by relating objects to their sounds, and communicate using language grounded in physical experience. Current AI systems largely process modalities in isolation, with multimodal models treating cross-modal understanding as a downstream fusion task.

Jin (Japanese: \begin{CJK}{UTF8}{min}神\end{CJK}, ``spirit'' or ``god'') introduces a hypermodal architecture where all modalities share a unified latent space from the ground up. The key insight is that joint embedding predictive architectures (JEPA)~\cite{jepa2022} can be extended beyond single modalities: by training encoders to predict masked representations across modalities, the model learns modality-agnostic features suitable for any downstream task.

\paragraph{Contributions.} This paper makes the following contributions:
\begin{itemize}
    \item A joint embedding space architecture with modality-specific encoders and learnable modality tokens enabling seamless cross-modal fusion.
    \item A diffusion transformer backbone with Mixture of Experts providing efficient routing for modality-specific and cross-modal computation.
    \item z-JEPA, a hierarchical zooming extension of JEPA for multi-scale visual understanding with adaptive computation.
    \item Runtime modality extension enabling new modalities without core model retraining.
    \item Comprehensive benchmarks demonstrating state-of-the-art performance on multimodal understanding tasks.
\end{itemize}