papers/hanzo-aso/sections/01-introduction.tex

\section{Introduction}

\label{sec:intro}

Modern large language models (LLMs) achieve remarkable performance across a wide
range of tasks~\cite{brown2020language, openai2023gpt4, anthropic2024claude},
yet adapting them to specific domains or user preferences remains expensive. The
dominant paradigms---fine-tuning, reinforcement learning from human feedback
(RLHF)~\cite{ouyang2022training}, and parameter-efficient methods like
LoRA~\cite{hu2022lora}---all require gradient computation, specialized
infrastructure, and careful hyperparameter tuning. Even the most efficient
approaches demand thousands of GPU-hours and risk catastrophic
forgetting~\cite{kirkpatrick2017overcoming}.

In-context learning (ICL)~\cite{brown2020language} offers a gradient-free
alternative but is limited by context window size, lacks systematic improvement
mechanisms, and cannot accumulate knowledge across sessions. Retrieval-augmented
generation (RAG)~\cite{lewis2020retrieval} extends ICL with external memory but
still treats retrieved documents as passive context rather than active adaptation
signals.

\paragraph{The Adaptation Gap.}
There exists a significant gap between the performance achievable with
fine-tuning and what gradient-free methods deliver. On SWE-bench
Verified~\cite{jimenez2024swebench}, for instance, fine-tuned agents achieve
20--30\% resolution rates while zero-shot prompting yields only 8--12\%. This
gap represents real economic value: organizations that cannot afford fine-tuning
are locked out of state-of-the-art performance.

\paragraph{Our Contribution.}
We introduce \textbf{Active Semantic Optimization (ASO)}, a framework that
closes this gap without any parameter updates. ASO operates in three phases:

\begin{enumerate}[leftmargin=1.5em]
    \item \textbf{TF-GRPO Rollouts:} Generate groups of candidate solutions,
          execute them against test suites, and extract semantic advantages via
          LLM introspection.
    \item \textbf{Prior Compression:} Distill advantages into token-level expert
          factors and compress them to 1-bit representations ($29.5\times$
          storage savings).
    \item \textbf{PoE Decoding:} Apply compressed priors as multiplicative
          factors in the autoregressive decoding distribution, producing adapted
          outputs without modifying model weights.
\end{enumerate}

The theoretical foundation is a Bayesian Product-of-Experts
model~\cite{hinton2002training} where each experiential prior acts as a
likelihood factor. We prove convergence guarantees and bound the approximation
error introduced by 1-bit compression.

\paragraph{Paper Outline.}
Section~\ref{sec:background} reviews necessary background. Section~\ref{sec:tfgrpo}
formalizes TF-GRPO. Section~\ref{sec:poe} develops the PoE decoding framework.
Section~\ref{sec:bitdelta} presents the 1-bit compression scheme.
Section~\ref{sec:convergence} proves convergence results. Section~\ref{sec:cost}
analyzes costs. Section~\ref{sec:agent} describes the Hanzo Dev agent.
Section~\ref{sec:experiments} presents experimental results.
Section~\ref{sec:related} discusses related work. Section~\ref{sec:conclusion}
concludes.

% ============================================================================
% 2. BACKGROUND
% ============================================================================