papers/hanzo-aso/sections/10-related-work.tex

\section{Related Work}

\label{sec:related}

\paragraph{Code Generation Agents.}
SWE-agent~\cite{yang2024sweagent} and AutoCodeRover~\cite{zhang2024autocoderover}
use agentic loops with tool use for software engineering tasks.
Aider~\cite{aider2024} provides a chat-based interface for code editing.
Claude Code and Cursor~\cite{cursor2024} integrate LLMs into development
workflows. These systems rely on in-context learning without systematic
adaptation mechanisms.

\paragraph{Reinforcement Learning for LLMs.}
RLHF~\cite{ouyang2022training} and DPO~\cite{rafailov2024direct} optimize
language model behavior via reward signals but require gradient updates.
GRPO~\cite{shao2024deepseekmath} introduces group-relative advantages for
more stable training. Our TF-GRPO extends GRPO to the training-free regime.

\paragraph{Parameter-Efficient Adaptation.}
LoRA~\cite{hu2022lora} and QLoRA~\cite{dettmers2024qlora} reduce the cost
of fine-tuning via low-rank parameter updates. Prefix
tuning~\cite{li2021prefix} and prompt tuning~\cite{lester2021power} adapt
models by learning soft prompts. These methods still require gradient
computation and risk forgetting.

\paragraph{Product-of-Experts.}
PoE models~\cite{hinton2002training} combine expert distributions
multiplicatively. \cite{du2023reducing} use PoE for reducing sycophancy in
LLMs. \cite{liu2024tuning} apply PoE for controllable generation. Our work
extends PoE to incorporate experiential priors derived from task execution.

\paragraph{Training-Free Adaptation.}
In-context learning~\cite{brown2020language}, chain-of-thought
prompting~\cite{wei2022chain}, and retrieval-augmented
generation~\cite{lewis2020retrieval} adapt models without training. Self-refine
~\cite{madaan2023selfrefine} uses iterative feedback loops. These methods lack
the systematic advantage extraction and compression that ASO provides.

\paragraph{Active Inference.}
Active inference~\cite{friston2017active, parr2022active} frames
decision-making as free energy minimization. \cite{sajid2021active} apply
active inference to planning. Our epistemic reward term is inspired by the
expected free energy formulation in active inference.

\paragraph{Model Compression.}
BitDelta~\cite{liu2024bitdelta} compresses fine-tuning deltas to 1 bit.
GPTQ~\cite{frantar2023gptq} and AWQ~\cite{lin2024awq} quantize model weights
for inference efficiency. Our 1-bit prior compression applies similar ideas to
experiential factors rather than model parameters.

% ============================================================================
% 11. CONCLUSION
% ============================================================================