papers/hanzo-agent-grpo/sections/08-experiments.tex

\section{Experiments}

\label{sec:experiments}

\subsection{Experimental Setup}

\paragraph{Benchmark.}
We evaluate on \textbf{HB-Bench}, a tool-use benchmark comprising 150 tasks
across five domains: software engineering (40 tasks), DevOps (30), data
processing (30), web research (30), and system administration (20). Each task
requires 3--15 tool invocations and has a binary completion criterion evaluated
by an automated judge.

\paragraph{Agent Implementation.}
The base agent is a Hanzo Dev agent~\cite{hanzo2026aso} backed by a 32B
parameter frozen LLM with access to 24 tools including file I/O, shell
execution, code search, Kubernetes operations, and web search. All experiments
use $G = 4$ trajectories per group and $K = 5$ retrieved experiences per
session.

\paragraph{Baselines.}
We compare against:
\begin{enumerate}[leftmargin=1.5em]
    \item \textbf{Zero-shot:} No experience injection; frozen base model.
    \item \textbf{RAG-Static:} Fixed tool documentation injected as context.
    \item \textbf{Reflexion:} Session-level reflection without persistent memory~\cite{shinn2023reflexion}.
    \item \textbf{ASO (text-only):} TF-GRPO on text outputs only, ignoring
          tool structure~\cite{hanzo2026aso}.
    \item \textbf{Agent-GRPO (ours):} Full tool trajectory extraction with
          semantic experience library.
\end{enumerate}

\subsection{Main Results}

\begin{table}[H]
\centering
\caption{Task completion rate (\%) on HB-Bench after 50 sessions. All results
are averaged over 5 independent runs; standard deviations in parentheses.}
\label{tab:main-results}
\begin{tabularx}{\textwidth}{lXrrr}
\toprule
\textbf{Method} & \textbf{SWE} & \textbf{DevOps} & \textbf{Data} & \textbf{Overall} \\
\midrule
Zero-shot           & 42.1 (1.8) & 38.6 (2.1) & 51.3 (1.5) & 44.0 (1.4) \\
RAG-Static          & 47.3 (1.6) & 43.2 (1.9) & 54.7 (1.7) & 48.4 (1.2) \\
Reflexion           & 52.8 (2.0) & 49.7 (2.3) & 58.2 (1.8) & 53.6 (1.5) \\
ASO (text-only)     & 56.1 (1.9) & 51.4 (2.0) & 61.8 (1.6) & 56.4 (1.4) \\
\textbf{Agent-GRPO} & \textbf{63.7 (1.7)} & \textbf{60.2 (1.8)} & \textbf{68.4 (1.5)} & \textbf{64.1 (1.3)} \\
\bottomrule
\end{tabularx}
\end{table}

Agent-GRPO achieves a \textbf{20.1 percentage point} improvement over zero-shot
and a \textbf{7.7 percentage point} improvement over ASO (text-only). The
improvement is most pronounced in the DevOps domain (+21.6\% over zero-shot),
where domain-specific convention experiences (Section~\ref{sec:adaptations})
provide substantial guidance on deployment sequences and rollback procedures.

\subsection{Tool-Call Error Rate}

\begin{table}[H]
\centering
\caption{Tool-call error rate (\%) across methods. Lower is better.}
\label{tab:error-rate}
\begin{tabularx}{\textwidth}{lXrr}
\toprule
\textbf{Method} & \textbf{Session 1} & \textbf{Session 25} & \textbf{Session 50} \\
\midrule
Zero-shot           & 18.4 & 18.2 & 18.5 \\
RAG-Static          & 15.7 & 15.5 & 15.9 \\
Reflexion           & 17.1 & 14.3 & 13.9 \\
ASO (text-only)     & 16.9 & 13.8 & 12.7 \\
\textbf{Agent-GRPO} & 18.1 & \textbf{10.2} & \textbf{12.6} \\
\bottomrule
\end{tabularx}
\end{table}

Agent-GRPO shows a marked error rate reduction between sessions 1 and 25
(\textbf{44.8\% relative reduction}), reflecting the rapid accumulation of
failure pattern experiences in the early epochs. The slight increase from
session 25 to 50 reflects library compression removing some overfitted
failure-pattern experiences.

\subsection{Ablation Studies}

\begin{table}[H]
\centering
\caption{Ablation on HB-Bench overall task completion rate (\%) after 50 sessions.
Each variant removes one component of Agent-GRPO.}
\label{tab:ablation}
\begin{tabular}{lc}
\toprule
\textbf{Configuration} & \textbf{Overall (\%)} \\
\midrule
Full Agent-GRPO                       & 64.1 \\
\quad w/o Stage 1 summarization       & 59.8 (-4.3) \\
\quad w/o Stage 3 consolidation       & 61.2 (-2.9) \\
\quad w/o domain filtering            & 62.0 (-2.1) \\
\quad w/o hybrid compression          & 63.1 (-1.0) \\
\quad w/o confidence weighting        & 62.5 (-1.6) \\
\quad w/o failure pattern experiences & 60.9 (-3.2) \\
\quad w/o Loop 2 (user corrections)   & 62.7 (-1.4) \\
\bottomrule
\end{tabular}
\end{table}

Stage~1 summarization contributes the largest individual gain (4.3 pp), as
it enables meaningful group comparison across trajectories of variable length.
Failure pattern experiences contribute 3.2 pp, confirming that negative
examples carry significant learning signal.

\subsection{Library Growth and Compression}

The library grows from 0 to approximately 320 entries over 50 sessions, with
two compression events at sessions 30 and 48 (when $|\mathcal{E}| > N_{\max} = 300$).
Post-compression library quality (measured by mean retrieval confidence)
recovers within two sessions, consistent with Proposition~\ref{prop:compression}.

% ============================================================================
% 9. RELATED WORK
% ============================================================================