papers/hanzo-agent-grpo/sections/10-discussion-and-future-work.tex

\section{Discussion and Future Work}

\label{sec:discussion}

\paragraph{Multi-Agent GRPO.}
The most natural extension of Agent-GRPO is to the multi-agent setting. When
$n$ agents operate on similar tasks, their experience libraries contain
complementary knowledge. DSO~\cite{hanzo2026dso} provides the aggregation
protocol, but the quality of aggregated experiences depends critically on the
quality of each agent's library---precisely what Agent-GRPO optimizes. Future
work should quantify the marginal benefit of multi-agent sharing as a function
of the single-agent library quality.

\paragraph{Adversarial Robustness.}
The experience library is a potential attack surface: a user who consistently
provides false corrections (Loop 2) or an adversarial tool that returns
misleading outputs could pollute the library with harmful experiences. Several
defenses are available: (i) rate-limiting the confidence increment in
Equation~\eqref{eq:confidence-update}, (ii) requiring consistency across
multiple independent sessions before accepting high-confidence experiences,
(iii) cryptographic attestation of tool outputs as in DSO. A formal analysis
of adversarial robustness is deferred to future work.

\paragraph{Scaling to Millions of Experiences.}
The current library architecture with flat cosine retrieval scales to
$|\mathcal{E}| \lesssim 10{,}000$ entries before retrieval latency becomes a
concern. For larger libraries, hierarchical memory structures (e.g.,
HNSW~\cite{malkov2018efficient} approximate nearest neighbor search) would
maintain sub-millisecond retrieval. Hierarchical compression---grouping
experiences into topic clusters and applying compression at the cluster
level---offers another avenue for scaling.

\paragraph{Curriculum and Experience Ordering.}
Theorem~\ref{thm:monotone} treats all sessions as equally informative. In
practice, there may be a curriculum effect: early sessions on simple tasks
produce foundational experiences (basic tool preferences) that enable better
performance on complex tasks later. Designing session orderings that maximize
$p_c$ in early epochs is an open problem.

\paragraph{Cross-Modal Extensions.}
The current framework assumes text-based tool outputs. Many real-world tools
produce structured data (JSON, CSV, images, audio). Extending the experience
representation to encode modality-specific patterns---for example, ``when
processing image outputs from OCR tools, always validate bounding box
coordinates before downstream use''---requires multi-modal embedding models
and a richer experience schema.

\paragraph{Relationship to Reinforcement Learning.}
Agent-GRPO can be interpreted as a form of case-based reasoning augmented with
group-relative advantage weighting. Unlike RL approaches that update policy
parameters, Agent-GRPO accumulates a growing case library indexed by task
embedding. The theoretical guarantees (Theorems~\ref{thm:monotone}
and~\ref{thm:sample}) are analogous to sample complexity bounds in online
learning, but differ in that the ``policy'' is the retrieval+injection
mechanism rather than model parameters.

% ============================================================================
% 11. CONCLUSION
% ============================================================================