papers/zen/zen-agent.tex

\documentclass[11pt,a4paper]{article}

% ─── Packages ────────────────────────────────────────────────────────────────
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[margin=1in]{geometry}
\usepackage{amsmath,amssymb,amsthm}
\usepackage{graphicx}
\usepackage[colorlinks=true,linkcolor=blue,citecolor=blue,urlcolor=blue]{hyperref}
\usepackage[ruled,vlined,linesnumbered]{algorithm2e}
\usepackage{algpseudocode}
\usepackage{booktabs}
\usepackage{listings}
\usepackage{xcolor}
\usepackage[numbers,sort&compress]{natbib}
\usepackage{tikz}
\usepackage{enumitem}
\usepackage{subcaption}
\usepackage{multirow}
\usepackage{tabularx}
\usepackage{fancyhdr}
\usepackage{float}

\usetikzlibrary{arrows.meta,positioning,shapes.geometric,calc,fit,backgrounds}

% ─── Colors ──────────────────────────────────────────────────────────────────
\definecolor{hanzo-red}{HTML}{FD4444}
\definecolor{codebg}{HTML}{F5F5F5}
\definecolor{codegreen}{HTML}{2E7D32}
\definecolor{codepurple}{HTML}{7B1FA2}
\definecolor{codegray}{HTML}{616161}

% ─── Listings ────────────────────────────────────────────────────────────────
\lstset{
  backgroundcolor=\color{codebg},
  basicstyle=\ttfamily\small,
  breaklines=true,
  commentstyle=\color{codegreen},
  keywordstyle=\color{codepurple}\bfseries,
  stringstyle=\color{hanzo-red},
  numberstyle=\tiny\color{codegray},
  numbers=left,
  frame=single,
  rulecolor=\color{codegray},
  tabsize=2,
  showstringspaces=false,
}

% ─── Theorem Environments ────────────────────────────────────────────────────
\newtheorem{definition}{Definition}
\newtheorem{theorem}{Theorem}
\newtheorem{proposition}{Proposition}
\newtheorem{lemma}{Lemma}

% ─── Header ──────────────────────────────────────────────────────────────────
\pagestyle{fancy}
\fancyhf{}
\fancyhead[L]{\small Zen-Agent: Multi-Agent Orchestration}
\fancyhead[R]{\small Hanzo AI Inc}
\fancyfoot[C]{\thepage}

% ─── Title ───────────────────────────────────────────────────────────────────
\title{%
  \textbf{Zen-Agent: Multi-Agent Orchestration\\with Hierarchical Planning}\\[0.5em]
  \large Technical Report
}

\author{
  Hanzo AI Inc\thanks{Correspondence: [email protected]}
  \and
  Zoo Labs Foundation
}

\date{February 2026}

\begin{document}

\maketitle

% ═════════════════════════════════════════════════════════════════════════════
% ABSTRACT
% ═════════════════════════════════════════════════════════════════════════════
\begin{abstract}
We present \textbf{Zen-Agent}, a multi-agent orchestration framework that achieves
state-of-the-art performance on complex, long-horizon reasoning and tool-use tasks
through hierarchical task decomposition, structured memory systems, and principled
inter-agent communication protocols. Zen-Agent introduces three core innovations:
(1)~a \emph{Hierarchical Task Graph} (HTG) planner that decomposes ambiguous natural
language goals into dependency-aware subgoal trees with provable completeness
guarantees; (2)~a \emph{Reflective Action Cycle} (RAC) that extends the
Reason-Act-Observe paradigm with self-critique, backtracking, and adaptive re-planning;
and (3)~a \emph{Shared Episodic Memory} (SEM) architecture that enables efficient
knowledge transfer across heterogeneous agent populations. On SWE-bench Verified,
Zen-Agent resolves 54.2\% of issues (a 7.3 percentage-point improvement over the
previous best open system), achieves 42.8\% task success on WebArena, and scores
61.4\% on the GAIA Level-3 benchmark. We release the full framework, trained
orchestrator weights, and evaluation harness under a permissive open-source license
to accelerate community research in agentic AI systems.
\end{abstract}

\vspace{0.5em}
\noindent\textbf{Keywords:} multi-agent systems, hierarchical planning, tool use,
large language models, ReAct reasoning, software engineering agents

% ═════════════════════════════════════════════════════════════════════════════
% 1. INTRODUCTION
% ═════════════════════════════════════════════════════════════════════════════
\section{Introduction}
\label{sec:introduction}

The deployment of large language models (LLMs) as autonomous agents capable of
interacting with software environments, browsing the web, and writing production
code has progressed rapidly over the past two years
\citep{yao2023react,shinn2023reflexion,wang2024opendevin}. Despite this progress,
single-agent systems encounter fundamental limitations when confronted with tasks
that require (a)~coordinated exploration of large state spaces, (b)~simultaneous
reasoning across heterogeneous tool modalities, or (c)~persistent memory over
extended interaction horizons exceeding tens of thousands of tokens.

Multi-agent approaches address these limitations by distributing cognitive load
across specialized agents, each optimized for a particular reasoning modality or
tool domain \citep{hong2024metagpt,wu2024autogen}. However, existing multi-agent
frameworks suffer from three systematic weaknesses:

\begin{enumerate}[leftmargin=2em]
  \item \textbf{Flat task decomposition.} Most systems decompose goals into
    independent subtasks without modeling dependencies, leading to redundant
    computation and ordering violations that require expensive re-planning.
  \item \textbf{Stateless action cycles.} The standard ReAct loop
    \citep{yao2023react} lacks mechanisms for self-critique and backtracking,
    causing agents to persist on failing strategies rather than exploring
    alternatives.
  \item \textbf{Isolated memory.} Per-agent memory stores prevent knowledge
    sharing, forcing each agent to re-discover environmental facts that
    sibling agents have already established.
\end{enumerate}

Zen-Agent addresses each of these weaknesses through principled architectural
innovations. Our \emph{Hierarchical Task Graph} (HTG) planner represents goals as
directed acyclic graphs with typed dependency edges, enabling parallel execution of
independent subtasks while enforcing ordering constraints. The \emph{Reflective
Action Cycle} (RAC) augments the standard reason-act-observe loop with a critique
phase that triggers backtracking when progress stalls, reducing wasted computation
by up to 38\%. The \emph{Shared Episodic Memory} (SEM) system provides a
transactional key-value store accessible to all agents, with conflict resolution
via vector-clock timestamps.

We evaluate Zen-Agent across three challenging benchmarks spanning software
engineering, web navigation, and general-purpose reasoning. Our system achieves
new state-of-the-art results on all three, demonstrating that principled
multi-agent orchestration with hierarchical planning yields consistent gains
over both single-agent and flat multi-agent baselines.

The contributions of this paper are:
\begin{itemize}[leftmargin=2em]
  \item A formal specification of the Hierarchical Task Graph planner with
    completeness and soundness guarantees (Section~\ref{sec:htg}).
  \item The Reflective Action Cycle, a self-correcting extension of ReAct with
    provably bounded retry budgets (Section~\ref{sec:rac}).
  \item Shared Episodic Memory with transactional semantics for multi-agent
    knowledge sharing (Section~\ref{sec:sem}).
  \item State-of-the-art results on SWE-bench Verified (54.2\%), WebArena
    (42.8\%), and GAIA Level-3 (61.4\%) (Section~\ref{sec:evaluation}).
  \item Open-source release of all code, weights, and evaluation harnesses.
\end{itemize}

% ═════════════════════════════════════════════════════════════════════════════
% 2. BACKGROUND AND RELATED WORK
% ═════════════════════════════════════════════════════════════════════════════
\section{Background and Related Work}
\label{sec:background}

\subsection{LLM-Based Agents}

The use of LLMs as autonomous agents was catalyzed by the ReAct framework
\citep{yao2023react}, which interleaves chain-of-thought reasoning with
environment actions. Subsequent work introduced reflection
\citep{shinn2023reflexion}, allowing agents to learn from past failures within
an episode. Toolformer \citep{schick2023toolformer} demonstrated that LLMs can
learn to invoke external tools during generation. These approaches established
the single-agent paradigm that Zen-Agent extends to the multi-agent setting.

\subsection{Multi-Agent Frameworks}

MetaGPT \citep{hong2024metagpt} assigns software engineering roles (product
manager, architect, engineer) to separate LLM agents and coordinates them
through a shared message board. AutoGen \citep{wu2024autogen} provides a
conversation-based protocol for agent interaction. ChatDev
\citep{qian2024chatdev} simulates a software company with role-playing agents.
While effective, these systems rely on flat task decomposition and
role-based (rather than capability-based) agent specialization.

CAMEL \citep{li2023camel} explores communicative agents with inception prompting
for autonomous cooperation. CrewAI provides sequential and hierarchical process
models but lacks formal planning guarantees. Zen-Agent differs from all of these
by introducing a formally specified task graph with dependency typing, a
self-correcting action cycle, and transactional shared memory.

\subsection{Hierarchical Planning in AI}

Hierarchical Task Networks (HTNs) \citep{erol1994htn} decompose tasks into
subtasks via methods, with ordering constraints. STRIPS-style planners
\citep{fikes1971strips} operate on state-space search. Recent work on
LLM-based planning includes SayCan \citep{ahn2022saycan}, which grounds language
in robotic affordances, and Inner Monologue \citep{huang2023inner}, which uses
environment feedback for re-planning. Zen-Agent's HTG planner draws on classical
HTN theory while leveraging LLM-generated decompositions for open-ended goals.

\subsection{Memory Systems for Agents}

MemGPT \citep{packer2023memgpt} introduced a virtual memory hierarchy for LLM
agents, enabling management of context beyond the model's window. Generative
Agents \citep{park2023generative} use a retrieval-augmented memory stream for
believable agent behavior. Voyager \citep{wang2023voyager} maintains a skill
library as a form of procedural memory. Zen-Agent's SEM system unifies episodic,
semantic, and procedural memory in a shared transactional store.

\subsection{Benchmarks}

We evaluate on three benchmarks:
\begin{itemize}[leftmargin=2em]
  \item \textbf{SWE-bench Verified} \citep{jimenez2024swebench}: 500 verified
    real-world GitHub issues requiring code patches.
  \item \textbf{WebArena} \citep{zhou2024webarena}: 812 web navigation tasks
    across realistic self-hosted web applications.
  \item \textbf{GAIA} \citep{mialon2024gaia}: 466 questions requiring
    multi-step reasoning with tool use, at three difficulty levels.
\end{itemize}

% ═════════════════════════════════════════════════════════════════════════════
% 3. ARCHITECTURE
% ═════════════════════════════════════════════════════════════════════════════
\section{System Architecture}
\label{sec:architecture}

Zen-Agent comprises four interconnected subsystems: the \emph{Orchestrator}, the
\emph{Hierarchical Task Graph Planner}, the \emph{Agent Pool}, and the
\emph{Shared Episodic Memory}. Figure~\ref{fig:architecture} provides an overview.

\begin{figure}[t]
\centering
\begin{tikzpicture}[
    box/.style={rectangle, draw=black, rounded corners=3pt, minimum width=3cm,
                minimum height=1cm, align=center, font=\small},
    arrow/.style={-{Stealth[length=3mm]}, thick},
    darrow/.style={<->{Stealth[length=3mm]}, thick},
    node distance=1.5cm
]
  % Orchestrator
  \node[box, fill=hanzo-red!20] (orch) {Orchestrator\\(Zen-MoDE 72B)};

  % HTG Planner
  \node[box, fill=blue!10, below left=1.5cm and 1cm of orch] (htg)
    {HTG Planner};

  % Agent Pool
  \node[box, fill=green!10, below=1.5cm of orch] (pool)
    {Agent Pool\\(N specialized agents)};

  % Shared Memory
  \node[box, fill=orange!10, below right=1.5cm and 1cm of orch] (sem)
    {Shared Episodic\\Memory (SEM)};

  % Environment
  \node[box, fill=gray!10, below=1.5cm of pool] (env)
    {Environment\\(Tools, APIs, Files)};

  % Arrows
  \draw[arrow] (orch) -- (htg);
  \draw[arrow] (htg) -- (pool);
  \draw[darrow] (pool) -- (sem);
  \draw[darrow] (pool) -- (env);
  \draw[arrow] (orch) -- (pool);
  \draw[darrow] (orch) -- (sem);

\end{tikzpicture}
\caption{High-level architecture of the Zen-Agent system. The Orchestrator
receives user goals and delegates to the HTG Planner for decomposition. The
Agent Pool executes subgoals using the Reflective Action Cycle, reading and
writing shared state through the Shared Episodic Memory.}
\label{fig:architecture}
\end{figure}

\subsection{Orchestrator}

The Orchestrator is the top-level controller instantiated with a Zen-MoDE 72B
model fine-tuned for planning and delegation. It receives a natural-language
goal $g$ from the user and performs three functions:

\begin{enumerate}[leftmargin=2em]
  \item \textbf{Goal analysis}: Classifies the goal by domain (code, web,
    reasoning, creative) and estimates complexity on a 1--5 scale.
  \item \textbf{Planner invocation}: Passes the goal and complexity estimate to
    the HTG Planner for decomposition.
  \item \textbf{Execution monitoring}: Tracks the status of all subgoals,
    detects stalls, and triggers re-planning when necessary.
\end{enumerate}

The Orchestrator maintains a \emph{global context window} that summarizes the
current state of all active subgoals, agent assignments, and memory snapshots.
This summary is compressed via an extractive summarization pass every $K=50$
steps to prevent context overflow.

\subsection{Hierarchical Task Graph Planner}
\label{sec:htg}

\begin{definition}[Hierarchical Task Graph]
A Hierarchical Task Graph is a tuple $\mathcal{G} = (V, E, \tau, \delta)$ where:
\begin{itemize}[leftmargin=2em]
  \item $V$ is a set of \emph{task nodes}, each representing a subgoal.
  \item $E \subseteq V \times V$ is a set of directed \emph{dependency edges}.
  \item $\tau: V \to \{\texttt{atomic}, \texttt{composite}\}$ assigns a type to
    each node.
  \item $\delta: V \to 2^V$ maps composite nodes to their child decompositions.
\end{itemize}
\end{definition}

The HTG Planner operates in three phases:

\paragraph{Phase 1: Initial Decomposition.}
Given goal $g$, the planner generates a root node $v_0$ and recursively
decomposes it into child nodes. Each decomposition step is performed by a
single LLM call with the following prompt structure:

\begin{lstlisting}[language={},title={HTG Decomposition Prompt}]
Goal: {goal_description}
Context: {environment_state}
Constraints: {dependency_constraints}

Decompose this goal into ordered subtasks.
For each subtask, specify:
- description: what to accomplish
- dependencies: list of prerequisite subtask IDs
- agent_type: required specialization
- estimated_steps: expected action count
\end{lstlisting}

The output is parsed into a DAG, validated for cycle-freedom, and stored as the
initial HTG.

\paragraph{Phase 2: Dependency Resolution.}
After initial decomposition, the planner runs a topological sort on the DAG to
determine a valid execution order. Tasks with no unresolved dependencies are
marked as \emph{ready} and can be dispatched in parallel.

\begin{proposition}[Completeness]
If the initial decomposition produces a valid DAG covering all aspects of goal
$g$, and each atomic task is achievable by at least one agent in the pool, then
the HTG Planner produces a complete execution plan.
\end{proposition}

\paragraph{Phase 3: Dynamic Re-planning.}
When an agent reports failure on an atomic task after exhausting its retry
budget, the planner is re-invoked with the failure context. It may:
(a)~generate an alternative decomposition for the failed subtree,
(b)~merge the failed task with a sibling to create a coarser-grained task, or
(c)~escalate to the Orchestrator for goal reformulation.

\subsection{Agent Pool}
\label{sec:agent-pool}

The Agent Pool contains $N$ specialized agents, each instantiated with a
domain-specific system prompt, tool set, and reasoning strategy. We define five
canonical agent types:

\begin{table}[H]
\centering
\caption{Agent specializations in the Zen-Agent pool.}
\label{tab:agents}
\begin{tabular}{@{}llll@{}}
\toprule
\textbf{Agent Type} & \textbf{Model} & \textbf{Tools} & \textbf{Strategy} \\
\midrule
\textsc{Coder} & Zen-MoDE 72B & shell, editor, LSP, git & RAC + tests \\
\textsc{Browser} & Zen-MoDE 32B & click, type, scroll, screenshot & RAC + visual \\
\textsc{Researcher} & Zen-MoDE 72B & search, read, cite & RAC + retrieval \\
\textsc{Analyst} & Zen-MoDE 14B & calculator, Python, SQL & RAC + verify \\
\textsc{Critic} & Zen-MoDE 72B & read-only access & critique only \\
\bottomrule
\end{tabular}
\end{table}

Agent selection for a given task node is determined by a learned routing
function $\rho: \mathcal{D} \times \mathcal{T} \to [0,1]^N$, where $\mathcal{D}$
is the task description embedding space and $\mathcal{T}$ is the tool
requirement space. The agent with the highest score is assigned to the task.

\subsection{Reflective Action Cycle}
\label{sec:rac}

The Reflective Action Cycle (RAC) extends the standard ReAct
\citep{yao2023react} loop with two additional phases: \emph{Critique} and
\emph{Backtrack}. The full cycle is:

\begin{enumerate}[leftmargin=2em]
  \item \textbf{Reason}: Generate a chain-of-thought analysis of the current
    state and formulate a plan for the next action.
  \item \textbf{Act}: Execute the chosen action (tool call, code edit, web
    interaction) in the environment.
  \item \textbf{Observe}: Capture the environment response (stdout, screenshot,
    API response, error message).
  \item \textbf{Critique}: Evaluate whether the observation constitutes progress
    toward the subgoal. Assign a progress score $p \in [0,1]$.
  \item \textbf{Backtrack} (conditional): If $p < \theta$ for $k$ consecutive
    steps, revert to a prior checkpoint and try an alternative strategy.
\end{enumerate}

\begin{algorithm}[t]
\SetAlgoLined
\KwIn{Task $t$, Agent $a$, Memory $\mathcal{M}$, max\_steps $T$, threshold $\theta$, patience $k$}
\KwOut{Result $r$, status $s$}
$\text{state} \gets \text{env.reset}(t)$\;
$\text{checkpoints} \gets [\text{state}]$\;
$\text{stall\_count} \gets 0$\;
\For{$i \gets 1$ \KwTo $T$}{
  $\text{context} \gets \text{state} \oplus \mathcal{M}.\text{retrieve}(t)$\;
  $\text{thought} \gets a.\text{reason}(\text{context})$\;
  $\text{action} \gets a.\text{act}(\text{thought})$\;
  $\text{observation} \gets \text{env.step}(\text{action})$\;
  $p \gets a.\text{critique}(\text{state}, \text{action}, \text{observation})$\;
  $\mathcal{M}.\text{write}(t, \text{thought}, \text{action}, \text{observation}, p)$\;
  \eIf{$p \geq \theta$}{
    $\text{stall\_count} \gets 0$\;
    $\text{checkpoints.append}(\text{state})$\;
  }{
    $\text{stall\_count} \gets \text{stall\_count} + 1$\;
    \If{$\text{stall\_count} \geq k$}{
      $\text{state} \gets \text{checkpoints.pop}()$\;
      $\text{stall\_count} \gets 0$\;
    }
  }
  $\text{state} \gets \text{observation}$\;
  \If{$a.\text{is\_complete}(\text{state}, t)$}{
    \Return{$(\text{state}, \textsc{success})$}\;
  }
}
\Return{$(\text{state}, \textsc{timeout})$}\;
\caption{Reflective Action Cycle (RAC)}
\label{alg:rac}
\end{algorithm}

\begin{theorem}[Bounded Retry]
Under the RAC protocol with patience $k$, maximum steps $T$, and checkpoint
stack depth $D$, the total number of backtrack events is bounded by
$\lfloor T / k \rfloor$ and the total number of revisited states is bounded
by $\min(D, \lfloor T / k \rfloor)$.
\end{theorem}

\begin{proof}
Each backtrack event requires exactly $k$ consecutive low-progress steps.
Since each step is counted once, the maximum number of backtrack events is
$\lfloor T/k \rfloor$. Each backtrack pops at most one checkpoint, and the
checkpoint stack has depth at most $D$, giving the second bound.
\end{proof}

\subsection{Shared Episodic Memory}
\label{sec:sem}

The Shared Episodic Memory (SEM) is a transactional key-value store that serves
as the collective knowledge base for all agents. It stores three types of records:

\begin{definition}[Memory Record]
A memory record is a tuple $(k, v, t, a, c)$ where $k$ is the key (a natural
language description), $v$ is the value (observation, fact, or procedure),
$t$ is the timestamp (vector clock), $a$ is the authoring agent ID, and $c$ is
the confidence score.
\end{definition}

\paragraph{Write Operations.}
When an agent discovers a new fact or completes a subtask, it writes a record to
SEM. If a record with a matching key already exists, the conflict resolution
policy is:
\begin{itemize}[leftmargin=2em]
  \item If the new record has a higher confidence, it overwrites the old one.
  \item If confidence is equal, the record with the later vector-clock timestamp
    wins.
  \item If both are equal, both records are retained and flagged for Orchestrator
    resolution.
\end{itemize}

\paragraph{Read Operations.}
Agents query SEM using natural-language retrieval. Queries are embedded via a
shared encoder (Zen-MoDE-Embed, 384-dimensional), and the top-$m$ records by
cosine similarity are returned. We use HNSW indexing \citep{malkov2020hnsw}
for sub-millisecond retrieval at scale.

\paragraph{Memory Consolidation.}
Every $C=100$ steps, a background process runs memory consolidation:
\begin{enumerate}[leftmargin=2em]
  \item \textbf{Deduplication}: Records with cosine similarity $> 0.95$ are
    merged, retaining the higher-confidence version.
  \item \textbf{Summarization}: Clusters of related records are summarized into
    a single high-level record by the Orchestrator.
  \item \textbf{Pruning}: Records with confidence below 0.3 and no recent reads
    are archived.
\end{enumerate}

% ═════════════════════════════════════════════════════════════════════════════
% 4. INTER-AGENT COMMUNICATION
% ═════════════════════════════════════════════════════════════════════════════
\section{Inter-Agent Communication Protocol}
\label{sec:communication}

Zen-Agent uses a structured message-passing protocol inspired by the Actor
model \citep{hewitt1973actor}. Each message has the form:

\begin{lstlisting}[language={},title={Message Schema}]
{
  "from": "agent_id",
  "to": "agent_id | broadcast",
  "type": "request | response | notify | escalate",
  "task_ref": "htg_node_id",
  "payload": { ... },
  "timestamp": "vector_clock"
}
\end{lstlisting}

\subsection{Message Types}

\begin{description}[leftmargin=2em]
  \item[\texttt{request}:] Agent $A$ asks agent $B$ to perform a subtask or
    provide information. Includes a deadline (step count) and priority level.
  \item[\texttt{response}:] Agent $B$ replies with results, partial results, or
    a failure report.
  \item[\texttt{notify}:] Broadcast message informing all agents of a state
    change (e.g., ``file X has been modified'').
  \item[\texttt{escalate}:] Agent reports an unresolvable issue to the
    Orchestrator, triggering re-planning.
\end{description}

\subsection{Communication Topology}

We evaluate three communication topologies:

\begin{enumerate}[leftmargin=2em]
  \item \textbf{Star}: All messages route through the Orchestrator. Simple but
    creates a bottleneck.
  \item \textbf{Mesh}: Agents communicate directly. Reduces latency but
    increases message volume.
  \item \textbf{Hierarchical}: Agents within the same HTG subtree communicate
    directly; cross-subtree messages route through the Orchestrator.
\end{enumerate}

Our experiments (Section~\ref{sec:ablation}) show that the hierarchical
topology achieves the best trade-off between latency and message efficiency.

\subsection{Deadlock Prevention}

Circular dependencies in communication can cause deadlocks. We prevent this
through two mechanisms:
\begin{itemize}[leftmargin=2em]
  \item \textbf{Timeout}: Every request has a deadline. If no response arrives
    by the deadline, the request is cancelled and escalated.
  \item \textbf{Dependency tracking}: The Orchestrator maintains a wait-for
    graph and breaks cycles by cancelling the lowest-priority request.
\end{itemize}

% ═════════════════════════════════════════════════════════════════════════════
% 5. TOOL-USE PLANNING
% ═════════════════════════════════════════════════════════════════════════════
\section{Tool-Use Planning}
\label{sec:tools}

Zen-Agent provides agents with access to over 260 tools organized into
categories derived from the Model Context Protocol (MCP) \citep{mcp2024}. Each
tool is specified by a typed schema:

\begin{definition}[Tool Schema]
A tool schema is a tuple $(\text{name}, \text{desc}, \text{params}, \text{returns},
\text{side\_effects})$ where \text{params} and \text{returns} are JSON Schema
types, and \text{side\_effects} is a set of environment state components that the
tool may modify.
\end{definition}

\subsection{Tool Selection}

Given an action intent from the Reason phase of RAC, the agent selects a tool
via a two-stage process:

\begin{enumerate}[leftmargin=2em]
  \item \textbf{Candidate retrieval}: The intent is embedded and matched against
    tool description embeddings to retrieve the top-10 candidates.
  \item \textbf{Schema-guided selection}: The agent evaluates each candidate's
    parameter schema against the available context to determine which tools
    can be invoked with complete arguments.
\end{enumerate}

\subsection{Compound Tool Calls}

For efficiency, Zen-Agent supports \emph{compound tool calls} where multiple
independent tools are invoked in parallel within a single Act step. The planner
determines independence by checking that the side-effect sets of the tools are
disjoint:

\begin{equation}
\text{parallel}(t_1, t_2) \iff \text{side\_effects}(t_1) \cap
\text{side\_effects}(t_2) = \emptyset
\end{equation}

This optimization reduces the average number of RAC steps by 23\% on
SWE-bench tasks.

\subsection{Tool Execution Sandboxing}

All tool executions occur within isolated sandboxes (Docker containers for
code execution, headless browsers for web interaction). Each sandbox maintains
its own filesystem and network namespace. The Orchestrator enforces resource
limits:

\begin{table}[H]
\centering
\caption{Sandbox resource limits per agent.}
\label{tab:sandbox}
\begin{tabular}{@{}lcc@{}}
\toprule
\textbf{Resource} & \textbf{Limit} & \textbf{Timeout} \\
\midrule
CPU & 4 cores & -- \\
Memory & 8 GB & -- \\
Disk & 20 GB & -- \\
Network & 10 Mbps & 30s per request \\
Execution & -- & 120s per tool call \\
\bottomrule
\end{tabular}
\end{table}

% ═════════════════════════════════════════════════════════════════════════════
% 6. TRAINING
% ═════════════════════════════════════════════════════════════════════════════
\section{Training}
\label{sec:training}

Zen-Agent's learned components include the Orchestrator's planning model, the
agent routing function, and the critique model used in RAC. We describe the
training procedure for each.

\subsection{Data Collection}

We collect training data from three sources:

\begin{enumerate}[leftmargin=2em]
  \item \textbf{Human demonstrations}: 12,000 expert trajectories collected
    from professional software engineers solving GitHub issues, web tasks, and
    research questions. Each trajectory includes the full action-observation
    sequence, tool calls, and final outcomes.
  \item \textbf{Self-play}: 85,000 trajectories generated by running an earlier
    version of Zen-Agent on held-out tasks. Successful trajectories are
    retained; failed trajectories are annotated with failure reasons by
    human reviewers.
  \item \textbf{Synthetic tasks}: 200,000 procedurally generated tasks covering
    file manipulation, API interaction, web navigation, and multi-step
    arithmetic. These provide broad coverage of tool-use patterns.
\end{enumerate}

\subsection{Orchestrator Training}

The Orchestrator model is trained in three stages:

\paragraph{Stage 1: Supervised Fine-tuning (SFT).}
The base Zen-MoDE 72B model is fine-tuned on the demonstration dataset using
standard next-token prediction. The training data is formatted as interleaved
thought-action-observation sequences with special tokens delineating each phase.

Training hyperparameters:
\begin{itemize}[leftmargin=2em]
  \item Learning rate: $2 \times 10^{-5}$ with cosine decay
  \item Batch size: 128 (accumulated over 8 GPUs)
  \item Sequence length: 32,768 tokens
  \item Epochs: 3
  \item Optimizer: AdamW ($\beta_1=0.9$, $\beta_2=0.95$, $\epsilon=10^{-8}$)
  \item Weight decay: 0.1
\end{itemize}

\paragraph{Stage 2: Reinforcement Learning from Execution Feedback (RLEF).}
We train a reward model on (trajectory, outcome) pairs and use Proximal Policy
Optimization (PPO) \citep{schulman2017ppo} to optimize the Orchestrator's
planning decisions. The reward function is:

\begin{equation}
R(g, \tau) = \alpha \cdot \mathbb{1}[\text{success}(g, \tau)]
           + \beta \cdot \frac{1}{|\tau|}
           - \gamma \cdot \text{cost}(\tau)
\label{eq:reward}
\end{equation}

where $\tau$ is the trajectory, $\alpha=1.0$ rewards task success,
$\beta=0.1$ rewards efficiency (shorter trajectories), and $\gamma=0.01$
penalizes computational cost (total tokens generated).

\paragraph{Stage 3: Iterative Self-Improvement.}
After RLEF, the improved Orchestrator generates new trajectories on the
self-play task set. High-quality trajectories (success with $< 50$ steps) are
added to the training set, and Stages 1--2 are repeated. We perform 3 rounds of
this loop, observing consistent improvements.

\subsection{Routing Function Training}

The agent routing function $\rho$ is trained as a multi-label classifier on
(task\_description, optimal\_agent\_type) pairs extracted from the demonstration
data. We use a two-layer MLP on top of frozen Zen-MoDE-Embed representations:

\begin{equation}
\rho(d) = \text{softmax}(W_2 \cdot \text{ReLU}(W_1 \cdot \text{Embed}(d) + b_1) + b_2)
\end{equation}

The classifier achieves 94.7\% top-1 accuracy on a held-out validation set.

\subsection{Critique Model Training}

The critique model used in the RAC's Critique phase is trained on
(state, action, observation, progress\_label) tuples. Progress labels are
derived from the demonstration data by comparing each step's state against the
final successful state using a learned state-similarity metric. We fine-tune a
Zen-MoDE 14B model for this task, achieving 0.89 Spearman correlation with
human progress judgments.

\subsection{Training Infrastructure}

All training is conducted on a cluster of 64 NVIDIA H100 GPUs using DeepSpeed
ZeRO Stage 3 \citep{rajbhandari2020zero} with activation checkpointing. The
full training pipeline (SFT + RLEF + 3 self-improvement rounds) requires
approximately 2,400 GPU-hours.

% ═════════════════════════════════════════════════════════════════════════════
% 7. EVALUATION
% ═════════════════════════════════════════════════════════════════════════════
\section{Evaluation}
\label{sec:evaluation}

We evaluate Zen-Agent on three benchmarks: SWE-bench Verified, WebArena, and
GAIA. All experiments use greedy decoding (temperature 0) for reproducibility.

\subsection{SWE-bench Verified}

SWE-bench Verified \citep{jimenez2024swebench} consists of 500 real-world GitHub
issues from 12 popular Python repositories. Each issue requires understanding
the codebase, localizing the bug, and generating a correct patch.

\begin{table}[t]
\centering
\caption{Results on SWE-bench Verified (500 instances). \% Resolved indicates
the fraction of issues where the generated patch passes all tests.}
\label{tab:swebench}
\begin{tabular}{@{}lcc@{}}
\toprule
\textbf{System} & \textbf{\% Resolved} & \textbf{Avg. Steps} \\
\midrule
SWE-Agent \citep{yang2024sweagent} & 23.0 & 42.1 \\
AutoCodeRover \citep{zhang2024autocoderover} & 30.7 & 38.4 \\
Agentless \citep{xia2024agentless} & 33.2 & 12.6 \\
OpenDevin \citep{wang2024opendevin} & 38.5 & 51.3 \\
Aider + GPT-4o & 41.4 & 28.7 \\
Devin (commercial) & 46.9 & -- \\
\midrule
Zen-Agent (single-agent) & 47.3 & 35.2 \\
Zen-Agent (multi-agent, flat) & 50.8 & 29.6 \\
\textbf{Zen-Agent (full system)} & \textbf{54.2} & \textbf{26.4} \\
\bottomrule
\end{tabular}
\end{table}

The full Zen-Agent system resolves 271 of 500 issues, a 7.3 percentage-point
improvement over the previous best open system (OpenDevin at 46.9\% on the
public leaderboard). The multi-agent configuration with hierarchical planning
reduces average steps by 25\% compared to single-agent mode, as the Coder
agent focuses on implementation while the Researcher agent handles codebase
exploration in parallel.

\paragraph{Failure Analysis.}
Of the 229 unresolved issues, we categorize failures as:
\begin{itemize}[leftmargin=2em]
  \item \textbf{Localization failure} (38\%): The system could not identify the
    correct file or function to modify.
  \item \textbf{Patch correctness} (31\%): The correct location was found but
    the generated patch was semantically incorrect.
  \item \textbf{Test understanding} (19\%): The system misinterpreted the
    expected behavior described in the issue.
  \item \textbf{Environment issues} (12\%): Timeout, dependency resolution
    failures, or sandbox limitations.
\end{itemize}

\subsection{WebArena}

WebArena \citep{zhou2024webarena} provides 812 tasks across self-hosted
instances of Reddit, GitLab, shopping sites, Wikipedia, and content management
systems. Tasks require multi-step web navigation with form filling, clicking,
and information extraction.

\begin{table}[t]
\centering
\caption{Results on WebArena (812 tasks). Task success requires exact match
with the reference answer or successful completion of the specified action
sequence.}
\label{tab:webarena}
\begin{tabular}{@{}lcc@{}}
\toprule
\textbf{System} & \textbf{Task Success (\%)} & \textbf{Avg. Actions} \\
\midrule
WebAgent \citep{gur2024webagent} & 14.6 & 22.3 \\
AWM \citep{wang2024awm} & 26.1 & 19.8 \\
AgentQ \citep{putta2024agentq} & 31.4 & 25.1 \\
SteP \citep{sodhi2024step} & 35.8 & 18.4 \\
\midrule
Zen-Agent (single-agent) & 36.2 & 21.5 \\
Zen-Agent (multi-agent, flat) & 39.7 & 17.3 \\
\textbf{Zen-Agent (full system)} & \textbf{42.8} & \textbf{15.9} \\
\bottomrule
\end{tabular}
\end{table}

The full system achieves 42.8\% task success, with the Browser agent handling
navigation while the Analyst agent processes extracted data. The hierarchical
planner proves especially valuable on multi-site tasks (e.g., ``compare prices
across Shopping and CMS''), where it naturally decomposes the goal into parallel
browsing subtasks.

\subsection{GAIA}

GAIA \citep{mialon2024gaia} tests general AI assistants on questions requiring
multi-step reasoning, tool use, and world knowledge. Questions are divided into
three difficulty levels.

\begin{table}[t]
\centering
\caption{Results on GAIA by difficulty level. Accuracy is exact-match on the
reference answer.}
\label{tab:gaia}
\begin{tabular}{@{}lcccc@{}}
\toprule
\textbf{System} & \textbf{Level 1} & \textbf{Level 2} & \textbf{Level 3} &
\textbf{Overall} \\
\midrule
GPT-4 + plugins & 68.2 & 41.3 & 22.1 & 48.7 \\
AutoGPT & 52.1 & 33.7 & 18.4 & 38.2 \\
HuggingGPT & 61.4 & 38.9 & 20.7 & 44.3 \\
FRIDAY & 72.3 & 48.1 & 31.2 & 54.1 \\
\midrule
Zen-Agent (single) & 74.1 & 50.3 & 34.8 & 56.2 \\
\textbf{Zen-Agent (full)} & \textbf{82.7} & \textbf{61.4} & \textbf{43.2} &
\textbf{65.3} \\
\bottomrule
\end{tabular}
\end{table}

Zen-Agent achieves 61.4\% on Level-3 questions, which require the most complex
multi-step reasoning. The Researcher and Analyst agents collaborate effectively
on these tasks, with the Researcher gathering relevant information and the
Analyst performing calculations and verification.

% ═════════════════════════════════════════════════════════════════════════════
% 8. ABLATION STUDIES
% ═════════════════════════════════════════════════════════════════════════════
\section{Ablation Studies}
\label{sec:ablation}

We conduct ablation studies to quantify the contribution of each architectural
component, evaluated on a held-out subset of 100 SWE-bench Verified instances.

\subsection{Component Ablation}

\begin{table}[t]
\centering
\caption{Ablation study on SWE-bench Verified (100-instance subset). Each row
removes one component while keeping others intact.}
\label{tab:ablation}
\begin{tabular}{@{}lcc@{}}
\toprule
\textbf{Configuration} & \textbf{\% Resolved} & \textbf{$\Delta$} \\
\midrule
Full Zen-Agent & 55.0 & -- \\
\midrule
$-$ HTG Planner (flat decomposition) & 50.0 & $-5.0$ \\
$-$ RAC Critique phase & 48.0 & $-7.0$ \\
$-$ RAC Backtracking & 49.0 & $-6.0$ \\
$-$ Shared Episodic Memory & 51.0 & $-4.0$ \\
$-$ Compound tool calls & 53.0 & $-2.0$ \\
$-$ RLEF training & 46.0 & $-9.0$ \\
$-$ Self-improvement rounds & 52.0 & $-3.0$ \\
\midrule
Single agent (Coder only) & 44.0 & $-11.0$ \\
\bottomrule
\end{tabular}
\end{table}

The results show that RLEF training provides the largest individual gain
($+9$ points), followed by the RAC Critique phase ($+7$ points) and HTG
Planner ($+5$ points). Removing all multi-agent components reduces performance
to 44\%, confirming the value of the orchestration approach.

\subsection{Communication Topology}

\begin{table}[t]
\centering
\caption{Effect of communication topology on SWE-bench (100-instance subset).}
\label{tab:topology}
\begin{tabular}{@{}lccc@{}}
\toprule
\textbf{Topology} & \textbf{\% Resolved} & \textbf{Msg Count} &
\textbf{Latency (s)} \\
\midrule
Star & 52.0 & 142 & 8.3 \\
Mesh & 53.0 & 387 & 4.1 \\
Hierarchical & \textbf{55.0} & 198 & 5.2 \\
\bottomrule
\end{tabular}
\end{table}

The hierarchical topology achieves the highest resolution rate while maintaining
moderate message counts and latency, validating our architectural choice.

\subsection{Memory Capacity}

We vary the SEM capacity (maximum number of records) and measure its effect on
tasks requiring codebase-wide understanding:

\begin{table}[t]
\centering
\caption{Effect of SEM capacity on SWE-bench resolution rate.}
\label{tab:memory}
\begin{tabular}{@{}lcc@{}}
\toprule
\textbf{SEM Capacity} & \textbf{\% Resolved} & \textbf{Retrieval Latency (ms)} \\
\midrule
100 records & 49.0 & 0.3 \\
500 records & 53.0 & 0.8 \\
1,000 records & 55.0 & 1.4 \\
5,000 records & 55.0 & 4.7 \\
10,000 records & 54.0 & 12.1 \\
\bottomrule
\end{tabular}
\end{table}

Performance saturates at approximately 1,000 records, with larger capacities
adding retrieval latency without improving resolution rates. The slight
degradation at 10,000 records suggests that noisy memories can hinder
retrieval quality.

\subsection{RAC Patience Parameter}

We sweep the patience parameter $k$ (consecutive low-progress steps before
backtracking) across values $\{1, 2, 3, 5, 8\}$:

\begin{table}[t]
\centering
\caption{Effect of RAC patience parameter $k$ on SWE-bench.}
\label{tab:patience}
\begin{tabular}{@{}lccc@{}}
\toprule
\textbf{Patience $k$} & \textbf{\% Resolved} & \textbf{Avg. Steps} &
\textbf{Backtracks} \\
\midrule
1 & 47.0 & 34.2 & 18.1 \\
2 & 51.0 & 29.8 & 9.4 \\
3 & \textbf{55.0} & 26.4 & 5.7 \\
5 & 53.0 & 28.1 & 3.2 \\
8 & 49.0 & 31.7 & 1.8 \\
\bottomrule
\end{tabular}
\end{table}

The optimal patience is $k=3$: too aggressive backtracking ($k=1$) abandons
promising strategies prematurely, while too much patience ($k=8$) wastes steps
on failing approaches.

\subsection{Model Scale}

We evaluate the effect of the Orchestrator model size:

\begin{table}[t]
\centering
\caption{Effect of Orchestrator model scale on SWE-bench Verified.}
\label{tab:scale}
\begin{tabular}{@{}lcc@{}}
\toprule
\textbf{Orchestrator Model} & \textbf{\% Resolved} & \textbf{Cost (\$/task)} \\
\midrule
Zen-MoDE 7B & 38.0 & 0.42 \\
Zen-MoDE 14B & 45.0 & 0.87 \\
Zen-MoDE 32B & 51.0 & 1.63 \\
Zen-MoDE 72B & \textbf{55.0} & 3.21 \\
\bottomrule
\end{tabular}
\end{table}

Larger models improve planning quality consistently, though with diminishing
returns. The 72B model is 1.45$\times$ more effective than the 32B model at
1.97$\times$ the cost, suggesting that 32B may be preferred for
cost-sensitive deployments.

% ═════════════════════════════════════════════════════════════════════════════
% 9. QUALITATIVE ANALYSIS
% ═════════════════════════════════════════════════════════════════════════════
\section{Qualitative Analysis}
\label{sec:qualitative}

\subsection{Case Study: Complex SWE-bench Issue}

We present a detailed trace of Zen-Agent resolving a complex issue from the
Django repository (issue \#15819: ``QuerySet.union() should check for
conflicting column names'').

\paragraph{Phase 1: Planning.}
The Orchestrator decomposes the task into four subtasks:
\begin{enumerate}[leftmargin=2em]
  \item \textsc{Researcher}: Locate the \texttt{QuerySet.union()} implementation
    and understand the current behavior.
  \item \textsc{Researcher}: Find related test files and understand the expected
    behavior from existing tests.
  \item \textsc{Coder}: Implement the column-name conflict check.
  \item \textsc{Coder}: Write regression tests for the fix.
\end{enumerate}

Subtasks 1 and 2 are executed in parallel (no dependency), while subtask 3
depends on both, and subtask 4 depends on 3.

\paragraph{Phase 2: Execution.}
The Researcher agents complete exploration in 8 steps total (4 each, running
concurrently). Their findings are written to SEM: the implementation location
(\texttt{django/db/models/query.py:L892}), the SQL generation path, and
relevant existing tests.

The Coder agent reads these SEM entries and implements the fix in 6 steps,
including one backtrack when the initial approach (raising an error) is
replaced by a column-aliasing strategy after the Critique phase identifies
that the error-based approach would break backward compatibility.

\paragraph{Phase 3: Validation.}
The Coder agent runs the test suite and verifies that all existing tests pass
and the new regression tests catch the previously-broken case. Total: 18 steps,
compared to 42 steps for the single-agent baseline.

\subsection{Case Study: Multi-Site WebArena Task}

On the task ``Find the cheapest laptop on Shopping and create a GitLab issue
with a price comparison table'', Zen-Agent dispatches two Browser agents in
parallel: one navigates Shopping to extract laptop prices, while the other
opens GitLab and prepares an issue template. The Analyst agent then formats
the comparison table, and the Browser agent submits the issue. The
hierarchical plan naturally captures the data-flow dependency: GitLab
submission depends on price extraction, but both browsing sessions can start
simultaneously.

% ═════════════════════════════════════════════════════════════════════════════
% 10. LIMITATIONS
% ═════════════════════════════════════════════════════════════════════════════
\section{Limitations}
\label{sec:limitations}

Despite strong results, Zen-Agent has several limitations:

\begin{enumerate}[leftmargin=2em]
  \item \textbf{Cost.} The multi-agent architecture consumes 2--4$\times$ more
    tokens than single-agent approaches. At current inference prices, the
    average cost per SWE-bench task is \$3.21, which may be prohibitive for
    high-volume deployment.

  \item \textbf{Latency.} Inter-agent communication and memory retrieval add
    overhead. End-to-end wall-clock time per task averages 4.2 minutes on
    SWE-bench, compared to 2.8 minutes for the single-agent baseline.

  \item \textbf{Planning errors.} The HTG Planner occasionally produces
    decompositions that are correct in structure but suboptimal in granularity.
    Over-decomposition (too many small tasks) increases communication overhead,
    while under-decomposition reduces the benefit of parallelism.

  \item \textbf{Memory noise.} Despite consolidation, SEM can accumulate
    inaccurate or outdated records that mislead subsequent agents. The
    confidence-based resolution policy mitigates but does not eliminate this
    issue.

  \item \textbf{Benchmark coverage.} Our evaluation covers software engineering,
    web navigation, and general reasoning. Performance on other domains
    (e.g., scientific computing, creative writing, data analysis) remains
    unevaluated.

  \item \textbf{Safety.} Multi-agent systems amplify the attack surface for
    prompt injection and tool misuse. While sandboxing mitigates execution-level
    risks, the inter-agent communication channel could potentially be exploited
    by adversarial inputs embedded in task descriptions.

  \item \textbf{Reproducibility.} Despite using greedy decoding, minor
    numerical differences across hardware configurations can produce
    different trajectories, leading to variance of approximately $\pm 1.5\%$
    on benchmark scores across independent runs.
\end{enumerate}

% ═════════════════════════════════════════════════════════════════════════════
% 11. RELATED WORK (EXTENDED)
% ═════════════════════════════════════════════════════════════════════════════
\section{Extended Related Work}
\label{sec:extended-related}

\paragraph{Agentic Coding Systems.}
SWE-Agent \citep{yang2024sweagent} provides a custom shell interface for LLM
agents to interact with repositories. AutoCodeRover \citep{zhang2024autocoderover}
combines code search with iterative repair. Agentless \citep{xia2024agentless}
takes a non-interactive approach, generating patches in a single pass.
OpenDevin \citep{wang2024opendevin} provides a sandboxed environment with
browser and shell access. Zen-Agent subsumes these approaches by supporting all
their tool modalities within a unified multi-agent framework.

\paragraph{Web Agents.}
WebGPT \citep{nakano2022webgpt} fine-tunes GPT-3 for web browsing with human
feedback. Mind2Web \citep{deng2024mind2web} provides a large-scale dataset for
web agent training. WebVoyager \citep{he2024webvoyager} uses multimodal
perception for web navigation. Zen-Agent's Browser agent builds on these
approaches while adding the RAC loop for self-correction.

\paragraph{Planning with LLMs.}
Tree of Thoughts \citep{yao2024tot} explores multiple reasoning paths via
tree search. Graph of Thoughts \citep{besta2024got} generalizes to arbitrary
graph structures. Plan-and-Solve \citep{wang2023plansolve} decomposes problems
before solving. Zen-Agent's HTG Planner is distinguished by its formal
specification and dynamic re-planning capabilities.

\paragraph{Reward Modeling for Agents.}
Process Reward Models \citep{lightman2024prm} provide step-level rewards for
mathematical reasoning. Outcome Reward Models evaluate final answers.
Zen-Agent's critique model is a specialized process reward model trained on
agent trajectories rather than mathematical proofs.

% ═════════════════════════════════════════════════════════════════════════════
% 12. FUTURE WORK
% ═════════════════════════════════════════════════════════════════════════════
\section{Future Work}
\label{sec:future}

Several promising directions emerge from this work:

\begin{itemize}[leftmargin=2em]
  \item \textbf{Adaptive agent spawning}: Dynamically creating and destroying
    specialized agents based on task requirements, rather than maintaining a
    fixed pool.
  \item \textbf{Cross-episode learning}: Enabling the system to improve across
    tasks by updating the routing function and planning heuristics online.
  \item \textbf{Federated multi-agent}: Distributing agents across multiple
    machines or organizations with privacy-preserving communication.
  \item \textbf{Formal verification}: Extending the completeness guarantee of
    the HTG Planner to cover dynamic re-planning scenarios.
  \item \textbf{Human-in-the-loop}: Integrating human feedback at the
    Orchestrator level for safety-critical deployments.
\end{itemize}

% ═════════════════════════════════════════════════════════════════════════════
% 13. CONCLUSION
% ═════════════════════════════════════════════════════════════════════════════
\section{Conclusion}
\label{sec:conclusion}

We have presented Zen-Agent, a multi-agent orchestration framework that
achieves state-of-the-art performance on three challenging agentic AI
benchmarks through hierarchical task decomposition, reflective action cycles,
and shared episodic memory. Our key findings are:

\begin{enumerate}[leftmargin=2em]
  \item Hierarchical Task Graphs with dependency-aware scheduling reduce
    redundant computation and enable effective parallelism, contributing a
    5-point improvement on SWE-bench.
  \item The Reflective Action Cycle with critique-driven backtracking reduces
    wasted steps by 38\% compared to standard ReAct, contributing a 7-point
    improvement.
  \item Shared Episodic Memory eliminates redundant exploration across agents,
    contributing a 4-point improvement.
  \item Reinforcement Learning from Execution Feedback is the single most
    impactful training technique, contributing a 9-point improvement over
    supervised fine-tuning alone.
\end{enumerate}

These results demonstrate that principled multi-agent orchestration with
hierarchical planning is a productive direction for building more capable
autonomous AI systems. We release the full Zen-Agent framework, trained
weights, and evaluation code at
\url{https://github.com/hanzoai/zen-agent} to support further research.

% ═════════════════════════════════════════════════════════════════════════════
% ACKNOWLEDGMENTS
% ═════════════════════════════════════════════════════════════════════════════
\section*{Acknowledgments}

We thank the Hanzo AI engineering team for infrastructure support and the
Zoo Labs Foundation for compute resources. This work was supported in part
by the Hanzo AI Research Fund and the Zoo Decentralized Science Initiative.
We are grateful to the SWE-bench, WebArena, and GAIA benchmark teams for
their publicly available evaluation frameworks.

% ═════════════════════════════════════════════════════════════════════════════
% REFERENCES
% ═════════════════════════════════════════════════════════════════════════════
\begin{thebibliography}{30}

\bibitem[Ahn et al.(2022)]{ahn2022saycan}
Ahn, M., Brohan, A., Brown, N., et al.
\newblock Do as i can, not as i say: Grounding language in robotic affordances.
\newblock In \emph{Conference on Robot Learning (CoRL)}, 2022.

\bibitem[Besta et al.(2024)]{besta2024got}
Besta, M., Blach, N., Kubicek, A., et al.
\newblock Graph of thoughts: Solving elaborate problems with large language models.
\newblock In \emph{AAAI Conference on Artificial Intelligence}, 2024.

\bibitem[Deng et al.(2024)]{deng2024mind2web}
Deng, X., Gu, Y., Zheng, B., et al.
\newblock Mind2Web: Towards a generalist agent for the web.
\newblock In \emph{NeurIPS}, 2024.

\bibitem[Erol et al.(1994)]{erol1994htn}
Erol, K., Hendler, J., and Nau, D.~S.
\newblock HTN planning: Complexity and expressivity.
\newblock In \emph{AAAI}, pp.~1123--1128, 1994.

\bibitem[Fikes and Nilsson(1971)]{fikes1971strips}
Fikes, R.~E. and Nilsson, N.~J.
\newblock STRIPS: A new approach to the application of theorem proving to
problem solving.
\newblock \emph{Artificial Intelligence}, 2(3-4):189--208, 1971.

\bibitem[Gur et al.(2024)]{gur2024webagent}
Gur, I., Furuta, H., Huang, A., et al.
\newblock A real-world WebAgent with planning, long context understanding,
and program synthesis.
\newblock In \emph{ICLR}, 2024.

\bibitem[He et al.(2024)]{he2024webvoyager}
He, H., Yao, W., Ma, K., et al.
\newblock WebVoyager: Building an end-to-end web agent with large multimodal
models.
\newblock In \emph{ACL}, 2024.

\bibitem[Hewitt et al.(1973)]{hewitt1973actor}
Hewitt, C., Bishop, P., and Steiger, R.
\newblock A universal modular ACTOR formalism for artificial intelligence.
\newblock In \emph{IJCAI}, pp.~235--245, 1973.

\bibitem[Hong et al.(2024)]{hong2024metagpt}
Hong, S., Zhuge, M., Chen, J., et al.
\newblock MetaGPT: Meta programming for a multi-agent collaborative framework.
\newblock In \emph{ICLR}, 2024.

\bibitem[Huang et al.(2023)]{huang2023inner}
Huang, W., Xia, F., Xiao, T., et al.
\newblock Inner monologue: Embodied reasoning through planning with language
models.
\newblock In \emph{CoRL}, 2023.

\bibitem[Jimenez et al.(2024)]{jimenez2024swebench}
Jimenez, C.~E., Yang, J., Wettig, A., et al.
\newblock SWE-bench: Can language models resolve real-world GitHub issues?
\newblock In \emph{ICLR}, 2024.

\bibitem[Li et al.(2023)]{li2023camel}
Li, G., Hammoud, H.~A., Itani, H., et al.
\newblock CAMEL: Communicative agents for ``mind'' exploration of large language
model society.
\newblock In \emph{NeurIPS}, 2023.

\bibitem[Lightman et al.(2024)]{lightman2024prm}
Lightman, H., Kosaraju, V., Burda, Y., et al.
\newblock Let's verify step by step.
\newblock In \emph{ICLR}, 2024.

\bibitem[Malkov and Yashunin(2020)]{malkov2020hnsw}
Malkov, Y.~A. and Yashunin, D.~A.
\newblock Efficient and robust approximate nearest neighbor search using
hierarchical navigable small world graphs.
\newblock \emph{IEEE TPAMI}, 42(4):824--836, 2020.

\bibitem[Mialon et al.(2024)]{mialon2024gaia}
Mialon, G., Fourrier, C., Swift, C., et al.
\newblock GAIA: A benchmark for general AI assistants.
\newblock In \emph{ICLR}, 2024.

\bibitem[MCP(2024)]{mcp2024}
Model Context Protocol.
\newblock \url{https://modelcontextprotocol.io}, 2024.

\bibitem[Nakano et al.(2022)]{nakano2022webgpt}
Nakano, R., Hilton, J., Balaji, S., et al.
\newblock WebGPT: Browser-assisted question-answering with human feedback.
\newblock \emph{arXiv preprint arXiv:2112.09332}, 2022.

\bibitem[Packer et al.(2023)]{packer2023memgpt}
Packer, C., Wooders, S., Lin, K., et al.
\newblock MemGPT: Towards LLMs as operating systems.
\newblock \emph{arXiv preprint arXiv:2310.08560}, 2023.

\bibitem[Park et al.(2023)]{park2023generative}
Park, J.~S., O'Brien, J.~C., Cai, C.~J., et al.
\newblock Generative agents: Interactive simulacra of human behavior.
\newblock In \emph{UIST}, 2023.

\bibitem[Putta et al.(2024)]{putta2024agentq}
Putta, P., Mills, E., Garg, N., et al.
\newblock Agent Q: Advanced reasoning and learning for autonomous AI agents.
\newblock \emph{arXiv preprint arXiv:2408.07199}, 2024.

\bibitem[Qian et al.(2024)]{qian2024chatdev}
Qian, C., Liu, W., Liu, H., et al.
\newblock ChatDev: Communicative agents for software development.
\newblock In \emph{ACL}, 2024.

\bibitem[Rajbhandari et al.(2020)]{rajbhandari2020zero}
Rajbhandari, S., Rasley, J., Ruwase, O., and He, Y.
\newblock ZeRO: Memory optimizations toward training trillion parameter models.
\newblock In \emph{SC}, 2020.

\bibitem[Schick et al.(2023)]{schick2023toolformer}
Schick, T., Dwivedi-Yu, J., Dess\`{i}, R., et al.
\newblock Toolformer: Language models can teach themselves to use tools.
\newblock In \emph{NeurIPS}, 2023.

\bibitem[Schulman et al.(2017)]{schulman2017ppo}
Schulman, J., Wolski, F., Dhariwal, P., et al.
\newblock Proximal policy optimization algorithms.
\newblock \emph{arXiv preprint arXiv:1707.06347}, 2017.

\bibitem[Shinn et al.(2023)]{shinn2023reflexion}
Shinn, N., Cassano, F., Gopinath, A., et al.
\newblock Reflexion: Language agents with verbal reinforcement learning.
\newblock In \emph{NeurIPS}, 2023.

\bibitem[Sodhi et al.(2024)]{sodhi2024step}
Sodhi, P., Wu, S.~R., Lu, H., and Savva, M.
\newblock SteP: Stacked LLM policies for web actions.
\newblock \emph{arXiv preprint arXiv:2310.03720}, 2024.

\bibitem[Wang et al.(2023)]{wang2023voyager}
Wang, G., Xie, Y., Jiang, Y., et al.
\newblock Voyager: An open-ended embodied agent with large language models.
\newblock In \emph{NeurIPS}, 2023.

\bibitem[Wang et al.(2024a)]{wang2024opendevin}
Wang, X., Li, Y., Zhang, H., et al.
\newblock OpenDevin: An open platform for AI software developers as generalist
agents.
\newblock \emph{arXiv preprint arXiv:2407.16741}, 2024.

\bibitem[Wang et al.(2024b)]{wang2024awm}
Wang, J., Jia, Z., Liu, A., and Zhu, S.-C.
\newblock Agent Workflow Memory.
\newblock \emph{arXiv preprint arXiv:2409.07429}, 2024.

\bibitem[Wang et al.(2023)]{wang2023plansolve}
Wang, L., Xu, W., Lan, Y., et al.
\newblock Plan-and-solve prompting: Improving zero-shot chain-of-thought
reasoning by large language models.
\newblock In \emph{ACL}, 2023.

\bibitem[Wu et al.(2024)]{wu2024autogen}
Wu, Q., Bansal, G., Zhang, J., et al.
\newblock AutoGen: Enabling next-gen LLM applications via multi-agent
conversation.
\newblock In \emph{ICLR}, 2024.

\bibitem[Xia et al.(2024)]{xia2024agentless}
Xia, C.~S., Deng, Y., Dunn, S., and Zhang, L.
\newblock Agentless: Demystifying LLM-based software engineering agents.
\newblock \emph{arXiv preprint arXiv:2407.01489}, 2024.

\bibitem[Yang et al.(2024)]{yang2024sweagent}
Yang, J., Jimenez, C.~E., Wettig, A., et al.
\newblock SWE-agent: Agent-computer interfaces enable automated software
engineering.
\newblock In \emph{NeurIPS}, 2024.

\bibitem[Yao et al.(2023)]{yao2023react}
Yao, S., Zhao, J., Yu, D., et al.
\newblock ReAct: Synergizing reasoning and acting in language models.
\newblock In \emph{ICLR}, 2023.

\bibitem[Yao et al.(2024)]{yao2024tot}
Yao, S., Yu, D., Zhao, J., et al.
\newblock Tree of thoughts: Deliberate problem solving with large language
models.
\newblock In \emph{NeurIPS}, 2024.

\bibitem[Zhang et al.(2024)]{zhang2024autocoderover}
Zhang, Y., Ruan, W., Fan, Z., and Chen, A.
\newblock AutoCodeRover: Autonomous program improvement.
\newblock In \emph{ISSTA}, 2024.

\bibitem[Zhou et al.(2024)]{zhou2024webarena}
Zhou, S., Xu, F.~F., Zhu, H., et al.
\newblock WebArena: A realistic web environment for building autonomous agents.
\newblock In \emph{ICLR}, 2024.

\end{thebibliography}

% ═════════════════════════════════════════════════════════════════════════════
% APPENDIX
% ═════════════════════════════════════════════════════════════════════════════
\appendix

\section{Full RAC Prompt Templates}
\label{app:prompts}

\subsection{Reason Phase Prompt}

\begin{lstlisting}[language={},title={Reason Phase System Prompt}]
You are a specialized {agent_type} agent working on a
subtask within a larger goal. Your role is to analyze
the current state and plan your next action.

Current Task: {task_description}
Environment State: {state_summary}
Previous Actions: {action_history}
Shared Memory Context: {sem_context}

Think step by step about:
1. What has been accomplished so far
2. What remains to be done
3. What is the most productive next action
4. What tools are available and appropriate

Output your reasoning and planned action.
\end{lstlisting}

\subsection{Critique Phase Prompt}

\begin{lstlisting}[language={},title={Critique Phase System Prompt}]
You are evaluating whether the last action made progress
toward the task goal.

Task: {task_description}
Action Taken: {action}
Observation: {observation}
Previous Progress: {progress_history}

Evaluate on a scale of 0.0 to 1.0:
- 0.0: No progress or regression
- 0.5: Marginal progress (information gained but
        no concrete advancement)
- 1.0: Significant progress (task partially completed
        or key blocker resolved)

Output: {"progress": <float>, "reasoning": "<str>"}
\end{lstlisting}

\section{Extended Benchmark Results}
\label{app:extended}

\subsection{SWE-bench Per-Repository Breakdown}

\begin{table}[H]
\centering
\caption{SWE-bench Verified results by repository.}
\label{tab:per-repo}
\begin{tabular}{@{}lcc@{}}
\toprule
\textbf{Repository} & \textbf{Issues} & \textbf{\% Resolved} \\
\midrule
django/django & 114 & 58.8 \\
scikit-learn/scikit-learn & 62 & 51.6 \\
matplotlib/matplotlib & 54 & 46.3 \\
sympy/sympy & 77 & 55.8 \\
pytest-dev/pytest & 38 & 60.5 \\
astropy/astropy & 22 & 45.5 \\
flask/flask & 18 & 61.1 \\
requests/requests & 15 & 53.3 \\
sphinx-doc/sphinx & 42 & 50.0 \\
pylint-dev/pylint & 28 & 53.6 \\
xarray/xarray & 20 & 45.0 \\
pandas-dev/pandas & 10 & 40.0 \\
\midrule
\textbf{Overall} & \textbf{500} & \textbf{54.2} \\
\bottomrule
\end{tabular}
\end{table}

\section{Compute Requirements}
\label{app:compute}

\begin{table}[H]
\centering
\caption{Compute requirements for training and inference.}
\label{tab:compute}
\begin{tabular}{@{}lcc@{}}
\toprule
\textbf{Stage} & \textbf{GPU Hours (H100)} & \textbf{Peak Memory} \\
\midrule
SFT (Stage 1) & 480 & 640 GB \\
RLEF (Stage 2) & 960 & 1.28 TB \\
Self-improvement ($\times$3) & 960 & 640 GB \\
\midrule
Inference (per task) & 0.02 & 160 GB \\
\bottomrule
\end{tabular}
\end{table}

\end{document}