mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-29 07:34:31 +00:00
zeekayandHanzo Dev
1d1e7be5ab
Both report benchmarks that already live in hanzoai/cloud under bench/, so the numbers have a script behind them rather than a spreadsheet. hanzo-dormant-agents takes apart the estimate that costs an agent as a process. State, execution and isolation are separable and differ by five orders of magnitude: a dormant agent is 477 bytes on disk and resumes in 0.034 ms, a parked loop is 601 bytes of heap, and the isolation boundary — the expensive one — is only owed by an agent actually running. The paper argues the container is worth its price where it is needed and says so; the claim is about which agents are charged for it. hanzo-multi-hop-retrieval is a negative result. On LoCoMo a dense retriever finds every required turn for 78.2% of single-hop questions and 22.7% of multi-hop ones, but finds at least one required turn about 80% of the time in both — so it understands the questions and cannot reach all of the answer. Four retrieval policies over one fixed index move that by at most 0.7 points and two make it worse. A diagnostic says why it was never going to work: 95.4% of multi-hop evidence spans sessions and the median worst-ranked gold turn sits at rank 67, so the missing signal is linkage rather than similarity. Publishing the dead end is the useful part. Each paper carries results-data.tex, generated from the runs, and prints XXX in place of any constant that file does not supply — a number cannot drift from its measurement without the drift showing on the page. Both compile under the CI gate's flags, -Werror included. INDEX.md is regenerated by make index, which also picks up the papers added since it was last written and the header its current generator emits. Co-authored-by: Hanzo Dev <[email protected]>
63 lines
3.3 KiB
TeX
63 lines
3.3 KiB
TeX
\section{Code: three language runtimes, none of them a container}
|
|
\label{sec:code}
|
|
|
|
An agent that only waits is not interesting. At some point it runs code --- a
|
|
tool implementation, a transformation, a policy the tenant wrote. The reflex is
|
|
that running tenant code requires a container, and \S\ref{sec:isolation} is
|
|
about when that reflex is right. But there is a tier between ``call a Go
|
|
function'' and ``start Linux'', and it is not exotic: an interpreter, in the
|
|
host process, with the host deciding what it can reach.
|
|
|
|
We measured three, all pure Go, all linked into the same binary that holds the
|
|
agent loops.
|
|
|
|
\begin{table}[h]
|
|
\centering
|
|
\begin{tabular}{llrl}
|
|
\toprule
|
|
\textbf{Runtime} & \textbf{Runs} & \textbf{Instantiate} & \textbf{Range} \\
|
|
\midrule
|
|
wazero~\cite{wazero} & WebAssembly & \rWazero{}\,\textmu s & \rWazero{}--\rWazeroHi{}\,\textmu s \meas{} \\
|
|
goja~\cite{goja} & JavaScript (ES5.1+) & \rGoja{}\,\textmu s & \rGojaLo{}--\rGojaHi{}\,\textmu s \meas{} \\
|
|
gpython~\cite{gpython}& Python & \rGpython{}\,\textmu s & \rGpythonLo{}--\rGpythonHi{}\,\textmu s \meas{} \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\caption{Cost of a fresh execution context for tenant code, in-process. Calling
|
|
into an instantiated wazero module costs \rWazeroCall{}\,ns; re-evaluating in an
|
|
already-warm goja VM costs \rGojaWarm{}\,ns
|
|
(\rGojaWarmLo{}--\rGojaWarmHi{}\,ns).}
|
|
\label{tab:runtimes}
|
|
\end{table}
|
|
|
|
All three are microseconds. The slowest, Python, is \rGpython{}\,\textmu s ---
|
|
still four orders of magnitude below a container cold start, and roughly the
|
|
cost of a single disk seek on hardware that has not had one in a decade.
|
|
|
|
\paragraph{WebAssembly is not one language.}
|
|
The wazero row deserves emphasis because it is easy to file under ``Rust and
|
|
Go''. WebAssembly is a compilation target, and CPython, TypeScript via QuickJS,
|
|
Rust, Go, C and C++ all reach it. A \rWazero{}\,\textmu s instantiation is
|
|
therefore not a JavaScript-shaped capability; it is a general one, with a
|
|
capability model that is deny-by-default --- a module reaches exactly the host
|
|
functions it was linked against and nothing else~\cite{wasm}. That is a stronger
|
|
default than a container, which starts with a filesystem and a network stack and
|
|
must have them taken away.
|
|
|
|
\paragraph{Warm is a different number, and usually the relevant one.}
|
|
\rGojaWarm{}\,ns to evaluate in an existing VM against \rGoja{}\,\textmu s to
|
|
make a new one is a factor of about three thousand. A fleet that creates one
|
|
context per call is paying the left column; a fleet that keeps a context per
|
|
active agent pays the right. Since \S\ref{sec:state} showed that agent state is
|
|
a row that can be rehydrated in \rFleetResume{}\,ms, the natural arrangement is
|
|
to key a pooled context by agent and let it fall away when the agent goes
|
|
dormant --- which converts the instantiation cost from per-call to per-wake.
|
|
|
|
\paragraph{The honest limit.}
|
|
None of these is a security boundary against an actively hostile tenant in the
|
|
way a virtual machine is. goja and gpython are interpreters in the host address
|
|
space: a bug in one of them is a bug in the host process. wazero is stronger ---
|
|
the sandbox is the WebAssembly semantics themselves, and the module cannot name
|
|
what it was not given --- but it is still one process. Where the threat model
|
|
includes an adversary who will try to escape, the answer is the next section,
|
|
and it costs a thousand times more for exactly that reason.
|