Files
zeekay f3160b9e60 benchmarks: a harness published nowhere a reader can open is named, not linked
brain/ and cronquestions/ are in no branch of hanzoai/benchmarks on
GitHub, so five links 404ed. harness.url is optional, Harness renders the
one way (a link where there is one), and a result's source is the page
itself when the harness has no public home.
2026-10-30 01:02:34 -07:00

112 lines
5.8 KiB
TypeScript
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/**
* /benchmarks/goroutine — what a RUNNING agent costs.
*
* The tables render the goroutine pass of lib/data/benchmarks-runtime.json. The
* prose quotes figures from that pass by hand, so a rerun edits both.
*/
import { T } from '@hanzo/i18n/react'
import { pageMeta } from '@/lib/page-meta'
import { BENCHMARKS, DORMANT, bench, paperUrl } from '@/lib/benchmarks'
import { BenchJsonLd } from '@/components/benchmarks/JsonLd'
import { Grid, H2, H3, Harness, Meta, More, Note, P, Shell, Spec, type Col } from '@/components/benchmarks/kit'
import { HOSTS, RUNTIME, shots, type Host, type Shot } from '@/lib/bench-data'
const B = bench('goroutine')!
export const metadata = pageMeta({
title: 'Live agent footprint benchmark',
description:
'Benchmark AI agents as goroutines: 601 bytes heap, 125 ns spawn, and 107 ms to wake 1M agents vs V8 isolates. Ultra-light concurrency.',
path: '/benchmarks/goroutine',
})
const MEASURES = ['heap bytes per live agent', 'spawn latency', 'wake latency', 'runtime instantiation']
const AGENT = shots('goroutine')
const aCols: Col<Shot>[] = [
{ head: <T>measurement</T>, get: (r) => r.what, left: true },
{ head: <T>Hanzo, measured</T>, get: (r) => r.ours },
{ head: <T>the published figure</T>, get: (r) => r.theirs ?? '—', left: true },
]
const lCols: Col<Host>[] = [
{ head: <T>runtime</T>, get: (r) => r.runtime, left: true },
{ head: <T>language</T>, get: (r) => r.language, left: true },
{ head: <T>instantiate</T>, get: (r) => r.instantiate },
{ head: <T>call</T>, get: (r) => r.call },
]
export default function Page() {
return (
<>
<BenchJsonLd b={B} measures={MEASURES} />
<Shell b={B}>
<Spec b={B} />
<H2><T>What is measured</T></H2>
<P>
<T values={{ running: <em><T>running</T></em> }}>
{'A dormant agent is a row on disk. An agent that is {running} still has to hold its place, and the question is what that place costs. Here it is a goroutine: the heap it occupies, the time to start one, and the time to wake a million of them at once. The comparison column is what a platform built on V8 isolates publishes for the same two quantities.'}
</T>
</P>
<H2><T>Results</T></H2>
<Meta>
<T values={{ machine: RUNTIME.passes.goroutine.machine }}>
{'{machine} · the pass of 2026-09-07 · memory was identical on all five runs; timings are medians'}
</T>
</Meta>
<Grid cols={aCols} rows={AGENT} keyOf={(r) => r.what} />
<Note>
<T>{'601 bytes against a 128 MB container floor is about 223,000×, and against a V8 isolate about 1,745× — and that second one is a goroutine against a JavaScript VM, so read it as what each primitive costs rather than as one beating the other. The isolate column is measured here rather than cited: a fresh isolated-vm isolate is 1.00 MiB and 0.35 ms on this laptop, close to the 1.2 MB attributed to it, because a megabyte is what any isolate costs including one of ours. The number that matters more for an interactive system is the 125 ns: an agent that costs nothing to start does not need to be kept warm, and a pool is a thing you maintain because starting was expensive.'}</T>
</Note>
<H3><T>What can run inside one</T></H3>
<Meta><T>the same pass · instantiate is per context; call is a warm invocation</T></Meta>
<Grid cols={lCols} rows={HOSTS} keyOf={(r) => r.runtime} />
<Note>
<T>{'WASM is not one language: CPython, QuickJS for TypeScript, Rust and Go all target it. A V8 isolate is JavaScript, and only JavaScript.'}</T>
</Note>
<H2><T>How to get this wrong</T></H2>
<P>
<strong>
<T>Benchmark a built binary.</T>
</strong>{' '}
<T values={{ cmd: <code>go run</code> }}>
{'Under {cmd} the compile is counted and spawn reads 253 ns instead of 125 ns, an error from the command you used to start the measurement. The reproduce line on this page builds first for exactly that reason.'}
</T>
</P>
<P>
<strong>
<T>Throw the first round away.</T>
</strong>{' '}
<T>
{"A freshly built binary is slower the first time it runs, and not slightly: three fresh builds read 202, 210 and 201 ns, and the same three read 123, 125 and 123 ns when run again. The goroutines did not get cheaper. The window is 41 ms rather than 25 ms for the same 200,000 spawns, so it is one fixed cost inside the measurement, paid once per binary for first-touch paging and the kernel's signature check on first exec. Divided by 200,000 and printed per goroutine, it is the cost of starting the program wearing the name of the thing being measured. This page read 187 ns for exactly that reason; the harness now runs the round twice and reports the second."}
</T>
</P>
<P>
<strong>
<T>A goroutine is not a sandbox.</T>
</strong>{' '}
<T>
{'These agents share an address space. Nothing here is a security boundary, and an agent that must run untrusted code gets a microVM, which is a separate measurement. The two answer different questions and a platform needs both.'}
</T>
</P>
<P>
<strong>
<T>One machine, one day.</T>
</strong>{' '}
<T>{'A single pass on one laptop, reproducible by one command. It establishes a floor; it is not a service level under load.'}</T>
</P>
<More b={B} all={BENCHMARKS} />
<Meta>
<T values={{ harness: <Harness of={B.harness} />, paper: <a href={paperUrl(DORMANT)}>{DORMANT.title}</a> }}>
{'harness and the committed table: {harness} · the paper: {paper}'}
</T>
</Meta>
</Shell>
</>
)
}