mirror of
https://github.com/hanzoai/site.git
synced 2026-11-20 21:21:13 +00:00
platform.hanzo.ai is the platform console (with cloud.hanzo.ai redirecting to it at the edge). Point the remaining cloud.hanzo.ai links on /benchmarks and /philosophy directly at https://platform.hanzo.ai. Claude-Session: https://claude.ai/code/session_015XYKLix9PSyn6Jw2HWKprk
1 line
29 KiB
HTML
1 line
29 KiB
HTML
<div class="is_View _col-_platformweb_text-primar121 _fd-column _dsp-block _minH-100vh _bg-background"><main class="is_View _col-_platformweb_text-primar121 _fd-column _maxW-1080px _width-10037 _mr-auto _ml-auto _pt-64px _pr-24px _pl-24px _pb-96px _bg-background"><p class="is_Text _col-muted-foreg108704142 _dsp-block _mb-4px _fs-f-size-1 _lh-f-lineHeigh112922" data-slot="text"><a href="/benchmarks">Benchmarks</a> / <!-- -->MemoryAgentBench</p><span class="is_Text _col-muted-foreg108704142 _dsp-block _fs-f-size-1 _lh-f-lineHeigh112922 _ls-0--88px _tt-uppercase" data-slot="text">measured</span><h1 class="is_Text font_heading _fs-_sm_f-size-11 _fs-_lg_text-6xl _textWrap-_platformweb_balance _col-foreground _dsp-revert-laye114 _ws-inherit _ff-f-family _fs-f-size-10 _fw-497 _lh-1--19 _ls--0--012em _textWrap-balance _mt-8px _mb-16px" data-slot="display">MemoryAgentBench</h1><p class="is_Text _col-muted-foreg108704142 _dsp-revert-laye114 _ws-inherit _fs-f-size-3 _lh-1--65 _ls-0px _textWrap-pretty _maxW-720px" data-slot="lede">FactConsolidation: haystacks of 6k to 262k tokens where a later line silently overwrites an earlier one. A typed search over parsed facts resolves the question with no language model reading anything.</p><dl class="is_Text _col-text-primar121 _dsp-grid _gridTemplateColumns-minmax120px329255122 _rowGap-8px _columnGap-20px _mt-20px _mr-0px _mb-0px _ml-0px _maxW-760px _fs-f-size-2 _lh-f-lineHeigh112922" data-slot="text"><dt class="is_Text _col-muted-foreg108704142" data-slot="text">dataset</dt><dd class="is_Text _col-inherit _mt-0px _mr-0px _mb-0px _ml-0px" data-slot="text"><a href="https://huggingface.co/datasets/ai-hyz/MemoryAgentBench">MemoryAgentBench</a></dd><dt class="is_Text _col-muted-foreg108704142" data-slot="text">licence</dt><dd class="is_Text _col-inherit _mt-0px _mr-0px _mb-0px _ml-0px" data-slot="text">MIT</dd><dt class="is_Text _col-muted-foreg108704142" data-slot="text">introduced by</dt><dd class="is_Text _col-inherit _mt-0px _mr-0px _mb-0px _ml-0px" data-slot="text"><a href="https://arxiv.org/abs/2507.05257">Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions</a> — <!-- -->Hu, Wang, McAuley<!-- -->, <!-- -->UC San Diego, 2025</dd><dt class="is_Text _col-muted-foreg108704142" data-slot="text">harness</dt><dd class="is_Text _col-inherit _mt-0px _mr-0px _mb-0px _ml-0px" data-slot="text">hanzoai/benchmarks · brain</dd><dt class="is_Text _col-muted-foreg108704142" data-slot="text">our paper</dt><dd class="is_Text _col-inherit _mt-0px _mr-0px _mb-0px _ml-0px" data-slot="text"><span class="is_Text _col-inherit" data-slot="text"><a href="https://github.com/hanzoai/papers/blob/main/site/public/pdfs/hanzo-context-memory.pdf">Linkage, Not Similarity</a></span></dd></dl><code class="is_Text font_mono _col-text-primar121 _dsp-block _mt-10px _pt-10px _pb-10px _pr-14px _pl-14px _btw-1px _brw-1px _borderBottomWidth-1px _borderLeftWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _btlr-6px _btrr-6px _bbrr-6px _bblr-6px _ox-auto _bg-card _ff-f-family _fs-f-size-1 _bbs-solid _bts-solid _bls-solid _brs-solid" data-slot="text">node results.mjs</code><p class="is_Text font_mono _col-muted-foreg108704142 _dsp-block _mb-10px _ff-f-family _fs-f-size-1 _lh-f-lineHeigh112922 _mt-8px" data-slot="text">regenerates every table on this page from the raw runs</p><h2 class="is_Text font_heading _col-foreground _dsp-revert-laye114 _ws-inherit _ff-f-family _fs-f-size-7 _fw-497 _lh-1--2 _ls--0--01em _textWrap-balance _mt-56px _mb-8px" data-slot="title">What is measured</h2><p class="is_Text _col-text-second96872 _dsp-block _maxW-720px _lh-1--3563rem" data-slot="text">FactConsolidation. A haystack is a numbered list of templated facts in which a later line silently replaces an earlier one about the same subject and relation, and nothing marks the update. Questions ask for the current value, single-hop and multi-hop, over haystacks of 6k to 262k tokens. One hundred questions per haystack. The metric is substring exact match, the benchmark’s own; an unanswered question scores zero. Dev is the 6k haystacks, test is 32k, 64k and 262k.</p><p class="is_Text _col-text-second96872 _dsp-block _maxW-720px _lh-1--3563rem" data-slot="text">No language model reads the memory or writes the answer in any row here. The question is compiled into a plan sketch and a deterministic typed search over the parsed fact store resolves it — 45 templates, no line unparsed at any size, the supersession chain keyed by subject and relation. That is why these rows are not comparable to a pipeline that reads the haystack with a frontier model, and it is also the point: the retrieval is doing the work.</p><h2 class="is_Text font_heading _col-foreground _dsp-revert-laye114 _ws-inherit _ff-f-family _fs-f-size-7 _fw-497 _lh-1--2 _ls--0--01em _textWrap-balance _mt-56px _mb-8px" data-slot="title">Results</h2><p class="is_Text font_mono _col-muted-foreg108704142 _dsp-block _mb-10px _ff-f-family _fs-f-size-1 _lh-f-lineHeigh112922" data-slot="text">substring exact match · 6k from dev, 32k / 64k / 262k from test · no reader · pooled = the mean of the four haystacks</p><h3 class="is_Text font_mono _col-text-primar121 _dsp-block _mt-28px _mb-6px _fs-f-size-3 _lh-f-lineHeigh112922 _ff-f-family _fw-500" data-slot="text">single-hop</h3><div class="is_View _ox-auto _btw-1px _brw-1px _borderBottomWidth-1px _borderLeftWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _btlr-6px _btrr-6px _bbrr-6px _bblr-6px _bg-card _bbs-solid _bts-solid _bls-solid _brs-solid"><table class="is_Text _dsp-_platformweb_table _borderCollapse-_platformweb_collapse _fontVariantNumeric-_platformweb_tabular-num115 _col-text-primar121 _width-10037 _fs-f-size-2 _lh-f-lineHeigh112922" data-slot="text"><thead><tr><th class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-muted-foreg108704142 _text-left _pt-8px _pb-8px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _fw-500 _ws-nowrap _bbs-solid" data-slot="text">configuration</th><th class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-muted-foreg108704142 _text-right _pt-8px _pb-8px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _fw-500 _ws-nowrap _bbs-solid" data-slot="text">6k</th><th class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-muted-foreg108704142 _text-right _pt-8px _pb-8px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _fw-500 _ws-nowrap _bbs-solid" data-slot="text">32k</th><th class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-muted-foreg108704142 _text-right _pt-8px _pb-8px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _fw-500 _ws-nowrap _bbs-solid" data-slot="text">64k</th><th class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-muted-foreg108704142 _text-right _pt-8px _pb-8px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _fw-500 _ws-nowrap _bbs-solid" data-slot="text">262k</th><th class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-muted-foreg108704142 _text-right _pt-8px _pb-8px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _fw-500 _ws-nowrap _bbs-solid" data-slot="text">pooled</th></tr></thead><tbody><tr class="is_Text _dsp-_platformweb_table-row _col-inherit" data-slot="text"><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-left _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">plain resolver</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">96.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">97.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">95.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">94.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">95.5</td></tr><tr class="is_Text _dsp-_platformweb_table-row _col-inherit" data-slot="text"><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-left _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">typed search</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">100.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">98.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">98.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">97.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">98.3</td></tr></tbody></table></div><h3 class="is_Text font_mono _col-text-primar121 _dsp-block _mt-28px _mb-6px _fs-f-size-3 _lh-f-lineHeigh112922 _ff-f-family _fw-500" data-slot="text">multi-hop</h3><div class="is_View _ox-auto _btw-1px _brw-1px _borderBottomWidth-1px _borderLeftWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _btlr-6px _btrr-6px _bbrr-6px _bblr-6px _bg-card _bbs-solid _bts-solid _bls-solid _brs-solid"><table class="is_Text _dsp-_platformweb_table _borderCollapse-_platformweb_collapse _fontVariantNumeric-_platformweb_tabular-num115 _col-text-primar121 _width-10037 _fs-f-size-2 _lh-f-lineHeigh112922" data-slot="text"><thead><tr><th class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-muted-foreg108704142 _text-left _pt-8px _pb-8px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _fw-500 _ws-nowrap _bbs-solid" data-slot="text">configuration</th><th class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-muted-foreg108704142 _text-right _pt-8px _pb-8px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _fw-500 _ws-nowrap _bbs-solid" data-slot="text">6k</th><th class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-muted-foreg108704142 _text-right _pt-8px _pb-8px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _fw-500 _ws-nowrap _bbs-solid" data-slot="text">32k</th><th class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-muted-foreg108704142 _text-right _pt-8px _pb-8px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _fw-500 _ws-nowrap _bbs-solid" data-slot="text">64k</th><th class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-muted-foreg108704142 _text-right _pt-8px _pb-8px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _fw-500 _ws-nowrap _bbs-solid" data-slot="text">262k</th><th class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-muted-foreg108704142 _text-right _pt-8px _pb-8px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _fw-500 _ws-nowrap _bbs-solid" data-slot="text">pooled</th></tr></thead><tbody><tr class="is_Text _dsp-_platformweb_table-row _col-inherit" data-slot="text"><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-left _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">plain resolver</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">76.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">59.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">59.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">33.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">56.8</td></tr><tr class="is_Text _dsp-_platformweb_table-row _col-inherit" data-slot="text"><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-left _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">typed search</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">93.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">89.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">89.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">67.0</td><td class="is_Text _dsp-_platformweb_table-cell _fontVariantNumeric-_platformweb_tabular-num115 _col-inherit _text-right _pt-7px _pb-7px _pr-12px _pl-12px _borderBottomWidth-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _ws-nowrap _bbs-solid" data-slot="text">84.5</td></tr></tbody></table></div><p class="is_Text _col-text-second96872 _dsp-block _maxW-720px _lh-1--2594rem _mt-14px _fs-f-size-2 _borderLeftWidth-2px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _pl-14px _bls-solid" data-slot="text">Typed search pools to 98.3 single-hop and 84.5 multi-hop against the plain resolver’s 95.5 and 56.8. The largest single move is 262k multi-hop, from 33.0 to 67.0.</p><h2 class="is_Text font_heading _col-foreground _dsp-revert-laye114 _ws-inherit _ff-f-family _fs-f-size-7 _fw-497 _lh-1--2 _ls--0--01em _textWrap-balance _mt-56px _mb-8px" data-slot="title">Numbers other systems publish</h2><p class="is_Text _col-text-second96872 _dsp-block _maxW-720px _lh-1--3563rem" data-slot="text">Quoted with their protocols, because the protocols differ, and none of them re-run here. CAR (arXiv 2606.01435) reports 78.0 single-hop / 30.2 multi-hop pooled over 6k–262k with gpt-4o-mini and 94.8 / 51.5 with gpt-4o, and 82 / 27 and 93 / 41 respectively at 262k. Its own baselines at 262k multi-hop: HippoRAG-v2 5, gpt-4o whole-context 5, BM25 3, Cognee and MemGPT 3, Mem0 2, Zep 3. That is the nearest thing to a like-for-like comparison anywhere on this site — same data, same split, same substring-EM metric — and it is still not one, because CAR reads the haystack with a frontier model and the rows above use no model at all.</p><p class="is_Text _col-text-second96872 _dsp-block _maxW-720px _lh-1--3563rem" data-slot="text">Pith publishes per-lane multi-hop figures on a vendor page: 83 / 84 at 32k and 64k, where typed search scores 89.0 / 89.0.</p><div class="is_View _fd-column _mt-48px _pt-16px"><div class="is_View _text-_platformweb_center _fd-column _mt-56px _mb-24px _pt-36px _pb-36px _pr-32px _pl-32px _btlr-12px _btrr-12px _bbrr-12px _bblr-12px _btw-1px _brw-1px _borderBottomWidth-1px _borderLeftWidth-1px _btc-border-stro3513 _brc-border-stro3513 _borderBottomColor-border-stro3513 _borderLeftColor-border-stro3513 _backgroundImage-linear-grad1889052070 _items-center _gap-16px _bbs-solid _bts-solid _bls-solid _brs-solid"><p class="is_Text font_mono _col-muted-foreg108704142 _dsp-block _mb-0px _ff-f-family _fs-f-size-1 _lh-f-lineHeigh112922 _mt-0px _mr-0px _ml-0px _ls-1--32px _tt-uppercase" data-slot="text">Hanzo AI Cloud</p><h3 class="is_Text _fs-_md_f-size-10 _lh-_md_2--5rem _col-text-primar121 _dsp-block _mt-0px _mr-0px _mb-0px _ml-0px _fs-f-size-7 _lh-1--6406rem _fw-600 _ls-f-letterSpa1360334199" data-slot="text">Build on Hanzo Cloud</h3><div class="is_View _fd-row _fwr-wrap _gap-12px _mt-8px _justify-center"><a class="is_Text _col-background _dsp-inline-flex _items-center _gap-8px _pt-12px _pb-12px _pr-24px _pl-24px _btlr-9999px _btrr-9999px _bbrr-9999px _bblr-9999px _btw-1px _brw-1px _borderBottomWidth-1px _borderLeftWidth-1px _btc-transparent _brc-transparent _borderBottomColor-transparent _borderLeftColor-transparent _bg-text-primar121 _fs-f-size-3 _lh-f-lineHeigh112922 _fw-600 _bbs-solid _bts-solid _bls-solid _brs-solid" data-slot="text" href="https://platform.hanzo.ai">Start on Hanzo Cloud<svg aria-hidden="true" data-i18n-mirror="true" fill="none" height="14" stroke="var(--background)" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" style="color:var(--background);flex-shrink:0" viewbox="0 0 24 24" width="14"><path d="M5 12h14" stroke="var(--background)"></path><path d="m12 5 7 7-7 7" stroke="var(--background)"></path></svg></a><a class="is_Text _col-text-primar121 _dsp-inline-flex _items-center _gap-8px _pt-12px _pb-12px _pr-24px _pl-24px _btlr-9999px _btrr-9999px _bbrr-9999px _bblr-9999px _btw-1px _brw-1px _borderBottomWidth-1px _borderLeftWidth-1px _btc-border-stro3513 _brc-border-stro3513 _borderBottomColor-border-stro3513 _borderLeftColor-border-stro3513 _bg-transparent _fs-f-size-3 _lh-f-lineHeigh112922 _fw-500 _bbs-solid _bts-solid _bls-solid _brs-solid" data-slot="text" href="/pricing">View Models & Pricing</a></div></div><div class="is_View _fd-column _btw-1px _btc-edge _brc-edge _borderBottomColor-edge _borderLeftColor-edge _pt-32px _bts-solid"><div class="is_View _fd-row _fwr-wrap _justify-space-betwe3241 _items-center _gap-16px _mb-32px _pt-16px _pb-16px _pr-20px _pl-20px _bg-card _btw-1px _brw-1px _borderBottomWidth-1px _borderLeftWidth-1px _btc-border _brc-border _borderBottomColor-border _borderLeftColor-border _btlr-8px _btrr-8px _bbrr-8px _bblr-8px _bbs-solid _bts-solid _bls-solid _brs-solid"><a class="is_View _fd-column _gap-2px _items-flex-start" href="/benchmarks/locomo"><div class="is_View _fd-row _items-center _gap-4px"><svg aria-hidden="true" data-i18n-mirror="true" fill="none" height="12" stroke="var(--muted-foreground)" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" style="color:var(--muted-foreground);flex-shrink:0" viewbox="0 0 24 24" width="12"><path d="m12 19-7-7 7-7" stroke="var(--muted-foreground)"></path><path d="M19 12H5" stroke="var(--muted-foreground)"></path></svg><span class="is_Text _col-muted-foreg108704142 _fs-f-size-1 _lh-f-lineHeigh112922 _tt-uppercase _ls-0--44px" data-slot="text">Previous</span></div><span class="is_Text _col-text-primar121 _fs-f-size-3 _lh-f-lineHeigh112922 _fw-500" data-slot="text">LoCoMo</span></a><a class="is_Text _col-muted-foreg108704142 _pt-6px _pb-6px _pr-12px _pl-12px _btlr-4px _btrr-4px _bbrr-4px _bblr-4px _bg-edge _btw-1px _brw-1px _borderBottomWidth-1px _borderLeftWidth-1px _btc-border _brc-border _borderBottomColor-border _borderLeftColor-border _fs-f-size-2 _lh-f-lineHeigh112922 _bbs-solid _bts-solid _bls-solid _brs-solid" data-slot="text" href="/benchmarks">All Benchmarks Board</a><a class="is_View _fd-column _gap-2px _items-flex-end" href="/benchmarks/longmemeval"><div class="is_View _fd-row _items-center _gap-4px"><span class="is_Text _col-muted-foreg108704142 _fs-f-size-1 _lh-f-lineHeigh112922 _tt-uppercase _ls-0--44px" data-slot="text">Next</span><svg aria-hidden="true" data-i18n-mirror="true" fill="none" height="12" stroke="var(--muted-foreground)" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" style="color:var(--muted-foreground);flex-shrink:0" viewbox="0 0 24 24" width="12"><path d="M5 12h14" stroke="var(--muted-foreground)"></path><path d="m12 5 7 7-7 7" stroke="var(--muted-foreground)"></path></svg></div><span class="is_Text _col-text-primar121 _fs-f-size-3 _lh-f-lineHeigh112922 _fw-500" data-slot="text">LongMemEval</span></a></div><div class="is_View _fd-column _mb-24px"><h3 class="is_Text font_mono _col-text-primar121 _dsp-block _mt-0px _mb-12px _fs-f-size-3 _lh-f-lineHeigh112922 _ff-f-family _fw-500" data-slot="text">Related in memory</h3><div class="is_View _fd-row _fwr-wrap _gap-10px"><a class="is_Text _col-text-primar121 _dsp-inline-flex _items-center _gap-6px _pt-6px _pb-6px _pr-12px _pl-12px _btlr-6px _btrr-6px _bbrr-6px _bblr-6px _bg-edge _btw-1px _brw-1px _borderBottomWidth-1px _borderLeftWidth-1px _btc-border _brc-border _borderBottomColor-border _borderLeftColor-border _fs-f-size-2 _lh-f-lineHeigh112922 _bbs-solid _bts-solid _bls-solid _brs-solid" data-slot="text" href="/benchmarks/locomo"><span class="is_Text _col-inherit" data-slot="text">LoCoMo</span><svg aria-hidden="true" data-i18n-mirror="true" fill="none" height="12" stroke="var(--muted-foreground)" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" style="color:var(--muted-foreground);flex-shrink:0" viewbox="0 0 24 24" width="12"><path d="M5 12h14" stroke="var(--muted-foreground)"></path><path d="m12 5 7 7-7 7" stroke="var(--muted-foreground)"></path></svg></a><a class="is_Text _col-text-primar121 _dsp-inline-flex _items-center _gap-6px _pt-6px _pb-6px _pr-12px _pl-12px _btlr-6px _btrr-6px _bbrr-6px _bblr-6px _bg-edge _btw-1px _brw-1px _borderBottomWidth-1px _borderLeftWidth-1px _btc-border _brc-border _borderBottomColor-border _borderLeftColor-border _fs-f-size-2 _lh-f-lineHeigh112922 _bbs-solid _bts-solid _bls-solid _brs-solid" data-slot="text" href="/benchmarks/longmemeval"><span class="is_Text _col-inherit" data-slot="text">LongMemEval</span><svg aria-hidden="true" data-i18n-mirror="true" fill="none" height="12" stroke="var(--muted-foreground)" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" style="color:var(--muted-foreground);flex-shrink:0" viewbox="0 0 24 24" width="12"><path d="M5 12h14" stroke="var(--muted-foreground)"></path><path d="m12 5 7 7-7 7" stroke="var(--muted-foreground)"></path></svg></a><a class="is_Text _col-text-primar121 _dsp-inline-flex _items-center _gap-6px _pt-6px _pb-6px _pr-12px _pl-12px _btlr-6px _btrr-6px _bbrr-6px _bblr-6px _bg-edge _btw-1px _brw-1px _borderBottomWidth-1px _borderLeftWidth-1px _btc-border _brc-border _borderBottomColor-border _borderLeftColor-border _fs-f-size-2 _lh-f-lineHeigh112922 _bbs-solid _bts-solid _bls-solid _brs-solid" data-slot="text" href="/benchmarks/locomo-subject-scope"><span class="is_Text _col-inherit" data-slot="text">LoCoMo · subject scope</span><svg aria-hidden="true" data-i18n-mirror="true" fill="none" height="12" stroke="var(--muted-foreground)" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" style="color:var(--muted-foreground);flex-shrink:0" viewbox="0 0 24 24" width="12"><path d="M5 12h14" stroke="var(--muted-foreground)"></path><path d="m12 5 7 7-7 7" stroke="var(--muted-foreground)"></path></svg></a><a class="is_Text _col-text-primar121 _dsp-inline-flex _items-center _gap-6px _pt-6px _pb-6px _pr-12px _pl-12px _btlr-6px _btrr-6px _bbrr-6px _bblr-6px _bg-edge _btw-1px _brw-1px _borderBottomWidth-1px _borderLeftWidth-1px _btc-border _brc-border _borderBottomColor-border _borderLeftColor-border _fs-f-size-2 _lh-f-lineHeigh112922 _bbs-solid _bts-solid _bls-solid _brs-solid" data-slot="text" href="/benchmarks/locomo-conv"><span class="is_Text _col-inherit" data-slot="text">LoCoMo-Conv</span><svg aria-hidden="true" data-i18n-mirror="true" fill="none" height="12" stroke="var(--muted-foreground)" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" style="color:var(--muted-foreground);flex-shrink:0" viewbox="0 0 24 24" width="12"><path d="M5 12h14" stroke="var(--muted-foreground)"></path><path d="m12 5 7 7-7 7" stroke="var(--muted-foreground)"></path></svg></a></div></div><div class="is_View _fd-column"><span class="is_Text font_mono _col-muted-foreg108704142 _dsp-block _mb-8px _ff-f-family _fs-f-size-2 _lh-f-lineHeigh112922 _tt-uppercase _ls-0--52px" data-slot="text">Other benchmarks</span><p class="is_Text _col-text-second96872 _dsp-block _maxW-720px _lh-1--3563rem _mt-0px _mr-0px _mb-0px _ml-0px _fs-f-size-2" data-slot="text"><span class="is_Text _col-inherit" data-slot="text"><a class="is_Text _col-text-second96872" data-slot="text" href="/benchmarks/cronquestions">CronQuestions</a></span><span class="is_Text _col-inherit" data-slot="text"> · <a class="is_Text _col-text-second96872" data-slot="text" href="/benchmarks/repobench-r">RepoBench-R</a></span><span class="is_Text _col-inherit" data-slot="text"> · <a class="is_Text _col-text-second96872" data-slot="text" href="/benchmarks/fleet">Fleet residency</a></span><span class="is_Text _col-inherit" data-slot="text"> · <a class="is_Text _col-text-second96872" data-slot="text" href="/benchmarks/goroutine">Live agent footprint</a></span><span class="is_Text _col-inherit" data-slot="text"> · <a class="is_Text _col-text-second96872" data-slot="text" href="/benchmarks/sandbox">Sandbox cold start</a></span><span class="is_Text _col-inherit" data-slot="text"> · <a class="is_Text _col-text-second96872" data-slot="text" href="/benchmarks/inference">Engine prefill on CUDA vs llama.cpp</a></span><span class="is_Text _col-inherit" data-slot="text"> · <a class="is_Text _col-text-second96872" data-slot="text" href="/benchmarks/gpqa">GPQA-Diamond</a></span></p></div></div></div><p class="is_Text font_mono _col-muted-foreg108704142 _dsp-block _mb-10px _ff-f-family _fs-f-size-1 _lh-f-lineHeigh112922" data-slot="text">raw runs, plans, traces and scoring: <!-- -->hanzoai/benchmarks · brain<!-- --> · every table above regenerated by <code>node results.mjs</code></p></main><!--$--><!--/$--></div> |