mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-29 07:34:31 +00:00
zooqueenandHanzo Dev
c9d9ad5d3f
The MemoryAgentBench section now reports the exhaustive typed search over the lexicon's and Qwen3.8's plan sets (test multi-hop 89 / 89 / 67, single-hop 98 / 98 / 97, pooled 98.3 / 84.5, no reader), a table of plan recall@N against execution success@N, and the ceiling analysis: 33 questions at 262K need an older version of a key a later, unasked-for edit overwrites, six store-readable signals separate nothing, latest-wins caps at 67 and the search is on that line. Pith is compared lane by lane with no best-published claim. Co-authored-by: Hanzo Dev <[email protected]>
11 lines
555 B
TeX
11 lines
555 B
TeX
% generated by hanzoai/cloud bench/brain/tables.mjs — FC-MH, plan recall@N / execution success@N out of 100, no reader; sets in lane order: the lexicon set, then the model sets alphabetically, each also in every relation order; 6k is dev
|
|
\begin{tabular}{lrrrrr}
|
|
\toprule
|
|
Haystack & sets & plans / question & N=1 & N=3 & N=9 \\
|
|
\midrule
|
|
6k & 9 & 13.8 & 88 / 88 & 90 / 90 & 94 / 93 \\
|
|
32k & 9 & 14.6 & 79 / 79 & 84 / 81 & 89 / 89 \\
|
|
64k & 9 & 16.6 & 83 / 83 & 86 / 85 & 89 / 89 \\
|
|
262k & 9 & 15.3 & 53 / 53 & 62 / 59 & 68 / 67 \\
|
|
\bottomrule
|
|
\end{tabular} |