mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-22 17:20:17 +00:00
zooqueen
67816f32ac
The engine's ablation on all ten conversations, the test split run once after the dev freeze, the same rows over all-MiniLM-L6-v2 at k=10, and the frozen configuration by k are generated by hanzoai/cloud bench/brain/tables.mjs. LoCoMo-Conv is unreleased, so that section now says so and reports the protocol check on the original questions instead of a number nobody can reproduce.
16 lines
712 B
TeX
16 lines
712 B
TeX
% generated by hanzoai/cloud bench/brain/tables.mjs — LoCoMo-Conv data is unreleased (github.com/MiuLab/LoCoMo-Conv answers 404); this is the paper's protocol on the original LoCoMo questions the rewrites came from
|
|
\begin{tabular}{lrrr}
|
|
\toprule
|
|
Memory unit, all-MiniLM-L6-v2, k=10 & R@10 all (1,977) & no adversarial (1,531) & tokens/q \\
|
|
\midrule
|
|
single turn & 42.8 & 45.4 & 279 \\
|
|
2-turn exchange & 56.3 & 54.3 & 571 \\
|
|
3-turn window & 62.5 & 60.7 & 829 \\
|
|
5-turn window & 60.3 & 58.2 & 1258 \\
|
|
\midrule
|
|
single turn, zen-embedding-0.6b & 59.7 & 63.5 & 315 \\
|
|
\midrule
|
|
\multicolumn{4}{l}{paper's Naive RAG row: dialog 0.533, implicit 0.312, counterfactual 0.573, composed 0.266} \\
|
|
\bottomrule
|
|
\end{tabular}
|