mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-29 07:34:31 +00:00
zooqueen
67816f32ac
The engine's ablation on all ten conversations, the test split run once after the dev freeze, the same rows over all-MiniLM-L6-v2 at k=10, and the frozen configuration by k are generated by hanzoai/cloud bench/brain/tables.mjs. LoCoMo-Conv is unreleased, so that section now says so and reports the protocol check on the original questions instead of a number nobody can reproduce.
17 lines
949 B
TeX
17 lines
949 B
TeX
% generated by hanzoai/cloud bench/brain/tables.mjs — all ten conversations, k=20, LoCoMo facts, frozen weights
|
|
\begin{tabular}{lrrrrrrrrrr}
|
|
\toprule
|
|
& \multicolumn{2}{c}{multi-hop} & single & temporal & open & all & & & pool & p50 \\
|
|
\cmidrule(lr){2-3}
|
|
Configuration & ALL@20 & ANY@20 & ALL@20 & ALL@20 & ALL@20 & R@20 & nDCG & supported & cands & ms \\
|
|
\midrule
|
|
cosine over turns & 22.7 & 79.8 & 78.2 & 77.3 & 33.7 & 71.9 & 49.6 & 81.0 & 40 & 0.55 \\
|
|
+ BM25 & 22.7 & 78.0 & 82.5 & 78.8 & 37.0 & 74.6 & 53.2 & 83.3 & 52 & 0.61 \\
|
|
+ atomic fact index & 36.9 & 88.3 & 82.3 & 82.6 & 42.4 & 78.2 & 59.6 & 86.4 & 71 & 0.83 \\
|
|
+ adjacency & 39.0 & 88.7 & 84.2 & 83.5 & 44.6 & 79.7 & 61.4 & 87.8 & 74 & 0.83 \\
|
|
+ second hop from resolved state & 39.0 & 87.9 & 86.8 & 84.4 & 45.7 & 80.9 & 62.6 & 88.0 & 76 & 1.65 \\
|
|
\midrule
|
|
global reciprocal-rank fusion & 33.7 & 89.0 & 86.8 & 83.8 & 40.2 & 80.5 & 56.2 & 89.0 & 137 & 1.73 \\
|
|
\bottomrule
|
|
\end{tabular}
|