mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-30 14:58:42 +00:00
zooqueen
67816f32ac
The engine's ablation on all ten conversations, the test split run once after the dev freeze, the same rows over all-MiniLM-L6-v2 at k=10, and the frozen configuration by k are generated by hanzoai/cloud bench/brain/tables.mjs. LoCoMo-Conv is unreleased, so that section now says so and reports the protocol check on the original questions instead of a number nobody can reproduce.
17 lines
952 B
TeX
17 lines
952 B
TeX
% generated by hanzoai/cloud bench/brain/tables.mjs — all-MiniLM-L6-v2 vectors, k=10, LoCoMo-Conv protocol embedder
|
|
\begin{tabular}{lrrrrrrrrrr}
|
|
\toprule
|
|
& \multicolumn{2}{c}{multi-hop} & single & temporal & open & all & & & pool & p50 \\
|
|
\cmidrule(lr){2-3}
|
|
Configuration & ALL@10 & ANY@10 & ALL@10 & ALL@10 & ALL@10 & R@10 & nDCG & supported & cands & ms \\
|
|
\midrule
|
|
cosine over turns & 9.6 & 49.6 & 51.4 & 46.1 & 16.3 & 45.3 & 31.1 & 53.1 & 40 & 0.25 \\
|
|
+ BM25 & 13.8 & 58.9 & 67.2 & 62.0 & 23.9 & 59.2 & 43.7 & 67.6 & 54 & 0.31 \\
|
|
+ atomic fact index & 20.9 & 73.4 & 73.1 & 72.0 & 25.0 & 66.7 & 52.0 & 75.6 & 73 & 0.42 \\
|
|
+ adjacency & 21.6 & 74.5 & 74.6 & 72.6 & 27.2 & 67.7 & 53.3 & 76.3 & 77 & 0.42 \\
|
|
+ second hop from resolved state & 24.1 & 78.0 & 77.5 & 73.5 & 27.2 & 70.1 & 55.8 & 78.5 & 78 & 0.76 \\
|
|
\midrule
|
|
global reciprocal-rank fusion & 20.2 & 74.8 & 73.4 & 74.1 & 28.3 & 67.2 & 48.6 & 76.1 & 143 & 0.43 \\
|
|
\bottomrule
|
|
\end{tabular}
|