mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-22 17:20:17 +00:00
zooqueen
67816f32ac
The engine's ablation on all ten conversations, the test split run once after the dev freeze, the same rows over all-MiniLM-L6-v2 at k=10, and the frozen configuration by k are generated by hanzoai/cloud bench/brain/tables.mjs. LoCoMo-Conv is unreleased, so that section now says so and reports the protocol check on the original questions instead of a number nobody can reproduce.
23 lines
1.0 KiB
TeX
23 lines
1.0 KiB
TeX
% generated by hanzoai/cloud bench/brain/tables.mjs — dev split only (conversations 0–2); coordinate descent
|
||
\begin{tabular}{lrrr}
|
||
\toprule
|
||
Parameter & range searched & start & chosen \\
|
||
\midrule
|
||
fact weight & 0.10–0.40 & 0.2 & \textbf{0.3} \\
|
||
BM25 weight & 0–0.30 & 0.15 & \textbf{0.2} \\
|
||
entity weight & 0–0.20 & 0.1 & 0.1 \\
|
||
temporal weight & 0–0.12 & 0.05 & \textbf{0.12} \\
|
||
adjacency weight & 0–0.10 & 0.06 & \textbf{0.1} \\
|
||
expansion weight & 0–0.15 & 0.08 & 0.08 \\
|
||
second-hop weight & 0–0.45 & 0.3 & 0.3 \\
|
||
facet weight & 0–0.30 & 0.2 & 0.2 \\
|
||
dense pool & 20–60 & 40 & 40 \\
|
||
fact pool & 10–30 & 20 & \textbf{30} \\
|
||
BM25 pool & 10–30 & 20 & 20 \\
|
||
second-hop pool & 5–15 & 10 & \textbf{15} \\
|
||
\midrule
|
||
\multicolumn{4}{l}{dev objective (mean over categories of (ALL@20 + nDCG@20)/2): 61.3 $\rightarrow$ 63.7 over 145 evaluations} \\
|
||
\multicolumn{4}{l}{frozen at commit \texttt{33d0f8749bd1}; test split, run once: multi-hop ALL@20 37.5, ANY@20 88.0, single-hop ALL@20 87.7} \\
|
||
\bottomrule
|
||
\end{tabular}
|