Enso: three of four best-arm cells come from the replay, and one does not

The router repository's stored replay names a strongest single arm per
benchmark. We print fable-5 at 38.2 on HLE and gpt-5.5 at 85.5 on CharXiv,
both of which agree with it to the digit, and we leave LiveCodeBench blank.
On GPQA we print gpt-5.6-sol at 91.9 where the replay names grok-4.5 at 92.9,
over a wider pool than this gateway serves. Three cells sharing a source and
the fourth not is worth saying out loud next to the row whose margin the
fourth cell decides.

Co-authored-by: Hanzo Dev <[email protected]>
This commit is contained in:
2026-09-10 19:57:34 -07:00
co-authored by Hanzo Dev
parent 7513388e56
commit 39770926be
2 changed files with 6 additions and 3 deletions
BIN
View File
Binary file not shown.
+6 -3
View File
@@ -704,9 +704,12 @@ and replays the router against it with the fingerprint index refit five-fold so
router is scored on questions absent from its own route table. Over the same item
counts that replay reports 92.9 (GPQA, $n{=}198$), 91.4 (LiveCodeBench, $n{=}175$),
38.2 (HLE, $n{=}500$) and 85.5 (CharXiv, $n{=}200$)---and on all four the strongest
single arm scores exactly the same as the router. It also names \texttt{grok-4.5},
not \texttt{\hBestGPQAname}, as the strongest arm on GPQA, over a wider eleven-arm
pool than the one this paper serves. A stored-table replay is a different
single arm scores exactly the same as the router. Three of the four best-arm cells we print agree with
that replay to the digit---\texttt{fable-5} at 38.2 on HLE, \texttt{gpt-5.5} at
85.5 on CharXiv, and the LiveCodeBench cell we leave blank. The fourth does not:
the replay names \texttt{grok-4.5} at 92.9 as the strongest arm on GPQA, over a
wider eleven-arm pool than the one this paper serves, where we print
\texttt{\hBestGPQAname} at \hBestGPQAacc. A stored-table replay is a different
measurement from the live-gateway run reported here and we do not merge the two.
It does support reading the GPQA row as a match rather than a lead, which is how
\S\ref{sec:eval} now reads it.