zeekay 5a0f823482 Say what the repair row cost and what it bought ...

mab-fc.tex carries a row — typed search with one repair call — whose test cells
are ellipses and which the paper never mentioned. A reader meets an unexplained,
mostly empty row and has no way to tell a missing run from a rejected one.

It is a rejected one, and the traces say so precisely: across 196 dev questions
the repair fired ONCE, that single plan was chosen, and it is the entire
difference between 88 and 89 multi-hop. One question, for a 27B model held
resident beside the search. The row beneath it — exhaustive enumeration with a
lexical tie-break — buys five points with no model in the loop at all.

So it was not promoted to test, and the empty cells are the method working
rather than a gap in it: a configuration that loses on dev does not get a second
chance on held-out data. Reporting the cells empty and saying why is cheaper
than dropping the row and leaving no trace that the idea was tried.

The paper already reports its losing LoCoMo variants this way. This one had the
numbers and not the sentence.

2026-09-15 15:45:43 -07:00

532 KiB

Raw History

View Raw