zeekay 9411e59870 decision: the trolley benchmark as rows 17 and 18 of the suite, with its reader, scorer and Kai's reads ...

Zach Landes's trolley-bench (github.com/zachlandes/trolley-bench@cde97d7,
5,240 setups) read on kai-1.3, kai-1.4 and kai15a once each on the CPU, Jev
and GPT-6 Luna from the repo's recorded answers; the scorer reproduces the
article's statistics. Kai fails both checks with a right answer (650 of 672
dominated choices against Jev's 69; 2,036 of 5,240 contradictions against
942), which the suite now says.

2026-10-16 10:41:22 -07:00

..

kai15a.jsonl.gz

decision: the trolley benchmark as rows 17 and 18 of the suite, with its reader, scorer and Kai's reads

2026-10-16 10:41:22 -07:00

kai-1.3.jsonl.gz

decision: the trolley benchmark as rows 17 and 18 of the suite, with its reader, scorer and Kai's reads

2026-10-16 10:41:22 -07:00

kai-1.4.jsonl.gz

decision: the trolley benchmark as rows 17 and 18 of the suite, with its reader, scorer and Kai's reads

2026-10-16 10:41:22 -07:00

report.md

decision: the trolley benchmark as rows 17 and 18 of the suite, with its reader, scorer and Kai's reads

2026-10-16 10:41:22 -07:00