zeekayandHanzo Dev 62c1ad4c32 decision: Kai, Laya and Jev side by side, with calibration and risk-coverage. ...

harness/states.py writes results/states.json.gz, every case with its state, and checks
it reproduces questions.json.gz. results/laya/preds.json.gz is three_way.py kai on all
62 suites. merge.py reads the committed results and adds AURC and risk at coverage,
Kai when results/kai/preds.json.gz exists, and MASSIVE by language; the upstream metric
code is unchanged. results/scores.json and results/table.md are its output.
test_merge.py checks the metrics on hand-computed cases.

Co-authored-by: Hanzo Dev <[email protected]>

2026-09-25 10:33:56 -07:00

1.1 MiB

Raw Permalink History

View Raw