mirror of
https://github.com/hanzoai/site.git
synced 2026-11-21 00:14:36 +00:00
1 line
2.3 KiB
JSON
1 line
2.3 KiB
JSON
{"text":["MASSIVE","Benchmarks","Kai","Decision harness","Kai and Jev on the same 5,100 questions of MASSIVE, one run per Kai revision.","Latest","90.7%","Accuracy · kai-1.4 · c1856a77","Jev 89.0% · Win · Oct 16, 2026 · Hanzo-measured","0.064","Calibration error (ECE) · kai-1.4 · c1856a77","Jev 0.064 · Win · Oct 16, 2026 · Hanzo-measured","History","9 results across 2 metrics, newest first. Superseded runs stay, so a revision's progress can be read.","88.7%","90.0%","91.3%","09-28","10-03","10-16","Kai a8 · 0cfeef05: 90.6%, Sep 28, 2026","Kai a7 · 0834a74f: 90.9%, Sep 28, 2026","Kai a6 · a211cc70: 91.0%, Sep 28, 2026","kai-1 · 0834a74f: 91.0%, Oct 3, 2026","kai-1.4 · c1856a77: 90.7%, Oct 16, 2026","Measured","Subject","Metric","Value","Baseline","Result","Status","Oct 16, 2026","kai-1.4 · c1856a77","Accuracy","Jev 89.0%","Win","Live","Hanzo-measured","Calibration error (ECE)","Jev 0.064","Oct 3, 2026","kai-1 · 0834a74f","91.0%","Superseded","0.060","Sep 28, 2026","Kai a8 · 0cfeef05","90.6%","Kai a7 · 0834a74f","90.9%","Kai a6 · a211cc70","0.059","Conditions","split: harness","5,100 questions, 5,100 answered","dataset dde140a63fc5","6 Kai revisions measured","Jev: typesafe/jev-1.13-20260917, measured Sep 25, 2026","Hardware","CPU AS SERVED","Source","research run massive-harness-kai-1.4.c1856a77-kai","Reproduce","bench score --harness . --kai results/kai/preds.json.gz --out results/scores.json # in hanzoai/benchmarks/decision @ 5f616dd","More Kai benchmarks","AG News","DAIR Emotion","Banking77","Support triage","Email spam","Phishing","Toxicity","RAG relevance","Model routing","Typed decisions","Choosing among many options","Latency of one decision","Every result for MASSIVE on the shelf","How these are measured"],"links":["/benchmarks","/benchmarks?product=Kai#all","#history","https://api.hanzo.ai/v1/research/runs?org=hanzo&id=massive-harness-kai-1.4.c1856a77-kai","/benchmarks/kai-ag-news","/benchmarks/kai-dair-emotion","/benchmarks/kai-banking77","/benchmarks/kai-support-triage","/benchmarks/kai-email-spam","/benchmarks/kai-phishing","/benchmarks/kai-toxicity","/benchmarks/kai-rag-relevance","/benchmarks/kai-model-routing","/benchmarks/kai-typed-decisions","/benchmarks/kai-choice-size","/benchmarks/kai-latency","/benchmarks?q=MASSIVE#all","/benchmarks#methods"]} |