24 KiB
Claims table: Kai, typed decisions under deterministic policy
Every number in kai.tex and sections/ maps to a committed file of the
implementation repository at commit f814147 (2026-10-21). Paths below are
relative to that repository's root. LLM.md §X names the section of the
project's running record where a number is stated in prose and no other
committed file states it. A "derived" entry says how the number was computed
from the named file. Lab hosts are named as in the paper: the arm64 host (GB10)
and the x86-64 host (Strix Halo); some source file and field names below carry
the hosts' internal names, which are written here as <arm64> and <x86-64>.
Section references, equation numbers, page geometry and the version numbers of releases are not claims and are not listed.
Abstract and Introduction (kai.tex, 01-introduction.tex)
| Claim | Source | Field / line |
|---|---|---|
kai-1.8 serves /v1/decisions as kai on CPUs since 2026-10-20 |
results/ground/run12r/live.md; release.json |
live.md first line (2026-10-20 18:56Z); deployed |
| blind suite of 14,832 scenes, floor 0.9 | results/ground/run12r/README.md |
"Ground-8, the blind read" |
| 99.94% of grounded scenes right; 5 wrong courses (0.03%) | results/ground/run12r/README.md; gate.txt |
"accuracy of grounded 0.9994, wrong courses 5 (0.03% of rows)"; gate line blind accuracy of grounded |
| 15,280 scenes with no exact count, 2 grounded | results/ground/run12r/README.md |
derived: 5,680 gold-incomplete blind scenes (false-grounded 2 of 5,680) + 9,600 negatives (0 grounded) |
| Q8_0 flipped 3 of 400 policy outcomes between arm64 and x86-64 | results/parity/kai-1.8-f32/summary.json |
pairs["<arm64>-q8 vs <x86-64>-q8"].outcome_flips = 3 |
| model's own count right on 142 of 3,814 courses | results/ground/g14-invalid/audit.json |
model argmax equals gold = 142; grounded people courses = 3814 |
| beats neither Jev nor Clef on any of 32 Decision Index rows | results/decision-index/kai-1/gap.json; kai-1-reach32768/gap.json |
beats jev = 0, beats clef = 0, of = 32 |
| trails Laya on sealed transfer suites | results/transfer/kai-1.2/against-laya.json |
groups.pooled.accuracy.verdict = worse |
| trails Jev on Jev's own benchmarks | LLM.md §Research |
finding 1: B1 0.48 against Jev 0.98 |
| catches fewer jailbreaks than Jev at fewer false alarms | bench/jailbreak-suite/results/report.txt |
pool real+optimized: caught 26.9% / 77.4%, f.alarm 3.6% / 13.0% |
2 Typed decisions (02-decisions.tex)
| Claim | Source | Field / line |
|---|---|---|
| choice ≥ 2 labels, score ≥ 1 level, boolean 2 sides | LLM.md §Runtime |
Schema bullet |
| 1 to 100 questions a request | LLM.md §Runtime |
Schema bullet |
| κ = (k·p_max − 1)/(k − 1); 0.75 (two-way) and 0.55 (ten-way) at κ ≥ 0.5 | LLM.md §Runtime |
Numerics bullet; derived from the formula |
| complement logit (z_a − z_b)/2, P(b) = 1 − P(a) | LLM.md §Runtime |
Complements bullet |
3 The model (03-model.tex, figures/model.tex)
| Claim | Source | Field / line |
|---|---|---|
| mmBERT-base, 22 layers of width 768, 8,192 trained positions | LLM.md §Models |
Encoder bullet; §Runtime Reach ("mmBERT: 8,192") |
| 256,000-token vocabulary | LLM.md §kai-1 and capabilities |
Memory table, "token table, 256,000 x 768" |
| base trained at 1,024 tokens | LLM.md §kai-1 and capabilities |
Reach bullet ("kai-1 was trained to read 1,024 tokens") |
| head scores at most k = 160 options | LLM.md §Models |
Retrieval bullet ("160") |
| cumulative-link head lost about thirty points on score questions | LLM.md §Stages and experiments |
row t1, t2: score 0.431 against Laya's 0.723 |
| yes/no reader k = 12, three outputs | results/bernoulli/reader12/Z2/report.json |
k = 12, outputs = 3 |
| kind and count readers k = 8 | results/ground/run12r/README.md |
"Fit: reader.py over kai-1.3's encoder, k 8" |
| count labels "0" to "100" and "more" | results/kai-1.7/README.md; LLM.md §Runtime |
"What it is"; Policy bullet |
| Banking77 0.02 to 0.21 under an update applied to every question | LLM.md §kai-1 and capabilities |
Satori bullet |
| HelpSteer2 head: 50 tensors, 15,170,049 values; resident 2,761 → 1,564 MiB; 468 requests; 11,099 answers same bits | LLM.md §kai-1 and capabilities |
One encoder bullet |
| kai-1.4 retrains the base on states to 32,000 tokens | LLM.md §kai-1.4 |
first paragraph and Data bullet ("states of up to 32,000 tokens") |
| stage a7: one epoch, 30,928 batches, three machines | benchmarks/decision/results/kai-a7/README.md |
"stage" row |
| barrier holds 109,818 items | LLM.md §Training |
Barrier paragraph |
| Jaccard ≥ 0.5, 80% of 50-character spans | LLM.md §Training |
Barrier paragraph |
| B1: Laya 0.770, laya-multilingual 0.690, a4 0.470, a7 0.480 | LLM.md §Research |
finding 2 |
| unit 924 of layer 11; [CLS] norm 19,330; about 100 at a4 | LLM.md §Research |
finding 3 |
| grafting layers 11 to 21 recovers B1 to 0.560 | LLM.md §Research |
finding 3 |
4 Policy (04-policy.tex)
| Claim | Source | Field / line |
|---|---|---|
| kai-1.2 held 25.6% of attacked routing decisions, Jev 51.5% | bench/hijack/report.txt |
"routing alone (post hoc) ... Kai held 25.6 Jev held 51.5" |
| default floor 0.9; caller may name 0.5 to 1 | LLM.md §Runtime; decision-core/src/policy.rs |
Policy bullet; pub const FLOOR: f64 = 0.9 |
| fourteen end-to-end tests; eight mutations each fail at least one | bench/nio/results/qualify.json |
baseline.tests.passed = 14; mutations (8 keys), each tests.failed ≥ 1 |
5 Runtime (05-runtime.tex)
| Claim | Source | Field / line |
|---|---|---|
| queue of 256; deadline 10 s | LLM.md §Runtime |
Batching bullet |
| live flip: production kind 0.913, x86-64 host 0.897 | results/ground/run12r/live.md |
second paragraph |
| Table 2, the four pairs (agreement, max | Δp | , flips) |
| Table 2, latency 574/630, 964/1,210 ms; resident 2,590/3,299, peak 2,999/3,320 MiB | results/parity/kai-1.8-f32/README.md |
cost table (x86-64 rows) |
| canary 0.86928 / 0.14429 (F32); 0.86187 / 0.12720 (x86-64 Q8_0); 0.87346 / 0.13493 (arm64 Q8_0) | results/parity/kai-1.8-f32/rereads.md |
canary table |
| 951 retained rows: 0 flips in F32, 7 in Q8_0, largest change 0.595 | results/parity/kai-1.8-f32/rereads.md |
retained table |
| decision recorded 2026-10-21; 1.68× median latency; 710 MiB | LLM.md (Deployed paragraph); results/parity/kai-1.8-f32/README.md |
"1.68x the latency at p50"; "710 MiB more resident" |
| production on release 0.4.31 (Q8_0) at the time of writing | release.json |
deployed.image = v0.4.31; F32 change b7f6c0f is not an ancestor of 0742d63 |
| CI tests now hold unrouted answers to 1e-5 | kai/tests/witness.rs |
fn same (commit 8ac8a5d) |
| two production replicas on different x86-64 CPUs answered the canary in different third decimals | LLM.md §kai-1 and capabilities |
v0.4.4 in production bullet (.8683 / .1263 against .8619 / .1272) |
| five-question grounding request | results/parity/kai-1.8-f32/README.md |
"server latency a request of five questions" |
| YaRN stretch 4; reach to 32,768 | LLM.md §Runtime |
Reach bullet |
| 11,085 harness questions within 1,024 tokens bit for bit at reach 8,192 | LLM.md §kai-1 and capabilities |
Reach bullet |
| 100 questions; four sequences of the reach; head 153,600; encoder 500,000 | LLM.md §Runtime |
Reach bullet |
| 0.47 ms an option token on four cores; about four minutes | LLM.md §Runtime |
Reach bullet |
| Table 3: 1,000 → 0.4 s, 4,000 → 1.2 s, 8,000 → 4.3 s, 13,400 → 13.7 s, 20,100 → 28 s, 31,800 → 65 s; 24 threads; 4.8 GiB | LLM.md §Runtime |
Reach bullet ("One question through kai serve on the CPU ...") |
| gateway allows 120 s | LLM.md §Runtime |
Reach bullet |
| planted facts: 36–41% at 1,000 tokens, 25–29% at 8,000, 100 decisions a cell, chance 25% | LLM.md §kai-1 and capabilities |
Reach bullet |
| tile: next multiple of an eighth of the octave, at least 8; 1,024 option tokens at a time; rows among 3, 9 and 140 others | LLM.md §Inference speed |
tensor bullet |
| 1,233 isolation requests moved none; each wire refusal 422 | results/ground/run12r/iso.json |
isolation.requests = 1233, moved = 0; fail_closed.* status 422 |
| seven configurations, 2,270 requests each, no answer outside the route moved | results/guard3/iso.json |
urls (7), requests = 2270, each comparison moved = 0 |
| teaching gate: 10%, four repairs per damage, 100 µs, 256 KiB | LLM.md §Per-org capabilities |
Teach bullet |
| fixture: 78 corrections, 2.4 s, 61 repairs, 8 of 157 broken, 0 of 235 moved; 48 rows, 33 against 45 | LLM.md §Per-org capabilities |
The fixture bullet |
| yes/no question 50.7 ms median, 57.2 ms p95 | results/bernoulli/reader12/Z2/served.json |
metrics.latency.p50_ms, latency.p95_ms |
| retained rows 105 ms (Q8_0) and 180 ms (F32) median on x86-64 | results/parity/kai-1.8-f32/rereads.md |
latency sentence |
| pods 1,826 and 2,891 MiB to about 922 MiB | LLM.md §kai-1 and capabilities |
v0.4.5 in production bullet ("922 and 923 MiB") |
6 Typed grounding (06-grounding.tex)
| Claim | Source | Field / line |
|---|---|---|
| 47 wrong courses, one defect (span inside a two-digit number) | LLM.md §kai-1.4 |
ground-2 bullet; results/ground/run4/errors.md |
| ground-3: 97 wrong, all hedged with the span after the hedge | LLM.md §kai-1.4 |
ground-3 bullet |
| ground-4: 286 of 293 one hedged form | train/ground/spec.json |
amendment "ground-4 read: FAIL; correction 3; ground-5 registered" |
| Table 4 ground-3: 12,752; .9883; .76%; 2.69% | LLM.md §kai-1.4 |
ground-3 bullet (floor .5) |
| Table 4 ground-4: 13,152; .964; 2.2%; 7.3% | train/ground/spec.json |
same amendment: "accuracy of grounded .964, wrong 293 of 13,152, false-grounded 293 of 4,000 (7.3%)"; 2.2% derived (293/13,152) |
| Table 4 ground-5: 13,632; .9943; .34%; 1.03% | results/ground/run8k/README.md |
served row |
| Table 4 ground-6: 13,632; .9003; 5.76%; 3.17% | results/ground/run9r/README.md |
"Ground-6, the blind read" |
| ground-6 kind 0.883, count 0.834 | results/ground/run9r/README.md |
same bullet |
| Table 4 ground-7: 14,832; .9984; .09%; .19% | train/ground/spec.json |
amendment "run11r's verdict is final": wrong 14, false-grounded 11 of 5,680; shares derived |
| ground-7 negatives: corrupted 35, infra 8, hedged 4 of 1,600; one parity flip of 400 | train/ground/spec.json |
same amendment |
| kai-1.7 default by owner's decision over NO-GO | train/ground/spec.json |
amendment "kai-1.8 (run12r) registered before its fit" (after field) |
| Table 4 ground-8 | results/ground/run12r/README.md |
blind paragraph |
| Table 4 ground-9: 14,832; 3 wrong; 93 pass / 2 fail | LLM.md §kai-1.4 |
ground-9 and ground-10 bullet |
| Table 4 ground-10: .9782; 1.16%; 79 pass / 13 fail | LLM.md §kai-1.4; results/ground/run15/errors.md |
same bullet |
| Table 4 ground-14: 3,317 read | results/ground/g14-invalid/audit.json |
rows = 3317 |
| training 62,159, development 19,508, fourteen registers | results/ground/run12r/README.md |
first paragraph |
| k 8, three epochs, seed 13 | results/ground/run12r/README.md |
first paragraph |
| dev at 0.9: coverage 57.4%, accuracy 0.9999, 1 wrong, 1 false-grounded of 8,305 | results/ground/run12r/README.md |
first paragraph |
| Table 5 rows | results/ground/run12r/README.md |
family table; totals line |
| served verdict = rule on 14,832 | results/ground/run12r/README.md; gate.txt |
"served verdict = rule 14832 of 14832" |
| negatives 9,600, six families of 1,600, 0 grounded | results/ground/run12r/README.md; gate.txt |
"The safety negatives" |
| counterfactuals 2,720; 480 quadruples; life-support 0 of 400 (previous 98) | results/ground/run12r/README.md |
"The counterfactual suite" |
| parity 1.000, 0.9888, 1.000, 0 disagreements, max 0.412 | results/ground/run12r/README.md; gate.txt |
"CPU parity" |
| gate 94 PASS, 0 FAIL, 1 INFO | results/ground/run12r/README.md |
"The gate" |
| live: 28 PASS, 1 FAIL; 399 of 400; 0.913 vs 0.897; six within 0.03 | results/ground/run12r/live.md |
second and third paragraphs |
| default floor 0.5 before; eight scenes 8 of 8 at 0.9 after | results/ground/run12r/floor/after.json; before.json |
none_eq_f90 = 8 (after), none_eq_f50 = 8 (before) |
| boatyard: 4 of 8 binding errors; floor caught 2; 0.9004 and 1.0; one confidently wrong course; 1 person where 4 at 0.5 | results/ground/run12r/floor/README.md |
both paragraphs |
| blind wrong courses 5 at 0.9, 2 at 0.99; causes 2 / 1 / 2 | results/ground/run12r/floor/README.md |
last paragraph |
7 Integrity (07-integrity.tex)
| Claim | Source | Field / line |
|---|---|---|
| Table 6 (3,317; 3,814; 3,814; 3,814; 142; 0) | results/ground/g14-invalid/audit.json |
rows, grounded people courses, count_p exactly 1.0, served count equals gold, model argmax equals gold, model argmax equals gold and p >= .9 |
| 14,832 ground-14 scenes | results/ground/g14-invalid/audit.json |
of = 14832 |
| markers committed after the suite file; binary from an uncommitted tree | results/ground/g14-invalid/README.md |
"What the read measured"; "What is kept" |
| production never served the parser | release.json; git history |
deployed commit 0742d63 precedes 16da1f6 and e92109d |
| cand15: 13 of 92 lines failed; .9782; 1.16% | LLM.md §kai-1.4 |
ground-9 and ground-10 bullet (79 PASS / 13 FAIL) |
| Table 7 (139, 19, 11, 3; 158 = 92%) | results/ground/run15/errors.md |
first table; "So 158 of 172 (92%)" |
| 168 of 171; 2,549 scenes (6.6%); 3 of 4,869 under the other cues | results/ground/run15/errors.md |
cue table (168/2,549; 3/2,436; 0/2,433) |
| median 0.953; 155 of 172 under 0.99; right reads 0.9999 | results/ground/run15/errors.md |
"How sure it was" |
| floors 0.9 / 0.95 / 0.99: 172 / 88 / 17 | results/ground/run15/errors.md |
floor table |
| 18 grounded negatives; four alternatives at probability 1.0; 15 of 18 survive 0.99 | results/ground/run15/errors.md |
"The 18 grounded negatives" |
| 162 role misses; counts of 2 to 19; previous candidate none of 1,920 | results/ground/run15/errors.md |
"The role quadruples" |
| parity defect: same rows read twice on arm64, largest difference 0 | results/ground/run15/errors.md; LLM.md §kai-1.4 |
item 4 of "What follows"; ground-10 bullet |
8 Results (08-results.tex)
| Claim | Source | Field / line |
|---|---|---|
| harness: seed 13, 62 suites, 9,499 states, 11,099 questions; reproduces upstream Part B to the fourth decimal | LLM.md §Evaluation |
Frozen harness paragraph |
| ten application suites of 400 (one of 399); typed decisions 2,000 questions over 400 states; MASSIVE 51 languages of 100 | benchmarks/decision/results/kai-a7/scores.json |
suites[s].kai.n (400; model routing 399; typed 2000); massive_summary.kai.n = 5100, languages = 51; 400 typed states derived (9,499 − 3,999 − 5,100) |
| Laya port parity: 21 states, 12 languages, 2.86e-5, 108 of 108 | results/laya/parity.json |
states, languages, max_logit, labels_equal |
| Table 8 (every cell) | benchmarks/decision/results/kai-a7/scores.json |
suites[s][b].accuracy for b in kai, laya (laya:typed-decisions for typed), jev; massive_summary[b].macro_accuracy; shown as in that folder's README |
| Laya English on typed decisions .362 | benchmarks/decision/results/kai-a7/scores.json |
suites.typed_decisions["laya:english"].accuracy = 0.3615 |
| means .857, .675, .781 | benchmarks/decision/results/kai-a7/scores.json |
derived: mean of the 12 table rows |
| gate rejects (Laya: support triage; Jev: typed, toxicity, 11 MASSIVE languages) | benchmarks/decision/results/kai-a7/README.md |
"Gates" |
| retained 951: reader .883, base head .893, Jev .875 | results/bernoulli/reader12/Z2/served.json; results/bernoulli/reader2/blend/retention-jev.txt |
metrics.retained.pooled; pooled accuracy line |
| base canary .507 / .290 | results/bernoulli/reader2/templates/legacy.json |
sets.critical |
| Table 9, base head column | results/bernoulli/reader2/templates/legacy.json; results/bernoulli/reader2/matrix-kai-1-deployed.json; retention-jev.txt |
sets.*; in band = 13, ordered states = 4; legacy pooled .893 |
| Table 9, X column | results/bernoulli/reader8/X/verdict.txt |
"Metrics" and "Numbers" lines |
| Table 9, Z2 column | results/bernoulli/reader12/Z2/served.json |
metrics.* |
| Table 9, Jev column | results/bernoulli/reader2/templates/jev.json; matrix-jev.json; retention-jev.txt |
sets.*; in band = 40, ordered states = 8; jev pooled .875 |
| reader generations (depth; business states; replay; square-root balance; Jev teacher) | LLM.md §kai-1 and capabilities |
Bernoulli reader bullets (k = 2, 4, 6; arms B, F, Z, Z2) |
| Z2 fails calibration (ECE .143), verify, retain (five suites), matrix (40 vs 41) | results/bernoulli/reader12/Z2/verdict.txt; LLM.md §kai-1 and capabilities |
(A) and (B) lines; "the matrix at 41" (Arm Z2 read) |
| answer suite .853 vs Jev .947 | results/bernoulli/reader12/Z2/verdict.txt |
"answer 0.853 (Jev 0.947)" |
| 50.7 ms vs the 30 ms floor line | results/bernoulli/reader12/Z2/served.json; verdict.txt |
latency.p50_ms; "latency p50 <= 30 ms" |
| transfer suite sizes 866; 48 / 72 / 111; 300; 52 | LLM.md §Evaluation |
Transfer paragraph |
| Table 10 | results/transfer/{kai-1.2,kai-1,laya}/transfer.json; results/transfer/kai-1.2/against-von-1.2.json, against-von-2-nano-r2.json |
suites[s].accuracy, jabr.clean.accuracy, pooled.accuracy; recorded rows from groups[s].base |
| paired: 1,449 cases; −0.135 [−0.170, −0.103], Holm 6e-14; worse on jabr and standard | results/transfer/kai-1.2/against-laya.json |
groups.pooled.accuracy; groups.jabr, groups["jevbench.standard"] verdicts |
| vs Von 1.2 −0.206; vs Von 2 nano r2 −0.271 | results/transfer/kai-1.2/against-von-1.2.json; against-von-2-nano-r2.json |
groups.pooled.accuracy.delta |
| polarity: 74 items; same bits in all three forms | results/transfer/kai-1.2/transfer.json; LLM.md §Evaluation |
suites.polarity.metrics (n 74, negation 1.0, swap 0.0); Transfer paragraph |
| B1 a7 0.48, Jev 0.98 | LLM.md §Research |
finding 1 |
| Decision Index 0.2.1, 38 benchmarks | LLM.md §kai-1 and capabilities; LLM.md §Training |
Decision Index bullet; Barrier ("131,616 rows of 38 benchmarks") |
| Table 11 | results/decision-index/kai-1/gap.json; kai-1-reach32768/gap.json |
areas[a].{kai,jev,clef}, refused, requests |
| jailbreak suite: 91 attacks, 309 benign | results/guard/score400.json |
read.attacks, read.benign |
| Table 12 rows | results/guard/score400.json (base head = references.legacy; kai-1.2 = references.kai12; guard-1 = read; Jev = references.jev); results/guard2/score.json (v1.candidate); results/guard3/score.diag.txt (v1 line) |
correct, caught, false_alarms, auroc |
| public pool: 1,163 attacks, 1,006 benign; AUROC .725 / .896; caught 26.9% / 77.4%; false alarms 3.6% / 13.0% | bench/jailbreak-suite/results/report.txt |
"by pool", real+optimized rows |
| guard-3 sealed: 1,191 attacks, 1,532 benign; 42.7% at 1.70%; kai-1.3 42.9% at 4.05%; Jev 88.1% at 17.6% | results/guard3/score.sealed.txt |
guard3, kai13, jev lines |
| production bar 95% at 1% | LLM.md §kai-1 and capabilities |
guard-3 bullet ("the bar (95% at 1%)") |
| InjecAgent: 141 of 250 at 5 of 207 | LLM.md §kai-1 and capabilities |
kai-1.3 in production bullet |
| trolley-bench: 5,240 setups, eleven families, 24 wordings (4 for one family), 0 to 100 | LLM.md §Evaluation |
Trolley paragraph |
| scorer reproduces 34 of 34 values | LLM.md §Evaluation |
Trolley paragraph |
| Table 13, first five rows | bench/trolley/results/results.json; report.md |
systems[s].dominated.acted, contradictions.n, spread.count_response; report rows server20, self |
| Table 13, last row | results/ground/run12r/trolley.score.json |
systems["kai-1.8-ground"]: dominated.acted 17, contradictions.n 2291, spread.count_response .357, server20.kept 19, self.n 297 |
| flat within 0.05 | LLM.md §Evaluation |
Trolley paragraph |
| act + leave 1.33 to 1.47 | bench/trolley/results/mechanism.md |
Polarity lines (1.331, 1.473, 1.405) |
| logit difference +0.03 (kai-1.4) | bench/trolley/results/mechanism.md |
kai-1.4 Consequence line (+0.027) |
| wording range 0.50 to 0.63 | bench/trolley/results/mechanism.md |
Wording lines (0.626, 0.513, 0.503) |
| 2,291 of 5,240 escalated; 16 of 24 wordings out of order | results/ground/run12r/trolley.score.json |
contradictions.n; monotone.n = 16, of = 24 |
| conditional: 2,432 cases, four families | LLM.md §Evaluation |
Conditional paragraph |
| conditional hazard prior −0.041 / −0.026 / −0.017 against line .50; polarity within .10 on 247 / 87 / 143 of 512 against line .95; P(act) at zero .83 / .88 / .87 | bench/conditional/results/report.md |
hazard rows |
| Table 14, kai-1.4 row and base row | LLM.md §kai-1.4 |
"One Kai, unified": Needle on the x86-64 host (900 .800 ... 31,000 .733; kai-1.3's base .33 ... .30) |
| bars .80 at 8,000 and .70 at 31,000; interval [.648, .804] | LLM.md §kai-1.4 |
same paragraph |
| 40 choice, 40 score, 40 yes/no a cell | LLM.md §kai-1.4 |
"120 states a cell ... (choice / score / yes-no of 40)" |
| kai-1.4 canary .535 / .467 | LLM.md §kai-1.4 |
gauge sentence (".5349 / .4665") |
| 312 s at the median for 31,000 tokens | LLM.md §kai-1.4 |
"31K latency on four x86 threads p50 312 s" |
| tools: 5,658 cases; nDCG@10 .8937 [.8891, .8984]; top-1 .7580; BM25 .7805 / .5411; head .8437 / .6545 | results/kai-1.5a/tools-dev.txt |
overall line and the BM25 line |
| when to call: 31 of 254 (.122); .919 kept | results/kai-1.5a/tools-dev.txt |
when line |
9 Limits (09-limits.tex)
| Claim | Source | Field / line |
|---|---|---|
| eight sets; five above text where six required | results/kai-1.7-evidence/README.md; release.json |
table; versions["kai-1.7-evidence"].verdict |
| bearing .906 vs .389; speech .961 vs .194 | results/kai-1.7-evidence/README.md |
table rows |
| ECE .143 | results/bernoulli/reader12/Z2/verdict.txt |
"ECE 0.143" |
| temperatures at the search bound | train/ground/spec.json |
amendment "the freeze is atomic ...": calibration |
| typed decisions under-confident | LLM.md §Models |
a7-512 paragraph (confidence 0.59, accuracy 0.76) |
| typed decisions .356 vs .759; exact .035 to .072; Jev .438 | LLM.md §Research; LLM.md §Evaluation |
finding 4; Joint panel ("exact 0.438") |
| suffix and role spoofing flip Kai | bench/hijack/report.txt |
Reading paragraph |
| each reader trained with one seed | results/ground/run12r/README.md; LLM.md §kai-1 and capabilities |
"seed 13"; Arm Z2 read ("One seed") |
| escalated two of three binding failures | results/ground/run12r/floor/README.md |
binding paragraph |
10 Future work (10-future.tex)
| Claim | Source | Field / line |
|---|---|---|
| every development line and threshold (0.999; 1 to 10^18 and past u64; 0.99; 0.98; 0; 0.99 isolated assets; 400 rows, two hosts, no flip) | train/ground/cand21.json |
dev_acceptance |
| blind suite only after every line holds; sealed by an independent evaluator; read once | train/ground/cand21.json |
blind_suite |
| kai-1.8 stays until a successor passes | train/ground/cand21.json |
production |
Appendices (a1-wire.tex, a2-sources.tex)
The two listings are request shapes with states written for this paper; they
carry no measured value. The count labels "0" to "100" and "more" are as in
section 3. Commit f814147 is the implementation repository's head when this
paper was written.
Release 0.4.32 (added after the roll)
| claim | source |
|---|---|
| no argmax flip over 955 answers, largest difference 1.8e-5, arm64 vs x86-64 | decision results/release/v0.4.32/rows.json (argmax_flips, max_dp) |
| 400 typed policy rows, 0 policy-outcome flips | decision results/release/v0.4.32/parity.json (outcome_flips) |
| mid-roll: new replica within 1.6e-6, old replica .015 to .043 off | decision results/release/v0.4.32/canary-production-midroll.jsonl against canary-reference-midroll.jsonl |
| after the roll: 16 of 16 within 1.5e-6 | decision results/release/v0.4.32/canary-production.jsonl against canary-reference.jsonl |