papers/kai/sections/09-limits.tex

\section{Limits}
\label{sec:limits}

\paragraph{What is not measured.}
Until \texttt{bench tex} writes the capability and speed tables for the final
checkpoint, this paper claims no capability or speed figure for it.
Stage B's adapters and stage C's decoder have code, data and stage files but no
trained checkpoint, so the media rows measure a text rendering unless the
scored checkpoint reads evidence, and the joint rows measure refinement over
the factorized head, not the slot decoder. The Enso evaluation is a design.

\paragraph{What is not built.}
Latents kept across calls, refinement inside the program executor, the program
form of the wire, the closed-form weight-change bound, sampled flip
probabilities, rejecting a narrative that states an uncited number, and
distilling refinement into one or two passes are \proposed.

\paragraph{What the harness cannot show.}
Kai trains on the train split of every harness source, so its leads over a
model that did not are in-distribution; with Jev's training data unknown, no
table here measures generalisation to a new task. Laya's typed-decisions score
rests on templates shared with half the test set
(Section~\ref{sec:overlap}), and the harness's MASSIVE suites hold 100
utterances per language, so a language's accuracy moves by 0.01 per question.
The capability suites were built by us, for properties Kai was designed to
have; they show what the baselines cannot do by construction, not that Kai
does it well, which only their Kai rows say.

\paragraph{What stage A leaves open.}
The collapse of Section~\ref{sec:collapse} is located (encoder, full mixture
under a $7\times$ outer step) but not attributed to a part of the mixture;
the fix removes the aggressive step and restarts from Laya's weights rather than
explaining it. Local SGD at a gentle outer step may still move the encoder
further than one process does at the same learning rates, and only the harness
and the context similarity say whether it did.

\paragraph{What stage a7 leaves open.}
Support triage and jailbreak did not move under repetition: triage's four
overlapping queues need pairs of near tickets from different queues, and
jailbreak needs benign near-neighbours of attacks, not more attacks; the next
stages add both from pinned public data. The joint-coherence suite is lost
outright until stage C trains the decoder. The serving modes of the design, a
decision over a cached latent and an incremental encoder over evidence chunks,
are built (the cache of question, state and option latents is exact and
tested) but not yet timed on a production backend; every latency in this paper
is a full encode per call.

\paragraph{Known failure modes of Laya.}
Its authors report that boolean-word labels can be followed instead of their
descriptions, negated requests answered as unnegated, \op{noul} answers driven
by the labels rather than the state, and a score's first level rarely picked
by the multilingual checkpoint~\cite{laya} \reported. A program's tests should
add a negated and a relabelled variant of each case, which Kai's robustness
transforms do in training.

\paragraph{Calibration is assumed, then checked.}
Thresholds, abstention, conformal sets and flip probabilities take
probabilities at their word; a shift in the evidence distribution breaks fitted
temperatures and pinned calibrations alike, and only shadow records detect
it.

\paragraph{Adversarial state beyond the lattice.}
Proposition~\ref{prop:tighten} protects constraints, not preferences: an
adversary can push routing toward a cheaper model or completion toward an early
finish. Those need deterministic backstops, such as requiring a passing test
run in the transcript. Deliberate friction is bounded by policy, not
prevented.

\paragraph{The study model has assumptions.}
Additive scores need preferentially independent criteria; one-parameter flips
hold the rest fixed; assumption sweeps are a grid; sampled flip probabilities
treat cells as independent, which fails when two read the same evidence. A
study's figures are its sources'; the JLTV trade cites a company figure for the
hybrid's fuel economy, and its package says so rather than resolving it.