Kai: stage a6 on the frozen harness, what it wins and loses and why, the score-head probe and its fix, the sibling rows for a5 and a6, and what a6 leaves open.
Hanzo CI/CD / test (linux-amd64) (push) Skipped
Hanzo CI/CD / test (linux-arm64) (push) Skipped
Hanzo CI/CD / test-advisory (push) Skipped
ci / papers (push) Failing after 27s
Hanzo CI/CD / test-amd64 (push) Successful in 1m12s
Build LaTeX Papers / build-pdfs (push) Failing after 32s
Hanzo CI/CD / cicd (push) Successful in 1m43s
CI/CD / cicd (push) Successful in 3m0s

Claude-Session: https://claude.ai/code/session_018Gt5TvrE2rxXuZikWSauv3
This commit is contained in:
2026-09-27 20:30:01 -07:00
parent cf7d5d7fe8
commit aff739a295
5 changed files with 71 additions and 15 deletions
+4 -2
View File
@@ -47,9 +47,11 @@ designed, not built.
floor, license classes and a contamination barrier
(Section~\ref{sec:training}).
\item \textbf{What went wrong}: stage A's encoder stopped relating two texts;
the measurements, the ablations that locate the cause, and the fix; and how
the measurements, the ablations that locate the cause, and the fix; how
much of the typed-decisions test set Laya's checkpoint had seen in near
form (Section~\ref{sec:findings}).
form, and that it did not help it; and a score head that lost thirty
points to a softmax over level texts on the same encoder and data
(Section~\ref{sec:findings}).
\item \textbf{Policy precedence} as a lattice join, with a proof that
enforcing a model only adds friction, and \textbf{programs and packages}
that replay, diff, refresh and say what flips a decision, with SysML~v2
+23 -2
View File
@@ -112,8 +112,9 @@ states without a sibling and the 165 with one apart
(Table~\ref{tab:siblings}). Laya's typed checkpoint scores alike on both
halves, so its lead is not memorized from the siblings; Jev, which never saw
them, is 8 points better without them; and Kai falls on exactly the states
whose siblings the barrier removed from its training. The lead is the head and
the training, and this section is a caveat about Kai's data, not Laya's score.
whose siblings the barrier removed from its training. Stage a5 read the split
whole and recovered the with-sibling half; the lead is the head and the
training, and this section is a caveat about Kai's data, not Laya's score.
\begin{table}[h]
\centering
@@ -123,6 +124,8 @@ the training, and this section is a caveat about Kai's data, not Laya's score.
typed decisions & all 400 & without a sibling (235) & with a sibling (165) \\
\midrule
Kai (stage a4) & 0.523 & 0.605 & 0.406 \\
Kai (stage a5) & 0.705 & 0.697 & 0.716 \\
Kai (stage a6) & 0.750 & 0.734 & 0.773 \\
Laya typed & 0.766 & 0.773 & 0.756 \\
Jev & 0.736 & 0.770 & 0.686 \\
\bottomrule
@@ -133,6 +136,24 @@ the training, and this section is a caveat about Kai's data, not Laya's score.
\label{tab:siblings}
\end{table}
\subsection{The score head}
\label{sec:score}
With typed decisions at 0.705 in stage a5 against Laya's 0.766, one probe put
the gap in the head rather than the encoder or the data: Kai's head trained on
Laya's own typed encoder (\texttt{kai-1-agent}) and typed decisions' train
split alone, six epochs, scored 0.605 on the harness, choice 0.608, \op{noul}
0.833, score 0.431, where Laya's checkpoint scores 0.733, 0.857 and 0.723
\meas. The cumulative-link score head of Section~\ref{sec:model}, one location
against ordered cutpoints, lost thirty points on score questions to Laya's
formulation, a softmax over the level texts. Kai now answers a score question as
a choice over its levels when the checkpoint says so (\texttt{ordinal: false}),
trained with the ranked probability score in place of the Brier so that mass
far from the gold level costs more than mass beside it; the same probe with
that head reached 0.742 on the typed validation split at the point where the
ordinal head sat at 0.571, and stage a6 raised typed decisions' score-question
accuracy on the harness from 0.603 to 0.711 (Section~\ref{sec:accuracy}).
\subsection{Other defects, each fixed}
\label{sec:other}
+30 -8
View File
@@ -3,15 +3,16 @@
Every table in this section is written by \texttt{bench tex} in
\texttt{hanzoai/decision} from the kept results of the runs it names, into
\texttt{tables/}; no number in a table is typed by hand. The capability and speed tables
are stubs until the scored checkpoint is timed; each says so. Each table's last row
\texttt{tables/}; no number in a table is typed by hand. Each table's last row
names the Kai revision, the device and the harness and benchmark commits it
reports.
\paragraph{Backends.}
\textbf{Kai}: stage a5, one epoch on from the final stage-A checkpoint
(Section~\ref{sec:collapse}) with typed decisions' train split read whole
(Section~\ref{sec:overlap}), on the native runtime. \textbf{Laya}: Laya 0.3.20's published weights served natively
\textbf{Kai}: stage a6, two stages on from the final stage-A checkpoint
(Section~\ref{sec:collapse}): a5 read the suites a4 trailed with more rows and
typed decisions' train split whole (Section~\ref{sec:overlap}); a6 answers
score questions as a choice over their level texts (Section~\ref{sec:score})
and repeats the suites still behind; one epoch each, on the native runtime. \textbf{Laya}: Laya 0.3.20's published weights served natively
by the Rust port of its joint head (parity in Section~\ref{sec:checkpoints}),
each case on the checkpoint Laya's router picks, except typed decisions, which
are answered by Laya's typed-decisions checkpoint: the router sends them to the
@@ -61,6 +62,21 @@ tex}; the last row names the revisions.}
\label{tab:harness}
\end{table}
\paragraph{What the table says.}
Stage a6 leads on seven suites: emotion 0.935, Banking77 0.922, phishing
0.993, toxicity 0.807, RAG relevance 0.652, model routing 1.000 and MASSIVE
0.910 across 51 languages against Jev's 0.890 and Laya's 0.382. It ties email
spam (0.998 to 0.998). It trails on four: AG News 0.940 to Laya's 0.950,
jailbreak 0.920 to Jev's 0.940, typed decisions 0.750 to Laya's typed checkpoint
at 0.766 (above Jev's 0.736), and support triage 0.435 to Laya's 0.502.
Between a5 and a6 the score-question accuracy of typed decisions rose from
0.603 to 0.711 (Section~\ref{sec:score}), choice and \op{noul} within two
points of Laya's; support triage fell from 0.490 to 0.435 while its validation
rose from 0.672 to 0.718: five passes over a train split whose labels are noisy
overfit its conventions, and the next stage repeats it twice and adds rows
written by our own model instead. Every figure here is
\texttt{hanzoai/benchmarks} \texttt{decision/results/kai-a6} \meas.
\paragraph{The gate.}
A checkpoint replaces a baseline in a deployment only when \texttt{bench gate}
accepts it: on up to 100 drawn cases per harness and extension suite and their
@@ -69,8 +85,12 @@ gain more than 0.03 ECE, whatever the macro average, and for Laya the native
runtime's parity must hold. The first stage-A checkpoint was rejected: macro
accuracy 0.639 over the 62 suites against Laya's 0.438, parity held, and seven
suites regressed, RAG relevance, typed decisions, Banking77 and jailbreak among
them \meas{} (\texttt{hanzoai/benchmarks} \texttt{9aa6a46}). The final
checkpoint's verdict is reported with its tables.
them \meas{} (\texttt{hanzoai/benchmarks} \texttt{9aa6a46}). Stage a6 is
rejected against Laya on one suite, support triage, and against Jev on
jailbreak, typed decisions, toxicity (its ECE on the drawn cases, at 0.807
accuracy to Jev's 0.662 over the suite) and eight of the 51 MASSIVE languages
\meas; so it does not replace either baseline in a deployment, and this paper
claims no such thing.
\subsection{Capabilities the baselines lack by construction}
\label{sec:capabilities}
@@ -95,7 +115,9 @@ kept runs are at \texttt{4e57fb2}.
readiness and review), with gold computed from a latent world by a policy
the state spells out, so every gold answer keeps every rule. The baselines
answer each question independently; Kai runs the refinement of
Section~\ref{sec:diffusion}. Per-variable accuracy, the share of states with
Section~\ref{sec:diffusion} over a decoder stage a6 has not trained, and
loses this suite to Jev (0.306 per variable to 0.837, and a rule broken in
every state before projection): the decoder is what stage C trains. Per-variable accuracy, the share of states with
all five right, and the share whose answers break a rule, as answered and
after the same exact projection applied to every backend's marginals.
\item[Invariance.] 200 Banking77 questions of 16 options asked in five
+9
View File
@@ -34,6 +34,15 @@ explaining it. Local SGD at a gentle outer step may still move the encoder
further than one process does at the same learning rates, and only the harness
and the context similarity say whether it did.
\paragraph{What stage a6 leaves open.}
Support triage fell under repetition while its validation rose, so the
harness, not validation, must pick a mixture. The joint-coherence suite is lost
outright until stage C trains the decoder. The serving modes of the design, a
decision over a cached latent in microseconds and an incremental encoder over
evidence chunks, are designed and partly built (the cache of question, state
and option latents is exact and tested), not measured; every latency in this
paper is a full encode per call.
\paragraph{Known failure modes of Laya.}
Its authors report that boolean-word labels can be followed instead of their
descriptions, negated requests answered as unnegated, \op{noul} answers driven
+5 -3
View File
@@ -20,9 +20,11 @@ set had same-wording siblings in its training data, and measuring the halves
apart showed the siblings did not help it. Both are written down here with the
runs that show them.
Kai's case does not rest on winning every benchmark: Laya leads where it was
trained and trails Jev off its mixture, and Kai enters this paper only through
the tables its benchmark writes. It rests on what a hosted model cannot offer:
Kai's case does not rest on winning every benchmark: after stage a6 Laya leads
on support triage and AG News, where its mixture holds the sources, Jev on
jailbreak and by a point and a half on typed decisions through its typed
checkpoint, and Kai on the other eight, MASSIVE among them; Kai enters this
paper only through the tables its benchmark writes. It rests on what a hosted model cannot offer:
a pinned revision, deployment where the data live, no per-call fee, a runtime
linked into the system it governs, a package per run that says what was
decided, from what, and what would change it, and, in Enso, less generation per