mirror of
https://github.com/hanzoai/papers.git
synced 2026-11-02 13:32:30 +00:00
72 lines
3.8 KiB
TeX
72 lines
3.8 KiB
TeX
\section{Related work}
|
|
\label{sec:related}
|
|
|
|
\paragraph{Typed decision models.}
|
|
TypeSafe's Jev~\cite{typesafejev} introduced the typed interface adopted
|
|
here, served through OpenRouter~\cite{openrouterdecisions}; its design
|
|
notes~\cite{jevnote} argue for typed state and permissions as decisions.
|
|
Laya~\cite{laya} is an open encoder implementation trained with proper scoring
|
|
rules~\cite{gneiting2007}.
|
|
|
|
\paragraph{Encoders that score options.}
|
|
Laya's joint head is a cross-encoder, as in passage
|
|
re-ranking~\cite{nogueira2019} and entailment-based zero-shot
|
|
classification~\cite{yin2019}, after BERT~\cite{devlin2019}. Encoding
|
|
candidates independently, caching them, and letting them attend to a context is
|
|
the poly-encoder~\cite{humeau2020poly} and, per token, late
|
|
interaction~\cite{khattab2020colbert}; Kai adds a law per type, a
|
|
cumulative-link model for ordinal answers~\cite{mccullagh1980}, and retrieval
|
|
that keeps the mass it cuts.
|
|
|
|
\paragraph{Parallel refinement.}
|
|
Masked discrete diffusion~\cite{austin2021d3pm},
|
|
mask-predict~\cite{ghazvininejad2019} and MaskGIT~\cite{chang2022maskgit}
|
|
predict masked tokens in parallel and re-predict the uncertain ones. Kai's unit
|
|
is a typed variable with its own domain, ordered by the program's DAG and
|
|
projected onto its rules.
|
|
|
|
\paragraph{Multimodal front ends.}
|
|
Whisper~\cite{radford2023whisper} and SigLIP~\cite{zhai2023siglip} give
|
|
frozen audio and image features; time-series
|
|
models~\cite{das2024timesfm,ansari2024chronos} give forecasts, which enter Kai
|
|
as evidence, not answers.
|
|
|
|
\paragraph{Structured output and guards.}
|
|
Constrained decoding~\cite{willard2023} removes parse failures, not the
|
|
generation, and its probability is a sequence's, not a distribution over the
|
|
caller's options. Tool-using agents~\cite{yao2023react,schick2023toolformer}
|
|
make the decisions of Table~\ref{tab:programs} inside the generative loop.
|
|
Guard models such as Llama Guard~\cite{inan2023llamaguard} judge text against
|
|
a harm taxonomy; Section~\ref{sec:authority} places any guard below policy.
|
|
|
|
\paragraph{Calibration and abstention.}
|
|
Temperature scaling~\cite{guo2017}, the reject option~\cite{chow1970},
|
|
selective classification~\cite{geifman2017} and conformal
|
|
prediction~\cite{vovk2005} are the theory behind fitted temperatures,
|
|
abstention to \ask{} and the prediction sets of Table~\ref{tab:capability}.
|
|
|
|
\paragraph{Training across machines.}
|
|
Local SGD with an outer Nesterov step is DiLoCo~\cite{douillard2023diloco}; the
|
|
outer step's effective multiplier (Proposition~\ref{prop:outer}) is the
|
|
quantity its authors tune. Carrying the rounding of a compressed update into
|
|
the next is error feedback~\cite{seide2014onebit,karimireddy2019ef}; bf16
|
|
compute over F32 masters is mixed-precision training~\cite{micikevicius2018mixed}.
|
|
Kai's trainer adds exact bookkeeping: join-order summation, a broadcast change
|
|
that every copy applies bit for bit, and resume equal to the uninterrupted run.
|
|
|
|
\paragraph{Contamination.}
|
|
Near-duplicate detection by min-hashed shingles~\cite{broder1997resemblance} is
|
|
the standard tool for deduplicating training corpora against each other and
|
|
against benchmarks~\cite{lee2022dedup}. Kai's barrier adds sampled-span
|
|
containment, which finds a test item embedded in a longer training text, and
|
|
treats evaluation as a role that no license decision admits.
|
|
|
|
\paragraph{Authority and decision analysis.}
|
|
Enforcing the join of two verdicts is the lattice model of information
|
|
flow~\cite{denning1976} applied to decisions, and a policy evaluated before
|
|
every action is a reference monitor~\cite{anderson1972}. Package scores are
|
|
additive multi-attribute utility~\cite{keeney1976}; weight sensitivity is
|
|
studied by Triantaphyllou and S\'anchez~\cite{triantaphyllou1997}, global
|
|
sensitivity by Saltelli et al.~\cite{saltelli2008}, and what is worth
|
|
resolving by the value of information~\cite{howard1966}.
|