Files
ml/hanzo-learn/src/data.rs
zeekay 026ac25fcf hanzo-learn: fitting is a transformation, a model is a value, predicting is a query
A crate whose whole interface is two verbs. `Config` holds hyperparameters and has no
`predict`. `Fit` maps `Config -> Data -> Model`. `Model` is immutable, has private fields
and no public constructor, so an unfitted model is not a state to be checked for — it is
a value that cannot be spoken, and `check_is_fitted` has no job to do. `Predict::Answer`
is `f64` for a regressor and `Class` for a classifier: the two do not unify, so a
regressor and a classifier are different types rather than one type with a mode flag.
There is no `get_params`, no `BaseEstimator`, no clone protocol and no `Pipeline` — those
exist in Python because a config and a fitted model are the same object there.

Every fitted model is named by `Address`, a content address over its shape, its
hyperparameters and its learned parameters, hashing floats as IEEE-754 bits and
length-prefixing every sequence so the encoding is injective. Same construction and same
reasoning as the risk plane's published model values, so a fit can be stored, cited by a
decision, compared against a challenger and rolled back to without a registry.

Three estimators: least squares by Householder QR on centred data (QR not the normal
equations, which square the condition number; centring not a ones column, so the
intercept never enters the factorisation), binary logistic regression by Newton steps on
a strictly convex penalised objective, and gradient boosted CART regression trees.

A TREE READS FEATURES AT IEEE-754 BINARY32, and that is the algorithm rather than a
detail. scikit-learn casts X to float32 before its splitter sees it, XGBoost and LightGBM
do the same, and ai.onnx.ml's TreeEnsembleRegressor stores nodes_values as floats. A tree
never does arithmetic on a feature value — it only orders values and places thresholds
between them — so precision decides only which side of a boundary a sample falls on.
Placing the threshold at single(a)/2 + single(b)/2 instead of a/2 + b/2 moves it by 2e-8,
which is enough to send a sample down the other branch. Measured over the 40 depth-1
trees of the stumps fixture, whose roots all see the full sample set: the single-precision
midpoint reproduces scikit-learn's threshold bit-for-bit 40 times out of 40, the
double-precision midpoint 0 times out of 40.

Correctness is asserted against scikit-learn itself. Fixtures generated by 1.9.0 are
committed and read as data, so `cargo test` needs no Python and the oracle's version is a
recorded fact. 195 boosted trees / 1,977 split nodes agree node for node — split feature,
threshold and both children all exact — with leaf values to 1.3e-15 and predictions to
8.9e-16 against a 1e-10 bound; least squares agrees to 1.1e-15 against 1e-10 and logistic
to 1.2e-8 against 1e-7, with hard labels exact. Each tolerance is derived from the
mathematics and every assertion prints the deviation it observed, so a regression that
stays inside a bound is still visible. The generator refuses to record an oracle that is
not a function of its data: at scikit-learn's default min_samples_leaf a tree's structure
varies with random_state (20 structures over 40 seeds, prediction spread 3.9), so every
case pins the tie away and proves determinism over 12 seeds before writing a number. This
crate breaks split ties by lowest feature index instead, because a content address over a
model an RNG helped choose would name one draw rather than one fit.

The exhaustive split search is not lowered to hanzo-kernel, and the obstacle is specific:
a launch geometry must be stated before the launch, and across all 69 launch sites in
that crate Grid::Static is the only variant used, every geometry computed on the host from
a count it already knows. A CART node's sample count is not such a number. The histogram
formulation dissolves this — bin count is a hyperparameter, so the per-level geometry is
host-knowable — but it quantises thresholds and so cannot hold the node-for-node bar. It
is named, not built.

scikit-learn is a test oracle only. No code or docstring is taken from it, and NOTICE
records that, along with the obligation to add BSD-3 attribution if that ever changes.

The crate also carries, unchanged and unclaimed, the anomaly-detection and preprocessing
modules written concurrently on another track (isolation, neighbour, transform, scale,
impute, encode, metric, moment, split, twister, plus benches and the perf oracle). They
share lib.rs and tests/sklearn.rs with the estimator seam, so the crate cannot be
committed in halves. Their `Transform` and `Outlier` traits extend this seam without
changing `Fit` or `Predict`. One of their fixture tests,
`the_imbalanced_metrics_match_scikit_learn`, is red on a fixture key that measure.json
does not yet carry — which is why this is a branch and not main.
2026-08-04 17:03:22 -07:00

196 lines
6.4 KiB
Rust

//! The data plane: a design matrix, and observations paired with their targets.
use crate::error::{Error, Result};
/// A design matrix: `n` observations of `p` features, dense, row-major.
///
/// RANK 2 IS IN THE TYPE. A general tensor reaching a least-squares solver carries its
/// rank as a runtime fact, so every solver opens by asking whether it was handed a
/// matrix at all. Here that question has one answer, established once, at
/// construction.
///
/// Row-major because every consumer walks observations: the tree splitter gathers one
/// feature at a time but does so through an index permutation, and the linear solvers
/// walk rows. A column-major variant would help the splitter and hurt everything else;
/// one layout, chosen for the common walk, is simpler than two that must agree.
#[derive(Clone, PartialEq, Debug)]
pub struct Matrix {
rows: usize,
cols: usize,
data: Vec<f64>,
}
impl Matrix {
/// A matrix from row-major values.
///
/// Fails unless `data.len() == rows * cols`, and unless both are non-zero — a
/// matrix with no columns has no features to fit against and one with no rows has
/// nothing to fit from, so both are refused here rather than producing a model
/// that answers everything with a constant.
pub fn new(rows: usize, cols: usize, data: Vec<f64>) -> Result<Self> {
if rows == 0 || cols == 0 {
return Err(Error::Shape(format!(
"a design matrix is {rows}x{cols}; both dimensions must be non-zero"
)));
}
if data.len() != rows * cols {
return Err(Error::Shape(format!(
"{rows}x{cols} needs {} values, got {}",
rows * cols,
data.len()
)));
}
if let Some(i) = data.iter().position(|v| !v.is_finite()) {
// Refused at the boundary, with the position, because the alternative is a
// model whose every parameter is NaN and whose predictions are all NaN,
// discovered downstream with nothing left pointing at the cause.
return Err(Error::Shape(format!(
"value at row {} column {} is not finite",
i / cols,
i % cols
)));
}
Ok(Self { rows, cols, data })
}
/// From rows, which is how callers usually hold their data.
pub fn rows(rows: &[Vec<f64>]) -> Result<Self> {
let n = rows.len();
let p = rows.first().map_or(0, Vec::len);
if let Some(i) = rows.iter().position(|r| r.len() != p) {
return Err(Error::Shape(format!(
"row 0 has {p} features but row {i} has {}",
rows[i].len()
)));
}
Self::new(n, p, rows.concat())
}
/// Number of observations.
pub fn n(&self) -> usize {
self.rows
}
/// Number of features.
pub fn p(&self) -> usize {
self.cols
}
/// One observation.
pub fn row(&self, i: usize) -> &[f64] {
&self.data[i * self.cols..(i + 1) * self.cols]
}
/// One cell. Hot in the tree splitter, which reaches for a single feature across a
/// permutation of rows rather than for whole rows.
pub fn at(&self, i: usize, j: usize) -> f64 {
self.data[i * self.cols + j]
}
/// Column means — the term both linear fits centre on.
pub(crate) fn column_means(&self) -> Vec<f64> {
let mut m = vec![0.0; self.cols];
for i in 0..self.rows {
for (acc, v) in m.iter_mut().zip(self.row(i)) {
*acc += v;
}
}
let n = self.rows as f64;
for v in &mut m {
*v /= n;
}
m
}
}
/// Observations paired with their targets.
///
/// THE ONE PLACE their agreement in `n` is established. Without this type every fit
/// would open by comparing two lengths, and every fit would be a partial function of
/// two arguments that must agree. With it, `fit` is total in `n`: a `Samples` that
/// exists is a dataset whose rows have targets, so no solver below re-checks it and
/// none can forget to.
///
/// It is the closest this crate gets to a dependent type. The agreement is a property
/// of a PAIR, so it is checked where the pair is made.
#[derive(Clone, PartialEq, Debug)]
pub struct Samples<T> {
x: Matrix,
y: Vec<T>,
}
impl<T> Samples<T> {
/// Pair a design matrix with its targets.
pub fn new(x: Matrix, y: Vec<T>) -> Result<Self> {
if x.n() != y.len() {
return Err(Error::Shape(format!(
"{} observations but {} targets",
x.n(),
y.len()
)));
}
Ok(Self { x, y })
}
/// The observations.
pub fn x(&self) -> &Matrix {
&self.x
}
/// The targets.
pub fn y(&self) -> &[T] {
&self.y
}
/// Number of observations. Equal to `y().len()` by construction.
pub fn n(&self) -> usize {
self.x.n()
}
/// Number of features.
pub fn p(&self) -> usize {
self.x.p()
}
}
/// The mean of a slice, accumulated in index order.
///
/// Summation order is part of the answer at this precision, so it is stated here once
/// and every caller inherits the same one: two fits of the same data must agree
/// bit-for-bit with each other, which a parallel reduction over an unspecified tree
/// would not guarantee.
pub(crate) fn mean(v: &[f64]) -> f64 {
v.iter().sum::<f64>() / v.len() as f64
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_matrix_refuses_a_length_that_is_not_its_shape() {
assert!(Matrix::new(2, 3, vec![1.0; 5]).is_err());
assert!(Matrix::new(0, 3, vec![]).is_err());
assert!(Matrix::new(2, 3, vec![1.0; 6]).is_ok());
}
#[test]
fn a_matrix_refuses_a_value_that_is_not_finite_and_says_where() {
let e = Matrix::new(2, 2, vec![1.0, 2.0, f64::NAN, 4.0]).unwrap_err();
assert!(format!("{e}").contains("row 1 column 0"), "{e}");
}
#[test]
fn ragged_rows_are_refused_with_the_offending_row() {
let e = Matrix::rows(&[vec![1.0, 2.0], vec![3.0]]).unwrap_err();
assert!(format!("{e}").contains("row 1"), "{e}");
}
#[test]
fn samples_are_the_one_place_n_agreement_is_checked() {
let x = Matrix::new(3, 2, vec![1.0; 6]).unwrap();
assert!(Samples::new(x.clone(), vec![1.0, 2.0]).is_err());
assert!(Samples::new(x, vec![1.0, 2.0, 3.0]).is_ok());
}
}