Files
ml/hanzo-learn/src/lib.rs
zeekay 026ac25fcf hanzo-learn: fitting is a transformation, a model is a value, predicting is a query
A crate whose whole interface is two verbs. `Config` holds hyperparameters and has no
`predict`. `Fit` maps `Config -> Data -> Model`. `Model` is immutable, has private fields
and no public constructor, so an unfitted model is not a state to be checked for — it is
a value that cannot be spoken, and `check_is_fitted` has no job to do. `Predict::Answer`
is `f64` for a regressor and `Class` for a classifier: the two do not unify, so a
regressor and a classifier are different types rather than one type with a mode flag.
There is no `get_params`, no `BaseEstimator`, no clone protocol and no `Pipeline` — those
exist in Python because a config and a fitted model are the same object there.

Every fitted model is named by `Address`, a content address over its shape, its
hyperparameters and its learned parameters, hashing floats as IEEE-754 bits and
length-prefixing every sequence so the encoding is injective. Same construction and same
reasoning as the risk plane's published model values, so a fit can be stored, cited by a
decision, compared against a challenger and rolled back to without a registry.

Three estimators: least squares by Householder QR on centred data (QR not the normal
equations, which square the condition number; centring not a ones column, so the
intercept never enters the factorisation), binary logistic regression by Newton steps on
a strictly convex penalised objective, and gradient boosted CART regression trees.

A TREE READS FEATURES AT IEEE-754 BINARY32, and that is the algorithm rather than a
detail. scikit-learn casts X to float32 before its splitter sees it, XGBoost and LightGBM
do the same, and ai.onnx.ml's TreeEnsembleRegressor stores nodes_values as floats. A tree
never does arithmetic on a feature value — it only orders values and places thresholds
between them — so precision decides only which side of a boundary a sample falls on.
Placing the threshold at single(a)/2 + single(b)/2 instead of a/2 + b/2 moves it by 2e-8,
which is enough to send a sample down the other branch. Measured over the 40 depth-1
trees of the stumps fixture, whose roots all see the full sample set: the single-precision
midpoint reproduces scikit-learn's threshold bit-for-bit 40 times out of 40, the
double-precision midpoint 0 times out of 40.

Correctness is asserted against scikit-learn itself. Fixtures generated by 1.9.0 are
committed and read as data, so `cargo test` needs no Python and the oracle's version is a
recorded fact. 195 boosted trees / 1,977 split nodes agree node for node — split feature,
threshold and both children all exact — with leaf values to 1.3e-15 and predictions to
8.9e-16 against a 1e-10 bound; least squares agrees to 1.1e-15 against 1e-10 and logistic
to 1.2e-8 against 1e-7, with hard labels exact. Each tolerance is derived from the
mathematics and every assertion prints the deviation it observed, so a regression that
stays inside a bound is still visible. The generator refuses to record an oracle that is
not a function of its data: at scikit-learn's default min_samples_leaf a tree's structure
varies with random_state (20 structures over 40 seeds, prediction spread 3.9), so every
case pins the tie away and proves determinism over 12 seeds before writing a number. This
crate breaks split ties by lowest feature index instead, because a content address over a
model an RNG helped choose would name one draw rather than one fit.

The exhaustive split search is not lowered to hanzo-kernel, and the obstacle is specific:
a launch geometry must be stated before the launch, and across all 69 launch sites in
that crate Grid::Static is the only variant used, every geometry computed on the host from
a count it already knows. A CART node's sample count is not such a number. The histogram
formulation dissolves this — bin count is a hyperparameter, so the per-level geometry is
host-knowable — but it quantises thresholds and so cannot hold the node-for-node bar. It
is named, not built.

scikit-learn is a test oracle only. No code or docstring is taken from it, and NOTICE
records that, along with the obligation to add BSD-3 attribution if that ever changes.

The crate also carries, unchanged and unclaimed, the anomaly-detection and preprocessing
modules written concurrently on another track (isolation, neighbour, transform, scale,
impute, encode, metric, moment, split, twister, plus benches and the perf oracle). They
share lib.rs and tests/sklearn.rs with the estimator seam, so the crate cannot be
committed in halves. Their `Transform` and `Outlier` traits extend this seam without
changing `Fit` or `Predict`. One of their fixture tests,
`the_imbalanced_metrics_match_scikit_learn`, is red on a fixture key that measure.json
does not yet carry — which is why this is a branch and not main.
2026-08-04 17:03:22 -07:00

172 lines
7.1 KiB
Rust

//! Statistical learning as values: fitting is a transformation, a model is a value,
//! predicting is a query.
//!
//! # The two verbs
//!
//! scikit-learn's surface is enormous and its load-bearing part is two verbs. This
//! crate is those two verbs — [`Fit`] and [`Predict`] — and nothing else. There is no
//! `get_params`, no `set_params`, no `**kwargs`, no `BaseEstimator`, no `Pipeline`, no
//! clone protocol, no tags dictionary. Those exist in Python because a config and a
//! fitted model are THE SAME OBJECT there, so Python needs a reflective protocol to
//! ask an object which of its attributes are hyperparameters and which are learned
//! state. Split the two apart and the entire protocol stops having a job.
//!
//! # What is separated from what
//!
//! ```text
//! Config hyperparameters. A plain value the caller writes down. Has no
//! predict, has no learned state, and cannot acquire either.
//! |
//! | fit(&Samples) Config -> Data -> Model
//! v
//! Model the fitted value: parameters, and the shape they are only
//! meaningful against. Immutable. Nameable ([`Address`]).
//! |
//! | predict(&Matrix) a QUERY against a value
//! v
//! Answer f64 for a regressor, [`Class`] for a classifier — a DIFFERENT
//! TYPE, which is what makes the two kinds of model different
//! kinds of thing rather than one thing with a mode flag.
//! ```
//!
//! # `check_is_fitted` is not implemented here, and cannot be needed
//!
//! sklearn ships `check_is_fitted` because `LinearRegression()` is a predictable-from
//! object before it has seen data, so every `predict` must open by asking whether the
//! object is in a state its own type permits but its mathematics does not. That is a
//! runtime tax on a type error.
//!
//! Here, a `Model` is produced only by [`Fit::fit`]. Every `Model` in this crate holds
//! its parameters in private fields with no public constructor, no `Default`, and no
//! mutator, so an unfitted one is not a state to be checked for — it is a value that
//! cannot be spoken. The error does not need handling because it cannot be written.
//!
//! # A model is a value, and it has a name
//!
//! [`Model::address`] is a content address over the model's shape, its
//! hyperparameters and its learned parameters — the same construction, and the same
//! reasoning, as the risk plane's published model values. Two fits that agree in
//! every term that changes an answer get one name; a fit under different
//! hyperparameters gets a different name even when the parameters land identically.
//! So a fitted model can be stored, cited by a decision, compared against a
//! challenger and rolled back to, without a registry handing out identifiers.
//!
//! # What is deliberately absent
//!
//! Multi-class logistic regression, sparse matrices, sample weights, cross
//! validation, pipelines and the `ai.onnx.ml` operator surface are all NOT here. Each
//! is named where it would go, and refused with a specific error where a caller could
//! otherwise believe it happened silently. An absent feature that says so is worth
//! more than one that guesses.
//!
//! # Licence
//!
//! Clean-room. The algorithms are the published literature (Householder 1958,
//! Friedman 2001, Breiman et al. 1984) and the numerical conventions are matched to
//! scikit-learn's BEHAVIOUR, observed through fixtures its own code produced. No
//! scikit-learn source or documentation text is reproduced here, so the BSD-3-Clause
//! attribution obligation is not triggered; the fixtures record which version of it
//! was consulted as an oracle.
#![forbid(unsafe_code)]
#![deny(missing_docs)]
mod address;
mod data;
mod error;
mod transform;
pub mod boost;
pub mod encode;
pub mod impute;
pub mod isolation;
pub mod linear;
pub mod logistic;
pub mod metric;
pub mod moment;
pub mod neighbour;
pub mod scale;
pub mod split;
pub mod tree;
pub mod twister;
pub use address::{Address, Digest};
pub use data::{Matrix, Samples};
pub use error::{Error, Result};
pub use transform::{Outlier, Transform};
/// Fitting: `Config -> Data -> Model`.
///
/// Implemented on CONFIG types, never on model types — a fitted model has nothing
/// left to fit, and refitting is producing a second value rather than mutating the
/// first. `&self` and not `self` because a config is a plain value worth reusing
/// across folds, and `fit` consumes nothing.
///
/// `T` is the target's own type, so the trait is what pairs a config with the kind of
/// question it can be asked: [`linear::Config`] fits against `f64` targets and
/// [`logistic::Config`] against `i64` labels, and neither can be handed the other's
/// data.
pub trait Fit<T> {
/// The value this config produces. It is the only way to obtain one.
type Model: Predict;
/// Fit one model value from these observations.
fn fit(&self, data: &Samples<T>) -> Result<Self::Model>;
}
/// Predicting: a QUERY against a fitted value.
///
/// [`Predict::Answer`] is the whole reason a regressor and a classifier are different
/// types. `Answer = f64` and `Answer = Class` do not unify, so no caller can write
/// code that treats one as the other, and no model needs a flag saying which it is.
pub trait Predict {
/// What one row's answer is.
type Answer;
/// Answer for every row of `x`.
///
/// Fails if `x` has a different number of features than the model was fitted on.
/// This is the one shape agreement this crate checks at runtime rather than in the
/// type: the feature count is not known until data is read, so putting it in the
/// type would mean a const generic on every matrix, every config and every model,
/// to catch a mismatch that occurs at exactly one call site.
fn predict(&self, x: &Matrix) -> Result<Vec<Self::Answer>>;
}
/// What every fitted value in this crate is: named by its content, and honest about
/// the shape its parameters are meaningful against.
pub trait Model {
/// This model's name, computed from its content.
///
/// A PURE function of the value: no clock, no identity, no counter. Two processes
/// holding the same model agree on the name without talking.
fn address(&self) -> Address;
/// How many features an input row must have.
fn features(&self) -> usize;
}
/// One class, as an index into the model that predicted it.
///
/// A caller cannot construct one. That is the point: a `Class` in hand is always a
/// class some model was actually fitted on, so the perennial confusion between "the
/// label 2" and "the third class" is not a bug that can be written. Resolve it to the
/// caller's own label with [`logistic::Model::label`].
///
/// Labels go IN raw (`&[i64]`, the caller's data) and come OUT model-owned. The
/// asymmetry is deliberate.
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Debug, Hash)]
pub struct Class(u32);
impl Class {
/// Position of this class in its model's own class list.
pub fn index(self) -> usize {
self.0 as usize
}
/// Only a fitted model mints these.
pub(crate) fn at(index: usize) -> Self {
Self(index as u32)
}
}