mirror of
https://github.com/hanzoai/ml.git
synced 2026-11-20 23:08:29 +00:00
A crate whose whole interface is two verbs. `Config` holds hyperparameters and has no `predict`. `Fit` maps `Config -> Data -> Model`. `Model` is immutable, has private fields and no public constructor, so an unfitted model is not a state to be checked for — it is a value that cannot be spoken, and `check_is_fitted` has no job to do. `Predict::Answer` is `f64` for a regressor and `Class` for a classifier: the two do not unify, so a regressor and a classifier are different types rather than one type with a mode flag. There is no `get_params`, no `BaseEstimator`, no clone protocol and no `Pipeline` — those exist in Python because a config and a fitted model are the same object there. Every fitted model is named by `Address`, a content address over its shape, its hyperparameters and its learned parameters, hashing floats as IEEE-754 bits and length-prefixing every sequence so the encoding is injective. Same construction and same reasoning as the risk plane's published model values, so a fit can be stored, cited by a decision, compared against a challenger and rolled back to without a registry. Three estimators: least squares by Householder QR on centred data (QR not the normal equations, which square the condition number; centring not a ones column, so the intercept never enters the factorisation), binary logistic regression by Newton steps on a strictly convex penalised objective, and gradient boosted CART regression trees. A TREE READS FEATURES AT IEEE-754 BINARY32, and that is the algorithm rather than a detail. scikit-learn casts X to float32 before its splitter sees it, XGBoost and LightGBM do the same, and ai.onnx.ml's TreeEnsembleRegressor stores nodes_values as floats. A tree never does arithmetic on a feature value — it only orders values and places thresholds between them — so precision decides only which side of a boundary a sample falls on. Placing the threshold at single(a)/2 + single(b)/2 instead of a/2 + b/2 moves it by 2e-8, which is enough to send a sample down the other branch. Measured over the 40 depth-1 trees of the stumps fixture, whose roots all see the full sample set: the single-precision midpoint reproduces scikit-learn's threshold bit-for-bit 40 times out of 40, the double-precision midpoint 0 times out of 40. Correctness is asserted against scikit-learn itself. Fixtures generated by 1.9.0 are committed and read as data, so `cargo test` needs no Python and the oracle's version is a recorded fact. 195 boosted trees / 1,977 split nodes agree node for node — split feature, threshold and both children all exact — with leaf values to 1.3e-15 and predictions to 8.9e-16 against a 1e-10 bound; least squares agrees to 1.1e-15 against 1e-10 and logistic to 1.2e-8 against 1e-7, with hard labels exact. Each tolerance is derived from the mathematics and every assertion prints the deviation it observed, so a regression that stays inside a bound is still visible. The generator refuses to record an oracle that is not a function of its data: at scikit-learn's default min_samples_leaf a tree's structure varies with random_state (20 structures over 40 seeds, prediction spread 3.9), so every case pins the tie away and proves determinism over 12 seeds before writing a number. This crate breaks split ties by lowest feature index instead, because a content address over a model an RNG helped choose would name one draw rather than one fit. The exhaustive split search is not lowered to hanzo-kernel, and the obstacle is specific: a launch geometry must be stated before the launch, and across all 69 launch sites in that crate Grid::Static is the only variant used, every geometry computed on the host from a count it already knows. A CART node's sample count is not such a number. The histogram formulation dissolves this — bin count is a hyperparameter, so the per-level geometry is host-knowable — but it quantises thresholds and so cannot hold the node-for-node bar. It is named, not built. scikit-learn is a test oracle only. No code or docstring is taken from it, and NOTICE records that, along with the obligation to add BSD-3 attribution if that ever changes. The crate also carries, unchanged and unclaimed, the anomaly-detection and preprocessing modules written concurrently on another track (isolation, neighbour, transform, scale, impute, encode, metric, moment, split, twister, plus benches and the perf oracle). They share lib.rs and tests/sklearn.rs with the estimator seam, so the crate cannot be committed in halves. Their `Transform` and `Outlier` traits extend this seam without changing `Fit` or `Predict`. One of their fixture tests, `the_imbalanced_metrics_match_scikit_learn`, is red on a fixture key that measure.json does not yet carry — which is why this is a branch and not main.
172 lines
7.1 KiB
Rust
172 lines
7.1 KiB
Rust
//! Statistical learning as values: fitting is a transformation, a model is a value,
|
|
//! predicting is a query.
|
|
//!
|
|
//! # The two verbs
|
|
//!
|
|
//! scikit-learn's surface is enormous and its load-bearing part is two verbs. This
|
|
//! crate is those two verbs — [`Fit`] and [`Predict`] — and nothing else. There is no
|
|
//! `get_params`, no `set_params`, no `**kwargs`, no `BaseEstimator`, no `Pipeline`, no
|
|
//! clone protocol, no tags dictionary. Those exist in Python because a config and a
|
|
//! fitted model are THE SAME OBJECT there, so Python needs a reflective protocol to
|
|
//! ask an object which of its attributes are hyperparameters and which are learned
|
|
//! state. Split the two apart and the entire protocol stops having a job.
|
|
//!
|
|
//! # What is separated from what
|
|
//!
|
|
//! ```text
|
|
//! Config hyperparameters. A plain value the caller writes down. Has no
|
|
//! predict, has no learned state, and cannot acquire either.
|
|
//! |
|
|
//! | fit(&Samples) Config -> Data -> Model
|
|
//! v
|
|
//! Model the fitted value: parameters, and the shape they are only
|
|
//! meaningful against. Immutable. Nameable ([`Address`]).
|
|
//! |
|
|
//! | predict(&Matrix) a QUERY against a value
|
|
//! v
|
|
//! Answer f64 for a regressor, [`Class`] for a classifier — a DIFFERENT
|
|
//! TYPE, which is what makes the two kinds of model different
|
|
//! kinds of thing rather than one thing with a mode flag.
|
|
//! ```
|
|
//!
|
|
//! # `check_is_fitted` is not implemented here, and cannot be needed
|
|
//!
|
|
//! sklearn ships `check_is_fitted` because `LinearRegression()` is a predictable-from
|
|
//! object before it has seen data, so every `predict` must open by asking whether the
|
|
//! object is in a state its own type permits but its mathematics does not. That is a
|
|
//! runtime tax on a type error.
|
|
//!
|
|
//! Here, a `Model` is produced only by [`Fit::fit`]. Every `Model` in this crate holds
|
|
//! its parameters in private fields with no public constructor, no `Default`, and no
|
|
//! mutator, so an unfitted one is not a state to be checked for — it is a value that
|
|
//! cannot be spoken. The error does not need handling because it cannot be written.
|
|
//!
|
|
//! # A model is a value, and it has a name
|
|
//!
|
|
//! [`Model::address`] is a content address over the model's shape, its
|
|
//! hyperparameters and its learned parameters — the same construction, and the same
|
|
//! reasoning, as the risk plane's published model values. Two fits that agree in
|
|
//! every term that changes an answer get one name; a fit under different
|
|
//! hyperparameters gets a different name even when the parameters land identically.
|
|
//! So a fitted model can be stored, cited by a decision, compared against a
|
|
//! challenger and rolled back to, without a registry handing out identifiers.
|
|
//!
|
|
//! # What is deliberately absent
|
|
//!
|
|
//! Multi-class logistic regression, sparse matrices, sample weights, cross
|
|
//! validation, pipelines and the `ai.onnx.ml` operator surface are all NOT here. Each
|
|
//! is named where it would go, and refused with a specific error where a caller could
|
|
//! otherwise believe it happened silently. An absent feature that says so is worth
|
|
//! more than one that guesses.
|
|
//!
|
|
//! # Licence
|
|
//!
|
|
//! Clean-room. The algorithms are the published literature (Householder 1958,
|
|
//! Friedman 2001, Breiman et al. 1984) and the numerical conventions are matched to
|
|
//! scikit-learn's BEHAVIOUR, observed through fixtures its own code produced. No
|
|
//! scikit-learn source or documentation text is reproduced here, so the BSD-3-Clause
|
|
//! attribution obligation is not triggered; the fixtures record which version of it
|
|
//! was consulted as an oracle.
|
|
|
|
#![forbid(unsafe_code)]
|
|
#![deny(missing_docs)]
|
|
|
|
mod address;
|
|
mod data;
|
|
mod error;
|
|
mod transform;
|
|
|
|
pub mod boost;
|
|
pub mod encode;
|
|
pub mod impute;
|
|
pub mod isolation;
|
|
pub mod linear;
|
|
pub mod logistic;
|
|
pub mod metric;
|
|
pub mod moment;
|
|
pub mod neighbour;
|
|
pub mod scale;
|
|
pub mod split;
|
|
pub mod tree;
|
|
pub mod twister;
|
|
|
|
pub use address::{Address, Digest};
|
|
pub use data::{Matrix, Samples};
|
|
pub use error::{Error, Result};
|
|
pub use transform::{Outlier, Transform};
|
|
|
|
/// Fitting: `Config -> Data -> Model`.
|
|
///
|
|
/// Implemented on CONFIG types, never on model types — a fitted model has nothing
|
|
/// left to fit, and refitting is producing a second value rather than mutating the
|
|
/// first. `&self` and not `self` because a config is a plain value worth reusing
|
|
/// across folds, and `fit` consumes nothing.
|
|
///
|
|
/// `T` is the target's own type, so the trait is what pairs a config with the kind of
|
|
/// question it can be asked: [`linear::Config`] fits against `f64` targets and
|
|
/// [`logistic::Config`] against `i64` labels, and neither can be handed the other's
|
|
/// data.
|
|
pub trait Fit<T> {
|
|
/// The value this config produces. It is the only way to obtain one.
|
|
type Model: Predict;
|
|
|
|
/// Fit one model value from these observations.
|
|
fn fit(&self, data: &Samples<T>) -> Result<Self::Model>;
|
|
}
|
|
|
|
/// Predicting: a QUERY against a fitted value.
|
|
///
|
|
/// [`Predict::Answer`] is the whole reason a regressor and a classifier are different
|
|
/// types. `Answer = f64` and `Answer = Class` do not unify, so no caller can write
|
|
/// code that treats one as the other, and no model needs a flag saying which it is.
|
|
pub trait Predict {
|
|
/// What one row's answer is.
|
|
type Answer;
|
|
|
|
/// Answer for every row of `x`.
|
|
///
|
|
/// Fails if `x` has a different number of features than the model was fitted on.
|
|
/// This is the one shape agreement this crate checks at runtime rather than in the
|
|
/// type: the feature count is not known until data is read, so putting it in the
|
|
/// type would mean a const generic on every matrix, every config and every model,
|
|
/// to catch a mismatch that occurs at exactly one call site.
|
|
fn predict(&self, x: &Matrix) -> Result<Vec<Self::Answer>>;
|
|
}
|
|
|
|
/// What every fitted value in this crate is: named by its content, and honest about
|
|
/// the shape its parameters are meaningful against.
|
|
pub trait Model {
|
|
/// This model's name, computed from its content.
|
|
///
|
|
/// A PURE function of the value: no clock, no identity, no counter. Two processes
|
|
/// holding the same model agree on the name without talking.
|
|
fn address(&self) -> Address;
|
|
|
|
/// How many features an input row must have.
|
|
fn features(&self) -> usize;
|
|
}
|
|
|
|
/// One class, as an index into the model that predicted it.
|
|
///
|
|
/// A caller cannot construct one. That is the point: a `Class` in hand is always a
|
|
/// class some model was actually fitted on, so the perennial confusion between "the
|
|
/// label 2" and "the third class" is not a bug that can be written. Resolve it to the
|
|
/// caller's own label with [`logistic::Model::label`].
|
|
///
|
|
/// Labels go IN raw (`&[i64]`, the caller's data) and come OUT model-owned. The
|
|
/// asymmetry is deliberate.
|
|
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Debug, Hash)]
|
|
pub struct Class(u32);
|
|
|
|
impl Class {
|
|
/// Position of this class in its model's own class list.
|
|
pub fn index(self) -> usize {
|
|
self.0 as usize
|
|
}
|
|
|
|
/// Only a fitted model mints these.
|
|
pub(crate) fn at(index: usize) -> Self {
|
|
Self(index as u32)
|
|
}
|
|
}
|