mirror of
https://github.com/hanzoai/ml.git
synced 2026-11-20 23:08:29 +00:00
A crate whose whole interface is two verbs. `Config` holds hyperparameters and has no `predict`. `Fit` maps `Config -> Data -> Model`. `Model` is immutable, has private fields and no public constructor, so an unfitted model is not a state to be checked for — it is a value that cannot be spoken, and `check_is_fitted` has no job to do. `Predict::Answer` is `f64` for a regressor and `Class` for a classifier: the two do not unify, so a regressor and a classifier are different types rather than one type with a mode flag. There is no `get_params`, no `BaseEstimator`, no clone protocol and no `Pipeline` — those exist in Python because a config and a fitted model are the same object there. Every fitted model is named by `Address`, a content address over its shape, its hyperparameters and its learned parameters, hashing floats as IEEE-754 bits and length-prefixing every sequence so the encoding is injective. Same construction and same reasoning as the risk plane's published model values, so a fit can be stored, cited by a decision, compared against a challenger and rolled back to without a registry. Three estimators: least squares by Householder QR on centred data (QR not the normal equations, which square the condition number; centring not a ones column, so the intercept never enters the factorisation), binary logistic regression by Newton steps on a strictly convex penalised objective, and gradient boosted CART regression trees. A TREE READS FEATURES AT IEEE-754 BINARY32, and that is the algorithm rather than a detail. scikit-learn casts X to float32 before its splitter sees it, XGBoost and LightGBM do the same, and ai.onnx.ml's TreeEnsembleRegressor stores nodes_values as floats. A tree never does arithmetic on a feature value — it only orders values and places thresholds between them — so precision decides only which side of a boundary a sample falls on. Placing the threshold at single(a)/2 + single(b)/2 instead of a/2 + b/2 moves it by 2e-8, which is enough to send a sample down the other branch. Measured over the 40 depth-1 trees of the stumps fixture, whose roots all see the full sample set: the single-precision midpoint reproduces scikit-learn's threshold bit-for-bit 40 times out of 40, the double-precision midpoint 0 times out of 40. Correctness is asserted against scikit-learn itself. Fixtures generated by 1.9.0 are committed and read as data, so `cargo test` needs no Python and the oracle's version is a recorded fact. 195 boosted trees / 1,977 split nodes agree node for node — split feature, threshold and both children all exact — with leaf values to 1.3e-15 and predictions to 8.9e-16 against a 1e-10 bound; least squares agrees to 1.1e-15 against 1e-10 and logistic to 1.2e-8 against 1e-7, with hard labels exact. Each tolerance is derived from the mathematics and every assertion prints the deviation it observed, so a regression that stays inside a bound is still visible. The generator refuses to record an oracle that is not a function of its data: at scikit-learn's default min_samples_leaf a tree's structure varies with random_state (20 structures over 40 seeds, prediction spread 3.9), so every case pins the tie away and proves determinism over 12 seeds before writing a number. This crate breaks split ties by lowest feature index instead, because a content address over a model an RNG helped choose would name one draw rather than one fit. The exhaustive split search is not lowered to hanzo-kernel, and the obstacle is specific: a launch geometry must be stated before the launch, and across all 69 launch sites in that crate Grid::Static is the only variant used, every geometry computed on the host from a count it already knows. A CART node's sample count is not such a number. The histogram formulation dissolves this — bin count is a hyperparameter, so the per-level geometry is host-knowable — but it quantises thresholds and so cannot hold the node-for-node bar. It is named, not built. scikit-learn is a test oracle only. No code or docstring is taken from it, and NOTICE records that, along with the obligation to add BSD-3 attribution if that ever changes. The crate also carries, unchanged and unclaimed, the anomaly-detection and preprocessing modules written concurrently on another track (isolation, neighbour, transform, scale, impute, encode, metric, moment, split, twister, plus benches and the perf oracle). They share lib.rs and tests/sklearn.rs with the estimator seam, so the crate cannot be committed in halves. Their `Transform` and `Outlier` traits extend this seam without changing `Fit` or `Predict`. One of their fixture tests, `the_imbalanced_metrics_match_scikit_learn`, is red on a fixture key that measure.json does not yet carry — which is why this is a branch and not main.
67 lines
3.3 KiB
Rust
67 lines
3.3 KiB
Rust
//! The unsupervised half of the seam.
|
|
|
|
use crate::{Matrix, Result};
|
|
|
|
/// A fitted value that rewrites a design matrix: a scaler, an encoding, a projection.
|
|
///
|
|
/// The counterpart to [`crate::Fit`] for estimators that have no target to fit against.
|
|
/// [`crate::Fit`] is parameterised by the target type because pairing a config with the
|
|
/// kind of question it can be asked is that trait's whole job; a transform has no target,
|
|
/// so there is nothing to pair and the trait carries no parameter.
|
|
///
|
|
/// Both widths are known before any data arrives, which is what lets a chain of transforms
|
|
/// be checked once when it is built rather than on every batch that flows through it.
|
|
///
|
|
/// # Why fitting is not part of this trait
|
|
///
|
|
/// `fit` is an associated function on each fitted type — [`crate::scale::Standard::fit`],
|
|
/// [`crate::scale::Range::fit`] — and not a trait method. Each takes its own arguments and
|
|
/// returns its own type, so a `Fit`-shaped trait over them would have exactly one
|
|
/// implementor per signature and no call site could ever dispatch through it. Fitting is a
|
|
/// constructor, and constructors do not share a type.
|
|
pub trait Transform {
|
|
/// How many features an input row must have: what it was fitted on.
|
|
fn features(&self) -> usize;
|
|
|
|
/// How many columns come out. Equal to [`Transform::features`] for a scaler, larger for
|
|
/// an expansion, smaller for a projection.
|
|
fn width(&self) -> usize;
|
|
|
|
/// Rewrite `x`, producing a fresh matrix of `x.n()` rows by [`Transform::width`]
|
|
/// columns.
|
|
///
|
|
/// Fails when `x` is not as wide as this value was fitted on. Named `apply` rather than
|
|
/// `transform` so that a call reads `scaler.apply(&x)`; scikit-learn's verb for it is
|
|
/// `transform`.
|
|
fn apply(&self, x: &Matrix) -> Result<Matrix>;
|
|
}
|
|
|
|
/// A fitted value that rates how unlike the fitted data each row is.
|
|
///
|
|
/// The third and last shape of estimator here, and the one an anomaly detector is:
|
|
/// [`crate::Fit`] needs a target, [`Transform`] rewrites a design, and this answers a
|
|
/// question about rows without either. [`crate::isolation::Forest`] and
|
|
/// [`crate::neighbour::Local`] are the implementors, and a caller can hold them behind this
|
|
/// trait precisely because that is the only thing they agree about — how they arrive at the
|
|
/// number could not be less alike.
|
|
///
|
|
/// # LARGER IS MORE ANOMALOUS, and why that is the opposite of scikit-learn
|
|
///
|
|
/// scikit-learn's `score_samples` returns larger for more NORMAL, because its whole API
|
|
/// obeys a "greater is better" convention inherited from scoring classifiers. That
|
|
/// convention costs a sign here: the quantity everyone actually wants is how suspicious a
|
|
/// row is, and a risk desk that feeds `score_samples` straight into a ranking gets its
|
|
/// alert queue upside down. So this trait reports suspicion directly, which is
|
|
/// `-score_samples` exactly, and the number composes with
|
|
/// [`crate::metric::Curve::of`] — whose positive class is the anomaly — with no negation at
|
|
/// the call site to forget.
|
|
pub trait Outlier {
|
|
/// How many features an input row must have: what it was fitted on.
|
|
fn features(&self) -> usize;
|
|
|
|
/// How anomalous each row of `x` is. Larger is more anomalous.
|
|
///
|
|
/// Fails when `x` is not as wide as this value was fitted on.
|
|
fn outlier(&self, x: &Matrix) -> Result<Vec<f64>>;
|
|
}
|