mirror of
https://github.com/hanzoai/ml.git
synced 2026-11-20 23:08:29 +00:00
A crate whose whole interface is two verbs. `Config` holds hyperparameters and has no `predict`. `Fit` maps `Config -> Data -> Model`. `Model` is immutable, has private fields and no public constructor, so an unfitted model is not a state to be checked for — it is a value that cannot be spoken, and `check_is_fitted` has no job to do. `Predict::Answer` is `f64` for a regressor and `Class` for a classifier: the two do not unify, so a regressor and a classifier are different types rather than one type with a mode flag. There is no `get_params`, no `BaseEstimator`, no clone protocol and no `Pipeline` — those exist in Python because a config and a fitted model are the same object there. Every fitted model is named by `Address`, a content address over its shape, its hyperparameters and its learned parameters, hashing floats as IEEE-754 bits and length-prefixing every sequence so the encoding is injective. Same construction and same reasoning as the risk plane's published model values, so a fit can be stored, cited by a decision, compared against a challenger and rolled back to without a registry. Three estimators: least squares by Householder QR on centred data (QR not the normal equations, which square the condition number; centring not a ones column, so the intercept never enters the factorisation), binary logistic regression by Newton steps on a strictly convex penalised objective, and gradient boosted CART regression trees. A TREE READS FEATURES AT IEEE-754 BINARY32, and that is the algorithm rather than a detail. scikit-learn casts X to float32 before its splitter sees it, XGBoost and LightGBM do the same, and ai.onnx.ml's TreeEnsembleRegressor stores nodes_values as floats. A tree never does arithmetic on a feature value — it only orders values and places thresholds between them — so precision decides only which side of a boundary a sample falls on. Placing the threshold at single(a)/2 + single(b)/2 instead of a/2 + b/2 moves it by 2e-8, which is enough to send a sample down the other branch. Measured over the 40 depth-1 trees of the stumps fixture, whose roots all see the full sample set: the single-precision midpoint reproduces scikit-learn's threshold bit-for-bit 40 times out of 40, the double-precision midpoint 0 times out of 40. Correctness is asserted against scikit-learn itself. Fixtures generated by 1.9.0 are committed and read as data, so `cargo test` needs no Python and the oracle's version is a recorded fact. 195 boosted trees / 1,977 split nodes agree node for node — split feature, threshold and both children all exact — with leaf values to 1.3e-15 and predictions to 8.9e-16 against a 1e-10 bound; least squares agrees to 1.1e-15 against 1e-10 and logistic to 1.2e-8 against 1e-7, with hard labels exact. Each tolerance is derived from the mathematics and every assertion prints the deviation it observed, so a regression that stays inside a bound is still visible. The generator refuses to record an oracle that is not a function of its data: at scikit-learn's default min_samples_leaf a tree's structure varies with random_state (20 structures over 40 seeds, prediction spread 3.9), so every case pins the tie away and proves determinism over 12 seeds before writing a number. This crate breaks split ties by lowest feature index instead, because a content address over a model an RNG helped choose would name one draw rather than one fit. The exhaustive split search is not lowered to hanzo-kernel, and the obstacle is specific: a launch geometry must be stated before the launch, and across all 69 launch sites in that crate Grid::Static is the only variant used, every geometry computed on the host from a count it already knows. A CART node's sample count is not such a number. The histogram formulation dissolves this — bin count is a hyperparameter, so the per-level geometry is host-knowable — but it quantises thresholds and so cannot hold the node-for-node bar. It is named, not built. scikit-learn is a test oracle only. No code or docstring is taken from it, and NOTICE records that, along with the obligation to add BSD-3 attribution if that ever changes. The crate also carries, unchanged and unclaimed, the anomaly-detection and preprocessing modules written concurrently on another track (isolation, neighbour, transform, scale, impute, encode, metric, moment, split, twister, plus benches and the perf oracle). They share lib.rs and tests/sklearn.rs with the estimator seam, so the crate cannot be committed in halves. Their `Transform` and `Outlier` traits extend this seam without changing `Fit` or `Predict`. One of their fixture tests, `the_imbalanced_metrics_match_scikit_learn`, is red on a fixture key that measure.json does not yet carry — which is why this is a branch and not main.
190 lines
6.2 KiB
Rust
190 lines
6.2 KiB
Rust
//! A fitted model's NAME, computed from its content.
|
|
//!
|
|
//! This is the risk plane's construction (`cloud/apps/risk/address.go`) applied to
|
|
//! fitted estimators, and the reasoning transfers unchanged:
|
|
//!
|
|
//! * The address covers everything that makes two models answer the same input
|
|
//! differently, and NOTHING else. The hyperparameters are in it even though they
|
|
//! are not consulted at prediction time, because two models fitted under different
|
|
//! configurations are two different values even in the rare case their parameters
|
|
//! coincide — a name that ignored the config would call them one.
|
|
//!
|
|
//! * Floats are hashed as IEEE-754 BITS and never as text. A decimal rendering is a
|
|
//! lossy function of an `f64`, so two runs holding the same number could print it
|
|
//! differently and be named apart.
|
|
//!
|
|
//! * No tenant, no clock, no counter. It is a pure function of the value, so two
|
|
//! processes agree on the name without talking and publication is idempotent.
|
|
//! Isolation is a predicate held elsewhere; a name is never an authority.
|
|
//!
|
|
//! # Length prefixes are load-bearing
|
|
//!
|
|
//! Every sequence is written with its length ahead of it. Without that, concatenation
|
|
//! is ambiguous: `[[1],[2,3]]` and `[[1,2],[3]]` flatten to the same bytes, so two
|
|
//! models of genuinely different shape would collide. The prefix is what makes the
|
|
//! encoding injective, and injectivity is the whole property a content address rests
|
|
//! on.
|
|
|
|
use sha2::{Digest as _, Sha256};
|
|
|
|
/// A model value's name: SHA-256 over its content.
|
|
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
|
|
pub struct Address([u8; 32]);
|
|
|
|
impl Address {
|
|
/// The raw digest.
|
|
pub fn bytes(&self) -> [u8; 32] {
|
|
self.0
|
|
}
|
|
|
|
/// Lower-case hex, which is the form a store keys on and a report cites.
|
|
pub fn hex(&self) -> String {
|
|
let mut s = String::with_capacity(64);
|
|
for b in self.0 {
|
|
s.push(char::from_digit((b >> 4) as u32, 16).unwrap());
|
|
s.push(char::from_digit((b & 0xf) as u32, 16).unwrap());
|
|
}
|
|
s
|
|
}
|
|
}
|
|
|
|
impl std::fmt::Display for Address {
|
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
|
f.write_str(&self.hex())
|
|
}
|
|
}
|
|
|
|
impl std::fmt::Debug for Address {
|
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
|
std::fmt::Display::fmt(self, f)
|
|
}
|
|
}
|
|
|
|
/// Builds one [`Address`].
|
|
///
|
|
/// ONE implementation, shared by every model in this crate, so the domain separation,
|
|
/// the float encoding and the length prefixing are a single decision rather than three
|
|
/// that happen to agree today.
|
|
pub struct Digest(Sha256);
|
|
|
|
impl Digest {
|
|
/// Open a digest in one model kind's own domain.
|
|
///
|
|
/// The domain is separated with a NUL, so a digest from `linear` can never be
|
|
/// mistaken for one from `boost` — even if their remaining bytes coincided, which
|
|
/// for a two-parameter linear model and a one-node tree is not far-fetched.
|
|
pub fn new(domain: &str) -> Self {
|
|
let mut h = Sha256::new();
|
|
h.update(domain.as_bytes());
|
|
h.update([0u8]);
|
|
Self(h)
|
|
}
|
|
|
|
/// A count or a dimension.
|
|
pub fn size(mut self, v: usize) -> Self {
|
|
self.0.update((v as u64).to_be_bytes());
|
|
self
|
|
}
|
|
|
|
/// An integer — a class label, or a hyperparameter that counts.
|
|
pub fn int(mut self, v: i64) -> Self {
|
|
self.0.update(v.to_be_bytes());
|
|
self
|
|
}
|
|
|
|
/// A flag.
|
|
pub fn flag(mut self, v: bool) -> Self {
|
|
self.0.update([u8::from(v)]);
|
|
self
|
|
}
|
|
|
|
/// A real number, as its bits.
|
|
pub fn real(mut self, v: f64) -> Self {
|
|
self.0.update(v.to_bits().to_be_bytes());
|
|
self
|
|
}
|
|
|
|
/// A sequence of real numbers, length-prefixed.
|
|
pub fn reals(mut self, v: &[f64]) -> Self {
|
|
self.0.update((v.len() as u64).to_be_bytes());
|
|
for x in v {
|
|
self.0.update(x.to_bits().to_be_bytes());
|
|
}
|
|
self
|
|
}
|
|
|
|
/// A sequence of integers, length-prefixed.
|
|
pub fn ints(mut self, v: &[i64]) -> Self {
|
|
self.0.update((v.len() as u64).to_be_bytes());
|
|
for x in v {
|
|
self.0.update(x.to_be_bytes());
|
|
}
|
|
self
|
|
}
|
|
|
|
/// A sequence of sizes, length-prefixed.
|
|
pub fn sizes(mut self, v: &[usize]) -> Self {
|
|
self.0.update((v.len() as u64).to_be_bytes());
|
|
for x in v {
|
|
self.0.update((*x as u64).to_be_bytes());
|
|
}
|
|
self
|
|
}
|
|
|
|
/// Seal it.
|
|
pub fn finish(self) -> Address {
|
|
Address(self.0.finalize().into())
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn an_address_is_sixty_four_hex_characters() {
|
|
let a = Digest::new("hanzo.learn.test").real(1.5).finish();
|
|
assert_eq!(a.hex().len(), 64);
|
|
assert!(a
|
|
.hex()
|
|
.chars()
|
|
.all(|c| c.is_ascii_hexdigit() && !c.is_uppercase()));
|
|
}
|
|
|
|
#[test]
|
|
fn the_domain_separates() {
|
|
let a = Digest::new("hanzo.learn.linear").reals(&[1.0]).finish();
|
|
let b = Digest::new("hanzo.learn.boost").reals(&[1.0]).finish();
|
|
assert_ne!(a, b);
|
|
}
|
|
|
|
#[test]
|
|
fn length_prefixes_make_the_encoding_injective() {
|
|
// The claim this file rests on: two different shapes cannot flatten to one
|
|
// name. Without the prefix these two are the same byte stream.
|
|
let a = Digest::new("d").reals(&[1.0]).reals(&[2.0, 3.0]).finish();
|
|
let b = Digest::new("d").reals(&[1.0, 2.0]).reals(&[3.0]).finish();
|
|
assert_ne!(a, b);
|
|
}
|
|
|
|
#[test]
|
|
fn zero_and_minus_zero_are_different_bit_patterns_and_stay_different_names() {
|
|
// They compare equal as numbers, so a text encoding could easily fold them
|
|
// together. They are different bits and this is a bit-level encoding.
|
|
assert_ne!(
|
|
Digest::new("d").real(0.0).finish(),
|
|
Digest::new("d").real(-0.0).finish()
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn a_nan_names_itself_consistently() {
|
|
// Floats that are not equal to themselves would break a naming scheme built on
|
|
// numeric comparison. This one is built on bits, so it does not care.
|
|
assert_eq!(
|
|
Digest::new("d").real(f64::NAN).finish(),
|
|
Digest::new("d").real(f64::NAN).finish()
|
|
);
|
|
}
|
|
}
|