Files
zeekay 026ac25fcf hanzo-learn: fitting is a transformation, a model is a value, predicting is a query
A crate whose whole interface is two verbs. `Config` holds hyperparameters and has no
`predict`. `Fit` maps `Config -> Data -> Model`. `Model` is immutable, has private fields
and no public constructor, so an unfitted model is not a state to be checked for — it is
a value that cannot be spoken, and `check_is_fitted` has no job to do. `Predict::Answer`
is `f64` for a regressor and `Class` for a classifier: the two do not unify, so a
regressor and a classifier are different types rather than one type with a mode flag.
There is no `get_params`, no `BaseEstimator`, no clone protocol and no `Pipeline` — those
exist in Python because a config and a fitted model are the same object there.

Every fitted model is named by `Address`, a content address over its shape, its
hyperparameters and its learned parameters, hashing floats as IEEE-754 bits and
length-prefixing every sequence so the encoding is injective. Same construction and same
reasoning as the risk plane's published model values, so a fit can be stored, cited by a
decision, compared against a challenger and rolled back to without a registry.

Three estimators: least squares by Householder QR on centred data (QR not the normal
equations, which square the condition number; centring not a ones column, so the
intercept never enters the factorisation), binary logistic regression by Newton steps on
a strictly convex penalised objective, and gradient boosted CART regression trees.

A TREE READS FEATURES AT IEEE-754 BINARY32, and that is the algorithm rather than a
detail. scikit-learn casts X to float32 before its splitter sees it, XGBoost and LightGBM
do the same, and ai.onnx.ml's TreeEnsembleRegressor stores nodes_values as floats. A tree
never does arithmetic on a feature value — it only orders values and places thresholds
between them — so precision decides only which side of a boundary a sample falls on.
Placing the threshold at single(a)/2 + single(b)/2 instead of a/2 + b/2 moves it by 2e-8,
which is enough to send a sample down the other branch. Measured over the 40 depth-1
trees of the stumps fixture, whose roots all see the full sample set: the single-precision
midpoint reproduces scikit-learn's threshold bit-for-bit 40 times out of 40, the
double-precision midpoint 0 times out of 40.

Correctness is asserted against scikit-learn itself. Fixtures generated by 1.9.0 are
committed and read as data, so `cargo test` needs no Python and the oracle's version is a
recorded fact. 195 boosted trees / 1,977 split nodes agree node for node — split feature,
threshold and both children all exact — with leaf values to 1.3e-15 and predictions to
8.9e-16 against a 1e-10 bound; least squares agrees to 1.1e-15 against 1e-10 and logistic
to 1.2e-8 against 1e-7, with hard labels exact. Each tolerance is derived from the
mathematics and every assertion prints the deviation it observed, so a regression that
stays inside a bound is still visible. The generator refuses to record an oracle that is
not a function of its data: at scikit-learn's default min_samples_leaf a tree's structure
varies with random_state (20 structures over 40 seeds, prediction spread 3.9), so every
case pins the tie away and proves determinism over 12 seeds before writing a number. This
crate breaks split ties by lowest feature index instead, because a content address over a
model an RNG helped choose would name one draw rather than one fit.

The exhaustive split search is not lowered to hanzo-kernel, and the obstacle is specific:
a launch geometry must be stated before the launch, and across all 69 launch sites in
that crate Grid::Static is the only variant used, every geometry computed on the host from
a count it already knows. A CART node's sample count is not such a number. The histogram
formulation dissolves this — bin count is a hyperparameter, so the per-level geometry is
host-knowable — but it quantises thresholds and so cannot hold the node-for-node bar. It
is named, not built.

scikit-learn is a test oracle only. No code or docstring is taken from it, and NOTICE
records that, along with the obligation to add BSD-3 attribution if that ever changes.

The crate also carries, unchanged and unclaimed, the anomaly-detection and preprocessing
modules written concurrently on another track (isolation, neighbour, transform, scale,
impute, encode, metric, moment, split, twister, plus benches and the perf oracle). They
share lib.rs and tests/sklearn.rs with the estimator seam, so the crate cannot be
committed in halves. Their `Transform` and `Outlier` traits extend this seam without
changing `Fit` or `Predict`. One of their fixture tests,
`the_imbalanced_metrics_match_scikit_learn`, is red on a fixture key that measure.json
does not yet carry — which is why this is a branch and not main.
2026-08-04 17:03:22 -07:00

116 lines
6.2 KiB
Plaintext

Hanzo ML
Copyright (c) 2024 Hanzo AI, Inc.
This product includes software from candle (https://github.com/huggingface/candle),
licensed under the MIT License or the Apache License 2.0:
Copyright (c) The candle authors
The upstream candle license texts are retained in this repository as LICENSE-MIT
and LICENSE-APACHE. This work is a derivative of candle and remains dual-licensed
under "MIT OR Apache-2.0"; the upstream license has not been changed.
This product also includes software PORTED FROM llama.cpp / ggml
(https://github.com/ggml-org/llama.cpp), licensed under the MIT License:
MIT License
Copyright (c) 2023-2026 The ggml authors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Ported from https://github.com/ggml-org/llama.cpp at commit
0821c5fcfd729af70037bc1e9e60769d42c081ba. These are ports, not reimplementations:
the GGUF/GGML on-disk quantization formats and their reference decode define the
bit layout our kernels must reproduce, so the upstream source is the specification.
The ported material is quantization-related and spans hanzo-ml, hanzo-kernel,
hanzo-kernels, hanzo-metal-kernels, hanzo-rocm-kernels, hanzo-transformers and
tensor-tools; the per-file provenance is recorded in source comments naming the
upstream file (and, where useful, the line). It includes at least:
- GGUF/GGML block formats and their CPU reference decode (quantized/k_quants.rs,
ggml_file.rs, gguf_file.rs) -- e.g. kvalues_mxfp4 from ggml-common.h and a
bit-for-bit replica of ggml_e8m0_to_fp32_half from ggml-impl.h.
- The IQ-quant codebook grids and sign table (quantized/iq_grids.rs), extracted
verbatim from ggml/src/ggml-common.h, and the CUDA MMQ codebook tables.
- GPU quantized matmul/matvec kernels whose structure follows the corresponding
ggml kernels (the q8_1 block-quantized dot-product decode used by the CUDA/ROCm
dp4a cores and the Vulkan dp4a shaders, and the expert-grouped MoE GEMM).
Copyright in the ported material remains with The ggml authors; Hanzo AI claims
no copyright over it. Hanzo's own contributions around it (the backends, the
kernel DSL, and the original kernels) are covered by the Hanzo copyright above.
llama.cpp ships no NOTICE file; there are no further attributions to propagate.
PROVENANCE
Forked from https://github.com/huggingface/candle. The candle sources are vendored
and renamed to hanzo-* crates, so no pinned candle crate version is declared; this
fork tracks upstream candle main. The nearest upstream reference point on this line
of development is candle commit 3091caf9 (upstream PR #3408).
Upstream candle ships no NOTICE file; no Apache-2.0 section 4(d) attributions to
propagate.
DEVIATIONS
Changes from upstream candle in this fork:
- Rebranded to "Hanzo ML"; the top-level crate candle-core is renamed to hanzo-ml
(core module hanzo_ml_core).
- Companion crates renamed: candle-nn -> hanzo-nn, candle-transformers ->
hanzo-transformers, candle-datasets -> hanzo-datasets, candle-onnx -> hanzo-onnx,
candle-flash-attn -> hanzo-flash-attn, candle-kernels -> hanzo-kernels,
candle-metal-kernels -> hanzo-metal-kernels, candle-pyo3 -> hanzo-ml-pyo3,
candle-examples -> hanzo-ml-examples, candle-book -> hanzo-ml-book.
- Added a first-party GPU kernel DSL (hanzo-kernel + hanzo-kernel-macros): one Rust
source lowered to CUDA, ROCm, Vulkan, and Metal.
- Added a ROCm/HIP backend and kernels (hanzo-rocm-kernels; rocm-rs vendored and
renamed to the hanzo-rocm package; hanzo-cubecl-hip fork).
- Added a Vulkan compute backend.
- Added FlashAttention-3 for Hopper/sm90 (hanzo-flash-attn-v3).
- Added NVIDIA GB10 / DGX Spark unified/managed-memory support in the CUDA backend.
- Extended quantization: the full GGUF quant zoo (Q/K, legacy, IQ1-4, TQ; 22 types)
decodes resident and bit-exact on AMD gfx1151 through one unified compute core.
- Added hanzo-3d (generative-3D geometry and mesh I/O: Pixal3D / TRELLIS / TripoSR)
and native-Rust training crates (hanzo-train, hanzo-training) targeting the
zen-agentic and zen-identity datasets and DSpark speculative-draft models.
- Integrated with Hanzo Engine (OpenAI + Anthropic + MCP serving APIs) and the
Hanzo/zen model hubs.
- Dependency and layout changes: cudarc pinned at 0.19.4; an orphaned duplicate
crate cluster is excluded from the workspace build.
- Added hanzo-learn (statistical learning: least squares, logistic regression,
gradient boosted regression trees).
hanzo-learn and scikit-learn
----------------------------
hanzo-learn is a CLEAN-ROOM implementation. Its algorithms come from the published
literature (Householder 1958; Breiman, Friedman, Olshen & Stone 1984; Friedman 2001)
and no scikit-learn source code or documentation text is reproduced in it.
scikit-learn (https://scikit-learn.org, BSD-3-Clause) is used as a TEST ORACLE: the
fixtures under hanzo-learn/tests/fixture/ hold numerical results produced by running
it (version recorded in each fixture), and hanzo-learn's own results are asserted
against them. The generator that produced them is committed beside them.
Because no scikit-learn code is included or distributed here, its BSD-3-Clause notice
is not carried. If any scikit-learn code or docstring is ever incorporated, that
attribution must be added to this file at the same time.