mirror of
https://github.com/hanzoai/ml.git
synced 2026-11-20 23:08:29 +00:00
A crate whose whole interface is two verbs. `Config` holds hyperparameters and has no `predict`. `Fit` maps `Config -> Data -> Model`. `Model` is immutable, has private fields and no public constructor, so an unfitted model is not a state to be checked for — it is a value that cannot be spoken, and `check_is_fitted` has no job to do. `Predict::Answer` is `f64` for a regressor and `Class` for a classifier: the two do not unify, so a regressor and a classifier are different types rather than one type with a mode flag. There is no `get_params`, no `BaseEstimator`, no clone protocol and no `Pipeline` — those exist in Python because a config and a fitted model are the same object there. Every fitted model is named by `Address`, a content address over its shape, its hyperparameters and its learned parameters, hashing floats as IEEE-754 bits and length-prefixing every sequence so the encoding is injective. Same construction and same reasoning as the risk plane's published model values, so a fit can be stored, cited by a decision, compared against a challenger and rolled back to without a registry. Three estimators: least squares by Householder QR on centred data (QR not the normal equations, which square the condition number; centring not a ones column, so the intercept never enters the factorisation), binary logistic regression by Newton steps on a strictly convex penalised objective, and gradient boosted CART regression trees. A TREE READS FEATURES AT IEEE-754 BINARY32, and that is the algorithm rather than a detail. scikit-learn casts X to float32 before its splitter sees it, XGBoost and LightGBM do the same, and ai.onnx.ml's TreeEnsembleRegressor stores nodes_values as floats. A tree never does arithmetic on a feature value — it only orders values and places thresholds between them — so precision decides only which side of a boundary a sample falls on. Placing the threshold at single(a)/2 + single(b)/2 instead of a/2 + b/2 moves it by 2e-8, which is enough to send a sample down the other branch. Measured over the 40 depth-1 trees of the stumps fixture, whose roots all see the full sample set: the single-precision midpoint reproduces scikit-learn's threshold bit-for-bit 40 times out of 40, the double-precision midpoint 0 times out of 40. Correctness is asserted against scikit-learn itself. Fixtures generated by 1.9.0 are committed and read as data, so `cargo test` needs no Python and the oracle's version is a recorded fact. 195 boosted trees / 1,977 split nodes agree node for node — split feature, threshold and both children all exact — with leaf values to 1.3e-15 and predictions to 8.9e-16 against a 1e-10 bound; least squares agrees to 1.1e-15 against 1e-10 and logistic to 1.2e-8 against 1e-7, with hard labels exact. Each tolerance is derived from the mathematics and every assertion prints the deviation it observed, so a regression that stays inside a bound is still visible. The generator refuses to record an oracle that is not a function of its data: at scikit-learn's default min_samples_leaf a tree's structure varies with random_state (20 structures over 40 seeds, prediction spread 3.9), so every case pins the tie away and proves determinism over 12 seeds before writing a number. This crate breaks split ties by lowest feature index instead, because a content address over a model an RNG helped choose would name one draw rather than one fit. The exhaustive split search is not lowered to hanzo-kernel, and the obstacle is specific: a launch geometry must be stated before the launch, and across all 69 launch sites in that crate Grid::Static is the only variant used, every geometry computed on the host from a count it already knows. A CART node's sample count is not such a number. The histogram formulation dissolves this — bin count is a hyperparameter, so the per-level geometry is host-knowable — but it quantises thresholds and so cannot hold the node-for-node bar. It is named, not built. scikit-learn is a test oracle only. No code or docstring is taken from it, and NOTICE records that, along with the obligation to add BSD-3 attribution if that ever changes. The crate also carries, unchanged and unclaimed, the anomaly-detection and preprocessing modules written concurrently on another track (isolation, neighbour, transform, scale, impute, encode, metric, moment, split, twister, plus benches and the perf oracle). They share lib.rs and tests/sklearn.rs with the estimator seam, so the crate cannot be committed in halves. Their `Transform` and `Outlier` traits extend this seam without changing `Fit` or `Predict`. One of their fixture tests, `the_imbalanced_metrics_match_scikit_learn`, is red on a fixture key that measure.json does not yet carry — which is why this is a branch and not main.
116 lines
6.2 KiB
Plaintext
116 lines
6.2 KiB
Plaintext
Hanzo ML
|
|
Copyright (c) 2024 Hanzo AI, Inc.
|
|
|
|
This product includes software from candle (https://github.com/huggingface/candle),
|
|
licensed under the MIT License or the Apache License 2.0:
|
|
|
|
Copyright (c) The candle authors
|
|
|
|
The upstream candle license texts are retained in this repository as LICENSE-MIT
|
|
and LICENSE-APACHE. This work is a derivative of candle and remains dual-licensed
|
|
under "MIT OR Apache-2.0"; the upstream license has not been changed.
|
|
|
|
This product also includes software PORTED FROM llama.cpp / ggml
|
|
(https://github.com/ggml-org/llama.cpp), licensed under the MIT License:
|
|
|
|
MIT License
|
|
|
|
Copyright (c) 2023-2026 The ggml authors
|
|
|
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
of this software and associated documentation files (the "Software"), to deal
|
|
in the Software without restriction, including without limitation the rights
|
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
copies of the Software, and to permit persons to whom the Software is
|
|
furnished to do so, subject to the following conditions:
|
|
|
|
The above copyright notice and this permission notice shall be included in all
|
|
copies or substantial portions of the Software.
|
|
|
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
SOFTWARE.
|
|
|
|
Ported from https://github.com/ggml-org/llama.cpp at commit
|
|
0821c5fcfd729af70037bc1e9e60769d42c081ba. These are ports, not reimplementations:
|
|
the GGUF/GGML on-disk quantization formats and their reference decode define the
|
|
bit layout our kernels must reproduce, so the upstream source is the specification.
|
|
The ported material is quantization-related and spans hanzo-ml, hanzo-kernel,
|
|
hanzo-kernels, hanzo-metal-kernels, hanzo-rocm-kernels, hanzo-transformers and
|
|
tensor-tools; the per-file provenance is recorded in source comments naming the
|
|
upstream file (and, where useful, the line). It includes at least:
|
|
|
|
- GGUF/GGML block formats and their CPU reference decode (quantized/k_quants.rs,
|
|
ggml_file.rs, gguf_file.rs) -- e.g. kvalues_mxfp4 from ggml-common.h and a
|
|
bit-for-bit replica of ggml_e8m0_to_fp32_half from ggml-impl.h.
|
|
- The IQ-quant codebook grids and sign table (quantized/iq_grids.rs), extracted
|
|
verbatim from ggml/src/ggml-common.h, and the CUDA MMQ codebook tables.
|
|
- GPU quantized matmul/matvec kernels whose structure follows the corresponding
|
|
ggml kernels (the q8_1 block-quantized dot-product decode used by the CUDA/ROCm
|
|
dp4a cores and the Vulkan dp4a shaders, and the expert-grouped MoE GEMM).
|
|
|
|
Copyright in the ported material remains with The ggml authors; Hanzo AI claims
|
|
no copyright over it. Hanzo's own contributions around it (the backends, the
|
|
kernel DSL, and the original kernels) are covered by the Hanzo copyright above.
|
|
llama.cpp ships no NOTICE file; there are no further attributions to propagate.
|
|
|
|
PROVENANCE
|
|
|
|
Forked from https://github.com/huggingface/candle. The candle sources are vendored
|
|
and renamed to hanzo-* crates, so no pinned candle crate version is declared; this
|
|
fork tracks upstream candle main. The nearest upstream reference point on this line
|
|
of development is candle commit 3091caf9 (upstream PR #3408).
|
|
|
|
Upstream candle ships no NOTICE file; no Apache-2.0 section 4(d) attributions to
|
|
propagate.
|
|
|
|
DEVIATIONS
|
|
|
|
Changes from upstream candle in this fork:
|
|
|
|
- Rebranded to "Hanzo ML"; the top-level crate candle-core is renamed to hanzo-ml
|
|
(core module hanzo_ml_core).
|
|
- Companion crates renamed: candle-nn -> hanzo-nn, candle-transformers ->
|
|
hanzo-transformers, candle-datasets -> hanzo-datasets, candle-onnx -> hanzo-onnx,
|
|
candle-flash-attn -> hanzo-flash-attn, candle-kernels -> hanzo-kernels,
|
|
candle-metal-kernels -> hanzo-metal-kernels, candle-pyo3 -> hanzo-ml-pyo3,
|
|
candle-examples -> hanzo-ml-examples, candle-book -> hanzo-ml-book.
|
|
- Added a first-party GPU kernel DSL (hanzo-kernel + hanzo-kernel-macros): one Rust
|
|
source lowered to CUDA, ROCm, Vulkan, and Metal.
|
|
- Added a ROCm/HIP backend and kernels (hanzo-rocm-kernels; rocm-rs vendored and
|
|
renamed to the hanzo-rocm package; hanzo-cubecl-hip fork).
|
|
- Added a Vulkan compute backend.
|
|
- Added FlashAttention-3 for Hopper/sm90 (hanzo-flash-attn-v3).
|
|
- Added NVIDIA GB10 / DGX Spark unified/managed-memory support in the CUDA backend.
|
|
- Extended quantization: the full GGUF quant zoo (Q/K, legacy, IQ1-4, TQ; 22 types)
|
|
decodes resident and bit-exact on AMD gfx1151 through one unified compute core.
|
|
- Added hanzo-3d (generative-3D geometry and mesh I/O: Pixal3D / TRELLIS / TripoSR)
|
|
and native-Rust training crates (hanzo-train, hanzo-training) targeting the
|
|
zen-agentic and zen-identity datasets and DSpark speculative-draft models.
|
|
- Integrated with Hanzo Engine (OpenAI + Anthropic + MCP serving APIs) and the
|
|
Hanzo/zen model hubs.
|
|
- Dependency and layout changes: cudarc pinned at 0.19.4; an orphaned duplicate
|
|
crate cluster is excluded from the workspace build.
|
|
- Added hanzo-learn (statistical learning: least squares, logistic regression,
|
|
gradient boosted regression trees).
|
|
|
|
hanzo-learn and scikit-learn
|
|
----------------------------
|
|
|
|
hanzo-learn is a CLEAN-ROOM implementation. Its algorithms come from the published
|
|
literature (Householder 1958; Breiman, Friedman, Olshen & Stone 1984; Friedman 2001)
|
|
and no scikit-learn source code or documentation text is reproduced in it.
|
|
|
|
scikit-learn (https://scikit-learn.org, BSD-3-Clause) is used as a TEST ORACLE: the
|
|
fixtures under hanzo-learn/tests/fixture/ hold numerical results produced by running
|
|
it (version recorded in each fixture), and hanzo-learn's own results are asserted
|
|
against them. The generator that produced them is committed beside them.
|
|
|
|
Because no scikit-learn code is included or distributed here, its BSD-3-Clause notice
|
|
is not carried. If any scikit-learn code or docstring is ever incorporated, that
|
|
attribution must be added to this file at the same time.
|