mirror of
https://github.com/hanzoai/ml.git
synced 2026-11-20 22:46:50 +00:00
The rule: each crate carries its own version and moves by a PATCH bump from the version it last released to crates.io. Crates change at different rates, so their numbers differ -- hanzo-ml has shipped 95 times and hanzo-3d once. That is information, not drift, and renumbering a crate to match another would both discard it and burn a breaking-position bump (0.2 -> 0.11 is nine of them) to say nothing new. The tag names the release event; the manifests name the artifacts. crates.io is the registry. No [registries] in .cargo/config.toml, no publish allow-list in any manifest, and hanzoai/engine publishes the same way. Publishing walked a hand-written crate list, and hanzo-kernel was missing from it: only hanzo-kernels (plural, the CUDA kernels) was listed, never hanzo-kernel (singular, the kernel DSL). A crate absent from that list never reaches the registry, so it drifted to 0.2.20 on crates.io against 0.2.39 in tree. Adding the one name back would leave the mechanism that lost it, and seven more crates were already missing the same way -- hanzo-datasets, hanzo-onnx, hanzo-transformers, hanzo-training, hanzo-3d, hanzo-bindgen-cuda, hanzo-research. So the list is now derived. scripts/publish-order reads every manifest, drops the ones that say `publish = false`, and topologically sorts what is left over the dependency edges already written there. A crate publishes because it exists. cargo metadata cannot answer this alone: the kernel crates carry `links=` build scripts and so sit in the workspace `exclude` list, invisible as members while still being crates we publish. `publish = false` becomes the one way to opt out, and says why in each manifest. Fifteen crates take it -- the examples, the nine browser demos, the book, the wasm test harness, hanzo-train (a binary), hanzo-ml-pyo3 (it ships to PyPI via maturin) -- and tensor-tools, whose crates.io name belongs to the upstream candle author, so that one can never be published from here at all. The release set is 18. Also: `cargo add hanzo-ml-core` was the documented way in, and no such crate exists in the tree or on crates.io. It is `hanzo-ml`.
2.9 KiB
2.9 KiB
LLM.md — hanzoai/ml
Fast, multi-backend tensor & ML framework for Rust (CPU · CUDA · Metal · ROCm · Vulkan) with quantization (GGUF/GGML/AFQ/GPTQ/AWQ). The compute core beneath Hanzo inference.
Canonical role
- This repo is the canonical implementation of the Hanzo Rust ML/tensor core. One impl, one place — discovery/wrapper repos link here, never copy the code.
- Rust is the 2nd-most-complete ecosystem (Python → Rust → C++ → Go → …).
- Crate
hanzo-ml(crates.io) · docs at docs.rs/hanzo-ml.
Install / run
- Core:
cargo add hanzo-ml(Tensor/Device); addhanzo-nnto build models. - GPU:
--features cuda(+cudnn), ormetal/rocm/vulkan. - Examples:
cargo run --example quantized --release(seehanzo-ml-examples/).
Key entry points
hanzo-ml/— core ops, devices,Tensor.hanzo-nn/— layers & model building.hanzo-transformers/— model implementations.hanzo-kernels/,hanzo-flash-attn/— CUDA kernels & FlashAttention v2.hanzo-onnx/,hanzo-datasets/,hanzo-ml-wasm-examples/.
Releasing
- Registry is crates.io, owner
zeekay. There is no Hanzo cargo registry: no[registries]in.cargo/config.toml, nopublish = [...]allow-list, and the sibling Rust repo (hanzoai/engine) publishes the same way. - Each crate carries its own version and moves by a patch bump from the version it last released. Crates change at different rates, so their numbers differ — that is information, not drift. Never renumber a crate to match another.
scripts/publish-orderis the release set: every crate that does not saypublish = false, topologically sorted.publish = falseis the one way to keep a crate off the registry (examples, demos, the book, the PyPI extension module, andtensor-tools, whose crates.io name belongs to the upstream candle author).- Publishing is CI's job: push a
N.N.Ntag to the forge and.hanzo/workflows/publish.ymlwalks that order withcargo publish --no-verify(GPU build scripts can't run on crates.io builders). The tag names the release event; the manifests name the artifacts. Re-running is safe — a crate already at its manifest version is skipped. - Run a crate's own tests before its version moves. ROCm needs
LD_LIBRARY_PATH=/opt/rocm/core-7.13/lib—libhiprtc.so.7lives there, not in/opt/rocm/lib, socargo test -p hanzo-kernel --features rocmotherwise dies at load time with the test binary already built.
Brand rules (enforce in all docs)
- Hanzo is the Open AI Cloud / full AI SDK — never an "LLM gateway", never positioned vs LiteLLM, never an "OpenAI-compatible proxy". Purge that framing.
- Paths are
/v1/, never/api/. - Zen models are our own family — don't present upstream model names as ours.
Spec: ~/work/hanzo/SDK-ARCHITECTURE.md — the canonical one-way SDK model.