mirror of
https://github.com/hanzoai/ml.git
synced 2026-11-19 23:50:51 +00:00
canonical is JCS: Ryū digits in ECMAScript's layout, names in UTF-16 order, and a duplicate name, NaN or an integer no double holds refused; RFC 8785's number, example and sorting vectors are its tests. lineage walks parents by id and reports bounded, present, acyclic, lineage, purpose, derivation, eval and clean, so an eval_only grandparent cannot reach a train grandchild. wal refuses a torn, reordered, re-chained or byte-flipped line, one log per (run, worker), fsync'd per append and held by one writer. Api::artifact checks the bytes it answers against the address asked for.
5.5 KiB
5.5 KiB
LLM.md — hanzoai/ml
Fast, multi-backend tensor & ML framework for Rust (CPU · CUDA · Metal · ROCm · Vulkan) with quantization (GGUF/GGML/AFQ/GPTQ/AWQ). The compute core beneath Hanzo inference.
Canonical role
- This repo is the canonical implementation of the Hanzo Rust ML/tensor core. One impl, one place — discovery/wrapper repos link here, never copy the code.
- Rust is the 2nd-most-complete ecosystem (Python → Rust → C++ → Go → …).
- Crate
hanzo-ml(crates.io) · docs at docs.rs/hanzo-ml.
Install / run
- Core:
cargo add hanzo-ml(Tensor/Device); addhanzo-nnto build models. - GPU:
--features cuda(+cudnn), ormetal/rocm/vulkan. - Examples:
cargo run --example quantized --release(seehanzo-ml-examples/).
Key entry points
hanzo-ml/— core ops, devices,Tensor.hanzo-nn/— layers & model building.hanzo-transformers/— model implementations.hanzo-kernels/,hanzo-flash-attn/— CUDA kernels & FlashAttention v2.hanzo-onnx/,hanzo-datasets/,hanzo-ml-wasm-examples/.hanzo-train/— training on hanzo-ml.cluster: one model across heterogeneous machines (Metal, CUDA, ROCm, CPU) by local SGD with an outer Nesterov step (DiLoCo). A coordinator owns the plan, θ_global in F32, the outer step and checkpoints; workers join any time, take batches one at a time, send θ_local − θ_global in bf16 with error feedback; deltas sum in join order, so a run equals its rounds replayed in one process (cluster::replay) bit for bit, and a checkpoint (outer momentum and carry, merged batches, round, every worker's AdamW state) resumes bit-identically. A worker that drops has its batches requeued; a round waits for no one past its deadline plusgrace(a silent member is dropped, its link shut), a join under a member's name replaces it, links have keepalive and bounded reads/writes (net::link), and a worker whose link fails joins again on its own with backoff. A model plugs in behindcluster::Model(varsF32 masters,rateper parameter —Nonefrozen,stepover a batch of row indices the caller planned); the coordinator's owner behindcluster::Keep(round line, validation, save).adam: AdamW with per-parameter rates and open moments (hanzo-nn'soptim::adamwupdate, fused on Metal).gpu: free-memory floor.dspark: the DSpark draft on the runtime (hanzo-train fit/join). Tests:cargo test -p hanzo-train(tiny classifier and synthetic DSpark cache, all CPU). Kai (hanzoai/decisiontrain) is one user.hanzo-research/— the research runtime's Rust client (HIP-1334).canonical: RFC 8785 JSON (Ryū digits in ECMAScript layout, UTF-16 key order, duplicate names, NaN and inexact integers refused; needs serde_json'sfloat_roundtrip).lineage:Purpose, the §3.2 admission matrix, §3.3derive, andverify_promotion_eligibility, a walk of the artifact graph by id and parent edge reporting eight checks (bounded, present, acyclic, lineage, purpose, derivation, eval, clean).wal: a run's log,wal-<worker>.jsonl, one canonical line per event, hash-chained per (run, worker), fsync'd per append, locked to one writer, and verified line by line on open (readrefuses the whole log at the first bad line;mergeorders by sequence then worker).environment:stamp()for a binary's build script andbuild!()/version!()to read it back;Environmentmeasured (ofa build,repoa checkout).wire: the/v1/researchclient (Api: artifacts put and get, both checked against the address; lineage; claims). Tests:cargo test -p hanzo-research. The session that drafted it in the main checkout left it uncommitted; this is where it landed.
Releasing
- Registry is crates.io, owner
zeekay. There is no Hanzo cargo registry: no[registries]in.cargo/config.toml, nopublish = [...]allow-list, and the sibling Rust repo (hanzoai/engine) publishes the same way. - Each crate carries its own version and moves by a patch bump from the version it last released. Crates change at different rates, so their numbers differ — that is information, not drift. Never renumber a crate to match another.
scripts/publish-orderis the release set: every crate that does not saypublish = false, topologically sorted.publish = falseis the one way to keep a crate off the registry (examples, demos, the book, the PyPI extension module, andtensor-tools, whose crates.io name belongs to the upstream candle author).- Publishing is CI's job: push a
N.N.Ntag to the forge and.hanzo/workflows/publish.ymlwalks that order withcargo publish --no-verify(GPU build scripts can't run on crates.io builders). The tag names the release event; the manifests name the artifacts. Re-running is safe — a crate already at its manifest version is skipped. - Run a crate's own tests before its version moves. ROCm needs
LD_LIBRARY_PATH=/opt/rocm/core-7.13/lib—libhiprtc.so.7lives there, not in/opt/rocm/lib, socargo test -p hanzo-kernel --features rocmotherwise dies at load time with the test binary already built.
Brand rules (enforce in all docs)
- Hanzo is the Open AI Cloud / full AI SDK — never an "LLM gateway", never positioned vs LiteLLM, never an "OpenAI-compatible proxy". Purge that framing.
- Paths are
/v1/, never/api/. - Zen models are our own family — don't present upstream model names as ours.
Spec: ~/work/hanzo/SDK-ARCHITECTURE.md — the canonical one-way SDK model.