mirror of
https://github.com/hanzoai/engine.git
synced 2026-11-20 14:17:52 +00:00
Builders for the fused router bit-exact to vLLM: src/cuda/route.rs, lane.rs, branch.rs and the exact kernels, plus the shared-expert overlap and the goldens. The router top-k moved out of ops.rs, which is why that file loses lines. Worktree checkpoint — the build had not happened when the stream stopped. Co-Authored-By: Claude Code <[email protected]>
95 lines
5.4 KiB
Plaintext
95 lines
5.4 KiB
Plaintext
Hanzo Engine
|
|
Copyright (c) 2024 Hanzo AI, Inc.
|
|
|
|
This product is licensed under the MIT License (see LICENSE).
|
|
|
|
-------------------------------------------------------------------------------
|
|
Third-party attribution
|
|
-------------------------------------------------------------------------------
|
|
|
|
Hanzo Engine is derived from mistral.rs (https://github.com/EricLBuehler/mistral.rs),
|
|
Copyright (c) 2024 Eric Buehler, used under the MIT License:
|
|
|
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
of this software and associated documentation files (the "Software"), to deal
|
|
in the Software without restriction, including without limitation the rights
|
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
copies of the Software, and to permit persons to whom the Software is
|
|
furnished to do so, subject to the following conditions:
|
|
|
|
The above copyright notice and this permission notice shall be included in all
|
|
copies or substantial portions of the Software.
|
|
|
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
SOFTWARE.
|
|
|
|
Hanzo Engine is an independent fork: all crates, binaries, and APIs are
|
|
Hanzo-branded (hanzo-engine), and Hanzo AI is not affiliated with Mistral AI.
|
|
|
|
Forked from https://github.com/EricLBuehler/mistral.rs. Upstream history is
|
|
retained in full, not squashed: this tree's root commit is upstream's own
|
|
1ecfb70842640b12c8321d69be58bec5331ddffc (EricLBuehler, 2024-02-26), and 3,697
|
|
of the 5,495 commits reachable from main are authored by Eric Buehler. The
|
|
crate rename (mistralrs-* -> hanzo-*) rewrote commit hashes from
|
|
7483b396fa0513362977a6e033e5259d1a7319e7 (2024-02-27) onward, so beyond that
|
|
point our SHAs no longer equal upstream's even where the content is theirs;
|
|
tools/upstream-map.sh maps the paths. Upstream work is merged through
|
|
mistral.rs 0a14af4a43e6f61afbed71f72f22c162ab944c0d (PR #2188, 2026-06-02).
|
|
Hanzo Engine tracks its own release line (1.7.x) rather than upstream tags.
|
|
|
|
Hanzo Engine is built on candle (https://github.com/huggingface/candle),
|
|
Copyright (c) 2023 HuggingFace Inc., dual-licensed under the MIT License OR the
|
|
Apache License, Version 2.0; used here under the MIT License (full text above).
|
|
The candle crates are consumed as Hanzo's fork (https://github.com/hanzoai/ml),
|
|
published as hanzo-ml (candle-core), hanzo-nn (candle-nn), hanzo-transformers
|
|
(candle-transformers), hanzo-flash-attn, hanzo-flash-attn-v3, and
|
|
hanzo-metal-kernels. That fork carries candle's own LICENSE-MIT and
|
|
LICENSE-APACHE and its own NOTICE; this file does not restate them.
|
|
|
|
Hanzo Engine bundles third-party GPU kernels, each under its own permissive
|
|
license (per-file copyright headers retained in-source):
|
|
- vLLM (Apache-2.0), Copyright (c) 2023 The vLLM team
|
|
- NVIDIA TensorRT-LLM (Apache-2.0), Copyright (c) 1993-2023 NVIDIA
|
|
CORPORATION & AFFILIATES (the MoE top-k gating kernel, reached via vLLM:
|
|
hanzo-engine/src/cuda/exact/route.cu)
|
|
- FlashInfer (Apache-2.0), Copyright (c) 2023-2025 FlashInfer team
|
|
- Apple MLX (MIT), Copyright (c) 2023-2024 Apple Inc.
|
|
- NVIDIA FasterTransformer (Apache-2.0), Copyright (c) 2020-2023 NVIDIA
|
|
CORPORATION (reached via the vLLM kernels; no CUTLASS source is vendored,
|
|
and no BSD-3-Clause-licensed source is present in this tree)
|
|
- Marlin (Apache-2.0), Copyright (c) 2024 Elias Frantar
|
|
- ggml / llama.cpp (MIT), Copyright (c) ggml authors (Georgi Gerganov et al.)
|
|
|
|
-------------------------------------------------------------------------------
|
|
DEVIATIONS
|
|
-------------------------------------------------------------------------------
|
|
|
|
Hanzo Engine has diverged substantially from upstream mistral.rs:
|
|
|
|
- Full Hanzo rebrand: every workspace crate is renamed hanzo-* (hanzo-engine,
|
|
hanzo-server-core, hanzo-router, hanzo-quant, hanzo-vision, hanzo-audio,
|
|
hanzo-paged-attn, hanzo-llm-mcp, hanzo-sandbox, hanzo-code-exec, hanzo-cli,
|
|
hanzo-pyo3, hanzo-film), and the CLI/binary is `hanzo`.
|
|
- Candle is consumed as Hanzo's fork (hanzo-ml, hanzo-nn, hanzo-flash-attn,
|
|
hanzo-metal-kernels) rather than upstream candle-core / candle-nn.
|
|
- OpenAI-compatible `/v1` server plus an Anthropic-compatible
|
|
`POST /v1/messages` endpoint (streaming + tool use), served from one router.
|
|
- All-modality serving: text/chat, vision (hanzo-vision), audio in and 24 kHz
|
|
speech out (hanzo-audio), video and image/diffusion models, long-form film
|
|
orchestration (hanzo-film), and 3D (hanzo-3d).
|
|
- GPU backend coverage across CUDA, Apple Metal, and AMD ROCm/HIP
|
|
(Dockerfile.cuda, Dockerfile.rocm, and the Metal kernels).
|
|
- Broad quantization support: GGUF, ISQ, UQFF, Q8_0/Q4_K, IQ4, MXFP4, HQQ,
|
|
AFQ, and bitsandbytes FP4.
|
|
- Zen model serving (native/zen3-serving and the zen model paths).
|
|
- Adapter primitives: AnyMoE plus LoRA and QLoRA adapters.
|
|
- Hanzo ZAP transport integration (hanzo-engine/src/zap) and a disk-first KV
|
|
cache for cross-restart sessions and agent prefix reuse.
|
|
- Sandboxed local code execution (hanzo-sandbox, hanzo-code-exec) for the
|
|
agentic runtime.
|