mirror of
https://github.com/hanzoai/engine.git
synced 2026-11-20 14:17:52 +00:00
crates.io `hanzo-cli` is the Hanzo CLI (hanzoai/cli: bins hanzo, driftgate, genproduct, genspec), not this repo's engine CLI, so `cargo install hanzo-cli --features cuda` installed the wrong crate or failed on the feature. Every such line now reads `cargo install --locked --git https://github.com/hanzoai/engine hanzo-cli`, which builds `hanzo-engine` from this repo as the root install.sh does. site/install.sh (engine.hanzo.ai/install.sh) fell back to `cargo install hanzo-engine`; crates.io hanzo-engine 0.6.1 is a library with no binary. It now uses the same git install.
Hanzo Engine
Fast, flexible LLM inference. Run 40+ model families (text, vision, video, audio, speech, image generation, and embeddings) with automatic hardware optimization, OpenAI-compatible APIs, and built-in agentic features.
Quick Start
# Install (Linux/macOS, auto-detects CUDA, Metal, MKL)
curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/hanzoai/engine/master/install.sh | sh
# Install (Windows PowerShell)
irm https://raw.githubusercontent.com/hanzoai/engine/master/install.ps1 | iex
# Run a model interactively
hanzo run --isq 4 -m Qwen/Qwen3-4B
# Or serve it as an OpenAI-compatible API with a web UI
hanzo serve --ui --isq 4 -m Qwen/Qwen3-4B
New to Hanzo Engine? Follow the Getting Started tutorial for a guided walkthrough.
Choose Your Path
| I want to... | Start here |
|---|---|
| Follow a step-by-step tutorial | Getting Started |
| Run a model from the command line | CLI Reference |
| Serve models over HTTP | HTTP Server |
| Build a Python application | Python SDK |
| Build a Rust application | Rust SDK |
| Optimize performance and memory | Performance Guide |
| Choose a quantization method | Quantization Overview |
| Use agentic features (tools, search, MCP) | Agentic Features Guide |
| Find a specific model | Supported Models |
| Troubleshoot an issue | Troubleshooting |
Highlights
- Zero config:
hanzo run -m <model>auto-detects architecture, chat template, and hardware - Quantization control: ISQ, GGUF, GPTQ, AWQ, HQQ, AFQ, FP8, MXFP4, or just
--isq 4 - Auto-tuning:
hanzo tunebenchmarks your hardware and recommends optimal settings - Agentic: Server-side tool calling loop, web search, MCP client, HTTP tool dispatch
- Multi-GPU: Automatic tensor parallelism via NCCL, or cross-machine via Ring backend
- Web UI:
--uiflag for instant browser-based chat