crates.io `hanzo-cli` is the Hanzo CLI (hanzoai/cli: bins hanzo, driftgate, genproduct, genspec), not this repo's engine CLI, so `cargo install hanzo-cli --features cuda` installed the wrong crate or failed on the feature. Every such line now reads `cargo install --locked --git https://github.com/hanzoai/engine hanzo-cli`, which builds `hanzo-engine` from this repo as the root install.sh does. site/install.sh (engine.hanzo.ai/install.sh) fell back to `cargo install hanzo-engine`; crates.io hanzo-engine 0.6.1 is a library with no binary. It now uses the same git install.
4.9 KiB
Installation Guide
Quick install (prebuilt)
npm i -g hanzo # any OS; or, on macOS/Linux: curl -fsSL https://hanzo.sh | sh
hanzo engine serve <model>
The first hanzo engine fetches the prebuilt hanzo-engine binary (linux amd64/arm64, macOS) and checks
its SHA-256. For CUDA, ROCm or any other feature set, use the install script or build from source below.
Install script (builds from source)
The install script automatically detects your hardware (CUDA, Metal, MKL) and builds with optimal features.
Linux/macOS:
curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/hanzoai/engine/main/install.sh | sh
Windows (PowerShell):
irm https://raw.githubusercontent.com/hanzoai/engine/main/install.ps1 | iex
Prerequisites
-
Install required packages:
- OpenSSL:
sudo apt install libssl-dev(Ubuntu) - pkg-config (Linux only):
sudo apt install pkg-config
- OpenSSL:
-
Install Rust from https://rustup.rs/
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh source $HOME/.cargo/env -
(Optional) Install FFmpeg for video input support:
- Linux:
sudo apt install ffmpegorsudo dnf install ffmpeg - macOS:
brew install ffmpeg - See Video Input for details
- Linux:
-
(Optional) Set up HuggingFace authentication:
hanzo loginOr use
huggingface-cli loginas documented here.
Supported Accelerators
| Accelerator | Feature Flag | Additional Flags |
|---|---|---|
| NVIDIA GPUs (CUDA) | cuda |
flash-attn, flash-attn-v3, cudnn |
| Apple Silicon GPU (Metal) | metal |
|
| CPU (Intel) | mkl |
|
| CPU (Apple Accelerate) | accelerate |
|
| Generic CPU (ARM/AVX) | none | ARM NEON / AVX enabled by default |
Note for Linux users: The
metalfeature is macOS-only. Use--features "cuda flash-attn cudnn"for NVIDIA GPUs or--features mklfor Intel CPUs instead of--all-features.
Feature Detection
Determine which features to enable based on your hardware:
| Hardware | Features |
|---|---|
| NVIDIA GPU (Ampere+, compute >=80) | cuda cudnn flash-attn |
| NVIDIA GPU (Hopper, compute 90) | cuda cudnn flash-attn flash-attn-v3 |
| NVIDIA GPU (older) | cuda cudnn |
| Apple Silicon (macOS) | metal accelerate |
| Intel CPU with MKL | mkl |
| CPU only | (no features needed) |
Install from crates.io
cargo install --locked --git https://github.com/hanzoai/engine hanzo-cli --features "<your-features>"
Example:
cargo install --locked --git https://github.com/hanzoai/engine hanzo-cli --features "cuda flash-attn cudnn"
Build from Source
git clone https://github.com/hanzoai/engine.git
cd engine
cargo install --path hanzo-cli --features "<your-features>"
Example:
cargo build --release --features "cuda flash-attn cudnn"
Docker
Docker images are available for quick deployment:
docker pull ghcr.io/hanzoai/engine:latest
docker run --gpus all -p 1234:1234 ghcr.io/hanzoai/engine:latest \
serve -m Qwen/Qwen3-4B
Docker images on GitHub Container Registry
Learn more about running Docker containers: https://docs.docker.com/engine/reference/run/
Python SDK
Install the Python package:
pip install hanzo-cuda # For NVIDIA GPUs
pip install hanzo-metal # For Apple Silicon
pip install hanzo-mkl # For Intel CPUs
pip install hanzo # CPU-only
Verify Installation
After installation, verify everything works:
# Check CLI is installed
hanzo --help
# Run system diagnostics
hanzo doctor
# Test with a small model
hanzo run -m Qwen/Qwen3-0.6B
Getting Models
From Hugging Face Hub (Default)
Models download automatically from Hugging Face Hub:
hanzo run -m meta-llama/Llama-3.2-3B-Instruct
For gated models, authenticate first:
hanzo login
# Or: hanzo run --token-source env:HF_TOKEN -m <model>
From Local Files
Pass a path to a downloaded model:
hanzo run -m /path/to/model
Running GGUF Models
hanzo run --format gguf -m author/model-repo -f model-quant.gguf
Specify tokenizer if needed:
hanzo run --format gguf -m author/model-repo -f file.gguf -t author/official-tokenizer
Next Steps
- CLI Reference: All commands and options
- HTTP API: Run as an OpenAI-compatible server
- Python SDK: Python package documentation
- Troubleshooting: Common issues and solutions