Files
engine/docs/INSTALLATION.md
zeekay 9c9e5a8003 docs: cargo install of the engine CLI names the git source
crates.io `hanzo-cli` is the Hanzo CLI (hanzoai/cli: bins hanzo,
driftgate, genproduct, genspec), not this repo's engine CLI, so
`cargo install hanzo-cli --features cuda` installed the wrong crate or
failed on the feature. Every such line now reads
`cargo install --locked --git https://github.com/hanzoai/engine hanzo-cli`,
which builds `hanzo-engine` from this repo as the root install.sh does.

site/install.sh (engine.hanzo.ai/install.sh) fell back to
`cargo install hanzo-engine`; crates.io hanzo-engine 0.6.1 is a library
with no binary. It now uses the same git install.
2026-11-03 19:44:54 -08:00

4.9 KiB

Installation Guide

Quick install (prebuilt)

npm i -g hanzo                    # any OS; or, on macOS/Linux: curl -fsSL https://hanzo.sh | sh
hanzo engine serve <model>

The first hanzo engine fetches the prebuilt hanzo-engine binary (linux amd64/arm64, macOS) and checks its SHA-256. For CUDA, ROCm or any other feature set, use the install script or build from source below.

Install script (builds from source)

The install script automatically detects your hardware (CUDA, Metal, MKL) and builds with optimal features.

Linux/macOS:

curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/hanzoai/engine/main/install.sh | sh

Windows (PowerShell):

irm https://raw.githubusercontent.com/hanzoai/engine/main/install.ps1 | iex

Prerequisites

  1. Install required packages:

    • OpenSSL: sudo apt install libssl-dev (Ubuntu)
    • pkg-config (Linux only): sudo apt install pkg-config
  2. Install Rust from https://rustup.rs/

    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
    source $HOME/.cargo/env
    
  3. (Optional) Install FFmpeg for video input support:

    • Linux: sudo apt install ffmpeg or sudo dnf install ffmpeg
    • macOS: brew install ffmpeg
    • See Video Input for details
  4. (Optional) Set up HuggingFace authentication:

    hanzo login
    

    Or use huggingface-cli login as documented here.

Supported Accelerators

Accelerator Feature Flag Additional Flags
NVIDIA GPUs (CUDA) cuda flash-attn, flash-attn-v3, cudnn
Apple Silicon GPU (Metal) metal
CPU (Intel) mkl
CPU (Apple Accelerate) accelerate
Generic CPU (ARM/AVX) none ARM NEON / AVX enabled by default

Note for Linux users: The metal feature is macOS-only. Use --features "cuda flash-attn cudnn" for NVIDIA GPUs or --features mkl for Intel CPUs instead of --all-features.

Feature Detection

Determine which features to enable based on your hardware:

Hardware Features
NVIDIA GPU (Ampere+, compute >=80) cuda cudnn flash-attn
NVIDIA GPU (Hopper, compute 90) cuda cudnn flash-attn flash-attn-v3
NVIDIA GPU (older) cuda cudnn
Apple Silicon (macOS) metal accelerate
Intel CPU with MKL mkl
CPU only (no features needed)

Install from crates.io

cargo install --locked --git https://github.com/hanzoai/engine hanzo-cli --features "<your-features>"

Example:

cargo install --locked --git https://github.com/hanzoai/engine hanzo-cli --features "cuda flash-attn cudnn"

Build from Source

git clone https://github.com/hanzoai/engine.git
cd engine
cargo install --path hanzo-cli --features "<your-features>"

Example:

cargo build --release --features "cuda flash-attn cudnn"

Docker

Docker images are available for quick deployment:

docker pull ghcr.io/hanzoai/engine:latest
docker run --gpus all -p 1234:1234 ghcr.io/hanzoai/engine:latest \
  serve -m Qwen/Qwen3-4B

Docker images on GitHub Container Registry

Learn more about running Docker containers: https://docs.docker.com/engine/reference/run/

Python SDK

Install the Python package:

pip install hanzo-cuda    # For NVIDIA GPUs
pip install hanzo-metal   # For Apple Silicon
pip install hanzo-mkl     # For Intel CPUs
pip install hanzo         # CPU-only

Verify Installation

After installation, verify everything works:

# Check CLI is installed
hanzo --help

# Run system diagnostics
hanzo doctor

# Test with a small model
hanzo run -m Qwen/Qwen3-0.6B

Getting Models

From Hugging Face Hub (Default)

Models download automatically from Hugging Face Hub:

hanzo run -m meta-llama/Llama-3.2-3B-Instruct

For gated models, authenticate first:

hanzo login
# Or: hanzo run --token-source env:HF_TOKEN -m <model>

From Local Files

Pass a path to a downloaded model:

hanzo run -m /path/to/model

Running GGUF Models

hanzo run --format gguf -m author/model-repo -f model-quant.gguf

Specify tokenizer if needed:

hanzo run --format gguf -m author/model-repo -f file.gguf -t author/official-tokenizer

Next Steps