Files
engine/docs/PYTHON_INSTALLATION.md

4.2 KiB

Python SDK Installation

Pre-built wheels are available for common platforms. Choose the package that matches your hardware:

Hardware Install Command
Recommended (auto-optimized) pip install hanzo
NVIDIA GPUs (CUDA) pip install hanzo-cuda
Apple Silicon (Metal) pip install hanzo-metal
Apple Accelerate pip install hanzo-accelerate
Intel CPUs (MKL) pip install hanzo-mkl

Platform-Specific Optimizations

The hanzo base package includes platform-specific optimizations:

  • macOS Apple Silicon: Metal GPU support built-in
  • Linux/Windows x86_64: Intel MKL optimizations built-in
  • Linux aarch64: CPU-only (use hanzo-cuda for GPU support)

All packages install the hanzo Python module. The package suffix controls which accelerator features are enabled.

Supported Platforms

Package Linux x86_64 Linux aarch64 Windows x86_64 macOS aarch64
hanzo MKL CPU MKL Metal
hanzo-cuda CUDA CUDA CUDA -
hanzo-metal - - - Metal
hanzo-accelerate - - - Accelerate
hanzo-mkl MKL - MKL -

Python version: 3.10+ (wheels use abi3 for forward compatibility)

Windows Requirements

It is recommended to use WSL2 on Windows machines.

On Windows, additional runtime dependencies may be required:

# Example: Install with CUDA support
pip install hanzo-cuda -v

Build from Source

Building from source gives you access to the latest features and allows customization of build options.

Prerequisites

  1. Install system packages:

    Ubuntu/Debian:

    sudo apt install libssl-dev pkg-config
    

    macOS:

    brew install openssl pkg-config
    
  2. Install Rust from https://rustup.rs/:

    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
    source $HOME/.cargo/env
    
  3. (Optional) Set up HuggingFace authentication for gated models:

    mkdir -p ~/.cache/huggingface
    echo "YOUR_HF_TOKEN" > ~/.cache/huggingface/token
    

    Or use huggingface-cli login.

Build Steps

  1. Clone the repository:

    git clone https://github.com/hanzoai/engine.git
    cd Hanzo Engine/hanzo-pyo3
    
  2. Create and activate a virtual environment:

    python -m venv .venv
    source .venv/bin/activate  # Linux/macOS
    # or: .venv\Scripts\activate  # Windows
    
  3. Install maturin (Rust + Python build tool):

    pip install maturin[patchelf]
    
  4. Build and install:

    maturin develop -r --features <your-features>
    

Feature Flags

Feature Description
cuda NVIDIA GPU support
flash-attn Flash Attention (CUDA, Ampere+)
flash-attn-v3 Flash Attention v3 (CUDA, Hopper)
cudnn cuDNN optimizations
metal Apple Silicon GPU (macOS only)
accelerate Apple Accelerate framework
mkl Intel MKL

Example with CUDA and Flash Attention:

maturin develop -r --features "cuda flash-attn cudnn"

Verify Installation

import hanzo
print(hanzo.__version__)

Quick test:

from hanzo import Runner, Which, ChatCompletionRequest

runner = Runner(
    which=Which.Plain(model_id="Qwen/Qwen3-0.6B"),
)

response = runner.send_chat_completion_request(
    ChatCompletionRequest(
        model="default",
        messages=[{"role": "user", "content": "Hello!"}],
        max_tokens=50,
    )
)
print(response.choices[0].message.content)

Next Steps