Files
zeekayandHanzo Dev b2832f1c55 cli: hanzo-engine bench is the one instrument, and board scores it
hanzo-bench is the library behind `hanzo-engine bench` (its `measure` loop) and
`hanzo-engine board score|publish|manifest`; its binary is gone, as is the CLI's
earlier bench that read the response's self-reported timing. The dossier calls
the CLI. Documented in docs/CLI.md and the CLI reference.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-17 19:15:05 -07:00
..
2026-09-15 09:42:19 -07:00

Hanzo Engine

Fast, flexible LLM inference. Run 40+ model families (text, vision, video, audio, speech, image generation, and embeddings) with automatic hardware optimization, OpenAI-compatible APIs, and built-in agentic features.

Quick Start

# Install (Linux/macOS, auto-detects CUDA, Metal, MKL)
curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/hanzoai/engine/master/install.sh | sh

# Install (Windows PowerShell)
irm https://raw.githubusercontent.com/hanzoai/engine/master/install.ps1 | iex

# Run a model interactively
hanzo run --isq 4 -m Qwen/Qwen3-4B

# Or serve it as an OpenAI-compatible API with a web UI
hanzo serve --ui --isq 4 -m Qwen/Qwen3-4B

New to Hanzo Engine? Follow the Getting Started tutorial for a guided walkthrough.

Choose Your Path

I want to... Start here
Follow a step-by-step tutorial Getting Started
Run a model from the command line CLI Reference
Serve models over HTTP HTTP Server
Build a Python application Python SDK
Build a Rust application Rust SDK
Optimize performance and memory Performance Guide
Choose a quantization method Quantization Overview
Use agentic features (tools, search, MCP) Agentic Features Guide
Find a specific model Supported Models
Troubleshoot an issue Troubleshooting

Highlights

  • Zero config: hanzo run -m <model> auto-detects architecture, chat template, and hardware
  • Quantization control: ISQ, GGUF, GPTQ, AWQ, HQQ, AFQ, FP8, MXFP4, or just --isq 4
  • Auto-tuning: hanzo tune benchmarks your hardware and recommends optimal settings
  • Agentic: Server-side tool calling loop, web search, MCP client, HTTP tool dispatch
  • Multi-GPU: Automatic tensor parallelism via NCCL, or cross-machine via Ring backend
  • Web UI: --ui flag for instant browser-based chat

Community