mirror of
https://github.com/hanzoai/site.git
synced 2026-11-20 21:21:13 +00:00
1 line
1.9 KiB
JSON
1 line
1.9 KiB
JSON
{"text":["hanzoai/engine","Hanzo Engine","The inference engine, in one binary","View on GitHub","One process, laptop to GPU cluster","01 / 06","run, serve, bench","One binary, three verbs. Chat with a model, put it behind an HTTP API, or measure it — the thing you benchmarked is the thing you shipped.","Built to serve, not just to run","Paged attention and continuous batching keep the card busy across concurrent requests. A disk-first KV cache carries a session across a restart, so an agent that reuses a long prefix does not pay for it twice.","More than text","Vision, audio, speech, image and embeddings go through the same modality pipeline as generation, in the same process. A multimodal app is one deployment, not four.","It quantizes on the way in","In-situ quantization of any Hugging Face model, plus GGUF from 2 to 8 bits, GPTQ, AWQ, HQQ, FP8 and bitsandbytes. Ask for a level and it takes a published prebuilt when one exists, or makes one when it doesn't.","The silicon you have","CUDA with FlashAttention and multi-GPU tensor parallelism, Metal on Apple Silicon, or plain CPU. Same engine and same commands on each, and a doctor command that reports what it found.","Tools, and a UI to watch them","It can call MCP servers over a process, HTTP or a WebSocket. The built-in UI at /ui shows reasoning, code execution and plots inline, and edits branch a conversation with its own state. Pass --no-ui and it is a plain API server.","Paste and ship","Paste this into any agent. It reads the skill manifest and calls Engine from there.","Agent prompt","Copy","Read https://hanzo.ai/skill.md and use Hanzo Engine in my project. Start with: hanzo engine serve <model>","Get started with Hanzo Engine","Get Engine","AI inference engine","Apache-2.0","License:","hanzoai","Deploy to Cloud","02 / 06","03 / 06","04 / 06","05 / 06","06 / 06"],"links":["https://github.com/hanzoai","https://platform.hanzo.ai/deploy"]} |