Files

199 lines
4.1 KiB
JSON

[
[
{
"h1": "Hanzo Edge"
},
{
"p": "Run a model on the machine in front of you."
},
{
"a": "View on GitHub",
"href": "https://github.com/hanzoai/edge"
},
{
"a": "View Documentation",
"href": "https://docs.hanzo.ai"
},
{
"img": "/models/oto.webp",
"alt": ""
},
{
"p": "macOS on Apple Silicon and Intel, Linux on x86 and ARM, and modern browsers through WebAssembly"
},
{
"p": "Weights on disk, tokens out of local memory. There is no request, so there is nothing to be slow"
},
{
"p": "Name a Hugging Face repo and it downloads and caches the weights. After that, offline"
},
{
"li": "Where it runs"
},
{
"li": "No network at all"
},
{
"li": "Fetched once"
}
],
[
{
"p": "Paste and ship"
},
{
"p": "Paste this into any agent. It reads the skill manifest and calls Edge from there."
},
{
"button": "Copy"
},
{
"code": "Read https://hanzo.ai/skill.md and use Hanzo Edge in my project."
}
],
[
{
"h2": "What the runtime does"
},
{
"p": "One Rust binary, a core crate to embed, a WASM build for the browser, and a local server for everything else"
},
{
"h3": "A server on localhost"
},
{
"p": "A local inference endpoint on a port you pick. Point a client at it and the rest of your code does not change."
},
{
"h3": "Quantized as a first language"
},
{
"p": "GGUF is the native format — Q4_K, Q5_K, Q8_0. A 4B model at Q4_K_M is about 2.5GB on disk, which is what makes a phone a plausible place to run one."
},
{
"h3": "It uses what it finds"
},
{
"p": "Metal on Apple Silicon, CUDA where there is an NVIDIA card, AVX2 or AVX-512 on CPU. Detected at startup rather than configured by you."
},
{
"h3": "Zen models sized for devices"
},
{
"p": "zen3-nano at 600M for embedded work and zen-eco at 4B for phones and tablets, both published pre-quantized."
},
{
"h3": "Token by token"
},
{
"p": "Streaming over server-sent events from the local server, or a callback from the Rust API, so a UI starts drawing before generation finishes."
},
{
"h3": "The prompt never leaves"
},
{
"p": "Nothing is uploaded and there is no key to leak. The prompt and the output stay in the process that made them, which is the whole reason to run a model here."
}
],
[
{
"h2": "When to reach for it"
},
{
"p": "Use Edge when the data cannot leave, the network cannot be relied on, or a per-call bill is the wrong shape. Use Hanzo Engine when you need full precision or many users at once."
},
{
"h3": "Apps that work with no signal"
},
{
"p": "The model is a file on disk, so the feature does not stop existing when the connection does."
},
{
"li": "• A desktop app that drafts and summarizes without a round trip"
},
{
"li": "• Field tools on a laptop, on a site with no coverage"
},
{
"li": "• Anything that has to keep working on a plane"
},
{
"h3": "Text that cannot leave the building"
},
{
"p": "Some prompts are a legal problem the moment they cross a network boundary. This is the answer to those."
},
{
"li": "• Clinical and legal text processed on the workstation that holds it"
},
{
"li": "• On-premise deployments with no egress at all"
},
{
"li": "• Personal data handled on the device it belongs to"
},
{
"h3": "Inside the browser tab"
},
{
"p": "The WebAssembly build runs the model in the page, with nothing behind it — so a visitor costs you bandwidth once and compute never."
},
{
"li": "• A demo that does not bill per visitor"
},
{
"li": "• An assistant inside a web app you already ship"
},
{
"li": "• Classification done client-side, before anything is uploaded"
},
{
"h3": "Small hardware"
},
{
"p": "ARM64 Linux is a production target. Cortex-A class boards are experimental, and the 600M model is the one that fits them."
},
{
"li": "• A 600M model on a single-board computer"
},
{
"li": "• An ARM server doing batch work on its own"
},
{
"li": "• Kiosks and appliances with no account to sign in to"
}
],
[
{
"p": "Related capabilities"
},
{
"a": "GPUs On-demand accelerators",
"href": "/cloud/gpus"
},
{
"a": "Machines VMs by the second",
"href": "/machines"
},
{
"a": "Containers",
"href": "/cloud/containers"
}
],
[
{
"h2": "Get Edge"
},
{
"p": "Edge compute"
},
{
"a": "hanzoai",
"href": "https://github.com/hanzoai"
},
{
"a": "Deploy to Cloud",
"href": "https://platform.hanzo.ai/deploy"
}
]
]