mirror of
https://github.com/hanzoai/router.git
synced 2026-10-30 18:23:11 +00:00
zeekay
f970ecc5ae
learner/ (the Enso policy), retrain/ (its fit and eval), heads/ (the fitted bundle), docs/ (the routing corpus and its mean-field results) and STATE-router-field.md now live in hanzo-inc/enso under policy/, with their history. hanzo-router keeps the mechanism: the registry, the SLO gate, the RoutePolicy trait, the heads loader and the proxy.
1.8 KiB
1.8 KiB
LLM.md — hanzoai/router
Guidance for AI agents working in this repo.
What this is
hanzo-router — the pure-Rust model-routing core. Value-in / decision-out:
classify a request, then pick from a pool of local + cloud models. No hardware
access of its own → deterministic, unit-testable. Optional proxy feature adds an
axum replica load-balancing front (prefix-affinity + least-loaded + health probe).
Canonical role
Real code lives here (canonical impl). It is the routing brain that hanzoai/node
uses with hanzoai/ml + hanzoai/engine. This is Rust ecosystem infra, not an SDK
wrapper — completeness order across langs is Python → Rust → C++ → Go. Discovery/
wrapper repos link OUT to this; never duplicate the impl. One impl, one place (DRY).
Build / run
cargo test— pure core.cargo test --features proxy— + the proxy front.- Proxy binary:
cargo run --features proxy --bin hanzo-router -- --model M --replica URL ... - Learned heads: a serve bundle mounted with
ROUTER_HEADS=<path>(src/heads.rs).
Key entry points
src/lib.rs— public API (route,load_policy), module map.src/policy.rs—Policy::select(Reuse → LoadLocal → Cloud → NoFit).src/route.rs—RoutePolicyseam (mechanism vs. learned brain).src/proxy.rs+src/bin/proxy.rs— the replica proxy front.src/heads.rs— the heads bundle a learned policy serves from, and the mean-field load term.
Brand rules (hard — enforce in all docs)
- Never call Hanzo an "LLM gateway" or position vs LiteLLM. It is a routing core within Hanzo — the Open AI Cloud, not a proxy product.
- Paths are
/v1/...only — never/api/. - Zen models are our own family; never name upstream models in public copy.
Full model: ~/work/hanzo/SDK-ARCHITECTURE.md.