Files
zeekay 531e9f423a lane: no platform ceiling on the paid lane; a payer spends what they bought
The paid lane capped every paid-lane call together at PAID_LANE_DAILY ($10 a UTC
day by default) and each org at PAID_LANE_ORG_SHARE (25% of it), and counted
wallet-paid calls against both: a customer with $1,000 prepaid was held to $2.50 a
day, and every payer together to $10. The rule is no ceiling but what a payer
prepays, or their usage. Free users get free models; payers spend their own money.

Deleted: the platform day and the org share (paidLaneDaily, orgShare, the day,
spent and per-org accounting in laneBook), the paid_day store (object.PaidDay, its
table registration, the writer, flush, drain, load and turn), Settled's drain of
it, laneSpend, PaidLaneOpen (the gate reads !FreeOnly()), and the refusals only the
day produced ("closed", "after 00:00 UTC"). paid_lane_ceiling named a platform
ceiling that no longer exists: a payer the lane has no room for is told
paid_lane_full, the code its refusal already carried, in X-Hanzo-Lane-Reason and
X-Hanzo-Usage-Reason. Stores keep an orphan paid_day table nothing reads.

Kept: the wallet hold (reserveFor), the plan class's LimitGrant.Spend less the
payer's calls in flight, quote sizing against it, 8 seats a payer, lapse and
stream-idle give-back, race losers holding the seat, BYO, and the free-model
hand-off for a payer with no room.

The replica guard. The lane was closed unless CLOUD_API_REPLICAS=1 because the day
lived in this process. No deployment sets CLOUD_API_REPLICAS: the cloud chart
injects CLOUD_REPLICAS (1 in production). So the boot assertion meant to stop a
second replica double-spending the in-pod wallet ledger never fired, and the lane
guard was the one live check; it also kept the paid lane closed in production. The
plan holds that remain are in-pod exactly as the wallet ledger is, so they take its
guard: SinglePodReplicaHint reads CLOUD_REPLICAS, and bootstrap refuses to start
when the deployment states more than one replica. A declared second replica stops
the process instead of closing the lane, and no switch is left whose only purpose
was the day.
2026-10-09 14:08:38 -07:00

6.3 KiB

Hanzo AI

ai

ai is the AI subsystem of the Hanzo cloud: it serves the /v1 inference surface, routes each request to a model, meters it, and carries the RAG and model-hub surfaces alongside. Pure Go.

Release License

It used to be the whole hanzoai/cloud repository. When the unified binary took the cloud name (HIP-0106), this became ai and now mounts as the ai subsystem inside it. That is why the root of this repository is package ai, a library, and not a main.

Call it

The hosted instance is api.hanzo.ai. Get a key at hanzo.ai:

curl -H "Authorization: Bearer sk-YOUR-API-KEY" \
  https://api.hanzo.ai/v1/chat/completions \
  -d '{"model":"zen5","messages":[{"role":"user","content":"Hello"}]}'

Three auth modes are accepted: a Hanzo API key (sk-*), a JWT from hanzo.id OAuth, or a provider key you already hold, in which case the request is routed with your account rather than metered on ours.

Run it yourself

docker run -p 8000:8000 ghcr.io/hanzoai/ai:vX.Y.Z    # pin a release, never :latest

Or build the server binary. It is cmd/aid — building the repository root gives you a library archive, not something you can run:

CGO_ENABLED=0 go build -o aid ./cmd/aid
./aid -h

compose.yml brings up the server with its Postgres.

Models

The gateway owns the catalog; this service does not keep a second list. Ask it:

curl -H "Authorization: Bearer sk-YOUR-API-KEY" https://api.hanzo.ai/v1/models

https://catalog.hanzo.ai/v1/models answers the same question without a key. Zen is our own family — enso, enso-ultra and the zen5 line, owned_by: hanzo. Models you connect with your own provider keys are routed as well, resolved from Hanzo KMS and scoped to your org.

Auto-routing

Send "model": "auto" (alias "zen-router") and the request is classified into a coarse task — code, reasoning, math, creative, vision, long-context, cheap-chat, general — and mapped to the first servable model in the matching preference list, before pricing and billing resolve. So an auto request is billed and reported as exactly the model that served it.

Turn it on in conf/models.yaml (the cloud-api-models ConfigMap in production), or with ROUTER_ENABLED=true / ROUTER_ENDPOINT=<url>:

router:
  enabled: true
  endpoint: ""          # "" = built-in heuristic; a URL = the learned router
  cost_ceiling: 0.0
  prefer:
    code:       [zen5-coder]
    reasoning:  [enso-ultra, enso]
    cheap_chat: [zen5-mini, zen5-flash]
    default:    [zen5]
curl -H "Authorization: Bearer sk-YOUR-API-KEY" \
  -H "X-Max-Cost: 2.0" -H "X-Max-Latency-Ms: 800" \
  https://api.hanzo.ai/v1/chat/completions \
  -d '{"model":"auto","messages":[{"role":"user","content":"refactor this function"}]}'
# response header: X-Routed-Model: <the model that served it>

X-Routed-Model and the model field in the body always report what actually ran. X-Max-Cost (per 1k, float) and X-Max-Latency-Ms (int) express a per-request budget.

Two strategies sit behind one interface. With endpoint set, ai POSTs {prompt, tasks?, slo} to {endpoint}/route and takes its {model, task, confidence} reply — that is the learned router, zenlm/zen-router. On any error, or with no endpoint configured, it falls back to a built-in Go heuristic, so auto works with no extra infrastructure.

Per-org override

Orgs can opt in or out through the admin OrgSettings surface (/v1/*-org-settings, global-admin-gated like /v1/*-model-route). The org row carries autoRouting: "" (unset), "enabled" or "disabled", and blends with the global flag:

global router.enabled org "" org "enabled" org "disabled"
true route route off — no rewrite, no header
false off route¹ off

¹ Per-org opt-in routes even when the global flag is off, but only when router config is present (a prefer table or an endpoint) — otherwise there is nothing to route with and auto is left unchanged.

Configuration

conf/app.conf, or environment variables:

Variable Description
HANZO_API_KEY Service token for internal IAM and KMS calls (billing, usage, auth)
iamEndpoint IAM service URL
clientId / clientSecret IAM OAuth client credentials for ai
dataSourceName Database DSN — inject from KMS, never commit it
KMS_CLIENT_ID / KMS_CLIENT_SECRET KMS Universal Auth credentials
KMS_PROJECT_ID Default KMS project
KMS_ENVIRONMENT KMS environment (default production)

Secrets come from Hanzo KMS; the deploy workflow resolves them at run time from a single HANZO_API_KEY GitHub secret, with Universal Auth as the fallback.

ai runs single-replica: its balance ledger is an in-pod invariant. Deploy with strategy: Recreate and min=max=1 — a boot assertion panics if CLOUD_REPLICAS > 1. LLM.md has the scale-out path.

Development

go build -race ./...                       # build
go test $(go list ./...) -tags skipCi      # test (needs MySQL)
golangci-lint run                          # lint

cd web && yarn install && yarn start       # the admin UI

LLM.md is the deep reference — architecture, the mount seam into cloud, and the conventions that apply here. ZAP.md covers the transport.

Lineage

NOTICE. The routing, auth, billing and model surfaces are ours; the admin UI and knowledge-base scaffolding started there.

License

Apache-2.0 — see LICENSE.


Hanzo — the open AI cloud. hanzo.ai · docs.hanzo.ai

SDKs: Python · TypeScript · Go · Rust · C++ · Swift · Kotlin · umbrella