Files
ai/LLM.md
zeekay 4c6933c2a0 A refused Enso tier falls to enso-free first
Live 2026-10-27: enso-ultra refused (its Anthropic key out of credit) fell
to the pool's nemotron-3-ultra:free, relayed as it answered, and forced
calls came back as text. enso-free is the engine's free lane, which judges
a free answer and reads a forced call written out as text as the call; a
refused Enso tier now asks it before the pool's own routes.
2026-10-27 01:54:20 -07:00

121 KiB
Raw Permalink Blame History

ai — agent guide

hanzoai/ai is the canonical AI control plane for the Hanzo platform: model hub, native Go model routing, RAG, and MCP/A2A management. It speaks the OpenAI-compatible /v1 API, routes 66+ models to upstream providers, and meters every request. Renamed from hanzoai/cloud (HIP-0106); mounts as the ai subsystem inside hanzoai/cloud. In prod it runs as cloud-api on hanzo-k8s, fronted by hanzoai/gateway at api.hanzo.ai.

Canonical role. This is a Hanzo service/infra repo — one impl, one place. It is NOT an SDK; SDKs link out to it. Completeness order across languages is Python → Rust → C++ → Go. Canonical spec: ~/work/hanzo/SDK-ARCHITECTURE.md.

Brand rules (hard — enforce in every edit).

  • Never call this an "LLM gateway" and never position it against LiteLLM — it is a full AI cloud / control plane, not a proxy.
  • /v1/ only, never an /api/ prefix.
  • Zen models are Hanzo's own family (owned_by: hanzo) — never name upstream models.
  • Voice: "Hanzo — the Open AI Cloud." Modern, crisp, developer-first.

Install / run.

go build -race -ldflags "-extldflags '-static'"   # build
./cloud-api-server                                 # run (env-configured)
go test -v $(go list ./...) -tags skipCi           # test (requires MySQL)
docker compose up                                  # local stack

Key entry points. main.go (entry) · bootstrap.go (boot + replica assertion) · routers/router.go (all /v1 routes) · controllers/ (HTTP handlers) · object/init.go (init + LLM provider seeding) · model/ (provider integrations) · object/kms.go (secret resolution) · web/ (React admin UI).

Architecture

Full-stack application:

  • Backend: Go 1.26 on github.com/zap-proto/zip (over fiber v3 / fasthttp), MySQL/MariaDB/PostgreSQL
  • Frontend: React + Ant Design v5, located in web/ (the folder is the React app; the in-house Go router that used to share the name is gone)
  • Auth: Hanzo IAM SSO integration

Directory Structure

cloud/
├── main.go                 # Entry point
├── go.mod                  # Go module
├── conf/app.conf           # Beego config
├── controllers/            # HTTP handlers (60+ files)
├── object/                 # Business logic & data models (85+ files)
├── model/                  # AI model provider integrations (38+ files)
├── routers/                # Routes and middleware filters
├── embedding/              # Embedding provider implementations
├── agent/                  # MCP agent support
├── split/                  # Text splitting strategies
├── txt/                    # Document parsing (PDF, CSV, XLSX, PPTX)
├── storage/                # File storage abstractions
├── util/                   # Utilities
└── web/                    # React frontend
    ├── src/
    └── package.json

Build & Run

Backend

# Build
go build -race -ldflags "-extldflags '-static'"

# Cross-platform build (linux amd64/arm64/riscv64)
./build.sh

Frontend

cd web
yarn install
yarn start       # Dev server
yarn run build   # Production build

Docker

docker compose up

Testing

# Go tests (requires MySQL)
go test -v $(go list ./...) -tags skipCi

# Frontend tests
cd web && yarn test

Linting

# Go - uses golangci-lint with gofumpt
golangci-lint run

# Frontend
cd web && yarn lint

Key Conventions

Backend (Go)

  • Framework: zip. An ApiController IS its request — it embeds *zip.Ctx, so there is no context to marry to a controller and no recorder to read afterwards. Controllers handle HTTP, objects hold the business logic.
  • New AI providers go in model/ (implement the provider interface)
  • Database access via the native store in object/
  • Route registration: CRUD resources are GENERATED from the one table in routers/resources.go; routers/router.go holds only what is NOT a resource (the OpenAI-compatible surface and a few singletons). See "The /v1 resource surface" below.
  • i18n strings: avoid duplicate keys across frontend and backend

Five things zip decides that the old router did not

Each cost a real defect during the migration, and each is invisible: the code compiles, the types check, and the behaviour is wrong.

  • A WRITE CARRIES ITS STATUS. c.JSON(code, v) / c.Bytes(code, b) set the status as they write, so setting one beforehand and writing after replaces it. c.Status is for a response whose body is written by something that takes no status — a stream — or that has no body at all. Everything else passes the status to the one write. A refusal that arrives as 200 is worse than an outage: the client reads it and believes it.

  • A STREAMED BODY IS PRODUCED AFTER THE HANDLER RETURNS. fasthttp drains the writer handed to SendStreamWriter while it serialises the response, so the callback has NOT run when the call comes back — measured, not assumed. Anything the stream reads must outlive the handler (hand the upstream body to the callback; a defer in the enclosing function closes it first and the client gets nothing), and anything the stream learns must be used inside it (token counts read outside are still zero, so a streamed answer bills no completion).

    AND THE CTX IN THERE IS NOT THE REQUEST'S — this line used to say it was, and production refuted it. The claim was that fasthttp finishes writing before fiber releases the request, so reading c from the callback was safe. Measured on api.hanzo.ai: DefaultCtx.App() nil-dereferenced under recordFamilyUsage -> billingOrg -> Ctx.Header, twelve times an hour. It does not fail the one request either — a SIGSEGV takes the whole plugin process, so every other call in flight dies with it and each one answers mount /v1: http: read response: EOF. Read what the stream needs from the request BEFORE the handler returns and let it travel by value; whence in zen_client.go is that shape, and recordFamilyUsage takes no receiver so there is nothing to reach through.

  • RequestURI IS SET BY A SERVER, NOT BY A CALLER. The fiber adaptor reads that field, so a request CONSTRUCTED in process — which is every request crossing ZAP — arrives at the router as / and 404s. controllers.target wraps a handler to fill it in from the URL; both ZAP installers go through it.

  • A BUFFERED WRITE IS NOT A DELIVERY. Bytes leave at the flush. A Write that returns n > 0 has reached a buffer, and an unchecked Flush reports a broken connection as success — which is how AnthropicWriter.StreamSent came to tell the failover loop an answer had reached a client it never reached.

  • A PRINCIPAL IS A VERIFIED TOKEN, and there is nothing else to seed. No session store exists; GetSessionClaims parses the credential the request presents. So a test that needs a caller PRESENTS one: presenting(visit(method, target), authtest.Bearer(t, user)). internal/authtest is the ONE installer of the verifier's certificate — two of them race over the same global. Corollary: any predicate that used to mean "there is a session" now fires for ANY verified token, a sibling brand's included.

Frontend (React)

  • UI components use Ant Design v5
  • i18n via i18next; translation files in web/src/locales/
  • State via React Context/Hooks (no Redux)
  • API calls go through web/src/backend/ helper modules

Commits & PRs

  • Branch naming: claude/<name>
  • Main branch: main
  • Follows semantic versioning (.releaserc.json)
  • PR titles should be semantic (feat/fix/chore/docs/refactor)

CI/CD (GitHub Actions)

.github/workflows/build.yml runs:

  1. Go unit tests (with MySQL service)
  2. Frontend build (Node.js 20 + Yarn)
  3. Go binary build
  4. golangci-lint
  5. Semantic release + Docker push to ghcr.io/hanzoai/cloud

Configuration

  • Backend config: conf/app.conf read by the native conf.AppConfig ini reader (env-first; absent in the embedded runtime, which runs env-only)
  • Database: MySQL 8.0+ or MariaDB
  • Default Docker DB: PostgreSQL via compose.yml

Important Files

File Purpose
main.go Application entry point
routers/router.go All API route definitions
object/init.go Application initialization + LLM provider seeding
conf/app.conf Runtime configuration
web/src/App.js Frontend root component
web/src/backend/ API client helpers

hedge — ask several providers, keep the first answer

hedge.Race(ctx, dst, attempts...) is the primitive behind fast mode. Tail latency is not the average: a provider that usually answers in 400ms occasionally takes eight seconds, and a request cannot know which it drew until it has already waited. Asking three and keeping the first cuts that tail.

No provider changes, and none are needed. Every provider already writes into an io.Writer, so Race hands each attempt a different one and only lets the winner's reach the caller. ModelProvider is untouched — the seam was already the right shape.

FIRST BYTE WINS, the only definition of fastest a stream can act on. A provider that will finish sooner but has not started is not the winner: the caller waits on bytes, not completion, and by the time completion is knowable the response has arrived and there was nothing to race.

Two cases decide whether hedging helps or hurts, and both are pinned by tests verified against a deliberately broken implementation:

  • An EMPTY write does not claim. A provider flushing a zero-length chunk while still waiting upstream would otherwise win having said nothing, leaving the caller on the SLOWEST attempt with the others cancelled — the opposite of the point.
  • A FAST FAILURE is not a win. Race returns when the attempt that CLAIMED the stream finishes, not when the first attempt returns; an upstream 503 in 1ms produced no bytes, so the caller is still waiting on whoever answers.

Losers are cancelled — that is what stops the upstream call, which is what stops the cost — and Race does not wait for them to unwind, because waiting on a provider we are abandoning hands it exactly the latency this removes.

One ledger path each, because the two losers are opposite cases

ask.fan (controllers/hedged.go) carries a request into fast mode: how many to ask, where the winner's bytes go, and how to make and adopt a writer. serve() dispatches on it, so a caller sets a NUMBER rather than choosing a code path, and a fan too narrow to race falls back to the cascade with its retry and demotion intact. Nothing sets it yet — the mechanism ships inert, and TestWithoutAFanNothingChanges is what says so.

A raced loser and a failover refusal wear the same shape and mean the opposite thing, which is the whole design:

  • A REFUSAL errored. No tokens, no upstream charge — recordRefusals writes a "failover" row that bills nothing, and that is correct.
  • A RACED LOSER was CANCELLED mid-generation. The vendor produced tokens and charged us. Filing it as a refusal records a real charge as free and makes fast mode invisible in the ledger the team reads.

So a beaten provider never appears in the attempts race() returns — the caller records those as refusals — and settles through fan.bill instead.

Each attempt needs its OWN writer. The loser's token count is on ITS writer, and one writer shared across attempts holds exactly one answer: the winner's. A shared writer therefore bills every loser for the winner's tokens AND glues its half-sentence to the winner's answer on the client's wire — both failures show up in TestEachRacerIsBilledFromItsOwnWriter, verified against a deliberately shared fork. OpenAIWriter.fork/adopt are that pair: forks copy the dialect and start the buffers empty; the winner's is adopted so the non-streaming body and the tail chunk come from the provider that actually served.

THE BILL ARRIVES LATE, and that is the shape rather than a wrinkle. Race returns when the winner finishes, not when the losers do — so at that moment a loser has typically not even started, and reading its outcome is both a missing bill and a data race. (The first draft did exactly this; the test caught it.) settle therefore runs on its own goroutine, and fan.bill must close over VALUES: by the time it fires the handler has returned and, on fasthttp, the request has been recycled.

Cancellation is a WRITE ERROR, not a context. QueryText takes no context, so a loser's only stop signal is hedge.ErrLost on its next write. One that checks its write error stops there; one that does not runs to completion and bills us for all of it. Either way the tokens are real — which is the argument for billing them.

The switch, and why it is decided with the reservation

A request asks with "fast": true in the body, or X-Fast: 1 for callers with no browser in the way — the body form exists because a page cannot send a custom header past the edge's CORS allow-list, exactly as retrieval works (controllers/fast.go).

What a race costs is settled WITH the hold, never after. widthFor reserves est × fastWidth before anything is spent, because N attempts is N completions and a hold sized for one is a promise the ledger cannot keep — the losers' rows land later and take the balance past the point the gate said it would stop. fastWidth is 2: the tail this cuts is one provider drawing a slow response while another would have answered normally, and a second opinion removes most of it for one extra completion rather than two.

Two refusals that are deliberately NOT refusals:

  • Cannot afford the race → serve the ordinary request. The caller asked to go faster, not to be turned away, and a new way for a funded account to be told no is worse than quietly being normal speed.
  • Nobody to bill → never race. The public lane makes no reservation, so hedging there spends N completions on a stranger and settles none.

Still open: the toggle in the products. hanzo.chat follows web_search; hanzo.app follows base and must resolve its in-flight merge on the three AI-path files first. The API accepts the field today, so either can ship the control without waiting on the gateway.

The /v1 resource surface — ONE table, no compound routes

The published OpenAPI description is GENERATED from that same table:

go run ./cmd/openapi -spec ../openapi/ai/openapi.yaml

It writes only the region between the # BEGIN/END generated markers in the canonical spec (hanzoai/openapi, OpenAPI 3.1) — the inference paths above it are hand-authored with real OpenAI request/response schemas and stay that way. cmd/openapi -verify and TestSpecMatchesTheTable fail when the spec drifts from the table, so the contract customers and SDKs read cannot describe a surface this service does not serve.

The old swagger/ directory is DELETED. It was a second description, and it was wrong: basePath: /api with 135 /<verb>-<noun> paths, a base path this service has never served. Nothing referenced it, nothing embedded it, and no test noticed when it stopped being true — which is the argument for generating the replacement rather than hand-maintaining it.

Every CRUD route is generated from routers/resources.go. There is no second registration path, and no route is written by hand.

GET    /v1/<ns>/<resource>                 list      POST   /v1/<ns>/<resource>          create
GET    /v1/<ns>/<resource>/<owner>/<name>  read      PATCH  …/<owner>/<name>            update
                                                     DELETE …/<owner>/<name>            delete
GET    /v1/<ns>/<resource>/global          cross-tenant list (where a resource has one)

The namespace is ai for every row, and for every other route this service serves. HIP-0139 §3.1: a capability answers at ONE address, /v1/<name>, and a second top-level address is a second capability or a misfiled route — never an alias. So /v1/router/*, /v1/org/settings, /v1/finetune/*, /v1/memory/*, /v1/rag/*, /v1/docs/ingest, /v1/feedback and /v1/traffic/globe are all under /v1/ai/… now (ingest under /v1/ai/rag/, since it writes the index rag reads). The groupings in the table — retrieval, chat, models, content, compute, work, ops — are how it is READ, not a second prefix.

Two families stay outside /v1/ai because a protocol, not this repo, fixes their spelling (§3.2): the OpenAI/Anthropic wire (/v1/chat/completions, /v1/completions, /v1/embeddings, /v1/messages, /v1/models, /v1/images/*, /v1/audio/*, /v1/videos/*, /v1/responses, /v1/rerank), and POST /v1/embed, the multipart form hanzo.chat posts.

/v1/ai/<noun> is a resource collection address. Nothing may register a ZAP gateway PREFIX there: the registry matches on path alone, so a prefix at a collection swallows that collection's :owner/:name members and answers with the LIST handler — a wrong answer, not an error. Deeper addresses (/v1/ai/rag/query, /v1/ai/router/defaults) are routes, not collections, and are safe. TestNoGatewayPrefixOverTheResourceNamespace holds the line; it is why /v1/ai/feedback has no gateway prefix and reaches routers.App through the fallback instead.

Three things about this that are easy to get wrong:

The member key is <owner>/<name> — two path segments, not one. Every object is keyed by that pair (util.GetOwnerAndNameFromIdWithError requires EXACTLY two tokens, so a name containing a slash was never valid — the two-segment URL is lossless, not a narrowing). They are recomposed into id in the one place route params are bound (web.Router.ServeHTTP), so handlers keep reading c.Input().Get("id") unchanged. A single :id segment cannot work: Go decodes %2F back into a separator before routing.

/v1/iam belongs to the IAM service, not to this one. cloud proxies that whole subtree away (cloud/iam_edge.go). Anything registered there is shadowed and unreachable — it fails as a wrong answer, not an error. TestNoResourceClaimsForeignNamespace holds that line.

The policy filters key on names, not paths. authz_filter and filter_balance hold hand-curated sets (super-admin endpoints, balance exemptions, demo-mode allowances). Those names are DERIVED from the same table by policyKey, never restated — so a policy entry cannot drift from the route it governs. Do not hand-edit a path into those maps.

Adding a resource is one table entry. The tests will tell you if you got it wrong: every controller method the table names is checked to exist by reflection, every route must emit a policy key, no path may contain a verb, and an action whose handler reads no id must sit on the collection (a member action's route could never match).

An OpenAI-COMPATIBLE vendor is upstream Local, never OpenAI

Provider.Type IS the dialect declaration, and the two OpenAI-shaped upstreams are not interchangeable:

type client dialect price from
OpenAI NewOpenAiModelProvider /v1/responses — OpenAI's own surface CalculateOpenAIModelPrice, a table of what OpenAI RETAILS
Local NewLocalModelProvider /v1/chat/completions (local.go, CreateChatCompletionStream) the provider ROW (inputPricePerThousandTokens), plus compatibleProvider for the API it speaks

DigitalOcean already resolves to the Local client; so should every vendor that merely SPEAKS OpenAI. Typing one OpenAI sends it to the Responses API, and every compatible vendor shims that differently.

Measured, one provider (the Hugging Face router), one credential, one URL — typed OpenAI: meta-llama/Llama-3.3-70B-Instruct answered, while zai-org/GLM-5.2, deepseek-ai/DeepSeek-V4-Flash-0731 and moonshotai/Kimi-K3 returned an EMPTY message with a 200 and zero usage. All four answer directly, in both dialects. Retyped Local, all four answer through the plane, reasoning models included, with no <think> leaking into content — and no code change. The reasoning streams carry response.reasoning_text.delta, which the Responses loop does not case on (it handles only response.reasoning_summary_text.delta).

So the rule is a ROW, not a patch: an OpenAI-compatible vendor is Local with its own URL and its own price. Reasoning stays a FIELD the dialect carries, never something sniffed out of the content stream.

Three things still dispatch on substrings of the model NAME, and a model in none of the lists gets wrong behaviour rather than an error — getOpenAiModelType (which dialect), InlinesReasoning (strings.Contains(name, "deepseek"), directly under a comment saying which SKU inlines is "a property of the catalog (HIP-0039) … never a list here"), and the retail price table. Each is a place a new model silently misbehaves; the Local row above is what they were compensating for.

LLM serving path (OpenAI-compatible /v1) — READ THIS

The ai module is mounted by hanzoai/cloud (HIP-0106). In prod it runs as cloud-api in hanzo-k8s, fronted by gateway (api.hanzo.ai). Both hanzo.chat (LibreChat, baseURL https://api.hanzo.ai/v1) and hanzo.app (build) consume this exact surface.

Request flow for a completion:

  1. gateway (api.hanzo.ai) validates the IAM JWT, mints identity headers, forwards /v1/chat/completions → cloud-api:8000. /v1/models is also proxied here; admin endpoints (/v1/get-providers, /v1/update-provider, /v1/*-model-route) are NOT gateway-exposed (use the direct api.cloud.hanzo.ai ingress + a session).
  2. controllers.ApiController.ListModels (/v1/models) requires a valid Bearer token — unauthenticated returns 401 with no data array (this looks like "0 models"; it is not — authenticate to see the real list).
  3. Model → provider routing: resolveModelRouteForOrg resolves in order DB routes (/v1/*-model-route, per-org → global "admin") → YAML config (conf/models.yaml, the cloud-api-models ConfigMap — runtime source of truth in prod) → static controllers/model_routes.go map. The YAML wins, so changing routing live = edit the ConfigMap (then /v1/reload-model-config or restart). The static map is the fallback when no YAML is present. A models.yaml entry with alias_of: <id> is an alias: not a route, not listed. AliasFilter rewrites a POST body naming it to name <id> ahead of every other reader, so it is routed, gated and billed as <id>. An alias of a zen or enso SKU belongs in that family's catalog instead: its /v1/models states it (aliases) and the family route resolves it, after this chain, so a row here for the same id wins. /v1/decisions sends the service the route's upstream and answers naming the id asked for.
  4. Provider records live in the DB (object/init.go initLLMProviders), keyed by name: do-ai (DigitalOcean GenAI, the primary — backs OpenAI/ Anthropic/Llama/DeepSeek/Qwen/GLM/Kimi via inference.do-ai.run), fireworks, openai-direct, zen. Each call does object.GetModelProviderByName(route.providerName) → reads ClientSecret.
  5. Provider keys: ClientSecret is kms://SECRET_NAME. ResolveProviderSecret (object/kms.go) resolves it env-var-first (os.Getenv(SECRET_NAME)), falling back to a real KMS call — BUT only runs when kms != nil, which needs KMS_CLIENT_ID or KMS_SERVICE_TOKEN set. So the working prod recipe is: set KMS_CLIENT_ID (flips kms on) + provide DO_AI_API_KEY/OPENAI_API_KEY/ ANTHROPIC_API_KEY/FIREWORKS_API_KEY as env (from a K8s secret sourced from KMS-managed values). The env-first path means no live KMS dependency on the hot path.
  6. Provider re-seed self-heals ClientSecret/ProviderUrl/State/Type/ SubType on every boot from the initLLMProviders table — so fixing a stale key or upstream URL = edit the seed table and restart (no manual DB edit).

Vendor video (video_spec.go, video_vendor.go). The resale family lists OpenRouter's video models too (lists: /v1/models, /v1/videos/models, one catalog; a second list that answers 404 lists nothing): each entry with price SKUs and durations is a clip SKU (zenModel.Video, outputs: ["video"], Variable, both paid floors, its card the SKUs × markup); an editor or upscaler (no durations) is not listed. /v1/models carries its video object (durations, resolutions, aspect ratios, sizes, frame images, audio). POST /v1/videos/generations for one is admitted against that object (400 names what it takes), reserved at its ceiling (the dearest SKU rate that applies to the resolution, audio and images asked, × seconds, × markup: clipQuoteNano, which the paid lane's quote reads too), and posted to OpenRouter's POST /v1/videos on the first account not cooling. The job is a row (object.MediaJob) and a watcher (clipJob.watch, every clipPoll): done, it copies the file into the host's object store (object.Assets, Hanzo S3 from cloud), reads duration and frame size from the MP4 (mp4.go), and bills once what the vendor stated (usage.cost × markup; the quote when none is stated), on the create's context; failed, it releases the hold and bills nothing. GET /v1/videos/{id} answers the row (Accept: text/event-stream streams each change), /content 302s to a fresh signed URL, both held to the payer's subject. ResumeVideos (bootstrap) watches every unsettled row a process before left, billing it to the payer's cash. A listed media model with no token price (Lyria) is perCall: Variable, priced, never free.

Vendor voice (voice_vendor.go, audio_api.go vendorVoice). The same family reads OpenRouter's speech list (/v1/models?output_modalities=speech): each speech-only entry is a voice SKU (zenModel.Voice: its voices, its per-character and per-output rates; Variable and both paid floors unless the vendor serves it free). POST /v1/audio/speech for one takes the speech admission, reserves the most its text can cost (voiceCeiling: every character, and the longest the speech can run at its output rate), posts OpenRouter's /v1/audio/speech on the first account not cooling (orSend, which clipCall uses too), answers the audio, and bills GET /v1/generation's total_cost × markup, else the listed per-character rate, else the reservation.

Async video (/v1/videos) — Sora-style, NOT sync. Text-to-video generation takes minutes, so /v1/videos/generations is ASYNC (the sync handler used to hold the request ~104s → console /ai proxy timed out → browser 502 while the generation actually succeeded). The lifecycle is three fast requests: POST /v1/videos/generations (create → returns a video_<uuid> job id + queued object immediately), GET /v1/videos/{id} (one status poll), and GET /v1/videos/{id}/content (stream the finished MP4). zen3-video* route to the spark-video provider (our GB10, spark-video.hanzo.ai/v1), whose OWN /v1/videos API is already the same async create→poll→download shape the three model.{Create,Retrieve,Download}VideoDOAI primitives proxy. Metering is per-user and lands EXACTLY ONCE on completion: create RESERVES the per-video budget and carries the *budgetHold in the in-pod videoJobStore (valid — the pod is single-replica, the balance ledger's own invariant); the first request that observes completed (poll or download) settles the hold + records the single Commerce usage event (videoJob.markCompleted, idempotent); a failed job releases the hold and bills nothing; a reaper releases abandoned/wedged jobs' holds (TTL) so budget never leaks. Every poll/download re-runs the shared auth+routing policy and enforces OWNERSHIP (caller subject == job subject, else 404 — never confirm another user's job); the provider key is re-resolved per request, never stored. The gateway passes every /v1/* sub-path through to cloud-api, so the poll/ content routes need no gateway change; the console /ai proxy allow-list adds the v1/videos/{id}[/content] pattern (hanzoai/console#92).

zen models: branded, owned_by: hanzo, premium: true. Route to do-ai upstreams (qwen3+/glm/kimi/deepseek — same working key, no GPU). Each zen model declares its public owner in owned_by; the identity is injected at call time via zenIdentityPrompt (hip-00NN). Do NOT route zen to Fireworks serverless paths (accounts/fireworks/models/*) — that account returns 404 "not deployed" for most of them.

Billing gate (routers/filter_balance.go + openai_api.go): non-premium models need balance > 0; premium models need balance > starter credit. Balance comes from commerce at /v1/billing/balance?user=<org>&currency=usd. No principal bypasses it: the gate always consults the balance, and a lookup that fails returns 500 rather than free access. To verify premium/zen, credit the org.

/v1/responses turns a namespace tool (Codex declares each MCP server's tools as one) into one chat function per tool it holds, named <namespace>__<name> and hashed to fit 64 characters (flatName); a call comes back with the namespace and name the client routes by (unflat), and a namespaced call in the history is named the same way. Sending the namespace as one empty function left a model one schemaless tool per server, and Codex refused every call it made (unsupported call: mcp__hanzo).

The chat relay — the caller's body, as written (controllers/forward.go)

Every non-family /v1/chat/completions (and /v1/responses, which becomes one) whose route row relays — an OpenAI-compatible chat address, not Anthropic's dialect — is sent the CALLER'S OWN BODY. Plain text, tools and images take the one path. The relay writes four things and only these: model (each provider's own id), the completion ceiling when the hold covers less than the caller asked (under the caller's key; when they named none, max_completion_tokens to an OpenAI row and max_tokens elsewhere), stream_options.include_usage on a stream, and retrieved knowledge as one leading system message. ours (fast/retrieval/retrieval_store) is read here and not sent on. unpriced (models, provider, transforms, route, plugins, web_search_options, service_tier: fields that buy something on OUR account the SKU price does not cover) is REFUSED 400 by name on every path, before any vendor is asked, never stripped. Nothing else is touched: every turn, temperature 0, seed, response_format, stop, vendor extras. chatRequest holds response_format and stop raw because go-openai cannot decode a json_schema or a string stop, which used to 400 before routing.

It fails over the way ask.cascade does — same candidates, retryTransient, cooled, announce, recordRefusals, exhausted — and every decision is made on the STATUS, before a byte is written. A stream is judged by its opening frames (opening, the family's rule): an error inside a 200 moves the request; the first answer frame commits it, and nothing after that moves it. dial is the seam the cascade tests script. Billing is by the SKU, with prompt_tokens_details.cached_tokens split out and priced as cached.

One prompt, on every path. usageRecord.PromptTokens is EVERY input token; CacheReadTokens and CacheWriteTokens are parts of it, read from and written to the upstream's prompt cache. Each usage shape is read into that one split (sniffZenUsage into tokens{fresh, cached, written}, AnthropicUsage.billed into tally.prompt()): OpenAI counts prompt_tokens_details.cached_tokens inside prompt_tokens, OpenRouter its cache_write_tokens there too, Anthropic counts cache_read_input_tokens and cache_creation_input_tokens beside input_tokens. Pricing bills the input rate on the part read fresh (prompt less reads and writes), reads at the cache-read rate and writes at the cache-write rate: a family SKU's cache_read / cache_write (a resold model's input_cache_read / input_cache_write times the markup), else the input rate; the table path's CacheReadPerMillion / cacheWriteRate. Before, the family and relay paths recorded the fresh part as the prompt while the table path read the whole, so cache_read > prompt on most cached rows, and no path recorded a cache write: a write, priced above input, billed as a fresh read. Held by TestEveryUsageShapeCountsThePromptWhole and TestACacheWriteBillsAtTheWriteRate.

A relayed stream OWNS ITS HOLD. The callback settles the real cost after the handler has returned, and budgetHold.settle is one-shot, so the handler's deferred release would win and the local ledger would never see a stream's spend. forward reports that it handed the hold to a stream and the handler lets go; the callback releases it on the way out if it ends without settling. The text pipeline and the family pipe still release before their streams settle.

The relay does not race. A relayed attempt is committed at its status, so a fast request takes the cascade; the reservation widthFor made covers it. Anthropic-type rows and endpoint-less rows keep the older paths (proxyToolRequestAnthropic, the QueryText cascade). An Anthropic-type FALLBACK on a relayed route is passed over, not converted.

A family service: what ai reads from it (controllers/zen_client.go, zen_media.go)

Zen and Enso are served OUTSIDE ai, by a family service at ZEN_URL / ENSO_URL (the Go zen binary today; the Rust one next). ai authenticates, gates, meters and forwards; the service owns which upstream answers. The contract, one address each:

  • GET /v1/models: {"object":"list","paid":<bool>,"aliases":{"<id>":"<sku>"}, "data":[...]}. paid is the catalog's line, the platform's paid switch (zen's only). aliases are the other ids the family answers to (claude-zen → zen6, hanzoai/zen → hanzo/zen): routed, priced and gated as their SKU, sent upstream as the SKU, answered wearing the alias, never listed; one that spells a listed SKU is that SKU. Each data row: id, owned_by, mode (chat | embedding | rerank | image | audio | video), context_window, pricing {input, output, cache_read, cache_write} as decimal strings (per 1M tokens; per unit for media), pricing_tiers, access (waitlist), min_tier, funding (prepaid where any path spends a prepaid balance), plan, capabilities.vision.
  • POST /v1/chat/completions, /v1/messages, /v1/embeddings: the body ai sends (familyBody), X-Org-Id the org the call is for (a machine credential acting for a person: that person's org, tenant), X-Hanzo-Fronted-By: ai (ai bills; the family meters nothing), X-Hanzo-Spend + TE: trailers for a plan's paid rungs.
  • POST /v1/rerank, /v1/images/generations, /v1/audio/{voice,music,foley}, /v1/videos/generations: buffered, billed per unit (zenMedia, the one media relay for HTTP and ZAP).
  • Back, on a fronted call: X-Hanzo-Served (the Hanzo SKU), X-Hanzo-Arm (the upstream), X-Hanzo-Provider, X-Hanzo-Free: true when the rung bills nothing (the call then bills nothing), X-Hanzo-Failover (each arm that failed, <upstream> (<provider>): <reason>, one field an arm or joined by "; "), and X-Hanzo-Cogs: what the request cost Hanzo upstream in USD, every arm it ran included — a header on a whole answer, a trailer (declared in Trailer:) on a stream. A committed plan answer adds X-Hanzo-Cost-Bound and the X-Hanzo-Cost trailer (plan.go).
  • GET /v1/admin/zen/catalog, PUT the same, GET /v1/admin/zen/keys, GET /v1/admin/zen/free, under Authorization: Bearer <ZEN_ADMIN_TOKEN>: the catalog a SuperAdmin edits at /v1/ai/router/catalog (routing.go), for zen whenever no zen is linked into the process. The engine (enso) answers the same under /v1/admin (/v1/admin/catalog, /keys, /free), and routing.go asks each family at its own.

What ai does with it: every priced answer settles EXACTLY, in nano-dollars (retailNano, unitNano, rounded up, never floored to a cent): the hold (settleNano), the debit (BilledNanoExact) and the row read one number, so an embedding at $0.01/MTok bills $0.00001 a thousand tokens and a $0.001 rerank bills $0.001. A hold still reserves whole cents (centsUp). X-Hanzo-Cogs is the row's and the debit's COGS (CostNanoExact; UsageEvent.CostUSD reads usageMargin, the same number), and each failed arm is a "failover" row beside the answer under the same request id, billed nothing (failoverRows), as the in-process zen filed them. A test that drives this whole contract stands up zenService in family_service_test.go.

A family in process (plugin/)

A family service may run in this process instead: its engine linked as a Go package (hanzo-inc/zen lib/zen and lib/enso: the Rust engine compiled to wasm32-wasip1, translated by wasm2go, behind an http.Handler that pumps the module, makes its upstream calls with the keys it is opened with, and writes its answers) and registered with plugin.Register(name, handler), once per family per process (one instance holds the family's key pool, lane health and sessions). The address decides: ZEN_URL=plugin://zen is the handler registered as zen, any http(s) address the service over the network. ai's family, discovery and decision clients carry the scheme (reaching), so relay, tells, billing, failover rows, caching and attribution are the HTTP path's, unchanged. One transport per hop.

  • plugin.Transport serves the request on the registered handler under the caller's context, which also ends when the caller closes the body early, so the engine hangs up on its upstream when its caller leaves. The body crosses through a pipe (a caller that stops reading stops the handler's writes); the head is handed over at the first WriteHeader, Write or Flush; declared trailers (and http.TrailerPrefix ones) arrive once the body ends; a handler that has not begun within Head (the family client's ResponseHeaderTimeout) is ended; a panic fails the call or breaks the body.
  • The host that registers the engines (cloud) holds the keys (KMS) and the catalog.

Acceptance bar before a plugin serves production (owner review): (1) it answers as the native Rust service on the whole conformance set (the zen suite's 25 scripted cases byte for byte, and streaming); (2) the amd64 and arm64 builds agree (dgx arm64, evo amd64); (3) CPU, RSS, allocation rate and per-request latency measured in the cloud pod's limits, plugin against today's relay; (4) Memory64 only for a module past 4 GiB, and then growth past 4 GiB and the bounds around the 32-bit wrap tested; (5) nothing untrusted runs through wasm2go: guest code stays in the hanzoai/wasm sandbox, which refuses Memory64. wasm2go is ahead-of-time translation: it removes the VM's overhead, not the module's own execution cost.

A model asked of its maker first (controllers/native.go, native_chat.go, anthropic_chat.go)

Owner, 2026-10-09/11: "in waterfall we should try all native anthropic api keys, then fall to OR, then free models". A resale family names makers: the APIs of the vendors that MAKE some of its SKUs, each asked for the SKUs its prefix names (maker). openrouterFam.makers are Anthropic's (ANTHROPIC_URL, anthropic/) and OpenAI's (OPENAI_URL, openai/, keys OPENAI_API_KEY, _2, _3, OPENAI_API_KEYS); an unset address asks that maker nothing. OpenRouter out of credit took every OpenAI SKU and Enso's OpenAI rungs down on 2026-10-16; a maker's key is what keeps one empty account from doing that again. The SKU, its catalog price and every gate are the family's, unchanged. dispatch asks the maker first and the family's own address only once no key answered:

  • OpenAI speaks the request's own dialect (native.chat): a chat is sent as written under OpenAI's id (dots kept: gpt-5.4-mini), its ceiling as max_completion_tokens, streamed with stream_options.include_usage; the stream comes back as the family's, its cost read from the usage chunk (chatTap). 429 insufficient_quota and 402 rest the key as an account out of credit; a messages body is the reseller's.

  • An account that cannot pay is said on the host's bus (emptied, object.PublishDrained): a maker's key, an OpenRouter key (sendKeyed) or a route row, by its NAME, once per ten minutes while it stays so. cloud puts it on the SuperAdmin's feed as a supply pulse, which admin.hanzo.ai shows as a critical incident.

  • Paid only, and only the route named. A maker bills every call, so it is asked only by a request seated on the paid lane (!FreeOnlyFor(ctx)), for the route the caller named (f == fam && s == model), where the family lists and charges for it. A free route, the free pool, enso-free/zen-free/free, a route standing in for another and every unseated request never reach it (TestTheFreeLaneNeverReachesAMaker).

  • Every key, in order. The pool is object.Accounts(AnthropicKeys, AnthropicKeyList): ANTHROPIC_API_KEY, _2, _3, then each line of ANTHROPIC_API_KEYS. A key's place is its name's (2 for _2) whatever the names before it hold. Always from the first key that can serve, never round the ring: a prompt cache is the account's, so one account leads. What a refusal does to its key:

    • 401: benched nativeDead (10 min), back at once when its list reads again.
    • 403, and an account that cannot pay (402, billing_error, or the vendor's 400 whose sentence STARTS "Your credit balance is too low" or "You have reached your specified API usage limits"): benched nativeDead, by time alone. A 400 that merely contains those words is a validation error quoting the caller, and benches nothing — read loosely, one crafted request benched the whole pool.
    • 429: benched until Retry-After (5 s when unstated), never past nativeBusy (1 min).
    • 404, 408, 5xx, 529: the next key, nothing benched.
    • Anything else (400, 413, 422), an answer that ended having said nothing, and a 200 that is not the stream asked for: the request's own. No other key is asked and the reseller is, so nothing that worked through the reseller stops working. A stream's head is waited for nativeHead (30 s); a vendor that cannot be reached is not asked on another key.
  • What a key lists is what it is asked for. GET /v1/models per key at start and every nativeEvery (5 min): anthropic/claude-haiku-4.5 is claude-haiku-4-5, or the newest claude-haiku-4-5-<date>; a SKU with a :variant is the reseller's. A key whose list was never read is not asked. A key's standing is logged when it changes (anthropic key ANTHROPIC_API_KEY is not serving (401: …)) and counted once in cloud_supply_refused{provider="anthropic"}.

  • As itself. Nothing here picks another model. When no key answers, the reseller is asked for the same model; when it refuses too, fallback returns nothing for a third-party SKU and candidates has nobody left, so the caller gets 503 providers_exhausted (TestAClaudeModelNobodyServesIsRefusedNeverStoodInFor).

  • Dialects. /v1/messages bodies go as they came (model respelled, the caller's anthropic-version, the betas the shared account serves, shared() deciding whether the body is the shared account's at all). A chat, and so /v1/responses, is translated by chatAsMessages and read back by claudeChat — always as a stream, which the pipe assembles for a whole answer. A sampling parameter and a way of thinking are sent only to a model whose listed capabilities take them; what the Messages API has no word for (audio, a stored file, JSON mode with no schema, an answer begun for the model) is errUntranslated and goes to the reseller untouched. A request with no cache_control is asked to cache its prompt, as on the reseller's path.

  • Money. The answer bills the catalog price of its tokens (zenModel.bill). The maker states what it cost at the vendor's own price (the SKU's rates less the family's markup, an hour-long cache write at the vendor's hour rate) as X-Hanzo-Cogs, header or trailer, exactly as a family service does; bill never goes below that times the markup. That a maker gave the answer is the pipe's own knowledge (serving.maker, set from the response dispatch was handed): a cost or a vendor in the headers of any other answer floors no bill and relabels no row.

  • A stream its caller hangs up on is billed for what the vendor had written: the vendor invoices it either way. claudeChat translates every line already received, tap counts every byte it passes, each states its cost when it is closed, and settle closes the body before it reads the trailer. sniffZenUsage no longer takes a Messages stream's opening count (message_start, an output of one token) as the answer's own: that billed a cut stream for one token, on every family (TestAStreamItsCallerHangsUpOnIsBilledForWhatWasWritten). An answer the VENDOR breaks off ends finish_reason: "length" and then an error frame, so the whole answer assembled from it is never taken for one that finished.

  • On the books. X-Hanzo-Arm (the vendor's id), X-Hanzo-Provider: anthropic, X-Hanzo-Key (the key's NAME) and X-Hanzo-Failover (<id> (anthropic): key 1: 401 authentication_error, one a key that refused) ride the answer in process. The usage row's origin is api.anthropic.com#<place> (admin lens only), its span carries gen_ai.hanzo.via = anthropic#<place>, gen_ai.provider.name = anthropic and the vendor's id as gen_ai.response.model; provider stays the family (openrouter). Margin by route is GROUP BY origin over billed_nano/cost_nano. Never a key.

Enso and Zen rungs are walked by their family service, not here (hanzo-inc/zen): its catalog asks a rung's upstreams in the order written, and its Go engine takes a provider of kind: anthropic with key_env/key_envs, so the same order there is that rung written before the OpenRouter rung for the same model. No rung in either production catalog names a Claude model today (universe charts/app/files/{zen,enso}/catalog.yaml), so there is nothing to reorder.

Strict mode — served unchanged or refused (controllers/strict.go)

X-Hanzo-Strict: 1 on /v1/chat/completions asks for the request to reach the route's FIRST row exactly as sent, or be refused 409 strict_violation with the broken invariant in X-Hanzo-Strict-Violation: route_auto (auto-routing), translation (a family pipe, an Anthropic/QueryText row, /v1/responses), ceiling_unset (no max_tokens named: the relay would write one), ceiling_lowered, rag, param_dropped, rewritten, added, not_canonical (a repeated name). No fallback, no cooled reordering: a vendor refusal is the answer. Served, it carries X-Hanzo-Request-Sha256 and X-Hanzo-Upstream-Sha256 — SHA-256 of the RFC 8785 form (go-json-experiment jsontext.Canonicalize) of the client body and the upstream body, each without model and stream_options.include_usage; they are equal on every strict answer. /v1/messages answers strict requests in its own file (anthropic_api.go).

A forced tool choice reaches its upstream forced (forced_test.go)

tool_choice: "required" or a named function forces a call, and every path carries it forced or refuses it; none sends it on as a choice that forces nothing.

  • Chat upstreams (a family: Enso, Zen; OpenAI as maker; the relay): sent as written, with parallel_tool_calls and each function's strict (familyBody, chatBody).
  • Anthropic's Messages API (chatAsMessages): required → {"type":"any"}, named → {"type":"tool","name"}, parallel_tool_calls: false → disable_parallel_tool_use, strict on the tool where the listing takes a schema; no thinking beside a forced call.
  • Anthropic in (anthropicToolChoiceToOpenAI, anthropicSerial): any → required, tool → named, disable_parallel_tool_use → parallel_tool_calls: false.
  • Responses in (responsesToolChoiceToChat): the flat {"type":"function","name"} and custom become chat's nested form, a namespaced tool by its flatName (named with or without its namespace), allowed_tools its mode over only the tools it allows. A hosted tool, an MCP server, an undeclared name or an unknown word is a 400.
  • Which model can be forced is the engine's to know. Claude Opus 5.5, Sonnet 5.5 and Fable 5.1 answer 400 to any forced choice, thinking or not, and no listing says so; the Enso catalog marks those rungs unforced and hanzo-inc/engine walks a forcing request past them. Before that, enso-pro and enso-ultra (whose live supply was Sonnet 5.5 on Anthropic's key once OpenAI and OpenRouter ran dry) answered every forced call 400 the model could not serve this request: the engine masks any sentence holding :, and the vendor's is tool_choice: type "tool" and "any" are not supported for this model. The engine now names the field instead. The same tiers refused any temperature below 1 ( `temperature` is deprecated for this model., Claude 4.7 and later); the catalog's Claude rungs now drops: [temperature, top_p].
  • A refused Enso tier falls to enso-free first (standIn), then the pool's routes: the engine judges a free answer and reads a forced call a model wrote out as text ([[{"name":…,"parameters":{…}}]], <tool_call> blocks) as the call when it names a declared tool with schema-valid arguments; the pool's routes are relayed as they answer. With no credit anywhere, that is how enso-pro and enso-ultra still return tool_calls.
  • Not done here: a maker's 400 on anthropic/claude-sonnet-5.5 (resale family) hands the request to OpenRouter, so with OpenRouter out of credit the caller reads 503 providers_exhausted, not the vendor's refusal.

Decisions — /v1/decisions, the one decision path

controllers/decisions.go. It reaches the decision service (KAI_URL) through one core, decide, which the HTTP handler and the ZAP twins (zap_decisions.go) call after resolving the principal. It serves Kai (kai and its versions, below) and Jev by OpenRouter's vendor ids (typesafe/jev-1.13, ~typesafe/jev-latest), selected by model; the vendor ids reach Jev itself. Nothing named Jev is ever answered by Kai (jevNamed), and a bare jev-* has no route, so it is 400. Every refusal is {"error":{"code","message"}}. Kai bills $0.021 and Jev its list $0.042 per million input tokens — the same row in model_pricing.go, conf/models.yaml and hanzoai/pricing's decisionCatalog. zenlm/zen6 and zenlm/zen6-flash are Zen answering through the same service by a chat completion from Hanzo's router; they bill input and output ($2.28 / $7.26 per million, a PROVISIONAL launch price the owner has not confirmed: the published Zen chat row). decisionCostNano prices input and output for every model; Kai's and Jev's output rows are zero, so their bills are input alone.

  • One principal pays. The ledger org is what the reservation, the debit and every handle name: billingOrg on HTTP; on ZAP the same rule over the gateway request's X-Org-Id (carried on ctx, orgAsked) and the JWT's signed orgs. An IAM or vendor key has no membership and pays from its own org.
  • A handle belongs to the org that observed it. observe/handle are sent as <org>/<id>; unscope takes the prefix off anything said back. An answer with no handle in it goes back byte for byte.
  • The service is told the org. Every call send makes carries X-Org-Id: the ledger org, the same one handles are prefixed with. The service routes by it. The request is built in send, so the caller's own X-Org-Id never reaches it.
  • One body bound, shared with the service. decisionBodyBytes (16 MiB) is the decision service's BODY_BYTES; change both together. A body past it is 422 request_too_long, after authentication. ai's socket admits more (26 MiB), so the handler sees it; a refusal a layer RETURNS (the framework reading an over-limit body included) is worded by Dialect via controllers.Refusing. A transport-level refusal (zip's raw fasthttp server, above the socket's own limit — or the cloud edge's GATEWAY_BODY_LIMIT in front of ai) is written by fasthttp before any handler and cannot be worded here.
  • The body is read key for key, and only after the credential (fieldsOf): the size bound is the one thing asked before vouched accepts the credential; then an unknown top-level field (a case variant of a known one included) or a repeated one is 400 the moment it is seen, so the read is one pass over at most ten keys and the gateway prices and scopes exactly what the service reads. The forwarded model is always the priced route's upstream id; a model asked in any case is filed as its route id.
  • A handle call holds what its handle bills. handled remembers, per org-scoped handle, the input tokens its observe and each later call billed, and a decision over it holds at least that — its own body is a few bytes.
  • A hold is at least a cent, spend is carried in nano. The ledger holds cents and a decision costs a fraction of one: decide holds max(1¢, estimate) and settles with settleNano; BalanceLedger carries the sub-cent remainder, which counts against what is available as the cent it has begun. A cent covers a whole wire's worth of state at Kai's or Jev's price.
  • An LLM decision bills no more than it was admitted at. A model whose output is priced is held at a token a body byte plus decisionPromptTokens (1,024) and decisionOutputTokens (4,096) of output, and is settled and debited at most that hold whoever pays (BilledNanoExact), a plan included. Counts are clamped where priced: none below 0, output at 4,096, input at 2^30. An answer whose usage is not two whole token counts is 502 and billed nothing (decisionUsage). Known loss: the service writes the state out again before the router counts it (1e15 becomes 1000000000000000.0), so a number-heavy body can cost past its hold; the excess is ours and logged (decide), about $1.76 a call at worst, until the service passes the state's bytes through. A refusal after the router spent is ours too, bounded by the service's max_tokens and 64 in flight a model.
  • The reply never waits on the books. The debit goes to 16 settlers through a 1024-deep queue (full: filed inline, never dropped). A debit the ledger refuses or does not answer in usageTimeout (5 s) is tried 3 times with backoff; a retry after a lost ack is charged once: each usage record mints one Ref the first time it is filed (object.UsageEvent.Ref), and the host keys the ledger on it. Not the record's RequestID — a hedge's winner and its losers share that one. controllers.Settled waits SettleBudget(); cmd/aid stops taking requests first, and cloud's ai plugin runs it as its Shutdown hook after zip drains.
  • Held answers are per org. decisioncache.go is an LRU of Kai answers for an identical body (whitespace-insensitive, order kept) from the same org within a minute, under a fresh id, billed like a miss. Each org holds at most an eighth of it and evicts its own. Never across orgs. Handle requests are never held. The key names the model asked (the id the answer names, so kai and a version asked of its own service as kai never share an answer); its revision is the weights (routing.sha256), plus routing.capability {name, sha256} when the org's capability answered. A lookup uses the capability that last answered that org (kept on its share), so once a capability answers, the base answers the org held stop being served. An answer without the field keeps the weights-only key.
  • Restate is the wording rule: every refusal in the service's shape, Retry-After + Retry-After-Ms on 402/429/529, X-Request-Id on everything; the service's X-Request-Id passes through. The Dialect filter sits OUTSIDE Recovered, so a panic's 500 is worded too, and matches the path in any case, trailing slash or not. The ZAP cloud wire (MsgType 100) has no header slot: its replies carry status, body and error text only, never headers folded into a body. The usage row's request_id column is the id the caller saw (ClientRequestID); the row's own id stays minted here. A 502 names no address.
  • created on /v1/models is a release time where one is recorded: a models.yaml released: (RFC 3339; kai and the Jev ids carry theirs) or an OpenRouter SKU's own created. A model nothing records keeps the listing's time; none is invented. The shape is unchanged (Codex decodes it).
  • pricing on /v1/models names its unit in every key: prompt/completion are USD per token as decimal strings (OpenRouter's keys and unit); input_per_million/output_per_million are the same rates per 1M. A bare input/output is per token to OpenRouter- and Vercel-style readers, so no per-million figure goes there (hanzoai/pricing's own document keeps input per 1M; it is not an OpenAI-compatible catalog).
  • A listed price is the billed price: build prices every row with getModelPriceForOrgOK(id, ""), the lookup billing charges from, so a bare id an OpenRouter alias serves lists the SKU's retail, not its config figure. No price is listed where billing has none.
  • OpenRouter's routers are variable (priced -1, e.g. openrouter/auto, typesafe/jev-router): premium, paid-floored, held at the catalog's dearest rate on each side, and billed at the answer's usage.cost × the markup (the ceiling when no cost is stated). /v1/models lists that ceiling with "variable": true in the pricing block: the most a token bills, never unpriced or $0. A catalog with nothing priced drops them.
  • canonical_slug identifies; id routes: owned_by/id (hanzo/kai), the id when already qualified, absent for an unbranded passthrough and when another route owns that spelling. A published slug is refused (400), never taken by a family prefix (zenlm/zen5 starts with zen) and billed at a default. Every entry path bills and gates on the raw id, so routing a slug needs canonicalizing at the entry first.
  • A UI reads a model's kind from /v1/models, never from its id: every row carries class (ClassOf, the policy's own answer) and, for Hanzo's, family (lineage: enso/zen from FamilyOf, kai, zoo; Jev is TypeSafe's, premium, no family). OpenRouter rows add name (vendor lead cut), description, inputs, supports_tools/supports_reasoning (from supported_parameters); family rows add outputs from their mode and supports_vision; a models.yaml row may state a name and a description. Class and family resolve routes, so build stamps them after it stores the slugs — before, the first build waits on its own lock (TestFirstBuildDoesNotWaitOnItself).
  • A Kai version is kai-<first 12 lowercase hex of the weights' sha256> or a release, kai-<major>.<minor> or kai-<major>.<minor>.<patch> (each part digits, no leading zero; kaiVersion). With no route of its own it is Kai: priced, gated (DecisionAs, which the balance gate reads a decision's model through, so the plan is asked about kai in class ours) and filed as kai, and its answer names the version. Any other kai-… no route names is 400.
  • Each Kai version may run as its own deployment: KAI_URLS. Comma-separated id=url pairs (kai-1.4=http://kai-1-4.hanzo.svc:8080,kai-1.5=http://kai-1-5.hanzo.svc:8080), read once at boot (InitKaiURLs, first in Bootstrap): an id that is no Kai version (kai itself included), an id named twice, or a url that is not http(s) with a host (no credential, query or fragment) refuses the boot, naming the pair. A decision whose id is a pair's goes to that url (kaiFor: Kai's own provider, only the address moved, so the same client, timeouts, KAI_API_KEY, X-Org-Id, X-Org-Capabilities, capture and billing), asked there for kai — the one Kai that deployment runs, which is how the stock image serves its folder — and the answer is renamed to the id asked. Every other id, kai included, and a version no pair names (sent as asked) go to KAI_URL. /v1/models lists each pair's id as kai's row under that id (owned_by hanzo, kai's price, class and family, slug hanzo/<id>); no models.yaml row is needed. Blank or unset, nothing changes. Start a version: add its deployment and its pair; kill it: drop both.
  • Nothing named Jev reaches Kai: a route or a models.yaml alias_of that would send a Jev-named id to a Kai upstream is refused where it resolves (Canonical, decisionModel).
  • A yes/no question is boolean; noul is Jev's spelling of it. One kind, two spellings, and the answer mirrors the one asked: {"type":"boolean","probability":p} or {"type":"noul","noul":p}; a boolean carries no confidence/answer_confidence (choice and score keep theirs), and Kai v0.3.x still adds both to a noul and refuses boolean (422) until v0.4.0 serves. Kai models get the body as written. Jev knows only noul: for a Jev model consult respells each boolean question noul (spell) and puts its answer back as exactly {"type":"boolean","probability":p} (unspell), in place over members, every other byte kept. A Jev model holds no state (the service refuses observe/handle for it), so no handle crosses this.
  • The spec is derived. routers/shape.go reads validate:"required,min=,max=" and enum:"..." tags, a type's Schema() (Content kinds) and Variants() (a question is a boolean, a noul, a choice or a score, by type); controllers.Answer.Refusals states each refusal and its headers.

AI Login Manager — universal metering + connected accounts + 1% BYO fee

ai is the centralized login manager for every user's AI providers. Every AI call from every surface (@hanzo/dev, Hanzo Desktop, hanzo.chat, hanzo.app) routes through THIS router at api.hanzo.ai/v1. For the caller's org the router resolves WHICH account executes the call, runs it, and meters ONE usage event — regardless of whose credits paid for it. There is exactly one meter and one credential store.

Two account modes (one router, one meter):

  • Hanzo credits (default). No org-owned provider for the requested model → the global built-in (Owner == "admin") serves it. A route row (models.yaml) bills its token list price; a resold OpenRouter SKU or a maker's answer bills its cost and the platform's fee, and an Enso SKU its fixed price (A pass-through call carries the fee, below).
  • Connected third-party account (BYO). The org connected its OWN OpenAI / Anthropic / Google account via /v1/ai/connections. The router uses the org's KMS-sealed key, the customer pays that provider directly, and Hanzo bills the hanzo_byok platform fee (1%, the host's setting, fee.go) on the provider list price for routing + metering + management.

BYO is the EXISTING BYOK seam, not new plumbing. resolveProviderForUser → object.GetModelProviderByNameForOrg(org, name) returns the org's own provider row when it has one (Owner == org), else the global admin built-in. providerBYO (controllers/platform_fee.go) classifies the resolved pair: BYO iff provider.Owner != "" && != "admin" && == user.Owner.

Metering — one path, extended not rebuilt. Every completion terminal already calls recordUsage (→ commerce POST /v1/billing/usage) + recordTrace (→ the hanzo.cloud_usage warehouse). This feature adds three dimensions to that ONE event: byo, fee_cents, account. platformFeeCents(costCents, byo) = byo ? ceil(costCents × hanzo_byok%) : 0. Billed amount = byo ? fee_cents : cost_cents; the full cost_cents is recorded on every row for analytics even when only the fee is billed. hanzo.cloud_usage is the ONE global usage store (its organization column is the row-level tenant filter — a tenant reads only its own org; the admin org reads unfiltered). The rich per-org analytics lens is entitlement-gated in hanzoai/cloud (analytics.datastore); basic own-org usage is always readable.

Connected accounts API (/v1/ai/connections, IAM-authed, org-scoped). Curated login-manager surface over the per-org object.Provider store (controllers/connections_api.go). Keys are sealed via object.StoreProviderSecret → kms://… ref (fails closed if KMS is down); the raw key is NEVER stored in a row, returned, or logged.

  • GET /v1/ai/connections → [{provider, connected, account_label, updated_at}] (never the key)
  • POST /v1/ai/connections {provider, apiKey} → seals key, activates the org row; returns metadata only
  • DELETE /v1/ai/connections/:provider → deactivates the org row (resolution reverts to Hanzo credits)

The org is the one the caller acts in (GetOrg, below). Reads are every member's; connect, reconnect, disconnect and the OAuth start are the org's admins' (requireConnectionAdmin → administers: authz.Claims.OrgAdmin over the signed set, or the SuperAdmin), on HTTP and on the ZAP door (zapConnectionAdmin) alike. An sk- key administers nothing. Pinned by controllers/acting_test.go.

Enforce "even if logged in". All AI traffic must reach this router; no surface bills a third party off-meter. hanzo.app's workspace default was flipped openrouter → hanzo; hanzo.chat routes via its gateway config; @hanzo/dev defaults to api.hanzo.ai/v1 under Hanzo login. Claude/Opus for @hanzo/dev is served HERE (Anthropic-native /v1/messages + OpenAI-compat /v1/chat/completions) against the org's connected Anthropic account + 1% fee — no direct-to-provider bypass.

Next slice (route selection). Connections are stored/sealed/listed and the 1%-fee metering fires the instant the resolver returns an org-owned provider. But the route table maps all models to providerName: "do-ai" (the aggregating upstream), so a connection row named openai/anthropic/google is not yet SELECTED for its model family. Making a connected consumer account actually serve its models is a bounded route-table change (model-family → org-connection selection

  • real upstream model-id/base-url mapping) — deliberately not in this slice.

RAG — ONE unified surface (retires the standalone chat-rag-api)

ai is the single RAG layer for the whole platform. There is exactly ONE retrieval pipeline; the standalone hanzoai/chat-rag-api (danny-avila/rag_api fork) is now redundant and slated for archive.

One index, one embedder, one retrieval path. Every document — doc/crawl/ github/s3 ingest AND per-file uploads from hanzo.chat — lands in the SAME per-tenant index {owner}-{store}-docs, fanned out to BOTH Hanzo Search (keyword, Meilisearch) and Hanzo Vector (semantic, Qdrant) via object.IndexDocuments, and retrieved via object.SearchDocuments (hybrid RRF). See object/search_docs.go.

File-scoped RAG is a FILTER, not a parallel store. file_id is a filterable dimension on that one index (DocIndex.FileID; Meili filterable attr file_id; Qdrant payload file_id). Uploaded-file retrieval = a file_id filter over the tenant index. object/rag.go holds the file-scoped logic (RagEmbedFile, RagQuery, DeleteRagFile, RagFileContext); chunking uses split.RecursiveSplitProvider (parity with rag-api's RecursiveCharacterTextSplitter, 1500/100), parsing uses txt.GetParsedTextFromUrl (PDF/CSV/XLSX/PPTX/…). Uploaded files default to the rag-files store so they don't pollute curated docs.

One address family (controllers/rag.go): POST /v1/ai/rag/ingest, /v1/ai/rag/embed, /v1/ai/rag/query, /v1/ai/rag/query-multiple, /v1/ai/rag/delete, GET /v1/ai/rag/context. Ingest and embed are two ways in to the one index; the rest read it back.

The one exception is POST /v1/embed (controllers/rag_librechat.go) — the MULTIPART form hanzo.chat's uploadVectors() posts. Its request line is fixed by that client, not by us, which is why it is not a spelling of /v1/ai/rag/embed. Everything hanzo.chat READS it reads from the native family.

Migration: hanzo.chat's RAG client calls ${origin}/v1/ai/rag/* directly. Auth/billing reuse the doc-RAG path (resolveSearchAuth/requireIndexAuth + recordSearchUsage); owner is the authenticated principal (tenant isolation), never a client-supplied field.

Search / Crawl / Web-search — THREE orthogonal products, ONE way each

Three distinct products, no overlap. Do NOT add a fourth crawl path.

  • POST /v1/search = native full-text index search (SearchDocs → object.SearchDocuments). Hanzo Search (Meilisearch, keyword, hanzoai/search-go at searchHost:7700) + Hanzo Vector (Qdrant semantic, vectorHost:6333) fused hybrid-RRF over the per-tenant index {owner}-{store}-docs. Returns {hits:[…]}. Needs a populated index — an empty/new tenant index returns {hits:[]} (no error). GET /v1/search/stats reports index size.

  • POST /v1/crawl = crawl a URL, get content back — THE single canonical crawl (controllers/crawl.go Crawl → object.Crawl → CrawlWithCrawl4AI). Backed by the self-hosted Crawl4AI (hanzoai/crawl, crawl.hanzo.svc:11235, Apache-2.0). Body {url} or {urls} (merged, deduped, capped at 10); returns {results:[{url,title,description,markdown,success,metadata}]} with LLM-ready markdown (prefers Crawl4AI fit_markdown, falls back to raw_markdown). Auth: requireIndexAuth (fail-closed, rejects read-only pk-/hz_ keys) + balance gate; usage recorded via recordSearchUsage(…, "crawl", "crawl4ai", …). object/crawl4ai.go handles the deployed Crawl4AI 0.8.6 wire shapes: markdown is an OBJECT (MarkdownField polymorphic decode), the /crawl envelope is synchronous with a boolean success and no status/task_id (return inline Results whenever present), and links/media values are heterogeneous (map[string][]map[string]interface{}).

  • POST /v1/scrape = crawl-and-INDEX (ingest) — NOT a general crawl. It crawls a site and WRITES structured content into the tenant search index, returning ScrapeStats (ScrapeDocs → object.ScrapeAndIndex). Sibling of POST /v1/index (index caller-supplied docs). Stays because its output/effect (index write) is distinct from /v1/crawl (fetch + return).

  • POST /v1/scrape/preview is a DEPRECATED alias of /v1/crawl — its handler now forwards to Crawl (one implementation, no parallel path). Use /v1/crawl.

  • /v1/websearch/* (lives in hanzoai/cloud, not ai) = web meta-search — a separate product: SearXNG-compat /v1/websearch/search + a firecrawl-compat /v1/websearch/v1/scrape leg that hanzo.chat's web-search plugin calls. That leg is a fixed EXTERNAL contract (firecrawl {success,data:{markdown}}) and reuses the SAME Crawl4AI backend — it is an integration shim for the web-search flow, not a general crawl endpoint. The general "crawl a URL" is ONLY /v1/crawl.

Security invariants (READ before touching auth/billing/scaling)

  • JWT iss/aud — every request-auth JWT path goes through object.ParseAndValidateJWT (signature via IAM and iss/aud policy), never raw iam.ParseJwtToken. object/jwt_validate.go sources the issuer from JWT_ISSUER/IAM_ISSUER/CLOUD_IAM_ISSUER/AUTH_ISSUER (any, default https://hanzo.id) and the audience allowlist from GATEWAY_ALLOWED_AUDIENCES (+ folds in IAM_AUDIENCE/AUTH_AUDIENCE), mirroring the gateway. A foreign-aud token → 401. The OAuth code-exchange callback (account.go Signin) is NOT a request-auth path and is intentionally exempt.

  • A delegated token is the HOST's to verify — a token whose scope names ai:inference (iam.Confined) is a person's credential IAM delegated for model calls (iam LLM.md "Delegation"), signed by a key the public JWKS never carries. ParseAndValidateJWT sends it to the verifier the host installed (object.SetConfinedVerifier — hanzo-inc/cloud passes its identity boundary's verdict on the exact token bytes) and reads its claims from the verified bytes (iam.ClaimsOf); with no verifier installed it is refused (ErrJWTConfined). Never a header: the module asks about the token it is about to believe.

  • No ZAP door serves a delegated token — the host's boundary, which keeps one to model calls, stands in front of HTTP alone. The gateway (MsgType 200), the native node (100), the ops node (110) and the forward bridge refuse one 401 before any verifier is asked, wherever in the request it rides, and so do the identity seams they share (zapResolveAuth, zapResolveUser, zapPrincipal). controllers/zap_confined.go; TestNoZapDoorServesAConfinedToken.

  • A delegated token names no app — its aud is the API (RFC 8707), so controllers.Apps answers none for it, as for an API key: a program that named its model gets that model or the 402 saying what pays, never limited mode's free model in its place. TestADelegatedTokenNamesNoApp.

  • Retrieval is stated once — retrieval.Asked / retrieval.Store (a leaf) is when a model call searches the org's documents into its prompt; the controllers and the host that refuses retrieval to a delegated token read the same function.

  • Balance ledger — single-pod invariant — object.GlobalBalanceLedger is in-pod memory. Reserve/Settle are correct ONLY at replicas: 1. Two pods double-spend (each reserves against its own cached Commerce balance). Enforced by deploy strategy: Recreate + HPA min=max=1 AND a boot assertion (bootstrap.go panics if CLOUD_REPLICAS > 1, the count the cloud chart injects). The paid lane's plan holds (controllers/lane.go) are in-pod too. To scale out you MUST first move Reserve/Settle behind a Commerce-atomic conditional reserve.

  • Address lanes (address.Buckets) — IPv4 whole; IPv6 its /64, then its /48 at 16×, the wider charged only for what the narrower admitted. The router asks only the /64 before buying an IAM round trip for an unresolved key, and charges the /48 only to a caller who stays anonymous, so a neighbour cannot close a paying key's /48. A resolved key stays named past its 5-minute TTL (up to 2×) while one background re-check replaces it. The free lane's DAY holds a /48 to siteDay (256) visitors, not 16: sixteen strangers a day is a quiet carrier /48.

  • Single-request reservation (controllers/billing_reserve.go) — an uncapped max_tokens is clamped (clampMaxTokens) and the reservation covers the larger of the clamped ceiling and the QueryText pipeline's fixed completion cap (reserveCompletionTokens / reserveCompletionFloor=4096, mirrored in model/openai_util.go), so actual spend can never exceed the hold on any path.

  • Native /v1/messages is the caller's bytes (nativeRequest, HTTP and ZAP, text-only included — QueryText never serves a native upstream) — only model and max_tokens (the reserved ceiling) are spliced; anthropic-version/-beta forwarded; only content-type and our X-Request-Id come back. splice refuses a top-level key repeated or case-folded (encoding/json folds case, keeps the last). The shared account serves an ALLOWLIST (sharedFields/sharedTools/ sharedBetas, no held file_id); a BYO account is sent anything. The hold is body bytes/3.5 + ceiling; an answer without the vendor's final count bills no less (lost); upstreams get an idle deadline, not a total one; no SSE line cap. Billing: SKU price, every usage.iterations run, cache writes 1.25x (5m) / 2x (1h, folded into write units until usageRecord carries a 1h count). A stream takes its hold with carry (anthropic + family), or the handler's settle(0) wins.

  • A tier is read through object.TierReader, and the host owes a reader that answers — there are two consumers and they must ask in the same order: controllers.commerceFamilyTier (the per-SKU gate) and routers.TierCache.commerceTierLookup (the rate limiter). Both read the installed reader FIRST; the HTTP route is what a STANDALONE ai has instead of one, not a second way to ask. InitTierCache builds the cache when EITHER exists, not when the endpoint alone is set.

    The reader is NOT necessarily in-process. Every cloud app is its own child PROCESS, so a Go func var cannot cross that boundary: what cloud installs is metering.Client.Tier, which is an HTTP call to GET /v1/billing/tier carrying COMMERCE_SERVICE_TOKEN and X-Org-Id. That call has no user session, so it only succeeds where the handler resolves its tenant with billing's readerOrg (session OR trusted service naming an org) rather than principal.OrgFrom alone. balance had that rule; the typed ops behind tier did not, and answered 401 sign in to view finance to a token the same request proved good on /v1/billing/balance. Fixed in cloud apps/billing/typed.go (payer falls through to readerOrg), guarded by apps/billing/tier_s2s_test.go.

    The failure is silent in both directions, which is why it lasted: the rate limiter maps a failed read to TierZenFree (every paying org shaped as free) and the SKU gate maps one to unknown and ALLOWS (it stops gating). Neither logs an error anyone watches, so assert the tier a caller actually gets rather than that the lookup "worked".

  • A failed call bills only what a vendor actually ran — recordUsage bills a failure because by then the vendor has run the request and invoiced us. That premise does not hold for a refusal it can decide from the request alone (400, 401, 402, 403, 404, 413, 422, 429) or for an error carrying no status at all (nothing was dialled). controllers.ran(err) answers that, and it is NOT faultOf: faultOf asks where the request goes NEXT, and the two come apart both ways — a 401 fails over having run nothing, a partial write stops the cascade having run plenty. spent() consults ran only when nothing came back, so a meter the vendor reported and any text that reached the caller still settle it first. Without this the quantity billed is the prompt measured on the way out, so the charge grows with the request: 400k tokens refused for being too long billed a dollar.

  • Exemption is by ROUTE, never by identity. BALANCE_EXEMPT_USERS is read by nothing; TestEnforceBalanceGate_NoExemptBypass and TestCheckBalanceNoExemption set it and require the formerly-listed subjects to fail closed. What is exempt is a set of paths — isBalanceExempt in routers/filter_balance.go — plus balanceExemptNames, three usage reads named as POLICY rather than URL so a $0 org can still see the panel telling it to add credit, and so moving the route cannot silently re-gate them. Exempt from balance is not exempt from auth.

  • SuperAdmin is the host boundary's verdict, never a reading of an org name. The host installs it (object.SetSuper; hanzo-inc/cloud installs principal.Super, which reads the X-User-IsAdmin its boundary deletes from every request and re-mints from authz.Claims.Sudo — a PERSON whose own org is admin). Every HTTP seam that turns a request's token into a principal goes through object.Verify(c, token), which stamps the verdict on iam.User.Sudo (json:"-", never decoded); util.IsSuperAdmin(user) reads that field and nothing else, and util.ScopeOwner(user, requested) asks it. No code here compares an org to "admin" to grant authority; the reserved org's NAME (authz.AdminOrg) is used only to say which ROWS are the platform's (provider rows, the platform's index). What this replaced decided on its own and disagreed with the boundary: an admin-org application (no orgs, owner claim admin), an admin-org person's sk- key, and an org spelled admin (the old parse trimmed orgs[0]) were each SuperAdmin here and nowhere else. isAdmin is not part of it — an org admin (hanzo/z) administers hanzo and holds no platform power.

  • The org a request acts in is the signed set's answer, by cloud's rule. GetOrg → actingOrg (controllers/org_resolver.go): the SuperAdmin selects any org; a credential with no membership set (sk- key, run key, a program's token) and a confined delegation act in the org that holds them; everyone else gets authz.Claims.EffectiveOrg(X-Org-Id) over the verified token — the same predicate cloud's SanitizeIdentity asks, so embedded and standalone ai agree. A membership opens its org unless it is the admin org, a brand's directory (hanzo-id), or an org the platform runs (hanzo, lux, …) held as a plain member; a set that opens nothing acts in no org. It used to answer user.Owner (orgs[0]) whenever nothing was selected and account.EffectiveOrg (membership only) otherwise, so a plain member of a brand org acted in the brand's org and could repoint its shared AI connections. The money resolvers (billingOrg, openai_api.go, filter_balance.go, zap_decisions.go, zap_router-policy-stats.go) still ask account.EffectiveOrg.

    With no host — standalone aid — there is no verdict and no SuperAdmin. A ZAP door is not the boundary either: target (the one place a ZAP-built request reaches the router) deletes X-User-IsAdmin, and zapPrincipal(ctx, auth) carries a verdict only when ctx does, which only the HTTP bridge (RouterConfigBridge, vouch) sets, from object.Super on the request it holds. A key is never the SuperAdmin on any path, as at the host. controllers/superadmin_test.go, routers/authz_filter_test.go (asSuper).

  • A tenant's URL is fetched by ONE rule: github.com/hanzoai/outbound. proxy.Public is outbound.Client (judged at the dial on the address the socket connects to, so DNS rebinding is refused on the answer that would have been used; every redirect a new dial judged the same way; no proxy; the IANA special-purpose registries, so NAT64 64:ff9b::a9fe:a9fe, 6to4, Teredo, fd00:ec2::254, CGNAT and 240/4 are refused). It fetches every address a request names: a message's href and images, /v1/ai/rag/embed's url, /v1/ai/rag/ingest's files[].url, and every page the Go scraper crawls; the browser engine (crawl4ai) is handed only what outbound.Host finds public. A tenant document goes through txt.GetParsedTextFromAddress, which reads http/https only — txt.GetParsedTextFromUrl reads a scheme-less string as a LOCAL PATH and is for addresses this system minted (stored files) alone. A test swaps proxy.Public; outbound has no switch. object/address_test.go, object/address_redirect_test.go, object/scrape_address_test.go, proxy/public_test.go.

  • Authz gate (routers/authz_filter.go) — platform-sensitive endpoints (superAdminEndpoints) are gated FIRST (before preview-mode + exempt reads). There is deliberately NO adminDomain Host bypass (a removed full-authz-bypass primitive). No principal → 401; wrong principal → 403.

  • User redaction (object/redact.go) — RedactUserSecrets is an allowlist projection over string/[]string fields (exposableUserFields): a field not on the list is zeroed, so a NEW upstream secret field is fail-secure. Credential struct slices (MfaAccounts/ManagedAccounts/MfaItems/FaceIds) are nilled.

Cost: known, zero, or not known — three answers, never two

An unregistered COGS used to read back as the CUSTOMER PRICE. costInputPerMillion() returned the price when no cost rate was set, so tokenProviderCostNano computed cost == billed and every such call recorded margin_nano == 0. For the Enso family that was every call: 1,296 rows checked, all of them zero. It is not a rounding error — it is a ledger reporting a business that earns nothing on everything it sells.

The shape of the fix is that cost is optional and margin follows it:

  • controllers/model_pricing.go — modelPrice.costed() reports whether the model states what it costs us, and it requires BOTH legs. Half a COGS is not a COGS: an input rate with no output rate prices the completion — the expensive half — at zero. costInputPerMillion() / costOutputPerMillion() now return the configured rate and nothing else; there is no path that reads one without first asking costed().
  • controllers/billing_nano.go — tokenProviderCostNano and providerCostNano return *int64, nil where no cost is known. costMargin.CostNano is a pointer for the same reason MarginNano already was: the row has to be able to say it does not know, and an int64 can only say zero. usageMargin reports a margin only when both sides are real.
  • controllers/zap_native.go + object/cloud_usage.go — the warehouse column is Int64, so an unknown cost writes 0 and sets uncosted = 1 beside it. Query uncosted = 0 for rows whose cost is a fact.
  • cache_saved_nano (cacheSavedNano, controllers/billing_nano.go) — what the prompt cache took off the call's bill: cache-read tokens at the model's input price less the same at its cache-read price (cacheReadRate, shared with tokenNanoAt), from the billed table. 0 for a free, BYO, stated-amount, decision, image, video or audio call, and for a row written before the column: 0 there is "not measured", so a sum is a floor. cloud's /v1/usage/summary reads it as llm.cacheSavedCents.
  • controllers/telemetry.go — the span emits provider_cost only when there is one, the way it already treated margin. It no longer asserts a COGS it does not have.

Zero and unknown are different answers. Zero is a fact: BYO (the customer paid the upstream with their own key), a free route, speech on hardware we already own. Nil is the absence of one: image and video have a per-unit PRICE table and no vendor invoice behind it, and a token model that states no COGS has told us nothing.

Speech sells at the published rates, in nano. sttNanoPerSecond (100,000 = $0.006/min) and ttsNanoPerChar (15,000 = $15 per 1M characters) are hanzo.ai/pricing's Speech-to-Text and Text-to-Speech rows, keyed by every id that reaches the speech service. Nano because a dictated sentence is seconds long and a per-call cent would bill it at twelve times the rate. The speech routes carry no token price, so the balance gate asks for a funded caller.

Three speech doors, one service. /v1/audio/transcriptions and /v1/audio/speech are controllers; /v1/audio/transcript (the growing transcript) is the ZAP gateway handler in zap_transcript.go, bound to HTTP by AudioTranscript through the in-process bridge; /v1/voice is hanzoai/voice behind zip.AdaptNetHTTP, which takes the upgrade's connection (zip v1.37.26+). The socket's ticket is its credential, so voice is anonymous at the bearer filter; its gate trusts object.TrustedJWTIssuers (IAM_URL is where IAM is reached, not what it signs as) and admits the browser origins in VOICE_ORIGINS. The speech provider row speech is operated: the paid lane being off never refuses it.

The visitor's mic is a public lane (controllers/public_scribe.go, POST /v1/audio/transcriptions/public): no credential, the model assigned (zen-scribe), the audio held to 60 s by the speech service's max_seconds (it stops decoding just past it, stt.WithLongest), and a day per visitor in its own dayCount keyed by Lanes (the /64 and the /48 at siteDay). PUBLIC_SCRIBE_DAILY is the switch and the ceiling; 0 closes it. It never reaches a vendor: a zen-scribe route to a provider that is not operated closes the lane. Nothing is kept or logged.

Where a cost is registered

object.ModelRoute.CostInPerMillion / CostOutPerMillion — the route row, which the admin cockpit already reads and writes. It is the only place, and the configured PRICE tables are deliberately not consulted as a substitute.

The family leg used to make that unreachable for zen/enso: familyModelPrice returned first and short-circuited the resolver, and the family wire (zenWireModel) carries no cost field, so an enso model could not be given a COGS at all without restating its price. getModelPriceForOrgOK now completes an uncosted family price from registeredCost. Price and cost resolve independently, because they are independent facts: the family owns what we charge, the route row owns what it costs us.

What still has no COGS

The mechanism no longer lies, but a rate nobody has entered is still a rate nobody has entered. A route row (models.yaml, the static table) states no cost until an operator sets both legs; one leg alone is ignored by costed(). Until then the rows read uncosted = 1, which is the honest answer and a visible one: SELECT count() FROM hanzo.cloud_usage WHERE uncosted = 1 is the backlog. A family's answer (Zen, Enso, the OpenRouter resale) states its own cost on every call (X-Hanzo-Cogs, or the vendor's usage cost), which wins over the rate table in usageMargin: rate math cannot express a fan-out or a failover's cost at all, and only the family can total it.

The admin cockpit's below-cost check compared price against a "cost" that WAS the price, so its difference was always exactly zero and it could neither fire honestly nor stay quiet honestly. It now has a cost to compare against, or a flag saying there is none.

A pass-through call carries the fee; Enso sells at a fixed price (fee.go, rates.go)

Owner, 2026-10-15: a model served through the gateway bills the price of the arm that answered, passed through, and the platform's fee on top: 5.5% (OpenRouter's own rate) on Hanzo's accounts, 1% on the org's own provider key. The fee is the margin there; nothing is ever billed below cost. Enso is the exception, decided after: NO adaptive or dynamic Enso pricing. Each Enso SKU has one fixed list price, and nothing measured moves it.

  • Fees are the host's (object.Fee, cloud apps/flags SeedFees, edited at admin.hanzo.ai Usage policy): openrouter (5.5, the vendor's fee on its credit, part of the cost), hanzo (5.5, the platform's) and hanzo_byok (1). fee.go reads each at most once a minute (levy), a stale read answering while the next runs; a read that is no percent keeps the last good one, and a platform fee of zero is refused there and at the admin write. A host that states none bills at cost: never below it.

  • Cost of record (costed): what the family (X-Hanzo-Cogs) or the vendor's usage stated, times the fee of the costliest vendor whose credit bought any arm of it (buyers: OpenRouter for the resale family's own answers; the answering arm's provider and every failed arm's for a family's; none for a maker's, whose price is its own). It is cost_nano, on every call, Enso's included.

  • Pass-through (not Enso): a resold OpenRouter SKU bills its vendor's price times (1 + openrouter fee) × (1 + hanzo fee) (openrouterMarkup), never below its stated cost times that. A maker's answer (native.go) bills plus(cost), its cost of record times the platform's fee. A BYO call (providerBYO) bills the hanzo_byok fee alone.

  • Enso's price list (ensoPrices, the one catalog): each paid SKU is the list price of the upstream its route typically answers on (the cost the served catalog, universe charts/app/files/enso/catalog.yaml, states for its first rung) times 2 to 3, and well under Claude Fable 5.1 and GPT-6 Astra ($10 / $50 at their vendors). Per million tokens, input / output / cache read / cache write:

    SKU typical upstream, list × price
    enso-flash GPT-6 Luna, 0.11 / 0.55 / 0.011 / 0.1375 3 0.33 / 1.65 / 0.033 / 0.4125
    enso-pro GPT-6.1 Sol, 2.20 / 11.00 / 0.11 / 2.75 2 4.40 / 22.00 / 0.22 / 5.50
    enso-ultra Claude Sonnet 5.5, 2.20 / 11.00 / 0.11 / 2.75 2.5 5.50 / 27.50 / 0.275 / 6.875

    Flash is 3x because Luna past a 272k prompt costs 0.20 / 0.75; ultra 2.5x because its Opus 5.5 arm (4.40 / 22.00) with OpenRouter's fee would sit above twice Sonnet. enso-free, the one SKU the catalog serves free, has no line and bills nothing; enso, enso-auto and hanzo/enso are no SKUs but the router over these three (Enso's router, below). modelFamily.fix lays the list over the family's discovery, so /v1/models lists it, zenModel.bill bills it, the hold reserves it, and the price list reads it (Prices, which cloud lays over /v1/pricing): one number in every place. A SKU the service prices that the list does not is dropped from the listing and never served, said in the log at each discovery. A price change is an edit of ensoPrices and a release; the engine's own quote (its catalog margins) is not read.

  • The guard (modelFamily.guard): a SKU is sent only while every arm of its route costs us no more than the SKU bills, in every token class, at a long prompt's price where an arm charges one, and with the fee of a reseller whose credit buys it. The route is the engine's admin catalog (GET {ENSO_URL}/v1/admin/catalog under ZEN_ADMIN_TOKEN, modelFamily.route): the first read is waited for, then read again beside the requests once a minute (routeEvery); a failed read keeps the last; a first read that failed answers its failure for 15 s. What cannot be checked is not sent: no price, no route read, or no arm for the SKU is a refusal. A refused SKU answers 503 naming no upstream (its route costs more than it bills / its route cannot be checked), is asked of no other provider, and is skipped as a stand-in (pool); each misfit is a guard: error in the log at every read. After the fact, an answer on an arm the route as read does not hold, or that costs more than the SKU bills, is a guard: error and reads the route again on the next request (trail). The free SKUs pass the same guard at a price of zero, so a free SKU is never sent to a paid arm, and a pod that has never read the route serves no Enso at all. TestTheServedRoutesFitUnderTheirPrices holds the list to the served catalog's arms.

  • Never silent: a call whose cost exceeds what it billed (a stand-in capped at the named model's price, a free rung that cost something) writes loss_nano on its row and a loss: warning. The ledger counts every one.

  • The row (hanzo.cloud_usage, one a call): beside model (the SKU), cost_nano, billed_nano and margin_nano, a family's rows carry served and vendor (the arm and the provider whose account ran it), reasoning (the depth asked, depthOf, mirroring hanzo-inc/engine thinking.rs: none, low, medium, high, max, or default), reasoning_tokens, loss_nano, and on Enso's arms: JSON, the arm that answered with its rate at the time (rate, the vendor's fee as fee) and the tokens it was paid for, then each arm that failed first and why. Failed arms keep their failover rows too.

  • The ledger is a report (Prices(ctx, admin)): to a SuperAdmin, each SKU's typical upstream, its list and the multiple, its route as last read with each arm's over classes and refused, and the last day's Ledger read from the usage table on demand (object.RoutedRows, one row per SKU, arm, vendor and status, deduplicated by id first): answers, failovers, OpenRouter-bought answers (fallbacks, fallback_share), cost, billed, margin_nano (billed less cost over the answers whose cost is known; uncosted counts the rest), losses, and the same by route. Nothing reads it to price.

  • Resolution is stated (resolve.go): a model is reached on a direct key first (its maker's API for a SKU its keys list, native.go, ahead of the family that resells it; a route row's own provider), then DigitalOcean's inference (do-ai), then OpenRouter, the fallback (openrouterTail, a route's last candidate). Paths(id, org) states that order for one model, each path's kind (direct, family, do, openrouter) and whether a key serves it. Between a row's own providers a call keeps the row's declared order (candidates), so every row of conf/models.yaml lists its direct provider before do-ai, held by TestEveryShippedRouteAsksADirectKeyBeforeDigitalOcean.

  • Gone (v1.833.348): the from / typical / up-to quote, its depth quotes, Price, Blend, Reprice, StartRates and the daily measure, and /v1/models' pricing.from, pricing.up_to, pricing.fee, pricing.window and variable on Enso. Cloud's /v1/pricing/enso quote and POST /v1/pricing/sync read them and go with its next pin of this module.

The routing catalog (routing.go, /v1/ai/router/catalog, SuperAdmin): zen is read and written at the Go zen's /v1/admin/zen/*, enso at the engine's /v1/admin/* (until v1.833.351 enso was asked at the zen address and every read answered 404).

  • An apply the guard would refuse is not sent: guardRefuses holds every priced model the edited catalog serves to misfit against ensoPrices, the guard's own rule, and an apply answers 400 with the arm and classes; a proposal carries the same sentence as refused, so it shows before anyone confirms. Without it an edit that put an arm over its model's price applied cleanly and the model answered 503 within a minute, when the guard re-read the route.
  • Convergence leaves a new deployment standing. Each version records boot, the deployment digest of the catalog its run of versions was made over: the catalog less every field the admin view derives (keyPresent; effectiveMargin, minTier, funding; a rung's retail and profit), so a key that resolves or an engine that derives another figure is no new deployment. A version made while the family served the newest keeps the newest's boot, one made over anything else starts a run. converge (under routingMu, one step with any apply) re-applies the newest version only when the family serves that boot (it restarted onto the file the run was made over); a family that restarted onto a new release of its file is left serving it, warned once, and the read says redeployed: true beside drift. Before, the first stored version would have been pushed over every later release of the family's file, a minute after each restart.
  • An edit over a restart waits. An apply made while the family serves the newest version's boot but not the version itself (restarted, not yet converged) is refused 409: made from what serves, it would drop the newest version's own edits. Convergence sends the version bare (less its derived fields, which a family that dropped or renamed one would refuse), and once it has failed on a deployment (unconverged) an edit is let through, so a version the family will no longer take never locks edits out.
  • The engine's admin view carries no long price, so neither the guard nor guardRefuses sees a rung's long-prompt price; both read the same view, so they agree.
  • RoutingRead, RoutingPropose and RoutingApply are the one implementation, exported for cloud's plane (ai_routing, ai_routing_apply), which the Enso policy registry (cloud apps/train /v1/train/policies) promotes through.

Enso's router: one tier a request, enso-free when nothing pays (depth_route.go, routers/filter_balance.go)

Owner, 2026-10-15: only enso-free is free. enso, enso-auto and hanzo/enso are the paid router, and they are ai's, not the engine's: the engine serves leaf SKUs only, and universe's enso catalog lists none of the three.

  • The router is router.depth (cloud-api-models, universe values/hanzo/cloud.yaml): a row per router id, reasoning depth → tier. Up to low (and a request that states no depth) enso-flash, medium and high enso-pro, xhigh and max enso-ultra. The depth is the request's own (requestDepth: reasoning_effort, reasoning.effort, a token budget, thinking). No learned router is in the path: the engine's POST /route is empty in conf/models.yaml and its policy serves nothing in production.
  • Resolved before anything prices it (BalanceGateFilter): a request naming a router id is rewritten to its tier and then gated, held, seated, billed and guarded exactly as a request naming the tier, at the tier's fixed price (ensoPrices). Explicit tiers and enso-free are unchanged.
  • Nothing pays the tier, enso-free answers at $0, never a refusal for want of payment: the family serves the tier to no one at a price; a program in this process made the call (object.Caller: its meter charges for the id it named); with no usage policy, the wallet cannot cover it; with one, the paid lane's switch is off, the policy refuses the tier or names nothing, names a payer the paid lane does not seat (the free plan, a free cap, granted credit: paid_plan_required), names prepaid cash the wallet cannot cover (insufficient_balance), or the paid lane has no room (paid_lane_full). A paid plan's included usage and a payer's own prepaid cash pay, plan or none (owes). What admitting the tier took is given back; a refusal enso-free meets as well (a request window, the free allowance) is enso-free's.
  • The answer says so: X-Routed-Model names what answered (the tier or enso-free); the chat envelope's routing object says router: enso, chosen, the depth and key it read, policy funded | unfunded | n/a, and a sentence of why. A refusal for want of payment adds limited mode's headers (fallBack): X-Hanzo-Fallback: enso-free, X-Hanzo-Usage-Reason (the code), X-Hanzo-Usage: limited, X-Hanzo-Topup-Url. The usage row's model is the SKU that answered.
  • ZAP has no gate, so its chat, responses and messages handlers serve a router id as enso-free (unpaid).
  • Listed (depthListings): each router id is a row of /v1/models, its default tier's row and price (listedAs) under its own id, the narrowest window of its tiers, and a description of where it routes. Never free.
  • Tests: TestARouterIdIsServedOnItsTierOrItsFreeSKU (every payer case, the switch, a program's call, a tier named outright), TestDepthRoute*, TestDepthRouting*, TestTheRouterIdsAreListed, TestARouterIdListsAtItsDefaultTiersPrice.

Who pays is the host's call — object.Limits (routers/filter_balance.go)

A call names the API key it arrived on. The identity boundary in front of this module writes X-Key-Id (<org>/<id>: the org the key is filed in and its id there; stripped on ingress, written only from a resolved key); TenantContextFilter threads it as GenAIAttribution.Key, usageRecord.bind copies it, and the native debit carries it as UsageEvent.Key. The host keeps each key's own spend by it, which is what a per-key budget is held to.

On every chat path and /v1/decisions the gate asks the host's usage policy (object.LimitFunc, cloud apps/ai/limits) with the model's Class (controllers.ClassOf: free when it costs nothing, ours for Enso/Zen and any route owned_by: hanzo — Kai, not the Jev ids the decision service forwards — premium otherwise) and whether it is Priced. The answer is a LimitGrant with Pays:

  • plan / free — covered: Cover puts the grant on the request locals AND context; reserveFor, enforceBalanceGate and the decision hold skip the wallet; usageRecord.bind picks the grant up from the context, the record carries Plan, and recordUsage settles planUse (list price; COGS for a model sold at zero) against the grant.
  • prepaid / credits — the wallet gates as before; Cash on the grant rides the record to UsageEvent.Cash so the host draws cash only.

A LimitHit refuses by Code with no figure: 429 usage_cap_exceeded (a plan window, the free lane included — limited mode cannot be farmed), 429 free_plan_cap, 402 plan_allowance_used, 402 paid_plan_required, 402 model_cap (the model used its share of the plan; Model names it or its pattern, Fallback the Hanzo model that answers instead). A conversation from a signed-in app (token aud) or a client sending X-Hanzo-Fallback: allow — and, while the paid lane's switch is on, a chat past a paid plan's included usage (plan_allowance_used, model_cap) from any client — is handed on instead — model_cap to Fallback, the other two to FreeModel — marked X-Hanzo-Fallback and X-Hanzo-Usage-Reason, and the model it lands on is asked of the policy again (up to three asks). Refusal actions: upgrade, switch (to the fallback), and credits ("Continue with credits", which turns on the org's PUT /v1/ai/limits opt-in) when Credits says the payer holds what could pay, else topup. Served calls carry X-Hanzo-Usage, X-Hanzo-Usage-Class, X-Hanzo-Paid-By (plan|credits|free — prepaid reads credits). An unreadable policy decides nothing (the wallet gates). ZAP twins have no gate verdict, so they stay wallet-only.

How a refused caller raises its limit is the host's to say too (object.RaiseFunc, SetRaise / RaiseHook). Every refusal a limit lifts — 429 allowance_spent (freeRefused), every LimitHit (limitReached), the wallet's 402 (denied) — is written by refuse, which asks the hook once with the person (person), subject, namespace and the refusal's code, bounded by raiseWait (500ms; a panic or no answer leaves the refusal as it was). A Raise adds error.tier and error.raise {method, path, body, message} after the site's own fields, and on a 429 X-RateLimit-Limit = Raise.Limit (only when > 0), Remaining 0, Reset = the refusal's reset, unless standing already set them. nil — no hook, or a caller the host does not single out — is byte-identical to before; a 503 never asks. The path and the copy are the host's: this module names neither.

Every priced endpoint asks the host, not only chat (controllers.Entitled): chat, /v1/decisions, embeddings, rerank, speech, transcriptions (the model is a form field), the live transcript's open, voice/music/foley, images and videos. A push to or the close of a live transcript names no model; it is asked as the model its open named (controllers.TranscriptModel) with Session set, and counts nothing. The media and retrieval controllers hold through reserveFor, so a covered call reserves no wallet, and the transcript's per-push metering settles against the push's grant. A request the host counted carries Release; the gate calls it when the answer is 400 or above (a failure or a refusal), so an unserved call keeps no window, cap or share. A stream is served from its first byte.

auto, zen-router and a request naming no model are the router's to resolve (isAutoModel), and routable is the one eligibility rule it folds into Known and the last-resort Allow floor: with an org enabled-models allowlist, exactly what it names; without one, only Hanzo classes (ours + free). A premium model answers auto only when the org allowlisted it; otherwise a caller names it.

A call a program in this process makes for a customer (object.Caller, ai.For)

A sibling app's embeddings, rerank or completion bought over the host's plane (cloud apps/ai/chat_rpc.go) reaches ai through ai.For(object.Caller{Org, Person, Project}): the same app and filters, with the caller stated as a fasthttp user value, which only code in this process can write. CallerFilter moves it onto the context. Then:

  • Attribution is the customer's: TenantContextFilter takes org, person and project from the caller, never headers; billingOrg is the caller's org, so the usage row is on the customer's books; tenant tells a family the caller's org.
  • Billing happens once, by the program, at ai's price: the program meters the call and charges the customer itself (cloud metered_ai.go), so the gate covers it with a PaysCaller grant (callerPays): no wallet is asked, the deployment's whose token carried it least of all, the plan is not asked, the paid lane seats it as the customer's, and the usage event is Plan: true, PaidBy: "caller" (no debit), priced as usual for the record. The price is TOLD to the caller: Caller.Bill (object.Bill) rides the grant (LimitGrant.Bill), and recordUsage adds each answer's usageMargin(record).BilledNano to it — what the same answer debits any other caller, a hedge's losing provider included. The program charges exactly that; it has no rate of its own. controllers.PriceOf(model, org) is the billed row as a Rate, for a program that must reserve before it calls; an unpriced model answers false, never the conservative default. Pinned by TestAProgramIsToldWhatItsCallCost, TestOnlyACallersGrantIsToldItsPrice, TestPriceOfIsTheBilledRow.
  • A request from the network states no caller whatever headers it carries (TestARequestFromTheNetworkStatesNoCaller), and For carries no identity header in (authz.Headers, authz.Retired): the identity is the caller and the credential (TestForCarriesNoIdentityHeader).

The paid lane: ONE switch, then the plan that pays

controllers.FreeOnly is the platform's switch, and ai does not own it: it is the zen catalog's paid line as the zen service's own discovery states it (GET ZEN_URL/v1/models, top-level "paid": true; zenFam.open), free unless the catalog, universe charts/app/files/zen/catalog.yaml, says paid: true. Spending is an opt-in: a catalog not yet read, or silent on it, keeps the paid lane off, and a failed refresh keeps the last catalog's line. GitOps is the source of truth; there is no settings row, no FREE_ONLY env and no second line (the enso service reads the same line through ZEN_SWITCH). A cloud that still links zen in process assigns it from that zen (aicontrollers.FreeOnly = z.Free) until its zen moves out; nothing else sets it.

A closed lane bounds the platform's cash, so it reaches a priced Hanzo SKU only where one could be spent: an embedding no stand-in answers is sent to its family when the catalog funds it from grants and our own compute on every path (spends: its listing says no "funding": "prepaid"), and refused when it does not.

With the switch on, each request is on one lane (controllers/lane.go), decided once by controllers.Seat inside routers.BalanceGateFilter from the host's grant: paid when the grant names a paid plan (LimitGrant.Plan — cloud apps/ai/limits planOf: an active subscription whose period was paid by card or recorded externally, with figures from apps/flags PlanSeeds, per seat for team rungs), the plan's included usage or the payer's prepaid/granted credit past it pays for a priced call, and the call's estimate fits; free otherwise (no plan, a free model, a model's free daily cap, a policy that said nothing, a call that did not fit, and a request that never passed the gate — every ZAP handler; the forward bridge is the ZAP route that runs the gate). routers.LaneFilter only labels a request the gate did not seat. Every read on the request path is FreeOnlyFor(ctx) (or shut(ctx, p), which also lets the org's own key through while the switch is on); candidates, openrouterTail, paying, spendOf and every ZAP handler (zapLane) take its answer.

A seat holds the call's quote (controllers/quote.go, quoteOf): every body byte as a token (a tokenizer never reads fewer than one byte per token), plus 1,024 for the vendor's framing and 2,000 per vendor-defined tool; a decodable picture (PNG, JPEG, GIF, WebP, ≤ 20 MiB) in an image part as its 768-px tiles at 258 tokens (at least 6,000, never below $0.01) in place of its bytes; a remote picture as an 8,000-px one; a PDF, a remote document or a stored file as a whole 1M-token context (or the model's declared window when larger); a prompt with cache_control at the hour-long cache-write rate; the completion ceiling the handler sends (reserveCompletionTokens) × n, plus a predicted output at the completion rate — at the most of list price and cost, the platform's and the org's, ×2 for fast mode; media at unit price.

The paid lane has no ceiling of its own (owner, 2026-10-09: no paid on free; no ceiling but what people prepay, or their usage). Nothing caps what paid-lane calls spend together or what one org's calls spend; there is no platform day and no org share. A seat holds, for a plan-paid call, against the plan class's LimitGrant.Spend less the payer's calls in flight; a wallet-paid call (prepaid cash, the org's credit line) is held against the wallet by its handler (reserveFor); one payer holds at most 8 seats (paidLaneCalls). A plan-paid chat that does not fit at the ceiling it asked for is sent with the highest one that fits (never below 4,096), named in X-Hanzo-Lane-Max-Tokens (laneTokens in the handlers). A family is sent X-Hanzo-Spend only on a seat, capped at what it holds. A seat closes exactly once, when its answer is counted (recordUsage → seat.settle, after the plan's settle); a fast mode loser gives nothing back (seat.lost; ask.race holds the seat until every loser is counted); Unseat closes one that recorded nothing after its handler; a stream whose client takes nothing for 2 minutes (seat.paced, applied in ApiController.SendStreamWriter) and any seat after 30 minutes lapse. Race losers and a video's later poll bill the creating request's context only on a seat (billing); off it they bill as before. The holds live in this process, as the wallet ledger's do, under the same single-replica invariant (bootstrap.go refuses to start at CLOUD_REPLICAS > 1). Provider-side loops — web fetch, tool search, compaction — are not served on the shared Anthropic account. A payer the lane had no room for gets X-Hanzo-Lane-Reason: paid_lane_full; a chat for a third-party model is handed to the free model in limited mode (never one on the org's own key, OwnKey), and any other call only the paid lane serves is refused paid_lane_full (laneOff): 413 when the plan has less left than the call's least, 429 when the payer's calls in flight hold the rest or it holds 8 seats. Responses carry X-Hanzo-Lane: paid|free only while the switch is on. The enso/zen service's own paid lane (zen lib: the catalog's paid, or the ZEN_SWITCH file's) spends what a fronted request's X-Hanzo-Spend allows; what zen spends for callers that reach it directly is bounded by the host's plan limiter (cloud apps/zen planSpend).

On the free lane no chat request reaches a priced route, and the routes that stand in for it answer, named as what they are; a tool or media request for one, and every embeddings, rerank, image, audio and video call to a provider that is not a family's own service (paying), is refused, and so is a decision that names Jev. Only zen and enso, whose catalogs decide for themselves, serve.

OpenRouter's list, kept in the store (controllers/listing.go, object/listing.go)

OpenRouter's /v1/models is synced into the store and the family reads its catalog from there, never from the vendor: every process lists, routes and bills from the same rows, and the free lane reads the same rows.

  • When: a process's first read syncs unless any process did within 5 minutes (listingStart); then once an hour (listingEvery), fleet-wide. object.TakeLease is a compare-and-set on the lease table, so one process runs an interval's sync; a failed sync gives the interval back. keepListed reads the family every TTL so the schedule does not wait on traffic.
  • What: object.Listing per model, Item the vendor's entry verbatim (every price, context_length, top_provider.max_completion_tokens, modalities, supported_parameters, created, expiration_date), Pricing canonical (keys sorted, exact decimals: a respelled number is no change), Free (prompt and completion zero, not a -1 router), Seen, Synced, Gone. A list of nothing is refused, never synced as every model gone.
  • Changes: object.ListingEvent rows new, back, gone, price, free (priced → free), paid (free → priced), with Was/Now price lists, then object.PublishListing (the host installs SetListingPublisher: cloud puts them on its bus). A sync that seeds an empty store records its rows and tells the bus nothing. A gone model leaves the catalog and is served no longer; its row and history stay.
  • Billing a resold SKU (zenModel.bill): tokens at the tier the prompt reaches (min_prompt_tokens overrides are tiers, the dearer side of the line; an hours-of-day override is a discount and is never billed), cached tokens at input_cache_read, all × the markup; and never below the answer's stated usage.cost × the markup, which covers what tokens do not price (cache writes, web search, a dearer provider). A variable router bills the stated cost alone.
  • The markup is 1 + the vendor's fee, a percent the host's operator sets (object.SetFees; cloud reads it from admin.hanzo.ai's usage policy, seeded at OpenRouter's 5.5%). ai keeps no fee of its own: with none set a resold model bills at its vendor's price, and a fee that is not a percent of 0 or more fails the catalog's refresh, which keeps the prices it had. Zen and Enso SKUs carry Hanzo's margin in their own catalogs; a resold model carries only the vendor's fee.
  • Listed: /v1/models rows carry free, expires, max_output_tokens, and pricing.rates (every vendor rate × the markup, exact, in the vendor's names and units) and pricing.overrides (each condition with its rates × the markup).
  • Read: GET /v1/models/vendors/openrouter[?free=1] (anonymous) answers {vendor, synced, data:[the vendor's entries]}, the shape the free lane parses; GET /v1/models/vendors/openrouter/events[?id=&limit=] the history; SuperAdmin GET /v1/admin/ai/free the free set with the last 24 h per model (answered, failed, rate) and per account. Counts are object.Served (vendor, model, account name, UTC hour; 48 h kept): each free request ai sends on its ring (counting), each free arm a family ran on a listed vendor (X-Hanzo-Key names the account), and each failed free arm of a failover chain (no account).

The free pool — every OpenRouter account, spent as one

The Free plan is limited usage from ONE pool every free user shares: the OpenRouter accounts named by object.OpenRouterKeys (OPENROUTER_API_KEY, _2, _3, resolved KMS-first), spent by controllers/family_keys.go.

  • A free route turns the ring; a priced route does not. OpenRouter's free allowance is per ACCOUNT (20/min; 50/day, or 1000/day once $10 of credit was bought), so a free request starts one key further round each time. A priced request starts at the first key, the funded one. An admin row's single key is sent as before and never benched (one tenant's 402 must not bench it for all).
  • A 429 is read, not guessed. freeQuota benches a key for free routes only when the message STARTS Rate limit exceeded: free-models-per-min|day, until X-RateLimit-Reset (header or error.metadata.headers, ms epoch), else the window's own length; a bench is never shortened. Any other 429 on a FREE route is the model limited upstream: answered after one account (asking all three is three calls for one refusal) and the pool moves it to the next route. On a priced route it still asks the next account.
  • FreePool() is the pool's standing: state (available | busy | exhausted) and when it refills, per process. Cloud's ai_pool plane op publishes state + refill only (no account counts) and GET /v1/allowance carries it to the Free usage page.
  • A busy FREE route moves to the pool (fallback): enso-auto on the enso service's one account answering 429 while three accounts sit idle made a shared pool a queue for one key. A busy PRICED route still stays itself.
  • The pool answers for itself only when an account is benched. A free-lane request the pool could not serve gets object.PoolRefused (429 pool_busy / pool_exhausted, upgrade_url = object.PayURL, hanzo.ai/pay) only when FreePool() is not available; a walk that failed with every account ready (502s, nothing discovered) keeps its path to the route's alternates. Busy never carries a reset past a minute; exhausted carries the refill and x-should-retry: false. A caller on a paid plan is told to pick a paid model, not to upgrade. A person's own share spent is 402 allowance_spent (object.AllowanceSpent) naming the window; object.SpentFunc answers a Standing. All three codes are billingNotice: relayed, they are never moved to the pool.
  • Free is held for seconds, not minutes (familyFreeTTL, freeTierTTL, 10s), and the rate limiter re-asks a free entry's tier: it is the one answer a payment changes, so a paying org is rated as what it bought by the time checkout brings it back.

No escape hatch: the gate refuses what it cannot decide, and every call is a row

  • Fail closed. A usage policy (object.LimitFunc) or free allowance (object.SpentFunc) that errors refuses 503 usage_unavailable (object.UsageUnavailable), on free and priced models alike — an outage of the counts is never a way past them.
  • No configured tier. RATE_LIMIT_TIERS is deleted: a tier is the caller's recorded subscription (commerce), never an operator mapping.
  • Shared counts. object.SetCounters(QuotaFunc, TakeFunc) puts RateLimiter (per-minute buckets) and Quota (8h/week/month) on the host's store, so every replica counts together and a restart forgets nothing; with none installed they count in process (standalone).
  • One transcript lookup. controllers.TranscriptModel is the only one; the rate filter and the plan gate both ask it.
  • Limited mode is one rule. fallback(c, path, code): in chat, from a signed-in app or X-Hanzo-Fallback: allow, any refusal for want of payment — plan_allowance_used, paid_plan_required, model_cap, insufficient_balance (the wallet's own 402) — is answered by the free model, with X-Hanzo-Fallback (lane), X-Hanzo-Usage-Reason (the real code), X-Hanzo-Usage: limited, X-Hanzo-Topup-Url, X-Hanzo-Upgrade-Url; the handler names the concrete model in X-Hanzo-Served. Non-chat callers keep their 402.
  • Every served call is billed and described. A unit-priced call (Zen media, scrape, crawl, RAG chat) carries its charge as BilledNanoExact — Cost alone was read by nothing on the money path and debited $0. A bi-encoder rerank bills the tokens it embedded. A vendor key pays as its row's owner on the ZAP wire as on HTTP. UsageEvent carries Class, Units, CostUSD and PaidBy ("plan", "hanzo" for a free model the platform absorbs, "" for the wallet) so the host writes a row for every call.