Files
hips/HIPs/hip-0410-llm-crd.md
zeekayandHanzo Dev f4684d1b16 One workflow directory per repository, and the shared tier everywhere else
The CI rule needed correcting twice in one pass, and the second correction came
from measuring rather than from the brief. `hanzoai/hips` moved its own
workflows to `.github/workflows` on 2026-09-08 with a sound reason: the forge
reads that directory natively and github.com reads nothing else, so one
directory serves both planes. Measured across the first-party checkouts, twelve
keep their caller in `.hanzo/workflows/` and two forks carry an upstream's
inherited `.github/workflows` — 24 files in datastore, six in kms, none of them
ours.

So the rule is not the directory's name. It is that a repository keeps exactly
ONE, because the forge reads the first that exists rather than the union: a
split silently retires half the pipeline, and the checks that would go red are
the ones no longer running.

The rest of this pass takes the per-app stores out of six more HIPs. The admin
console had Postgres+Prisma, ClickHouse, Redis+BullMQ and MinIO of its own;
serverless had a `hanzo_functions` database; commerce tracked idempotency in
"Redis"; the zero-trust HIP named SPIFFE identities for `postgres` and `redis`,
which are the upstreams we fork rather than the services we run. Payment
processing also had Commerce POSTing balances into IAM through `add-balance` and
`add-transaction`, which contradicts both HIP-0111's forbidden-verb list and the
correction that Commerce owns the position and IAM mirrors it.

Deployment carried the last `/api/` paths and the corpus's only `v2`
(`api.hanzo.ai/v2`, offered as an example of path routing), plus a `resources:`
block that provisioned a per-app Postgres and Redis. It now names which rank of
the shared tier an app is a tenant of.

HIP-0144 replaces the number this HIP was drafted at: 0138 was published as
"Cloud — Unified Hanzo Binary" and retired, and the index generator refused the
reuse — correctly, since every existing reference to HIP-0138 would have
resolved to a different specification.

Two self-test guards were reported as not holding. Both were the harness's own:
the FM011 case appended a second `implementation-go:` key to a subject that
already declared one, so the later key won and the contradiction was never
built; and test-index restored HANZO_CAPABILITIES only when the caller had
exported it, so every case after the capabilities case ran against a missing
input. A check whose test is a no-op has never been shown to work.

The corpus now passes lint, index --check and both self-tests, from 23 errors.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-09 12:50:48 -07:00

2.3 KiB

hip, title, author, type, category, status, implementation-rust, created
hip title author type category status implementation-rust created
0410 LLM CRD Hanzo Platform Team Standards Track Operator Final partial 2026-05-19

HIP-0410: LLM CRD

Abstract

The LLM CRD is the facade Kind for the Hanzo LLM gateway (hanzoai/gateway, which historically also lived as hanzoai/llm). It proxies 100+ upstream model providers and exposes a unified OpenAI-compatible API. The reconciler delegates to the Service controller. See HIP-004 for design.

Specification

Group + version

hanzo.ai/v1, plural llms, shortname llm.

Spec fields

Same shape as Service (HIP-400). Conventionally:

  • image.repository: oci.hanzo.ai/hanzoai/gateway
  • ports: containerPort: 4000
  • kmsSecrets supplies its own machine identity — IAM_CLIENT_ID / IAM_CLIENT_SECRET at hanzo/gateway/<NAME>@prod (HIP-0136) — and nothing that spends. It carries no upstream provider key. An earlier revision of this HIP had it consume gateway-secrets via envFrom with ANTHROPIC_API_KEY, OPENAI_API_KEY and the rest; those keys are in KMS custody behind egress now, and the gateway asks egress for a call rather than holding a credential (HIP-0143).
  • depends on egress, and on the shared stores per HIP-0144 rather than on instances of its own

Example CR

apiVersion: hanzo.ai/v1
kind: LLM
metadata:
  name: gateway
  namespace: hanzo
spec:
  image:
    repository: oci.hanzo.ai/hanzoai/gateway
    tag: 2.14.1
  replicas: 2
  ports:
    - name: http
      containerPort: 4000
      servicePort: 80
  kmsSecrets:
    - name: gateway-env-kms-sync
      secretsPath: /gateway
      keys: [IAM_CLIENT_ID, IAM_CLIENT_SECRET]
      secretName: gateway-env
  resources:
    requests: { cpu: 500m, memory: 1Gi }
    limits:   { cpu: 2,    memory: 4Gi }

Generated K8s resources

Deployment, Service, optional HPA.

Operator reconciler

~/work/hanzo/operator/src/controllers/service.rs (via LLM facade).

  • HIP-0004, HIP-0043 (LLM gateway, inference standards)
  • HIP-0143 (egress — where the upstream credentials live)
  • HIP-0136 (the path the machine identity is addressed at)
  • HIP-0144 (the stores it is a tenant of)

Status

Implemented in hanzoai/operator v0.3.0+. CR gateway active in cluster.