Files
zeekayandHanzo Dev 6ade4b0a89 api.lux.cloud could not resolve any API key
Three causes, each sufficient on its own: no IAM client credential (so the
resolver never made a call), an image below the tag where a machine credential
resolves its org from the token subject, and probes aimed at a port this binary
does not open. Pin sits under v1.801.487, where the console became a published
site and boot starts requiring one.

Co-authored-by: Hanzo Dev <[email protected]>
2026-08-10 16:09:51 -07:00

225 lines
8.8 KiB
YAML

# Lux Cloud API — the unified `cloud` binary (ghcr.io/hanzoai/cloud) serving
# api.lux.cloud. ONE unified binary on :8000 (product API) + :9090 (ops/health),
# white-labelled to Lux purely by env. Replaces the retired `cloud-api`.
#
# Gotchas: runs non-root → needs securityContext.fsGroup + a writable PVC at
# CLOUD_DATA_DIR (embedded Base); RWO PVC → replicas=1 + strategy=Recreate
# (single writer). Distroless image.
apiVersion: apps/v1
kind: Deployment
metadata:
name: cloud
namespace: hanzo
labels:
app: cloud
app.kubernetes.io/managed-by: lux-universe
app.kubernetes.io/part-of: lux-cloud
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: cloud
template:
metadata:
labels:
app: cloud
app.kubernetes.io/part-of: lux-cloud
spec:
imagePullSecrets:
- name: ghcr-secret
securityContext:
fsGroup: 1000
runAsNonRoot: true
runAsUser: 1000
containers:
- name: cloud
# The ONE proven unified cloud binary (same image hanzo prod runs),
# white-labelled to lux.cloud purely via env.
#
# Floor >= v1.801.245: that is where a MACHINE credential resolves its
# org from the token subject. Before it, an sk- key resolved to a
# principal with no `orgs` claim and the identity boundary — correctly
# refusing a human token that lost its claim — refused the key too, so
# every API-key request on this host answered "X-Org-Id required" no
# matter how good the key was.
#
# Ceiling < v1.801.487: there the console stopped being bytes in the
# binary and became a published site, so boot REQUIRES a released
# `lux/hanzo-console`. This front door has none, and the console load
# is fatal by design — v1.801.496 crash-looped on exactly that.
#
# v1.801.476 is the newest tag inside that window with a PUBLISHED
# IMAGE. Tags above it exist in git and resolve to nothing in the
# registry, so a pin must be checked against ghcr, never `git tag`.
image: ghcr.io/hanzoai/cloud:v1.801.476
imagePullPolicy: IfNotPresent
env:
- name: CLOUD_LISTEN
value: ":8000"
- name: CLOUD_DATA_DIR
value: /var/lib/cloud
- name: RUNNING_IN_DOCKER
value: "true"
# NO CLOUD_ENABLE -> mount ALL subsystems EXCEPT the staged `iam`
# (HIP-0106). This is what makes ai (/v1/models), tasks (/v1/tasks)
# and visor (/v1/machines) all mount, matching hanzo prod. A curated
# CLOUD_ENABLE list silently drops ai's special mount gate -> 503.
# Sovereign lux IAM: validate against lux.id (its own JWKS). The
# issuer is token-visible, so it must match what prod already minted
# under — flipping it invalidates every live session.
- name: IAM_URL
value: https://lux.id
- name: IAM_ISSUER
value: https://lux.id
- name: CLOUD_IAM_ISSUER
value: https://lux.id
- name: AUTH_ISSUER
value: https://lux.id
- name: JWT_ISSUER
value: https://lux.id
- name: JWKS_URL
value: https://lux.id/v1/iam/.well-known/jwks
- name: IAM_KEYS_URL
value: https://lux.id/v1/iam/.well-known/jwks
# The confidential client cloud presents to resolve an sk- API key
# (get-user?accessKey). Without it iamCred() is "" and the resolver
# short-circuits to nil, so EVERY sk-/pk- arrives anonymous and the
# answer is "X-Org-Id required" — a 403 that reads like a bad key
# while the key is perfectly good. A JWT still worked, which is why
# this went unnoticed: only key auth was dead.
# `hanzo-cloud` is the binary's own service identity, the same one
# it presents on api.hanzo.ai, and one of the two clients IAM's
# IAM_KEY_RESOLVE_APPS admits at that door. A brand app cannot be
# used here: lux-cloud is a SPA and holds no secret.
- name: IAM_MINT_CLIENT_ID
valueFrom:
secretKeyRef:
name: cloud-iam-client
key: clientId
- name: IAM_MINT_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: cloud-iam-client
key: clientSecret
- name: CLOUD_KMS_MASTER_KEY_REF
valueFrom:
secretKeyRef:
name: cloud-kms-master-key
key: MASTER_KEY_B64
- name: IAM_AUDIENCE
value: lux-cloud
- name: GATEWAY_ALLOWED_AUDIENCES
value: lux-cloud,lux-console,lux-id,https://api.lux.cloud
- name: AUTH_PUBLIC_PATHS
value: /healthz,/readyz,/__stats,/.well-known/,/favicon
# Published-site edge (clients/sites) — LUX apexes, never Hanzo's.
# The apex is per-PROCESS, not per-request: sites.New falls back to
# hanzo.app when CLOUD_SITES_APEX is empty and pins first-party sites
# to org `hanzo`, so an unset Lux deployment silently serves Hanzo's
# site namespace off a Lux host. Naming them is the white-label.
# lux.link is the multi-tenant apex (`<slug>.<org>.lux.link`);
# lux.network is the brand apex, where sites are OPT-IN only
# (FIRSTPARTY allowlist) because it also carries api/docs/explore —
# one missing denylist label there would let a project shadow a real
# host. Self domains keep every real Lux host out of custom-domain
# binding entirely.
- name: CLOUD_SITES_APEX
value: lux.link
- name: CLOUD_SITES_SELF_DOMAINS
value: lux.link,lux.network,lux.cloud,lux.id
- name: CLOUD_SITES_FIRSTPARTY_APEX
value: lux.network
# Empty = fail-closed (the default is Hanzo's own cd,flow,gallery).
# Add a label here only once that site is published through
# POST /v1/sites/deploy and its host is routed to this edge.
- name: CLOUD_SITES_FIRSTPARTY
value: ""
- name: CLOUD_SITES_FIRSTPARTY_ORG
value: lux
# ai (casibase) runtime boot: driverName gates it (empty => 503
# "ai runtime not initialized"). sqlite lives on the RWO PVC.
- name: driverName
value: sqlite
- name: dbName
value: hanzo_cloud
- name: modelConfigPath
value: /conf/models.yaml
- name: TIKTOKEN_CACHE_DIR
value: /var/lib/cloud/tiktoken
ports:
- containerPort: 8000
name: http
# The probe belongs to the IMAGE. This binary answers /healthz on the
# product listener and opens no second ops port — CLOUD_HEALTH_LISTEN
# is read by nothing here — so probing :9090 fails every time and the
# liveness probe kills a process that is serving traffic perfectly.
# /readyz does not exist either; /healthz is the one door.
livenessProbe:
httpGet:
path: /healthz
port: http
initialDelaySeconds: 20
periodSeconds: 30
readinessProbe:
httpGet:
path: /healthz
port: http
initialDelaySeconds: 10
periodSeconds: 10
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: "2"
memory: 2Gi
volumeMounts:
# Durable tasks/fleet store + ai sqlite. Mounted at BOTH paths:
# CLOUD_DATA_DIR and the ai runtime's CWD-relative ./data (=/data/cloud).
- name: cloud-data
mountPath: /var/lib/cloud
- name: cloud-data
mountPath: /data/cloud
volumes:
- name: cloud-data
persistentVolumeClaim:
claimName: cloud-data
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: cloud-data
namespace: hanzo
labels:
app.kubernetes.io/part-of: lux-cloud
spec:
accessModes: [ReadWriteOnce]
storageClassName: do-block-storage
resources:
requests:
storage: 5Gi
---
apiVersion: v1
kind: Service
metadata:
name: cloud
namespace: hanzo
labels:
app: cloud
app.kubernetes.io/part-of: lux-cloud
spec:
selector:
app: cloud
ports:
- name: http
port: 8000
targetPort: 8000
- name: zap
port: 9653
targetPort: 9653
- name: tasks-zap
port: 9999
targetPort: 9999