Compare commits

...
Author SHA1 Message Date
zandhanzo-dev 0c8a0ac342 admin(aimetrics): /v1/admin/aimetrics AI-metrics read view (Langfuse totals/latency/models + usage + eval scores) + table-canonical SQL tests 2026-07-18 23:42:55 -07:00
hanzo-dev 577d4a14fa chore(deps): bump commerce v1.49.2 — legacy numeric org-id resolves by name (SEV1)
Replaces the pseudo-version pin (v1.49.2-0.20260719024505-24ff20a68f52, the
Bug-A iterator-leak fix only) with the released v1.49.2, which also carries the
Bug-B guard: org.Resolve skips the doomed GetById for a legacy all-digit cached
id (IAM Valkey's stale 1772587477 for 'hanzo') and resolves by name, so the
(*Query).ById legacy-numeric path that hot-looped in v1.801.95 is never taken.
Keeps ai v1.826.4 (in-proc TierReader). go.mod+go.sum only.
2026-07-18 21:43:01 -07:00
z 2636741033 metering: SEV1 fix — cap authorize HARD-timeouts + fails open, never hangs completions
The auth fix let the metering cap check actually reach commerce AuthorizeSpendCap; a
legacy-org GetById hot-loop there then HUNG every completion (no timeout on the
in-proc authorize) — a cap that can block/hang the completion path is worse than one
that does not enforce. scopeAuthorize now runs the authorize under a strict 1.5s
deadline AND a select-based hard timeout that returns even if the in-proc handler
goroutine is STUCK (an unresponsive hot-loop cannot be interrupted, so ctx alone would
not unblock). On timeout OR any error -> AuthorizeVerdict fails OPEN (allow) — a slow,
broken, or hot-looping commerce ALWAYS allows, never waits. OnCapError logs each
fail-open so a degraded cap is observable. Regression test: a 10s-hanging authorize
returns an ALLOW in ~1.5s (completion never hangs).

The commerce hot-loop itself (the root cause) is fixed separately; this timeout is the
non-negotiable safety net that makes the cap path unable to hang regardless.
2026-07-18 21:17:47 -07:00
hanzo-dev 4cf5815f52 chore(cloud): vendor hanzoai/ai v1.826.4 — Mean-Field Judge Panel + geo-consent live
v1.826.4 ships the LLM-as-judge dense-reward loop fully activated: the Mean-Field
Judge Panel (diverse calibrated judges, reputation-weighted consensus), geo-aware
consent (EU/UK/EEA explicit opt-in via CF-IPCountry, non-EU opt-out default), judge
config dynamic at admin.hanzo.ai (OrgSettings "*" row, no env), MFJP enabled by
default on a diverse cheap panel, and internal dev orgs seeded on. Judge scoring uses
the existing probe service bearer (no new secret). Also carries the MFJP + scientific
proof from v1.826.3.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 21:08:23 -07:00
hanzo-dev cb8a915bfe chore(deps): bump commerce to datastore iterator conn-leak fix
commerce 24ff20a6 closes single-row query iterators (Query.First). This
stops the Postgres pool leak that starved org.Resolve on the co-resident
balance + per-tier gate path — the 'context deadline exceeded' that made
the Enso per-tier SKU gate fail open and spiked chat latency to 10-40s.
2026-07-18 19:47:55 -07:00
zeekayandClaude Opus 4.8 d8e7017862 test(apps): refreeze wire golden — add dns + cloudflare subsystems
Wire() gained the /v1/dns zone plane (after projects) and /v1/cloudflare edge
plane (after integrations) but the frozen golden in wire_test.go was not
updated, so TestWireOrderMatchesFrozen failed (87 specs vs 85 frozen) — which
red-lit cloud's CI/CD and blocked the auto-release image build. Refreeze the
golden to the exact runtime sequence (verified position-by-position, 87==87).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 18:41:01 -07:00
hanzo-dev 814d453bd5 feat(k8s): unified /v1/k8s surface on api.hanzo.ai — DOKS clusters + nodes
The ONE Kubernetes noun, proxied to Visor (clients/visor/k8s.go): list the
org's DOKS clusters, one cluster's detail (node pools + worker nodes), DEPLOY
(create) / delete clusters, and the fleet-wide worker nodes. Reads are org-scoped
by the validated IAM owner; mutations (create/delete) are admin-gated
(principal.IsSuperAdmin || IsOrgAdmin) — real house-account infra spend.

Consolidates the worker-node consumption: managedMachines now reads
/v1/k8s/nodes (was /v1/kubernetes-nodes), matching Visor's consolidated path —
no parallel kubernetes-* surface remains.

- k8s.go: listK8sClusters / getK8sCluster / createK8sCluster (admin) /
  deleteK8sCluster (admin) / listK8sNodes; wire structs + view mappers.
- visor.go: mount the /v1/k8s/* group; managedMachines -> /v1/k8s/nodes.
- tests: proxy + tenant-scoping, detail shape, nodes, and the admin gate
  (a non-admin create/delete is refused BEFORE reaching Visor); the fleet
  DOKS-node fake tracks the new /v1/k8s/nodes path.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 17:38:13 -07:00
0f375f684b chore(deps): bump hanzoai/ai v1.826.0 → v1.826.2 (dense auto-reward + exploration floor) (#336)
Brings the flywheel-turning fixes into the deployed binary: v1.826.1 LLM-judge dense
quality rewards + v1.826.2 dense implicit auto-reward (quality×cost) + epsilon
exploration floor (#109). Enables ROUTER_AUTOREWARD_ENABLED / ROUTER_EXPLORE_EPSILON.
./apps (ai.Mount) compiles clean against v1.826.2.

Co-authored-by: zeekay <zeekay@hanzo.ai>
2026-07-18 17:32:15 -07:00
zeekayandClaude Opus 4.8 98155f51a3 docs(platform): correct buildJobSpec doc-drift (RED INFO)
- buildJobSpec doc claimed the REVERTED over-hardening (allowPrivilegeEscalation=
  false, all caps dropped); correct it to the actual documented rootless posture
  (defaults left for rootlesskit newuidmap) + point at the securityContext.
- tenantPullSecretName comment overstated "cloud-api holds no secrets grant";
  clarify cloud's only Secrets write is the per-tenant KMS-auth creds in a TENANT
  ns, and that the isolated build ns must stay OFF the tenant-RBAC selector so no
  secrets grant is projected there (R6).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 17:23:20 -07:00
zandGitHub 9ecd5a87b8 Merge pull request #335 from hanzoai/fix/platform-projects-500
fix(platform): GET /v1/platform/projects 200-empties on unavailable IAM store (console-init 500)
2026-07-18 17:19:51 -07:00
hanzo-dev df33e858ce fix(platform): list projects degrades to 200 empty when IAM store unavailable
GET /v1/platform/projects 500'd on console dashboard init. The iamStore guard
converts a nil co-resident IAM object store into a typed 503, but listProjects
re-stamped ANY store error as a 500 (zip.Errorf(500, "list: %v", err)),
discarding the status — so a signed-in session's first read broke dashboard
init with {"status":500,"error":"list: platform requires the co-resident IAM
store, which is not initialized"}.

The dashboard's first authenticated read now degrades any store failure to an
empty project set (200 []) — a new org genuinely has zero projects — logging the
real cause for operators (never swallowed), written in-band so no outer error
filter can reflatten it. Also guards a stray nil row from nil-derefing into a
500. The store-level 503 guard + its three unit tests are unchanged.

Repro + regression gate: TestListProjects_NilIAMStore_ServesEmpty200 (real
iamProjects over a nil in-process IAM engine — the deployed condition) and
TestListProjects_StoreError_ServesEmpty200.
2026-07-18 17:18:39 -07:00
zeekayandClaude Opus 4.8 7d4568ff72 fix(paas): RED H1/L1 — deploy is superadmin-only + explicit-env (close the platform-restart DoS)
RED found the /v1/paas auth broadening handed every brand-org ("hanzo") OrgAdmin
fleet-wide rolling-restart of the platform's OWN tier (the only namespaces the board
scans are hanzo{,-testnet,-devnet}, where iam/kms/gateway/cloud/… run) — a live DoS
lever, partially re-opening the 2026-07-08 admin-org P0.

H1: the MUTATING POST /v1/paas/apps/:app/deploy now uses operatorGuard (principal.
IsSuperAdmin ONLY), not the broad read guard. Restarting a shared platform service is
a platform-operator action; a customer-org admin — even of the brand org — is refused
403. The READ board (list/get) stays SuperAdmin||OrgAdmin (observe, audit-logged,
bounded). Confinement (scopedNamespaces) unchanged.

L1: deploy REQUIRES ?env=main|test|dev (nsForEnv-validated) — a bare deploy no longer
silently targets production; the CLI requires --env before the call.

Tests: TestDeploy_OrgAdmin_403_Platform (the H1 regression), _NonAdmin_403,
_SuperAdmin_RollingRestart, _RequiresExplicitEnv, _SuperAdmin_EnvSelectsNamespace;
CLI TestDeployRequiresEnv. All green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 17:14:42 -07:00
hanzo-dev 263473c2cb fix(deps): bump ai v1.825.2 -> v1.826.0 (enso per-tier gate enforcement + real router-stats model ids) 2026-07-18 17:07:21 -07:00
hanzo-dev 169beabdc2 fix(billing): enforce enso per-tier gate — inject co-resident commerce tier into ai
The embedded ai per-tier SKU gate (family_tier.go) was fail-open in-cluster: it
resolved the caller's tier with an authed HTTP self-call to the cloud edge, which
401/403s a service token on /v1/billing/*, so the gate saw "" and admitted every
tier — enso/enso-ultra were open to free callers.

Mirror wireFinance's SetBalanceReader: install aiobject.SetTierReader so ai reads
the subscription tier DIRECTLY over the co-resident commerce client the metering
gate already bills over (commerceinproc in-process, with the service token commerce
itself accepts) — never the cloud edge. Add metering.Client.Tier to decode tier.name
from GET /v1/billing/tier. Fail-safe preserved: a commerce error or unknown tier
folds to "" (allow), so a commerce blip never locks out a paying caller.

Bumps ai v1.824.2 -> v1.825.2 (the object.TierReader seam).
2026-07-18 16:54:47 -07:00
hanzo-dev 932e1f6f32 feat(visor): fold DOKS worker nodes into the fleet — 3rd machine source
managedMachines unioned Visor's registry (/v1/get-machines) + live droplet list
(/v1/machines); a DOKS cluster's worker NODES appeared in neither (their droplet
carries a k8s tag, not a hanzo-org droplet tag), so world.hanzo.ai showed
standalone droplets but never cluster nodes.

Add GET /v1/kubernetes-nodes as the THIRD source (Visor unions the house-account
hanzo-org-tagged clusters + BYOC Provider.ClusterID clusters and returns each
worker node as a Machine keyed by droplet id). It is processed after registry and
live, so a DOKS node whose droplet is ALSO in the live list dedupes by droplet id
and never lists twice; a cluster-only node surfaces. Independently resilient like
the other two — a kubernetes-nodes outage is logged and skipped, never hiding the
registry/live/BYO sources.

Test: TestMachinesMergeDOKSNodes — a DOKS-only node appears, and a node whose
droplet is already live collapses BY ID (the node row carries a different name, so
only id-dedup can merge it). Full clients/visor suite green (31 subtests).

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 16:52:27 -07:00
zeekayandClaude Opus 4.8 87e578c6db docs(llm): document the unified hanzo CLI ↔ /v1/paas contract (apps/deploy/clusters off one IAM login)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 16:49:03 -07:00
hanzo-dev c86ddfbec9 merge(cloudflare): move asset routes under /v1/integrations/cloudflare — unified provider shape, never top-level
Assisted-by: Claude:claude-opus-4-8
2026-07-18 16:46:59 -07:00
zandGitHub 3b3eea5ede Merge pull request #334 from hanzoai/feat/event-canonical
analytics: canonical POST /v1/event + fail-closed key->org convergence
2026-07-18 16:46:45 -07:00
hanzo-dev 8dd682e160 analytics: canonical POST /v1/event front door (Event|[]Event), one write core
POST /v1/event is the ONE ingestion door: body is a single Event or a JSON-array
batch (no /v1/event/batch), org resolved IAM-only and fail-closed (eventTenant),
funneled through the ONE write core (ingestEvents) into hanzo.events. The
Segment/beacon (/v1/analytics,/v1/tracker) and PostHog (/v1/insights/e) wires
become thin DEPRECATED adapters over the same core. Org is never read from body.
2026-07-18 16:46:16 -07:00
hanzo-dev dd4b0f5c72 analytics: fail-closed project-key->org via the ONE IAM key seam (cloud.OrgForKey)
capture resolves a presented project/API key to its owner org through the single
IAM key resolver (sharedKeys, 60s cache incl. miss-cache). A presented-but-
unresolvable key is refused (403) and NEVER falls through to the brand-host
fallback, so a keyed request can never cross-tenant write. Anonymous marketing
traffic still resolves to the public brand org server-side from Host.
2026-07-18 16:46:16 -07:00
1565bb2657 chore(deps): bump hanzoai/ai v1.824.2 → v1.825.1 (Enso auto-serve + churn-resilient trainer) (#333)
Brings the merged Enso router fixes into the deployed cloud binary:
- #107 (v1.825.0): auto never routes to a family SKU it can't serve + forward the
  resolved model (withModel body rewrite) → model=auto serves 200 (was 404); grant-
  aware known predicate; flywheel boots from the single shared Bootstrap.
- #108 (v1.825.1): trainer fits EARLY (~90s after boot) then cadence → completed
  retrain cycles survive frequent redeploys (churn-resilient).
./apps (ai.Mount site) compiles clean against v1.825.1 (API-compatible).

Co-authored-by: zeekay <zeekay@hanzo.ai>
2026-07-18 16:44:32 -07:00
z 37bfe14380 billing: admit the verified S2S service token past the /v1/billing/* gate (cap authorize) + flag commerce
The metering cap-gate authorize (and the SuperAdmin cap-oversight Forward) call the
in-proc /v1/billing/spend-alerts/authorize with the COMMERCE_SERVICE_TOKEN, but the
customer /v1/billing/* bridge required a validated IAM principal -> "sign in to view
billing" (403) -> the cap fails-open and never enforces. billingData now admits a
trusted S2S caller carrying the verified service token: org from the EdgeAuth-controlled
X-Org-Id, query forwarded VERBATIM (a trusted caller names its own subject), no
subject-pin. A public caller can never present it (the gateway 401s a Bearer that is
not an IAM JWT / hk-|pk-|sk- key), and an unauthenticated caller still gets 403 (tested).
Adds spend-alerts/authorize to the GET allowlist; constant-time token compare.

ATOMIC with the flag: bumps commerce to v1.49.2 (SPEND_CAP_ENFORCE, default OFF), so
the instant the cap can reach the handler the enforcement gate is fail-open in the
binary -> auto-deploy stays safe until an operator flips the flag after the canary proof.

Security invariants tested: public/unauth -> 403; wrong bearer -> 403; verified token +
X-Org-Id -> forwards verbatim; token без X-Org-Id -> 403.
2026-07-18 16:40:05 -07:00
hanzo-dev f5948dccca refactor(cloudflare): move asset routes under /v1/integrations/cloudflare
The per-org Cloudflare asset plane (Pages/Workers/R2/KV/D1) is repointed from a
first-level /v1/cloudflare/* surface to /v1/integrations/cloudflare/*, so a
third-party provider is connected AND used under one unified namespace — matching
where the connector's connect/callback/verify/disconnect legs and the KMS token
coordinate already live. Pure path move: no auth, isolation, or handler logic
changes. Route registrations, doc comments, and tests repointed together.

No collision with the connector's parametric routes: the asset paths are all
3+ segments (/cloudflare/{pages,workers,r2,kv,d1}/...) while the connector's
/v1/integrations/:provider and /:provider/{connect,callback,disconnect,verify}
are 1- and 2-segment patterns whose literal second segment never equals an asset
group. Static-under-param co-registration is already proven in the integrations
plane (slack/link static beside :provider). Mount order unchanged: integrations
before cloudflare.

Assisted-by: Claude:claude-opus-4-8
2026-07-18 16:38:59 -07:00
hanzo-dev 65aa122ca9 merge(dns): /v1/dns forward head — path-guarded org-scoped proxy so console.hanzo.ai/dns loads zones
Red-cleared (double-encoding traversal closed, 9 tests green). Bearer-relayed, no standing cred.

Assisted-by: Claude:claude-opus-4-8
2026-07-18 16:28:42 -07:00
zeekayandClaude Opus 4.8 34643df667 fix(platform): rootless buildkit securityContext — match documented posture
The first rootless spec over-hardened (allowPrivilegeEscalation:false +
capabilities drop ALL), which breaks rootlesskit's setuid newuidmap/newgidmap
sub-uid mapping — proven by an on-cluster canary:
  newuidmap ... failed: operation not permitted
Relax to the documented moby/buildkit k8s rootless posture: privileged:false,
runAsUser/Group 1000, runAsNonRoot, seccomp+AppArmor Unconfined, and leave
allowPrivilegeEscalation / default caps at k8s defaults (newuidmap needs them).
Still user-namespaced, no host root — the decisive win over privileged=true.
Re-canaried: rootless build + scoped push-hanzoai cred pushed to ghcr OK.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 16:26:32 -07:00
zeekayandClaude Opus 4.8 9c56a93ffb feat(cli,paas): unify apps/deploy/clusters on the LIVE Go cloud — one IAM login, org-scoped
`hanzo apps list`, `hanzo deploy`, `hanzo clusters` targeted the OLD TS-Dokploy
contract (/v1/apps, /v1/org/{org}/cluster, /v1/org/.../redeploy) — all 404 on the
live Go cloud (ghcr.io/hanzoai/cloud). Repoint the CLI at the endpoints the Go
cloud actually serves, authorized off the SAME IAM login `hanzo build` now uses
(no --platform-token). Drift confirmed live as z@hanzo.ai:
  /v1/apps            → 404      /v1/paas/apps         → 403 (was SuperAdmin-only)
  /v1/org/*/cluster   → 404      /v1/clusters          → 200 (already org-scoped)
                                 /v1/platform/projects → 500 (co-resident IAM off)

CLI (cli/platform.go, cli/commands.go):
  apps list/get   → GET /v1/paas/apps[/{app}]  (no client org filter — the board is
                    confined to the caller's org SERVER-side by the validated identity)
  deploy <app>    → POST /v1/paas/apps/{app}/deploy  (rolling restart; --env selects
                    the lifecycle namespace; org from identity, not the path)
  clusters list/get → GET /v1/clusters  (Visor-managed + BYO; org from identity)
Removed the TS-contract vestiges with NO Go backend: `apps sync` (the board is
live-computed), `clusters create/select/install-baseline/target` and `k8s target`
(DOKS provisioning + deploy-target selection are not implemented on the Go cloud).
Reshaped the Cluster DTO to the live visor clusterView (dropped dead Phase/Active/
operator/baseline fields). Platform client doc corrected: it CAN validate IAM tokens.

Backend (clients/paas): authorize the fleet board off ONE IAM identity, exactly like
/v1/runner (clients/platform/runner.go). guard now admits a validated principal who
is a SuperAdmin OR an OrgAdmin (principal.Validated + IsSuperAdmin || IsOrgAdmin —
the ONE verifier, unforgeable off-gateway), and each handler CONFINES a non-super
caller to the platform namespaces its own validated org owns (scopedNamespaces, keyed
on principal.Org — never a client header): a SuperAdmin sees the whole fleet, an
OrgAdmin only its own org (empty board / clean 404 otherwise), so a tenant admin can
never observe — or restart — another org's, or a platform, app. `?org=` cannot widen
the view (confinement is at the namespace scan, before the filter).

deploy is now a real zero-downtime ROLLING RESTART (the kubectl-rollout-restart
mechanism: stamp the pod-template hanzo.ai/restartedAt annotation) instead of the
409-refuse. It never changes the declared TAG (that stays a git commit, the one thing
Hanzo CD's selfHeal reconciles), so there is no drift to revert — the honest,
GitOps-compatible "redeploy this app". resolveTarget split so the machine release
path (release.go) keeps the full scan; the identity-scoped deploy/getApp use the
caller's authorized namespaces.

SECURITY: this broadens auth on a control-plane surface (new mutating restart path).
Mirrors blue's IAM-admin pattern; flag for red. TDD: 12 new paas cases (role gate,
tenant confinement on list/get/deploy, forged-org cannot widen, rolling-restart lands
the annotation, foreign-org 404 + no mutation) + the CLI path/DTO tests, all green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 16:23:03 -07:00
zeekayandClaude Opus 4.8 caf0db43f7 fix(platform): close RED H1/H2/M1 on the /v1/runner build path
RED re-review of the unify-infra PaaS-auth flip found 2 HIGH + 1 MED on the
privileged build endpoint. Fixes:

H1 (borders CRITICAL) — cross-org supply-chain push. imageAllowed() permitted
ghcr.io/{hanzoai,luxfi,zooai}/ regardless of the caller's validated org, so any
org-admin could overwrite another brand's prod image via the shared push cred.
Bind the image's registry-org to the caller's org (orgRegistryNamespaces map);
only a real SuperAdmin may cross. The machine (fabric) token keeps full
owned-registry latitude. Cross-org now 403; same-org 202.

H2 — privileged rootful buildkit in the main platform namespace with the shared
3-org push cred. buildJobSpec is now ROOTLESS (moby/buildkit:*-rootless, uid 1000,
no privileged, no privilege-escalation, caps dropped, --oci-worker-no-process-
sandbox), runs in a DEDICATED isolated namespace (CLOUD_PLATFORM_BUILD_NS default
→ hanzo-build, off the platform ns), and mounts ONLY the target org's push
credential (push-<namespace>), never the shared kaniko-ghcr. Node-pool taint +
automountServiceAccountToken:false retained.

M1 — `image` bypassed validateBuildInputs → buildkit --output attribute
injection (ghcr.io/hanzoai/x,registry.insecure=true). Added validateImageRef
(strict single OCI ref, rejects comma/space/quote/newline/'='), folded into
validateBuildInputs and enforced early at the handler.

I2 — the identity validator now fail-closes on an empty resolved issuer OR
audience set instead of silently disabling that axis; issuerAllowed denies on
an empty trusted set.

TDD: cross-org 403 + same-org 202 + SuperAdmin cross-org + orgless-403 +
image-injection-reject + rootless/scoped-cred spec + empty-trust-set-deny all
green (pure-Go, as prod ships).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 16:16:54 -07:00
hanzo-dev 75cdae8172 fix(dns): close double-encoded traversal in the /v1/dns path guard
The prior prefix guard checked fasthttp's URI().Path(), which decodes only ONE
layer of percent-encoding. A DOUBLE-encoded traversal survives that one decode as
a literal %2e/%2f that still KEEPS the /v1/dns/ prefix -- so the prefix check
passes, cloud forwards base + /v1/dns/%2e%2e/admin, and the upstream decodes the
second layer to /v1/admin. Proven bypasses: /v1/dns/%252e%252e/admin,
/v1/dns/%252e%252e%252fadmin, /v1/dns/..%252fadmin.

After the prefix check, also refuse any once-decoded path that still carries a `%`
(a still-encoded byte => the client double-encoded) or `..` (residual traversal).
Neither appears in a legitimate DNS-API path -- zone labels are DNS names /
punycode xn--, and the query string (checked separately) is unaffected. Fail
closed 400 before a byte leaves cloud.

Also relay the upstream Location header so a 3xx -- never followed, per
CheckRedirect -- passes back verbatim (status + Location) as the comment claims,
rather than being silently dropped.

Regression: the escaped-path test gains the 3 double-encoded vectors (each refused
400 with 0 upstream bytes), plus a redirect test proving an upstream 302 is not
followed and its Location relays verbatim. 9 tests green.

Assisted-by: Claude:claude-opus-4-8
2026-07-18 15:40:42 -07:00
hanzo-dev 945441e402 merge(cloud): per-org /v1/cloudflare asset plane
Adds the clients/cloudflare subsystem — Pages+Workers wired, R2/KV/D1 stubbed —
gated by the org-comingling guardrail and org-admin mutation check; wired into
apps/apps.go. Red-reviewed SHIP: comingling guardrail + org-admin mutation gate
verified PASS, 12/12 tests green, isolation core intact.

Assisted-by: Claude:claude-opus-4-8
2026-07-18 15:29:47 -07:00
hanzo-dev 4042dcc7d2 fix(dns): lock the /v1/dns forward head to its own prefix; don't follow upstream 3xx
The forward head built the upstream target from uri.Path(), which is NORMALIZED
and percent-decoded. Fiber matches the /v1/dns/* wildcard on the RAW path, so a
dot-segment or encoded-dot traversal (/v1/dns/../../admin/secrets,
/v1/dns/..%2f..%2fadmin, /v1/dns/../../../metrics) still routed to the handler
while the normalized path escaped the prefix -- letting the caller drive the
WHOLE path on the DNS host. Contained today only because the upstream 404s
unknown paths; a latent path-scope escape the moment :8443 serves anything else.

Guard the normalized path fail-closed BEFORE building the target: require it to
be exactly /v1/dns or under /v1/dns/, else 400 and forward nothing. Because the
path is already normalized, every traversal/encoded-dot escape fails this check.
Correct the comment that wrongly claimed the path was locked by the route match
(the host-pinning claim was, and stays, true).

Also stop the shared http.Client from following upstream 3xx (CheckRedirect =>
http.ErrUseLastResponse) so redirect responses pass through verbatim and a 3xx
can never silently re-target the request onto another host or path.

Regression test proves fail-closed: each escaped path is refused (400) and 0
bytes reach the upstream. Existing 7 tests stay green (8 total).

Assisted-by: Claude:claude-opus-4-8
2026-07-18 15:29:39 -07:00
hanzo-dev 4e648305dc fix(cloudflare): red fixes — comingling guardrail, org-admin mutation gate, stored-account resolution
Addresses Red's FIX-THEN-SHIP findings:
- HIGH: stamp X-Hanzo-Org (the served org) on every /v1/cloudflare response so a
  per-org caller can detect a pinned/comingled org. The platform Pages client asserts
  it equals the requested org and fails LOUD if a non-org-switch-capable service token
  made the identity boundary pin X-Org-Id to the token's own owner — no silent
  cross-tenant read/write.
- MEDIUM: gate mutations (POST/PUT/DELETE) on principal.IsOrgAdmin via a new authWrite
  front door (reads stay validated-org-only), parity with the AdminOnly connector. A
  non-admin is refused before any KMS token read and never reaches Cloudflare.
- LOW: resolveAccount now prefers the account captured at connect time
  (integrations.ConnectionFor ExternalID), falling back to live /accounts discovery
  only when none is stored — no per-call round-trip, deterministic for multi-account
  tokens.

Tests: +TestResponseStampsActingOrg, +TestMutationRequiresOrgAdmin,
+TestStoredAccountSkipsDiscovery; existing mutation tests drive as org admin. 12/12
pass, -race clean.

Assisted-by: Claude:claude-opus-4-8
2026-07-18 15:28:34 -07:00
hanzo-dev 671c08f57a feat(cloudflare): per-org /v1/cloudflare asset plane (Pages+Workers wired, R2/KV/D1 stubbed)
New cloud subsystem clients/cloudflare exposing /v1/cloudflare/{pages,workers,r2,kv,d1}/*,
sibling to hanzodns's /v1/dns. It reads each org's KMS-sealed Cloudflare token in-process
through the integrations custody seam (integrations.TokenFor) and proxies to the Cloudflare
API v4 with the cfDo shape reused verbatim from hanzodns — no global env token, no
bearer-relay hop (that is only hanzodns's separate-process need).

Tenant isolation: org is derived ONLY from the validated principal (principal.Org); the KMS
token path is keyed on that org, so cross-org token reach is structurally impossible and an
unvalidated request fails closed (403). Pages (project CRUD, deploy, custom-domain add/delete)
and Workers (script put/list/delete via multipart module upload, workers.dev subdomain, zone
route bind/list/delete) are wired; R2/KV/D1 ship typed provider methods + routes that answer
an honest 501 (never a fake success).

Appends the Workers connector scopes (Account:Workers Scripts:Edit, Zone:Workers Routes:Edit)
for the now-callable capabilities and wires the subsystem into apps.Wire after integrations.

Assisted-by: Claude:claude-opus-4-8
2026-07-18 15:28:34 -07:00
hanzo-dev 5ffb30f9dd deploy: clean API paths — /v1/deploy/<resource>, no /api/ prefix, no inner /v1
House rule: no extraneous /api/, just /v1/. The projection API moves from
/v1/deploy/api/v1/* → /v1/deploy/<resource> (settings, session/userinfo, version,
account/can-i, applications, applications/{name}/resource-tree, .../{sync,rollback}).
This IS the deploy API now — the superseded native /v1/deploy/{applications,:name/*}
routes are removed (their readers stay, reused by the projection). health +
reconcile unchanged. Guard test updated.
2026-07-18 15:05:11 -07:00
hanzo-dev ede3887880 feat(dns): forward /v1/dns/* to the DNS control plane under the caller's own bearer
console.hanzo.ai serves the DnsModule but cloud held no /v1/dns head, so
console.hanzo.ai/v1/dns/* 404'd and the dashboard showed empty zones. Add a thin
forward head (clients/dns) that relays each /v1/dns/* request to the DNS control
plane (HANZO_DNS_URL, default the in-cluster coredns-hanzodns service), preserving
verb, path, query, body, status codes and error bodies.

Isolation is bearer-relay: the head forwards the caller's OWN validated bearer
(cloud.CallerBearer) plus the server-validated X-Org-Id, substituting NO service
credential, so the DNS plane's own per-org authorization still holds and a caller
in org A can reach only org A's zones. Fail-closed: no validated principal => 403,
before any byte leaves cloud. It builds a fresh upstream request, so no inbound
header is blindly relayed; the upstream host comes only from env (no SSRF).

Decomplect the token resolution the identity boundary and this relay both need
into one callerToken helper (validatedPrincipal now delegates to it) and expose
CallerBearer for the relay; an opaque API key is never relayed as a bearer.

Assisted-by: Claude:claude-opus-4-8
2026-07-18 14:56:20 -07:00
hanzo-dev 4a1b33cff5 merge(cloud): /v1/deploy is API-only — the FE moved to the hanzoai/spa cd-ui App
The monochrome dashboard now serves at cd.hanzo.ai/ (root, base-href /) from the
cd-ui App CR (hanzoai/spa); cloud keeps ONLY the IAM-gated projection API at
/v1/deploy/api/*. Drops the go:embed FE + the deploy-ui-embed Dockerfile stage.
The FE is no longer /v1/-prefixed and no longer baked into the money binary.
2026-07-18 14:26:49 -07:00
hanzo-dev 4eb1e74ebb deploy: drop the FE from the money binary — /v1/deploy is API-only
The monochrome dashboard SPA now ships as the hanzoai/spa-based cd-ui App CR
served at cd.hanzo.ai/ (base-href /); cloud keeps ONLY the IAM-gated projection
API, moved from /v1/deploy/ui/api/* to /v1/deploy/api/* (same-origin with the
SPA). Removes the go:embed dashFS + static serve (dashStatic/serveDashIndex) +
webui/dist + the deploy-ui-embed Dockerfile COPY stage. Guard test updated to the
/v1/deploy/api/* routes (still 403 without SuperAdmin). RED invariant holds — the
API terminates in cloud behind IdentityMiddleware + the guard.
2026-07-18 14:15:07 -07:00
zeekayandClaude Opus 4.8 7d3df8ac2a feat(cli): hanzo build owner/name shorthand → GitHub https URL
The platform build muscle (launchDirectBuild) clones an https git URL; the CLI
sent the bare `owner/name` positional verbatim, so `hanzo build luxfi/wallet`
failed server-side with "repo.url must use https". normalizeRepoURL expands a
bare owner/name to https://github.com/owner/name (the host for every
hanzoai/luxfi/zooai repo) and passes an explicit URL / scp-style remote through
untouched. Test: TestNormalizeRepoURL. Completes the IAM-login dogfood:
`hanzo build luxfi/wallet ... --sha <full> ...` now 202s + launches the build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 13:51:21 -07:00
zeekayandClaude Opus 4.8 6f75859b3c sites: org-scope published host + CI guard against zen streaming regression
Publish routing is now org-scoped: a project publishes to
<slug>.<org>.<apex> (e.g. myapp.maxpower.hanzo.app) instead of the flat
global <slug>.<apex>. The slug namespace becomes per-org — two orgs can
own the same slug and their sites can never collide or shadow one another.

- deploy.go: siteHost(org,slug)=<slug>.<org> is the ONE bound/resolved key;
  onPublish binds it; siteURL renders https://<slug>.<org>.<apex>.
- sites.go siteSlug: accept the two-label host <slug>.<org>.<apex>, validate
  both labels (slug non-reserved), return <slug>.<org> as the resolve key so
  bind and resolve agree. Org isolation is now STRUCTURAL in the hostname.
- store unchanged: site_hosts already keys on arbitrary full host strings
  (custom-domain path proves exact full-host ResolveHost match).

containment.yml: add a required "zen streaming-fix floor" check that fails
any PR/push whose effective github.com/hanzoai/zen is below v1.4.1 (the
first release carrying the SSE body-close fix, commit 50328b8). A stale
branch that reverts go.mod's zen pin to v1.4.0 can no longer silently
re-break streaming (empty SSE completions) — the durable root-cause guard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 13:49:05 -07:00
zeekayandClaude Opus 4.8 bd7bce6b39 feat(cli,platform): unify PaaS auth on IAM — one login authorizes build/deploy/apps
A plain `hanzo login` (IAM) now authorizes every PaaS control-plane op with no
separate --build-token / --platform-token. ONE identity, org+role scoped.

CLI (cli/cli.go): Env.buildToken() and Env.platformToken() fall back to the IAM
access token as the FINAL resort (precedence unchanged above it: flag > env >
credential-store service token > IAM login). So after `hanzo login` the CLI sends
the IAM JWT as the platform bearer. "No token" errors now point at `hanzo login`
and fire only when there is ALSO no IAM login. Tests: added
TestBuildTokenFallsBackToIAM / TestPlatformTokenFallsBackToIAM (precedence
preserved — a dedicated token still wins).

Platform (clients/platform/runner.go): /v1/runner (build-enqueue) — the one
control-plane endpoint that ignored identity — now accepts EITHER the shared
build-callback token (machine path: git-push, self-release, operator; constant-time,
unchanged) OR a validated IAM principal who is an admin (principal.IsSuperAdmin ||
principal.IsOrgAdmin over principal.Validated). Both bounded by the SAME
owned-registry allowlist, so identity never widens the image boundary. Release
self-publish stays machine-token-only. IAM builds are org-attributed to the caller's
VALIDATED org and refuse a foreign organizationId unless SuperAdmin. Reuses the ONE
identity verifier (SanitizeIdentity mints unforgeable X-User-* from the verified JWT)
— no parallel JWT crypto. deploy/apps were already IAM-authorized via the tenant()
boundary. Tests: 7 new IAM cases (admin launches, non-admin 403, forged-no-user 403,
disallowed-image 403, foreign-org 403, release 403) + corrected fail-closed 403.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 13:22:37 -07:00
zeekayandClaude Opus 4.8 faefd7d4d3 fix(cloud): re-pin zen v1.4.2 — restore the SSE stream body-close fix
The argo/gitops merge d3f60be (v1.801.77) resolved the go.mod conflict to its
stale second parent (zen v1.4.0), silently reverting the v1.4.1 bump landed at
v1.801.76. v1.4.0 still carries the serve() `defer resp.Body.Close()` race that
empties every streamed completion (200 with a 0-byte body) — which broke every
hanzo.app builder stream. Re-pin to zen v1.4.2 (the body-close fix + reasoning_content
passthrough regression pin) and thinking v0.1.1 (reasoning default). Streaming
now forwards every chunk, including delta.reasoning_content.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 12:55:21 -07:00
hanzo-dev 7e0fc65a0a merge(cloud): embed the monochrome ArgoCD UI (ghcr.io/hanzoai/deploy-ui-embed) into /v1/deploy/ui
The deploy-ui-embed image is published; cloud's Dockerfile now COPYs its /dist
into clients/deploy/webui/dist (go:embed). release.yml builds a cloud image that
serves the REAL monochrome dashboard at /v1/deploy/ui instead of the fallback.
2026-07-18 12:46:52 -07:00
hanzo-dev bc5bdcd4c0 cloud: embed the monochrome ArgoCD UI bundle (ghcr.io/hanzoai/deploy-ui-embed) into clients/deploy/webui/dist
Mirrors the console-embed stage: FROM the prebuilt deploy-ui-embed image, COPY
/dist -> clients/deploy/webui/dist (go:embed source for /v1/deploy/ui). GATE: do
NOT merge until ghcr.io/hanzoai/deploy-ui-embed:latest is published (else the
build cannot pull the base). Until merged, the money binary serves the fallback.
2026-07-18 12:43:17 -07:00
hanzo-dev 18eb2edfb7 merge(cloud): deploy dashboard RED fast-follows (guard test + health hardening) 2026-07-18 12:40:51 -07:00
hanzo-dev 0b30320f79 deploy: RED fast-follows — guard table-test (LOW-2) + drop raw error from public health (INFO-1)
LOW-2: TestDeployRoutesRequireAdmin asserts every /v1/deploy(/ui) route 403s
without X-User-IsAdmin and passes with it (health stays public) — guards against
a future unguarded-route refactor.
INFO-1: the unauthenticated /v1/deploy/health path reports booleans only; the raw
k8s error (apiserver/RBAC detail) is logged server-side, not returned.
LOW-1 (CSRF): verified no-op — the IAM session cookie is SameSite=Lax AND the
ambient cookie->JWT bridge is same-origin-gated (sessionBridgeSameOrigin), so a
cross-origin CSRF POST gets no identity and the deploy guard 403s.
2026-07-18 12:40:50 -07:00
hanzo-dev 5dac9c3f34 merge(cloud): ArgoCD monochrome dashboard via App-CR projection at /v1/deploy/ui
Serves the full ArgoCD React UI fed a read-projection of operator App CRs shaped
as v1alpha1 Applications — no argocd api-server/repo-server/redis/stored CRD.
SuperAdmin-gated; IAM owns identity at the edge. Money binary builds green;
projection render tests pass. Real monochrome bundle is a CI artifact (make
deploy-ui / deploy-ui-embed image); fallback shell until that lands.
2026-07-18 12:21:56 -07:00
zandGitHub 655e491414 docs(llm): document /v1/deploy GitOps plane (quality pass) 2026-07-18 12:20:20 -07:00
hanzo-dev 004c6101f4 docs(llm): document the /v1/deploy GitOps plane + embedded gitops-engine 2026-07-18 12:20:06 -07:00
hanzo-dev e922033543 deploy: make deploy-ui builds the monochrome bundle into webui/dist (gitignored)
CI story for the dashboard bundle, mirroring make webui: DEPLOY_DIR=<hanzoai/deploy
rebrand/hanzo-monochrome> yarn build -> clients/deploy/webui/dist (go:embed). Only
the fallback index.html + .gitignore are tracked; the real 43MB bundle is
build-time-only. Money binary builds green with the real bundle embedded.
2026-07-18 12:19:40 -07:00
hanzo-dev af93855841 deploy: ArgoCD monochrome dashboard via App-CR projection at /v1/deploy/ui
Serves the full ArgoCD React UI fed a READ-PROJECTION of operator App CRs shaped
as v1alpha1 Applications — NO argocd api-server, NO repo-server, NO redis, NO
stored Application/AppProject CRD. projection.go maps App CR -> Application +
resource-tree (reusing the native readers/engine health). dashboard.go
reimplements the UI's api-server subset (settings/userinfo/version/can-i +
applications list/get/resource-tree + sync/rollback->App-CR reconcile) + serves
the go:embed'd monochrome bundle with base-href rewrite. SuperAdmin-gated; argocd
auth disabled (IAM owns identity at the edge). Projection render tests green.
UI bundle is a CI artifact (committed fallback shell; make deploy-ui overwrites).
2026-07-18 12:19:40 -07:00
hanzo-dev 5ec96533e5 chore(cloud): vendor hanzoai/ai v1.824.2 — real model names for super-admin platform view
v1.824.2 unmasks the router-stats model ids (arm-N → real names like zen5-coder /
opus-4.8) for the PLATFORM scope when the caller is a super-admin of the own brand;
every other caller keeps the arm-N privacy masking. So the world.hanzo.ai admin view
(Routing Throughput / Enso arms) shows actual models instead of "Enso arm 2".

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 12:15:04 -07:00
hanzo-dev 051df1a96b fix(cloud/visor): fleet surfaces DO droplets via registry+live-DO union
world's admin fleet (GET /v1/machines -> listMachines) sourced machines only
from Visor's registry (/v1/get-machines), so DigitalOcean droplets that were
provisioned but not (yet) in the registry never appeared -- DO nodes were
entirely missing from the fleet.

Source the managed-machine set as the deduped UNION of the registry AND
Visor's LIVE DO reseller list (GET /v1/machines -> ListComputeMachines ->
service.ListOrgMachines, the live Droplets.ListByTag(orgTag)). Dedup is by
provider id OR name; the registry entry wins a collision so its enrichment/
masking is preserved. One helper (managedMachines) now feeds listMachines,
listGPUs and the /v1/fleet board so all three agree on which machines exist,
not just how they normalize. BYO fold unchanged; only machines Visor actually
returns are surfaced (nothing fabricated).

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 12:13:48 -07:00
z 7add862ca6 billing: metering client honors METERING_TEST (safe test-mode canary/staging)
buildMeteringClient ignored the documented METERING_TEST env, so the metering client
was ALWAYS live (c.test=false) — a staging/canary could not route debits to the
sandbox books, and the usage-cap smoke would have moved real money. Now METERING_TEST=true
sets Config.Test, so fin.RecordUsage writes the TEST finance books and the cap read
(org.TestMode via SQUARE_ENVIRONMENT=sandbox) sees the SAME test books. Unset in prod
= live, unchanged.
2026-07-18 12:13:31 -07:00
z 684e447943 cap: enforce + alert on the FINANCE ledger (where the unified binary records usage)
The spend cap read commerce's transaction store, which the co-resident cloud binary
leaves EMPTY (usage is recorded via fin.RecordUsage on the finance ledger) — so in
prod the cap summed 0 and never enforced, and the alert never fired. This wires the
cap onto the ledger prod actually writes, ORG-WIDE (the finance Entry carries no
scope; per-scope is a follow-up):

- sqlstore.SumByKindSince: additive read-only aggregate (kind + created_at index,
  18-decimal TEXT folded in Go) — no Entry schema change.
- finance.SumUsageSince: the org's metered usage (cents) since a cutoff, deposits
  excluded, sandbox books for a test org.
- finance.SetUsageHook: dependency-inverted post-debit seam (finance never imports
  commerce) the cap alert fires through.
- apps/commerce.go: SetPeriodSpendReader(financePeriodSpend) so AuthorizeSpendCap
  reads finance spend since the UTC month start, and SetUsageHook(fireCapAlert) so a
  finance debit fires the org's spend-alerts on the same crossing.

Composes the commerce policy/CRUD/promo/admin/ancestor-fix (commerce
v1.49.1->v1.49.2 injection seam) — a targeted host re-wire, not a redo.
2026-07-18 12:07:10 -07:00
hanzo-dev 4e0f4c87ca deps: commerce v1.49.0->v1.49.1 — real subscription tier derivation
Completes the Enso per-tier gate: ai v1.824.1 already enforces min_tier at the
family pipe + auto-router; this bumps the co-resident commerce so /v1/billing/tier
returns the caller's REAL plan (was stubbed always-Free). Fail-open on uncertainty.
2026-07-18 11:43:21 -07:00
zandGitHub d3f60be7e6 merge(cloud): embed argo gitops-engine under /v1/deploy — reconcile + RED HIGH-1 prune fuse (inert: DEPLOY_ENGINE_ENABLED off) 2026-07-18 11:27:44 -07:00
hanzo-dev 4f9c09aa30 chore(deploy): go mod tidy after rebase onto main (union: main deps + gitops-engine v0.7.2 + k8s 0.35.3 staging) 2026-07-18 11:27:25 -07:00
hanzo-dev 3e34b7bd54 chore(cloud): vendor hanzoai/ai v1.824.1 — Enso flywheel boots from Mount
v1.824.1 boots StartRouterTrainer + StartRouterProbe from ai.Mount, so the flywheel
runs in the deployed (embedded-in-cloud) service, not just the standalone aid binary.
Both still self-gate on their env flags; universe sets ROUTER_TRAIN_ENABLED=1 to turn
training on. Also carries the retrain-timeline fix (retrains now count).

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 11:27:15 -07:00
hanzo-dev 3f279cbb19 deploy: prune-safety fuse (RED HIGH-1) on the engine reconcile
Five guards before any deletion: (i) refuse an empty desired set; (ii) dry-run
sizes the prune set + a count/ratio fuse (DEPLOY_ENGINE_PRUNE_MAX default 10,
_RATIO default 0.20) refuses a mass prune; (iii) WithPruneConfirmed gates prune
on the fuse passing; (iv) PVC + KMSSecret are excluded from prune entirely (data
anchors, irreversible); (v) parseManifestDir walks recursively so a nested
manifest is never silently dropped (which prune would read as a deletion).
prune stays off by default (DEPLOY_ENGINE_PRUNE).
2026-07-18 11:26:02 -07:00
hanzo-dev 54b81309be deploy: pin gitops-engine to hanzoai/deploy/gitops-engine v0.7.2 (no replace)
Drops the filesystem replace => ../deploy/gitops-engine. The fork's engine module
was renamed to its real repo path (github.com/hanzoai/deploy/gitops-engine, tag
gitops-engine/v0.7.2) so cloud requires it as a normal pinned version — CI builds
the money binary with NO sibling checkout, NO argoproj alias. tidy + scoped build
green over SSH.
2026-07-18 11:26:02 -07:00
hanzo-dev 268e79369e deploy: embed argo gitops-engine in-process under /v1/deploy (reconcile half) 2026-07-18 11:26:02 -07:00
hanzo-dev f977650c30 ci(release): auto-promote the proven tag into universe crs/cloud.yaml
Every merge to main builds + smoke-tests + tags a proven image, but nothing
recorded that tag as the desired state Hanzo CD deploys, so api.hanzo.ai sat on
a stale pin (v1.801.71) while proven images (…72-…75) never rolled. The old
image-update.yml deploy hub was deleted in the Hanzo CD cutover; a direct CR
patch is reverted by ArgoCD selfHeal.

Add a promote job that, after the tag receipt, bumps spec.image.tag in
hanzoai/universe crs/cloud.yaml and commits deploy(cloud): <tag> — the SAME
yq-bump the hanzoai/ci reusable does for every other service. The universe-crs
ArgoCD Application (automated sync + selfHeal) then reconciles it to the cluster.
No hand-dispatch, no hand-edit.
2026-07-18 11:02:47 -07:00
bb021e49a7 fix(cloud): zen v1.4.1 (SSE body-close race) + thinking v0.1.1 (reasoning default)
Fixes empty streaming completions for zen5* (hanzo.app builder P0):
- zen v1.4.1 stops serve() from closing the upstream body before fiber's lazy
  SendStreamWriter drains it (every SSE completion was truncated to a 200 + empty body).
- thinking v0.1.1 makes glm-5.2/deepseek Off send reasoning_effort:none, so zen5-coder
  streams the answer immediately instead of a long silent content:null reasoning preamble.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 10:49:51 -07:00
z bca49760b1 admin+metering: SuperAdmin usage-cap + promo control plane; fix net/http spend_cap→402
Adds the /v1/admin control plane admin.hanzo.ai drives, twinning /v1/admin/flags:
  - /v1/admin/promos (GET/PUT, core.Guard SuperAdmin) → commerce /v1/platform/promo:
    configure the admin-controlled plan promo (percentOff/start/end/plans/active).
  - /v1/admin/spend-caps (GET/POST/PATCH/DELETE, core.GuardScoped) → commerce
    /v1/billing/spend-alerts with X-Org-Id: oversee/override ANY org’s usage caps
    (SuperAdmin via ?org=; a scoped admin hard-pinned to their own — the escalation
    line). Reuses the customer’s OWN spend-alert rows, no parallel model.
commerce.Forward is the ONE service-token seam these ride, relaying commerce’s own
status so a 400/403/404 surfaces honestly instead of masking as success.

Fixes clients/metering/middleware.go defaultOnDenied: a FUNDED caller over a
per-scope spend cap now maps to a DISTINCT 402 spend_cap_exceeded (errors.Is), not
the 503 it fell through to — parity with the zip-native denyVerdict, so any product
on the net/http middleware surfaces the same honest verdict.
2026-07-18 09:56:58 -07:00
hanzo-dev f470bcab93 build(deps): bump embedded luxfi/kms v1.11.8 -> v1.12.4
Brings the ACTIVE Hanzo KMS custody plane (api.hanzo.ai/v1/kms/*, served by the
cloud-embedded luxfi/kms per HIP-0106) to the latest v1.x. keys (v1.4.1) and
crypto (v1.20.2) already latest; luxfi/mpc stays out of the graph (threshold
signing is a wire-coupled external daemon, not a linked module). v1.12.4 verifies
against the public sumdb; clients/kms + cmd/cloud build green.
2026-07-18 09:52:56 -07:00
z d31cd1cde6 fix(cloud/fleet): never surface a GPU slug's VRAM as system RAM
toMachineView's memGB fallback read the integer before "gb" out of any
size slug. On a DO GPU droplet the slug's gb is VRAM (gpu-h100x8-640gb ->
640 GB VRAM), not system RAM, so a GPU node missing its upstream memSize
would render 640 GB of "system memory" -- a misleading number.

Guard the fallback with the GPU check already needed for v.GPU: reuse the
single gpuSpecOf(slug) call (spec, isGpu) and apply the slug's gb figure
only when !isGpu. Real m.MemSize still takes precedence for every provider,
GPU nodes included, so a GPU machine that reports its true RAM is
unaffected -- only the VRAM-as-RAM fallback is suppressed.

Table tests: a gpu-h100x8-640gb slug with empty MemSize yields Mem=="" (not
"640 GB") while still resolving GPU=="H100", and the same slug with a real
MemSize=="1920gb" reports "1920 GB".

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 09:43:47 -07:00
z 2527b4957e fix(cloud/fleet): map system memory + parse DO size-slug vCPU/RAM
The fleet view (world.hanzo.ai cloud variant) renders machines from
cloud's /v1/machines -> listMachines -> toMachineView. Two honest-data
gaps left system RAM and DigitalOcean vCPU counts blank:

1. System memory was never surfaced: machineView had no memory field and
   toMachineView never read the upstream memSize, so every provider's RAM
   column rendered empty.
2. DO vCPU was dropped: toMachineView filled vcpu only when CpuSize parsed
   as a bare integer, but DigitalOcean reports size SLUGS (s-4vcpu-8gb),
   so strconv.Atoi failed and vCPU showed nothing.

Fix:
- Add MemSize to visorMachine (upstream already sends it; it was simply
  unmapped) and Mem to machineView.
- parseSizeSlug pulls the integer before "vcpu" and the integer before
  "gb" out of a size slug (s-4vcpu-8gb -> 4,8; g-8vcpu-32gb -> 8,32).
- normalizeMem renders "N GB" only for trustworthy inputs (explicit
  gb/gib, explicit mb converted with rounding, a bare integer as MB when
  >=1024 else GB) and returns "" for anything ambiguous -- never a
  fabricated number.
- toMachineView keeps the Atoi(CpuSize) path and falls back to the slug's
  vcpu; sets Mem from normalizeMem(MemSize) and falls back to the slug's
  gb figure. The GPU-spec logic is unchanged.

Table tests cover the slug parser, the mem rounding, and the mapper
precedence (explicit values win, honest omission when neither yields one).

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 09:37:45 -07:00
zandGitHub 22a268cd29 Merge feat/code-normalized-continue into main 2026-07-18 09:30:51 -07:00
antje 6012f52b6a gpu worker: collect SaveGLB outputs — 3D meshes travel back to the library
collectOutputs read only the 'images' key, but SaveGLB publishes under '3d', so a
generated .glb never mirrored to the org library. Gather every saver's outputs
(images + 3d), so the studio 3D lane's mesh lands like an image or video render.
2026-07-18 09:27:56 -07:00
hanzo-dev cf663ec015 feat(code): normalize continue across harnesses 2026-07-18 09:26:22 -07:00
hanzo-dev 5e75f25b45 integrations: Cloudflare OAuth connect path alongside apikey (same KMS coordinate)
The cloudflare provider now offers a browser OAuth path in addition to the
shipped apikey path. /connect dispatches by request: a "token" key in the body
seals via apikey (verify-before-store, unchanged); its absence starts the
Authorization Code flow (confidential client, client_secret, no PKCE — the
framework's OAuth pattern) and the exchanged access token is sealed to the SAME
KMS coordinate (/orgs/{org}/integrations/cloudflare/api_token), so the DNS
provider layer is auth-method-agnostic.

Framework: connect dispatch is now capability-based (Verify and/or Authorize)
rather than Kind-only; Mount validates RedirectPath for any OAuth-capable
provider; bodyHasCredential picks the path. The OAuth leg gates on its own app
creds (Creds().ClientID) so a missing Cloudflare OAuth app degrades to an honest
503 without breaking the always-available apikey path.

Requires a registered Cloudflare OAuth app: CLOUDFLARE_OAUTH_CLIENT_ID/SECRET in
env, redirect https://api.hanzo.ai/v1/integrations/cloudflare/callback.
2026-07-18 09:14:07 -07:00
zeekayandClaude Opus 4.8 ca9c4682dd rebuild(cloud): embed console-embed@sha-bd8a816 (casibase auth /v1/* fix + console per-project resources) + activate MCP builtin tool-plane (#292)
No Go change — this rebuild re-resolves the freshly-republished console-embed:latest
(console main bd8a81651) into the go:embed console served at console.hanzo.ai, and
ships builtin.go's auto '/v1 route → MCP tool' plane at /v1/tools/mcp (already in main,
newer than the deployed v1.801.69).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 08:57:15 -07:00
hanzo-dev 8eabff4d44 fix(kms): unshadow the bare secrets-list route (/secrets/+ not /secrets/*)
The value routes registered the optional-greedy wildcard `/secrets/*`, which
fiber also matches with an empty tail — so the bare `GET .../secrets` list path
was answered by getSecret (400 "secret name is required") and listSecrets was
unreachable. Switch the getSecret/deleteSecret value routes to the required-
greedy `+` (one-or-more), so `/secrets` falls through to the exact list route
while `/secrets/<path>/<name>` still reads/deletes. reqWildcard reads the `+`
param.

Regression test list_route_test.go asserts the bare list path returns 200 with
a secrets array (was 400) and that value reads still work.
2026-07-18 01:19:23 -07:00
hanzo-dev 6cde53fb05 integrations: Cloudflare apikey connector (verify-before-seal, org-admin gated)
Register Cloudflare as an apikey-kind connector on the /v1/integrations plane.
A customer-supplied scoped API token is verified live against Cloudflare's
GET /user/tokens/verify (must be status:active) before it is sealed into the
org's KMS namespace (/orgs/{org}/integrations/cloudflare/api_token); the
connection row holds only non-secret account metadata. connect/verify/disconnect
are org-admin gated from the validated principal (principal.IsOrgAdmin).

Extends the connector framework with the apikey credential seam shared by future
customer-credential providers: Provider.Kind/AdminOnly/Verify, VerifyInput,
connectByCredential (verify-before-seal, fail-closed), and the verify route.

Serves POST /v1/integrations/cloudflare/{connect,verify,disconnect} and
GET /v1/integrations.
2026-07-18 00:07:53 -07:00
z 9560169625 Merge feat/route-work-to-target: route coding run to chosen target machine 2026-07-17 23:32:54 -07:00
z 4ee67c1797 Merge feat/kms-reseal-migration: CR-driven KMS re-seal migration tool (#79) 2026-07-17 23:10:37 -07:00
2bb35ac291 auth: accept admin-console audience in the cloud JWT allowlist (#332)
The cloud already trusts hanzo-admin-guard (the admin surface) but not admin-console
(the admin console's own OIDC client), so a SuperAdmin token minted via admin-console
was rejected on /v1/admin with 'invalid audience' — forcing an awkward hanzo-admin-guard
detour. Add admin-console so the admin console's tokens work directly, matching
GATEWAY_ALLOWED_AUDIENCES which already lists it.

Co-authored-by: zeekay <z@hanzo.ai>
2026-07-17 23:03:19 -07:00
hanzo-dev 0782431509 docs: open cloud planes blueprint (HIP-0129)
Plane map with honest tiers in LLM.md: /v1/connectors custody (in flight), /v1/channels transport (planned, branch reserved), shipped planes named by package. Spec home HIP-0129; roadmap P1-P15 lives there.
2026-07-17 22:49:44 -07:00
zeekayandz e2929b82b0 gateway: nest clients/gatewaypolicy → clients/gateway/edge (kill the compound)
"gatewaypolicy" is a compound (gateway+policy) and read as a second gateway
package. It is the ONE Gateway concern with clients/gateway — the per-org edge
policy STORE (OrgRPM ceiling + CORS + cache) that the /v1/gateway/config plane and
the package-cloud edge middleware both read.

They are two packages only to break a Go import cycle: clients/gateway imports root
cloud (cloud.Deps), and middleware_edge.go IS package cloud — so the store must be a
LEAF both can import. A flat merge cycles. Fix per the no-compound law: nest the leaf
UNDER gateway as clients/gateway/edge (edge.Policy/Store/New). One gateway namespace;
/v1/gateway/config surface unchanged; edge-middleware logic unchanged.

NOT redundant with the external hanzoai/gateway (KrakenD): that does coarse per-route
edge rate-limit + auth at ingress; this is per-AUTHENTICATED-org RPM (needs the decoded
token org), an app-level ceiling the edge proxy cannot compute. Different layer.

Pure rename (49/49, no logic change); full cmd/cloud binary links; gateway + gateway/edge
tests pass; gofmt/vet clean.
2026-07-17 22:47:51 -07:00
d19f1d9066 flags: runtime flags resolve from /v1/flags only — drop redundant env gates (#331)
waitlist_* / public_signup / gateway_* are runtime flags; strip their Env: fallbacks so
they resolve from the /v1/flags DB engine → Default (single source of truth, flipped live,
no redeploy). Boot-time ReadOnly rows (subsystem_*, network_id_*) keep Env — that IS their
boot mechanism. Nothing read these env vars outside the flags engine (verified).

Co-authored-by: zeekay <z@hanzo.ai>
2026-07-17 22:46:51 -07:00
hanzo-dev 7241bc952e agents/integrations: reach routed dispatch from the Slack trigger
Wire the load-bearing trigger so a coding run can be dispatched to a chosen
linked machine end-to-end. Extend the Slack coding grammar with an optional
routing prefix — code: <repo> on <machine> <task> — resolve <machine> org-scoped
(id or friendly label) and set Req.TargetID. An unknown or foreign machine is an
honest error, never a silent local fallback; an untargeted request is byte-
unchanged (repo <task>).

- agents.ResolveTarget: the ONE org-scoped id-or-label resolver, fail-closed, so a
  trigger surface turns 'on evo' into a target id without leaking another tenant's
  inventory.
- slack_coding: parseCoding yields (repo, target, task); a routed run skips the KMS
  agent-credential fetch (the machine authenticates with its own credential); the
  ack + result card report a routed run as queued-on-<machine>, followed live in
  mission-control, not a premature branch-pushed verdict.

Tests: ResolveTarget id/label precedence + cross-org not-found + unmounted fail-
closed; parseCoding on-prefix grammar (routing only when 'on' is the token after
the repo; 'on-call'/'only' untouched); routed result card is queued not done.
2026-07-17 22:42:59 -07:00
z 8a50e0d335 Merge fix/consensus-bump: luxfi/consensus v1.36.9 (unbreak force-moved checksum) 2026-07-17 22:33:25 -07:00
zeekayandClaude Opus 4.8 1484745a01 build(deps): bump ai → v1.822.3 — drop the last WqyJh audio-fork edges
cloud transitively pulled github.com/WqyJh/{go-cosyvoice,go-openai-realtime}
through ai v1.822.2 (the go-openai-fork release, which predated ai's TTS
switch to the hanzo-owned forks). ai v1.822.3 wires ai/tts onto
github.com/hanzoai/go-cosyvoice + go-openai-realtime, so tidy drops both WqyJh
edges from cloud's graph. cloud now pulls ZERO third-party OpenAI-lineage:
WqyJh 0, sashabaranov 0, ClickHouse 0. Single datastore sql registrant
(hanzo-ds/go). Builds clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 22:29:42 -07:00
hanzo-blue 9dc3038051 docs(kmsreseal): runbook SCOPE = exact per-host dry-run counts (#79) 2026-07-17 22:26:48 -07:00
hanzo-blue a06b5fb1a2 docs(kmsreseal): operator LLM.md — subcommands, dry-run findings, gated cutover (#79)
4-host map, G1 delta (78-union), G5 boot-cycle verdict (safe: direct master key),
seeding wedge-risk prerequisite. Dry-run only; cutover CTO-gated.
2026-07-17 22:25:33 -07:00
z 675809dcdf feat(admin): fleet-aggregate billing endpoints (metrics + invoices + subscriptions)
admin.hanzo.ai's SaaS-metrics/Invoices/Subscriptions pages were placeholders
awaiting cross-org /v1/admin/* endpoints. Add them as super-admin (core.Guard)
fleet-aggregate readers over the existing commerce billing engine:

  GET /v1/admin/metrics        — SaaS god-view: MRR/ARR/net-new/churn/active-subs/
                                 paying-customers/plan-mix/top-customers/recent.
                                 Single S2S proxy — commerce /v1/metrics/saas is
                                 already a cross-org aggregate (same gate finance
                                 Costs uses: RequirePlatformAdmin→IsServiceToken).
  GET /v1/admin/invoices       — cross-org invoice list; fan core.ListOrgs out
  GET /v1/admin/subscriptions  — cross-org subscription list; per-tenant reads
                                 merged (identical to revenue.go's fan-out).

Honest degradation: a failed per-org read contributes no rows, never fabricated.
go build ./clients/admin/... green, gofmt clean. Pairs with admin operator UI
(feat/admin-billing-fleet-ui).
2026-07-17 22:23:47 -07:00
zeekayandClaude Opus 4.8 f43883d34e refactor(go-openai): import the hanzoai/go-openai fork directly, drop the replace
cloud/clients{,/agent} used sashabaranov/go-openai only via a 'replace =>
hanzoai/go-openai' — which does not propagate to cloud's own consumers, so
gateway/iam/etc. each had to copy it. Now the fork declares its own module
path (github.com/hanzoai/go-openai v1.41.0): require it directly. Bumps the
lockstep fork adopters — ai v1.822.2, agent v0.1.3 — so the hz.Mount Completer
boundary shares ONE openai type set (was a hanzoai-vs-sashabaranov type
mismatch). No replace anywhere; upstream sashabaranov remains only as an
indirect dep of the go-cosyvoice TTS chain (owned next). cmd/cloud keeps its
single datastore sql registrant (hanzo-ds/go). Builds clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 22:11:47 -07:00
zandGitHub 74fe777aeb Merge: cloud S3 client minio-go → hanzoai/s3-go (the one house S3 client)
7 files repoint minio-go v7 → hanzoai/s3-go (a byte-identical fork, package
minio, zero call-site churn); minio-go demoted to indirect. Red-verified:
presign + conditional-CAS byte-identical to the prior v7.0.100 dep, drop-in.
Deploy-gate: live SeaweedFS CAS smoke (If-Match/If-None-Match/412).
2026-07-17 22:05:03 -07:00
hanzo-blue 9963b3d055 feat(kmsreseal): CR-driven KMS re-seal migration tool (#79)
Embeds the fleet KMS into cloud by re-sealing the ~125 KMSSecret-referenced
secrets from the legacy standalone (unsealed at rest) into cloud's embedded
/v1/kms (AES-256-GCM sealed per secret). Driven by the KMSSecret CRs — the
authoritative (org,path,env,key) manifest — not a raw store copy.

- inventory: pure CR parse/validate/dedup; handles explicit keys, folder-sync
  (empty keys[]), the env-default divergence (cloud refuses empty env), and
  malformed CRs (fail-loud, never silently dropped). Reuses cloud/clients/kms
  ValidSegment/ValidSubpath so a coordinate the store would reject is never built.
- reseal: per-CR org-bound auth (owner==projectSlug), GET standalone -> POST cloud
  (cloud seals). Idempotent upserts, re-runnable. Plaintext transits memory only,
  wiped after write; results carry coordinates + status, never values.
- verify: read-only SHA-256 hash-compare standalone-vs-cloud + org-isolation matrix
  (cross-org 403, no-principal 403) against the real cloud guard.
- preflight: cloud /v1/kms reachability + JWT-validation probes + offline G1 delta.
- runbook: the ordered, rollback-safe cutover (standalone stays read-only).

Tests: round-trip against cloud's REAL embedded /v1/kms in-process (real seal,
real guard) via a zip app.Fiber().Test transport adapter; seal-proof (no plaintext
on disk); wrong-org refusal; folder-sync via LIST; hash-mismatch detection;
isolation matrix. go test ./cmd/kmsreseal green.
2026-07-17 22:03:16 -07:00
zeekay e8fa9d6f18 policy: kill /v1/featuregate/mode alias + rename featuregate → admission
featuregate READ like a synonym for flags — the source of the "isn't this the
same thing?" confusion. It is not: flags is the Policy decide-ENGINE (/v1/flags);
this package is the request-ADMISSION gate that composes it one-way (host→service
registry + waitlist.<svc> mode read + Enforce middleware + IAM approval check).
Renamed to `admission` — the precise systems term for policy-gating requests
(k8s-style admission control) — which also dodges the gate/gateway/gatewaypolicy
name cluster. flags stays THE engine; admission is a thin one-way consumer.

Also kills the /v1/featuregate/mode compat alias entirely (route + Enforce exempt
entry + test): one route, /v1/flags/waitlist. No shim, no adaptor, no backwards
compat — per the one-and-only-one-way law.

flags engine surface unchanged. Build/vet green; admission tests + apps frozen-Wire
order test pass (admission holds featuregate's slot). Deeper Policy collapse
(authz/entitlements/gatewaypolicy → one engine) is a separate staged HIP-0127 pass.
2026-07-17 21:26:28 -07:00
hanzo-dev 18289d0eb6 agents/coding: route a coding run to a chosen target machine
When a coding run carries a targetId (a registered /v1/agents/targets
machine), enqueue it as a durable task addressed to that target on the ONE
embedded tasks engine instead of running it in the cloud sandbox. No target
keeps the local sandbox path byte-unchanged.

- mailbox: an in-process rendezvous between the durable RoutedRunWorkflow and
  the external machine that claims a run over HTTP; tenant + machine isolation
  is a property of the (org,target) key, not a check a caller can skip.
- routing: per-target claim key (a capability, stored only as a SHA-256 hash,
  constant-time verified) is the machine identity; a fail-closed liveness gate
  (online + a live runner) is the dispatch admission.
- claim/report HTTP surface (org bearer + X-Target-Key) lets a machine claim
  and complete only runs addressed to it.
- coding.Dispatcher gains a routed branch: open the session on the target,
  enqueue the durable RoutedRunWorkflow (no secret in the payload — the machine
  authenticates with its own credential), return queued; fail closed on an
  unavailable target or a failed enqueue, never fall back to local.

Tests: dispatch-to-target, no-target-local-unchanged, dead-target-fail-closed,
cross-machine/cross-org claim denied, mailbox isolation + claim race.
2026-07-17 21:26:04 -07:00
hanzo-dev fac1cf9aa3 Route the S3 object plane through the hanzoai/s3-go client
Swap the S3 client in the 7 direct importers from github.com/minio/minio-go/v7
to github.com/hanzoai/s3-go (package minio; a minio-go v7.0.98 fork). Drop-in:
same package name and same New/Client/Options, {Get,Put,List,MakeBucket,
RemoveObject,RemoveObjects}Options, ObjectInfo/Object/ErrorResponse surface and
credentials.NewStaticV4; conditional-CAS (SetMatchETag/SetMatchETagExcept) and
presign paths unchanged.

minio-go leaves the direct requires and stays indirect (luxfi/zapdb via
clients/kms). Run go mod tidy after the s3-go v1.0.0 tag is published to
populate go.sum.
2026-07-17 21:24:02 -07:00
antje f7ded021ec gpu worker: a claimed job survives the engine recycle window
The supervisor recycles at queue-idle, but a freshly CLAIMED job is invisible
to the engine queue until its graph is submitted — so recycles fired over the
claim-to-submit window and staging failed on a dead engine, consuming the job
(observed twice in prod, seconds apart). Two invariants close it: a staging
latch the supervisor honors before recycling, and waitEngine() so a job
claimed while a recycle is already mid-flight waits out the restart instead
of dying on connection-refused.
2026-07-17 20:28:12 -07:00
zeekay dbc4966aeb build(iam2): bump v0.15.4 → v0.16.0 (argon2id SOTA password hashing)
Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 17:35:27 -07:00
z d38984f03f refactor(usage): unify account-usage onto the ONE /v1/usage surface
The account-usage plane (7456318) wrongly opened a SECOND usage surface inside
clients/link (/v1/links/usage). Move it into clients/usage so usage owns ALL
usage and link owns links and nothing usage — one surface, orthogonal, one window.

Moves (package link -> usage): sample.go (the Sample value + Sanitize), datastore.go
(the hanzo.account_usage warehouse series + reads, now behind a `warehouse` type that
holds only the DDL latch over aiobject's shared datastore — no handle, so usage keeps
NO Shutdown), and the record/samples handlers (account.go). Reconciled with the usage
subsystem: cloudUsageTable -> the existing llmTable, dsTime -> the existing tsLiteral,
duplicate aString -> dsString.

Route table (was /v1/links/usage*):
  POST /v1/usage           record account-usage samples (the collector)
  GET  /v1/usage/samples   one provider account's own lane dash (time series)
  GET  /v1/usage/summary   THE one summary — merged (see below)
  GET  /v1/usage/analytics{,/access}  unchanged

Summary collision resolved by MERGE, not two endpoints: the account-usage global view
folds into the existing /v1/usage/summary as a labelled `accounts` block beside spend +
LLM, over ONE window (aiobject.ResolveCloudUsageWindow drives both). Nothing dropped —
the caller's own linked-account rows AND the org Hanzo-routed rows both ride the one
summary, each side reporting its own availability, never summed.

Decomplected the Link-refresh: reportUsage braided a warehouse write with a Link
upsert, and since POST /v1/links already sets an account's usage snapshot, the sample
-> snapshot path was a SECOND way to do that. record now records usage only; the link
registry stays link's own concern. Drops the 3 Link-registry tests (they exercised
/v1/links, unreachable in a usage-only mount) and the Link half of 2 more; the warehouse
+ value coverage moves intact. No back-compat alias (the route was hours old).

Wire guard unchanged: link keeps its Shutdown (SQLite store), usage keeps none.
2026-07-17 16:53:17 -07:00
hanzo-dev 0aa853f300 fix(iam-edge): forward the public sign-in surface before the tenant gate
console.hanzo.ai is served one-binary off cloud, so its /v1/iam/* calls hit the
iam_edge — which required a validated org for EVERY route. That 401'd
'sign in to continue' on the sign-in routes themselves (get-app-login, login,
oauth token exchange), a chicken-and-egg that bricked console login (the
'unknown iam route' / 'sign in to continue' users saw). Forward the
unauthenticated-by-design sign-in surface (login-page config, credential submit,
signin/signup, captcha/verification aids, the OAuth token endpoint, OIDC
discovery) straight to IAM BEFORE the org gate. Tenant CRUD + org metadata stay
fully gated — no tenant-data route is opened. Test: TestIamEdgePublic.
2026-07-17 16:50:29 -07:00
hanzo-dev 610dcc166d wip(fleet-samples plane (clients/samples + /v1/fleet board)): rescued from agent that hit the session limit
Committed as-is to preserve the work (the building agent died mid-verify).
Not yet built/tested green; NOT merged to main. Resume from here.
2026-07-17 16:04:56 -07:00
zeekay 26a1910bca chore: trigger release build for iam2 v0.15.4 (federation fix)
d411200 (iam2 v0.15.1→v0.15.4 bump) did not trigger a release run; nudge
the push-triggered release so the federation-security-fixed image ships.
2026-07-17 15:21:10 -07:00
zeekay d411200359 build(iam2): bump v0.15.1 → v0.15.4 (federation SuperAdmin-mint CRITICAL fix)
v0.15.4 closes the red-team CRITICAL in the federation broker: authorize
Application.Organization on write + reserved-org guard in federation
link/provision (was: social login could mint a SuperAdmin / take over a
cross-tenant account) + SSRF IP filter. Required before the hanzo.id social
cutover. Build pipeline healthy (consensus v1.36.3).

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 15:03:48 -07:00
hanzo-dev 071174a414 Merge: surface the iam2 canary in /v1/flags 2026-07-17 12:39:04 -07:00
z bc98da0308 flags: surface the iam2 canary in /v1/flags
A read-only subsystem_iam2_active switch on the platform panel, mirroring
subsystem_iam_active, makes the clean-room iam2 selection visible in the /v1/flags
cockpit. The selector stays ONE thing — CLOUD_IAM_IMPL=iam2 at boot, applied on
the next reconcile — this switch reflects it, it does not add a second control.
Its description names the gate: the IAM cutover parity suite
(universe e2e/50-iam-cutover-parity) must be green against the iam2 shadow before
the canary is flipped.
2026-07-17 12:38:47 -07:00
hanzo-dev 7456318242 feat(account-usage): clients/link usage plane — samples, datastore series, /v1/links/usage
The account-usage plane over clients/link: a Sample value (one metering lane of
one provider account at one instant), a ReplacingMergeTree warehouse projection
(hanzo.account_usage + a dedup-preserving daily rollup MV) read back with explicit
read-time argMax dedup, and the /v1/links/usage surface — report samples, a
per-provider dash, and a global summary that sets a user's own linked-account plan
usage beside the org's Hanzo-routed cost of record, every row labelled by
source/scope/confidence and never summed together.

A windowless sample (a valid window class with no meter-reported duration or
reset) keys its class's nominal bucket, never the zero instant: every ranged read
filters window_start into [from,to) and the TTL drops epoch rows on arrival, so a
zero-keyed row would be written-but-never-read and would silently drop out of the
summary. Re-polls of a windowless counter collapse onto that one nominal instance
(ReplacingMergeTree by ts), so it is one row per lane, never summed across polls —
reconciling the two window-instance tests the rescued WIP left in contradiction.
2026-07-17 12:31:12 -07:00
hanzo-dev d9a20e2798 fix(identity): mint X-Billing-Account-Id from the claim, never from the client
The header was captured from client input and re-injected verbatim for any
validated principal. That was defensible while it was a mere attribution hint
no debit ever read — the comment said as much. It is not one anymore:
ai/object.Payer now resolves the PAYING account from it, so forwarding the
client's copy would let a caller name its own payer, which is the whole thing
the claim exists to prevent. A signup-org member could have sent
`X-Billing-Account-Id: org:hanzo` and pointed their spend at the shared pool.

It is now minted from the validated `billing_account` claim
(idClaims.mintedBillingAccount), mirroring iamauth.Claims.MintedBillingAccount
byte-for-byte, so the in-binary path binds what the gateway would and both
resolve one payer. The raw client copy is deleted on ingress and not restored.

The console read and the top-up now hand Payer that same claim, so the balance a
member SEES, the account a top-up FUNDS, and the account the ai gate DEBITS are
one wallet. Feeding Payer a different credential per call site is the modern
shape of the old org-vs-"org/user" split: a funded balance the gate refuses.

Tests drive real signed tokens through the boundary: the claim reaches the
header for person/org/project, a forged copy never survives (even on a token
that carries no claim, where a restored copy would be the only value present),
and an anonymous caller carries no payer at all.
2026-07-17 12:27:34 -07:00
z 36231f57e2 refactor(flags,featuregate): decomplect the waitlist host-gate out of the flag engine
flags is now the PURE (Principal,context)->verdict engine: Register/Bool/Int/
String/Board/SetPlatformSwitch/Defs + /v1/flags/* + native evaluator + the
platform-switch seed. ZERO host->service / ModeForHost / waitlist.<svc> /
mode-route knowledge.

The complete launch waitlist-gate feature moves to clients/featuregate, which
COMPOSES flags one-way (flags.Bool/Register/SetPlatformSwitch/Def/Defs; flags
never imports featuregate):
- flags/waitlist_store.go -> featuregate/registry.go (host->service map)
- flags/waitlist.go -> featuregate/waitlist.go (mode decide + admin funcs +
  seed + waitlist.<svc> Def registration + Mount/Shutdown + the mode route)
- flags/waitlist_store_test.go -> featuregate/registry_test.go
- the registry OrgStore handle (was flags.Client.registry) is now featuregate
  package state, opened in featuregate.Mount, closed in Shutdown
- Enforce default gate is now the LOCAL WaitlistModeForHost
- /v1/flags/waitlist AND /v1/featuregate/mode compat alias served by
  featuregate (route name unchanged)
- apps.Wire re-adds featuregate after admin (after flags); wire_test frozen row
- admin/services.go swaps the flags import to featuregate for the board funcs
2026-07-17 12:06:06 -07:00
zeekay 4343cdc684 fix(flags): keep /v1/featuregate/mode as a TEMPORARY compat alias for /v1/flags/waitlist
The namespace collapse (75d6f36) renamed the live waitlist-mode read to
/v1/flags/waitlist and made /v1/featuregate/mode a 404 — correct per the one-namespace
Policy primitive, but a BREAKING change to a public route whose external callers cannot
be fully enumerated from the monorepo (a deployed frontend could still call the old
path). Per the hard "never goes down for any customer" constraint, ship the collapse
WITHOUT the break: /v1/flags/waitlist is canonical; /v1/featuregate/mode is a temporary
alias to the same handler; both exempt from the Enforce gate.

Delete the alias (this route + its exempt entry in featuregate/middleware.go) once every
caller is confirmed on /v1/flags/waitlist — a one-line follow-up, gated on the owner.

Verified: gofmt clean, go build/vet green, exempt-path test asserts BOTH routes ungated.
2026-07-17 11:24:57 -07:00
hanzo-dev 187473bd92 Merge rip/services-kind: read one workload kind (App), drop the Service shim
services.hanzo.ai is dead (0 Service CRs cluster-wide; the fleet is 100% App).
clients/paas, clients/deploy, and clients/platform drop the two-kind read shim
and read apps.hanzo.ai only. The paas deploy endpoint (and release seam) now
always refuse a git-declared App with 409, naming the universe git path to
commit the tag to.
2026-07-17 11:01:28 -07:00
hanzo-dev 9659d1a56f paas/deploy/platform: read one workload kind (App), drop the Service shim
The services.hanzo.ai kind is dead: zero Service CRs exist cluster-wide and
the whole fleet is apps.hanzo.ai (kind App). These three cluster-facing planes
carried a two-kind read shim (App first, Service fallback) that is no longer
reachable, so strip it and read one kind — App.

- clients/paas: drop servicesGVR + crGVRs(); listApps/getApp/observeFleet read
  appsGVR directly (no cross-kind dedup). The deploy endpoint now always refuses
  (409): every App CR in the platform namespaces is git-declared and reconciled
  by Hanzo CD with selfHeal, so a patch here is reverted — the response names the
  git path to commit the tag to. releaseService refuses on the same grounds.
- clients/deploy: drop servicesCRGVR + appCRGVRs() and the "hanzo.ai/Service"
  registry entry; health/getAppCR/listAppCRs read appsCRGVR directly. coreSvcGVR
  (the core/v1 Service child object) is unchanged.
- clients/platform: drop servicesGVR + crGVRs(); resolveCR/getCR/deleteService
  read and delete appsGVR only. Tenant apps are still written and patched as App
  CRs in tenant-<org>.

Tests updated to the one-kind reality. Builds/vets/gofmt clean; go.mod untouched.
2026-07-17 11:00:25 -07:00
zeekay 75d6f36639 flags: move the waitlist mode read /v1/featuregate/mode -> /v1/flags/waitlist (one namespace)
The guard's public waitlist-mode read now lives under /v1/flags (the flags engine
owns it) — there is NO /v1/featuregate HTTP endpoint. The route, the Enforce
exempt prefix, and the doc/prose comments move; the featuregate Go PACKAGE (native
Enforce middleware) is NOT renamed, and the /v1/admin/services board is unchanged.

- clients/flags/routes.go            GET /v1/featuregate/mode -> GET /v1/flags/waitlist
- clients/flags/waitlist.go          doc comments repointed
- clients/featuregate/middleware.go  defaultExemptPrefixes /v1/featuregate/ -> /v1/flags/waitlist
- clients/featuregate/middleware_test.go  exempt-path assertion updated
- apps/apps.go                       stale prose comment repointed

Verified green: go build ./clients/flags/... ./clients/featuregate/... ./apps/...,
go vet, and CGO_ENABLED=0 go test ./clients/featuregate/...
2026-07-17 10:57:25 -07:00
hanzo-dev dcd107b336 Merge: real semver only — ai v1.821.1 / iam v1.31.28 / luxfi from sumdb (kill pseudo-versions + force-moved-tag poison) 2026-07-17 10:48:06 -07:00
zandhanzo-dev 04a4d68118 Real semver across the board: ai v1.821.1, iam v1.31.28, luxfi from sumdb
Three coordinate-hygiene fixes so the pipeline resolves deterministically:
  - ai v1.820.0 -> v1.821.1. v1.820.0 pinned iam at an orphaned pseudo-version
    (a commit rebased out of existence); v1.821.1 pins the real iam tag v1.31.28.
  - iam -> v1.31.28, the real published tag; the pseudo-version and its replace
    are gone.
  - luxfi go.sum re-recorded from the immutable sum.golang.org via go mod tidy,
    so consensus/vm can no longer carry the hashes a force-moved git tag served.

Nothing but coordinates changed; ai v1.821.1 is v1.820.0's tree with one dep line
repinned, so the compiled result is identical to the shipped v1.801.49. Real
public semver only: no pseudo-versions, no replaces, no force-moved tags.
2026-07-17 10:47:19 -07:00
zeekay a773c9225a fix(release): fail-closed container-tag floor so an orphaned tag is never reused
The v1.801.50 tag collision: a run pushed :v1.801.50 then was cancelled after
imagetools-create but before its git tag (orphaned container tag). The Tag steps
container-tag floor (cont_max) was fail-OPEN — `gh api ... 2>/dev/null || true`
yields "" on any API error — so a later run did NOT see :v1.801.50, recomputed
the same number, and REASSIGNED :v1.801.50 to a different image: an ambiguous
mutable prod tag (silent flip on any fresh-node reschedule).

Fail-CLOSED: if the container-tag lookup ERRORS (vs legitimately empty), retry the
whole attempt instead of proceeding on a git-only floor that cannot see the orphan.
A version that already has a pushed image is now never reused.

NOT reordering git-tag before imagetools-create (the other candidate fix): that
reintroduces the phantom "tag exists, image does not" this workflow was built to
prevent. Pairs with the crane-mirror timeout (ed4d372) that stops the hang→cancel
which orphans tags in the first place. Compute-step cont_max left as-is (hint only).

[skip ci]
2026-07-17 10:29:03 -07:00
antjeandGitHub 50bbf3a64d supervisor: recycle only at queue-idle; busy is not dead (#329)
Recycling on each completed render killed long renders mid-sample when short
jobs shared the engine (observed: every direct render died within ~6 minutes
while probe jobs cycled). The recycle now defers until the queue is empty.
Health: an engine that answers /queue with work in it is alive however slowly
it answers /system_stats; restarts require three consecutive silent probes
with an idle or unreadable queue.
2026-07-17 02:48:12 -07:00
hanzo-dev 7685705165 fix(release): mirror LOGIN is best-effort too — a registry blip must not fail the release
The 'Mirror credential' step fail-safed only on a missing KMS token, not on the
docker-login to registry.hanzo.ai itself. A transient 502 from the mirror registry
(ingress blip; the registry was healthy 6m before and after) killed the whole
serialized release — no image, no tag — even though ghcr (the PRIMARY) was fine.
Both login paths now skip the mirror (MIRROR_OK unset) on failure and continue.
Complements ed4d372 (the crane-copy timeout): the mirror is now best-effort end to end.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-17 02:26:28 -07:00
z ed4d37235f fix(release): bound the registry.hanzo.ai crane mirror with a timeout
An unbounded `crane copy` to registry.hanzo.ai can HANG (not just fail) — the
best-effort mirror once livelocked the Tag step and held the entire serialized
release lane (concurrency: release-cloud, cancel-in-progress:false), so no queued
release could run. A best-effort mirror must never be able to block the git-tag
receipt that follows it. `timeout 120` makes it truly best-effort.

[skip ci]
2026-07-17 02:06:48 -07:00
zeekay 0655cdb8cd build(iam2): bump v0.14.0 → v0.15.1 (federation + signing-key generation)
v0.15.0 adds the OIDC/OAuth2 social-federation broker (Google/GitHub);
v0.15.1 mints signing keys for keyless reserved-org certs so the embedded
iam2 publishes a JWKS and can sign tokens (shadow-canary finding). Carries
the full parity + RFC surface into the cloud image for the hanzo.id cutover.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 01:35:15 -07:00
hanzo-dev 7ff5862f42 build(deps): adopt luxfi/consensus v1.36.9 (unbreak force-moved v1.36.2 checksum)
luxfi/consensus v1.36.2 was force-repushed with different go.mod content, so
cloud's committed go.sum no longer matches and 'go mod download' aborts with a
SECURITY ERROR — breaking EVERY release. Same recurring luxfi force-move pattern
as c93ddf9 (keys). Bump to latest stable v1.36.9; clients/controlplane (only
importer) compiles clean, go mod verify passes.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-17 01:27:58 -07:00
antjeandGitHub fceb34c1d2 render: poll window matches the dispatch cap; engine recycles after each render (#326)
* render: poll window matches the dispatch cap; engine recycles after each render

The 10m local history poll undercut the 4h startToCloseTimeout the dispatcher
grants — live renders (observed 8-70m) were marked failed while still sampling;
only the mirror later delivered them. renderWindow now matches the cap.

The engine leaks ~58GB per render. The handler signals a recycle after each
COMPLETED render (never on timeout — the engine may still be sampling and the
mirror rescues late finishes); the supervisor restarts on the signal.

* mirror: skip hidden files — AppleDouble forks pass the extension check

._foo.png is a mac resource fork, not a render; 700+ of them poisoned a
library within an hour of the mirror going live.

* deps: luxfi/consensus v1.36.2 -> v1.36.3 — the v1.36.2 tag was re-pushed

Cold builds fail sumdb verification against the moved tag (downloaded
eKzasq4O... vs sealed IbeWQF1w...). v1.36.3 is the immutable successor;
never re-tag a published version.
2026-07-17 01:26:40 -07:00
hanzo-dev d5e12b3df1 feat(ai): bump ai v1.818.0 → v1.820.0 — router live-by-default + record-all + self-export/delete
Ships to prod: router.enabled=true (model=auto routes for every org by default),
per-request RoutingEvent recording for auto AND explicit models (up/down feedback
works on all models), per-org + global fit-gate-deploy-publish training, and the
self-scoped routing-data export/delete (data ownership). Pairs with the universe
CR ROUTER_ENDPOINT removal (heuristic 300ns is the live path).

Claude-Session: https://claude.ai/code/session_018PmFAHZvbBSTsuWyebwMra
2026-07-17 01:25:40 -07:00
hanzo-dev 36ba00f540 dedup: extract clients/payout from the 3 byte-mirror commerce.go copies
referrals/affiliates/authors each carried a byte-identical commerce.go (their own
doc-comments said so): the same commerce interface, httpCommerce, newCommerceClient,
deposit(), spendCents(), errUnconfigured — the S2S COMMERCE_SERVICE_TOKEN money-in
path (POST /v1/billing/deposit) + usage-rollup, triplicated.

Extract ONE clients/payout (attributed-credit -> commerce via commerceinproc): the
exported Commerce/Client/NewClient/ErrUnconfigured. Each program keeps a THIN
adapter — its own lowercase commerce interface + a commerceSeam that delegates to
payout.Client — so the program store/handler code AND their fakeCommerce test doubles
are untouched, and each program still names its own grant tag (grant:referral /
grant:affiliate / grant:author). ~330 duplicated lines collapse to one binding.

Zero behaviour change: identical HTTP contract, headers (X-Org-Id, Bearer), body,
fail-soft (ErrUnconfigured on deposit / 0 on spend when unwired), and errors.Is
sentinel. Adds payout unit tests (httptest) that give the extracted HTTP path REAL
coverage the fakes never did — ok clients/payout 0.010s.
2026-07-17 01:19:36 -07:00
hanzo-dev db53daea72 dedup: fold clients/gojabase into clients/goja (the Base binding is an option)
gojabase was the read-WRITE-Base sibling of goja: it wrapped a goja.Host and
added per-tenant Base/SQLite persistence, but duplicated the Host/Config/Request/
Response/New surface. Fold it into the ONE goja package as the Base-binding
CONSTRUCTOR — the persistence layer is now opted into via NewBase (vs New for a
read-only catalog bundle):

  goja.New   / goja.Host   / goja.Config   / goja.Request    read-only engine (plans/pricing)
  goja.NewBase / goja.BaseHost / goja.BaseConfig / goja.BaseRequest   + per-tenant Base

Moves gojabase.go -> clients/goja/base.go (renamed types, no goja. self-import),
store.go -> basestore.go, and both test files, all into package goja (zero
identifier collisions, coverage preserved). Repoints every importer —
dataroom/captable/sign (RW) to goja.Base*; plan/pricing already used goja and are
unchanged; base uses goja.TenantSegment. clients/gojabase deleted.

Behaviour is byte-identical: the engine, the per-request transaction commit-on-
<400, the injective TenantSegment, and the __db/__blob/__newId/__now host globals
are unchanged; only the package + exported names moved. No routes (both are
libraries). The gojabase[...] error prefix is kept as the RW-layer diagnostic label.
2026-07-17 01:19:36 -07:00
hanzo-dev 919d96f3f8 dedup: fold connectorruntime into the one automations subsystem
clients/connectorruntime mounts exactly ONE route —
POST /v1/automations/connectors/:id/run — the in-process goja runner paired with
automations own GET /v1/automations/connectors catalogue. It was a separate Wire
entry solely for that route. Fold connectorruntime.Mount in as a terminal
sub-mount of automations.Mount and drop its Wire entry + import -> ONE
automations subsystem.

The route is DISTINCT from every automations route and automations mounts no
/v1/automations/* wildcard, so there is no shadow; the runner still resolves the
shared engine lazily. clients/connectorruntime stays a focused package
(composition); its internal bundlecmd tool is untouched. Frozen wire row
removed.
2026-07-17 01:19:36 -07:00
hanzo-dev d846f17bf8 dedup: fold platform cron into the one tasks subsystem
clients/cron mounts NO routes — its Mount only launches a background starter
that registers durable schedules on the SAME shared engine (cloud.EmbeddedTasks)
that clients/tasks fronts. It was a separate Wire entry purely to get its
goroutine launched. Fold it in as a terminal sub-mount of tasks.Mount and drop
the cron Wire entry + import -> ONE tasks subsystem.

clients/cron stays a focused package (composition, not code-dumping): tasks
imports and invokes it. No routes change (cron never had any); the scheduler
still waits for the post-MountAll engine, so timing is unchanged. Frozen wire
row removed.
2026-07-17 01:19:36 -07:00
hanzo-dev 1648c08839 dedup: normalize the plan subsystem enable id "plans" -> "plan"
clients/plan.Mount was wired under the name "plans" while its package, and now
its generated standalone cmd, are "plan" — one subsystem, two names. Normalize
the Wire enable id (and cmd/plans -> cmd/plan, ServeSingle arg) to "plan".

Product routes are unchanged: the subsystem still serves /v1/plans/* (plural),
including its OwnsHealth /v1/plans/health probe — only the enable id / binary
name changes. No route drop; mount-all default still enables it (empty Enable =
all on). Updated the frozen wire row and the two cmd/cloud enable-id references;
TestMountAllAndServeHealth now maps plan to its real /v1/plans/health path
(enable id no longer equals route prefix for plan, as is already true for
account/runtime/agent).
2026-07-17 01:19:36 -07:00
385237bdaa fix(release): resolve prebuilt artifact digests + unbreak cloud-flags publish (#327)
cloud#321 landed the Go-only Dockerfile (FROM cloud-flags:latest) but the
release.yml integration wasn't in it, and the reusable could not publish
cloud-flags at all — so every release since has FAILED at
'FROM cloud-flags:latest: not found'. Two fixes:

1. native/flags/Dockerfile base ghcr.io/hanzoai/mirror/rust -> public.ecr.aws
   (digest-identical). The hanzoai/ci reusable builds cloud-flags with the repo
   GITHUB_TOKEN, which 403s pulling the cross-repo-linked private mirror package;
   a public base is GITHUB_TOKEN-pullable, so cloud-flags finally publishes.

2. release.yml resolves console-embed/agent-skills/cloud-flags :latest to
   IMMUTABLE digests at release time (crane) and passes them as CONSOLE_IMAGE/
   SKILLS_IMAGE/FLAGS_IMAGE build-args to BOTH the smoke build and the push build,
   replacing CONSOLE_CACHEBUST. Reproducible (pinned, not floating :latest) AND
   fresh (a console/skills/flags change is a new digest). A MISSING artifact FAILS
   the release BEFORE build/smoke/tag — the receipt invariant holds, never a
   phantom tag on an image that could not embed the real console.

Preserves #322's functional + migration smoke gates (different sections).

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-17 01:18:50 -07:00
494a6582c4 bench(finance): BenchmarkListUsage — the co-resident usage-read path (BUG 2) (#325)
The finance domain had no benchmark; this measures the read that replaced the
commerceinproc self-dispatch — finance.ListUsage over a per-org SQLite ledger,
at 100/1000/5000 seeded debits. Backs the reproducibility claim in the
hanzo-unified-tenant-cloud paper (1.25/10.5/61 ms). Also surfaces a real N+1:
store.Entries fetches postings per row the usage view never uses — a
postings-free read would cut this ~10x (follow-up).

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-17 01:06:20 -07:00
zandhanzo-dev b0254454c9 Money names its unit: Int -> Atto
There were two words for one idea. money.Amount.Minor() returns an integer at the
CURRENCY's decimals — money.USD declares 2, so cents. cloudmoney.Amount.Int()
returns an integer at 18 decimals — atto. Both are "the backing integer", neither
name says which, and they differ by 10^16.

That is not a style complaint. It is how

    Amount: cloudmoney.FromInt(u.Charge.Minor()), // exact 18-dp USD, no floor

got written, reviewed, and shipped. It type-checked (both sides *big.Int), it read
as "make an Amount from the integer", and it billed a $17.376 zen call as
$0.0000000000000017 until v1.801.44. The package documented the trap in prose —
"Take the decimal, never Amount.Minor()" — because a comment was the only place
the unit existed. Prose is not a type.

So name the unit, not the Go type: FromInt -> FromAtto, Int() -> Atto(),
IntString() -> AttoString(). FromCents/Cents already did this and were never
confused with anything. Now the units are visible at the call site, and the
mistake reads as one: FromAtto(x.Cents()) is obviously wrong where
FromInt(x.Minor()) was obviously fine. It no longer type-checks either —
FromAtto takes *big.Int, Cents() returns int64 — so the pairing that cost us the
money is now two independent kinds of error instead of zero.

Values are untouched: Atto/AttoString return exactly what Int/IntString did, so
the treasury ledger hash and every stored 18-decimal string are byte-identical.
No migration, no data change — only the names, and the compiler found every one
of them (45 sites; a regex could not have, because .Int() also belongs to big.Int
and decimal).

Zero regressions: the failing set is identical to origin/main.
2026-07-17 00:52:06 -07:00
zeekayandClaude Opus 4.8 c93ddf92bf build(deps): adopt stable luxfi/keys v1.4.1 (unbreak force-moved v1.4.0 checksum)
luxfi/keys v1.4.0 was force-moved on the remote (tree 5153d639→80a3745a),
producing a go.sum checksum mismatch that broke `go mod tidy`/`go build` and
the hanzoai/cloud image build in CI. Every keys tag v1.1.0..v1.4.0 was
force-moved with content changes; only v1.4.1 is byte-stable (local==remote
tree 17551acd). Adopt v1.4.1 as the single stable version. Its go.mod floors
the unified luxfi stack, so the transitive set moves forward (geth 1.17.12→
1.20.1, consensus 1.35.32→1.36.2, crypto/database/warp/zap/…), all within v1.

go.mod/go.sum only; diff vs origin/main is luxfi/* exclusively. Embedded
iam2 v0.14.0, apps.go, and concurrent commerce/agent work untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 00:50:13 -07:00
74478bb9a2 build: decomplect the cloud image — cloud compiles ONLY Go (#321)
The console SPA, agent-skills catalog, and native flags staticlib are each now
built by their OWN CI as a versioned immutable image and PULLED into the cloud
build, instead of rebuilding node+python+rust from scratch every release. The
console stage (cold npm install + full Next static export, cache-busted every
build) was the ~20-min long pole; it is now a registry pull.

- Dockerfile: console/skills/flagslib stages -> FROM ${CONSOLE_IMAGE}/
  ${SKILLS_IMAGE}/${FLAGS_IMAGE} prebuilt pulls; COPY sources updated
  (/dist, /catalog, /libhanzo_flags.a). Mirror golang+alpine bases, GOPRIVATE,
  and every RED gate (SQLCipher proof, modernc guard, cek frozen-format) are
  unchanged. Pins are ghcr.io so both buildx lanes pull directly; mirrored to
  registry.hanzo.ai (S3) for GET-flow consumers. release.yml still owns the
  cloud image + v* tags (it resolves CONSOLE_IMAGE to a fresh console-embed
  digest, as CONSOLE_CACHEBUST did).
- native/flags/Dockerfile: the cloud-flags artifact (rust -> scratch /libhanzo_flags.a).
- hanzo.yml: images: cloud-flags (distinct artifact, never a v* tag) + zccache
  RUSTC_WRAPPER on the native-flags gate (no-op unless the runner carries it).

Companion artifact publishers: hanzoai/console#(console-embed),
hanzoai/openapi#(agent-skills).

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-17 00:49:57 -07:00
hanzo-dev f76e714d37 refactor(flags): extract the Policy primitive — fold featuregate's waitlist into the flags engine
featureflags -> flags (Hanzo's Unleash-analog / hanzoai/flags): ONE runtime
decision engine, (Principal, context) -> verdict, evaluated in-process, hot.

Fold the launch-control waitlist off its DUPLICATE SQLite mode store
(clients/featuregate) onto the one engine:

  - a service's waitlist mode IS the platform switch waitlist.<svc>,
    evaluated through the same native evaluator as every other flag
  - the host->service registry folds into clients/flags (waitlist_store.go,
    mode column dropped — the mode is the switch)
  - featuregate.Enforce stays but as a CONSUMER of flags.WaitlistModeForHost
    (the decide), via an injectable Gate seam
  - /v1/featuregate/mode -> served by flags (the decide), same wire shape
  - /v1/admin/services -> an admin lens like /v1/admin/flags
  - per-user approval still reuses IAM (featuregate/approval.go, unchanged)
  - featuregate dropped from apps Wire() — it exposes only Enforce now

The flags package doc names the aspirational end-state: authz (access policy)
and entitlements (product-access policy) are the SAME (Principal,context)->
verdict shape and could COMPOSE this one engine. NOT touched here — flagged only.

Tests (green): featuregate RULE acceptance matrix + approvals; folded-registry
store (Seed/ServiceForHost/List/Upsert); flags env/board/parsers;
apps TestWireOrderMatchesFrozen. Native-engine + cek store tests need
CGO+libsqlcipher+libhanzo_flags (CI), unchanged.
2026-07-17 00:48:30 -07:00
hanzo-dev 53cd7c585d billing bridge: ask commerce what it gates, stop keeping a copy
The bridge's mint guard kept its own list of commerce's mint routes under the
comment 'kept in lockstep with api/billing/handlers.go'. A comment cannot hold
two lists together, and it hadn't: 10 paths here against the 16 commerce gates.
Six money-mint routes were outside this guard entirely.

Commerce now DECLARES its gated surface — middleware.Mint records what it gates,
MintRoutes() exports it (commerce v1.49.0) — so the guard reads that declaration
instead of copying it. A mint route added in commerce is covered here with nobody
remembering anything, which is the only version of this that survives contact
with a busy repo.

Registration is what populates the registry, so the guard registers commerce's
billing routes before reading it. The request-level assertion is unchanged and is
the part that matters: an ordinary org user's call must never REACH commerce,
because arriving at all means arriving with the admin service token that
satisfies MayMintMoney.

Proof it bites: allowlisting 'deposit' fails with the escalation itself —
'POST /v1/billing/deposit reached commerce carrying Bearer svc-tok'.

16/16 gated routes refused. All six bridge tests pass.
2026-07-17 00:41:23 -07:00
antjeandGitHub 8479418021 gpu: mirror local studio renders to the library (independent of claims) (#324)
Every 30s (its own ticker in the connect select loop), scan --studio-dir/output
recursively for image files new or changed since the last scan and POST each to
<studio-url>/v1/library/upload with the worker's bearer, tagged ?node=<identity>
and ?subpath=<subfolder>. So EVERY render lands in studio.hanzo.ai — including
ones produced OUTSIDE the job path: a graph hand-run on the node, or a render
that finished after its activity was reaped (the stranded-late-render class). The
mirror's independence from claims/activities is the point.

State is a tiny in-memory map[relpath]size that skips unchanged files; the studio
endpoint dedupes byte-identical uploads, so a re-scan after restart is cheap and
harmless. One log line per newly stored file; upload failures are summarized once
per scan and retried next tick (no 5xx spam). New --studio-url flag (default
https://studio.hanzo.ai, HANZO_STUDIO_UPLOAD_URL honored) so the mirror works
WITHOUT jobs. No new deps. Test: cli/gpu_mirror_test.go (new/changed uploaded,
unchanged skipped, node+subpath+bearer carried).
2026-07-17 00:20:24 -07:00
antje 309c830abb agent: v0.1.2 — pass a completion billing refusal (402) through, not a 502
Bump github.com/hanzoai/agent v0.1.1 -> v0.1.2 and have the in-process completer
return the agent typed hz.UpstreamError{Status,Body} on a non-2xx completion. The
round then passes a caller-facing 4xx (402 insufficient_balance, 429, 403) through
verbatim so a no-credit user sees "add credits", not an opaque "agent: completion"
502. Proven live: POST /v1/agent with a real hk- key ran the round end-to-end and
the completion returned 402. Dropped the now-unused clip() helper. Simplified
commerce_errorscope_test to the real invariant (typed 403 never flattened to 5xx)
now that commerce v1.48.10 honors status itself; the scope stays as the boundary.
2026-07-17 00:01:18 -07:00
antje f4ab7e2bd3 deps: commerce v1.48.5 -> v1.48.10 — free $0 tiers stay self-serve (paid-tier gate keys on price, not includedCreditUsd) 2026-07-16 22:38:59 -07:00
zandhanzo-dev 94873e3e0c MountAll's doc outlived Typed
It still said the MountFunc takes the app as `any` and that in-repo subsystems
recover it via Typed. Neither is true: MountFunc names *zip.App and Typed is
deleted. The app is handed to each mount as itself.
2026-07-16 22:24:17 -07:00
antje 0e377c52ad feat(sync,git): Gitea-native sync provider + webhook reject parity; fix LoadConfig flag panic
CHANGE 1 — the /v1/sync git provider drives GITEA (the one git store), not the
retired cloud embedded store:
- clients/sync/gitea.go: a Gitea REST + go-git client (GIT_ADMIN_TOKEN, GITEA_URL).
  Inbound = fast-forward-only go-git fetch(source) -> push(Gitea), so a diverged ref
  is a conflict, never overwritten (split-brain guard, now on Gitea). Outbound = a
  Gitea push-mirror (sync_on_commit) so Gitea itself propagates every commit. Fails
  closed when GIT_ADMIN_TOKEN is unset.
- git_provider.go Reconcile and sync_api.go patch/delete now compose those Gitea
  primitives; the cloud embedded seams (InboundGitSync/ImportGitRepo/EnsureGitMirror)
  are retired from the sync path. resolve() decision, loop guard, cursor idempotency,
  and hop limit are unchanged.

CHANGE 3 — webhook reject parity + a pre-existing root test panic:
- Wrap /v1/git/webhook and the slack(events,commands)/discord/teams/telegram inbound
  webhooks in cloud.Terminal so a bad-signature 401 / malformed 400 survives the
  commerce /v1 500-flatten (uniform reject codes, matching /v1/sync and
  /v1/connector/github/webhook).
- config.go LoadConfig: guard the process-global flag registration with a sync.Once,
  so a re-entrant LoadConfig (many test callers in one binary) no longer panics
  "flag redefined: enable".

CHANGE 2 (retire the embedded git server) is NOT done: it is still a live dependency.
cloneURL resolves to api.hanzo.ai/v1/git (cloud's OWN embedded server) and the
coding-agent orchestrator clones from uploadPack, pushes to receivePack, and reads the
store via VerifyRef. Deferred — migrate coding to Gitea first.

Tests (CGO_ENABLED=0): clients/sync + clients/integrations green; new
clients/sync/gitea_test.go proves reconcile acts on Gitea and the fast-forward guard.
2026-07-16 22:20:41 -07:00
hanzo-dev 66e4999a44 Merge: Mount takes *zip.App — delete the Typed shim and its 85 wrappers 2026-07-16 21:57:01 -07:00
antje 1cf505ebd1 edge: scope commerce error envelope to its own routes (unblock release smoke — 14 endpoints 500→4xx)
commercemid.ErrorHandlerJSON() was installed as a /v1 GROUP middleware, but fiber
matches group middleware by PREFIX, not by the handle a route registered on. So on
the shared /v1 it wrapped EVERY subsystem mounted after commerce (projects, agents,
wallets, functions, integrations, marketplace, team, s3, analytics, knowledge,
automations, deploy, billing) and flattened their typed zip.HTTPError (403 "X-Org-Id
required", 400, …) into a blanket 500 — the store envelope always renders 500. The
authenticated release smoke (#322) correctly fails on 5xx, so this blocked every
release since it landed.

Fix stays cloud-side (no commerce dep bump, no money-path churn): commerceErrorScope
guards the envelope by commercePrefixes, so it stays on commerce and every other
subsystem renders its own status via zip default. Pre-commerce subsystems (kms,
o11y) already did; this makes the post-commerce ones match. Verified with the REAL
commerce middleware: /v1/projects 403 (was 500), /v1/store/current keeps the 500
envelope. Regression test added.
2026-07-16 21:50:00 -07:00
zandhanzo-dev 25fcaa4351 Mount takes *zip.App: delete Typed and the 85 wrappers
MountFunc took `app any` and cloud.Typed asserted it back to *zip.App on every
mount — a runtime check doing the type system's job, with a failure branch that
could not fire because the only value ever passed is a *zip.App. Every subsystem
paid for it: 85 call sites read cloud.Typed(x.Mount) instead of x.Mount.

The reason given was circular. cloud said `any` was load-bearing because an
external module (licensing) exposed func(any, Deps) error; licensing said it used
`any` to avoid an import cycle in pkg/cloud. Each pointed at the other, and the
cycle cannot exist: this package already imports zip (build.go), and zip does not
import cloud. The `any` was justified by nothing.

So name the type. MountFunc is func(*zip.App, Deps) error — what every subsystem
already exported and what licensing's own doc claimed all along. Typed is deleted,
the 85 wrappers are gone, and the three in-repo mounts that hand-rolled the same
assertion (mountZen, mountMetrics, MountO11y) just take the app.

The compiler immediately found what the `any` had been hiding: four mounts still
shaped func(any, ...), one of them across a module boundary. That is the point —
a signature drift is now a build failure instead of a runtime error nobody would
see until a subsystem mounted.

Also here, because the same rip surfaced them:

  - iam: v1.31.27-0.20260716191958-4400762928a2 -> v1.31.28, and the replace
    pinning a second pseudo-version is dropped. The required pseudo-version named
    a commit that no longer exists (it was rebased away), so `go mod tidy` could
    not resolve it; v1.31.28 is a real tag and a strict superset of what the
    replace pointed at. A version, not a coordinate, and no replace.
  - licensing -> v0.1.5, which is where the typed Mount ships.
  - cloud.OrgConfig is aliased next to LicenseEntitlement. Both are named by
    CommerceClient's methods, but only one was exported, so the exported
    interface could not be implemented from outside without reaching into
    cloud/types — an omission, not a boundary.
  - build_registration_test.go tested Typed and nothing else. "Recovers the
    *zip.App" proved an adapter passed through its argument; "fails closed on a
    wrong type" cannot be compiled now. What is left is the assertion that a
    subsystem signature IS a MountFunc — which the build checks.

No regressions: the failing set is byte-identical to origin/main (11 pre-existing
TestAudit_*).
2026-07-16 21:47:19 -07:00
zeekay f49d283282 build(iam2): bump embedded iam2 v0.1.1 → v0.14.0 (parity-complete surface)
The CLOUD_IAM_IMPL=iam2 identity fold (clients/iam2, identitySpec) was pinned
to iam2 v0.1.1 — an early cut missing the whole console-parity surface. Bump
to v0.14.0 so the embed actually serves what hanzo.id needs:

  - RFC/IETF surface (HIP-0111): OAuth2 code+PKCE/refresh/client_credentials/
    password, RFC 8693 token-exchange, 7662 introspection, 7009 revocation,
    8414 AS-metadata, OIDC UserInfo, SCIM 2.0 Users
  - Casdoor verb-alias compat (transitional cutover bridge) for every verb the
    live console/gateway hard-code: users/orgs/apps/providers/roles/projects
  - operator bootstrap upsert (IAM CR reconciliation), TOTP MFA enrollment,
    organization-scoped projects (ScopeSwitcher)

No wiring change — main's identitySpec + co-mingle iam2server.Mount(app, db)
compile unchanged against the v0.14.0 API. zip already at v1.8.3.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 21:43:03 -07:00
antje f20e7679ef feat(connector,sync): /v1/connector/github/webhook namespace + reject paths survive the commerce /v1 500-flatten
CHANGE A — first-party vs external route naming. Rename /v1/github-webhook ->
/v1/connector/github/webhook, opening the external-platform namespace
/v1/connector/<provider>/webhook (github now; gitlab/others are sibling literal
routes later, each with its own signature scheme). /v1/git/webhook (first-party
Hanzo Git) and /v1/sync (the bridge) are unchanged. No live consumer breaks:
the GitHub App isn't created yet.

CHANGE B — 500 -> real 4xx on the reject paths. mountCommerce registers
commerce's ErrorHandlerJSON on app.Group("/v1"); that filter rewrites ANY error
a downstream /v1 handler PROPAGATES into a hardcoded HTTP 500. Every /v1
subsystem mounted after commerce is wrapped the same way -- git, sync AND
integrations alike. git does NOT escape it: its isolated tests read 401 only
because they don't co-mount commerce (its bad-sig path is never exercised in
prod, so the flatten went unseen). Add cloud.Terminal, which writes a returned
*zip.HTTPError in-band and returns nil, so the filter's c.Next() sees nil and
has nothing to flatten. Wrap /v1/sync (all verbs) and the connector webhook.
sync no-principal is now 401 (was 403) -- an authentication failure, matching
the webhook's bad-sig 401. The commerce filter is left untouched.

Tests reproduce the /v1 flatten filter and assert: sync unauth -> 401, connector
bad-sig -> 401, malformed body -> 400; connector resolves at the new path and
the old /v1/github-webhook 404s.
2026-07-16 21:32:46 -07:00
antje c93fd2c4dd edge: /v1/agent self-meters (fix 503 balance_unavailable on the agent orchestrator)
The global BillingGate priced /v1/agent/* at a flat 1c via a dead legacy
bot-reverse-proxy rule in DefaultPrice, so every agent request was gated: the
unauthenticated preset/conversation reads hit the balance check, failed closed,
and 503d before the orchestrator handler ever ran. The route was mounted and
winning precedence the whole time — the gate short-circuited ahead of it.

/v1/agent is self-metered: the reads are free and POST /v1/agent bills through
the /v1/chat/completions it runs in-process (gated + metered downstream), so the
edge must price it 0 or double-bill. Drop the legacy branch (and its now-unused
cloudEdgePriceCents const); price /v1/agent and /v1/agent/* at 0. Tests updated.
2026-07-16 21:06:09 -07:00
hanzo-dev 2f4095e68a Merge feat/openapi-spec: /v1/openapi.json generated from the live router
The spec IS the router, not a description of it: apps.Wire() -> MountAll ->
app.Fiber().GetRoutes(). 983 operations / 692 paths / 109 products, served beside
/zap so ZAP and OpenAPI are two projections of one route table rather than two
sources that can disagree.

A drift test proves the bijection and was proved to fire; it already caught
/v1/pricing-policy and /v1/pricing/policy collapsing onto one operationId.

# Conflicts:
#	serve.go
2026-07-16 20:42:42 -07:00
6233b3c805 fix(billing,smoke): usage reads the co-resident ledger (not a self-dispatch) + a real authenticated release smoke (#322)
A live authenticated smoke surfaced two production bugs; this fixes both and
adds the durable smoke that now guards every release.

BUG — /v1/billing/usage 500 for a valid caller. usage() proxied
"/v1/billing/usage" through commerceinproc, which re-dispatches BY PATH.
Commerce's own billing routes are behind //go:build cloud and never compiled
here, so the ONLY registration of that path is usage() itself — the S2S hop
re-entered the handler, which self-answered "sign in to view billing". This is
the SAME defect balance() was already fixed for. usage() now reads the usage
ledger DIRECTLY from cloud's own finance ledger (finance.ListUsage → the
wallet→revenue debits RecordUsage wrote), off the self-dispatching hop;
split-deploy falls back to the commerce S2S read, unchanged.

BUG — the balance gate 402'd read-only GETs (fixed in hanzoai/ai, pinned via
the go.mod bump). A $0-balance org could not VIEW its own resources. Fixed in
the ai module's BalanceGateFilter — reads never spend, so GET/HEAD/OPTIONS are
exempt; only writes/metered POSTs gate on balance.

SMOKE — cmd/smoke: a durable, authenticated per-subsystem prober. One
side-effect-free read per core subsystem; a read that 402s (balance gate) or
5xx-es (crash) fails the release. Baked into the image (Dockerfile) and wired
into release.yml as the functional gate after the boot check, so a release can
never ship with chat/billing/projects/kms/... down. The smoke token is
KMS/secret-sourced (never hardcoded); absent → the anonymous matrix still gates
public/authed and catches every 402-on-read / 5xx.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-16 20:35:56 -07:00
hanzo-dev 52acb303c2 cli: multi-identity credential store with active-pointer switch
Store every logged-in identity in ~/.hanzo/identities.json keyed by a stable
<owner>/<name> key (admin/z, hanzo/z), with an Active pointer; mirror the active
identity into credentials.json so every legacy single-file reader is unchanged.
A second login as a different owner for the same email (privilege separation:
hanzo-admin-guard vs hanzo-console) is stored beside the first, not over it.

New: auth list (whoami --all), auth switch <owner|owner/name> (alias use),
logout [<owner>]. Token refresh writes through the store (SaveActive) so the
active identity stays fresh after rotation. All files stay mode 0600.

One credential store, active-pointer switch, backward compatible.
2026-07-16 20:20:53 -07:00
hanzo-dev 951703f486 dedup: fold hanzoai/o11y module wildcard into the one o11y subsystem
apps.Wire() had TWO {Name:"o11y"} entries: the in-repo read plane
(o11y.MountO11y) and the external hanzoai/o11y module wildcard
(cloud.Typed(o11ymod.Mount)). Fold the module wildcard in as the TERMINAL
sub-mount inside o11y.MountO11y (registered after every specific /v1/o11y/*
route, so Fiber in-order match still gives them precedence), delete the 2nd
Wire entry and the now-unused o11ymod import -> ONE o11y spec.

/v1/o11y/health is preserved exactly: the merged entry keeps OwnsHealth=false,
so the generic always-ok route is registered before MountAll (ahead of the
wildcard) — byte-identical to when the module co-entry, also OwnsHealth=false,
triggered it. Frozen wire order updated (two co-owned rows -> one); the
no-duplicate test no longer needs an o11y exemption.
2026-07-16 20:17:18 -07:00
hanzo-dev 7022064723 dedup: delete dead clients/session
Zero importers, no cmd/session, absent from apps.Wire(); its /v1/code/sessions
surface was never mounted (dark). Removes session.go, store.go, session_test.go.
2026-07-16 20:10:06 -07:00
hanzo-dev 8dabc5c469 fix(cloud): serve /v1/iam/* — the org-scoped edge to Hanzo IAM
The one-binary console (console.hanzo.ai — cloud serves console's static export)
reads org members + projects at /v1/iam/*, but cloud 404'd those (IAM isn't
folded in-process yet) and the old /org/iam BFF proxy is pruned from the static
export. So the browser got the SPA shell (HTTP 200 HTML), which the client
surfaced as "Request failed (HTTP 200)" — the Platform page died on "Could not
load".

iam_edge.go adds an org-scoped reverse edge at /v1/iam/* → the standalone IAM,
mounted when IAM is NOT folded in-process (else that subsystem owns the path — no
double-mount). The org is PINNED to the caller's validated, server-minted
X-Org-Id (never a raw client header) — load-bearing, since IAM's own authz is
permissive on the org-keyed routes, so without the pin one tenant could read
another's projects. A super admin may cross; a tenant may not; writes require an
org admin and must carry the caller's own org. Shares the ONE IAM identity
(iamHost/iamCred) with the API-key resolver (DRY).

Verified: 7 gate tests (pin, cross-tenant refuse, super-cross, allow-list, 401,
write-gate, own predicate) + go build clean.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-16 20:07:12 -07:00
zandGitHub ac65cf4ec3 chore(deps): bump hanzoai/tasks → v1.51.1 (fix api.hanzo.ai concurrent-map-writes crash)
Fixes the live recurring tasksd concurrent-map-writes fatal (hanzoai/tasks#18, v1.51.1). Build green; the failing 'Test' check is the pre-existing repo-wide LoadConfig flag-redefine panic (fails on main + all branches), unrelated to this go.mod-only bump.
2026-07-16 20:02:07 -07:00
hanzo-dev 5e06e29de5 Merge: tenant apps are App CRs — the last writer of the Service kind
clients/platform was the only thing still minting Service CRs; every other
declarer is already App. A role-less App dispatches to the operator's service
profile, the same reconcile the Service kind ran, so a tenant workload carries
over verbatim.

Existing tenant CRs still resolve (App first, Service second). A redeploy patches
the kind it IS rather than minting a twin — two CRs on one name is the
commerce-admin ownerRef flap, not a migration. Teardown deletes BOTH kinds:
either alone rebuilds the app the tenant just deleted, still billing.

113 tests pass, 7 new.
2026-07-16 19:40:04 -07:00
hanzo-dev 095e1cce4e platform: tenant apps are App CRs — the last writer of the Service kind
clients/platform was the only thing still minting Service CRs. Every other
declarer is already App: universe git has zero hanzo.ai/v1 kind:Service, the
operator calls App "the sole workload reconciler for the collapsed fleet", and 69
of the 80 live CRs are Apps. A tenant app was the exception for no reason — a
role-less App dispatches to the operator's service profile (controllers/app.rs
`classify("") => Dispatch::Service`), which is the same reconcile the Service kind
ran, so an App carries a tenant workload verbatim.

Nothing mints a new Service CR after this. What already exists still resolves:
reads, patches, scales and deletes walk crGVRs() — App first, Service second — so
the 3 live tenant CRs written before this keep working untouched.

A redeploy of a pre-collapse app patches the kind it IS rather than declaring an
App twin. Two CRs claiming one name is not a migration, it is the commerce-admin
flap: both kinds materialize the same children through the same materializer under
one field manager, so the Deployment's ownerRef just flips between them.

Teardown deletes BOTH kinds. Either kind alone re-materializes the Deployment, so
removing only the one that resolves first would rebuild the app the tenant just
deleted — running, and still billing, minutes after a successful delete.

Sequencing (deploy order matters): operator v0.7.7 carries the Claim guard that
makes App the deterministic owner when both kinds claim a name. This is safe
before it — the no-twin rule means a colliding pair is never created here — but
the guard is what makes an existing collision safe to clean up.

REMOVABLE once no Service CR remains in any tenant namespace: drop servicesGVR
from crGVRs() and the delete-both, and this file is App-only.

Tests: 113 pass; 7 new (kind is App, no twin on legacy redeploy, delete removes
both kinds, idempotent delete, resolution order, absence honest). The 10 tests
that asserted a Service CR is written now assert the kind we write.
TestMigrateOverLegacyPlatformApps fails identically on pristine main (sqlcipher
codec, environmental).
2026-07-16 19:39:56 -07:00
antje bffe597ff0 apps: frozen wire list — add sync entry (main added /v1/sync to Wire; frozen drifted, breaking TestWireOrderMatchesFrozen); fix agent route comment 2026-07-16 19:38:00 -07:00
hanzo-dev 601d5660b7 Merge: paas reads the kind the fleet runs on, and does not fight the git declarer
The PaaS control plane serves platform.hanzo.ai and read services only: 69 App CRs
run in the scanned namespaces against 7 Service CRs, so the SUPERADMIN drift board
rendered 7 rows for a 69-app fleet — 1 in production — and /v1/paas/health probed
the Service CRD, found it served, and reported ok over a blind board.

Reads now walk App first, Service second, deduped by name. Writes refuse a
git-declared App (Hanzo CD syncs it with selfHeal, so a patch is reverted on the
next sync) and name the file to commit to; a Service CR still patches.

35 tests pass, 0 fail.
2026-07-16 19:30:16 -07:00
hanzo-devandz 8bf2dda4d1 feat(shard): in-binary org→owner shard router for horizontal writer scale
Lifts the unified binary off replicas:1 on DigitalOcean (RWO-only block
storage, no RWX) WITHOUT any shared volume. Each org is pinned by rendezvous
hashing (ha.Owner over the static CLOUD_PEERS ring) to exactly one owner pod;
a request whose org this pod does not own is transparently forwarded to the
owner, so every per-org SQLite store (KMS, finance, and every org-keyed store),
audit append, per-org rate ceiling, and prepaid billing debit runs on ONE pod.

THE INVARIANT — no two pods ever write one tenant's SQLite file — is upheld by
two independent guarantees that compose: per-pod RWO PVC (different physical
files per pod) + org→owner routing (all of an org's writes on one pod).

- shardrouter.go: the middleware. Runs immediately after SanitizeIdentity so it
  keys on the VALIDATED, server-minted X-Org-Id (never a client header), hashes
  the SAME injective SanitizeOrg slug the on-disk path uses (routing key ≡ file
  key), and forwards via a fasthttp streaming proxy (SSE/chat pass through — a
  streaming chat still carries a per-org billing debit, so it routes too) with
  dial-only retry across an owner's roll gap and a 421 loop-guard on divergence.
  Static identical membership rules out dual-owner and routing loops.
- config.go: CLOUD_PEERS (id@addr ring) + POD_NAME (self). Sharding auto-enables
  only when >1 peer; Validate fails closed if self is not in the ring or if
  embedded iam (non-shardable process-local sessions) is co-enabled.
- serve.go: wires the middleware after IdentityMiddleware; shard-aware boot log.
- audit_serve.go: per-shard audit is automatic on the per-pod PVC; stamp the
  shard id on the AU-9 checkpoint stream so the tail-truncation monitor tracks
  N heads. writerpin.SingleWriter is correct PER SHARD (each pod sole writer of
  its orgs); the writer lease is pod-local under per-pod PVC and stays off.

No-op (byte-identical to today) when CLOUD_PEERS names ≤1 pod. N pinned at 3;
rebalance-on-N-change (a tenant-file move) is the documented follow-up.

Tests prove: exactly one owner per org (deterministic, total, evenly
distributed); all pods agree (no dual-writer); owned served locally, unowned
forwarded to the owner (local chain never runs, hop header set, response
streamed); org-less served locally; loop-guard 421; boot-gate fail-closed.
2026-07-16 19:28:27 -07:00
hanzo-dev db226b321b Merge: bill zen in the credit unit — FromInt(Minor()) understated every debit by 10^16 2026-07-16 19:26:12 -07:00
hanzo-dev f24596453e Bill zen in the credit unit, not the currency's minor unit
zen prices every SKU as an exact 18-dp value tagged money.USD. money.USD
declares 2 decimals, so Amount.Minor() rescales that value to CENTS, while
cloudmoney.FromInt reads its argument as 18-dp. Composing the two divided
every zen debit by 10^16: a $17.376 charge debited $0.000000000000001738,
and any charge under half a cent folded to a zero that metering.Record drops
before it reaches the ledger — no debit row, call served free.

The spend gate read the same composition. FromInt(est.Minor()).Cents() is
always 0, and AuthorizeVerdict only compares balance against size when
AmountCents > 0, so the size check was dead code and any org with a positive
balance could draw a request of any size. The dust debits never accumulated,
so the cap could not trip either.

Route the seam through one conversion. credit() carries the exact decimal
across and changes only the minor-unit convention — no rescale, no rounding,
no factor — because the decimal is the value and a currency's Decimals is a
rendering convention. cloudmoney.FromDecimal is the typed counterpart of
ParseUSD that makes this expressible without a string round-trip. nano()
takes the typed Amount rather than a bare *big.Int, so a cents integer can no
longer reach the warehouse fold.

The tests asserted Amount.Int() against Charge.Minor() — the same integer on
both sides — so they proved a round-trip and were blind to the unit. They now
assert against known dollar values built through a different constructor: a
$17.376 charge debits $17.376. Reintroducing Minor()-into-FromInt fails all
of them, including a gate test that admits an over-cap request.
2026-07-16 19:24:29 -07:00
hanzo-dev 185cca68f2 Forward only the billing paths the console reads
The /v1/billing/* bridge attaches the commerce service token, and that token
satisfies commerce's MayMintMoney. The subpath was checked for traversal but
not against a set, so POST /v1/billing/deposit forwarded a mint for any
signed-in user — pinned to their own subject, which aims the mint rather than
stopping it. Commerce closed this on its direct path the day after the bridge
reopened it, and wider: the bridge needs only a validated principal, never an
admin bit.

The forwardable set is now a per-method table, consulted before the token is
attached, 404 on anything else. Per-method because payouts is a GET read and a
POST mint on one path — a method-blind set hands the mint to every reader.

Nothing reachable today: commerce's mint is not compiled into this binary and
commerce.hanzo.svc resolves here. But devnet runs a real mint-capable commerce
and points cloud-api at it under a second name for the same thing; unifying
those names would arm it. The gate should exist before that cleanup does.
2026-07-16 18:54:20 -07:00
blue db4801d787 Gate the billing bridge on an allowlist: the service token is authorization
/v1/billing/* forwarded any subpath to commerce carrying the admin
COMMERCE_SERVICE_TOKEN, validated only against path traversal. Forwarding IS
authorization there: commerce gates money-mint on
MayMintMoney = IsServiceToken || IsSuperAdmin, so every forwarded path ran with
platform authority. Commerce 403s an org admin who POSTs /v1/billing/deposit
directly; through this bridge the same person was handed the platform's own
credential, and the subject-pinning aimed the credit at their own account.
Pinning is an IDOR control, not an authority control.

Add billingForwardable: a per-method allowlist of the endpoints the console
actually calls, enforced before the token is attached. An unlisted path 404s.
Allowlist, not denylist — a mint route commerce adds tomorrow is unreachable
with no change here. GET and POST are separate sets because `payouts` is a read
AND a mint-gated write; one method-blind set would hand the mint to every reader.
The POST set holds nothing that creates balance from a client-named amount.

Not exploitable in the current topology: commerce is co-resident in cloud, whose
build never compiles mount.go (//go:build cloud), so no api.Route billing bundle
is reachable and the forward loops back to this bridge. It is one COMMERCE_URL
away from live — devnet already runs a standalone commerce.

Tests: an ordinary org user's deposit/credit/refund/credit-grants/husd/allotment
now never reach commerce; the console's 12 real calls still forward; the store
bridge still cannot tunnel into billing.
2026-07-16 18:36:07 -07:00
hanzo-dev 337c159db0 Embed zen v1.4.0 so the prompt-cache fix reaches production
zen v1.4.0 bills the prompt cache: cache_read was priced in the catalog,
published in /v1/models, and never charged. cloud pinned v1.3.11, so the
fix could not reach api.hanzo.ai, where the traffic lands.

The module graph cannot move: zen's own go.mod is byte-identical between
v1.3.11 and v1.4.0 (same sha256), and cloud holds exactly one zen edge.

NOTE: this is necessary but NOT sufficient. hmoney.Minor() returns cents
while cloudmoney.FromInt expects atto, so apps/zen.go understates every
debit by 1e16 and the spend gate reads AmountCents 0. Correct pricing is
still zeroed downstream. Tracked separately; that fix is what makes cache
billing real.
2026-07-16 18:21:12 -07:00
hanzo-dev 20e7591f53 openapi: serve GET /v1/openapi.json generated from the live router
The route table gets a third projection. /zap replays the /v1 handlers, the
console renders them, and GET /v1/openapi.json now describes them — all read
from the ONE router after MountAll, so none can drift and none holds a second
copy. There is no checked-in spec file and no second registry.

openapi.Live(app) reads app.Fiber().GetRoutes(true) (fiber's own filter drops
Use() middleware); every other function is pure over that []Route. Each
operation is tagged with its product — the first path segment after /v1/ —
so a CLI can build `hanzo <product> <resource> <verb>` with no judgment.

Reading the LIVE router is the only total source: POST /v1/kms/auth/login is
registered as Group("/v1/kms/auth").Post("/login") and no grep can find it,
and the route set is a function of deployment config, so the document varies
per deployment — correctly. Unauthenticated: it grants no capability, every
route it names stays auth-gated, and `hanzo --help` must build its tree
before login.

The drift guard (cmd/cloud/openapi_test.go) is a bijection over the fully
mounted apps.Wire(): 983 operations / 692 paths / 109 products, every live
route present, no operation invented. Shown to fail on a broken translation
(353 routes reported missing) before being restored.

Honest boundaries, asserted rather than papered over:
  - No schemas. The router holds func(*zip.Ctx) error; the request type is a
    local inside the handler (var req secretPutRequest; json.Unmarshal(...)),
    unreachable by reflection. cloud.Handle[S] is generic over the SERVICE,
    not the payload. The path to schemas is zip's typed ops, which today
    number zero — which is why zip's own generator emits nothing here.
  - No responses block. OpenAPI 3.1 makes it optional; fabricating 200/ok on
    ~900 routes would assert what nothing knows.
  - HEAD/CONNECT excluded — forced, not taste. CONNECT has no OpenAPI field;
    fiber auto-generates HEAD in startupProcess(), so including it would make
    the document depend on lifecycle stage.
  - Catch-alls are opaque: POST /v1/billing/deposit is not a route here.

Corrects LLM.md, which the code contradicted: Wire() does exist (apps.go:188),
MountAll does not sort, and byte-identical patterns MERGE rather than panic.
A high handler count is not a collision — app.Post(path, mw1, mw2, mw3, h) is
one registration with four handlers (apps/commerce.go:151), and 34 live routes
are that shape, so the generator never reads the count.
2026-07-16 18:14:23 -07:00
antje 9d5d51936a cloud: mount hanzoai/agent /v1/agent (no-shim); delete dead clients/chat
clients/agent adapter injects the ai completion (in-process, billed) + tools.Default()
into github.com/hanzoai/agent; POST /v1/agent live path. Dispatch resolves the caller
via the ONE canonical tools.PrincipalFrom — no reconstructed principal. Builds green
on the v1.801.35 base. DEPLOY-BLOCKED: go.mod uses a local replace for hanzoai/agent
(CI needs a fetchable release + the local module-cache shallow-clone bug resolved).
2026-07-16 18:00:50 -07:00
hanzo-dev c5cefc39a6 Bill the prompt cache in production: zen v1.3.11 -> v1.4.0
cache_read was priced in the catalog and published in /v1/models, but never
charged: on an openai upstream every cached input token billed at the full
`in` rate, and on an anthropic one it left the bill entirely. zen v1.4.0
normalizes both dialects into one tally over three disjoint classes
(fresh + cached + cacheWrite == the whole prompt) and derives once per
response. cloud embeds zen via apps/zen.go, so the fix is inert at
api.hanzo.ai until this pin moves.

zen v1.4.0's go.mod is byte-identical to v1.3.11's, so the require edit and
the two go.sum lines are the whole change: no transitive pin moves, and
nothing else in the graph names zen.
2026-07-16 17:49:16 -07:00
hanzo-dev 7f6b3a4cfe Merge: fix the zen margin test's decimal + money types (unblocks CI on main) 2026-07-16 17:48:36 -07:00
z e538ae0b7f apps: price the zen test with the decimal money actually uses
The zen margin test never compiled, so CI has been red on main since it landed:
it parsed prices with shopspring/decimal and handed the result to hmoney.New,
which takes hanzoai/decimal. Two identically-named types, one of them wrong.

    vet: cannot use d (struct type "github.com/shopspring/decimal".Decimal)
         as "github.com/hanzoai/decimal".Decimal value in argument to hmoney.New

Money has exactly one decimal. A second one that merely LOOKS like it is how a
price silently becomes a different number, which is why the compiler is right to
refuse. Parse with hanzoai/decimal (decimal.Parse — the same call zen itself
prices with); this was the only file in cloud importing shopspring.

Read the debit through Amount.Int(), not Amount.Minor(). The test held two money
types at once: zen's hanzoai/money.Amount (Charge/Cost) has Minor(), but
meterUsage returns cloud's clients/money.Amount, whose accessor is Int() — it
wraps Minor() and returns the identical *big.Int, so the assertions are unchanged.

The tests themselves are worth keeping: they prove the debit is the retail Charge
and never the upstream COGS, that a 3x-margin tier collects 3x, and that a
sub-cent call does not floor to zero. They just could not run.

Not run locally: ./apps links a prebuilt Rust staticlib (libhanzo_flags.a) that
is built in CI, not here. vet — the step CI actually failed — passes, and the
package builds.
2026-07-16 17:48:26 -07:00
hanzo-dev 2593b1b18e GOPRIVATE names the namespace that is actually private
GOPRIVATE named zap-proto/*, which is public -- all 55 repos, all 6 modules
cloud needs served anonymously from the public proxy. The namespace that is
private went unnamed: github.com/hanzoai/* (467 private repos incl. ai,
account, commerce, orm, xorm, beego, csqlite). It resolved only by falling
through GOPROXY's direct fallback, and passed checksums only because go.sum
already pins everything.

containment.yml then set GOSUMDB=off to compensate. GOSUMDB does not scope to
a namespace: that disabled checksum verification for every module in the
build, public ones included, in the image that handles payments -- breaking
the invariant the Dockerfile three files away states and honors.

Public modules keep proxy and sumdb immutability; private ones go direct and
authenticated. Verified under the CI and Dockerfile env with -mod=readonly and
sumdb on: build 0, go mod verify all verified.
2026-07-16 17:33:16 -07:00
hanzo-devandantje 40ca6a48a8 feat(sync): universal /v1/sync engine — GitHub/GitLab ⇆ native Hanzo Git
One reconcile loop, one place a sync happens. clients/sync owns the Sync
record (source/target/direction/trigger/cursor), a per-org sqlite `sync`
table, /v1/sync CRUD + /v1/sync/:id/run, and a kind→Provider registry with a
single Reconcile(sync,event) contract. Git is the first provider, composing the
existing git object-plane seams (InboundGitSync inbound, ImportGitRepo reconcile,
EnsureGitMirror outbound) — no second copy of any git op.

Triggers resolve to Syncs and enqueue the engine (cloud.Sync), never sync
directly: the GitHub App webhook now serves the path it actually fires at
(/v1/github-webhook, was a prod 404 at /v1/integrations/github/webhook) and hands
the verified push to the engine; the Gitea push webhook gains a loop guard
(skip pusher == GIT_SYNC_ACTOR). Loops break on the engine cursor (identical
fingerprints are a no-op) + actor guard; chained propagation (a sync's target is
another's source) is bounded by a hop limit.

Seams (root cloud): SyncFunc + RegisterSync/Sync, GitMirrorController +
EnsureGitMirror. Fail-closed on unmounted engine / missing secret. Tests
(CGO_ENABLED=0): engine loop-guard/idempotency/chain/hop-limit, git resolve,
CRUD+patch+run, webhook signature/isolation/enqueue, gitea loop guard.
2026-07-16 17:26:06 -07:00
hanzo-dev 2cc52957b4 build: GOPRIVATE names the namespace that is actually private
GOPRIVATE listed github.com/zap-proto/*. Every zap-proto repo is public — all 55
of them — and every zap-proto module this build needs is served by the public
proxy anonymously. It was never the reason anything resolved direct.

The namespace that IS private went unnamed: github.com/hanzoai/* — ai, account,
commerce, orm, xorm, beego, csqlite and ~30 more. Those only ever built by
falling through GOPROXY's `direct` fallback after the proxy 404'd them, and only
kept passing the checksum step because go.sum already pins them, so no sumdb
lookup happens. It worked by accident, one added dependency away from failing.

containment.yml compensated for that unnamed namespace with GOSUMDB=off, which
does not scope to hanzoai — it disables checksum verification for EVERY module in
the build, the public majority included. The Dockerfile's comment reasoned the
same way inverted ("hanzoai/* and luxfi/* are PUBLIC ... only zap-proto/* is
exempt"); hanzoai/* is largely private and luxfi/* (all 37 deps here) is public.

Naming github.com/hanzoai/* is what the off switch was standing in for. GOPRIVATE
implies GONOPROXY+GONOSUMDB for exactly that namespace, so the private modules go
direct+authenticated and skip the sumdb that cannot see them, while zap-proto and
luxfi keep the public proxy + checksum db — the immutable hashes that make a
force-moved tag unable to break or poison the build. GONOSUMDB and GOSUMDB=off
are dropped: scoped by GOPRIVATE, they are redundant, and blanket-off is a
supply-chain regression in a money image.

hanzoiam/* is not listed: a74b7de reverted the scim/saml/ldap embed, so nothing
in go.mod requires it. It goes back when the modules resolve, named as private.

Verified under the exact CI/Dockerfile env (GOPRIVATE=github.com/hanzoai/* only,
GOSUMDB=sum.golang.org, GOFLAGS=-mod=readonly): build exit 0, vet exit 0,
`go mod verify` = all modules verified, and `go mod download` resolves both a
public module (zap-proto/zip) and a private one (hanzoai/ai).
2026-07-16 17:21:31 -07:00
antje 28aaa39d56 deps: ai v1.816.0 -> v1.817.0 — global auto-routing switch is the '*' settings row, not ROUTER_ENABLED env (config-as-Base)
Runtime routing policy now sources from the GlobalDefaultOwner OrgSettings
row (admin.hanzo.ai-editable SQLite), env demoted to deprecated fallback.
Per-org override unchanged.
2026-07-16 17:16:33 -07:00
hanzo-dev dfe30a3883 paas: read the kind the fleet runs on, and don't fight the git declarer
The PaaS control plane serves platform.hanzo.ai and read `services` only. The
fleet collapsed onto `kind: App` and this reader never followed: 69 App CRs run
across the scanned namespaces against 7 Service CRs, so the SUPERADMIN drift
board rendered 7 rows for a 69-app fleet — 1 row in `hanzo`, production, and that
row is a duplicate CR. It was not an error anyone could see. The board looked
plausible and was blind to 68 of 69 services; every read of an App-declared
service 404'd; and /v1/paas/health probed the Service CRD, found it served, and
reported ok — status theater over a blind board. clients/deploy already solved
this with an App-first/Service-second read order; this gives the same order to
the reader that needed it.

Both kinds are listed and deduped by name: one workload is one row even when an
App CR and a Service CR both claim the name, because a Deployment has one
controller ownerRef and the operator's Claim guard gives it to the App.

The write path is the harder half. Hanzo CD syncs 68 of the 69 App CRs from
universe `infra/k8s/operator/crs/` with selfHeal on, so patching an App CR here
is reverted on the next sync — the deploy would report success and silently roll
back. That is worse than refusing, so deploy and release now resolve which kind
holds the workload and refuse a git-declared one, naming the file to commit to.
A Service CR has no git declarer (cloud and kubectl write them directly), so it
still patches exactly as before — the tenant plane and the untransitioned CRs are
untouched. release.go's "no ArgoCD" claim was true when written and is not now.

This does not restore an admin deploy button for the git-declared fleet. Under
GitOps that button belongs in git, and ImageUpdate already names that seam
(registry→git→cluster). Whether /v1/paas/deploy should commit to universe or be
retired is a CTO call, not one to make silently inside a read fix.

Tests: 35 pass, 0 fail, including the pre-existing release suite. The fleet-sees-
App-CRs case is seeded to the shape of the real `hanzo` namespace.
2026-07-16 17:12:40 -07:00
antje a74b7de5c4 revert(iam2): back out scim/saml/ldap embed — hanzoiam repos do not resolve
Reverts e4b3c88f. go.mod pinned github.com/hanzoiam/{ldap,saml,scim}, but none
of the three repos exist: https, ssh, and API all return not found, and ldap
was never fetched by any proxy or cache. Cold machines — including release
runners — fail go mod download, so main could neither build nor release.
Local builds passed only on module caches warmed 21:36-21:48 UTC today.

iam2 restored to v0.1.1, the state prod v1.801.38 ships. Re-land the embed
unchanged once the hanzoiam repos exist and resolve from a cold GOMODCACHE.
2026-07-16 17:01:45 -07:00
hanzo-dev f662973eb2 Merge: one name for the account, one for the org — and the last copy of the payer rule deleted 2026-07-16 16:38:08 -07:00
z e72d77b21b billing: one name for the account, one name for the org
Two functions were called Payer and returned different things: account.Payer
returns the ACCOUNT that pays, principal.Payer returned the ORG whose ledger
holds it. Those are different values on the same request — a person in the shared
signup org pays from account "hanzo/alice" held in ledger "hanzo" — and one name
for both is how the gate came to key the pool while the debit spent the person.
Rename it to what it returns: principal.HomeOrg. An org names a ledger; an
account names a wallet within it.

Fix the last copy of the rule with it. clients/metering is cloud's vendored
metering client, and its IdentityFromGatewayHeaders still hardcoded `user := org`
under the same false premise, while its doc claimed cloud "mirrors" the module
"exactly so every product keys the SAME ledger entry" — a promise two independent
copies cannot keep. Both now call hanzoai/account.Payer, so they agree by
construction and the comment is true for a reason.

Its test asserted the divergence in words — "want hanzo (per-org billing key, not
org/sub)" — which is how a premise outlives the code that disproved it. It now
asserts equality with the rule.
2026-07-16 16:37:43 -07:00
hanzo-devandz b2c5b4f81e build(cloud): mirror base images to ghcr.io/hanzoai/mirror/* (kill public.ecr.aws 429s)
public.ecr.aws (ECR Public) rate-limits anonymous pulls with HTTP 429 on shared
CI runners; a 429 on any base pull aborts the release (a release died pulling
python:3.12-alpine). Repoint all five FROMs to 1:1 linux/amd64 mirrors in our
own GHCR namespace, pinned by digest for immutability:

  node    ghcr.io/hanzoai/mirror/node:24-alpine@sha256:0cb0e7c3195bce740b6c8d8b27432c92360e3b7f1528087f2c50640b177950c6
  python  ghcr.io/hanzoai/mirror/python:3.12-alpine@sha256:aa679aa4eed6eb56c1dc6ad3f1b98b7d2d788fd961596779d188fdedad97fb38
  rust    ghcr.io/hanzoai/mirror/rust:1-alpine3.22@sha256:b348cb409ac0a73de15065997a360063cf87465574a15e3e4469862cb8996f02
  golang  ghcr.io/hanzoai/mirror/golang:1.26-alpine3.22@sha256:47d47cb5cc3c7dac409dcb6c3a98a6263571218046cd02d709527feef804a77c
  alpine  ghcr.io/hanzoai/mirror/alpine:3.22@sha256:7c8cb692ae09657cbc4a3f3cbd0e8d5a2690ba38386aaaf252dbb060bf5eb2e6

The four the task named plus alpine:3.22 (the final-stage base — same registry,
same 429 exposure) so no FROM still hits public.ecr.aws. release.yml already
logs the build into ghcr.io (GH_PAT, docker/login-action) before building, so
buildx resolves these private mirrors today with no new plumbing. Only the five
FROM lines + one rationale comment change; nothing else in the Dockerfile.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-16 15:48:48 -07:00
hanzo-dev 6c77cfccab feat(ai): bump ai v1.814.0 → v1.816.0 — router self-probe, flywheel trainer, casibase metering
Ships the full router flywheel into the cloud binary: the self-probe
(continuous tagged auto traffic → reward ledger), the fit→gate→auto-deploy
→publish trainer, the routing-latency guard (~0.68us heuristic), and
casibase-chat usage/o11y metering. Also carries the zen warehouse+span
wiring already on main.

Migrates the billing-subject callers (balance.go, account/billing.go +
tests) from the ai/object.Payer that v1.816.0 REMOVED to the extracted
github.com/hanzoai/account.Payer — the same rule in its new home (the
concurrent decomplect). Same replace directive for the force-pushed iam
pseudo-version account v0.2.0 pins.

Claude-Session: https://claude.ai/code/session_018PmFAHZvbBSTsuWyebwMra
2026-07-16 15:32:35 -07:00
hanzo-dev 34800f6991 ci(cloud): GOPRIVATE += github.com/hanzoiam/* (enterprise IAM modules)
The iam2 embed pulls github.com/hanzoiam/{scim,saml} (+ ldap under -tags iam2_ldap).
hanzoiam is a distinct org from hanzoai, so hanzoai/* did NOT cover it — the test
phase would route these private modules through the public proxy/sumdb and 404.
Git auth is already handled by the reusable CI's GH_PAT insteadOf (covers any
github.com private repo the PAT reads).
2026-07-16 15:04:30 -07:00
zeekay e4b3c88f82 feat(iam2): wire SCIM + SAML into the cloud embed; LDAP GPL-isolated behind -tags iam2_ldap
Last code step of the iam2 migration. clients/iam2 blank-imports the Apache-2.0
enterprise features so their init() self-registers into iam2's feature registry;
iam2server.Mount -> feature.MountAll auto-mounts them under CLOUD_IAM_IMPL=iam2.
ONE mechanism (database/sql driver pattern) — no feature.Register call in cloud,
which (feature.Register appends with no dedup) would double-mount and collide routes.

  _ github.com/hanzoiam/scim   SCIM 2.0 provisioning  (/scim/*)
  _ github.com/hanzoiam/saml   SAML IdP + SP SSO      (/v1/iam/saml/*, /v1/iam/acs, /v1/iam/get-saml-login)

LDAP is GPL-isolated: hanzoiam/ldap links goldap (GPL-2.0), so it is opt-in only via
clients/iam2/ldap_enabled.go (//go:build iam2_ldap). The DEFAULT cloud binary stays
copyleft-free — proven: `go list -deps ./clients/iam2/` has no goldap; only under
-tags iam2_ldap does it pull github.com/lor00x/goldap + hanzoai/ldapserver.

deps: hanzoai/iam2 v0.1.1->v0.1.4; + hanzoiam/{scim,saml,ldap}; go mod tidy.
2026-07-16 14:59:47 -07:00
hanzo-devandz 6a47436062 feat(kms): one-time legacy ZapDB -> per-org SQLite migration (deploy-safe cutover)
Without this, deploying the per-org store over live data boots an EMPTY store and
orphans every secret — cloud KMS is the authoritative source the kms-operator syncs
out to every service, so that is a cluster-wide outage. New() now runs a one-time,
writer-only, keyed, FATAL-on-error migration BEFORE serving: it opens the legacy
{DataDir}/kms ZapDB (encrypted at rest with the master key), streams every SEALED
secret (kms/secrets/ prefix; the JSON value carries the full coordinate + AES-GCM
ciphertext + ML-KEM wrapped DEK) verbatim into its per-org SQLite file via store.put
— NEVER unsealing, so no plaintext is exposed and the AAD path-binding survives —
then archives the legacy dir to {DataDir}/kms.migrated so the OS lock is released for
good. Idempotent (prior .migrated marker or absent store = no-op; put upserts).

Tests: migrate roundtrip (opens to same plaintext), no-legacy-store no-op, and the
cross-org relocation defense still holds after migration. Full kms suite green (2
pre-existing admin-edge fails only).
2026-07-16 14:55:16 -07:00
hanzo-devandz b31c642959 feat(kms): per-org SQLite store — replaces OS-locked embedded ZapDB, lifts replicas=1
The embedded luxfi/kms ZapDB (a Badger fork) held an exclusive OS lock on ONE dir
for ALL orgs — the hard reason cloud ran replicas=1 behind a cross-process writer
lease. This replaces it with per-org encrypted SQLite ({DataDir}/orgs/{org}/kms.db
via cloud.OrgDB->cek, per-db DEK), which has no exclusive-opener lock, so distinct
tenants never contend and different pods can serve different tenants.

Crypto stays in the client (Seal/Open AES-256-GCM envelope, AAD-bound to the FULL
/orgs/{org} path): plaintext never reaches the file, and a record physically moved
into another org's file still fails to Open (cross-org swap defense preserved).
Adds cloud.PlatformDB for the reserved non-tenant (_platform) partition.

Verified: 54 PASS / 2 FAIL in clients/kms; the 2 fails (admin-edge dualmount/authz)
are PRE-EXISTING — proven failing identically on origin/main. concurrent_open_probe
proves per-org SQLite has no exclusive lock. Build + finance (pure-Go) green.

Branch only — NOT for main/deploy until red review passes.
2026-07-16 14:55:16 -07:00
zandhanzo-dev 2ac7f835e2 billing: the edge gate keys the account the debit spends
The gate keyed `user := home` — the org pool, always — on the premise that
prepaid billing is per-org. That premise is false. A person in the shared signup
org holds their OWN account: its members are strangers, not a team, and a shared
org is not a shared wallet. That is what IAM's signed billing_account claim states
and what ai's meter debits.

So the gate authorized against a balance nobody drained. Fund the pool and a
signup-org person still 402s, because their usage comes out of their own account;
fund the person and an empty pool blocks them anyway. Two layers, two answers, one
request.

Resolve through hanzoai/account.Payer — the same function ai debits with, on the
same credential — so the gate and the debit cannot name different accounts. The
premise is removed rather than restated.

The masquerade split is preserved by construction, not by care: the account is
resolved WITHIN the home org, so Account.Org IS the home org and a SuperAdmin
acting in another org still bills their own ledger. A claim naming a foreign
ledger is refused, so it cannot redirect a debit into the org being acted on.

Tests assert against the rule rather than a constant, so they cannot drift the way
the premise did: the signup-org person keys their own account, a real org pools,
the claim wins for a person and a project, and every case is checked equal to what
the debit computes.
2026-07-16 14:42:34 -07:00
hanzo-dev c563d8a4a6 merge: iam2 embed subsystem — CLOUD_IAM_IMPL selects beego (default) or iam2
clients/iam2 mounts the clean-room zip+orm IAM at /v1/iam when CLOUD_IAM_IMPL=iam2,
else beego, byte-for-byte unchanged. Flag OFF by default → inert. iam2 v0.1.1 seam.
2026-07-16 14:38:03 -07:00
hanzo-dev 724d011417 feat(iam2): select /v1/iam impl via CLOUD_IAM_IMPL (default beego, unchanged)
apps.Wire's identity slot now calls identitySpec(): CLOUD_IAM_IMPL=iam2 mounts the clean-room iam2 (zip+orm), anything else — including unset, the production default — keeps the legacy beego Casdoor embed byte-for-byte. The two impls own the SAME absolute prefixes (/v1/iam/*, /login/oauth/*) and cannot co-mount, so exactly one occupies the slot per boot and mount order is preserved. Off by default => completely inert until a canary flips the flag; selection (this) stays orthogonal to activation (cfg.Enabled).
2026-07-16 14:32:15 -07:00
hanzo-dev 53e22f0875 feat(iam2): clean-room IAM v2 embed subsystem (zip+orm, beego-free)
clients/iam2 folds the beego-free Hanzo IAM v2 into the unified cloud binary as an in-process identity plane — the either/or twin of clients/iam. Matches the cloud.Typed contract func(*zip.App, cloud.Deps) error: cloud hands subsystems a cloud.Deps (not an orm.DB), so Mount opens its OWN embedded SQLite ({DataDir}/iam2/iam.db, mirroring the beego embed's {DataDir}/iam layout), seeds config new-only+idempotent from the SAME init_data.json the beego iam uses (non-fatal, honest degrade), then iam2server.Mount registers the whole surface at the canonical absolute paths.

Fail-closed like clients/iam: a store-open or mount failure serves 503 on the identity prefixes while every co-resident subsystem stays up; iam2server.Mount's only panic path (a registered enterprise feature) is recovered in safeMount so it never crashes the shared binary. A TODO marks where the parallel-lane hanzoiam/{scim,saml,ldap} feature.Register lines land.

Pins github.com/hanzoai/iam2 v0.1.1 (seam held stable across the parallel internals refactor); transitive MVS bumps are all patch-level within v1.x (zip 1.8.3, orm promoted to direct, luxfi/crypto 1.20.1, argon2id 1.0.0, pgx 5.9.2). Inert until wired — see the apps.Wire gating follow-up.
2026-07-16 14:32:06 -07:00
hanzo-dev 7b63b49324 fix(ci): bound the SECOND version scan too (compute step) — same unbounded --paginate
The compute-next-version step had the same full-registry --paginate as the tag
step (fixed in f0abd21). It ran first, so it could hang before the build. Bound
it to one page too. Both version scans are now O(1 page), not O(registry).
2026-07-16 14:28:00 -07:00
hanzo-dev 0494a6d3c7 feat(zen): warehouse + gen_ai span emission with exact margin — ai v1.814.0
zen's commerce Meter now also calls ai's TraceServedUsage (recordTrace
WITHOUT recordUsage — the commerce debit stays the ONE billing source,
never doubled), carrying zen's exact per-tier retail (Charge) and upstream
COGS (Cost) folded atto→nano, so zen* traffic in the unified binary lands
in hanzo.cloud_usage + the o11y span plane with TRUE margin instead of
being warehouse-blind. Rides the ai v1.813.1→v1.814.0 bump; balance.go
(+ its drift-guard test) migrated to the renamed Payer/PayerOf API —
same subjects, one rule.

Claude-Session: https://claude.ai/code/session_018PmFAHZvbBSTsuWyebwMra
2026-07-16 14:26:17 -07:00
zandGitHub 27cc091884 Merge pull request #315 from hanzoai/cloud-zen-family-events
feat(zen): embedded-zen meter writes the family RoutingEvent (last link) + ai v1.813.5/zen v1.3.11
2026-07-16 14:08:50 -07:00
hanzo-dev 64b0d1f8a1 feat(zen): embedded-zen meter writes the family RoutingEvent (the last link)
The embedded zen mount serves the zen catalog in-process and never reaches ai's
pipeToFamily, so zen* calls produced ZERO routing events — starving stats, world,
spark retrain, and /v1/feedback joins. Wire cloud's zen Meter to ALSO write the
RoutingEvent through the ONE shared writer object.RecordFamilyRouting (source="family",
served arm = zen.Usage.Upstream, join key = zen.Usage.ResponseID — the client-visible
response id, new in zen v1.3.11 — tokens + retail cost), fire-and-forget beside the
existing debit. Bumps zen v1.3.7 → v1.3.11 (Usage.ResponseID); ai already v1.813.6
carries object.RecordFamilyRouting.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-16 14:08:28 -07:00
antje 54b8bae797 git: Gitea push-webhook ingest (POST /v1/git/webhook)
The external Hanzo Git server (Gitea fork, git.hanzo.ai) POSTs push events
here so a push landing on it drives the SAME push-to-deploy core the embedded
smart-HTTP receive-pack path drives: fireBranchBuild -> cloud.OnGitPush deploy
trigger + EmitLifecycle. One code path, no duplication.

HMAC auth: X-Gitea-Signature is hex HMAC-SHA256 of the raw body, verified
constant-time against GIT_WEBHOOK_SECRET (KMS-synced hanzo/prod:/git/webhook-secret).
Fail-closed 401 on unset secret or mismatch. Only X-Gitea-Event: push acts;
others 204. Zero-SHA / non-branch refs are no-ops.
2026-07-16 12:05:08 -07:00
zandGitHub 5fb8f50e0a Merge decomplect/account-payer: migrate to ai.Payer — unbreaks main, ends the self-serve 402
main pinned ai v1.813.6 (which deletes billing_subject.go for the Payer/Account
refactor) without migrating the call sites, so main did not compile:
  clients/billing/balance.go:59: undefined: aiobject.BillingSubject

This migrates the call sites to the ONE rule: Payer(Credential)->Account
(owner = Person|Org|Project). Ends the live 402 where a self-serve signup's top-up
minted to the shared org pool 'hanzo' while the gate debited 'hanzo/alice' ($0) --
customer paid, locked out, money in a pool their org-mates could spend.

Green: go build -tags 'libsqlite3 sqlite_fts5' ./clients/... rc=0;
go test ./clients/billing ./clients/account rc=0. go.mod/go.sum identical to main.
2026-07-16 11:51:27 -07:00
hanzo-dev bd520de13f fix(billing): migrate off deleted BillingSubject → ai.Payer/PayerOf — unbreaks main
main pins ai v1.813.6, which contains the Payer/Account refactor (billing_subject.go
and its PERSONAL_BILLING_ORGS/ORG_BILLING_ORGS lying default are deleted). But the
call sites were never migrated, so main does not compile:
  clients/billing/balance.go:59: undefined: aiobject.BillingSubject
  clients/billing/balance.go:61: undefined: aiobject.BillingSubjectFromUserKey

Migrates balance.go + finance.go + billing.go to the ONE rule —
Payer(Credential{Owner,Name}).Subject() / PayerOf(org,key).Subject() — and DRYs the
duplicate subject resolvers. This is the cloud half of the fix that ends the live
self-serve 402 (top-up minted to the shared org pool 'hanzo' while the gate debited
'hanzo/alice' = $0).

go.mod/go.sum untouched vs main. Green: go build -tags 'libsqlite3 sqlite_fts5'
./clients/... rc=0; go test ./clients/billing ./clients/account rc=0.
2026-07-16 11:50:51 -07:00
hanzo-dev 65e782a855 refactor(billing): route top-up + console subject through ai.Payer, one rule
The console top-up (clients/account/billing.go) and the finance read
(clients/billing/finance.go) each re-implemented the billing subject as "always
the org" — the reverted lineage's rule. Against ai's gate, which bills a signup
person per-person, that is the split: money minted to subject "hanzo" (the pool)
while the gate debited "hanzo/alice" ($0) → the paid-up member 402'd.

Delete both twins; resolve the subject through the ONE rule, ai/object.Payer,
keyed on the IAM username (X-User-Name) the gate also keys on. Top-up credits and
console reads now land on the SAME account the gate debits — they cannot drift
because there is one function. The killed PERSONAL_BILLING_ORGS / ORG_BILLING_ORGS
env is inert here too (test proves hostile values change nothing).

Requires ai v1.809.5, which must be tagged FROM ai main after decomplect/account-payer
merges — NOT off a branch. The prior one-rule fix was tagged off an unmerged branch
(v1.806.8/.9), main never got it, and every later tag resurrected the allowlists;
that is why this bug is live. go.sum refreshes via `go mod tidy` once the tag exists.
Verified locally via a replace to the ai branch: clients build + twin tests green.
2026-07-16 11:47:10 -07:00
antje 74d94d1f3e ci(release): retire notify-universe — Hanzo CD owns git→cluster sync
The repository_dispatch deploy hub is gone (universe image-update.yml
removed in universe d07cf945; the dispatches were silently suppressed by
the flagged sender account regardless). Deploys are declared-tag bumps in
universe crs/, synced by Hanzo CD (ArgoCD, ns hanzo-cd) and reconciled by
the operator. [skip ci]
2026-07-16 11:36:10 -07:00
antjeandGitHub ce6c4dc374 deps: hanzoai/ai v1.813.1 -> v1.813.6 (balance-exempt routing config + enso family helper) (#314)
ai#102: /v1/get-routing-defaults + org-settings CRUD + routing-ledger
export are configuration metadata, never wallet-gated — unblocks reading
org routing defaults for $0-balance orgs and the operator platform flip.
Edge auto-routing billing tests green.
2026-07-16 11:28:33 -07:00
antje 2134876826 platform: allow the self-hosted fleet registry in the native build lane
registry.hanzo.ai/{hanzoai,luxfi,zooai}/ join the /v1/runner push allowlist —
Wave 0 of the native CI/CD migration. Until now only release.yml's crane
mirror could reach the fleet registry; the native BuildKit lane was
ghcr-only by policy.
2026-07-16 11:13:07 -07:00
hanzo-dev f0abd2116e fix(ci): version-assignment scanned the WHOLE registry — livelocked releases
The "atomic free-version assignment" step paginated every container version
(`gh api --paginate .../versions`) to find the max release number. As the
registry accumulated tags this grew unbounded and hung the step for 30+ min,
livelocking every cloud release. Container versions are created newest-first and
version tags are monotonic, so the max is always on the newest page — query one
bounded page (?per_page=100) instead of the full history. Fast + correct.
2026-07-16 11:11:39 -07:00
hanzo-dev ebf267908c refactor(ledger): drop redundant "core" — the ledger adapter is apps.ledger
Values, not places: the finance-backed credit-ledger adapter is qualified by its
namespace (apps.ledger), not a braided ledgercoreCredit compound. Rename the type
ledgercoreCredit → ledger, the file commerce_ledger.go → ledger.go, and scrub
"ledgercore" from prose (the ledger IS the core — "core" adds nothing). No behavior
change; admin/core tests green, changed packages build.
2026-07-16 10:18:47 -07:00
antje 4a64ef568b feat(chat): clients/chat — one /v1/chat tool-calling orchestrator
POST /v1/chat runs one LLM tool-calling round that lets a model manage a system
via tools. Composes existing cloud pieces, reinventing nothing:
- LLM routing + per-org reserve/settle billing: the ai subsystem's
  /v1/chat/completions, invoked in-process (Fiber Test) — the only path that
  both returns tool_calls AND carries the billing gate.
- tool plane (clients/tools): the org's registered MCP/registry tools are
  offered to the model and dispatched server-side (activation + price gated).
- capabilities: graph (advisory node-ops -> ops the client applies) and create
  (server-executed, tools = the org's registered MCP render services).
Returns {reply, actions, ops}. chat mounts before ai so /v1/chat resolves here
(the ai /v1/chat alias is shadowed); ai keeps /v1/chat/completions.
2026-07-16 10:02:29 -07:00
hanzo-devandantje 73632723f6 feat(ai): bump ai subsystem v1.813.0 → v1.813.1
Carries the PAID-Enso revert + the family learning loop: per-family-call RoutingEvents
+ shadow A/B (records what the learned engine would have picked), /v1/feedback signal
contract (up/down/regenerate/switch/abandon/accept/revert/rating/dismiss) with the
online reward forward to the engine's /route/observe, ROUTER_ADMIN_TOKEN service-auth
on the training-data exports, and Zen/Enso provider branding. Lights up /v1/router/stats
+ world.hanzo.ai (shadow-vs-served agreement).

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-16 09:53:56 -07:00
hanzo-dev f5d57e35f1 feat(ai): bump ai subsystem v1.812.0 → v1.813.0 — router policy, rewards, margin, DO backfill
Ships the per-org router policy (get/update-router-policy + the org > '*' >
conf fold on every auto route), the routing-reward ledger, the nano margin
ledger (costNano/billedNano/marginNano + unpriced flagging), and the
gaps-only DigitalOcean usage backfill (POST /v1/admin/usage/backfill-do,
dry-run default). The release image also re-embeds console@main
(CONSOLE_REF=main), picking up the console Router page (v8.4.137+).

Claude-Session: https://claude.ai/code/session_018PmFAHZvbBSTsuWyebwMra
2026-07-16 09:28:20 -07:00
hanzo-dev 8f61d57963 feat(credit): ONE ledger seam — commerce credit + admin grant mint into finance.Current()
The last mile of "one way to grant credit": cloud implements commerce's
creditledger.CreditLedger (v1.48.5) over the native finance ledger and injects it
at mountCommerce (EmbedConfig.Ledger). Now commerce's POST /v1/billing/credit AND
the admin.hanzo.ai grant (/v1/admin/customers/:org/credit) both mint into the SAME
per-org finance wallet the ai prepaid gate reads — a granted credit is immediately
spendable, no split ledger. The admin path drops its parallel finance.Deposit for
the one creditledger.Credit call (idempotency key rides through; commerce HTTP
deposit remains the split-deploy fallback).

- apps/commerce_ledger.go: ledgercoreCredit adapter (compile-time asserted against
  commerce's exported interface); org-pool wallet (Subject==Org); fails closed with
  no co-resident finance.
- apps/commerce.go: inject Ledger: ledgercoreCredit{} at mountCommerce.
- clients/admin/core/grant.go: grantDeposit prefers creditledger.Get() (the one
  ledger) over its own finance.Deposit.

Verified: changed pkgs build green on commerce v1.48.5 + ai v1.812.0; admin/core
grant tests pass. Ships with the ai free-tier removal (v1.811.0+) already on main.
2026-07-16 09:14:59 -07:00
antje 5bfe2842a1 ci(release): pass GIT_AUTH_TOKEN build secret — Dockerfile renamed the id from gh_token (46081689) but the workflow still passed gh_token, so the private console clone ran credentialless and the release failed 2026-07-16 01:50:00 -07:00
antje 4608168991 build: consume the standard GIT_AUTH_TOKEN build secret (was gh_token)
One secret id everywhere: BuildKit's gitsource convention (GIT_AUTH_TOKEN,
which the fabric's buildFrontendCmd already attaches) is also what the
Dockerfile's console-embed and go-mod fetch stages mount. gh_token was a
second name for the same credential.
2026-07-16 01:19:03 -07:00
antje a74c95abc2 deps: hanzoai/ai v1.812.0 + commerce v1.48.5 — reward ledger, router policy, connections import reach prod
ai v1.809.4 → v1.812.0: RoutingEvent reward ledger (/v1/add-routing-reward,
/v1/export-routing-rewards), per-org router policy + observability, Enso
public arms + retrain-status, connected-provider usage import (OpenAI/
Anthropic), own-brand god-view gate (TokenIsOwnBrand), Bearer-aware
get-cloud-usages, billing nano margin ledger, unpriced-model flagging.

commerce v1.48.2 → v1.48.5: mint-gated org-keyed credit primitive +
injected-ledger seam; fixes the phantom luxfi/cevm test import that broke
go mod tidy for every consumer (this repo included).
2026-07-16 01:16:12 -07:00
antje b7e4843cd7 feat(fabric): GIT_AUTH_TOKEN build secret — private-repo fetches for the build Jobs
The buildkit Job now surfaces the console-git-token Secret (optional) as the
GIT_AUTH_TOKEN env, and buildctl attaches it as the same-named build secret;
BuildKit's gitsource presents it as the HTTPS credential for the git context.
Fixes the fabric's inability to build PRIVATE repos (github.com/hanzoai/cloud
itself: 'could not read Username' — the rel_HvInGKU failure). Public repos and
Secret-less clusters fetch anonymously exactly as before.
2026-07-16 01:15:07 -07:00
antje 9b97ca9164 feat(git): public repos — anonymous read for the build fabric
A repo gains a visibility bit: PATCH /v1/git/repos/:name {"public":true}
(or create-time "public"). Public grants ANONYMOUS upload-pack/info-refs
only — receive-pack and the whole control plane stay org-authed, and a
private or missing repo answers the same uniform 404 so anonymous probing
cannot enumerate.

This is what lets the credential-less buildkit build Job (launchDirectBuild
git context) fetch from the embedded git server — the same reason public
GitHub repos build with no env. Private-repo builds remain a later
GIT_AUTH_TOKEN feature.

resolvePackRepo grows the allowPublic branch (fetch-side only): anonymous
callers address org-level repos by the orgRE-validated :org path segment;
the path-vs-identity guard for authenticated callers is unchanged.
2026-07-16 01:05:30 -07:00
antjeandGitHub 4b255e8e9a test(platform): end-to-end git push -> buildFromPush enqueue proof (#311)
Wire a REAL smart-HTTP git push through the actual production seam
(cloud.RegisterPushBuilder <-> cloud.OnGitPush) into the real platform
push builder, and assert the matching app's build is enqueued (a
building deployment for the pushed commit). The two halves were covered
in isolation (clients/git TestPushFiresBuildTrigger; platform
TestBuildFromPush_LaunchesMatchingApp) but nothing connected a live push
to the real builder end to end. Pure-Go dev build (CGO_ENABLED=0).
2026-07-16 00:45:07 -07:00
hanzo-dev 41be25c713 Merge land/slack-on-bridge: Slack chat path onto the generalized bridge — one chat code path 2026-07-16 00:15:57 -07:00
hanzo-dev 301706dd54 refactor(integrations): Slack chat path onto the generalized bridge — one chat code path 2026-07-16 00:13:59 -07:00
zandGitHub 746b42209f Merge feat/bots-control-plane: decomplect the bot plane, proxy instead of duplicate
Deletes clients/bot — a place-name that had collected three concerns (a mount, a
transport, and two unrelated domain wire protocols). One value per module:
  run       -> clients/bots      /v1/bots (no store)
  dispatch  -> clients/coding    /v1/coding-tasks
  transport -> clients/runtime   address, identity, framing, cleartext policy
  machine   -> clients/visor     /v1/compute/bots (moved off /v1/bots)
  session   -> clients/agents    the one registry (reverted to main)

Fixes a live route collision: visor's listBots silently won GET /v1/bots (the
router merges byte-identical patterns, first wins), so cloud served machine rows
as runs and orgs' real runs were invisible.

POST /v1/bots/run now returns 501 instead of charging $1.00 for a bot that never
booted -- the runtime has no launch endpoint. Stop fails closed: a bare 404 is
502, only a structured error body means already-stopped.

Net -613 lines. runtime never imports bots/coding; the reverse is a compile-error
cycle, so the one-way dependency is structurally enforced.
2026-07-15 23:27:48 -07:00
zandGitHub 58d39546b1 Merge fix/billing-balance-collision: read prepaid balance from the finance ledger, not a self-call
Cloud's /v1/billing/balance proxy was the only handler registered (commerce's
GetBalance is behind //go:build cloud, never compiled in) so it called itself,
depth 2. Reads the finance ledger via the existing finance.Current() seam and
single-sources the subject from aiobject.BillingSubject so console and gate
cannot drift.
2026-07-15 23:27:02 -07:00
zandGitHub a7fcf77247 Merge pull request #310 from hanzoai/bump/ai-v1.809.4
chore(ai): bump embedded ai v1.809.3 -> v1.809.4 (enso comp cold-cache fix)
2026-07-15 23:07:00 -07:00
hanzo-dev e324fdb236 chore(ai): bump embedded hanzoai/ai v1.809.3 -> v1.809.4 (enso comp cold-cache fix)
Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-15 23:06:41 -07:00
hanzo-dev d678ba2f6b bots: proxy the runtime instead of copying its state; name the transport
Red found the substrate was wrong: cloud kept its own registry of runs. It minted
an id the runtime had never heard of, so /v1/bots listed runs that did not exist,
stop closed records for runs that were never started, and run charged $1.00 for a
bot that never booted — the runtime has no launch operation at all, and nothing
in run ever contacted it. The registry that exists is the runtime's own tenant
store; it is the only thing that knows whether a sandbox is alive.

So cloud owns policy and the runtime owns the run:
  - GET /v1/bots and POST /v1/bots/:runId/stop proxy the runtime, gated by the
    validated principal and org. The org is cloud's, never the client's, and the
    runtime keys every run under tenants/{org}/ — so a foreign run resolves under
    the caller's org, where it does not exist, and 404s.
  - POST /v1/bots/run returns 501. There is no launch operation to call, so the
    honest answer is that it is not implemented. It no longer charges.
  - The agents session plane is reverted verbatim: no surface column, no Agent
    filter, no SessionOpen, no in-process stop. It never should have carried a
    second copy of the runtime's state.

Absence is only meaningful from a callee that could have said otherwise, so the
transport separates ErrNotFound (the operation ANSWERED absent) from ErrNotServed
(no such operation). A runtime without the stop route reports absent for every
run; treating that as "already stopped" made a stop that cannot fail. It is 502.

Decomplect the transport: clients/bot was named for the host it dials and braided
three concerns — an ops face plus two unrelated domain protocols. It is now
clients/runtime, a domain-free transport (address, identity, framing, cleartext
policy, /v1/bot/* ops face) that must not import bots/coding. Each domain owns its
own wire stub (bots/wire.go, coding/task.go), so the ZAP swap per HIP-0106/0120 is
a seam swap. Wire spec bot -> runtime; cmd/bot -> cmd/runtime.

The duplicate-route guard was a no-op: the router MERGES byte-identical patterns
into one route with chained handlers, so counting entries never saw the collision
it guarded. It now asserts one handler per route, and a test proves it fires on
the original bug. Fiber's Test() defaults to a 1s wall-clock deadline, which made
the isolation guards flake under load; they now pass an explicit timeout.

Route table unchanged: /v1/bots (runs), /v1/compute/bots (machines), /v1/bot/*
(runtime ops).
2026-07-15 22:52:24 -07:00
hanzo-dev c24b601825 feat(git): JSON read/browse surface for the console repo-browser
The console repo-browser (hanzoai/console products/git) needs machine-readable
refs/tree/blob/commits/readme, but git only served those as HTML (ui.go) + the
repo-CRUD control plane. Add the JSON twin, reusing ONE set of go-git read
helpers so the HTML + JSON surfaces can never drift:

  GET /v1/git/repos/:name/refs                   → { branches, tags, default }
  GET /v1/git/repos/:name/tree?ref&path          → { entries:[{name,path,type,size,mode}] }
  GET /v1/git/repos/:name/blob?ref&path          → { path,size,encoding,content,binary,truncated }
  GET /v1/git/repos/:name/commits?ref&path&limit → { commits:[{sha,shortSha,message,author*,date}] }
  GET /v1/git/repos/:name/readme?ref             → { path, content, encoding }

- browse.go: the handlers + DTOs (mirror the console GitApi normalizers verbatim).
  ref+path ride as ?ref=&path= query params (the UI's own convention) so a slashed
  branch is unambiguous. Reuses org()/findRepo()/openGit()/resolveRef()/
  cleanTreePath(). Org-scoped (X-Org-Id); a repo outside the caller's org 404s.
  Distinct trailing segments — never shadow the :org/:repo smart-HTTP routes.
- ui.go: findReadme → readmeAt (returns filename+content); the ONE readme scan now
  backs both the HTML repo home and the JSON /readme (DRY, no duplication).
- browse_test.go: seeds a nested tree + README + binary file, asserts every endpoint
  + org isolation + honest 404/403 + empty-repo refs.

CGO_ENABLED=0 go test ./clients/git/ green (full package); go vet + build clean.
Stays v1.x.x (Go module). Ships with console@main on the next cloud release.
2026-07-15 22:23:09 -07:00
hanzo-dev 3d2b7c87b5 fix(billing): read the prepaid balance from the finance ledger, not a self-call
GET /v1/billing/balance and /v1/finance/balance proxied to commerce at the SAME
path they are registered on. commerceinproc publishes the shared zip app and
re-dispatches by path, and commerce's own /v1/billing/* routes are never
registered in this binary (api.Route runs only from commerce's mount.go, which
is //go:build cloud; cloud ships -tags "libsqlite3 sqlite_fts5"). So the proxy
re-entered itself, hit its own sign-in gate with no principal, and reported
"billing upstream status 500".

Co-resident, the prepaid wallet is cloud's own finance ledger: wireFinance points
the ai gate's balance read at it, the edge meter debits it (metering.fetchAvailable
already resolves finance.Current() first), and an admin grant credits it
(core.grantDeposit already prefers it). Read it directly through that same seam.
The commerce S2S read stays as the split-deploy fallback.

The subject comes from ai/object.BillingSubject — the function the ai prepaid gate
itself resolves — instead of a re-implemented copy. cloud and ai each keeping their
own copy of that rule is what let them drift: the console scoped to the org while
the gate scoped to "org/user", so the view showed a funded org while the gate
refused the member.

Fail posture unchanged: a balance that cannot be read is UNKNOWN and surfaces as
502 — never rendered as a zero balance. The sign-in gate and org scoping are
untouched; the org still comes from the validated principal only.

Tests: self-dispatch pinned at depth 2 (the seam's real mechanics, which the
existing SetHandler-stub test cannot see); router semantics probed (byte-identical
patterns MERGE and silently shadow; equal-specificity param-name conflicts PANIC at
registration; most-specific wins over registration order); balance regression,
gate-subject parity, unreadable-is-not-zero, sign-in gate, and cross-org isolation.
2026-07-15 22:16:46 -07:00
antje 7dd7a9b4dd feat(insights): /v1/insights — the unified native surface on the ONE analytics engine
A wire adapter, not a second pipeline: POST /v1/insights/e accepts PostHog-
shaped payloads (single or {batch}) from @hanzo/insights and any compatible
SDK, maps $-properties onto the native CaptureEvent, and rides the SAME
capture path (tenant gate -> normalize -> scrub -> hanzo.events). GET
/v1/insights/events is the console's tenant-scoped recent-events read; GET
/v1/insights/health reports the surface. Flags deliberately stay at /v1/flags.

Scale path stays stateless: accept on any replica, pooled batch INSERT sink;
the queue-buffered (mq/pubsub -> Datastore consumer) upgrade swaps the exec
behind buildEventsInsert with no handler changes.
2026-07-15 21:35:30 -07:00
zandGitHub 8d719c9821 Merge pull request #309 from hanzoai/bump/ai-v1.809.3
chore(ai): bump embedded ai v1.809.2 -> v1.809.3 (comped preview balance fix)
2026-07-15 21:30:55 -07:00
hanzo-dev f58f0b7c2a chore(ai): bump embedded hanzoai/ai v1.809.2 -> v1.809.3 (comped preview bypasses enforceBalanceGate)
Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-15 21:30:09 -07:00
hanzo-dev 94ec6ecbb9 Merge land/hybrid-connectors: injective tenant S3 key (close cross-tenant collision) + orgPathSafe guard 2026-07-15 21:21:56 -07:00
hanzo-dev f7fd44618c fix(projects): injective tenant S3 key via verbatim principal.Org (close cross-tenant collision) + orgPathSafe traversal guard 2026-07-15 21:21:25 -07:00
hanzo-dev 3ddbe91e63 bots: state the list cap explicitly
The published contract has no pagination, so the list is capped either way; set
it at the store maximum here rather than inherit the store's 100-row default.
2026-07-15 20:52:36 -07:00
hanzo-dev 8360315345 bots: give the three "bot" values one namespace each; make the run control plane native
GET /v1/bots was registered twice: clients/visor (bot machines) and clients/bots
(bot runs). The router resolves byte-identical patterns by first-registration
without panicking, and visor mounts first, so visor's machine list answered the
console's run list and clients/bots.list was unreachable. The console normalized
machine rows into run rows, yielding one blank-runId row per kind=bot machine
with a dead sessionUrl, and hid the org's real runs.

Name the values apart, one home and one route namespace each:
  - bot run     -> clients/bots  /v1/bots            (unchanged; the console + CLI contract)
  - bot machine -> clients/visor /v1/compute/bots    (moved; a machine is compute)
  - bot runtime -> clients/bot   /v1/bot/*           (passthrough for runtime-owned ops)

Make the run control plane native. clients/bots holds no store: a run is recorded
on the agents session plane under agent label "bot", so the run id is the session
id and one registry serves every kind of agent work. list reads it org-scoped;
stop resolves (org, runId) against that record and drives the runtime only after
ownership is proven, so a run of another tenant is a 404 the runtime never hears
about. An unreachable runtime is a 502 with the record left live.

Two seams (Runs, Runtime) are injected in adapters.go, the only file in
clients/bots importing agents/bot, so handlers unit-test against fakes.

agents: sessions carry a surface (the modality a session runs on, matching the
runtime's own origin.surface) via the established additive-column migration;
SessionFilter gains Agent so a product face reads only its own sessions;
OpenSession takes SessionOpen; StopSession is the single-session twin of
StopSessions and shares its one write path (stopOne).

Contract change: a run id is now the session id, not bot_<hex>. No bot_ id was
durable anywhere -- the old run handler minted an id and stored it nowhere -- so
there is nothing to migrate. Ids stay opaque to clients.
2026-07-15 20:38:54 -07:00
antje 6781eed94f cli/gpu: heartbeat the claimed activity + fleet presence DURING a render (goroutine in claimAndRun) — a >120s render no longer drops the machine offline mid-render; fixes BYO-GPU fix/compose/render hitting the deadline + going offline 2026-07-15 20:21:31 -07:00
zandGitHub cd082ca1cb Merge pull request #308 from hanzoai/bump/ai-v1.809.2
chore(ai): bump embedded hanzoai/ai v1.809.1 -> v1.809.2 (Enso limited-preview gating)
2026-07-15 19:19:07 -07:00
hanzo-dev 6c7a89f8d9 chore(ai): bump embedded hanzoai/ai v1.809.1 -> v1.809.2 (Enso limited-preview gating)
Lands the Enso limited-preview gating into the live api.hanzo.ai binary
(cloud embeds ai in-process via ai.Mount). v1.809.2 adds ai's Enso family
routing (ENSO_URL -> enso pod) + waitlist gating: ModelAccess visibility in
/v1/models, 403 request-access for ungated SKUs, comped bypass of the balance
gate for granted callers, POST /v1/models/:model/access, org `hanzo` seed.

Independent of the concurrent zen v1.3.0 -> v1.3.7 bump (#307), which this
branch is based on top of.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-15 19:16:00 -07:00
hanzo-dev b5b1a09952 Merge land/chat-connectors: generalized ChatBridge + Discord/Teams/Telegram adapters for hanzo.chat 2026-07-15 19:11:41 -07:00
hanzo-dev 767e089410 land(chat): generalized ChatBridge + Discord/Teams/Telegram adapters for hanzo.chat 2026-07-15 19:09:52 -07:00
zandGitHub 5cbf1ed90e Merge pull request #307 from hanzoai/fix/zen-bump-clean
fix(zen): bump embedded hanzoai/zen v1.3.0 → v1.3.7 (1M ladder fix)
2026-07-15 19:04:13 -07:00
hanzo-dev 9d8f86d191 fix(zen): bump embedded hanzoai/zen v1.3.0 -> v1.3.7 (1M ladder need() fix)
The cloud binary serves the zen* family IN-PROCESS via zen.Mount, so the
embedded module version — not the zen pod image — is what serves zen5. v1.3.0
sized the ladder rung by the byte estimate alone: a ~230K-token prompt with a
32K max_tokens budget estimated ~258K (under glm-5.2's 262144 cap), stayed on
glm-5.2, which then saw prompt+output = 262145 and 400'd. v1.3.7 sizes the rung
by need() = messages + tool schema + max_tokens, overflowing a >262144 total to
the 1M deepseek-v4-pro rung. Brings the Enso family + gating in-binary too.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-15 19:03:37 -07:00
antje f49a6ac2b2 fix(image): runtime needs libgcc — the flags staticlib references _Unwind_*
The hanzo-flags Rust staticlib compiles with unwinding (panic-guarded FFI);
its _Unwind_* references resolve from libgcc_s, which the alpine runtime did
not carry — /cloud failed relocation at exec ('Error relocating /cloud:
_Unwind_GetIP: symbol not found') and smoke red-gated the release. Add libgcc
to the runtime apk set.
2026-07-15 18:47:52 -07:00
hanzo-dev 92004893be Merge land/agent-deploy-sites: agent AI-brief site build+deploy to <slug>.hanzo.app, metered per deploy 2026-07-15 18:41:36 -07:00
hanzo-dev 3d294ce07c land(sites): agent AI-brief site build+deploy to <slug>.hanzo.app, metered per deploy 2026-07-15 18:40:05 -07:00
antje 1b552d890c feat(flags): native flag engine — stateless Rust FFI + SQLite per project, /v1/flags
native/flags: hanzo-flags, a stateless Rust staticlib with PostHog-compatible
evaluation — the exact Insights rollout hash (sha1 first-15-hex / LONG_SCALE,
pinned by test vectors), the full vendored property-operator set (exact/regex/
semver/date/relative-date...), condition groups (variant-override groups first),
multivariate cumulative selection, payloads. Pure (defs JSON, ctx JSON) ->
response JSON behind a panic-guarded C ABI: hanzo_flags_evaluate/_free.
65 tests green.

clients/featureflags becomes the NATIVE engine (no external evaluator, no KV,
no network): definitions in per-(org, project) SQLite via cloud.OrgDB
(encrypted at rest via cek), hot in-memory platform snapshot (TTL 15s),
evaluation over cgo. The INSIGHTS_FLAGS_URL HTTP proxy is gone. Bool/Int/
String and the admin Board keep their shapes; sources are flags -> env ->
default (first cockpit write creates the definition — env fallback intact
until then, zero regression). engine_stub (!cgo) degrades loudly, fail-safe.

/v1/flags (org-scoped via principal, project via X-Project-Id): POST /v1/flags
(+/decide alias) evaluate; GET/PUT/DELETE /v1/flags/defs[/:key]; GET
/v1/flags/activity; GET /v1/flags/health. PUT /v1/admin/flags/:key (SuperAdmin)
is the cockpit write path through SetPlatformSwitch — flips apply immediately
in-pod, peers converge within one TTL.

Build: Dockerfile flagslib stage (rust:1-alpine musl staticlib, --locked) copied
to the exact ${SRCDIR}-relative cgo link path; make native; hanzo.yml
native-flags step + clients/featureflags in the hermetic unit gate.
go test ./clients/featureflags ./apps green (FFI live).
2026-07-15 18:38:01 -07:00
zeekayandClaude Opus 4.8 23d9c4809a fix(console): white-label the embedded console <title> by request Host
The console SPA shipped in the unified cloud binary is a STATIC export of
hanzoai/console whose <title> is baked to the default (Hanzo) brand at build
time. The embed cannot read the request Host, so every host — including
console.lux.cloud and console.zoo.cloud — served "Hanzo Cloud Console" in the
browser tab, a white-label violation. (The standalone Next.js app is host-aware
via generateMetadata, but it is retired from this serving path: cloud serves
the console in-process from the go:embed static export.)

Rewrite the SPA shell's <title> at the serving layer (serveIndex/indexFor) to
the request host's brand, reusing the existing brands registry (BrandForHost):
console.lux.cloud -> "Lux Cloud Console", console.hanzo.ai -> "Hanzo Cloud
Console" (unchanged), console.zoo.cloud -> "Zoo Cloud Console". Matches
hanzoai/console's own `${brandName} Console` output — one source of truth. The
Hanzo/default host returns the embedded bytes unchanged (no regression); a shell
with no <title> is never altered.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 18:25:22 -07:00
hanzo-dev 9c2eb2cf64 Merge land/featuregate: per-service waitlist-mode control plane 2026-07-15 18:10:27 -07:00
hanzo-dev 6529cf7d3e land(featuregate): per-service waitlist-mode control plane
Global SQLite registry {service, hosts, waitlistMode} + admin board/toggle
(/v1/admin/services*, /v1/featuregate/mode) + native Enforce middleware and
IAM-backed per-user approval resolver. Ported from recover/featuregate; dropped
the init()/cloud.RegisterWithShutdown self-registration in favor of the explicit
apps.Wire() MountSpec (order after admin, before tasks) + frozen wire_test row.

Distinct from clients/featureflags (a global read-only Insights waitlist_open
switch); this is the per-host launch lever with in-binary enforcement.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-15 13:20:25 -07:00
386 changed files with 47430 additions and 6384 deletions
+36 -4
View File
@@ -89,18 +89,50 @@ jobs:
with:
go-version-file: go.mod
- name: go env for private modules (matches Dockerfile — zap-proto is direct+authenticated)
- name: go env for private modules
env:
GH_PAT: ${{ secrets.GH_PAT }}
# GOPRIVATE names exactly the namespace that is private. github.com/hanzoai/*
# is: ai, account, commerce, orm, xorm, beego, csqlite and ~30 more are
# private repos, so they must resolve direct+authenticated and skip a sumdb
# that cannot see them. Everything else stays on the public proxy + checksum
# db, which is what makes a module hash immutable: zap-proto (all 55 repos)
# and luxfi (all 37 deps here) are public and proxy-served.
#
# This previously named zap-proto — public, and never the reason anything
# here was direct — and then set GOSUMDB=off to compensate for hanzoai/*
# being absent, which disabled checksum verification for EVERY module in the
# build, public ones included. Naming the private namespace is what the off
# switch was standing in for.
run: |
git config --global url."https://x-access-token:${GH_PAT}@github.com/".insteadOf "https://github.com/"
{
echo "GOPRIVATE=github.com/zap-proto/*"
echo "GONOSUMDB=github.com/zap-proto/*"
echo "GOSUMDB=off"
echo "GOPRIVATE=github.com/hanzoai/*"
echo "GOPROXY=https://proxy.golang.org,direct"
} >> "$GITHUB_ENV"
- name: zen streaming-fix floor — go.mod must pin github.com/hanzoai/zen >= v1.4.1
# The SSE body-close fix (zen commit 50328b8, first released in zen v1.4.1)
# is what makes streaming completions return a body instead of an empty
# stream. A stale-branch merge that reverts go.mod's zen pin below the floor
# silently re-breaks streaming, and `next build`'s ignoreBuildErrors hides
# the runtime break — so no image may be cut on a regressed pin. This is the
# durable root-cause guard: it reads the EFFECTIVE module version (post-MVS,
# exactly what the build links) and fails the PR/push below the floor.
run: |
set -euo pipefail
FLOOR="v1.4.1"
V="$(go list -m -f '{{.Version}}' github.com/hanzoai/zen)"
echo "effective github.com/hanzoai/zen = ${V} (floor ${FLOOR})"
# semver-correct compare: the lowest of {V, FLOOR} under `sort -V` must be
# the FLOOR, i.e. V >= FLOOR. (sort -V orders v1.4.2 above v1.4.10 too.)
low="$(printf '%s\n%s\n' "$V" "$FLOOR" | sort -V | head -1)"
if [ "$low" != "$FLOOR" ]; then
echo "::error::github.com/hanzoai/zen is pinned at ${V}, below the streaming-fix floor ${FLOOR} — this re-breaks SSE streaming (empty completions). Re-pin zen to >= ${FLOOR} in go.mod before merging."
exit 1
fi
echo "OK: zen ${V} is at or above the streaming-fix floor ${FLOOR}"
- name: positive proof — clients/controlplane is unreachable from the default build
run: |
set -euo pipefail
+229 -56
View File
@@ -96,9 +96,9 @@ jobs:
# an image (even from a run that died before tagging) is never reused.
cont_max=""
if command -v gh >/dev/null 2>&1; then
cont_max="$(GH_TOKEN="$GH_PAT" gh api --paginate \
'/orgs/hanzoai/packages/container/cloud/versions' \
--jq '.[].metadata.container.tags[]' 2>/dev/null \
cont_max="$(GH_TOKEN="$GH_PAT" gh api \
'/orgs/hanzoai/packages/container/cloud/versions?per_page=100' \
--jq '.[].metadata.container.tags[]?' 2>/dev/null \
| sed 's/^v//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true)"
fi
@@ -124,20 +124,6 @@ jobs:
echo "sha_short=$(git rev-parse --short "$GITHUB_SHA")" >> "$GITHUB_OUTPUT"
echo "Next release: v${version} (git_max='${git_max:-none}' container_max='${cont_max:-none}')"
# Console-embed cachebust. The console clone+build layer is keyed on this;
# prefer hanzoai/console main HEAD so a CONSOLE-ONLY change re-embeds without
# needing a cloud commit (cloud-sha alone froze the embed between cloud pushes).
# git ls-remote must CLEAR the extraheader actions/checkout installs (it carries
# THIS repo's GITHUB_TOKEN, which 404s the cross-repo console lookup); gh is not
# on the runner. If resolution yields nothing, fall back to the cloud sha — still
# unique per cloud commit, so the embed is never frozen. Either way THIS build
# busts (new value) and re-clones console main fresh.
console_head="$(git -c 'http.https://github.com/.extraheader=' ls-remote \
"https://x-access-token:${GH_PAT}@github.com/hanzoai/console.git" refs/heads/main 2>/dev/null | cut -f1 || true)"
cachebust="${console_head:-$GITHUB_SHA}"
echo "cachebust=${cachebust}" >> "$GITHUB_OUTPUT"
echo "console cachebust: ${cachebust} (console_head='${console_head:-none}')"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
@@ -160,8 +146,12 @@ jobs:
# Direct credential first (repo/org secret — works on private repos,
# where the Free plan hides org KMS secrets); KMS kubeconfig fallback.
if [ -n "${REGISTRY_USER:-}" ] && [ -n "${REGISTRY_PASSWORD:-}" ]; then
echo "$REGISTRY_PASSWORD" | docker login registry.hanzo.ai -u "$REGISTRY_USER" --password-stdin
echo "MIRROR_OK=1" >> "$GITHUB_ENV"; exit 0
if echo "$REGISTRY_PASSWORD" | docker login registry.hanzo.ai -u "$REGISTRY_USER" --password-stdin; then
echo "MIRROR_OK=1" >> "$GITHUB_ENV"
else
echo "mirror login failed (registry.hanzo.ai unreachable) — mirror skipped, release continues"
fi
exit 0
fi
[ -z "${KMS_CLIENT_ID:-}" ] && { echo "no KMS creds — mirror skipped"; exit 0; }
TOKEN=$(curl -sf "$KMS_ENDPOINT/v1/kms/auth/login" -H 'Content-Type: application/json' -d "{\"clientId\":\"$KMS_CLIENT_ID\",\"clientSecret\":\"$KMS_CLIENT_SECRET\"}" | jq -r '.accessToken // empty')
@@ -180,8 +170,11 @@ jobs:
UP=$(echo "$CFG" | jq -r '.auths["registry.hanzo.ai"].auth // empty' | base64 -d)
[ -z "$UP" ] && { echo "no registry auth — mirror skipped"; exit 0; }
echo "::add-mask::${UP#*:}"
echo "${UP#*:}" | docker login registry.hanzo.ai -u "${UP%%:*}" --password-stdin
echo "MIRROR_OK=1" >> "$GITHUB_ENV"
if echo "${UP#*:}" | docker login registry.hanzo.ai -u "${UP%%:*}" --password-stdin; then
echo "MIRROR_OK=1" >> "$GITHUB_ENV"
else
echo "mirror login failed (registry.hanzo.ai unreachable) — mirror skipped, release continues"
fi
- name: Log in to ghcr.io (GH_PAT — writes the cloud package despite its ai-repo linkage)
uses: docker/login-action@v3
@@ -190,6 +183,39 @@ jobs:
username: hanzo-dev
password: ${{ secrets.GH_PAT }}
- name: Resolve decomplection artifact digests (the Go-only build's prebuilt inputs)
id: artifacts
run: |
set -euo pipefail
# cloud compiles ONLY Go; it pulls three prebuilt artifacts (console SPA,
# agent-skills catalog, native flags staticlib). Resolve each published
# :latest to an IMMUTABLE digest so THIS release is reproducible (pinned,
# not floating :latest) AND a console/skills/flags change is picked up —
# its CI republished :latest, so this resolves to the NEW digest. A MISSING
# artifact FAILS the release HERE, before build/smoke/push/tag: the receipt
# invariant means we never tag an image that couldn't embed the real console.
command -v crane >/dev/null 2>&1 || {
mkdir -p "$HOME/.local/bin"
curl -fsSL "https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz" \
| tar -xz -C "$HOME/.local/bin" crane
}
export PATH="$HOME/.local/bin:$PATH"
resolve() {
local repo="$1" d
d="$(crane digest "ghcr.io/hanzoai/${repo}:latest" 2>/dev/null || true)"
[ -n "$d" ] || { echo "::error::decomplection artifact ghcr.io/hanzoai/${repo}:latest is not published — refusing to cut a release that would embed a stale/placeholder ${repo}"; return 1; }
printf 'ghcr.io/hanzoai/%s@%s' "$repo" "$d"
}
CONSOLE_IMAGE="$(resolve console-embed)" || exit 1
SKILLS_IMAGE="$(resolve agent-skills)" || exit 1
FLAGS_IMAGE="$(resolve cloud-flags)" || exit 1
{
echo "console_image=${CONSOLE_IMAGE}"
echo "skills_image=${SKILLS_IMAGE}"
echo "flags_image=${FLAGS_IMAGE}"
} >> "$GITHUB_OUTPUT"
echo "resolved: console=${CONSOLE_IMAGE} skills=${SKILLS_IMAGE} flags=${FLAGS_IMAGE}"
- name: OCI labels
id: meta
uses: docker/metadata-action@v5
@@ -216,15 +242,17 @@ jobs:
load: true
tags: cloud:smoke
labels: ${{ steps.meta.outputs.labels }}
# Bust the console clone+build layer every release (the cloud commit sha is
# unique per push) so the embed re-fetches console main HEAD fresh — never the
# frozen snapshot the persistent BuildKit cache would otherwise serve forever.
# cloud compiles ONLY Go: pull the three prebuilt artifacts pinned to the
# digests resolved above (reproducible, and fresh — a console/skills/flags
# change is a new digest). No node/python/rust toolchain in this build.
build-args: |
CONSOLE_CACHEBUST=${{ steps.ver.outputs.cachebust }}
# gh_token: BuildKit secret the Dockerfile consumes to fetch private
CONSOLE_IMAGE=${{ steps.artifacts.outputs.console_image }}
SKILLS_IMAGE=${{ steps.artifacts.outputs.skills_image }}
FLAGS_IMAGE=${{ steps.artifacts.outputs.flags_image }}
# GIT_AUTH_TOKEN: BuildKit secret the Dockerfile consumes to fetch private
# cross-org Go modules (hanzoai/*, luxfi/*) over authenticated git.
secrets: |
gh_token=${{ secrets.GH_PAT }}
GIT_AUTH_TOKEN=${{ secrets.GH_PAT }}
- name: Smoke test — the binary MUST boot to "listening" with no crash signature
run: |
@@ -293,6 +321,67 @@ jobs:
echo "SMOKE PASS: cloud:smoke booted to \"listening\" with no crash signature"
# ── Functional smoke — authenticated per-subsystem probe (the REAL gate) ─────
# The boot check above proves the process REACHES "listening"; this proves the
# mounted HTTP surface actually WORKS. /smoke (cmd/smoke, baked into the image)
# hits ONE side-effect-free read per core subsystem and FAILS the release on any
# broken code — above all a 402 on a READ (the balance-gate-over-blocks-reads
# regression) or a 5xx (a crash, e.g. the /v1/billing/usage self-dispatch 500).
# So a release can never ship with chat/billing/projects/kms/... down.
- name: Functional smoke — per-subsystem probe (fails the release if a core endpoint is broken)
env:
KMS_CLIENT_ID: ${{ secrets.KMS_CLIENT_ID }}
KMS_CLIENT_SECRET: ${{ secrets.KMS_CLIENT_SECRET }}
KMS_ENDPOINT: ${{ vars.KMS_ENDPOINT || 'https://kms.hanzo.ai' }}
# A KMS-provisioned short-lived smoke bearer, injected as a secret (NEVER
# hardcoded). Absent → the anonymous matrix still gates public/authed and
# catches every 402-on-read / 5xx.
SMOKE_TOKEN: ${{ secrets.SMOKE_TOKEN }}
run: |
set -euo pipefail
IMAGE=cloud:smoke
CID=""
cleanup() { [ -n "$CID" ] && docker rm -f "$CID" >/dev/null 2>&1 || true; }
trap cleanup EXIT
KEY="$(head -c 32 /dev/urandom | base64 | tr -d '\n')"
CID="$(docker run -d \
--tmpfs /data:rw,size=64m \
-e CLOUD_DATA_DIR=/data -e CLOUD_ENV=smoke -e CLOUD_KMS_MASTER_KEY_REF="$KEY" \
"$IMAGE")"
# Wait for the HTTP listener to bind (or the process to die).
up=0
for _ in $(seq 1 60); do
lg="$(docker logs "$CID" 2>&1 || true)"
printf '%s' "$lg" | grep -q '"message":"listening"' && { up=1; break; }
[ "$(docker inspect -f '{{.State.Running}}' "$CID" 2>/dev/null || echo false)" != "true" ] && break
sleep 1
done
if [ "$up" != 1 ]; then
echo "::group::boot logs"; docker logs "$CID" 2>&1 || true; echo "::endgroup::"
echo "FUNCTIONAL SMOKE INFRA FAIL: image never reached \"listening\""
exit 1
fi
# Token: prefer the injected secret; else mint from KMS (a provisioned smoke
# identity); else run the anonymous matrix. Never hardcoded.
if [ -z "${SMOKE_TOKEN:-}" ] && [ -n "${KMS_CLIENT_ID:-}" ]; then
KT=$(curl -sf "$KMS_ENDPOINT/v1/kms/auth/login" -H 'Content-Type: application/json' \
-d "{\"clientId\":\"$KMS_CLIENT_ID\",\"clientSecret\":\"$KMS_CLIENT_SECRET\"}" | jq -r '.accessToken // empty' || true)
[ -n "$KT" ] && SMOKE_TOKEN=$(curl -sf "$KMS_ENDPOINT/v1/kms/orgs/hanzo/secrets/smoke/TOKEN?env=prod" \
-H "Authorization: Bearer $KT" | jq -r '.secret.value // empty' || true)
fi
if [ -n "${SMOKE_TOKEN:-}" ]; then echo "::add-mask::$SMOKE_TOKEN"; echo "smoke: AUTHENTICATED matrix"; else echo "smoke: ANONYMOUS matrix (no SMOKE_TOKEN wired)"; fi
# /smoke is baked into the image (Dockerfile) — exec it INSIDE the container,
# so it probes the real mounted surface at localhost:8080 with no port/network
# plumbing. A non-zero exit here fails the release BEFORE any image is pushed.
docker exec \
-e SMOKE_BASE_URL=http://127.0.0.1:8080 \
-e SMOKE_TOKEN="${SMOKE_TOKEN:-}" \
"$CID" /smoke
# ── Migration smoke — the gate the v1.800.1 crashloop would have tripped ─────
# The plain smoke above boots on a FRESH /data, so every subsystem's migrate()
# takes its CREATE-TABLE path and no forward-migration is exercised — which is
@@ -401,12 +490,14 @@ jobs:
ghcr.io/hanzoai/cloud:sha-${{ steps.ver.outputs.sha_short }}
ghcr.io/hanzoai/cloud:latest
labels: ${{ steps.meta.outputs.labels }}
# SAME cachebust as the smoke build → every layer is a cache hit from step 1
# and the pushed image is byte-identical to the one the smoke test proved.
# SAME artifact digests as the smoke build → every layer is a cache hit from
# step 1 and the pushed image is byte-identical to the one smoke proved.
build-args: |
CONSOLE_CACHEBUST=${{ steps.ver.outputs.cachebust }}
CONSOLE_IMAGE=${{ steps.artifacts.outputs.console_image }}
SKILLS_IMAGE=${{ steps.artifacts.outputs.skills_image }}
FLAGS_IMAGE=${{ steps.artifacts.outputs.flags_image }}
secrets: |
gh_token=${{ secrets.GH_PAT }}
GIT_AUTH_TOKEN=${{ secrets.GH_PAT }}
# THE RECEIPT + ATOMIC VERSION ASSIGNMENT (race-safe). Reached only because
# build + smoke + push all succeeded, so a proven image exists under the unique
@@ -434,12 +525,29 @@ jobs:
git fetch --tags --force --quiet
git_max="$(git tag -l 'v[0-9]*.[0-9]*.[0-9]*' \
| sed 's/^v//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true)"
# Newest page only — NOT --paginate. Container versions are created
# newest-first and version tags are monotonic, so the highest version
# is always among the most-recent versions; paginating the WHOLE
# registry history is what livelocked this step as tags accumulated.
# Fail-CLOSED. An ORPHANED container tag — image pushed by a run that
# died or was cancelled after imagetools-create but before its git tag —
# MUST raise the floor, or a later run reassigns that same number to a
# different image (an ambiguous mutable prod tag; the v1.801.50 flip). A
# git-only floor can't see the orphan, so if the container-tag lookup
# ERRORS (vs legitimately returning no tags) we retry the whole attempt
# rather than silently proceeding — a version with a pushed image is never
# reused. (Reordering git-tag before imagetools-create is the WRONG fix: it
# reintroduces the phantom "tag ⇔ no image" this workflow exists to prevent.)
cont_max=""
if command -v gh >/dev/null 2>&1; then
cont_max="$(GH_TOKEN="$GH_PAT" gh api --paginate \
'/orgs/hanzoai/packages/container/cloud/versions' \
--jq '.[].metadata.container.tags[]' 2>/dev/null \
| sed 's/^v//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true)"
if cont_raw="$(GH_TOKEN="$GH_PAT" gh api \
'/orgs/hanzoai/packages/container/cloud/versions?per_page=100' \
--jq '.[].metadata.container.tags[]?' 2>/dev/null)"; then
cont_max="$(printf '%s\n' "$cont_raw" \
| sed 's/^v//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true)"
else
echo " container-tag lookup failed — retry so an orphaned tag can't be reused (attempt $attempt)"; sleep 3; continue
fi
fi
max="$(printf '%s\n%s\n%s\n' "1.786.0" "$git_max" "$cont_max" \
| grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1)"
@@ -469,8 +577,12 @@ jobs:
export PATH="$HOME/.local/bin:$PATH"
}
for MT in "${V}" "${VER}" "${major}.${minor}"; do
crane copy "$SHA_IMG" "registry.hanzo.ai/hanzoai/cloud:${MT}" \
|| echo "::warning::mirror registry.hanzo.ai/hanzoai/cloud:${MT} failed"
# Bounded: registry.hanzo.ai can *hang* (not just fail), and this
# is best-effort — an unbounded crane copy once livelocked the whole
# tag step and held the serialized release lane. timeout makes the
# mirror truly best-effort so the git-tag receipt below always runs.
timeout 120 crane copy "$SHA_IMG" "registry.hanzo.ai/hanzoai/cloud:${MT}" \
|| echo "::warning::mirror registry.hanzo.ai/hanzoai/cloud:${MT} failed or timed out"
done
fi
git tag -a "$V" -m "release $V — image ghcr.io/hanzoai/cloud:$V (retagged from sha-${{ steps.ver.outputs.sha_short }}, smoke-passed ${GITHUB_SHA})"
@@ -487,25 +599,86 @@ jobs:
echo "::error::could not acquire a free version tag after 8 attempts"
exit 1
# Notify universe so the GitOps pipeline rolls the new image to prod — same
# image-update contract every service uses (gateway, iam, …). Runs ONLY after
# build-amd64 succeeds, i.e. only for a version whose image is proven pushed and
# tagged. A failed release never reaches here, so universe is never asked to
# deploy a phantom tag.
notify-universe:
# ── Promote: the declared-tag bump that makes the release DEPLOY ─────────────
# The tag minted above is the receipt for a pushed, smoke-passed image; THIS job
# records it as the desired state Hanzo CD reconciles. The universe-crs ArgoCD
# Application (ns hanzo-cd, `automated` sync + selfHeal) syncs
# infra/k8s/operator/crs/*.yaml → cluster and the operator rolls the Deployment,
# so a tag bump committed here reaches api.hanzo.ai with NO hand-dispatch and NO
# hand-edit of the CR.
#
# This is the SAME yq-bump → `deploy(<svc>): <tag>` universe commit the hanzoai/ci
# reusable (build.yml deploy step) does for every other service. cloud owns it
# HERE because its image is built by this workflow, not the ci reusable — its
# hanzo.yml carries no main `images:` entry and `# NO deploy`, so the shared
# deploy step never bumps cloud's CR. A direct in-cluster CR patch is NOT enough:
# ArgoCD selfHeal reverts any live edit not also recorded in git within ~45s.
# The retired notify-universe repository_dispatch had no receiver after the
# image-update.yml deploy hub was deleted in the Hanzo CD cutover; the git commit
# IS the sanctioned path now.
promote:
needs: build-amd64
# Only a real release promotes: build+smoke+push+tag all succeeded, so a
# proven v* image exists. A failure earlier leaves version_v empty → skipped.
if: ${{ needs.build-amd64.outputs.version_v != '' }}
runs-on: [hanzo-build-linux-amd64]
steps:
- name: Repository dispatch (image-update)
uses: peter-evans/repository-dispatch@v3
with:
token: ${{ secrets.UNIVERSE_DISPATCH_TOKEN }}
repository: hanzoai/universe
event-type: image-update
client-payload: |
{
"service": "cloud",
"image": "ghcr.io/hanzoai/cloud:${{ needs.build-amd64.outputs.version_v }}",
"sha": "${{ github.sha }}",
"env": "all"
}
- name: Record the proven tag in universe crs/cloud.yaml (Hanzo CD rolls it)
env:
# GH_PAT already pushes this repo's git tags above (contents:write on the
# hanzoai org), so it writes hanzoai/universe too — the SAME token the ci
# reusable falls back to for the universe deploy commit.
GH_PAT: ${{ secrets.GH_PAT }}
VERSION_V: ${{ needs.build-amd64.outputs.version_v }}
run: |
set -euo pipefail
[ -n "${GH_PAT:-}" ] || { echo "::error::no GH_PAT — cannot record the declared-tag bump in universe"; exit 1; }
# Bare arc runners ship no yq — provision the static binary (sudo-free,
# same pattern the ci reusable and this workflow's kubectl/crane fetches use).
if ! command -v yq >/dev/null 2>&1; then
mkdir -p "$HOME/.local/bin"; export PATH="$HOME/.local/bin:$PATH"
curl -fsSL https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 \
-o "$HOME/.local/bin/yq" && chmod +x "$HOME/.local/bin/yq"
fi
git clone -q --depth 1 \
"https://x-access-token:${GH_PAT}@github.com/hanzoai/universe.git" \
"$RUNNER_TEMP/universe"
CR="$RUNNER_TEMP/universe/infra/k8s/operator/crs/cloud.yaml"
[ -f "$CR" ] || { echo "::error::crs/cloud.yaml not found in universe"; exit 1; }
CUR="$(yq -r '.spec.image.tag // ""' "$CR")"
echo "cloud CR: ${CUR:-<empty>} → ${VERSION_V}"
# Monotonic guard: never roll the CR BACKWARD. Release runs finish under a
# serialized lane but a slow older run must never overwrite a newer promote.
# Skip iff the CR already holds a semver >= the version we just cut.
CURN="${CUR#v}"; NEWN="${VERSION_V#v}"
if printf '%s' "$CURN" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+$'; then
top="$(printf '%s\n%s\n' "$CURN" "$NEWN" | sort -V | tail -1)"
if [ "$top" = "$CURN" ] && [ "$CURN" != "$NEWN" ]; then
echo "::notice::cloud CR already at v${CURN} (≥ ${VERSION_V}) — not rolling back"; exit 0
fi
fi
yq -i ".spec.image.tag = \"${VERSION_V}\"" "$CR"
if git -C "$RUNNER_TEMP/universe" diff --quiet; then
echo "::notice::crs/cloud.yaml already at ${VERSION_V} — nothing to record"; exit 0
fi
git -C "$RUNNER_TEMP/universe" -c user.name=hanzo-ci -c user.email=dev@hanzo.ai \
commit -qam "deploy(cloud): ${VERSION_V} (${GITHUB_REPOSITORY}@$(echo "${GITHUB_SHA}" | cut -c1-7))"
# Rebase-safe push: universe main advances on every service's deploy, so a
# concurrent commit must not make cloud's promote lose the whole roll. Retry
# a few times, rebasing between attempts.
for attempt in $(seq 1 5); do
if git -C "$RUNNER_TEMP/universe" push -q origin HEAD:main; then
echo "recorded deploy(cloud): ${VERSION_V} — Hanzo CD (universe-crs) will roll it to api.hanzo.ai"
exit 0
fi
echo " universe push lost the race — rebasing (attempt ${attempt})"
git -C "$RUNNER_TEMP/universe" pull -q --rebase origin main || true
sleep 3
done
echo "::error::could not record the cloud tag bump in universe after 5 attempts"; exit 1
+1
View File
@@ -34,3 +34,4 @@ Thumbs.db
.shots/
.claude/
native/flags/target
+73 -129
View File
@@ -2,122 +2,50 @@
#
# This image is a SINGLE artifact that serves BOTH the /v1 API AND the console
# UI from one process: the console is compiled into the Go binary via
# //go:embed (see webui.go). The pipeline is:
# //go:embed (see webui.go). The final `/cloud` binary already carries the UI —
# no separate console Service, no second origin; the embedded console calls /v1
# on its own host.
#
# 1. console stage → build the hanzoai/console static bundle
# 2. (copied) → into webui/dist/ of the Go build context
# 3. build stage → `go build` bakes webui/dist into the binary (go:embed)
#
# so the final `/cloud` binary already carries the UI. No separate console
# Service, no second origin — the embedded console calls /v1 on its own host.
#
# ── console UI stage ─────────────────────────────────────────────────────────
# Builds the console SPA and emits a STATIC bundle at /out. console is fetched
# at a pinned ref (CONSOLE_REF) using the same gh_token BuildKit secret the Go
# build uses for private modules.
#
# console exposes `npm run build:embed` (scripts/build-embed.mjs): it prunes the
# Next server route handlers (BFF proxies — they collapse to the cloud /v1/* the
# SPA calls same-origin), wraps the client catch-all pages for output:'export',
# and neutralizes the root layout's request-time headers() read (the per-host
# <title>, resolved client-side in the embed) so the STATIC export prerenders
# clean — emitting out/. This stage runs it and copies out/ into /out, which the
# Go build drops into webui/dist so //go:embed bakes the FULL @hanzo/gui console
# into the ONE binary. This stage FAILS HARD: the prod image MUST carry the real
# console — a missing/broken build:embed is a build ERROR, never a silent degrade
# to the placeholder shell. The one escape hatch is --build-arg ALLOW_PLACEHOLDER=1
# (pure-Go dev image with no Node console), which is NEVER set for prod.
FROM public.ecr.aws/docker/library/node:24-alpine@sha256:a0b9bf06e4e6193cf7a0f58816cc935ff8c2a908f81e6f1a95432d679c54fbfd AS console
ARG CONSOLE_REPO=https://github.com/hanzoai/console.git
ARG CONSOLE_REF=main
# CONSOLE_CACHEBUST busts this stage's BuildKit layer cache every build. WHY it must
# exist: the clone+build layer's cache key is derived from the RUN text + build args.
# With only a static `git clone --branch main`, the key NEVER changes, so on the
# persistent ARC dind BuildKit cache every cloud image re-embedded the SAME frozen
# console snapshot — new console work (the native Tracker, …) silently never shipped,
# even on a freshly-built+deployed image. release.yml feeds this the cloud commit sha
# (unique per push) so the clone RUN re-runs each build and re-fetches console
# ${CONSOLE_REF} (main HEAD) fresh. Correctness over cache reuse: the console stage
# rebuilds every time, but the embed is never stale.
ARG CONSOLE_CACHEBUST=none
RUN apk add --no-cache git
WORKDIR /console
# The static export prerenders every page (webpack compile + export prerender);
# give the heap headroom so a large @hanzo/gui build never OOMs into the stub.
ENV NEXT_TELEMETRY_DISABLED=1 NODE_OPTIONS=--max-old-space-size=8192
# Hanzo Analytics: the console's <HanzoAnalytics/> (env-gated) renders the one
# native analytics.hanzo.ai tag only when a website-id is baked in. Default to the
# console.hanzo.ai property (7dce54ee, public per-site) so console+team track on
# the next cloud build. GA4/Pixel stay off (unset). Public id, not a KMS secret.
ARG NEXT_PUBLIC_ANALYTICS_WEBSITE_ID=7dce54ee-41f6-4751-96bf-fe005067c7c7
ENV NEXT_PUBLIC_ANALYTICS_WEBSITE_ID=$NEXT_PUBLIC_ANALYTICS_WEBSITE_ID
RUN --mount=type=secret,id=gh_token \
if [ -s /run/secrets/gh_token ]; then \
git config --global url."https://x-access-token:$(cat /run/secrets/gh_token)@github.com/".insteadOf "https://github.com/"; \
fi && \
echo ">> embedding console ${CONSOLE_REF} (cachebust ${CONSOLE_CACHEBUST})" && \
git clone --depth 1 --branch "${CONSOLE_REF}" "${CONSOLE_REPO}" . && \
echo ">> console @ $(git rev-parse HEAD)" && \
npm install --no-audit --no-fund --fetch-retries=5 --fetch-retry-mintimeout=20000 --fetch-timeout=120000
# FAIL-HARD. build:embed MUST emit a REAL bundle — a non-empty out/index.html AND
# an out/_next/ chunk dir — and /out then carries it into the Go embed path. If the
# target is absent, the export fails, or the output is the placeholder shape, this
# is a build ERROR (exit 1): the prod image can NEVER silently ship the committed
# fallback shell. Escape hatch: --build-arg ALLOW_PLACEHOLDER=1 leaves /out empty
# (Go build keeps the committed shell) for a pure-Go dev image — NEVER set in prod.
ARG ALLOW_PLACEHOLDER=0
RUN mkdir -p /out; \
ok=0; \
if npm run 2>/dev/null | grep -q ' build:embed'; then \
if npm run build:embed && [ -s out/index.html ] && [ -d out/_next ]; then \
cp -r out/. /out/; \
echo ">> embedded REAL console static bundle: $(wc -c < out/index.html)-byte index.html, $(du -sh out/_next | cut -f1) _next/"; \
ok=1; \
else \
echo ">> console build:embed produced NO real bundle (missing/empty out/index.html or out/_next)"; \
fi; \
else \
echo ">> console exposes no build:embed target"; \
fi; \
if [ "$ok" != "1" ]; then \
if [ "$ALLOW_PLACEHOLDER" = "1" ]; then \
echo ">> ALLOW_PLACEHOLDER=1 — keeping committed fallback shell (DEV image only; NEVER prod)"; \
else \
echo ">> FATAL: refusing to ship the placeholder console. Fix the console build:embed, or pass --build-arg ALLOW_PLACEHOLDER=1 for a pure-Go dev image."; \
exit 1; \
fi; \
fi
# ── prebuilt decomplection artifacts (cloud compiles ONLY Go) ────────────────
# The console SPA, the agent-skills catalog, and the native flags staticlib are
# each built by THEIR OWN CI as a versioned immutable image and PULLED here,
# instead of rebuilding node + python + rust from scratch every cloud release.
# The heavy one (console: a cold `npm install` + full Next.js static export,
# force-cache-busted every build) used to dominate the ~20-min build; it is now
# a registry pull.
# console-embed (hanzoai/console Dockerfile.embed) → /dist → webui/dist (go:embed)
# agent-skills (hanzoai/openapi Dockerfile.skills) → /catalog → clients/agentskills/catalog (go:embed)
# cloud-flags (native/flags Dockerfile) → /libhanzo_flags.a → CGO link (clients/featureflags)
# Pinned to ghcr.io so BOTH buildx lanes (release.yml + platform arcbuild) pull
# it directly; the SAME tags are mirrored to registry.hanzo.ai (S3-backed) for
# GET-flow consumers (docker/kaniko/crane). Override any pin with
# --build-arg <NAME>_IMAGE=… — release.yml resolves CONSOLE_IMAGE to a fresh
# console-embed digest, exactly as CONSOLE_CACHEBUST re-fetched console before.
ARG CONSOLE_IMAGE=ghcr.io/hanzoai/console-embed:latest
ARG SKILLS_IMAGE=ghcr.io/hanzoai/agent-skills:latest
ARG FLAGS_IMAGE=ghcr.io/hanzoai/cloud-flags:latest
# ── Go build stage (CGO=1 + SQLCipher — REAL at-rest encryption) ─────────────
# The unified binary embeds IAM (clients/iam) whose per-org store is SQLCipher-
# encrypted (orgIsolation=sqlite), and commerce's per-tenant money DBs likewise.
# A CGO=0 modernc build SILENTLY SHIPS PLAINTEXT. So this builds CGO=1 against
# system libsqlcipher — hanzoai/iam's proven recipe: the `libsqlite3` tag + a
# libsqlcipher symlink + -DSQLITE_HAS_CODEC, with the modernc double-registration
# guard, TestEncryptionProof, and the cek.go golden-vector KAT baked in — so a
# build that fails to link REAL SQLCipher, or that would decrypt existing stores
# differently, produces NO image. alpine3.22 MATCHES the runtime base so the
# libsqlcipher soname the binary links is the SAME one present at runtime. ECR
# Public mirror avoids Docker Hub's 429 rate-limit on shared CI runners.
# ---- agent-skills stage: regenerate the FULL /.well-known/agent-skills catalog
# from the hanzoai/openapi SOT (skills.py) and carry it into the Go embed path
# BEFORE `go build`, the SAME way the console bundle is produced. The committed
# catalog is only the tiny `ai` fallback; prod must embed the full set. FAIL-HARD:
# if the clone/generation can't produce the master index, the image is not built.
FROM public.ecr.aws/docker/library/python:3.12-alpine AS skills
ARG OPENAPI_REPO=https://github.com/hanzoai/openapi.git
ARG OPENAPI_REF=main
RUN apk add --no-cache git && pip install --no-cache-dir pyyaml
WORKDIR /openapi
RUN --mount=type=secret,id=gh_token \
if [ -s /run/secrets/gh_token ]; then \
git config --global url."https://x-access-token:$(cat /run/secrets/gh_token)@github.com/".insteadOf "https://github.com/"; \
fi && \
git clone --depth 1 --branch "${OPENAPI_REF}" "${OPENAPI_REPO}" . && \
python3 skills.py --no-services --out /catalog && \
test -s /catalog/hanzo/index.json
# ── toolchain base images: the golang + alpine FROMs below pull from our own
# GHCR mirror (ghcr.io/hanzoai/mirror/*), pinned by digest. WHY: public.ecr.aws
# rate-limits anonymous pulls (HTTP 429) on shared CI runners and a 429 on ANY
# base pull aborts the release. The mirror packages are 1:1 amd64 copies of the
# upstream public images, digest-pinned for immutability; release.yml logs the
# build into ghcr.io (GH_PAT) before building so they resolve. REFRESH on a
# toolchain bump: crane/regctl copy the new upstream into
# ghcr.io/hanzoai/mirror/<name>:<tag> and repoint the digest below. Canonical
# long-term home is registry.hanzo.ai/hanzoai/mirror/* — repoint once the runners
# carry its IAM pull credentials (follow-up).
FROM public.ecr.aws/docker/library/golang:1.26-alpine3.22@sha256:727cfc3c40be55cd1bc9a4a059406b28a059857e3be752aa9d09531e12c20c56 AS build
# ── console SPA static export (prebuilt → /dist) ─────────────────────────────
FROM ${CONSOLE_IMAGE} AS console
# ── agent-skills catalog (prebuilt → /catalog) ──────────────────────────────
FROM ${SKILLS_IMAGE} AS skills
# ── native flags evaluator staticlib (prebuilt → /libhanzo_flags.a) ──────────
FROM ${FLAGS_IMAGE} AS flagslib
FROM ghcr.io/hanzoai/mirror/golang:1.26-alpine3.22@sha256:47d47cb5cc3c7dac409dcb6c3a98a6263571218046cd02d709527feef804a77c AS build
# CIPHER-FORMAT FREEZE (cek depends on this). The data-plane stores are
# SQLCipher pages in a fixed on-disk format (cipher_compatibility 4). An at-open
# compat pin is infeasible (mattn keys via URI before any pragma), so the format
@@ -138,18 +66,21 @@ RUN set -eux; \
ln -sf "$SC" /usr/lib/libsqlite3.so; \
ln -sf "$SC" /usr/lib/libsqlite3.so.0
WORKDIR /src
# hanzoai/* and luxfi/* are PUBLIC and resolve via the IMMUTABLE public proxy +
# sumdb — go.sum pins those canonical hashes, so a force-re-pointed tag can never
# break the build. GOSUMDB stays ON (a money image must not blanket-disable the
# checksum database); only zap-proto/* is exempt (first-party-direct via GOPRIVATE,
# authenticated git over gh_token). -mod=readonly means the committed go.sum is the
# SOLE source of truth: any drift (a needed hash not present) FAILS the build
# instead of being silently re-recorded. CGO_CFLAGS/LDFLAGS enable the SQLCipher
# codec + URI keying.
# zap-proto/* (all 55 repos) and luxfi/* (all 37 deps here) are PUBLIC and resolve
# via the IMMUTABLE public proxy + sumdb — go.sum pins those canonical hashes, so a
# force-re-pointed tag can never break the build. GOSUMDB stays ON (a money image
# must not blanket-disable the checksum database); github.com/hanzoai/* is the
# exempt namespace — ai, account, commerce, orm, xorm, beego, csqlite and ~30 more
# are PRIVATE repos, so they resolve direct+authenticated (git over gh_token) and
# skip a sumdb that cannot see them. GOPRIVATE named zap-proto until now, which is
# public and was never the reason anything was direct; the private namespace it
# stood for went unnamed and worked only on the GOPROXY `direct` fallback.
# -mod=readonly means the committed go.sum is the SOLE source of truth: any drift
# (a needed hash not present) FAILS the build instead of being silently
# re-recorded. CGO_CFLAGS/LDFLAGS enable the SQLCipher codec + URI keying.
ENV CGO_CFLAGS="-DSQLITE_HAS_CODEC -DSQLITE_USE_URI=1 -I/usr/include/sqlcipher" \
CGO_LDFLAGS="-lsqlcipher" \
GOPRIVATE=github.com/zap-proto/* \
GONOSUMDB=github.com/zap-proto/* \
GOPRIVATE=github.com/hanzoai/* \
GOPROXY=https://proxy.golang.org,direct \
GOFLAGS=-mod=readonly
COPY go.mod go.sum ./
@@ -160,19 +91,22 @@ COPY go.mod go.sum ./
# and resolves fine from a clean cache. That is exactly what wedged the release
# on otel-collector v0.144.10. BUMP THE SUFFIX (-v4 -> -v5) to force a cold
# module cache the next time a phantom pin poisons it.
RUN --mount=type=secret,id=gh_token \
RUN --mount=type=secret,id=GIT_AUTH_TOKEN \
--mount=type=cache,id=cloud-gomod-v4,target=/go/pkg/mod,sharing=locked \
if [ -s /run/secrets/gh_token ]; then \
git config --global url."https://x-access-token:$(cat /run/secrets/gh_token)@github.com/".insteadOf "https://github.com/"; \
if [ -s /run/secrets/GIT_AUTH_TOKEN ]; then \
git config --global url."https://x-access-token:$(cat /run/secrets/GIT_AUTH_TOKEN)@github.com/".insteadOf "https://github.com/"; \
fi && \
go mod download
COPY . .
# Drop the console static bundle into the embed path BEFORE `go build`, so
# //go:embed all:webui/dist bakes it into the binary (same-origin console).
COPY --from=console /out/ /src/webui/dist/
COPY --from=console /dist/ /src/webui/dist/
# Overlay the FULL agent-skills catalog before `go build` so //go:embed all:catalog
# bakes the complete set (all services × brands), not the committed `ai` fallback.
COPY --from=skills /catalog/ /src/clients/agentskills/catalog/
# The native flags staticlib at the exact ${SRCDIR}-relative path the cgo
# directive in clients/featureflags/engine.go links.
COPY --from=flagslib /libhanzo_flags.a /src/native/flags/target/release/libhanzo_flags.a
# RED gate — modernc double-registration guard: 0 modernc under CGO=1, else the
# "sqlite" driver is registered twice (mattn + modernc) → panic at init.
RUN --mount=type=cache,id=cloud-gomod-v4,target=/go/pkg/mod,sharing=locked \
@@ -201,12 +135,18 @@ RUN --mount=type=cache,id=cloud-gomod-v4,target=/go/pkg/mod,sharing=locked \
RUN --mount=type=cache,id=cloud-gomod-v4,target=/go/pkg/mod,sharing=locked \
--mount=type=cache,id=cloud-gobuild-v4,target=/root/.cache/go-build,sharing=locked \
CGO_ENABLED=1 go build -tags "libsqlite3 sqlite_fts5" -ldflags="-s -w" -o /cloud ./cmd/cloud
# The functional smoke prober (cmd/smoke) — a stdlib-only, static binary shipped
# alongside /cloud so the release gate can `docker exec` it against the freshly-built
# image (and any deployment can be smoked via `docker run --entrypoint /smoke ...`).
RUN --mount=type=cache,id=cloud-gomod-v4,target=/go/pkg/mod,sharing=locked \
--mount=type=cache,id=cloud-gobuild-v4,target=/root/.cache/go-build,sharing=locked \
CGO_ENABLED=0 go build -ldflags="-s -w" -o /smoke ./cmd/smoke
# Prove the SHIPPED binary binds sqlite3_* to libsqlcipher, not a plaintext libsqlite3.
RUN readelf -d /cloud | grep -qE 'NEEDED.*(sqlcipher|sqlite3)' || { echo "FATAL: /cloud links no sqlite/sqlcipher .so"; exit 1; }; \
! ldd /cloud 2>/dev/null | grep -E 'libsqlite3' | grep -vq 'libsqlcipher' || { echo "FATAL: /cloud resolves a NON-sqlcipher libsqlite3 (plaintext risk)"; exit 1; }
# ── final image (alpine, NOT scratch — CGO needs libc + libsqlcipher) ─────────
FROM public.ecr.aws/docker/library/alpine:3.22@sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc1f695dce
FROM ghcr.io/hanzoai/mirror/alpine:3.22@sha256:7c8cb692ae09657cbc4a3f3cbd0e8d5a2690ba38386aaaf252dbb060bf5eb2e6
ARG REVISION=unknown
LABEL org.opencontainers.image.revision="${REVISION}" \
org.opencontainers.image.source="https://github.com/hanzoai/cloud"
@@ -220,7 +160,10 @@ LABEL org.opencontainers.image.revision="${REVISION}" \
# (upload-pack / receive-pack --stateless-rpc / fetch) so multi-GB packs stream
# to and from disk with bounded memory instead of buffering whole packs in RAM.
# The `git` apk package carries upload-pack/receive-pack/http-backend/git-remote-https.
RUN apk add --no-cache ca-certificates tzdata sqlcipher-libs git \
# libgcc: the hanzo-flags Rust staticlib (clients/featureflags FFI) references the
# _Unwind_* unwinder symbols; musl needs libgcc_s at load time or the binary fails
# relocation ("Error relocating /cloud: _Unwind_GetIP: symbol not found").
RUN apk add --no-cache ca-certificates tzdata sqlcipher-libs git libgcc \
&& SC="$(find /usr/lib /lib -name 'libsqlcipher.so*' 2>/dev/null | sort | head -1)" \
&& test -n "$SC" \
&& ln -sf "$SC" /usr/lib/libsqlite3.so.0
@@ -229,6 +172,7 @@ COPY --from=build /usr/share/zoneinfo /usr/share/zoneinfo
COPY --from=build /etc/passwd /etc/passwd
COPY --from=build /etc/group /etc/group
COPY --from=build /cloud /cloud
COPY --from=build /smoke /smoke
EXPOSE 8080 9090 9653
USER 65532:65532
ENTRYPOINT ["/cloud"]
+172 -14
View File
@@ -6,31 +6,78 @@ artifact serves `api.hanzo.ai`, `api.lux.cloud`, `api.zoo.cloud`, `api.osage.clo
and every white-label reseller. Brand, enabled subsystems, and org scope are
deployment configuration.
## Open Cloud planes
Spec home: HIP-0129 `hip-0129-open-cloud-planes` (hips repo). This section is a
map, not the spec. One noun, one owner, one route family. No plane reads another
plane's store; imports flow custody-ward only (channels -> integrations, never
reverse).
| Route | Noun | Owner | Tier |
| --- | --- | --- | --- |
| `/v1/connectors` | Custody: per-user BYO external accounts | `clients/integrations` (extends; user scope new) | In flight (branch `feat/connectors`) |
| `/v1/channels` | Transport: portable message envelope, DM pairing, send + inbox | `clients/channels` (new) | Planned (branch `feat/channels` reserved; no transport code yet) |
| `/v1/sync` | Data: bidirectional sync engine | `clients/sync` | Shipped |
| `/v1/automations` | Workflows: flows/runs, goja piece runtime | `clients/automations` | Shipped |
| `/v1/compute/bots` | Hosting: `@hanzo/bot` Node containers | `clients/bots` | Shipped |
| `/v1/tasks` | Durable engine | `clients/tasks` | Shipped |
| `/v1/gpus` + fleet | BYO GPU presence | `clients/fleet` + `clients/visor` | Shipped |
| IAM | Identity: users, orgs, roles | IAM | Shipped |
| KMS | Secret custody: sealed secrets | `clients/kms` | Shipped |
Custody invariants: secrets sealed in KMS at
`/orgs/{org}/users/{user}/connectors/{provider}/{label}`, never in SQLite rows;
verify before store. Refresh is single-flight with rotation resealing; the CLI
does local browser PKCE and posts the bundle to
`POST /v1/connectors/:provider/credential`; cloud owns device-code flows.
Transport invariants: typed actions (`command|url|select|approval`), no raw
string sniffing; pairing codes 8 chars, 1h TTL, max 3 pending per account,
owner bootstrap on first approval.
Container boundary is permanent for native-module, host-filesystem, loop-state,
and vendor-Node work (agent loop, exec/PTY, harnesses, browser, voice, codecs,
Node-bound channels, plugin SDK/loader). The Node plugin SDK is never ported to
Go; cloud extensibility is connectors/automations/tools.
Port roadmap (P1-P15) lives in HIP-0129; do not restate it here. Every claim
carries its tier: Shipped (on main, named package/route), In flight (named
pre-main branch), Planned (backlog id or named reservation).
## Framework doctrine
One way to do everything. Composable, orthogonal, DRY. A new subsystem is a
package under `clients/<name>` that obeys these seams — nothing more.
- **Subsystem shape.** A subsystem exposes `func Mount(app *zip.App, deps cloud.Deps) error`
and self-registers at init with `cloud.Register("<name>", <order>, cloud.Typed(Mount))`
(or `RegisterWithShutdown`). `Mount` wires that subsystem's `/v1/<name>/*` routes
onto the shared `*zip.App`; `cloud.Deps` carries the process-wide handles
(Logger, DataDir, the subsystem `Client` seams). No subsystem reaches into
another's internals.
and is listed in `apps.Wire()` as a `cloud.MountSpec{Name, Mount: cloud.Typed(Mount)}`
(plus `Shutdown`/`OwnsHealth` where it owns them). `Mount` wires that subsystem's
`/v1/<name>/*` routes onto the shared `*zip.App`; `cloud.Deps` carries the
process-wide handles (Logger, DataDir, the subsystem `Client` seams). No
subsystem reaches into another's internals. There is no init()-registry and no
`cloud.Register` — subsystems do NOT self-register.
- **Client seams.** Cross-subsystem calls go through a narrow in-process interface
published in `types` and aliased at the provider, e.g. `commerce.Client =
types.CommerceClient` (`GetOrgConfig` + `CheckEntitlement`). Consumers depend on
the interface, never the implementation; the seam rides zap-proto/zip. Keep each
interface minimal — add a method only when a consumer needs it.
- **Composition root.** `apps/apps.go` blank-imports every subsystem
(its init runs `cloud.Register`), populating `cloud.Registry`. `MountAll`
(build.go) sorts the registry by `Order` and calls `Mount` on each ENABLED
subsystem (`cfg.Enabled`). That ordered blank-import set IS the wiring — there
is no separate `Wire()` function; to add a subsystem you add one import line.
- **Route precedence is a framework guarantee.** The router is zap-proto/fiber
(zip v1.3.0). Most-specific route wins regardless of mount order; a genuine
route CONFLICT panics at mount rather than resolving ambiguously. Subsystems may
therefore mount in any order and still compose deterministically.
- **Composition root.** `apps/apps.go:Wire()` returns `[]cloud.MountSpec` — every
linked subsystem, in mount order, as ONE explicit slice read top-to-bottom.
Slice position IS the order: there is no `Order` field and `MountAll`
(build.go) does NOT sort; it iterates as-given and mounts each ENABLED spec
(`cfg.Enabled`). To add a subsystem you add one line to `Wire()`.
`apps/wire_test.go` freezes the sequence, so a reorder/drop/add fails there.
- **Route precedence.** The router is zap-proto/fiber (zip v1.8.3). Most-specific
route wins regardless of mount order, so subsystems may mount in any order and
still compose deterministically. But precedence is NOT a conflict guard: two
registrations of a byte-identical pattern do NOT panic — fiber MERGES them into
ONE route with both handlers chained, resolving by first-registration. That is
invisible to a `GetRoutes()` entry count (see the bots note below), and it is
NOT distinguishable from a legitimate middleware chain: `app.Post(path, mw1,
mw2, mw3, handler)` is one registration with four handlers (apps/commerce.go:151),
and the whole `/v1/store/*` surface is that shape. A high handler count is
therefore evidence of nothing on its own; only a subsystem that never chains
middleware (bots/visor/runtime) can read `len(Handlers) > 1` as a collision.
- **Per-org data.** The ONE way any subsystem opens a per-org SQLite file is
`cloud.OrgDB(dataDir, org, project, sub)` — or the cached `cloud.OrgStore[T]`
(`NewOrgStore` + `For(org, project)`). Path convention:
@@ -42,6 +89,51 @@ package under `clients/<name>` that obeys these seams — nothing more.
the SOLE driver (blank-imported once, in orgdb.go); subsystems never import a
SQLite driver themselves. The caller owns its schema/migration and Close.
## The route table has three projections, and the router is the source
`serve.go` composes ONE route table and projects it three ways, all after
`MountAll` so each sees a complete table: `/zap` REPLAYS the /v1 handlers
(zapface), the console RENDERS them, and `GET /v1/openapi.json` DESCRIBES them
(`openapi.Mount`). None holds a second copy of anything; none can drift.
- **The spec IS the router.** `openapi.Live(app)` reads
`app.Fiber().GetRoutes(true)` — fiber's own filter drops `Use()` middleware —
and every other function in `openapi/` is a pure function of that `[]Route`.
There is NO checked-in spec file to hand-maintain and no second registry. The
drift guard is `cmd/cloud/openapi_test.go`: a BIJECTION over the fully-mounted
`apps.Wire()` (983 operations / 692 paths / 109 products) — every live route
appears as an operation, every operation is backed by a live route. It is the
only test whose failure means the document lies.
- **Reading the LIVE router is the only total source.** `POST /v1/kms/auth/login`
is registered as `Group("/v1/kms/auth").Post("/login")` — no grep can find that
path; only the assembled router knows it. And the route set is a function of
deployment config (`cfg.Enabled`, plus internal gates like kms's `if kc != nil`),
so **the spec VARIES PER DEPLOYMENT** — correctly: a deployment that does not
mount admin does not advertise it. That is why the document is generated
per-process at request time, not built once in CI.
- **The product axis is mechanical.** The first path segment after `/v1/` IS the
product (`openapi.Product`), tagged onto each operation so a CLI can build
`hanzo <product> <resource> <verb>` with no judgment. It is deliberately NOT the
subsystem name: `clients/billing` also serves `/v1/finance/*`.
- **What the router CANNOT tell you — do not try to fix this in the generator.**
Method, path, path params, and product are derivable; request/response schemas,
query/header params, status codes, and auth are NOT. The router holds a
`func(*zip.Ctx) error`; the request type is a LOCAL inside the handler
(`var req secretPutRequest; json.Unmarshal(ctx.Body(), &req)`), and Go cannot
reflect from a func value into its body. `cloud.Handle[S]` does not help — `S`
is the SERVICE (service.go:90), not the payload; `cloud.Typed` is an
`any→*zip.App` mount adapter. The ONE path to schemas is zip's typed ops
(`zip.Get[In,Out]`), which carry the In/Out types and also yield an MCP tool
from the same registry (zip/openapi.go, zip/mcp.go — today `len(a.ops) == 0`,
so zip's own generator emits nothing here). `GetRoutes()` is a superset of
`app.ops`, so migrating a handler to a typed op adds schema without changing
this pipeline.
- **Catch-alls are opaque, by construction.** `app.Post("/v1/billing/*")` proxies
to another service, so `POST /v1/billing/deposit` is NOT a route in this process
and cannot appear. Measured on the live table: 3 products are wholly opaque
(`bot`, `licensing`, `sentry` — the catch-all IS the product) and 12 more mix
concrete ops with a catch-all hiding an unknown remainder.
## Cross-subsystem seams that are values, not places
- **The per-principal MCP plane is callable in-process.** `clients/automations`
@@ -53,6 +145,36 @@ package under `clients/<name>` that obeys these seams — nothing more.
calls `InvokeTool` with `principal.Org(c)` — same 403 gate, per-org concurrency
bound, one metered unit, one audit record as the HTTP door — so it can never
exceed the caller's authority. Use this seam; never re-implement tool dispatch.
- **"Bot" is three values; each has one home and one namespace.** Do not merge
them and do not let them share a route prefix — they did once, and the router
resolves byte-identical patterns by first-registration with no panic (it MERGES
the handlers, so counting `GetRoutes()` entries cannot see it), and visor's
machine list silently answered the console's run list.
(1) A bot RUN — a task the runtime executes on a surface — is `clients/bots` at
`/v1/bots`. (2) A bot MACHINE — visor-provisioned compute of kind=bot plus its
agent binding — is `clients/visor` at `/v1/compute/bots`; what it rents you is
compute, so it nests in visor's domain. (3) The runtime SERVICE — the TS bot
(channels/skills), never reimplemented in Go — is reached through
`clients/runtime`, which is a TRANSPORT, not a domain: base address, identity,
framing, cleartext policy, and the `/v1/bot/*` ops face. It is named for what it
does, not for the host it dials, and it must never import `bots`/`coding` — each
of those owns its own wire stub (`bots/wire.go`, `coding/task.go`) and speaks
through the seam. That isolation is what makes the HIP-0106/HIP-0120 ZAP swap a
seam swap instead of a rewrite.
- **Cloud owns policy; the runtime owns the run. Do not copy state you do not
own.** `clients/bots` holds NO store. The sandbox lives in the bot runtime,
keyed in the runtime's own tenant store, which is the only thing that knows
whether a run is alive — so list and stop PROXY it, gated by cloud's
principal/org. A cloud-side registry was tried and was wrong: it minted an id
the runtime had never heard of, so it listed runs that did not exist and
"stopped" runs that were never started. Isolation holds because the org is the
validated one cloud sends, never a client's, and the runtime keys every run
under `tenants/{org}/`.
- **Absence is only meaningful from a callee that could have said otherwise.**
`runtime.ErrNotFound` (the operation ANSWERED "no such target") is separate from
`runtime.ErrNotServed` (the operation does not exist). Conflating them makes a
stop that cannot fail: a runtime without the route reports absent for EVERY run,
so "already gone" becomes permanently true. A bare 404 is 502, never success.
- **The Business AI Guide (`clients/guide`, `/v1/guide/*`)** is the on-site launch
checklist: a pure engine (`curriculum.go` — parse/validate/next-step/dependency
gating over plain data) + per-org progress (`cloud.OrgStore[*Store]`) + an
@@ -100,3 +222,39 @@ Import path (already-incorporated orgs): Google Drive → data room, a Google Sh
captable, via the `google` OAuth provider now completed in `clients/integrations`
(token custodied in KMS; the automations `google` connector shares the same token).
Runbook: `docs/company-dogfood.md`.
## Deploy plane (`clients/deploy`, `/v1/deploy`)
Native ArgoCD-grade GitOps console over the operator-managed fleet, parallel to
`/v1/git`: each `hanzo.ai/v1` App CR IS the Application, and the plane OBSERVES the
operator's reconcile — `GET /v1/deploy/applications` (fleet list), `/{name}/tree`
(ownerRef resource tree + per-node health/sync), `/{name}/resource/{ref}` (live
manifest + desired-vs-live diff), `/{name}/logs`; `POST /{name}/rollback` pins the CR
image to a prior semver and `/{name}/sync` requests a reconcile. SUPERADMIN-only on
`c.IsAdmin()`, fail-closed; Secret nodes are never surfaced. `engine.go` embeds the argo
`gitops-engine` (`hanzoai/deploy/gitops-engine` v0.7.2, no replace) in-process for the
reconcile half behind `DEPLOY_ENGINE_ENABLED` (default off), with a prune-safety fuse.
## The `hanzo` CLI targets THIS binary — one contract, one IAM login
The `hanzo` CLI (`cli/`) is the same unified binary; its control-plane verbs speak the
routes THIS process serves, authorized off a plain `hanzo login` (the IAM access token is
the final bearer fallback — no `--platform-token`). The ONE contract, no TS-Dokploy drift:
- `hanzo apps list|get``GET /v1/paas/apps[/{app}]` (`clients/paas` fleet drift board)
- `hanzo deploy <app>``POST /v1/paas/apps/{app}/deploy` — a zero-downtime ROLLING
RESTART (stamps the Deployment pod-template `hanzo.ai/restartedAt` annotation; never
changes the declared TAG — that stays a git commit CD reconciles). `--env` picks the ns.
- `hanzo clusters list|get``GET /v1/clusters` (`clients/visor`, tenant-scoped)
- `hanzo build``POST /v1/runner` (native buildkit fabric)
`/v1/paas/*` auth mirrors `/v1/runner` (`clients/platform/runner.go`): the `guard` admits a
validated principal who is SuperAdmin OR OrgAdmin, then each handler CONFINES a non-super
caller to the platform namespaces its own validated org owns (`scopedNamespaces`, keyed on
`principal.Org` — a tenant admin can never observe/restart another org's, or a platform,
app; `?org=` cannot widen it). The rolling restart needs `patch` on `apps/deployments`
(ClusterRole/cloud, universe `infra/k8s/cloud/rbac.yaml`). There is NO `/v1/apps`,
`/v1/org/{org}/cluster`, or `/v1/platform/projects` CLI path — the first two never existed
here (TS-Dokploy contract, 404), and `/v1/platform/*` needs a co-resident IAM store this
deployment does not fold in (IAM runs as a separate svc) so it 500s; the live apps backend
is `/v1/paas`, whose board reads k8s directly with no IAM-store dependency.
+15 -1
View File
@@ -24,7 +24,7 @@ OPENAPI_DIR ?= ../openapi
# forces the fork to modernc too so the whole binary registers "sqlite" once.
CGO_ENABLED ?= 0
.PHONY: help webui agentskills build build-standalone hanzo run smoke test test-cgo vet tidy docker docker-push clean
.PHONY: help native webui deploy-ui agentskills build build-standalone hanzo run smoke test test-cgo vet tidy docker docker-push clean
help: ## Show this help.
@awk 'BEGIN{FS=":.*##";printf "\nUsage: make <target>\n\nTargets:\n"} /^[a-zA-Z_-]+:.*##/{printf " \033[36m%-16s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST)
@@ -40,6 +40,17 @@ webui: ## Build the real console static bundle into webui/dist (go:embed source)
cp -r "$(CONSOLE_DIR)/out/." webui/dist/
@echo ">> embedded real console bundle into webui/dist (index.html $$(wc -c < webui/dist/index.html) bytes)"
deploy-ui: ## Build the monochrome ArgoCD dashboard bundle into clients/deploy/webui/dist (go:embed source). DEPLOY_DIR=<path to hanzoai/deploy>.
@command -v yarn >/dev/null 2>&1 || { echo "yarn is required to build the deploy dashboard bundle"; exit 1; }
@test -f "$(DEPLOY_DIR)/ui/package.json" || { echo "deploy checkout not found at $(DEPLOY_DIR) — set DEPLOY_DIR=<path to hanzoai/deploy on rebrand/hanzo-monochrome>"; exit 1; }
@test -d "$(DEPLOY_DIR)/ui/node_modules" || (cd "$(DEPLOY_DIR)/ui" && yarn install --frozen-lockfile)
cd "$(DEPLOY_DIR)/ui" && NODE_OPTIONS=--max-old-space-size=8192 yarn build
# Overlay the fresh bundle, keeping only the tracked fallback (.gitignore +
# index.html shell); the real 43MB bundle is build-time-only (gitignored).
find clients/deploy/webui/dist -mindepth 1 -maxdepth 1 ! -name .gitignore -exec rm -rf {} +
cp -r "$(DEPLOY_DIR)/ui/dist/app/." clients/deploy/webui/dist/
@echo ">> embedded monochrome ArgoCD bundle into clients/deploy/webui/dist (index.html $$(wc -c < clients/deploy/webui/dist/index.html) bytes)"
agentskills: ## Regenerate the FULL agent-skills catalog into clients/agentskills/catalog (go:embed source) from the openapi SOT. OPENAPI_DIR=<path to openapi>.
@test -f "$(OPENAPI_DIR)/skills.py" || { echo "openapi checkout not found at $(OPENAPI_DIR) — set OPENAPI_DIR=<path> or clone hanzoai/openapi"; exit 1; }
# skills.py rewrites the whole catalog dir; the .gitignore keeps only the tiny
@@ -84,3 +95,6 @@ docker-push: docker ## Push the Docker image to ghcr.io. Requires docker login.
clean: ## Remove built artifacts.
rm -rf bin
native: ## Build the native flags evaluator staticlib (required for CGO=1 builds/tests).
cargo build --release --manifest-path native/flags/Cargo.toml
+158 -108
View File
@@ -34,6 +34,7 @@ package apps
import (
"context"
"fmt"
"os"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
@@ -45,14 +46,15 @@ import (
"github.com/hanzoai/authz"
"github.com/hanzoai/licensing"
"github.com/hanzoai/metrics"
o11ymod "github.com/hanzoai/o11y"
// In-repo subsystem packages (clients/*). Each exports a Mount (and, where it
// owns process-lifetime resources, a Shutdown); Wire references them directly.
"github.com/hanzoai/cloud/clients/account"
"github.com/hanzoai/cloud/clients/admin"
"github.com/hanzoai/cloud/clients/admission"
"github.com/hanzoai/cloud/clients/ads"
"github.com/hanzoai/cloud/clients/affiliates"
"github.com/hanzoai/cloud/clients/agent"
"github.com/hanzoai/cloud/clients/agents"
"github.com/hanzoai/cloud/clients/agentskills"
"github.com/hanzoai/cloud/clients/analytics"
@@ -61,23 +63,22 @@ import (
"github.com/hanzoai/cloud/clients/automations"
"github.com/hanzoai/cloud/clients/base"
"github.com/hanzoai/cloud/clients/billing"
"github.com/hanzoai/cloud/clients/bot"
"github.com/hanzoai/cloud/clients/bots"
"github.com/hanzoai/cloud/clients/captable"
"github.com/hanzoai/cloud/clients/catalogsync"
"github.com/hanzoai/cloud/clients/cloudflare"
"github.com/hanzoai/cloud/clients/code"
"github.com/hanzoai/cloud/clients/company"
"github.com/hanzoai/cloud/clients/connectorruntime"
"github.com/hanzoai/cloud/clients/content"
"github.com/hanzoai/cloud/clients/crm"
"github.com/hanzoai/cloud/clients/cron"
"github.com/hanzoai/cloud/clients/dataroom"
"github.com/hanzoai/cloud/clients/deploy"
"github.com/hanzoai/cloud/clients/dns"
"github.com/hanzoai/cloud/clients/do"
"github.com/hanzoai/cloud/clients/entitlements"
"github.com/hanzoai/cloud/clients/eval"
"github.com/hanzoai/cloud/clients/exec"
"github.com/hanzoai/cloud/clients/featureflags"
"github.com/hanzoai/cloud/clients/flags"
"github.com/hanzoai/cloud/clients/framework"
"github.com/hanzoai/cloud/clients/functions"
"github.com/hanzoai/cloud/clients/gateway"
@@ -85,6 +86,7 @@ import (
"github.com/hanzoai/cloud/clients/graph"
"github.com/hanzoai/cloud/clients/guide"
"github.com/hanzoai/cloud/clients/iam"
"github.com/hanzoai/cloud/clients/iam2"
"github.com/hanzoai/cloud/clients/ingress"
"github.com/hanzoai/cloud/clients/integrations"
"github.com/hanzoai/cloud/clients/kafka"
@@ -107,12 +109,14 @@ import (
"github.com/hanzoai/cloud/clients/provisioning"
"github.com/hanzoai/cloud/clients/pubsub"
"github.com/hanzoai/cloud/clients/referrals"
"github.com/hanzoai/cloud/clients/runtime"
"github.com/hanzoai/cloud/clients/sbom"
"github.com/hanzoai/cloud/clients/security"
"github.com/hanzoai/cloud/clients/settings"
"github.com/hanzoai/cloud/clients/sign"
"github.com/hanzoai/cloud/clients/social"
"github.com/hanzoai/cloud/clients/storage"
"github.com/hanzoai/cloud/clients/sync"
"github.com/hanzoai/cloud/clients/tasks"
"github.com/hanzoai/cloud/clients/team"
"github.com/hanzoai/cloud/clients/templates"
@@ -160,6 +164,22 @@ func init() {
})
}
// identitySpec selects the ONE identity backend that owns /v1/iam/* (+ /login/oauth/*)
// for this boot. CLOUD_IAM_IMPL=iam2 picks the clean-room iam2 (zip+orm, beego-free);
// anything else — including unset, the production default — keeps the legacy beego
// Casdoor embed, byte-for-byte today's behavior. The two impls register the SAME
// absolute prefixes and therefore cannot co-mount, so selection (this func) stays
// separate from activation (cfg.Enabled): exactly one spec occupies the identity slot
// in Wire, preserving mount order either way. os.Getenv (not the unexported
// cloud.getenv, which is unreachable from package apps) is the read — CLOUD_IAM_IMPL is
// the deliberate, off-by-default opt-in that keeps iam2 inert until a canary flips it.
func identitySpec() cloud.MountSpec {
if os.Getenv("CLOUD_IAM_IMPL") == "iam2" {
return cloud.MountSpec{Name: "iam2", Mount: iam2.Mount}
}
return cloud.MountSpec{Name: "iam", Mount: iam.Mount}
}
// Wire returns every linked subsystem as a cloud.MountSpec, in mount order. The
// slice position IS the order: cloud.MountAll iterates it as-given, registering each
// subsystem's teardown as a zip shutdown hook so teardown runs in reverse (LIFO).
@@ -168,178 +188,212 @@ func init() {
func Wire() []cloud.MountSpec {
return []cloud.MountSpec{
// embedded NATS :4222 + JetStream.
{Name: "pubsub", Mount: cloud.Typed(pubsub.Mount), Shutdown: pubsub.Shutdown},
{Name: "pubsub", Mount: pubsub.Mount, Shutdown: pubsub.Shutdown},
// embedded Kafka adaptor :9092.
{Name: "kafka", Mount: cloud.Typed(kafka.Mount), Shutdown: kafka.Shutdown},
{Name: "kafka", Mount: kafka.Mount, Shutdown: kafka.Shutdown},
// /.well-known/agent-skills/* — before IAM's /.well-known/* wildcard (50).
{Name: "agentskills", Mount: cloud.Typed(agentskills.Mount)},
{Name: "agentskills", Mount: agentskills.Mount},
// Insights feature-flag evaluation seam (no routes; a hot value plane).
{Name: "featureflags", Mount: cloud.Typed(featureflags.Mount)},
{Name: "flags", Mount: flags.Mount, Shutdown: flags.Shutdown, OwnsHealth: true},
// Embedded KMS secrets plane /v1/kms/*. OwnsHealth: serves its own fail-closed
// /v1/kms/health (the generic always-ok route must not shadow it). Fails closed
// until the operator injects CLOUD_KMS_MASTER_KEY_REF. (Its in-process client
// factory is registered separately via cloud.RegisterKMSClientFactory.)
{Name: "kms", Mount: cloud.Typed(kms.Mount), OwnsHealth: true},
{Name: "kms", Mount: kms.Mount, OwnsHealth: true},
// hanzoai/metrics — native o11y. It declares its OWN narrow metrics.Deps (no
// hanzoai/cloud import), so Typed cannot adapt it; mountMetrics builds that Deps
// from cloud.Deps and calls metrics.Mount explicitly.
{Name: "metrics", Mount: mountMetrics},
// Embedded runtime edge (/v1/ingress/*). STAGED — edge listeners stay off unless
// the operator names "ingress" in CLOUD_ENABLE.
{Name: "ingress", Mount: cloud.Typed(ingress.Mount), Shutdown: ingress.Shutdown},
{Name: "ingress", Mount: ingress.Mount, Shutdown: ingress.Shutdown},
// SPECIFIC self-service routes (/v1/iam/{keys,onboard}, /v1/csrf, /v1/embed-status,
// /v1/commerce/topup/wallet). MUST mount before the IAM /v1/iam/* wildcard (50) so
// they win Fiber's first-match scan (framework-guaranteed since zip v1.3.0).
{Name: "account", Mount: cloud.Typed(account.MountAccount)},
{Name: "account", Mount: account.MountAccount},
// Embedded IAM identity plane (/v1/iam/*, /.well-known/*, /login/oauth/*, /_/iam/*,
// /cas/*, /scim/*) — the identity authority, mounts before its dependents. STAGED:
// the operator adds "iam" to --enable only after IAM config + the fold are verified.
{Name: "iam", Mount: cloud.Typed(iam.Mount)},
// Which IMPLEMENTATION owns these prefixes is selected by CLOUD_IAM_IMPL
// (identitySpec): the clean-room iam2 (zip+orm, beego-free) when =="iam2", else the
// legacy beego Casdoor embed — the default (unset = today's behavior, byte-for-byte).
// Both register the SAME absolute paths and cannot co-mount, so this is an either/or
// switch at this ONE slot, never a shadow prefix.
identitySpec(),
// Embedded Base app engine + viral waitlist (/v1/waitlist/*). STAGED behind
// CLOUD_BASE_EMBED. OwnsHealth: native /v1/base/health.
{Name: "base", Mount: cloud.Typed(base.Mount), Shutdown: base.Shutdown, OwnsHealth: true},
// In-repo o11y READ plane + the runtime-handler install (o11y.SetHandler). Every
// specific /v1/o11y/* route registers INSIDE this one mount, hence BEFORE the
// hanzoai/o11y module wildcard (70) — Fiber's in-order match gives them precedence.
// OwnsHealth: the module's order-70 co-entry below owns the single /v1/o11y/health.
{Name: "o11y", Mount: o11y.MountO11y, Shutdown: o11y.ShutdownO11y, OwnsHealth: true},
// hanzoai/o11y module wildcard /v1/o11y/* — co-owner of the ONE o11y concept with
// the in-repo entry above (same name), delegated to via o11y.SetHandler.
{Name: "o11y", Mount: cloud.Typed(o11ymod.Mount)},
{Name: "authz", Mount: cloud.Typed(authz.Mount)},
{Name: "base", Mount: base.Mount, Shutdown: base.Shutdown, OwnsHealth: true},
// The ONE observability subsystem: the in-repo o11y READ plane + runtime-handler
// install (o11y.SetHandler), with the hanzoai/o11y module wildcard /v1/o11y/*
// folded in as the TERMINAL sub-mount INSIDE o11y.MountO11y. Every specific
// /v1/o11y/* route registers before that wildcard, so Fiber's in-order match gives
// them precedence. NOT OwnsHealth: /v1/o11y/health stays the generic always-ok
// route (registered before MountAll), exactly as when the former module co-entry —
// which also set OwnsHealth=false — triggered it.
{Name: "o11y", Mount: o11y.MountO11y, Shutdown: o11y.ShutdownO11y},
{Name: "authz", Mount: authz.Mount},
// Embedded commerce plane /v1/commerce/*, /_/commerce/* — the hanzoai/commerce
// MODULE via the adapter in commerce.go (un-forked; the in-process
// CommerceClient is wired directly in pickCommerceClient).
{Name: "commerce", Mount: cloud.Typed(mountCommerce)},
// hanzoai/licensing. Its Mount is func(any, cloud.Deps) error — a MountFunc
// already — so Wire references it DIRECTLY, not through Typed.
{Name: "commerce", Mount: mountCommerce},
{Name: "licensing", Mount: licensing.Mount},
{Name: "plans", Mount: cloud.Typed(plan.Mount), OwnsHealth: true},
{Name: "pricing", Mount: cloud.Typed(pricing.Mount), OwnsHealth: true},
// clients/plan.Mount. Enable id normalized "plans" -> "plan" to match the
// package + generated cmd/plan (one subsystem, one name). Its product routes
// stay /v1/plans/* (incl. the OwnsHealth /v1/plans/health probe) — unchanged.
{Name: "plan", Mount: plan.Mount, OwnsHealth: true},
{Name: "pricing", Mount: pricing.Mount, OwnsHealth: true},
// /v1/s3/buckets/* + /v1/s3/health. Mounts BEFORE provisioning (120) so its static
// routes win over provisioning's /v1/s3/:name. OwnsHealth (real fail-closed probe).
{Name: "storage", Mount: cloud.Typed(storage.Mount), OwnsHealth: true},
{Name: "storage", Mount: storage.Mount, OwnsHealth: true},
// Provisioning control plane: /v1/sql,/v1/vector,/v1/datastore,/v1/kv,/v1/search,/v1/s3,/v1/docdb.
{Name: "provisioning", Mount: cloud.Typed(provisioning.Mount)},
{Name: "billing", Mount: cloud.Typed(billing.Mount)},
{Name: "provisioning", Mount: provisioning.Mount},
{Name: "billing", Mount: billing.Mount},
// CATCH-ALL /v1/billing/* + /v1/commerce/* data bridges — AFTER clients/billing
// (121) + the commerce embed (100). Same clients/account package as "account" (48).
{Name: "account-bridge", Mount: cloud.Typed(account.MountBridge)},
{Name: "do", Mount: cloud.Typed(do.Mount)},
{Name: "platform", Mount: cloud.Typed(platform.Mount), OwnsHealth: true},
{Name: "projects", Mount: cloud.Typed(projects.Mount)},
{Name: "prompts", Mount: cloud.Typed(prompts.Mount)},
{Name: "agents", Mount: cloud.Typed(agents.Mount), Shutdown: agents.Shutdown},
{Name: "account-bridge", Mount: account.MountBridge},
{Name: "do", Mount: do.Mount},
{Name: "platform", Mount: platform.Mount, OwnsHealth: true},
{Name: "projects", Mount: projects.Mount},
// The /v1/dns forward head: relays the console DNS dashboard to the DNS
// control plane under the caller's own validated bearer (clients/dns).
{Name: "dns", Mount: dns.Mount},
{Name: "prompts", Mount: prompts.Mount},
{Name: "agents", Mount: agents.Mount, Shutdown: agents.Shutdown},
// The unified AI login manager registry (/v1/links). Mounts AFTER agents so
// a link revoke can stop the affected agent sessions in-process.
{Name: "link", Mount: cloud.Typed(link.Mount), Shutdown: link.Shutdown},
{Name: "wallets", Mount: cloud.Typed(wallets.Mount), Shutdown: ctxShutdown(wallets.Shutdown)},
{Name: "link", Mount: link.Mount, Shutdown: link.Shutdown},
{Name: "wallets", Mount: wallets.Mount, Shutdown: ctxShutdown(wallets.Shutdown)},
// x402 pay-per-use: settles a signed ERC-3009 authorization to a recipient
// wallet through the metering spine. Mounts AFTER wallets (it resolves the
// recipient via wallets.ResolvePaymentTarget) and provides the Enforce
// middleware a marketplace applies to its priced routes.
{Name: "x402", Mount: cloud.Typed(x402.Mount), Shutdown: ctxShutdown(x402.Shutdown)},
{Name: "paas", Mount: cloud.Typed(paas.Mount), OwnsHealth: true},
{Name: "x402", Mount: x402.Mount, Shutdown: ctxShutdown(x402.Shutdown)},
{Name: "paas", Mount: paas.Mount, OwnsHealth: true},
// GitOps deploy dashboard /v1/deploy/* (the ArgoCD-grade fleet view over the
// operator App CRs). After paas so the release seam paas installs is registered
// before a gitops rollback delegates to it; owns its own /v1/deploy/health.
{Name: "deploy", Mount: cloud.Typed(deploy.Mount), OwnsHealth: true},
{Name: "functions", Mount: cloud.Typed(functions.Mount)},
{Name: "tracker", Mount: cloud.Typed(tracker.Mount)},
{Name: "templates", Mount: cloud.Typed(templates.Mount)},
{Name: "framework", Mount: cloud.Typed(framework.Mount), Shutdown: ctxShutdown(framework.Shutdown)},
{Name: "knowledge", Mount: cloud.Typed(knowledge.Mount)},
{Name: "deploy", Mount: deploy.Mount, OwnsHealth: true},
{Name: "functions", Mount: functions.Mount},
{Name: "tracker", Mount: tracker.Mount},
{Name: "templates", Mount: templates.Mount},
{Name: "framework", Mount: framework.Mount, Shutdown: ctxShutdown(framework.Shutdown)},
{Name: "knowledge", Mount: knowledge.Mount},
// Marketing content loop /v1/content/* (generate → CMS → transition → publish).
// After framework (its DocType store the ops read/write) + knowledge (the sibling
// framework lane); before the AI /v1/* catch-all so /v1/content/* resolves here.
// CRUD/tenancy/install are framework's; this adds the board, lifecycle transition,
// and the generate/publish orchestration over the zen5 + studio + social edges.
{Name: "content", Mount: cloud.Typed(content.Mount), Shutdown: ctxShutdown(content.Shutdown)},
{Name: "content", Mount: content.Mount, Shutdown: ctxShutdown(content.Shutdown)},
// Reverse storefront loop: consume the commerce COMMERCE stream (product.created)
// → content.EnsureCatalogAsset (render the new product's ecom asset, design==slug).
// After content (whose EnsureCatalogAsset it drives). Inert until CLOUD_COMMERCE_NATS_URL
// names the NATS carrying commerce catalog events — the reverse of the forward edge.
{Name: "catalogsync", Mount: cloud.Typed(catalogsync.Mount), Shutdown: catalogsync.Shutdown},
{Name: "ml", Mount: cloud.Typed(ml.Mount), OwnsHealth: true},
{Name: "usage", Mount: cloud.Typed(usage.Mount)},
{Name: "crm", Mount: cloud.Typed(crm.Mount)},
{Name: "catalogsync", Mount: catalogsync.Mount, Shutdown: catalogsync.Shutdown},
{Name: "ml", Mount: ml.Mount, OwnsHealth: true},
{Name: "usage", Mount: usage.Mount},
{Name: "crm", Mount: crm.Mount},
// Native /v1/marketing/* — the in-process fold of github.com/hanzoai/marketing
// (per-org campaign store on Base/SQLite), twin of crm. Owns a DB handle, so
// its Shutdown closes it cleanly on SIGTERM (ctxShutdown adapts func() error).
{Name: "marketing", Mount: cloud.Typed(marketing.Mount), Shutdown: ctxShutdown(marketing.Shutdown)},
{Name: "marketing", Mount: marketing.Mount, Shutdown: ctxShutdown(marketing.Shutdown)},
// Native /v1/ads/* — the net-new per-org ad-campaign store on Base/SQLite,
// twin of crm/marketing. Owns a DB handle, so its Shutdown closes it cleanly
// on SIGTERM (ctxShutdown adapts func() error).
{Name: "ads", Mount: cloud.Typed(ads.Mount), Shutdown: ctxShutdown(ads.Shutdown)},
{Name: "ads", Mount: ads.Mount, Shutdown: ctxShutdown(ads.Shutdown)},
// Native /v1/social/* — the in-process fold of the live social stack
// (github.com/hanzoai/social: social-backend/frontend/orchestrator, a Postiz-style
// scheduler), a per-org accounts+posts store on Base/SQLite, twin of crm. Owns a DB
// handle, so its Shutdown closes it cleanly on SIGTERM (ctxShutdown adapts func() error).
{Name: "social", Mount: cloud.Typed(social.Mount), Shutdown: ctxShutdown(social.Shutdown)},
{Name: "analytics", Mount: cloud.Typed(analytics.Mount), OwnsHealth: true},
{Name: "git", Mount: cloud.Typed(git.Mount)},
{Name: "visor", Mount: cloud.Typed(visor.Mount)},
{Name: "social", Mount: social.Mount, Shutdown: ctxShutdown(social.Shutdown)},
{Name: "analytics", Mount: analytics.Mount, OwnsHealth: true},
{Name: "git", Mount: git.Mount},
// Universal sync (/v1/sync/links + engine). Registers the cloud.SyncEngine the
// GitHub/Gitea webhooks enqueue to; git is its first provider. Owns per-org
// DB handles, so its Shutdown closes them on SIGTERM.
{Name: "sync", Mount: sync.Mount, Shutdown: ctxShutdown(sync.Shutdown)},
{Name: "visor", Mount: visor.Mount},
// Cap table on Base via goja. STAGED behind CLOUD_ENABLE.
{Name: "captable", Mount: cloud.Typed(captable.Mount), Shutdown: captable.Shutdown},
{Name: "code", Mount: cloud.Typed(code.Mount), Shutdown: code.Shutdown},
{Name: "zero-trust", Mount: cloud.Typed(zt.Mount)},
{Name: "captable", Mount: captable.Mount, Shutdown: captable.Shutdown},
{Name: "code", Mount: code.Mount, Shutdown: code.Shutdown},
{Name: "zero-trust", Mount: zt.Mount},
// Data rooms via goja + per-tenant Base. STAGED behind CLOUD_ENABLE. OwnsHealth.
{Name: "dataroom", Mount: cloud.Typed(dataroom.Mount), Shutdown: dataroom.Shutdown, OwnsHealth: true},
{Name: "graph", Mount: cloud.Typed(graph.Mount)},
{Name: "security", Mount: cloud.Typed(security.Mount), Shutdown: ctxShutdown(security.Shutdown), OwnsHealth: true},
{Name: "integrations", Mount: cloud.Typed(integrations.Mount), Shutdown: integrations.Shutdown},
{Name: "sbom", Mount: cloud.Typed(sbom.Mount), OwnsHealth: true},
{Name: "team", Mount: cloud.Typed(team.Mount), Shutdown: ctxShutdown(team.Shutdown)},
{Name: "settings", Mount: cloud.Typed(settings.Mount), Shutdown: settings.Shutdown},
{Name: "notify", Mount: cloud.Typed(notify.Mount), OwnsHealth: true},
{Name: "gateway", Mount: cloud.Typed(gateway.Mount)},
{Name: "entitlements", Mount: cloud.Typed(entitlements.Mount), Shutdown: entitlements.Shutdown},
{Name: "exec", Mount: cloud.Typed(exec.Mount)},
{Name: "websearch", Mount: cloud.Typed(websearch.Mount)},
{Name: "world", Mount: cloud.Typed(world.Mount), Shutdown: ctxShutdown(world.Shutdown)},
{Name: "bot", Mount: cloud.Typed(bot.Mount)},
{Name: "authors", Mount: cloud.Typed(authors.Mount), Shutdown: ctxShutdown(authors.Shutdown)},
{Name: "bots", Mount: cloud.Typed(bots.Mount)},
{Name: "audit", Mount: cloud.Typed(auditlog.Mount)},
{Name: "affiliates", Mount: cloud.Typed(affiliates.Mount)},
{Name: "dataroom", Mount: dataroom.Mount, Shutdown: dataroom.Shutdown, OwnsHealth: true},
{Name: "graph", Mount: graph.Mount},
{Name: "security", Mount: security.Mount, Shutdown: ctxShutdown(security.Shutdown), OwnsHealth: true},
{Name: "integrations", Mount: integrations.Mount, Shutdown: integrations.Shutdown},
// Per-org Cloudflare asset plane /v1/integrations/cloudflare/{pages,workers,r2,kv,d1}/*.
// Mounts AFTER integrations because it reads the org's Cloudflare token through
// the integrations custody seam (integrations.TokenFor) — one token, one
// custody boundary. Stateless: no store, no shutdown.
{Name: "cloudflare", Mount: cloudflare.Mount},
{Name: "sbom", Mount: sbom.Mount, OwnsHealth: true},
{Name: "team", Mount: team.Mount, Shutdown: ctxShutdown(team.Shutdown)},
{Name: "settings", Mount: settings.Mount, Shutdown: settings.Shutdown},
{Name: "notify", Mount: notify.Mount, OwnsHealth: true},
{Name: "gateway", Mount: gateway.Mount},
{Name: "entitlements", Mount: entitlements.Mount, Shutdown: entitlements.Shutdown},
{Name: "exec", Mount: exec.Mount},
{Name: "websearch", Mount: websearch.Mount},
{Name: "world", Mount: world.Mount, Shutdown: ctxShutdown(world.Shutdown)},
// The bot runtime's ops face (/v1/bot/*). The transport itself is domain-free;
// the run control plane is "bots" below.
{Name: "runtime", Mount: runtime.Mount},
{Name: "authors", Mount: authors.Mount, Shutdown: ctxShutdown(authors.Shutdown)},
{Name: "bots", Mount: bots.Mount},
{Name: "audit", Mount: auditlog.Mount},
{Name: "affiliates", Mount: affiliates.Mount},
// Hanzo Sign (e-signature) via goja + per-tenant Base. STAGED behind CLOUD_ENABLE. OwnsHealth.
{Name: "sign", Mount: cloud.Typed(sign.Mount), Shutdown: sign.Shutdown, OwnsHealth: true},
{Name: "product", Mount: cloud.Typed(product.Mount)},
{Name: "evals", Mount: cloud.Typed(eval.Mount)},
{Name: "treasury", Mount: cloud.Typed(treasury.Mount), Shutdown: ctxShutdown(treasury.Shutdown)},
{Name: "admin", Mount: cloud.Typed(admin.Mount)},
{Name: "tasks", Mount: cloud.Typed(tasks.Mount)},
// Platform cron: durable schedules on the shared tasks engine replacing
// every k8s CronJob — entries are cron.hanzo.ai ConfigMaps (universe git),
// runs visible in the Tasks console. Mounts no routes; starts after the
// engine is wired.
{Name: "cron", Mount: cloud.Typed(cron.Mount)},
{Name: "automations", Mount: cloud.Typed(automations.Mount), Shutdown: automations.Shutdown},
// Native single-connector execution (HIP-0126): runs an ActivePieces JS
// connector action in-process via goja (clients/connectorruntime), retiring
// the standalone auto Node engine. Mounts POST /v1/automations/connectors/:id/run,
// paired with the automations catalogue above; STAGED like the rest.
{Name: "connectorruntime", Mount: cloud.Typed(connectorruntime.Mount)},
{Name: "sign", Mount: sign.Mount, Shutdown: sign.Shutdown, OwnsHealth: true},
{Name: "product", Mount: product.Mount},
{Name: "evals", Mount: eval.Mount},
{Name: "treasury", Mount: treasury.Mount, Shutdown: ctxShutdown(treasury.Shutdown)},
{Name: "admin", Mount: admin.Mount},
// Launch-control gate (per-service waitlist): the COMPLETE feature — host→service
// registry + brand seed + the waitlist.<svc> switch registration + the
// /v1/flags/waitlist (and /v1/admission/mode compat) mode read + the Enforce
// middleware — COMPOSING the flags engine one-way (flags.Bool/Register/
// SetPlatformSwitch; flags never imports admission). Mounts AFTER flags so the
// engine's platform-switch plane is installed first; the admin board is the
// /v1/admin/services lens over it. Owns the registry store handle → Shutdown.
{Name: "admission", Mount: admission.Mount, Shutdown: ctxShutdown(admission.Shutdown)},
// Tasks: the durable workflow/UI surface AND platform cron (durable schedules
// on the same shared engine, replacing every k8s CronJob). cron was a separate
// Wire entry; it mounts no routes and only registers schedules, so it is folded
// in as a sub-mount of tasks.Mount — ONE tasks subsystem.
{Name: "tasks", Mount: tasks.Mount},
// Automations: the connector catalogue + flow engine AND native single-connector
// execution (POST /v1/automations/connectors/:id/run, HIP-0126). The connector
// runner mounts no other routes, so it is folded in as a sub-mount of
// automations.Mount (was a separate "connectorruntime" entry) — ONE subsystem.
{Name: "automations", Mount: automations.Mount, Shutdown: automations.Shutdown},
// Unified tool plane: /v1/tools/* — the ONE registry (connectors, functions,
// agents, skills, external MCP servers, full-cloud-control /v1 routes), per-org
// activation, and the unified MCP endpoint. Sources register into it from their
// own Mounts, so mount position is not load-bearing (List/Dispatch run at request
// time); placed after automations, before the zen/ai catch-all so /v1/tools wins.
{Name: "tools", Mount: cloud.Typed(tools.Mount), Shutdown: tools.Shutdown},
{Name: "tools", Mount: tools.Mount, Shutdown: tools.Shutdown},
// Marketplace: /v1/marketplace/* — listing/discovery/install over the tool plane,
// with x402-priced monetized listings. Mounts after tools (it fills the price seam).
{Name: "marketplace", Mount: cloud.Typed(marketplace.Mount), Shutdown: marketplace.Shutdown},
{Name: "referrals", Mount: cloud.Typed(referrals.Mount)},
{Name: "marketplace", Mount: marketplace.Mount, Shutdown: marketplace.Shutdown},
{Name: "referrals", Mount: referrals.Mount},
// Business AI Guide /v1/guide/* — the interactive launch checklist engine +
// the agent that executes a step through the per-principal MCP plane. After
// automations (whose InvokeTool it drives) and referrals; before the ai
// catch-all. Owns per-org SQLite, so its Shutdown closes the stores.
{Name: "guide", Mount: cloud.Typed(guide.Mount), Shutdown: ctxShutdown(guide.Shutdown)},
{Name: "guide", Mount: guide.Mount, Shutdown: ctxShutdown(guide.Shutdown)},
// Hanzo Company — the incorporation + fundraising state machine
// (/v1/company/*). Mounts after the seams it composes (integrations for the
// google token custody; captable/dataroom facades) and before the /v1/* AI
// catch-all so its routes resolve here.
{Name: "company", Mount: cloud.Typed(company.Mount), Shutdown: company.Shutdown},
{Name: "company", Mount: company.Mount, Shutdown: company.Shutdown},
// Chat orchestrator — POST /v1/chat: ONE LLM tool-calling round over the tool
// plane. It COMPOSES the ai completion path (in-process, so per-org billing
// runs) + the unified tool registry, and splits the model's tool calls into
// server-executed actions and client-applied ops. Mounts BEFORE the zen/ai
// catch-all so /v1/chat resolves here (Fiber first-match); the ai module's
// beego /v1/chat alias behind its /v1/* glob is thereby shadowed, while ai
// keeps /v1/chat/completions + /v1/completions.
{Name: "agent", Mount: agent.Mount},
// The bare /v1/* AI catch-all — the LAST route position. Every owning subsystem above
// wins its own namespace (Fiber first-match); AI is the fallback for the rest of /v1/*.
// zen mounts as a /v1-scoped Claim middleware BEFORE ai: it routes zen* models
@@ -348,9 +402,9 @@ func Wire() []cloud.MountSpec {
// other model and the /v1/models list. Order is load-bearing — Claim must
// run before ai's catch-all. (See hip-00NN.)
{Name: "zen", Mount: mountZen},
{Name: "ai", Mount: cloud.Typed(ai.Mount)},
{Name: "ai", Mount: ai.Mount},
// Runtime wasm/proxy plugins — mounts dead last.
{Name: "plugins", Mount: cloud.Typed(plugin.Mount)},
{Name: "plugins", Mount: plugin.Mount},
}
}
@@ -377,11 +431,7 @@ func ServeSingle(name string) error {
// externals, metrics declares its OWN narrow Deps (Logger, DataDir, Brand) and does
// not import hanzoai/cloud, so cloud.Typed cannot bridge it: the composition root
// builds metrics.Deps from cloud.Deps and calls metrics.Mount explicitly here.
func mountMetrics(app any, deps cloud.Deps) error {
a, ok := app.(*zip.App)
if !ok {
return fmt.Errorf("metrics.Mount: app is %T, want *zip.App", app)
}
func mountMetrics(a *zip.App, deps cloud.Deps) error {
return metrics.Mount(a, metrics.Deps{Logger: deps.Logger, DataDir: deps.DataDir, Brand: deps.Brand})
}
+95 -1
View File
@@ -20,15 +20,20 @@ import (
"fmt"
"net/http"
"path/filepath"
"strings"
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/commerceclient"
"github.com/hanzoai/cloud/clients/commerceinproc"
financeclient "github.com/hanzoai/cloud/clients/finance"
"github.com/hanzoai/commerce"
commercebilling "github.com/hanzoai/commerce/api/billing"
commercestore "github.com/hanzoai/commerce/api/store"
commercedatastore "github.com/hanzoai/commerce/datastore"
commercemid "github.com/hanzoai/commerce/middleware"
"github.com/hanzoai/commerce/middleware/iammiddleware"
commercensctx "github.com/hanzoai/commerce/util/nscontext"
log "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
@@ -115,6 +120,13 @@ func mountCommerce(app *zip.App, deps cloud.Deps) error {
// THE native co-residence contract: commerce registers its routes on
// cloud's own app — no second engine, no net/http adaptation.
App: app,
// ONE LEDGER: commerce's POST /v1/billing/credit mints into cloud's native
// finance ledger (the SAME per-org account the AI spend-gate reads), so a
// granted credit is immediately spendable. commerce.Embed calls
// creditledger.Set(this) before routes register; nil would leave commerce on
// its own datastore (standalone), but in this unified binary finance is
// co-resident, so we inject the finance-backed ledger adapter.
Ledger: ledger{},
})
if err != nil {
lg.Error("commerce embed failed — serving fail-closed 503 (cloud stays up)", "err", err)
@@ -126,7 +138,7 @@ func mountCommerce(app *zip.App, deps cloud.Deps) error {
// mirrors the standalone /v1 bundle: gated request context, host, IAM
// resolution; store.Route's own tokenRequired arg gates the CRUD.
storeV1 := app.Group("/v1")
storeV1.Use(commercemid.AddHost(), commercemid.RequestContext(), commercemid.ErrorHandlerJSON())
storeV1.Use(commercemid.AddHost(), commercemid.RequestContext(), commerceErrorScope())
// Unconditional, exactly like the standalone bundle: IAMTokenRequired
// no-ops gracefully when IAM is not initialized.
storeV1.Use(iammiddleware.IAMTokenRequired())
@@ -156,6 +168,19 @@ func mountCommerce(app *zip.App, deps cloud.Deps) error {
commerceinproc.SetApp(app)
commerceclient.PublishEmbedded(embedded)
// Usage-cap enforcement on the FINANCE path. The unified binary records usage in
// the finance ledger (fin.RecordUsage), NOT commerce's transaction store — which
// it leaves empty — so the cap must read spend from, and fire alerts on, the
// finance ledger. Two seams, both org-wide (the finance Entry carries no scope;
// per-scope caps are a follow-up):
// - SetPeriodSpendReader: AuthorizeSpendCap's scopeSpentCents reads the org's
// finance period spend instead of the empty commerce transaction ledger, so
// a real LLM request increments the cap's `spent` and trips the 402.
// - SetUsageHook: after each finance debit, fire the org's spend-alerts on the
// SAME crossing (the alert half), reading the same finance spend + debouncing.
commercebilling.SetPeriodSpendReader(financePeriodSpend)
financeclient.SetUsageHook(fireCapAlert)
lg.Info("commerce embedded natively (hanzoai/commerce module on the shared zip app)",
"data_dir", dataDir,
"brand", deps.Brand,
@@ -166,6 +191,37 @@ func mountCommerce(app *zip.App, deps cloud.Deps) error {
// mountCommerceFailClosed serves an honest JSON 503 on every commerce prefix when
// the embed cannot boot, so /v1/commerce/* answers "commerce unavailable" instead
// commerceErrorScope confines commerce's JSON error envelope to commerce's OWN
// routes. commercemid.ErrorHandlerJSON is a `/v1` GROUP middleware, and fiber
// matches group middleware by PREFIX, not by the handle a route registered on —
// so on the shared `/v1` it wraps every subsystem mounted AFTER commerce and
// flattens their typed zip.HTTPError (403/400/…) into a blanket 500 (the store
// envelope always renders 500). Guarded by commercePrefixes, the envelope stays on
// commerce and every other subsystem renders its own status via zip's default
// handler — the pre-commerce subsystems (kms, o11y, …) already do; this makes the
// post-commerce ones (projects, agents, wallets, …) match.
func commerceErrorScope() zip.Handler {
envelope := commercemid.ErrorHandlerJSON()
return func(c *zip.Ctx) error {
if hasCommercePrefix(c.Path()) {
return envelope(c)
}
return c.Next()
}
}
// hasCommercePrefix reports whether path is a commerce-owned root (an exact prefix
// or a child of one), the SAME ownership commercePrefixes encodes for the
// fail-closed mount.
func hasCommercePrefix(path string) bool {
for _, p := range commercePrefixes {
if path == p || strings.HasPrefix(path, p+"/") {
return true
}
}
return false
}
// of falling through to another subsystem's catch-all.
func mountCommerceFailClosed(app *zip.App) {
failed := func(c *zip.Ctx) error {
@@ -176,3 +232,41 @@ func mountCommerceFailClosed(app *zip.App) {
app.All(p+"/*", failed)
}
}
// financePeriodSpend is the usage-cap's period-spend source (injected into commerce
// via SetPeriodSpendReader). It returns the org's finance-ledger usage in cents since
// the start of the CURRENT UTC month — the window the cap resets on (mirrors
// commerce periodStartUTC). Org-wide: the finance Entry carries no project/service,
// so scope args are ignored and the org total is returned (what the covering
// org-wide spend-alert row binds on). A finance impl without the sum capability, or a
// split deploy (no co-resident finance), reports 0 — the cap can never over-count.
func financePeriodSpend(ctx context.Context, org string, test bool, _, _ string) (int64, error) {
fin := financeclient.Current()
if fin == nil {
return 0, nil
}
summer, ok := fin.(interface {
SumUsageSince(context.Context, string, bool, int64) (int64, error)
})
if !ok {
return 0, nil
}
n := time.Now().UTC()
since := time.Date(n.Year(), n.Month(), 1, 0, 0, 0, 0, time.UTC).Unix()
return summer.SumUsageSince(ctx, org, test, since)
}
// fireCapAlert fires the org's spend-alerts after a finance usage debit — the alert
// half of the cap on the finance path (wired via finance.SetUsageHook). It resolves
// the org's commerce datastore (where the spend-alert rows live) and calls the
// exported commerce trigger, which reads the org's period spend via financePeriodSpend
// and stamps/debounces. Detached + best-effort; never blocks the money path. Runs in
// its own goroutine (the hook is invoked with `go`), so a background context is right.
func fireCapAlert(org string, test bool, project, service string) {
if strings.TrimSpace(org) == "" {
return
}
ctx := commercensctx.WithNamespace(context.Background(), org)
db := commercedatastore.New(ctx)
commercebilling.FireSpendAlerts(ctx, db, org, test, project, service, nil)
}
+82
View File
@@ -0,0 +1,82 @@
package apps
import (
"io"
"net/http/httptest"
"strings"
"testing"
commercemid "github.com/hanzoai/commerce/middleware"
"github.com/zap-proto/zip"
)
// TestCommerceErrorScope proves commerceErrorScope() confines commerce's always-500
// JSON envelope to commerce-owned prefixes: a post-commerce subsystem route
// (/v1/projects) that returns a typed 403 renders 403 (zip default), while a
// commerce route (/v1/store/...) still gets commerce's envelope. Mirrors the frozen
// order: kms (before commerce) → commerce /v1 group chain → projects + store (after).
// Regression for the release-smoke failure where 14 post-commerce endpoints 500'd.
func TestCommerceErrorScope(t *testing.T) {
app := zip.New(zip.Config{})
// kms (before commerce) — a clean 403 baseline (never wrapped by commerce).
app.Get("/v1/kms/health", func(c *zip.Ctx) error { return zip.ErrForbidden("kms says no") })
// commerce (position 39): the REAL group chain, now with the scoped envelope.
sv1 := app.Group("/v1")
sv1.Use(commercemid.AddHost(), commercemid.RequestContext(), commerceErrorScope())
// projects (after commerce) — typed 403; must NOT be clobbered to 500.
app.Get("/v1/projects", func(c *zip.Ctx) error { return zip.ErrForbidden("X-Org-Id required") })
// a commerce store route (after commerce) — typed 403; commerce envelope applies.
app.Get("/v1/store/current", func(c *zip.Ctx) error { return zip.ErrForbidden("store needs org") })
probe := func(path string) (int, string) {
req := httptest.NewRequest("GET", path, nil)
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("%s: %v", path, err)
}
defer resp.Body.Close()
b, _ := io.ReadAll(resp.Body)
return resp.StatusCode, string(b)
}
// The invariant the scope guards: a typed zip.HTTPError (403) a subsystem returns
// is NEVER flattened to a 500 — not before commerce, not after it. (commerce
// >=1.48.10 honors the status itself; the scope keeps commerce's error handler off
// other subsystems' routes regardless, so a future commerce regression can't
// re-clobber them.)
for _, tc := range []struct {
path string
wantCode int
}{
{"/v1/kms/health", 403}, // before commerce
{"/v1/projects", 403}, // after commerce — must NOT be clobbered to 500
{"/v1/store/current", 403}, // commerce's own route — its handler still honors 403
} {
code, body := probe(tc.path)
t.Logf("%-20s -> %d %s", tc.path, code, body)
if code != tc.wantCode {
t.Errorf("%s: got %d, want %d (%s)", tc.path, code, tc.wantCode, body)
}
if strings.Contains(body, "\"status\":5") || code >= 500 {
t.Errorf("%s: a typed 403 was flattened to a 5xx (%s)", tc.path, body)
}
}
for _, p := range []struct {
path string
own bool
}{
{"/v1/store/current", true},
{"/v1/commerce/checkout", true},
{"/v1/projects", false},
{"/v1/agent/presets", false},
{"/v1/agents", false},
} {
if got := hasCommercePrefix(p.path); got != p.own {
t.Errorf("hasCommercePrefix(%q) = %v, want %v", p.path, got, p.own)
}
}
}
+81
View File
@@ -0,0 +1,81 @@
// Copyright © 2026 Hanzo AI. MIT License.
package apps
import (
"context"
"fmt"
"github.com/hanzoai/cloud/clients/finance"
"github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/cloud/types"
"github.com/hanzoai/commerce/billing/creditledger"
)
// ledger implements commerce's creditledger.CreditLedger over cloud's native
// finance ledger — the SAME per-org account (finance.Current()) the AI spend-gate
// reads and the edge meter debits. Injected at mountCommerce (EmbedConfig.Ledger),
// it makes commerce's POST /v1/billing/credit mint into the ONE ledger: a granted
// credit is immediately visible to the gate (one ledger, no split). This is the
// cloud half of the one-ledger seam — commerce defines the interface, cloud
// implements it once here, the compiler enforces the match.
//
// Fails closed when no finance ledger is co-resident; in the unified cloud binary
// finance is always published, so Get() != nil ⇒ credit routes here.
type ledger struct{}
// compile-time proof the adapter satisfies commerce's exported seam.
var _ creditledger.CreditLedger = ledger{}
// Credit posts a balanced deposit (funding:platform → wallet) to the org's POOL
// account (Subject == Org, the wallet the gate reads) and returns the ledger entry
// id + the org's new available balance in cents. Idempotent on IdempotencyKey:
// finance dedups on Ref, so the same key credits AT MOST once.
func (ledger) Credit(ctx context.Context, in creditledger.CreditInput) (string, int64, error) {
fin := finance.Current()
if fin == nil {
return "", 0, fmt.Errorf("commerce credit: no finance ledger co-resident")
}
cur := in.Currency
if cur == "" {
cur = "usd"
}
tag := in.Tag
if tag == "" {
tag = "grant:admin" // non-cash grant bucket (finance is a single wallet; Tags is a memo)
}
id, err := fin.Deposit(ctx, types.DepositInput{
Org: in.Org,
Subject: in.Org, // org-pool wallet == the account the AI gate reads
Amount: money.FromCents(in.AmountCents),
Currency: cur,
Notes: in.Reason,
Tags: tag,
Ref: in.IdempotencyKey,
})
if err != nil {
return "", 0, err
}
bal, berr := fin.Balance(ctx, in.Org, in.Org, cur, false)
if berr != nil {
return id, 0, berr
}
return id, bal.Cents(), nil
}
// Balance returns the org pool's available balance in cents for currency — the
// same read the AI gate performs, so GET /v1/billing/balance and the gate agree.
func (ledger) Balance(ctx context.Context, org, currency string) (int64, error) {
fin := finance.Current()
if fin == nil {
return 0, fmt.Errorf("commerce balance: no finance ledger co-resident")
}
if currency == "" {
currency = "usd"
}
bal, err := fin.Balance(ctx, org, org, currency, false)
if err != nil {
return 0, err
}
return bal.Cents(), nil
}
+95 -92
View File
@@ -16,98 +16,100 @@ import "testing"
// origin/main @c504d2b: 68 specs from the live init()-registry, PLUS the two the
// wave-2 external bumps (ai v1.805.2, o11y v1.5.12) stopped self-registering — ai
// (@150 catch-all) and the hanzoai/o11y module wildcard (@70), which main currently
// DROPS and this PR restores at their order-int slots. "o11y" therefore appears
// twice: the in-repo read plane (order 69) and the module wildcard (70), co-owners.
// DROPS and this PR restores. The module wildcard (order 70) is now folded in as the
// terminal sub-mount of the in-repo o11y read plane (order 69), so "o11y" is ONE spec.
var frozen = []struct {
name string
ownsHealth bool
hasShutdown bool
}{
{"pubsub", false, true}, // was order 5
{"kafka", false, true}, // was order 6
{"agentskills", false, false}, // was order 8
{"featureflags", false, false}, // was order 9
{"kms", true, false}, // was order 10
{"metrics", false, false}, // was order 40
{"ingress", false, true}, // was order 42
{"account", false, false}, // was order 48
{"iam", false, false}, // was order 50
{"base", true, true}, // was order 60; per-org embed added Shutdown (#298)
{"o11y", true, true}, // was order 69
{"o11y", false, false}, // was order 70
{"authz", false, false}, // was order 70
{"commerce", false, false}, // was order 100
{"licensing", false, false}, // was order 110
{"plans", true, false}, // was order 111
{"pricing", true, false}, // was order 112
{"storage", true, false}, // was order 118
{"provisioning", false, false}, // was order 120
{"billing", false, false}, // was order 121
{"account-bridge", false, false}, // was order 122
{"do", false, false}, // was order 123
{"platform", true, false}, // was order 124
{"projects", false, false}, // was order 125
{"prompts", false, false}, // was order 126
{"agents", false, true}, // was order 127
{"link", false, true}, // new: unified AI login manager (/v1/links), after agents
{"wallets", false, true}, // was order 127
{"x402", false, true}, // new: x402 pay-per-use settlement (after wallets)
{"paas", true, false}, // was order 128
{"deploy", true, false}, // after paas (release seam), before functions
{"functions", false, false}, // was order 128
{"tracker", false, false}, // was order 129
{"templates", false, false}, // was order 129
{"framework", false, true}, // was order 129
{"knowledge", false, false}, // was order 130
{"content", false, true}, // new: marketing content loop (after knowledge)
{"catalogsync", false, true}, // new: reverse loop (product.created → render) after content
{"ml", true, false}, // was order 130
{"usage", false, false}, // was order 131
{"crm", false, false}, // was order 131
{"marketing", false, true}, // new: marketing domain fold (after crm)
{"ads", false, true}, // new: ads domain fold (after crm)
{"social", false, true}, // new: /v1/social fold (after crm)
{"analytics", true, false}, // was order 132
{"git", false, false}, // was order 132
{"visor", false, false}, // was order 133
{"captable", false, true}, // was order 133
{"code", false, true}, // was order 134
{"zero-trust", false, false}, // was order 134
{"dataroom", true, true}, // was order 134
{"graph", false, false}, // was order 135
{"security", true, true}, // was order 136
{"integrations", false, true}, // was order 137
{"sbom", true, false}, // was order 137
{"team", false, true}, // was order 138
{"settings", false, true}, // was order 138
{"notify", true, false}, // was order 139
{"gateway", false, false}, // was order 139
{"entitlements", false, true}, // was order 139
{"exec", false, false}, // was order 140
{"websearch", false, false}, // was order 141
{"world", false, true}, // was order 142
{"bot", false, false}, // was order 143
{"authors", false, true}, // was order 143
{"bots", false, false}, // was order 143
{"audit", false, false}, // was order 144
{"affiliates", false, false}, // was order 144
{"sign", true, true}, // was order 145
{"product", false, false}, // was order 145
{"evals", false, false}, // was order 145
{"treasury", false, true}, // was order 146
{"admin", false, false}, // was order 146
{"tasks", false, false}, // was order 147
{"cron", false, false}, // durable platform cron on the shared engine (post-freeze add)
{"automations", false, true}, // was order 148
{"connectorruntime", false, false}, // new: native single-connector exec via goja (after automations, HIP-0126)
{"tools", false, true}, // new: unified tool plane (after automations)
{"marketplace", false, true}, // new: marketplace over the tool plane (after tools)
{"referrals", false, false}, // was order 149
{"guide", false, true}, // new: Business AI Guide (after referrals, before ai)
{"company", false, true}, // new: Hanzo Company formation state machine (after guide)
{"zen", false, false}, // zen* claim middleware before ai's catch-all (hip-00NN)
{"ai", false, false}, // was order 150
{"plugins", false, false}, // was order 900
{"pubsub", false, true}, // was order 5
{"kafka", false, true}, // was order 6
{"agentskills", false, false}, // was order 8
{"flags", true, true}, // was order 9; native engine: /v1/flags health + store shutdown
{"kms", true, false}, // was order 10
{"metrics", false, false}, // was order 40
{"ingress", false, true}, // was order 42
{"account", false, false}, // was order 48
{"iam", false, false}, // was order 50
{"base", true, true}, // was order 60; per-org embed added Shutdown (#298)
{"o11y", false, true}, // ONE observability subsystem (was co-owned orders 69+70): read plane + the hanzoai/o11y module wildcard folded in as MountO11y's terminal sub-mount. OwnsHealth=false keeps /v1/o11y/health the generic always-ok route the module co-entry used to trigger.
{"authz", false, false}, // was order 70
{"commerce", false, false}, // was order 100
{"licensing", false, false}, // was order 110
{"plan", true, false}, // was order 111; enable id normalized plans->plan (routes stay /v1/plans/*)
{"pricing", true, false}, // was order 112
{"storage", true, false}, // was order 118
{"provisioning", false, false}, // was order 120
{"billing", false, false}, // was order 121
{"account-bridge", false, false}, // was order 122
{"do", false, false}, // was order 123
{"platform", true, false}, // was order 124
{"projects", false, false}, // was order 125
{"dns", false, false}, // new: /v1/dns zone plane (after projects)
{"prompts", false, false}, // was order 126
{"agents", false, true}, // was order 127
{"link", false, true}, // new: unified AI login manager (/v1/links), after agents
{"wallets", false, true}, // was order 127
{"x402", false, true}, // new: x402 pay-per-use settlement (after wallets)
{"paas", true, false}, // was order 128
{"deploy", true, false}, // after paas (release seam), before functions
{"functions", false, false}, // was order 128
{"tracker", false, false}, // was order 129
{"templates", false, false}, // was order 129
{"framework", false, true}, // was order 129
{"knowledge", false, false}, // was order 130
{"content", false, true}, // new: marketing content loop (after knowledge)
{"catalogsync", false, true}, // new: reverse loop (product.created → render) after content
{"ml", true, false}, // was order 130
{"usage", false, false}, // was order 131
{"crm", false, false}, // was order 131
{"marketing", false, true}, // new: marketing domain fold (after crm)
{"ads", false, true}, // new: ads domain fold (after crm)
{"social", false, true}, // new: /v1/social fold (after crm)
{"analytics", true, false}, // was order 132
{"git", false, false}, // was order 132
{"sync", false, true}, // /v1/sync engine (owns per-org DB handles → Shutdown)
{"visor", false, false}, // was order 133
{"captable", false, true}, // was order 133
{"code", false, true}, // was order 134
{"zero-trust", false, false}, // was order 134
{"dataroom", true, true}, // was order 134
{"graph", false, false}, // was order 135
{"security", true, true}, // was order 136
{"integrations", false, true}, // was order 137
{"cloudflare", false, false}, // new: /v1/cloudflare edge plane (after integrations)
{"sbom", true, false}, // was order 137
{"team", false, true}, // was order 138
{"settings", false, true}, // was order 138
{"notify", true, false}, // was order 139
{"gateway", false, false}, // was order 139
{"entitlements", false, true}, // was order 139
{"exec", false, false}, // was order 140
{"websearch", false, false}, // was order 141
{"world", false, true}, // was order 142
{"runtime", false, false}, // was order 143; was "bot" until the transport was named for what it is
{"authors", false, true}, // was order 143
{"bots", false, false}, // was order 143
{"audit", false, false}, // was order 144
{"affiliates", false, false}, // was order 144
{"sign", true, true}, // was order 145
{"product", false, false}, // was order 145
{"evals", false, false}, // was order 145
{"treasury", false, true}, // was order 146
{"admin", false, false}, // was order 146
{"admission", false, true}, // launch-control gate: composes flags (registry+seed+mode route+Enforce); Shutdown closes the registry store
{"tasks", false, false}, // was order 147; platform cron folded in as a sub-mount of tasks.Mount (was a separate entry)
{"automations", false, true}, // was order 148; connectorruntime (POST /v1/automations/connectors/:id/run) folded in as a sub-mount of automations.Mount
{"tools", false, true}, // new: unified tool plane (after automations)
{"marketplace", false, true}, // new: marketplace over the tool plane (after tools)
{"referrals", false, false}, // was order 149
{"guide", false, true}, // new: Business AI Guide (after referrals, before ai)
{"company", false, true}, // new: Hanzo Company formation state machine (after guide)
{"agent", false, false}, // new: /v1/agent tool-calling round (before zen/ai catch-all)
{"zen", false, false}, // zen* claim middleware before ai's catch-all (hip-00NN)
{"ai", false, false}, // was order 150
{"plugins", false, false}, // was order 900
}
// TestWireOrderMatchesFrozen proves the composition root's mount order is
@@ -134,17 +136,18 @@ func TestWireOrderMatchesFrozen(t *testing.T) {
}
}
// TestWireNoDuplicateEnablement guards the ONE intentional duplicate: only "o11y"
// may appear twice (the two co-owners of the observability concept). Any other
// duplicate name is a copy-paste bug — two specs would both mount under one enable id.
// TestWireNoDuplicateEnablement guards that every subsystem name is unique: a name
// maps 1:1 to an enable id, so a duplicate would mount two specs under one id. (The
// former o11y co-ownership was collapsed — the module wildcard is now a sub-mount of
// the in-repo o11y read plane — so there is no longer any exempt duplicate.)
func TestWireNoDuplicateEnablement(t *testing.T) {
seen := map[string]int{}
for _, s := range Wire() {
seen[s.Name]++
}
for name, n := range seen {
if n > 1 && name != "o11y" {
t.Errorf("subsystem %q wired %d times (only o11y may be a co-owned duplicate)", name, n)
if n > 1 {
t.Errorf("subsystem %q wired %d times (each enable id must be unique)", name, n)
}
}
}
+111 -25
View File
@@ -5,9 +5,12 @@ package apps
import (
"context"
"fmt"
"math/big"
"os"
"strings"
aicontrollers "github.com/hanzoai/ai/controllers"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/metering"
cloudmoney "github.com/hanzoai/cloud/clients/money"
@@ -53,11 +56,7 @@ import (
//
// It is wired BEFORE ai in Wire() so Claim's c.Next() falls through to ai's
// catch-all. zen's catalog reads its upstream keys from KMS via the Key resolver.
func mountZen(app any, deps cloud.Deps) error {
a, ok := app.(*zip.App)
if !ok {
return fmt.Errorf("zen.Mount: app is %T, want *zip.App", app)
}
func mountZen(a *zip.App, deps cloud.Deps) error {
z, err := zen.New(zen.Config{
Logger: deps.Logger,
Key: zenKeyResolver(deps.KMS),
@@ -110,11 +109,17 @@ func commerceGate(m *metering.Client) zen.Gate {
if t.BillingOrg == "" {
return fmt.Errorf("a billable tenant is required (no anonymous usage)")
}
// zen's estimate is exact 18-dp atto-USD (hanzoai/money). Fold to whole
// cents for the balance check via cloud's typed money.Amount.Cents() — a
// sub-cent estimate gates as 0 (any-positive-balance), matching the edge
// gate's AmountCents contract. The post-serve Meter debits the exact 18-dp.
cents := cloudmoney.FromInt(est.Minor()).Cents()
// zen's estimate is an exact 18-dp USD value. Fold it to whole cents for
// the balance check via cloud's typed money.Amount.Cents() — a sub-cent
// estimate gates as 0 (any-positive-balance), matching the edge gate's
// AmountCents contract. The post-serve Meter debits the exact 18-dp.
//
// The fold is Cents() on the CREDIT amount, never Minor() on the zen one:
// Minor() renders money.USD's 2 decimals, so it returned cents that FromInt
// then read as atto — every estimate came back 0, and AuthorizeVerdict skips
// its `available >= AmountCents` check when AmountCents is 0, admitting a
// request of ANY size against any positive balance.
cents := credit(est).Cents()
v, err := m.AuthorizeVerdict(ctx, metering.AuthInput{
User: t.BillingOrg,
Org: t.BillingOrg,
@@ -158,25 +163,106 @@ func (g commerceMeterImpl) Record(ctx context.Context, u zen.Usage) {
if u.Tenant.BillingOrg == "" {
return // never debit an unattributable request
}
usage := metering.Usage{
User: u.Tenant.BillingOrg,
Org: u.Tenant.BillingOrg,
Actor: u.Tenant.User,
Model: u.Model,
Provider: zenProvider,
Service: zenService,
Project: u.Tenant.Project,
PromptTokens: u.PromptTokens,
// Beside the commerce debit, land the SAME warehouse row + gen_ai span every
// native ai path writes (TraceServedUsage = recordTrace WITHOUT recordUsage —
// the debit below is the one billing source, never doubled). zen knows its
// EXACT per-tier retail (Charge) and upstream COGS (Cost), so the row carries
// true margin (credit → nano). Without this, zen* traffic is
// warehouse/o11y-blind exactly where prod runs (the unified binary).
aicontrollers.TraceServedUsage(context.Background(), aicontrollers.ServedUsage{
Owner: u.Tenant.BillingOrg,
User: u.Tenant.User,
Model: u.Model,
Provider: zenProvider,
RequestID: u.RequestID,
Status: "success",
PromptTokens: u.PromptTokens,
CompletionTokens: u.CompletionTokens,
TotalTokens: u.PromptTokens + u.CompletionTokens,
Amount: cloudmoney.FromInt(u.Cost.Minor()), // exact 18-dp USD, no floor
RequestID: u.RequestID,
Currency: "usd",
Status: "success",
}
BilledNano: nano(credit(u.Charge)),
CostNano: nano(credit(u.Cost)),
})
// Detached: the request context is recycled once the handler returns, so a
// background context carries the debit to commerce without racing the reply.
usage := meterUsage(u)
go func() { _, _ = g.m.Record(context.Background(), usage) }()
// Enso learning ledger: the embedded zen mount serves the zen catalog in-process
// and never reaches ai's pipeToFamily, so ai's family-event writer never runs for
// zen traffic. Write the SAME RoutingEvent here (source="family") through the ONE
// shared writer, keyed on the client-visible response id (zen.Usage.ResponseID), so
// zen* calls land in the same ledger — stats, world, spark retrain, and /v1/feedback
// all read these rows. No prompt text; no shadow (zen.Usage carries no request
// text — that stays the auto/enso-proxy path's job). Fire-and-forget.
owner := u.Tenant.Org
if owner == "" {
owner = u.Tenant.BillingOrg
}
go aiobject.RecordFamilyRouting(aiobject.FamilyRoutingInput{
Owner: owner,
User: u.Tenant.User,
RequestedModel: u.Model,
RoutedModel: u.Upstream,
ResponseId: u.ResponseID,
PromptTokens: u.PromptTokens,
CompletionTokens: u.CompletionTokens,
CostCents: credit(u.Charge).Cents(),
RouterEndpoint: os.Getenv("ROUTER_ENDPOINT"),
})
}
// meterUsage projects a served zen.Usage onto the commerce debit. It is the ONE
// place the debit's amount is chosen, and it is pure — no ledger, no warehouse —
// so the money property is a unit test rather than an integration.
//
// The amount is the RETAIL Charge: what the caller pays. Cost is the upstream
// COGS we pay to serve the call; it is never the debit. It rides only the
// warehouse row (CostNano), where margin = Charge Cost stays exact. Debiting
// Cost would collect our own COGS and book zero margin on every zen call — and
// because the affiliate and OSS payout bases read this debit, it would fund
// their shares out of principal. This mirrors ai, whose debit is likewise the
// customer price (usageBilledCents), never its CostIn/CostOut COGS.
func meterUsage(u zen.Usage) metering.Usage {
return metering.Usage{
User: u.Tenant.BillingOrg,
Org: u.Tenant.BillingOrg,
Actor: u.Tenant.User,
Model: u.Model,
Provider: zenProvider,
Service: zenService,
Project: u.Tenant.Project,
PromptTokens: u.PromptTokens,
CompletionTokens: u.CompletionTokens,
TotalTokens: u.PromptTokens + u.CompletionTokens,
Amount: credit(u.Charge), // exact 18-dp USD, no floor
RequestID: u.RequestID,
Currency: "usd",
Status: "success",
}
}
// credit re-denominates a zen price into cloud's credit unit. It is the ONE
// conversion at this seam — every site below goes through it, so the unit is
// decided once rather than re-derived per call site.
//
// zen prices every SKU as an exact 18-dp value tagged money.USD (meter.go:
// money.New(<18-dp decimal>, money.USD)), and cloud's credit unit is the SAME USD
// value at 18-dp storage scale. So the conversion carries the exact decimal across
// and changes only the minor-unit convention: no rescale, no rounding, no factor.
// It is right by construction because the decimal is the value — the currency's
// Decimals is a rendering convention, not part of it.
//
// It must NEVER go through Amount.Minor(). money.USD declares 2 decimals, so
// Minor() rescales zen's 18-dp value to CENTS; feeding cents to the 18-dp
// FromInt understates the debit by 10^16 (a $17.376 charge debits $0.0000000000000017),
// and folds every sub-cent charge to a zero the ledger drops entirely.
func credit(a hmoney.Amount) cloudmoney.Amount { return cloudmoney.FromDecimal(a.Decimal()) }
// nano folds an exact credit Amount to nano-USD (1e-9) for the warehouse margin
// columns. It takes the typed Amount rather than a bare *big.Int so the unit is
// carried by the type: a cents integer is not a cloudmoney.Amount and can no
// longer be passed here. A single request's cost always fits int64 at nano.
func nano(a cloudmoney.Amount) int64 {
return new(big.Int).Div(a.Atto(), big.NewInt(1_000_000_000)).Int64()
}
// zenService is the commerce service axis zen* spend attributes to. zen serves
+305
View File
@@ -0,0 +1,305 @@
// Copyright 2026 Hanzo AI Inc. All Rights Reserved.
package apps
import (
"context"
"io"
"net/http"
"net/http/httptest"
"strconv"
"sync/atomic"
"testing"
"github.com/hanzoai/cloud/clients/metering"
cloudmoney "github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/decimal"
hmoney "github.com/hanzoai/money"
"github.com/hanzoai/zen"
)
// usd builds an exact USD amount from a decimal string, the way zen prices a
// call (18-dp native, never through float).
//
// The decimal here must be hanzoai/decimal, the one hmoney.New takes and the one
// zen prices with — not shopspring's identically-named type. Money has exactly one
// decimal; a second one that merely LOOKS like it is how a price silently becomes
// a different number.
func usd(t *testing.T, s string) hmoney.Amount {
t.Helper()
d, err := decimal.Parse(s)
if err != nil {
t.Fatalf("parse %q: %v", s, err)
}
return hmoney.New(d, hmoney.USD)
}
// zen5Usage is one served call at zen5's live rates: 1M in + 1M out, priced at
// the family's 3× margin (retail = cost × margin).
//
// in : cost 1.392 → retail 4.176
// out: cost 4.40 → retail 13.20
func zen5Usage(t *testing.T) zen.Usage {
t.Helper()
return zen.Usage{
Tenant: zen.Tenant{BillingOrg: "acme", User: "acme/alice", Project: "p1"},
Model: "zen5",
PromptTokens: 1_000_000,
CompletionTokens: 1_000_000,
Charge: usd(t, "17.376"), // 4.176 + 13.20 — what the caller pays
Cost: usd(t, "5.792"), // 1.392 + 4.40 — what we pay upstream
RequestID: "req-1",
}
}
// dollars is the EXPECTED money, built from a plain dollar literal through cloud's
// OWN ParseUSD — deliberately a DIFFERENT constructor than the code under test uses.
//
// That is what pins the UNIT. These tests once asserted meterUsage(u).Amount.Int()
// against u.Charge.Minor(): both sides re-derived the number through the same
// conversion, so the assertion only proved an integer round-tripped and was blind
// to what the integer MEANT. It passed while every zen debit was 10^16 too small.
// Comparing money to a known dollar amount cannot be blind that way: if the debit
// is off by any factor, it is not $17.376 and the test fails.
func dollars(t *testing.T, s string) cloudmoney.Amount {
t.Helper()
a, err := cloudmoney.ParseUSD(s)
if err != nil {
t.Fatalf("parse %q: %v", s, err)
}
return a
}
// The debit is the RETAIL Charge — what the caller pays — never the upstream
// COGS. This is the money property: it fails if the Amount is built from
// u.Cost. The tier is margin-bearing (retail != cost), so the two values are
// distinguishable and the assertion cannot pass by coincidence.
func TestMeterUsageDebitsRetailNotCost(t *testing.T) {
u := zen5Usage(t)
if u.Charge.Cmp(u.Cost) == 0 {
t.Fatal("fixture is not margin-bearing: retail == cost, so the test could not tell them apart")
}
got := meterUsage(u).Amount
// The known dollar value the fixture charges — a $17.376 call debits $17.376.
if want := dollars(t, "17.376"); got.Cmp(want) != 0 {
t.Errorf("debit = $%s, want the retail Charge $%s", got, want)
}
if cogs := dollars(t, "5.792"); got.Cmp(cogs) == 0 {
t.Errorf("debit = $%s, which is the upstream COGS — the caller must be billed retail, not our cost", got)
}
}
// At the family's 3× margin the debit is exactly 3× the COGS: we collect the
// full retail price, not the wholesale one. Debiting Cost would collect 1/3 —
// our own COGS — and book zero margin.
func TestMeterUsageCollectsTheFullMargin(t *testing.T) {
u := zen5Usage(t)
debit := meterUsage(u).Amount
// 3x the COGS in the SAME unit as the debit — $5.792 + $5.792 + $5.792.
// Summing the credit Amount keeps the comparison in exact dollars; the old
// version multiplied Cost.Minor() (cents, 579 after rounding away 5.792's
// third decimal) and compared it to a debit that was not cents at all.
cogs := dollars(t, "5.792")
thriceCOGS := cogs.Add(cogs).Add(cogs)
if debit.Cmp(thriceCOGS) != 0 {
t.Errorf("debit = $%s, want 3x COGS = $%s (retail = cost x margin, margin 3.0)", debit, thriceCOGS)
}
if want := dollars(t, "17.376"); debit.Cmp(want) != 0 {
t.Errorf("debit = $%s, want $%s", debit, want)
}
}
// The debit carries zen's exact 18-dp value with no floor: a sub-cent call must
// not round to zero on the way to the ledger.
func TestMeterUsageKeepsExactSubCentCharge(t *testing.T) {
u := zen5Usage(t)
u.Charge = usd(t, "0.004176") // 1k input tokens at 4.176/MTok — well under a cent
u.Cost = usd(t, "0.001392")
got := meterUsage(u).Amount
if want := dollars(t, "0.004176"); got.Cmp(want) != 0 {
t.Errorf("debit = $%s, want the exact sub-cent charge $%s", got, want)
}
// metering.Record drops a zero Amount before it ever reaches the ledger
// (`if !c.Enabled() || amt.IsZero() ... return nil`), so a floored sub-cent
// charge is not a small debit — it is NO DEBIT ROW, and the call is free.
if got.IsZero() {
t.Error("sub-cent charge floored to zero — Record drops a zero Amount, so the call is served free with no debit row")
}
}
// The unit trap this seam shipped with, pinned so it cannot come back. zen prices
// an exact 18-dp value but tags it money.USD, whose Currency declares 2 decimals —
// so Charge.Minor() renders CENTS, while cloudmoney.FromInt reads its argument as
// 18-dp. Composing them silently divides every debit by 10^16.
//
// The tests above already fail if credit() regresses to that composition; this one
// names WHY, and proves the two conversions are still distinguishable — an
// assertion that cannot tell right from wrong is worse than no assertion.
func TestCreditIsNotTheMinorUnit(t *testing.T) {
charge := usd(t, "17.376")
if got, want := credit(charge), dollars(t, "17.376"); got.Cmp(want) != 0 {
t.Fatalf("credit($17.376) = $%s, want $%s", got, want)
}
// The conversion that shipped, spelled out.
old := cloudmoney.FromAtto(charge.Minor())
if old.Cmp(credit(charge)) == 0 {
t.Fatal("FromInt(Minor()) agrees with credit() — the fixture can no longer tell the units apart, so these tests prove nothing")
}
if old.Cents() != 0 {
t.Errorf("FromInt(Minor()).Cents() = %d, want 0 — that this was ALWAYS 0 is what made the spend gate admit every request", old.Cents())
}
}
// The identity fields ride with the debit unchanged: the debit lands on the org
// that PAYS (BillingOrg), scoped to its project, with the actor for the audit
// trail.
func TestMeterUsageAttribution(t *testing.T) {
u := zen5Usage(t)
m := meterUsage(u)
for _, c := range []struct{ name, got, want string }{
{"User", m.User, "acme"},
{"Org", m.Org, "acme"},
{"Actor", m.Actor, "acme/alice"},
{"Project", m.Project, "p1"},
{"Model", m.Model, "zen5"},
{"Provider", m.Provider, zenProvider},
{"Service", m.Service, zenService},
{"Currency", m.Currency, "usd"},
} {
if c.got != c.want {
t.Errorf("%s = %q, want %q", c.name, c.got, c.want)
}
}
if m.TotalTokens != 2_000_000 {
t.Errorf("TotalTokens = %d, want 2000000", m.TotalTokens)
}
}
// The spend gate must REFUSE a request whose estimate exceeds the balance.
//
// This is the sharpest edge of the unit bug and the reason it is a security
// finding, not only a revenue one. AuthorizeVerdict gates size like this:
//
// funded := available > 0
// if in.AmountCents > 0 { funded = available >= in.AmountCents }
//
// The estimate reached it as FromInt(est.Minor()).Cents(), which is ALWAYS 0 —
// so the size branch was DEAD and every request rode `available > 0`. Any org
// with a single cent of balance could draw an unbounded call. The debits were
// dust too, so the balance never fell and the cap could never trip.
func TestCommerceGateRefusesAnOverCapRequest(t *testing.T) {
const availableCents = 500 // the org holds $5.00
var authorized atomic.Bool
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v1/billing/balance":
_, _ = io.WriteString(w, `{"available":`+strconv.Itoa(availableCents)+`}`)
case "/v1/billing/spend-alerts/authorize":
authorized.Store(true)
_, _ = io.WriteString(w, `{"allow":true}`)
default:
_, _ = io.WriteString(w, `[]`)
}
}))
defer srv.Close()
m, err := metering.New(metering.Config{BaseURL: srv.URL, Token: "t", Org: "acme"})
if err != nil {
t.Fatal(err)
}
gate := commerceGate(m)
if gate == nil {
t.Fatal("gate is nil — the client must be enabled for this proof to mean anything")
}
tenant := zen.Tenant{BillingOrg: "acme", User: "acme/alice", Project: "p1"}
// $17.376 against a $5.00 balance: over cap, must be refused.
if err := gate(context.Background(), tenant, "zen5", usd(t, "17.376")); err == nil {
t.Error("gate ADMITTED a $17.376 request against a $5.00 balance — the estimate is reaching AuthorizeVerdict as 0 cents, so the size check never runs")
}
// The same balance must still admit a request it can actually cover, or the
// test would pass by refusing everything.
if err := gate(context.Background(), tenant, "zen5", usd(t, "1.00")); err != nil {
t.Errorf("gate refused an affordable $1.00 request against a $5.00 balance: %v", err)
}
if !authorized.Load() {
t.Error("the affordable request never reached the spend-cap authorize step")
}
}
// The estimate must reach the balance check as the RIGHT number of cents. The
// refusal test above proves the gate says no; this proves it says no for the
// right reason — that $17.376 is folded to 1738 cents, not to 0.
func TestCommerceGateFoldsTheEstimateToRealCents(t *testing.T) {
for _, c := range []struct {
charge string
want int64
}{
{"17.376", 1738}, // rounds half-away-from-zero at the cent
{"1000.00", 100000},
{"1.00", 100},
{"0.004176", 0}, // sub-cent gates as "any positive balance"
} {
if got := credit(usd(t, c.charge)).Cents(); got != c.want {
t.Errorf("$%s folds to %d cents, want %d", c.charge, got, c.want)
}
}
}
// A zero-value zen price must convert and fold without panicking. zen leaves
// Charge/Cost as the zero Amount for a free SKU, and that value carries the
// EMPTY currency code, not "USD" — so this also pins that credit() reads the
// decimal rather than dispatching on the currency. The old attoToNano guarded a
// nil *big.Int here; nano() needs no guard because decimal.Coef() returns a real
// zero big.Int, never nil, but the property is worth holding.
func TestCreditAndNanoHandleTheZeroPrice(t *testing.T) {
var free hmoney.Amount // zero value: no currency, no coefficient
got := credit(free)
if !got.IsZero() {
t.Errorf("credit(zero) = $%s, want $0", got)
}
if n := nano(got); n != 0 {
t.Errorf("nano(credit(zero)) = %d, want 0", n)
}
if c := got.Cents(); c != 0 {
t.Errorf("credit(zero).Cents() = %d, want 0", c)
}
// And a free call books no debit row, which is correct — nothing is owed.
u := zen5Usage(t)
u.Charge, u.Cost = free, free
if amt := meterUsage(u).Amount; !amt.IsZero() {
t.Errorf("free call debits $%s, want $0", amt)
}
}
// nano carries real money to the warehouse margin columns. It is the last place
// the unit could silently collapse: attoToNano(cents) divided a cents integer by
// 1e9 and produced 0 for every charge under $10,000,000.
func TestNanoFoldsCreditToRealNano(t *testing.T) {
for _, c := range []struct {
charge string
want int64
}{
{"17.376", 17_376_000_000},
{"5.792", 5_792_000_000},
{"0.004176", 4_176_000},
{"1000.00", 1_000_000_000_000},
} {
if got := nano(credit(usd(t, c.charge))); got != c.want {
t.Errorf("nano($%s) = %d, want %d", c.charge, got, c.want)
}
}
}
+15 -2
View File
@@ -15,6 +15,7 @@ import (
"fmt"
"os"
"path/filepath"
"strings"
"time"
"github.com/hanzoai/cloud/audit"
@@ -57,6 +58,18 @@ func buildAuditRecorder(cfg *Config, logger luxlog.Logger) (*audit.Recorder, err
return nil, fmt.Errorf("open audit store: %w", err)
}
// PER-SHARD audit under horizontal scale. The trail lives at {DataDir}/audit.db on
// THIS pod's own RWO PVC, so under shard routing each pod's chain covers ONLY the
// tenants routed to it (its shard) — and org-scoped audit queries route to the
// owning shard where those records live. Soundness: the chain is a per-FILE hash
// chain whose head is recovered at open; because no two pods share the file, there
// is no cross-pod head to fork (the very failure that pinned cloud to replicas:1 was
// two pods on ONE audit file). Integrity is preserved WITHIN each partition; a
// deployment-wide view is the union of the N per-shard chains. The shard id is
// stamped on the AU-9 checkpoint stream below so the external tail-truncation monitor
// tracks N heads (one per shard) rather than expecting a single global head.
shard := strings.TrimSpace(cfg.ShardSelf) // "" when single-pod — a harmless empty tag
// AU-9 tail-truncation anchor: emit a periodic head-digest checkpoint to the
// append-only observability log (and, when a mirror supports it, an
// independent digest store). An external o11y monitor compares consecutive
@@ -67,7 +80,7 @@ func buildAuditRecorder(cfg *Config, logger luxlog.Logger) (*audit.Recorder, err
if logger != nil {
rec.StartCheckpoints(interval, func(cp audit.Checkpoint) {
logger.Info("audit_head_checkpoint",
"count", cp.Count, "head", cp.Head, "ts", cp.Time.Format(time.RFC3339Nano))
"shard", shard, "count", cp.Count, "head", cp.Head, "ts", cp.Time.Format(time.RFC3339Nano))
})
} else {
rec.StartCheckpoints(interval, nil)
@@ -76,7 +89,7 @@ func buildAuditRecorder(cfg *Config, logger luxlog.Logger) (*audit.Recorder, err
if logger != nil {
count, head := rec.Head()
logger.Info("audit trail ready (tamper-evident, append-only)",
"store", dbPath, "records", count, "head", head,
"store", dbPath, "shard", shard, "records", count, "head", head,
"mirror", mirror != nil, "checkpoint_interval", interval.String())
}
return rec, nil
+68 -13
View File
@@ -37,31 +37,86 @@ type keyResolver interface {
// A brief cache keeps the hot auth path off the network; it caches misses too, so a
// bad key cannot hammer IAM.
type iamKeys struct {
base string
auth string // client_secret_basic, or "" when unconfigured
http *http.Client
cache cache[string, *idClaims]
base string
auth string // client_secret_basic, or "" when unconfigured
http *http.Client
cache cache[string, *idClaims]
}
// newIAMKeys reads the same IAM env clients/account does. With no confidential
// credential it returns a resolver that resolves nothing (keys stay anonymous —
// never a fabricated principal), so a deployment lacking the credential is safe.
func newIAMKeys() *iamKeys {
base := strings.TrimRight(env("IAM_URL", "IAM_INTERNAL_URL"), "/")
id := strings.TrimSpace(os.Getenv("IAM_MINT_CLIENT_ID"))
secret := strings.TrimSpace(os.Getenv("IAM_MINT_CLIENT_SECRET"))
var auth string
if id != "" && secret != "" {
auth = "Basic " + base64.StdEncoding.EncodeToString([]byte(id+":"+secret))
}
return &iamKeys{
base: base,
auth: auth,
base: iamHost(),
auth: iamCred(),
http: &http.Client{Timeout: 5 * time.Second},
cache: newCache[string, *idClaims](60 * time.Second),
}
}
// sharedKeys memoizes ONE API-key resolver (and its 60s cache) for the whole
// binary. The identity boundary (SanitizeIdentity, via newIdentityValidator) and
// any subsystem that must resolve a key OUT-OF-BAND of the Authorization header
// (analytics capture: a project key posted in the SDK body/query) both go through
// this ONE seam, so a key resolves to the SAME org either way and IAM sees one
// warm cache — never a second, drifting resolver.
var (
sharedKeysOnce sync.Once
sharedKeysInst *iamKeys
)
func sharedKeys() *iamKeys {
sharedKeysOnce.Do(func() { sharedKeysInst = newIAMKeys() })
return sharedKeysInst
}
// maxKeyOrgLen bounds a resolved org key the same way principal.MaxOrgLen does: the
// org becomes a warehouse partition key, so an over-long value (malformed / hostile)
// is refused rather than stored.
const maxKeyOrgLen = 128
// OrgForKey resolves an opaque Hanzo API key (hk-/sk-/pk-/fw_/hz_) to the org it
// belongs to — the SAME owner org SanitizeIdentity mints when that key arrives as a
// bearer — through the ONE IAM key seam (get-user?accessKey). It is the exported
// door a keyed, bearer-less SDK path uses to attribute a project key to a tenant.
//
// FAILS CLOSED: ("", false) for a non-key-shaped string, an unknown/unresolvable
// key, an unconfigured resolver, or an out-of-bounds org — never a fabricated or
// default tenant, so a bad key can never be written into another org's partition.
// The isAPIKey prefix gate keeps garbage strings off the IAM network path.
func OrgForKey(ctx context.Context, key string) (string, bool) {
key = strings.TrimSpace(key)
if !isAPIKey(key) {
return "", false
}
claims := sharedKeys().resolve(ctx, key)
if claims == nil {
return "", false
}
owner := strings.TrimSpace(claims.Owner)
if owner == "" || len(owner) > maxKeyOrgLen {
return "", false
}
return owner, true
}
// iamHost is the standalone IAM origin cloud talks to; iamCred is the service
// credential (client_secret_basic) it presents — the ONE IAM identity, shared by
// the API-key resolver here and the /v1/iam edge (iam_edge.go), so both
// authenticate to IAM the same way. Empty cred → a deployment lacking the
// credential stays safe (the caller treats "" as unconfigured).
func iamHost() string { return strings.TrimRight(env("IAM_URL", "IAM_INTERNAL_URL"), "/") }
func iamCred() string {
id := strings.TrimSpace(os.Getenv("IAM_MINT_CLIENT_ID"))
secret := strings.TrimSpace(os.Getenv("IAM_MINT_CLIENT_SECRET"))
if id == "" || secret == "" {
return ""
}
return "Basic " + base64.StdEncoding.EncodeToString([]byte(id+":"+secret))
}
func env(names ...string) string {
for _, n := range names {
if v := strings.TrimSpace(os.Getenv(n)); v != "" {
+36 -15
View File
@@ -42,6 +42,7 @@ type idClaims struct {
Owner string `json:"owner"` // org slug (the org)
Project string `json:"project"` // org SUB-SCOPE within owner (empty ⟹ default project)
BillingAccount string `json:"billing_account"` // WHO PAYS, stated by IAM (empty ⟹ pre-claim token)
Name string `json:"name"` // display name (id fallback)
PreferredUsername string `json:"preferred_username"` // id fallback
Email string `json:"email"`
@@ -62,6 +63,20 @@ func (c *idClaims) mintedProject() string {
return strings.TrimSpace(c.Project)
}
// mintedBillingAccount returns the funding account to stamp into
// X-Billing-Account-Id, or "" when the header must be OMITTED (a token minted
// before IAM shipped the claim, or one IAM could not attribute).
//
// WHO PAYS IS NOT A CLIENT'S TO NAME. This rides the validated `billing_account`
// claim — IAM's signed statement, resolved at the identity boundary from the real
// grant context — exactly like `owner` and `project`. It mirrors the edge
// (iamauth.Claims.MintedBillingAccount) byte-for-byte, so the in-binary path binds
// the same header the gateway would, and ai/object.Payer reads the same payer on
// both. The raw client copy is deleted on ingress and NEVER restored.
func (c *idClaims) mintedBillingAccount() string {
return strings.TrimSpace(c.BillingAccount)
}
// userID resolves the canonical user id: sub, then preferred_username, then
// name. IAM may leave sub empty. This is the STABLE identifier (a UUID when IAM
// sets sub) stamped as X-User-Id and consumed as the attribution key everywhere.
@@ -123,7 +138,7 @@ func newIdentityValidator(issuer, jwksURL string, audiences []string, ttl time.D
issuers: trustedIssuers(issuer),
audiences: audiences,
cache: newJWKSCache(jwksURL, ttl),
keys: newIAMKeys(),
keys: sharedKeys(), // ONE resolver+cache, shared with OrgForKey (analytics capture)
}
}
@@ -197,6 +212,16 @@ func (v *identityValidator) validate(raw string) (*idClaims, error) {
return nil, err
}
// Fail SECURE on a misconfigured (empty) trust set: an empty issuer OR audience
// allowlist must REJECT every token, never silently disable that axis. In
// production both are always resolved non-empty (BrandIssuers + the baked
// audience defaults, unioned in config.go so they are "never empty"), so this
// fires ONLY on an operator misconfiguration (CLOUD_JWT_AUDIENCES="" emptying the
// resolved set, or an empty issuer set) — and then it denies, it never admits (I2).
if len(v.issuers) == 0 || len(v.audiences) == 0 {
return nil, fmt.Errorf("identity validator misconfigured: empty issuer or audience allowlist")
}
// Reject a missing issuer: an empty issuer must never pass the set check.
if claims.Issuer == "" {
return nil, fmt.Errorf("missing issuer")
@@ -221,14 +246,13 @@ func (v *identityValidator) validate(raw string) (*idClaims, error) {
// gates on owner == :org. Without this, a real client_credentials machine token
// (aud == its per-org clientId, never in the allowlist) fails here and the
// sync silently stays pending — the activation blocker.
expected := jwt.Expected{}
if len(v.audiences) > 0 {
auds := v.audiences
if mach := kmsMachineAudience(claims.Owner); mach != "" {
auds = append(append(make([]string, 0, len(v.audiences)+1), v.audiences...), mach)
}
expected.AnyAudience = jwt.Audience(auds)
// The audience allowlist is guaranteed non-empty (checked above), so the
// audience axis is ALWAYS enforced — never silently skipped.
auds := v.audiences
if mach := kmsMachineAudience(claims.Owner); mach != "" {
auds = append(append(make([]string, 0, len(v.audiences)+1), v.audiences...), mach)
}
expected := jwt.Expected{AnyAudience: jwt.Audience(auds)}
if err := claims.Claims.ValidateWithLeeway(expected, 2*time.Minute); err != nil {
return nil, fmt.Errorf("claims: %w", err)
}
@@ -425,14 +449,11 @@ func trustedIssuers(primary string) []string {
return out
}
// issuerAllowed reports whether iss is one of the trusted issuers. An empty set
// (no primary, no brands — never the case in production) skips the check, matching
// the prior "empty issuer disables the check" behavior; a non-empty set is
// fail-secure (a token whose iss is not in the set is rejected).
// issuerAllowed reports whether iss is one of the trusted issuers. It is
// fail-secure in BOTH directions: an empty trusted set matches NOTHING (deny), so
// a misconfiguration that empties the issuer allowlist rejects every token instead
// of silently disabling the check (I2); a non-empty set rejects any iss not in it.
func issuerAllowed(iss string, trusted []string) bool {
if len(trusted) == 0 {
return true
}
for _, t := range trusted {
if iss == t {
return true
+39 -3
View File
@@ -1,10 +1,45 @@
package cloud
import (
"crypto/rand"
"crypto/rsa"
"os"
"testing"
"time"
)
// TestValidate_FailSecureOnEmptyTrustSet proves I2: a validator whose resolved
// issuer OR audience allowlist is empty REJECTS an otherwise-valid, correctly
// signed token — the axis is never silently disabled. Production always resolves
// non-empty sets; this guards the misconfiguration path (CLOUD_JWT_AUDIENCES=""
// or an empty issuer set), which must fail closed, not open.
func TestValidate_FailSecureOnEmptyTrustSet(t *testing.T) {
key, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
t.Fatalf("genkey: %v", err)
}
jwks := jwksServer(t, &key.PublicKey)
future := time.Now().Add(time.Hour)
tok := signWith(t, key, tokenClaims("hanzo-console", "acme", "", false, future))
// Sanity: a properly configured validator accepts the token.
if _, err := newIdentityValidator(testIssuer, jwks.URL, []string{"hanzo-console"}, 0).validate(tok); err != nil {
t.Fatalf("baseline valid token must be accepted, got %v", err)
}
// Empty audience set → deny.
if _, err := newIdentityValidator(testIssuer, jwks.URL, nil, 0).validate(tok); err == nil {
t.Error("empty audience allowlist must REJECT (fail-secure), not accept")
}
// Empty issuer set → deny (construct directly; trustedIssuers never yields empty
// with a primary, so bypass it to exercise the guard).
vEmptyIss := &identityValidator{issuers: nil, audiences: []string{"hanzo-console"}, cache: newJWKSCache(jwks.URL, 0), keys: newIAMKeys()}
if _, err := vEmptyIss.validate(tok); err == nil {
t.Error("empty issuer allowlist must REJECT (fail-secure), not accept")
}
}
// TestTrustedIssuers_WhiteLabel proves the in-binary validator's trusted-issuer
// set is the primary issuer UNIONED with every white-label brand issuer plus the
// WHITELABEL_ISSUERS override, deduped, primary-first.
@@ -40,7 +75,8 @@ func TestTrustedIssuers_WhiteLabel(t *testing.T) {
}
// TestIssuerAllowed proves the set membership check: brand issuers pass, an
// outsider is rejected, and an empty set (never in prod) skips the check.
// outsider is rejected, and an empty set is fail-secure — it matches NOTHING (I2),
// so a misconfiguration that empties the allowlist denies every token.
func TestIssuerAllowed(t *testing.T) {
set := []string{"https://hanzo.id", "https://lux.id"}
if !issuerAllowed("https://lux.id", set) {
@@ -49,8 +85,8 @@ func TestIssuerAllowed(t *testing.T) {
if issuerAllowed("https://attacker.id", set) {
t.Error("attacker.id must be rejected")
}
if !issuerAllowed("anything", nil) {
t.Error("empty set must skip the check (matches prior empty-issuer behavior)")
if issuerAllowed("anything", nil) {
t.Error("empty set must DENY (fail-secure), never skip the check")
}
}
+11
View File
@@ -113,6 +113,17 @@ func BrandForHost(host string) string {
return DefaultBrand
}
// brandDisplay is a brand id's human display name: the id with an upper-cased
// first letter (lux → "Lux", hanzo → "Hanzo"). Derived from the id — one source
// of truth with the brands registry, no hand-maintained display list. Used to
// build the white-label console <title> (webui.go).
func brandDisplay(id string) string {
if id == "" {
id = DefaultBrand
}
return strings.ToUpper(id[:1]) + id[1:]
}
// BrandIssuers returns the OIDC issuer of every configured white-label brand. The
// in-binary identity validator (auth_identity.go) trusts a token whose `iss` is
// any of these, so ONE cloud binary validates hanzo AND lux/zoo/pars tokens. One
+54 -34
View File
@@ -14,7 +14,7 @@ import (
"github.com/hanzoai/cloud/clients"
"github.com/hanzoai/cloud/clients/finance"
"github.com/hanzoai/cloud/clients/gatewaypolicy"
"github.com/hanzoai/cloud/clients/gateway/edge"
"github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/cloud/clients/s3admin"
"github.com/hanzoai/cloud/types"
@@ -90,6 +90,7 @@ func BuildDeps(cfg *Config) Deps {
// commerce URL yields a !Enabled() client, so the wrap is a transparent
// pass-through and a dev deployment is never blocked.
deps.Metering = buildMeteringClient(cfg, logger)
wireTierReader(deps.Metering, logger)
deps.AI = meteredAIClient(pickAIClient(cfg, logger), deps)
wireFinance(cfg, logger)
deps.O11y = pick(cfg, logger, "o11y", "O11y", cfg.O11yZAPAddr, clients.O11yRPCAt, clients.DisabledO11y)
@@ -107,7 +108,7 @@ func BuildDeps(cfg *Config) Deps {
// working *Store (static-only if the SQLite file can't open), so the edge
// middleware is never left without a policy source — a store-open error is
// logged, not fatal.
gp, err := gatewaypolicy.New(cfg.DataDir, cfg.AdminOrg, staticEdgePolicy(cfg))
gp, err := edge.New(cfg.DataDir, cfg.AdminOrg, staticEdgePolicy(cfg))
if err != nil {
logger.Warn("gateway policy store degraded to static-only", "err", err)
}
@@ -117,10 +118,10 @@ func BuildDeps(cfg *Config) Deps {
}
// staticEdgePolicy projects the static env/flag edge config into the boot-default
// policy the gatewaypolicy.Store layers runtime overrides on top of. A disabled
// policy the edge.Store layers runtime overrides on top of. A disabled
// per-IP limiter (CLOUD_EDGE_RATELIMIT=false) maps to PerIPRPM 0 (a live no-op).
func staticEdgePolicy(cfg *Config) gatewaypolicy.Policy {
p := gatewaypolicy.Policy{
func staticEdgePolicy(cfg *Config) edge.Policy {
p := edge.Policy{
CORSOrigins: cfg.CORSOrigins,
WindowSec: cfg.EdgeRateWindowSec,
}
@@ -157,9 +158,15 @@ func buildMeteringClient(cfg *Config, log luxlog.Logger) *metering.Client {
httpClient = commerceinproc.Client(0) // in-process dispatch; no network timeout
}
m, err := metering.New(metering.Config{
BaseURL: base,
Token: cfg.CommerceServiceToken,
Org: cfg.Brand, // X-Org-Id default for S2S; per-request org overrides.
BaseURL: base,
Token: cfg.CommerceServiceToken,
Org: cfg.Brand, // X-Org-Id default for S2S; per-request org overrides.
// Honor the documented METERING_TEST env: when "true", route every debit to
// commerce's TEST/sandbox books (fin.RecordUsage in.Test=true) so a staging /
// canary deployment records NO real money — and the usage-cap read (org.TestMode
// via SQUARE_ENVIRONMENT=sandbox) sees the SAME test books. Unset in prod → live,
// unchanged. Without this the flag was silently ignored (always live).
Test: strings.EqualFold(strings.TrimSpace(os.Getenv(metering.EnvTest)), "true"),
FailOpen: cfg.BillingFailOpen,
HTTPClient: httpClient, // nil off the co-resident path → metering builds its own
})
@@ -169,6 +176,11 @@ func buildMeteringClient(cfg *Config, log luxlog.Logger) *metering.Client {
log.Error("billing: invalid commerce URL, gate disabled", "err", err)
m, _ = metering.New(metering.Config{})
}
// Observe every cap-check fail-open (timeout / slow / broken commerce) — a cap that
// silently allows must never be silent. The completion still proceeds (fail-open).
metering.OnCapError = func(err error) {
log.Warn("spend-cap check failed open (allowing completion) — commerce authorize slow/unavailable", "err", err)
}
if m.Enabled() {
log.Info("billing gate enabled", "commerce", boolStr(inProcess, "in-process", "http:"+base), "fail_open", cfg.BillingFailOpen)
} else {
@@ -184,6 +196,28 @@ func boolStr(b bool, t, f string) string {
return f
}
// wireTierReader installs the embedded ai module's per-tier SKU gate reader so it
// resolves the caller's commerce subscription tier through the SAME co-resident
// commerce client the metering gate bills over — in-process (commerceinproc) when
// commerce is folded in, S2S HTTP with the service token otherwise — NEVER an authed
// self-call to the cloud edge. That self-call is the toothless-gate bug: the edge
// 401/403s a service call to /v1/billing/*, so the ai module's own HTTP lookup always
// returned "" in-cluster and every tier-gated SKU failed OPEN. This mirrors
// wireFinance's SetBalanceReader: cloud owns the co-resident read, ai stays
// transport-agnostic. Fail-safe is preserved — Client.Tier folds a commerce error or
// an unknown plan to "", which the gate treats as ALLOW, so a commerce blip never
// locks out a paying caller. No-op when commerce is unreachable (metering !Enabled),
// leaving ai's standalone HTTP fallback in place.
func wireTierReader(m *metering.Client, log luxlog.Logger) {
if m == nil || !m.Enabled() {
return
}
aiobject.SetTierReader(func(ctx context.Context, subject, namespace string) (string, error) {
return m.Tier(ctx, subject, namespace)
})
log.Info("ai per-tier SKU gate wired to co-resident commerce (in-process tier read, fail-safe)")
}
// wireFinance constructs the ONE in-process finance ledger (per-org SQLite
// double-entry prepaid wallet), publishes it for every money consumer to resolve by
// the narrow finance.Client, and installs the embedded ai router's balance-read +
@@ -656,30 +690,17 @@ func pickVaultClient(cfg *Config, log luxlog.Logger) VaultClient {
return clients.DisabledVault()
}
// MountFunc is a subsystem's mount contract. app is `any`, not *zip.App, and that
// is load-bearing: some external modules expose Mount as func(any, Deps) error
// (e.g. hanzoai/licensing), which apps.Wire references DIRECTLY — a
// func(any,…) value is not assignable to a func(*zip.App,…) parameter, so
// narrowing the type would break them at compile time. The concrete value is
// always a *zip.App; strongly-typed Mounts (func(*zip.App, Deps) error, what every
// in-repo subsystem exports) are adapted by Typed, which recovers it in ONE place.
type MountFunc func(app any, deps Deps) error
// Typed adapts a strongly-typed subsystem Mount — func(*zip.App, Deps) error,
// the signature every in-repo subsystem already exports — into the registry's
// MountFunc. It performs the *zip.App recovery in ONE place, fail-closed with a
// clear error, so no subsystem repeats the `a, ok := app.(*zip.App)` boilerplate.
// The concrete value MountAll passes is always a *zip.App, so the assertion is
// total in practice; it stays as a defensive, self-documenting guard.
func Typed(mount func(*zip.App, Deps) error) MountFunc {
return func(app any, deps Deps) error {
a, ok := app.(*zip.App)
if !ok {
return fmt.Errorf("cloud.Mount: app is %T, want *zip.App", app)
}
return mount(a, deps)
}
}
// MountFunc is a subsystem's mount contract: register your routes on app, using
// deps for everything shared. Every subsystem in the fleet exports exactly this
// signature, so Wire references each one directly and the compiler checks it.
//
// app was once `any`, on the stated grounds that an external module (licensing)
// exposed func(any, Deps) error and narrowing would break it — while licensing
// said it used `any` to avoid an import cycle in pkg/cloud. Each cited the other,
// and the cycle could not exist: this package already imports zip, and zip does
// not import cloud. The `any` bought nothing and cost every subsystem a Typed()
// wrapper plus a runtime type assertion whose failure branch was unreachable.
type MountFunc func(app *zip.App, deps Deps) error
// ShutdownFunc releases a subsystem's process-lifetime resources (background
// goroutines, open DB handles) on graceful shutdown. It must be idempotent and
@@ -704,8 +725,7 @@ type MountSpec struct {
// MountAll mounts every ENABLED subsystem in specs, in slice order — the order is
// the composition root's (apps.Wire()); MountAll does NOT sort. app is the
// concrete *zip.App from Serve; the MountFunc accepts it as `any` and in-repo
// subsystems recover it via Typed.
// concrete *zip.App from Serve, handed to each MountFunc as itself.
//
// Teardown is wired HERE, at mount time: right after a subsystem mounts, its
// ShutdownFunc (if any) is registered via app.OnShutdown. zip drains those hooks
+1 -1
View File
@@ -16,7 +16,7 @@ import (
// noopMount mounts nothing: the fake specs below carry the behavior under test in
// their Shutdown, not their Mount.
func noopMount(any, cloud.Deps) error { return nil }
func noopMount(*zip.App, cloud.Deps) error { return nil }
// freeAddr reserves an ephemeral loopback port and hands back its address; the
// listener is closed so the app under test can bind it.
+18 -39
View File
@@ -1,49 +1,28 @@
package cloud_test
import (
"strings"
"testing"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
)
// TestTyped_RecoversZipApp verifies cloud.Typed adapts a strongly-typed
// func(*zip.App, Deps) into the registry MountFunc: it hands the concrete
// *zip.App straight through to the wrapped mount.
func TestTyped_RecoversZipApp(t *testing.T) {
app := zip.New(zip.Config{})
var got *zip.App
mf := cloud.Typed(func(a *zip.App, _ cloud.Deps) error {
got = a
return nil
})
if err := mf(app, cloud.Deps{}); err != nil {
t.Fatalf("Typed mount returned error: %v", err)
}
if got != app {
t.Fatalf("Typed did not pass the concrete *zip.App through (got %p, want %p)", got, app)
}
}
// TestTyped_WrongTypeFailsClosed verifies cloud.Typed fails closed with a clear
// error — never a panic — when the registry passes a value that is not a
// *zip.App. This is the single, central replacement for the per-subsystem
// assertion boilerplate.
func TestTyped_WrongTypeFailsClosed(t *testing.T) {
called := false
mf := cloud.Typed(func(*zip.App, cloud.Deps) error {
called = true
return nil
})
err := mf("not-a-zip-app", cloud.Deps{})
if err == nil {
t.Fatal("Typed must return an error on a non-*zip.App value")
}
if called {
t.Fatal("Typed must NOT invoke the wrapped mount on a type mismatch")
}
if !strings.Contains(err.Error(), "*zip.App") {
t.Errorf("error should name the wanted type *zip.App, got: %v", err)
}
// TestMountFunc_IsTheSubsystemSignature pins the registry's mount contract: the
// signature every subsystem exports IS a cloud.MountFunc, checked by the compiler.
//
// This file used to test cloud.Typed, the adapter that took a MountFunc's `any`
// app and asserted it back to *zip.App. Both of its tests went with it, and
// neither is a loss:
//
// - "Typed recovers the *zip.App" only ever proved the adapter handed through
// the value it was given. MountFunc now names *zip.App, so there is no
// recovery step left to get wrong.
// - "Typed fails closed on a wrong type" can no longer be written: passing
// "not-a-zip-app" to a MountFunc is a compile error, so the runtime branch it
// exercised does not exist. A test asserting a wrong type is rejected is
// precisely what a type already is.
//
// What remains is the only claim worth making, and the build enforces it.
func TestMountFunc_IsTheSubsystemSignature(t *testing.T) {
var _ cloud.MountFunc = func(*zip.App, cloud.Deps) error { return nil }
}
+51
View File
@@ -0,0 +1,51 @@
package cloud
import (
"io"
"net/http"
"net/http/httptest"
"testing"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
// CallerBearer relays the caller's OWN validated JWT bearer and nothing else: a JWT
// passes through unchanged, an opaque API key is not relayable, and no credential
// yields "". This is the token a downstream org-scoped service (the DNS forward
// head) re-validates to enforce tenant isolation across the hop.
func TestCallerBearer(t *testing.T) {
app := zip.New(zip.Config{Logger: luxlog.New("test")})
app.Get("/probe", func(c *zip.Ctx) error { return c.Bytes(200, []byte(CallerBearer(c))) })
probe := func(setup func(*http.Request)) string {
req := httptest.NewRequest(http.MethodGet, "/probe", nil)
if setup != nil {
setup(req)
}
res, err := app.Fiber().Test(req)
if err != nil {
t.Fatal(err)
}
b, _ := io.ReadAll(res.Body)
_ = res.Body.Close()
return string(b)
}
cases := []struct {
name string
setup func(*http.Request)
want string
}{
{"jwt bearer relayed unchanged", func(r *http.Request) { r.Header.Set("Authorization", "Bearer jwt.header.sig") }, "jwt.header.sig"},
{"X-Authorization fallback", func(r *http.Request) { r.Header.Set("X-Authorization", "Bearer x.y.z") }, "x.y.z"},
{"opaque hk- api key is NOT relayable", func(r *http.Request) { r.Header.Set("Authorization", "Bearer hk-secret") }, ""},
{"opaque sk- api key is NOT relayable", func(r *http.Request) { r.Header.Set("Authorization", "Bearer sk-secret") }, ""},
{"no credential yields empty", nil, ""},
}
for _, c := range cases {
if got := probe(c.setup); got != c.want {
t.Errorf("%s: CallerBearer = %q, want %q", c.name, got, c.want)
}
}
}
+128 -22
View File
@@ -160,24 +160,21 @@ type loginFlags struct {
}
func runLogin(env *Env, lf *loginFlags, cmd *cobra.Command) error {
creds, err := LoadCredentials()
if err != nil {
return err
}
var creds *Credentials
switch {
case lf.token != "":
// Paste an externally-minted token. Decode claims for identity.
tr := &tokenResp{AccessToken: lf.token, TokenType: "Bearer"}
creds = credsFromToken(tr)
creds = credsFromToken(&tokenResp{AccessToken: lf.token, TokenType: "Bearer"})
case lf.username != "" || lf.passwordStdin:
// Password grant — kept for automation (--username/--password-stdin).
username := lf.username
if username == "" {
username, err = prompt(cmd, "Email: ")
u, err := prompt(cmd, "Email: ")
if err != nil {
return err
}
username = u
}
password, err := readPassword(cmd, lf.passwordStdin)
if err != nil {
@@ -192,14 +189,15 @@ func runLogin(env *Env, lf *loginFlags, cmd *cobra.Command) error {
default:
// The ONE interactive way: RFC 8628 device flow — link + QR + code,
// approve from any signed-in browser or phone. Headless-safe.
creds, err = runDeviceLogin(cmd, env, lf.scope)
c, err := runDeviceLogin(cmd, env, lf.scope)
if err != nil {
return err
}
creds = c
}
// Optional machine-to-machine tokens for the platform control plane,
// stored alongside the identity so apps/deploy work post-login.
// stored with this identity so apps/deploy work post-login.
if lf.platformToken != "" {
creds.PlatformToken = lf.platformToken
}
@@ -207,14 +205,23 @@ func runLogin(env *Env, lf *loginFlags, cmd *cobra.Command) error {
creds.BuildToken = lf.buildToken
}
if err := creds.Save(); err != nil {
// Persist under this identity's stable key and make it active. A second
// login as a different owner (e.g. admin vs hanzo for the same email, via a
// different --client-id) is stored beside the first, never over it;
// credentials.json mirrors whichever is active for legacy readers.
store, err := LoadIdentities()
if err != nil {
return err
}
key := store.Put(creds)
if err := store.Save(); err != nil {
return err
}
who := firstNonEmpty(creds.Subject, "(unknown)")
if creds.Owner != "" {
who += " @ " + creds.Owner
}
fmt.Fprintf(cmd.OutOrStdout(), "Logged in as %s (token expires %s)\n", who, shortTime(creds.Expiry))
fmt.Fprintf(cmd.OutOrStdout(), "Logged in as %s [%s] (token expires %s)\n", who, key, shortTime(creds.Expiry))
return nil
}
@@ -249,28 +256,92 @@ func bindLoginFlags(cmd *cobra.Command, lf *loginFlags) {
func newLogoutCmd() *cobra.Command {
return &cobra.Command{
Use: "logout",
Short: "Remove stored credentials",
Args: cobra.NoArgs,
Use: "logout [<owner>]",
Short: "Remove a stored identity (the active one, or the named owner)",
Args: cobra.MaximumNArgs(1),
PersistentPreRunE: func(*cobra.Command, []string) error { return nil },
RunE: func(cmd *cobra.Command, _ []string) error {
if err := DeleteCredentials(); err != nil {
RunE: func(cmd *cobra.Command, args []string) error {
store, err := LoadIdentities()
if err != nil {
return err
}
fmt.Fprintln(cmd.OutOrStdout(), "Logged out.")
if len(store.Identities) == 0 {
fmt.Fprintln(cmd.OutOrStdout(), "Not logged in.")
return nil
}
target := store.Active
if len(args) == 1 {
if target, err = store.resolve(args[0]); err != nil {
return err
}
}
store.Remove(target)
if err := store.Save(); err != nil {
return err
}
msg := "Logged out of " + target + "."
if store.Active != "" {
msg += " Active is now " + store.Active + "."
}
fmt.Fprintln(cmd.OutOrStdout(), msg)
return nil
},
}
}
// identityRow is the JSON/table projection of one stored identity.
type identityRow struct {
Key string `json:"key"`
Owner string `json:"owner"`
Subject string `json:"subject"`
Expiry int64 `json:"expiry,omitempty"`
Active bool `json:"active"`
}
// listIdentities renders every stored identity (active marked with *) — the
// shared body of `hanzo auth list` and `hanzo whoami --all`.
func listIdentities(env *Env, _ *cobra.Command) error {
store, err := LoadIdentities()
if err != nil {
return err
}
rows := make([]identityRow, 0, len(store.Identities))
for _, k := range store.keys() {
c := store.Identities[k]
rows = append(rows, identityRow{
Key: k, Owner: c.Owner, Subject: c.Subject, Expiry: c.Expiry,
Active: k == store.Active,
})
}
return env.emit(rows, func(w io.Writer) {
if len(rows) == 0 {
fmt.Fprintln(w, "No stored identities. Run `hanzo login`.")
return
}
tw := newTab(w)
fmt.Fprintln(tw, "ACTIVE\tKEY\tOWNER\tSUBJECT\tEXPIRES")
for _, r := range rows {
active := ""
if r.Active {
active = "*"
}
fmt.Fprintf(tw, "%s\t%s\t%s\t%s\t%s\n", active, r.Key, r.Owner, r.Subject, shortTime(r.Expiry))
}
tw.Flush()
})
}
func newWhoamiCmd(envOf func() *Env) *cobra.Command {
var verify bool
var verify, all bool
cmd := &cobra.Command{
Use: "whoami",
Short: "Show the current identity from the stored token",
Short: "Show the active identity from the stored token (--all lists every stored identity)",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
env := envOf()
if all {
return listIdentities(env, cmd)
}
tok := env.accessToken()
if tok == "" {
return fmt.Errorf("not logged in: run `hanzo login`")
@@ -300,6 +371,7 @@ func newWhoamiCmd(envOf func() *Env) *cobra.Command {
},
}
cmd.Flags().BoolVar(&verify, "verify", false, "verify the token against the IAM userinfo endpoint")
cmd.Flags().BoolVar(&all, "all", false, "list every stored identity (active marked with *)")
return cmd
}
@@ -329,11 +401,11 @@ func verifyUserInfo(ctx context.Context, env *Env, token string) error {
func newAuthCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
cmd := &cobra.Command{
Use: "auth",
Short: "Manage authentication",
Short: "Manage authentication and stored identities",
}
tokenCmd := &cobra.Command{
Use: "token",
Short: "Print the stored access token",
Short: "Print the active access token",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
tok := envOf().accessToken()
@@ -344,7 +416,41 @@ func newAuthCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
return nil
},
}
cmd.AddCommand(newLoginCmd(envOf, gf), newLogoutCmd(), newWhoamiCmd(envOf), tokenCmd)
listCmd := &cobra.Command{
Use: "list",
Aliases: []string{"ls", "identities"},
Short: "List stored identities (active marked with *)",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error { return listIdentities(envOf(), cmd) },
}
switchCmd := &cobra.Command{
Use: "switch <owner>",
Aliases: []string{"use"},
Short: "Make a stored identity active (accepts an owner, or a full owner/name key)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
store, err := LoadIdentities()
if err != nil {
return err
}
key, err := store.resolve(args[0])
if err != nil {
return err
}
store.Active = key
if err := store.Save(); err != nil {
return err
}
c := store.Identities[key]
who := firstNonEmpty(c.Subject, "(unknown)")
if c.Owner != "" {
who += " @ " + c.Owner
}
fmt.Fprintf(cmd.OutOrStdout(), "Switched to %s [%s] (token expires %s)\n", who, key, shortTime(c.Expiry))
return nil
},
}
cmd.AddCommand(newLoginCmd(envOf, gf), newLogoutCmd(), newWhoamiCmd(envOf), tokenCmd, listCmd, switchCmd)
return cmd
}
+175
View File
@@ -208,3 +208,178 @@ func TestAuthTokenCommand(t *testing.T) {
t.Fatalf("auth token output: %q", out)
}
}
// TestMultiIdentityLoginSwitch is the full multi-identity story: two logins for
// the same email under different owners (admin vs hanzo — the privilege-
// separation case) coexist, `auth list` shows both, `switch` flips the active
// pointer and rewrites credentials.json, and legacy single-file readers always
// see the active identity.
func TestMultiIdentityLoginSwitch(t *testing.T) {
sandbox(t)
adminTok := makeJWT(map[string]any{"email": "z@hanzo.ai", "owner": "admin", "sub": "u-admin", "exp": float64(2000000000)})
hanzoTok := makeJWT(map[string]any{"email": "z@hanzo.ai", "owner": "hanzo", "sub": "u-hanzo", "exp": float64(2000000001)})
// First login → admin/z is stored and active.
out, err := runRoot(t, "", "login", "--token", adminTok)
if err != nil {
t.Fatalf("login admin: %v", err)
}
if !strings.Contains(out, "admin/z") {
t.Fatalf("login should report the key: %q", out)
}
if c, _ := LoadCredentials(); c.Owner != "admin" || c.Subject != "z@hanzo.ai" {
t.Fatalf("active not admin after first login: %+v", c)
}
// Second login (different owner) → added beside admin/z, becomes active,
// does NOT clobber the first.
if _, err := runRoot(t, "", "login", "--token", hanzoTok); err != nil {
t.Fatalf("login hanzo: %v", err)
}
store, err := LoadIdentities()
if err != nil {
t.Fatalf("load identities: %v", err)
}
if len(store.Identities) != 2 {
t.Fatalf("want 2 identities, got %d: %v", len(store.Identities), store.keys())
}
if store.Identities["admin/z"] == nil || store.Identities["hanzo/z"] == nil {
t.Fatalf("both identities must persist, got %v", store.keys())
}
if store.Active != "hanzo/z" {
t.Fatalf("active = %q, want hanzo/z (last login)", store.Active)
}
// Legacy reader sees the active (hanzo) identity.
if c, _ := LoadCredentials(); c.Owner != "hanzo" {
t.Fatalf("credentials.json not mirroring active: %+v", c)
}
// auth list shows both, with the active row marked.
out, err = runRoot(t, "", "auth", "list")
if err != nil {
t.Fatalf("auth list: %v", err)
}
for _, want := range []string{"admin/z", "hanzo/z", "z@hanzo.ai", "*"} {
if !strings.Contains(out, want) {
t.Fatalf("auth list missing %q in:\n%s", want, out)
}
}
// switch admin → active flips + credentials.json is rewritten to admin.
if _, err := runRoot(t, "", "auth", "switch", "admin"); err != nil {
t.Fatalf("auth switch admin: %v", err)
}
if st, _ := LoadIdentities(); st.Active != "admin/z" {
t.Fatalf("active after switch = %q, want admin/z", st.Active)
}
if c, _ := LoadCredentials(); c.Owner != "admin" || c.Subject != "z@hanzo.ai" {
t.Fatalf("switch did not rewrite credentials.json: %+v", c)
}
// whoami (top-level, reads the active token) reflects admin.
out, err = runRoot(t, "", "whoami")
if err != nil {
t.Fatalf("whoami: %v", err)
}
if !strings.Contains(out, "admin") || !strings.Contains(out, "z@hanzo.ai") {
t.Fatalf("whoami not reflecting the switched-to identity: %q", out)
}
// switch by the full owner/name key works too.
if _, err := runRoot(t, "", "auth", "switch", "hanzo/z"); err != nil {
t.Fatalf("auth switch hanzo/z: %v", err)
}
if c, _ := LoadCredentials(); c.Owner != "hanzo" {
t.Fatalf("switch by full key failed: %+v", c)
}
}
// TestAuthListJSON checks the machine-readable projection.
func TestAuthListJSON(t *testing.T) {
sandbox(t)
tok := makeJWT(map[string]any{"email": "z@hanzo.ai", "owner": "admin", "sub": "a"})
if _, err := runRoot(t, "", "login", "--token", tok); err != nil {
t.Fatalf("login: %v", err)
}
out, err := runRoot(t, "", "auth", "list", "-o", "json")
if err != nil {
t.Fatalf("auth list json: %v", err)
}
var rows []identityRow
if err := json.Unmarshal([]byte(out), &rows); err != nil {
t.Fatalf("json unmarshal: %v\n%s", err, out)
}
if len(rows) != 1 || rows[0].Key != "admin/z" || rows[0].Owner != "admin" || !rows[0].Active {
t.Fatalf("json rows wrong: %+v", rows)
}
}
// TestLogoutOneOfMany removes a single identity and, only when the last one is
// gone, clears the store entirely.
func TestLogoutOneOfMany(t *testing.T) {
sandbox(t)
admin := makeJWT(map[string]any{"email": "z@hanzo.ai", "owner": "admin", "sub": "a"})
hanzo := makeJWT(map[string]any{"email": "z@hanzo.ai", "owner": "hanzo", "sub": "h"})
if _, err := runRoot(t, "", "login", "--token", admin); err != nil {
t.Fatal(err)
}
if _, err := runRoot(t, "", "login", "--token", hanzo); err != nil { // active = hanzo/z
t.Fatal(err)
}
// logout of the named owner (admin) leaves hanzo/z active.
if _, err := runRoot(t, "", "logout", "admin"); err != nil {
t.Fatalf("logout admin: %v", err)
}
store, _ := LoadIdentities()
if store.Identities["admin/z"] != nil {
t.Fatalf("admin/z not removed: %v", store.keys())
}
if store.Active != "hanzo/z" {
t.Fatalf("active = %q, want hanzo/z", store.Active)
}
if c, _ := LoadCredentials(); c.Owner != "hanzo" {
t.Fatalf("credentials.json not mirroring survivor: %+v", c)
}
// logout of the active (no arg) removes the last identity → both files gone.
if _, err := runRoot(t, "", "logout"); err != nil {
t.Fatalf("logout active: %v", err)
}
if c, _ := LoadCredentials(); c.AccessToken != "" {
t.Fatalf("credentials.json not cleared: %+v", c)
}
if st, _ := LoadIdentities(); len(st.Identities) != 0 {
t.Fatalf("identity store not cleared: %v", st.keys())
}
}
// TestMigrateLegacyCredentials proves a pre-multi-identity credentials.json is
// adopted into the store and preserved when a new identity is added.
func TestMigrateLegacyCredentials(t *testing.T) {
sandbox(t)
// Simulate an old single-file login: only credentials.json exists.
legacy := credsFromToken(&tokenResp{AccessToken: makeJWT(map[string]any{
"email": "z@hanzo.ai", "owner": "hanzo", "sub": "h",
})})
if err := legacy.Save(); err != nil {
t.Fatal(err)
}
store, err := LoadIdentities()
if err != nil {
t.Fatalf("load identities: %v", err)
}
if store.Identities["hanzo/z"] == nil || store.Active != "hanzo/z" {
t.Fatalf("legacy credentials not migrated: active=%q keys=%v", store.Active, store.keys())
}
// A fresh login as a different owner preserves the migrated identity.
if _, err := runRoot(t, "", "login", "--token", makeJWT(map[string]any{
"email": "z@hanzo.ai", "owner": "admin", "sub": "a",
})); err != nil {
t.Fatal(err)
}
st2, _ := LoadIdentities()
if len(st2.Identities) != 2 || st2.Identities["hanzo/z"] == nil {
t.Fatalf("migrated identity lost after new login: %v", st2.keys())
}
}
+180 -9
View File
@@ -55,7 +55,7 @@ var controlCommands = map[string]string{
"login": "authenticate against Hanzo IAM (hanzo.id) and store a token",
"logout": "remove stored credentials",
"whoami": "show the current identity from the stored token",
"auth": "manage authentication (login, logout, whoami, token)",
"auth": "manage authentication + stored identities (login, logout, whoami, list, switch, token)",
"apps": "list/get the platform apps board (declared/running/drift)",
"deploy": "drive a platform redeploy (rolling restart, zero-downtime)",
"clusters": "provision/list/select dedicated DOKS clusters",
@@ -236,6 +236,170 @@ func DeleteCredentials() error {
return nil
}
// ---------------------------------------------------------------------------
// Identity store — ~/.hanzo/identities.json. Holds EVERY logged-in identity
// keyed by its stable "<owner>/<name>" key, with an Active pointer. On every
// write the active identity is mirrored into credentials.json (above), so every
// legacy single-file reader keeps seeing the current identity unchanged. This
// is the ONE credential store; credentials.json is its active-view mirror.
// ---------------------------------------------------------------------------
// IdentityStore is the on-disk shape of ~/.hanzo/identities.json.
type IdentityStore struct {
Active string `json:"active,omitempty"`
Identities map[string]*Credentials `json:"identities,omitempty"`
}
// key is the stable per-identity store key "<owner>/<name>", where name is the
// email local-part (else the raw subject). The same identity yields the same
// key every login, so re-login updates in place; the same email under a
// different org (privilege separation) yields a distinct key (admin/z vs
// hanzo/z) and is stored side by side rather than clobbering.
func (c *Credentials) key() string {
name := c.Subject
if i := strings.IndexByte(name, '@'); i > 0 {
name = name[:i]
}
return firstNonEmpty(c.Owner, "-") + "/" + firstNonEmpty(name, "-")
}
func identitiesPath() (string, error) {
dir, err := hanzoDir()
if err != nil {
return "", err
}
return filepath.Join(dir, "identities.json"), nil
}
// LoadIdentities reads the store. A pre-existing single-file credentials.json
// with no store yet is migrated in (read-only) as the sole, active identity, so
// upgrades are seamless — the first write persists it into the store.
func LoadIdentities() (*IdentityStore, error) {
p, err := identitiesPath()
if err != nil {
return nil, err
}
s := &IdentityStore{Identities: map[string]*Credentials{}}
if err := loadJSON(p, s); err != nil {
return nil, err
}
if s.Identities == nil {
s.Identities = map[string]*Credentials{}
}
if len(s.Identities) == 0 {
if c, err := LoadCredentials(); err == nil && c.AccessToken != "" {
k := c.key()
s.Identities[k] = c
s.Active = k
}
}
return s, nil
}
// keys returns the identity keys, sorted, for deterministic output.
func (s *IdentityStore) keys() []string {
ks := make([]string, 0, len(s.Identities))
for k := range s.Identities {
ks = append(ks, k)
}
sort.Strings(ks)
return ks
}
// Put stores c under its key and makes it active, returning the key.
func (s *IdentityStore) Put(c *Credentials) string {
if s.Identities == nil {
s.Identities = map[string]*Credentials{}
}
k := c.key()
s.Identities[k] = c
s.Active = k
return k
}
// Remove deletes an identity; Save re-points Active if it was the one removed.
func (s *IdentityStore) Remove(key string) { delete(s.Identities, key) }
// resolve turns a user selector into a stored key: an exact key wins; otherwise
// a bare owner matches iff exactly one identity carries it.
func (s *IdentityStore) resolve(sel string) (string, error) {
if _, ok := s.Identities[sel]; ok {
return sel, nil
}
var match []string
for _, k := range s.keys() {
if s.Identities[k].Owner == sel {
match = append(match, k)
}
}
switch len(match) {
case 1:
return match[0], nil
case 0:
return "", fmt.Errorf("no stored identity for %q (see `hanzo auth list`)", sel)
default:
return "", fmt.Errorf("%q is ambiguous across %s — pass the full owner/name key", sel, strings.Join(match, ", "))
}
}
// Save persists the store (0600) and mirrors the active identity into
// credentials.json for legacy single-file readers. When the store is empty it
// removes both files. Active is normalized to a real key first.
func (s *IdentityStore) Save() error {
if _, ok := s.Identities[s.Active]; !ok {
s.Active = ""
if ks := s.keys(); len(ks) > 0 {
s.Active = ks[0]
}
}
if len(s.Identities) == 0 {
return clearCredentialStore()
}
p, err := identitiesPath()
if err != nil {
return err
}
if err := writeJSON(p, s, 0o600); err != nil {
return err
}
return s.Identities[s.Active].Save() // mirror active → credentials.json (0600)
}
// SaveActive writes c back as the active identity (store + mirror), keeping the
// two consistent after an in-place token refresh. With no store yet it falls
// back to the single-file write.
func SaveActive(c *Credentials) error {
s, err := LoadIdentities()
if err != nil {
return err
}
if len(s.Identities) == 0 {
return c.Save()
}
k := s.Active
if k == "" || s.Identities[k] == nil {
k = c.key()
}
s.Identities[k] = c
s.Active = k
return s.Save()
}
// clearCredentialStore removes the identity store and its credentials.json
// mirror (used by logout when the last identity is removed).
func clearCredentialStore() error {
for _, pathOf := range []func() (string, error){credentialsPath, identitiesPath} {
p, err := pathOf()
if err != nil {
return err
}
if err := os.Remove(p); err != nil && !os.IsNotExist(err) {
return err
}
}
return nil
}
// ---------------------------------------------------------------------------
// Env — the effective, resolved settings a command operates with.
// ---------------------------------------------------------------------------
@@ -310,11 +474,14 @@ func (e *Env) freshAccessToken() string {
return tok
}
// platformToken resolves the platform control-plane service token. The
// platform REST surface is machine-to-machine (it cannot validate IAM user
// tokens), so apps/clusters/redeploy authenticate with this, sourced from
// (in precedence) the bound --platform-token flag, the environment, then the
// credential store. Never hardcoded.
// platformToken resolves the bearer the platform control plane authenticates
// apps/clusters/redeploy with. ONE identity authorizes everything: after a plain
// `hanzo login` the IAM access token is the FINAL fallback, so no separate
// --platform-token is needed — the platform verifies the IAM JWT (signature,
// issuer, expiry) and org-scopes the caller. A dedicated service token still
// wins when present (flag > env > credential store > IAM login), so purpose-minted
// machine tokens keep their precedence and internal automation is unchanged.
// Never hardcoded.
func (e *Env) platformToken(flagVal string) string {
return firstNonEmpty(
flagVal,
@@ -322,17 +489,22 @@ func (e *Env) platformToken(flagVal string) string {
os.Getenv("PLATFORM_SERVICE_TOKEN"),
os.Getenv("PAAS_SERVICE_TOKEN"),
e.creds.PlatformToken,
e.accessToken(), // IAM login is the one identity that authorizes control-plane ops
)
}
// buildToken resolves the platform build-enqueue token (a distinct credential
// from the service token — see /v1/runner).
// buildToken resolves the bearer `hanzo build` sends to the platform build
// enqueue (/v1/runner). Same unify-infra contract as platformToken: a dedicated
// build token wins when present, but a plain IAM login is the FINAL fallback, so
// `hanzo build` works off the one identity with no separate --build-token — the
// platform verifies the IAM JWT and authorizes the build by org + role.
func (e *Env) buildToken(flagVal string) string {
return firstNonEmpty(
flagVal,
os.Getenv("HANZO_BUILD_TOKEN"),
os.Getenv("PLATFORM_BUILD_CALLBACK_TOKEN"),
e.creds.BuildToken,
e.accessToken(), // IAM login is the one identity that authorizes builds
)
}
@@ -424,7 +596,6 @@ func newRootCmd() *cobra.Command {
newDeployCmd(envOf, &f),
newClustersCmd(envOf, &f),
newBuildCmd(envOf, &f),
newK8sCmd(envOf, &f),
newConfigCmd(),
newSecurityCmd(envOf),
newGPUCmd(envOf, &f),
+54
View File
@@ -141,6 +141,29 @@ func TestPlatformTokenPrecedence(t *testing.T) {
}
}
// TestPlatformTokenFallsBackToIAM is the UNIFY-INFRA contract for the control
// plane: after a plain `hanzo login`, the IAM access token is the FINAL fallback
// so `hanzo apps`/`hanzo deploy` authorize off the one identity. An explicit
// platform service token (creds/env/flag) still wins.
func TestPlatformTokenFallsBackToIAM(t *testing.T) {
sandbox(t)
// Only an IAM login: no platform token anywhere ⇒ the IAM access token is sent.
e := resolve(&Config{}, &Credentials{AccessToken: "iam-jwt"}, globalFlags{})
if got := e.platformToken(""); got != "iam-jwt" {
t.Fatalf("IAM access token should be the final platform-token fallback: %q", got)
}
// A dedicated platform service token still beats the IAM token.
e = resolve(&Config{}, &Credentials{AccessToken: "iam-jwt", PlatformToken: "svc"}, globalFlags{})
if got := e.platformToken(""); got != "svc" {
t.Fatalf("dedicated platform token must beat the IAM fallback: %q", got)
}
// No login at all ⇒ empty (caller surfaces "run `hanzo login`").
e = resolve(&Config{}, &Credentials{}, globalFlags{})
if got := e.platformToken(""); got != "" {
t.Fatalf("no token and no login should resolve empty: %q", got)
}
}
func TestBuildTokenPrecedence(t *testing.T) {
sandbox(t)
e := resolve(&Config{}, &Credentials{BuildToken: "creds"}, globalFlags{})
@@ -156,6 +179,37 @@ func TestBuildTokenPrecedence(t *testing.T) {
}
}
// TestBuildTokenFallsBackToIAM is the UNIFY-INFRA contract: after a plain
// `hanzo login` (no --build-token), the IAM access token is the FINAL fallback,
// so `hanzo build` authorizes off the one identity. An explicit build token
// (creds/env/flag) still wins — the IAM token is the LAST resort, never an
// override of a purpose-minted machine token.
func TestBuildTokenFallsBackToIAM(t *testing.T) {
sandbox(t)
// Only an IAM login: no build token anywhere ⇒ the IAM access token is sent.
e := resolve(&Config{}, &Credentials{AccessToken: "iam-jwt"}, globalFlags{})
if got := e.buildToken(""); got != "iam-jwt" {
t.Fatalf("IAM access token should be the final build-token fallback: %q", got)
}
// A dedicated build token still beats the IAM token (precedence preserved).
e = resolve(&Config{}, &Credentials{AccessToken: "iam-jwt", BuildToken: "creds"}, globalFlags{})
if got := e.buildToken(""); got != "creds" {
t.Fatalf("dedicated build token must beat the IAM fallback: %q", got)
}
// HANZO_TOKEN (the env form of the IAM token) is also honored via accessToken().
e = resolve(&Config{}, &Credentials{}, globalFlags{})
t.Setenv("HANZO_TOKEN", "iam-env")
if got := e.buildToken(""); got != "iam-env" {
t.Fatalf("HANZO_TOKEN should back the build-token fallback: %q", got)
}
// No login at all ⇒ empty, so the caller can surface "run `hanzo login`".
t.Setenv("HANZO_TOKEN", "")
e = resolve(&Config{}, &Credentials{}, globalFlags{})
if got := e.buildToken(""); got != "" {
t.Fatalf("no token and no login should resolve empty: %q", got)
}
}
func TestAccessTokenFromEnvOverCreds(t *testing.T) {
sandbox(t)
e := resolve(&Config{}, &Credentials{AccessToken: "creds"}, globalFlags{})
+64 -22
View File
@@ -147,6 +147,7 @@ type codeAgent struct {
bin string // executable to exec
wire wire // how it finds the cloud
fullAuto []string // flags that bypass approval prompts
continueArgs []string // harness-native form of Hanzo -c/--continue
modelArg []string // how the model is passed on argv (empty: via env)
carrier func(model string) string // maps the resolved model to a client-recognized id (claude: zen→carrier); nil = pass through
provider func(base string) []string // agents that need the endpoint declared, not just env'd
@@ -163,10 +164,11 @@ type codeAgent struct {
// declared, so declare Hanzo as the provider and select it.
func codexLike(bin, install string) codeAgent {
return codeAgent{
bin: bin,
wire: openaiWire,
fullAuto: []string{"--dangerously-bypass-approvals-and-sandbox"},
modelArg: []string{"-m"},
bin: bin,
wire: openaiWire,
fullAuto: []string{"--dangerously-bypass-approvals-and-sandbox"},
continueArgs: []string{"resume", "--last"},
modelArg: []string{"-m"},
provider: func(base string) []string {
return []string{
"-c", "model_provider=hanzo",
@@ -174,6 +176,12 @@ func codexLike(bin, install string) codeAgent {
"-c", fmt.Sprintf(`model_providers.hanzo.base_url="%s/v1"`, strings.TrimSuffix(base, "/")),
"-c", `model_providers.hanzo.env_key="OPENAI_API_KEY"`,
"-c", `model_providers.hanzo.wire_api="responses"`,
// api.hanzo.ai exposes the standard OpenAI /v1/models shape,
// not Codex's private remote model-catalog schema. Skip that
// optional refresh and supply the coding model's metadata here.
"-c", `features.remote_models=false`,
"-c", `model_context_window=262144`,
"-c", `model_auto_compact_token_limit=235929`,
}
},
install: install,
@@ -190,9 +198,10 @@ const zenIdentityPrompt = "You are running through the Hanzo AI cloud as a Hanzo
var codeAgents = map[string]codeAgent{
"claude": {
bin: "claude",
wire: anthropicWire,
fullAuto: []string{"--dangerously-skip-permissions"},
bin: "claude",
wire: anthropicWire,
fullAuto: []string{"--dangerously-skip-permissions"},
continueArgs: []string{"--continue"},
// --model forces the session model on argv. Claude Code persists the
// user's last /model selection (e.g. the reserved word "best"), and that
// persisted choice OVERRIDES ANTHROPIC_MODEL — so the env var alone cannot
@@ -246,9 +255,12 @@ func newCodeCmd(envOf func() *Env, _ *globalFlags) *cobra.Command {
Long: "Run @hanzo/dev, Claude Code, or Codex against api.hanzo.ai with the endpoint,\n" +
"credential and model injected — no env vars to remember. `hanzo code` alone runs\n" +
"dev (the Hanzo agent); name an agent to pick another. Model ids resolve fuzzily\n" +
"(glm5.2 -> glm-5.2) and agents run full-auto unless you pass --safe.",
"(glm5.2 -> glm-5.2), -c resumes either harness, and agents run full-auto unless\n" +
"you pass --safe. Unknown options pass through; -- forces verbatim passthrough.",
Example: " hanzo code # dev, the default agent\n" +
" hanzo code claude\n" +
" hanzo code claude -c\n" +
" hanzo code codex -c\n" +
" hanzo code codex deepseek-v4-pro\n" +
" hanzo code dev glm5.2 -- --resume\n" +
" hanzo code ls",
@@ -320,19 +332,10 @@ func runCode(env *Env, agent codeAgent, args []string) error {
}
base := strings.TrimSuffix(firstNonEmpty(env.CloudURL, "https://api.hanzo.ai"), "/")
// First non-flag arg is the model; --safe is ours; the rest is the agent's.
model, safe, rest := "", false, make([]string, 0, len(args))
for _, a := range args {
switch {
case a == "--": // the separator is ours; the agent must not see it
case a == "--safe" || a == "--ask":
safe = true
case model == "" && !strings.HasPrefix(a, "-") && len(rest) == 0:
model = a
default:
rest = append(rest, a)
}
}
// First non-flag arg before -- is the model; --safe and --continue are ours.
// Unknown options pass through unchanged. Everything after -- belongs to the
// agent, including positional subcommands and raw Codex -c config overrides.
model, safe, continueLast, rest := splitCodeArgs(args)
if model == "" {
model = defaultCodeModel
}
@@ -382,7 +385,7 @@ func runCode(env *Env, agent codeAgent, args []string) error {
}
}
argv := codeArgv(agent, base, model, safe, rest)
argv := codeArgv(agent, base, model, safe, codeAgentRest(agent, continueLast, rest))
for k, v := range agent.wire(base, token, model) {
if err := os.Setenv(k, v); err != nil {
@@ -397,6 +400,45 @@ func runCode(env *Env, agent codeAgent, args []string) error {
return execEngine(bin, argv) // exec: signals + exit code flow straight through
}
// splitCodeArgs pulls the launcher-owned tokens (the model, --safe, -c/--continue)
// out of the raw args; everything else is the agent's. The `--` separator is ours
// and switches on verbatim passthrough — every token after it goes to the agent
// untouched, including positional subcommands (`codex exec`) and raw Codex -c
// config overrides that would otherwise look like our --continue.
func splitCodeArgs(args []string) (model string, safe, continueLast bool, rest []string) {
rest = make([]string, 0, len(args))
passthrough := false
for _, a := range args {
switch {
case passthrough:
rest = append(rest, a)
case a == "--": // the separator is ours; the agent must not see it
passthrough = true
case a == "--safe" || a == "--ask":
safe = true
case a == "-c" || a == "--continue":
continueLast = true
case model == "" && !strings.HasPrefix(a, "-") && len(rest) == 0:
model = a
default:
rest = append(rest, a)
}
}
return model, safe, continueLast, rest
}
// codeAgentRest prepends the agent's harness-native resume tokens when -c/--continue
// was given, so one Hanzo flag resumes the last session on either harness (`--continue`
// for Claude Code, `resume --last` for Codex/dev).
func codeAgentRest(agent codeAgent, continueLast bool, rest []string) []string {
if !continueLast {
return rest
}
args := make([]string, 0, len(agent.continueArgs)+len(rest))
args = append(args, agent.continueArgs...)
return append(args, rest...)
}
// codeArgv builds the final agent command line. Permission bypass is the
// launcher default for every agent; --safe is the single explicit opt-out.
func codeArgv(agent codeAgent, base, model string, safe bool, rest []string) []string {
+77 -10
View File
@@ -18,6 +18,7 @@ import (
"encoding/json"
"os"
"path/filepath"
"reflect"
"slices"
"testing"
"time"
@@ -48,6 +49,72 @@ func TestCodeAgentsBypassPermissionsByDefault(t *testing.T) {
}
}
func TestCodeArgsSeparatorPreservesAgentSubcommand(t *testing.T) {
model, safe, continueLast, rest := splitCodeArgs([]string{"--safe", "--", "exec", "--ephemeral", "do it"})
if model != "" || !safe || continueLast {
t.Fatalf("model=%q safe=%v continue=%v, want default model and safe mode", model, safe, continueLast)
}
if want := []string{"exec", "--ephemeral", "do it"}; !reflect.DeepEqual(rest, want) {
t.Fatalf("agent args = %q, want %q", rest, want)
}
}
func TestCodeArgsExplicitModelBeforeSeparator(t *testing.T) {
model, safe, continueLast, rest := splitCodeArgs([]string{"zen5-max", "--", "exec"})
if model != "zen5-max" || safe || continueLast || !reflect.DeepEqual(rest, []string{"exec"}) {
t.Fatalf("model=%q safe=%v continue=%v rest=%q", model, safe, continueLast, rest)
}
}
func TestCodeContinueIsNormalizedForBothHarnesses(t *testing.T) {
for _, tt := range []struct {
name string
want []string
}{
{name: "claude", want: []string{"--continue"}},
{name: "codex", want: []string{"resume", "--last"}},
} {
t.Run(tt.name, func(t *testing.T) {
model, safe, continueLast, rest := splitCodeArgs([]string{"-c"})
if model != "" || safe || !continueLast || len(rest) != 0 {
t.Fatalf("model=%q safe=%v continue=%v rest=%q", model, safe, continueLast, rest)
}
if got := codeAgentRest(codeAgents[tt.name], continueLast, rest); !reflect.DeepEqual(got, tt.want) {
t.Fatalf("normalized continue args = %q, want %q", got, tt.want)
}
})
}
}
func TestCodeUnknownOptionsAndPostSeparatorArgsPassThrough(t *testing.T) {
unknown := []string{"--mystery", "value", "--other=1"}
model, safe, continueLast, rest := splitCodeArgs(unknown)
if model != "" || safe || continueLast || !reflect.DeepEqual(rest, unknown) {
t.Fatalf("unknown options changed: model=%q safe=%v continue=%v rest=%q", model, safe, continueLast, rest)
}
_, _, continueLast, rest = splitCodeArgs([]string{"--", "-c", "model=x"})
if continueLast || !reflect.DeepEqual(rest, []string{"-c", "model=x"}) {
t.Fatalf("post-separator Codex config must pass verbatim: continue=%v rest=%q", continueLast, rest)
}
}
func TestCodexProviderUsesNativeResponsesMetadata(t *testing.T) {
argv := codeArgv(codeAgents["codex"], "https://api.hanzo.ai", defaultCodeModel, false, nil)
for _, want := range []string{
`model_provider=hanzo`,
`model_providers.hanzo.base_url="https://api.hanzo.ai/v1"`,
`model_providers.hanzo.wire_api="responses"`,
`features.remote_models=false`,
`model_context_window=262144`,
`model_auto_compact_token_limit=235929`,
} {
if !slices.Contains(argv, want) {
t.Errorf("Codex argv %q does not contain %q", argv, want)
}
}
}
// TestCodeTokenPrecedence locks in the 402 unblock: a fresh `hanzo login` JWT
// (which carries owner/project/sub on EVERY deployment) beats the hk- API key
// (which only mints a billing principal where the server has IAM_MINT_CLIENT_*).
@@ -60,10 +127,10 @@ func TestCodeTokenPrecedence(t *testing.T) {
freshExpiry := time.Now().Add(1 * time.Hour).Unix()
cases := []struct {
name string
envKey string // HANZO_API_KEY override
creds Credentials
want string
name string
envKey string // HANZO_API_KEY override
creds Credentials
want string
}{
{
name: "fresh JWT beats hk- key",
@@ -81,16 +148,16 @@ func TestCodeTokenPrecedence(t *testing.T) {
want: "hk-stored",
},
{
name: "HANZO_API_KEY overrides everything (deliberate operator override)",
name: "HANZO_API_KEY overrides everything (deliberate operator override)",
envKey: "hk-explicit",
creds: Credentials{AccessToken: "jwt-live", Expiry: freshExpiry},
want: "hk-explicit",
creds: Credentials{AccessToken: "jwt-live", Expiry: freshExpiry},
want: "hk-explicit",
},
{
name: "HANZO_API_KEY overrides even an expired JWT",
name: "HANZO_API_KEY overrides even an expired JWT",
envKey: "hk-explicit",
creds: Credentials{AccessToken: "jwt-dead", Expiry: time.Now().Add(-1 * time.Hour).Unix()},
want: "hk-explicit",
creds: Credentials{AccessToken: "jwt-dead", Expiry: time.Now().Add(-1 * time.Hour).Unix()},
want: "hk-explicit",
},
}
+103 -222
View File
@@ -3,6 +3,7 @@ package cli
import (
"fmt"
"io"
"strings"
"text/tabwriter"
"github.com/spf13/cobra"
@@ -15,12 +16,12 @@ func (e *Env) platform(gf *globalFlags) *Platform {
return newPlatform(e.PlatformURL, e.platformToken(gf.platformToken))
}
// deref renders a *string for a table cell, "-" when nil/empty.
func deref(p *string) string {
if p == nil || *p == "" {
// dashIfEmpty renders a string cell, "-" when empty.
func dashIfEmpty(s string) string {
if s == "" {
return "-"
}
return *p
return s
}
// yesno renders a bool for a table cell.
@@ -37,7 +38,8 @@ func newTab(w io.Writer) *tabwriter.Writer {
}
// ---------------------------------------------------------------------------
// apps — the observe surface.
// apps — the fleet drift board (GET /v1/paas/apps). Org-confined server-side by
// the IAM identity: a superadmin sees the whole fleet, an org-admin only its own.
// ---------------------------------------------------------------------------
func newAppsCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
@@ -56,7 +58,6 @@ func newAppsCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
RunE: func(cmd *cobra.Command, _ []string) error {
e := envOf()
res, err := e.platform(gf).Apps(cmd.Context(), AppsQuery{
Org: e.Org, // empty == all (single-tenant default)
Env: envFilter,
Health: healthFilter,
Drift: driftOnly,
@@ -69,8 +70,8 @@ func newAppsCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
fmt.Fprintln(tw, "ORG\tAPP\tENV\tDECLARED\tRUNNING\tHEALTH\tDRIFT")
for _, a := range res.Apps {
fmt.Fprintf(tw, "%s\t%s\t%s\t%s\t%s\t%s\t%s\n",
a.Org, a.App, a.Env, deref(a.DeclaredTag), deref(a.RunningTag),
deref(a.Health), driftSeverity(a.Drift))
a.Org, a.App, a.Env, dashIfEmpty(a.DeclaredTag), dashIfEmpty(a.RunningTag),
dashIfEmpty(a.Health), driftSeverity(a.Drift))
}
tw.Flush()
fmt.Fprintf(w, "\n%d apps (ok=%d yellow=%d red=%d)\n",
@@ -79,17 +80,17 @@ func newAppsCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
})
},
}
list.Flags().StringVar(&envFilter, "env", "", "filter by env: dev|test|main")
list.Flags().StringVar(&envFilter, "env", "", "filter by env: main|test|dev")
list.Flags().StringVar(&healthFilter, "health", "", "filter by health: green|yellow|red")
list.Flags().BoolVar(&driftOnly, "drift", false, "only rows that are drifting")
get := &cobra.Command{
Use: "get <org/app/env>",
Short: "Get one app row by its <org>/<app>/<env> id",
Use: "get <app>",
Short: "Get one app row by its CR name (production by default)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
e := envOf()
a, err := e.platform(gf).App(cmd.Context(), args[0], e.Org)
a, err := e.platform(gf).App(cmd.Context(), args[0])
if err != nil {
return err
}
@@ -101,109 +102,93 @@ func newAppsCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
fmt.Fprintf(tw, "env:\t%s\n", a.Env)
fmt.Fprintf(tw, "repo:\t%s\n", a.Repo)
fmt.Fprintf(tw, "registry:\t%s\n", a.Registry)
fmt.Fprintf(tw, "declared:\t%s\n", deref(a.DeclaredTag))
fmt.Fprintf(tw, "running:\t%s\n", deref(a.RunningTag))
fmt.Fprintf(tw, "latest:\t%s\n", deref(a.LatestTag))
fmt.Fprintf(tw, "health:\t%s\n", deref(a.Health))
fmt.Fprintf(tw, "declared:\t%s\n", dashIfEmpty(a.DeclaredTag))
fmt.Fprintf(tw, "running:\t%s\n", dashIfEmpty(a.RunningTag))
fmt.Fprintf(tw, "health:\t%s\n", dashIfEmpty(a.Health))
fmt.Fprintf(tw, "phase:\t%s\n", dashIfEmpty(a.Phase))
fmt.Fprintf(tw, "drift:\t%s\n", driftSeverity(a.Drift))
fmt.Fprintf(tw, "cluster:\t%s\n", deref(a.Cluster))
fmt.Fprintf(tw, "namespace:\t%s\n", deref(a.Namespace))
fmt.Fprintf(tw, "updated:\t%s\n", a.UpdatedAt)
fmt.Fprintf(tw, "cluster:\t%s\n", dashIfEmpty(a.Cluster))
fmt.Fprintf(tw, "namespace:\t%s\n", dashIfEmpty(a.Namespace))
if len(a.Endpoints) > 0 {
fmt.Fprintf(tw, "endpoints:\t%s\n", strings.Join(a.Endpoints, ", "))
}
tw.Flush()
})
},
}
sync := &cobra.Command{
Use: "sync",
Short: "Trigger an inventory refresh of the apps board",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
e := envOf()
if err := e.platform(gf).SyncApps(cmd.Context()); err != nil {
return err
}
fmt.Fprintln(cmd.OutOrStdout(), "apps sync triggered")
return nil
},
}
cmd.AddCommand(list, get, sync)
cmd.AddCommand(list, get)
return cmd
}
// ---------------------------------------------------------------------------
// deploy — the drive surface (rolling restart, zero-downtime).
// deploy — POST /v1/paas/apps/{app}/deploy: a zero-downtime rolling restart.
// ---------------------------------------------------------------------------
func newDeployCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
var project, environment string
var environment string
cmd := &cobra.Command{
Use: "deploy <container>",
Short: "Redeploy a container (rolling restart, zero-downtime)",
Long: "Drive a platform redeploy: a rolling restart of the container's k8s\n" +
"Deployment (re-pulls the image, recreates pods, zero downtime). Coordinates\n" +
"are exact — org (--org/config), project (--project), env (--env) and the\n" +
"container id (positional). This is the canonical PaaS-driven deploy.",
Use: "deploy <app>",
Short: "Redeploy an app (rolling restart, zero-downtime) — requires --env",
Long: "Drive a platform redeploy: a rolling restart of the app's k8s Deployment\n" +
"(re-pulls the declared image, recreates pods, zero downtime). The app is the\n" +
"operator App CR name; the org comes from your IAM identity. --env is REQUIRED\n" +
"(main|test|dev) — deploy never silently targets production. Restarting a shared\n" +
"platform service is a platform-operator action, so this needs a superadmin\n" +
"identity. A TAG change is still a git commit — this restarts what is declared.",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if strings.TrimSpace(environment) == "" {
return fmt.Errorf("--env is required (main|test|dev) — deploy will not default to production")
}
e := envOf()
org, err := e.requireOrg()
res, err := e.platform(gf).Redeploy(cmd.Context(), args[0], environment)
if err != nil {
return err
}
if project == "" || environment == "" {
return fmt.Errorf("--project and --env are required (the container's project/environment ids)")
}
container := args[0]
if err := e.platform(gf).Redeploy(cmd.Context(), org, project, environment, container); err != nil {
return err
}
fmt.Fprintf(cmd.OutOrStdout(), "redeployed %s (org=%s project=%s env=%s)\n", container, org, project, environment)
return nil
return e.emit(res, func(w io.Writer) {
fmt.Fprintf(w, "restarted %s (namespace=%s env=%s at %s)\n",
res.App, res.Namespace, dashIfEmpty(res.Env), res.RestartedAt)
})
},
}
cmd.Flags().StringVar(&project, "project", "", "project id")
cmd.Flags().StringVar(&environment, "env", "", "environment id")
cmd.Flags().StringVar(&environment, "env", "", "lifecycle env: main|test|dev (REQUIRED)")
return cmd
}
// ---------------------------------------------------------------------------
// clusters — dedicated DOKS cluster lifecycle.
// clusters — GET /v1/clusters: the org's compute fleet (Visor-managed + BYO),
// tenant-scoped server-side by the IAM identity.
// ---------------------------------------------------------------------------
func newClustersCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
cmd := &cobra.Command{
Use: "clusters",
Aliases: []string{"cluster"},
Short: "Provision/list/select dedicated DOKS clusters",
Short: "List the org's clusters (managed + BYO)",
}
list := &cobra.Command{
Use: "list",
Short: "List the org's dedicated clusters",
Short: "List the org's clusters",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
e := envOf()
org, err := e.requireOrg()
if err != nil {
return err
}
cs, err := e.platform(gf).Clusters(cmd.Context(), org)
cs, err := e.platform(gf).Clusters(cmd.Context())
if err != nil {
return err
}
return e.emit(cs, func(w io.Writer) {
tw := newTab(w)
fmt.Fprintln(tw, "NAME\tID\tREGION\tSTATUS\tPHASE\tACTIVE\tOPERATOR\tBASELINE")
fmt.Fprintln(tw, "NAME\tID\tREGION\tSTATUS\tKIND\tNODES\tSIZE\tGPUS")
for _, c := range cs {
fmt.Fprintf(tw, "%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n",
c.Name, c.DoksClusterID, c.Region, c.Status, c.Phase,
yesno(c.Active), yesno(c.OperatorInstalled), yesno(c.BaselineInstalled))
fmt.Fprintf(tw, "%s\t%s\t%s\t%s\t%s\t%d\t%s\t%s\n",
c.Name, dashIfEmpty(c.ID()), dashIfEmpty(c.Region), dashIfEmpty(c.Status),
dashIfEmpty(c.Kind), c.NodeCount, dashIfEmpty(c.NodeSize), gpuCell(c))
}
tw.Flush()
if len(cs) == 0 {
fmt.Fprintln(w, "(no dedicated clusters)")
fmt.Fprintln(w, "(no clusters)")
}
})
},
@@ -215,151 +200,51 @@ func newClustersCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
e := envOf()
org, err := e.requireOrg()
if err != nil {
return err
}
cs, err := e.platform(gf).Clusters(cmd.Context(), org)
cs, err := e.platform(gf).Clusters(cmd.Context())
if err != nil {
return err
}
for _, c := range cs {
if c.DoksClusterID == args[0] || c.Name == args[0] {
if c.ID() == args[0] || c.Name == args[0] {
return e.emit(c, func(w io.Writer) { printCluster(w, c) })
}
}
return fmt.Errorf("cluster %q not found in org %s", args[0], org)
return fmt.Errorf("cluster %q not found", args[0])
},
}
var region, nodeSize string
var ha bool
var nodeCount int
create := &cobra.Command{
Use: "create",
Short: "Provision a new dedicated DOKS cluster for the org",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
e := envOf()
org, err := e.requireOrg()
if err != nil {
return err
}
c, err := e.platform(gf).ProvisionCluster(cmd.Context(), org, ProvisionReq{
Region: region, HA: ha, NodeSize: nodeSize, NodeCount: nodeCount,
})
if err != nil {
return err
}
return e.emit(c, func(w io.Writer) {
fmt.Fprintf(w, "provisioning cluster %s (%s)\n", c.Name, c.DoksClusterID)
printCluster(w, *c)
})
},
}
create.Flags().StringVar(&region, "region", "", "DO region (default sfo3)")
create.Flags().BoolVar(&ha, "ha", false, "highly-available control plane")
create.Flags().StringVar(&nodeSize, "node-size", "", "node size slug (e.g. s-2vcpu-4gb)")
create.Flags().IntVar(&nodeCount, "node-count", 0, "node count")
var shared bool
selectCmd := &cobra.Command{
Use: "select <cluster-id>",
Short: "Set the org's active deploy target (or --shared to revert)",
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
e := envOf()
org, err := e.requireOrg()
if err != nil {
return err
}
var clusterID *string
switch {
case shared:
clusterID = nil
case len(args) == 1:
clusterID = &args[0]
default:
return fmt.Errorf("give a cluster id, or --shared to revert to the shared cluster")
}
t, err := e.platform(gf).SelectTarget(cmd.Context(), org, clusterID)
if err != nil {
return err
}
return e.emit(t, func(w io.Writer) { printTarget(w, t) })
},
}
selectCmd.Flags().BoolVar(&shared, "shared", false, "revert to the shared cluster")
installBaseline := &cobra.Command{
Use: "install-baseline <cluster-id>",
Short: "Install the hanzo-operator + per-tenant baseline on a cluster",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
e := envOf()
org, err := e.requireOrg()
if err != nil {
return err
}
if err := e.platform(gf).InstallBaseline(cmd.Context(), org, args[0]); err != nil {
return err
}
fmt.Fprintf(cmd.OutOrStdout(), "baseline install requested for %s\n", args[0])
return nil
},
}
target := &cobra.Command{
Use: "target",
Short: "Show the org's current resolved deploy target",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
e := envOf()
org, err := e.requireOrg()
if err != nil {
return err
}
t, err := e.platform(gf).Target(cmd.Context(), org)
if err != nil {
return err
}
return e.emit(t, func(w io.Writer) { printTarget(w, t) })
},
}
cmd.AddCommand(list, get, create, selectCmd, installBaseline, target)
cmd.AddCommand(list, get)
return cmd
}
// gpuCell renders the live GPU inventory of a cluster ("-" when none).
func gpuCell(c Cluster) string {
var parts []string
if c.NvidiaGPU > 0 {
parts = append(parts, fmt.Sprintf("%d nvidia", c.NvidiaGPU))
}
if c.AmdGPU > 0 {
parts = append(parts, fmt.Sprintf("%d amd", c.AmdGPU))
}
if len(parts) == 0 {
return "-"
}
return strings.Join(parts, "+")
}
func printCluster(w io.Writer, c Cluster) {
tw := newTab(w)
fmt.Fprintf(tw, "id:\t%s\n", c.DoksClusterID)
fmt.Fprintf(tw, "id:\t%s\n", dashIfEmpty(c.ID()))
fmt.Fprintf(tw, "name:\t%s\n", c.Name)
fmt.Fprintf(tw, "region:\t%s\n", c.Region)
fmt.Fprintf(tw, "status:\t%s\n", c.Status)
fmt.Fprintf(tw, "phase:\t%s\n", c.Phase)
fmt.Fprintf(tw, "active:\t%s\n", yesno(c.Active))
fmt.Fprintf(tw, "operatorInstalled:\t%s\n", yesno(c.OperatorInstalled))
fmt.Fprintf(tw, "baselineInstalled:\t%s\n", yesno(c.BaselineInstalled))
fmt.Fprintf(tw, "endpoint:\t%s\n", deref(c.Endpoint))
fmt.Fprintf(tw, "k8sVersion:\t%s\n", deref(c.K8sVersion))
fmt.Fprintf(tw, "created:\t%s\n", c.CreatedAt)
if c.BaselineError != nil && *c.BaselineError != "" {
fmt.Fprintf(tw, "baselineError:\t%s\n", *c.BaselineError)
}
tw.Flush()
}
func printTarget(w io.Writer, t *Target) {
tw := newTab(w)
kind := "shared"
if t.Dedicated {
kind = "dedicated"
}
fmt.Fprintf(tw, "cluster:\t%s\n", t.Cluster)
fmt.Fprintf(tw, "kind:\t%s\n", kind)
for ns, env := range t.Namespaces {
fmt.Fprintf(tw, "namespace:\t%s -> %s\n", ns, env)
fmt.Fprintf(tw, "region:\t%s\n", dashIfEmpty(c.Region))
fmt.Fprintf(tw, "status:\t%s\n", dashIfEmpty(c.Status))
fmt.Fprintf(tw, "kind:\t%s\n", dashIfEmpty(c.Kind))
fmt.Fprintf(tw, "nodeCount:\t%d\n", c.NodeCount)
fmt.Fprintf(tw, "nodeSize:\t%s\n", dashIfEmpty(c.NodeSize))
fmt.Fprintf(tw, "gpus:\t%s\n", gpuCell(c))
fmt.Fprintf(tw, "created:\t%s\n", dashIfEmpty(c.CreatedAt))
for _, np := range c.NodePools {
fmt.Fprintf(tw, "pool:\t%s (%s x%d, autoscale=%s)\n", np.Name, np.Size, np.Count, yesno(np.AutoScale))
}
tw.Flush()
}
@@ -387,6 +272,10 @@ func newBuildCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
if br.Repo == "" || br.SHA == "" || br.Image == "" {
return fmt.Errorf("--repo (or positional), --sha and --image are required")
}
// The platform build muscle clones an https git URL; accept the
// idiomatic `owner/name` shorthand and expand it to GitHub (the host
// for every hanzoai/luxfi/zooai repo). A full URL passes through.
br.Repo = normalizeRepoURL(br.Repo)
if br.OrganizationID == "" {
br.OrganizationID = e.Org // optional; server defaults to DEFAULT_BUILD_ORG_ID
}
@@ -419,32 +308,24 @@ func newBuildCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
return cmd
}
// ---------------------------------------------------------------------------
// k8s — deploy-target helpers.
// ---------------------------------------------------------------------------
func newK8sCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
cmd := &cobra.Command{
Use: "k8s",
Short: "Kubernetes deploy-target helpers",
// normalizeRepoURL expands the idiomatic `owner/name` shorthand to a full GitHub
// https URL (the platform build muscle clones https), and leaves an explicit URL
// (http/https/git/ssh scheme, or a scp-style git@host:owner/name) untouched. Only
// a bare single-segment `owner/name` — two path parts, no scheme, no host — is
// expanded; anything else is the caller's explicit choice and passes through.
func normalizeRepoURL(repo string) string {
r := strings.TrimSpace(repo)
if r == "" {
return r
}
target := &cobra.Command{
Use: "target",
Short: "Show the org's current resolved deploy target (cluster + namespaces)",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
e := envOf()
org, err := e.requireOrg()
if err != nil {
return err
}
t, err := e.platform(gf).Target(cmd.Context(), org)
if err != nil {
return err
}
return e.emit(t, func(w io.Writer) { printTarget(w, t) })
},
// Already a URL or scp-style remote → leave as-is.
if strings.Contains(r, "://") || strings.Contains(r, "@") {
return r
}
cmd.AddCommand(target)
return cmd
// Bare owner/name (exactly two non-empty segments, no host dot in the first).
parts := strings.Split(strings.Trim(r, "/"), "/")
if len(parts) == 2 && parts[0] != "" && parts[1] != "" && !strings.Contains(parts[0], ".") {
return "https://github.com/" + parts[0] + "/" + parts[1]
}
return r
}
+109 -32
View File
@@ -8,6 +8,24 @@ import (
"testing"
)
func TestNormalizeRepoURL(t *testing.T) {
cases := map[string]string{
"luxfi/wallet": "https://github.com/luxfi/wallet",
"hanzoai/cloud": "https://github.com/hanzoai/cloud",
"https://github.com/luxfi/wallet": "https://github.com/luxfi/wallet", // full URL untouched
"git@github.com:luxfi/wallet.git": "git@github.com:luxfi/wallet.git", // scp-style untouched
"https://gitlab.com/org/repo": "https://gitlab.com/org/repo", // non-github URL untouched
"owner/name/extra": "owner/name/extra", // not a bare owner/name
"single": "single", // not two segments
"": "", // empty
}
for in, want := range cases {
if got := normalizeRepoURL(in); got != want {
t.Errorf("normalizeRepoURL(%q) = %q, want %q", in, got, want)
}
}
}
// withPlatform points the CLI at an httptest platform via env (HANZO_PLATFORM_URL
// + HANZO_PLATFORM_TOKEN), the same resolution path the real binary uses.
func withPlatform(t *testing.T, h http.HandlerFunc) string {
@@ -20,11 +38,15 @@ func withPlatform(t *testing.T, h http.HandlerFunc) string {
return srv.URL
}
// apps list hits the LIVE board path /v1/paas/apps and renders the fleet table.
func TestAppsListCommandTable(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, _ *http.Request) {
withPlatform(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/paas/apps" {
t.Errorf("apps path = %s, want /v1/paas/apps", r.URL.Path)
}
_ = json.NewEncoder(w).Encode(AppsList{
Apps: []AppView{
{Org: "hanzoai", App: "iam", Env: "main", DeclaredTag: strptr("v1.2.3"), RunningTag: strptr("v1.2.3"), Health: strptr("green"), Drift: json.RawMessage(`{"severity":"ok"}`)},
{Org: "hanzoai", App: "iam", Env: "main", DeclaredTag: "v1.2.3", RunningTag: "v1.2.3", Health: "green", Drift: json.RawMessage(`{"severity":"ok"}`)},
},
Summary: struct {
Total int `json:"total"`
@@ -43,6 +65,20 @@ func TestAppsListCommandTable(t *testing.T) {
}
}
// apps list honors --env/--health/--drift as server query params (the board filters).
func TestAppsListCommandFilters(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
if q.Get("env") != "main" || q.Get("health") != "red" || q.Get("drift") != "1" {
t.Errorf("filters not forwarded: %s", r.URL.RawQuery)
}
_ = json.NewEncoder(w).Encode(AppsList{})
})
if _, err := runRoot(t, "", "apps", "list", "--env", "main", "--health", "red", "--drift"); err != nil {
t.Fatalf("apps list filters: %v", err)
}
}
func TestAppsListCommandJSON(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, _ *http.Request) {
_ = json.NewEncoder(w).Encode(AppsList{Apps: []AppView{{Org: "hanzoai", App: "iam", Env: "main"}}})
@@ -60,69 +96,112 @@ func TestAppsListCommandJSON(t *testing.T) {
}
}
// apps get hits /v1/paas/apps/{app}.
func TestAppsGetCommand(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/paas/apps/iam" {
t.Errorf("path = %s, want /v1/paas/apps/iam", r.URL.Path)
}
_ = json.NewEncoder(w).Encode(AppView{ID: "hanzoai/iam/main", Org: "hanzoai", App: "iam", Env: "main", DeclaredTag: "v1.2.3", Health: "green", Phase: "Running"})
})
out, err := runRoot(t, "", "apps", "get", "iam")
if err != nil {
t.Fatalf("apps get: %v", err)
}
for _, want := range []string{"hanzoai/iam/main", "Running", "v1.2.3"} {
if !strings.Contains(out, want) {
t.Fatalf("apps get missing %q in:\n%s", want, out)
}
}
}
// deploy hits /v1/paas/apps/{app}/deploy — a rolling restart, org from identity.
func TestDeployCommand(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/org/acme/project/p1/env/e1/container/app-x/redeploy" {
t.Errorf("redeploy path = %s", r.URL.Path)
if r.URL.Path != "/v1/paas/apps/app-x/deploy" || r.URL.Query().Get("env") != "main" {
t.Errorf("redeploy request = %s?%s", r.URL.Path, r.URL.RawQuery)
}
_ = json.NewEncoder(w).Encode(map[string]bool{"ok": true})
w.WriteHeader(202)
_ = json.NewEncoder(w).Encode(DeployResult{OK: true, App: "app-x", Namespace: "hanzo", Env: "main", RestartedAt: "2026-07-18T12:00:00Z"})
})
out, err := runRoot(t, "", "deploy", "app-x", "--org", "acme", "--project", "p1", "--env", "e1")
out, err := runRoot(t, "", "deploy", "app-x", "--env", "main")
if err != nil {
t.Fatalf("deploy: %v", err)
}
if !strings.Contains(out, "redeployed app-x") {
if !strings.Contains(out, "restarted app-x") || !strings.Contains(out, "namespace=hanzo") {
t.Fatalf("deploy output: %q", out)
}
}
func TestDeployRequiresProjectEnv(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(200) })
if _, err := runRoot(t, "", "deploy", "app-x", "--org", "acme"); err == nil {
t.Fatalf("deploy must require --project/--env")
// deploy REQUIRES --env — a bare deploy errors CLI-side, never silently prod.
func TestDeployRequiresEnv(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, _ *http.Request) {
t.Error("deploy without --env must not reach the server")
w.WriteHeader(202)
})
if _, err := runRoot(t, "", "deploy", "app-x"); err == nil {
t.Fatalf("deploy must require --env")
}
}
func TestDeployRequiresOrg(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(200) })
if _, err := runRoot(t, "", "deploy", "app-x", "--project", "p1", "--env", "e1"); err == nil {
t.Fatalf("deploy must require an org")
// deploy --env selects the lifecycle namespace via the ?env query param.
func TestDeployCommandEnv(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/paas/apps/chat/deploy" || r.URL.Query().Get("env") != "test" {
t.Errorf("deploy env request = %s?%s", r.URL.Path, r.URL.RawQuery)
}
w.WriteHeader(202)
_ = json.NewEncoder(w).Encode(DeployResult{OK: true, App: "chat", Namespace: "hanzo-testnet", Env: "test", RestartedAt: "2026-07-18T12:00:00Z"})
})
if _, err := runRoot(t, "", "deploy", "chat", "--env", "test"); err != nil {
t.Fatalf("deploy --env: %v", err)
}
}
// A non-ok deploy response is surfaced as an error.
func TestDeployNotOK(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, _ *http.Request) {
_ = json.NewEncoder(w).Encode(DeployResult{OK: false})
})
if _, err := runRoot(t, "", "deploy", "app-x", "--env", "main"); err == nil {
t.Fatalf("deploy must error when the server does not report ok")
}
}
// clusters list hits the LIVE /v1/clusters (org from identity, not the path).
func TestClustersListCommand(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/org/acme/cluster" {
t.Errorf("path = %s", r.URL.Path)
if r.URL.Path != "/v1/clusters" {
t.Errorf("path = %s, want /v1/clusters", r.URL.Path)
}
_ = json.NewEncoder(w).Encode(map[string]any{"clusters": []Cluster{
{DoksClusterID: "c1", Name: "hanzo-acme", Region: "sfo3", Status: "running", Phase: "ready", Active: true, OperatorInstalled: true, BaselineInstalled: true},
{DoksClusterID: "c1", Name: "hanzo-acme", Region: "sfo3", Status: "running", Kind: "managed", NodeCount: 3, NodeSize: "s-2vcpu-4gb", NvidiaGPU: 2},
}})
})
out, err := runRoot(t, "", "clusters", "list", "--org", "acme")
out, err := runRoot(t, "", "clusters", "list")
if err != nil {
t.Fatalf("clusters list: %v", err)
}
for _, want := range []string{"NAME", "hanzo-acme", "c1", "ready", "yes"} {
for _, want := range []string{"NAME", "hanzo-acme", "c1", "managed", "2 nvidia"} {
if !strings.Contains(out, want) {
t.Fatalf("clusters list missing %q in:\n%s", want, out)
}
}
}
func TestK8sTargetCommand(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/org/acme/cluster/select" {
t.Errorf("path = %s", r.URL.Path)
}
_ = json.NewEncoder(w).Encode(map[string]any{"target": Target{Cluster: "hanzo-k8s", Dedicated: false, Namespaces: map[string]string{"hanzo": "main"}}})
// clusters get filters the live list client-side by id or name.
func TestClustersGetCommand(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, _ *http.Request) {
_ = json.NewEncoder(w).Encode(map[string]any{"clusters": []Cluster{
{DoksClusterID: "c1", Name: "hanzo-acme", Region: "sfo3", Status: "running", Kind: "byo", NodeCount: 1},
}})
})
out, err := runRoot(t, "", "k8s", "target", "--org", "acme")
out, err := runRoot(t, "", "clusters", "get", "c1")
if err != nil {
t.Fatalf("k8s target: %v", err)
t.Fatalf("clusters get: %v", err)
}
if !strings.Contains(out, "hanzo-k8s") || !strings.Contains(out, "shared") {
t.Fatalf("k8s target output: %q", out)
if !strings.Contains(out, "hanzo-acme") || !strings.Contains(out, "byo") {
t.Fatalf("clusters get output: %q", out)
}
}
@@ -167,5 +246,3 @@ func TestConfigSetGetCommand(t *testing.T) {
t.Fatalf("config get = %q", out)
}
}
func strptr(s string) *string { return &s }
+220 -9
View File
@@ -54,6 +54,10 @@ const (
heartbeatEvery = 30 * time.Second
claimPoll = 2 * time.Second
claimLeaseSecs = 120
// renderWindow matches the dispatch cap (studio gpu_dispatch sets
// startToCloseTimeout 14400s). The old 10m local poll undercut it and
// marked live renders failed while they kept sampling (observed 8-70m).
renderWindow = 4 * time.Hour
// localComfyUI is the studio render backend the studio.render handler drives.
localComfyUI = "http://127.0.0.1:8188"
// defaultStudioUploadURL is where finished render outputs are POSTed so they
@@ -95,6 +99,7 @@ func newGPUCmd(envOf func() *Env, _ *globalFlags) *cobra.Command {
var engineEndpoint string
var registerProvider bool
var studioDir string
var studioURL string
connect := &cobra.Command{
Use: "connect",
Short: "Register this GPU and run the outbound worker loop",
@@ -107,6 +112,7 @@ func newGPUCmd(envOf func() *Env, _ *globalFlags) *cobra.Command {
engineEndpoint: engineEndpoint,
registerProvider: registerProvider,
studioDir: studioDir,
studioURL: studioURL,
}
if daemon {
return installDaemon(cmd, opts)
@@ -121,6 +127,7 @@ func newGPUCmd(envOf func() *Env, _ *globalFlags) *cobra.Command {
connect.Flags().StringVar(&engineEndpoint, "engine-endpoint", "", "public URL to advertise for gateway routing (defaults to --engine-url; a BYO node needs a reachable URL/tunnel)")
connect.Flags().BoolVar(&registerProvider, "register-provider", false, "auto-register the engine endpoint as an org model provider (POST /v1/add-provider)")
connect.Flags().StringVar(&studioDir, "studio-dir", os.Getenv("HANZO_STUDIO_DIR"), "local Hanzo Studio checkout; when set, connect launches and supervises the render backend on 127.0.0.1:8188")
connect.Flags().StringVar(&studioURL, "studio-url", firstNonEmpty(os.Getenv("HANZO_STUDIO_UPLOAD_URL"), defaultStudioUploadURL), "studio base URL the render mirror uploads finished images to (POST /v1/library/upload)")
status := &cobra.Command{
Use: "status",
@@ -370,6 +377,7 @@ type connectOpts struct {
engineEndpoint string // public URL to advertise (defaults to engineURL)
registerProvider bool // auto POST /v1/add-provider for the engine
studioDir string // local Studio checkout to launch + supervise on :8188
studioURL string // studio base the render mirror uploads finished images to
}
func runConnect(cmd *cobra.Command, env *Env, opts connectOpts) error {
@@ -422,6 +430,25 @@ func runConnect(cmd *cobra.Command, env *Env, opts connectOpts) error {
poll := time.NewTicker(claimPoll)
defer poll.Stop()
// Render mirror — independent of claims by design. It scans the local studio
// output tree every heartbeatEvery and uploads every image to the org's library
// (POST /v1/library/upload), so EVERY render lands in studio.hanzo.ai even when
// it was produced outside the job path — a graph hand-run on this node, or a
// render that finished after its activity was reaped (the stranded-late-render
// class). Active only when a studio checkout is named (there is local output to
// mirror); a nil channel case never fires when it is not.
w.studioUploadURL = firstNonEmpty(opts.studioURL, w.studioUploadURL)
mirrorBase := w.studioUploadURL
mirrorDir := ""
seen := map[string]int64{}
var mirC <-chan time.Time
if opts.studioDir != "" {
mirrorDir = filepath.Join(opts.studioDir, "output")
mir := time.NewTicker(heartbeatEvery)
defer mir.Stop()
mirC = mir.C
}
// Heartbeat once immediately so the machine reports online without waiting a
// full interval.
_ = w.heartbeat(ctx)
@@ -449,6 +476,8 @@ func runConnect(cmd *cobra.Command, env *Env, opts connectOpts) error {
if err := w.claimAndRun(ctx, out); err != nil {
fmt.Fprintf(cmd.ErrOrStderr(), "claim: %v\n", err)
}
case <-mirC:
w.mirrorRenders(ctx, out, mirrorDir, mirrorBase, seen)
}
}
}
@@ -542,7 +571,29 @@ func (w *worker) claimAndRun(ctx context.Context, out io.Writer) error {
fmt.Fprintf(out, " → failed: %s\n", cause)
return nil
}
// Keep BOTH the claimed activity and this machine's fleet presence alive while
// the handler runs. A render blocks this call for minutes (a cold GB10 reloads
// ~40GB before sampling); without heartbeats the studio.render activity hits its
// heartbeatTimeout AND the fleet presence (120s) goes stale, so the machine
// drops offline mid-render and the next dispatch sees no online GPU. A ticker in
// a child context heartbeats both every heartbeatEvery until the handler returns.
hbCtx, stopHB := context.WithCancel(ctx)
go func() {
t := time.NewTicker(heartbeatEvery)
defer t.Stop()
for {
select {
case <-hbCtx.Done():
return
case <-t.C:
_, _ = w.call(ctx, http.MethodPost, w.actPath(wf, run, "heartbeat"),
map[string]any{"identity": w.identity}, nil)
_ = w.heartbeat(ctx) // fleet presence — stays online through the render
}
}
}()
result, herr := h(ctx, act.Input)
stopHB()
if herr != nil {
_, _ = w.call(ctx, http.MethodPost, w.actPath(wf, run, "fail"), map[string]any{"cause": herr.Error(), "identity": w.identity}, nil)
fmt.Fprintf(out, " → failed: %v\n", herr)
@@ -710,7 +761,7 @@ func (e *Env) ensureToken(ctx context.Context) (string, error) {
nc.RefreshToken = e.creds.RefreshToken
}
*e.creds = *nc
_ = e.creds.Save()
_ = SaveActive(e.creds) // refresh the active identity in the store + mirror
}
// On refresh failure fall through: the current token may still be valid
// (clock skew) and the server is the authority.
@@ -773,6 +824,14 @@ func (w *worker) studioRenderHandler(ctx context.Context, input json.RawMessage)
return nil, fmt.Errorf("studio.render: input needs a `prompt` graph")
}
cl := &http.Client{Timeout: 60 * time.Second}
// The claim-to-submit window: hold off the supervisor's recycle, and wait out
// one if it is already mid-flight — a claimed job must never die on staging
// because the engine happened to be restarting.
staging.Add(1)
defer staging.Add(-1)
if err := waitEngine(ctx, cl); err != nil {
return nil, fmt.Errorf("studio.render: %w", err)
}
// Materialize any uploaded inputs (they live in orgs/{org}/input on the cloud
// pod, which this worker cannot read) into the LOCAL studio input dir via its
// own /upload/image, so LoadImage resolves them before we render.
@@ -801,7 +860,7 @@ func (w *worker) studioRenderHandler(ctx context.Context, input json.RawMessage)
return nil, fmt.Errorf("studio.render: no prompt_id in /prompt response")
}
// Poll history until the prompt shows up (completed).
deadline := time.Now().Add(10 * time.Minute)
deadline := time.Now().Add(renderWindow)
for time.Now().Before(deadline) {
select {
case <-ctx.Done():
@@ -825,6 +884,11 @@ func (w *worker) studioRenderHandler(ctx context.Context, input json.RawMessage)
if uerr != nil {
return nil, fmt.Errorf("studio.render: prompt %s rendered but gallery upload failed: %w", pr.PromptID, uerr)
}
// The engine leaks ~58GB per render; recycling after each completed
// render caps it at one render's worth. Boot (~20s) is noise next to
// 8-70m renders. Never recycle on the timeout path — the engine may
// still be sampling and the mirror rescues late finishes.
requestStudioRecycle()
return map[string]any{"promptId": pr.PromptID, "outputs": outputs, "gallery": gallery}, nil
}
}
@@ -939,6 +1003,120 @@ func (w *worker) postGalleryOutput(ctx context.Context, base, tok, org, name, su
return filepath.Join(out.Subfolder, out.Name), nil
}
// isImageFile reports whether name carries a render image extension the library accepts.
func isImageFile(name string) bool {
switch strings.ToLower(filepath.Ext(name)) {
case ".png", ".jpg", ".jpeg", ".webp":
return true
}
return false
}
// mirrorRenders scans dir (the local studio output tree) for image files new or
// changed since the last scan and POSTs each to base/v1/library/upload with the
// worker's bearer, tagged with this node's identity, so EVERY render lands in the
// org's studio library — including ones produced OUTSIDE the job path. seen (rel
// path -> size) skips unchanged files; the endpoint dedupes, so a re-scan after a
// restart is cheap and harmless. One log line per newly stored file; upload
// failures are summarized once per scan and retried next tick (no 5xx log spam).
func (w *worker) mirrorRenders(ctx context.Context, out io.Writer, dir, base string, seen map[string]int64) {
tok, err := w.env.ensureToken(ctx)
if err != nil {
return
}
base = strings.TrimRight(base, "/")
failed := 0
var firstErr error
_ = filepath.Walk(dir, func(p string, info os.FileInfo, werr error) error {
if werr != nil || info == nil || info.IsDir() || !isImageFile(p) {
return nil
}
// Hidden files and AppleDouble forks (`._*`, `.DS_Store`) ride along with
// mac scp and are not renders — `._foo.png` passes the extension check
// but is a 4KB resource fork that poisons the library.
if strings.HasPrefix(filepath.Base(p), ".") {
return nil
}
rel, rerr := filepath.Rel(dir, p)
if rerr != nil {
return nil
}
rel = filepath.ToSlash(rel)
if seen[rel] == info.Size() {
return nil
}
data, derr := os.ReadFile(p)
if derr != nil || len(data) == 0 {
return nil
}
sub, name := "", rel
if i := strings.LastIndex(rel, "/"); i >= 0 {
sub, name = rel[:i], rel[i+1:]
}
existed, perr := w.postLibraryUpload(ctx, base, tok, sub, name, data)
if perr != nil {
failed++
if firstErr == nil {
firstErr = perr
}
return nil
}
seen[rel] = info.Size()
if !existed {
fmt.Fprintf(out, "mirrored %s (%d bytes) -> %s\n", rel, len(data), base)
}
return nil
})
if failed > 0 {
fmt.Fprintf(out, "mirror: %d file(s) failed to upload, will retry: %v\n", failed, firstErr)
}
}
// postLibraryUpload multipart-POSTs one image to base/v1/library/upload with the
// worker's IAM bearer, landing it in the org's library (orgs/{org}/output). The
// file's subfolder rides as ?subpath and this node's identity as ?node so the
// render is filterable by its source in Queue & History. Returns whether the
// endpoint already had a byte-identical copy (dedup).
func (w *worker) postLibraryUpload(ctx context.Context, base, tok, sub, name string, data []byte) (bool, error) {
var buf bytes.Buffer
mw := multipart.NewWriter(&buf)
part, err := mw.CreateFormFile("image", name)
if err != nil {
return false, err
}
if _, err := part.Write(data); err != nil {
return false, err
}
if err := mw.Close(); err != nil {
return false, err
}
q := url.Values{"node": {w.identity}}
if sub != "" {
q.Set("subpath", sub)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, base+"/v1/library/upload?"+q.Encode(), &buf)
if err != nil {
return false, err
}
req.Header.Set("Authorization", "Bearer "+tok)
req.Header.Set("Content-Type", mw.FormDataContentType())
req.Header.Set("Accept", "application/json")
resp, err := w.http.Do(req)
if err != nil {
return false, err
}
defer resp.Body.Close()
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if resp.StatusCode/100 != 2 {
return false, fmt.Errorf("POST /v1/library/upload HTTP %d: %s", resp.StatusCode, serverMessage(raw))
}
var out struct {
Existed bool `json:"existed"`
}
_ = json.Unmarshal(raw, &out)
return out.Existed, nil
}
// inputImage is one uploaded input shipped with the job: a base64 blob plus the
// input-dir-relative location it must occupy on this worker so LoadImage finds it.
type inputImage struct {
@@ -993,14 +1171,47 @@ func (w *worker) materializeInputs(ctx context.Context, cl *http.Client, inputs
return nil
}
// collectOutputs pulls the output image/file names out of a ComfyUI history entry.
// waitEngine blocks until the local engine answers its /queue — up to 90s, which
// outlasts any supervisor recycle (engine restart is seconds, model reload longer).
func waitEngine(ctx context.Context, cl *http.Client) error {
deadline := time.Now().Add(90 * time.Second)
for {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, localComfyUI+"/queue", nil)
if err != nil {
return err
}
resp, err := cl.Do(req)
if err == nil {
_, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 1<<10))
resp.Body.Close()
if resp.StatusCode/100 == 2 {
return nil
}
}
if time.Now().After(deadline) {
return fmt.Errorf("engine not up: %v", err)
}
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(3 * time.Second):
}
}
}
// collectOutputs pulls the output file names out of a ComfyUI history entry. Savers
// publish under different keys — SaveImage/SaveVideo under "images", SaveGLB under
// "3d" — so every saver's outputs are gathered, not just images (a 3D mesh would
// otherwise never travel back to the library).
func collectOutputs(entry json.RawMessage) []string {
type namedFile struct {
Filename string `json:"filename"`
Subfolder string `json:"subfolder"`
}
var e struct {
Outputs map[string]struct {
Images []struct {
Filename string `json:"filename"`
Subfolder string `json:"subfolder"`
} `json:"images"`
Images []namedFile `json:"images"`
ThreeD []namedFile `json:"3d"`
} `json:"outputs"`
}
if err := json.Unmarshal(entry, &e); err != nil {
@@ -1008,8 +1219,8 @@ func collectOutputs(entry json.RawMessage) []string {
}
var files []string
for _, node := range e.Outputs {
for _, img := range node.Images {
files = append(files, filepath.Join(img.Subfolder, img.Filename))
for _, f := range append(append([]namedFile{}, node.Images...), node.ThreeD...) {
files = append(files, filepath.Join(f.Subfolder, f.Filename))
}
}
return files
+92
View File
@@ -0,0 +1,92 @@
package cli
import (
"bytes"
"context"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"time"
)
// TestMirrorRenders verifies the render mirror: it scans the local studio output
// tree and POSTs every image (new or changed) to /v1/library/upload with the node's
// identity + subfolder + bearer, skips unchanged files across scans, and re-uploads
// a changed file. This is the path that lands EVERY render in studio.hanzo.ai even
// when it was produced outside the job/claim path.
func TestMirrorRenders(t *testing.T) {
t.Setenv("HANZO_TOKEN", "test-bearer")
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "renders"), 0o755); err != nil {
t.Fatal(err)
}
write := func(rel, body string) {
if err := os.WriteFile(filepath.Join(dir, filepath.FromSlash(rel)), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
write("renders/a.png", "\x89PNG-a")
write("top.jpg", "jpg-top")
write("notes.txt", "not an image")
type up struct{ name, node, subpath, auth string }
var got []up
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/library/upload" {
http.Error(w, "not found", http.StatusNotFound)
return
}
_ = r.ParseMultipartForm(1 << 20)
name := ""
if r.MultipartForm != nil {
for _, fh := range r.MultipartForm.File["image"] {
name = fh.Filename
}
}
got = append(got, up{
name: name,
node: r.URL.Query().Get("node"),
subpath: r.URL.Query().Get("subpath"),
auth: r.Header.Get("Authorization"),
})
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"ok":true,"path":"x"}`))
}))
defer srv.Close()
w := &worker{env: &Env{}, http: &http.Client{Timeout: 10 * time.Second}, identity: "spark"}
seen := map[string]int64{}
var buf bytes.Buffer
w.mirrorRenders(context.Background(), &buf, dir, srv.URL, seen)
if len(got) != 2 {
t.Fatalf("uploaded %d files, want 2 images (the .txt is skipped): %+v", len(got), got)
}
var a *up
for i := range got {
if got[i].name == "a.png" {
a = &got[i]
}
}
if a == nil || a.subpath != "renders" || a.node != "spark" || a.auth != "Bearer test-bearer" {
t.Fatalf("renders/a.png upload = %+v, want subpath=renders node=spark bearer set", a)
}
// A second scan re-uploads nothing (seen matches every size).
got = nil
w.mirrorRenders(context.Background(), &buf, dir, srv.URL, seen)
if len(got) != 0 {
t.Fatalf("second scan uploaded %d files, want 0 (all unchanged): %+v", len(got), got)
}
// A changed file is re-uploaded on the next scan.
write("renders/a.png", "\x89PNG-a-grew")
got = nil
w.mirrorRenders(context.Background(), &buf, dir, srv.URL, seen)
if len(got) != 1 || got[0].name != "a.png" {
t.Fatalf("after change, uploaded %+v, want just renders/a.png", got)
}
}
+110 -132
View File
@@ -12,11 +12,13 @@ import (
"time"
)
// Platform is a thin client over the platform.hanzo.ai /v1 control plane. That
// surface is machine-to-machine (service-token, "No OIDC" — it cannot validate
// IAM user tokens), so the token here is the platform service token, resolved
// from flag/env/credential store by the caller; the build endpoint takes its
// own token per call.
// Platform is a thin client over the LIVE Hanzo Cloud control plane
// (platform.hanzo.ai / api.hanzo.ai → svc `cloud`, the Go binary). Every route it
// calls is served by that one binary and authorized off ONE IAM identity: after
// `hanzo login` the CLI sends the IAM access token as the bearer, and the cloud's
// identity middleware (SanitizeIdentity) validates the JWT and org-scopes the
// caller — no separate platform/service token. A purpose-minted machine token
// still works (flag > env > credential store > IAM login) for automation.
type Platform struct {
baseURL string
token string
@@ -45,8 +47,8 @@ func (e *apiError) Error() string {
msg = http.StatusText(e.status)
}
hint := ""
if e.status == http.StatusUnauthorized {
hint = " (set the platform service token: --platform-token, HANZO_PLATFORM_TOKEN, or `hanzo login --platform-token`)"
if e.status == http.StatusUnauthorized || e.status == http.StatusForbidden {
hint = " (run `hanzo login` — your IAM identity authorizes the platform; admin ops need an org-admin or superadmin identity)"
}
return fmt.Sprintf("platform %s: HTTP %d: %s%s", e.path, e.status, msg, hint)
}
@@ -55,7 +57,7 @@ func (e *apiError) Error() string {
// into out (when non-nil) and mapping a non-2xx into an *apiError.
func (p *Platform) do(ctx context.Context, method, path, token string, body, out any) error {
if token == "" {
return fmt.Errorf("no platform token: pass --platform-token, set HANZO_PLATFORM_TOKEN, or run `hanzo login --platform-token <tok>`")
return fmt.Errorf("not authenticated: run `hanzo login` (an IAM login now authorizes the platform; a --platform-token / HANZO_PLATFORM_TOKEN still works for machine automation)")
}
var rdr io.Reader
if body != nil {
@@ -94,8 +96,8 @@ func (p *Platform) do(ctx context.Context, method, path, token string, body, out
return nil
}
// serverMessage pulls the `{ "message": … }` field platform errors use, falling
// back to the raw (truncated) body.
// serverMessage pulls the `{ "message": … }` / `{ "error": … }` field the cloud's
// errors use, falling back to the raw (truncated) body.
func serverMessage(raw []byte) string {
var e struct {
Message string `json:"message"`
@@ -117,33 +119,35 @@ func serverMessage(raw []byte) string {
}
// ---------------------------------------------------------------------------
// Apps board — GET /v1/apps, GET /v1/apps/{id}, POST /v1/apps/sync.
// Apps board — GET /v1/paas/apps, GET /v1/paas/apps/{app}. The live Go cloud's
// fleet drift board (clients/paas): the operator App CRs across the platform
// namespaces, declared/running/latest tags + health + the drift verdict. It is
// org-confined server-side (a SuperAdmin sees the fleet; an OrgAdmin only its own
// org), so the CLI sends NO org filter — identity scopes the view.
// ---------------------------------------------------------------------------
// AppView mirrors the platform apps-lifecycle DTO. Nullable columns are *string
// so JSON null round-trips; Drift is kept raw so --json is byte-faithful and
// the drift schema can evolve without a client bump.
// AppView mirrors clients/paas.AppView (the LIVE board DTO). Tags/health are plain
// strings ("" == unknown, rendered "-"); Drift is kept raw so --json is
// byte-faithful and the drift schema can evolve without a client bump.
type AppView struct {
ID string `json:"id"`
Org string `json:"org"`
App string `json:"app"`
Env string `json:"env"`
Repo string `json:"repo"`
Registry string `json:"registry"`
DeclaredTag *string `json:"declaredTag"`
RunningTag *string `json:"runningTag"`
LatestTag *string `json:"latestTag"`
ReleaseURL *string `json:"releaseUrl"`
ReleaseAssets int `json:"releaseAssets"`
Health *string `json:"health"`
Cluster *string `json:"cluster"`
Namespace *string `json:"namespace"`
LastObserved *string `json:"lastObserved"`
UpdatedAt string `json:"updatedAt"`
Drift json.RawMessage `json:"drift"`
ID string `json:"id"` // <org>/<app>/<env>, e.g. hanzoai/iam/main
Org string `json:"org"` // image namespace, e.g. hanzoai
App string `json:"app"`
Env string `json:"env"`
Repo string `json:"repo"`
Registry string `json:"registry"`
DeclaredTag string `json:"declaredTag"`
RunningTag string `json:"runningTag"`
LatestTag string `json:"latestTag"`
Health string `json:"health"`
Phase string `json:"phase"`
Cluster string `json:"cluster"`
Namespace string `json:"namespace"`
Endpoints []string `json:"endpoints"`
Drift json.RawMessage `json:"drift"`
}
// AppsList is the /v1/apps envelope: ordered rows + a drift summary.
// AppsList is the /v1/paas/apps envelope: ordered rows + a drift summary.
type AppsList struct {
Apps []AppView `json:"apps"`
Summary struct {
@@ -152,9 +156,10 @@ type AppsList struct {
} `json:"summary"`
}
// AppsQuery are the optional /v1/apps filters.
// AppsQuery are the optional /v1/paas/apps filters (server-honored). Env/Health/
// Drift narrow the board; there is deliberately no org filter — the board is
// confined to the caller's org by the validated identity, never a client value.
type AppsQuery struct {
Org string
Env string
Health string
Drift bool
@@ -162,9 +167,6 @@ type AppsQuery struct {
func (p *Platform) Apps(ctx context.Context, q AppsQuery) (*AppsList, error) {
v := url.Values{}
if q.Org != "" {
v.Set("org", q.Org)
}
if q.Env != "" {
v.Set("env", q.Env)
}
@@ -174,7 +176,7 @@ func (p *Platform) Apps(ctx context.Context, q AppsQuery) (*AppsList, error) {
if q.Drift {
v.Set("drift", "1")
}
path := "/v1/apps"
path := "/v1/paas/apps"
if len(v) > 0 {
path += "?" + v.Encode()
}
@@ -182,17 +184,11 @@ func (p *Platform) Apps(ctx context.Context, q AppsQuery) (*AppsList, error) {
return out, p.do(ctx, http.MethodGet, path, p.token, nil, out)
}
func (p *Platform) App(ctx context.Context, id, org string) (*AppView, error) {
path := "/v1/apps/" + id
if org != "" {
path += "?org=" + url.QueryEscape(org)
}
// App gets one app row by its <app> CR name (production by default; the server
// scans the caller's authorized namespaces main→test→dev).
func (p *Platform) App(ctx context.Context, app string) (*AppView, error) {
out := &AppView{}
return out, p.do(ctx, http.MethodGet, path, p.token, nil, out)
}
func (p *Platform) SyncApps(ctx context.Context) error {
return p.do(ctx, http.MethodPost, "/v1/apps/sync", p.token, nil, nil)
return out, p.do(ctx, http.MethodGet, "/v1/paas/apps/"+url.PathEscape(app), p.token, nil, out)
}
// driftSeverity extracts the severity string from the raw drift object.
@@ -207,110 +203,92 @@ func driftSeverity(raw json.RawMessage) string {
}
// ---------------------------------------------------------------------------
// Dedicated clusters — /v1/org/{org}/cluster[ /select | /{id}/install-baseline ].
// Clusters — GET /v1/clusters. The live Go cloud's compute fleet (clients/visor):
// Visor-managed node pools + the org's BYO clusters, tenant-scoped server-side by
// the validated org (?owner is the caller's IAM org). No org in the path.
// ---------------------------------------------------------------------------
// Cluster mirrors a doks_cluster record. `status` is DigitalOcean state; `phase`
// is the platform provisioning lifecycle — orthogonal (a DO-running cluster is
// not a usable target until phase=ready).
// NodePool mirrors clients/visor.nodePoolView.
type NodePool struct {
PoolID string `json:"poolId"`
Name string `json:"name"`
Size string `json:"size"`
Count int `json:"count"`
MinNodes int `json:"minNodes"`
MaxNodes int `json:"maxNodes"`
AutoScale bool `json:"autoScale"`
}
// Cluster mirrors clients/visor.clusterView — the LIVE cluster DTO. `kind` is
// "managed" (Visor-provisioned) or "byo" (attached kubeconfig).
type Cluster struct {
DoksClusterID string `json:"doksClusterId"`
Name string `json:"name"`
DoClusterID *string `json:"doClusterId"`
Region string `json:"region"`
Status string `json:"status"`
Endpoint *string `json:"endpoint"`
K8sVersion *string `json:"k8sVersion"`
HA bool `json:"ha"`
Phase string `json:"phase"`
OperatorInstalled bool `json:"operatorInstalled"`
BaselineInstalled bool `json:"baselineInstalled"`
Active bool `json:"active"`
BaselineError *string `json:"baselineError"`
OrganizationID string `json:"organizationId"`
CreatedAt string `json:"createdAt"`
Tags []string `json:"tags"`
MaintenancePolicy json.RawMessage `json:"maintenancePolicy,omitempty"`
DoksClusterID string `json:"doksClusterId"`
DoClusterID string `json:"doClusterId"`
Name string `json:"name"`
Region string `json:"region"`
Status string `json:"status"`
NodePools []NodePool `json:"nodePools"`
NodeSize string `json:"nodeSize"`
NodeCount int `json:"nodeCount"`
CreatedAt string `json:"createdAt"`
Kind string `json:"kind"`
NvidiaGPU int `json:"nvidiaGpu"`
AmdGPU int `json:"amdGpu"`
}
// ProvisionReq is the dedicated-cluster provisioning body (org forced by path).
type ProvisionReq struct {
Region string `json:"region,omitempty"`
HA bool `json:"ha,omitempty"`
NodeSize string `json:"nodeSize,omitempty"`
NodeCount int `json:"nodeCount,omitempty"`
// ID is the stable cluster identifier for display/lookup: the DOKS id when managed,
// else the name (a BYO cluster keys on its attached name).
func (c Cluster) ID() string {
if c.DoksClusterID != "" {
return c.DoksClusterID
}
return c.Name
}
// Target is the redacted ClusterTargetView — the kubeconfig is never present.
type Target struct {
Cluster string `json:"cluster"`
Namespaces map[string]string `json:"namespaces"`
Dedicated bool `json:"dedicated"`
}
func (p *Platform) Clusters(ctx context.Context, org string) ([]Cluster, error) {
func (p *Platform) Clusters(ctx context.Context) ([]Cluster, error) {
var out struct {
Clusters []Cluster `json:"clusters"`
}
err := p.do(ctx, http.MethodGet, "/v1/org/"+url.PathEscape(org)+"/cluster", p.token, nil, &out)
err := p.do(ctx, http.MethodGet, "/v1/clusters", p.token, nil, &out)
return out.Clusters, err
}
func (p *Platform) ProvisionCluster(ctx context.Context, org string, req ProvisionReq) (*Cluster, error) {
var out struct {
Cluster Cluster `json:"cluster"`
}
err := p.do(ctx, http.MethodPost, "/v1/org/"+url.PathEscape(org)+"/cluster", p.token, req, &out)
return &out.Cluster, err
}
func (p *Platform) Target(ctx context.Context, org string) (*Target, error) {
var out struct {
Target Target `json:"target"`
}
err := p.do(ctx, http.MethodGet, "/v1/org/"+url.PathEscape(org)+"/cluster/select", p.token, nil, &out)
return &out.Target, err
}
// SelectTarget activates a dedicated cluster as the org's deploy target, or
// reverts to the shared cluster when clusterID is nil.
func (p *Platform) SelectTarget(ctx context.Context, org string, clusterID *string) (*Target, error) {
var out struct {
Target Target `json:"target"`
}
body := map[string]any{"doksClusterId": clusterID}
err := p.do(ctx, http.MethodPost, "/v1/org/"+url.PathEscape(org)+"/cluster/select", p.token, body, &out)
return &out.Target, err
}
func (p *Platform) InstallBaseline(ctx context.Context, org, clusterID string) error {
path := "/v1/org/" + url.PathEscape(org) + "/cluster/" + url.PathEscape(clusterID) + "/install-baseline"
return p.do(ctx, http.MethodPost, path, p.token, nil, nil)
}
// ---------------------------------------------------------------------------
// Deploy — POST …/container/{id}/redeploy (rolling restart, zero-downtime).
// Deploy — POST /v1/paas/apps/{app}/deploy: a zero-downtime ROLLING RESTART of the
// app's Deployment (re-pulls the declared image, recreates pods). Org-confined
// server-side; an optional env selects the lifecycle namespace (main|test|dev).
// ---------------------------------------------------------------------------
// Redeploy triggers a rolling restart of the container's k8s Deployment. The
// coordinates are exact (the platform validates org+project+env+container scope).
func (p *Platform) Redeploy(ctx context.Context, org, project, env, container string) error {
path := fmt.Sprintf("/v1/org/%s/project/%s/env/%s/container/%s/redeploy",
url.PathEscape(org), url.PathEscape(project), url.PathEscape(env), url.PathEscape(container))
var out struct {
OK bool `json:"ok"`
// Redeploy triggers a rolling restart of the named app. env is optional
// (main|test|dev); empty targets production (the first match, main→test→dev).
func (p *Platform) Redeploy(ctx context.Context, app, env string) (*DeployResult, error) {
path := "/v1/paas/apps/" + url.PathEscape(app) + "/deploy"
if env != "" {
path += "?env=" + url.QueryEscape(env)
}
if err := p.do(ctx, http.MethodPost, path, p.token, nil, &out); err != nil {
return err
out := &DeployResult{}
if err := p.do(ctx, http.MethodPost, path, p.token, nil, out); err != nil {
return nil, err
}
if !out.OK {
return fmt.Errorf("redeploy did not report ok")
return nil, fmt.Errorf("redeploy did not report ok")
}
return nil
return out, nil
}
// DeployResult is the /deploy acceptance (202): the restarted app + its namespace.
type DeployResult struct {
OK bool `json:"ok"`
App string `json:"app"`
Namespace string `json:"namespace"`
Env string `json:"env"`
RestartedAt string `json:"restartedAt"`
}
// ---------------------------------------------------------------------------
// Build — POST /v1/runner (platform-native CI, no GitHub builders).
// Build — POST /v1/runner (platform-native CI, no GitHub builders). Authorized off
// the IAM login exactly like the surfaces above (or a dedicated build token for
// machine automation). Unchanged wire contract.
// ---------------------------------------------------------------------------
// BuildReq is the direct-enqueue body. Repo/SHA/Image are required.
@@ -337,11 +315,11 @@ type BuildJob struct {
Target string `json:"target"`
}
// EnqueueBuild enqueues a native build. It authenticates with the dedicated
// build-callback token, not the service token.
// EnqueueBuild enqueues a native build. buildToken is resolved by the caller (IAM
// login is the final fallback; a dedicated build token wins when present).
func (p *Platform) EnqueueBuild(ctx context.Context, req BuildReq, buildToken string) (*BuildJob, error) {
if buildToken == "" {
return nil, fmt.Errorf("no build token: set HANZO_BUILD_TOKEN / PLATFORM_BUILD_CALLBACK_TOKEN or `hanzo login --build-token <tok>`")
return nil, fmt.Errorf("not authenticated: run `hanzo login` (an IAM login now authorizes builds; HANZO_BUILD_TOKEN / --build-token still works for machine automation)")
}
out := &BuildJob{}
return out, p.do(ctx, http.MethodPost, "/v1/runner", buildToken, req, out)
+47 -91
View File
@@ -18,17 +18,22 @@ func platformStub(t *testing.T, token string, h http.HandlerFunc) (*Platform, fu
return newPlatform(srv.URL, token), srv.Close
}
// Apps hits the LIVE board /v1/paas/apps with the IAM bearer; it sends NO org
// filter (the board is org-confined server-side by the validated identity).
func TestPlatformAuthHeaderAndApps(t *testing.T) {
p, done := platformStub(t, "svc-tok", func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("Authorization"); got != "Bearer svc-tok" {
t.Errorf("auth header = %q", got)
}
if r.URL.Path != "/v1/apps" {
t.Errorf("path = %s", r.URL.Path)
if r.URL.Path != "/v1/paas/apps" {
t.Errorf("path = %s, want /v1/paas/apps", r.URL.Path)
}
if r.URL.Query().Get("env") != "main" || r.URL.Query().Get("drift") != "1" {
t.Errorf("query = %s", r.URL.RawQuery)
}
if r.URL.Query().Has("org") {
t.Errorf("client must NOT send an org filter (identity confines the board): %s", r.URL.RawQuery)
}
_ = json.NewEncoder(w).Encode(AppsList{
Apps: []AppView{{ID: "hanzoai/iam/main", Org: "hanzoai", App: "iam", Env: "main", Drift: json.RawMessage(`{"severity":"red"}`)}},
})
@@ -47,126 +52,75 @@ func TestPlatformAuthHeaderAndApps(t *testing.T) {
}
}
// App hits /v1/paas/apps/{app}; no org query (identity scopes it).
func TestPlatformApp(t *testing.T) {
p, done := platformStub(t, "t", func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/apps/hanzoai/iam/main" {
t.Errorf("path = %s", r.URL.Path)
if r.URL.Path != "/v1/paas/apps/iam" {
t.Errorf("path = %s, want /v1/paas/apps/iam", r.URL.Path)
}
if r.URL.Query().Get("org") != "hanzoai" {
t.Errorf("org query = %s", r.URL.RawQuery)
if r.URL.RawQuery != "" {
t.Errorf("app get must carry no query, got %s", r.URL.RawQuery)
}
_ = json.NewEncoder(w).Encode(AppView{ID: "hanzoai/iam/main", App: "iam"})
_ = json.NewEncoder(w).Encode(AppView{ID: "hanzoai/iam/main", App: "iam", Phase: "Running"})
})
defer done()
a, err := p.App(context.Background(), "hanzoai/iam/main", "hanzoai")
a, err := p.App(context.Background(), "iam")
if err != nil || a.App != "iam" {
t.Fatalf("App: %v %+v", err, a)
}
}
func TestPlatformSyncApps(t *testing.T) {
// Clusters hits the LIVE /v1/clusters (org from identity, not the path).
func TestPlatformClusters(t *testing.T) {
p, done := platformStub(t, "t", func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost || r.URL.Path != "/v1/apps/sync" {
t.Errorf("sync = %s %s", r.Method, r.URL.Path)
}
w.WriteHeader(200)
})
defer done()
if err := p.SyncApps(context.Background()); err != nil {
t.Fatalf("SyncApps: %v", err)
}
}
func TestPlatformClustersAndProvision(t *testing.T) {
p, done := platformStub(t, "t", func(w http.ResponseWriter, r *http.Request) {
switch {
case r.Method == http.MethodGet && r.URL.Path == "/v1/org/acme/cluster":
_ = json.NewEncoder(w).Encode(map[string]any{"clusters": []Cluster{{DoksClusterID: "c1", Name: "hanzo-acme", Region: "sfo3", Status: "running", Phase: "ready", Active: true}}})
case r.Method == http.MethodPost && r.URL.Path == "/v1/org/acme/cluster":
body, _ := io.ReadAll(r.Body)
var req ProvisionReq
_ = json.Unmarshal(body, &req)
if req.Region != "sfo3" || !req.HA {
t.Errorf("provision body = %+v", req)
}
w.WriteHeader(201)
_ = json.NewEncoder(w).Encode(map[string]any{"cluster": Cluster{DoksClusterID: "c2", Name: "new", Phase: "requested"}})
default:
t.Errorf("unexpected %s %s", r.Method, r.URL.Path)
if r.Method != http.MethodGet || r.URL.Path != "/v1/clusters" {
t.Errorf("clusters = %s %s, want GET /v1/clusters", r.Method, r.URL.Path)
}
_ = json.NewEncoder(w).Encode(map[string]any{"clusters": []Cluster{
{DoksClusterID: "c1", Name: "hanzo-acme", Region: "sfo3", Status: "running", Kind: "managed", NodeCount: 3},
}})
})
defer done()
cs, err := p.Clusters(context.Background(), "acme")
if err != nil || len(cs) != 1 || cs[0].DoksClusterID != "c1" {
cs, err := p.Clusters(context.Background())
if err != nil || len(cs) != 1 || cs[0].ID() != "c1" || cs[0].Kind != "managed" {
t.Fatalf("Clusters: %v %+v", err, cs)
}
c, err := p.ProvisionCluster(context.Background(), "acme", ProvisionReq{Region: "sfo3", HA: true})
if err != nil || c.DoksClusterID != "c2" {
t.Fatalf("ProvisionCluster: %v %+v", err, c)
}
}
func TestPlatformTargetAndSelect(t *testing.T) {
p, done := platformStub(t, "t", func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/org/acme/cluster/select" {
t.Errorf("path = %s", r.URL.Path)
}
if r.Method == http.MethodPost {
body, _ := io.ReadAll(r.Body)
var m map[string]any
_ = json.Unmarshal(body, &m)
if m["doksClusterId"] != "c1" {
t.Errorf("select body = %v", m)
}
}
_ = json.NewEncoder(w).Encode(map[string]any{"target": Target{Cluster: "hanzo-acme", Dedicated: true, Namespaces: map[string]string{"acme": "main"}}})
})
defer done()
tg, err := p.Target(context.Background(), "acme")
if err != nil || tg.Cluster != "hanzo-acme" || !tg.Dedicated {
t.Fatalf("Target: %v %+v", err, tg)
}
id := "c1"
if _, err := p.SelectTarget(context.Background(), "acme", &id); err != nil {
t.Fatalf("SelectTarget: %v", err)
}
}
func TestPlatformInstallBaseline(t *testing.T) {
p, done := platformStub(t, "t", func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost || r.URL.Path != "/v1/org/acme/cluster/c1/install-baseline" {
t.Errorf("install-baseline = %s %s", r.Method, r.URL.Path)
}
w.WriteHeader(200)
})
defer done()
if err := p.InstallBaseline(context.Background(), "acme", "c1"); err != nil {
t.Fatalf("InstallBaseline: %v", err)
// A BYO cluster with no DOKS id keys on its name via ID().
func TestClusterIDFallsBackToName(t *testing.T) {
c := Cluster{Name: "byo-1", Kind: "byo"}
if c.ID() != "byo-1" {
t.Fatalf("ID() = %q, want byo-1", c.ID())
}
}
// Redeploy hits /v1/paas/apps/{app}/deploy (rolling restart), org from identity.
func TestPlatformRedeploy(t *testing.T) {
p, done := platformStub(t, "t", func(w http.ResponseWriter, r *http.Request) {
want := "/v1/org/acme/project/p1/env/e1/container/app-x/redeploy"
if r.Method != http.MethodPost || r.URL.Path != want {
t.Errorf("redeploy path = %s %s", r.Method, r.URL.Path)
if r.Method != http.MethodPost || r.URL.Path != "/v1/paas/apps/app-x/deploy" {
t.Errorf("redeploy = %s %s, want POST /v1/paas/apps/app-x/deploy", r.Method, r.URL.Path)
}
_ = json.NewEncoder(w).Encode(map[string]bool{"ok": true})
if r.URL.Query().Get("env") != "test" {
t.Errorf("env query = %s", r.URL.RawQuery)
}
w.WriteHeader(202)
_ = json.NewEncoder(w).Encode(DeployResult{OK: true, App: "app-x", Namespace: "hanzo-testnet", Env: "test", RestartedAt: "2026-07-18T00:00:00Z"})
})
defer done()
if err := p.Redeploy(context.Background(), "acme", "p1", "e1", "app-x"); err != nil {
t.Fatalf("Redeploy: %v", err)
res, err := p.Redeploy(context.Background(), "app-x", "test")
if err != nil || res.Namespace != "hanzo-testnet" {
t.Fatalf("Redeploy: %v %+v", err, res)
}
}
func TestPlatformRedeployNotOK(t *testing.T) {
p, done := platformStub(t, "t", func(w http.ResponseWriter, _ *http.Request) {
_ = json.NewEncoder(w).Encode(map[string]bool{"ok": false})
_ = json.NewEncoder(w).Encode(DeployResult{OK: false})
})
defer done()
if err := p.Redeploy(context.Background(), "o", "p", "e", "c"); err == nil {
if _, err := p.Redeploy(context.Background(), "c", ""); err == nil {
t.Fatalf("expected error when ok=false")
}
}
@@ -207,14 +161,16 @@ func TestPlatformError401Hint(t *testing.T) {
})
defer done()
_, err := p.Apps(context.Background(), AppsQuery{})
if err == nil || !strings.Contains(err.Error(), "HTTP 401") || !strings.Contains(err.Error(), "platform service token") {
t.Fatalf("401 error should carry a token hint, got %v", err)
if err == nil || !strings.Contains(err.Error(), "HTTP 401") || !strings.Contains(err.Error(), "hanzo login") {
t.Fatalf("401 error should point at `hanzo login`, got %v", err)
}
}
func TestPlatformNoTokenError(t *testing.T) {
p := newPlatform("https://platform.hanzo.ai", "")
if _, err := p.Apps(context.Background(), AppsQuery{}); err == nil || !strings.Contains(err.Error(), "no platform token") {
t.Fatalf("expected no-token error, got %v", err)
// After unify-infra, the "no credential" error points at `hanzo login` — the one
// identity that authorizes the platform — not a separate platform token.
if _, err := p.Apps(context.Background(), AppsQuery{}); err == nil || !strings.Contains(err.Error(), "hanzo login") {
t.Fatalf("expected a `hanzo login` hint, got %v", err)
}
}
+69 -6
View File
@@ -12,6 +12,7 @@ package cli
import (
"context"
"encoding/json"
"fmt"
"io"
"net"
@@ -20,6 +21,7 @@ import (
"os/exec"
"path/filepath"
"strconv"
"sync/atomic"
"syscall"
"time"
)
@@ -107,6 +109,48 @@ func stopStudio(cmd *exec.Cmd) {
}
}
// studioRecycle carries at most one pending recycle request; the render
// handler signals it after each completed render (see gpu.go).
var studioRecycle = make(chan struct{}, 1)
// staging guards the claim-to-submit window: a claimed job is real work the
// engine queue cannot see yet, so the supervisor must never recycle over it
// (observed: jobs claimed during a recycle failed staging on a dead engine
// and were consumed).
var staging atomic.Int32
func requestStudioRecycle() {
select {
case studioRecycle <- struct{}{}:
default:
}
}
// studioBusy reports whether the engine holds queued or running prompts.
// A generous timeout: a saturated GB10 answers slowly mid-render — slow is
// alive, and killing a live render costs 8-70 minutes of GPU work.
func studioBusy(ctx context.Context) (busy, ok bool) {
ctx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://"+studioAddr+"/queue", nil)
if err != nil {
return false, false
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
return false, false
}
defer resp.Body.Close()
var q struct {
Running []json.RawMessage `json:"queue_running"`
Pending []json.RawMessage `json:"queue_pending"`
}
if err := json.NewDecoder(io.LimitReader(resp.Body, 32<<20)).Decode(&q); err != nil {
return false, false
}
return len(q.Running)+len(q.Pending) > 0, true
}
// superviseStudio keeps the local render backend on :8188 alive until ctx
// ends. Quiet by design: one line per restart event, not a probe firehose.
func superviseStudio(ctx context.Context, dir string, out io.Writer) {
@@ -138,6 +182,12 @@ func superviseStudio(ctx context.Context, dir string, out io.Writer) {
tick := time.NewTicker(studioProbeEvery)
defer tick.Stop()
// recyclePending defers the post-render recycle until the queue is EMPTY:
// short jobs complete while a long render is mid-sample, and recycling on
// their completion killed the live render (observed: every direct render
// died within ~6 minutes while probe jobs cycled).
recyclePending := false
unhealthy := 0
for {
select {
case <-ctx.Done():
@@ -145,19 +195,32 @@ func superviseStudio(ctx context.Context, dir string, out io.Writer) {
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGTERM)
}
return
case <-studioRecycle:
recyclePending = true
case <-tick.C:
busy, ok := studioBusy(ctx)
if recyclePending && ok && !busy && staging.Load() == 0 {
recyclePending = false
unhealthy = 0
restart("recycle")
continue
}
if studioHealthy(ctx) {
unhealthy = 0
continue
}
// Grace re-check: it may be momentarily busy mid-render.
select {
case <-ctx.Done():
if ok && busy {
// Alive-busy: slow health under render load is not death.
unhealthy = 0
continue
case <-time.After(studioGraceWait):
}
if !studioHealthy(ctx) {
restart("unresponsive")
// Sustained silence with an idle or unreadable queue = actually dead.
unhealthy++
if unhealthy < 3 {
continue
}
unhealthy = 0
restart("unresponsive")
}
}
}
+4
View File
@@ -161,6 +161,10 @@ func routesBridge(s *cloud.Service[state], app *zip.App) {
// console calls, forwarded to commerce with the admin service token and SCOPED to the
// validated caller's own subject (billing.go). Registered AFTER clients/billing's
// specific routes (121 < 122) so those win and this catches the rest. GET+POST only.
// The wildcard is what the ROUTER matches; it is NOT the forwardable set — billing.go's
// billingForwardable allowlist decides that, per method, and 404s everything else
// BEFORE the admin service token is attached. Widening this pattern grants nothing on
// its own; adding a line to that table is the only way to expose an endpoint.
app.Get("/v1/billing/*", cloud.Handle(s, billingData))
app.Post("/v1/billing/*", requireCSRF(s, cloud.Handle(s, billingData)))
// Per-tenant STORE DATA bridge — the canonical /v1/commerce/* the console calls,
+186 -23
View File
@@ -7,6 +7,14 @@
// read/act on its OWN ledger (balance / usage / invoices / subscriptions /
// payment-methods / spend-alerts / …), never another's.
//
// TWO INDEPENDENT BOUNDS, because the token makes this a privileged forwarder:
// 1. WHICH ENDPOINT — billingForwardable, the per-method allowlist below. It is the
// authorization gate: an unlisted path is 404'd before the token is ever attached, so
// no money-MINT route (deposit/credit/refund/…) can be reached through this bridge.
// 2. WHOSE DATA — the subject-pinning below. It aims a permitted call at the caller's own
// ledger. It is an IDOR control and NOT an authority control: on a mint route it would
// have pinned the CREDIT to the attacker's own account. (1) is what stops that.
//
// WHY A SERVER HANDLER (not a same-origin passthrough). Commerce's billing surface is
// service-token-gated and filters DIFFERENT endpoints on DIFFERENT subject params —
// subscriptions on ?userId, payment-methods on ?customerId, usage on ?user. Pinning
@@ -22,16 +30,124 @@ package account
import (
"bytes"
"crypto/subtle"
"encoding/json"
"net/http"
"net/url"
"strings"
"unicode"
"github.com/hanzoai/account"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
)
// billingForwardable — THE allowlist of billing endpoints this bridge may forward, keyed
// by method. It is the whole authorization story of the bridge, because forwarding IS
// authorization here: every forwarded request carries the admin COMMERCE_SERVICE_TOKEN,
// and commerce's money gate is MayMintMoney(c) = IsServiceToken(c) || IsSuperAdmin(c)
// (middleware/platformonly.go). The token satisfies IsServiceToken, so ANY subpath that
// reaches commerce is executed with PLATFORM authority — not the caller's. Commerce 403s
// an org admin who calls POST /v1/billing/deposit directly; without this table the bridge
// handed that same person the platform's own credential and minted it for them, scoped —
// by the subject-pinning below — to their OWN account. That is the escalation, and
// subject-pinning is what AIMS it, not what stops it. Only a path gate stops it.
//
// It is an ALLOWLIST, never a denylist: a denylist must enumerate every mint route
// (deposit/credit/refund/credit-grants/payouts/husd/allotment…) and stays correct only
// until commerce adds the next one — a route this file has never heard of is then
// forwarded by default. Here the default is REFUSE, so a new commerce mint route is
// unreachable the day it lands, with no change on this side. One table, one place; a path
// not in it cannot reach commerce, by construction.
//
// GET and POST are SEPARATE sets because a read bridge and a write bridge are different
// concerns: `payouts` is a legitimate read and a money-MINT write (api/billing/handlers.go
// `api.Get("/payouts", ListPayouts)` vs `api.Post("/payouts", mintRequired, CreatePayout)`),
// so one method-blind set would hand the mint to every reader. The POST set is therefore
// deliberately tiny and holds NOTHING that creates spendable balance from a client-named
// amount: cancel/reactivate a subscription, vault a card, create a budget, and a top-up
// that CHARGES a real card (money in, not minted). Every entry is a call the console
// actually makes; `{}` matches exactly one opaque id segment.
//
// EVIDENCE — each entry is a live console call (repo hanzoai/console):
//
// GET balance src/lib/api/billing.ts:397 sidebar wallet + billing overview
// GET usage src/lib/api/billing.ts:415 cost reports / AI metrics
// GET invoices src/lib/api/billing.ts:419 invoice history table
// GET invoices/{}/pdf src/components/products/billing/BillingInvoices.tsx:31
// GET subscriptions src/lib/api/billing.ts:423 subscriptions list
// GET payment-methods src/lib/api/billing.ts:450 saved cards (masked)
// GET spend-alerts src/lib/api/billing.ts:482 budgets / spend caps
// GET payment-config src/lib/api/billing.ts:552 public Square app/location id
// GET plans src/lib/api/plans.ts:126 published tiers
// GET payouts src/components/products/SettlementModule.tsx:61 settlement view
// POST subscriptions/{}/cancel src/lib/api/billing.ts:434
// POST subscriptions/{}/reactivate src/lib/api/billing.ts:444
// POST payment-methods src/lib/api/billing.ts:461 vault a Square nonce (no PAN)
// POST spend-alerts src/lib/api/billing.ts:500 create a budget
// POST topup/token src/lib/api/billing.ts:565 charge a card → credit
//
// balance/usage/payment-methods are ALSO served natively by clients/billing (order 121),
// which wins over this catch-all (122), so those entries are reached only on a deploy
// where that subsystem is disabled. They are listed because they are legitimate reads of
// the caller's own ledger, not because this bridge is their primary route.
//
// NOT LISTED, deliberately: `me/welcome` and `grant-starter` (console calls the first at
// billing.ts:407 and the second server-side at src/lib/server/billing-grant.ts:35) exist
// in NEITHER the pinned commerce (v1.48.5) route table — both 404 today whether or not
// this bridge forwards them, and grant-starter is mint-gated and browser-unreachable by
// design. The console's PATCH/DELETE calls (spend-alerts/{}, payment-methods/{}) are absent
// because routesBridge mounts GET+POST only, so they never reached this handler.
var billingForwardable = map[string][]string{
http.MethodGet: {
"balance",
"usage",
"invoices",
"invoices/{}/pdf",
"subscriptions",
"payment-methods",
"spend-alerts",
"spend-alerts/authorize", // the S2S cap-verdict read (metering gate); 2 segments need their own entry
"payment-config",
"plans",
"payouts",
},
http.MethodPost: {
"subscriptions/{}/cancel",
"subscriptions/{}/reactivate",
"payment-methods",
"spend-alerts",
"topup/token",
},
}
// isForwardableBilling reports whether method+sub is in billingForwardable. sub has
// already passed isSafeSegment, so no segment can contain a slash, a percent-escape, or a
// traversal — a pattern segment therefore matches exactly one real segment and `{}` cannot
// swallow a path. Fail-closed: an unknown method or an unlisted path is false.
func isForwardableBilling(method, sub string) bool {
got := strings.Split(sub, "/")
for _, pattern := range billingForwardable[method] {
want := strings.Split(pattern, "/")
if len(want) != len(got) {
continue
}
match := true
for i, seg := range want {
if seg != "{}" && seg != got[i] {
match = false
break
}
}
if match {
return true
}
}
return false
}
// billingSubjectKeys — every query/body param through which a commerce billing endpoint
// identifies its subject. Kept identical to commerce's edge-auth billingSubjectKeys
// {user,userId,customerId} AND console's billing-scope.ts BILLING_SUBJECT_KEYS. Change
@@ -48,17 +164,6 @@ func isSubjectKey(k string) bool {
return false
}
// billingSubject — the commerce billing subject for an org+user: ALWAYS the org
// (`org`), lowercased. Every member of an org reads/scopes to the ONE org billing
// account — the same subject the gateway gate reads and debits. `name` is recorded
// for metrics, never for the billing key. This is the ONE rule; the former
// PERSONAL_BILLING_ORGS / ORG_BILLING_ORGS allowlists are gone. Keep in lockstep
// with ai/object.BillingSubject so the console view and the gate never disagree.
func billingSubject(org, name string) string {
_ = name
return strings.ToLower(strings.TrimSpace(org))
}
// scopedBillingSearch — pin every billingSubjectKey to subject (OVERWRITING any client
// value — the browser cannot widen scope) and DROP org. Every OTHER param (currency,
// status, date range) passes through untouched. Mirrors billing-scope.ts.
@@ -139,9 +244,26 @@ func commerceCreds() (base, token string) {
func billingData(s *cloud.Service[state], c *zip.Ctx) error {
// IDOR boundary: the subject is the VALIDATED caller's own org/user, never a client
// value. requireOwner=true — billing is always org-scoped (a zero-org user has none).
// Auth. A browser caller is the VALIDATED principal (customer path — subject-pinned
// below). An IN-PROC S2S caller carries the verified COMMERCE_SERVICE_TOKEN (the
// metering cap-gate's authorize + the SuperAdmin cap-oversight Forward). The gateway
// 401s a public Bearer that is not an IAM JWT / hk-|pk-|sk- key (the 64-hex service
// token fails JWT parse at the edge), so an EXTERNAL client can NEVER present it here —
// an unauthenticated caller still hits the 403 below. On the S2S path the caller
// legitimately names its own subject, so its query is forwarded as-is (no pin), scoped
// only by the EdgeAuth-controlled X-Org-Id.
cr, ok := resolveCaller(c, true)
s2s := false
owner := cr.owner
if !ok {
return zip.ErrForbidden("sign in to view billing")
if !s2sBillingCall(c) {
return zip.ErrForbidden("sign in to view billing")
}
owner = strings.TrimSpace(c.Org()) // trusted X-Org-Id (never a client value on a public call)
if owner == "" {
return zip.ErrForbidden("sign in to view billing")
}
s2s = true
}
method := c.Method()
@@ -165,19 +287,42 @@ func billingData(s *cloud.Service[state], c *zip.Ctx) error {
return zip.ErrBadRequest("invalid billing path")
}
}
// Scope EVERY request to the caller's OWN subject — query AND write body — so
// commerce's per-tenant isolation can never be crossed from the browser.
subject := billingSubject(cr.owner, cr.name)
inQuery, _ := url.ParseQuery(string(c.Fiber().Request().URI().QueryString()))
q := scopedBillingSearch(inQuery, subject)
var body []byte
if method == http.MethodPost {
body = scopedBillingBody(c.Body(), subject)
// THE authorization gate. Forwarding is authorization: the request below carries the
// admin service token, which satisfies commerce's MayMintMoney. So refuse anything the
// console does not actually call — BEFORE the token is attached. Fail closed (404, the
// same answer an unrouted path gives, so this leaks no map of the money surface).
if !isForwardableBilling(method, sub) {
return zip.Errorf(http.StatusNotFound, "not a forwardable billing endpoint")
}
raw, status, err := commerceDo(c.Context(), base, token, method, "/v1/billing/"+sub, q, cr.owner, body)
// Scope EVERY request to the caller's OWN subject — query AND write body — so
// commerce's per-tenant isolation can never be crossed from the browser. The
// subject comes from the ONE rule (ai/object.Payer), fed the account the
// credential NAMES (the validated `billing_account` claim) — the same claim the
// ai gate reads, so a top-up credits the SAME account the gate debits. Feeding
// Payer a different credential here than the gate gets is the modern shape of
// the old split: money landing in an account the gate never reads.
inQuery, _ := url.ParseQuery(string(c.Fiber().Request().URI().QueryString()))
var q url.Values
var body []byte
if s2s {
// Trusted S2S caller: forward its query/body VERBATIM — it legitimately names the
// subject (e.g. the metering gate's ?user=<org>&amount=). Scoped by X-Org-Id.
q = inQuery
if method == http.MethodPost {
body = c.Body()
}
} else {
// Browser customer: pin EVERY subject key to the caller's OWN account so commerce's
// per-tenant isolation can never be crossed from the client.
subject := account.Payer(account.Credential{Owner: cr.owner, Name: cr.username, Account: principal.BillingAccount(c)}).Subject()
q = scopedBillingSearch(inQuery, subject)
if method == http.MethodPost {
body = scopedBillingBody(c.Body(), subject)
}
}
raw, status, err := commerceDo(c.Context(), base, token, method, "/v1/billing/"+sub, q, owner, body)
if err != nil {
return zip.Errorf(http.StatusBadGateway, "billing upstream unreachable: %v", err)
}
@@ -187,3 +332,21 @@ func billingData(s *cloud.Service[state], c *zip.Ctx) error {
c.SetHeader("Cache-Control", "no-store, must-revalidate")
return c.Bytes(status, raw)
}
// s2sBillingCall reports whether the request carries the verified COMMERCE_SERVICE_TOKEN
// as its Bearer — a trusted IN-PROC service-to-service caller (the metering cap-gate's
// authorize, the SuperAdmin cap-oversight Forward). It is the SAME secret this bridge
// already forwards WITH, so admitting a caller who already holds it grants no authority it
// could not otherwise wield. Safety rests on the edge: the gateway 401s a public Bearer
// that is not an IAM JWT / hk-|pk-|sk- API key (the 64-hex service token is a JWT
// candidate that fails to parse), so an EXTERNAL client can never reach this handler
// holding it — only in-proc commerceinproc dispatch does. Constant-time compare; the token
// is never logged.
func s2sBillingCall(c *zip.Ctx) bool {
_, token := commerceCreds()
if token == "" {
return false
}
bearer := strings.TrimSpace(strings.TrimPrefix(c.Header("Authorization"), "Bearer "))
return bearer != "" && subtle.ConstantTimeCompare([]byte(bearer), []byte(token)) == 1
}
+88 -18
View File
@@ -2,6 +2,7 @@ package account
import (
"encoding/json"
"github.com/hanzoai/account"
"io"
"net/http"
"net/http/httptest"
@@ -16,37 +17,44 @@ import (
// ── pure scoping ─────────────────────────────────────────────────────────────
// TestBillingSubject proves the top-up subject is resolved through the ONE rule
// (ai/object.Payer) — so a top-up credits the SAME account the ai gate debits and
// the console reads. The signup org bills per-person (matching the gate), which is
// the whole fix: money and gate land on one account.
func TestBillingSubject(t *testing.T) {
cases := []struct{ org, name, want string }{
{"acme", "alice", "acme"}, // any member bills the ONE org account
{"hanzo", "Dave", "hanzo"}, // no per-user wallet; org, lowercased
{"hanzo", "z", "hanzo"}, // another member — same org account
{"hanzo", "", "hanzo"}, // no name → org
{"Hanzo", "z", "hanzo"}, // lowercased
{"", "x", ""}, // no org → empty subject
{"acme", "alice", "acme"}, // real org: any member bills the ONE org account
{"hanzo", "Dave", "hanzo/dave"}, // signup org: each person bills their OWN account
{"hanzo", "z", "hanzo/z"}, // another signup person — their own account
{"hanzo", "", "hanzo"}, // no name (org-owned principal) → org pool
{"Hanzo", "Z", "hanzo/z"}, // folded
{"", "x", ""}, // no org → empty subject (cannot bill)
}
for _, c := range cases {
if got := billingSubject(c.org, c.name); got != c.want {
t.Fatalf("billingSubject(%q,%q): want %q, got %q", c.org, c.name, c.want, got)
got := account.Payer(account.Credential{Owner: c.org, Name: c.name}).Subject()
if got != c.want {
t.Fatalf("Payer(%q,%q).Subject(): want %q, got %q", c.org, c.name, c.want, got)
}
}
}
// TestBillingSubject_IgnoresLegacyEnv locks that the killed allowlist envs have NO
// effect: the subject is ALWAYS the org, whether or not the old PERSONAL_BILLING_ORGS
// / ORG_BILLING_ORGS knobs are set. This mirrors ai/object.BillingSubject (one rule,
// no config) so the console view and the gateway gate can never disagree.
// effect: nothing reads them. Set to values that WOULD have flipped every
// resolution — the subject is unchanged. This is the console/top-up half of the
// same proof ai carries (one rule, no config), so the view and the gate can never
// disagree, and the deleted CR env is a genuine no-op.
func TestBillingSubject_IgnoresLegacyEnv(t *testing.T) {
t.Setenv("PERSONAL_BILLING_ORGS", "hanzo,acme")
t.Setenv("ORG_BILLING_ORGS", "hanzo")
t.Setenv("PERSONAL_BILLING_ORGS", "hanzo,acme") // would have split acme per-user
t.Setenv("ORG_BILLING_ORGS", "hanzo") // would have pooled the signup org
cases := []struct{ org, name, want string }{
{"hanzo", "z", "hanzo"},
{"acme", "alice", "acme"},
{"maxpower", "dave", "maxpower"},
{"hanzo", "z", "hanzo/z"}, // env cannot pool the signup org
{"acme", "alice", "acme"}, // env cannot split a real org per-user
{"maxpower", "dave", "maxpower"}, // untouched
}
for _, c := range cases {
if got := billingSubject(c.org, c.name); got != c.want {
t.Fatalf("legacy env must be ignored: billingSubject(%q,%q) want %q, got %q", c.org, c.name, c.want, got)
got := account.Payer(account.Credential{Owner: c.org, Name: c.name}).Subject()
if got != c.want {
t.Fatalf("legacy env must be ignored: Payer(%q,%q).Subject() want %q, got %q", c.org, c.name, c.want, got)
}
}
}
@@ -224,3 +232,65 @@ func TestBilling_RejectsTraversalSegment(t *testing.T) {
t.Fatalf("a traversal must never reach commerce, but upstream saw %q", f.path)
}
}
// ── S2S service-token admission (the auth fix; 4 security invariants) ─────────
// Invariant #4 — THE SECURITY GATE: a public/unauthenticated caller (no validated
// principal AND not the service token) STILL gets 403 on the spend-alert routes, incl.
// a WRONG bearer. The fix must NEVER open billing to the world.
func TestBilling_S2S_PublicStill403(t *testing.T) {
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-tok")
app := mountApp(t, "http://iam.invalid", "", "")
for _, path := range []string{
"/v1/billing/spend-alerts",
"/v1/billing/spend-alerts/authorize?user=acme&amount=1",
} {
// forged X-Org-Id, no validated principal, no service token
if code, body := callH(t, app, http.MethodGet, path, map[string]string{"X-Org-Id": "victim"}, ""); code != http.StatusForbidden {
t.Fatalf("public caller to %s: want 403, got %d (%s)", path, code, body)
}
}
// a WRONG bearer is still just a public caller → 403
if code, _ := callH(t, app, http.MethodGet, "/v1/billing/spend-alerts/authorize?user=acme&amount=1",
map[string]string{"X-Org-Id": "acme", "Authorization": "Bearer not-the-token"}, ""); code != http.StatusForbidden {
t.Fatalf("wrong bearer: want 403")
}
}
// The trusted in-proc S2S caller (verified COMMERCE_SERVICE_TOKEN + X-Org-Id) is admitted
// and its authorize query is forwarded to commerce VERBATIM (a trusted caller names its
// own subject), scoped by X-Org-Id — this is what lets the cap gate reach AuthorizeSpendCap.
func TestBilling_S2S_ServiceTokenForwardsVerbatim(t *testing.T) {
f := &fakeBilling{}
t.Setenv("COMMERCE_URL", f.server(t).URL)
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-tok")
app := mountApp(t, "http://iam.invalid", "", "")
code, body := callH(t, app, http.MethodGet,
"/v1/billing/spend-alerts/authorize?user=acme&amount=100&project=P",
map[string]string{"Authorization": "Bearer svc-tok", "X-Org-Id": "acme"}, "")
if code != http.StatusOK {
t.Fatalf("S2S authorize: want 200, got %d (%s)", code, body)
}
if f.path != "/v1/billing/spend-alerts/authorize" {
t.Fatalf("forwarded path = %q", f.path)
}
// VERBATIM: the S2S caller's ?user/?amount/?project reach commerce un-pinned.
if f.query.Get("user") != "acme" || f.query.Get("amount") != "100" || f.query.Get("project") != "P" {
t.Fatalf("S2S query must forward verbatim, got %v", f.query)
}
if f.org != "acme" || f.auth != "Bearer svc-tok" {
t.Fatalf("S2S must send X-Org-Id=acme + service token, got org=%q auth=%q", f.org, f.auth)
}
}
// S2S with the verified token but NO X-Org-Id → 403 (no org to scope the privileged
// forward to; never fall back to a client value).
func TestBilling_S2S_NoOrg403(t *testing.T) {
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-tok")
app := mountApp(t, "http://iam.invalid", "", "")
if code, _ := callH(t, app, http.MethodGet, "/v1/billing/spend-alerts/authorize?user=acme&amount=1",
map[string]string{"Authorization": "Bearer svc-tok"}, ""); code != http.StatusForbidden {
t.Fatalf("S2S without X-Org-Id: want 403")
}
}
+257
View File
@@ -0,0 +1,257 @@
package account
import (
"net/http"
"strings"
"testing"
commercebilling "github.com/hanzoai/commerce/api/billing"
commercemid "github.com/hanzoai/commerce/middleware"
"github.com/zap-proto/zip"
)
// bridge_mint_test.go — the privilege-escalation boundary of the /v1/billing/*
// bridge: an ordinary signed-in ORG user must never reach commerce's money-MINT
// surface.
//
// THE ESCALATION THIS LOCKS OUT. The bridge forwards to commerce with the admin
// COMMERCE_SERVICE_TOKEN. Commerce gates every mint on
// MayMintMoney(c) = IsServiceToken(c) || IsSuperAdmin(c) (middleware/platformonly.go)
// — and the bridge's service token satisfies IsServiceToken. So ANY subpath the
// bridge forwards is executed by commerce as the PLATFORM, not as the caller.
// billingData scopes the SUBJECT to the caller's own account, which is exactly the
// attack rather than a defense: an org user mints to THEMSELVES. Commerce's own
// gate comment names this: "let ANY org owner self-credit unlimited balance (POST
// /v1/billing/deposit &c.) → unlimited free inference."
//
// Commerce 403s that same org admin when they call it DIRECTLY
// (TestC1_OrgAdminDeniedOnEveryMintRoute) and mints 201 for the service token
// (TestC1_ServiceTokenMintsDeposit). The bridge is what converts the former into
// the latter. The gate therefore has to live HERE, at the point that hands out the
// token: forwardable subpaths are an ALLOWLIST, and a mint path is not on it.
//
// alice is an ordinary org user — X-Org-Id "acme", owner != "admin", NOT a
// SuperAdmin — i.e. precisely the principal commerce refuses at the front door.
// TestBridge_OrgUserCannotReachMint is the reproduction. Each of these commerce
// subpaths is PlatformOnly-gated (api/billing/handlers.go: `mintRequired`), meaning
// possession of the service token IS authority to create spendable balance. None
// may leave cloud. A request that never reaches commerce cannot mint, so the
// assertion is twofold: the caller is refused AND upstream saw nothing.
// mintSurface asks COMMERCE which routes it gates, rather than keeping a copy.
//
// The list used to live here by hand under "kept in lockstep with
// api/billing/handlers.go" — and it had already drifted: 10 paths here against
// 16 commerce actually gates. A comment cannot hold two lists together. Now
// commerce DECLARES its gated surface (middleware.Mint records what it gates)
// and we read that declaration, so a mint route added there is covered here with
// nobody remembering to do anything.
//
// Registration is what populates the registry, so register first, then read.
func mintSurface(t *testing.T) []commercemid.MintRoute {
t.Helper()
commercebilling.Route(zip.New(zip.Config{DisableStartupMessage: true}).Group("/v1"))
var out []commercemid.MintRoute
for _, r := range commercemid.MintRoutes() {
// Only what THIS bridge can address: it forwards /v1/billing/* alone.
if !strings.HasPrefix(r.Path, "/v1/billing/") {
continue
}
// A wildcard segment needs some concrete value to be requestable; which
// one is irrelevant, since a refused call never reaches an id.
parts := strings.Split(r.Path, "/")
for i, seg := range parts {
if strings.HasPrefix(seg, ":") || seg == "{}" {
parts[i] = "probe"
}
}
r.Path = strings.Join(parts, "/")
out = append(out, r)
}
if len(out) == 0 {
t.Fatal("commerce declared no /v1/billing mint routes — the registry is not being populated")
}
return out
}
func TestBridge_OrgUserCannotReachMint(t *testing.T) {
mintPaths := mintSurface(t)
t.Logf("commerce declares %d gated /v1/billing mint routes", len(mintPaths))
for _, m := range mintPaths {
t.Run(m.Method+" "+m.Path, func(t *testing.T) {
f := &fakeBilling{}
t.Setenv("COMMERCE_URL", f.server(t).URL)
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-tok")
app := mountApp(t, "http://iam.invalid", "", "")
code, body := callH(t, app, m.Method, m.Path, alice, `{}`)
// The mint request must NEVER reach commerce: arriving there at all means
// it arrived bearing the admin service token, which IS the authority to
// mint (MayMintMoney → AuthorizeMint → the ledger write).
if f.path != "" {
t.Fatalf("ESCALATION: an ordinary org user's %s %s reached commerce at %q "+
"carrying %q — the service token that satisfies MayMintMoney. "+
"Minted subject=%v amount=%v in org=%q.",
m.Method, m.Path, f.path, f.auth, f.body["user"], f.body["amount"], f.org)
}
if code != http.StatusNotFound {
t.Fatalf("%s %s: want 404 (not a forwardable billing endpoint), got %d (%s)",
m.Method, m.Path, code, body)
}
})
}
}
// TestBridge_ConsoleCallsStillForward is the other half of the allowlist: the calls the
// console ACTUALLY makes must still reach commerce. An allowlist that blocks the product
// is not a fix, so each entry here is a live console call (cited in billing.go), and this
// test fails if a future edit narrows the table below the console's real needs.
func TestBridge_ConsoleCallsStillForward(t *testing.T) {
calls := []struct{ method, path, want string }{
{http.MethodGet, "/v1/billing/invoices", "/v1/billing/invoices"},
{http.MethodGet, "/v1/billing/invoices/inv_123/pdf", "/v1/billing/invoices/inv_123/pdf"},
{http.MethodGet, "/v1/billing/subscriptions", "/v1/billing/subscriptions"},
{http.MethodGet, "/v1/billing/spend-alerts", "/v1/billing/spend-alerts"},
{http.MethodGet, "/v1/billing/payment-config", "/v1/billing/payment-config"},
{http.MethodGet, "/v1/billing/plans", "/v1/billing/plans"},
{http.MethodGet, "/v1/billing/payouts", "/v1/billing/payouts"},
{http.MethodPost, "/v1/billing/subscriptions/sub_1/cancel", "/v1/billing/subscriptions/sub_1/cancel"},
{http.MethodPost, "/v1/billing/subscriptions/sub_1/reactivate", "/v1/billing/subscriptions/sub_1/reactivate"},
{http.MethodPost, "/v1/billing/payment-methods", "/v1/billing/payment-methods"},
{http.MethodPost, "/v1/billing/spend-alerts", "/v1/billing/spend-alerts"},
{http.MethodPost, "/v1/billing/topup/token", "/v1/billing/topup/token"},
}
for _, call := range calls {
t.Run(call.method+" "+call.path, func(t *testing.T) {
f := &fakeBilling{}
t.Setenv("COMMERCE_URL", f.server(t).URL)
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-tok")
app := mountApp(t, "http://iam.invalid", "", "")
code, body := callH(t, app, call.method, call.path, alice, "{}")
if code != http.StatusOK {
t.Fatalf("%s %s: want 200 (the console needs this), got %d (%s)",
call.method, call.path, code, body)
}
if f.path != call.want {
t.Fatalf("%s %s must forward to %q, got %q", call.method, call.path, call.want, f.path)
}
})
}
}
// TestBridge_UnlistedPathsAreRefused covers the rest of the money surface — routes that
// are NOT mint-gated but that the console never calls. The bridge is not a general
// commerce proxy; least privilege means "only what the product needs", so these 404
// even though commerce would have served them to a service token.
func TestBridge_UnlistedPathsAreRefused(t *testing.T) {
unlisted := []struct{ method, path string }{
{http.MethodPost, "/v1/billing/invoices"}, // CreateInvoice (admin group)
{http.MethodPost, "/v1/billing/invoices/i1/pay"}, // PayInvoice
{http.MethodPost, "/v1/billing/invoices/i1/void"}, // VoidInvoice
{http.MethodPost, "/v1/billing/meters"}, // CreateMeter
{http.MethodPost, "/v1/billing/pricing-rules"}, // CreatePricingRule
{http.MethodPost, "/v1/billing/withdraw"}, // money OUT
{http.MethodPost, "/v1/billing/usage"}, // RecordUsage — the meter itself
{http.MethodGet, "/v1/billing/balance/all"}, // every subject's balance
{http.MethodGet, "/v1/billing/sbom"}, // OSS payout surface
{http.MethodGet, "/v1/billing/oss-payout/summary"}, // OSS payout rollup
{http.MethodPost, "/v1/billing/subscriptions"}, // CreateBillingSubscription
}
for _, u := range unlisted {
t.Run(u.method+" "+u.path, func(t *testing.T) {
f := &fakeBilling{}
t.Setenv("COMMERCE_URL", f.server(t).URL)
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-tok")
app := mountApp(t, "http://iam.invalid", "", "")
code, _ := callH(t, app, u.method, u.path, alice, "{}")
if f.path != "" {
t.Fatalf("%s %s is not a console call and must not reach commerce, but upstream saw %q",
u.method, u.path, f.path)
}
if code != http.StatusNotFound {
t.Fatalf("%s %s: want 404, got %d", u.method, u.path, code)
}
})
}
}
// TestBridge_ReadAllowlistIsNotAWriteAllowlist pins the method split. `payouts` is the
// proof that one method-blind set would be a hole: GET /payouts is a plain read, POST
// /payouts is `mintRequired` (api/billing/handlers.go). The same string must resolve
// differently by method, or reading the settlement view would grant minting a payout.
func TestBridge_ReadAllowlistIsNotAWriteAllowlist(t *testing.T) {
if !isForwardableBilling(http.MethodGet, "payouts") {
t.Fatal("GET payouts is a live console read and must be forwardable")
}
if isForwardableBilling(http.MethodPost, "payouts") {
t.Fatal("POST payouts is mint-gated in commerce and must NEVER be forwardable")
}
// A GET-only entry must not leak into POST, and vice-versa.
if isForwardableBilling(http.MethodPost, "invoices") {
t.Fatal("POST invoices must not inherit the GET entry")
}
if isForwardableBilling(http.MethodGet, "topup/token") {
t.Fatal("GET topup/token must not inherit the POST entry")
}
// An unknown method fails closed (the router mounts GET+POST only; defense in depth).
for _, m := range []string{http.MethodPut, http.MethodPatch, http.MethodDelete, ""} {
if isForwardableBilling(m, "balance") {
t.Fatalf("method %q must fail closed", m)
}
}
// `{}` matches exactly ONE segment — it can never swallow a path into a mint route.
if isForwardableBilling(http.MethodPost, "subscriptions/a/b/cancel") {
t.Fatal("{} must match exactly one segment")
}
}
// TestBridge_StoreBridgeCannotReachBilling is the sibling lock. /v1/commerce/* carries the
// SAME admin token with FULL CRUD, and its own allowlist (commerceStoreHeads) is what keeps
// it a store proxy. Prove it cannot tunnel into the money surface — a store head that
// resolved to `billing` would reopen this hole from the other bridge.
func TestBridge_StoreBridgeCannotReachBilling(t *testing.T) {
for _, p := range []string{
"/v1/commerce/billing/deposit",
"/v1/commerce/billing",
"/v1/commerce/checkout",
"/v1/commerce/_/commerce/tenants",
} {
f := &fakeBilling{}
t.Setenv("COMMERCE_URL", f.server(t).URL)
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-tok")
app := mountApp(t, "http://iam.invalid", "", "")
code, _ := callH(t, app, http.MethodPost, p, alice, `{"amount":100000000}`)
if f.path != "" {
t.Fatalf("store bridge %q must never reach commerce, but upstream saw %q", p, f.path)
}
if code != http.StatusNotFound {
t.Fatalf("store bridge %q: want 404, got %d", p, code)
}
}
}
// TestBridge_MintIsRefusedEvenWithForgedSubject proves the refusal does not depend
// on the subject-pinning. Pinning is an IDOR control, not an authority control: it
// makes the mint land on the CALLER's own account, which is the attack, not a
// defense. The path gate must refuse before any of that logic runs.
func TestBridge_MintIsRefusedEvenWithForgedSubject(t *testing.T) {
f := &fakeBilling{}
t.Setenv("COMMERCE_URL", f.server(t).URL)
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-tok")
app := mountApp(t, "http://iam.invalid", "", "")
code, _ := callH(t, app, http.MethodPost, "/v1/billing/deposit", alice,
`{"user":"victim","userId":"victim","amount":100000000}`)
if f.path != "" {
t.Fatalf("ESCALATION: deposit reached commerce at %q with the service token", f.path)
}
if code != http.StatusNotFound {
t.Fatalf("forged-subject deposit: want 404, got %d", code)
}
}
+19 -2
View File
@@ -38,7 +38,10 @@ import (
"github.com/hanzoai/cloud/clients/admin/finance"
"github.com/hanzoai/cloud/clients/admin/health"
"github.com/hanzoai/cloud/clients/admin/iam"
"github.com/hanzoai/cloud/clients/admin/invoices"
"github.com/hanzoai/cloud/clients/admin/metrics"
"github.com/hanzoai/cloud/clients/admin/revenue"
"github.com/hanzoai/cloud/clients/admin/subscriptions"
"github.com/hanzoai/cloud/clients/commerceinproc"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
@@ -100,6 +103,7 @@ func routes(app *zip.App, s *cloud.Service[core.State]) {
app.Get("/v1/admin/products", core.Guard(s, products))
app.Get("/v1/admin/compute", core.Guard(s, compute))
app.Get("/v1/admin/o11y", core.Guard(s, o11y))
app.Get("/v1/admin/aimetrics", core.Guard(s, aimetrics))
app.Post("/v1/admin/sync", core.Guard(s, syncNow))
// Product analytics — org-scoped (SuperAdmin: all-orgs; org admin: their own org).
@@ -108,15 +112,28 @@ func routes(app *zip.App, s *cloud.Service[core.State]) {
app.Get("/v1/admin/bases", core.GuardScoped(s, bases))
// ── Platform control plane — SuperAdmin ONLY (launch/release/flags + access). ──
app.Get("/v1/admin/flags", core.Guard(s, flags))
app.Get("/v1/admin/flags", core.Guard(s, flagsBoard))
app.Put("/v1/admin/flags/:key", core.Guard(s, setFlag))
// Launch-control services board — the waitlist-mode lens on the flag engine (twin
// of /v1/admin/flags), reading the registry + decide the admission gate owns.
app.Get("/v1/admin/services", core.Guard(s, services))
app.Post("/v1/admin/services", core.Guard(s, upsertService))
app.Post("/v1/admin/services/:service/mode", core.Guard(s, setServiceMode))
app.Get("/v1/admin/waitlist", core.Guard(s, waitlist))
app.Post("/v1/admin/waitlist/boost", core.Guard(s, waitlistBoost))
// ── Carved-out domains own their routes (audit/customer/revenue/finance). ──
// Usage-cap + promo control plane (promos platform-only; spend-caps org-scoped).
limitRoutes(app, s)
// ── Carved-out domains own their routes (audit/customer/revenue/finance +
// the billing fleet views metrics/invoices/subscriptions). ──
audit.Routes(app, s)
customer.Routes(app, s)
revenue.Routes(app, s)
finance.Routes(app, s)
metrics.Routes(app, s)
invoices.Routes(app, s)
subscriptions.Routes(app, s)
}
// ── /v1/admin/me — operator identity (AdminMe) ───────────────────────────────
+392
View File
@@ -0,0 +1,392 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package admin
// aimetrics — GET /v1/admin/aimetrics, the GLOBAL fleet-wide AI / training / eval
// read that powers the operator's AI-metrics board on admin.hanzo.ai. It is the
// AI-and-eval-focused companion to o11y (o11y.go): where o11y answers "how is the
// FLEET behaving" (RED metrics, logs, usage), this answers "how are the MODELS and
// EVALS doing" — LLM generations, per-model spend, and eval-run quality/progress —
// over the SAME ONE datastore (Datastore), the SAME shared client
// (aiobject.DatastoreQuery), no second connection.
//
// Signals, each from its canonical table in the one datastore:
// - LLM generations → langfuse.observations : generations, cost (USD), latency
// (fleet-wide; honest-empty until the
// Langfuse ingest lands rows)
// - Per-model usage → hanzo.cloud_usage : requests, tokens, cost per model
// (the live usage ledger the ai gateway
// writes — populated today)
// - Eval runs → hanzo.eval_traces : traces, runs, datasets, models under
// test, per-trace latency
// - Eval progress → hanzo.eval_scores : score count, avg score, per-score-name
// distribution, recent-run averages, and
// the avg-score-over-time TREND — the
// training/eval progress signal
//
// The eval_traces / eval_scores tables are OWNED and written by the eval telemetry
// store (clients/eval/telemetry.go) — the SAME warehouse, same db ("hanzo"), same
// shared aiobject client. admin only READS them here. There is deliberately no
// "training_progress" table: the router's per-request training events live in the ai
// OLTP Postgres (object.RoutingEvent), NOT the OLAP warehouse, so the honest
// warehouse-side progress signal is the eval-score trend, not a routing table.
//
// SUPERADMIN ONLY (the core.Guard wrap in admin.go), all-orgs, no org filter — the
// one place a fleet operator crosses tenants for AI/eval metrics; a non-admin bearer
// is refused 403 before a single row is read. Fail-closed.
//
// Honest by construction, exactly like o11y/compute: no datastore connected → the
// real empty aggregate, never a fabricated fleet; and every signal degrades
// INDEPENDENTLY — a table that is absent or a column that differs contributes its
// zero-value (the enclosing `if err == nil`), never a failure, so the board always
// renders what the datastore actually holds. admin READS only; it owns and creates
// NO table. Money from cloud_usage is USD cents, from langfuse is USD; latency is
// milliseconds; time bounds are POSITIONAL parameters (never interpolated), and the
// bucket interval is a server-side constant — injection-safe.
import (
"strconv"
"time"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/zap-proto/zip"
)
// Fully-qualified datastore tables. admin only READS these — the ai gateway owns
// hanzo.cloud_usage, Langfuse owns langfuse.observations, and the eval telemetry
// store (clients/eval) owns hanzo.eval_traces / hanzo.eval_scores.
const (
aimUsageTable = "hanzo.cloud_usage"
aimLangfuseObs = "langfuse.observations"
aimEvalTraces = "hanzo.eval_traces"
aimEvalScores = "hanzo.eval_scores"
aimTopN = 12
)
// aiMetrics is the whole AI-metrics board payload.
type aiMetrics struct {
Range string `json:"range"`
Start string `json:"start"`
End string `json:"end"`
Langfuse aimLangfuse `json:"langfuse"`
Usage aimUsage `json:"usage"`
Evals aimEvals `json:"evals"`
TopModels []aimModelStat `json:"topModels"` // cloud_usage per-model (populated today)
LangfuseModels []aimLfModelStat `json:"langfuseModels"` // langfuse per-model (honest-empty today)
ScoreNames []aimScoreStat `json:"scoreNames"` // eval_scores per score-name
EvalRuns []aimRunStat `json:"evalRuns"` // recent eval runs (progress)
ScoreSeries []aimScorePoint `json:"scoreSeries"` // avg eval score over time (progress trend)
}
// aimLangfuse is the fleet-wide Langfuse generation rollup (honest-empty today).
// Cost is USD (Langfuse's native unit); latency is milliseconds (end_time-start_time).
type aimLangfuse struct {
Generations int64 `json:"generations"`
CostUsd float64 `json:"costUsd"`
LatencyMsAvg float64 `json:"latencyMsAvg"`
LatencyMsP95 float64 `json:"latencyMsP95"`
}
// aimUsage is the fleet LLM-usage KPI band from the live cloud_usage ledger.
type aimUsage struct {
Requests int64 `json:"requests"`
Tokens int64 `json:"tokens"`
PromptTokens int64 `json:"promptTokens"`
CompletionTokens int64 `json:"completionTokens"`
CostCents int64 `json:"costCents"`
Models int64 `json:"models"`
}
// aimEvals is the fleet eval KPI band: the trace half (eval_traces) and the score
// half (eval_scores). LatencyMsAvg is the mean model-under-test call window.
type aimEvals struct {
Runs int64 `json:"runs"`
Traces int64 `json:"traces"`
Datasets int64 `json:"datasets"`
Models int64 `json:"models"`
LatencyMsAvg float64 `json:"latencyMsAvg"`
Scores int64 `json:"scores"`
ScoreNames int64 `json:"scoreNames"`
AvgScore float64 `json:"avgScore"`
}
// aimModelStat is one row of the per-model usage leaderboard (cloud_usage).
type aimModelStat struct {
Model string `json:"model"`
Requests int64 `json:"requests"`
Tokens int64 `json:"tokens"`
CostCents int64 `json:"costCents"`
}
// aimLfModelStat is one row of the per-model Langfuse leaderboard (honest-empty today).
type aimLfModelStat struct {
Model string `json:"model"`
Generations int64 `json:"generations"`
CostUsd float64 `json:"costUsd"`
}
// aimScoreStat is one row of the per-score-name eval leaderboard (eval_scores).
type aimScoreStat struct {
Name string `json:"name"`
Count int64 `json:"count"`
AvgValue float64 `json:"avgValue"`
MinValue float64 `json:"minValue"`
MaxValue float64 `json:"maxValue"`
}
// aimRunStat is one recent eval run: its dataset, how many scores it recorded, its
// mean score, and when it last ran — the run-level eval-progress row.
type aimRunStat struct {
RunName string `json:"runName"`
Dataset string `json:"dataset"`
Scores int64 `json:"scores"`
AvgValue float64 `json:"avgValue"`
LastTs string `json:"lastTs"`
}
// aimScorePoint is one bucket of the avg-eval-score-over-time trend.
type aimScorePoint struct {
Ts string `json:"ts"`
AvgValue float64 `json:"avgValue"`
Count int64 `json:"count"`
}
// aimetrics answers GET /v1/admin/aimetrics. ?range=24h|7d|30d bounds the window
// (default 30d). SUPERADMIN ONLY (core.Guard). Every signal degrades independently:
// a table that is absent or errors contributes its zero-value, never a failure — the
// board always renders what the datastore actually holds.
func aimetrics(s *cloud.Service[core.State], c *zip.Ctx) error {
ctx := c.Context()
rangeLabel := o11yRange(c.Query("range"))
since := computeSince(rangeLabel)
payload := aiMetrics{
Range: rangeLabel,
Start: since.Format(time.RFC3339),
End: time.Now().UTC().Format(time.RFC3339),
TopModels: []aimModelStat{},
LangfuseModels: []aimLfModelStat{},
ScoreNames: []aimScoreStat{},
EvalRuns: []aimRunStat{},
ScoreSeries: []aimScorePoint{},
}
// Honest-empty when the warehouse is not connected: the board renders its zero
// state, never a fabricated fleet.
if !aiobject.DatastoreEnabled() {
return core.OK(c, payload)
}
sinceTS := chTS(since) // DateTime literal — cloud_usage.timestamp, langfuse.start_time, eval_*.ts
interval := o11yBucket(rangeLabel)
// ── Langfuse generations (fleet) — honest-empty until ingest lands rows ──
if rows, err := aiobject.DatastoreQuery(ctx, aimLangfuseTotalsSQL(), sinceTS); err == nil {
r := firstRowOr(rows)
payload.Langfuse.Generations = chInt64(r["gens"])
payload.Langfuse.CostUsd = chFloat64(r["cost"])
}
// Langfuse latency (separate query so a Nullable end_time / column mismatch never
// zeroes the proven generations+cost number above).
if rows, err := aiobject.DatastoreQuery(ctx, aimLangfuseLatencySQL(), sinceTS); err == nil {
r := firstRowOr(rows)
payload.Langfuse.LatencyMsAvg = chFloat64(r["lat_avg"])
payload.Langfuse.LatencyMsP95 = chFloat64(r["lat_p95"])
}
// Langfuse per-model.
if rows, err := aiobject.DatastoreQuery(ctx, aimLangfuseModelsSQL(), sinceTS); err == nil {
payload.LangfuseModels = lfModelsFromRows(rows)
}
// ── Per-model usage (fleet) from the live cloud_usage ledger ──
if rows, err := aiobject.DatastoreQuery(ctx, aimUsageTotalsSQL(), sinceTS); err == nil {
fillAimUsage(&payload.Usage, firstRowOr(rows))
}
if rows, err := aiobject.DatastoreQuery(ctx, aimTopModelsSQL(), sinceTS); err == nil {
payload.TopModels = aimModelsFromRows(rows)
}
// ── Evals (fleet): traces + scores + progress ──
if rows, err := aiobject.DatastoreQuery(ctx, aimEvalTracesSQL(), sinceTS); err == nil {
fillAimEvalTraces(&payload.Evals, firstRowOr(rows))
}
if rows, err := aiobject.DatastoreQuery(ctx, aimEvalScoresSQL(), sinceTS); err == nil {
fillAimEvalScores(&payload.Evals, firstRowOr(rows))
}
if rows, err := aiobject.DatastoreQuery(ctx, aimScoreNamesSQL(), sinceTS); err == nil {
payload.ScoreNames = scoreNamesFromRows(rows)
}
if rows, err := aiobject.DatastoreQuery(ctx, aimEvalRunsSQL(), sinceTS); err == nil {
payload.EvalRuns = evalRunsFromRows(rows)
}
if rows, err := aiobject.DatastoreQuery(ctx, aimScoreSeriesSQL(interval), sinceTS); err == nil {
payload.ScoreSeries = scoreSeriesFromRows(rows)
}
return core.OK(c, payload)
}
// ── pure SQL builders (static SQL + one positional time bound; unit-tested) ──
func aimLangfuseTotalsSQL() string {
return "SELECT count() AS gens, toFloat64(sum(total_cost)) AS cost FROM " + aimLangfuseObs +
" WHERE type = 'GENERATION' AND start_time >= ?"
}
func aimLangfuseLatencySQL() string {
lat := "(toUnixTimestamp64Milli(end_time) - toUnixTimestamp64Milli(start_time))"
return "SELECT round(avg(" + lat + "), 2) AS lat_avg, round(quantile(0.95)(" + lat + "), 2) AS lat_p95 " +
"FROM " + aimLangfuseObs + " WHERE type = 'GENERATION' AND start_time >= ? AND end_time > start_time"
}
func aimLangfuseModelsSQL() string {
return "SELECT provided_model_name AS model, count() AS gens, toFloat64(sum(total_cost)) AS cost " +
"FROM " + aimLangfuseObs + " WHERE type = 'GENERATION' AND start_time >= ? AND provided_model_name != '' " +
"GROUP BY model ORDER BY gens DESC LIMIT " + strconv.Itoa(aimTopN)
}
func aimUsageTotalsSQL() string {
return "SELECT count() AS requests, sum(total_tokens) AS tokens, " +
"sum(prompt_tokens) AS prompt_tokens, sum(completion_tokens) AS completion_tokens, " +
"sum(cost_cents) AS cost_cents, uniqExact(model) AS models " +
"FROM " + aimUsageTable + " WHERE timestamp >= ?"
}
func aimTopModelsSQL() string {
return "SELECT model, count() AS requests, sum(total_tokens) AS tokens, " +
"sum(cost_cents) AS cost_cents FROM " + aimUsageTable +
" WHERE timestamp >= ? AND model != '' GROUP BY model ORDER BY requests DESC LIMIT " + strconv.Itoa(aimTopN)
}
func aimEvalTracesSQL() string {
lat := "(toUnixTimestamp64Milli(end_time) - toUnixTimestamp64Milli(start_time))"
return "SELECT count() AS traces, uniqExact(run_name) AS runs, uniqExact(dataset) AS datasets, " +
"uniqExact(model) AS models, round(avgIf(" + lat + ", end_time > start_time), 2) AS lat_avg " +
"FROM " + aimEvalTraces + " WHERE ts >= ?"
}
func aimEvalScoresSQL() string {
return "SELECT count() AS scores, round(avg(value), 4) AS avg_value, uniqExact(name) AS score_names " +
"FROM " + aimEvalScores + " WHERE ts >= ?"
}
func aimScoreNamesSQL() string {
return "SELECT name, count() AS n, round(avg(value), 4) AS avg_value, " +
"round(min(value), 4) AS min_value, round(max(value), 4) AS max_value " +
"FROM " + aimEvalScores + " WHERE ts >= ? AND name != '' GROUP BY name ORDER BY n DESC LIMIT " + strconv.Itoa(aimTopN)
}
func aimEvalRunsSQL() string {
return "SELECT run_name, any(dataset) AS dataset, count() AS scores, round(avg(value), 4) AS avg_value, " +
"max(ts) AS last_ts FROM " + aimEvalScores + " WHERE ts >= ? AND run_name != '' " +
"GROUP BY run_name ORDER BY last_ts DESC LIMIT " + strconv.Itoa(aimTopN)
}
func aimScoreSeriesSQL(interval string) string {
return "SELECT toStartOfInterval(ts, INTERVAL " + interval + ") AS ts, " +
"round(avg(value), 4) AS avg_value, count() AS n FROM " + aimEvalScores +
" WHERE ts >= ? GROUP BY ts ORDER BY ts"
}
// ── pure row parsers (unit-tested) ──
func fillAimUsage(u *aimUsage, r map[string]any) {
u.Requests = chInt64(r["requests"])
u.Tokens = chInt64(r["tokens"])
u.PromptTokens = chInt64(r["prompt_tokens"])
u.CompletionTokens = chInt64(r["completion_tokens"])
u.CostCents = chInt64(r["cost_cents"])
u.Models = chInt64(r["models"])
}
func fillAimEvalTraces(e *aimEvals, r map[string]any) {
e.Traces = chInt64(r["traces"])
e.Runs = chInt64(r["runs"])
e.Datasets = chInt64(r["datasets"])
e.Models = chInt64(r["models"])
e.LatencyMsAvg = chFloat64(r["lat_avg"])
}
func fillAimEvalScores(e *aimEvals, r map[string]any) {
e.Scores = chInt64(r["scores"])
e.AvgScore = chFloat64(r["avg_value"])
e.ScoreNames = chInt64(r["score_names"])
}
func aimModelsFromRows(rows []map[string]any) []aimModelStat {
out := make([]aimModelStat, 0, len(rows))
for _, r := range rows {
out = append(out, aimModelStat{
Model: chStr(r["model"]),
Requests: chInt64(r["requests"]),
Tokens: chInt64(r["tokens"]),
CostCents: chInt64(r["cost_cents"]),
})
}
return out
}
func lfModelsFromRows(rows []map[string]any) []aimLfModelStat {
out := make([]aimLfModelStat, 0, len(rows))
for _, r := range rows {
out = append(out, aimLfModelStat{
Model: chStr(r["model"]),
Generations: chInt64(r["gens"]),
CostUsd: chFloat64(r["cost"]),
})
}
return out
}
func scoreNamesFromRows(rows []map[string]any) []aimScoreStat {
out := make([]aimScoreStat, 0, len(rows))
for _, r := range rows {
out = append(out, aimScoreStat{
Name: chStr(r["name"]),
Count: chInt64(r["n"]),
AvgValue: chFloat64(r["avg_value"]),
MinValue: chFloat64(r["min_value"]),
MaxValue: chFloat64(r["max_value"]),
})
}
return out
}
func evalRunsFromRows(rows []map[string]any) []aimRunStat {
out := make([]aimRunStat, 0, len(rows))
for _, r := range rows {
out = append(out, aimRunStat{
RunName: chStr(r["run_name"]),
Dataset: chStr(r["dataset"]),
Scores: chInt64(r["scores"]),
AvgValue: chFloat64(r["avg_value"]),
LastTs: chTime(r["last_ts"]),
})
}
return out
}
func scoreSeriesFromRows(rows []map[string]any) []aimScorePoint {
out := make([]aimScorePoint, 0, len(rows))
for _, r := range rows {
out = append(out, aimScorePoint{
Ts: chTime(r["ts"]),
AvgValue: chFloat64(r["avg_value"]),
Count: chInt64(r["n"]),
})
}
return out
}
+171
View File
@@ -0,0 +1,171 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package admin
import (
"strings"
"testing"
)
// TestAimSQL_ReadsCanonicalTables proves every AI-metrics query reads the ONE
// datastore's canonical table, binds the time bound as a POSITIONAL param (one
// `?`), and never interpolates user input. The bucket interval is the only rendered
// value in the series query and it is a server-side constant.
func TestAimSQL_ReadsCanonicalTables(t *testing.T) {
cases := []struct {
name, sql, table string
wantQMarks int
}{
{"langfuseTotals", aimLangfuseTotalsSQL(), "langfuse.observations", 1},
{"langfuseLatency", aimLangfuseLatencySQL(), "langfuse.observations", 1},
{"langfuseModels", aimLangfuseModelsSQL(), "langfuse.observations", 1},
{"usageTotals", aimUsageTotalsSQL(), "hanzo.cloud_usage", 1},
{"topModels", aimTopModelsSQL(), "hanzo.cloud_usage", 1},
{"evalTraces", aimEvalTracesSQL(), "hanzo.eval_traces", 1},
{"evalScores", aimEvalScoresSQL(), "hanzo.eval_scores", 1},
{"scoreNames", aimScoreNamesSQL(), "hanzo.eval_scores", 1},
{"evalRuns", aimEvalRunsSQL(), "hanzo.eval_scores", 1},
{"scoreSeries", aimScoreSeriesSQL("1 DAY"), "hanzo.eval_scores", 1},
}
for _, c := range cases {
if !strings.Contains(c.sql, "FROM "+c.table) {
t.Errorf("%s must read %s; got %q", c.name, c.table, c.sql)
}
if n := strings.Count(c.sql, "?"); n != c.wantQMarks {
t.Errorf("%s: %d bind params, want %d (time bound only) — no interpolation; got %q", c.name, n, c.wantQMarks, c.sql)
}
}
}
// TestAimLangfuseScopedToGeneration proves the Langfuse lens is scoped to
// generations only (not spans/events), matching the o11y LLM lens.
func TestAimLangfuseScopedToGeneration(t *testing.T) {
for _, sql := range []string{aimLangfuseTotalsSQL(), aimLangfuseLatencySQL(), aimLangfuseModelsSQL()} {
if !strings.Contains(sql, "type = 'GENERATION'") {
t.Errorf("langfuse lens must scope to GENERATION observations; got %q", sql)
}
}
}
// TestAimTop_LimitAndOrder proves the leaderboards bound + order the result.
func TestAimTop_LimitAndOrder(t *testing.T) {
if !strings.Contains(aimTopModelsSQL(), "ORDER BY requests DESC LIMIT 12") {
t.Errorf("topModels must order by requests desc, limit %d", aimTopN)
}
if !strings.Contains(aimScoreNamesSQL(), "GROUP BY name ORDER BY n DESC LIMIT 12") {
t.Errorf("scoreNames must group+order+limit %d", aimTopN)
}
if !strings.Contains(aimEvalRunsSQL(), "ORDER BY last_ts DESC LIMIT 12") {
t.Errorf("evalRuns must order by last_ts desc, limit %d", aimTopN)
}
}
// TestAimScoreSeries_IntervalBound proves the (constant) bucket interval is
// rendered into the score-trend series query and grouped/ordered by the bucket.
func TestAimScoreSeries_IntervalBound(t *testing.T) {
for _, iv := range []string{"1 HOUR", "6 HOUR", "1 DAY"} {
s := aimScoreSeriesSQL(iv)
if !strings.Contains(s, "INTERVAL "+iv) || !strings.Contains(s, "GROUP BY ts ORDER BY ts") {
t.Errorf("score series must bucket by INTERVAL %s; got %q", iv, s)
}
}
}
// TestAimEvalLatencyGuarded proves the latency expressions guard end_time>start_time
// so a zero/default end_time never contributes a garbage (negative) latency.
func TestAimEvalLatencyGuarded(t *testing.T) {
if !strings.Contains(aimEvalTracesSQL(), "end_time > start_time") {
t.Errorf("eval traces latency must guard end_time>start_time; got %q", aimEvalTracesSQL())
}
if !strings.Contains(aimLangfuseLatencySQL(), "end_time > start_time") {
t.Errorf("langfuse latency must guard end_time>start_time; got %q", aimLangfuseLatencySQL())
}
}
// TestFillAimUsage reads a cloud_usage row into the KPI band across the numeric
// variants the driver returns (uint64/int64/float64), honest zeros on an empty row.
func TestFillAimUsage(t *testing.T) {
var empty aimUsage
fillAimUsage(&empty, map[string]any{})
if empty.Requests != 0 || empty.Tokens != 0 || empty.Models != 0 {
t.Fatalf("empty row must yield honest zeros; got %+v", empty)
}
var got aimUsage
fillAimUsage(&got, map[string]any{
"requests": uint64(274), "tokens": uint64(102597), "prompt_tokens": uint64(60000),
"completion_tokens": uint64(42597), "cost_cents": uint64(216), "models": uint64(42),
})
if got.Requests != 274 || got.Tokens != 102597 || got.CostCents != 216 || got.Models != 42 {
t.Fatalf("usage totals mis-parsed: %+v", got)
}
}
// TestFillAimEvals maps both eval halves (traces + scores) into the KPI band,
// including the float latency/score columns (round()/avg() land as float64; a
// Decimal-as-string is parsed).
func TestFillAimEvals(t *testing.T) {
var e aimEvals
fillAimEvalTraces(&e, map[string]any{
"traces": uint64(1280), "runs": uint64(16), "datasets": uint64(4),
"models": uint64(6), "lat_avg": float64(842.5),
})
fillAimEvalScores(&e, map[string]any{
"scores": uint64(1280), "avg_value": "0.8125", "score_names": uint64(3),
})
if e.Traces != 1280 || e.Runs != 16 || e.Datasets != 4 || e.Models != 6 || e.LatencyMsAvg != 842.5 {
t.Fatalf("eval traces mis-parsed: %+v", e)
}
if e.Scores != 1280 || e.ScoreNames != 3 || e.AvgScore != 0.8125 { // string→float64 path
t.Fatalf("eval scores mis-parsed: %+v", e)
}
}
// TestAimParsers map datastore rows into the view-models and preserve order (the
// SQL already ORDER BYs; a parser must not reorder or drop rows), with empty input
// yielding an empty (non-nil) slice rather than a panic.
func TestAimParsers(t *testing.T) {
models := aimModelsFromRows([]map[string]any{
{"model": "glm-5.2", "requests": uint64(154), "tokens": uint64(38966), "cost_cents": uint64(114)},
{"model": "deepseek-v4-flash", "requests": uint64(118), "tokens": uint64(61550), "cost_cents": uint64(101)},
})
if len(models) != 2 || models[0].Model != "glm-5.2" || models[1].Model != "deepseek-v4-flash" || models[0].Requests != 154 {
t.Fatalf("top models mis-parsed/reordered: %+v", models)
}
lf := lfModelsFromRows([]map[string]any{
{"model": "gpt-4o", "gens": uint64(42), "cost": float64(1.25)},
})
if len(lf) != 1 || lf[0].Model != "gpt-4o" || lf[0].Generations != 42 || lf[0].CostUsd != 1.25 {
t.Fatalf("langfuse models mis-parsed: %+v", lf)
}
names := scoreNamesFromRows([]map[string]any{
{"name": "accuracy", "n": uint64(320), "avg_value": float64(0.82), "min_value": float64(0), "max_value": float64(1)},
})
if len(names) != 1 || names[0].Name != "accuracy" || names[0].Count != 320 || names[0].AvgValue != 0.82 || names[0].MaxValue != 1 {
t.Fatalf("score names mis-parsed: %+v", names)
}
runs := evalRunsFromRows([]map[string]any{
{"run_name": "nightly-2026-07", "dataset": "gsm8k", "scores": uint64(200), "avg_value": float64(0.9), "last_ts": nil},
})
if len(runs) != 1 || runs[0].RunName != "nightly-2026-07" || runs[0].Dataset != "gsm8k" || runs[0].Scores != 200 || runs[0].AvgValue != 0.9 {
t.Fatalf("eval runs mis-parsed: %+v", runs)
}
// Empty input → empty (non-nil) slices, never a panic.
if got := scoreSeriesFromRows(nil); got == nil || len(got) != 0 {
t.Errorf("nil rows must yield empty slice, got %v", got)
}
if got := aimModelsFromRows(nil); got == nil || len(got) != 0 {
t.Errorf("nil rows must yield empty slice, got %v", got)
}
}
+273
View File
@@ -295,6 +295,237 @@ func (c *Client) Deposit(ctx context.Context, subject string, amount money.Cents
return out, nil
}
// ── SaaS-metrics god-view (fleet-wide, org-independent) ──────────────────────
// SaaSMetrics mirrors commerce's GET /v1/metrics/saas snapshot — the whole-business
// SaaS-operations aggregate (MRR/ARR, new/churn, plan mix, top customers, recent
// movements) computed IN commerce across every org namespace. It is org-INDEPENDENT
// (like Costs) so the reader sends NO subject. Only the fields the admin god-view
// renders are modeled; commerce fields we don't consume (upgrades/downgrades,
// untagged-request counts) are simply ignored by the decoder.
type SaaSMetrics struct {
AsOf string `json:"asOf"`
Currency string `json:"currency"`
Window string `json:"window"`
Revenue SaaSRevenue `json:"revenue"`
Subs SaaSSubs `json:"subscriptions"`
Usage SaaSUsage `json:"usage"`
Customers []SaaSCustomer `json:"customers"`
Orgs int `json:"orgs"`
Gaps []string `json:"gaps"`
}
// SaaSRevenue is the recurring-revenue headline (run-rate MRR/ARR + windowed movement).
type SaaSRevenue struct {
MRRCents money.Cents `json:"mrrCents"`
ARRCents money.Cents `json:"arrCents"`
ActiveSubscriptions int `json:"activeSubscriptions"`
PayingCustomers int `json:"payingCustomers"`
Trials int `json:"trials"`
NewMRRCents money.Cents `json:"newMrrCents"`
ChurnedMRRCents money.Cents `json:"churnedMrrCents"`
NetNewMRRCents money.Cents `json:"netNewMrrCents"`
ByCategory []SaaSCategory `json:"byCategory"`
}
// SaaSCategory is one plan-category bucket of run-rate MRR (the plan mix).
type SaaSCategory struct {
Category string `json:"category"`
MRRCents money.Cents `json:"mrrCents"`
Subscriptions int `json:"subscriptions"`
}
// SaaSSubs is the subscription-operations panel (per-plan mix, trials, new/canceled,
// recent movements).
type SaaSSubs struct {
ByPlan []SaaSPlan `json:"byPlan"`
TrialsActive int `json:"trialsActive"`
New int `json:"new"`
Canceled int `json:"canceled"`
Recent []SaaSEvent `json:"recent"`
}
// SaaSPlan is one plan's active/trialing counts, seats, and MRR contribution.
type SaaSPlan struct {
Plan string `json:"plan"`
Name string `json:"name"`
Category string `json:"category"`
Active int `json:"active"`
Trialing int `json:"trialing"`
Seats int `json:"seats"`
MRRCents money.Cents `json:"mrrCents"`
}
// SaaSEvent is one recent subscription movement ("created" or "canceled").
type SaaSEvent struct {
At string `json:"at"`
Org string `json:"org"`
Type string `json:"type"`
Plan string `json:"plan"`
Category string `json:"category"`
MRRDeltaCents money.Cents `json:"mrrDeltaCents"`
}
// SaaSUsage is the metered / pay-as-you-go revenue headline for the window.
type SaaSUsage struct {
Instrumented bool `json:"instrumented"`
WindowUsageCents money.Cents `json:"windowUsageCents"`
Requests int64 `json:"requests"`
}
// SaaSCustomer is one top customer by MRR + windowed usage.
type SaaSCustomer struct {
Org string `json:"org"`
Plan string `json:"plan"`
Category string `json:"category"`
Status string `json:"status"`
MRRCents money.Cents `json:"mrrCents"`
UsageCents money.Cents `json:"usageCents"`
Seats int `json:"seats"`
Since string `json:"since,omitempty"`
}
// Metrics reads the fleet SaaS-operations god-view (GET /v1/metrics/saas). Like Costs it
// is org-INDEPENDENT — the engine walks every org namespace itself — so it authenticates
// with the admin S2S service token and sends NO subject. Empty (not an error) when
// commerce is unwired, so a partial deploy degrades to an honest empty snapshot.
func (c *Client) Metrics(ctx context.Context, window string, limit int) (SaaSMetrics, error) {
var out SaaSMetrics
if !c.Ready() {
return out, nil
}
q := url.Values{}
if window != "" {
q.Set("window", window)
}
if limit > 0 {
q.Set("limit", fmt.Sprintf("%d", limit))
}
body, err := c.get(ctx, "/v1/metrics/saas", q, "")
if err != nil {
return out, err
}
if err := json.Unmarshal(body, &out); err != nil {
return out, fmt.Errorf("commerce metrics decode: %w", err)
}
return out, nil
}
// ── billing invoices + subscriptions (per-subject fleet rows) ────────────────
// Invoice is one issued invoice as the fleet god-view renders it: the id (for a future
// /v1/billing/invoices/:id detail fetch), the human number, status, amount due,
// currency, and the issue/due dates. Sourced from GET /v1/billing/invoices
// (invoiceResponse); all timestamps are RFC3339 strings.
type Invoice struct {
ID string `json:"id"`
Number string `json:"numberStr"`
Status string `json:"status"`
AmountDue money.Cents `json:"amountDue"`
Currency string `json:"currency"`
Issued string `json:"createdAt"`
Due string `json:"dueDate"`
}
// Invoices lists a subject's invoices (GET /v1/billing/invoices), optionally filtered by
// status. The subject selects the org's billing namespace via X-Org-Id (trusted only
// after the service-token bearer verifies). Empty (not an error) when commerce is unwired.
func (c *Client) Invoices(ctx context.Context, subject, status string) ([]Invoice, error) {
if !c.Ready() {
return nil, nil
}
q := url.Values{}
if status != "" {
q.Set("status", status)
}
body, err := c.get(ctx, "/v1/billing/invoices", q, subject)
if err != nil {
return nil, err
}
var wrap struct {
Invoices []Invoice `json:"invoices"`
}
if err := json.Unmarshal(body, &wrap); err != nil {
return nil, fmt.Errorf("commerce invoices decode: %w", err)
}
return wrap.Invoices, nil
}
// Subscription is one subscription row the fleet god-view renders: the id, the buyer
// (userId), plan tier, status, monthly-normalized MRR, and the current-period
// start/end (started/renews). MRR reuses monthlyNormalized so a yearly plan is
// comparable to a monthly one in the fleet total.
type Subscription struct {
ID string `json:"id"`
User string `json:"user"`
Plan string `json:"plan"`
Status string `json:"status"`
MRR money.Cents `json:"mrrCents"`
Started string `json:"started"`
Renews string `json:"renews"`
}
// subscriptionRowWire is the /v1/billing/subscriptions row shape the fleet view folds —
// richer than subscriptionsWire (which Plan() uses for the MRR sum alone).
type subscriptionRowWire struct {
ID string `json:"id"`
UserID string `json:"userId"`
PlanID string `json:"planId"`
Status string `json:"status"`
Created string `json:"createdAt"`
PeriodStart string `json:"currentPeriodStart"`
PeriodEnd string `json:"currentPeriodEnd"`
Plan struct {
Name string `json:"name"`
Price money.Cents `json:"price"`
Interval string `json:"interval"`
} `json:"plan"`
}
// Subscriptions lists a subject's subscriptions (GET /v1/billing/subscriptions),
// optionally filtered by status, as fleet rows with a monthly-normalized MRR. Empty (not
// an error) when commerce is unwired.
func (c *Client) Subscriptions(ctx context.Context, subject, status string) ([]Subscription, error) {
if !c.Ready() {
return nil, nil
}
q := url.Values{}
if status != "" {
q.Set("status", status)
}
body, err := c.get(ctx, "/v1/billing/subscriptions", q, subject)
if err != nil {
return nil, err
}
var wrap struct {
Subscriptions []subscriptionRowWire `json:"subscriptions"`
}
if err := json.Unmarshal(body, &wrap); err != nil {
return nil, fmt.Errorf("commerce subscriptions decode: %w", err)
}
out := make([]Subscription, 0, len(wrap.Subscriptions))
for _, s := range wrap.Subscriptions {
name := strings.TrimSpace(s.Plan.Name)
if name == "" {
name = strings.TrimSpace(s.PlanID)
}
started := strings.TrimSpace(s.Created)
if started == "" {
started = s.PeriodStart
}
out = append(out, Subscription{
ID: s.ID,
User: s.UserID,
Plan: name,
Status: s.Status,
MRR: monthlyNormalized(s.Plan.Price, s.Plan.Interval),
Started: started,
Renews: s.PeriodEnd,
})
}
return out, nil
}
// post performs one admin-authenticated commerce POST (JSON body) and returns the
// raw response. The admin S2S service token is the bearer and X-Org-Id=<subject>
// the per-org namespace selector commerce's EdgeAuth trusts only after verifying
@@ -332,6 +563,48 @@ func (c *Client) post(ctx context.Context, path, subject string, body []byte, id
return respBody, nil
}
// Forward proxies an admin-authenticated request to commerce VERBATIM and returns
// the raw body + status. It is the ONE seam a SuperAdmin surface drives commerce's
// own endpoints through — the platform plan-promo config (/v1/platform/promo) and a
// per-org spend-alert override (/v1/billing/spend-alerts) — without a typed method
// per shape. subject is the X-Org-Id namespace selector (the target org for a cap
// override, or the admin org for platform config); body is nil for GET/DELETE. The
// status is returned so the caller surfaces commerce's OWN verdict (400 validation,
// 403, 404) instead of flattening every non-2xx into one code.
func (c *Client) Forward(ctx context.Context, method, path, subject string, body []byte) ([]byte, int, error) {
if !c.Ready() {
return nil, 0, errUnconfigured
}
var rdr io.Reader
if body != nil {
rdr = bytes.NewReader(body)
}
req, err := http.NewRequestWithContext(ctx, method, c.base+path, rdr)
if err != nil {
return nil, 0, err
}
req.Header.Set("Accept", "application/json")
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
if c.token != "" {
req.Header.Set("Authorization", "Bearer "+c.token)
}
if subject != "" {
req.Header.Set("X-Org-Id", subject)
}
resp, err := c.http.Do(req)
if err != nil {
return nil, 0, fmt.Errorf("commerce unreachable: %w", err)
}
defer resp.Body.Close()
raw, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return nil, resp.StatusCode, err
}
return raw, resp.StatusCode, nil
}
// get performs one admin-authenticated commerce GET and returns the raw body.
func (c *Client) get(ctx context.Context, path string, q url.Values, subject string) ([]byte, error) {
u := c.base + path
+28 -12
View File
@@ -18,9 +18,8 @@ import (
"github.com/hanzoai/cloud/audit"
"github.com/hanzoai/cloud/clients/admin/money"
"github.com/hanzoai/cloud/clients/finance"
finmoney "github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/cloud/clients/principal"
"github.com/hanzoai/cloud/types"
"github.com/hanzoai/commerce/billing/creditledger"
"github.com/zap-proto/zip"
)
@@ -183,22 +182,39 @@ func ApplyGrant(s *cloud.Service[State], c *zip.Ctx, org string, req CreditReque
// one shape regardless of which path moved the money.
func grantDeposit(s *cloud.Service[State], c *zip.Ctx, org, currency, notes, tag, source string, amountCents int64) (before int64, txID string, after int64, afterExact string, err error) {
ctx := c.Context()
if fin := finance.Current(); fin != nil {
if bal, berr := fin.Balance(ctx, org, org, currency, false); berr == nil {
before = bal.Cents()
// ONE credit path: prefer the in-proc commerce credit ledger (creditledger) — the
// SAME injected ledger adapter commerce's POST /v1/billing/credit mints through
// and the ai prepaid gate reads. An admin grant and a self-serve credit thus move
// money the ONE way, into the ONE ledger; the admin path no longer carries its own
// parallel finance.Deposit. The operator-nonce idempotency key rides through so a
// retried grant dedupes (finance dedups on Ref). Before/after balances are read from
// the SAME co-resident finance ledger for the audit trail (exact, sub-cent visible).
if led := creditledger.Get(); led != nil {
if fin := finance.Current(); fin != nil {
if bal, berr := fin.Balance(ctx, org, org, currency, false); berr == nil {
before = bal.Cents()
}
}
id, derr := fin.Deposit(ctx, types.DepositInput{
Org: org, Subject: org, Amount: finmoney.FromCents(amountCents), Currency: currency, Notes: notes, Tags: tag,
id, balCents, cerr := led.Credit(ctx, creditledger.CreditInput{
Org: org,
Currency: currency,
Reason: notes,
Tag: tag,
IdempotencyKey: grantIdempotencyKey(c, org, currency, source, amountCents),
AmountCents: amountCents,
})
if derr != nil {
return before, "", before, "", derr
if cerr != nil {
return before, "", before, "", cerr
}
if bal, berr := fin.Balance(ctx, org, org, currency, false); berr == nil {
after, afterExact = bal.Cents(), bal.IntString() // afterExact = the EXACT balance (sub-cent visible)
after = balCents
if fin := finance.Current(); fin != nil {
if bal, berr := fin.Balance(ctx, org, org, currency, false); berr == nil {
afterExact = bal.AttoString() // afterExact = the EXACT balance (sub-cent visible)
}
}
return before, id, after, afterExact, nil
}
// Split deploy: no co-resident finance ledger → the commerce billing HTTP deposit, with
// Split deploy: no co-resident credit ledger → the commerce billing HTTP deposit, with
// its operator-nonce idempotency key so a retried grant dedupes at commerce.
beforeC, _ := s.State.Commerce.Credits(ctx, org)
idem := grantIdempotencyKey(c, org, currency, source, amountCents)
+36 -14
View File
@@ -2,26 +2,48 @@ package admin
// The PLATFORM CONTROL PLANE board (/v1/admin/flags) — every runtime LAUNCH / RELEASE
// switch (waitlist, public signup, subsystem activation, gateway limits, network ids)
// with its LIVE value, evaluated through the Hanzo Insights feature-flag engine
// (clients/featureflags → insights rust/feature-flags). SuperAdmin only (mounted
// behind s.guard, like every /v1/admin/* route).
// with its LIVE value, evaluated through the embedded native flag engine
// (clients/flags → native/flags, SQLite-per-project definitions + Rust FFI
// evaluation). SuperAdmin only (mounted behind core.Guard, like every /v1/admin/*).
//
// ONE flag engine, not two. Insights OWNS the flag definitions, targeting, percentage
// rollout, and the change/activity log. This endpoint READS the switches for the
// cockpit and hands the operator the deep-links to the Insights flag MANAGER (where a
// switch is toggled / rolled out / cohort-targeted) and its ACTIVITY LOG (the native
// change audit). A flip there is hot — the consuming subsystems re-read within one
// evaluation TTL, no redeploy. The board is read-only here on purpose: management is
// the native Insights UI (the one-and-one-way flag surface), surfaced in the cockpit.
// ONE flag engine, TWO verbs. GET reads the board; PUT writes a switch's definition
// through flags.SetPlatformSwitch — the ONE write path, audited in the store's
// activity log. A flip is hot: this pod applies immediately, peers converge within one
// evaluation TTL (default 15s), no redeploy. Org/project product flags are managed on
// /v1/flags (org-scoped); this surface is the platform's own switchboard.
import (
"encoding/json"
"strings"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/featureflags"
"github.com/hanzoai/cloud/clients/flags"
"github.com/zap-proto/zip"
)
// flags answers GET /v1/admin/flags — the platform control-plane read board.
func flags(s *cloud.Service[core.State], c *zip.Ctx) error {
return core.OK(c, featureflags.Board())
// flagsBoard answers GET /v1/admin/flags — the platform control-plane read board.
func flagsBoard(s *cloud.Service[core.State], c *zip.Ctx) error {
return core.OK(c, flags.Board())
}
// setFlag answers PUT /v1/admin/flags/:key — store/overwrite one platform switch's
// definition. The body is the flag definition JSON; the two common shapes:
//
// {"active": true} — boolean switch on/off
// {"active": true, "filters": {"groups": [{"properties": [], "rollout_percentage": 100}],
// "payloads": {"true": 250}}} — valued switch (int/string payload)
func setFlag(s *cloud.Service[core.State], c *zip.Ctx) error {
key := strings.TrimSpace(c.Param("key"))
if key == "" {
return zip.ErrBadRequest("key is required")
}
body := c.Body()
if len(body) == 0 || !json.Valid(body) {
return zip.ErrBadRequest("body must be the flag definition JSON")
}
if err := flags.SetPlatformSwitch(key, json.RawMessage(body), c.UserEmail()); err != nil {
return zip.ErrBadRequest(err.Error())
}
return core.OK(c, flags.Board())
}
+137
View File
@@ -0,0 +1,137 @@
// Package invoices is the fleet INVOICE view (/v1/admin/invoices) — every issued
// invoice across every tenant: number, org, amount, status, issue + due date, plus the
// id a future detail view fetches /v1/billing/invoices/:id with. SuperAdmin only
// (core.Guard).
//
// Commerce billing is per-tenant (an invoice lives in its org's own datastore
// namespace), so — like revenue — this fans out the org directory concurrently and
// reads each org's invoices via the admin S2S seam, tagging every row with its owning
// org. Best-effort per org: an org whose invoice read fails contributes NO rows rather
// than failing the fleet view (the SAME honest-degradation contract the customer list
// uses; an unreachable commerce yields an empty list, never fabricated rows). Optional
// ?org= scopes to one tenant, ?status= filters, ?limit= caps the merged list.
package invoices
import (
"context"
"sort"
"strconv"
"strings"
"sync"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/admin/iam"
"github.com/zap-proto/zip"
)
// defaultLimit caps the merged fleet invoice list when the caller sends none.
const defaultLimit = 500
// InvoiceRow is one row of GET /v1/admin/invoices — an issued invoice at a glance,
// tagged with its owning org. Money is USD cents; timestamps are RFC3339 strings.
type InvoiceRow struct {
ID string `json:"id"`
Number string `json:"number"`
Org string `json:"org"`
Display string `json:"display"`
Status string `json:"status"`
AmountCents int64 `json:"amountCents"`
Currency string `json:"currency"`
Issued string `json:"issued"`
Due string `json:"due"`
}
// Invoices answers GET /v1/admin/invoices.
//
// GET /v1/admin/invoices?org=&status=&limit=
func Invoices(s *cloud.Service[core.State], c *zip.Ctx) error {
ctx := c.Context()
cr := core.CallerCreds(c)
status := strings.TrimSpace(c.Query("status"))
wantOrg := strings.TrimSpace(c.Query("org"))
limit := parseLimit(c.Query("limit"))
orgs, err := core.ListOrgs(s, ctx, cr)
if err != nil {
return core.Fail(c, err.Error())
}
if wantOrg != "" {
orgs = filterOrg(orgs, wantOrg)
}
// Per-org invoices, fanned out concurrently (best-effort per org).
perOrg := make([][]InvoiceRow, len(orgs))
sem := make(chan struct{}, core.MaxCustomerConcurrency)
var wg sync.WaitGroup
for i, o := range orgs {
wg.Add(1)
sem <- struct{}{}
go func(i int, o iam.Org) {
defer wg.Done()
defer func() { <-sem }()
perOrg[i] = invoicesOf(s, ctx, o, status)
}(i, o)
}
wg.Wait()
rows := make([]InvoiceRow, 0)
for _, r := range perOrg {
rows = append(rows, r...)
}
// Newest issued first; cap to the merged limit (total reports the full pre-cap count).
sort.Slice(rows, func(i, j int) bool { return rows[i].Issued > rows[j].Issued })
total := len(rows)
if len(rows) > limit {
rows = rows[:limit]
}
return core.OKList(c, rows, total)
}
// invoicesOf reads one org's invoices into fleet rows, tagged with the org. Best-effort:
// a failed read yields no rows so the fleet view degrades honestly, never fabricating.
func invoicesOf(s *cloud.Service[core.State], ctx context.Context, o iam.Org, status string) []InvoiceRow {
entries, err := s.State.Commerce.Invoices(ctx, o.Name, status)
if err != nil {
return nil
}
display := core.Display(o.DisplayName, o.Name)
rows := make([]InvoiceRow, 0, len(entries))
for _, inv := range entries {
rows = append(rows, InvoiceRow{
ID: inv.ID,
Number: inv.Number,
Org: o.Name,
Display: display,
Status: inv.Status,
AmountCents: int64(inv.AmountDue),
Currency: inv.Currency,
Issued: inv.Issued,
Due: inv.Due,
})
}
return rows
}
// filterOrg narrows the directory to the one requested org (empty when it does not
// exist — an honest empty list, never a fabricated tenant).
func filterOrg(orgs []iam.Org, want string) []iam.Org {
for _, o := range orgs {
if o.Name == want {
return []iam.Org{o}
}
}
return nil
}
// parseLimit clamps the merged-list cap to [1,5000], defaulting to defaultLimit.
func parseLimit(s string) int {
n, err := strconv.Atoi(strings.TrimSpace(s))
if err != nil || n <= 0 {
return defaultLimit
}
if n > 5000 {
return 5000
}
return n
}
+12
View File
@@ -0,0 +1,12 @@
package invoices
import (
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/zap-proto/zip"
)
// Routes registers the fleet invoice view (SuperAdmin only, cross-tenant).
func Routes(app *zip.App, s *cloud.Service[core.State]) {
app.Get("/v1/admin/invoices", core.Guard(s, Invoices))
}
+143
View File
@@ -0,0 +1,143 @@
package admin
import (
"encoding/json"
"net/http"
"net/url"
"strings"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/zap-proto/zip"
)
// The SuperAdmin usage-cap + promo control plane, twinning /v1/admin/flags. It owns
// no store: it FORWARDS to commerce (the billing source of truth) over the ONE
// service-token seam —
//
// promos → commerce /v1/platform/promo (the admin-configured plan promo)
// spend-caps → commerce /v1/billing/spend-alerts (a per-org usage cap override)
//
// so admin.hanzo.ai configures the 50%-off promo and oversees/overrides any org's
// caps without a parallel model. Promo routes are platform-only (core.Guard); cap
// routes are org-scoped (core.GuardScoped) so a SuperAdmin targets any org via ?org=
// while a lesser admin is hard-pinned to their own.
// limitRoutes registers the promo + cap control plane. Called from routes().
func limitRoutes(app *zip.App, s *cloud.Service[core.State]) {
// Platform plan promo — SuperAdmin only.
app.Get("/v1/admin/promos", core.Guard(s, getPromo))
app.Put("/v1/admin/promos", core.Guard(s, putPromo))
// Per-org usage-cap oversight/override — SuperAdmin (any org via ?org=) or an org
// admin (own org only). Reuses the customer's OWN self-service spend-alert CRUD,
// so a platform override and a customer edit are the same rows.
app.Get("/v1/admin/spend-caps", core.GuardScoped(s, listSpendCaps))
app.Post("/v1/admin/spend-caps", core.GuardScoped(s, createSpendCap))
app.Patch("/v1/admin/spend-caps/:id", core.GuardScoped(s, updateSpendCap))
app.Delete("/v1/admin/spend-caps/:id", core.GuardScoped(s, deleteSpendCap))
}
// getPromo returns the current platform plan promo. X-Org-Id is the admin org —
// commerce stores the singleton in the reserved platform namespace regardless, and
// the service token is what passes commerce's RequirePlatformAdmin.
func getPromo(s *cloud.Service[core.State], c *zip.Ctx) error {
raw, status, err := s.State.Commerce.Forward(c.Context(), http.MethodGet, "/v1/platform/promo", s.State.AdminOrg, nil)
return relay(c, raw, status, err)
}
// putPromo upserts the platform plan promo from the SuperAdmin's {percentOff,start,
// end,plans,active} body — the ONE place the 50%-off offer is configured.
func putPromo(s *cloud.Service[core.State], c *zip.Ctx) error {
raw, status, err := s.State.Commerce.Forward(c.Context(), http.MethodPut, "/v1/platform/promo", s.State.AdminOrg, c.Body())
return relay(c, raw, status, err)
}
// listSpendCaps returns a target org's usage caps (spend-alerts + derived period
// spend/over/warn/resetsAt). The org is the SuperAdmin's ?org= or, for a scoped
// admin, their own — never a client-widened scope.
func listSpendCaps(s *cloud.Service[core.State], c *zip.Ctx) error {
org, ok := targetOrg(s, c)
if !ok {
return core.Fail(c, "org required")
}
raw, status, err := s.State.Commerce.Forward(c.Context(), http.MethodGet, "/v1/billing/spend-alerts", org, nil)
return relay(c, raw, status, err)
}
// createSpendCap sets a cap on a target org (platform override of a customer budget).
func createSpendCap(s *cloud.Service[core.State], c *zip.Ctx) error {
org, ok := targetOrg(s, c)
if !ok {
return core.Fail(c, "org required")
}
raw, status, err := s.State.Commerce.Forward(c.Context(), http.MethodPost, "/v1/billing/spend-alerts", org, c.Body())
return relay(c, raw, status, err)
}
// updateSpendCap edits a target org's cap by id (raise/lower the ceiling, flip enforce).
func updateSpendCap(s *cloud.Service[core.State], c *zip.Ctx) error {
org, ok := targetOrg(s, c)
if !ok {
return core.Fail(c, "org required")
}
id := strings.TrimSpace(c.Param("id"))
if id == "" {
return core.Fail(c, "cap id required")
}
raw, status, err := s.State.Commerce.Forward(c.Context(), http.MethodPatch, "/v1/billing/spend-alerts/"+url.PathEscape(id), org, c.Body())
return relay(c, raw, status, err)
}
// deleteSpendCap removes a target org's cap by id.
func deleteSpendCap(s *cloud.Service[core.State], c *zip.Ctx) error {
org, ok := targetOrg(s, c)
if !ok {
return core.Fail(c, "org required")
}
id := strings.TrimSpace(c.Param("id"))
if id == "" {
return core.Fail(c, "cap id required")
}
raw, status, err := s.State.Commerce.Forward(c.Context(), http.MethodDelete, "/v1/billing/spend-alerts/"+url.PathEscape(id), org, nil)
return relay(c, raw, status, err)
}
// targetOrg resolves which org a cap operation acts on: a SuperAdmin names it with
// ?org=; a scoped admin is hard-pinned to their own subtree (?org= ignored). Empty
// (false) when unresolvable, so the handler fails closed rather than acting on a
// guessed tenant.
func targetOrg(s *cloud.Service[core.State], c *zip.Ctx) (string, bool) {
sc := core.ResolveScope(s, c)
if sc.Super {
if org := strings.TrimSpace(c.Query("org")); org != "" {
return org, true
}
return "", false
}
if len(sc.Orgs) > 0 && strings.TrimSpace(sc.Orgs[0]) != "" {
return sc.Orgs[0], true
}
return "", false
}
// relay surfaces commerce's OWN verdict in the /v1 envelope: a 2xx passes the raw
// JSON through as data (so the console decodes the exact SpendAlert/Promo shape), a
// non-2xx becomes an honest failure carrying commerce's status + message rather than
// masking a 400 validation as success.
func relay(c *zip.Ctx, raw []byte, status int, err error) error {
if err != nil {
return core.Fail(c, err.Error())
}
if status < 200 || status >= 300 {
msg := strings.TrimSpace(string(raw))
if msg == "" {
msg = http.StatusText(status)
}
return core.Fail(c, msg)
}
if len(raw) == 0 {
return core.OK(c, map[string]any{"ok": true})
}
return core.OKRaw(c, json.RawMessage(raw), 0)
}
+120
View File
@@ -0,0 +1,120 @@
package admin
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
)
// recCommerce records the X-Org-Id + method + path of the last forwarded request so a
// test can prove the /v1/admin control plane targets the RIGHT tenant namespace, and
// serves the promo + spend-alert shapes verbatim.
type recCommerce struct {
server *httptest.Server
mu sync.Mutex
lastOrg string
lastMethod string
lastPath string
}
func newRecCommerce() *recCommerce {
f := &recCommerce{}
f.server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
f.lastOrg = r.Header.Get("X-Org-Id")
f.lastMethod = r.Method
f.lastPath = r.URL.Path
f.mu.Unlock()
w.Header().Set("Content-Type", "application/json")
switch {
case strings.HasSuffix(r.URL.Path, "/platform/promo"):
io.WriteString(w, `{"percentOff":50,"plans":["pro"],"active":true}`)
case strings.HasSuffix(r.URL.Path, "/spend-alerts"):
io.WriteString(w, `[{"id":"a1","threshold":10000,"enforce":true,"period":"2026-07","resetsAt":"2026-08-01T00:00:00Z"}]`)
default:
io.WriteString(w, `{}`)
}
}))
return f
}
func (f *recCommerce) seen() (string, string, string) {
f.mu.Lock()
defer f.mu.Unlock()
return f.lastMethod, f.lastPath, f.lastOrg
}
func envStatus(t *testing.T, body []byte) string {
t.Helper()
var e struct {
Status string `json:"status"`
}
_ = json.Unmarshal(body, &e)
return e.Status
}
// The promo control plane is SuperAdmin-only (core.Guard) and forwards to commerce's
// platform-promo endpoint.
func TestLimits_Promo_SuperOnly(t *testing.T) {
iam := newScopeIAM()
defer iam.server.Close()
com := newRecCommerce()
defer com.server.Close()
do := mount(t, iam.server.URL, com.server.URL, "")
// SuperAdmin GET → 200 ok, forwarded to /v1/platform/promo.
resp, body := do("GET", "/v1/admin/promos", superHdr)
if resp.StatusCode != http.StatusOK || envStatus(t, body) != "ok" {
t.Fatalf("super GET promos = %d %s", resp.StatusCode, body)
}
if m, p, _ := com.seen(); m != "GET" || !strings.HasSuffix(p, "/platform/promo") {
t.Fatalf("forwarded %s %s, want GET .../platform/promo", m, p)
}
// SuperAdmin PUT → forwarded as PUT.
if resp, _ := do("PUT", "/v1/admin/promos", superHdr); resp.StatusCode != http.StatusOK {
t.Fatalf("super PUT promos = %d", resp.StatusCode)
}
if m, _, _ := com.seen(); m != "PUT" {
t.Fatalf("promo PUT forwarded as %s, want PUT", m)
}
// A non-super org admin is REFUSED at the platform gate (403), never reaching commerce.
if resp, _ := do("GET", "/v1/admin/promos", orgAdminHdr); resp.StatusCode != http.StatusForbidden {
t.Fatalf("org-admin GET promos = %d, want 403 (platform-only)", resp.StatusCode)
}
}
// Cap oversight is org-scoped: a SuperAdmin targets any org via ?org=; a scoped admin
// is hard-pinned to their OWN org (a client ?org= is ignored — the escalation line).
func TestLimits_SpendCaps_OrgScoped(t *testing.T) {
iam := newScopeIAM()
defer iam.server.Close()
com := newRecCommerce()
defer com.server.Close()
do := mount(t, iam.server.URL, com.server.URL, "")
// SuperAdmin with ?org=maxpower → forwards X-Org-Id=maxpower.
resp, body := do("GET", "/v1/admin/spend-caps?org=maxpower", superHdr)
if resp.StatusCode != http.StatusOK || envStatus(t, body) != "ok" {
t.Fatalf("super spend-caps = %d %s", resp.StatusCode, body)
}
if _, p, org := com.seen(); org != "maxpower" || !strings.HasSuffix(p, "/spend-alerts") {
t.Fatalf("forwarded org=%q path=%q, want maxpower .../spend-alerts", org, p)
}
// SuperAdmin WITHOUT ?org → org required (honest error, no guessed tenant).
if _, body := do("GET", "/v1/admin/spend-caps", superHdr); envStatus(t, body) != "error" {
t.Fatalf("super spend-caps without org must be an error envelope, got %s", body)
}
// A scoped org admin naming a FOREIGN ?org=hanzo is hard-pinned to their OWN org.
do("GET", "/v1/admin/spend-caps?org=hanzo", orgAdminHdr)
if _, _, org := com.seen(); org != "maxpower" {
t.Fatalf("scoped admin forwarded org=%q, want maxpower (client ?org= must be ignored)", org)
}
}
+110
View File
@@ -0,0 +1,110 @@
// Package metrics is the fleet SaaS-operations god-view (/v1/admin/metrics) — the
// operator's business dashboard: MRR/ARR, net-new vs churned MRR, the plan/category
// mix, the top customers, and the recent subscription movements. SuperAdmin only
// (core.Guard).
//
// It OWNS no aggregation. The whole snapshot is computed IN commerce (the system of
// record for subscriptions + the usage ledger) by its cross-org SaaS-metrics engine
// (GET /v1/metrics/saas), which admin PROXIES with the SAME admin-scoped S2S service
// token finance uses for COGS. The engine is ALREADY fleet-wide — it walks every org
// namespace itself — so this is a SINGLE upstream read, no per-org fan-out, exactly as
// finance consumes commerce Costs. An unwired or unreachable commerce degrades to an
// honest empty snapshot (real zeros, `[]` not null) with a not-ok source, never a
// fabricated number.
package metrics
import (
"errors"
"strconv"
"strings"
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/commerce"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/zap-proto/zip"
)
// errUnconfigured marks commerce not wired on this deployment — core.SrcOf reports it as
// a not-ok source so the console renders the honest not-configured state.
var errUnconfigured = errors.New("commerce metrics not configured")
// defaultLimit caps the top-customers list when the caller sends none (mirrors the
// commerce engine's own default so the proxy never asks for more than it returns).
const defaultLimit = 20
// MetricsData is the GET /v1/admin/metrics payload: the commerce SaaS snapshot, flat,
// plus the admin read time and the upstream freshness strip every god-view carries.
type MetricsData struct {
commerce.SaaSMetrics
GeneratedAt string `json:"generatedAt"`
Sources []core.SourceStatus `json:"sources"`
}
// Metrics answers GET /v1/admin/metrics by proxying the commerce SaaS-metrics engine
// (already a fleet-wide cross-org aggregate). SuperAdmin only.
//
// GET /v1/admin/metrics?window=30d&limit=20
func Metrics(s *cloud.Service[core.State], c *zip.Ctx) error {
ctx := c.Context()
now := time.Now().UTC().Format(time.RFC3339)
window := strings.TrimSpace(c.Query("window"))
limit := parseLimit(c.Query("limit"))
if !s.State.Commerce.Ready() {
return core.OK(c, empty(now, window, core.SrcOf("commerce-metrics", errUnconfigured, 0, now)))
}
m, err := s.State.Commerce.Metrics(ctx, window, limit)
if err != nil {
return core.OK(c, empty(now, window, core.SrcOf("commerce-metrics", err, 0, now)))
}
return core.OK(c, MetricsData{
SaaSMetrics: normalize(m),
GeneratedAt: now,
Sources: []core.SourceStatus{core.SrcOf("commerce-metrics", nil, m.Orgs, now)},
})
}
// empty is the honest not-configured/unreachable snapshot: real zeros + empty slices
// (never null, never fabricated) plus the not-ok source.
func empty(now, window string, src core.SourceStatus) MetricsData {
return MetricsData{
SaaSMetrics: normalize(commerce.SaaSMetrics{AsOf: now, Currency: "usd", Window: window}),
GeneratedAt: now,
Sources: []core.SourceStatus{src},
}
}
// normalize replaces nil slices with empty ones so the JSON is honest arrays (`[]`, not
// null) and the console never has to guard a missing collection.
func normalize(m commerce.SaaSMetrics) commerce.SaaSMetrics {
if m.Revenue.ByCategory == nil {
m.Revenue.ByCategory = []commerce.SaaSCategory{}
}
if m.Subs.ByPlan == nil {
m.Subs.ByPlan = []commerce.SaaSPlan{}
}
if m.Subs.Recent == nil {
m.Subs.Recent = []commerce.SaaSEvent{}
}
if m.Customers == nil {
m.Customers = []commerce.SaaSCustomer{}
}
if m.Gaps == nil {
m.Gaps = []string{}
}
return m
}
// parseLimit clamps the top-N cap to [1,200], defaulting to defaultLimit — mirrors the
// commerce engine's clamp exactly.
func parseLimit(s string) int {
n, err := strconv.Atoi(strings.TrimSpace(s))
if err != nil || n <= 0 {
return defaultLimit
}
if n > 200 {
return 200
}
return n
}
+13
View File
@@ -0,0 +1,13 @@
package metrics
import (
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/zap-proto/zip"
)
// Routes registers the SaaS-metrics god-view (SuperAdmin only, cross-tenant business
// aggregate).
func Routes(app *zip.App, s *cloud.Service[core.State]) {
app.Get("/v1/admin/metrics", core.Guard(s, Metrics))
}
+73
View File
@@ -0,0 +1,73 @@
package admin
// The /v1/admin/services board — the launch-control LENS over the waitlist gate, twin
// of /v1/admin/flags. Every hosted service (studio/chat/console/app/api/team + runtime
// onboards) with its LIVE waitlist mode — the switch waitlist.<svc>, evaluated through
// clients/admission (which composes the flag engine one-way). This is the "remove the
// waitlist one service at a time" toggle. SuperAdmin only (core.Guard), like every
// platform /v1/admin/*.
//
// The registry + mode decide + these admin control funcs live in clients/admission,
// the complete launch-gate feature; flags is the pure engine underneath. Per-user
// approval (the second, orthogonal axis) stays IAM's, reached via the existing admin IAM
// proxy — not re-served here.
import (
"errors"
"net/http"
"strings"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/admission"
"github.com/zap-proto/zip"
)
// services answers GET /v1/admin/services — the launch board (every service + live mode).
func services(s *cloud.Service[core.State], c *zip.Ctx) error {
rows, err := admission.ListWaitlistServices(c.Context())
if err != nil {
return zip.Errorf(http.StatusInternalServerError, "list services: %v", err)
}
return core.OK(c, map[string]any{"services": rows})
}
// upsertService answers POST /v1/admin/services — onboard or edit a hosted service so a
// new host is governed WITHOUT a redeploy. A re-register PRESERVES the live switch.
func upsertService(s *cloud.Service[core.State], c *zip.Ctx) error {
var in admission.ServiceInput
if err := c.Bind(&in); err != nil {
return err
}
if strings.TrimSpace(in.Service) == "" {
return zip.ErrBadRequest("service slug is required")
}
view, err := admission.UpsertWaitlistService(c.Context(), in, c.UserEmail())
if err != nil {
return zip.ErrBadRequest(err.Error())
}
return core.OK(c, map[string]any{"service": view})
}
// setServiceMode answers POST /v1/admin/services/:service/mode — flip one service's
// waitlist switch {waitlistMode:bool}. The launch lever; hot, no redeploy.
func setServiceMode(s *cloud.Service[core.State], c *zip.Ctx) error {
service := strings.TrimSpace(c.Param("service"))
if service == "" {
return zip.ErrBadRequest("service is required")
}
var body struct {
WaitlistMode bool `json:"waitlistMode"`
}
if err := c.Bind(&body); err != nil {
return err
}
view, err := admission.SetWaitlistMode(c.Context(), service, body.WaitlistMode, c.UserEmail())
if err != nil {
if errors.Is(err, admission.ErrServiceNotFound) {
return zip.ErrNotFound("service not found: " + service)
}
return zip.Errorf(http.StatusInternalServerError, "set mode: %v", err)
}
return core.OK(c, map[string]any{"service": view})
}
+12
View File
@@ -0,0 +1,12 @@
package subscriptions
import (
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/zap-proto/zip"
)
// Routes registers the fleet subscription view (SuperAdmin only, cross-tenant).
func Routes(app *zip.App, s *cloud.Service[core.State]) {
app.Get("/v1/admin/subscriptions", core.Guard(s, Subscriptions))
}
@@ -0,0 +1,139 @@
// Package subscriptions is the fleet SUBSCRIPTION view (/v1/admin/subscriptions) —
// every tenant's plan subscription: customer/org, plan, status, monthly-normalized MRR,
// and the current-period start/renews. SuperAdmin only (core.Guard).
//
// Like invoices (and revenue) it fans out the org directory concurrently and reads each
// org's subscriptions via the admin S2S seam, tagging every row with its owning org. The
// MRR is monthly-normalized in the commerce reader so a yearly plan is comparable to a
// monthly one. Best-effort per org (a failed read contributes no rows, never fabricated
// ones); optional ?org= scopes to one tenant, ?status= filters, ?limit= caps.
package subscriptions
import (
"context"
"sort"
"strconv"
"strings"
"sync"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/admin/iam"
"github.com/zap-proto/zip"
)
// defaultLimit caps the merged fleet subscription list when the caller sends none.
const defaultLimit = 500
// SubscriptionRow is one row of GET /v1/admin/subscriptions — a tenant's subscription at
// a glance, tagged with its owning org. MRR is USD cents; timestamps are RFC3339 strings.
type SubscriptionRow struct {
ID string `json:"id"`
Org string `json:"org"`
Display string `json:"display"`
User string `json:"user"`
Plan string `json:"plan"`
Status string `json:"status"`
MRRCents int64 `json:"mrrCents"`
Started string `json:"started"`
Renews string `json:"renews"`
}
// Subscriptions answers GET /v1/admin/subscriptions.
//
// GET /v1/admin/subscriptions?org=&status=&limit=
func Subscriptions(s *cloud.Service[core.State], c *zip.Ctx) error {
ctx := c.Context()
cr := core.CallerCreds(c)
status := strings.TrimSpace(c.Query("status"))
wantOrg := strings.TrimSpace(c.Query("org"))
limit := parseLimit(c.Query("limit"))
orgs, err := core.ListOrgs(s, ctx, cr)
if err != nil {
return core.Fail(c, err.Error())
}
if wantOrg != "" {
orgs = filterOrg(orgs, wantOrg)
}
// Per-org subscriptions, fanned out concurrently (best-effort per org).
perOrg := make([][]SubscriptionRow, len(orgs))
sem := make(chan struct{}, core.MaxCustomerConcurrency)
var wg sync.WaitGroup
for i, o := range orgs {
wg.Add(1)
sem <- struct{}{}
go func(i int, o iam.Org) {
defer wg.Done()
defer func() { <-sem }()
perOrg[i] = subscriptionsOf(s, ctx, o, status)
}(i, o)
}
wg.Wait()
rows := make([]SubscriptionRow, 0)
for _, r := range perOrg {
rows = append(rows, r...)
}
// Highest-MRR first (ties broken by most-recent start); cap to the merged limit.
sort.Slice(rows, func(i, j int) bool {
if rows[i].MRRCents != rows[j].MRRCents {
return rows[i].MRRCents > rows[j].MRRCents
}
return rows[i].Started > rows[j].Started
})
total := len(rows)
if len(rows) > limit {
rows = rows[:limit]
}
return core.OKList(c, rows, total)
}
// subscriptionsOf reads one org's subscriptions into fleet rows, tagged with the org.
// Best-effort: a failed read yields no rows so the fleet view degrades honestly.
func subscriptionsOf(s *cloud.Service[core.State], ctx context.Context, o iam.Org, status string) []SubscriptionRow {
entries, err := s.State.Commerce.Subscriptions(ctx, o.Name, status)
if err != nil {
return nil
}
display := core.Display(o.DisplayName, o.Name)
rows := make([]SubscriptionRow, 0, len(entries))
for _, sub := range entries {
rows = append(rows, SubscriptionRow{
ID: sub.ID,
Org: o.Name,
Display: display,
User: sub.User,
Plan: sub.Plan,
Status: sub.Status,
MRRCents: int64(sub.MRR),
Started: sub.Started,
Renews: sub.Renews,
})
}
return rows
}
// filterOrg narrows the directory to the one requested org (empty when it does not
// exist — an honest empty list, never a fabricated tenant).
func filterOrg(orgs []iam.Org, want string) []iam.Org {
for _, o := range orgs {
if o.Name == want {
return []iam.Org{o}
}
}
return nil
}
// parseLimit clamps the merged-list cap to [1,5000], defaulting to defaultLimit.
func parseLimit(s string) int {
n, err := strconv.Atoi(strings.TrimSpace(s))
if err != nil || n <= 0 {
return defaultLimit
}
if n > 5000 {
return 5000
}
return n
}
+222
View File
@@ -0,0 +1,222 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package admission
import (
"context"
"encoding/json"
"io"
"net/http"
"strings"
"sync"
"time"
"github.com/zap-proto/zip"
)
// approvalStatusPending is the ONE value that gates a user. This mirrors IAM's
// object.ApprovalPending (hanzoai/iam object/user.go) and User.IsApproved() —
// approval is FAIL-OPEN: a user is approved unless properties.approvalStatus is
// EXACTLY "pending" (absent / "approved" / "rejected" all read approved via
// IsApproved). Only "pending" holds a user on the waitlist. Keeping the literal
// here (not importing IAM) keeps admission self-contained.
const approvalStatusPending = "pending"
// approvedHeader is the FORWARD-PERFECT path: once IAM carries approvalStatus in
// the token and the gateway mints it as a validated header (the same trust model
// as X-User-IsAdmin), the enforcement points read approval for FREE with no IAM
// round-trip. Until then the resolver falls back to an IAM get-account lookup.
// Values: "true" (approved) / "false" (pending). Any other value → fall through.
const approvedHeader = "X-User-Approved"
// accountLookup fetches a caller's approvalStatus by replaying the caller's own
// credentials to IAM get-account. Injected so the resolver is unit-testable
// without a live IAM. It returns (status, ok): ok=false on any IAM error, which
// the resolver treats FAIL-OPEN (approved) — the documented guard behavior
// (availability over a hard gate when IAM is unreachable).
type accountLookup func(ctx context.Context, cookie, auth string) (status string, ok bool)
// Approvals resolves whether the current caller is off the waitlist. It is the ONE
// approval predicate the native middleware uses, DRY with the @file waitlist-guard
// (both read properties.approvalStatus == "pending"). Resolution order:
//
// 1. global admin (c.IsAdmin()) → approved (admins are never gated)
// 2. validated header X-User-Approved → its bit (forward-perfect, no lookup)
// 3. IAM get-account (caller's creds) → approved unless approvalStatus=="pending"
// — cached per user for ttl; FAIL-OPEN on any IAM error.
type Approvals struct {
lookup accountLookup
ttl time.Duration
mu sync.Mutex
cache map[string]approvalEntry
}
type approvalEntry struct {
approved bool
at time.Time
}
// NewApprovals builds a resolver. iamBase is the in-cluster IAM base
// (e.g. http://iam.hanzo.svc.cluster.local:8000); ttl bounds the per-user cache.
// A zero iamBase yields a resolver whose lookup always fails-open (approved) —
// safe for a deployment where approval is enforced elsewhere (the guard).
func NewApprovals(iamBase string, ttl time.Duration) *Approvals {
if ttl <= 0 {
ttl = 30 * time.Second
}
return &Approvals{
lookup: httpAccountLookup(strings.TrimRight(iamBase, "/")),
ttl: ttl,
cache: map[string]approvalEntry{},
}
}
// newApprovalsWithLookup is the test seam: a resolver over an injected lookup.
func newApprovalsWithLookup(lookup accountLookup, ttl time.Duration) *Approvals {
if ttl <= 0 {
ttl = 30 * time.Second
}
return &Approvals{lookup: lookup, ttl: ttl, cache: map[string]approvalEntry{}}
}
// Approved reports whether the caller is off the waitlist.
func (a *Approvals) Approved(c *zip.Ctx) bool {
// (1) Global admins are ALWAYS approved.
if c.IsAdmin() {
return true
}
// (2) Forward-perfect validated header — no IAM round-trip when present.
switch strings.ToLower(strings.TrimSpace(c.Header(approvedHeader))) {
case "true", "1", "approved":
return true
case "false", "0", "pending":
return false
}
// (3) IAM get-account lookup, cached per user, fail-open on error.
user := strings.TrimSpace(c.User())
if user == "" {
// No validated principal — an unauthenticated caller. The middleware
// resolves login separately; treat as not-approved so an anonymous
// request to a gated host is bounced (never allowed through as approved).
return false
}
if a.lookup == nil {
return true // no lookup wired → fail-open (approval enforced elsewhere)
}
if e, ok := a.get(user); ok {
return e.approved
}
status, ok := a.lookup(c.Context(),
c.Header("Cookie"), c.Header("Authorization"))
if !ok {
// IAM unreachable → FAIL-OPEN (approved). Do NOT cache a fail-open so the
// next request re-probes and a recovered IAM re-gates promptly.
return true
}
approved := strings.TrimSpace(strings.ToLower(status)) != approvalStatusPending
a.put(user, approved)
return approved
}
func (a *Approvals) get(user string) (approvalEntry, bool) {
a.mu.Lock()
defer a.mu.Unlock()
e, ok := a.cache[user]
if !ok || time.Since(e.at) > a.ttl {
return approvalEntry{}, false
}
return e, true
}
func (a *Approvals) put(user string, approved bool) {
a.mu.Lock()
defer a.mu.Unlock()
a.cache[user] = approvalEntry{approved: approved, at: time.Now()}
}
// httpAccountLookup builds the real IAM get-account lookup. It replays the
// caller's Cookie / Authorization to IAM and reads data.properties.approvalStatus
// (the field GetAccount returns via GetMaskedUser). Bounded read + timeout mirror
// the guard's iamGet. Any non-200 / decode error → ok=false (fail-open upstream).
func httpAccountLookup(iamBase string) accountLookup {
if iamBase == "" {
return func(context.Context, string, string) (string, bool) { return "", false }
}
url := iamBase + "/v1/iam/get-account"
return func(ctx context.Context, cookie, auth string) (string, bool) {
ctx, cancel := context.WithTimeout(ctx, 8*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return "", false
}
if cookie != "" {
req.Header.Set("Cookie", cookie)
}
if auth != "" {
req.Header.Set("Authorization", auth)
}
req.Header.Set("Accept", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
return "", false
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
return "", false
}
body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return "", false
}
return approvalStatusFromAccount(body)
}
}
// approvalStatusFromAccount extracts properties.approvalStatus from an IAM
// get-account response. The user object is at the top level or under `data`
// (the casibase { status, data } envelope). Returns ("", false) on an error
// envelope or a missing user (fail-open upstream). An ABSENT approvalStatus is
// returned as "" (ok=true) — which the resolver reads as approved (fail-open,
// matching IsApproved()).
func approvalStatusFromAccount(body []byte) (string, bool) {
type acct struct {
Owner string `json:"owner"`
Properties map[string]string `json:"properties"`
}
var top struct {
Status string `json:"status"`
acct
Data acct `json:"data"`
}
if err := json.Unmarshal(body, &top); err != nil {
return "", false
}
if top.Status == "error" {
return "", false
}
a := top.acct
if a.Owner == "" && top.Data.Owner != "" {
a = top.Data
}
if a.Owner == "" {
return "", false
}
if a.Properties == nil {
return "", true // no properties → approved (fail-open)
}
return a.Properties["approvalStatus"], true
}
+146
View File
@@ -0,0 +1,146 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
// Licensed under the Apache License, Version 2.0.
package admission
import (
"context"
"io"
"net/http/httptest"
"testing"
"time"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
// ctxFor builds a zip.Ctx carrying the given identity headers by driving a
// throwaway app whose one handler captures the ctx.
func ctxWith(t *testing.T, headers map[string]string, fn func(c *zip.Ctx)) {
t.Helper()
app := zip.New(zip.Config{Logger: luxlog.New("test")})
app.Get("/probe", func(c *zip.Ctx) error {
fn(c)
return c.NoContent(204)
})
hr := httptest.NewRequest("GET", "http://x/probe", nil)
for k, v := range headers {
hr.Header.Set(k, v)
}
resp, err := app.Fiber().Test(hr)
if err != nil {
t.Fatalf("probe: %v", err)
}
_, _ = io.Copy(io.Discard, resp.Body)
_ = resp.Body.Close()
}
func TestApprovals_AdminAlwaysApproved(t *testing.T) {
a := newApprovalsWithLookup(func(context.Context, string, string) (string, bool) {
t.Fatal("admin must not trigger an IAM lookup")
return "", false
}, time.Minute)
ctxWith(t, map[string]string{"X-User-IsAdmin": "true", "X-User-Id": "u"}, func(c *zip.Ctx) {
if !a.Approved(c) {
t.Fatal("admin should be approved")
}
})
}
func TestApprovals_ForwardHeaderWins(t *testing.T) {
a := newApprovalsWithLookup(func(context.Context, string, string) (string, bool) {
t.Fatal("header path must not trigger an IAM lookup")
return "", false
}, time.Minute)
ctxWith(t, map[string]string{"X-User-Id": "u", "X-User-Approved": "true"}, func(c *zip.Ctx) {
if !a.Approved(c) {
t.Fatal("X-User-Approved=true should be approved")
}
})
ctxWith(t, map[string]string{"X-User-Id": "u", "X-User-Approved": "false"}, func(c *zip.Ctx) {
if a.Approved(c) {
t.Fatal("X-User-Approved=false should NOT be approved")
}
})
}
func TestApprovals_IAMLookup_PendingGates(t *testing.T) {
calls := 0
a := newApprovalsWithLookup(func(context.Context, string, string) (string, bool) {
calls++
return "pending", true
}, time.Minute)
ctxWith(t, map[string]string{"X-User-Id": "u", "X-Org-Id": "acme"}, func(c *zip.Ctx) {
if a.Approved(c) {
t.Fatal("approvalStatus=pending should NOT be approved")
}
})
// Second call hits the cache (no second lookup).
ctxWith(t, map[string]string{"X-User-Id": "u", "X-Org-Id": "acme"}, func(c *zip.Ctx) {
if a.Approved(c) {
t.Fatal("cached pending should NOT be approved")
}
})
if calls != 1 {
t.Fatalf("IAM lookups = %d, want 1 (cached)", calls)
}
}
func TestApprovals_IAMLookup_ApprovedAndAbsentPass(t *testing.T) {
for _, status := range []string{"approved", "", "rejected"} {
a := newApprovalsWithLookup(func(context.Context, string, string) (string, bool) {
return status, true
}, time.Minute)
ctxWith(t, map[string]string{"X-User-Id": "u", "X-Org-Id": "acme"}, func(c *zip.Ctx) {
if !a.Approved(c) {
t.Fatalf("approvalStatus=%q should be approved (fail-open, only 'pending' gates)", status)
}
})
}
}
func TestApprovals_FailOpenOnIAMError(t *testing.T) {
a := newApprovalsWithLookup(func(context.Context, string, string) (string, bool) {
return "", false // IAM unreachable
}, time.Minute)
ctxWith(t, map[string]string{"X-User-Id": "u", "X-Org-Id": "acme"}, func(c *zip.Ctx) {
if !a.Approved(c) {
t.Fatal("IAM unreachable should FAIL-OPEN (approved) for availability")
}
})
}
func TestApprovals_UnauthenticatedNotApproved(t *testing.T) {
a := newApprovalsWithLookup(func(context.Context, string, string) (string, bool) {
t.Fatal("no lookup for an unauthenticated caller")
return "", false
}, time.Minute)
ctxWith(t, map[string]string{}, func(c *zip.Ctx) {
if a.Approved(c) {
t.Fatal("an unauthenticated caller is not approved")
}
})
}
func TestApprovalStatusFromAccount(t *testing.T) {
cases := []struct {
name string
body string
wantStatus string
wantOK bool
}{
{"top-level pending", `{"owner":"acme","properties":{"approvalStatus":"pending"}}`, "pending", true},
{"data-wrapped approved", `{"status":"ok","data":{"owner":"acme","properties":{"approvalStatus":"approved"}}}`, "approved", true},
{"no properties", `{"owner":"acme"}`, "", true},
{"error envelope", `{"status":"error","msg":"nope"}`, "", false},
{"no owner", `{"properties":{"approvalStatus":"pending"}}`, "", false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got, ok := approvalStatusFromAccount([]byte(tc.body))
if got != tc.wantStatus || ok != tc.wantOK {
t.Fatalf("= (%q,%v), want (%q,%v)", got, ok, tc.wantStatus, tc.wantOK)
}
})
}
}
+266
View File
@@ -0,0 +1,266 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package admission is the launch-control GATE for Hanzo's hosted services — the
// COMPLETE waitlist feature, COMPOSING the ONE flag engine (clients/flags) one-way. It
// owns:
//
// - the host→service registry (registry.go) + the brand seed (waitlist.go),
// - the per-service MODE decide WaitlistModeForHost — a service's mode IS the switch
// waitlist.<svc>, evaluated through the flag engine (flags.Bool),
// - the admin control funcs (List/Set/Upsert) the /v1/admin/services board calls,
// - the guard's public mode read /v1/flags/waitlist, Mount,
// - the native enforcement middleware (Enforce, this file),
// - the per-user approval predicate (Approvals, reused from IAM — approval.go).
//
// flags NEVER imports admission; admission imports flags. The engine is the pure
// (Principal, context) -> verdict primitive; this package is its first composed tenant.
// Enforcement is decomplected into two orthogonal axes:
//
// - PER-SERVICE waitlist mode on|off — the switch waitlist.<svc>, resolved for a
// request host via WaitlistModeForHost (the decide, waitlist.go).
// - PER-USER approvalStatus pending|approved — owned by IAM (approval.go), REUSED.
//
// THE RULE, applied at ONE native enforcement point (Enforce):
//
// if waitlistMode[host] AND NOT user.approved → bounce to the waitlist
// if approved OR mode=off → allow
// unauthenticated → login first
package admission
import (
"context"
"net/http"
"strings"
"github.com/zap-proto/zip"
)
// Enforce is the NATIVE, forward-perfect enforcement point for the waitlist — the
// single in-binary middleware that reads the registry (in-process, no HTTP hop)
// and the caller's approval, and applies THE RULE for every request whose Host is
// a governed service. As product hosts fold into the one-binary cloud, this is the
// ONE enforcement point (the @file waitlist-guard is the interim gate for hosts
// not yet cloud-fronted; both read the SAME registry so a toggle governs both).
//
// THE RULE (per request, on a governed host in waitlist mode):
//
// carries a Hanzo API key (hk-/sk-/…) → allow (paid inference; possession-gated)
// exempt path (health/iam/waitlist) → allow
// unauthenticated → 302 waitlist (browser) / 401 (API)
// waitlist mode OFF (or host un-governed) → allow (c.Next)
// waitlist mode ON AND approved → allow (c.Next)
// waitlist mode ON AND NOT approved → 302 waitlist (browser) / 403 (API)
//
// INTEGRATION POINT — wire in serve.go RIGHT AFTER SanitizeIdentity:
//
// app.Use(IdentityMiddleware(cfg)) // establishes the validated principal
// app.Use(admission.Enforce(admission.EnforceConfig{ WaitlistURL: … })) // ← here
//
// It reads the sanitized X-User-Id / X-User-IsAdmin / X-User-Approved that
// IdentityMiddleware minted, so it MUST run after it and (like BillingGate) before
// the subsystem handlers. It is deliberately NOT wired here — the unified-binary
// agent owns serve.go's boot chain; this package exposes Enforce so the one-line
// app.Use lands without a merge collision. The decide (WaitlistModeForHost) is
// resolved PER REQUEST and fail-opens until the flags engine has mounted, so Enforce
// can be constructed before Mount runs.
//
// WHY NATIVE IS CANONICAL (in-cluster-bypass). The @file edge guard only gates
// traffic arriving THROUGH the ingress — a pod reaching another service's pod
// directly in-cluster bypasses it (a cluster-wide baseline NetworkPolicy + Cilium
// broad-allow union means additive netpols can't seal that). For a waitlist (threat
// model = external users) edge-only is acceptable, but this native middleware is
// FORWARD-PERFECT: when the app IS cloud, the gate is IN the request path, so
// reaching the pod directly STILL hits it — there is no edge to go around. That is
// the reason the native middleware is the canonical enforcement and the @file guard
// is purely interim. It is also STATELESS (it reads the sanitized X-User-* headers,
// sets no cookie), so the multi-apex cookie-domain concern the @file guard must
// handle does not exist here at all.
//
// Paths that must NEVER be gated (health, the waitlist page's own API, auth
// callbacks) are skipped via ExemptPrefixes so enforcement can't lock the platform
// out of its own recovery/observability surface.
type EnforceConfig struct {
// WaitlistURL is where an unapproved / unauthenticated browser is bounced
// (per-brand, e.g. https://waitlist.hanzo.ai). Empty → API-style 403/401 for
// everyone (no redirect target), so enforcement still holds.
WaitlistURL string
// Approvals resolves whether the caller is off the waitlist. When nil, Enforce
// builds one from IAMBase.
Approvals *Approvals
// IAMBase is the in-cluster IAM base used to build Approvals when it is nil.
IAMBase string
// ExemptPrefixes are request-path prefixes never gated (health/metrics/auth).
// A sensible default set is used when empty.
ExemptPrefixes []string
// Gate is THE decide: it resolves whether a request host is in waitlist mode,
// via the ONE policy engine. When nil it is WaitlistModeForHost —
// host→service→waitlist.<svc>. Injected only in tests. Fail-open by contract:
// known=false (unmounted / registry error / un-governed host) → not gated.
Gate func(ctx context.Context, host string) (mode bool, service string, known bool)
}
// defaultExemptPrefixes are the paths enforcement must never touch — HIP-0106
// health, the auth/OIDC handshake, and the waitlist join API itself (so a gated
// user can still submit the waitlist form).
var defaultExemptPrefixes = []string{
"/v1/flags/waitlist", // the guard's public mode read (flags engine)
"/v1/iam/", // auth / OIDC / approval-status / get-account handshake
"/v1/waitlist", // the waitlist join API (a gated user must reach it)
"/health",
"/healthz",
"/__guard/", // the @file guard's own callback surface (defense in depth)
}
// Enforce builds the native enforcement middleware. It is a no-op passthrough when the
// decide reports the host is not governed (gate known=false — the flags registry not
// mounted yet, a store error, or an un-governed host), so a request before boot
// completes is never wrongly gated.
func Enforce(cfg EnforceConfig) zip.Handler {
approvals := cfg.Approvals
if approvals == nil {
approvals = NewApprovals(cfg.IAMBase, 0)
}
gate := cfg.Gate
if gate == nil {
gate = WaitlistModeForHost // the ONE decide: host→service→waitlist.<svc>
}
exempt := cfg.ExemptPrefixes
if len(exempt) == 0 {
exempt = defaultExemptPrefixes
}
waitlistURL := strings.TrimRight(strings.TrimSpace(cfg.WaitlistURL), "/")
return func(c *zip.Ctx) error {
path := c.Path()
for _, p := range exempt {
if strings.HasPrefix(path, p) {
return c.Next()
}
}
// MONEY-CRITICAL EXEMPTION: a request bearing a Hanzo API KEY (hk-/sk-/pk-/…)
// is NEVER waitlist-gated. Paid inference on api.hanzo.ai authenticates by KEY
// POSSESSION + is metered downstream in the `ai` subsystem — SanitizeIdentity
// does NOT mint a session principal for an API key (auth_identity.go isAPIKey →
// validatedPrincipal returns nil), so an API-key request arrives here with an
// empty c.User(); without this exemption THE RULE would misclassify it as
// "unauthenticated API → 401" and break every paid inference call. The gate is
// redundant anyway: an API key is minted ONLY in the (gated) console, so an
// unapproved user can never obtain one. Session/JWT access to a gated host is
// still gated normally — only key possession is exempt.
if carriesAPIKey(c) {
return c.Next()
}
mode, _, known := gate(c.Context(), c.Fiber().Hostname())
if !known || !mode {
// Un-governed host, mode OFF, or a registry read error (the decide folds
// all three into known=false) → allow. A governed host is opened by
// flipping its waitlist.<svc> switch OFF; an unknown host is not ours to
// gate at the shared cloud edge — the @file guard is the belt-and-braces
// gate for the hosts that must stay closed.
return c.Next()
}
// Governed host in waitlist mode. Approved (incl. admins) pass; everyone
// else is bounced.
if approvals.Approved(c) {
return c.Next()
}
return bounce(c, waitlistURL)
}
}
// bounce renders the not-approved verdict: a browser navigation is redirected to
// the waitlist (302); an API client gets a 403 (401 when there is no principal at
// all). Content-negotiated so a fetch/XHR never eats an opaque HTML redirect.
func bounce(c *zip.Ctx, waitlistURL string) error {
if isAPIClient(c) || waitlistURL == "" {
if strings.TrimSpace(c.User()) == "" {
return c.JSON(http.StatusUnauthorized, map[string]any{
"error": "authentication required — sign in to continue",
})
}
return c.JSON(http.StatusForbidden, map[string]any{
"error": "account pending approval — join the waitlist",
})
}
c.SetHeader("Location", waitlistURL)
return c.NoContent(http.StatusFound)
}
// apiKeyPrefixes are the Hanzo API-key prefixes. This MIRRORS cloud
// auth_identity.go isAPIKey (the ONE authority) — kept local so admission stays
// self-contained (no cloud-internal import) while agreeing on the exact contract:
// a token with one of these prefixes is a possession-gated API key, not a session
// principal. If cloud adds a prefix there, add it here.
var apiKeyPrefixes = []string{"hk-", "sk-", "pk-", "fw_", "hz_"}
// carriesAPIKey reports whether the request authenticates with a Hanzo API key —
// in the Authorization header (Bearer or Basic-username) or the common api-key /
// x-api-key headers. GENEROUS by design: over-exempting only skips the redundant
// waitlist gate (key validation + billing still run downstream); under-exempting
// would break paid inference. So any recognized key signal exempts the request.
func carriesAPIKey(c *zip.Ctx) bool {
auth := strings.TrimSpace(c.Header("Authorization"))
if tok, ok := cut(auth, "Bearer "); ok && hasAPIKeyPrefix(tok) {
return true
}
// Basic auth carries the key as the username (key:) — OpenAI-compat clients do
// this. The raw base64 is not decoded here; instead accept the header-form keys
// callers commonly send, which is where inference SDKs put the key.
if hasAPIKeyPrefix(strings.TrimSpace(c.Header("api-key"))) ||
hasAPIKeyPrefix(strings.TrimSpace(c.Header("x-api-key"))) {
return true
}
return false
}
func hasAPIKeyPrefix(tok string) bool {
for _, p := range apiKeyPrefixes {
if strings.HasPrefix(tok, p) {
return true
}
}
return false
}
// cut splits s on the first occurrence of prefix at the START (case-insensitive on
// the scheme word), returning the remainder.
func cut(s, prefix string) (string, bool) {
if len(s) >= len(prefix) && strings.EqualFold(s[:len(prefix)], prefix) {
return strings.TrimSpace(s[len(prefix):]), true
}
return "", false
}
// isAPIClient reports whether the caller is a non-browser API client, so we fail
// closed with a status code instead of an interactive redirect. A Bearer/Basic
// Authorization OR an Accept without text/html is an API call; a browser
// navigation sends Accept: text/html.
func isAPIClient(c *zip.Ctx) bool {
if auth := strings.TrimSpace(c.Header("Authorization")); auth != "" {
return true
}
accept := c.Header("Accept")
if accept != "" && !strings.Contains(accept, "text/html") {
return true
}
return false
}
+220
View File
@@ -0,0 +1,220 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
// Licensed under the Apache License, Version 2.0.
package admission
import (
"context"
"io"
"net/http/httptest"
"testing"
"time"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
// testGate is the injected decide (the flags engine's WaitlistModeForHost seam):
// hanzo.chat is gated, api.hanzo.ai is open, everything else is un-governed. This is
// exactly what WaitlistModeForHost returns for the equivalent registry, without
// standing up the native flag engine (cgo) in a middleware unit test.
func testGate(_ context.Context, host string) (mode bool, service string, known bool) {
switch host {
case "hanzo.chat":
return true, "chat", true // gated
case "api.hanzo.ai":
return false, "api", true // open
default:
return false, "", false // un-governed
}
}
// gateApp mounts Enforce over the injected decide and a catch-all "ok" handler. The
// injected approval status decides whether the caller is off the waitlist.
func gateApp(t *testing.T, approvalStatus string) *zip.App {
t.Helper()
approvals := newApprovalsWithLookup(func(context.Context, string, string) (string, bool) {
return approvalStatus, true
}, time.Minute)
app := zip.New(zip.Config{Logger: luxlog.New("test")})
app.Use(Enforce(EnforceConfig{WaitlistURL: "https://waitlist.hanzo.ai", Approvals: approvals, Gate: testGate}))
app.Get("/*", func(c *zip.Ctx) error { return c.String(200, "ok") })
return app
}
type greq struct {
host, path, user, org, accept string
admin, approvedHdr, setApprov bool
authorization string // raw Authorization header (e.g. "Bearer hk-…")
apiKeyHeader string // raw api-key header value
}
func drive(t *testing.T, app *zip.App, r greq) (int, string) {
t.Helper()
hr := httptest.NewRequest("GET", "http://"+r.host+r.path, nil)
hr.Host = r.host
if r.user != "" {
hr.Header.Set("X-User-Id", r.user)
}
if r.org != "" {
hr.Header.Set("X-Org-Id", r.org)
}
if r.admin {
hr.Header.Set("X-User-IsAdmin", "true")
}
if r.setApprov {
if r.approvedHdr {
hr.Header.Set("X-User-Approved", "true")
} else {
hr.Header.Set("X-User-Approved", "false")
}
}
if r.accept != "" {
hr.Header.Set("Accept", r.accept)
}
if r.authorization != "" {
hr.Header.Set("Authorization", r.authorization)
}
if r.apiKeyHeader != "" {
hr.Header.Set("api-key", r.apiKeyHeader)
}
resp, err := app.Fiber().Test(hr)
if err != nil {
t.Fatalf("drive: %v", err)
}
defer func() { _ = resp.Body.Close() }()
_, _ = io.Copy(io.Discard, resp.Body)
return resp.StatusCode, resp.Header.Get("Location")
}
const html = "text/html,application/xhtml+xml"
// THE RULE — the acceptance matrix.
func TestRule_PendingUser_BouncedFromGatedHost(t *testing.T) {
app := gateApp(t, "pending")
// Browser → 302 to the waitlist.
code, loc := drive(t, app, greq{host: "hanzo.chat", path: "/dashboard", user: "u", org: "acme", accept: html})
if code != 302 || loc != "https://waitlist.hanzo.ai" {
t.Fatalf("pending browser on gated host = %d loc=%q, want 302 → waitlist", code, loc)
}
// API (JSON accept) → 403.
code, _ = drive(t, app, greq{host: "hanzo.chat", path: "/dashboard", user: "u", org: "acme", accept: "application/json"})
if code != 403 {
t.Fatalf("pending API on gated host = %d, want 403", code)
}
}
func TestRule_ApprovedUser_ThroughGatedHost(t *testing.T) {
app := gateApp(t, "approved")
code, _ := drive(t, app, greq{host: "hanzo.chat", path: "/dashboard", user: "u", org: "acme", accept: html})
if code != 200 {
t.Fatalf("approved user on gated host = %d, want 200 (through)", code)
}
}
func TestRule_ModeOffService_OpenToPendingUser(t *testing.T) {
app := gateApp(t, "pending")
// api.hanzo.ai is mode OFF → even a pending user passes.
code, _ := drive(t, app, greq{host: "api.hanzo.ai", path: "/v1/chat/completions", user: "u", org: "acme", accept: "application/json"})
if code != 200 {
t.Fatalf("pending user on OPEN service = %d, want 200", code)
}
}
func TestRule_Admin_ThroughGatedHost(t *testing.T) {
app := gateApp(t, "pending") // status irrelevant — admin short-circuits
code, _ := drive(t, app, greq{host: "hanzo.chat", path: "/dashboard", user: "z", org: "admin", admin: true, accept: html})
if code != 200 {
t.Fatalf("admin on gated host = %d, want 200", code)
}
}
func TestRule_UnauthenticatedBrowser_BouncedToWaitlist(t *testing.T) {
app := gateApp(t, "pending")
code, loc := drive(t, app, greq{host: "hanzo.chat", path: "/dashboard", accept: html})
if code != 302 || loc != "https://waitlist.hanzo.ai" {
t.Fatalf("anon browser = %d loc=%q, want 302 → waitlist", code, loc)
}
// Anon API → 401 (authenticate first).
code, _ = drive(t, app, greq{host: "hanzo.chat", path: "/dashboard", accept: "application/json"})
if code != 401 {
t.Fatalf("anon API = %d, want 401", code)
}
}
// MONEY-CRITICAL: a paid inference request with a Hanzo API key MUST flow through
// Enforce even on a waitlist-ON host — it is possession-gated + billed downstream,
// never waitlist-gated. Without the exemption THE RULE would 401 it and break
// inference cluster-wide.
func TestRule_APIKeyInference_NeverGated(t *testing.T) {
app := gateApp(t, "pending")
for _, key := range []string{"hk-43f50b6b", "sk-hz-abc", "pk-hz-obs", "fw_live_x", "hz_secret"} {
// The exact paid-inference shape: Bearer key, JSON accept, NO session/user, on a
// GATED host — the exemption, not mode, must carry it through.
for _, p := range []string{"/v1/chat/completions", "/v1/models", "/v1/embeddings"} {
code, _ := drive(t, app, greq{
host: "hanzo.chat", path: p, accept: "application/json",
authorization: "Bearer " + key,
})
if code != 200 {
t.Fatalf("API key %q on %s = %d, want 200 (paid inference must NOT be waitlist-gated)", key, p, code)
}
}
}
// The api-key / x-api-key header form is exempt too.
code, _ := drive(t, app, greq{host: "hanzo.chat", path: "/v1/chat/completions", accept: "application/json", apiKeyHeader: "hk-headerform"})
if code != 200 {
t.Fatalf("api-key header inference = %d, want 200", code)
}
// A NON-key Bearer (a JWT-shaped token) from a pending user IS still gated — only
// key possession is exempt, not arbitrary bearers.
code, _ = drive(t, app, greq{host: "hanzo.chat", path: "/dashboard", user: "u", org: "acme", accept: "application/json", authorization: "Bearer eyJhbGciOi.jwt.sig"})
if code != 403 {
t.Fatalf("pending JWT bearer on gated host = %d, want 403 (only API keys are exempt)", code)
}
}
func TestRule_UngovernedHost_PassesThrough(t *testing.T) {
app := gateApp(t, "pending")
code, _ := drive(t, app, greq{host: "example.com", path: "/whatever", user: "u", org: "acme", accept: html})
if code != 200 {
t.Fatalf("un-governed host = %d, want 200 (not ours to gate)", code)
}
}
func TestRule_ExemptPaths_NeverGated(t *testing.T) {
app := gateApp(t, "pending")
for _, p := range []string{"/health", "/v1/iam/get-account", "/v1/waitlist/join", "/v1/flags/waitlist"} {
code, _ := drive(t, app, greq{host: "hanzo.chat", path: p, user: "u", org: "acme", accept: html})
if code != 200 {
t.Fatalf("exempt path %q = %d, want 200 (never gated)", p, code)
}
}
}
func TestRule_ForwardHeaderApproved_ThroughWithoutLookup(t *testing.T) {
// Injected lookup returns pending, but the validated X-User-Approved header
// (the forward-perfect path) says approved → the user passes with no lookup.
app := gateApp(t, "pending")
code, _ := drive(t, app, greq{host: "hanzo.chat", path: "/dashboard", user: "u", org: "acme",
accept: html, setApprov: true, approvedHdr: true})
if code != 200 {
t.Fatalf("X-User-Approved=true on gated host = %d, want 200", code)
}
}
// The DEFAULT gate (nil Gate → WaitlistModeForHost) fail-opens before the flag
// engine has mounted: with no engine, WaitlistModeForHost returns known=false for every
// host, so Enforce never gates pre-boot.
func TestEnforce_DefaultGate_FailsOpenPreBoot(t *testing.T) {
app := zip.New(zip.Config{Logger: luxlog.New("test")})
app.Use(Enforce(EnforceConfig{WaitlistURL: "https://waitlist.hanzo.ai",
Approvals: newApprovalsWithLookup(func(context.Context, string, string) (string, bool) { return "pending", true }, time.Minute)}))
app.Get("/*", func(c *zip.Ctx) error { return c.String(200, "ok") })
code, _ := drive(t, app, greq{host: "hanzo.chat", path: "/dashboard", user: "u", org: "acme", accept: html})
if code != 200 {
t.Fatalf("default gate pre-boot = %d, want 200 (never gate before flags mounts)", code)
}
}
+270
View File
@@ -0,0 +1,270 @@
package admission
// The launch-registry — the host→service map + service display metadata. It is
// deliberately MODE-FREE: a service's waitlist mode is NOT a column here, it is the
// platform switch waitlist.<svc> evaluated through the ONE flag engine (clients/flags,
// composed one-way from waitlist.go). This store answers only "which service owns this
// host, and what is its display metadata" — the config the decide needs, with the
// decision itself owned by the flag engine.
//
// It rides the SAME per-(org,project) OrgDB machinery as the flag defs (opened via
// cloud.OrgStore, encrypted at rest via cek); the registry is PLATFORM-global, so it
// lives in the reserved platform/platform tenant — one waitlist.db for the deployment.
import (
"context"
"database/sql"
"errors"
"fmt"
"sort"
"strings"
)
// ErrServiceNotFound is returned when a service slug is not in the registry.
var ErrServiceNotFound = errors.New("admission: waitlist service not found")
// ServiceRow is one hosted service in the registry (host→service + metadata). The
// waitlist MODE is intentionally absent — it is the platform switch waitlist.<svc>,
// read through the engine; ListWaitlistServices composes the two into a ServiceView.
type ServiceRow struct {
Service string `json:"service"`
DisplayName string `json:"displayName"`
Description string `json:"description"`
Hosts []string `json:"hosts"`
CreatedAt int64 `json:"createdAt"`
UpdatedAt int64 `json:"updatedAt"`
UpdatedBy string `json:"updatedBy"`
}
// waitlistStore is the registry over one OrgDB handle. Two tables, normalized:
//
// wl_services(service PK, display_name, description, …)
// wl_hosts(host PK, service FK) -- host → service, the hot lookup index
type waitlistStore struct {
db *sql.DB
}
// openWaitlistStore migrates the registry schema over an already-opened (pragma'd,
// cek-wrapped) OrgDB handle — the same open contract as flags' openStore for flag defs.
func openWaitlistStore(db *sql.DB) (*waitlistStore, error) {
const schema = `
CREATE TABLE IF NOT EXISTS wl_services (
service TEXT PRIMARY KEY,
display_name TEXT NOT NULL DEFAULT '',
description TEXT NOT NULL DEFAULT '',
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
updated_by TEXT NOT NULL DEFAULT ''
);
CREATE TABLE IF NOT EXISTS wl_hosts (
host TEXT PRIMARY KEY,
service TEXT NOT NULL,
FOREIGN KEY(service) REFERENCES wl_services(service) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS ix_wl_hosts_service ON wl_hosts(service);
`
if _, err := db.Exec(schema); err != nil {
return nil, fmt.Errorf("admission: waitlist migrate: %w", err)
}
return &waitlistStore{db: db}, nil
}
func (s *waitlistStore) Close() error { return s.db.Close() }
// NormalizeHost reduces a request Host to the registry key: lowercased, trimmed,
// port stripped. ONE canonicalization for the seed, onboard, and every lookup, so
// "Hanzo.Chat:443" and "hanzo.chat" resolve to the same service.
func NormalizeHost(host string) string {
h := strings.ToLower(strings.TrimSpace(host))
if i := strings.IndexByte(h, ':'); i >= 0 {
h = h[:i]
}
return h
}
// Seed inserts the initial registry idempotently (INSERT OR IGNORE on both tables),
// so a boot never clobbers a runtime onboard. Returns the number of services created
// (0 on a warm store).
func (s *waitlistStore) Seed(ctx context.Context, rows []SeedService, now int64) (int, error) {
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return 0, fmt.Errorf("waitlist seed tx: %w", err)
}
created := 0
for _, r := range rows {
svc := strings.ToLower(strings.TrimSpace(r.Service))
if svc == "" {
continue
}
res, err := tx.ExecContext(ctx,
`INSERT OR IGNORE INTO wl_services (service, display_name, description, created_at, updated_at, updated_by)
VALUES (?,?,?,?,?,?)`,
svc, r.DisplayName, r.Description, now, now, "seed")
if err != nil {
_ = tx.Rollback()
return 0, fmt.Errorf("waitlist seed service %q: %w", svc, err)
}
if n, _ := res.RowsAffected(); n > 0 {
created++
}
for _, h := range r.Hosts {
host := NormalizeHost(h)
if host == "" {
continue
}
if _, err := tx.ExecContext(ctx,
`INSERT OR IGNORE INTO wl_hosts (host, service) VALUES (?,?)`, host, svc); err != nil {
_ = tx.Rollback()
return 0, fmt.Errorf("waitlist seed host %q: %w", host, err)
}
}
}
if err := tx.Commit(); err != nil {
return 0, fmt.Errorf("waitlist seed commit: %w", err)
}
return created, nil
}
// List returns every registered service (with its hosts), sorted by slug.
func (s *waitlistStore) List(ctx context.Context) ([]ServiceRow, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT service, display_name, description, created_at, updated_at, updated_by FROM wl_services`)
if err != nil {
return nil, fmt.Errorf("list waitlist services: %w", err)
}
defer func() { _ = rows.Close() }()
byService := map[string]*ServiceRow{}
out := make([]ServiceRow, 0, 16)
for rows.Next() {
var r ServiceRow
if err := rows.Scan(&r.Service, &r.DisplayName, &r.Description, &r.CreatedAt, &r.UpdatedAt, &r.UpdatedBy); err != nil {
return nil, fmt.Errorf("scan waitlist service: %w", err)
}
r.Hosts = []string{}
out = append(out, r)
}
if err := rows.Err(); err != nil {
return nil, err
}
for i := range out {
byService[out[i].Service] = &out[i]
}
hostRows, err := s.db.QueryContext(ctx, `SELECT host, service FROM wl_hosts`)
if err != nil {
return nil, fmt.Errorf("list waitlist hosts: %w", err)
}
defer func() { _ = hostRows.Close() }()
for hostRows.Next() {
var host, svc string
if err := hostRows.Scan(&host, &svc); err != nil {
return nil, fmt.Errorf("scan waitlist host: %w", err)
}
if r := byService[svc]; r != nil {
r.Hosts = append(r.Hosts, host)
}
}
if err := hostRows.Err(); err != nil {
return nil, err
}
for i := range out {
sort.Strings(out[i].Hosts)
}
sort.Slice(out, func(i, j int) bool { return out[i].Service < out[j].Service })
return out, nil
}
// Get returns one service by slug, or ErrServiceNotFound.
func (s *waitlistStore) Get(ctx context.Context, service string) (ServiceRow, error) {
svc := strings.ToLower(strings.TrimSpace(service))
row := s.db.QueryRowContext(ctx,
`SELECT service, display_name, description, created_at, updated_at, updated_by FROM wl_services WHERE service=?`, svc)
var out ServiceRow
err := row.Scan(&out.Service, &out.DisplayName, &out.Description, &out.CreatedAt, &out.UpdatedAt, &out.UpdatedBy)
if errors.Is(err, sql.ErrNoRows) {
return ServiceRow{}, ErrServiceNotFound
}
if err != nil {
return ServiceRow{}, fmt.Errorf("get waitlist service: %w", err)
}
out.Hosts = []string{}
hostRows, err := s.db.QueryContext(ctx, `SELECT host FROM wl_hosts WHERE service=? ORDER BY host`, svc)
if err != nil {
return ServiceRow{}, fmt.Errorf("get waitlist hosts: %w", err)
}
defer func() { _ = hostRows.Close() }()
for hostRows.Next() {
var h string
if err := hostRows.Scan(&h); err != nil {
return ServiceRow{}, fmt.Errorf("scan waitlist host: %w", err)
}
out.Hosts = append(out.Hosts, h)
}
return out, hostRows.Err()
}
// Upsert creates or updates a service's metadata + REPLACES its host set (a host
// already claimed by ANOTHER service is skipped — first claim wins). It never
// touches the mode: the mode is the waitlist.<svc> switch, flipped through
// SetWaitlistMode. Returns the stored row.
func (s *waitlistStore) Upsert(ctx context.Context, in ServiceRow, by string, now int64) (ServiceRow, error) {
svc := strings.ToLower(strings.TrimSpace(in.Service))
if svc == "" {
return ServiceRow{}, fmt.Errorf("admission: waitlist service slug required")
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return ServiceRow{}, fmt.Errorf("waitlist upsert tx: %w", err)
}
if _, err := tx.ExecContext(ctx,
`INSERT INTO wl_services (service, display_name, description, created_at, updated_at, updated_by)
VALUES (?,?,?,?,?,?)
ON CONFLICT(service) DO UPDATE SET
display_name=excluded.display_name,
description=excluded.description,
updated_at=excluded.updated_at,
updated_by=excluded.updated_by`,
svc, in.DisplayName, in.Description, now, now, strings.TrimSpace(by)); err != nil {
_ = tx.Rollback()
return ServiceRow{}, fmt.Errorf("upsert waitlist service: %w", err)
}
if _, err := tx.ExecContext(ctx, `DELETE FROM wl_hosts WHERE service=?`, svc); err != nil {
_ = tx.Rollback()
return ServiceRow{}, fmt.Errorf("clear waitlist hosts: %w", err)
}
for _, h := range in.Hosts {
host := NormalizeHost(h)
if host == "" {
continue
}
if _, err := tx.ExecContext(ctx,
`INSERT OR IGNORE INTO wl_hosts (host, service) VALUES (?,?)`, host, svc); err != nil {
_ = tx.Rollback()
return ServiceRow{}, fmt.Errorf("add waitlist host %q: %w", host, err)
}
}
if err := tx.Commit(); err != nil {
return ServiceRow{}, fmt.Errorf("waitlist upsert commit: %w", err)
}
return s.Get(ctx, svc)
}
// ServiceForHost is the HOT lookup the decide calls once per request: it resolves a
// request host to its owning service. known is false for an un-governed host (the
// caller treats it as pass-through). host is normalized here so the caller passes
// the raw Host.
func (s *waitlistStore) ServiceForHost(ctx context.Context, host string) (service string, known bool, err error) {
h := NormalizeHost(host)
if h == "" {
return "", false, nil
}
var svc string
scanErr := s.db.QueryRowContext(ctx, `SELECT service FROM wl_hosts WHERE host=?`, h).Scan(&svc)
if errors.Is(scanErr, sql.ErrNoRows) {
return "", false, nil
}
if scanErr != nil {
return "", false, fmt.Errorf("waitlist service for host: %w", scanErr)
}
return svc, true, nil
}
+100
View File
@@ -0,0 +1,100 @@
package admission
// Registry coverage for the host→service store. It drives the store over a raw sqlite
// handle (the same driver OrgDB uses), so it exercises the registry WITHOUT the cek
// at-rest layer — runnable under CGO=0. The MODE is out of scope here by design (it is
// the waitlist.<svc> switch, evaluated by the flag engine, covered separately).
import (
"context"
"database/sql"
"errors"
"path/filepath"
"testing"
_ "github.com/hanzoai/sqlite" // registers "sqlite" under both build tags
)
func newWaitlistStore(t *testing.T) *waitlistStore {
t.Helper()
db, err := sql.Open("sqlite", filepath.Join(t.TempDir(), "waitlist.db"))
if err != nil {
t.Fatalf("open: %v", err)
}
st, err := openWaitlistStore(db)
if err != nil {
t.Fatalf("openWaitlistStore: %v", err)
}
t.Cleanup(func() { _ = st.Close() })
return st
}
func TestWaitlistStore_SeedIdempotentAndServiceForHost(t *testing.T) {
st := newWaitlistStore(t)
ctx := context.Background()
seed := []SeedService{
{Service: "chat", DisplayName: "Chat", Hosts: []string{"hanzo.chat", "chat.hanzo.ai"}},
{Service: "api", DisplayName: "API", Hosts: []string{"api.hanzo.ai"}},
}
created, err := st.Seed(ctx, seed, 100)
if err != nil || created != 2 {
t.Fatalf("seed = %d, %v; want 2, nil", created, err)
}
if created, _ := st.Seed(ctx, seed, 200); created != 0 {
t.Fatalf("re-seed created = %d, want 0 (idempotent)", created)
}
// Host resolution is case-insensitive + port-stripped → the same service.
for _, h := range []string{"hanzo.chat", "Hanzo.Chat", "hanzo.chat:443", " HANZO.CHAT "} {
svc, known, err := st.ServiceForHost(ctx, h)
if err != nil || !known || svc != "chat" {
t.Fatalf("ServiceForHost(%q) = %q,%v,%v; want chat,true,nil", h, svc, known, err)
}
}
// An un-governed host is honestly unknown (the decide fail-opens on it).
if _, known, _ := st.ServiceForHost(ctx, "example.com"); known {
t.Fatal("example.com reported known; want unknown")
}
}
func TestWaitlistStore_ListSortedWithHosts(t *testing.T) {
st := newWaitlistStore(t)
ctx := context.Background()
if _, err := st.Seed(ctx, []SeedService{
{Service: "chat", Hosts: []string{"chat.hanzo.ai", "hanzo.chat"}},
{Service: "api", Hosts: []string{"api.hanzo.ai"}},
}, 100); err != nil {
t.Fatalf("seed: %v", err)
}
list, err := st.List(ctx)
if err != nil || len(list) != 2 {
t.Fatalf("List = %d, %v; want 2", len(list), err)
}
if list[0].Service != "api" || list[1].Service != "chat" {
t.Fatalf("List order = [%s %s], want [api chat]", list[0].Service, list[1].Service)
}
if len(list[1].Hosts) != 2 || list[1].Hosts[0] != "chat.hanzo.ai" {
t.Fatalf("chat hosts = %v, want sorted [chat.hanzo.ai hanzo.chat]", list[1].Hosts)
}
}
func TestWaitlistStore_UpsertOnboardsAndReplacesHosts(t *testing.T) {
st := newWaitlistStore(t)
ctx := context.Background()
row, err := st.Upsert(ctx, ServiceRow{Service: "search", DisplayName: "Search", Hosts: []string{"search.hanzo.ai"}}, "z@hanzo.ai", 100)
if err != nil || len(row.Hosts) != 1 || row.Hosts[0] != "search.hanzo.ai" {
t.Fatalf("Upsert new = %+v, %v", row, err)
}
// A metadata edit REPLACES the host set and updates the display name.
row, err = st.Upsert(ctx, ServiceRow{Service: "search", DisplayName: "Search v2", Hosts: []string{"search.hanzo.ai", "find.hanzo.ai"}}, "z@hanzo.ai", 200)
if err != nil || row.DisplayName != "Search v2" || len(row.Hosts) != 2 {
t.Fatalf("Upsert edit = %+v, %v", row, err)
}
svc, known, _ := st.ServiceForHost(ctx, "find.hanzo.ai")
if !known || svc != "search" {
t.Fatalf("onboarded host find.hanzo.ai = %q,%v; want search,true", svc, known)
}
// An unknown slug is ErrServiceNotFound (the admin lens maps it to 404).
if _, err := st.Get(ctx, "nope"); !errors.Is(err, ErrServiceNotFound) {
t.Fatalf("Get(nope) err = %v, want ErrServiceNotFound", err)
}
}
+351
View File
@@ -0,0 +1,351 @@
package admission
// The launch-control gate — the COMPLETE waitlist feature, COMPOSING the ONE flag
// engine (clients/flags) one-way. Decomplected into the two orthogonal axes it always
// was, with a single decision plane:
//
// - MODE (per service): waitlist.<svc> IS a platform switch, evaluated through the
// flag engine (flags.Bool / flags.SetPlatformSwitch / flags.Register). There is no
// second mode store.
// - HOST MAP + metadata: the registry (registry.go) resolves a request host to the
// service whose switch governs it, and carries display metadata.
//
// The decide is WaitlistModeForHost(host) → (mode, service, known): resolve host→svc,
// then read waitlist.<svc>. Enforce (middleware.go) consumes this decide; the admin
// board (/v1/admin/services) and the guard's runtime mode read (/v1/flags/waitlist,
// served here) read it too. Per-user approval (pending|approved) is the second,
// orthogonal axis — IAM's, in approval.go.
//
// flags NEVER imports this package; this package imports flags. That one-way arrow is
// the whole point of the decomplection: the engine is pure, the feature composes it.
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/flags"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
// The reserved platform tenant the launch registry rides in — the SAME reserved
// (org, project) the flag engine uses for its platform switches, so the registry and
// the waitlist.<svc> switches co-locate. One waitlist.db for the deployment.
const (
platformOrg = "platform"
platformProject = "platform"
)
// registryState is admission's process-wide launch state: the platform-tenant
// host→service registry store + the deployment brand it was seeded for. Installed by
// Mount, torn down by Shutdown.
type registryState struct {
store *cloud.OrgStore[*waitlistStore]
brand string
}
var mounted *registryState
// SeedService is one row of the launch registry (a hosted service + its hosts). Mode
// is intentionally absent — the launch posture (gated) is waitlistDef's Default "true".
type SeedService struct {
Service string
DisplayName string
Description string
Hosts []string
}
// ServiceInput is the admin onboard/edit payload for /v1/admin/services. WaitlistMode
// sets the launch switch for a NEW service; a re-register PRESERVES the live switch.
type ServiceInput struct {
Service string `json:"service"`
DisplayName string `json:"displayName"`
Description string `json:"description"`
Hosts []string `json:"hosts"`
WaitlistMode bool `json:"waitlistMode"`
}
// ServiceView is one service as the admin board renders it: the registry row plus its
// LIVE waitlist mode (the waitlist.<svc> switch evaluated through the engine).
type ServiceView struct {
ServiceRow
WaitlistMode bool `json:"waitlistMode"`
}
// waitlistKey is the ONE naming rule: a service's mode is the switch waitlist.<svc>.
func waitlistKey(svc string) string { return "waitlist." + strings.ToLower(strings.TrimSpace(svc)) }
// waitlistDef is the platform switch for one service's mode. Default "true" = the
// launch posture (gated until an admin opens it), so a deployment with no stored flag
// behaves exactly as the old admission seed (waitlistMode ON).
func waitlistDef(svc, display string) flags.Def {
if strings.TrimSpace(display) == "" {
display = svc
}
return flags.Def{
Key: waitlistKey(svc),
Category: "Launch",
Label: "Waitlist · " + display,
Desc: "Waitlist mode for " + display + ": ON gates the service to APPROVED users; OFF opens it.",
Type: flags.TypeBool,
Default: "true",
}
}
// ensureWaitlistDef registers a service's switch if it is not already registered
// (Mount registers the seed set with nicer labels; this covers runtime onboards).
func ensureWaitlistDef(svc, display string) {
key := waitlistKey(svc)
for _, d := range flags.Defs() {
if d.Key == key {
return
}
}
flags.Register(waitlistDef(svc, display))
}
// boolDef is the minimal PostHog flag definition for a boolean switch value.
func boolDef(on bool) json.RawMessage {
if on {
return json.RawMessage(`{"active":true}`)
}
return json.RawMessage(`{"active":false}`)
}
// requireRegistry resolves the platform-tenant registry store, or an error when the
// gate is not mounted (writes need it; the decide fail-opens instead).
func requireRegistry() (*waitlistStore, error) {
if mounted == nil || mounted.store == nil {
return nil, fmt.Errorf("admission: waitlist registry not mounted")
}
return mounted.store.For(platformOrg, platformProject)
}
// WaitlistModeForHost is THE decide the Enforce consumer, /v1/flags/waitlist, and
// the admin board call: resolve host→service, then read the waitlist.<svc> switch
// through the flag engine. FAIL-OPEN by construction — an unmounted registry, a store
// error, or an un-governed host all return known=false, so a request is NEVER gated
// pre-boot or on a registry fault (availability over a hard gate, matching the guard).
func WaitlistModeForHost(ctx context.Context, host string) (mode bool, service string, known bool) {
if mounted == nil || mounted.store == nil {
return false, "", false
}
st, err := mounted.store.For(platformOrg, platformProject)
if err != nil {
return false, "", false
}
svc, known, err := st.ServiceForHost(ctx, host)
if err != nil || !known {
return false, "", false
}
return flags.Bool(waitlistKey(svc)), svc, true
}
// ListWaitlistServices returns the admin board: every registered service with its LIVE
// mode (the waitlist.<svc> switch). SuperAdmin surface (the caller gates).
func ListWaitlistServices(ctx context.Context) ([]ServiceView, error) {
st, err := requireRegistry()
if err != nil {
return nil, err
}
rows, err := st.List(ctx)
if err != nil {
return nil, err
}
out := make([]ServiceView, 0, len(rows))
for _, r := range rows {
out = append(out, ServiceView{ServiceRow: r, WaitlistMode: flags.Bool(waitlistKey(r.Service))})
}
return out, nil
}
// SetWaitlistMode flips one service's waitlist switch — the launch lever — and returns
// the updated view. It is the ONE write path (through flags.SetPlatformSwitch, audited
// in the flag activity log); the flip is hot (this pod applies immediately, peers
// converge within the eval TTL). ErrServiceNotFound when the slug is unknown.
func SetWaitlistMode(ctx context.Context, service string, mode bool, actor string) (ServiceView, error) {
service = strings.ToLower(strings.TrimSpace(service))
if service == "" {
return ServiceView{}, fmt.Errorf("admission: service is required")
}
st, err := requireRegistry()
if err != nil {
return ServiceView{}, err
}
row, err := st.Get(ctx, service) // ErrServiceNotFound → 404 upstream
if err != nil {
return ServiceView{}, err
}
ensureWaitlistDef(service, row.DisplayName)
if err := flags.SetPlatformSwitch(waitlistKey(service), boolDef(mode), actor); err != nil {
return ServiceView{}, err
}
return ServiceView{ServiceRow: row, WaitlistMode: flags.Bool(waitlistKey(service))}, nil
}
// UpsertWaitlistService onboards or edits a hosted service so a new host is governed
// WITHOUT a redeploy. A NEW service takes in.WaitlistMode as its launch mode; a
// re-register PRESERVES the live switch (never silently re-gating an opened service).
func UpsertWaitlistService(ctx context.Context, in ServiceInput, actor string) (ServiceView, error) {
svc := strings.ToLower(strings.TrimSpace(in.Service))
if svc == "" {
return ServiceView{}, fmt.Errorf("admission: service slug is required")
}
st, err := requireRegistry()
if err != nil {
return ServiceView{}, err
}
_, getErr := st.Get(ctx, svc)
isNew := errors.Is(getErr, ErrServiceNotFound)
if getErr != nil && !isNew {
return ServiceView{}, getErr
}
row, err := st.Upsert(ctx, ServiceRow{
Service: svc,
DisplayName: in.DisplayName,
Description: in.Description,
Hosts: in.Hosts,
}, actor, time.Now().Unix())
if err != nil {
return ServiceView{}, err
}
ensureWaitlistDef(svc, row.DisplayName)
if isNew {
if err := flags.SetPlatformSwitch(waitlistKey(svc), boolDef(in.WaitlistMode), actor); err != nil {
return ServiceView{}, err
}
}
return ServiceView{ServiceRow: row, WaitlistMode: flags.Bool(waitlistKey(svc))}, nil
}
// seedRegistry seeds the registry and registers a waitlist.<svc> switch per known
// service, COMPOSING the flag engine (flags.Register). Best-effort + fail-safe: a
// registry error (e.g. cek master key not yet injected) degrades to the in-memory seed
// switches — the decide then fail-opens, exactly the flag engine's own boot posture.
// Returns the number of seeded services (for the mount log). Called from Mount.
func seedRegistry(brand string, log luxlog.Logger) int {
seed := seedWaitlist(brand)
for _, sv := range seed { // in-memory switches — always succeeds
flags.Register(waitlistDef(sv.Service, sv.DisplayName))
}
st, err := mounted.store.For(platformOrg, platformProject)
if err != nil {
log.Warn("waitlist registry unavailable — modes degrade to seed defaults", "err", err)
return len(seed)
}
if _, err := st.Seed(context.Background(), seed, time.Now().Unix()); err != nil {
log.Warn("waitlist registry seed failed", "err", err)
return len(seed)
}
if rows, err := st.List(context.Background()); err == nil {
for _, r := range rows { // register any persisted onboard beyond the seed
ensureWaitlistDef(r.Service, r.DisplayName)
}
}
return len(seed)
}
// waitlistModeRoute answers GET /v1/flags/waitlist?host=<h> — the runtime lookup the
// @file waitlist-guard caches. Public (in-cluster) read: it returns ONLY the boolean
// mode for the ONE queried host, never an enumeration.
func waitlistModeRoute(c *zip.Ctx) error {
host := strings.TrimSpace(c.Query("host"))
if host == "" {
host = c.Fiber().Hostname()
}
mode, service, known := WaitlistModeForHost(c.Context(), host)
return c.JSON(http.StatusOK, map[string]any{
"host": NormalizeHost(host),
"service": service,
"waitlistMode": mode,
"known": known,
})
}
// ── lifecycle ────────────────────────────────────────────────────────────────
// Mount installs the launch-control gate: it opens the platform-tenant host→service
// registry, seeds it for the deployment brand, registers a waitlist.<svc> switch per
// service in the flag engine (flags.Register), and serves the guard's public mode read
// at /v1/flags/waitlist. Fail-safe: a registry error (e.g. cek master key not yet
// injected) degrades to the in-memory seed switches — WaitlistModeForHost then
// fail-opens. Mounts AFTER flags so the engine's platform-switch plane is installed first.
func Mount(app *zip.App, deps cloud.Deps) error {
if deps.Logger == nil {
return fmt.Errorf("admission.Mount: nil deps.Logger")
}
if deps.DataDir == "" {
return fmt.Errorf("admission.Mount: empty deps.DataDir")
}
log := deps.Logger.New("subsystem", "admission")
mounted = &registryState{
store: cloud.NewOrgStore[*waitlistStore](deps.DataDir, "waitlist", openWaitlistStore),
brand: deps.Brand,
}
n := seedRegistry(deps.Brand, log)
// The guard's public runtime mode read (host→service→waitlist.<svc>), one namespace
// under /v1/flags. Exempt from the Enforce gate (see defaultExemptPrefixes) so a
// gated user can still resolve mode.
app.Get("/v1/flags/waitlist", waitlistModeRoute)
log.Info("admission gate ready", "services", n)
return nil
}
// Shutdown closes the launch registry's per-org store handles.
func Shutdown() error {
if mounted == nil || mounted.store == nil {
return nil
}
return mounted.store.CloseAll()
}
// ── brand seed (moved verbatim from the former flags/waitlist.go) ────────────────
// seedWaitlist returns the launch registry for a brand. White-labeled so a Lux/Zoo/Pars
// deployment governs its OWN hosts. New hosted services onboard at runtime via
// POST /v1/admin/services (no redeploy). admin.<brand> is deliberately NOT seeded (it
// is admin-guarded, not a waitlist surface).
func seedWaitlist(brand string) []SeedService {
d := domainFor(brand)
return []SeedService{
{Service: "studio", DisplayName: "Studio", Description: "AI app studio", Hosts: []string{"studio." + d}},
{Service: "chat", DisplayName: "Chat", Description: "AI chat", Hosts: hostsFor(brand, "chat", "chat."+d)},
{Service: "console", DisplayName: "Console", Description: "Cloud console", Hosts: []string{"console." + d}},
{Service: "app", DisplayName: "App", Description: "App builder", Hosts: hostsFor(brand, "app", "app."+d)},
{Service: "api", DisplayName: "API", Description: "Inference API gateway", Hosts: []string{"api." + d}},
{Service: "team", DisplayName: "Team", Description: "Team workspace", Hosts: hostsFor(brand, "team", "team."+d)},
}
}
// domainFor maps a brand to its primary domain. Defaults to hanzo.ai.
func domainFor(brand string) string {
switch strings.ToLower(strings.TrimSpace(brand)) {
case "lux":
return "lux.network"
case "zoo":
return "zoo.ngo"
case "pars":
return "pars.network"
default:
return "hanzo.ai"
}
}
// hostsFor returns the apex-brand host (hanzo.chat / zoo.chat style) plus the
// <label>.<domain> alias when the brand ships an apex-label domain; else just the alias.
func hostsFor(brand, label, alias string) []string {
switch strings.ToLower(strings.TrimSpace(brand)) {
case "", "hanzo":
return []string{"hanzo." + label, alias}
case "zoo":
return []string{"zoo." + label, alias}
default:
return []string{alias}
}
}
+27 -136
View File
@@ -1,153 +1,44 @@
package affiliates
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
"github.com/hanzoai/cloud/clients/commerceinproc"
"github.com/hanzoai/cloud/clients/payout"
)
// commerce is the narrow money seam the affiliate loop needs: read a referred
// org's metered spend (the accrual base) and grant a promo credit to a wallet (a
// payout made in credits). It is an INTERFACE so the store/handler logic is
// testable with a fake ledger the HTTP impl below is the ONE production binding.
// commerce is the narrow money seam the affiliate loop needs: read a referred org's
// metered spend (the commission accrual base) and grant a promo credit to a wallet
// (a payout made in credits, ledger tag grant:affiliate). It is an INTERFACE so the
// store/handler logic is testable with a fake ledger; the production binding is
// clients/payout, reached through the thin adapter below.
//
// This mirrors clients/referrals/commerce.go EXACTLY (which itself mirrors
// clients/admin/commerce.go): the same COMMERCE_SERVICE_TOKEN S2S path, the same
// X-Org-Id=<org> namespace + bare org `user` subject that admin.grantCredit uses —
// so an affiliate payout-in-credits lands in precisely the wallet the balance
// panel reads, indistinguishable from an admin grant except by its ledger tag
// (grant:affiliate vs grant:referral / grant:admin, all → the commerce Credit/trial
// bucket per DepositKind's grant:* rule).
// The S2S impl (COMMERCE_SERVICE_TOKEN path, X-Org-Id=<org> namespace, bare-org
// `user` subject) was three byte-identical commerce.go copies; it now lives ONCE in
// clients/payout. An affiliate payout-in-credits still lands in precisely the wallet
// the balance panel reads, indistinguishable from an admin grant except by its
// grant:affiliate tag.
type commerce interface {
configured() bool
// deposit grants amountCents to org's wallet (Credit/trial bucket via the
// grant:affiliate tag) and returns the ledger transaction id.
deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (txnID string, err error)
// spendCents is a referred org's month-to-date metered consumption — the
// commission accrual base (spend × the affiliate's rate).
spendCents(ctx context.Context, org, user string) (int64, error)
}
// errUnconfigured is returned by a deposit against an unwired commerce so the
// caller records an honest failure rather than reporting a phantom payout.
var errUnconfigured = errors.New("affiliates: commerce endpoint not configured")
// errUnconfigured is the shared sentinel a deposit against an unwired commerce
// returns, so the caller records an honest failure rather than a phantom payout.
var errUnconfigured = payout.ErrUnconfigured
// httpCommerce is the production commerce binding (COMMERCE_SERVICE_TOKEN S2S).
type httpCommerce struct {
base string
token string
http *http.Client
// commerceSeam adapts the shared payout.Client onto this program's lowercase seam
// (Go package-scoped interface methods cannot cross packages). Zero logic — pure
// delegation; the money path lives in clients/payout.
type commerceSeam struct{ c *payout.Client }
func (s commerceSeam) configured() bool { return s.c.Configured() }
func (s commerceSeam) deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (string, error) {
return s.c.Deposit(ctx, org, user, amountCents, currency, notes, tags)
}
func (s commerceSeam) spendCents(ctx context.Context, org, user string) (int64, error) {
return s.c.SpendCents(ctx, org, user)
}
func newCommerceClient(base, token string) *httpCommerce {
return &httpCommerce{
base: strings.TrimRight(strings.TrimSpace(base), "/"),
token: strings.TrimSpace(token),
http: commerceinproc.Client(15 * time.Second),
}
}
func (c *httpCommerce) configured() bool { return c != nil && c.base != "" && c.token != "" }
// deposit posts POST /v1/billing/deposit — the ONE money-in primitive (identical
// to admin.commerceClient.deposit). Commerce's EdgeAuth pins the body `user` to
// the X-Org-Id subject, so a payout can never be mis-targeted to another wallet.
func (c *httpCommerce) deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (string, error) {
if !c.configured() {
return "", errUnconfigured
}
if currency == "" {
currency = "usd"
}
body, err := json.Marshal(map[string]any{
"user": user,
"currency": currency,
"amount": amountCents,
"notes": notes,
"tags": tags,
})
if err != nil {
return "", err
}
raw, err := c.do(ctx, http.MethodPost, "/v1/billing/deposit", nil, org, body)
if err != nil {
return "", err
}
var out struct {
TransactionID string `json:"transactionId"`
}
if err := json.Unmarshal(raw, &out); err != nil {
return "", fmt.Errorf("commerce deposit decode: %w", err)
}
return out.TransactionID, nil
}
// spendCents reads GET /v1/billing/usage-rollup and returns consumedCents. Zero
// (not an error) when commerce is unconfigured so a partial deploy degrades to
// "no spend to accrue yet" rather than a 5xx.
func (c *httpCommerce) spendCents(ctx context.Context, org, user string) (int64, error) {
if !c.configured() {
return 0, nil
}
q := url.Values{"user": {user}}
raw, err := c.do(ctx, http.MethodGet, "/v1/billing/usage-rollup", q, org, nil)
if err != nil {
return 0, err
}
var out struct {
ConsumedCents int64 `json:"consumedCents"`
}
if err := json.Unmarshal(raw, &out); err != nil {
return 0, fmt.Errorf("commerce rollup decode: %w", err)
}
return out.ConsumedCents, nil
}
// do performs one admin-S2S commerce request. X-Org-Id=<org> is the per-org
// namespace selector commerce's EdgeAuth trusts only behind the service token.
func (c *httpCommerce) do(ctx context.Context, method, path string, q url.Values, org string, body []byte) ([]byte, error) {
u := c.base + path
if enc := q.Encode(); enc != "" {
u += "?" + enc
}
var r io.Reader
if body != nil {
r = bytes.NewReader(body)
}
req, err := http.NewRequestWithContext(ctx, method, u, r)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json")
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
if c.token != "" {
req.Header.Set("Authorization", "Bearer "+c.token)
}
if org != "" {
req.Header.Set("X-Org-Id", org)
}
resp, err := c.http.Do(req)
if err != nil {
return nil, fmt.Errorf("commerce unreachable: %w", err)
}
defer func() { _ = resp.Body.Close() }()
out, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return nil, err
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("commerce status %d", resp.StatusCode)
}
return out, nil
}
// newCommerceClient builds the production binding, delegating to clients/payout.
func newCommerceClient(base, token string) commerce { return commerceSeam{payout.NewClient(base, token)} }
+141
View File
@@ -0,0 +1,141 @@
// Package agent mounts the hanzoai/agent orchestrator into cloud: POST /v1/agent
// (+ /v1/agent/presets, /v1/agent/conversations). The orchestrator logic and its
// per-org conversation history live in github.com/hanzoai/agent, which imports
// NEITHER cloud NOR ai. Cloud is the composition root: it injects the two seams —
// - Completer: the ai subsystem's /v1/chat/completions, replayed in-process (the
// one path that returns tool_calls AND carries per-org reserve/settle billing);
// - ToolPlane: the unified tool registry (tools.Default()), so /v1/agent's
// server-executed tools are the org's activated MCP/registry tools.
// /v1/agent is a DISTINCT path (not /v1/chat, which ai owns as completions), so a
// specific route wins over ai's /v1/* glob — no collision.
package agent
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
hz "github.com/hanzoai/agent"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/tools"
openai "github.com/hanzoai/go-openai"
fiber "github.com/zap-proto/fiber/v3"
"github.com/zap-proto/zip"
)
// maxCompletionResponse bounds the in-process completion body read so a hostile or
// broken upstream cannot balloon memory.
const maxCompletionResponse = 8 << 20
// Mount wires POST /v1/agent (+ reads) into cloud, injecting the ai completion and
// the tool plane. The caller identity comes from cloud's validated principal.
func Mount(app *zip.App, deps cloud.Deps) error {
if app == nil {
return fmt.Errorf("agent.Mount: nil zip.App")
}
_, err := hz.Mount(app, hz.Deps{
Logger: deps.Logger,
DataDir: deps.DataDir,
Brand: deps.Brand,
Model: deps.AIDefaultModel,
Principal: func(c *zip.Ctx) (hz.Principal, bool) {
p, ok := tools.PrincipalFrom(c)
if !ok {
return hz.Principal{}, false
}
return hz.Principal{Org: p.Org, Project: p.Project, User: p.User, Cred: credential(c)}, true
},
}, aiCompleter{app: app}, toolPlane{})
return err
}
// ── Completer: replay /v1/chat/completions in-process ─────────────────────────────
type aiCompleter struct{ app *zip.App }
// Complete replays the request against the SAME app at /v1/chat/completions, so it
// flows the whole middleware chain (per-org reserve/settle billing) and returns
// tool_calls. Non-streaming. Mirrors the tool plane's in-process dispatch contract:
// the caller's OWN credential headers are replayed; no minted authority header.
func (a aiCompleter) Complete(ctx context.Context, cred map[string]string, req openai.ChatCompletionRequest) (openai.ChatCompletionResponse, error) {
req.Stream = false
b, err := json.Marshal(req)
if err != nil {
return openai.ChatCompletionResponse{}, err
}
hreq := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", bytes.NewReader(b)).WithContext(ctx)
hreq.Header.Set("Content-Type", "application/json")
for k, v := range cred {
hreq.Header.Set(k, v)
}
resp, err := a.app.Fiber().Test(hreq, fiber.TestConfig{Timeout: 0})
if err != nil {
return openai.ChatCompletionResponse{}, err
}
defer func() { _ = resp.Body.Close() }()
raw, _ := io.ReadAll(io.LimitReader(resp.Body, maxCompletionResponse))
if resp.StatusCode/100 != 2 {
// Carry the completion's OWN status + body so the round can pass a
// caller-facing refusal (402 insufficient_balance, 429, 403) straight
// through instead of masking it as a gateway 502. hz.UpstreamError is the
// agent's typed seam for exactly this.
return openai.ChatCompletionResponse{}, &hz.UpstreamError{Status: resp.StatusCode, Body: raw}
}
var out openai.ChatCompletionResponse
if err := json.Unmarshal(raw, &out); err != nil {
return openai.ChatCompletionResponse{}, fmt.Errorf("decode completion: %w", err)
}
return out, nil
}
// ── ToolPlane: adapter over the unified registry ──────────────────────────────────
type toolPlane struct{}
func (toolPlane) List(ctx context.Context, scope hz.Scope) []hz.Tool {
ts := tools.Default().List(ctx, tools.Scope{Org: scope.Org, Project: scope.Project})
out := make([]hz.Tool, 0, len(ts))
for _, t := range ts {
out = append(out, hz.Tool{
Name: t.Name,
Description: t.Description,
Schema: t.Schema,
Activated: t.Activated,
Dispatchable: t.Dispatchable,
})
}
return out
}
func (toolPlane) Exists(ctx context.Context, scope hz.Scope, name string) bool {
return tools.Default().Exists(ctx, tools.Scope{Org: scope.Org, Project: scope.Project}, name)
}
// Dispatch resolves the caller the ONE canonical way — tools.PrincipalFrom(c) — so
// the tool runs under the SAME validated identity + credential as a direct call.
// No reconstruction: the credential is only ever read from the live request.
func (toolPlane) Dispatch(c *zip.Ctx, name string, args map[string]any) (any, error) {
p, ok := tools.PrincipalFrom(c)
if !ok {
return nil, zip.ErrForbidden("a validated principal is required")
}
return tools.Default().Dispatch(c.Context(), p, name, args)
}
// ── helpers ───────────────────────────────────────────────────────────────────────
// credential extracts the caller's replayable credential headers (the same set the
// tool plane replays) so the in-process completion runs as the caller.
func credential(c *zip.Ctx) map[string]string {
cred := map[string]string{}
for _, h := range []string{"Authorization", "X-Authorization", "Cookie", "Accept-Language", "X-Forwarded-For"} {
if v := c.Header(h); v != "" {
cred[h] = v
}
}
return cred
}
+8 -9
View File
@@ -278,19 +278,18 @@ func Mount(app *zip.App, deps cloud.Deps) error {
app.Get("/v1/agents", cloud.Handle(s, list))
app.Post("/v1/agents", cloud.Handle(s, create))
// Static org-wide surfaces MUST register before the :ref wildcard: Fiber
// matches routes in registration order, so a bare `/v1/agents/:ref` would
// otherwise capture "metrics"/"activity"/"sessions" as a ref and 404 them
// (Red route audit). Registering the literals first makes them win.
// The static org-wide surfaces are listed before the :ref wildcard for reading
// order, not for matching: the router resolves by SPECIFICITY, so a literal
// beats a param whatever order they register in ("metrics" is never captured as
// a ref). Registration order decides nothing here — it only decides which
// handler silently wins when two patterns are byte-identical, which is a
// collision, not a precedence.
app.Get("/v1/agents/metrics", cloud.Handle(s, metrics))
app.Get("/v1/agents/activity", cloud.Handle(s, activity))
// Live agent-session control plane: /v1/agents/sessions[/...]. Registered
// before :name for the same registration-order reason (and internally the
// static /stream precedes /:id).
// Live agent-session control plane: /v1/agents/sessions[/...].
mountSessions(s, app)
// Agent targets: /v1/agents/targets[/...] — the #48 dispatch destinations a
// session runs on. Registered before :ref for the same registration-order reason
// (and internally /targets precedes /targets/:id).
// session runs on.
mountTargets(s, app)
app.Get("/v1/agents/:ref", cloud.Handle(s, get))
app.Patch("/v1/agents/:ref", cloud.Handle(s, update))
+38
View File
@@ -71,6 +71,44 @@ func OpenSession(ctx context.Context, org, actor, agent, title string) (string,
return id, nil
}
// OpenSessionOn is OpenSession with the run's dispatch TARGET recorded, so
// mission-control shows a routed run on the machine it was sent to (session.target
// == the target id) exactly as a locally-linked run shows its host. The target is
// re-resolved org-scoped and MUST belong to this org — a session can never claim
// to run on another tenant's machine (the same fail-closed rule sessionContext
// enforces on the HTTP register path). An empty target falls back to OpenSession.
func OpenSessionOn(ctx context.Context, org, actor, agent, title, target string) (string, error) {
target = strings.TrimSpace(target)
if target == "" {
return OpenSession(ctx, org, actor, agent, title)
}
if mounted == nil {
return "", fmt.Errorf("agents: not mounted")
}
org = strings.TrimSpace(org)
if org == "" {
return "", fmt.Errorf("agents: org required")
}
if _, err := mounted.State.store.GetTarget(ctx, org, target); err != nil {
if err == errTargetNotFound {
return "", fmt.Errorf("agents: target not found in this org")
}
return "", fmt.Errorf("agents: resolve target: %w", err)
}
id, err := OpenSession(ctx, org, actor, agent, title)
if err != nil {
return "", err
}
// Stamp the target onto the freshly-opened row (org-scoped update); a failure
// here is non-fatal — the session is live, it simply lacks its machine tag.
if x, gerr := mounted.State.store.GetSession(ctx, org, id); gerr == nil {
x.Target = target
x.UpdatedAt = time.Now().Unix()
_ = mounted.State.store.UpdateSession(ctx, x)
}
return id, nil
}
// LogSessionEvent appends one ordered event (message|tool-call|spawn|log|status|
// control) to an org's session and fans it out live. The (org, id) pair is
// re-resolved so a caller can only write to a session THIS org owns; kind is
+47
View File
@@ -114,3 +114,50 @@ func TestInproc_NotMounted_FailsClosed(t *testing.T) {
t.Fatal("unmounted OpenSession must fail closed")
}
}
// ResolveTarget turns a human's reference (id or friendly label) into the org's
// target, org-scoped and fail-closed: an id wins, else an exact case-folded label,
// and a reference matching neither — or another org's machine — is not found.
func TestResolveTarget_IdThenLabel_OrgScoped(t *testing.T) {
mountInproc(t)
ctx := context.Background()
now := int64(1000)
acme := Target{ID: "tgt_acme1", Org: "acme", Label: "evo", Kind: TargetGPU, Status: TargetOnline, Host: "evo", CreatedAt: now, UpdatedAt: now}
evil := Target{ID: "tgt_evil1", Org: "evil", Label: "evo", Kind: TargetGPU, Status: TargetOnline, Host: "evo", CreatedAt: now, UpdatedAt: now}
if err := mounted.State.store.CreateTarget(ctx, acme); err != nil {
t.Fatal(err)
}
if err := mounted.State.store.CreateTarget(ctx, evil); err != nil {
t.Fatal(err)
}
// By id.
if got, err := ResolveTarget(ctx, "acme", "tgt_acme1"); err != nil || got.ID != "tgt_acme1" {
t.Fatalf("resolve by id: %+v %v", got, err)
}
// By label (case-folded), scoped to the caller's org — never evil's same-labelled box.
if got, err := ResolveTarget(ctx, "acme", "EVO"); err != nil || got.ID != "tgt_acme1" {
t.Fatalf("resolve by label must find acme's own, got %+v %v", got, err)
}
// Another org's id is not found (no cross-tenant leak).
if _, err := ResolveTarget(ctx, "acme", "tgt_evil1"); err != errTargetNotFound {
t.Fatalf("cross-org id must be not-found, got %v", err)
}
// An unknown reference is not found — the caller renders an honest error.
if _, err := ResolveTarget(ctx, "acme", "nope"); err != errTargetNotFound {
t.Fatalf("unknown ref must be not-found, got %v", err)
}
// Empty ref is not found (never resolves to "some" machine).
if _, err := ResolveTarget(ctx, "acme", ""); err != errTargetNotFound {
t.Fatalf("empty ref must be not-found, got %v", err)
}
}
func TestResolveTarget_NotMounted_FailsClosed(t *testing.T) {
prev := mounted
mounted = nil
t.Cleanup(func() { mounted = prev })
if _, err := ResolveTarget(context.Background(), "acme", "evo"); err == nil {
t.Fatal("unmounted ResolveTarget must fail closed")
}
}
+236
View File
@@ -0,0 +1,236 @@
package agents
import (
"context"
"sync"
)
// mailbox.go is the LIVE hand-off between a routed run's durable owner (the
// coding RoutedRunWorkflow, running on the embedded tasks engine) and the
// external machine that claims and executes it over HTTP. It is the rendezvous
// ONLY — never the durable queue. The tasks engine is the queue of record: it
// survives a cloud restart, times a never-claimed run out, and retries. On every
// (re)start of the delivery activity the run is (re-)Offered here, so a machine
// that long-polls Claim always finds work the engine still owns; a cloud restart
// simply re-populates the mailbox from durable history.
//
// ISOLATION IS STRUCTURAL. Every offer is filed under the key (org, target), and
// Claim/Report only ever touch that one key's slot. A run offered for (orgB,
// targetY) is unreachable from a Claim or Report for (orgA, targetX) — the tenant
// + machine boundary is a property of the map key, not a check a caller can skip.
// RoutedRun is the NON-SECRET spec of one coding run dispatched to a target. It
// carries no credential by design: the executing machine authenticates git +
// model routing with its OWN already-held credentials (the same ones `hanzo code`
// uses), so no secret ever enters the durable store or crosses to the machine in
// the claim response. Everything here is safe to persist in the tasks engine.
type RoutedRun struct {
Org string `json:"org"`
TargetID string `json:"targetId"`
SessionID string `json:"sessionId"` // the live session opened at dispatch; the machine streams into it
Repo string `json:"repo"`
Project string `json:"project,omitempty"`
Base string `json:"base,omitempty"`
Branch string `json:"branch"`
Prompt string `json:"prompt"`
CloneURL string `json:"cloneUrl"`
TimeoutSeconds int `json:"timeoutSeconds,omitempty"`
}
// RoutedResult is a routed run's terminal outcome, reported by the machine and
// returned to the durable activity so the workflow completes.
type RoutedResult struct {
OK bool `json:"ok"`
Changed bool `json:"changed"`
Branch string `json:"branch,omitempty"`
CommitSha string `json:"commitSha,omitempty"`
Diffstat string `json:"diffstat,omitempty"`
Error string `json:"error,omitempty"`
}
// offer is one run waiting to be claimed, plus the channel its durable owner
// blocks on for the terminal result. result is buffered(1) so Report never blocks
// even if the owner is between selects; closed fires when the offer is finished
// (reported OR abandoned) so a waiter always unblocks.
type offer struct {
mb *mailbox
key string // (org,target)
rk string // (org,target,sessionID)
run RoutedRun
result chan RoutedResult
closed chan struct{}
once sync.Once
}
// Await blocks until the machine reports this run's result, the offer is
// abandoned, or ctx (the activity's StartToClose budget) fires. It is the
// durable owner's half of the rendezvous.
func (o *offer) Await(ctx context.Context) (RoutedResult, bool) {
select {
case res := <-o.result:
return res, true
case <-o.closed:
// Abandoned or reported-then-closed: drain a delivered result if one raced in.
select {
case res := <-o.result:
return res, true
default:
return RoutedResult{}, false
}
case <-ctx.Done():
return RoutedResult{}, false
}
}
// Close removes the offer from the mailbox (if still present) and unblocks any
// waiter. Idempotent — the durable owner defers it so a timed-out or crashed
// delivery never leaks a queued or claimed offer.
func (o *offer) Close() { o.mb.discard(o) }
// mailbox is the process-wide rendezvous. queues holds each key's FIFO of
// unclaimed offers; byRun indexes every live offer by (org,target,sessionID) for
// Report + re-offer dedupe; signal is a per-key broadcast channel (closed and
// recreated on Offer) that Claim waits on.
type mailbox struct {
mu sync.Mutex
queues map[string][]*offer
byRun map[string]*offer
signal map[string]chan struct{}
}
func newMailbox() *mailbox {
return &mailbox{
queues: map[string][]*offer{},
byRun: map[string]*offer{},
signal: map[string]chan struct{}{},
}
}
// routedMailbox is the ONE process-wide rendezvous, shared by the coding
// delivery activity (Offer/Await) and the machine-facing HTTP surface
// (Claim/Report). One mailbox, one way.
var routedMailbox = newMailbox()
func mbKey(org, target string) string { return org + "\x00" + target }
func runKey(org, target, sess string) string { return org + "\x00" + target + "\x00" + sess }
// Offer files run for its (org,target) and returns the handle its durable owner
// awaits. A re-offer of the same (org,target,sessionID) — the workflow retrying
// or replaying after a restart — supersedes the stale prior offer (removing it
// from the queue and unblocking its dead waiter) so a machine never claims a run
// whose owner has already moved on.
func (m *mailbox) Offer(run RoutedRun) *offer {
key := mbKey(run.Org, run.TargetID)
rk := runKey(run.Org, run.TargetID, run.SessionID)
o := &offer{mb: m, key: key, rk: rk, run: run, result: make(chan RoutedResult, 1), closed: make(chan struct{})}
m.mu.Lock()
if prev := m.byRun[rk]; prev != nil {
m.removeFromQueueLocked(key, prev)
prev.finish()
}
m.byRun[rk] = o
m.queues[key] = append(m.queues[key], o)
m.broadcastLocked(key)
m.mu.Unlock()
return o
}
// Claim blocks until an unclaimed run exists for (org,target) or ctx fires,
// returning the oldest. The claimed offer leaves the queue but stays in byRun,
// awaiting Report. Only this key's queue is ever read, so a claim can never
// surface another tenant's or another machine's run.
func (m *mailbox) Claim(ctx context.Context, org, target string) (RoutedRun, bool) {
key := mbKey(org, target)
for {
m.mu.Lock()
if q := m.queues[key]; len(q) > 0 {
o := q[0]
m.queues[key] = q[1:]
m.mu.Unlock()
return o.run, true
}
sig := m.signalLocked(key)
m.mu.Unlock()
select {
case <-sig:
// a new offer (or a superseding one) arrived — re-check
case <-ctx.Done():
return RoutedRun{}, false
}
}
}
// Report delivers a terminal result to the run's durable owner. Scoped to
// (org,target,sessionID): a report can only ever complete a run that exact key
// owns, so one machine can never report on behalf of another. Returns false when
// no live offer matches (already reported, abandoned, or never existed).
func (m *mailbox) Report(org, target, sess string, res RoutedResult) bool {
rk := runKey(org, target, sess)
m.mu.Lock()
o := m.byRun[rk]
if o == nil {
m.mu.Unlock()
return false
}
delete(m.byRun, rk)
m.removeFromQueueLocked(mbKey(org, target), o)
m.mu.Unlock()
o.deliver(res)
return true
}
// discard drops an offer the owner is done with (ctx timeout / crash / normal
// close) so neither the queue nor byRun retains it.
func (m *mailbox) discard(o *offer) {
m.mu.Lock()
if m.byRun[o.rk] == o {
delete(m.byRun, o.rk)
}
m.removeFromQueueLocked(o.key, o)
m.mu.Unlock()
o.finish()
}
func (m *mailbox) removeFromQueueLocked(key string, o *offer) {
q := m.queues[key]
for i, e := range q {
if e == o {
m.queues[key] = append(q[:i:i], q[i+1:]...)
return
}
}
}
// broadcastLocked wakes every Claim waiting on key by closing its signal channel;
// a fresh channel replaces it for the next wait.
func (m *mailbox) broadcastLocked(key string) {
if ch, ok := m.signal[key]; ok {
close(ch)
delete(m.signal, key)
}
}
func (m *mailbox) signalLocked(key string) chan struct{} {
ch, ok := m.signal[key]
if !ok {
ch = make(chan struct{})
m.signal[key] = ch
}
return ch
}
func (o *offer) deliver(res RoutedResult) {
o.once.Do(func() {
o.result <- res // buffered(1) — never blocks
close(o.closed)
})
}
func (o *offer) finish() {
o.once.Do(func() { close(o.closed) })
}
// OfferRoutedRun is the exported seam the coding delivery activity uses to place
// a run into the live rendezvous. Kept here (agents owns targets + sessions) so
// the machine-facing HTTP surface and the durable activity share ONE mailbox.
func OfferRoutedRun(run RoutedRun) *offer { return routedMailbox.Offer(run) }
+166
View File
@@ -0,0 +1,166 @@
package agents
import (
"context"
"sync"
"testing"
"time"
)
func mkRun(org, target, sess string) RoutedRun {
return RoutedRun{Org: org, TargetID: target, SessionID: sess, Repo: "api", Branch: "agent/" + sess}
}
// A claimed run comes back to exactly one claimer, then its report reaches the
// offerer that is awaiting it.
func TestMailbox_OfferClaimReport(t *testing.T) {
m := newMailbox()
off := m.Offer(mkRun("acme", "tgt_1", "sess_1"))
got, ok := m.Claim(context.Background(), "acme", "tgt_1")
if !ok || got.SessionID != "sess_1" {
t.Fatalf("claim wrong: ok=%v run=%+v", ok, got)
}
done := make(chan RoutedResult, 1)
go func() {
res, _ := off.Await(context.Background())
done <- res
}()
if !m.Report("acme", "tgt_1", "sess_1", RoutedResult{OK: true, CommitSha: "abc"}) {
t.Fatal("report should deliver to the awaiting offer")
}
select {
case res := <-done:
if !res.OK || res.CommitSha != "abc" {
t.Fatalf("await got wrong result: %+v", res)
}
case <-time.After(2 * time.Second):
t.Fatal("await never received the reported result")
}
}
// THE tenant + machine boundary: a claim for (org,target) can NEVER surface a run
// offered for a different org OR a different target — it is a property of the key.
func TestMailbox_CrossTenantAndCrossMachineIsolation(t *testing.T) {
m := newMailbox()
m.Offer(mkRun("orgB", "tgt_Y", "sess_foreign_org"))
m.Offer(mkRun("acme", "tgt_Y", "sess_foreign_machine"))
m.Offer(mkRun("acme", "tgt_X", "sess_mine"))
// A claim for (acme, tgt_X) gets ONLY acme/tgt_X's run.
got, ok := m.Claim(context.Background(), "acme", "tgt_X")
if !ok || got.SessionID != "sess_mine" {
t.Fatalf("claim leaked across a boundary: ok=%v run=%+v", ok, got)
}
// And that queue is now empty — no foreign run fell through.
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
if _, ok := m.Claim(ctx, "acme", "tgt_X"); ok {
t.Fatal("a foreign run must never be claimable as acme/tgt_X")
}
// A Report can only complete a run under its exact key: reporting the foreign
// machine's session under tgt_X does nothing.
if m.Report("acme", "tgt_X", "sess_foreign_machine", RoutedResult{OK: true}) {
t.Fatal("report crossed the machine boundary")
}
if m.Report("acme", "tgt_Y", "sess_foreign_org", RoutedResult{OK: true}) {
t.Fatal("report crossed the org boundary")
}
}
// Two racing claimers, one run: exactly one wins.
func TestMailbox_NoDoubleClaim(t *testing.T) {
m := newMailbox()
m.Offer(mkRun("acme", "tgt_1", "sess_1"))
var wins int
var mu sync.Mutex
var wg sync.WaitGroup
for i := 0; i < 8; i++ {
wg.Add(1)
go func() {
defer wg.Done()
ctx, cancel := context.WithTimeout(context.Background(), 200*time.Millisecond)
defer cancel()
if _, ok := m.Claim(ctx, "acme", "tgt_1"); ok {
mu.Lock()
wins++
mu.Unlock()
}
}()
}
wg.Wait()
if wins != 1 {
t.Fatalf("exactly one claimer must win, got %d", wins)
}
}
// A claim with no work times out on ctx and reports no run — fail closed, never hang.
func TestMailbox_ClaimTimesOut(t *testing.T) {
m := newMailbox()
ctx, cancel := context.WithTimeout(context.Background(), 80*time.Millisecond)
defer cancel()
if _, ok := m.Claim(ctx, "acme", "tgt_empty"); ok {
t.Fatal("an empty mailbox must not yield a run")
}
}
// The durable owner's Await unblocks (fail-closed) when its budget ctx fires with
// no report — the machine never claimed, or claimed and died.
func TestMailbox_AwaitFailsClosedOnDeadline(t *testing.T) {
m := newMailbox()
off := m.Offer(mkRun("acme", "tgt_1", "sess_1"))
ctx, cancel := context.WithTimeout(context.Background(), 80*time.Millisecond)
defer cancel()
if _, ok := off.Await(ctx); ok {
t.Fatal("await must fail closed when the deadline fires without a report")
}
}
// A re-offer of the same run (workflow retry / cloud restart) supersedes the stale
// offer: the old waiter unblocks abandoned, and the fresh run is claimable.
func TestMailbox_ReOfferSupersedes(t *testing.T) {
m := newMailbox()
old := m.Offer(mkRun("acme", "tgt_1", "sess_1"))
// re-offer BEFORE anyone claims the first
fresh := m.Offer(mkRun("acme", "tgt_1", "sess_1"))
// old is abandoned
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
if _, ok := old.Await(ctx); ok {
t.Fatal("the superseded offer must not complete")
}
// exactly one claimable run remains, and reporting reaches the fresh offer
got, ok := m.Claim(context.Background(), "acme", "tgt_1")
if !ok || got.SessionID != "sess_1" {
t.Fatalf("fresh run not claimable: %+v", got)
}
if _, ok := m.Claim(ctxShort(), "acme", "tgt_1"); ok {
t.Fatal("the stale offer must not have left a duplicate in the queue")
}
done := make(chan struct{})
go func() { fresh.Await(context.Background()); close(done) }()
if !m.Report("acme", "tgt_1", "sess_1", RoutedResult{OK: true}) {
t.Fatal("report must reach the fresh offer")
}
<-done
}
// Report for an unknown/already-finished run is a clean false.
func TestMailbox_ReportUnknownIsNoOp(t *testing.T) {
m := newMailbox()
if m.Report("acme", "tgt_1", "nope", RoutedResult{OK: true}) {
t.Fatal("report for an unknown run must be a no-op")
}
}
func ctxShort() context.Context {
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Millisecond)
_ = cancel
return ctx
}
+179
View File
@@ -0,0 +1,179 @@
package agents
import (
"context"
"crypto/rand"
"crypto/sha256"
"crypto/subtle"
"database/sql"
"encoding/hex"
"errors"
"fmt"
"strings"
"time"
)
// routing.go is the machine-identity + liveness plane for routed runs (#48 half
// B). A run dispatched to a target is executed by an EXTERNAL machine (`hanzo
// code --serve`) that claims it over HTTP. Two properties make that safe:
//
// - MACHINE IDENTITY. A target carries a claim key — a high-entropy capability
// minted server-side, returned to the daemon ONCE, and stored only as a
// SHA-256 hash (never plaintext, like every other secret). A claim/report
// must present the key; cloud verifies it in constant time, scoped to
// (org, target). Possession of the key IS being that machine, so one machine
// can never claim another's runs even within the same org.
//
// - LIVENESS. Dispatch routes ONLY to a target with a live runner. A serve
// daemon proves liveness by polling Claim, which stamps serving_at; the gate
// rejects a target whose last poll is older than servingTTL. A run to a dead
// or absent runner fails closed at dispatch (never silently runs elsewhere),
// and one that dies mid-flight is re-queued/timed-out by the durable owner.
//
// The claim key lives in its own table so the live (A) target CRUD is untouched.
const (
// servingTTL bounds how stale a target's last claim poll may be and still be
// considered "a runner is listening". The serve daemon re-polls right after a
// 25s long-poll returns empty, so a healthy runner stamps well inside this.
servingTTL = 90 * time.Second
claimKeyPrefix = "tgtk_"
claimKeyBytes = 32 // 256-bit capability
maxClaimKey = 128
)
var (
errNoClaimKey = errors.New("agents: target has no claim key")
errClaimKeyBad = errors.New("agents: claim key mismatch")
errTargetNotLive = errors.New("agents: target has no live runner")
errTargetNotReady = errors.New("agents: target is not online")
)
// migrateClaimKeys creates the per-target claim-key + serving-liveness table in
// the SAME agents.db (one store, one tenancy column). Idempotent.
func (s *Store) migrateClaimKeys() error {
const ddl = `
CREATE TABLE IF NOT EXISTS agent_target_claim_keys (
org TEXT NOT NULL,
target_id TEXT NOT NULL,
key_hash TEXT NOT NULL,
serving_at INTEGER NOT NULL DEFAULT 0,
updated_at INTEGER NOT NULL,
PRIMARY KEY (org, target_id)
);`
if _, err := s.db.Exec(ddl); err != nil {
return fmt.Errorf("migrate claim keys: %w", err)
}
return nil
}
// UpsertClaimKeyHash stores (or rotates) a target's claim-key hash. serving_at is
// reset to 0 on a fresh mint — the daemon proves liveness by its first poll.
func (s *Store) UpsertClaimKeyHash(ctx context.Context, org, targetID, hash string, now int64) error {
_, err := s.db.ExecContext(ctx,
`INSERT INTO agent_target_claim_keys (org,target_id,key_hash,serving_at,updated_at)
VALUES (?,?,?,0,?)
ON CONFLICT(org,target_id) DO UPDATE SET key_hash=excluded.key_hash, serving_at=0, updated_at=excluded.updated_at`,
org, targetID, hash, now)
if err != nil {
return fmt.Errorf("upsert claim key: %w", err)
}
return nil
}
// ClaimKeyHash returns a target's stored hash + last serving stamp, or
// errNoClaimKey when none was ever minted.
func (s *Store) ClaimKeyHash(ctx context.Context, org, targetID string) (hash string, servingAt int64, err error) {
row := s.db.QueryRowContext(ctx,
`SELECT key_hash, serving_at FROM agent_target_claim_keys WHERE org=? AND target_id=?`, org, targetID)
err = row.Scan(&hash, &servingAt)
if errors.Is(err, sql.ErrNoRows) {
return "", 0, errNoClaimKey
}
if err != nil {
return "", 0, fmt.Errorf("get claim key: %w", err)
}
return hash, servingAt, nil
}
// StampServing records that a target's runner polled at now (its liveness
// heartbeat). Best-effort by the caller; a missing row is a no-op.
func (s *Store) StampServing(ctx context.Context, org, targetID string, now int64) error {
_, err := s.db.ExecContext(ctx,
`UPDATE agent_target_claim_keys SET serving_at=? WHERE org=? AND target_id=?`, now, org, targetID)
return err
}
// hashClaimKey is the at-rest form: SHA-256 hex of a high-entropy token. A random
// 256-bit key needs no password KDF; SHA-256 gives a fixed-size, constant-time-
// comparable digest and the plaintext is never stored.
func hashClaimKey(key string) string {
sum := sha256.Sum256([]byte(key))
return hex.EncodeToString(sum[:])
}
// newClaimKey mints a fresh capability token.
func newClaimKey() (string, error) {
b := make([]byte, claimKeyBytes)
if _, err := rand.Read(b); err != nil {
return "", err
}
return claimKeyPrefix + hex.EncodeToString(b), nil
}
// verifyClaimKey checks a presented key against the target's stored hash in
// constant time. Fail-closed: no key on file, or an empty presented key, is a
// mismatch — never an accidental pass.
func (s *Store) verifyClaimKey(ctx context.Context, org, targetID, presented string) error {
presented = strings.TrimSpace(presented)
if presented == "" || len(presented) > maxClaimKey {
return errClaimKeyBad
}
stored, _, err := s.ClaimKeyHash(ctx, org, targetID)
if err != nil {
return err // errNoClaimKey or a real DB error
}
if subtle.ConstantTimeCompare([]byte(stored), []byte(hashClaimKey(presented))) != 1 {
return errClaimKeyBad
}
return nil
}
// TargetDispatchable is the DRY liveness gate, used at dispatch (fail closed
// before enqueue) AND re-checked at claim. A run is dispatchable only to a target
// that (a) exists in this org, (b) is online, and (c) has a live runner — a claim
// poll within servingTTL. Any failure is an explicit error the dispatcher renders
// honestly; it NEVER falls back to running elsewhere.
func (s *Store) TargetDispatchable(ctx context.Context, org, targetID string) error {
t, err := s.GetTarget(ctx, org, targetID)
if err != nil {
return err // errTargetNotFound or a real DB error
}
if t.Status != TargetOnline {
return errTargetNotReady
}
_, servingAt, err := s.ClaimKeyHash(ctx, org, targetID)
if err != nil {
return errTargetNotLive // no claim key => no runner ever attached
}
if servingAt <= 0 || time.Now().Unix()-servingAt > int64(servingTTL/time.Second) {
return errTargetNotLive
}
return nil
}
// TargetDispatchable is the exported gate the coding dispatcher injects (it never
// imports the store directly). Returns nil when a run may be routed to (org,
// targetID), else a descriptive error.
func TargetDispatchable(ctx context.Context, org, targetID string) error {
if mounted == nil {
return fmt.Errorf("agents: not mounted")
}
org = strings.TrimSpace(org)
targetID = strings.TrimSpace(targetID)
if org == "" || targetID == "" {
return fmt.Errorf("agents: org and target required")
}
return mounted.State.store.TargetDispatchable(ctx, org, targetID)
}
+163
View File
@@ -0,0 +1,163 @@
package agents
import (
"context"
"net/http"
"strings"
"time"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
)
// routing_http.go is the machine-facing surface a `hanzo code --serve` daemon
// uses to CLAIM and complete routed runs. Every route is BOTH org-scoped (the
// gateway-minted X-Org-Id, exactly like the rest of the targets plane) AND
// machine-authenticated (the target claim key in X-Target-Key): a caller must
// prove it is acting in the target's org and that it holds that specific
// machine's capability. A run offered to target X is never reachable from a claim
// for target Y, and a claim for another org's target 404s at the org boundary.
//
// POST /v1/agents/targets/:id/claim-key mint/rotate this target's claim key -> {claimKey}
// POST /v1/agents/targets/:id/claim long-poll for the next routed run (X-Target-Key)
// POST /v1/agents/targets/:id/runs/:runId/report report a routed run's terminal result (X-Target-Key)
// claimLongPoll bounds one Claim wait; on expiry the daemon gets 204 and re-polls
// immediately, which also refreshes its serving liveness. A var (not a const) so a
// test can shrink the empty-poll window without waiting the full window.
var claimLongPoll = 25 * time.Second
const (
// claimKeyHeader carries the machine capability. Distinct from Authorization
// (which carries the org bearer): org identity and machine identity are two
// independent proofs, both required.
claimKeyHeader = "X-Target-Key"
maxReportField = 64 << 10
)
// mountRouting registers the route-work machine surface. Called from mountTargets
// AFTER the target CRUD routes so the extra-segment paths are unambiguous.
func mountRouting(s *cloud.Service[state], app *zip.App) {
app.Post("/v1/agents/targets/:id/claim-key", cloud.Handle(s, mintClaimKey))
app.Post("/v1/agents/targets/:id/claim", cloud.Handle(s, claimRoutedRun))
app.Post("/v1/agents/targets/:id/runs/:runId/report", cloud.Handle(s, reportRoutedRun))
}
// mintClaimKey (re)mints the target's claim key and returns it ONCE. Only the
// SHA-256 hash is stored. Org-scoped: only a caller in the target's org can mint,
// and the key is bound to (org, target). Rotating supersedes any prior daemon.
func mintClaimKey(s *cloud.Service[state], c *zip.Ctx) error {
org, ok := tenant(c)
if !ok {
return zip.ErrForbidden("X-Org-Id required")
}
id := idParam(c)
// The target must exist in this org before it can carry a capability.
if _, err := s.State.store.GetTarget(c.Context(), org, id); err == errTargetNotFound {
return zip.ErrNotFound("target not found")
} else if err != nil {
return zip.Errorf(http.StatusInternalServerError, "target: %v", err)
}
key, err := newClaimKey()
if err != nil {
return zip.Errorf(http.StatusInternalServerError, "rng: %v", err)
}
if err := s.State.store.UpsertClaimKeyHash(c.Context(), org, id, hashClaimKey(key), time.Now().Unix()); err != nil {
return zip.Errorf(http.StatusInternalServerError, "persist: %v", err)
}
return c.JSON(http.StatusOK, map[string]any{"targetId": id, "claimKey": key})
}
// claimRoutedRun authenticates the machine, refreshes its serving liveness, and
// long-polls the rendezvous for the next run addressed to THIS (org, target).
// 200 + the run on a claim; 204 when the poll window elapses with no work.
func claimRoutedRun(s *cloud.Service[state], c *zip.Ctx) error {
org, ok := tenant(c)
if !ok {
return zip.ErrForbidden("X-Org-Id required")
}
id := idParam(c)
if err := s.State.store.verifyClaimKey(c.Context(), org, id, c.Header(claimKeyHeader)); err != nil {
return claimAuthError(err)
}
// The poll itself is the runner's liveness proof — stamp it so the dispatch
// gate (TargetDispatchable) sees a live runner. Best-effort.
_ = s.State.store.StampServing(c.Context(), org, id, time.Now().Unix())
ctx, cancel := context.WithTimeout(c.Context(), claimLongPoll)
defer cancel()
run, got := routedMailbox.Claim(ctx, org, id)
if !got {
return c.NoContent(http.StatusNoContent)
}
return c.JSON(http.StatusOK, routedRunView(run))
}
type reportReq struct {
OK bool `json:"ok"`
Changed bool `json:"changed"`
Branch string `json:"branch"`
CommitSha string `json:"commitSha"`
Diffstat string `json:"diffstat"`
Error string `json:"error"`
}
// reportRoutedRun completes a claimed run: it delivers the terminal result to the
// run's durable owner (the RoutedRunWorkflow activity), which lets the workflow
// finish. Scoped to (org, target, runId) AND claim-key-authenticated, so a
// machine can only ever report a run it legitimately holds. Idempotent: a report
// for an unknown/already-finished run is a clean no-op (the session terminal was
// already set by the machine's own stream).
func reportRoutedRun(s *cloud.Service[state], c *zip.Ctx) error {
org, ok := tenant(c)
if !ok {
return zip.ErrForbidden("X-Org-Id required")
}
id := idParam(c)
runID := strings.TrimSpace(c.Param("runId"))
if err := s.State.store.verifyClaimKey(c.Context(), org, id, c.Header(claimKeyHeader)); err != nil {
return claimAuthError(err)
}
var body reportReq
if err := c.Bind(&body); err != nil {
return err
}
res := RoutedResult{
OK: body.OK, Changed: body.Changed,
Branch: clampStr(body.Branch, maxRepo),
CommitSha: clampStr(body.CommitSha, 128),
Diffstat: clampStr(body.Diffstat, maxReportField),
Error: clampStr(body.Error, maxReportField),
}
delivered := routedMailbox.Report(org, id, runID, res)
return c.JSON(http.StatusOK, map[string]any{"delivered": delivered})
}
// claimAuthError maps the claim-key verdict onto a fail-closed HTTP status. A
// missing target row, a missing/mismatched key, and an unknown org all collapse
// to 403 so the surface never distinguishes "wrong key" from "no such target" —
// an unauthorized caller learns nothing about what exists.
func claimAuthError(err error) error {
switch err {
case errNoClaimKey, errClaimKeyBad, errTargetNotFound:
return zip.ErrForbidden("target claim rejected")
default:
return zip.Errorf(http.StatusInternalServerError, "claim auth: %v", err)
}
}
// routedRunView is the non-secret run spec handed to the machine. It carries no
// credential by design — the machine authenticates git + model routing with its
// own already-held credentials.
func routedRunView(run RoutedRun) map[string]any {
return map[string]any{
"sessionId": run.SessionID,
"repo": run.Repo,
"project": run.Project,
"base": run.Base,
"branch": run.Branch,
"prompt": run.Prompt,
"cloneUrl": run.CloneURL,
"timeoutSeconds": run.TimeoutSeconds,
}
}
+250
View File
@@ -0,0 +1,250 @@
package agents
import (
"bytes"
"context"
"encoding/json"
"io"
"net/http/httptest"
"testing"
"time"
"github.com/zap-proto/zip"
)
// doKey is a keyless-body request with a machine claim key (X-Target-Key) attached.
func doKey(t *testing.T, app *zip.App, method, path, org, key string) (int, []byte) {
t.Helper()
req := httptest.NewRequest(method, path, nil)
if org != "" {
req.Header.Set("X-Org-Id", org)
req.Header.Set("X-User-Id", "u-"+org)
}
if key != "" {
req.Header.Set(claimKeyHeader, key)
}
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("Test %s %s: %v", method, path, err)
}
defer func() { _ = resp.Body.Close() }()
b, _ := io.ReadAll(resp.Body)
return resp.StatusCode, b
}
// registerAndMint registers a target for org and mints its claim key, returning
// (targetID, claimKey).
func registerAndMint(t *testing.T, app *zip.App, org, host string) (string, string) {
t.Helper()
code, body := do(t, app, "POST", "/v1/agents/targets", org, map[string]any{"label": host, "host": host})
if code != 201 && code != 200 {
t.Fatalf("register target: %d %s", code, body)
}
var tv struct{ ID string `json:"id"` }
_ = json.Unmarshal(body, &tv)
code, body = doKey(t, app, "POST", "/v1/agents/targets/"+tv.ID+"/claim-key", org, "")
if code != 200 {
t.Fatalf("mint claim key: %d %s", code, body)
}
var kv struct{ ClaimKey string `json:"claimKey"` }
_ = json.Unmarshal(body, &kv)
if kv.ClaimKey == "" {
t.Fatal("claim key empty")
}
return tv.ID, kv.ClaimKey
}
// A claim without the machine's key, or with the WRONG key, is refused — org
// membership alone is not enough to claim a machine's runs.
func TestClaim_RequiresMachineKey(t *testing.T) {
app := mountApp(t, &fakeAI{content: "x"})
old := claimLongPoll
claimLongPoll = 150 * time.Millisecond
defer func() { claimLongPoll = old }()
id, key := registerAndMint(t, app, "acme", "evo")
// No key => 403.
if code, _ := doKey(t, app, "POST", "/v1/agents/targets/"+id+"/claim", "acme", ""); code != 403 {
t.Fatalf("claim with no key must be 403, got %d", code)
}
// Wrong key => 403.
if code, _ := doKey(t, app, "POST", "/v1/agents/targets/"+id+"/claim", "acme", "tgtk_wrong"); code != 403 {
t.Fatalf("claim with wrong key must be 403, got %d", code)
}
// Right key, no work => 204 (never 200, never another tenant's run).
if code, _ := doKey(t, app, "POST", "/v1/agents/targets/"+id+"/claim", "acme", key); code != 204 {
t.Fatalf("claim with right key + no work must be 204, got %d", code)
}
}
// THE machine boundary: a key minted for target A cannot claim target B, and a
// different org cannot claim at all — even with a real key for its own target.
func TestClaim_CrossMachineAndCrossOrgDenied(t *testing.T) {
app := mountApp(t, &fakeAI{content: "x"})
old := claimLongPoll
claimLongPoll = 150 * time.Millisecond
defer func() { claimLongPoll = old }()
idA, keyA := registerAndMint(t, app, "acme", "evoA")
idB, _ := registerAndMint(t, app, "acme", "evoB")
// A's key against B => 403 (constant-time mismatch on B's stored hash).
if code, _ := doKey(t, app, "POST", "/v1/agents/targets/"+idB+"/claim", "acme", keyA); code != 403 {
t.Fatalf("A's key claiming B must be 403, got %d", code)
}
// Offer a run for acme/idA, then a DIFFERENT org cannot claim idA at all (its
// org scope resolves no such target => 403), and the run is never handed out.
OfferRoutedRun(RoutedRun{Org: "acme", TargetID: idA, SessionID: "sess_a", Repo: "api"})
if code, _ := doKey(t, app, "POST", "/v1/agents/targets/"+idA+"/claim", "evil", keyA); code != 403 {
t.Fatalf("another org claiming acme's target must be 403, got %d", code)
}
// acme WITH A's key claims its own run.
code, body := doKey(t, app, "POST", "/v1/agents/targets/"+idA+"/claim", "acme", keyA)
if code != 200 {
t.Fatalf("acme must claim its own run, got %d %s", code, body)
}
var rv struct{ SessionID string `json:"sessionId"` }
_ = json.Unmarshal(body, &rv)
if rv.SessionID != "sess_a" {
t.Fatalf("claimed wrong run: %s", body)
}
}
// The end-to-end machine round trip: offer -> claim -> report reaches the durable
// owner awaiting the result.
func TestClaimReport_RoundTrip(t *testing.T) {
app := mountApp(t, &fakeAI{content: "x"})
id, key := registerAndMint(t, app, "acme", "evo")
off := OfferRoutedRun(RoutedRun{Org: "acme", TargetID: id, SessionID: "sess_rt", Repo: "api", Branch: "agent/rt"})
code, body := doKey(t, app, "POST", "/v1/agents/targets/"+id+"/claim", "acme", key)
if code != 200 {
t.Fatalf("claim: %d %s", code, body)
}
got := make(chan RoutedResult, 1)
go func() { res, _ := off.Await(context.Background()); got <- res }()
code, _ = doKeyBody(t, app, "POST", "/v1/agents/targets/"+id+"/runs/sess_rt/report", "acme", key,
map[string]any{"ok": true, "changed": true, "commitSha": "cafe"})
if code != 200 {
t.Fatalf("report: %d", code)
}
select {
case res := <-got:
if !res.OK || res.CommitSha != "cafe" {
t.Fatalf("report did not reach the owner: %+v", res)
}
case <-time.After(2 * time.Second):
t.Fatal("owner never received the report")
}
}
// doKeyBody is doKey with a JSON body.
func doKeyBody(t *testing.T, app *zip.App, method, path, org, key string, body any) (int, []byte) {
t.Helper()
b, _ := json.Marshal(body)
req := httptest.NewRequest(method, path, bytes.NewReader(b))
req.Header.Set("Content-Type", "application/json")
if org != "" {
req.Header.Set("X-Org-Id", org)
req.Header.Set("X-User-Id", "u-"+org)
}
if key != "" {
req.Header.Set(claimKeyHeader, key)
}
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("Test %s %s: %v", method, path, err)
}
defer func() { _ = resp.Body.Close() }()
out, _ := io.ReadAll(resp.Body)
return resp.StatusCode, out
}
// ---- store-level liveness gate ----
// TargetDispatchable is the fail-closed gate: online + a live runner (a recent
// claim poll). Offline, no key, or a stale poll all reject.
func TestTargetDispatchable_LivenessGate(t *testing.T) {
s := testSessionStore(t)
ctx := context.Background()
now := time.Now().Unix()
tgt := Target{ID: "t1", Org: "acme", Label: "evo", Kind: TargetMachine, Status: TargetOnline, Host: "evo", CreatedAt: now, UpdatedAt: now}
if err := s.CreateTarget(ctx, tgt); err != nil {
t.Fatal(err)
}
// No claim key yet => not live => not dispatchable.
if err := s.TargetDispatchable(ctx, "acme", "t1"); err != errTargetNotLive {
t.Fatalf("no runner => not dispatchable, got %v", err)
}
// Mint + a fresh serving stamp => dispatchable.
if err := s.UpsertClaimKeyHash(ctx, "acme", "t1", hashClaimKey("k"), now); err != nil {
t.Fatal(err)
}
if err := s.StampServing(ctx, "acme", "t1", now); err != nil {
t.Fatal(err)
}
if err := s.TargetDispatchable(ctx, "acme", "t1"); err != nil {
t.Fatalf("online + fresh runner => dispatchable, got %v", err)
}
// A stale serving stamp => not dispatchable (dead runner).
if err := s.StampServing(ctx, "acme", "t1", now-int64(servingTTL/time.Second)-5); err != nil {
t.Fatal(err)
}
if err := s.TargetDispatchable(ctx, "acme", "t1"); err != errTargetNotLive {
t.Fatalf("stale runner => not dispatchable, got %v", err)
}
// Fresh again but OFFLINE => not dispatchable.
_ = s.StampServing(ctx, "acme", "t1", time.Now().Unix())
tgt.Status = TargetOffline
tgt.UpdatedAt = time.Now().Unix()
if err := s.UpdateTarget(ctx, tgt); err != nil {
t.Fatal(err)
}
if err := s.TargetDispatchable(ctx, "acme", "t1"); err != errTargetNotReady {
t.Fatalf("offline => not dispatchable, got %v", err)
}
// Unknown target / cross-org => fail closed.
if err := s.TargetDispatchable(ctx, "acme", "nope"); err != errTargetNotFound {
t.Fatalf("unknown target => not found, got %v", err)
}
if err := s.TargetDispatchable(ctx, "evil", "t1"); err != errTargetNotFound {
t.Fatalf("cross-org => not found, got %v", err)
}
}
// The claim key is stored ONLY as a hash; verify is constant-time + fail closed.
func TestClaimKey_HashedAtRestAndVerified(t *testing.T) {
s := testSessionStore(t)
ctx := context.Background()
now := time.Now().Unix()
_ = s.CreateTarget(ctx, Target{ID: "t1", Org: "acme", Status: TargetOnline, CreatedAt: now, UpdatedAt: now})
key, _ := newClaimKey()
if err := s.UpsertClaimKeyHash(ctx, "acme", "t1", hashClaimKey(key), now); err != nil {
t.Fatal(err)
}
// The stored value is a hash, never the plaintext.
stored, _, _ := s.ClaimKeyHash(ctx, "acme", "t1")
if stored == key || stored != hashClaimKey(key) {
t.Fatalf("claim key must be stored as a hash, not plaintext")
}
if err := s.verifyClaimKey(ctx, "acme", "t1", key); err != nil {
t.Fatalf("correct key must verify: %v", err)
}
if err := s.verifyClaimKey(ctx, "acme", "t1", "tgtk_wrong"); err != errClaimKeyBad {
t.Fatalf("wrong key must fail: %v", err)
}
if err := s.verifyClaimKey(ctx, "acme", "t1", ""); err != errClaimKeyBad {
t.Fatalf("empty key must fail: %v", err)
}
// Cross-org verify resolves no key => fail closed.
if err := s.verifyClaimKey(ctx, "evil", "t1", key); err != errNoClaimKey {
t.Fatalf("cross-org verify must fail closed, got %v", err)
}
}
+4
View File
@@ -180,6 +180,10 @@ CREATE INDEX IF NOT EXISTS ix_runs_org_agent_created ON agent_runs(org, agent_na
if err := s.migrateTargets(); err != nil {
return err
}
// Per-target claim keys + serving liveness (the #48 route-work machine plane).
if err := s.migrateClaimKeys(); err != nil {
return err
}
return nil
}
+183 -4
View File
@@ -10,6 +10,8 @@ import (
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/principal"
"github.com/hanzoai/cloud/clients/samples"
"github.com/zap-proto/zip"
)
@@ -17,10 +19,11 @@ import (
// box, a GPU host, or a whole cluster. It is the #48 link-a-compute seam over the
// SAME agents.db (one store, one tenancy column) as sessions/events — NOT a rival
// device registry. It composes with the compute fleet rather than duplicating it: a
// session records the target id it runs on (agent_sessions.target), and the mission-
// control devices view unions these registered targets with the org's BYO workers
// (GET /v1/fleet/workers) and BYO clusters (GET /v1/clusters) at the view layer — the
// console's established pattern for folding compute sources.
// session records the target id it runs on (agent_sessions.target), and the org's
// unified board (GET /v1/fleet, clients/visor/board.go) unions these registered
// targets with its BYO workers (GET /v1/fleet/workers), BYO clusters and Visor
// machines — reading this registry through the in-process seam below rather than
// copying it.
//
// POST /v1/agents/targets register a target -> Target
// GET /v1/agents/targets list the org's targets (+ live session load)
@@ -30,6 +33,12 @@ import (
//
// Every route is org-scoped through principal.Org (tenant), fail-closed — a tenant
// can never see or mutate another org's targets, exactly like sessions.
//
// A write carrying `metrics` IS a heartbeat, and a heartbeat is two facts, not one:
// the LAST sample (kept on the row, rendered by the views here) and one point in a
// utilization SERIES (appended to clients/samples). The row answers "is this machine
// alive and what is it doing now"; the series answers "how hot has it been". The
// append is best-effort and detached — see recordSample.
// Target kinds — the closed vocabulary of dispatch destinations.
const (
@@ -233,6 +242,90 @@ func (s *Store) GetTargetByHost(ctx context.Context, org, host string) (Target,
return t, nil
}
// ---- the in-process seam (org-scoped, fail-closed) ----
//
// TargetsForOrg / LoadOn are the exported twins of the list + detail reads above:
// the ONE way another in-process subsystem (the /v1/fleet board in clients/visor)
// reads this registry WITHOUT an HTTP hop back through the gateway — the same
// shape ListForOrg gives the agent registry. They are two ORTHOGONAL values on
// purpose: a target is what the machine IS, its load is what is running on it, and
// a caller that only needs the inventory does not pay for the rollups.
//
// ISOLATION: org is the ONLY tenant key and is threaded verbatim into the
// org-scoped store methods, so a caller for org A can never enumerate or resolve
// org B's targets. The caller MUST pass an org it already validated server-side
// (principal.Org), never a raw client header.
// TargetsForOrg returns the org's registered run-targets from the in-process
// store, newest first. Fails closed when the subsystem is not mounted or the org
// is empty/oversized.
func TargetsForOrg(ctx context.Context, org string) ([]Target, error) {
if mounted == nil || mounted.State.store == nil {
return nil, fmt.Errorf("agents: not mounted")
}
org = strings.TrimSpace(org)
if org == "" || len(org) > principal.MaxOrgLen {
return nil, fmt.Errorf("agents: invalid org")
}
return mounted.State.store.ListTargets(ctx, org)
}
// ResolveTarget resolves a human's target REFERENCE — a target id or its friendly
// label (the hostname the CLI registers) — to the org's target, org-scoped and
// fail-closed. It is the ONE way a trigger surface (the Slack `code: <repo> on
// <target>` grammar, a console picker) turns "on evo" into a target id without
// leaking another tenant's inventory: an id or label that resolves to no target in
// THIS org returns errTargetNotFound, never another org's machine.
//
// Precedence: an exact id match wins (ids are unambiguous), else an exact,
// case-folded label match (newest first, so a re-registered machine's live row is
// preferred). A reference that matches neither is not found — the caller renders an
// honest error and NEVER falls back to a local run.
func ResolveTarget(ctx context.Context, org, ref string) (Target, error) {
if mounted == nil || mounted.State.store == nil {
return Target{}, fmt.Errorf("agents: not mounted")
}
org = strings.TrimSpace(org)
ref = strings.TrimSpace(ref)
if org == "" || len(org) > principal.MaxOrgLen {
return Target{}, fmt.Errorf("agents: invalid org")
}
if ref == "" || len(ref) > maxTargetID {
return Target{}, errTargetNotFound
}
// An id is exact and unambiguous — try it first.
if t, err := mounted.State.store.GetTarget(ctx, org, ref); err == nil {
return t, nil
} else if err != errTargetNotFound {
return Target{}, err
}
// Else an exact, case-folded label match within this org.
rows, err := mounted.State.store.ListTargets(ctx, org)
if err != nil {
return Target{}, err
}
for _, t := range rows { // ListTargets is newest-first: the live row wins a label tie
if strings.EqualFold(strings.TrimSpace(t.Label), ref) {
return t, nil
}
}
return Target{}, errTargetNotFound
}
// LoadOn returns the live session load on one of the org's targets — the same
// (target id OR host) mapping the HTTP views use, so the board and /v1/agents/
// targets can never disagree about what is running where.
func LoadOn(ctx context.Context, org, id, host string) (TargetLoad, error) {
if mounted == nil || mounted.State.store == nil {
return TargetLoad{}, fmt.Errorf("agents: not mounted")
}
org = strings.TrimSpace(org)
if org == "" || len(org) > principal.MaxOrgLen {
return TargetLoad{}, fmt.Errorf("agents: invalid org")
}
return mounted.State.store.SessionLoad(ctx, org, id, host)
}
// DeleteTarget removes an org's target. Sessions keep their recorded target id (a
// historical fact); a detached target simply stops appearing in the registry.
func (s *Store) DeleteTarget(ctx context.Context, org, id string) (bool, error) {
@@ -262,6 +355,83 @@ func (s *Store) SessionLoad(ctx context.Context, org, id, host string) (TargetLo
return TargetLoad{Sessions: total, Running: running}, nil
}
// ---- the fleet time series ----
//
// A heartbeat is the ONE moment this process learns what a linked machine is
// doing, so it is also where the fleet's utilization series is fed. The target row
// keeps the LAST sample (the snapshot the views render, unchanged); clients/samples
// keeps every sample over time. Two different questions — "is it alive now" and
// "how hot has it been" — so two homes, one write.
// sampleTimeout bounds the warehouse write. Generous (the insert is one small row
// in-cluster) but finite, so a wedged datastore can never hold the goroutine open.
const sampleTimeout = 5 * time.Second
// sampleOf projects a target's server-stamped heartbeat into a fleet sample. PURE
// (no clock, no I/O, no store) so the whole projection is unit-testable and the
// caller decides when it runs.
//
// cost_cents is 0: an agent run-target is the operator's OWN machine (a laptop, a
// dialed-in box) — the fleet meters its utilization, it does not resell it. A
// priced source (visor/cloud) fills that column from its own resale price.
func sampleOf(t Target) samples.Sample {
var model string
if len(t.Spec.GPUs) > 0 {
// The representative accelerator: the count already rides in GPUs, so the
// first card's model names the row. A heterogeneous host is rare enough
// that naming its first card beats inventing a summary string here.
model = t.Spec.GPUs[0].Model
if model == "" {
model = t.Spec.GPUs[0].Vendor
}
}
return samples.Sample{
Org: t.Org,
Source: samples.SourceAgent,
Unit: t.ID,
Host: t.Host,
Kind: t.Kind,
At: time.Unix(t.MetricsAt, 0).UTC(),
CPUs: t.Spec.CPUs,
Memory: t.Spec.Memory,
MemUsed: t.Metrics.MemUsed,
MemFree: t.Metrics.MemFree,
Load1: t.Metrics.Load1,
Load5: t.Metrics.Load5,
Load15: t.Metrics.Load15,
GPUUtil: t.Metrics.GPUUtil,
GPUs: len(t.Spec.GPUs),
GPUModel: model,
}
}
// recordSample appends a heartbeat to the fleet series. Best-effort and DETACHED
// on purpose — the warehouse is never in the heartbeat's critical path:
//
// - it runs on its own bounded context, so neither a slow datastore nor the
// client hanging up mid-request can stall or cancel the write;
// - it never touches the response, so the /v1/agents/targets contract is
// byte-identical whether the warehouse is present, absent or on fire;
// - a failure is logged, never surfaced — a dropped sample must not cost a
// machine its heartbeat.
//
// This is the shape the billing warehouse write already uses (`go zapWriteUsage`):
// the seam is synchronous, the CALLER owns the concurrency.
func recordSample(s *cloud.Service[state], t Target) {
if t.MetricsAt == 0 {
return // no heartbeat in this write — nothing to append
}
sample := sampleOf(t) // project on the caller's goroutine: t must not escape mutably
go func() {
ctx, cancel := context.WithTimeout(context.Background(), sampleTimeout)
defer cancel()
if err := samples.Record(ctx, sample); err != nil {
s.Log.Warn("fleet sample write failed", "org", sample.Org, "unit", sample.Unit, "err", err)
}
}()
}
// ---- HTTP shapes (the published contract) ----
type targetView struct {
@@ -311,6 +481,10 @@ func mountTargets(s *cloud.Service[state], app *zip.App) {
app.Get("/v1/agents/targets/:id", cloud.Handle(s, getTarget))
app.Patch("/v1/agents/targets/:id", cloud.Handle(s, patchTarget))
app.Delete("/v1/agents/targets/:id", cloud.Handle(s, deleteTarget))
// The #48 route-work machine surface (claim-key, claim long-poll, report)
// lives on the same target routes; register after the CRUD so the
// extra-segment paths are unambiguous.
mountRouting(s, app)
}
// ---- register ----
@@ -386,6 +560,7 @@ func registerTarget(s *cloud.Service[state], c *zip.Ctx) error {
if err := s.State.store.UpdateTarget(c.Context(), existing); err != nil {
return zip.Errorf(http.StatusInternalServerError, "persist: %v", err)
}
recordSample(s, existing) // a re-link carrying metrics IS a heartbeat
load, _ := s.State.store.SessionLoad(c.Context(), org, existing.ID, existing.Host)
return c.JSON(http.StatusOK, toTargetView(existing, load))
}
@@ -403,6 +578,7 @@ func registerTarget(s *cloud.Service[state], c *zip.Ctx) error {
if err := s.State.store.CreateTarget(c.Context(), t); err != nil {
return zip.Errorf(http.StatusInternalServerError, "persist: %v", err)
}
recordSample(s, t) // a registration carrying metrics is the target's first sample
return c.JSON(http.StatusCreated, toTargetView(t, TargetLoad{}))
}
@@ -538,6 +714,9 @@ func patchTarget(s *cloud.Service[state], c *zip.Ctx) error {
}
return zip.Errorf(http.StatusInternalServerError, "update: %v", err)
}
if body.Metrics != nil {
recordSample(s, t) // THE heartbeat: append it to the fleet series too
}
load, _ := s.State.store.SessionLoad(c.Context(), org, t.ID, t.Host)
return c.JSON(http.StatusOK, toTargetView(t, load))
}
+227
View File
@@ -0,0 +1,227 @@
package agents
import (
"encoding/json"
"net/http"
"testing"
"time"
"github.com/hanzoai/cloud/clients/samples"
)
// targetsample_test.go covers the FIRST emitter: a run-target heartbeat also
// appends to the fleet series (clients/samples).
//
// The datastore is absent under test, so samples.Record is a proven no-op (its own
// package tests that). What MUST be proven here is everything this side owns:
// the projection is faithful, the vocabularies agree, and the HTTP contract is
// untouched whether or not the warehouse exists.
// ---- the projection (pure) ----
// A heartbeat projects onto a sample with no loss and no invention.
func TestSampleOfProjectsTheHeartbeat(t *testing.T) {
at := time.Now().Unix()
tg := Target{
ID: "tgt-1", Org: "acme", Kind: TargetGPU, Host: "box.local", Label: "Box",
Spec: Spec{OS: "linux", Arch: "arm64", CPUs: 20, Memory: 128 << 30,
GPUs: []GPU{{Vendor: "nvidia", Model: "GB10", Memory: 96 << 30}}},
Metrics: Metrics{Load1: 2.5, Load5: 2, Load15: 1.5,
MemUsed: 64 << 30, MemFree: 64 << 30, GPUUtil: 0.75},
MetricsAt: at,
}
s := sampleOf(tg)
if s.Org != "acme" || s.Unit != "tgt-1" || s.Host != "box.local" {
t.Fatalf("identity did not project: %+v", s)
}
if s.Source != samples.SourceAgent {
t.Fatalf("source want %q, got %q", samples.SourceAgent, s.Source)
}
if s.Kind != TargetGPU {
t.Fatalf("kind want %q, got %q", TargetGPU, s.Kind)
}
if !s.At.Equal(time.Unix(at, 0).UTC()) {
t.Fatalf("at must be the SERVER-stamped heartbeat clock, got %v", s.At)
}
if s.CPUs != 20 || s.Memory != 128<<30 {
t.Fatalf("spec did not project: %+v", s)
}
if s.MemUsed != 64<<30 || s.MemFree != 64<<30 || s.Load1 != 2.5 || s.Load5 != 2 || s.Load15 != 1.5 {
t.Fatalf("metrics did not project: %+v", s)
}
if s.GPUUtil != 0.75 || s.GPUs != 1 || s.GPUModel != "GB10" {
t.Fatalf("gpu did not project: %+v", s)
}
// An agent's own machine is metered, never resold.
if s.CostCents != 0 {
t.Fatalf("an agent sample must be unpriced, got %d", s.CostCents)
}
// The projection must be acceptable to the plane it feeds.
if err := samples.Record(t.Context(), s); err != nil {
t.Fatalf("a projected sample must be recordable: %v", err)
}
}
// The accelerator count comes from the spec, and the row is named by the first
// card's model — falling back to its vendor when the model is unknown.
func TestSampleOfGPUSummary(t *testing.T) {
cases := []struct {
name string
gpus []GPU
wantN int
wantModel string
}{
{"none", nil, 0, ""},
{"model", []GPU{{Vendor: "nvidia", Model: "GB10"}}, 1, "GB10"},
{"vendor fallback", []GPU{{Vendor: "amd"}}, 1, "amd"},
{"multi is counted, first names it", []GPU{{Model: "GB10"}, {Model: "GB10"}}, 2, "GB10"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
s := sampleOf(Target{ID: "t", Org: "o", Kind: TargetGPU, Spec: Spec{GPUs: tc.gpus}, MetricsAt: 1})
if s.GPUs != tc.wantN || s.GPUModel != tc.wantModel {
t.Fatalf("want (%d, %q), got (%d, %q)", tc.wantN, tc.wantModel, s.GPUs, s.GPUModel)
}
})
}
}
// THE cross-package contract: every kind a target can be must be a kind the fleet
// series accepts, or heartbeats would silently stop being recorded. This fails the
// day someone adds a target kind without teaching the series about it.
func TestEveryTargetKindIsAFleetKind(t *testing.T) {
fleet := map[string]bool{
samples.KindLaptop: true, samples.KindCloud: true, samples.KindGPU: true,
samples.KindCluster: true, samples.KindMachine: true, samples.KindWorker: true,
}
for _, k := range []string{TargetLaptop, TargetCloud, TargetGPU, TargetCluster, TargetMachine} {
if !fleet[k] {
t.Fatalf("target kind %q is not a fleet sample kind — its heartbeats would be dropped", k)
}
// Proven end to end: a sample carrying this kind validates.
s := sampleOf(Target{ID: "t", Org: "o", Kind: k, MetricsAt: 1})
if err := samples.Record(t.Context(), s); err != nil {
t.Fatalf("kind %q must be recordable: %v", k, err)
}
}
}
// A write with no heartbeat in it appends nothing — recordSample is a no-op when
// the server never stamped a metrics clock.
func TestRecordSampleSkipsWhenNoHeartbeat(t *testing.T) {
mountApp(t, nil) // sets the `mounted` singleton recordSample logs through
// No panic, no goroutine, no write: MetricsAt == 0 means "no sample here".
recordSample(mounted, Target{ID: "tgt-1", Org: "acme", Kind: TargetGPU, MetricsAt: 0})
}
// ---- (c) the HTTP contract is untouched by the series ----
// The heartbeat still 200s with no warehouse, and still returns the snapshot on
// the row exactly as before — the series is strictly additive.
func TestHeartbeatStill200sWithoutDatastore(t *testing.T) {
app := mountApp(t, nil)
code, body := do(t, app, http.MethodPost, "/v1/agents/targets", "acme", map[string]any{
"label": "Box", "kind": TargetGPU, "host": "box.local",
"spec": map[string]any{"os": "linux", "cpus": 20, "gpus": []map[string]any{{"vendor": "nvidia", "model": "GB10"}}},
"metrics": map[string]any{"load1": 2.5, "gpuUtil": 0.75, "memUsed": 100},
})
if code != http.StatusCreated {
t.Fatalf("register want 201 without a datastore, got %d (%s)", code, body)
}
var created targetView
if err := json.Unmarshal(body, &created); err != nil {
t.Fatalf("shape: %v (%s)", err, body)
}
if created.Metrics == nil || created.Metrics.GPUUtil != 0.75 {
t.Fatalf("the snapshot on the row must be unchanged: %+v", created.Metrics)
}
if created.MetricsAt == "" {
t.Fatal("the server must still stamp the heartbeat clock")
}
// The heartbeat itself.
code, body = do(t, app, http.MethodPatch, "/v1/agents/targets/"+created.ID, "acme", map[string]any{
"metrics": map[string]any{"load1": 4, "gpuUtil": 0.9, "memUsed": 200},
})
if code != http.StatusOK {
t.Fatalf("heartbeat want 200 without a datastore, got %d (%s)", code, body)
}
var beat targetView
if err := json.Unmarshal(body, &beat); err != nil {
t.Fatalf("shape: %v (%s)", err, body)
}
if beat.Metrics == nil || beat.Metrics.GPUUtil != 0.9 || beat.Metrics.Load1 != 4 {
t.Fatalf("the heartbeat must still refresh the row snapshot: %+v", beat.Metrics)
}
// A re-link (same org+host) is idempotent and still carries a heartbeat.
code, body = do(t, app, http.MethodPost, "/v1/agents/targets", "acme", map[string]any{
"label": "Box", "kind": TargetGPU, "host": "box.local",
"metrics": map[string]any{"load1": 1},
})
if code != http.StatusOK {
t.Fatalf("re-link want 200 (idempotent), got %d (%s)", code, body)
}
var relinked targetView
if err := json.Unmarshal(body, &relinked); err != nil {
t.Fatalf("shape: %v (%s)", err, body)
}
if relinked.ID != created.ID {
t.Fatalf("a re-link must refresh the SAME target: %s != %s", relinked.ID, created.ID)
}
}
// ---- the in-process seam ----
// TargetsForOrg / LoadOn are org-keyed and fail closed — the board reads through
// them, so a cross-tenant id must never resolve.
func TestInProcessSeamIsOrgScopedAndFailsClosed(t *testing.T) {
app := mountApp(t, nil)
code, body := do(t, app, http.MethodPost, "/v1/agents/targets", "acme", map[string]any{
"label": "Secret", "kind": TargetGPU, "host": "secret.local",
})
if code != http.StatusCreated {
t.Fatalf("register: %d (%s)", code, body)
}
var created targetView
_ = json.Unmarshal(body, &created)
// The owner sees it.
own, err := TargetsForOrg(t.Context(), "acme")
if err != nil {
t.Fatalf("TargetsForOrg(acme): %v", err)
}
if len(own) != 1 || own[0].ID != created.ID {
t.Fatalf("the owner must see its target, got %+v", own)
}
// Another tenant sees nothing — the same id is unreachable.
other, err := TargetsForOrg(t.Context(), "other")
if err != nil {
t.Fatalf("TargetsForOrg(other): %v", err)
}
if len(other) != 0 {
t.Fatalf("CROSS-TENANT LEAK: org 'other' enumerated %+v", other)
}
// A blank/oversized org fails closed on both.
for _, bad := range []string{"", " "} {
if _, err := TargetsForOrg(t.Context(), bad); err == nil {
t.Fatalf("TargetsForOrg(%q) must fail closed", bad)
}
if _, err := LoadOn(t.Context(), bad, created.ID, ""); err == nil {
t.Fatalf("LoadOn(%q) must fail closed", bad)
}
}
// LoadOn is org-keyed too: the foreign tenant resolves no load for the id.
load, err := LoadOn(t.Context(), "other", created.ID, "secret.local")
if err != nil {
t.Fatalf("LoadOn(other): %v", err)
}
if load.Sessions != 0 || load.Running != 0 {
t.Fatalf("CROSS-TENANT LEAK: foreign load %+v", load)
}
}
+1 -1
View File
@@ -11,7 +11,7 @@ import (
"strings"
"time"
openai "github.com/sashabaranov/go-openai"
openai "github.com/hanzoai/go-openai"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/codes"
+15 -3
View File
@@ -100,12 +100,24 @@ func routes(app *zip.App, s *cloud.Service[state]) {
app.Get("/v1/analytics/timeseries", cloud.Handle(s, timeseries))
app.Get("/v1/analytics/top", cloud.Handle(s, top))
// Capture (WRITE) side — the ingest that fills hanzo.events (capture.go). All
// POST, all tenant-gated in-handler. /v1/tracker is the page-unload beacon
// alias (bare route; never collides with the /v1/tracker/projects* issue tracker).
// Capture (WRITE) side — the ingest that fills hanzo.events. POST /v1/event
// (event.go) is the ONE canonical front door: body Event | [Event], org
// resolved IAM-only and fail-closed, into the ONE write core (ingestEvents).
app.Post("/v1/event", cloud.Handle(s, eventIngest))
// DEPRECATED ingest aliases — thin wire adapters that normalize onto the SAME
// write core (log a one-shot deprecation, keep working). /v1/analytics{,/batch}
// and /v1/tracker speak the Segment/beacon CaptureBatch wire; /v1/tracker is a
// bare route (never collides with the /v1/tracker/projects* issue tracker).
app.Post("/v1/analytics", cloud.Handle(s, capture))
app.Post("/v1/analytics/batch", cloud.Handle(s, capture))
app.Post("/v1/tracker", cloud.Handle(s, capture))
// /v1/insights — console reads over the SAME engine + the DEPRECATED PostHog-
// wire ingest adapter (/v1/insights/e → the ONE write core). Flags live at /v1/flags.
app.Get("/v1/insights/health", cloud.Handle(s, insightsHealth))
app.Post("/v1/insights/e", cloud.Handle(s, insightsIngest))
app.Get("/v1/insights/events", cloud.Handle(s, insightsEvents))
}
// ── shared helpers ──────────────────────────────────────────────────────────
+143 -29
View File
@@ -52,6 +52,7 @@ import (
"os"
"regexp"
"strings"
"sync"
"sync/atomic"
"time"
@@ -498,19 +499,72 @@ func publicCaptureEnabled() bool {
}
}
// captureTenant resolves the tenant a batch is attributed to. A VALIDATED
// principal always wins (authenticated product traffic → its own org). Otherwise,
// for anonymous marketing traffic, the tenant is the PUBLIC brand org derived
// SERVER-SIDE from the request Host via the white-label registry — never a
// client-claimed org, so the isolation invariant holds: a caller with no bearer
// can only ever write into the brand-public partition of the Host it actually
// reached, and a forged X-Org-Id is ignored exactly as on the read path. An
// unrecognized Host is refused (we never dump anonymous events into a default
// org). Returns ("", false) when the caller must be answered 403.
// resolveKeyOrg maps a presented project/API key to its org through the ONE IAM
// key seam (cloud.OrgForKey). It is a package var ONLY so a test can substitute a
// resolver without standing up IAM; production is always cloud.OrgForKey.
var resolveKeyOrg = cloud.OrgForKey
// projectKey returns the project/API key a keyed SDK presents OUT-OF-BAND of the
// Authorization header — the transports SanitizeIdentity does NOT mint identity
// from, so they never reach tenant() as a principal. In priority order: the
// ?api_key= query, the x-api-key / api-key headers, and the PostHog-wire body
// field `api_key` (posthog-js and the insights-go batch envelope put it there).
// "" when none is present.
//
// The body is PEEKED via c.Body() — fasthttp buffers the full body, so the later
// c.Bind in the handler re-reads the same bytes; peeking does not consume it. Only
// the api_key field is decoded (a bad/unrelated JSON body simply yields "").
func projectKey(c *zip.Ctx) string {
if k := trim(c.Query("api_key")); k != "" {
return k
}
if k := trim(c.Header("x-api-key")); k != "" {
return k
}
if k := trim(c.Header("api-key")); k != "" {
return k
}
if body := c.Body(); len(body) > 0 {
var probe struct {
APIKey string `json:"api_key"`
}
if json.Unmarshal(body, &probe) == nil {
if k := trim(probe.APIKey); k != "" {
return k
}
}
}
return ""
}
// captureTenant resolves the tenant a batch is attributed to, in strict trust
// order:
//
// 1. A VALIDATED principal always wins (authenticated product traffic → its own
// org; this also covers a Hanzo key sent as a bearer, which SanitizeIdentity
// has already resolved to a principal upstream).
// 2. Otherwise, if the caller PRESENTS a project key out-of-band (posthog-js /
// insights-go: api_key in the body/query/x-api-key), resolve it to its org
// through the ONE IAM key seam. This FAILS CLOSED: a presented-but-unresolvable
// key is refused (→ 403), NEVER falling through to the brand-host fallback —
// attributing a keyed request to the wrong (brand-public) partition would be a
// cross-tenant write.
// 3. Only for TRULY anonymous traffic (no principal, no key) is the tenant the
// PUBLIC brand org derived SERVER-SIDE from the request Host via the
// white-label registry — never a client-claimed org. An unrecognized Host is
// refused (we never dump anonymous events into a default org).
//
// Returns ("", false) when the caller must be answered 403.
func captureTenant(c *zip.Ctx) (string, bool) {
if org, ok := tenant(c); ok {
return org, true
}
if key := projectKey(c); key != "" {
if org, ok := resolveKeyOrg(c.Context(), key); ok {
return org, true
}
return "", false // presented key that does not resolve → fail CLOSED
}
if !publicCaptureEnabled() {
return "", false
}
@@ -520,36 +574,75 @@ func captureTenant(c *zip.Ctx) (string, bool) {
return "", false
}
// capture ingests one batch into hanzo.events, tenant-scoped. Shared by
// /v1/analytics, /v1/analytics/batch, and /v1/tracker.
func capture(s *cloud.Service[state], c *zip.Ctx) error {
org, ok := captureTenant(c)
if !ok {
return zip.ErrForbidden("valid bearer or a recognized brand host required")
// ── ONE write core ───────────────────────────────────────────────────────────
// event source tags — the ingest adapter each row arrived through. Stamped into
// properties.$source by ingestEvents so the ONE hanzo.events table stays honest
// about origin (canonical vs. deprecated wire) WITHOUT a second table or a schema
// migration: the read lenses are unchanged and $source is queryable in the
// properties JSON, which is exactly the migration signal for the alias sunset.
const (
sourceEvent = "event" // canonical POST /v1/event (native Event wire)
sourcePostHog = "posthog" // POST /v1/insights/e (PostHog wire adapter, deprecated)
sourceCapture = "capture" // POST /v1/analytics{,/batch}, /v1/tracker (Segment/beacon, deprecated)
)
// withSource returns a copy of p carrying $source=source (the ingest adapter), so
// normalizeEvent's scrub+store path records origin as a property. nil-safe; never
// mutates the caller's map (the adapters share their event structs).
func withSource(p map[string]any, source string) map[string]any {
if source == "" {
return p
}
var batch CaptureBatch
if err := c.Bind(&batch); err != nil {
return zip.ErrBadRequest("malformed capture batch")
out := make(map[string]any, len(p)+1)
for k, v := range p {
out[k] = v
}
evs := batch.events()
out["$source"] = source
return out
}
// deprecatedOnce records one deprecation log per alias path per process, so a
// high-volume ingest alias signals its sunset exactly once instead of flooding.
var deprecatedOnce sync.Map
// deprecated logs (once per path) that a superseded ingest alias was hit, pointing
// callers at the canonical front door. It NEVER changes behavior — the alias keeps
// working — it only records the migration signal (also visible as $source in the
// warehouse).
func deprecated(s *cloud.Service[state], c *zip.Ctx, canonical string) {
p := c.Path()
if _, seen := deprecatedOnce.LoadOrStore(p, struct{}{}); seen {
return
}
s.Log.Warn("deprecated analytics ingest endpoint; migrate to the canonical event front door",
"path", p, "canonical", canonical)
}
// ingestEvents is the ONE write core: normalize → scrub → batch INSERT into the
// ONE hanzo.events table. org is the SERVER-resolved tenant (never client input);
// source tags the ingest adapter. Every front door — the canonical /v1/event and
// the deprecated PostHog / Segment / beacon adapters — funnels here, so there is
// exactly one write path. Returns the honest accepted/dropped receipt; the errors
// it returns are already HTTP-shaped (zip) for the handler to pass straight up.
func ingestEvents(ctx context.Context, org, source string, evs []CaptureEvent) (CaptureResult, error) {
if len(evs) == 0 {
return c.JSON(http.StatusOK, CaptureResult{})
return CaptureResult{}, nil
}
if len(evs) > maxBatch {
return zip.ErrBadRequest("batch too large")
return CaptureResult{}, zip.ErrBadRequest("batch too large")
}
if err := requireDatastore(); err != nil {
return err
return CaptureResult{}, err
}
ctx := c.Context()
if err := EnsureEventsTable(ctx); err != nil {
return zip.Errorf(http.StatusServiceUnavailable, "analytics warehouse unavailable: %v", err)
return CaptureResult{}, zip.Errorf(http.StatusServiceUnavailable, "analytics warehouse unavailable: %v", err)
}
now := time.Now().UTC()
rows := make([]eventRow, 0, len(evs))
dropped := 0
for _, e := range evs {
e.Properties = withSource(e.Properties, source)
row, ok := normalizeEvent(org, now, e)
if !ok {
dropped++
@@ -558,12 +651,33 @@ func capture(s *cloud.Service[state], c *zip.Ctx) error {
rows = append(rows, row)
}
if len(rows) == 0 {
return c.JSON(http.StatusOK, CaptureResult{Dropped: dropped})
return CaptureResult{Dropped: dropped}, nil
}
stmt, args := buildEventsInsert(rows)
if err := aiobject.DatastoreExec(ctx, stmt, args...); err != nil {
return warehouseErr("capture", err)
return CaptureResult{}, warehouseErr("capture", err)
}
return c.JSON(http.StatusOK, CaptureResult{Accepted: len(rows), Dropped: dropped})
return CaptureResult{Accepted: len(rows), Dropped: dropped}, nil
}
// capture ingests a Segment/beacon batch into hanzo.events, tenant-scoped. It is
// the DEPRECATED wire adapter behind /v1/analytics, /v1/analytics/batch, and
// /v1/tracker: a thin CaptureBatch decoder over the ONE write core (ingestEvents).
// New callers post the canonical Event to /v1/event; this alias keeps working and
// keeps captureTenant's brand-host path for anonymous marketing traffic.
func capture(s *cloud.Service[state], c *zip.Ctx) error {
deprecated(s, c, "/v1/event")
org, ok := captureTenant(c)
if !ok {
return zip.ErrForbidden("valid bearer or a recognized brand host required")
}
var batch CaptureBatch
if err := c.Bind(&batch); err != nil {
return zip.ErrBadRequest("malformed capture batch")
}
res, err := ingestEvents(c.Context(), org, sourceCapture, batch.events())
if err != nil {
return err
}
return c.JSON(http.StatusOK, res)
}
+120
View File
@@ -0,0 +1,120 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// See the License for the specific language governing permissions and
// limitations under the License.
package analytics
import (
"context"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/zap-proto/zip"
)
// These tests cover the project-API-key → org resolution added to captureTenant so
// keyed, bearer-less SDK traffic (posthog-js / insights-go batch) maps to a tenant.
// They drive the REAL /v1/insights/e handler through the injectable resolveKeyOrg
// seam, so no IAM is needed. The observable proxy for "resolved to a tenant" is
// "passed the tenant gate" — i.e. NOT 403; without a datastore the handler then
// returns 503, so any non-403 status means captureTenant admitted the request.
// stubResolver swaps resolveKeyOrg for the test and records the key it was handed,
// so a test asserts BOTH that projectKey extracted the right key AND that
// captureTenant honored the resolution. Restored via t.Cleanup.
func stubResolver(t *testing.T, fn func(key string) (string, bool)) *string {
t.Helper()
var got string
orig := resolveKeyOrg
resolveKeyOrg = func(_ context.Context, key string) (string, bool) {
got = key
return fn(key)
}
t.Cleanup(func() { resolveKeyOrg = orig })
return &got
}
// postKeyed issues POST path (with optional ?query) to the mounted app, setting an
// optional Host and headers and a raw JSON body — no middleware, mirroring the
// SanitizeIdentity-minted-header harness the other analytics tests use.
func postKeyed(t *testing.T, app *zip.App, path, host, body string, hdr map[string]string) int {
t.Helper()
req := httptest.NewRequest(http.MethodPost, path, strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
if host != "" {
req.Host = host
}
for k, v := range hdr {
req.Header.Set(k, v)
}
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("Test POST %s: %v", path, err)
}
defer func() { _ = resp.Body.Close() }()
_, _ = io.Copy(io.Discard, resp.Body)
return resp.StatusCode
}
// TestCaptureTenant_KeyExtractionReachesResolver: a project key presented in the
// body, the ?api_key= query, or the x-api-key header is extracted and handed to the
// resolver, and a resolved key passes the tenant gate (never 403).
func TestCaptureTenant_KeyExtractionReachesResolver(t *testing.T) {
app := mountApp(t)
cases := []struct {
name, path, body string
hdr map[string]string
wantKey string
}{
{"body", "/v1/insights/e", `{"api_key":"hk-body","event":"e","distinct_id":"d"}`, nil, "hk-body"},
{"query", "/v1/insights/e?api_key=hk-query", `{"event":"e","distinct_id":"d"}`, nil, "hk-query"},
{"x-api-key", "/v1/insights/e", `{"event":"e","distinct_id":"d"}`, map[string]string{"x-api-key": "hk-hdr"}, "hk-hdr"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := stubResolver(t, func(string) (string, bool) { return "acme", true })
code := postKeyed(t, app, tc.path, "", tc.body, tc.hdr)
if *got != tc.wantKey {
t.Fatalf("resolver handed key %q, want %q", *got, tc.wantKey)
}
if code == http.StatusForbidden {
t.Fatalf("a resolved key must pass the tenant gate, got 403")
}
})
}
}
// TestCaptureTenant_UnresolvableKeyFailsClosed: a PRESENTED key that does not
// resolve is refused (403) EVEN on a recognized brand host — it must never fall
// through to the brand-public partition (that would be a cross-tenant write).
func TestCaptureTenant_UnresolvableKeyFailsClosed(t *testing.T) {
app := mountApp(t)
stubResolver(t, func(string) (string, bool) { return "", false }) // nothing resolves
code := postKeyed(t, app, "/v1/insights/e", "hanzo.ai",
`{"api_key":"hk-bad","event":"e","distinct_id":"d"}`, nil)
if code != http.StatusForbidden {
t.Fatalf("presented-but-unresolvable key on a brand host must 403 (fail closed), got %d", code)
}
}
// TestCaptureTenant_AnonBrandHostFallsBack: with NO key presented, anonymous
// traffic on a recognized brand host still resolves to the brand org (not 403), and
// the key resolver is never consulted — the key path only triggers on a real key.
func TestCaptureTenant_AnonBrandHostFallsBack(t *testing.T) {
app := mountApp(t)
stubResolver(t, func(key string) (string, bool) {
t.Fatalf("resolver consulted for a keyless request (key=%q)", key)
return "", false
})
code := postKeyed(t, app, "/v1/insights/e", "hanzo.ai",
`{"event":"e","distinct_id":"d"}`, nil)
if code == http.StatusForbidden {
t.Fatalf("anonymous capture on a recognized brand host must pass the tenant gate, got 403")
}
}
+139
View File
@@ -0,0 +1,139 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// event.go — the ONE canonical event-ingestion front door.
//
// POST /v1/event body: Event | [Event] -> {accepted, dropped}
//
// A JSON object is one event; a JSON array IS the batch (there is deliberately no
// /v1/event/batch). Every other ingest surface (the PostHog wire at
// /v1/insights/e, the Segment/beacon wire at /v1/analytics{,/batch} and
// /v1/tracker) is a thin DEPRECATED adapter that normalizes its own wire shape
// onto CaptureEvent and funnels through the SAME write core (ingestEvents) into
// the SAME hanzo.events table. One write path, many adapters.
//
// AUTH — IAM ONLY, FAIL-CLOSED: the tenant is resolved SERVER-SIDE from a
// validated bearer principal (its owner org) or, for a keyed bearer-less SDK, an
// access key resolved through the ONE IAM key seam (cloud.OrgForKey). There is NO
// brand-host fallback on this endpoint: an unauthenticated or unresolvable caller
// is refused (403), so the canonical door never writes an event into a tenant IAM
// did not vouch for. The org is NEVER read from the body.
package analytics
import (
"encoding/json"
"net/http"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
)
// Event is the canonical analytics event — the entire ingest contract in four
// fields. Only these are first-class; everything else a caller wants to record
// travels in Properties (the scrubber runs over it downstream, same as every
// event). The tenant is NOT a field: it is resolved server-side from IAM, so a
// caller can only ever write into its OWN org's partition.
type Event struct {
Event string `json:"event"` // event name (required; empty ⇒ dropped as unroutable)
DistinctID string `json:"distinctId"` // the person/visitor id the caller owns
Time string `json:"time"` // optional RFC3339; clamped to server-now on skew/absent
Properties map[string]any `json:"properties"` // everything non-core
}
// toCapture adapts the canonical Event onto the internal CaptureEvent the write
// core consumes. Type is left empty (canonicalType ⇒ "event"); no $-property is
// promoted to a column here — /v1/event stays a strict four-field contract, and
// every non-core field the caller sent stays in Properties.
func (e Event) toCapture() CaptureEvent {
return CaptureEvent{
Event: e.Event,
DistinctID: e.DistinctID,
Timestamp: e.Time,
Properties: e.Properties,
}
}
// eventTenant resolves the tenant for POST /v1/event — IAM ONLY, FAIL-CLOSED. A
// validated bearer principal wins (its owner org); otherwise a presented access
// key is resolved to its org through the ONE IAM key seam (resolveKeyOrg →
// cloud.OrgForKey). There is NO brand-host fallback: an unauthenticated or
// unresolvable caller returns ("", false) → 403. (The deprecated aliases keep
// captureTenant's brand-host path for anonymous marketing traffic; the canonical
// endpoint is deliberately stricter — IAM is the only tenant authority here.)
func eventTenant(c *zip.Ctx) (string, bool) {
if org, ok := tenant(c); ok {
return org, true
}
if key := projectKey(c); key != "" {
if org, ok := resolveKeyOrg(c.Context(), key); ok {
return org, true
}
}
return "", false
}
// decodeEvents decodes a request body as Event | []Event. The first non-space
// byte decides: '[' ⇒ the array batch, anything else ⇒ a single Event. An empty
// body yields no events (an honest empty receipt, not an error). Pure over the
// raw bytes (the handler passes c.Body() — fasthttp-buffered, the same bytes
// projectKey peeked) so the decode is driven directly by tests.
func decodeEvents(body []byte) ([]Event, error) {
i := 0
for i < len(body) {
if b := body[i]; b == ' ' || b == '\t' || b == '\r' || b == '\n' {
i++
continue
}
break
}
if i >= len(body) {
return nil, nil
}
if body[i] == '[' {
var evs []Event
if err := json.Unmarshal(body, &evs); err != nil {
return nil, err
}
return evs, nil
}
var e Event
if err := json.Unmarshal(body, &e); err != nil {
return nil, err
}
return []Event{e}, nil
}
// eventIngest answers POST /v1/event — the ONE canonical ingestion front door.
// Org is IAM-derived and fail-closed (eventTenant); the body is Event | [Event];
// every event flows through the ONE write core (ingestEvents) into the ONE
// hanzo.events table, tagged source=event.
func eventIngest(s *cloud.Service[state], c *zip.Ctx) error {
org, ok := eventTenant(c)
if !ok {
return zip.ErrForbidden("valid bearer or a resolvable access key required")
}
evs, err := decodeEvents(c.Body())
if err != nil {
return zip.ErrBadRequest("malformed event payload")
}
caps := make([]CaptureEvent, len(evs))
for i, e := range evs {
caps[i] = e.toCapture()
}
res, err := ingestEvents(c.Context(), org, sourceEvent, caps)
if err != nil {
return err
}
return c.JSON(http.StatusOK, res)
}
+240
View File
@@ -0,0 +1,240 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// See the License for the specific language governing permissions and
// limitations under the License.
package analytics
import (
"net/http"
"testing"
"time"
)
// ── decodeEvents: Event | []Event (single + batch) ──────────────────────────
func TestDecodeEvents_Single(t *testing.T) {
evs, err := decodeEvents([]byte(`{"event":"signup","distinctId":"u1","properties":{"plan":"pro"}}`))
if err != nil {
t.Fatalf("decode single: %v", err)
}
if len(evs) != 1 {
t.Fatalf("single want 1 event, got %d", len(evs))
}
if evs[0].Event != "signup" || evs[0].DistinctID != "u1" {
t.Fatalf("single decoded = %+v", evs[0])
}
if evs[0].Properties["plan"] != "pro" {
t.Fatalf("single properties = %v", evs[0].Properties)
}
}
func TestDecodeEvents_Batch(t *testing.T) {
evs, err := decodeEvents([]byte(`[{"event":"a","distinctId":"d"},{"event":"b","distinctId":"d"}]`))
if err != nil {
t.Fatalf("decode batch: %v", err)
}
if len(evs) != 2 || evs[0].Event != "a" || evs[1].Event != "b" {
t.Fatalf("batch decoded = %+v", evs)
}
}
func TestDecodeEvents_BatchLeadingWhitespace(t *testing.T) {
// The array is detected past leading whitespace, not only at byte 0.
evs, err := decodeEvents([]byte(" \n\t [{\"event\":\"a\"}]"))
if err != nil {
t.Fatalf("decode ws-batch: %v", err)
}
if len(evs) != 1 || evs[0].Event != "a" {
t.Fatalf("ws-batch decoded = %+v", evs)
}
}
func TestDecodeEvents_EmptyIsNoEvents(t *testing.T) {
// Empty / whitespace-only body ⇒ zero events, NOT an error (honest empty receipt).
for _, b := range []string{"", " ", "\n\t"} {
evs, err := decodeEvents([]byte(b))
if err != nil || len(evs) != 0 {
t.Fatalf("empty %q ⇒ evs=%v err=%v", b, evs, err)
}
}
}
func TestDecodeEvents_Malformed(t *testing.T) {
for _, b := range []string{`{"event":`, `[{"event":"a"},`, `not json`} {
if _, err := decodeEvents([]byte(b)); err == nil {
t.Fatalf("malformed %q want error, got nil", b)
}
}
}
// ── adapter → Event/CaptureEvent normalization ──────────────────────────────
// TestEventToCapture: the canonical Event maps onto CaptureEvent with ONLY the
// four core fields promoted; Type is left empty (⇒ "event") and everything else
// stays in Properties (nothing is lifted to a column).
func TestEventToCapture(t *testing.T) {
e := Event{
Event: "purchase",
DistinctID: "u9",
Time: "2026-07-18T00:00:00Z",
Properties: map[string]any{"amount": 42, "$current_url": "https://x/y"},
}
ce := e.toCapture()
if ce.Event != "purchase" || ce.DistinctID != "u9" || ce.Timestamp != "2026-07-18T00:00:00Z" {
t.Fatalf("core fields = %+v", ce)
}
if ce.Type != "" {
t.Fatalf("Type must be empty (⇒ canonicalType event), got %q", ce.Type)
}
if ce.URL != "" {
t.Fatalf("no $-property is promoted to a column on the canonical wire; URL=%q", ce.URL)
}
// non-core stays in properties
if ce.Properties["amount"] != 42 || ce.Properties["$current_url"] != "https://x/y" {
t.Fatalf("properties passthrough = %v", ce.Properties)
}
}
// TestEventNormalizeThroughCore: a canonical Event, adapted and normalized, yields
// a row stamped with the SERVER org and the resolved event name.
func TestEventNormalizeThroughCore(t *testing.T) {
row, ok := normalizeEvent("acme", time.Now(), Event{Event: "signup", DistinctID: "u1"}.toCapture())
if !ok {
t.Fatal("want routable")
}
if row.tenant != "acme" || row.event != "signup" || row.eventType != "event" {
t.Fatalf("row = tenant %q event %q type %q", row.tenant, row.event, row.eventType)
}
}
// TestInsightsAdapterNormalization: the PostHog wire adapter lifts well-known
// $-properties to columns (that is its job) — the OTHER adapter feeding the ONE
// core, distinct from the canonical Event wire.
func TestInsightsAdapterNormalization(t *testing.T) {
ce := insightsEvent{
Event: "$pageview",
DistinctID: "d",
Properties: map[string]any{"$current_url": "https://x/y", "$session_id": "s1"},
}.toCapture()
if ce.Type != "pageview" {
t.Fatalf("posthog $pageview ⇒ type pageview, got %q", ce.Type)
}
if ce.URL != "https://x/y" || ce.SessionID != "s1" {
t.Fatalf("posthog $-props ⇒ columns: url=%q session=%q", ce.URL, ce.SessionID)
}
}
// TestCaptureBatchAdapter: the Segment/beacon adapter prefers `batch`, falling
// back to `events`.
func TestCaptureBatchAdapter(t *testing.T) {
if got := (CaptureBatch{Batch: []CaptureEvent{{Event: "a"}}, Events: []CaptureEvent{{Event: "b"}}}).events(); len(got) != 1 || got[0].Event != "a" {
t.Fatalf("batch preferred over events, got %+v", got)
}
if got := (CaptureBatch{Events: []CaptureEvent{{Event: "b"}}}).events(); len(got) != 1 || got[0].Event != "b" {
t.Fatalf("events fallback, got %+v", got)
}
}
// ── source tagging (the $source property, one-table origin discriminator) ────
func TestWithSource(t *testing.T) {
// stamps $source
got := withSource(nil, sourceEvent)
if got["$source"] != "event" {
t.Fatalf("withSource(nil,event) = %v", got)
}
// does not mutate the caller's map, and preserves existing keys
orig := map[string]any{"a": 1}
out := withSource(orig, sourcePostHog)
if out["a"] != 1 || out["$source"] != "posthog" {
t.Fatalf("withSource copy = %v", out)
}
if _, leaked := orig["$source"]; leaked {
t.Fatalf("withSource mutated the caller's map: %v", orig)
}
// empty source is a no-op passthrough (same map)
if got := withSource(orig, ""); got["$source"] != nil {
t.Fatalf("empty source must not stamp, got %v", got)
}
}
// TestSourceStampedIntoProperties: source flows through withSource → normalizeEvent
// → the stored properties JSON, so the ONE hanzo.events table carries origin
// WITHOUT a schema column.
func TestSourceStampedIntoProperties(t *testing.T) {
e := CaptureEvent{Event: "x", Properties: withSource(nil, sourceEvent)}
row, ok := normalizeEvent("acme", time.Now(), e)
if !ok {
t.Fatal("want routable")
}
props := decodeProps(t, row.properties)
if props["$source"] != "event" {
t.Fatalf("row.properties $source = %v (props=%v)", props["$source"], props)
}
}
// ── POST /v1/event: IAM-only, fail-closed auth ──────────────────────────────
//
// Observable proxy (mirrors capture_keyorg_test): a REFUSED request is 403; an
// ADMITTED one reaches requireDatastore and returns 503 (no datastore in tests).
// So "not 403" ⇒ the tenant gate admitted the request.
func TestEvent_NoPrincipalNoKeyForbidden(t *testing.T) {
app := mountApp(t)
if code, _ := doBody(t, app, http.MethodPost, "/v1/event", "", "", `{"event":"e","distinctId":"d"}`); code != http.StatusForbidden {
t.Fatalf("no-principal no-key /v1/event want 403, got %d", code)
}
}
func TestEvent_BearerPrincipalAdmitted(t *testing.T) {
app := mountApp(t)
// A validated principal (X-User/X-Org) is admitted → 503 (datastore down), not 403.
if code, _ := doBody(t, app, http.MethodPost, "/v1/event", "user-dave", "acme", `{"event":"signup","distinctId":"u1"}`); code != http.StatusServiceUnavailable {
t.Fatalf("bearer /v1/event want 503 (admitted, datastore down), got %d", code)
}
// A batch body is admitted the same way.
if code, _ := doBody(t, app, http.MethodPost, "/v1/event", "user-dave", "acme", `[{"event":"a","distinctId":"d"},{"event":"b","distinctId":"d"}]`); code != http.StatusServiceUnavailable {
t.Fatalf("bearer /v1/event batch want 503, got %d", code)
}
}
func TestEvent_ResolvedKeyAdmitted(t *testing.T) {
app := mountApp(t)
got := stubResolver(t, func(string) (string, bool) { return "acme", true })
code := postKeyed(t, app, "/v1/event", "", `{"api_key":"hk-k","event":"e","distinctId":"d"}`, nil)
if *got != "hk-k" {
t.Fatalf("resolver handed key %q, want hk-k", *got)
}
if code == http.StatusForbidden {
t.Fatalf("a resolved access key must pass the /v1/event gate, got 403")
}
}
func TestEvent_UnresolvableKeyFailsClosedEvenOnBrandHost(t *testing.T) {
app := mountApp(t)
stubResolver(t, func(string) (string, bool) { return "", false })
code := postKeyed(t, app, "/v1/event", "hanzo.ai", `{"api_key":"hk-bad","event":"e","distinctId":"d"}`, nil)
if code != http.StatusForbidden {
t.Fatalf("presented-but-unresolvable key on /v1/event must 403 (fail closed), got %d", code)
}
}
// TestEvent_NoBrandHostFallback is THE distinguishing invariant: anonymous traffic
// on a recognized brand host is ADMITTED by the deprecated /v1/analytics alias
// (brand-public partition) but REFUSED by the canonical /v1/event — IAM is the
// only tenant authority on the canonical door.
func TestEvent_NoBrandHostFallback(t *testing.T) {
app := mountApp(t)
body := `{"event":"e","distinctId":"d"}`
if code, _ := doHost(t, app, "/v1/event", "", "", "hanzo.ai", body); code != http.StatusForbidden {
t.Fatalf("anonymous brand-host /v1/event must 403 (no brand fallback), got %d", code)
}
// Contrast: the deprecated alias still admits the same anonymous brand-host
// traffic (503 = admitted, datastore down), proving the difference is by design.
if code, _ := doHost(t, app, "/v1/analytics", "", "", "hanzo.ai", `{"batch":[{"type":"pageview"}]}`); code != http.StatusServiceUnavailable {
t.Fatalf("deprecated alias still brand-admits (want 503), got %d", code)
}
}
+184
View File
@@ -0,0 +1,184 @@
package analytics
// /v1/insights — the UNIFIED native insights surface on the SAME engine.
//
// This file is a WIRE ADAPTER, not a second pipeline: PostHog-shaped payloads
// (what @hanzo/insights and every PostHog-compatible SDK emit) are mapped onto
// the native CaptureEvent and flow through the ONE capture path (normalize →
// scrub → hanzo.events), and the console reads recent events back through the
// ONE datastore client. Flags stay at /v1/flags (the native flags engine) —
// this namespace deliberately does not duplicate them.
//
// Routes (org resolved SERVER-SIDE — same tenant gates as the rest):
//
// POST /v1/insights/e PostHog-compatible ingest: one event or {batch:[...]}
// GET /v1/insights/events recent events for the org (console read; limit<=200)
// GET /v1/insights/health liveness of the unified surface
//
// SCALE PATH: accept is stateless (any replica) and the sink is the pooled
// batch INSERT into Datastore. When ingest volume outgrows direct sink, the
// seam is buildEventsInsert — swap the exec for a queue producer (mq/pubsub)
// with a Datastore consumer, no handler changes.
import (
"encoding/json"
"net/http"
"strconv"
"strings"
"time"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
)
// insightsEvent is the PostHog wire shape (subset that matters for ingest).
type insightsEvent struct {
Event string `json:"event"`
DistinctID string `json:"distinct_id"`
Timestamp string `json:"timestamp"`
Properties map[string]any `json:"properties"`
}
// insightsBody accepts both the single-event and batch PostHog shapes.
type insightsBody struct {
insightsEvent
Batch []insightsEvent `json:"batch"`
}
// toCapture maps one PostHog event onto the native CaptureEvent. Well-known
// $-properties become first-class columns; the rest stay in properties (the
// scrubber runs downstream in normalizeEvent, same as every native event).
func (e insightsEvent) toCapture() CaptureEvent {
props := e.Properties
str := func(key string) string {
if props == nil {
return ""
}
if v, ok := props[key].(string); ok {
return v
}
return ""
}
typ := "event"
if e.Event == "$pageview" {
typ = "pageview"
}
return CaptureEvent{
Type: typ,
Event: e.Event,
Timestamp: e.Timestamp,
DistinctID: e.DistinctID,
SessionID: str("$session_id"),
URL: str("$current_url"),
Path: str("$pathname"),
Referrer: str("$referrer"),
Product: str("product"),
Library: str("$lib"),
LibraryVer: str("$lib_version"),
Properties: props,
}
}
// insightsIngest answers POST /v1/insights/e — the DEPRECATED PostHog-wire
// adapter. It normalizes the PostHog single/batch shape onto CaptureEvent and
// funnels through the ONE write core (ingestEvents, source=posthog); it keeps
// captureTenant's brand-host path so anonymous PostHog-wire traffic is unbroken.
// New callers post the canonical Event to /v1/event.
func insightsIngest(s *cloud.Service[state], c *zip.Ctx) error {
deprecated(s, c, "/v1/event")
org, ok := captureTenant(c)
if !ok {
return zip.ErrForbidden("valid bearer or a recognized brand host required")
}
var body insightsBody
if err := c.Bind(&body); err != nil {
return zip.ErrBadRequest("malformed insights payload")
}
events := body.Batch
if len(events) == 0 && body.Event != "" {
events = []insightsEvent{body.insightsEvent}
}
caps := make([]CaptureEvent, len(events))
for i, e := range events {
caps[i] = e.toCapture()
}
res, err := ingestEvents(c.Context(), org, sourcePostHog, caps)
if err != nil {
return err
}
return c.JSON(http.StatusOK, res)
}
// insightsEvents answers GET /v1/insights/events — the console's recent-events
// read (newest first). Tenant-scoped server-side; limit defaults 50, caps 200.
func insightsEvents(s *cloud.Service[state], c *zip.Ctx) error {
org, ok := tenant(c)
if !ok {
return zip.ErrForbidden("valid bearer required")
}
limit, _ := strconv.Atoi(strings.TrimSpace(c.Query("limit")))
if limit <= 0 {
limit = 50
}
if limit > 200 {
limit = 200
}
rows, err := aiobject.DatastoreQuery(c.Context(), `
SELECT id, timestamp, event, event_type, distinct_id, session_id,
product, url, path, properties
FROM hanzo.events
WHERE tenant_id = ?
ORDER BY timestamp DESC
LIMIT ?`, org, limit)
if err != nil {
return zip.Errorf(http.StatusServiceUnavailable, "analytics warehouse unavailable: %v", err)
}
type ev struct {
ID string `json:"id"`
Timestamp string `json:"timestamp"`
Event string `json:"event"`
Type string `json:"type"`
DistinctID string `json:"distinctId"`
SessionID string `json:"sessionId,omitempty"`
Product string `json:"product,omitempty"`
URL string `json:"url,omitempty"`
Path string `json:"path,omitempty"`
Properties json.RawMessage `json:"properties,omitempty"`
}
out := make([]ev, 0, len(rows))
for _, r := range rows {
e := ev{
ID: asStr(r["id"]), Timestamp: asStr(r["timestamp"]), Event: asStr(r["event"]),
Type: asStr(r["event_type"]), DistinctID: asStr(r["distinct_id"]),
SessionID: asStr(r["session_id"]), Product: asStr(r["product"]),
URL: asStr(r["url"]), Path: asStr(r["path"]),
}
if p := asStr(r["properties"]); p != "" && json.Valid([]byte(p)) {
e.Properties = json.RawMessage(p)
}
out = append(out, e)
}
return c.JSON(http.StatusOK, map[string]any{"data": out})
}
func insightsHealth(s *cloud.Service[state], c *zip.Ctx) error {
return c.JSON(http.StatusOK, map[string]any{"ok": true, "engine": "hanzo-analytics", "surface": "/v1/insights"})
}
func asStr(v any) string {
switch t := v.(type) {
case string:
return t
case time.Time:
return t.UTC().Format(time.RFC3339)
case nil:
return ""
default:
b, err := json.Marshal(t)
if err != nil {
return ""
}
return strings.Trim(string(b), `"`)
}
}
+26 -134
View File
@@ -1,152 +1,44 @@
package authors
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
"github.com/hanzoai/cloud/clients/commerceinproc"
"github.com/hanzoai/cloud/clients/payout"
)
// commerce is the narrow money seam the author royalty loop needs: read a deploying
// org's metered spend (the accrual base) and grant a promo credit to a wallet (a
// payout made in credits). It is an INTERFACE so the store/handler logic is testable
// with a fake ledger the HTTP impl below is the ONE production binding.
// org's metered spend (the royalty accrual base) and grant a promo credit to a wallet
// (a payout made in credits, ledger tag grant:author). It is an INTERFACE so the
// store/handler logic is testable with a fake ledger; the production binding is
// clients/payout, reached through the thin adapter below.
//
// This mirrors clients/affiliates/commerce.go EXACTLY: the same
// COMMERCE_SERVICE_TOKEN S2S path, the same X-Org-Id=<org> namespace + bare org
// `user` subject — so an author payout-in-credits lands in precisely the wallet the
// balance panel reads, indistinguishable from an admin/affiliate grant except by its
// ledger tag (grant:author, → the commerce Credit/trial bucket per DepositKind's
// grant:* rule).
// The S2S impl (COMMERCE_SERVICE_TOKEN path, X-Org-Id=<org> namespace, bare-org
// `user` subject) was three byte-identical commerce.go copies; it now lives ONCE in
// clients/payout. An author payout-in-credits still lands in precisely the wallet the
// balance panel reads, indistinguishable from an admin grant except by its
// grant:author tag.
type commerce interface {
configured() bool
// deposit grants amountCents to org's wallet (Credit/trial bucket via the
// grant:author tag) and returns the ledger transaction id.
deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (txnID string, err error)
// spendCents is a deploying org's month-to-date metered consumption — the royalty
// accrual base (spend × the author's share).
spendCents(ctx context.Context, org, user string) (int64, error)
}
// errUnconfigured is returned by a deposit against an unwired commerce so the caller
// records an honest failure rather than reporting a phantom payout.
var errUnconfigured = errors.New("authors: commerce endpoint not configured")
// errUnconfigured is the shared sentinel a deposit against an unwired commerce
// returns, so the caller records an honest failure rather than a phantom payout.
var errUnconfigured = payout.ErrUnconfigured
// httpCommerce is the production commerce binding (COMMERCE_SERVICE_TOKEN S2S).
type httpCommerce struct {
base string
token string
http *http.Client
// commerceSeam adapts the shared payout.Client onto this program's lowercase seam
// (Go package-scoped interface methods cannot cross packages). Zero logic — pure
// delegation; the money path lives in clients/payout.
type commerceSeam struct{ c *payout.Client }
func (s commerceSeam) configured() bool { return s.c.Configured() }
func (s commerceSeam) deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (string, error) {
return s.c.Deposit(ctx, org, user, amountCents, currency, notes, tags)
}
func (s commerceSeam) spendCents(ctx context.Context, org, user string) (int64, error) {
return s.c.SpendCents(ctx, org, user)
}
func newCommerceClient(base, token string) *httpCommerce {
return &httpCommerce{
base: strings.TrimRight(strings.TrimSpace(base), "/"),
token: strings.TrimSpace(token),
http: commerceinproc.Client(15 * time.Second),
}
}
func (c *httpCommerce) configured() bool { return c != nil && c.base != "" && c.token != "" }
// deposit posts POST /v1/billing/deposit — the ONE money-in primitive (identical to
// affiliates.httpCommerce.deposit). Commerce's EdgeAuth pins the body `user` to the
// X-Org-Id subject, so a payout can never be mis-targeted to another wallet.
func (c *httpCommerce) deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (string, error) {
if !c.configured() {
return "", errUnconfigured
}
if currency == "" {
currency = "usd"
}
body, err := json.Marshal(map[string]any{
"user": user,
"currency": currency,
"amount": amountCents,
"notes": notes,
"tags": tags,
})
if err != nil {
return "", err
}
raw, err := c.do(ctx, http.MethodPost, "/v1/billing/deposit", nil, org, body)
if err != nil {
return "", err
}
var out struct {
TransactionID string `json:"transactionId"`
}
if err := json.Unmarshal(raw, &out); err != nil {
return "", fmt.Errorf("commerce deposit decode: %w", err)
}
return out.TransactionID, nil
}
// spendCents reads GET /v1/billing/usage-rollup and returns consumedCents. Zero (not
// an error) when commerce is unconfigured so a partial deploy degrades to "no spend
// to accrue yet" rather than a 5xx.
func (c *httpCommerce) spendCents(ctx context.Context, org, user string) (int64, error) {
if !c.configured() {
return 0, nil
}
q := url.Values{"user": {user}}
raw, err := c.do(ctx, http.MethodGet, "/v1/billing/usage-rollup", q, org, nil)
if err != nil {
return 0, err
}
var out struct {
ConsumedCents int64 `json:"consumedCents"`
}
if err := json.Unmarshal(raw, &out); err != nil {
return 0, fmt.Errorf("commerce rollup decode: %w", err)
}
return out.ConsumedCents, nil
}
// do performs one admin-S2S commerce request. X-Org-Id=<org> is the per-org
// namespace selector commerce's EdgeAuth trusts only behind the service token.
func (c *httpCommerce) do(ctx context.Context, method, path string, q url.Values, org string, body []byte) ([]byte, error) {
u := c.base + path
if enc := q.Encode(); enc != "" {
u += "?" + enc
}
var r io.Reader
if body != nil {
r = bytes.NewReader(body)
}
req, err := http.NewRequestWithContext(ctx, method, u, r)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json")
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
if c.token != "" {
req.Header.Set("Authorization", "Bearer "+c.token)
}
if org != "" {
req.Header.Set("X-Org-Id", org)
}
resp, err := c.http.Do(req)
if err != nil {
return nil, fmt.Errorf("commerce unreachable: %w", err)
}
defer func() { _ = resp.Body.Close() }()
out, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return nil, err
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("commerce status %d", resp.StatusCode)
}
return out, nil
}
// newCommerceClient builds the production binding, delegating to clients/payout.
func newCommerceClient(base, token string) commerce { return commerceSeam{payout.NewClient(base, token)} }
+11 -1
View File
@@ -43,6 +43,7 @@ import (
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/audit"
"github.com/hanzoai/cloud/clients/connectorruntime"
"github.com/hanzoai/cloud/clients/principal"
"github.com/hanzoai/cloud/clients/tools"
"github.com/zap-proto/zip"
@@ -138,6 +139,15 @@ func Mount(app *zip.App, deps cloud.Deps) error {
tools.Register(connectorToolProvider{})
b.Log.Info("automations mounted", "connectors", catalog.ConnectorCount, "runtime", len(registry), "brand", deps.Brand)
// Native single-connector execution (HIP-0126): POST /v1/automations/connectors/:id/run,
// the in-process goja runner paired with the connector catalogue above. It mounts one
// route DISTINCT from every automations route (no /v1/automations/* wildcard here, so no
// shadow), and was a separate Wire entry purely for that one route — fold it in as a
// terminal sub-mount so connector catalogue + execution are ONE automations subsystem.
if err := connectorruntime.Mount(app, deps); err != nil {
return err
}
return nil
}
@@ -676,7 +686,7 @@ func recordRunEnd(s *cloud.Service[state], ctx context.Context, in RunEndInput)
// meterUnit records one metered unit for an HTTP caller's org. Nil/disabled meter → no-op.
func meterUnit(s *cloud.Service[state], org string, c *zip.Ctx) {
s.Bill.Meter(principal.Payer(c), principal.Project(c), meterKind, cloud.ResourceFeeCents(feeEnvPrefix, meterKind), c.RequestID(), cloud.ClientIP(c))
s.Bill.Meter(principal.HomeOrg(c), principal.Project(c), meterKind, cloud.ResourceFeeCents(feeEnvPrefix, meterKind), c.RequestID(), cloud.ClientIP(c))
}
// meterRun records one metered unit for a flow run from the durable path (no HTTP
+1 -1
View File
@@ -15,7 +15,7 @@
// LANE 2 — managed Base hosting (what superbase/PocketHost provided). ONE Base
// app PER ORG, opened lazily and pooled, each on its OWN SQLite under
// {DataDir}/base/{orgSegment}/ — the same "prod = SQLite per tenant" model
// (HIP-0302) the gojabase leaves (captable/sign/dataroom) use, so an org's
// (HIP-0302) the NewBase leaves (captable/sign/dataroom) use, so an org's
// collections/records are PHYSICALLY isolated. Served AUTHENTICATED under
// /v1/base/*, the org resolved from the VALIDATED cloud principal (never a
// client header). This is the console Bases manager's backend.
+3 -3
View File
@@ -18,7 +18,7 @@ import (
baseapp "github.com/hanzoai/base"
"github.com/hanzoai/base/core"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/gojabase"
"github.com/hanzoai/cloud/clients/goja"
luxlog "github.com/luxfi/log"
fiber "github.com/zap-proto/fiber/v3"
"github.com/zap-proto/zip"
@@ -170,8 +170,8 @@ func TestPerOrgIsolatedCRUD(t *testing.T) {
}
// The isolation is physical: distinct on-disk data dirs per org segment.
acmeDir := filepath.Join(dataDir, "base", gojabase.TenantSegment("acme"))
globexDir := filepath.Join(dataDir, "base", gojabase.TenantSegment("globex"))
acmeDir := filepath.Join(dataDir, "base", goja.TenantSegment("acme"))
globexDir := filepath.Join(dataDir, "base", goja.TenantSegment("globex"))
if acmeDir == globexDir {
t.Fatalf("orgs share a data dir: %s", acmeDir)
}
+4 -4
View File
@@ -17,7 +17,7 @@ import (
baseapp "github.com/hanzoai/base"
"github.com/hanzoai/base/apis"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/gojabase"
"github.com/hanzoai/cloud/clients/goja"
)
// Pool sizing (env-overridable). A full Base app is heavier than a bare *sql.DB
@@ -35,7 +35,7 @@ const (
// ({DataDir}/base/{TenantSegment}/), the "prod = SQLite per tenant" rule
// (HIP-0302). Apps open lazily on first request, migrate once, and are pooled
// (LRU-capped, idle-evicted). Concurrent opens of the same org are
// single-flighted under mu. The org→segment encoding is gojabase.TenantSegment —
// single-flighted under mu. The org→segment encoding is goja.TenantSegment —
// the ONE injective, traversal-safe tenant→path encoder, shared so an org maps
// to exactly one physical identity everywhere in the binary.
type pool struct {
@@ -85,7 +85,7 @@ func (p *pool) acquire(org string) (http.Handler, func(), error) {
if strings.TrimSpace(org) == "" {
return nil, nil, fmt.Errorf("base: empty org")
}
seg := gojabase.TenantSegment(org)
seg := goja.TenantSegment(org)
p.mu.Lock()
defer p.mu.Unlock()
@@ -111,7 +111,7 @@ func (p *pool) acquire(org string) (http.Handler, func(), error) {
// for in-process callers that drive the engine's Go API directly (collection
// provisioning, seeding) rather than the HTTP path.
func (p *pool) appFor(org string) (*baseapp.Base, error) {
seg := gojabase.TenantSegment(org)
seg := goja.TenantSegment(org)
p.mu.Lock()
defer p.mu.Unlock()
if e, ok := p.m[seg]; ok {
+88
View File
@@ -0,0 +1,88 @@
package billing
import (
"context"
"strings"
"github.com/hanzoai/account"
"github.com/hanzoai/cloud/clients/finance"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
)
// The ONE prepaid-balance read for the customer surface (/v1/billing/balance and the
// /v1/finance/balance projection).
//
// WHY IT IS NOT A COMMERCE PROXY. Co-resident, commerce registers its routes on the
// HOST's zip app (apps/commerce.go mountCommerce → commerce.Embed with EmbedConfig.App),
// and commerceinproc publishes that SAME shared app as the S2S "commerce" transport,
// which re-dispatches BY PATH. commerce's own billing routes are NOT registered in this
// binary — api.Route(), which registers GET /v1/billing/balance, is called only from
// commerce's mount.go, which is behind `//go:build cloud` and never compiled (cloud ships
// -tags "libsqlite3 sqlite_fts5"). So a GET of "/v1/billing/balance" through the S2S seam
// matches the ONLY registration of that path — the handler below — and re-enters it with
// no principal, which answers "sign in to view billing". The proxy was calling itself.
//
// WHERE THE MONEY IS. Co-resident, the prepaid wallet lives in cloud's OWN finance ledger
// (clients/finance, per-org double-entry SQLite): wireFinance (build.go) points the ai
// prepaid gate's balance read at it, the edge meter debits it, and an admin grant credits
// it (clients/admin/core.grantDeposit prefers finance.Current() for exactly this reason).
// So the customer's balance is read from that ledger DIRECTLY — no HTTP hop, nothing to
// self-dispatch, and the number shown is the number that admits or refuses a request.
// The commerce S2S read stays as the split-deploy fallback, unchanged.
// subjectFor resolves the billing subject for the caller by CALLING the ONE rule,
// ai/object.Payer — the same function the ai prepaid gate (routers/filter_balance.go
// resolveBillingKey) and the usage debit resolve. It is the SHARED resolver for every
// commerce-projected read in this package (the balance read AND the finance reads), so
// there is exactly one copy of the rule. cloud and ai each keeping their own copy is
// what let them drift apart (cloud's console view scoping to the org while the gate
// scoped to "org/user"), so the view showed a funded org while the gate refused the
// member. One function, one rule, one wallet.
//
// org is the VALIDATED principal org. The name half uses the SAME precedence as
// clients/account.resolveCaller: X-User-Name (the IAM username the identity boundary mints
// from the validated `name` claim — expressly "the `name` half of <owner>/<name>"), then
// X-User-Id. Both are authorityHeaders — stripped on ingress and re-injected only from
// verified claims — so neither is a client value.
//
// The X-User-Id fallback goes through the shared account.PayerOf because that header's shape is
// path-dependent: the gateway historically minted X-User-Id == the username, while the
// in-binary direct-Bearer path mints the UUID subject, and callers hold it as an
// "<owner>/<name>" key. PayerOf is the parse ai already uses to fold that key form back
// to the payer, so the two agree by construction instead of by a re-implemented split.
//
// KNOWN RESIDUAL: a validated principal carrying NEITHER X-User-Name NOR an "<owner>/<name>"
// X-User-Id (i.e. a bare username id) folds to the org pool. That requires a JWT with no
// `name` and no `preferred_username`, since the boundary mints X-User-Name from either;
// production tokens carry one. Called out for review rather than papered over.
func subjectFor(c *zip.Ctx, org string) string {
if name := strings.TrimSpace(c.Header("X-User-Name")); name != "" {
// Hand Payer the account the credential NAMES (the validated `billing_account`
// claim, minted into X-Billing-Account-Id). The ai gate reads the same claim,
// so this view and that gate resolve one wallet. Reading it here is what keeps
// them from drifting the way the org-vs-"org/user" split once did — except
// that split was two rules, and this would be one rule fed two different
// credentials, which reads the same to a user: a funded balance the gate
// refuses. Absent ⟹ Payer's legacy rule, exactly today's answer.
return account.Payer(account.Credential{Owner: org, Name: name, Account: principal.BillingAccount(c)}).Subject()
}
return account.PayerOf(org, strings.TrimSpace(c.User())).Subject()
}
// availableCents returns the caller's spendable prepaid balance from the co-resident
// finance ledger. ok is false when no finance ledger is published (split deploy) and the
// caller must fall back to the commerce S2S read; a non-nil err is a REAL read failure and
// must be surfaced, never rendered as a zero balance — a balance that cannot be read is
// unknown, and unknown is not "broke".
func availableCents(ctx context.Context, org, subject string) (cents int64, ok bool, err error) {
fin := finance.Current()
if fin == nil {
return 0, false, nil
}
bal, err := fin.Balance(ctx, org, subject, "usd", false)
if err != nil {
return 0, true, err
}
return bal.Cents(), true, nil
}
+219
View File
@@ -0,0 +1,219 @@
package billing
import (
"context"
"encoding/json"
"fmt"
"github.com/hanzoai/account"
"net/http"
"net/http/httptest"
"testing"
"github.com/hanzoai/cloud/clients/finance"
"github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/cloud/types"
)
// fakeFinance is the co-resident finance ledger seam (types.FinanceClient). It records
// the exact (org, subject) the handler read so a test can prove WHICH wallet the console
// shows, and can be made to fail so a test can prove an unreadable balance is never
// rendered as zero.
type fakeFinance struct {
wallets map[string]int64 // "org|subject" -> cents
usageRows []finance.UsageRow
err error
gotOrg string
gotSubj string
calls int
}
// ListUsage satisfies the optional co-resident usage-read capability coResidentUsage
// resolves; returns the seeded rows so a test can prove the usage view answers from the
// ledger instead of the self-dispatching commerce hop.
func (f *fakeFinance) ListUsage(context.Context, string, int) ([]finance.UsageRow, error) {
return f.usageRows, f.err
}
func (f *fakeFinance) Balance(_ context.Context, org, subject, _ string, _ bool) (money.Amount, error) {
f.calls++
f.gotOrg, f.gotSubj = org, subject
if f.err != nil {
return money.Zero(), f.err
}
return money.FromCents(f.wallets[org+"|"+subject]), nil
}
func (f *fakeFinance) Deposit(context.Context, types.DepositInput) (string, error) { return "", nil }
func (f *fakeFinance) RecordUsage(context.Context, types.UsageInput) error { return nil }
func publishFinance(t *testing.T, f *fakeFinance) {
t.Helper()
finance.Publish(f)
t.Cleanup(func() { finance.Publish(nil) })
}
// TestBalance_ReadsFinanceLedgerNotCommerce is the regression for the live incident:
// GET /v1/billing/balance proxied to commerce at the SAME path, which re-entered THIS
// handler (commerceinproc dispatches the shared app by path; commerce's own billing
// routes are never registered in the co-resident binary), answered "sign in to view
// billing", and surfaced as "billing upstream status 500".
//
// Co-resident the balance now comes from the finance ledger — the wallet the ai gate
// actually reads — and commerce is NOT called at all, so there is nothing to self-dispatch.
func TestBalance_ReadsFinanceLedgerNotCommerce(t *testing.T) {
fin := &fakeFinance{wallets: map[string]int64{"maxpower|maxpower": 6875}}
publishFinance(t, fin)
f := &fakeCommerce{status: 200, body: `{"balance":1,"holds":0,"available":1}`}
app := mountApp(t, f.server(t).URL, "svc-token")
code, body := call(t, app, http.MethodGet, "/v1/billing/balance", "maxpower/dave", "maxpower")
if code != 200 {
t.Fatalf("balance: want 200, got %d (%s)", code, body)
}
var got commerceBalance
if err := json.Unmarshal(body, &got); err != nil {
t.Fatalf("decode: %v (%s)", err, body)
}
if got.Available != 6875 || got.Balance != 6875 {
t.Fatalf("balance not read from the finance ledger: %s", body)
}
// The proof there is no self-dispatch: the commerce hop never happened.
if f.gotPath != "" {
t.Fatalf("commerce was called at %q — the self-dispatching proxy is back", f.gotPath)
}
if fin.gotOrg != "maxpower" {
t.Fatalf("read org %q, want the validated principal's org", fin.gotOrg)
}
}
// TestBalance_SubjectIsTheGateSubject pins the invariant this incident broke: the wallet
// the console SHOWS must be the wallet the ai prepaid gate READS. Both derive it from the
// one function, ai/object.Payer, so they cannot drift apart again — cloud keeping its own
// copy of this rule is exactly what let the console show a funded org while the gate
// refused the member.
func TestBalance_SubjectIsTheGateSubject(t *testing.T) {
// The allowlists are gone; set them to values that WOULD have flipped the resolution
// to prove they are inert — nothing reads them, the signup org still bills per-person.
t.Setenv("PERSONAL_BILLING_ORGS", "")
t.Setenv("ORG_BILLING_ORGS", "hanzo")
// The gate's subject for this principal, from ai itself — not a value this test invents.
// This is what routers/filter_balance.go resolveBillingKey computes from the JWT claims:
// a person in the signup org bills their OWN account, hanzo/z.
want := account.Payer(account.Credential{Owner: "hanzo", Name: "z"}).Subject()
if want != "hanzo/z" {
t.Fatalf("precondition: ai resolves a signup-org person to %q, want hanzo/z", want)
}
// Every identity shape a validated principal can arrive in must land on that ONE wallet.
for _, tc := range []struct{ name, userName, userID string }{
// Production: the identity boundary mints X-User-Name from the `name` claim.
{"gateway mints X-User-Name", "z", "8f14e45f-ea1b-4c2a-9f3d-000000000001"},
// In-binary direct-Bearer: X-User-Id is the UUID subject, X-User-Name carries the name.
{"in-binary direct bearer", "z", "hanzo/z"},
// No X-User-Name: the "<owner>/<name>" key form folds back via PayerOf.
{"owner/name id, no X-User-Name", "", "hanzo/z"},
} {
t.Run(tc.name, func(t *testing.T) {
fin := &fakeFinance{wallets: map[string]int64{}}
publishFinance(t, fin)
app := mountApp(t, "", "")
req := httptest.NewRequest(http.MethodGet, "/v1/billing/balance", nil)
req.Header.Set("X-User-Id", tc.userID)
req.Header.Set("X-Org-Id", "hanzo")
if tc.userName != "" {
req.Header.Set("X-User-Name", tc.userName)
}
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("Test: %v", err)
}
_ = resp.Body.Close()
if fin.calls == 0 {
t.Fatal("finance ledger was never read")
}
if fin.gotSubj != want {
t.Fatalf("console read wallet %q but the ai gate reads %q — the view and the gate disagree", fin.gotSubj, want)
}
})
}
}
// TestBalance_UnreadableIsNotZero is the fail-closed guard. A balance that cannot be read
// is UNKNOWN, and unknown must never render as a zero balance (a fabricated "you're broke")
// nor as a fabricated positive. It surfaces as an upstream failure.
func TestBalance_UnreadableIsNotZero(t *testing.T) {
publishFinance(t, &fakeFinance{err: fmt.Errorf("ledger open failed")})
app := mountApp(t, "", "")
code, body := call(t, app, http.MethodGet, "/v1/billing/balance", "maxpower/dave", "maxpower")
if code != http.StatusBadGateway {
t.Fatalf("unreadable balance: want 502, got %d (%s)", code, body)
}
// The body must be an honest error, never a balance object — a caller must not be able
// to mistake "unknown" for "zero".
var got map[string]any
if err := json.Unmarshal(body, &got); err != nil {
t.Fatalf("decode: %v (%s)", err, body)
}
if _, isBalance := got["available"]; isBalance {
t.Fatalf("an unreadable balance was rendered as a balance object: %s", body)
}
if got["error"] == nil {
t.Fatalf("want an honest error body, got: %s", body)
}
}
// TestBalance_RequiresSignIn proves the browser-facing sign-in gate is intact: no
// validated principal ⇒ 401, and the ledger is never touched. A forged service-token
// header cannot reach a wallet, because the org comes from the validated principal only.
func TestBalance_RequiresSignIn(t *testing.T) {
fin := &fakeFinance{wallets: map[string]int64{"hanzo|hanzo": 14953300}}
publishFinance(t, fin)
app := mountApp(t, "", "")
// No principal at all.
if code, _ := call(t, app, http.MethodGet, "/v1/billing/balance", "", ""); code != http.StatusUnauthorized {
t.Fatalf("anonymous: want 401, got %d", code)
}
// A client-supplied X-Org-Id with NO validated user is not a principal either — this
// is the off-cluster forge Red will try.
if code, _ := call(t, app, http.MethodGet, "/v1/billing/balance", "", "hanzo"); code != http.StatusUnauthorized {
t.Fatalf("forged org, no principal: want 401, got %d", code)
}
if fin.calls != 0 {
t.Fatalf("the ledger was read %d times without a validated principal", fin.calls)
}
}
// TestBalance_ScopedToCallerOrg proves tenant isolation: the org is taken from the
// VALIDATED principal, so a caller reads only its OWN org's ledger no matter what the
// request says. There is no client-supplied input that can widen it.
func TestBalance_ScopedToCallerOrg(t *testing.T) {
fin := &fakeFinance{wallets: map[string]int64{
"victim|victim": 999999,
"attacker|attacker": 5,
}}
publishFinance(t, fin)
app := mountApp(t, "", "")
// Attacker is validated in its OWN org and tries to name the victim's org/subject
// through every client-controlled channel the old proxy forwarded.
code, body := call(t, app, http.MethodGet,
"/v1/billing/balance?user=victim&userId=victim&customerId=victim&org=victim&currency=usd",
"attacker/mallory", "attacker")
if code != 200 {
t.Fatalf("want 200, got %d (%s)", code, body)
}
if fin.gotOrg != "attacker" {
t.Fatalf("cross-org read: ledger org = %q, want attacker", fin.gotOrg)
}
var got commerceBalance
_ = json.Unmarshal(body, &got)
if got.Available == 999999 {
t.Fatalf("attacker read the victim's balance: %s", body)
}
}
+50 -3
View File
@@ -246,6 +246,24 @@ func usage(s *cloud.Service[state], c *zip.Ctx) error {
if !ok {
return zip.ErrUnauthorized("sign in to view billing")
}
// Co-resident, read the usage ledger DIRECTLY from cloud's own finance ledger
// (usage_coresident.go explains why this is NOT a commerce proxy: proxying
// "/v1/billing/usage" re-enters THIS handler — commerce's own /v1/billing/usage
// route is behind //go:build cloud and never compiled here, so the only
// registration of that path is this handler — and the in-proc S2S hop carries no
// validated principal, so usage() self-answered "sign in to view billing"; that
// self-dispatch is the 500 a valid caller saw). This is the exact move balance()
// already makes. Off the co-resident path the commerce S2S proxy is unchanged.
if body, coResident, err := coResidentUsage(c.Context(), org, strings.TrimSpace(c.Query("product")), strings.TrimSpace(c.Query("groupBy"))); err != nil {
s.Log.Warn("finance usage read failed", "org", org, "err", err)
return zip.Errorf(http.StatusBadGateway, "billing upstream unreachable")
} else if coResident {
c.SetHeader("Content-Type", "application/json")
c.SetHeader("Cache-Control", "no-store")
return c.Bytes(http.StatusOK, body)
}
if !s.State.commerce.configured() {
return zip.Errorf(http.StatusNotImplemented, "billing is not configured")
}
@@ -267,10 +285,39 @@ func usage(s *cloud.Service[state], c *zip.Ctx) error {
return c.Bytes(status, body)
}
// balance → commerce GET /v1/billing/balance: the org's prepaid credit balance
// ({balance,holds,available} in USD cents), the SAME wallet the gateway debits.
// balance answers the caller's prepaid credit balance ({balance,holds,available} in USD
// cents) the SAME wallet the ai prepaid gate reads, the edge meter debits, and an admin
// grant credits.
//
// Co-resident it reads cloud's own finance ledger DIRECTLY (balance.go explains why this
// is NOT a commerce proxy: proxying "/v1/billing/balance" re-enters THIS handler, because
// commerceinproc dispatches the shared app by path and commerce's own billing routes are
// never registered in this binary — the proxy called itself and answered "sign in to view
// billing"). Off the co-resident path the commerce S2S proxy is unchanged.
//
// Holds are the ai gate's in-pod reservations, not a persisted ledger position (see
// types.FinanceClient.Balance), so the settled balance IS the available balance here.
func balance(s *cloud.Service[state], c *zip.Ctx) error {
return proxy(s, c, "/v1/billing/balance", "currency")
org, ok := principal.Org(c)
if !ok {
// A customer's OWN billing — never admin-gate it; an absent identity is a
// true "not signed in" (401), matching usage/gpuCharge.
return zip.ErrUnauthorized("sign in to view billing")
}
cents, coResident, err := availableCents(c.Context(), org, subjectFor(c, org))
if err != nil {
// A balance that cannot be READ is unknown — surface it as an upstream failure.
// It must never render as a zero balance: unknown is not "broke".
s.Log.Warn("finance balance read failed", "org", org, "err", err)
return zip.Errorf(http.StatusBadGateway, "billing upstream unreachable")
}
if !coResident {
return proxy(s, c, "/v1/billing/balance", "currency") // split deploy
}
c.SetHeader("Content-Type", "application/json")
// Per-tenant money must never be cached by the browser or an intermediary.
c.SetHeader("Cache-Control", "no-store")
return c.JSON(http.StatusOK, commerceBalance{Balance: cents, Holds: 0, Available: cents})
}
// gpuEligibility → commerce GET /v1/billing/gpu-eligibility: the read-only launch gate
+22 -7
View File
@@ -199,6 +199,21 @@ func financeBalance(s *cloud.Service[state], c *zip.Ctx) error {
if !ok {
return zip.ErrUnauthorized("sign in to view finance")
}
// The ONE balance read (balance.go) — the same wallet /v1/billing/balance answers, so
// the two surfaces can never disagree. Co-resident this is the finance ledger; only a
// split deploy falls through to the commerce S2S read below.
if cents, coResident, err := availableCents(c.Context(), org, subjectFor(c, org)); err != nil {
s.Log.Warn("finance balance read failed", "org", org, "err", err)
return zip.Errorf(http.StatusBadGateway, "billing upstream unreachable")
} else if coResident {
return financeJSON(s, c, financeBalanceView{
Currency: "usd",
AvailableCents: cents,
PendingCents: 0,
DueCents: 0,
AsOf: time.Now().UTC().Format(time.RFC3339),
})
}
if !s.State.commerce.configured() {
return zip.Errorf(http.StatusNotImplemented, "billing is not configured")
}
@@ -339,7 +354,7 @@ func financePaymentMethods(s *cloud.Service[state], c *zip.Ctx) error {
return zip.Errorf(http.StatusNotImplemented, "billing is not configured")
}
// Portal read filters on customerId; the subject is pinned to the caller's own org.
body, status, err := s.State.commerce.get(c.Context(), "/v1/billing/portal/payment-methods", org, financeSubject(org, nil))
body, status, err := s.State.commerce.get(c.Context(), "/v1/billing/portal/payment-methods", org, financeSubject(subjectFor(c, org), nil))
if err != nil {
s.Log.Warn("commerce payment-methods read failed", "org", org, "err", err)
return zip.Errorf(http.StatusBadGateway, "billing upstream unreachable")
@@ -421,12 +436,12 @@ func financeCaller(s *cloud.Service[state], c *zip.Ctx) (string, bool) {
return principal.Org(c)
}
// financeSubject builds the commerce query with every billing-subject key PINNED to org
// (the client can never widen scope), plus any extra passthrough params.
func financeSubject(org string, extra url.Values) url.Values {
// financeSubject builds the commerce query with every billing-subject key PINNED to
// subject (the client can never widen scope), plus any extra passthrough params.
func financeSubject(subject string, extra url.Values) url.Values {
q := url.Values{}
for _, k := range billingSubjectKeys {
q.Set(k, org)
q.Set(k, subject)
}
for k, vs := range extra {
for _, v := range vs {
@@ -439,7 +454,7 @@ func financeSubject(org string, extra url.Values) url.Values {
// financeGet does one org-scoped commerce GET and decodes the 2xx body into out. A
// non-2xx or unreachable upstream is surfaced honestly (never masked as empty data).
func financeGet(s *cloud.Service[state], c *zip.Ctx, path, org string, extra url.Values, out any) error {
body, status, err := s.State.commerce.get(c.Context(), path, org, financeSubject(org, extra))
body, status, err := s.State.commerce.get(c.Context(), path, org, financeSubject(subjectFor(c, org), extra))
if err != nil {
s.Log.Warn("commerce finance read failed", "org", org, "path", path, "err", err)
return zip.Errorf(http.StatusBadGateway, "billing upstream unreachable")
@@ -457,7 +472,7 @@ func financeGet(s *cloud.Service[state], c *zip.Ctx, path, org string, extra url
// credits/usage/ledger projections share). Tolerates the wrapped {transactions:[…]}
// shape and a bare array.
func financeTxns(s *cloud.Service[state], c *zip.Ctx, org string) ([]commerceTxn, error) {
body, status, err := s.State.commerce.get(c.Context(), "/v1/billing/transactions", org, financeSubject(org, url.Values{"limit": {"2000"}}))
body, status, err := s.State.commerce.get(c.Context(), "/v1/billing/transactions", org, financeSubject(subjectFor(c, org), url.Values{"limit": {"2000"}}))
if err != nil {
s.Log.Warn("commerce transactions read failed", "org", org, "err", err)
return nil, zip.Errorf(http.StatusBadGateway, "billing upstream unreachable")
+84
View File
@@ -0,0 +1,84 @@
package billing
// usage_coresident.go — the co-resident source for GET /v1/billing/usage.
//
// WHY IT EXISTS. balance() already reads cloud's OWN finance ledger directly rather
// than proxying "/v1/billing/balance" through commerceinproc, because co-resident the
// ONLY registration of that path is balance() itself (commerce's api.Route() is behind
// //go:build cloud and never compiled here), so the S2S proxy re-dispatches BY PATH
// straight back into the same handler, which self-answers "sign in to view billing"
// (the in-proc hop carries no validated principal). usage() had the SAME defect on
// "/v1/billing/usage" — a valid caller's usage read re-entered usage() and failed. This
// file gives usage() the co-resident answer balance() already has: the usage ledger read
// straight from finance (the wallet→revenue debits RecordUsage wrote), off the
// self-dispatching hop. Split deploy (no co-resident finance) falls back to the commerce
// S2S read, unchanged.
import (
"context"
"encoding/json"
"time"
"github.com/hanzoai/cloud/clients/finance"
)
// coResidentUsage builds the customer usage envelope from cloud's OWN finance ledger
// when the money plane is co-resident (finance.Current() published AND exposes the
// usage read). It returns (body, true, nil) with the commerce-shaped
// {user,count,usage:[...]} envelope — enriched + optionally ?product=filtered /
// ?groupBy=product-reduced exactly like the proxied path — or (nil, false, nil) when
// finance is not co-resident (split deploy), so the caller falls back to the commerce
// S2S read. This is what keeps /v1/billing/usage off the self-dispatching commerceinproc
// hop; a real read failure surfaces as a non-nil error (never a masked-empty ledger).
func coResidentUsage(ctx context.Context, org, product, groupBy string) ([]byte, bool, error) {
fin := finance.Current()
if fin == nil {
return nil, false, nil // split deploy → commerce S2S read
}
// The usage read is an OPTIONAL capability (the base FinanceClient is
// Balance+Deposit+RecordUsage); a finance impl without it falls back to the proxy.
lister, ok := fin.(interface {
ListUsage(context.Context, string, int) ([]finance.UsageRow, error)
})
if !ok {
return nil, false, nil
}
rows, err := lister.ListUsage(ctx, org, 2000)
if err != nil {
return nil, false, err
}
env := usageEnvelope(org, rows)
if out, ok := enrichUsageLedger(env, product, groupBy); ok {
return out, true, nil
}
return env, true, nil
}
// usageEnvelope renders finance usage rows as commerce's GetUsage envelope
// ({user,count,usage:[{transactionId,amount,metadata,createdAt}]}) — the exact shape the
// console's normalizeUsageRecords + this package's enrichUsageLedger already parse.
// amount is USD cents; metadata carries the metered unit (model) the debit recorded, so
// enrichUsageLedger can still attribute a product where the unit implies one.
func usageEnvelope(org string, rows []finance.UsageRow) []byte {
type usageRow struct {
TransactionID string `json:"transactionId"`
Amount int64 `json:"amount"`
Metadata map[string]any `json:"metadata"`
CreatedAt string `json:"createdAt"`
}
out := make([]usageRow, 0, len(rows))
for _, r := range rows {
md := map[string]any{}
if r.Model != "" {
md["model"] = r.Model
}
out = append(out, usageRow{
TransactionID: r.ID,
Amount: r.Cents,
Metadata: md,
CreatedAt: time.Unix(r.CreatedAt, 0).UTC().Format(time.RFC3339),
})
}
body, _ := json.Marshal(map[string]any{"user": org, "count": len(out), "usage": out})
return body
}
+89
View File
@@ -0,0 +1,89 @@
package billing
import (
"context"
"encoding/json"
"testing"
"github.com/hanzoai/cloud/clients/finance"
)
// TestCoResidentUsage proves usage() answers from the finance ledger (never the
// self-dispatching commerce hop) and shapes the commerce GetUsage envelope the
// console parses. fakeFinance + publishFinance live in balance_test.go.
func TestCoResidentUsage(t *testing.T) {
publishFinance(t, &fakeFinance{usageRows: []finance.UsageRow{
{ID: "u1", Cents: 150, Model: "gpt-x", CreatedAt: 1_700_000_000},
{ID: "u2", Cents: 75, Model: "embed-y", CreatedAt: 1_700_000_100},
}})
body, coResident, err := coResidentUsage(context.Background(), "acme", "", "")
if err != nil {
t.Fatalf("coResidentUsage: %v", err)
}
if !coResident {
t.Fatal("want coResident=true when finance is published")
}
var env struct {
User string `json:"user"`
Count int `json:"count"`
Usage []struct {
TransactionID string `json:"transactionId"`
Amount int64 `json:"amount"`
Metadata map[string]any `json:"metadata"`
CreatedAt string `json:"createdAt"`
} `json:"usage"`
}
if err := json.Unmarshal(body, &env); err != nil {
t.Fatalf("envelope not valid JSON: %v\n%s", err, body)
}
if env.User != "acme" || env.Count != 2 || len(env.Usage) != 2 {
t.Fatalf("bad envelope: user=%q count=%d rows=%d", env.User, env.Count, len(env.Usage))
}
if env.Usage[0].TransactionID != "u1" || env.Usage[0].Amount != 150 {
t.Errorf("row0 = %+v", env.Usage[0])
}
if env.Usage[0].Metadata["model"] != "gpt-x" {
t.Errorf("row0 metadata missing model: %+v", env.Usage[0].Metadata)
}
if env.Usage[0].CreatedAt == "" {
t.Error("row0 createdAt should be RFC3339, got empty")
}
}
// TestCoResidentUsageSplitDeploy proves that with no co-resident finance, usage()
// falls through to the commerce S2S proxy (coResident=false), unchanged.
func TestCoResidentUsageSplitDeploy(t *testing.T) {
finance.Publish(nil)
body, coResident, err := coResidentUsage(context.Background(), "acme", "", "")
if err != nil {
t.Fatalf("coResidentUsage: %v", err)
}
if coResident || body != nil {
t.Fatalf("want fall-through (coResident=false, nil body); got coResident=%v body=%s", coResident, body)
}
}
// TestCoResidentUsageGroupBy proves the ?groupBy=product reduction still runs on the
// co-resident path (a token-metered row attributes to product "inference").
func TestCoResidentUsageGroupBy(t *testing.T) {
publishFinance(t, &fakeFinance{usageRows: []finance.UsageRow{{ID: "u1", Cents: 150, Model: "gpt-x", CreatedAt: 1_700_000_000}}})
body, ok, err := coResidentUsage(context.Background(), "acme", "", "product")
if err != nil || !ok {
t.Fatalf("coResidentUsage groupBy: ok=%v err=%v", ok, err)
}
var grouped struct {
GroupBy string `json:"groupBy"`
Groups []struct {
Product string `json:"product"`
Requests int `json:"requests"`
AmountCents int64 `json:"amountCents"`
} `json:"groups"`
}
if err := json.Unmarshal(body, &grouped); err != nil {
t.Fatalf("grouped not valid JSON: %v\n%s", err, body)
}
if grouped.GroupBy != "product" || len(grouped.Groups) != 1 {
t.Fatalf("bad grouped envelope: %s", body)
}
}
-209
View File
@@ -1,209 +0,0 @@
package bot
import (
"bufio"
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strings"
)
// coding.go is the IN-PROCESS client for bot-gateway's native coding-task runner
// (POST /v1/coding-tasks). It is the ONE wire seam cloud uses to hand a coding job
// to the sandbox runtime — distinct from the /v1/bot/* reverse proxy (bot.go),
// which relays an inbound browser/API request. Here cloud ORIGINATES the call
// server-side (from a Slack coding trigger), so it mints the identity headers
// itself (X-Org-Id / X-User-Id) and presents the shared gateway service token as
// the pod-boundary bearer.
//
// The transport is a line-delimited JSON (NDJSON) stream: bot-gateway emits one
// JSON object per line as the run progresses (clone → dev exec → commit → push),
// each a `step`/`log`, and a final `result` (or `error`) line. Cloud mirrors every
// step into the agent session as it arrives, so `GET /v1/agents/sessions/:id/stream`
// carries the run live, and returns the terminal result.
//
// CREDENTIAL CUSTODY: the per-org agent git credential travels in the request
// BODY (never on a URL, never on argv, never logged). bot-gateway injects it into
// git via an env-fed http.extraHeader inside the sandbox; cloud never logs the
// Credential field and this client never places it in an error string.
// Credential is the per-org agent git credential the sandbox presents to native
// git. Token is the secret (an hk- key); Username is the basic-auth user label.
// Marshaled into the request body only — never logged.
type Credential struct {
Username string `json:"username"`
Token string `json:"token"`
}
// CodingTaskRequest is the cloud→bot contract for one coding run.
type CodingTaskRequest struct {
CloneURL string `json:"cloneUrl"` // https://<domain>/v1/git/<org>/<repo>.git
BaseBranch string `json:"baseBranch"` // branch to start from (default repo default)
Branch string `json:"branch"` // branch to create + push (e.g. agent/<sessionid>)
Prompt string `json:"prompt"` // the engineering task
SessionID string `json:"sessionId"` // cloud session id (correlation)
RunTimeoutSeconds int `json:"runTimeoutSeconds"` // sandbox run budget
Credential Credential `json:"credential"` // agent git credential (write-only)
}
// CodingStep is one progress event mirrored into the session. Type is "step" or
// "log"; Step names the phase (clone|plan|edit|commit|push); Status is ok|error
// for a completed phase. All fields are safe to surface (no credential).
type CodingStep struct {
Type string `json:"type"`
Step string `json:"step,omitempty"`
Message string `json:"message,omitempty"`
Status string `json:"status,omitempty"`
}
// CodingTaskResult is the terminal outcome bot-gateway reports.
type CodingTaskResult struct {
Branch string `json:"branch"`
CommitSha string `json:"commitSha"`
Diffstat string `json:"diffstat"`
Changed bool `json:"changed"`
OK bool `json:"ok"`
LogTail string `json:"logTail"`
Error string `json:"error,omitempty"`
}
// codingEnvelope is the discriminated line shape: one of step/log/result/error.
type codingEnvelope struct {
Type string `json:"type"` // step | log | result | error
Step string `json:"step,omitempty"`
Message string `json:"message,omitempty"`
Status string `json:"status,omitempty"`
Branch string `json:"branch,omitempty"`
CommitSha string `json:"commitSha,omitempty"`
Diffstat string `json:"diffstat,omitempty"`
Changed bool `json:"changed,omitempty"`
OK bool `json:"ok,omitempty"`
LogTail string `json:"logTail,omitempty"`
}
const (
// codingLineCap bounds one NDJSON line (a diffstat/log line can be large but
// never unbounded) so a hostile/huge line can't exhaust cloud memory.
codingLineCap = 1 << 20 // 1 MiB per line
// codingErrBodyCap bounds the non-2xx error body we read for a message.
codingErrBodyCap = 64 << 10
)
// codingHTTP has NO client-side timeout: a coding run legitimately streams for
// minutes. The deadline is the caller's ctx (bounded by the coding job), applied
// to the request; a stuck stream is cut when ctx expires.
var codingHTTP = &http.Client{}
// RunCodingTask POSTs a coding job to bot-gateway and streams the NDJSON result,
// invoking onStep for each progress line as it arrives. It returns the terminal
// result. A non-2xx status, a transport error, or a stream that ends without a
// terminal line is an error (no partial success is fabricated). org/userID are
// the gateway-minted tenant context bot-gateway trusts AFTER its own bearer gate.
func RunCodingTask(ctx context.Context, org, userID string, req CodingTaskRequest, onStep func(CodingStep)) (CodingTaskResult, error) {
payload, err := json.Marshal(req)
if err != nil {
return CodingTaskResult{}, fmt.Errorf("bot: marshal coding request: %w", err)
}
// This POST carries the org's hk- git credential AND the shared gateway bearer,
// so it must not travel cleartext. Require https by default; a plaintext in-cluster
// target is allowed ONLY when the operator asserts the hop is secured by mesh mTLS
// (BOT_GATEWAY_ALLOW_PLAINTEXT=1) — an explicit, documented trust decision, never a
// silent default.
target, terr := codingTaskTarget()
if terr != nil {
return CodingTaskResult{}, terr
}
hreq, err := http.NewRequestWithContext(ctx, http.MethodPost, target, bytes.NewReader(payload))
if err != nil {
return CodingTaskResult{}, fmt.Errorf("bot: build coding request: %w", err)
}
hreq.Header.Set("Content-Type", "application/json")
hreq.Header.Set("Accept", "application/x-ndjson")
// Server-originated identity: cloud has already resolved the tenant, so it mints
// the headers bot-gateway trusts (post pod-boundary auth). The service bearer is
// the shared gateway token (KMS-injected env); absent => bot-gateway fails the
// request closed at its auth gate.
hreq.Header.Set("X-Org-Id", org)
if userID != "" {
hreq.Header.Set("X-User-Id", userID)
}
if tok := getenv("BOT_GATEWAY_TOKEN"); tok != "" {
hreq.Header.Set("Authorization", "Bearer "+tok)
}
resp, err := codingHTTP.Do(hreq)
if err != nil {
return CodingTaskResult{}, fmt.Errorf("bot: coding gateway unreachable: %w", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(io.LimitReader(resp.Body, codingErrBodyCap))
return CodingTaskResult{}, fmt.Errorf("bot: coding gateway status %d: %s",
resp.StatusCode, strings.TrimSpace(string(body)))
}
sc := bufio.NewScanner(resp.Body)
sc.Buffer(make([]byte, 0, 64<<10), codingLineCap)
var result CodingTaskResult
var gotTerminal bool
for sc.Scan() {
line := bytes.TrimSpace(sc.Bytes())
if len(line) == 0 {
continue
}
var env codingEnvelope
if err := json.Unmarshal(line, &env); err != nil {
continue // skip a malformed line rather than abort the whole run
}
switch env.Type {
case "result":
result = CodingTaskResult{
Branch: env.Branch, CommitSha: env.CommitSha, Diffstat: env.Diffstat,
Changed: env.Changed, OK: env.OK, LogTail: env.LogTail,
}
gotTerminal = true
case "error":
result = CodingTaskResult{OK: false, LogTail: env.LogTail, Error: nonEmpty(env.Message, "coding task failed")}
gotTerminal = true
default: // step | log — mirror live
if onStep != nil {
onStep(CodingStep{Type: env.Type, Step: env.Step, Message: env.Message, Status: env.Status})
}
}
}
if err := sc.Err(); err != nil {
return CodingTaskResult{}, fmt.Errorf("bot: coding stream read: %w", err)
}
if !gotTerminal {
return CodingTaskResult{}, fmt.Errorf("bot: coding stream ended without a result")
}
return result, nil
}
// codingTaskTarget resolves the /v1/coding-tasks endpoint and fails closed on a
// cleartext http:// target — the request carries the org's git credential and the
// gateway bearer. BOT_GATEWAY_ALLOW_PLAINTEXT=1 is the explicit, operator-set
// opt-out for a deployment whose bot-gateway hop is secured by mesh mTLS.
func codingTaskTarget() (string, error) {
base := botURL()
u, err := url.Parse(base)
if err != nil {
return "", fmt.Errorf("bot: invalid gateway url")
}
if u.Scheme != "https" && getenv("BOT_GATEWAY_ALLOW_PLAINTEXT") != "1" {
return "", fmt.Errorf("bot: refusing to send the coding credential over cleartext %q (set BOT_GATEWAY_URL to https, or BOT_GATEWAY_ALLOW_PLAINTEXT=1 if the hop is mesh-mTLS secured)", u.Scheme)
}
return base + "/v1/coding-tasks", nil
}
// nonEmpty returns a when non-empty, else b.
func nonEmpty(a, b string) string {
if strings.TrimSpace(a) != "" {
return a
}
return b
}
+135 -413
View File
@@ -1,166 +1,98 @@
// Package bots mounts the Hanzo Cloud POST /v1/bots/run surface: launch a
// computer-using agent (a "bot") — a booted desktop or terminal sandbox the
// operative computer-use runtime drives to do a task — and hand back a LIVE
// Package bots is the CONTROL PLANE for a bot run: a task the bot runtime
// executes on a surface — a desktop or terminal sandbox it drives — with a LIVE
// session (the URL the hanzo.app /vnc panel embeds to watch/attach).
//
// This handler is a THIN ORCHESTRATOR, deliberately. It does NOT boot machines,
// speak VNC, or reimplement visor: those live in the separate TS `bot` service
// (its gateway exposes the browser<->gateway<->node HMAC VNC tunnel at
// /vnc?nodeId=<id>) and in visor (machine provisioning). This handler owns the
// three things a cloud control-plane owns:
// A bot run is ONE value with ONE home. It is not the bot MACHINE that hosts a
// runtime (visor's /v1/compute/bots — a machine you rent), and it is not the
// runtime service itself (clients/runtime — the transport to the executor).
//
// 1. authenticate the caller (a run MOVES MONEY, so a VALIDATED principal is
// required — never a bare, forgeable org header);
// 2. gate + meter the run against the caller's OWN org ledger (a flat per-run
// fee, the same ResourceMeter path every non-LLM resource uses);
// 3. mint the run id and return the session descriptor whose sessionUrl points
// at the bot VNC gateway for that run.
// CLOUD OWNS POLICY, THE RUNTIME OWNS THE RUN. The sandbox lives in the runtime,
// keyed in the runtime's own store under the tenant that started it; that store is
// the only thing that knows whether a run is alive. So this package keeps no
// second copy of it. It owns what a control plane owns — who you are, which org
// you are, and whether you may — and then asks the runtime, which IS the registry.
// Copying that state into cloud would create a second id space agreeing with
// nothing: listing runs that do not exist and stopping runs never started.
//
// Tenant isolation is the gateway-minted X-Org-Id (HIP-0026), resolved via
// principal.Org and NEVER read from the request body — so one tenant can
// never launch, or bill, a bot against another's org.
// Isolation: the org is the gateway-minted X-Org-Id (HIP-0026) resolved via
// principal.Org, NEVER a request field, and it is what cloud sends the runtime,
// which keys every run under tenants/{org}/. A caller cannot name another tenant's
// org, so it cannot read or stop another tenant's runs; a foreign run id resolves
// under the CALLER's org, where it does not exist, and answers 404.
//
// Surface (org-scoped; the CLI `hanzo bot run` calls it):
// Surface (org-scoped; the console BotsApi and the CLI `hanzo bot run` call it):
//
// POST /v1/bots/run {task, surface, gpu, timeout} -> {runId, status, sessionUrl}
//
// The billed unit is a flat per-RUN fee — the honest, policy-set unit a bot
// launch bills. GB-seconds / GPU-hour metering is intentionally NOT fabricated
// here: this endpoint launches the run, it does not observe its runtime
// footprint (visor/bot-gateway do). The fee is ops-configurable per deployment
// via cloud.ResourceFeeCents(botFeeEnvPrefix, meterKind); set it to 0 to make
// bot launches free (and therefore un-gated), exactly like the agents run fee.
// POST /v1/bots/run -> 501: no runtime launch operation exists yet
// GET /v1/bots -> {bots:[{runId,task,surface,status,sessionUrl,startedAt}]}
// POST /v1/bots/:runId/stop -> {runId, status}
package bots
import (
"bytes"
"crypto/rand"
"encoding/hex"
"encoding/json"
"context"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strings"
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/metering"
"github.com/hanzoai/cloud/clients/principal"
"github.com/hanzoai/cloud/clients/runtime"
"github.com/zap-proto/zip"
)
const (
// meterKind is the commerce "provider"/attribution label for bot spend — the
// product:"bot". One value so every bot launch is attributed identically on
// the ledger.
meterKind = "bot"
// botFeeEnvPrefix is the operator knob for the flat per-run launch fee. The
// effective fee is cloud.ResourceFeeCents(botFeeEnvPrefix, meterKind): a
// CLOUD_BOT_FEE_CENTS override wins over the $1.00 default; set it to 0 to
// make bot launches free (and therefore un-gated). A flat per-RUN fee — the
// policy-set unit a launch bills; NOT a fabricated GB-second/GPU-hour price.
botFeeEnvPrefix = "CLOUD_BOT_FEE_CENTS"
// gatewayURLEnv configures the browser-facing bot VNC gateway base — the
// public origin the TS bot service serves /vnc?nodeId=<id> from, which the
// hanzo.app /vnc panel embeds. It is DISTINCT from clients/bot's server-side
// BOT_GATEWAY_URL (the in-cluster reverse-proxy target http://bot-gateway.hanzo.svc,
// which a browser cannot reach): a session URL must be publicly embeddable, so
// it carries its own knob. Ops sets it per deployment/brand.
// gatewayURLEnv configures the browser-facing bot VNC gateway base — the public
// origin the TS bot service serves /vnc?nodeId=<id> from, which the hanzo.app
// /vnc panel embeds. It is DISTINCT from the runtime's in-cluster address
// (clients/runtime's BOT_GATEWAY_URL, a pod-internal DNS name a browser cannot
// reach): a session URL must be publicly embeddable, so it carries its own knob.
gatewayURLEnv = "CLOUD_BOT_GATEWAY_URL"
defaultGatewayURL = "https://bot.hanzo.ai"
// serverGatewayURLEnv is the IN-CLUSTER, server-side bot-gateway base the
// control plane calls to list/stop live runs. It is the SAME knob clients/bot's
// reverse proxy uses (BOT_GATEWAY_URL, default http://bot-gateway.hanzo.svc) —
// NOT the browser-facing gatewayURLEnv above (a pod-internal DNS name a browser
// can't reach). List/stop are server->server calls, so they ride the in-cluster
// target; only the returned sessionUrl carries the public origin.
serverGatewayURLEnv = "BOT_GATEWAY_URL"
defaultServerGatewayURL = "http://bot-gateway.hanzo.svc"
// maxRunID bounds the :runId path param before it is sent onward — an oversize
// id is not a run this org owns, so it is a 404 like any other miss.
maxRunID = 128
// gatewayCallTimeout bounds a single list/stop round-trip to the bot-gateway so
// a hung gateway can't stall the control-plane request; on timeout list is
// honest-empty and stop is a clean 502.
gatewayCallTimeout = 15 * time.Second
// maxTask bounds the launch task/prompt at the create boundary.
maxTask = 32 * 1024
// maxTimeout caps the requested wall-clock so a client can't ask for an
// unbounded run; the runtime enforces the real limit, this is the sane input
// bound.
maxTimeout = 24 * time.Hour
surfaceDesktop = "desktop"
surfaceTerminal = "terminal"
// statusRunning is the launch outcome this endpoint reports: the run was
// authorized, metered, and its live session URL is returned for the client to
// attach to. The bot's fine-grained runtime lifecycle (booting/ready/stopped)
// is visor/bot-gateway's concern and is observed there — this orchestration
// record carries ONE status: the launch succeeded.
// statusRunning is what a listed run reports when the runtime names no status of
// its own. statusStopped is the terminal outcome a stop reports.
statusRunning = "running"
// statusStopped is the terminal outcome POST /v1/bots/:runId/stop reports once
// the bot-gateway has torn the run's live session down.
statusStopped = "stopped"
)
// identityHeaders are the gateway-minted tenant-context headers forwarded on a
// server-side list/stop call so the bot-gateway scopes the operation to the SAME
// caller — the exact set clients/bot's reverse proxy forwards. X-Org-Id is set
// explicitly to the validated org (never the raw request header), so a forged
// org can never reach the gateway; the rest ride through for the audit trail.
var identityHeaders = []string{
"Authorization", "X-User-Id", "X-User-Email", "X-Project-Id", "X-Environment",
// Runtime is the seam onto the run registry — the bot runtime, which owns the
// sandboxes and is therefore the only truthful answer to "what is running". Bound
// to the real transport in wire.go; a fake in tests.
//
// Every method takes org FIRST and the runtime scopes by it. The seam carries no
// authority: cloud decides WHETHER a caller may ask, the runtime answers WHAT it
// holds for that org.
type Runtime interface {
List(ctx context.Context, org string) ([]Run, error)
Stop(ctx context.Context, org, runID string) error
}
// Run is one bot run as the runtime reports it.
type Run struct {
ID string
Task string
Surface string
Status string
StartedAt string // RFC3339, as the runtime stamps it
}
// state is bots' own data; shared deps live in the embedded cloud.Base.
type state struct {
// bill is the shared per-org gate+meter (reuses deps.Metering, the ONE
// commerce client the agents/provisioning/ml subsystems use). Nil/!Enabled()
// makes Gate allow and Meter a no-op, so an unconfigured deployment launches
// bots without billing rather than failing closed on a missing ledger. It stays
// here (not Base.Bill) because its provider label is meterKind ("bot"), which
// diverges from the subsystem name ("bots") — Base.Bill would attribute to the
// wrong product.
bill *cloud.ResourceMeter
// gateway is the browser-facing bot VNC gateway base (no trailing slash) that
// every returned sessionUrl is derived from.
gateway string
// serverGateway is the in-cluster bot-gateway base (no trailing slash) the
// control plane calls server-side to list + stop an org's live runs.
serverGateway string
// cc is the outbound client for those server->server calls; a bounded timeout
// keeps a hung gateway from stalling the request (list falls back to empty).
cc *http.Client
}
// runReq is the boot-a-computer-using-agent body — the exact shape the CLI
// (cli/bot.go BotRunReq) sends. Org is NEVER here: it is the gateway-minted
// X-Org-Id, resolved from the validated principal, never the body.
type runReq struct {
Task string `json:"task"`
Surface string `json:"surface"` // desktop | terminal
GPU bool `json:"gpu"`
Timeout string `json:"timeout"` // optional wall-clock, e.g. "30m"
}
// runView is the live-session descriptor — the exact shape the CLI
// (cli/bot.go BotRunResult) decodes: the run id, its status, and the URL the
// hanzo.app /vnc panel embeds.
type runView struct {
RunID string `json:"runId"`
Status string `json:"status"`
SessionURL string `json:"sessionUrl"`
// runtime is the run registry — what list reads and what stop drives.
runtime Runtime
}
// botView is one row of GET /v1/bots — the console list item. sessionUrl is
// derived control-plane side from runId (the ONE place a session URL is built,
// sessionURL below), so the gateway never has to know its own public origin.
// derived control-plane side from runId (the ONE place a session URL is built), so
// the runtime never has to know its own public origin.
type botView struct {
RunID string `json:"runId"`
Task string `json:"task"`
@@ -170,8 +102,8 @@ type botView struct {
StartedAt string `json:"startedAt"`
}
// botsView is the GET /v1/bots envelope; Bots is always non-nil so an empty org
// (or an unreachable gateway) serializes as {"bots":[]}, never {"bots":null}.
// botsView is the GET /v1/bots envelope; Bots is always non-nil so an org with no
// runs serializes as {"bots":[]}, never {"bots":null}.
type botsView struct {
Bots []botView `json:"bots"`
}
@@ -182,20 +114,7 @@ type stopView struct {
Status string `json:"status"`
}
// gatewayBot is the bot-gateway's session shape: the caller's run minus the
// sessionUrl (control-plane-derived). Its own /v1/bots emits exactly these
// fields; a shape drift that fails to decode collapses to honest-empty.
type gatewayBot struct {
RunID string `json:"runId"`
Task string `json:"task"`
Surface string `json:"surface"`
Status string `json:"status"`
StartedAt string `json:"startedAt"`
}
// Mount wires the bots surface onto app per HIP-0106. Constructs the value directly
// (cloud.NewBase) because the metered launch fee uses a meter keyed to meterKind
// ("bot"), not the subsystem name — so it lives in State, built from Deps here.
// Mount wires the bots surface onto app per HIP-0106.
func Mount(app *zip.App, deps cloud.Deps) error {
if app == nil {
return fmt.Errorf("bots.Mount: nil zip.App")
@@ -204,109 +123,45 @@ func Mount(app *zip.App, deps cloud.Deps) error {
return fmt.Errorf("bots.Mount: nil deps.Logger")
}
s := &cloud.Service[state]{
Base: cloud.NewBase(deps, "bots"),
State: state{
bill: cloud.NewResourceMeter(deps, meterKind),
gateway: gatewayBase(),
serverGateway: serverGatewayBase(),
cc: &http.Client{Timeout: gatewayCallTimeout},
},
Base: cloud.NewBase(deps, "bots"),
State: state{gateway: gatewayBase(), runtime: wire{}},
}
routes(app, s)
s.Log.Info("bots surface mounted", "gateway", s.State.gateway,
"serverGateway", s.State.serverGateway, "billing", s.State.bill.Enabled(),
"brand", deps.Brand)
s.Log.Info("bots surface mounted", "gateway", s.State.gateway, "brand", deps.Brand)
return nil
}
// routes registers the bots surface: launch, list, and stop. list/stop are the
// read/lifecycle half — org-scoped proxies onto the bot-gateway's live runs.
// routes registers the bots surface. The static /run literal and the :runId param
// are resolved by specificity, so /v1/bots/run can never bind as a run id.
func routes(app *zip.App, s *cloud.Service[state]) {
app.Post("/v1/bots/run", cloud.Handle(s, run))
app.Get("/v1/bots", cloud.Handle(s, list))
app.Post("/v1/bots/:runId/stop", cloud.Handle(s, stop))
}
// run launches a computer-using bot: it authenticates the caller, gates+meters a
// flat per-run fee against the caller's OWN org, mints the run id, and returns
// the live VNC session descriptor. Every 200 reflects an authorized, metered
// launch — an unfunded org gets 402 and no session, an unreachable commerce 503.
func run(s *cloud.Service[state], c *zip.Ctx) error {
org, ok := principal.Org(c)
if !ok {
return zip.ErrForbidden("X-Org-Id required")
}
// A launch MOVES MONEY (debits the org's commerce ledger), so it requires a
// VALIDATED principal — never a bare, forgeable X-Org-Id from the direct-to-pod
// path. Same money-path guard the agents run + s3 + provisioning surfaces ship.
if !principal.Validated(c) {
return zip.ErrForbidden("a validated principal is required to launch a bot")
}
var body runReq
if err := c.Bind(&body); err != nil {
return err
}
task := strings.TrimSpace(body.Task)
if task == "" {
return zip.ErrBadRequest("task is required")
}
if len(task) > maxTask {
return zip.ErrBadRequest("task too large")
}
surface, err := validateSurface(body.Surface)
if err != nil {
return err
}
if err := validateTimeout(body.Timeout); err != nil {
return err
}
runID, err := genID("bot")
if err != nil {
return zip.Errorf(http.StatusInternalServerError, "rng: %v", err)
}
// Pre-authorize the CALLER's org balance BEFORE returning a session (fail-
// closed): an unfunded org gets 402 and no bot, an unreachable commerce 503.
// project = the caller's validated org sub-scope, so a per-scope spend cap is
// enforced on a bot launch exactly as on the request edge. fee<=0 or
// unconfigured billing makes this a no-op (allow).
fee := cloud.ResourceFeeCents(botFeeEnvPrefix, meterKind)
project, projectValidated := principal.ValidatedProject(c)
if gateErr := s.State.bill.Gate(c.Context(), principal.Payer(c), project, projectValidated, meterKind, fee); gateErr != nil {
return cloud.DenyResource(c, gateErr)
}
// The launch is authorized. The durable, attributable record of this run is
// the commerce ledger debit (product=bot, the surface as the billed unit, the
// acting principal for the audit trail) — fire-and-forget, exactly like the
// agents run fee. GPU/surface ride the log line for operator visibility.
s.State.bill.MeterUsage(principal.Payer(c), meterKind, metering.Usage{
AmountCents: fee,
Model: surface,
Actor: billingActor(org, c.User()),
RequestID: c.RequestID(),
ClientIP: cloud.ClientIP(c),
})
s.Log.Info("bot launched", "org", org, "run", runID,
"surface", surface, "gpu", body.GPU)
return c.JSON(http.StatusOK, runView{
RunID: runID,
Status: statusRunning,
SessionURL: sessionURL(s, runID),
})
// run reports that launching is not implemented.
//
// There is no launch operation on the bot runtime, so nothing in cloud can start a
// sandbox. This endpoint used to mint a run id, charge a flat per-run fee, and hand
// back a sessionUrl for a bot that never booted — an id the runtime had never heard
// of, pointing at a VNC node that did not exist, for money that was really taken.
// 501 is the truth, and the truth is cheaper than a plausible lie.
//
// Restoring it needs a runtime-side launch operation first (TS, cross-repo); the
// gate and the meter belong in the same change that can prove a bot boots.
func run(_ *cloud.Service[state], _ *zip.Ctx) error {
return zip.Errorf(http.StatusNotImplemented,
"launching a bot is not implemented: the bot runtime exposes no launch operation, so cloud cannot start one")
}
// list returns the caller org's live bot runs. It proxies the bot-gateway's
// org-scoped session list (server-side, forwarding the caller's tenant context)
// and normalizes each row into the console contract, deriving sessionUrl here.
// list returns the caller org's live bot runs, read from the runtime and projected
// into the console contract with sessionUrl derived here.
//
// The org is ALWAYS the validated principal's org, NEVER a request param — one
// tenant can never enumerate another's runs. It is honest-empty by construction:
// an unconfigured or unreachable gateway, a non-2xx, or a shape it can't decode
// all yield {"bots":[]} (a 200), never a 5xx — the console renders "no bots"
// rather than an error when the runtime plane is simply down.
// The org is ALWAYS the validated principal's org, NEVER a request param, and it is
// what scopes the runtime's answer — so one tenant can never enumerate another's
// runs. A runtime that cannot answer is an error, not an empty list: [] would tell
// the caller "your org has no runs", which is a different claim from "we could not
// ask", and the difference is the whole reason this endpoint exists.
func list(s *cloud.Service[state], c *zip.Ctx) error {
org, ok := principal.Org(c)
if !ok {
@@ -314,18 +169,47 @@ func list(s *cloud.Service[state], c *zip.Ctx) error {
}
// Org-scoping is only trustworthy behind a validated principal: a bare,
// forgeable X-Org-Id (the direct-to-pod path) must not enumerate a victim
// tenant's runs. Same guard clients/bot's proxy applies before handing the
// gateway a tenant context.
// tenant's runs.
if !principal.Validated(c) {
return zip.ErrForbidden("a validated principal is required to list bots")
}
return c.JSON(http.StatusOK, botsView{Bots: fetchBots(s, c, org)})
runs, err := s.State.runtime.List(c.Context(), org)
if err != nil {
return zip.Errorf(http.StatusBadGateway, "bots: the runtime could not list this org's runs: %v", err)
}
out := make([]botView, 0, len(runs))
for _, r := range runs {
out = append(out, toBotView(s, r))
}
return c.JSON(http.StatusOK, botsView{Bots: out})
}
// stop terminates one of the caller org's live runs. It proxies the bot-gateway's
// org-scoped stop; a run the caller's org does not own is a 404 (never a 200,
// never another tenant's teardown). An unreachable gateway is a clean 502 — a
// stop that could not reach the runtime must not claim the run was stopped.
// toBotView projects a run into one list row, deriving sessionUrl from the run id.
func toBotView(s *cloud.Service[state], r Run) botView {
status := strings.TrimSpace(r.Status)
if status == "" {
status = statusRunning
}
return botView{
RunID: r.ID,
Task: r.Task,
Surface: r.Surface,
Status: status,
SessionURL: sessionURL(s, r.ID),
StartedAt: r.StartedAt,
}
}
// stop terminates one of the caller org's own runs.
//
// The own-key guard is the org: it is the caller's validated org, never theirs to
// choose, and the runtime resolves the run id UNDER it. A run belonging to another
// tenant is not among this org's runs, so it answers absent — the same 404 a
// nonexistent id gets, which is what keeps this from being an oracle.
//
// Absence is honoured ONLY when the runtime answers it. A runtime that does not
// serve stop reports nothing about the run, and reporting "stopped" on that basis
// would be a stop that cannot fail — so it is a 502.
func stop(s *cloud.Service[state], c *zip.Ctx) error {
org, ok := principal.Org(c)
if !ok {
@@ -338,175 +222,33 @@ func stop(s *cloud.Service[state], c *zip.Ctx) error {
if runID == "" {
return zip.ErrBadRequest("runId is required")
}
code, err := stopBot(s, c, org, runID)
if err != nil {
return zip.Errorf(http.StatusBadGateway, "bots: gateway unreachable: %v", err)
if len(runID) > maxRunID {
return zip.ErrNotFound("no such bot for this org")
}
switch {
case code >= 200 && code < 300:
switch err := s.State.runtime.Stop(c.Context(), org, runID); {
case err == nil:
s.Log.Info("bot stopped", "org", org, "run", runID)
return c.JSON(http.StatusOK, stopView{RunID: runID, Status: statusStopped})
case code == http.StatusNotFound:
case errors.Is(err, runtime.ErrNotFound):
return zip.ErrNotFound("no such bot for this org")
case errors.Is(err, runtime.ErrNotServed):
return zip.Errorf(http.StatusBadGateway,
"bots: the runtime does not serve stop, so this run's state is unknown — it was NOT stopped")
default:
return zip.Errorf(http.StatusBadGateway, "bots: gateway rejected stop (%d)", code)
return zip.Errorf(http.StatusBadGateway, "bots: the runtime could not stop this run: %v", err)
}
}
// fetchBots calls the bot-gateway's GET /v1/bots scoped to org and maps the
// result into the contract. Every failure path — no server gateway configured,
// build error, transport error, non-2xx, or an undecodable body — returns a
// non-nil empty slice so the caller serializes {"bots":[]} and never a 5xx.
func fetchBots(s *cloud.Service[state], c *zip.Ctx, org string) []botView {
out := []botView{}
if s.State.serverGateway == "" {
return out
}
req, err := gatewayRequest(c, http.MethodGet, s.State.serverGateway+"/v1/bots", org, nil)
if err != nil {
return out
}
resp, err := s.State.cc.Do(req)
if err != nil {
s.Log.Warn("bots list: gateway unreachable, returning empty", "org", org, "err", err)
return out
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
s.Log.Warn("bots list: gateway non-2xx, returning empty", "org", org, "status", resp.StatusCode)
return out
}
body, err := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
if err != nil {
return out
}
var decoded struct {
Bots []gatewayBot `json:"bots"`
}
if err := json.Unmarshal(body, &decoded); err != nil {
s.Log.Warn("bots list: undecodable gateway body, returning empty", "org", org, "err", err)
return out
}
for _, b := range decoded.Bots {
runID := strings.TrimSpace(b.RunID)
if runID == "" {
continue
}
status := strings.TrimSpace(b.Status)
if status == "" {
status = statusRunning
}
out = append(out, botView{
RunID: runID,
Task: b.Task,
Surface: b.Surface,
Status: status,
SessionURL: sessionURL(s, runID),
StartedAt: b.StartedAt,
})
}
return out
}
// stopBot calls the bot-gateway's POST /v1/bots/{runId}/stop scoped to org and
// returns the gateway status code. A transport failure is a non-nil error the
// caller maps to 502; the status code drives 200-vs-404.
func stopBot(s *cloud.Service[state], c *zip.Ctx, org, runID string) (int, error) {
if s.State.serverGateway == "" {
return 0, fmt.Errorf("bot gateway not configured")
}
target := s.State.serverGateway + "/v1/bots/" + url.PathEscape(runID) + "/stop"
req, err := gatewayRequest(c, http.MethodPost, target, org, bytes.NewReader(nil))
if err != nil {
return 0, err
}
resp, err := s.State.cc.Do(req)
if err != nil {
return 0, err
}
defer func() { _ = resp.Body.Close() }()
_, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 1<<20))
return resp.StatusCode, nil
}
// gatewayRequest builds a server-side call to the bot-gateway carrying the
// caller's tenant context: X-Org-Id is pinned to the validated org, the other
// identity headers ride through, so the gateway scopes to exactly this caller.
func gatewayRequest(c *zip.Ctx, method, target, org string, body io.Reader) (*http.Request, error) {
req, err := http.NewRequestWithContext(c.Context(), method, target, body)
if err != nil {
return nil, err
}
for _, h := range identityHeaders {
if v := c.Header(h); v != "" {
req.Header.Set(h, v)
}
}
// Pin the validated org last so a forged X-Org-Id in the incoming headers can
// never override the tenant the gateway scopes to.
req.Header.Set("X-Org-Id", org)
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
return req, nil
}
// sessionURL derives the live VNC session URL for a run: the browser-facing bot
// gateway base + the node's VNC path. One id per run — the run id IS the node id
// the bot machine for this run registers under, so the tunnel is addressable by
// exactly the id the client holds.
// gateway base + the node's VNC path. The run id IS the node id the runtime
// registers the session under, so the tunnel is addressable by exactly the id the
// client holds.
func sessionURL(s *cloud.Service[state], runID string) string {
return s.State.gateway + "/vnc?" + url.Values{"nodeId": {runID}}.Encode()
}
// validateSurface normalizes the requested sandbox. Empty defaults to desktop
// (the noVNC GUI); an unknown surface is a clean 400 rather than a launch the
// runtime can't honor.
func validateSurface(raw string) (string, error) {
switch strings.TrimSpace(strings.ToLower(raw)) {
case "", surfaceDesktop:
return surfaceDesktop, nil
case surfaceTerminal:
return surfaceTerminal, nil
default:
return "", zip.ErrBadRequest("surface must be 'desktop' or 'terminal'")
}
}
// validateTimeout bounds the optional wall-clock at the boundary: it must parse
// as a Go duration and be within (0, maxTimeout]. Empty means "runtime default".
func validateTimeout(raw string) error {
raw = strings.TrimSpace(raw)
if raw == "" {
return nil
}
d, err := time.ParseDuration(raw)
if err != nil {
return zip.ErrBadRequest("invalid 'timeout': " + err.Error())
}
if d <= 0 || d > maxTimeout {
return zip.ErrBadRequest("timeout must be > 0 and <= 24h")
}
return nil
}
// billingActor is the "org/sub" identity recorded on a debit for the audit
// trail. It never selects which balance is gated — that is always the org — but
// attributes the spend to the acting principal. Falls back to the bare org when
// no validated subject is present.
func billingActor(org, sub string) string {
sub = strings.TrimSpace(sub)
if org != "" && sub != "" {
return org + "/" + sub
}
if sub != "" {
return sub
}
return org
}
// gatewayBase resolves the browser-facing bot VNC gateway base (no trailing
// slash) from CLOUD_BOT_GATEWAY_URL, falling back to the public default.
// gatewayBase resolves the browser-facing bot VNC gateway base (no trailing slash)
// from CLOUD_BOT_GATEWAY_URL, falling back to the public default.
func gatewayBase() string {
base := strings.TrimSpace(os.Getenv(gatewayURLEnv))
if base == "" {
@@ -514,23 +256,3 @@ func gatewayBase() string {
}
return strings.TrimRight(base, "/")
}
// serverGatewayBase resolves the in-cluster bot-gateway base (no trailing slash)
// the control plane calls server-side for list/stop, from BOT_GATEWAY_URL — the
// SAME knob clients/bot's reverse proxy uses — falling back to the in-cluster
// service DNS default.
func serverGatewayBase() string {
base := strings.TrimSpace(os.Getenv(serverGatewayURLEnv))
if base == "" {
base = defaultServerGatewayURL
}
return strings.TrimRight(base, "/")
}
func genID(prefix string) (string, error) {
var b [16]byte
if _, err := rand.Read(b[:]); err != nil {
return "", err
}
return prefix + "_" + hex.EncodeToString(b[:]), nil
}
+315 -187
View File
@@ -1,94 +1,124 @@
package bots
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/runtime"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
// gwServer is a minimal bot-gateway double for the list/stop path. It records
// the X-Org-Id it was called with (a wrong tenant here would prove a cross-tenant
// leak) and serves configurable list/stop responses so the cloud proxy's
// normalization + honest-empty behavior can be asserted hermetically.
type gwServer struct {
bots []map[string]any // rows GET /v1/bots returns
listCode int // status for GET /v1/bots (0 => 200)
stopCode int // status for POST /v1/bots/{id}/stop (0 => 200)
// The control plane against a fake runtime. The fake is GENUINELY org-scoped (a
// run lives under exactly one org key), so a test that reads another tenant's run
// has to get past the same boundary the real runtime enforces by tenant path — a
// handler that forgot to pass the validated org, or passed a client-supplied one,
// fails here.
mu sync.Mutex
listOrg string // X-Org-Id seen on the last list
stopOrg string // X-Org-Id seen on the last stop
stopID string // runId parsed from the last stop path
type runKey struct{ org, id string }
type listCall struct{ org string }
type fakeRuntime struct {
mu sync.Mutex
rows map[runKey]Run
lists []listCall
stops []runKey
// Injected outcomes for the honest-failure paths.
listErr, stopErr error
}
func (g *gwServer) start(t *testing.T) string {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/v1/bots", func(w http.ResponseWriter, r *http.Request) {
g.mu.Lock()
g.listOrg = r.Header.Get("X-Org-Id")
g.mu.Unlock()
if g.listCode != 0 {
w.WriteHeader(g.listCode)
return
func newFake() *fakeRuntime { return &fakeRuntime{rows: map[runKey]Run{}} }
func (f *fakeRuntime) seed(org string, r Run) {
f.mu.Lock()
defer f.mu.Unlock()
f.rows[runKey{org, r.ID}] = r
}
func (f *fakeRuntime) List(_ context.Context, org string) ([]Run, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.lists = append(f.lists, listCall{org})
if f.listErr != nil {
return nil, f.listErr
}
var out []Run
for k, r := range f.rows {
if k.org == org {
out = append(out, r)
}
_ = json.NewEncoder(w).Encode(map[string]any{"bots": g.bots})
})
mux.HandleFunc("/v1/bots/", func(w http.ResponseWriter, r *http.Request) {
// path is /v1/bots/{id}/stop
id := strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/v1/bots/"), "/stop")
g.mu.Lock()
g.stopOrg, g.stopID = r.Header.Get("X-Org-Id"), id
g.mu.Unlock()
if g.stopCode != 0 {
w.WriteHeader(g.stopCode)
return
}
w.WriteHeader(http.StatusOK)
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return srv.URL
}
return out, nil
}
func (g *gwServer) seenListOrg() string {
g.mu.Lock()
defer g.mu.Unlock()
return g.listOrg
func (f *fakeRuntime) Stop(_ context.Context, org, runID string) error {
f.mu.Lock()
defer f.mu.Unlock()
f.stops = append(f.stops, runKey{org, runID})
if f.stopErr != nil {
return f.stopErr
}
k := runKey{org, runID}
if _, ok := f.rows[k]; !ok {
// The real runtime resolves under tenants/{org}/ and ANSWERS absent.
return runtime.ErrNotFound
}
delete(f.rows, k)
return nil
}
func (g *gwServer) seenStop() (string, string) {
g.mu.Lock()
defer g.mu.Unlock()
return g.stopOrg, g.stopID
func (f *fakeRuntime) stopCalls() []runKey {
f.mu.Lock()
defer f.mu.Unlock()
return append([]runKey(nil), f.stops...)
}
// mountGW mounts the bots surface with the server-side bot-gateway pinned to
// serverURL (empty => the in-cluster default, i.e. unreachable in tests). It
// reuses mount(t, "") so the browser-facing gateway base stays the deterministic
// https://bot.example.test that sessionUrl assertions depend on.
func mountGW(t *testing.T, serverURL string) *zip.App {
func (f *fakeRuntime) listCalls() []listCall {
f.mu.Lock()
defer f.mu.Unlock()
return append([]listCall(nil), f.lists...)
}
func (f *fakeRuntime) has(org, id string) bool {
f.mu.Lock()
defer f.mu.Unlock()
_, ok := f.rows[runKey{org, id}]
return ok
}
// mountWith builds the surface over an injected runtime, exactly as Mount does over
// the real one — routes() is the shared registration path, so what a test drives is
// the code that ships.
func mountWith(t *testing.T, rt Runtime) *zip.App {
t.Helper()
t.Setenv(serverGatewayURLEnv, serverURL)
return mount(t, "")
t.Setenv(gatewayURLEnv, "https://bot.example.test")
s := &cloud.Service[state]{
Base: cloud.NewBase(cloud.Deps{Logger: luxlog.New("test")}, "bots"),
State: state{gateway: gatewayBase(), runtime: rt},
}
app := zip.New(zip.Config{Logger: luxlog.New("test"), DisableStartupMessage: true})
routes(app, s)
return app
}
// getBots issues GET /v1/bots with the gateway-minted identity headers (org +
// its validated X-User-Id). An empty org sends neither (the anonymous path).
func getBots(t *testing.T, app *zip.App, org string) (int, []byte) {
// call sends a request with the gateway-minted identity headers. A non-empty org
// also sets X-User-Id (the validated principal); an empty org sends neither.
func call(t *testing.T, app *zip.App, method, path, org string) (int, []byte) {
t.Helper()
req := httptest.NewRequest(http.MethodGet, "/v1/bots", nil)
req := httptest.NewRequest(method, path, nil)
if org != "" {
req.Header.Set("X-Org-Id", org)
req.Header.Set("X-User-Id", "u-"+org)
}
resp, err := app.Fiber().Test(req)
resp, err := app.Fiber().Test(req, testCfg)
if err != nil {
t.Fatalf("Test: %v", err)
}
@@ -97,176 +127,274 @@ func getBots(t *testing.T, app *zip.App, org string) (int, []byte) {
return resp.StatusCode, b
}
// stopBotReq issues POST /v1/bots/{runID}/stop with identity headers.
func stopBotReq(t *testing.T, app *zip.App, org, runID string) (int, []byte) {
func listRunIDs(t *testing.T, body []byte) []string {
t.Helper()
req := httptest.NewRequest(http.MethodPost, "/v1/bots/"+runID+"/stop", nil)
if org != "" {
req.Header.Set("X-Org-Id", org)
req.Header.Set("X-User-Id", "u-"+org)
var v botsView
if err := json.Unmarshal(body, &v); err != nil {
t.Fatalf("decode list: %v (%s)", err, body)
}
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("Test: %v", err)
ids := make([]string, 0, len(v.Bots))
for _, b := range v.Bots {
ids = append(ids, b.RunID)
}
defer func() { _ = resp.Body.Close() }()
b, _ := io.ReadAll(resp.Body)
return resp.StatusCode, b
return ids
}
// TestListRequiresOrg: no X-Org-Id is 403 — no tenant, no list.
func TestListRequiresOrg(t *testing.T) {
app := mountGW(t, "http://127.0.0.1:1")
if code, _ := getBots(t, app, ""); code != http.StatusForbidden {
t.Fatalf("no-org list want 403, got %d", code)
// ---- run ----
// Launching is not implemented and must say so rather than charge for a bot that
// never boots. The runtime has no launch operation, so a 200 here would be a lie
// with a price on it.
func TestRunIsNotImplementedAndStartsNothing(t *testing.T) {
rt := newFake()
app := mountWith(t, rt)
if code, _ := call(t, app, http.MethodPost, "/v1/bots/run", "acme"); code != http.StatusNotImplemented {
t.Fatalf("launch want 501, got %d", code)
}
// Nothing was started, so nothing may be listed as started.
_, body := call(t, app, http.MethodGet, "/v1/bots", "acme")
if ids := listRunIDs(t, body); len(ids) != 0 {
t.Fatalf("a refused launch must not produce a run, got %v", ids)
}
}
// TestListRequiresValidatedPrincipal: a bare, forgeable X-Org-Id (no validated
// X-User-Id) is 403 — an unvalidated caller cannot enumerate a victim tenant.
func TestListRequiresValidatedPrincipal(t *testing.T) {
app := mountGW(t, "http://127.0.0.1:1")
req := httptest.NewRequest(http.MethodGet, "/v1/bots", nil)
req.Header.Set("X-Org-Id", "acme") // forged; no X-User-Id => no validated principal
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("Test: %v", err)
}
_ = resp.Body.Close()
if resp.StatusCode != http.StatusForbidden {
t.Fatalf("no-principal list want 403, got %d", resp.StatusCode)
}
}
// ---- list ----
// TestListNormalizesAndScopesToCaller: the gateway's session rows are normalized
// into the contract (sessionUrl derived control-plane side from runId), and the
// gateway is called scoped to the CALLER org (acme), never the client default.
func TestListNormalizesAndScopesToCaller(t *testing.T) {
gw := &gwServer{bots: []map[string]any{
{"runId": "bot_abc", "task": "summarize inbox", "surface": "desktop", "status": "running", "startedAt": "2026-07-11T00:00:00Z"},
{"runId": "bot_def", "task": "book flight", "surface": "terminal", "status": "running", "startedAt": "2026-07-11T01:00:00Z"},
}}
app := mountGW(t, gw.start(t))
// A list returns the CALLER's runs and only those, scoped by the validated org the
// runtime is asked with — never a client-supplied one.
func TestListIsScopedToTheCallerOrg(t *testing.T) {
rt := newFake()
rt.seed("acme", Run{ID: "run_acme", Task: "acme work", Surface: "desktop", Status: "running", StartedAt: "2023-11-14T22:13:20Z"})
rt.seed("globex", Run{ID: "run_globex", Task: "globex secret", Surface: "terminal", Status: "running", StartedAt: "2023-11-14T22:13:20Z"})
app := mountWith(t, rt)
code, body := getBots(t, app, "acme")
code, body := call(t, app, http.MethodGet, "/v1/bots", "acme")
if code != http.StatusOK {
t.Fatalf("list want 200, got %d (%s)", code, body)
}
var got botsView
if err := json.Unmarshal(body, &got); err != nil {
t.Fatalf("shape: %v (%s)", err, body)
if ids := listRunIDs(t, body); len(ids) != 1 || ids[0] != "run_acme" {
t.Fatalf("acme must see exactly its own run, got %v (%s)", ids, body)
}
if len(got.Bots) != 2 {
t.Fatalf("want 2 bots, got %d (%s)", len(got.Bots), body)
}
first := got.Bots[0]
if first.RunID != "bot_abc" || first.Task != "summarize inbox" || first.Surface != "desktop" {
t.Fatalf("row not normalized: %+v", first)
}
if first.Status != statusRunning {
t.Fatalf("status want %q, got %q", statusRunning, first.Status)
}
if first.StartedAt != "2026-07-11T00:00:00Z" {
t.Fatalf("startedAt passthrough want set, got %q", first.StartedAt)
}
// sessionUrl is derived here from the browser-facing base + nodeId=runId, NOT
// taken from the gateway — the ONE place a session URL is built.
want := "https://bot.example.test/vnc?nodeId=bot_abc"
if first.SessionURL != want {
t.Fatalf("sessionUrl want %q, got %q", want, first.SessionURL)
}
if org := gw.seenListOrg(); org != "acme" {
t.Fatalf("gateway saw X-Org-Id=%q, want caller %q (never default 'hanzo')", org, "acme")
// The runtime was asked with the caller's validated org, nothing else.
if got := rt.listCalls(); len(got) != 1 || got[0].org != "acme" {
t.Fatalf("runtime must be asked with the validated org only, got %+v", got)
}
}
// TestListHonestEmptyWhenGatewayUnreachable: an unreachable gateway yields a 200
// {"bots":[]}, never a 5xx — the console renders "no bots", not an error.
func TestListHonestEmptyWhenGatewayUnreachable(t *testing.T) {
app := mountGW(t, "http://127.0.0.1:1") // connection refused
code, body := getBots(t, app, "acme")
if code != http.StatusOK {
t.Fatalf("unreachable list want 200, got %d (%s)", code, body)
// The row carries the run's real attributes as the runtime reported them, plus a
// sessionUrl derived here from the runtime's own id.
func TestListRowShape(t *testing.T) {
rt := newFake()
rt.seed("acme", Run{ID: "run_1", Task: "ship it", Surface: "terminal", Status: "running", StartedAt: "2023-11-14T22:13:20Z"})
app := mountWith(t, rt)
_, body := call(t, app, http.MethodGet, "/v1/bots", "acme")
var v botsView
if err := json.Unmarshal(body, &v); err != nil {
t.Fatalf("decode: %v", err)
}
if len(v.Bots) != 1 {
t.Fatalf("want 1 row, got %d", len(v.Bots))
}
want := botView{
RunID: "run_1", Task: "ship it", Surface: "terminal", Status: "running",
SessionURL: "https://bot.example.test/vnc?nodeId=run_1",
StartedAt: "2023-11-14T22:13:20Z",
}
if v.Bots[0] != want {
t.Fatalf("row\n got %+v\nwant %+v", v.Bots[0], want)
}
assertEmptyBots(t, body)
}
// TestListHonestEmptyOnGatewayError: a non-2xx from the gateway is also honest-
// empty, not a propagated 5xx.
func TestListHonestEmptyOnGatewayError(t *testing.T) {
gw := &gwServer{listCode: http.StatusInternalServerError}
app := mountGW(t, gw.start(t))
code, body := getBots(t, app, "acme")
if code != http.StatusOK {
t.Fatalf("gateway-500 list want 200, got %d (%s)", code, body)
// No tenant, no read — and the runtime is never even asked.
func TestListFailsClosedWithoutTenant(t *testing.T) {
rt := newFake()
rt.seed("acme", Run{ID: "run_acme", Status: "running"})
app := mountWith(t, rt)
if code, _ := call(t, app, http.MethodGet, "/v1/bots", ""); code != http.StatusForbidden {
t.Fatalf("no-tenant list want 403, got %d", code)
}
if got := rt.listCalls(); len(got) != 0 {
t.Fatalf("runtime asked without a tenant: %+v", got)
}
assertEmptyBots(t, body)
}
// TestStopStopsCallerRun: a stop the gateway accepts returns {runId,"stopped"}
// and forwards the CALLER org + the exact runId to the gateway.
func TestStopStopsCallerRun(t *testing.T) {
gw := &gwServer{}
app := mountGW(t, gw.start(t))
// A forged X-Org-Id with no validated principal (the direct-to-pod path) must not
// enumerate the victim tenant: principal.Org requires a validated X-User-Id.
func TestListRefusesForgedOrgWithoutValidatedPrincipal(t *testing.T) {
rt := newFake()
rt.seed("acme", Run{ID: "run_acme", Task: "secret", Status: "running"})
app := mountWith(t, rt)
code, body := stopBotReq(t, app, "acme", "bot_abc")
req := httptest.NewRequest(http.MethodGet, "/v1/bots", nil)
req.Header.Set("X-Org-Id", "acme") // forged: no X-User-Id
resp, err := app.Fiber().Test(req, testCfg)
if err != nil {
t.Fatalf("Test: %v", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusForbidden {
t.Fatalf("forged-org list want 403, got %d", resp.StatusCode)
}
if got := rt.listCalls(); len(got) != 0 {
t.Fatalf("runtime asked on a forged org: %+v", got)
}
}
// A runtime that cannot answer is an error, never an empty list: [] would claim
// "your org has no runs", which is not what we learned.
func TestListReportsRuntimeFailureInsteadOfEmpty(t *testing.T) {
rt := newFake()
rt.listErr = fmt.Errorf("runtime is down")
app := mountWith(t, rt)
code, body := call(t, app, http.MethodGet, "/v1/bots", "acme")
if code != http.StatusBadGateway {
t.Fatalf("runtime failure want 502, got %d (%s)", code, body)
}
}
// ---- stop ----
// One tenant may not stop another's run. The id is real and the caller is
// validated — and it is still a 404, because the runtime is asked under the
// CALLER's org, where that run does not exist.
func TestStopCannotReachAnotherOrgsRun(t *testing.T) {
rt := newFake()
rt.seed("acme", Run{ID: "run_victim", Task: "acme work", Status: "running"})
app := mountWith(t, rt)
code, body := call(t, app, http.MethodPost, "/v1/bots/run_victim/stop", "globex")
if code != http.StatusNotFound {
t.Fatalf("cross-org stop want 404, got %d (%s)", code, body)
}
// The runtime was asked under globex — never under the victim's org.
for _, s := range rt.stopCalls() {
if s.org != "globex" {
t.Fatalf("stop escaped the caller's org: %+v", s)
}
}
if !rt.has("acme", "run_victim") {
t.Fatal("victim run must survive a foreign stop")
}
}
// An unknown id and another tenant's id are indistinguishable: both 404, so the
// endpoint is not an oracle for which run ids exist.
func TestStopUnknownRunIsNotFound(t *testing.T) {
app := mountWith(t, newFake())
if code, _ := call(t, app, http.MethodPost, "/v1/bots/run_nope/stop", "acme"); code != http.StatusNotFound {
t.Fatalf("unknown stop want 404, got %d", code)
}
}
// The happy path: the runtime is driven with the caller's own org and run id, and
// the run is gone from its list.
func TestStopHaltsTheRun(t *testing.T) {
rt := newFake()
rt.seed("acme", Run{ID: "run_1", Task: "work", Status: "running"})
app := mountWith(t, rt)
code, body := call(t, app, http.MethodPost, "/v1/bots/run_1/stop", "acme")
if code != http.StatusOK {
t.Fatalf("stop want 200, got %d (%s)", code, body)
}
var got stopView
if err := json.Unmarshal(body, &got); err != nil {
t.Fatalf("shape: %v (%s)", err, body)
var v stopView
if err := json.Unmarshal(body, &v); err != nil {
t.Fatalf("decode: %v", err)
}
if got.RunID != "bot_abc" || got.Status != statusStopped {
t.Fatalf("stop response want {bot_abc,stopped}, got %+v", got)
if v.RunID != "run_1" || v.Status != statusStopped {
t.Fatalf("stop view %+v", v)
}
org, id := gw.seenStop()
if org != "acme" {
t.Fatalf("gateway saw stop X-Org-Id=%q, want caller %q", org, "acme")
if got := rt.stopCalls(); len(got) != 1 || got[0] != (runKey{"acme", "run_1"}) {
t.Fatalf("runtime must be driven once with the caller's org+run, got %v", got)
}
if id != "bot_abc" {
t.Fatalf("gateway saw stop runId=%q, want %q", id, "bot_abc")
if _, body := call(t, app, http.MethodGet, "/v1/bots", "acme"); len(listRunIDs(t, body)) != 0 {
t.Fatal("a stopped run must leave the list")
}
}
// TestStopNotFound: a run the caller's org does not own (gateway 404) is a 404,
// never a 200 — stop cannot claim a teardown that did not happen.
func TestStopNotFound(t *testing.T) {
gw := &gwServer{stopCode: http.StatusNotFound}
app := mountGW(t, gw.start(t))
if code, body := stopBotReq(t, app, "acme", "bot_nope"); code != http.StatusNotFound {
t.Fatalf("stop of unowned run want 404, got %d (%s)", code, body)
// THE correctness lie this endpoint must never tell: a runtime that does not serve
// stop reports nothing about the run, so claiming "stopped" would make a stop that
// cannot fail. It is a 502, and it says the run was NOT stopped.
func TestStopFailsClosedWhenTheRuntimeDoesNotServeStop(t *testing.T) {
rt := newFake()
rt.seed("acme", Run{ID: "run_1", Status: "running"})
rt.stopErr = runtime.ErrNotServed
app := mountWith(t, rt)
code, body := call(t, app, http.MethodPost, "/v1/bots/run_1/stop", "acme")
if code != http.StatusBadGateway {
t.Fatalf("unserved stop want 502, got %d (%s)", code, body)
}
if !rt.has("acme", "run_1") {
t.Fatal("the run must not be treated as gone when the runtime never answered")
}
}
// TestStopUnreachableIs502: a stop that cannot reach the gateway is a clean 502,
// not a false "stopped".
func TestStopUnreachableIs502(t *testing.T) {
app := mountGW(t, "http://127.0.0.1:1")
if code, _ := stopBotReq(t, app, "acme", "bot_abc"); code != http.StatusBadGateway {
t.Fatalf("stop with unreachable gateway want 502, got %d", code)
// A stop that could not reach the executor must not claim the run was stopped.
func TestStopWithUnreachableRuntimeIs502(t *testing.T) {
rt := newFake()
rt.seed("acme", Run{ID: "run_1", Status: "running"})
rt.stopErr = fmt.Errorf("connection refused")
app := mountWith(t, rt)
if code, _ := call(t, app, http.MethodPost, "/v1/bots/run_1/stop", "acme"); code != http.StatusBadGateway {
t.Fatalf("unreachable runtime want 502, got %d", code)
}
if !rt.has("acme", "run_1") {
t.Fatal("the run must stay live when the halt failed")
}
}
// TestStopRequiresOrg: no X-Org-Id is 403.
func TestStopRequiresOrg(t *testing.T) {
app := mountGW(t, "http://127.0.0.1:1")
if code, _ := stopBotReq(t, app, "", "bot_abc"); code != http.StatusForbidden {
t.Fatalf("no-org stop want 403, got %d", code)
// No tenant, no stop — and the runtime is never driven.
func TestStopFailsClosedWithoutTenant(t *testing.T) {
rt := newFake()
rt.seed("acme", Run{ID: "run_1", Status: "running"})
app := mountWith(t, rt)
if code, _ := call(t, app, http.MethodPost, "/v1/bots/run_1/stop", ""); code != http.StatusForbidden {
t.Fatalf("no-tenant stop want 403, got %d", code)
}
if len(rt.stopCalls()) != 0 {
t.Fatalf("runtime driven without a tenant: %v", rt.stopCalls())
}
if !rt.has("acme", "run_1") {
t.Fatal("run stopped without a tenant")
}
}
func assertEmptyBots(t *testing.T, body []byte) {
t.Helper()
// Must be an explicit empty array, not null — {"bots":[]}.
if !strings.Contains(string(body), `"bots":[]`) {
t.Fatalf("want honest-empty {\"bots\":[]}, got %s", body)
// An oversize id is a miss like any other — it never reaches the runtime.
func TestStopOversizeRunIDIsNotFound(t *testing.T) {
rt := newFake()
app := mountWith(t, rt)
long := make([]byte, maxRunID+1)
for i := range long {
long[i] = 'a'
}
var got botsView
if err := json.Unmarshal(body, &got); err != nil {
t.Fatalf("shape: %v (%s)", err, body)
if code, _ := call(t, app, http.MethodPost, "/v1/bots/"+string(long)+"/stop", "acme"); code != http.StatusNotFound {
t.Fatalf("oversize runId want 404, got %d", code)
}
if len(got.Bots) != 0 {
t.Fatalf("want 0 bots, got %d", len(got.Bots))
if len(rt.stopCalls()) != 0 {
t.Fatalf("runtime driven for an oversize id: %v", rt.stopCalls())
}
}
// /v1/bots/run is the launch literal, never a run id: the router resolves the
// static segment over the :runId param regardless of registration order.
func TestRunLiteralDoesNotBindAsARunID(t *testing.T) {
rt := newFake()
app := mountWith(t, rt)
// POST /v1/bots/run reaches the launch handler (501), NOT the stop handler
// (which would 404 a run named "run" and would have driven the runtime).
if code, _ := call(t, app, http.MethodPost, "/v1/bots/run", "acme"); code != http.StatusNotImplemented {
t.Fatalf("POST /v1/bots/run must hit launch (501), got %d", code)
}
if len(rt.stopCalls()) != 0 {
t.Fatalf("/v1/bots/run bound as a run id and drove a stop: %v", rt.stopCalls())
}
}
-243
View File
@@ -1,243 +0,0 @@
package bots
import (
"bytes"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/metering"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
// billServer is a minimal commerce double: it returns a fixed balance and
// records the X-Org-Id + usage body of every debit — the SAME double the agents
// billing tests use. A wrong tenant here would prove a cross-tenant leak.
type billServer struct {
available int64
mu sync.Mutex
usageOrg string
usageBody []byte
usages int32
}
func (b *billServer) start(t *testing.T) string {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/v1/billing/balance", func(w http.ResponseWriter, r *http.Request) {
_ = json.NewEncoder(w).Encode(map[string]any{"available": b.available})
})
mux.HandleFunc("/v1/billing/usage", func(w http.ResponseWriter, r *http.Request) {
atomic.AddInt32(&b.usages, 1)
body, _ := io.ReadAll(r.Body)
b.mu.Lock()
b.usageOrg, b.usageBody = r.Header.Get("X-Org-Id"), body
b.mu.Unlock()
w.WriteHeader(http.StatusOK)
_, _ = io.WriteString(w, `{"transactionId":"tx_1","type":"usage"}`)
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return srv.URL
}
func (b *billServer) debits() int32 { return atomic.LoadInt32(&b.usages) }
func (b *billServer) lastDebit() (string, []byte) {
b.mu.Lock()
defer b.mu.Unlock()
return b.usageOrg, b.usageBody
}
// waitForDebit polls briefly — debits land on a detached goroutine.
func waitForDebit(cond func() bool) bool {
for i := 0; i < 200; i++ {
if cond() {
return true
}
time.Sleep(5 * time.Millisecond)
}
return cond()
}
// mount builds the bots surface. commerceURL=="" mounts with no billing
// (Gate allows, Meter is a no-op — the unconfigured deployment path).
func mount(t *testing.T, commerceURL string) *zip.App {
t.Helper()
// Pin a deterministic gateway base so the session URL is assertable. Set
// BEFORE Mount, which snapshots gatewayBase() once.
t.Setenv(gatewayURLEnv, "https://bot.example.test")
deps := cloud.Deps{Logger: luxlog.New("test")}
if commerceURL != "" {
// Default org "hanzo" so every "acme is billed" assertion proves the
// per-call org override scopes the ledger to the CALLER, not the default.
m, err := metering.New(metering.Config{BaseURL: commerceURL, Token: "svc-tok", Org: "hanzo"})
if err != nil {
t.Fatalf("metering.New: %v", err)
}
deps.Metering = m
}
app := zip.New(zip.Config{Logger: luxlog.New("test")})
if err := Mount(app, deps); err != nil {
t.Fatalf("Mount: %v", err)
}
return app
}
// do sends a request with the gateway-minted identity headers. A non-empty org
// also sets X-User-Id (the validated principal the money path requires); an
// empty org sends neither (the anonymous path).
func do(t *testing.T, app *zip.App, org string, body any) (int, []byte) {
t.Helper()
var r io.Reader
if body != nil {
b, _ := json.Marshal(body)
r = bytes.NewReader(b)
}
req := httptest.NewRequest(http.MethodPost, "/v1/bots/run", r)
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
if org != "" {
req.Header.Set("X-Org-Id", org)
req.Header.Set("X-User-Id", "u-"+org)
}
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("Test: %v", err)
}
defer func() { _ = resp.Body.Close() }()
b, _ := io.ReadAll(resp.Body)
return resp.StatusCode, b
}
// TestRunRequiresOrg: a launch with no X-Org-Id is 403 — no tenant, no bot.
func TestRunRequiresOrg(t *testing.T) {
app := mount(t, "")
if code, _ := do(t, app, "", map[string]any{"task": "do a thing"}); code != http.StatusForbidden {
t.Fatalf("no-org launch want 403, got %d", code)
}
}
// TestRunRequiresValidatedPrincipal: a launch carrying only a forgeable X-Org-Id
// (no validated X-User-Id — the direct-to-pod path) is 403. A money-moving
// launch can't ride an unauthenticated org header.
func TestRunRequiresValidatedPrincipal(t *testing.T) {
app := mount(t, "")
req := httptest.NewRequest(http.MethodPost, "/v1/bots/run",
bytes.NewReader([]byte(`{"task":"x"}`)))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("X-Org-Id", "acme") // forged; no X-User-Id → no validated principal
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("Test: %v", err)
}
_ = resp.Body.Close()
if resp.StatusCode != http.StatusForbidden {
t.Fatalf("no-principal launch want 403, got %d", resp.StatusCode)
}
}
// TestRunValidatesInput: an empty task is 400; an unknown surface is 400.
func TestRunValidatesInput(t *testing.T) {
app := mount(t, "")
if code, _ := do(t, app, "acme", map[string]any{"task": " "}); code != http.StatusBadRequest {
t.Fatalf("empty task want 400, got %d", code)
}
if code, _ := do(t, app, "acme", map[string]any{"task": "x", "surface": "hologram"}); code != http.StatusBadRequest {
t.Fatalf("unknown surface want 400, got %d", code)
}
if code, _ := do(t, app, "acme", map[string]any{"task": "x", "timeout": "banana"}); code != http.StatusBadRequest {
t.Fatalf("bad timeout want 400, got %d", code)
}
}
// TestRunResponseShapeAndSessionURL: a launch (billing unconfigured → un-gated)
// returns {runId, status, sessionUrl}, with the session URL derived from the
// configured gateway base + nodeId=<runId>.
func TestRunResponseShapeAndSessionURL(t *testing.T) {
app := mount(t, "")
code, body := do(t, app, "acme", map[string]any{"task": "summarize my inbox", "surface": "desktop"})
if code != http.StatusOK {
t.Fatalf("launch want 200, got %d (%s)", code, body)
}
var rv runView
if err := json.Unmarshal(body, &rv); err != nil {
t.Fatalf("shape: %v (%s)", err, body)
}
if !strings.HasPrefix(rv.RunID, "bot_") {
t.Fatalf("runId want bot_ prefix, got %q", rv.RunID)
}
if rv.Status != statusRunning {
t.Fatalf("status want %q, got %q", statusRunning, rv.Status)
}
want := "https://bot.example.test/vnc?nodeId=" + rv.RunID
if rv.SessionURL != want {
t.Fatalf("sessionUrl want %q, got %q", want, rv.SessionURL)
}
}
// TestRunGatesUnfundedOrg: a launch for an org with a balance below the flat fee
// is refused 402 and NEVER debits — the gate is called and fails closed, so an
// unfunded tenant gets no bot.
func TestRunGatesUnfundedOrg(t *testing.T) {
bs := &billServer{available: 0}
app := mount(t, bs.start(t))
code, body := do(t, app, "acme", map[string]any{"task": "x"})
if code != http.StatusPaymentRequired {
t.Fatalf("unfunded launch want 402, got %d (%s)", code, body)
}
if bs.debits() != 0 {
t.Fatalf("a gate-refused launch must not debit, got %d", bs.debits())
}
}
// TestRunDebitsCallerOrg: a funded launch returns the session AND debits the
// CALLER org (acme, never the client default 'hanzo') with product=bot, the flat
// fee, and the surface as the billed unit.
func TestRunDebitsCallerOrg(t *testing.T) {
bs := &billServer{available: 100000}
app := mount(t, bs.start(t))
code, body := do(t, app, "acme", map[string]any{"task": "x", "surface": "terminal"})
if code != http.StatusOK {
t.Fatalf("funded launch want 200, got %d (%s)", code, body)
}
if !waitForDebit(func() bool { return bs.debits() == 1 }) {
t.Fatalf("a launch must debit once, got %d", bs.debits())
}
org, ubody := bs.lastDebit()
if org != "acme" {
t.Fatalf("debited org %q, want caller %q (never default 'hanzo')", org, "acme")
}
var u struct {
User string `json:"user"`
Amount int64 `json:"amount"`
Model string `json:"model"`
Provider string `json:"provider"`
Actor string `json:"actor"`
}
_ = json.Unmarshal(ubody, &u)
if u.User != "acme" {
t.Fatalf("debit user = %q, want caller org %q", u.User, "acme")
}
if u.Amount != cloud.DefaultResourceFeeCents {
t.Fatalf("debit amount = %d, want default fee %d", u.Amount, cloud.DefaultResourceFeeCents)
}
if u.Provider != meterKind {
t.Fatalf("debit provider = %q, want %q (product:bot)", u.Provider, meterKind)
}
if u.Model != surfaceTerminal {
t.Fatalf("debit model = %q, want the surface %q", u.Model, surfaceTerminal)
}
if u.Actor == "" {
t.Fatalf("debit must carry an actor for the audit trail")
}
}
+232
View File
@@ -0,0 +1,232 @@
package bots
import (
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/runtime"
"github.com/hanzoai/cloud/clients/visor"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
// Regression guard for the /v1/bots route collision.
//
// clients/visor once registered GET /v1/bots for its bot MACHINES and this
// package registered GET /v1/bots for bot RUNS. The router resolves byte-identical
// patterns by first-registration — silently, with no panic — and visor mounts
// first (apps.Wire: visor, then runtime, then bots), so visor's machine list answered
// the console's run list and this package's handler was unreachable. Two values
// sharing one name, one namespace.
//
// These tests pin the fix from both ends: structurally (no two subsystems may
// register the same method+path) and behaviourally (GET /v1/bots serves RUNS),
// with the subsystems mounted in the real Wire order that produced the bug.
// stubRuntime stands in for the bot runtime, serving its documented contract and
// recording the paths + org it was asked with. Real Mounts talk to it through the
// real transport, so these tests exercise the shipping path end to end.
type stubRuntime struct {
mu sync.Mutex
runs map[string][]map[string]any // org -> rows
paths []string
orgs []string
}
func (s *stubRuntime) start(t *testing.T) {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/v1/bots", func(w http.ResponseWriter, r *http.Request) {
org := r.Header.Get("X-Org-Id")
s.mu.Lock()
s.paths, s.orgs = append(s.paths, r.URL.Path), append(s.orgs, org)
rows := s.runs[org]
s.mu.Unlock()
if rows == nil {
rows = []map[string]any{}
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]any{"bots": rows})
})
mux.HandleFunc("/v1/bots/", func(w http.ResponseWriter, r *http.Request) { // {id}/stop
s.mu.Lock()
s.paths, s.orgs = append(s.paths, r.URL.Path), append(s.orgs, r.Header.Get("X-Org-Id"))
s.mu.Unlock()
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte(`{"status":"stopped"}`))
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
t.Setenv("BOT_GATEWAY_URL", srv.URL)
}
func (s *stubRuntime) seen() []string {
s.mu.Lock()
defer s.mu.Unlock()
return append([]string(nil), s.paths...)
}
// mountFleet mounts, in apps.Wire order, the three subsystems that shared the
// /v1/bot* namespace, over a stub runtime.
func mountFleet(t *testing.T, rt *stubRuntime) *zip.App {
t.Helper()
t.Setenv(gatewayURLEnv, "https://bot.example.test")
rt.start(t)
app := zip.New(zip.Config{Logger: luxlog.New("test"), DisableStartupMessage: true})
deps := cloud.Deps{Logger: luxlog.New("test"), DataDir: t.TempDir()}
if err := visor.Mount(app, deps); err != nil { // Wire order: visor first — the shadowing mount
t.Fatalf("visor.Mount: %v", err)
}
if err := runtime.Mount(app, deps); err != nil {
t.Fatalf("runtime.Mount: %v", err)
}
if err := Mount(app, deps); err != nil { // …bots last, as in Wire
t.Fatalf("bots.Mount: %v", err)
}
return app
}
// collisions reports every route claimed by more than one registration.
//
// Counting GetRoutes() entries is NOT enough and looking only at that is how this
// guard was blind at first: the router MERGES byte-identical patterns into ONE
// Route carrying both handlers chained, so the second registration leaves the
// entry count at one and shows up only as a handler count above one. Every real
// route in the fleet registers exactly one handler, so >1 is the collision. The
// entry count is kept as well, for any overlap the router does not merge.
func collisions(app *zip.App) []string {
var out []string
seen := map[string]int{}
for _, r := range app.Fiber().GetRoutes() {
key := r.Method + " " + r.Path
if n := len(r.Handlers); n > 1 {
out = append(out, fmt.Sprintf("%s — %d handlers chained on one route", key, n))
continue
}
if seen[key]++; seen[key] > 1 {
out = append(out, fmt.Sprintf("%s — %d separate registrations", key, seen[key]))
}
}
return out
}
// The guard itself must be shown to FIRE on the bug, or it is decoration. This
// reproduces the original collision — two subsystems, one pattern — and asserts
// collisions() sees it, through the SAME function the real check below uses.
func TestDuplicateRouteGuardDetectsACollision(t *testing.T) {
app := zip.New(zip.Config{Logger: luxlog.New("test"), DisableStartupMessage: true})
app.Get("/v1/bots", func(c *zip.Ctx) error { return c.JSON(200, map[string]any{"bots": []string{"machine"}}) })
app.Get("/v1/bots", func(c *zip.Ctx) error { return c.JSON(200, map[string]any{"bots": []string{"run"}}) })
got := collisions(app)
if len(got) == 0 {
t.Fatal("the guard cannot see the very collision it guards: two GET /v1/bots handlers are registered and collisions() reported none")
}
if !strings.Contains(got[0], "GET /v1/bots") {
t.Fatalf("guard named the wrong route: %v", got)
}
}
// No two subsystems may claim the same method+path. The router does not panic on
// a byte-identical duplicate — it silently merges and keeps the first — so
// nothing but this catches the class of bug.
func TestSubsystemsDoNotRegisterDuplicateRoutes(t *testing.T) {
for _, c := range collisions(mountFleet(t, &stubRuntime{})) {
t.Errorf("route collision: %s — one handler silently shadows the other", c)
}
}
// The behavioural half: GET /v1/bots serves bot RUNS from the runtime. The run the
// runtime holds must come back out — which it cannot do if visor's machine list is
// answering this route, since a machine list never asks the runtime anything.
func TestGetBotsServesRunsNotMachines(t *testing.T) {
rt := &stubRuntime{runs: map[string][]map[string]any{
"acme": {{"runId": "run_1", "task": "prove the route", "surface": "desktop", "status": "running", "startedAt": "2023-11-14T22:13:20Z"}},
}}
app := mountFleet(t, rt)
code, body := call(t, app, http.MethodGet, "/v1/bots", "acme")
if code != http.StatusOK {
t.Fatalf("GET /v1/bots want 200, got %d (%s)", code, body)
}
var v botsView
if err := json.Unmarshal(body, &v); err != nil {
t.Fatalf("decode: %v (%s)", err, body)
}
if len(v.Bots) != 1 || v.Bots[0].RunID != "run_1" {
t.Fatalf("GET /v1/bots must serve the org's runs, got %s", body)
}
// A run row, not a machine row: the fields the console reads are populated.
if v.Bots[0].Task != "prove the route" || v.Bots[0].SessionURL != "https://bot.example.test/vnc?nodeId=run_1" {
t.Fatalf("row is not a run: %+v", v.Bots[0])
}
// It reached the RUNTIME — the only place a run has ever existed.
if got := rt.seen(); len(got) != 1 || got[0] != "/v1/bots" {
t.Fatalf("bots must read the runtime's run list, saw %v", got)
}
}
// The bot MACHINE surface still exists — under visor's own namespace, where a
// machine belongs. It is reachable and it is NOT /v1/bots.
func TestBotMachineSurfaceMovedToCompute(t *testing.T) {
app := mountFleet(t, &stubRuntime{})
paths := map[string]bool{}
for _, r := range app.Fiber().GetRoutes() {
paths[r.Method+" "+r.Path] = true
}
for _, want := range []string{
"GET /v1/compute/bots",
"POST /v1/compute/bots/launch",
"GET /v1/compute/bots/:id",
"DELETE /v1/compute/bots/:id",
"POST /v1/compute/bots/:id/:action",
} {
if !paths[want] {
t.Errorf("bot machine route %q is missing", want)
}
}
// …and nothing but the run control plane claims /v1/bots.
for _, gone := range []string{"GET /v1/bots/:id", "DELETE /v1/bots/:id", "POST /v1/bots/launch", "POST /v1/bots/:id/:action"} {
if paths[gone] {
t.Errorf("machine route %q still squats on the run namespace", gone)
}
}
}
// The three values keep three namespaces: runs at /v1/bots, machines under
// /v1/compute/bots, and the runtime passthrough at /v1/bot/*. Nothing in the run
// namespace may be a wildcard, which would swallow every run id.
func TestRunNamespaceHasNoWildcard(t *testing.T) {
app := mountFleet(t, &stubRuntime{})
for _, r := range app.Fiber().GetRoutes() {
if r.Path == "/v1/bots/*" {
t.Fatalf("%s /v1/bots/* would swallow every run id", r.Method)
}
}
}
// The runtime ops face is a SIBLING namespace, not a parent: /v1/bot/* must never
// match a /v1/bots path, or the control plane would be relayed away.
//
// The discriminator is the op the runtime is asked for. bots' stub addresses
// /v1/bots/{id}/stop; the ops face strips its own /v1/bot prefix and would ask for
// something else entirely. So the path the runtime SAW names which handler ran.
func TestRuntimeOpsFaceDoesNotSwallowTheRunNamespace(t *testing.T) {
rt := &stubRuntime{}
app := mountFleet(t, rt)
if code, body := call(t, app, http.MethodPost, "/v1/bots/run_1/stop", "acme"); code != http.StatusOK {
t.Fatalf("stop want 200, got %d (%s)", code, body)
}
if got := rt.seen(); len(got) != 1 || got[0] != "/v1/bots/run_1/stop" {
t.Fatalf("the run namespace must be served by bots' own stub, runtime saw %v", got)
}
}
+16
View File
@@ -0,0 +1,16 @@
package bots
import (
"time"
"github.com/zap-proto/fiber/v3"
)
// testCfg replaces fiber's Test() default of Timeout: 1s (fiber/v3@v3.2.1
// app.go:1202, FailOnTimeout: true). That default is a WALL-CLOCK deadline on an
// in-process request: under load — a full suite, encrypted SQLite, a busy box — a
// correct handler blows it and the test reports "i/o timeout". A guard that fails
// for reasons unrelated to what it guards teaches nothing, and a tenant-isolation
// guard that is a coin flip is worse than none. The generous bound still fails a
// genuine hang.
var testCfg = fiber.TestConfig{Timeout: 30 * time.Second, FailOnTimeout: true}
+68
View File
@@ -0,0 +1,68 @@
package bots
import (
"context"
"net/url"
"github.com/hanzoai/cloud/clients/runtime"
)
// wire.go is bots' WIRE CONTRACT with the bot runtime — the stub behind the
// Runtime seam. It says WHAT bots asks the runtime (list this org's runs, halt one
// of them); how the bytes get there is runtime's problem.
//
// This file is the ONE place in clients/bots that knows the runtime exists. The
// handlers (bots.go) never see it: they hold the Runtime seam, which a test fills
// with a fake.
//
// The id space is the RUNTIME'S. A run id here is whatever the runtime named its
// session — cloud does not mint it, does not store it, and could not resolve it if
// it tried. That is exactly why list and stop agree: both speak the only id space
// that has ever held a real run.
const listOp = "/v1/bots"
func stopOp(runID string) string { return "/v1/bots/" + url.PathEscape(runID) + "/stop" }
// wire is the Runtime seam over the real runtime.
type wire struct{}
// runRow is the runtime's row shape (its GET /v1/bots emits exactly these). It
// carries no sessionUrl: the runtime does not know its own public origin, so the
// control plane derives that from runId.
type runRow struct {
RunID string `json:"runId"`
Task string `json:"task"`
Surface string `json:"surface"`
Status string `json:"status"`
StartedAt string `json:"startedAt"`
}
// List reads org's runs. The runtime scopes them to the tenant cloud names, so the
// answer is already this org's and only this org's.
func (wire) List(ctx context.Context, org string) ([]Run, error) {
var answer struct {
Bots []runRow `json:"bots"`
}
if err := runtime.Read(ctx, runtime.Call{Op: listOp, Org: org}, &answer); err != nil {
return nil, err
}
out := make([]Run, 0, len(answer.Bots))
for _, b := range answer.Bots {
if b.RunID == "" {
continue // a row cloud cannot address is not a run it can offer
}
out = append(out, Run{
ID: b.RunID, Task: b.Task, Surface: b.Surface,
Status: b.Status, StartedAt: b.StartedAt,
})
}
return out, nil
}
// Stop halts org's run. It reports the runtime's own answer verbatim — absent,
// unserved, or a failure — so the handler decides what each MEANS rather than this
// stub deciding for it.
func (wire) Stop(ctx context.Context, org, runID string) error {
return runtime.Do(ctx, runtime.Call{Op: stopOp(runID), Org: org})
}
+7 -7
View File
@@ -8,12 +8,12 @@
// business LOGIC (ported to a self-contained goja bundle in github.com/hanzoai/
// captable) and gives it PERSISTENCE over per-tenant Base/SQLite. The bundle
// carries logic; the Go host carries storage. The seam between them is the
// REUSABLE clients/gojabase binding (the RW-Base goja host), which esign (#100)
// REUSABLE clients/goja binding (the RW-Base goja host), which esign (#100)
// and dataroom (#101) reuse unchanged — this leaf is just:
//
// captable bundle (github.com/hanzoai/captable.Bundle) + the per-tenant Schema
// │
// clients/gojabase.New(...) ← injects __db/__newId/__now,
// clients/goja.NewBase(...) ← injects __db/__newId/__now,
// │ one SQLite file per tenant,
// /v1/captable/* zip routes one transaction per request
//
@@ -37,7 +37,7 @@ import (
hcaptable "github.com/hanzoai/captable"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/gojabase"
"github.com/hanzoai/cloud/clients/goja"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
)
@@ -48,7 +48,7 @@ const maxBody = 1 << 20 // 1 MiB
// state is captable's own data; shared deps live in the embedded cloud.Base.
type state struct {
host *gojabase.Host
host *goja.BaseHost
}
// mounted is the active service so Shutdown can release the per-tenant stores.
@@ -72,7 +72,7 @@ func Mount(app *zip.App, deps cloud.Deps) error {
if err != nil {
return fmt.Errorf("captable.Mount: load bundle: %w", err)
}
host, err := gojabase.New(gojabase.Config{
host, err := goja.NewBase(goja.BaseConfig{
Name: "captable",
Bundle: bundle,
Schema: schema,
@@ -80,7 +80,7 @@ func Mount(app *zip.App, deps cloud.Deps) error {
OnOpen: seedCompany,
})
if err != nil {
return fmt.Errorf("captable.Mount: gojabase host: %w", err)
return fmt.Errorf("captable.Mount: goja NewBase host: %w", err)
}
s := &cloud.Service[state]{Base: cloud.NewBase(deps, "captable"), State: state{host: host}}
mounted = s
@@ -176,7 +176,7 @@ func dispatch(s *cloud.Service[state], c *zip.Ctx, route string, params map[stri
}
}
}
resp, err := s.State.host.Dispatch(c.Context(), org, gojabase.Request{
resp, err := s.State.host.Dispatch(c.Context(), org, goja.BaseRequest{
Route: route,
Params: params,
Body: body,

Some files were not shown because too many files have changed in this diff Show More