Compare commits

...
Author SHA1 Message Date
2bb35ac291 auth: accept admin-console audience in the cloud JWT allowlist (#332)
The cloud already trusts hanzo-admin-guard (the admin surface) but not admin-console
(the admin console's own OIDC client), so a SuperAdmin token minted via admin-console
was rejected on /v1/admin with 'invalid audience' — forcing an awkward hanzo-admin-guard
detour. Add admin-console so the admin console's tokens work directly, matching
GATEWAY_ALLOWED_AUDIENCES which already lists it.

Co-authored-by: zeekay <z@hanzo.ai>
2026-07-17 23:03:19 -07:00
hanzo-dev 0782431509 docs: open cloud planes blueprint (HIP-0129)
Plane map with honest tiers in LLM.md: /v1/connectors custody (in flight), /v1/channels transport (planned, branch reserved), shipped planes named by package. Spec home HIP-0129; roadmap P1-P15 lives there.
2026-07-17 22:49:44 -07:00
zeekayandz e2929b82b0 gateway: nest clients/gatewaypolicy → clients/gateway/edge (kill the compound)
"gatewaypolicy" is a compound (gateway+policy) and read as a second gateway
package. It is the ONE Gateway concern with clients/gateway — the per-org edge
policy STORE (OrgRPM ceiling + CORS + cache) that the /v1/gateway/config plane and
the package-cloud edge middleware both read.

They are two packages only to break a Go import cycle: clients/gateway imports root
cloud (cloud.Deps), and middleware_edge.go IS package cloud — so the store must be a
LEAF both can import. A flat merge cycles. Fix per the no-compound law: nest the leaf
UNDER gateway as clients/gateway/edge (edge.Policy/Store/New). One gateway namespace;
/v1/gateway/config surface unchanged; edge-middleware logic unchanged.

NOT redundant with the external hanzoai/gateway (KrakenD): that does coarse per-route
edge rate-limit + auth at ingress; this is per-AUTHENTICATED-org RPM (needs the decoded
token org), an app-level ceiling the edge proxy cannot compute. Different layer.

Pure rename (49/49, no logic change); full cmd/cloud binary links; gateway + gateway/edge
tests pass; gofmt/vet clean.
2026-07-17 22:47:51 -07:00
d19f1d9066 flags: runtime flags resolve from /v1/flags only — drop redundant env gates (#331)
waitlist_* / public_signup / gateway_* are runtime flags; strip their Env: fallbacks so
they resolve from the /v1/flags DB engine → Default (single source of truth, flipped live,
no redeploy). Boot-time ReadOnly rows (subsystem_*, network_id_*) keep Env — that IS their
boot mechanism. Nothing read these env vars outside the flags engine (verified).

Co-authored-by: zeekay <z@hanzo.ai>
2026-07-17 22:46:51 -07:00
z 8a50e0d335 Merge fix/consensus-bump: luxfi/consensus v1.36.9 (unbreak force-moved checksum) 2026-07-17 22:33:25 -07:00
zeekayandClaude Opus 4.8 1484745a01 build(deps): bump ai → v1.822.3 — drop the last WqyJh audio-fork edges
cloud transitively pulled github.com/WqyJh/{go-cosyvoice,go-openai-realtime}
through ai v1.822.2 (the go-openai-fork release, which predated ai's TTS
switch to the hanzo-owned forks). ai v1.822.3 wires ai/tts onto
github.com/hanzoai/go-cosyvoice + go-openai-realtime, so tidy drops both WqyJh
edges from cloud's graph. cloud now pulls ZERO third-party OpenAI-lineage:
WqyJh 0, sashabaranov 0, ClickHouse 0. Single datastore sql registrant
(hanzo-ds/go). Builds clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 22:29:42 -07:00
z 675809dcdf feat(admin): fleet-aggregate billing endpoints (metrics + invoices + subscriptions)
admin.hanzo.ai's SaaS-metrics/Invoices/Subscriptions pages were placeholders
awaiting cross-org /v1/admin/* endpoints. Add them as super-admin (core.Guard)
fleet-aggregate readers over the existing commerce billing engine:

  GET /v1/admin/metrics        — SaaS god-view: MRR/ARR/net-new/churn/active-subs/
                                 paying-customers/plan-mix/top-customers/recent.
                                 Single S2S proxy — commerce /v1/metrics/saas is
                                 already a cross-org aggregate (same gate finance
                                 Costs uses: RequirePlatformAdmin→IsServiceToken).
  GET /v1/admin/invoices       — cross-org invoice list; fan core.ListOrgs out
  GET /v1/admin/subscriptions  — cross-org subscription list; per-tenant reads
                                 merged (identical to revenue.go's fan-out).

Honest degradation: a failed per-org read contributes no rows, never fabricated.
go build ./clients/admin/... green, gofmt clean. Pairs with admin operator UI
(feat/admin-billing-fleet-ui).
2026-07-17 22:23:47 -07:00
zeekayandClaude Opus 4.8 f43883d34e refactor(go-openai): import the hanzoai/go-openai fork directly, drop the replace
cloud/clients{,/agent} used sashabaranov/go-openai only via a 'replace =>
hanzoai/go-openai' — which does not propagate to cloud's own consumers, so
gateway/iam/etc. each had to copy it. Now the fork declares its own module
path (github.com/hanzoai/go-openai v1.41.0): require it directly. Bumps the
lockstep fork adopters — ai v1.822.2, agent v0.1.3 — so the hz.Mount Completer
boundary shares ONE openai type set (was a hanzoai-vs-sashabaranov type
mismatch). No replace anywhere; upstream sashabaranov remains only as an
indirect dep of the go-cosyvoice TTS chain (owned next). cmd/cloud keeps its
single datastore sql registrant (hanzo-ds/go). Builds clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 22:11:47 -07:00
zandGitHub 74fe777aeb Merge: cloud S3 client minio-go → hanzoai/s3-go (the one house S3 client)
7 files repoint minio-go v7 → hanzoai/s3-go (a byte-identical fork, package
minio, zero call-site churn); minio-go demoted to indirect. Red-verified:
presign + conditional-CAS byte-identical to the prior v7.0.100 dep, drop-in.
Deploy-gate: live SeaweedFS CAS smoke (If-Match/If-None-Match/412).
2026-07-17 22:05:03 -07:00
zeekay e8fa9d6f18 policy: kill /v1/featuregate/mode alias + rename featuregate → admission
featuregate READ like a synonym for flags — the source of the "isn't this the
same thing?" confusion. It is not: flags is the Policy decide-ENGINE (/v1/flags);
this package is the request-ADMISSION gate that composes it one-way (host→service
registry + waitlist.<svc> mode read + Enforce middleware + IAM approval check).
Renamed to `admission` — the precise systems term for policy-gating requests
(k8s-style admission control) — which also dodges the gate/gateway/gatewaypolicy
name cluster. flags stays THE engine; admission is a thin one-way consumer.

Also kills the /v1/featuregate/mode compat alias entirely (route + Enforce exempt
entry + test): one route, /v1/flags/waitlist. No shim, no adaptor, no backwards
compat — per the one-and-only-one-way law.

flags engine surface unchanged. Build/vet green; admission tests + apps frozen-Wire
order test pass (admission holds featuregate's slot). Deeper Policy collapse
(authz/entitlements/gatewaypolicy → one engine) is a separate staged HIP-0127 pass.
2026-07-17 21:26:28 -07:00
hanzo-dev fac1cf9aa3 Route the S3 object plane through the hanzoai/s3-go client
Swap the S3 client in the 7 direct importers from github.com/minio/minio-go/v7
to github.com/hanzoai/s3-go (package minio; a minio-go v7.0.98 fork). Drop-in:
same package name and same New/Client/Options, {Get,Put,List,MakeBucket,
RemoveObject,RemoveObjects}Options, ObjectInfo/Object/ErrorResponse surface and
credentials.NewStaticV4; conditional-CAS (SetMatchETag/SetMatchETagExcept) and
presign paths unchanged.

minio-go leaves the direct requires and stays indirect (luxfi/zapdb via
clients/kms). Run go mod tidy after the s3-go v1.0.0 tag is published to
populate go.sum.
2026-07-17 21:24:02 -07:00
antje f7ded021ec gpu worker: a claimed job survives the engine recycle window
The supervisor recycles at queue-idle, but a freshly CLAIMED job is invisible
to the engine queue until its graph is submitted — so recycles fired over the
claim-to-submit window and staging failed on a dead engine, consuming the job
(observed twice in prod, seconds apart). Two invariants close it: a staging
latch the supervisor honors before recycling, and waitEngine() so a job
claimed while a recycle is already mid-flight waits out the restart instead
of dying on connection-refused.
2026-07-17 20:28:12 -07:00
zeekay dbc4966aeb build(iam2): bump v0.15.4 → v0.16.0 (argon2id SOTA password hashing)
Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 17:35:27 -07:00
z d38984f03f refactor(usage): unify account-usage onto the ONE /v1/usage surface
The account-usage plane (7456318) wrongly opened a SECOND usage surface inside
clients/link (/v1/links/usage). Move it into clients/usage so usage owns ALL
usage and link owns links and nothing usage — one surface, orthogonal, one window.

Moves (package link -> usage): sample.go (the Sample value + Sanitize), datastore.go
(the hanzo.account_usage warehouse series + reads, now behind a `warehouse` type that
holds only the DDL latch over aiobject's shared datastore — no handle, so usage keeps
NO Shutdown), and the record/samples handlers (account.go). Reconciled with the usage
subsystem: cloudUsageTable -> the existing llmTable, dsTime -> the existing tsLiteral,
duplicate aString -> dsString.

Route table (was /v1/links/usage*):
  POST /v1/usage           record account-usage samples (the collector)
  GET  /v1/usage/samples   one provider account's own lane dash (time series)
  GET  /v1/usage/summary   THE one summary — merged (see below)
  GET  /v1/usage/analytics{,/access}  unchanged

Summary collision resolved by MERGE, not two endpoints: the account-usage global view
folds into the existing /v1/usage/summary as a labelled `accounts` block beside spend +
LLM, over ONE window (aiobject.ResolveCloudUsageWindow drives both). Nothing dropped —
the caller's own linked-account rows AND the org Hanzo-routed rows both ride the one
summary, each side reporting its own availability, never summed.

Decomplected the Link-refresh: reportUsage braided a warehouse write with a Link
upsert, and since POST /v1/links already sets an account's usage snapshot, the sample
-> snapshot path was a SECOND way to do that. record now records usage only; the link
registry stays link's own concern. Drops the 3 Link-registry tests (they exercised
/v1/links, unreachable in a usage-only mount) and the Link half of 2 more; the warehouse
+ value coverage moves intact. No back-compat alias (the route was hours old).

Wire guard unchanged: link keeps its Shutdown (SQLite store), usage keeps none.
2026-07-17 16:53:17 -07:00
hanzo-dev 0aa853f300 fix(iam-edge): forward the public sign-in surface before the tenant gate
console.hanzo.ai is served one-binary off cloud, so its /v1/iam/* calls hit the
iam_edge — which required a validated org for EVERY route. That 401'd
'sign in to continue' on the sign-in routes themselves (get-app-login, login,
oauth token exchange), a chicken-and-egg that bricked console login (the
'unknown iam route' / 'sign in to continue' users saw). Forward the
unauthenticated-by-design sign-in surface (login-page config, credential submit,
signin/signup, captcha/verification aids, the OAuth token endpoint, OIDC
discovery) straight to IAM BEFORE the org gate. Tenant CRUD + org metadata stay
fully gated — no tenant-data route is opened. Test: TestIamEdgePublic.
2026-07-17 16:50:29 -07:00
hanzo-dev 610dcc166d wip(fleet-samples plane (clients/samples + /v1/fleet board)): rescued from agent that hit the session limit
Committed as-is to preserve the work (the building agent died mid-verify).
Not yet built/tested green; NOT merged to main. Resume from here.
2026-07-17 16:04:56 -07:00
zeekay 26a1910bca chore: trigger release build for iam2 v0.15.4 (federation fix)
d411200 (iam2 v0.15.1→v0.15.4 bump) did not trigger a release run; nudge
the push-triggered release so the federation-security-fixed image ships.
2026-07-17 15:21:10 -07:00
zeekay d411200359 build(iam2): bump v0.15.1 → v0.15.4 (federation SuperAdmin-mint CRITICAL fix)
v0.15.4 closes the red-team CRITICAL in the federation broker: authorize
Application.Organization on write + reserved-org guard in federation
link/provision (was: social login could mint a SuperAdmin / take over a
cross-tenant account) + SSRF IP filter. Required before the hanzo.id social
cutover. Build pipeline healthy (consensus v1.36.3).

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 15:03:48 -07:00
hanzo-dev 071174a414 Merge: surface the iam2 canary in /v1/flags 2026-07-17 12:39:04 -07:00
z bc98da0308 flags: surface the iam2 canary in /v1/flags
A read-only subsystem_iam2_active switch on the platform panel, mirroring
subsystem_iam_active, makes the clean-room iam2 selection visible in the /v1/flags
cockpit. The selector stays ONE thing — CLOUD_IAM_IMPL=iam2 at boot, applied on
the next reconcile — this switch reflects it, it does not add a second control.
Its description names the gate: the IAM cutover parity suite
(universe e2e/50-iam-cutover-parity) must be green against the iam2 shadow before
the canary is flipped.
2026-07-17 12:38:47 -07:00
hanzo-dev 7456318242 feat(account-usage): clients/link usage plane — samples, datastore series, /v1/links/usage
The account-usage plane over clients/link: a Sample value (one metering lane of
one provider account at one instant), a ReplacingMergeTree warehouse projection
(hanzo.account_usage + a dedup-preserving daily rollup MV) read back with explicit
read-time argMax dedup, and the /v1/links/usage surface — report samples, a
per-provider dash, and a global summary that sets a user's own linked-account plan
usage beside the org's Hanzo-routed cost of record, every row labelled by
source/scope/confidence and never summed together.

A windowless sample (a valid window class with no meter-reported duration or
reset) keys its class's nominal bucket, never the zero instant: every ranged read
filters window_start into [from,to) and the TTL drops epoch rows on arrival, so a
zero-keyed row would be written-but-never-read and would silently drop out of the
summary. Re-polls of a windowless counter collapse onto that one nominal instance
(ReplacingMergeTree by ts), so it is one row per lane, never summed across polls —
reconciling the two window-instance tests the rescued WIP left in contradiction.
2026-07-17 12:31:12 -07:00
hanzo-dev d9a20e2798 fix(identity): mint X-Billing-Account-Id from the claim, never from the client
The header was captured from client input and re-injected verbatim for any
validated principal. That was defensible while it was a mere attribution hint
no debit ever read — the comment said as much. It is not one anymore:
ai/object.Payer now resolves the PAYING account from it, so forwarding the
client's copy would let a caller name its own payer, which is the whole thing
the claim exists to prevent. A signup-org member could have sent
`X-Billing-Account-Id: org:hanzo` and pointed their spend at the shared pool.

It is now minted from the validated `billing_account` claim
(idClaims.mintedBillingAccount), mirroring iamauth.Claims.MintedBillingAccount
byte-for-byte, so the in-binary path binds what the gateway would and both
resolve one payer. The raw client copy is deleted on ingress and not restored.

The console read and the top-up now hand Payer that same claim, so the balance a
member SEES, the account a top-up FUNDS, and the account the ai gate DEBITS are
one wallet. Feeding Payer a different credential per call site is the modern
shape of the old org-vs-"org/user" split: a funded balance the gate refuses.

Tests drive real signed tokens through the boundary: the claim reaches the
header for person/org/project, a forged copy never survives (even on a token
that carries no claim, where a restored copy would be the only value present),
and an anonymous caller carries no payer at all.
2026-07-17 12:27:34 -07:00
z 36231f57e2 refactor(flags,featuregate): decomplect the waitlist host-gate out of the flag engine
flags is now the PURE (Principal,context)->verdict engine: Register/Bool/Int/
String/Board/SetPlatformSwitch/Defs + /v1/flags/* + native evaluator + the
platform-switch seed. ZERO host->service / ModeForHost / waitlist.<svc> /
mode-route knowledge.

The complete launch waitlist-gate feature moves to clients/featuregate, which
COMPOSES flags one-way (flags.Bool/Register/SetPlatformSwitch/Def/Defs; flags
never imports featuregate):
- flags/waitlist_store.go -> featuregate/registry.go (host->service map)
- flags/waitlist.go -> featuregate/waitlist.go (mode decide + admin funcs +
  seed + waitlist.<svc> Def registration + Mount/Shutdown + the mode route)
- flags/waitlist_store_test.go -> featuregate/registry_test.go
- the registry OrgStore handle (was flags.Client.registry) is now featuregate
  package state, opened in featuregate.Mount, closed in Shutdown
- Enforce default gate is now the LOCAL WaitlistModeForHost
- /v1/flags/waitlist AND /v1/featuregate/mode compat alias served by
  featuregate (route name unchanged)
- apps.Wire re-adds featuregate after admin (after flags); wire_test frozen row
- admin/services.go swaps the flags import to featuregate for the board funcs
2026-07-17 12:06:06 -07:00
zeekay 4343cdc684 fix(flags): keep /v1/featuregate/mode as a TEMPORARY compat alias for /v1/flags/waitlist
The namespace collapse (75d6f36) renamed the live waitlist-mode read to
/v1/flags/waitlist and made /v1/featuregate/mode a 404 — correct per the one-namespace
Policy primitive, but a BREAKING change to a public route whose external callers cannot
be fully enumerated from the monorepo (a deployed frontend could still call the old
path). Per the hard "never goes down for any customer" constraint, ship the collapse
WITHOUT the break: /v1/flags/waitlist is canonical; /v1/featuregate/mode is a temporary
alias to the same handler; both exempt from the Enforce gate.

Delete the alias (this route + its exempt entry in featuregate/middleware.go) once every
caller is confirmed on /v1/flags/waitlist — a one-line follow-up, gated on the owner.

Verified: gofmt clean, go build/vet green, exempt-path test asserts BOTH routes ungated.
2026-07-17 11:24:57 -07:00
hanzo-dev 187473bd92 Merge rip/services-kind: read one workload kind (App), drop the Service shim
services.hanzo.ai is dead (0 Service CRs cluster-wide; the fleet is 100% App).
clients/paas, clients/deploy, and clients/platform drop the two-kind read shim
and read apps.hanzo.ai only. The paas deploy endpoint (and release seam) now
always refuse a git-declared App with 409, naming the universe git path to
commit the tag to.
2026-07-17 11:01:28 -07:00
hanzo-dev 9659d1a56f paas/deploy/platform: read one workload kind (App), drop the Service shim
The services.hanzo.ai kind is dead: zero Service CRs exist cluster-wide and
the whole fleet is apps.hanzo.ai (kind App). These three cluster-facing planes
carried a two-kind read shim (App first, Service fallback) that is no longer
reachable, so strip it and read one kind — App.

- clients/paas: drop servicesGVR + crGVRs(); listApps/getApp/observeFleet read
  appsGVR directly (no cross-kind dedup). The deploy endpoint now always refuses
  (409): every App CR in the platform namespaces is git-declared and reconciled
  by Hanzo CD with selfHeal, so a patch here is reverted — the response names the
  git path to commit the tag to. releaseService refuses on the same grounds.
- clients/deploy: drop servicesCRGVR + appCRGVRs() and the "hanzo.ai/Service"
  registry entry; health/getAppCR/listAppCRs read appsCRGVR directly. coreSvcGVR
  (the core/v1 Service child object) is unchanged.
- clients/platform: drop servicesGVR + crGVRs(); resolveCR/getCR/deleteService
  read and delete appsGVR only. Tenant apps are still written and patched as App
  CRs in tenant-<org>.

Tests updated to the one-kind reality. Builds/vets/gofmt clean; go.mod untouched.
2026-07-17 11:00:25 -07:00
zeekay 75d6f36639 flags: move the waitlist mode read /v1/featuregate/mode -> /v1/flags/waitlist (one namespace)
The guard's public waitlist-mode read now lives under /v1/flags (the flags engine
owns it) — there is NO /v1/featuregate HTTP endpoint. The route, the Enforce
exempt prefix, and the doc/prose comments move; the featuregate Go PACKAGE (native
Enforce middleware) is NOT renamed, and the /v1/admin/services board is unchanged.

- clients/flags/routes.go            GET /v1/featuregate/mode -> GET /v1/flags/waitlist
- clients/flags/waitlist.go          doc comments repointed
- clients/featuregate/middleware.go  defaultExemptPrefixes /v1/featuregate/ -> /v1/flags/waitlist
- clients/featuregate/middleware_test.go  exempt-path assertion updated
- apps/apps.go                       stale prose comment repointed

Verified green: go build ./clients/flags/... ./clients/featuregate/... ./apps/...,
go vet, and CGO_ENABLED=0 go test ./clients/featuregate/...
2026-07-17 10:57:25 -07:00
hanzo-dev dcd107b336 Merge: real semver only — ai v1.821.1 / iam v1.31.28 / luxfi from sumdb (kill pseudo-versions + force-moved-tag poison) 2026-07-17 10:48:06 -07:00
zandhanzo-dev 04a4d68118 Real semver across the board: ai v1.821.1, iam v1.31.28, luxfi from sumdb
Three coordinate-hygiene fixes so the pipeline resolves deterministically:
  - ai v1.820.0 -> v1.821.1. v1.820.0 pinned iam at an orphaned pseudo-version
    (a commit rebased out of existence); v1.821.1 pins the real iam tag v1.31.28.
  - iam -> v1.31.28, the real published tag; the pseudo-version and its replace
    are gone.
  - luxfi go.sum re-recorded from the immutable sum.golang.org via go mod tidy,
    so consensus/vm can no longer carry the hashes a force-moved git tag served.

Nothing but coordinates changed; ai v1.821.1 is v1.820.0's tree with one dep line
repinned, so the compiled result is identical to the shipped v1.801.49. Real
public semver only: no pseudo-versions, no replaces, no force-moved tags.
2026-07-17 10:47:19 -07:00
zeekay a773c9225a fix(release): fail-closed container-tag floor so an orphaned tag is never reused
The v1.801.50 tag collision: a run pushed :v1.801.50 then was cancelled after
imagetools-create but before its git tag (orphaned container tag). The Tag steps
container-tag floor (cont_max) was fail-OPEN — `gh api ... 2>/dev/null || true`
yields "" on any API error — so a later run did NOT see :v1.801.50, recomputed
the same number, and REASSIGNED :v1.801.50 to a different image: an ambiguous
mutable prod tag (silent flip on any fresh-node reschedule).

Fail-CLOSED: if the container-tag lookup ERRORS (vs legitimately empty), retry the
whole attempt instead of proceeding on a git-only floor that cannot see the orphan.
A version that already has a pushed image is now never reused.

NOT reordering git-tag before imagetools-create (the other candidate fix): that
reintroduces the phantom "tag exists, image does not" this workflow was built to
prevent. Pairs with the crane-mirror timeout (ed4d372) that stops the hang→cancel
which orphans tags in the first place. Compute-step cont_max left as-is (hint only).

[skip ci]
2026-07-17 10:29:03 -07:00
antjeandGitHub 50bbf3a64d supervisor: recycle only at queue-idle; busy is not dead (#329)
Recycling on each completed render killed long renders mid-sample when short
jobs shared the engine (observed: every direct render died within ~6 minutes
while probe jobs cycled). The recycle now defers until the queue is empty.
Health: an engine that answers /queue with work in it is alive however slowly
it answers /system_stats; restarts require three consecutive silent probes
with an idle or unreadable queue.
2026-07-17 02:48:12 -07:00
hanzo-dev 7685705165 fix(release): mirror LOGIN is best-effort too — a registry blip must not fail the release
The 'Mirror credential' step fail-safed only on a missing KMS token, not on the
docker-login to registry.hanzo.ai itself. A transient 502 from the mirror registry
(ingress blip; the registry was healthy 6m before and after) killed the whole
serialized release — no image, no tag — even though ghcr (the PRIMARY) was fine.
Both login paths now skip the mirror (MIRROR_OK unset) on failure and continue.
Complements ed4d372 (the crane-copy timeout): the mirror is now best-effort end to end.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-17 02:26:28 -07:00
z ed4d37235f fix(release): bound the registry.hanzo.ai crane mirror with a timeout
An unbounded `crane copy` to registry.hanzo.ai can HANG (not just fail) — the
best-effort mirror once livelocked the Tag step and held the entire serialized
release lane (concurrency: release-cloud, cancel-in-progress:false), so no queued
release could run. A best-effort mirror must never be able to block the git-tag
receipt that follows it. `timeout 120` makes it truly best-effort.

[skip ci]
2026-07-17 02:06:48 -07:00
zeekay 0655cdb8cd build(iam2): bump v0.14.0 → v0.15.1 (federation + signing-key generation)
v0.15.0 adds the OIDC/OAuth2 social-federation broker (Google/GitHub);
v0.15.1 mints signing keys for keyless reserved-org certs so the embedded
iam2 publishes a JWKS and can sign tokens (shadow-canary finding). Carries
the full parity + RFC surface into the cloud image for the hanzo.id cutover.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 01:35:15 -07:00
hanzo-dev 7ff5862f42 build(deps): adopt luxfi/consensus v1.36.9 (unbreak force-moved v1.36.2 checksum)
luxfi/consensus v1.36.2 was force-repushed with different go.mod content, so
cloud's committed go.sum no longer matches and 'go mod download' aborts with a
SECURITY ERROR — breaking EVERY release. Same recurring luxfi force-move pattern
as c93ddf9 (keys). Bump to latest stable v1.36.9; clients/controlplane (only
importer) compiles clean, go mod verify passes.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-17 01:27:58 -07:00
antjeandGitHub fceb34c1d2 render: poll window matches the dispatch cap; engine recycles after each render (#326)
* render: poll window matches the dispatch cap; engine recycles after each render

The 10m local history poll undercut the 4h startToCloseTimeout the dispatcher
grants — live renders (observed 8-70m) were marked failed while still sampling;
only the mirror later delivered them. renderWindow now matches the cap.

The engine leaks ~58GB per render. The handler signals a recycle after each
COMPLETED render (never on timeout — the engine may still be sampling and the
mirror rescues late finishes); the supervisor restarts on the signal.

* mirror: skip hidden files — AppleDouble forks pass the extension check

._foo.png is a mac resource fork, not a render; 700+ of them poisoned a
library within an hour of the mirror going live.

* deps: luxfi/consensus v1.36.2 -> v1.36.3 — the v1.36.2 tag was re-pushed

Cold builds fail sumdb verification against the moved tag (downloaded
eKzasq4O... vs sealed IbeWQF1w...). v1.36.3 is the immutable successor;
never re-tag a published version.
2026-07-17 01:26:40 -07:00
hanzo-dev d5e12b3df1 feat(ai): bump ai v1.818.0 → v1.820.0 — router live-by-default + record-all + self-export/delete
Ships to prod: router.enabled=true (model=auto routes for every org by default),
per-request RoutingEvent recording for auto AND explicit models (up/down feedback
works on all models), per-org + global fit-gate-deploy-publish training, and the
self-scoped routing-data export/delete (data ownership). Pairs with the universe
CR ROUTER_ENDPOINT removal (heuristic 300ns is the live path).

Claude-Session: https://claude.ai/code/session_018PmFAHZvbBSTsuWyebwMra
2026-07-17 01:25:40 -07:00
hanzo-dev 36ba00f540 dedup: extract clients/payout from the 3 byte-mirror commerce.go copies
referrals/affiliates/authors each carried a byte-identical commerce.go (their own
doc-comments said so): the same commerce interface, httpCommerce, newCommerceClient,
deposit(), spendCents(), errUnconfigured — the S2S COMMERCE_SERVICE_TOKEN money-in
path (POST /v1/billing/deposit) + usage-rollup, triplicated.

Extract ONE clients/payout (attributed-credit -> commerce via commerceinproc): the
exported Commerce/Client/NewClient/ErrUnconfigured. Each program keeps a THIN
adapter — its own lowercase commerce interface + a commerceSeam that delegates to
payout.Client — so the program store/handler code AND their fakeCommerce test doubles
are untouched, and each program still names its own grant tag (grant:referral /
grant:affiliate / grant:author). ~330 duplicated lines collapse to one binding.

Zero behaviour change: identical HTTP contract, headers (X-Org-Id, Bearer), body,
fail-soft (ErrUnconfigured on deposit / 0 on spend when unwired), and errors.Is
sentinel. Adds payout unit tests (httptest) that give the extracted HTTP path REAL
coverage the fakes never did — ok clients/payout 0.010s.
2026-07-17 01:19:36 -07:00
hanzo-dev db53daea72 dedup: fold clients/gojabase into clients/goja (the Base binding is an option)
gojabase was the read-WRITE-Base sibling of goja: it wrapped a goja.Host and
added per-tenant Base/SQLite persistence, but duplicated the Host/Config/Request/
Response/New surface. Fold it into the ONE goja package as the Base-binding
CONSTRUCTOR — the persistence layer is now opted into via NewBase (vs New for a
read-only catalog bundle):

  goja.New   / goja.Host   / goja.Config   / goja.Request    read-only engine (plans/pricing)
  goja.NewBase / goja.BaseHost / goja.BaseConfig / goja.BaseRequest   + per-tenant Base

Moves gojabase.go -> clients/goja/base.go (renamed types, no goja. self-import),
store.go -> basestore.go, and both test files, all into package goja (zero
identifier collisions, coverage preserved). Repoints every importer —
dataroom/captable/sign (RW) to goja.Base*; plan/pricing already used goja and are
unchanged; base uses goja.TenantSegment. clients/gojabase deleted.

Behaviour is byte-identical: the engine, the per-request transaction commit-on-
<400, the injective TenantSegment, and the __db/__blob/__newId/__now host globals
are unchanged; only the package + exported names moved. No routes (both are
libraries). The gojabase[...] error prefix is kept as the RW-layer diagnostic label.
2026-07-17 01:19:36 -07:00
hanzo-dev 919d96f3f8 dedup: fold connectorruntime into the one automations subsystem
clients/connectorruntime mounts exactly ONE route —
POST /v1/automations/connectors/:id/run — the in-process goja runner paired with
automations own GET /v1/automations/connectors catalogue. It was a separate Wire
entry solely for that route. Fold connectorruntime.Mount in as a terminal
sub-mount of automations.Mount and drop its Wire entry + import -> ONE
automations subsystem.

The route is DISTINCT from every automations route and automations mounts no
/v1/automations/* wildcard, so there is no shadow; the runner still resolves the
shared engine lazily. clients/connectorruntime stays a focused package
(composition); its internal bundlecmd tool is untouched. Frozen wire row
removed.
2026-07-17 01:19:36 -07:00
hanzo-dev d846f17bf8 dedup: fold platform cron into the one tasks subsystem
clients/cron mounts NO routes — its Mount only launches a background starter
that registers durable schedules on the SAME shared engine (cloud.EmbeddedTasks)
that clients/tasks fronts. It was a separate Wire entry purely to get its
goroutine launched. Fold it in as a terminal sub-mount of tasks.Mount and drop
the cron Wire entry + import -> ONE tasks subsystem.

clients/cron stays a focused package (composition, not code-dumping): tasks
imports and invokes it. No routes change (cron never had any); the scheduler
still waits for the post-MountAll engine, so timing is unchanged. Frozen wire
row removed.
2026-07-17 01:19:36 -07:00
hanzo-dev 1648c08839 dedup: normalize the plan subsystem enable id "plans" -> "plan"
clients/plan.Mount was wired under the name "plans" while its package, and now
its generated standalone cmd, are "plan" — one subsystem, two names. Normalize
the Wire enable id (and cmd/plans -> cmd/plan, ServeSingle arg) to "plan".

Product routes are unchanged: the subsystem still serves /v1/plans/* (plural),
including its OwnsHealth /v1/plans/health probe — only the enable id / binary
name changes. No route drop; mount-all default still enables it (empty Enable =
all on). Updated the frozen wire row and the two cmd/cloud enable-id references;
TestMountAllAndServeHealth now maps plan to its real /v1/plans/health path
(enable id no longer equals route prefix for plan, as is already true for
account/runtime/agent).
2026-07-17 01:19:36 -07:00
385237bdaa fix(release): resolve prebuilt artifact digests + unbreak cloud-flags publish (#327)
cloud#321 landed the Go-only Dockerfile (FROM cloud-flags:latest) but the
release.yml integration wasn't in it, and the reusable could not publish
cloud-flags at all — so every release since has FAILED at
'FROM cloud-flags:latest: not found'. Two fixes:

1. native/flags/Dockerfile base ghcr.io/hanzoai/mirror/rust -> public.ecr.aws
   (digest-identical). The hanzoai/ci reusable builds cloud-flags with the repo
   GITHUB_TOKEN, which 403s pulling the cross-repo-linked private mirror package;
   a public base is GITHUB_TOKEN-pullable, so cloud-flags finally publishes.

2. release.yml resolves console-embed/agent-skills/cloud-flags :latest to
   IMMUTABLE digests at release time (crane) and passes them as CONSOLE_IMAGE/
   SKILLS_IMAGE/FLAGS_IMAGE build-args to BOTH the smoke build and the push build,
   replacing CONSOLE_CACHEBUST. Reproducible (pinned, not floating :latest) AND
   fresh (a console/skills/flags change is a new digest). A MISSING artifact FAILS
   the release BEFORE build/smoke/tag — the receipt invariant holds, never a
   phantom tag on an image that could not embed the real console.

Preserves #322's functional + migration smoke gates (different sections).

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-17 01:18:50 -07:00
118 changed files with 7307 additions and 1825 deletions
+78 -31
View File
@@ -124,20 +124,6 @@ jobs:
echo "sha_short=$(git rev-parse --short "$GITHUB_SHA")" >> "$GITHUB_OUTPUT"
echo "Next release: v${version} (git_max='${git_max:-none}' container_max='${cont_max:-none}')"
# Console-embed cachebust. The console clone+build layer is keyed on this;
# prefer hanzoai/console main HEAD so a CONSOLE-ONLY change re-embeds without
# needing a cloud commit (cloud-sha alone froze the embed between cloud pushes).
# git ls-remote must CLEAR the extraheader actions/checkout installs (it carries
# THIS repo's GITHUB_TOKEN, which 404s the cross-repo console lookup); gh is not
# on the runner. If resolution yields nothing, fall back to the cloud sha — still
# unique per cloud commit, so the embed is never frozen. Either way THIS build
# busts (new value) and re-clones console main fresh.
console_head="$(git -c 'http.https://github.com/.extraheader=' ls-remote \
"https://x-access-token:${GH_PAT}@github.com/hanzoai/console.git" refs/heads/main 2>/dev/null | cut -f1 || true)"
cachebust="${console_head:-$GITHUB_SHA}"
echo "cachebust=${cachebust}" >> "$GITHUB_OUTPUT"
echo "console cachebust: ${cachebust} (console_head='${console_head:-none}')"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
@@ -160,8 +146,12 @@ jobs:
# Direct credential first (repo/org secret — works on private repos,
# where the Free plan hides org KMS secrets); KMS kubeconfig fallback.
if [ -n "${REGISTRY_USER:-}" ] && [ -n "${REGISTRY_PASSWORD:-}" ]; then
echo "$REGISTRY_PASSWORD" | docker login registry.hanzo.ai -u "$REGISTRY_USER" --password-stdin
echo "MIRROR_OK=1" >> "$GITHUB_ENV"; exit 0
if echo "$REGISTRY_PASSWORD" | docker login registry.hanzo.ai -u "$REGISTRY_USER" --password-stdin; then
echo "MIRROR_OK=1" >> "$GITHUB_ENV"
else
echo "mirror login failed (registry.hanzo.ai unreachable) — mirror skipped, release continues"
fi
exit 0
fi
[ -z "${KMS_CLIENT_ID:-}" ] && { echo "no KMS creds — mirror skipped"; exit 0; }
TOKEN=$(curl -sf "$KMS_ENDPOINT/v1/kms/auth/login" -H 'Content-Type: application/json' -d "{\"clientId\":\"$KMS_CLIENT_ID\",\"clientSecret\":\"$KMS_CLIENT_SECRET\"}" | jq -r '.accessToken // empty')
@@ -180,8 +170,11 @@ jobs:
UP=$(echo "$CFG" | jq -r '.auths["registry.hanzo.ai"].auth // empty' | base64 -d)
[ -z "$UP" ] && { echo "no registry auth — mirror skipped"; exit 0; }
echo "::add-mask::${UP#*:}"
echo "${UP#*:}" | docker login registry.hanzo.ai -u "${UP%%:*}" --password-stdin
echo "MIRROR_OK=1" >> "$GITHUB_ENV"
if echo "${UP#*:}" | docker login registry.hanzo.ai -u "${UP%%:*}" --password-stdin; then
echo "MIRROR_OK=1" >> "$GITHUB_ENV"
else
echo "mirror login failed (registry.hanzo.ai unreachable) — mirror skipped, release continues"
fi
- name: Log in to ghcr.io (GH_PAT — writes the cloud package despite its ai-repo linkage)
uses: docker/login-action@v3
@@ -190,6 +183,39 @@ jobs:
username: hanzo-dev
password: ${{ secrets.GH_PAT }}
- name: Resolve decomplection artifact digests (the Go-only build's prebuilt inputs)
id: artifacts
run: |
set -euo pipefail
# cloud compiles ONLY Go; it pulls three prebuilt artifacts (console SPA,
# agent-skills catalog, native flags staticlib). Resolve each published
# :latest to an IMMUTABLE digest so THIS release is reproducible (pinned,
# not floating :latest) AND a console/skills/flags change is picked up —
# its CI republished :latest, so this resolves to the NEW digest. A MISSING
# artifact FAILS the release HERE, before build/smoke/push/tag: the receipt
# invariant means we never tag an image that couldn't embed the real console.
command -v crane >/dev/null 2>&1 || {
mkdir -p "$HOME/.local/bin"
curl -fsSL "https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz" \
| tar -xz -C "$HOME/.local/bin" crane
}
export PATH="$HOME/.local/bin:$PATH"
resolve() {
local repo="$1" d
d="$(crane digest "ghcr.io/hanzoai/${repo}:latest" 2>/dev/null || true)"
[ -n "$d" ] || { echo "::error::decomplection artifact ghcr.io/hanzoai/${repo}:latest is not published — refusing to cut a release that would embed a stale/placeholder ${repo}"; return 1; }
printf 'ghcr.io/hanzoai/%s@%s' "$repo" "$d"
}
CONSOLE_IMAGE="$(resolve console-embed)" || exit 1
SKILLS_IMAGE="$(resolve agent-skills)" || exit 1
FLAGS_IMAGE="$(resolve cloud-flags)" || exit 1
{
echo "console_image=${CONSOLE_IMAGE}"
echo "skills_image=${SKILLS_IMAGE}"
echo "flags_image=${FLAGS_IMAGE}"
} >> "$GITHUB_OUTPUT"
echo "resolved: console=${CONSOLE_IMAGE} skills=${SKILLS_IMAGE} flags=${FLAGS_IMAGE}"
- name: OCI labels
id: meta
uses: docker/metadata-action@v5
@@ -216,11 +242,13 @@ jobs:
load: true
tags: cloud:smoke
labels: ${{ steps.meta.outputs.labels }}
# Bust the console clone+build layer every release (the cloud commit sha is
# unique per push) so the embed re-fetches console main HEAD fresh — never the
# frozen snapshot the persistent BuildKit cache would otherwise serve forever.
# cloud compiles ONLY Go: pull the three prebuilt artifacts pinned to the
# digests resolved above (reproducible, and fresh — a console/skills/flags
# change is a new digest). No node/python/rust toolchain in this build.
build-args: |
CONSOLE_CACHEBUST=${{ steps.ver.outputs.cachebust }}
CONSOLE_IMAGE=${{ steps.artifacts.outputs.console_image }}
SKILLS_IMAGE=${{ steps.artifacts.outputs.skills_image }}
FLAGS_IMAGE=${{ steps.artifacts.outputs.flags_image }}
# GIT_AUTH_TOKEN: BuildKit secret the Dockerfile consumes to fetch private
# cross-org Go modules (hanzoai/*, luxfi/*) over authenticated git.
secrets: |
@@ -462,10 +490,12 @@ jobs:
ghcr.io/hanzoai/cloud:sha-${{ steps.ver.outputs.sha_short }}
ghcr.io/hanzoai/cloud:latest
labels: ${{ steps.meta.outputs.labels }}
# SAME cachebust as the smoke build → every layer is a cache hit from step 1
# and the pushed image is byte-identical to the one the smoke test proved.
# SAME artifact digests as the smoke build → every layer is a cache hit from
# step 1 and the pushed image is byte-identical to the one smoke proved.
build-args: |
CONSOLE_CACHEBUST=${{ steps.ver.outputs.cachebust }}
CONSOLE_IMAGE=${{ steps.artifacts.outputs.console_image }}
SKILLS_IMAGE=${{ steps.artifacts.outputs.skills_image }}
FLAGS_IMAGE=${{ steps.artifacts.outputs.flags_image }}
secrets: |
GIT_AUTH_TOKEN=${{ secrets.GH_PAT }}
@@ -499,12 +529,25 @@ jobs:
# newest-first and version tags are monotonic, so the highest version
# is always among the most-recent versions; paginating the WHOLE
# registry history is what livelocked this step as tags accumulated.
# Fail-CLOSED. An ORPHANED container tag — image pushed by a run that
# died or was cancelled after imagetools-create but before its git tag —
# MUST raise the floor, or a later run reassigns that same number to a
# different image (an ambiguous mutable prod tag; the v1.801.50 flip). A
# git-only floor can't see the orphan, so if the container-tag lookup
# ERRORS (vs legitimately returning no tags) we retry the whole attempt
# rather than silently proceeding — a version with a pushed image is never
# reused. (Reordering git-tag before imagetools-create is the WRONG fix: it
# reintroduces the phantom "tag ⇔ no image" this workflow exists to prevent.)
cont_max=""
if command -v gh >/dev/null 2>&1; then
cont_max="$(GH_TOKEN="$GH_PAT" gh api \
'/orgs/hanzoai/packages/container/cloud/versions?per_page=100' \
--jq '.[].metadata.container.tags[]?' 2>/dev/null \
| sed 's/^v//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true)"
if cont_raw="$(GH_TOKEN="$GH_PAT" gh api \
'/orgs/hanzoai/packages/container/cloud/versions?per_page=100' \
--jq '.[].metadata.container.tags[]?' 2>/dev/null)"; then
cont_max="$(printf '%s\n' "$cont_raw" \
| sed 's/^v//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true)"
else
echo " container-tag lookup failed — retry so an orphaned tag can't be reused (attempt $attempt)"; sleep 3; continue
fi
fi
max="$(printf '%s\n%s\n%s\n' "1.786.0" "$git_max" "$cont_max" \
| grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1)"
@@ -534,8 +577,12 @@ jobs:
export PATH="$HOME/.local/bin:$PATH"
}
for MT in "${V}" "${VER}" "${major}.${minor}"; do
crane copy "$SHA_IMG" "registry.hanzo.ai/hanzoai/cloud:${MT}" \
|| echo "::warning::mirror registry.hanzo.ai/hanzoai/cloud:${MT} failed"
# Bounded: registry.hanzo.ai can *hang* (not just fail), and this
# is best-effort — an unbounded crane copy once livelocked the whole
# tag step and held the serialized release lane. timeout makes the
# mirror truly best-effort so the git-tag receipt below always runs.
timeout 120 crane copy "$SHA_IMG" "registry.hanzo.ai/hanzoai/cloud:${MT}" \
|| echo "::warning::mirror registry.hanzo.ai/hanzoai/cloud:${MT} failed or timed out"
done
fi
git tag -a "$V" -m "release $V — image ghcr.io/hanzoai/cloud:$V (retagged from sha-${{ steps.ver.outputs.sha_short }}, smoke-passed ${GITHUB_SHA})"
+38
View File
@@ -6,6 +6,44 @@ artifact serves `api.hanzo.ai`, `api.lux.cloud`, `api.zoo.cloud`, `api.osage.clo
and every white-label reseller. Brand, enabled subsystems, and org scope are
deployment configuration.
## Open Cloud planes
Spec home: HIP-0129 `hip-0129-open-cloud-planes` (hips repo). This section is a
map, not the spec. One noun, one owner, one route family. No plane reads another
plane's store; imports flow custody-ward only (channels -> integrations, never
reverse).
| Route | Noun | Owner | Tier |
| --- | --- | --- | --- |
| `/v1/connectors` | Custody: per-user BYO external accounts | `clients/integrations` (extends; user scope new) | In flight (branch `feat/connectors`) |
| `/v1/channels` | Transport: portable message envelope, DM pairing, send + inbox | `clients/channels` (new) | Planned (branch `feat/channels` reserved; no transport code yet) |
| `/v1/sync` | Data: bidirectional sync engine | `clients/sync` | Shipped |
| `/v1/automations` | Workflows: flows/runs, goja piece runtime | `clients/automations` | Shipped |
| `/v1/compute/bots` | Hosting: `@hanzo/bot` Node containers | `clients/bots` | Shipped |
| `/v1/tasks` | Durable engine | `clients/tasks` | Shipped |
| `/v1/gpus` + fleet | BYO GPU presence | `clients/fleet` + `clients/visor` | Shipped |
| IAM | Identity: users, orgs, roles | IAM | Shipped |
| KMS | Secret custody: sealed secrets | `clients/kms` | Shipped |
Custody invariants: secrets sealed in KMS at
`/orgs/{org}/users/{user}/connectors/{provider}/{label}`, never in SQLite rows;
verify before store. Refresh is single-flight with rotation resealing; the CLI
does local browser PKCE and posts the bundle to
`POST /v1/connectors/:provider/credential`; cloud owns device-code flows.
Transport invariants: typed actions (`command|url|select|approval`), no raw
string sniffing; pairing codes 8 chars, 1h TTL, max 3 pending per account,
owner bootstrap on first approval.
Container boundary is permanent for native-module, host-filesystem, loop-state,
and vendor-Node work (agent loop, exec/PTY, harnesses, browser, voice, codecs,
Node-bound channels, plugin SDK/loader). The Node plugin SDK is never ported to
Go; cloud extensibility is connectors/automations/tools.
Port roadmap (P1-P15) lives in HIP-0129; do not restate it here. Every claim
carries its tier: Shipped (on main, named package/route), In flight (named
pre-main branch), Planned (backlog id or named reservation).
## Framework doctrine
One way to do everything. Composable, orthogonal, DRY. A new subsystem is a
+21 -18
View File
@@ -51,6 +51,7 @@ import (
// owns process-lifetime resources, a Shutdown); Wire references them directly.
"github.com/hanzoai/cloud/clients/account"
"github.com/hanzoai/cloud/clients/admin"
"github.com/hanzoai/cloud/clients/admission"
"github.com/hanzoai/cloud/clients/ads"
"github.com/hanzoai/cloud/clients/affiliates"
"github.com/hanzoai/cloud/clients/agent"
@@ -67,10 +68,8 @@ import (
"github.com/hanzoai/cloud/clients/catalogsync"
"github.com/hanzoai/cloud/clients/code"
"github.com/hanzoai/cloud/clients/company"
"github.com/hanzoai/cloud/clients/connectorruntime"
"github.com/hanzoai/cloud/clients/content"
"github.com/hanzoai/cloud/clients/crm"
"github.com/hanzoai/cloud/clients/cron"
"github.com/hanzoai/cloud/clients/dataroom"
"github.com/hanzoai/cloud/clients/deploy"
"github.com/hanzoai/cloud/clients/do"
@@ -236,7 +235,10 @@ func Wire() []cloud.MountSpec {
// CommerceClient is wired directly in pickCommerceClient).
{Name: "commerce", Mount: mountCommerce},
{Name: "licensing", Mount: licensing.Mount},
{Name: "plans", Mount: plan.Mount, OwnsHealth: true},
// clients/plan.Mount. Enable id normalized "plans" -> "plan" to match the
// package + generated cmd/plan (one subsystem, one name). Its product routes
// stay /v1/plans/* (incl. the OwnsHealth /v1/plans/health probe) — unchanged.
{Name: "plan", Mount: plan.Mount, OwnsHealth: true},
{Name: "pricing", Mount: pricing.Mount, OwnsHealth: true},
// /v1/s3/buckets/* + /v1/s3/health. Mounts BEFORE provisioning (120) so its static
// routes win over provisioning's /v1/s3/:name. OwnsHealth (real fail-closed probe).
@@ -336,23 +338,24 @@ func Wire() []cloud.MountSpec {
{Name: "evals", Mount: eval.Mount},
{Name: "treasury", Mount: treasury.Mount, Shutdown: ctxShutdown(treasury.Shutdown)},
{Name: "admin", Mount: admin.Mount},
// Launch-control (per-service waitlist mode) folded into the flags engine: the
// mode IS the switch waitlist.<svc>, the board is the /v1/admin/services lens,
// and /v1/featuregate/mode is served by flags. featuregate is no longer a mounted
// subsystem — it exposes only the native Enforce middleware (wired in serve.go),
// a consumer of flags.WaitlistModeForHost.
// Launch-control gate (per-service waitlist): the COMPLETE feature — host→service
// registry + brand seed + the waitlist.<svc> switch registration + the
// /v1/flags/waitlist (and /v1/admission/mode compat) mode read + the Enforce
// middleware — COMPOSING the flags engine one-way (flags.Bool/Register/
// SetPlatformSwitch; flags never imports admission). Mounts AFTER flags so the
// engine's platform-switch plane is installed first; the admin board is the
// /v1/admin/services lens over it. Owns the registry store handle → Shutdown.
{Name: "admission", Mount: admission.Mount, Shutdown: ctxShutdown(admission.Shutdown)},
// Tasks: the durable workflow/UI surface AND platform cron (durable schedules
// on the same shared engine, replacing every k8s CronJob). cron was a separate
// Wire entry; it mounts no routes and only registers schedules, so it is folded
// in as a sub-mount of tasks.Mount — ONE tasks subsystem.
{Name: "tasks", Mount: tasks.Mount},
// Platform cron: durable schedules on the shared tasks engine replacing
// every k8s CronJob — entries are cron.hanzo.ai ConfigMaps (universe git),
// runs visible in the Tasks console. Mounts no routes; starts after the
// engine is wired.
{Name: "cron", Mount: cron.Mount},
// Automations: the connector catalogue + flow engine AND native single-connector
// execution (POST /v1/automations/connectors/:id/run, HIP-0126). The connector
// runner mounts no other routes, so it is folded in as a sub-mount of
// automations.Mount (was a separate "connectorruntime" entry) — ONE subsystem.
{Name: "automations", Mount: automations.Mount, Shutdown: automations.Shutdown},
// Native single-connector execution (HIP-0126): runs an ActivePieces JS
// connector action in-process via goja (clients/connectorruntime), retiring
// the standalone auto Node engine. Mounts POST /v1/automations/connectors/:id/run,
// paired with the automations catalogue above; STAGED like the rest.
{Name: "connectorruntime", Mount: connectorruntime.Mount},
// Unified tool plane: /v1/tools/* — the ONE registry (connectors, functions,
// agents, skills, external MCP servers, full-cloud-control /v1 routes), per-org
// activation, and the unified MCP endpoint. Sources register into it from their
+85 -86
View File
@@ -23,92 +23,91 @@ var frozen = []struct {
ownsHealth bool
hasShutdown bool
}{
{"pubsub", false, true}, // was order 5
{"kafka", false, true}, // was order 6
{"agentskills", false, false}, // was order 8
{"flags", true, true}, // was order 9; native engine: /v1/flags health + store shutdown
{"kms", true, false}, // was order 10
{"metrics", false, false}, // was order 40
{"ingress", false, true}, // was order 42
{"account", false, false}, // was order 48
{"iam", false, false}, // was order 50
{"base", true, true}, // was order 60; per-org embed added Shutdown (#298)
{"o11y", false, true}, // ONE observability subsystem (was co-owned orders 69+70): read plane + the hanzoai/o11y module wildcard folded in as MountO11y's terminal sub-mount. OwnsHealth=false keeps /v1/o11y/health the generic always-ok route the module co-entry used to trigger.
{"authz", false, false}, // was order 70
{"commerce", false, false}, // was order 100
{"licensing", false, false}, // was order 110
{"plans", true, false}, // was order 111
{"pricing", true, false}, // was order 112
{"storage", true, false}, // was order 118
{"provisioning", false, false}, // was order 120
{"billing", false, false}, // was order 121
{"account-bridge", false, false}, // was order 122
{"do", false, false}, // was order 123
{"platform", true, false}, // was order 124
{"projects", false, false}, // was order 125
{"prompts", false, false}, // was order 126
{"agents", false, true}, // was order 127
{"link", false, true}, // new: unified AI login manager (/v1/links), after agents
{"wallets", false, true}, // was order 127
{"x402", false, true}, // new: x402 pay-per-use settlement (after wallets)
{"paas", true, false}, // was order 128
{"deploy", true, false}, // after paas (release seam), before functions
{"functions", false, false}, // was order 128
{"tracker", false, false}, // was order 129
{"templates", false, false}, // was order 129
{"framework", false, true}, // was order 129
{"knowledge", false, false}, // was order 130
{"content", false, true}, // new: marketing content loop (after knowledge)
{"catalogsync", false, true}, // new: reverse loop (product.created → render) after content
{"ml", true, false}, // was order 130
{"usage", false, false}, // was order 131
{"crm", false, false}, // was order 131
{"marketing", false, true}, // new: marketing domain fold (after crm)
{"ads", false, true}, // new: ads domain fold (after crm)
{"social", false, true}, // new: /v1/social fold (after crm)
{"analytics", true, false}, // was order 132
{"git", false, false}, // was order 132
{"sync", false, true}, // /v1/sync engine (owns per-org DB handles → Shutdown)
{"visor", false, false}, // was order 133
{"captable", false, true}, // was order 133
{"code", false, true}, // was order 134
{"zero-trust", false, false}, // was order 134
{"dataroom", true, true}, // was order 134
{"graph", false, false}, // was order 135
{"security", true, true}, // was order 136
{"integrations", false, true}, // was order 137
{"sbom", true, false}, // was order 137
{"team", false, true}, // was order 138
{"settings", false, true}, // was order 138
{"notify", true, false}, // was order 139
{"gateway", false, false}, // was order 139
{"entitlements", false, true}, // was order 139
{"exec", false, false}, // was order 140
{"websearch", false, false}, // was order 141
{"world", false, true}, // was order 142
{"runtime", false, false}, // was order 143; was "bot" until the transport was named for what it is
{"authors", false, true}, // was order 143
{"bots", false, false}, // was order 143
{"audit", false, false}, // was order 144
{"affiliates", false, false}, // was order 144
{"sign", true, true}, // was order 145
{"product", false, false}, // was order 145
{"evals", false, false}, // was order 145
{"treasury", false, true}, // was order 146
{"admin", false, false}, // was order 146
{"tasks", false, false}, // was order 147
{"cron", false, false}, // durable platform cron on the shared engine (post-freeze add)
{"automations", false, true}, // was order 148
{"connectorruntime", false, false}, // new: native single-connector exec via goja (after automations, HIP-0126)
{"tools", false, true}, // new: unified tool plane (after automations)
{"marketplace", false, true}, // new: marketplace over the tool plane (after tools)
{"referrals", false, false}, // was order 149
{"guide", false, true}, // new: Business AI Guide (after referrals, before ai)
{"company", false, true}, // new: Hanzo Company formation state machine (after guide)
{"agent", false, false}, // new: /v1/agent tool-calling round (before zen/ai catch-all)
{"zen", false, false}, // zen* claim middleware before ai's catch-all (hip-00NN)
{"ai", false, false}, // was order 150
{"plugins", false, false}, // was order 900
{"pubsub", false, true}, // was order 5
{"kafka", false, true}, // was order 6
{"agentskills", false, false}, // was order 8
{"flags", true, true}, // was order 9; native engine: /v1/flags health + store shutdown
{"kms", true, false}, // was order 10
{"metrics", false, false}, // was order 40
{"ingress", false, true}, // was order 42
{"account", false, false}, // was order 48
{"iam", false, false}, // was order 50
{"base", true, true}, // was order 60; per-org embed added Shutdown (#298)
{"o11y", false, true}, // ONE observability subsystem (was co-owned orders 69+70): read plane + the hanzoai/o11y module wildcard folded in as MountO11y's terminal sub-mount. OwnsHealth=false keeps /v1/o11y/health the generic always-ok route the module co-entry used to trigger.
{"authz", false, false}, // was order 70
{"commerce", false, false}, // was order 100
{"licensing", false, false}, // was order 110
{"plan", true, false}, // was order 111; enable id normalized plans->plan (routes stay /v1/plans/*)
{"pricing", true, false}, // was order 112
{"storage", true, false}, // was order 118
{"provisioning", false, false}, // was order 120
{"billing", false, false}, // was order 121
{"account-bridge", false, false}, // was order 122
{"do", false, false}, // was order 123
{"platform", true, false}, // was order 124
{"projects", false, false}, // was order 125
{"prompts", false, false}, // was order 126
{"agents", false, true}, // was order 127
{"link", false, true}, // new: unified AI login manager (/v1/links), after agents
{"wallets", false, true}, // was order 127
{"x402", false, true}, // new: x402 pay-per-use settlement (after wallets)
{"paas", true, false}, // was order 128
{"deploy", true, false}, // after paas (release seam), before functions
{"functions", false, false}, // was order 128
{"tracker", false, false}, // was order 129
{"templates", false, false}, // was order 129
{"framework", false, true}, // was order 129
{"knowledge", false, false}, // was order 130
{"content", false, true}, // new: marketing content loop (after knowledge)
{"catalogsync", false, true}, // new: reverse loop (product.created → render) after content
{"ml", true, false}, // was order 130
{"usage", false, false}, // was order 131
{"crm", false, false}, // was order 131
{"marketing", false, true}, // new: marketing domain fold (after crm)
{"ads", false, true}, // new: ads domain fold (after crm)
{"social", false, true}, // new: /v1/social fold (after crm)
{"analytics", true, false}, // was order 132
{"git", false, false}, // was order 132
{"sync", false, true}, // /v1/sync engine (owns per-org DB handles → Shutdown)
{"visor", false, false}, // was order 133
{"captable", false, true}, // was order 133
{"code", false, true}, // was order 134
{"zero-trust", false, false}, // was order 134
{"dataroom", true, true}, // was order 134
{"graph", false, false}, // was order 135
{"security", true, true}, // was order 136
{"integrations", false, true}, // was order 137
{"sbom", true, false}, // was order 137
{"team", false, true}, // was order 138
{"settings", false, true}, // was order 138
{"notify", true, false}, // was order 139
{"gateway", false, false}, // was order 139
{"entitlements", false, true}, // was order 139
{"exec", false, false}, // was order 140
{"websearch", false, false}, // was order 141
{"world", false, true}, // was order 142
{"runtime", false, false}, // was order 143; was "bot" until the transport was named for what it is
{"authors", false, true}, // was order 143
{"bots", false, false}, // was order 143
{"audit", false, false}, // was order 144
{"affiliates", false, false}, // was order 144
{"sign", true, true}, // was order 145
{"product", false, false}, // was order 145
{"evals", false, false}, // was order 145
{"treasury", false, true}, // was order 146
{"admin", false, false}, // was order 146
{"admission", false, true}, // launch-control gate: composes flags (registry+seed+mode route+Enforce); Shutdown closes the registry store
{"tasks", false, false}, // was order 147; platform cron folded in as a sub-mount of tasks.Mount (was a separate entry)
{"automations", false, true}, // was order 148; connectorruntime (POST /v1/automations/connectors/:id/run) folded in as a sub-mount of automations.Mount
{"tools", false, true}, // new: unified tool plane (after automations)
{"marketplace", false, true}, // new: marketplace over the tool plane (after tools)
{"referrals", false, false}, // was order 149
{"guide", false, true}, // new: Business AI Guide (after referrals, before ai)
{"company", false, true}, // new: Hanzo Company formation state machine (after guide)
{"agent", false, false}, // new: /v1/agent tool-calling round (before zen/ai catch-all)
{"zen", false, false}, // zen* claim middleware before ai's catch-all (hip-00NN)
{"ai", false, false}, // was order 150
{"plugins", false, false}, // was order 900
}
// TestWireOrderMatchesFrozen proves the composition root's mount order is
+15
View File
@@ -42,6 +42,7 @@ type idClaims struct {
Owner string `json:"owner"` // org slug (the org)
Project string `json:"project"` // org SUB-SCOPE within owner (empty ⟹ default project)
BillingAccount string `json:"billing_account"` // WHO PAYS, stated by IAM (empty ⟹ pre-claim token)
Name string `json:"name"` // display name (id fallback)
PreferredUsername string `json:"preferred_username"` // id fallback
Email string `json:"email"`
@@ -62,6 +63,20 @@ func (c *idClaims) mintedProject() string {
return strings.TrimSpace(c.Project)
}
// mintedBillingAccount returns the funding account to stamp into
// X-Billing-Account-Id, or "" when the header must be OMITTED (a token minted
// before IAM shipped the claim, or one IAM could not attribute).
//
// WHO PAYS IS NOT A CLIENT'S TO NAME. This rides the validated `billing_account`
// claim — IAM's signed statement, resolved at the identity boundary from the real
// grant context — exactly like `owner` and `project`. It mirrors the edge
// (iamauth.Claims.MintedBillingAccount) byte-for-byte, so the in-binary path binds
// the same header the gateway would, and ai/object.Payer reads the same payer on
// both. The raw client copy is deleted on ingress and NEVER restored.
func (c *idClaims) mintedBillingAccount() string {
return strings.TrimSpace(c.BillingAccount)
}
// userID resolves the canonical user id: sub, then preferred_username, then
// name. IAM may leave sub empty. This is the STABLE identifier (a UUID when IAM
// sets sub) stamped as X-User-Id and consumed as the attribution key everywhere.
+5 -5
View File
@@ -14,7 +14,7 @@ import (
"github.com/hanzoai/cloud/clients"
"github.com/hanzoai/cloud/clients/finance"
"github.com/hanzoai/cloud/clients/gatewaypolicy"
"github.com/hanzoai/cloud/clients/gateway/edge"
"github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/cloud/clients/s3admin"
"github.com/hanzoai/cloud/types"
@@ -107,7 +107,7 @@ func BuildDeps(cfg *Config) Deps {
// working *Store (static-only if the SQLite file can't open), so the edge
// middleware is never left without a policy source — a store-open error is
// logged, not fatal.
gp, err := gatewaypolicy.New(cfg.DataDir, cfg.AdminOrg, staticEdgePolicy(cfg))
gp, err := edge.New(cfg.DataDir, cfg.AdminOrg, staticEdgePolicy(cfg))
if err != nil {
logger.Warn("gateway policy store degraded to static-only", "err", err)
}
@@ -117,10 +117,10 @@ func BuildDeps(cfg *Config) Deps {
}
// staticEdgePolicy projects the static env/flag edge config into the boot-default
// policy the gatewaypolicy.Store layers runtime overrides on top of. A disabled
// policy the edge.Store layers runtime overrides on top of. A disabled
// per-IP limiter (CLOUD_EDGE_RATELIMIT=false) maps to PerIPRPM 0 (a live no-op).
func staticEdgePolicy(cfg *Config) gatewaypolicy.Policy {
p := gatewaypolicy.Policy{
func staticEdgePolicy(cfg *Config) edge.Policy {
p := edge.Policy{
CORSOrigins: cfg.CORSOrigins,
WindowSec: cfg.EdgeRateWindowSec,
}
+52 -1
View File
@@ -54,6 +54,10 @@ const (
heartbeatEvery = 30 * time.Second
claimPoll = 2 * time.Second
claimLeaseSecs = 120
// renderWindow matches the dispatch cap (studio gpu_dispatch sets
// startToCloseTimeout 14400s). The old 10m local poll undercut it and
// marked live renders failed while they kept sampling (observed 8-70m).
renderWindow = 4 * time.Hour
// localComfyUI is the studio render backend the studio.render handler drives.
localComfyUI = "http://127.0.0.1:8188"
// defaultStudioUploadURL is where finished render outputs are POSTed so they
@@ -820,6 +824,14 @@ func (w *worker) studioRenderHandler(ctx context.Context, input json.RawMessage)
return nil, fmt.Errorf("studio.render: input needs a `prompt` graph")
}
cl := &http.Client{Timeout: 60 * time.Second}
// The claim-to-submit window: hold off the supervisor's recycle, and wait out
// one if it is already mid-flight — a claimed job must never die on staging
// because the engine happened to be restarting.
staging.Add(1)
defer staging.Add(-1)
if err := waitEngine(ctx, cl); err != nil {
return nil, fmt.Errorf("studio.render: %w", err)
}
// Materialize any uploaded inputs (they live in orgs/{org}/input on the cloud
// pod, which this worker cannot read) into the LOCAL studio input dir via its
// own /upload/image, so LoadImage resolves them before we render.
@@ -848,7 +860,7 @@ func (w *worker) studioRenderHandler(ctx context.Context, input json.RawMessage)
return nil, fmt.Errorf("studio.render: no prompt_id in /prompt response")
}
// Poll history until the prompt shows up (completed).
deadline := time.Now().Add(10 * time.Minute)
deadline := time.Now().Add(renderWindow)
for time.Now().Before(deadline) {
select {
case <-ctx.Done():
@@ -872,6 +884,11 @@ func (w *worker) studioRenderHandler(ctx context.Context, input json.RawMessage)
if uerr != nil {
return nil, fmt.Errorf("studio.render: prompt %s rendered but gallery upload failed: %w", pr.PromptID, uerr)
}
// The engine leaks ~58GB per render; recycling after each completed
// render caps it at one render's worth. Boot (~20s) is noise next to
// 8-70m renders. Never recycle on the timeout path — the engine may
// still be sampling and the mirror rescues late finishes.
requestStudioRecycle()
return map[string]any{"promptId": pr.PromptID, "outputs": outputs, "gallery": gallery}, nil
}
}
@@ -1014,6 +1031,12 @@ func (w *worker) mirrorRenders(ctx context.Context, out io.Writer, dir, base str
if werr != nil || info == nil || info.IsDir() || !isImageFile(p) {
return nil
}
// Hidden files and AppleDouble forks (`._*`, `.DS_Store`) ride along with
// mac scp and are not renders — `._foo.png` passes the extension check
// but is a 4KB resource fork that poisons the library.
if strings.HasPrefix(filepath.Base(p), ".") {
return nil
}
rel, rerr := filepath.Rel(dir, p)
if rerr != nil {
return nil
@@ -1148,6 +1171,34 @@ func (w *worker) materializeInputs(ctx context.Context, cl *http.Client, inputs
return nil
}
// waitEngine blocks until the local engine answers its /queue — up to 90s, which
// outlasts any supervisor recycle (engine restart is seconds, model reload longer).
func waitEngine(ctx context.Context, cl *http.Client) error {
deadline := time.Now().Add(90 * time.Second)
for {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, localComfyUI+"/queue", nil)
if err != nil {
return err
}
resp, err := cl.Do(req)
if err == nil {
_, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 1<<10))
resp.Body.Close()
if resp.StatusCode/100 == 2 {
return nil
}
}
if time.Now().After(deadline) {
return fmt.Errorf("engine not up: %v", err)
}
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(3 * time.Second):
}
}
}
// collectOutputs pulls the output image/file names out of a ComfyUI history entry.
func collectOutputs(entry json.RawMessage) []string {
var e struct {
+69 -6
View File
@@ -12,6 +12,7 @@ package cli
import (
"context"
"encoding/json"
"fmt"
"io"
"net"
@@ -20,6 +21,7 @@ import (
"os/exec"
"path/filepath"
"strconv"
"sync/atomic"
"syscall"
"time"
)
@@ -107,6 +109,48 @@ func stopStudio(cmd *exec.Cmd) {
}
}
// studioRecycle carries at most one pending recycle request; the render
// handler signals it after each completed render (see gpu.go).
var studioRecycle = make(chan struct{}, 1)
// staging guards the claim-to-submit window: a claimed job is real work the
// engine queue cannot see yet, so the supervisor must never recycle over it
// (observed: jobs claimed during a recycle failed staging on a dead engine
// and were consumed).
var staging atomic.Int32
func requestStudioRecycle() {
select {
case studioRecycle <- struct{}{}:
default:
}
}
// studioBusy reports whether the engine holds queued or running prompts.
// A generous timeout: a saturated GB10 answers slowly mid-render — slow is
// alive, and killing a live render costs 8-70 minutes of GPU work.
func studioBusy(ctx context.Context) (busy, ok bool) {
ctx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://"+studioAddr+"/queue", nil)
if err != nil {
return false, false
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
return false, false
}
defer resp.Body.Close()
var q struct {
Running []json.RawMessage `json:"queue_running"`
Pending []json.RawMessage `json:"queue_pending"`
}
if err := json.NewDecoder(io.LimitReader(resp.Body, 32<<20)).Decode(&q); err != nil {
return false, false
}
return len(q.Running)+len(q.Pending) > 0, true
}
// superviseStudio keeps the local render backend on :8188 alive until ctx
// ends. Quiet by design: one line per restart event, not a probe firehose.
func superviseStudio(ctx context.Context, dir string, out io.Writer) {
@@ -138,6 +182,12 @@ func superviseStudio(ctx context.Context, dir string, out io.Writer) {
tick := time.NewTicker(studioProbeEvery)
defer tick.Stop()
// recyclePending defers the post-render recycle until the queue is EMPTY:
// short jobs complete while a long render is mid-sample, and recycling on
// their completion killed the live render (observed: every direct render
// died within ~6 minutes while probe jobs cycled).
recyclePending := false
unhealthy := 0
for {
select {
case <-ctx.Done():
@@ -145,19 +195,32 @@ func superviseStudio(ctx context.Context, dir string, out io.Writer) {
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGTERM)
}
return
case <-studioRecycle:
recyclePending = true
case <-tick.C:
busy, ok := studioBusy(ctx)
if recyclePending && ok && !busy && staging.Load() == 0 {
recyclePending = false
unhealthy = 0
restart("recycle")
continue
}
if studioHealthy(ctx) {
unhealthy = 0
continue
}
// Grace re-check: it may be momentarily busy mid-render.
select {
case <-ctx.Done():
if ok && busy {
// Alive-busy: slow health under render load is not death.
unhealthy = 0
continue
case <-time.After(studioGraceWait):
}
if !studioHealthy(ctx) {
restart("unresponsive")
// Sustained silence with an idle or unreadable queue = actually dead.
unhealthy++
if unhealthy < 3 {
continue
}
unhealthy = 0
restart("unresponsive")
}
}
}
+7 -5
View File
@@ -39,6 +39,7 @@ import (
"github.com/hanzoai/account"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
)
@@ -277,11 +278,12 @@ func billingData(s *cloud.Service[state], c *zip.Ctx) error {
// Scope EVERY request to the caller's OWN subject — query AND write body — so
// commerce's per-tenant isolation can never be crossed from the browser. The
// subject comes from the ONE rule (ai/object.Payer), keyed on the IAM username
// (cr.username = X-User-Name) the gate also keys on — so a top-up credits the
// SAME account the gate debits. Keying on cr.name (X-User-Id, a UUID on the
// direct-bearer path) would fund an account the gate never reads: the split.
subject := account.Payer(account.Credential{Owner: cr.owner, Name: cr.username}).Subject()
// subject comes from the ONE rule (ai/object.Payer), fed the account the
// credential NAMES (the validated `billing_account` claim) — the same claim the
// ai gate reads, so a top-up credits the SAME account the gate debits. Feeding
// Payer a different credential here than the gate gets is the modern shape of
// the old split: money landing in an account the gate never reads.
subject := account.Payer(account.Credential{Owner: cr.owner, Name: cr.username, Account: principal.BillingAccount(c)}).Subject()
inQuery, _ := url.ParseQuery(string(c.Fiber().Request().URI().QueryString()))
q := scopedBillingSearch(inQuery, subject)
+9 -2
View File
@@ -38,7 +38,10 @@ import (
"github.com/hanzoai/cloud/clients/admin/finance"
"github.com/hanzoai/cloud/clients/admin/health"
"github.com/hanzoai/cloud/clients/admin/iam"
"github.com/hanzoai/cloud/clients/admin/invoices"
"github.com/hanzoai/cloud/clients/admin/metrics"
"github.com/hanzoai/cloud/clients/admin/revenue"
"github.com/hanzoai/cloud/clients/admin/subscriptions"
"github.com/hanzoai/cloud/clients/commerceinproc"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
@@ -111,18 +114,22 @@ func routes(app *zip.App, s *cloud.Service[core.State]) {
app.Get("/v1/admin/flags", core.Guard(s, flagsBoard))
app.Put("/v1/admin/flags/:key", core.Guard(s, setFlag))
// Launch-control services board — the waitlist-mode lens on the flag engine (twin
// of /v1/admin/flags), folded in from the former featuregate control plane.
// of /v1/admin/flags), reading the registry + decide the admission gate owns.
app.Get("/v1/admin/services", core.Guard(s, services))
app.Post("/v1/admin/services", core.Guard(s, upsertService))
app.Post("/v1/admin/services/:service/mode", core.Guard(s, setServiceMode))
app.Get("/v1/admin/waitlist", core.Guard(s, waitlist))
app.Post("/v1/admin/waitlist/boost", core.Guard(s, waitlistBoost))
// ── Carved-out domains own their routes (audit/customer/revenue/finance). ──
// ── Carved-out domains own their routes (audit/customer/revenue/finance +
// the billing fleet views metrics/invoices/subscriptions). ──
audit.Routes(app, s)
customer.Routes(app, s)
revenue.Routes(app, s)
finance.Routes(app, s)
metrics.Routes(app, s)
invoices.Routes(app, s)
subscriptions.Routes(app, s)
}
// ── /v1/admin/me — operator identity (AdminMe) ───────────────────────────────
+231
View File
@@ -295,6 +295,237 @@ func (c *Client) Deposit(ctx context.Context, subject string, amount money.Cents
return out, nil
}
// ── SaaS-metrics god-view (fleet-wide, org-independent) ──────────────────────
// SaaSMetrics mirrors commerce's GET /v1/metrics/saas snapshot — the whole-business
// SaaS-operations aggregate (MRR/ARR, new/churn, plan mix, top customers, recent
// movements) computed IN commerce across every org namespace. It is org-INDEPENDENT
// (like Costs) so the reader sends NO subject. Only the fields the admin god-view
// renders are modeled; commerce fields we don't consume (upgrades/downgrades,
// untagged-request counts) are simply ignored by the decoder.
type SaaSMetrics struct {
AsOf string `json:"asOf"`
Currency string `json:"currency"`
Window string `json:"window"`
Revenue SaaSRevenue `json:"revenue"`
Subs SaaSSubs `json:"subscriptions"`
Usage SaaSUsage `json:"usage"`
Customers []SaaSCustomer `json:"customers"`
Orgs int `json:"orgs"`
Gaps []string `json:"gaps"`
}
// SaaSRevenue is the recurring-revenue headline (run-rate MRR/ARR + windowed movement).
type SaaSRevenue struct {
MRRCents money.Cents `json:"mrrCents"`
ARRCents money.Cents `json:"arrCents"`
ActiveSubscriptions int `json:"activeSubscriptions"`
PayingCustomers int `json:"payingCustomers"`
Trials int `json:"trials"`
NewMRRCents money.Cents `json:"newMrrCents"`
ChurnedMRRCents money.Cents `json:"churnedMrrCents"`
NetNewMRRCents money.Cents `json:"netNewMrrCents"`
ByCategory []SaaSCategory `json:"byCategory"`
}
// SaaSCategory is one plan-category bucket of run-rate MRR (the plan mix).
type SaaSCategory struct {
Category string `json:"category"`
MRRCents money.Cents `json:"mrrCents"`
Subscriptions int `json:"subscriptions"`
}
// SaaSSubs is the subscription-operations panel (per-plan mix, trials, new/canceled,
// recent movements).
type SaaSSubs struct {
ByPlan []SaaSPlan `json:"byPlan"`
TrialsActive int `json:"trialsActive"`
New int `json:"new"`
Canceled int `json:"canceled"`
Recent []SaaSEvent `json:"recent"`
}
// SaaSPlan is one plan's active/trialing counts, seats, and MRR contribution.
type SaaSPlan struct {
Plan string `json:"plan"`
Name string `json:"name"`
Category string `json:"category"`
Active int `json:"active"`
Trialing int `json:"trialing"`
Seats int `json:"seats"`
MRRCents money.Cents `json:"mrrCents"`
}
// SaaSEvent is one recent subscription movement ("created" or "canceled").
type SaaSEvent struct {
At string `json:"at"`
Org string `json:"org"`
Type string `json:"type"`
Plan string `json:"plan"`
Category string `json:"category"`
MRRDeltaCents money.Cents `json:"mrrDeltaCents"`
}
// SaaSUsage is the metered / pay-as-you-go revenue headline for the window.
type SaaSUsage struct {
Instrumented bool `json:"instrumented"`
WindowUsageCents money.Cents `json:"windowUsageCents"`
Requests int64 `json:"requests"`
}
// SaaSCustomer is one top customer by MRR + windowed usage.
type SaaSCustomer struct {
Org string `json:"org"`
Plan string `json:"plan"`
Category string `json:"category"`
Status string `json:"status"`
MRRCents money.Cents `json:"mrrCents"`
UsageCents money.Cents `json:"usageCents"`
Seats int `json:"seats"`
Since string `json:"since,omitempty"`
}
// Metrics reads the fleet SaaS-operations god-view (GET /v1/metrics/saas). Like Costs it
// is org-INDEPENDENT — the engine walks every org namespace itself — so it authenticates
// with the admin S2S service token and sends NO subject. Empty (not an error) when
// commerce is unwired, so a partial deploy degrades to an honest empty snapshot.
func (c *Client) Metrics(ctx context.Context, window string, limit int) (SaaSMetrics, error) {
var out SaaSMetrics
if !c.Ready() {
return out, nil
}
q := url.Values{}
if window != "" {
q.Set("window", window)
}
if limit > 0 {
q.Set("limit", fmt.Sprintf("%d", limit))
}
body, err := c.get(ctx, "/v1/metrics/saas", q, "")
if err != nil {
return out, err
}
if err := json.Unmarshal(body, &out); err != nil {
return out, fmt.Errorf("commerce metrics decode: %w", err)
}
return out, nil
}
// ── billing invoices + subscriptions (per-subject fleet rows) ────────────────
// Invoice is one issued invoice as the fleet god-view renders it: the id (for a future
// /v1/billing/invoices/:id detail fetch), the human number, status, amount due,
// currency, and the issue/due dates. Sourced from GET /v1/billing/invoices
// (invoiceResponse); all timestamps are RFC3339 strings.
type Invoice struct {
ID string `json:"id"`
Number string `json:"numberStr"`
Status string `json:"status"`
AmountDue money.Cents `json:"amountDue"`
Currency string `json:"currency"`
Issued string `json:"createdAt"`
Due string `json:"dueDate"`
}
// Invoices lists a subject's invoices (GET /v1/billing/invoices), optionally filtered by
// status. The subject selects the org's billing namespace via X-Org-Id (trusted only
// after the service-token bearer verifies). Empty (not an error) when commerce is unwired.
func (c *Client) Invoices(ctx context.Context, subject, status string) ([]Invoice, error) {
if !c.Ready() {
return nil, nil
}
q := url.Values{}
if status != "" {
q.Set("status", status)
}
body, err := c.get(ctx, "/v1/billing/invoices", q, subject)
if err != nil {
return nil, err
}
var wrap struct {
Invoices []Invoice `json:"invoices"`
}
if err := json.Unmarshal(body, &wrap); err != nil {
return nil, fmt.Errorf("commerce invoices decode: %w", err)
}
return wrap.Invoices, nil
}
// Subscription is one subscription row the fleet god-view renders: the id, the buyer
// (userId), plan tier, status, monthly-normalized MRR, and the current-period
// start/end (started/renews). MRR reuses monthlyNormalized so a yearly plan is
// comparable to a monthly one in the fleet total.
type Subscription struct {
ID string `json:"id"`
User string `json:"user"`
Plan string `json:"plan"`
Status string `json:"status"`
MRR money.Cents `json:"mrrCents"`
Started string `json:"started"`
Renews string `json:"renews"`
}
// subscriptionRowWire is the /v1/billing/subscriptions row shape the fleet view folds —
// richer than subscriptionsWire (which Plan() uses for the MRR sum alone).
type subscriptionRowWire struct {
ID string `json:"id"`
UserID string `json:"userId"`
PlanID string `json:"planId"`
Status string `json:"status"`
Created string `json:"createdAt"`
PeriodStart string `json:"currentPeriodStart"`
PeriodEnd string `json:"currentPeriodEnd"`
Plan struct {
Name string `json:"name"`
Price money.Cents `json:"price"`
Interval string `json:"interval"`
} `json:"plan"`
}
// Subscriptions lists a subject's subscriptions (GET /v1/billing/subscriptions),
// optionally filtered by status, as fleet rows with a monthly-normalized MRR. Empty (not
// an error) when commerce is unwired.
func (c *Client) Subscriptions(ctx context.Context, subject, status string) ([]Subscription, error) {
if !c.Ready() {
return nil, nil
}
q := url.Values{}
if status != "" {
q.Set("status", status)
}
body, err := c.get(ctx, "/v1/billing/subscriptions", q, subject)
if err != nil {
return nil, err
}
var wrap struct {
Subscriptions []subscriptionRowWire `json:"subscriptions"`
}
if err := json.Unmarshal(body, &wrap); err != nil {
return nil, fmt.Errorf("commerce subscriptions decode: %w", err)
}
out := make([]Subscription, 0, len(wrap.Subscriptions))
for _, s := range wrap.Subscriptions {
name := strings.TrimSpace(s.Plan.Name)
if name == "" {
name = strings.TrimSpace(s.PlanID)
}
started := strings.TrimSpace(s.Created)
if started == "" {
started = s.PeriodStart
}
out = append(out, Subscription{
ID: s.ID,
User: s.UserID,
Plan: name,
Status: s.Status,
MRR: monthlyNormalized(s.Plan.Price, s.Plan.Interval),
Started: started,
Renews: s.PeriodEnd,
})
}
return out, nil
}
// post performs one admin-authenticated commerce POST (JSON body) and returns the
// raw response. The admin S2S service token is the bearer and X-Org-Id=<subject>
// the per-org namespace selector commerce's EdgeAuth trusts only after verifying
+137
View File
@@ -0,0 +1,137 @@
// Package invoices is the fleet INVOICE view (/v1/admin/invoices) — every issued
// invoice across every tenant: number, org, amount, status, issue + due date, plus the
// id a future detail view fetches /v1/billing/invoices/:id with. SuperAdmin only
// (core.Guard).
//
// Commerce billing is per-tenant (an invoice lives in its org's own datastore
// namespace), so — like revenue — this fans out the org directory concurrently and
// reads each org's invoices via the admin S2S seam, tagging every row with its owning
// org. Best-effort per org: an org whose invoice read fails contributes NO rows rather
// than failing the fleet view (the SAME honest-degradation contract the customer list
// uses; an unreachable commerce yields an empty list, never fabricated rows). Optional
// ?org= scopes to one tenant, ?status= filters, ?limit= caps the merged list.
package invoices
import (
"context"
"sort"
"strconv"
"strings"
"sync"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/admin/iam"
"github.com/zap-proto/zip"
)
// defaultLimit caps the merged fleet invoice list when the caller sends none.
const defaultLimit = 500
// InvoiceRow is one row of GET /v1/admin/invoices — an issued invoice at a glance,
// tagged with its owning org. Money is USD cents; timestamps are RFC3339 strings.
type InvoiceRow struct {
ID string `json:"id"`
Number string `json:"number"`
Org string `json:"org"`
Display string `json:"display"`
Status string `json:"status"`
AmountCents int64 `json:"amountCents"`
Currency string `json:"currency"`
Issued string `json:"issued"`
Due string `json:"due"`
}
// Invoices answers GET /v1/admin/invoices.
//
// GET /v1/admin/invoices?org=&status=&limit=
func Invoices(s *cloud.Service[core.State], c *zip.Ctx) error {
ctx := c.Context()
cr := core.CallerCreds(c)
status := strings.TrimSpace(c.Query("status"))
wantOrg := strings.TrimSpace(c.Query("org"))
limit := parseLimit(c.Query("limit"))
orgs, err := core.ListOrgs(s, ctx, cr)
if err != nil {
return core.Fail(c, err.Error())
}
if wantOrg != "" {
orgs = filterOrg(orgs, wantOrg)
}
// Per-org invoices, fanned out concurrently (best-effort per org).
perOrg := make([][]InvoiceRow, len(orgs))
sem := make(chan struct{}, core.MaxCustomerConcurrency)
var wg sync.WaitGroup
for i, o := range orgs {
wg.Add(1)
sem <- struct{}{}
go func(i int, o iam.Org) {
defer wg.Done()
defer func() { <-sem }()
perOrg[i] = invoicesOf(s, ctx, o, status)
}(i, o)
}
wg.Wait()
rows := make([]InvoiceRow, 0)
for _, r := range perOrg {
rows = append(rows, r...)
}
// Newest issued first; cap to the merged limit (total reports the full pre-cap count).
sort.Slice(rows, func(i, j int) bool { return rows[i].Issued > rows[j].Issued })
total := len(rows)
if len(rows) > limit {
rows = rows[:limit]
}
return core.OKList(c, rows, total)
}
// invoicesOf reads one org's invoices into fleet rows, tagged with the org. Best-effort:
// a failed read yields no rows so the fleet view degrades honestly, never fabricating.
func invoicesOf(s *cloud.Service[core.State], ctx context.Context, o iam.Org, status string) []InvoiceRow {
entries, err := s.State.Commerce.Invoices(ctx, o.Name, status)
if err != nil {
return nil
}
display := core.Display(o.DisplayName, o.Name)
rows := make([]InvoiceRow, 0, len(entries))
for _, inv := range entries {
rows = append(rows, InvoiceRow{
ID: inv.ID,
Number: inv.Number,
Org: o.Name,
Display: display,
Status: inv.Status,
AmountCents: int64(inv.AmountDue),
Currency: inv.Currency,
Issued: inv.Issued,
Due: inv.Due,
})
}
return rows
}
// filterOrg narrows the directory to the one requested org (empty when it does not
// exist — an honest empty list, never a fabricated tenant).
func filterOrg(orgs []iam.Org, want string) []iam.Org {
for _, o := range orgs {
if o.Name == want {
return []iam.Org{o}
}
}
return nil
}
// parseLimit clamps the merged-list cap to [1,5000], defaulting to defaultLimit.
func parseLimit(s string) int {
n, err := strconv.Atoi(strings.TrimSpace(s))
if err != nil || n <= 0 {
return defaultLimit
}
if n > 5000 {
return 5000
}
return n
}
+12
View File
@@ -0,0 +1,12 @@
package invoices
import (
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/zap-proto/zip"
)
// Routes registers the fleet invoice view (SuperAdmin only, cross-tenant).
func Routes(app *zip.App, s *cloud.Service[core.State]) {
app.Get("/v1/admin/invoices", core.Guard(s, Invoices))
}
+110
View File
@@ -0,0 +1,110 @@
// Package metrics is the fleet SaaS-operations god-view (/v1/admin/metrics) — the
// operator's business dashboard: MRR/ARR, net-new vs churned MRR, the plan/category
// mix, the top customers, and the recent subscription movements. SuperAdmin only
// (core.Guard).
//
// It OWNS no aggregation. The whole snapshot is computed IN commerce (the system of
// record for subscriptions + the usage ledger) by its cross-org SaaS-metrics engine
// (GET /v1/metrics/saas), which admin PROXIES with the SAME admin-scoped S2S service
// token finance uses for COGS. The engine is ALREADY fleet-wide — it walks every org
// namespace itself — so this is a SINGLE upstream read, no per-org fan-out, exactly as
// finance consumes commerce Costs. An unwired or unreachable commerce degrades to an
// honest empty snapshot (real zeros, `[]` not null) with a not-ok source, never a
// fabricated number.
package metrics
import (
"errors"
"strconv"
"strings"
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/commerce"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/zap-proto/zip"
)
// errUnconfigured marks commerce not wired on this deployment — core.SrcOf reports it as
// a not-ok source so the console renders the honest not-configured state.
var errUnconfigured = errors.New("commerce metrics not configured")
// defaultLimit caps the top-customers list when the caller sends none (mirrors the
// commerce engine's own default so the proxy never asks for more than it returns).
const defaultLimit = 20
// MetricsData is the GET /v1/admin/metrics payload: the commerce SaaS snapshot, flat,
// plus the admin read time and the upstream freshness strip every god-view carries.
type MetricsData struct {
commerce.SaaSMetrics
GeneratedAt string `json:"generatedAt"`
Sources []core.SourceStatus `json:"sources"`
}
// Metrics answers GET /v1/admin/metrics by proxying the commerce SaaS-metrics engine
// (already a fleet-wide cross-org aggregate). SuperAdmin only.
//
// GET /v1/admin/metrics?window=30d&limit=20
func Metrics(s *cloud.Service[core.State], c *zip.Ctx) error {
ctx := c.Context()
now := time.Now().UTC().Format(time.RFC3339)
window := strings.TrimSpace(c.Query("window"))
limit := parseLimit(c.Query("limit"))
if !s.State.Commerce.Ready() {
return core.OK(c, empty(now, window, core.SrcOf("commerce-metrics", errUnconfigured, 0, now)))
}
m, err := s.State.Commerce.Metrics(ctx, window, limit)
if err != nil {
return core.OK(c, empty(now, window, core.SrcOf("commerce-metrics", err, 0, now)))
}
return core.OK(c, MetricsData{
SaaSMetrics: normalize(m),
GeneratedAt: now,
Sources: []core.SourceStatus{core.SrcOf("commerce-metrics", nil, m.Orgs, now)},
})
}
// empty is the honest not-configured/unreachable snapshot: real zeros + empty slices
// (never null, never fabricated) plus the not-ok source.
func empty(now, window string, src core.SourceStatus) MetricsData {
return MetricsData{
SaaSMetrics: normalize(commerce.SaaSMetrics{AsOf: now, Currency: "usd", Window: window}),
GeneratedAt: now,
Sources: []core.SourceStatus{src},
}
}
// normalize replaces nil slices with empty ones so the JSON is honest arrays (`[]`, not
// null) and the console never has to guard a missing collection.
func normalize(m commerce.SaaSMetrics) commerce.SaaSMetrics {
if m.Revenue.ByCategory == nil {
m.Revenue.ByCategory = []commerce.SaaSCategory{}
}
if m.Subs.ByPlan == nil {
m.Subs.ByPlan = []commerce.SaaSPlan{}
}
if m.Subs.Recent == nil {
m.Subs.Recent = []commerce.SaaSEvent{}
}
if m.Customers == nil {
m.Customers = []commerce.SaaSCustomer{}
}
if m.Gaps == nil {
m.Gaps = []string{}
}
return m
}
// parseLimit clamps the top-N cap to [1,200], defaulting to defaultLimit — mirrors the
// commerce engine's clamp exactly.
func parseLimit(s string) int {
n, err := strconv.Atoi(strings.TrimSpace(s))
if err != nil || n <= 0 {
return defaultLimit
}
if n > 200 {
return 200
}
return n
}
+13
View File
@@ -0,0 +1,13 @@
package metrics
import (
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/zap-proto/zip"
)
// Routes registers the SaaS-metrics god-view (SuperAdmin only, cross-tenant business
// aggregate).
func Routes(app *zip.App, s *cloud.Service[core.State]) {
app.Get("/v1/admin/metrics", core.Guard(s, Metrics))
}
+15 -15
View File
@@ -1,16 +1,16 @@
package admin
// The /v1/admin/services board — the launch-control LENS on the ONE flag engine, twin
// The /v1/admin/services board — the launch-control LENS over the waitlist gate, twin
// of /v1/admin/flags. Every hosted service (studio/chat/console/app/api/team + runtime
// onboards) with its LIVE waitlist mode — the switch waitlist.<svc> evaluated through
// clients/flags. This is the "remove the waitlist one service at a time" toggle.
// SuperAdmin only (core.Guard), like every platform /v1/admin/*.
// onboards) with its LIVE waitlist mode — the switch waitlist.<svc>, evaluated through
// clients/admission (which composes the flag engine one-way). This is the "remove the
// waitlist one service at a time" toggle. SuperAdmin only (core.Guard), like every
// platform /v1/admin/*.
//
// Formerly clients/featuregate owned its OWN SQLite mode store + this control plane;
// both folded onto the flag engine so the platform has ONE decision plane. featuregate
// now owns only the native Enforce middleware — a consumer of flags.WaitlistModeForHost.
// Per-user approval (the second, orthogonal axis) stays IAM's, reached via the existing
// admin IAM proxy — not re-served here.
// The registry + mode decide + these admin control funcs live in clients/admission,
// the complete launch-gate feature; flags is the pure engine underneath. Per-user
// approval (the second, orthogonal axis) stays IAM's, reached via the existing admin IAM
// proxy — not re-served here.
import (
"errors"
@@ -19,13 +19,13 @@ import (
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/flags"
"github.com/hanzoai/cloud/clients/admission"
"github.com/zap-proto/zip"
)
// services answers GET /v1/admin/services — the launch board (every service + live mode).
func services(s *cloud.Service[core.State], c *zip.Ctx) error {
rows, err := flags.ListWaitlistServices(c.Context())
rows, err := admission.ListWaitlistServices(c.Context())
if err != nil {
return zip.Errorf(http.StatusInternalServerError, "list services: %v", err)
}
@@ -35,14 +35,14 @@ func services(s *cloud.Service[core.State], c *zip.Ctx) error {
// upsertService answers POST /v1/admin/services — onboard or edit a hosted service so a
// new host is governed WITHOUT a redeploy. A re-register PRESERVES the live switch.
func upsertService(s *cloud.Service[core.State], c *zip.Ctx) error {
var in flags.ServiceInput
var in admission.ServiceInput
if err := c.Bind(&in); err != nil {
return err
}
if strings.TrimSpace(in.Service) == "" {
return zip.ErrBadRequest("service slug is required")
}
view, err := flags.UpsertWaitlistService(c.Context(), in, c.UserEmail())
view, err := admission.UpsertWaitlistService(c.Context(), in, c.UserEmail())
if err != nil {
return zip.ErrBadRequest(err.Error())
}
@@ -62,9 +62,9 @@ func setServiceMode(s *cloud.Service[core.State], c *zip.Ctx) error {
if err := c.Bind(&body); err != nil {
return err
}
view, err := flags.SetWaitlistMode(c.Context(), service, body.WaitlistMode, c.UserEmail())
view, err := admission.SetWaitlistMode(c.Context(), service, body.WaitlistMode, c.UserEmail())
if err != nil {
if errors.Is(err, flags.ErrServiceNotFound) {
if errors.Is(err, admission.ErrServiceNotFound) {
return zip.ErrNotFound("service not found: " + service)
}
return zip.Errorf(http.StatusInternalServerError, "set mode: %v", err)
+12
View File
@@ -0,0 +1,12 @@
package subscriptions
import (
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/zap-proto/zip"
)
// Routes registers the fleet subscription view (SuperAdmin only, cross-tenant).
func Routes(app *zip.App, s *cloud.Service[core.State]) {
app.Get("/v1/admin/subscriptions", core.Guard(s, Subscriptions))
}
@@ -0,0 +1,139 @@
// Package subscriptions is the fleet SUBSCRIPTION view (/v1/admin/subscriptions) —
// every tenant's plan subscription: customer/org, plan, status, monthly-normalized MRR,
// and the current-period start/renews. SuperAdmin only (core.Guard).
//
// Like invoices (and revenue) it fans out the org directory concurrently and reads each
// org's subscriptions via the admin S2S seam, tagging every row with its owning org. The
// MRR is monthly-normalized in the commerce reader so a yearly plan is comparable to a
// monthly one. Best-effort per org (a failed read contributes no rows, never fabricated
// ones); optional ?org= scopes to one tenant, ?status= filters, ?limit= caps.
package subscriptions
import (
"context"
"sort"
"strconv"
"strings"
"sync"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/admin/iam"
"github.com/zap-proto/zip"
)
// defaultLimit caps the merged fleet subscription list when the caller sends none.
const defaultLimit = 500
// SubscriptionRow is one row of GET /v1/admin/subscriptions — a tenant's subscription at
// a glance, tagged with its owning org. MRR is USD cents; timestamps are RFC3339 strings.
type SubscriptionRow struct {
ID string `json:"id"`
Org string `json:"org"`
Display string `json:"display"`
User string `json:"user"`
Plan string `json:"plan"`
Status string `json:"status"`
MRRCents int64 `json:"mrrCents"`
Started string `json:"started"`
Renews string `json:"renews"`
}
// Subscriptions answers GET /v1/admin/subscriptions.
//
// GET /v1/admin/subscriptions?org=&status=&limit=
func Subscriptions(s *cloud.Service[core.State], c *zip.Ctx) error {
ctx := c.Context()
cr := core.CallerCreds(c)
status := strings.TrimSpace(c.Query("status"))
wantOrg := strings.TrimSpace(c.Query("org"))
limit := parseLimit(c.Query("limit"))
orgs, err := core.ListOrgs(s, ctx, cr)
if err != nil {
return core.Fail(c, err.Error())
}
if wantOrg != "" {
orgs = filterOrg(orgs, wantOrg)
}
// Per-org subscriptions, fanned out concurrently (best-effort per org).
perOrg := make([][]SubscriptionRow, len(orgs))
sem := make(chan struct{}, core.MaxCustomerConcurrency)
var wg sync.WaitGroup
for i, o := range orgs {
wg.Add(1)
sem <- struct{}{}
go func(i int, o iam.Org) {
defer wg.Done()
defer func() { <-sem }()
perOrg[i] = subscriptionsOf(s, ctx, o, status)
}(i, o)
}
wg.Wait()
rows := make([]SubscriptionRow, 0)
for _, r := range perOrg {
rows = append(rows, r...)
}
// Highest-MRR first (ties broken by most-recent start); cap to the merged limit.
sort.Slice(rows, func(i, j int) bool {
if rows[i].MRRCents != rows[j].MRRCents {
return rows[i].MRRCents > rows[j].MRRCents
}
return rows[i].Started > rows[j].Started
})
total := len(rows)
if len(rows) > limit {
rows = rows[:limit]
}
return core.OKList(c, rows, total)
}
// subscriptionsOf reads one org's subscriptions into fleet rows, tagged with the org.
// Best-effort: a failed read yields no rows so the fleet view degrades honestly.
func subscriptionsOf(s *cloud.Service[core.State], ctx context.Context, o iam.Org, status string) []SubscriptionRow {
entries, err := s.State.Commerce.Subscriptions(ctx, o.Name, status)
if err != nil {
return nil
}
display := core.Display(o.DisplayName, o.Name)
rows := make([]SubscriptionRow, 0, len(entries))
for _, sub := range entries {
rows = append(rows, SubscriptionRow{
ID: sub.ID,
Org: o.Name,
Display: display,
User: sub.User,
Plan: sub.Plan,
Status: sub.Status,
MRRCents: int64(sub.MRR),
Started: sub.Started,
Renews: sub.Renews,
})
}
return rows
}
// filterOrg narrows the directory to the one requested org (empty when it does not
// exist — an honest empty list, never a fabricated tenant).
func filterOrg(orgs []iam.Org, want string) []iam.Org {
for _, o := range orgs {
if o.Name == want {
return []iam.Org{o}
}
}
return nil
}
// parseLimit clamps the merged-list cap to [1,5000], defaulting to defaultLimit.
func parseLimit(s string) int {
n, err := strconv.Atoi(strings.TrimSpace(s))
if err != nil || n <= 0 {
return defaultLimit
}
if n > 5000 {
return 5000
}
return n
}
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package featuregate
package admission
import (
"context"
@@ -31,7 +31,7 @@ import (
// approval is FAIL-OPEN: a user is approved unless properties.approvalStatus is
// EXACTLY "pending" (absent / "approved" / "rejected" all read approved via
// IsApproved). Only "pending" holds a user on the waitlist. Keeping the literal
// here (not importing IAM) keeps featuregate self-contained.
// here (not importing IAM) keeps admission self-contained.
const approvalStatusPending = "pending"
// approvedHeader is the FORWARD-PERFECT path: once IAM carries approvalStatus in
@@ -1,7 +1,7 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
// Licensed under the Apache License, Version 2.0.
package featuregate
package admission
import (
"context"
@@ -12,16 +12,24 @@
// See the License for the specific language governing permissions and
// limitations under the License.
// Package featuregate is the launch-control ENFORCEMENT for Hanzo's hosted services:
// the native middleware (Enforce) + the per-user approval predicate (Approvals, reused
// from IAM). It is a CONSUMER of the ONE policy engine — the per-service waitlist MODE
// and the host→service registry live in clients/flags (a service's mode IS the
// switch waitlist.<svc>, evaluated through the native engine); the admin board is the
// /v1/admin/services lens and the guard's runtime mode read is /v1/featuregate/mode,
// both served there. This package owns only enforcement, decomplected into two axes:
// Package admission is the launch-control GATE for Hanzo's hosted services — the
// COMPLETE waitlist feature, COMPOSING the ONE flag engine (clients/flags) one-way. It
// owns:
//
// - PER-SERVICE waitlist mode on|off — the flags switch waitlist.<svc>,
// resolved for a request host via flags.WaitlistModeForHost (the decide).
// - the host→service registry (registry.go) + the brand seed (waitlist.go),
// - the per-service MODE decide WaitlistModeForHost — a service's mode IS the switch
// waitlist.<svc>, evaluated through the flag engine (flags.Bool),
// - the admin control funcs (List/Set/Upsert) the /v1/admin/services board calls,
// - the guard's public mode read /v1/flags/waitlist, Mount,
// - the native enforcement middleware (Enforce, this file),
// - the per-user approval predicate (Approvals, reused from IAM — approval.go).
//
// flags NEVER imports admission; admission imports flags. The engine is the pure
// (Principal, context) -> verdict primitive; this package is its first composed tenant.
// Enforcement is decomplected into two orthogonal axes:
//
// - PER-SERVICE waitlist mode on|off — the switch waitlist.<svc>, resolved for a
// request host via WaitlistModeForHost (the decide, waitlist.go).
// - PER-USER approvalStatus pending|approved — owned by IAM (approval.go), REUSED.
//
// THE RULE, applied at ONE native enforcement point (Enforce):
@@ -29,14 +37,13 @@
// if waitlistMode[host] AND NOT user.approved → bounce to the waitlist
// if approved OR mode=off → allow
// unauthenticated → login first
package featuregate
package admission
import (
"context"
"net/http"
"strings"
"github.com/hanzoai/cloud/clients/flags"
"github.com/zap-proto/zip"
)
@@ -59,13 +66,13 @@ import (
// INTEGRATION POINT — wire in serve.go RIGHT AFTER SanitizeIdentity:
//
// app.Use(IdentityMiddleware(cfg)) // establishes the validated principal
// app.Use(featuregate.Enforce(featuregate.EnforceConfig{ WaitlistURL: … })) // ← here
// app.Use(admission.Enforce(admission.EnforceConfig{ WaitlistURL: … })) // ← here
//
// It reads the sanitized X-User-Id / X-User-IsAdmin / X-User-Approved that
// IdentityMiddleware minted, so it MUST run after it and (like BillingGate) before
// the subsystem handlers. It is deliberately NOT wired here — the unified-binary
// agent owns serve.go's boot chain; this package exposes Enforce so the one-line
// app.Use lands without a merge collision. The decide (flags.WaitlistModeForHost) is
// app.Use lands without a merge collision. The decide (WaitlistModeForHost) is
// resolved PER REQUEST and fail-opens until the flags engine has mounted, so Enforce
// can be constructed before Mount runs.
//
@@ -102,7 +109,7 @@ type EnforceConfig struct {
ExemptPrefixes []string
// Gate is THE decide: it resolves whether a request host is in waitlist mode,
// via the ONE policy engine. When nil it is flags.WaitlistModeForHost —
// via the ONE policy engine. When nil it is WaitlistModeForHost —
// host→service→waitlist.<svc>. Injected only in tests. Fail-open by contract:
// known=false (unmounted / registry error / un-governed host) → not gated.
Gate func(ctx context.Context, host string) (mode bool, service string, known bool)
@@ -112,9 +119,9 @@ type EnforceConfig struct {
// health, the auth/OIDC handshake, and the waitlist join API itself (so a gated
// user can still submit the waitlist form).
var defaultExemptPrefixes = []string{
"/v1/featuregate/", // the mode read + the health route
"/v1/iam/", // auth / OIDC / approval-status / get-account handshake
"/v1/waitlist", // the waitlist join API (a gated user must reach it)
"/v1/flags/waitlist", // the guard's public mode read (flags engine)
"/v1/iam/", // auth / OIDC / approval-status / get-account handshake
"/v1/waitlist", // the waitlist join API (a gated user must reach it)
"/health",
"/healthz",
"/__guard/", // the @file guard's own callback surface (defense in depth)
@@ -131,7 +138,7 @@ func Enforce(cfg EnforceConfig) zip.Handler {
}
gate := cfg.Gate
if gate == nil {
gate = flags.WaitlistModeForHost // the ONE decide: host→service→waitlist.<svc>
gate = WaitlistModeForHost // the ONE decide: host→service→waitlist.<svc>
}
exempt := cfg.ExemptPrefixes
if len(exempt) == 0 {
@@ -199,7 +206,7 @@ func bounce(c *zip.Ctx, waitlistURL string) error {
}
// apiKeyPrefixes are the Hanzo API-key prefixes. This MIRRORS cloud
// auth_identity.go isAPIKey (the ONE authority) — kept local so featuregate stays
// auth_identity.go isAPIKey (the ONE authority) — kept local so admission stays
// self-contained (no cloud-internal import) while agreeing on the exact contract:
// a token with one of these prefixes is a possession-gated API key, not a session
// principal. If cloud adds a prefix there, add it here.
@@ -1,7 +1,7 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
// Licensed under the Apache License, Version 2.0.
package featuregate
package admission
import (
"context"
@@ -16,7 +16,7 @@ import (
// testGate is the injected decide (the flags engine's WaitlistModeForHost seam):
// hanzo.chat is gated, api.hanzo.ai is open, everything else is un-governed. This is
// exactly what flags.WaitlistModeForHost returns for the equivalent registry, without
// exactly what WaitlistModeForHost returns for the equivalent registry, without
// standing up the native flag engine (cgo) in a middleware unit test.
func testGate(_ context.Context, host string) (mode bool, service string, known bool) {
switch host {
@@ -186,7 +186,7 @@ func TestRule_UngovernedHost_PassesThrough(t *testing.T) {
func TestRule_ExemptPaths_NeverGated(t *testing.T) {
app := gateApp(t, "pending")
for _, p := range []string{"/health", "/v1/iam/get-account", "/v1/waitlist/join", "/v1/featuregate/mode"} {
for _, p := range []string{"/health", "/v1/iam/get-account", "/v1/waitlist/join", "/v1/flags/waitlist"} {
code, _ := drive(t, app, greq{host: "hanzo.chat", path: p, user: "u", org: "acme", accept: html})
if code != 200 {
t.Fatalf("exempt path %q = %d, want 200 (never gated)", p, code)
@@ -205,7 +205,7 @@ func TestRule_ForwardHeaderApproved_ThroughWithoutLookup(t *testing.T) {
}
}
// The DEFAULT gate (nil Gate → flags.WaitlistModeForHost) fail-opens before the flags
// The DEFAULT gate (nil Gate → WaitlistModeForHost) fail-opens before the flag
// engine has mounted: with no engine, WaitlistModeForHost returns known=false for every
// host, so Enforce never gates pre-boot.
func TestEnforce_DefaultGate_FailsOpenPreBoot(t *testing.T) {
@@ -1,11 +1,11 @@
package flags
package admission
// The waitlist REGISTRY — the host→service map + service metadata folded in from
// the former clients/featuregate SQLite store. It is deliberately MODE-FREE: a
// service's waitlist mode is NOT a column here, it is the platform switch
// waitlist.<svc> evaluated through the ONE native engine (waitlist.go). This store
// answers only "which service owns this host, and what is its display metadata" —
// the config the decide needs, with the decision itself owned by the flag engine.
// The launch-registry — the host→service map + service display metadata. It is
// deliberately MODE-FREE: a service's waitlist mode is NOT a column here, it is the
// platform switch waitlist.<svc> evaluated through the ONE flag engine (clients/flags,
// composed one-way from waitlist.go). This store answers only "which service owns this
// host, and what is its display metadata" — the config the decide needs, with the
// decision itself owned by the flag engine.
//
// It rides the SAME per-(org,project) OrgDB machinery as the flag defs (opened via
// cloud.OrgStore, encrypted at rest via cek); the registry is PLATFORM-global, so it
@@ -21,7 +21,7 @@ import (
)
// ErrServiceNotFound is returned when a service slug is not in the registry.
var ErrServiceNotFound = errors.New("flags: waitlist service not found")
var ErrServiceNotFound = errors.New("admission: waitlist service not found")
// ServiceRow is one hosted service in the registry (host→service + metadata). The
// waitlist MODE is intentionally absent — it is the platform switch waitlist.<svc>,
@@ -45,7 +45,7 @@ type waitlistStore struct {
}
// openWaitlistStore migrates the registry schema over an already-opened (pragma'd,
// cek-wrapped) OrgDB handle — the same open contract as openStore for flag defs.
// cek-wrapped) OrgDB handle — the same open contract as flags' openStore for flag defs.
func openWaitlistStore(db *sql.DB) (*waitlistStore, error) {
const schema = `
CREATE TABLE IF NOT EXISTS wl_services (
@@ -64,7 +64,7 @@ CREATE TABLE IF NOT EXISTS wl_hosts (
CREATE INDEX IF NOT EXISTS ix_wl_hosts_service ON wl_hosts(service);
`
if _, err := db.Exec(schema); err != nil {
return nil, fmt.Errorf("flags: waitlist migrate: %w", err)
return nil, fmt.Errorf("admission: waitlist migrate: %w", err)
}
return &waitlistStore{db: db}, nil
}
@@ -210,7 +210,7 @@ func (s *waitlistStore) Get(ctx context.Context, service string) (ServiceRow, er
func (s *waitlistStore) Upsert(ctx context.Context, in ServiceRow, by string, now int64) (ServiceRow, error) {
svc := strings.ToLower(strings.TrimSpace(in.Service))
if svc == "" {
return ServiceRow{}, fmt.Errorf("flags: waitlist service slug required")
return ServiceRow{}, fmt.Errorf("admission: waitlist service slug required")
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
@@ -1,9 +1,9 @@
package flags
package admission
// Registry coverage for the folded host→service store. It drives the store over a raw
// sqlite handle (the same driver OrgDB uses), so it exercises the fold WITHOUT the cek
// Registry coverage for the host→service store. It drives the store over a raw sqlite
// handle (the same driver OrgDB uses), so it exercises the registry WITHOUT the cek
// at-rest layer — runnable under CGO=0. The MODE is out of scope here by design (it is
// the waitlist.<svc> switch, evaluated by the native engine, covered separately).
// the waitlist.<svc> switch, evaluated by the flag engine, covered separately).
import (
"context"
@@ -1,19 +1,23 @@
package flags
package admission
// The waitlist LENS on the ONE flag engine — the launch-control plane folded in from
// the former clients/featuregate. Decomplected into the two orthogonal axes it always
// was, now with a single decision plane:
// The launch-control gate — the COMPLETE waitlist feature, COMPOSING the ONE flag
// engine (clients/flags) one-way. Decomplected into the two orthogonal axes it always
// was, with a single decision plane:
//
// - MODE (per service): waitlist.<svc> IS a platform switch, evaluated through the
// SAME native engine as every other platform flag. There is no second mode store.
// - HOST MAP + metadata: the registry (waitlist_store.go) resolves a request host
// to the service whose switch governs it, and carries display metadata.
// flag engine (flags.Bool / flags.SetPlatformSwitch / flags.Register). There is no
// second mode store.
// - HOST MAP + metadata: the registry (registry.go) resolves a request host to the
// service whose switch governs it, and carries display metadata.
//
// The decide is WaitlistModeForHost(host) → (mode, service, known): resolve host→svc,
// then read waitlist.<svc>. featuregate.Enforce is now a CONSUMER of this decide, and
// /v1/featuregate/mode + the /v1/admin/services board read it too. Per-user approval
// (pending|approved) stays IAM's (featuregate/approval.go) — the second, orthogonal
// axis, unchanged.
// then read waitlist.<svc>. Enforce (middleware.go) consumes this decide; the admin
// board (/v1/admin/services) and the guard's runtime mode read (/v1/flags/waitlist,
// served here) read it too. Per-user approval (pending|approved) is the second,
// orthogonal axis — IAM's, in approval.go.
//
// flags NEVER imports this package; this package imports flags. That one-way arrow is
// the whole point of the decomplection: the engine is pure, the feature composes it.
import (
"context"
@@ -25,10 +29,29 @@ import (
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/flags"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
// The reserved platform tenant the launch registry rides in — the SAME reserved
// (org, project) the flag engine uses for its platform switches, so the registry and
// the waitlist.<svc> switches co-locate. One waitlist.db for the deployment.
const (
platformOrg = "platform"
platformProject = "platform"
)
// registryState is admission's process-wide launch state: the platform-tenant
// host→service registry store + the deployment brand it was seeded for. Installed by
// Mount, torn down by Shutdown.
type registryState struct {
store *cloud.OrgStore[*waitlistStore]
brand string
}
var mounted *registryState
// SeedService is one row of the launch registry (a hosted service + its hosts). Mode
// is intentionally absent — the launch posture (gated) is waitlistDef's Default "true".
type SeedService struct {
@@ -60,17 +83,17 @@ func waitlistKey(svc string) string { return "waitlist." + strings.ToLower(strin
// waitlistDef is the platform switch for one service's mode. Default "true" = the
// launch posture (gated until an admin opens it), so a deployment with no stored flag
// behaves exactly as the old featuregate seed (waitlistMode ON).
func waitlistDef(svc, display string) Def {
// behaves exactly as the old admission seed (waitlistMode ON).
func waitlistDef(svc, display string) flags.Def {
if strings.TrimSpace(display) == "" {
display = svc
}
return Def{
return flags.Def{
Key: waitlistKey(svc),
Category: "Launch",
Label: "Waitlist · " + display,
Desc: "Waitlist mode for " + display + ": ON gates the service to APPROVED users; OFF opens it.",
Type: TypeBool,
Type: flags.TypeBool,
Default: "true",
}
}
@@ -78,9 +101,13 @@ func waitlistDef(svc, display string) Def {
// ensureWaitlistDef registers a service's switch if it is not already registered
// (Mount registers the seed set with nicer labels; this covers runtime onboards).
func ensureWaitlistDef(svc, display string) {
if _, ok := lookupDef(waitlistKey(svc)); !ok {
Register(waitlistDef(svc, display))
key := waitlistKey(svc)
for _, d := range flags.Defs() {
if d.Key == key {
return
}
}
flags.Register(waitlistDef(svc, display))
}
// boolDef is the minimal PostHog flag definition for a boolean switch value.
@@ -92,26 +119,24 @@ func boolDef(on bool) json.RawMessage {
}
// requireRegistry resolves the platform-tenant registry store, or an error when the
// engine is not mounted (writes need it; the decide fail-opens instead).
// gate is not mounted (writes need it; the decide fail-opens instead).
func requireRegistry() (*waitlistStore, error) {
c := mounted
if c == nil || c.registry == nil {
return nil, fmt.Errorf("flags: waitlist registry not mounted")
if mounted == nil || mounted.store == nil {
return nil, fmt.Errorf("admission: waitlist registry not mounted")
}
return c.registry.For(platformOrg, platformProject)
return mounted.store.For(platformOrg, platformProject)
}
// WaitlistModeForHost is THE decide the Enforce consumer, /v1/featuregate/mode, and
// WaitlistModeForHost is THE decide the Enforce consumer, /v1/flags/waitlist, and
// the admin board call: resolve host→service, then read the waitlist.<svc> switch
// through the engine. FAIL-OPEN by construction — an unmounted registry, a store
// through the flag engine. FAIL-OPEN by construction — an unmounted registry, a store
// error, or an un-governed host all return known=false, so a request is NEVER gated
// pre-boot or on a registry fault (availability over a hard gate, matching the guard).
func WaitlistModeForHost(ctx context.Context, host string) (mode bool, service string, known bool) {
c := mounted
if c == nil || c.registry == nil {
if mounted == nil || mounted.store == nil {
return false, "", false
}
st, err := c.registry.For(platformOrg, platformProject)
st, err := mounted.store.For(platformOrg, platformProject)
if err != nil {
return false, "", false
}
@@ -119,7 +144,7 @@ func WaitlistModeForHost(ctx context.Context, host string) (mode bool, service s
if err != nil || !known {
return false, "", false
}
return Bool(waitlistKey(svc)), svc, true
return flags.Bool(waitlistKey(svc)), svc, true
}
// ListWaitlistServices returns the admin board: every registered service with its LIVE
@@ -135,19 +160,19 @@ func ListWaitlistServices(ctx context.Context) ([]ServiceView, error) {
}
out := make([]ServiceView, 0, len(rows))
for _, r := range rows {
out = append(out, ServiceView{ServiceRow: r, WaitlistMode: Bool(waitlistKey(r.Service))})
out = append(out, ServiceView{ServiceRow: r, WaitlistMode: flags.Bool(waitlistKey(r.Service))})
}
return out, nil
}
// SetWaitlistMode flips one service's waitlist switch — the launch lever — and returns
// the updated view. It is the ONE write path (through SetPlatformSwitch, audited in the
// flag activity log); the flip is hot (this pod applies immediately, peers converge
// within the eval TTL). ErrServiceNotFound when the slug is unknown.
// the updated view. It is the ONE write path (through flags.SetPlatformSwitch, audited
// in the flag activity log); the flip is hot (this pod applies immediately, peers
// converge within the eval TTL). ErrServiceNotFound when the slug is unknown.
func SetWaitlistMode(ctx context.Context, service string, mode bool, actor string) (ServiceView, error) {
service = strings.ToLower(strings.TrimSpace(service))
if service == "" {
return ServiceView{}, fmt.Errorf("flags: service is required")
return ServiceView{}, fmt.Errorf("admission: service is required")
}
st, err := requireRegistry()
if err != nil {
@@ -158,10 +183,10 @@ func SetWaitlistMode(ctx context.Context, service string, mode bool, actor strin
return ServiceView{}, err
}
ensureWaitlistDef(service, row.DisplayName)
if err := SetPlatformSwitch(waitlistKey(service), boolDef(mode), actor); err != nil {
if err := flags.SetPlatformSwitch(waitlistKey(service), boolDef(mode), actor); err != nil {
return ServiceView{}, err
}
return ServiceView{ServiceRow: row, WaitlistMode: Bool(waitlistKey(service))}, nil
return ServiceView{ServiceRow: row, WaitlistMode: flags.Bool(waitlistKey(service))}, nil
}
// UpsertWaitlistService onboards or edits a hosted service so a new host is governed
@@ -170,7 +195,7 @@ func SetWaitlistMode(ctx context.Context, service string, mode bool, actor strin
func UpsertWaitlistService(ctx context.Context, in ServiceInput, actor string) (ServiceView, error) {
svc := strings.ToLower(strings.TrimSpace(in.Service))
if svc == "" {
return ServiceView{}, fmt.Errorf("flags: service slug is required")
return ServiceView{}, fmt.Errorf("admission: service slug is required")
}
st, err := requireRegistry()
if err != nil {
@@ -192,43 +217,44 @@ func UpsertWaitlistService(ctx context.Context, in ServiceInput, actor string) (
}
ensureWaitlistDef(svc, row.DisplayName)
if isNew {
if err := SetPlatformSwitch(waitlistKey(svc), boolDef(in.WaitlistMode), actor); err != nil {
if err := flags.SetPlatformSwitch(waitlistKey(svc), boolDef(in.WaitlistMode), actor); err != nil {
return ServiceView{}, err
}
}
return ServiceView{ServiceRow: row, WaitlistMode: Bool(waitlistKey(svc))}, nil
return ServiceView{ServiceRow: row, WaitlistMode: flags.Bool(waitlistKey(svc))}, nil
}
// mountWaitlist seeds the registry and registers a waitlist.<svc> switch per known
// service. Best-effort + fail-safe: a registry error (e.g. cek master key not yet
// injected) degrades to the in-memory seed switches — the decide then fail-opens,
// exactly the flag engine's own boot posture. Called from Mount.
func mountWaitlist(c *Client, brand string, log luxlog.Logger) {
// seedRegistry seeds the registry and registers a waitlist.<svc> switch per known
// service, COMPOSING the flag engine (flags.Register). Best-effort + fail-safe: a
// registry error (e.g. cek master key not yet injected) degrades to the in-memory seed
// switches — the decide then fail-opens, exactly the flag engine's own boot posture.
// Returns the number of seeded services (for the mount log). Called from Mount.
func seedRegistry(brand string, log luxlog.Logger) int {
seed := seedWaitlist(brand)
for _, sv := range seed { // in-memory switches — always succeeds
Register(waitlistDef(sv.Service, sv.DisplayName))
flags.Register(waitlistDef(sv.Service, sv.DisplayName))
}
st, err := c.registry.For(platformOrg, platformProject)
st, err := mounted.store.For(platformOrg, platformProject)
if err != nil {
log.Warn("waitlist registry unavailable — modes degrade to seed defaults", "err", err)
return
return len(seed)
}
if _, err := st.Seed(context.Background(), seed, time.Now().Unix()); err != nil {
log.Warn("waitlist registry seed failed", "err", err)
return
return len(seed)
}
if rows, err := st.List(context.Background()); err == nil {
for _, r := range rows { // register any persisted onboard beyond the seed
ensureWaitlistDef(r.Service, r.DisplayName)
}
}
return len(seed)
}
// waitlistModeRoute answers GET /v1/featuregate/mode?host=<h> — the runtime lookup the
// waitlistModeRoute answers GET /v1/flags/waitlist?host=<h> — the runtime lookup the
// @file waitlist-guard caches. Public (in-cluster) read: it returns ONLY the boolean
// mode for the ONE queried host, never an enumeration. Same wire shape as the former
// featuregate route, so the interim guard ports 1:1.
func waitlistModeRoute(_ *cloud.Service[state], c *zip.Ctx) error {
// mode for the ONE queried host, never an enumeration.
func waitlistModeRoute(c *zip.Ctx) error {
host := strings.TrimSpace(c.Query("host"))
if host == "" {
host = c.Fiber().Hostname()
@@ -242,7 +268,44 @@ func waitlistModeRoute(_ *cloud.Service[state], c *zip.Ctx) error {
})
}
// ── brand seed (moved verbatim from the former featuregate/seed.go) ──────────────
// ── lifecycle ────────────────────────────────────────────────────────────────
// Mount installs the launch-control gate: it opens the platform-tenant host→service
// registry, seeds it for the deployment brand, registers a waitlist.<svc> switch per
// service in the flag engine (flags.Register), and serves the guard's public mode read
// at /v1/flags/waitlist. Fail-safe: a registry error (e.g. cek master key not yet
// injected) degrades to the in-memory seed switches — WaitlistModeForHost then
// fail-opens. Mounts AFTER flags so the engine's platform-switch plane is installed first.
func Mount(app *zip.App, deps cloud.Deps) error {
if deps.Logger == nil {
return fmt.Errorf("admission.Mount: nil deps.Logger")
}
if deps.DataDir == "" {
return fmt.Errorf("admission.Mount: empty deps.DataDir")
}
log := deps.Logger.New("subsystem", "admission")
mounted = &registryState{
store: cloud.NewOrgStore[*waitlistStore](deps.DataDir, "waitlist", openWaitlistStore),
brand: deps.Brand,
}
n := seedRegistry(deps.Brand, log)
// The guard's public runtime mode read (host→service→waitlist.<svc>), one namespace
// under /v1/flags. Exempt from the Enforce gate (see defaultExemptPrefixes) so a
// gated user can still resolve mode.
app.Get("/v1/flags/waitlist", waitlistModeRoute)
log.Info("admission gate ready", "services", n)
return nil
}
// Shutdown closes the launch registry's per-org store handles.
func Shutdown() error {
if mounted == nil || mounted.store == nil {
return nil
}
return mounted.store.CloseAll()
}
// ── brand seed (moved verbatim from the former flags/waitlist.go) ────────────────
// seedWaitlist returns the launch registry for a brand. White-labeled so a Lux/Zoo/Pars
// deployment governs its OWN hosts. New hosted services onboard at runtime via
+27 -136
View File
@@ -1,153 +1,44 @@
package affiliates
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
"github.com/hanzoai/cloud/clients/commerceinproc"
"github.com/hanzoai/cloud/clients/payout"
)
// commerce is the narrow money seam the affiliate loop needs: read a referred
// org's metered spend (the accrual base) and grant a promo credit to a wallet (a
// payout made in credits). It is an INTERFACE so the store/handler logic is
// testable with a fake ledger the HTTP impl below is the ONE production binding.
// commerce is the narrow money seam the affiliate loop needs: read a referred org's
// metered spend (the commission accrual base) and grant a promo credit to a wallet
// (a payout made in credits, ledger tag grant:affiliate). It is an INTERFACE so the
// store/handler logic is testable with a fake ledger; the production binding is
// clients/payout, reached through the thin adapter below.
//
// This mirrors clients/referrals/commerce.go EXACTLY (which itself mirrors
// clients/admin/commerce.go): the same COMMERCE_SERVICE_TOKEN S2S path, the same
// X-Org-Id=<org> namespace + bare org `user` subject that admin.grantCredit uses —
// so an affiliate payout-in-credits lands in precisely the wallet the balance
// panel reads, indistinguishable from an admin grant except by its ledger tag
// (grant:affiliate vs grant:referral / grant:admin, all → the commerce Credit/trial
// bucket per DepositKind's grant:* rule).
// The S2S impl (COMMERCE_SERVICE_TOKEN path, X-Org-Id=<org> namespace, bare-org
// `user` subject) was three byte-identical commerce.go copies; it now lives ONCE in
// clients/payout. An affiliate payout-in-credits still lands in precisely the wallet
// the balance panel reads, indistinguishable from an admin grant except by its
// grant:affiliate tag.
type commerce interface {
configured() bool
// deposit grants amountCents to org's wallet (Credit/trial bucket via the
// grant:affiliate tag) and returns the ledger transaction id.
deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (txnID string, err error)
// spendCents is a referred org's month-to-date metered consumption — the
// commission accrual base (spend × the affiliate's rate).
spendCents(ctx context.Context, org, user string) (int64, error)
}
// errUnconfigured is returned by a deposit against an unwired commerce so the
// caller records an honest failure rather than reporting a phantom payout.
var errUnconfigured = errors.New("affiliates: commerce endpoint not configured")
// errUnconfigured is the shared sentinel a deposit against an unwired commerce
// returns, so the caller records an honest failure rather than a phantom payout.
var errUnconfigured = payout.ErrUnconfigured
// httpCommerce is the production commerce binding (COMMERCE_SERVICE_TOKEN S2S).
type httpCommerce struct {
base string
token string
http *http.Client
// commerceSeam adapts the shared payout.Client onto this program's lowercase seam
// (Go package-scoped interface methods cannot cross packages). Zero logic — pure
// delegation; the money path lives in clients/payout.
type commerceSeam struct{ c *payout.Client }
func (s commerceSeam) configured() bool { return s.c.Configured() }
func (s commerceSeam) deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (string, error) {
return s.c.Deposit(ctx, org, user, amountCents, currency, notes, tags)
}
func (s commerceSeam) spendCents(ctx context.Context, org, user string) (int64, error) {
return s.c.SpendCents(ctx, org, user)
}
func newCommerceClient(base, token string) *httpCommerce {
return &httpCommerce{
base: strings.TrimRight(strings.TrimSpace(base), "/"),
token: strings.TrimSpace(token),
http: commerceinproc.Client(15 * time.Second),
}
}
func (c *httpCommerce) configured() bool { return c != nil && c.base != "" && c.token != "" }
// deposit posts POST /v1/billing/deposit — the ONE money-in primitive (identical
// to admin.commerceClient.deposit). Commerce's EdgeAuth pins the body `user` to
// the X-Org-Id subject, so a payout can never be mis-targeted to another wallet.
func (c *httpCommerce) deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (string, error) {
if !c.configured() {
return "", errUnconfigured
}
if currency == "" {
currency = "usd"
}
body, err := json.Marshal(map[string]any{
"user": user,
"currency": currency,
"amount": amountCents,
"notes": notes,
"tags": tags,
})
if err != nil {
return "", err
}
raw, err := c.do(ctx, http.MethodPost, "/v1/billing/deposit", nil, org, body)
if err != nil {
return "", err
}
var out struct {
TransactionID string `json:"transactionId"`
}
if err := json.Unmarshal(raw, &out); err != nil {
return "", fmt.Errorf("commerce deposit decode: %w", err)
}
return out.TransactionID, nil
}
// spendCents reads GET /v1/billing/usage-rollup and returns consumedCents. Zero
// (not an error) when commerce is unconfigured so a partial deploy degrades to
// "no spend to accrue yet" rather than a 5xx.
func (c *httpCommerce) spendCents(ctx context.Context, org, user string) (int64, error) {
if !c.configured() {
return 0, nil
}
q := url.Values{"user": {user}}
raw, err := c.do(ctx, http.MethodGet, "/v1/billing/usage-rollup", q, org, nil)
if err != nil {
return 0, err
}
var out struct {
ConsumedCents int64 `json:"consumedCents"`
}
if err := json.Unmarshal(raw, &out); err != nil {
return 0, fmt.Errorf("commerce rollup decode: %w", err)
}
return out.ConsumedCents, nil
}
// do performs one admin-S2S commerce request. X-Org-Id=<org> is the per-org
// namespace selector commerce's EdgeAuth trusts only behind the service token.
func (c *httpCommerce) do(ctx context.Context, method, path string, q url.Values, org string, body []byte) ([]byte, error) {
u := c.base + path
if enc := q.Encode(); enc != "" {
u += "?" + enc
}
var r io.Reader
if body != nil {
r = bytes.NewReader(body)
}
req, err := http.NewRequestWithContext(ctx, method, u, r)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json")
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
if c.token != "" {
req.Header.Set("Authorization", "Bearer "+c.token)
}
if org != "" {
req.Header.Set("X-Org-Id", org)
}
resp, err := c.http.Do(req)
if err != nil {
return nil, fmt.Errorf("commerce unreachable: %w", err)
}
defer func() { _ = resp.Body.Close() }()
out, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return nil, err
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("commerce status %d", resp.StatusCode)
}
return out, nil
}
// newCommerceClient builds the production binding, delegating to clients/payout.
func newCommerceClient(base, token string) commerce { return commerceSeam{payout.NewClient(base, token)} }
+1 -1
View File
@@ -22,7 +22,7 @@ import (
hz "github.com/hanzoai/agent"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/tools"
openai "github.com/sashabaranov/go-openai"
openai "github.com/hanzoai/go-openai"
fiber "github.com/zap-proto/fiber/v3"
"github.com/zap-proto/zip"
)
+137 -4
View File
@@ -10,6 +10,8 @@ import (
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/principal"
"github.com/hanzoai/cloud/clients/samples"
"github.com/zap-proto/zip"
)
@@ -17,10 +19,11 @@ import (
// box, a GPU host, or a whole cluster. It is the #48 link-a-compute seam over the
// SAME agents.db (one store, one tenancy column) as sessions/events — NOT a rival
// device registry. It composes with the compute fleet rather than duplicating it: a
// session records the target id it runs on (agent_sessions.target), and the mission-
// control devices view unions these registered targets with the org's BYO workers
// (GET /v1/fleet/workers) and BYO clusters (GET /v1/clusters) at the view layer — the
// console's established pattern for folding compute sources.
// session records the target id it runs on (agent_sessions.target), and the org's
// unified board (GET /v1/fleet, clients/visor/board.go) unions these registered
// targets with its BYO workers (GET /v1/fleet/workers), BYO clusters and Visor
// machines — reading this registry through the in-process seam below rather than
// copying it.
//
// POST /v1/agents/targets register a target -> Target
// GET /v1/agents/targets list the org's targets (+ live session load)
@@ -30,6 +33,12 @@ import (
//
// Every route is org-scoped through principal.Org (tenant), fail-closed — a tenant
// can never see or mutate another org's targets, exactly like sessions.
//
// A write carrying `metrics` IS a heartbeat, and a heartbeat is two facts, not one:
// the LAST sample (kept on the row, rendered by the views here) and one point in a
// utilization SERIES (appended to clients/samples). The row answers "is this machine
// alive and what is it doing now"; the series answers "how hot has it been". The
// append is best-effort and detached — see recordSample.
// Target kinds — the closed vocabulary of dispatch destinations.
const (
@@ -233,6 +242,48 @@ func (s *Store) GetTargetByHost(ctx context.Context, org, host string) (Target,
return t, nil
}
// ---- the in-process seam (org-scoped, fail-closed) ----
//
// TargetsForOrg / LoadOn are the exported twins of the list + detail reads above:
// the ONE way another in-process subsystem (the /v1/fleet board in clients/visor)
// reads this registry WITHOUT an HTTP hop back through the gateway — the same
// shape ListForOrg gives the agent registry. They are two ORTHOGONAL values on
// purpose: a target is what the machine IS, its load is what is running on it, and
// a caller that only needs the inventory does not pay for the rollups.
//
// ISOLATION: org is the ONLY tenant key and is threaded verbatim into the
// org-scoped store methods, so a caller for org A can never enumerate or resolve
// org B's targets. The caller MUST pass an org it already validated server-side
// (principal.Org), never a raw client header.
// TargetsForOrg returns the org's registered run-targets from the in-process
// store, newest first. Fails closed when the subsystem is not mounted or the org
// is empty/oversized.
func TargetsForOrg(ctx context.Context, org string) ([]Target, error) {
if mounted == nil || mounted.State.store == nil {
return nil, fmt.Errorf("agents: not mounted")
}
org = strings.TrimSpace(org)
if org == "" || len(org) > principal.MaxOrgLen {
return nil, fmt.Errorf("agents: invalid org")
}
return mounted.State.store.ListTargets(ctx, org)
}
// LoadOn returns the live session load on one of the org's targets — the same
// (target id OR host) mapping the HTTP views use, so the board and /v1/agents/
// targets can never disagree about what is running where.
func LoadOn(ctx context.Context, org, id, host string) (TargetLoad, error) {
if mounted == nil || mounted.State.store == nil {
return TargetLoad{}, fmt.Errorf("agents: not mounted")
}
org = strings.TrimSpace(org)
if org == "" || len(org) > principal.MaxOrgLen {
return TargetLoad{}, fmt.Errorf("agents: invalid org")
}
return mounted.State.store.SessionLoad(ctx, org, id, host)
}
// DeleteTarget removes an org's target. Sessions keep their recorded target id (a
// historical fact); a detached target simply stops appearing in the registry.
func (s *Store) DeleteTarget(ctx context.Context, org, id string) (bool, error) {
@@ -262,6 +313,83 @@ func (s *Store) SessionLoad(ctx context.Context, org, id, host string) (TargetLo
return TargetLoad{Sessions: total, Running: running}, nil
}
// ---- the fleet time series ----
//
// A heartbeat is the ONE moment this process learns what a linked machine is
// doing, so it is also where the fleet's utilization series is fed. The target row
// keeps the LAST sample (the snapshot the views render, unchanged); clients/samples
// keeps every sample over time. Two different questions — "is it alive now" and
// "how hot has it been" — so two homes, one write.
// sampleTimeout bounds the warehouse write. Generous (the insert is one small row
// in-cluster) but finite, so a wedged datastore can never hold the goroutine open.
const sampleTimeout = 5 * time.Second
// sampleOf projects a target's server-stamped heartbeat into a fleet sample. PURE
// (no clock, no I/O, no store) so the whole projection is unit-testable and the
// caller decides when it runs.
//
// cost_cents is 0: an agent run-target is the operator's OWN machine (a laptop, a
// dialed-in box) — the fleet meters its utilization, it does not resell it. A
// priced source (visor/cloud) fills that column from its own resale price.
func sampleOf(t Target) samples.Sample {
var model string
if len(t.Spec.GPUs) > 0 {
// The representative accelerator: the count already rides in GPUs, so the
// first card's model names the row. A heterogeneous host is rare enough
// that naming its first card beats inventing a summary string here.
model = t.Spec.GPUs[0].Model
if model == "" {
model = t.Spec.GPUs[0].Vendor
}
}
return samples.Sample{
Org: t.Org,
Source: samples.SourceAgent,
Unit: t.ID,
Host: t.Host,
Kind: t.Kind,
At: time.Unix(t.MetricsAt, 0).UTC(),
CPUs: t.Spec.CPUs,
Memory: t.Spec.Memory,
MemUsed: t.Metrics.MemUsed,
MemFree: t.Metrics.MemFree,
Load1: t.Metrics.Load1,
Load5: t.Metrics.Load5,
Load15: t.Metrics.Load15,
GPUUtil: t.Metrics.GPUUtil,
GPUs: len(t.Spec.GPUs),
GPUModel: model,
}
}
// recordSample appends a heartbeat to the fleet series. Best-effort and DETACHED
// on purpose — the warehouse is never in the heartbeat's critical path:
//
// - it runs on its own bounded context, so neither a slow datastore nor the
// client hanging up mid-request can stall or cancel the write;
// - it never touches the response, so the /v1/agents/targets contract is
// byte-identical whether the warehouse is present, absent or on fire;
// - a failure is logged, never surfaced — a dropped sample must not cost a
// machine its heartbeat.
//
// This is the shape the billing warehouse write already uses (`go zapWriteUsage`):
// the seam is synchronous, the CALLER owns the concurrency.
func recordSample(s *cloud.Service[state], t Target) {
if t.MetricsAt == 0 {
return // no heartbeat in this write — nothing to append
}
sample := sampleOf(t) // project on the caller's goroutine: t must not escape mutably
go func() {
ctx, cancel := context.WithTimeout(context.Background(), sampleTimeout)
defer cancel()
if err := samples.Record(ctx, sample); err != nil {
s.Log.Warn("fleet sample write failed", "org", sample.Org, "unit", sample.Unit, "err", err)
}
}()
}
// ---- HTTP shapes (the published contract) ----
type targetView struct {
@@ -386,6 +514,7 @@ func registerTarget(s *cloud.Service[state], c *zip.Ctx) error {
if err := s.State.store.UpdateTarget(c.Context(), existing); err != nil {
return zip.Errorf(http.StatusInternalServerError, "persist: %v", err)
}
recordSample(s, existing) // a re-link carrying metrics IS a heartbeat
load, _ := s.State.store.SessionLoad(c.Context(), org, existing.ID, existing.Host)
return c.JSON(http.StatusOK, toTargetView(existing, load))
}
@@ -403,6 +532,7 @@ func registerTarget(s *cloud.Service[state], c *zip.Ctx) error {
if err := s.State.store.CreateTarget(c.Context(), t); err != nil {
return zip.Errorf(http.StatusInternalServerError, "persist: %v", err)
}
recordSample(s, t) // a registration carrying metrics is the target's first sample
return c.JSON(http.StatusCreated, toTargetView(t, TargetLoad{}))
}
@@ -538,6 +668,9 @@ func patchTarget(s *cloud.Service[state], c *zip.Ctx) error {
}
return zip.Errorf(http.StatusInternalServerError, "update: %v", err)
}
if body.Metrics != nil {
recordSample(s, t) // THE heartbeat: append it to the fleet series too
}
load, _ := s.State.store.SessionLoad(c.Context(), org, t.ID, t.Host)
return c.JSON(http.StatusOK, toTargetView(t, load))
}
+227
View File
@@ -0,0 +1,227 @@
package agents
import (
"encoding/json"
"net/http"
"testing"
"time"
"github.com/hanzoai/cloud/clients/samples"
)
// targetsample_test.go covers the FIRST emitter: a run-target heartbeat also
// appends to the fleet series (clients/samples).
//
// The datastore is absent under test, so samples.Record is a proven no-op (its own
// package tests that). What MUST be proven here is everything this side owns:
// the projection is faithful, the vocabularies agree, and the HTTP contract is
// untouched whether or not the warehouse exists.
// ---- the projection (pure) ----
// A heartbeat projects onto a sample with no loss and no invention.
func TestSampleOfProjectsTheHeartbeat(t *testing.T) {
at := time.Now().Unix()
tg := Target{
ID: "tgt-1", Org: "acme", Kind: TargetGPU, Host: "box.local", Label: "Box",
Spec: Spec{OS: "linux", Arch: "arm64", CPUs: 20, Memory: 128 << 30,
GPUs: []GPU{{Vendor: "nvidia", Model: "GB10", Memory: 96 << 30}}},
Metrics: Metrics{Load1: 2.5, Load5: 2, Load15: 1.5,
MemUsed: 64 << 30, MemFree: 64 << 30, GPUUtil: 0.75},
MetricsAt: at,
}
s := sampleOf(tg)
if s.Org != "acme" || s.Unit != "tgt-1" || s.Host != "box.local" {
t.Fatalf("identity did not project: %+v", s)
}
if s.Source != samples.SourceAgent {
t.Fatalf("source want %q, got %q", samples.SourceAgent, s.Source)
}
if s.Kind != TargetGPU {
t.Fatalf("kind want %q, got %q", TargetGPU, s.Kind)
}
if !s.At.Equal(time.Unix(at, 0).UTC()) {
t.Fatalf("at must be the SERVER-stamped heartbeat clock, got %v", s.At)
}
if s.CPUs != 20 || s.Memory != 128<<30 {
t.Fatalf("spec did not project: %+v", s)
}
if s.MemUsed != 64<<30 || s.MemFree != 64<<30 || s.Load1 != 2.5 || s.Load5 != 2 || s.Load15 != 1.5 {
t.Fatalf("metrics did not project: %+v", s)
}
if s.GPUUtil != 0.75 || s.GPUs != 1 || s.GPUModel != "GB10" {
t.Fatalf("gpu did not project: %+v", s)
}
// An agent's own machine is metered, never resold.
if s.CostCents != 0 {
t.Fatalf("an agent sample must be unpriced, got %d", s.CostCents)
}
// The projection must be acceptable to the plane it feeds.
if err := samples.Record(t.Context(), s); err != nil {
t.Fatalf("a projected sample must be recordable: %v", err)
}
}
// The accelerator count comes from the spec, and the row is named by the first
// card's model — falling back to its vendor when the model is unknown.
func TestSampleOfGPUSummary(t *testing.T) {
cases := []struct {
name string
gpus []GPU
wantN int
wantModel string
}{
{"none", nil, 0, ""},
{"model", []GPU{{Vendor: "nvidia", Model: "GB10"}}, 1, "GB10"},
{"vendor fallback", []GPU{{Vendor: "amd"}}, 1, "amd"},
{"multi is counted, first names it", []GPU{{Model: "GB10"}, {Model: "GB10"}}, 2, "GB10"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
s := sampleOf(Target{ID: "t", Org: "o", Kind: TargetGPU, Spec: Spec{GPUs: tc.gpus}, MetricsAt: 1})
if s.GPUs != tc.wantN || s.GPUModel != tc.wantModel {
t.Fatalf("want (%d, %q), got (%d, %q)", tc.wantN, tc.wantModel, s.GPUs, s.GPUModel)
}
})
}
}
// THE cross-package contract: every kind a target can be must be a kind the fleet
// series accepts, or heartbeats would silently stop being recorded. This fails the
// day someone adds a target kind without teaching the series about it.
func TestEveryTargetKindIsAFleetKind(t *testing.T) {
fleet := map[string]bool{
samples.KindLaptop: true, samples.KindCloud: true, samples.KindGPU: true,
samples.KindCluster: true, samples.KindMachine: true, samples.KindWorker: true,
}
for _, k := range []string{TargetLaptop, TargetCloud, TargetGPU, TargetCluster, TargetMachine} {
if !fleet[k] {
t.Fatalf("target kind %q is not a fleet sample kind — its heartbeats would be dropped", k)
}
// Proven end to end: a sample carrying this kind validates.
s := sampleOf(Target{ID: "t", Org: "o", Kind: k, MetricsAt: 1})
if err := samples.Record(t.Context(), s); err != nil {
t.Fatalf("kind %q must be recordable: %v", k, err)
}
}
}
// A write with no heartbeat in it appends nothing — recordSample is a no-op when
// the server never stamped a metrics clock.
func TestRecordSampleSkipsWhenNoHeartbeat(t *testing.T) {
mountApp(t, nil) // sets the `mounted` singleton recordSample logs through
// No panic, no goroutine, no write: MetricsAt == 0 means "no sample here".
recordSample(mounted, Target{ID: "tgt-1", Org: "acme", Kind: TargetGPU, MetricsAt: 0})
}
// ---- (c) the HTTP contract is untouched by the series ----
// The heartbeat still 200s with no warehouse, and still returns the snapshot on
// the row exactly as before — the series is strictly additive.
func TestHeartbeatStill200sWithoutDatastore(t *testing.T) {
app := mountApp(t, nil)
code, body := do(t, app, http.MethodPost, "/v1/agents/targets", "acme", map[string]any{
"label": "Box", "kind": TargetGPU, "host": "box.local",
"spec": map[string]any{"os": "linux", "cpus": 20, "gpus": []map[string]any{{"vendor": "nvidia", "model": "GB10"}}},
"metrics": map[string]any{"load1": 2.5, "gpuUtil": 0.75, "memUsed": 100},
})
if code != http.StatusCreated {
t.Fatalf("register want 201 without a datastore, got %d (%s)", code, body)
}
var created targetView
if err := json.Unmarshal(body, &created); err != nil {
t.Fatalf("shape: %v (%s)", err, body)
}
if created.Metrics == nil || created.Metrics.GPUUtil != 0.75 {
t.Fatalf("the snapshot on the row must be unchanged: %+v", created.Metrics)
}
if created.MetricsAt == "" {
t.Fatal("the server must still stamp the heartbeat clock")
}
// The heartbeat itself.
code, body = do(t, app, http.MethodPatch, "/v1/agents/targets/"+created.ID, "acme", map[string]any{
"metrics": map[string]any{"load1": 4, "gpuUtil": 0.9, "memUsed": 200},
})
if code != http.StatusOK {
t.Fatalf("heartbeat want 200 without a datastore, got %d (%s)", code, body)
}
var beat targetView
if err := json.Unmarshal(body, &beat); err != nil {
t.Fatalf("shape: %v (%s)", err, body)
}
if beat.Metrics == nil || beat.Metrics.GPUUtil != 0.9 || beat.Metrics.Load1 != 4 {
t.Fatalf("the heartbeat must still refresh the row snapshot: %+v", beat.Metrics)
}
// A re-link (same org+host) is idempotent and still carries a heartbeat.
code, body = do(t, app, http.MethodPost, "/v1/agents/targets", "acme", map[string]any{
"label": "Box", "kind": TargetGPU, "host": "box.local",
"metrics": map[string]any{"load1": 1},
})
if code != http.StatusOK {
t.Fatalf("re-link want 200 (idempotent), got %d (%s)", code, body)
}
var relinked targetView
if err := json.Unmarshal(body, &relinked); err != nil {
t.Fatalf("shape: %v (%s)", err, body)
}
if relinked.ID != created.ID {
t.Fatalf("a re-link must refresh the SAME target: %s != %s", relinked.ID, created.ID)
}
}
// ---- the in-process seam ----
// TargetsForOrg / LoadOn are org-keyed and fail closed — the board reads through
// them, so a cross-tenant id must never resolve.
func TestInProcessSeamIsOrgScopedAndFailsClosed(t *testing.T) {
app := mountApp(t, nil)
code, body := do(t, app, http.MethodPost, "/v1/agents/targets", "acme", map[string]any{
"label": "Secret", "kind": TargetGPU, "host": "secret.local",
})
if code != http.StatusCreated {
t.Fatalf("register: %d (%s)", code, body)
}
var created targetView
_ = json.Unmarshal(body, &created)
// The owner sees it.
own, err := TargetsForOrg(t.Context(), "acme")
if err != nil {
t.Fatalf("TargetsForOrg(acme): %v", err)
}
if len(own) != 1 || own[0].ID != created.ID {
t.Fatalf("the owner must see its target, got %+v", own)
}
// Another tenant sees nothing — the same id is unreachable.
other, err := TargetsForOrg(t.Context(), "other")
if err != nil {
t.Fatalf("TargetsForOrg(other): %v", err)
}
if len(other) != 0 {
t.Fatalf("CROSS-TENANT LEAK: org 'other' enumerated %+v", other)
}
// A blank/oversized org fails closed on both.
for _, bad := range []string{"", " "} {
if _, err := TargetsForOrg(t.Context(), bad); err == nil {
t.Fatalf("TargetsForOrg(%q) must fail closed", bad)
}
if _, err := LoadOn(t.Context(), bad, created.ID, ""); err == nil {
t.Fatalf("LoadOn(%q) must fail closed", bad)
}
}
// LoadOn is org-keyed too: the foreign tenant resolves no load for the id.
load, err := LoadOn(t.Context(), "other", created.ID, "secret.local")
if err != nil {
t.Fatalf("LoadOn(other): %v", err)
}
if load.Sessions != 0 || load.Running != 0 {
t.Fatalf("CROSS-TENANT LEAK: foreign load %+v", load)
}
}
+1 -1
View File
@@ -11,7 +11,7 @@ import (
"strings"
"time"
openai "github.com/sashabaranov/go-openai"
openai "github.com/hanzoai/go-openai"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/codes"
+26 -134
View File
@@ -1,152 +1,44 @@
package authors
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
"github.com/hanzoai/cloud/clients/commerceinproc"
"github.com/hanzoai/cloud/clients/payout"
)
// commerce is the narrow money seam the author royalty loop needs: read a deploying
// org's metered spend (the accrual base) and grant a promo credit to a wallet (a
// payout made in credits). It is an INTERFACE so the store/handler logic is testable
// with a fake ledger the HTTP impl below is the ONE production binding.
// org's metered spend (the royalty accrual base) and grant a promo credit to a wallet
// (a payout made in credits, ledger tag grant:author). It is an INTERFACE so the
// store/handler logic is testable with a fake ledger; the production binding is
// clients/payout, reached through the thin adapter below.
//
// This mirrors clients/affiliates/commerce.go EXACTLY: the same
// COMMERCE_SERVICE_TOKEN S2S path, the same X-Org-Id=<org> namespace + bare org
// `user` subject — so an author payout-in-credits lands in precisely the wallet the
// balance panel reads, indistinguishable from an admin/affiliate grant except by its
// ledger tag (grant:author, → the commerce Credit/trial bucket per DepositKind's
// grant:* rule).
// The S2S impl (COMMERCE_SERVICE_TOKEN path, X-Org-Id=<org> namespace, bare-org
// `user` subject) was three byte-identical commerce.go copies; it now lives ONCE in
// clients/payout. An author payout-in-credits still lands in precisely the wallet the
// balance panel reads, indistinguishable from an admin grant except by its
// grant:author tag.
type commerce interface {
configured() bool
// deposit grants amountCents to org's wallet (Credit/trial bucket via the
// grant:author tag) and returns the ledger transaction id.
deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (txnID string, err error)
// spendCents is a deploying org's month-to-date metered consumption — the royalty
// accrual base (spend × the author's share).
spendCents(ctx context.Context, org, user string) (int64, error)
}
// errUnconfigured is returned by a deposit against an unwired commerce so the caller
// records an honest failure rather than reporting a phantom payout.
var errUnconfigured = errors.New("authors: commerce endpoint not configured")
// errUnconfigured is the shared sentinel a deposit against an unwired commerce
// returns, so the caller records an honest failure rather than a phantom payout.
var errUnconfigured = payout.ErrUnconfigured
// httpCommerce is the production commerce binding (COMMERCE_SERVICE_TOKEN S2S).
type httpCommerce struct {
base string
token string
http *http.Client
// commerceSeam adapts the shared payout.Client onto this program's lowercase seam
// (Go package-scoped interface methods cannot cross packages). Zero logic — pure
// delegation; the money path lives in clients/payout.
type commerceSeam struct{ c *payout.Client }
func (s commerceSeam) configured() bool { return s.c.Configured() }
func (s commerceSeam) deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (string, error) {
return s.c.Deposit(ctx, org, user, amountCents, currency, notes, tags)
}
func (s commerceSeam) spendCents(ctx context.Context, org, user string) (int64, error) {
return s.c.SpendCents(ctx, org, user)
}
func newCommerceClient(base, token string) *httpCommerce {
return &httpCommerce{
base: strings.TrimRight(strings.TrimSpace(base), "/"),
token: strings.TrimSpace(token),
http: commerceinproc.Client(15 * time.Second),
}
}
func (c *httpCommerce) configured() bool { return c != nil && c.base != "" && c.token != "" }
// deposit posts POST /v1/billing/deposit — the ONE money-in primitive (identical to
// affiliates.httpCommerce.deposit). Commerce's EdgeAuth pins the body `user` to the
// X-Org-Id subject, so a payout can never be mis-targeted to another wallet.
func (c *httpCommerce) deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (string, error) {
if !c.configured() {
return "", errUnconfigured
}
if currency == "" {
currency = "usd"
}
body, err := json.Marshal(map[string]any{
"user": user,
"currency": currency,
"amount": amountCents,
"notes": notes,
"tags": tags,
})
if err != nil {
return "", err
}
raw, err := c.do(ctx, http.MethodPost, "/v1/billing/deposit", nil, org, body)
if err != nil {
return "", err
}
var out struct {
TransactionID string `json:"transactionId"`
}
if err := json.Unmarshal(raw, &out); err != nil {
return "", fmt.Errorf("commerce deposit decode: %w", err)
}
return out.TransactionID, nil
}
// spendCents reads GET /v1/billing/usage-rollup and returns consumedCents. Zero (not
// an error) when commerce is unconfigured so a partial deploy degrades to "no spend
// to accrue yet" rather than a 5xx.
func (c *httpCommerce) spendCents(ctx context.Context, org, user string) (int64, error) {
if !c.configured() {
return 0, nil
}
q := url.Values{"user": {user}}
raw, err := c.do(ctx, http.MethodGet, "/v1/billing/usage-rollup", q, org, nil)
if err != nil {
return 0, err
}
var out struct {
ConsumedCents int64 `json:"consumedCents"`
}
if err := json.Unmarshal(raw, &out); err != nil {
return 0, fmt.Errorf("commerce rollup decode: %w", err)
}
return out.ConsumedCents, nil
}
// do performs one admin-S2S commerce request. X-Org-Id=<org> is the per-org
// namespace selector commerce's EdgeAuth trusts only behind the service token.
func (c *httpCommerce) do(ctx context.Context, method, path string, q url.Values, org string, body []byte) ([]byte, error) {
u := c.base + path
if enc := q.Encode(); enc != "" {
u += "?" + enc
}
var r io.Reader
if body != nil {
r = bytes.NewReader(body)
}
req, err := http.NewRequestWithContext(ctx, method, u, r)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json")
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
if c.token != "" {
req.Header.Set("Authorization", "Bearer "+c.token)
}
if org != "" {
req.Header.Set("X-Org-Id", org)
}
resp, err := c.http.Do(req)
if err != nil {
return nil, fmt.Errorf("commerce unreachable: %w", err)
}
defer func() { _ = resp.Body.Close() }()
out, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return nil, err
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("commerce status %d", resp.StatusCode)
}
return out, nil
}
// newCommerceClient builds the production binding, delegating to clients/payout.
func newCommerceClient(base, token string) commerce { return commerceSeam{payout.NewClient(base, token)} }
+10
View File
@@ -43,6 +43,7 @@ import (
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/audit"
"github.com/hanzoai/cloud/clients/connectorruntime"
"github.com/hanzoai/cloud/clients/principal"
"github.com/hanzoai/cloud/clients/tools"
"github.com/zap-proto/zip"
@@ -138,6 +139,15 @@ func Mount(app *zip.App, deps cloud.Deps) error {
tools.Register(connectorToolProvider{})
b.Log.Info("automations mounted", "connectors", catalog.ConnectorCount, "runtime", len(registry), "brand", deps.Brand)
// Native single-connector execution (HIP-0126): POST /v1/automations/connectors/:id/run,
// the in-process goja runner paired with the connector catalogue above. It mounts one
// route DISTINCT from every automations route (no /v1/automations/* wildcard here, so no
// shadow), and was a separate Wire entry purely for that one route — fold it in as a
// terminal sub-mount so connector catalogue + execution are ONE automations subsystem.
if err := connectorruntime.Mount(app, deps); err != nil {
return err
}
return nil
}
+1 -1
View File
@@ -15,7 +15,7 @@
// LANE 2 — managed Base hosting (what superbase/PocketHost provided). ONE Base
// app PER ORG, opened lazily and pooled, each on its OWN SQLite under
// {DataDir}/base/{orgSegment}/ — the same "prod = SQLite per tenant" model
// (HIP-0302) the gojabase leaves (captable/sign/dataroom) use, so an org's
// (HIP-0302) the NewBase leaves (captable/sign/dataroom) use, so an org's
// collections/records are PHYSICALLY isolated. Served AUTHENTICATED under
// /v1/base/*, the org resolved from the VALIDATED cloud principal (never a
// client header). This is the console Bases manager's backend.
+3 -3
View File
@@ -18,7 +18,7 @@ import (
baseapp "github.com/hanzoai/base"
"github.com/hanzoai/base/core"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/gojabase"
"github.com/hanzoai/cloud/clients/goja"
luxlog "github.com/luxfi/log"
fiber "github.com/zap-proto/fiber/v3"
"github.com/zap-proto/zip"
@@ -170,8 +170,8 @@ func TestPerOrgIsolatedCRUD(t *testing.T) {
}
// The isolation is physical: distinct on-disk data dirs per org segment.
acmeDir := filepath.Join(dataDir, "base", gojabase.TenantSegment("acme"))
globexDir := filepath.Join(dataDir, "base", gojabase.TenantSegment("globex"))
acmeDir := filepath.Join(dataDir, "base", goja.TenantSegment("acme"))
globexDir := filepath.Join(dataDir, "base", goja.TenantSegment("globex"))
if acmeDir == globexDir {
t.Fatalf("orgs share a data dir: %s", acmeDir)
}
+4 -4
View File
@@ -17,7 +17,7 @@ import (
baseapp "github.com/hanzoai/base"
"github.com/hanzoai/base/apis"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/gojabase"
"github.com/hanzoai/cloud/clients/goja"
)
// Pool sizing (env-overridable). A full Base app is heavier than a bare *sql.DB
@@ -35,7 +35,7 @@ const (
// ({DataDir}/base/{TenantSegment}/), the "prod = SQLite per tenant" rule
// (HIP-0302). Apps open lazily on first request, migrate once, and are pooled
// (LRU-capped, idle-evicted). Concurrent opens of the same org are
// single-flighted under mu. The org→segment encoding is gojabase.TenantSegment —
// single-flighted under mu. The org→segment encoding is goja.TenantSegment —
// the ONE injective, traversal-safe tenant→path encoder, shared so an org maps
// to exactly one physical identity everywhere in the binary.
type pool struct {
@@ -85,7 +85,7 @@ func (p *pool) acquire(org string) (http.Handler, func(), error) {
if strings.TrimSpace(org) == "" {
return nil, nil, fmt.Errorf("base: empty org")
}
seg := gojabase.TenantSegment(org)
seg := goja.TenantSegment(org)
p.mu.Lock()
defer p.mu.Unlock()
@@ -111,7 +111,7 @@ func (p *pool) acquire(org string) (http.Handler, func(), error) {
// for in-process callers that drive the engine's Go API directly (collection
// provisioning, seeding) rather than the HTTP path.
func (p *pool) appFor(org string) (*baseapp.Base, error) {
seg := gojabase.TenantSegment(org)
seg := goja.TenantSegment(org)
p.mu.Lock()
defer p.mu.Unlock()
if e, ok := p.m[seg]; ok {
+9 -1
View File
@@ -6,6 +6,7 @@ import (
"github.com/hanzoai/account"
"github.com/hanzoai/cloud/clients/finance"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
)
@@ -57,7 +58,14 @@ import (
// production tokens carry one. Called out for review rather than papered over.
func subjectFor(c *zip.Ctx, org string) string {
if name := strings.TrimSpace(c.Header("X-User-Name")); name != "" {
return account.Payer(account.Credential{Owner: org, Name: name}).Subject()
// Hand Payer the account the credential NAMES (the validated `billing_account`
// claim, minted into X-Billing-Account-Id). The ai gate reads the same claim,
// so this view and that gate resolve one wallet. Reading it here is what keeps
// them from drifting the way the org-vs-"org/user" split once did — except
// that split was two rules, and this would be one rule fed two different
// credentials, which reads the same to a user: a funded balance the gate
// refuses. Absent ⟹ Payer's legacy rule, exactly today's answer.
return account.Payer(account.Credential{Owner: org, Name: name, Account: principal.BillingAccount(c)}).Subject()
}
return account.PayerOf(org, strings.TrimSpace(c.User())).Subject()
}
+7 -7
View File
@@ -8,12 +8,12 @@
// business LOGIC (ported to a self-contained goja bundle in github.com/hanzoai/
// captable) and gives it PERSISTENCE over per-tenant Base/SQLite. The bundle
// carries logic; the Go host carries storage. The seam between them is the
// REUSABLE clients/gojabase binding (the RW-Base goja host), which esign (#100)
// REUSABLE clients/goja binding (the RW-Base goja host), which esign (#100)
// and dataroom (#101) reuse unchanged — this leaf is just:
//
// captable bundle (github.com/hanzoai/captable.Bundle) + the per-tenant Schema
// │
// clients/gojabase.New(...) ← injects __db/__newId/__now,
// clients/goja.NewBase(...) ← injects __db/__newId/__now,
// │ one SQLite file per tenant,
// /v1/captable/* zip routes one transaction per request
//
@@ -37,7 +37,7 @@ import (
hcaptable "github.com/hanzoai/captable"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/gojabase"
"github.com/hanzoai/cloud/clients/goja"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
)
@@ -48,7 +48,7 @@ const maxBody = 1 << 20 // 1 MiB
// state is captable's own data; shared deps live in the embedded cloud.Base.
type state struct {
host *gojabase.Host
host *goja.BaseHost
}
// mounted is the active service so Shutdown can release the per-tenant stores.
@@ -72,7 +72,7 @@ func Mount(app *zip.App, deps cloud.Deps) error {
if err != nil {
return fmt.Errorf("captable.Mount: load bundle: %w", err)
}
host, err := gojabase.New(gojabase.Config{
host, err := goja.NewBase(goja.BaseConfig{
Name: "captable",
Bundle: bundle,
Schema: schema,
@@ -80,7 +80,7 @@ func Mount(app *zip.App, deps cloud.Deps) error {
OnOpen: seedCompany,
})
if err != nil {
return fmt.Errorf("captable.Mount: gojabase host: %w", err)
return fmt.Errorf("captable.Mount: goja NewBase host: %w", err)
}
s := &cloud.Service[state]{Base: cloud.NewBase(deps, "captable"), State: state{host: host}}
mounted = s
@@ -176,7 +176,7 @@ func dispatch(s *cloud.Service[state], c *zip.Ctx, route string, params map[stri
}
}
}
resp, err := s.State.host.Dispatch(c.Context(), org, gojabase.Request{
resp, err := s.State.host.Dispatch(c.Context(), org, goja.BaseRequest{
Route: route,
Params: params,
Body: body,
+7 -7
View File
@@ -6,22 +6,22 @@ import (
"testing"
hcaptable "github.com/hanzoai/captable"
"github.com/hanzoai/cloud/clients/gojabase"
"github.com/hanzoai/cloud/clients/goja"
)
// TestFullLifecycle drives the REAL embedded captable bundle against a REAL
// per-tenant SQLite through gojabase — the end-to-end proof that the bundle's SQL
// per-tenant SQLite through NewBase — the end-to-end proof that the bundle's SQL
// matches the Go host schema and that the whole cap-table fold round-trips
// through Base: company → stakeholders → share class → equity plan → share
// issuance → options → SAFE → priced round + investment (dilution) → transfer →
// computed cap table. Any column/route drift fails here, not in production.
func newHost(t *testing.T) *gojabase.Host {
func newHost(t *testing.T) *goja.BaseHost {
t.Helper()
bundle, err := hcaptable.Bundle()
if err != nil {
t.Fatal(err)
}
h, err := gojabase.New(gojabase.Config{
h, err := goja.NewBase(goja.BaseConfig{
Name: "captable",
Bundle: bundle,
Schema: schema,
@@ -36,9 +36,9 @@ func newHost(t *testing.T) *gojabase.Host {
}
// do dispatches a route and returns (status, decoded body).
func do(t *testing.T, h *gojabase.Host, org, route string, params map[string]string, body any) (int, any) {
func do(t *testing.T, h *goja.BaseHost, org, route string, params map[string]string, body any) (int, any) {
t.Helper()
resp, err := h.Dispatch(context.Background(), org, gojabase.Request{Route: route, Params: params, Body: body})
resp, err := h.Dispatch(context.Background(), org, goja.BaseRequest{Route: route, Params: params, Body: body})
if err != nil {
t.Fatalf("dispatch %s: %v", route, err)
}
@@ -300,7 +300,7 @@ func TestDilutiveOptionsExcludeTerminal(t *testing.T) {
}
// firstShareID returns the founder's original certificate share id.
func firstShareID(t *testing.T, h *gojabase.Host, org string) string {
func firstShareID(t *testing.T, h *goja.BaseHost, org string) string {
t.Helper()
_, body := do(t, h, org, "shares.list", nil, nil)
data := body.(map[string]any)["data"].([]any)
+4 -4
View File
@@ -7,7 +7,7 @@ import (
"fmt"
"time"
"github.com/hanzoai/cloud/clients/gojabase"
"github.com/hanzoai/cloud/clients/goja"
)
// facade.go is the in-process cap-table seam: it lets a sibling subsystem (Hanzo
@@ -74,7 +74,7 @@ type RoundInput struct {
// response. The body is round-tripped through JSON to a generic value so the goja
// bundle sees the SAME wire shape (lower-case json keys) the HTTP path produces —
// passing a typed Go struct straight to goja would expose Go field names instead.
func facadeDispatch(ctx context.Context, org, route string, params map[string]string, body any) (*gojabase.Response, error) {
func facadeDispatch(ctx context.Context, org, route string, params map[string]string, body any) (*goja.Response, error) {
if mounted == nil || mounted.State.host == nil {
return nil, ErrNotMounted
}
@@ -85,7 +85,7 @@ func facadeDispatch(ctx context.Context, org, route string, params map[string]st
if err != nil {
return nil, err
}
return mounted.State.host.Dispatch(ctx, org, gojabase.Request{Route: route, Params: params, Body: wire})
return mounted.State.host.Dispatch(ctx, org, goja.BaseRequest{Route: route, Params: params, Body: wire})
}
// toWire normalizes a typed value to a generic JSON value (map[string]any /
@@ -107,7 +107,7 @@ func toWire(body any) (any, error) {
// okBody checks the response is 2xx and returns the body bytes, else a descriptive
// error carrying the bundle's own message.
func okBody(resp *gojabase.Response, route string) ([]byte, error) {
func okBody(resp *goja.Response, route string) ([]byte, error) {
if resp.Status/100 != 2 {
return nil, fmt.Errorf("captable %s: status %d: %s", route, resp.Status, string(resp.Body))
}
+1 -1
View File
@@ -19,7 +19,7 @@ import (
// mountApp builds a bare zip.App (no SanitizeIdentity middleware, so X-Org-Id +
// X-User-Id are trusted verbatim — the standard cloud leaf test harness) and
// mounts the captable leaf on it. This exercises the REAL HTTP path: routing →
// body decode → principal gate → gojabase dispatch → per-tenant Base → response,
// body decode → principal gate → NewBase dispatch → per-tenant Base → response,
// the same path the live binary serves under CLOUD_ENABLE=captable.
func mountApp(t *testing.T) *zip.App {
t.Helper()
+2 -2
View File
@@ -9,7 +9,7 @@ import (
// schema is the per-tenant SQLite DDL — the Go host owns migrations; the goja
// bundle only issues SQL against these tables. Column names MUST match the SQL in
// the captable bundle (github.com/hanzoai/captable goja/src/routes/*). Idempotent
// (IF NOT EXISTS), so it runs on every tenant DB open via gojabase.
// (IF NOT EXISTS), so it runs on every tenant DB open via NewBase.
//
// This is the Prisma data model (prisma/schema.prisma) translated to SQLite:
// DateTime → TEXT (ISO strings stored verbatim; the bundle never parses them),
@@ -217,7 +217,7 @@ CREATE INDEX IF NOT EXISTS ix_investment_company ON investment(company_id);
CREATE INDEX IF NOT EXISTS ix_investment_round ON investment(round_id);
`
// seedCompany is the gojabase OnOpen hook: it ensures the tenant's cap-table
// seedCompany is the NewBase OnOpen hook: it ensures the tenant's cap-table
// company row exists (id == the validated tenant), so the bundle's companyId
// always resolves. The name defaults to the tenant and is renamed via
// PUT /v1/captable/company. INSERT OR IGNORE makes it idempotent across reopens.
+4 -4
View File
@@ -1,4 +1,4 @@
// Hanzo Dataroom — goja bundle (read-WRITE, on clients/gojabase).
// Hanzo Dataroom — goja bundle (read-WRITE, on clients/goja).
//
// SELF-CONTAINED, NO ESM, NO node: imports. The complete dataroom business
// logic (documents, data rooms, shareable links with access controls, viewers,
@@ -8,7 +8,7 @@
// data model becomes Base/SQLite tables (see the leaf's schema.go), the handlers
// become the route table below. No Postgres, no Next.js.
//
// Host contract (clients/gojabase injects these per dispatch; each dispatch runs
// Host contract (clients/goja injects these per dispatch; each dispatch runs
// inside ONE per-tenant SQLite transaction that commits iff status < 400):
// globalThis.__db.query(sql, args) -> [ {col: val, ...}, ... ]
// globalThis.__db.exec(sql, args) -> { changes, lastId }
@@ -49,7 +49,7 @@
}
// err builds a route result carrying a non-200 status via __status. A >=400
// status also rolls back the dispatch transaction (gojabase), so a rejected
// status also rolls back the dispatch transaction (NewBase), so a rejected
// request leaves the tenant DB untouched.
function err(status, message) { return { __status: status, error: message }; }
@@ -121,7 +121,7 @@
}
// === route handlers ========================================================
// Admin routes are org-scoped by the per-tenant DB gojabase selects; the Go
// Admin routes are org-scoped by the per-tenant DB NewBase selects; the Go
// leaf refuses any request without a validated principal before dispatching.
// Viewer routes run under the org resolved from the public link id.
+13 -13
View File
@@ -8,7 +8,7 @@
// business logic (documents, data rooms, shareable links with access controls,
// viewers, per-page view analytics) is a self-contained goja bundle (bundle.js, the
// ESM-free port of the Papermark API handlers). It runs in-process on the REUSABLE
// clients/gojabase host — the SAME RW-Base binding captable (#97) pilots and esign
// clients/goja host — the SAME RW-Base binding captable (#97) pilots and esign
// (#100) reuses — which injects __db/__newId/__now and one SQLite file per tenant,
// one transaction per request. This leaf adds only: the per-tenant Schema, the
// object-storage seam for document bytes, a bcrypt HostFn for link passwords, and
@@ -16,7 +16,7 @@
//
// dataroom bundle (bundle.js, go:embed) + per-tenant Schema + __bcrypt HostFn
// │
// clients/gojabase.New(...) ← __db/__newId/__now, per-tenant Base,
// clients/goja.NewBase(...) ← __db/__newId/__now, per-tenant Base,
// │ one transaction per request
// /v1/dataroom/* zip routes
//
@@ -29,7 +29,7 @@
// AUTH. Admin routes require a validated cloud principal (principal.Org → org);
// public viewer routes carry no principal and resolve their org from the link index
// (a link id → org routing table — the one cross-tenant piece). Tenant isolation is
// the per-org SQLite file gojabase selects from that org.
// the per-org SQLite file NewBase selects from that org.
//
// ACTIVATION: dataroom is NOT staged — it mounts under the mount-all default
// (empty CLOUD_ENABLE), so the one binary serves /v1/dataroom/* from first boot.
@@ -51,7 +51,7 @@ import (
"golang.org/x/crypto/bcrypt"
hcloud "github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/gojabase"
"github.com/hanzoai/cloud/clients/goja"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
)
@@ -75,7 +75,7 @@ type blobStore interface {
// state is dataroom's own data; shared deps live in the embedded cloud.Base,
// reached as s.Log.
type state struct {
host *gojabase.Host
host *goja.BaseHost
index *linkIndex
blob blobStore
}
@@ -105,7 +105,7 @@ func Mount(app *zip.App, deps hcloud.Deps) error {
return c.JSON(http.StatusOK, map[string]string{"service": "dataroom", "status": "ok"})
})
host, err := gojabase.New(gojabase.Config{
host, err := goja.NewBase(goja.BaseConfig{
Name: "dataroom",
Bundle: bundleJS,
Schema: schema,
@@ -233,7 +233,7 @@ func uploadDocument(s *hcloud.Service[state], c *zip.Ctx) error {
ct = "application/octet-stream"
}
// ONE tenant encoding everywhere: the object-store key prefix uses the SAME
// injective, path-safe gojabase.TenantSegment the per-tenant SQLite filename
// injective, path-safe goja.TenantSegment the per-tenant SQLite filename
// does — never the raw org (which could carry a '/' and traverse the key
// namespace, and would drift from the DB's encoding).
rk, err := randKey()
@@ -241,7 +241,7 @@ func uploadDocument(s *hcloud.Service[state], c *zip.Ctx) error {
s.Log.Error("dataroom: crypto/rand unavailable", "err", err)
return zip.Errorf(http.StatusInternalServerError, "storage key generation failed")
}
key := "dataroom/" + gojabase.TenantSegment(org) + "/" + rk
key := "dataroom/" + goja.TenantSegment(org) + "/" + rk
if err := s.State.blob.Put(c.Context(), key, raw); err != nil {
s.Log.Error("dataroom storage put failed", "err", err)
return zip.Errorf(http.StatusBadGateway, "document storage unavailable")
@@ -259,7 +259,7 @@ func adminDownload(s *hcloud.Service[state], c *zip.Ctx) error {
if !ok {
return zip.ErrForbidden("X-Org-Id required")
}
resp, err := s.State.host.Dispatch(c.Context(), org, gojabase.Request{
resp, err := s.State.host.Dispatch(c.Context(), org, goja.BaseRequest{
Route: "documents.file", Params: map[string]string{"id": c.Param("id")},
})
if err != nil {
@@ -275,7 +275,7 @@ func viewerDownload(s *hcloud.Service[state], c *zip.Ctx) error {
if err != nil || !ok {
return zip.ErrNotFound("link not found")
}
resp, err := s.State.host.Dispatch(c.Context(), org, gojabase.Request{
resp, err := s.State.host.Dispatch(c.Context(), org, goja.BaseRequest{
Route: "view.file",
Params: map[string]string{"linkId": linkID, "documentId": c.Param("documentId")},
Query: map[string]string{"viewId": c.Query("viewId"), "download": c.Query("download")},
@@ -288,7 +288,7 @@ func viewerDownload(s *hcloud.Service[state], c *zip.Ctx) error {
// streamFile turns a {fileKey,contentType,name} bundle result into a byte stream
// from object storage. A non-200 bundle result (404/403) passes through as JSON.
func streamFile(s *hcloud.Service[state], c *zip.Ctx, resp *gojabase.Response) error {
func streamFile(s *hcloud.Service[state], c *zip.Ctx, resp *goja.Response) error {
if resp.Status != http.StatusOK {
c.SetHeader("Content-Type", "application/json")
return c.Bytes(resp.Status, resp.Body)
@@ -325,7 +325,7 @@ func createLink(s *hcloud.Service[state], c *zip.Ctx) error {
if err != nil {
return err
}
resp, err := s.State.host.Dispatch(c.Context(), org, gojabase.Request{Route: "links.create", Body: body})
resp, err := s.State.host.Dispatch(c.Context(), org, goja.BaseRequest{Route: "links.create", Body: body})
if err != nil {
return zip.Errorf(http.StatusInternalServerError, "dataroom dispatch failed")
}
@@ -349,7 +349,7 @@ func createLink(s *hcloud.Service[state], c *zip.Ctx) error {
// write dispatches one bundle route on the tenant's Base store (one transaction
// per request) and writes {status, body}.
func write(s *hcloud.Service[state], c *zip.Ctx, org, route string, params, query map[string]string, body any) error {
resp, err := s.State.host.Dispatch(c.Context(), org, gojabase.Request{
resp, err := s.State.host.Dispatch(c.Context(), org, goja.BaseRequest{
Route: route, Params: params, Query: query, Body: body,
})
if err != nil {
+1 -1
View File
@@ -19,7 +19,7 @@ import (
type linkIndex struct{ db *sql.DB }
func openLinkIndex(dataDir string) (*linkIndex, error) {
// Its OWN dir, distinct from gojabase's per-tenant tree ({dataDir}/dataroom/):
// Its OWN dir, distinct from NewBase's per-tenant tree ({dataDir}/dataroom/):
// this global routing table must never collide with a tenant's DB file.
dir := filepath.Join(dataDir, "dataroom_index")
if err := os.MkdirAll(dir, 0o700); err != nil {
+3 -3
View File
@@ -7,7 +7,7 @@ import (
"fmt"
"net/http"
"github.com/hanzoai/cloud/clients/gojabase"
"github.com/hanzoai/cloud/clients/goja"
)
// ingest.go is the in-process ingestion seam: it lets a sibling subsystem (Hanzo
@@ -48,11 +48,11 @@ func Ingest(ctx context.Context, org, name, contentType string, data []byte) (st
if err != nil {
return "", fmt.Errorf("dataroom.Ingest: storage key: %w", err)
}
key := "dataroom/" + gojabase.TenantSegment(org) + "/" + rk
key := "dataroom/" + goja.TenantSegment(org) + "/" + rk
if err := mounted.State.blob.Put(ctx, key, data); err != nil {
return "", fmt.Errorf("dataroom.Ingest: blob put: %w", err)
}
resp, err := mounted.State.host.Dispatch(ctx, org, gojabase.Request{
resp, err := mounted.State.host.Dispatch(ctx, org, goja.BaseRequest{
Route: "documents.create",
Body: map[string]any{"name": name, "fileKey": key, "contentType": contentType, "fileSize": len(data)},
})
+1 -1
View File
@@ -1,6 +1,6 @@
package dataroom
// schema is the per-tenant SQLite DDL — the Go host owns migrations (gojabase
// schema is the per-tenant SQLite DDL — the Go host owns migrations (NewBase
// runs this on every tenant DB open); the goja bundle only issues SQL against
// these tables. Column names MUST match the SQL in bundle.js. Idempotent
// (IF NOT EXISTS).
+28 -83
View File
@@ -2,7 +2,7 @@
// ArgoCD-grade deploy dashboard for the operator-managed fleet, made native to
// the cloud binary and parallel to /v1/git (the native git server).
//
// Each operator hanzo.ai/v1 Service CR IS a GitOps Application: the desired state
// Each operator hanzo.ai/v1 App CR IS a GitOps Application: the desired state
// declared for one workload, which the Hanzo operator reconciles into a
// Deployment + Service + Ingress (+ HPA/PDB/Pods). This plane OBSERVES that
// reconciliation the way ArgoCD observes a synced Application —
@@ -54,27 +54,12 @@ import (
"k8s.io/client-go/tools/clientcmd"
)
// The operator "Application" CR is undergoing a kind collapse: the specialized
// services.hanzo.ai (kind Service) + siblings become ONE apps.hanzo.ai (kind App),
// with the former kind carried as a value in spec.role. appsCRGVR is the FORWARD
// target; servicesCRGVR is the CURRENT live kind. The operator/CRD collapse lands
// on branches; the live cluster still serves kind Service until cutover, so this
// plane reads BOTH — App first (forward), Service as a transition shim. Group
// hanzo.ai disambiguates either from the core/v1 Service (a CHILD it reconciles),
// which is why a resource ref always carries its group.
//
// COMPAT SHIM (removable post-cutover): drop servicesCRGVR from appCRGVRs() and the
// "hanzo.ai/Service" registry entry once every cluster serves kind App.
var (
appsCRGVR = schema.GroupVersionResource{Group: "hanzo.ai", Version: "v1", Resource: "apps"}
servicesCRGVR = schema.GroupVersionResource{Group: "hanzo.ai", Version: "v1", Resource: "services"}
)
// appCRGVRs is the App-CR read order: the forward kind first, the transition kind
// second. Every CR read/resolve walks this list.
func appCRGVRs() []schema.GroupVersionResource {
return []schema.GroupVersionResource{appsCRGVR, servicesCRGVR}
}
// appsCRGVR is the operator App CR (apps.hanzo.ai) — the one workload kind this
// plane reads. Each App IS a GitOps Application: the desired state for one
// workload, which the operator reconciles into a Deployment + Service + Ingress
// (+ HPA/PDB/Pods). Group hanzo.ai disambiguates it from the core/v1 Service (a
// CHILD it reconciles), which is why a resource ref always carries its group.
var appsCRGVR = schema.GroupVersionResource{Group: "hanzo.ai", Version: "v1", Resource: "apps"}
// childGVRs are the operator-owned workload objects the tree walks at depth 1
// (owned by the Service CR) and their descendants (ReplicaSet → Pod). Secrets are
@@ -96,8 +81,7 @@ var (
// the resource endpoint can never be steered at an arbitrary cluster object.
// Keyed by "group/Kind" (group "" for the core API group).
var kindGVR = map[string]schema.GroupVersionResource{
"hanzo.ai/App": appsCRGVR, // forward kind
"hanzo.ai/Service": servicesCRGVR, // transition shim (removable post-cutover)
"hanzo.ai/App": appsCRGVR,
"apps/Deployment": deploymentsGVR,
"apps/ReplicaSet": replicaSetsGVR,
"/Pod": podsGVR,
@@ -179,33 +163,17 @@ func guard(s *cloud.Service[state], h zip.Handler) zip.Handler {
}
}
// health is a REAL probe: the API server is reachable AND the Service CRD is
// served. 200 only when both hold; 503 + the real reason otherwise. Not
// admin-gated — liveness must be probe-able without a JWT.
// health is a REAL probe: the API server is reachable AND the App CRD is served.
// 200 only when both hold; 503 + the real reason otherwise. Not admin-gated —
// liveness must be probe-able without a JWT.
func health(s *cloud.Service[state], c *zip.Ctx) error {
res := map[string]any{"service": "deploy", "status": "ok"}
if s.State.dyn == nil {
res["status"], res["k8s"], res["error"] = "degraded", false, s.State.initErr
return c.JSON(http.StatusServiceUnavailable, res)
}
// The App CRD is served if EITHER the forward (apps) or transition (services)
// kind lists without error — a NotFound on one during the collapse is not a
// degradation as long as the other answers.
var lastErr error
served := false
for _, gvr := range appCRGVRs() {
if _, err := s.State.dyn.Resource(gvr).Namespace("hanzo").List(c.Context(), metav1.ListOptions{Limit: 1}); err == nil {
served = true
break
} else if !apierrors.IsNotFound(err) {
lastErr = err
}
}
if !served {
res["status"], res["k8s"], res["crd"] = "degraded", true, false
if lastErr != nil {
res["error"] = lastErr.Error()
}
if _, err := s.State.dyn.Resource(appsCRGVR).Namespace("hanzo").List(c.Context(), metav1.ListOptions{Limit: 1}); err != nil {
res["status"], res["k8s"], res["crd"], res["error"] = "degraded", true, false, err.Error()
return c.JSON(http.StatusServiceUnavailable, res)
}
res["k8s"], res["crd"] = true, true
@@ -239,8 +207,7 @@ func regexpLower(s string) string {
}
// resolveNamespace finds the platform namespace an App CR lives in, scanning in
// env order (main first) across both CR kinds. Returns a clean 404 when found in
// none.
// env order (main first). Returns a clean 404 when found in none.
func resolveNamespace(s *cloud.Service[state], c *zip.Ctx, name string) (string, error) {
for _, ns := range scanOrder() {
if _, _, err := getAppCR(s, c.Context(), ns, name); err == nil {
@@ -252,49 +219,27 @@ func resolveNamespace(s *cloud.Service[state], c *zip.Ctx, name string) (string,
return "", zip.ErrNotFound("application " + name + " not found in the platform namespaces")
}
// getAppCR gets an App CR by name from ns, trying the forward kind (apps) then the
// transition kind (services). Returns the object and the GVR it was found under, so
// a mutation (sync) patches the SAME kind. A miss in both is an IsNotFound error.
// getAppCR gets an App CR by name from ns. Returns the object and its GVR so a
// mutation (sync/rollback) patches the App CR it read. A miss is an IsNotFound
// error.
func getAppCR(s *cloud.Service[state], ctx context.Context, ns, name string) (*unstructured.Unstructured, schema.GroupVersionResource, error) {
var readErr error
for _, gvr := range appCRGVRs() {
obj, err := s.State.dyn.Resource(gvr).Namespace(ns).Get(ctx, name, metav1.GetOptions{})
if err == nil {
return obj, gvr, nil
}
if !apierrors.IsNotFound(err) {
readErr = err
}
obj, err := s.State.dyn.Resource(appsCRGVR).Namespace(ns).Get(ctx, name, metav1.GetOptions{})
if err != nil {
return nil, schema.GroupVersionResource{}, err
}
if readErr != nil {
return nil, schema.GroupVersionResource{}, readErr
}
return nil, schema.GroupVersionResource{}, apierrors.NewNotFound(appsCRGVR.GroupResource(), name)
return obj, appsCRGVR, nil
}
// listAppCRs lists every App CR in ns across both kinds, App first, de-duplicated
// by name (a name served by both kinds during the collapse yields the App copy).
// listAppCRs lists every App CR in ns.
func listAppCRs(s *cloud.Service[state], ctx context.Context, ns string) ([]unstructured.Unstructured, error) {
seen := map[string]bool{}
var out []unstructured.Unstructured
for _, gvr := range appCRGVRs() {
list, err := s.State.dyn.Resource(gvr).Namespace(ns).List(ctx, metav1.ListOptions{})
if err != nil {
if apierrors.IsNotFound(err) {
continue
}
return nil, err
}
for i := range list.Items {
n := list.Items[i].GetName()
if seen[n] {
continue
}
seen[n] = true
out = append(out, list.Items[i])
list, err := s.State.dyn.Resource(appsCRGVR).Namespace(ns).List(ctx, metav1.ListOptions{})
if err != nil {
if apierrors.IsNotFound(err) {
return nil, nil
}
return nil, err
}
return out, nil
return list.Items, nil
}
// k8sErr maps a raw API error to an honest gateway error, naming the missing RBAC
+14 -29
View File
@@ -20,7 +20,6 @@ func fakeSvc(objs ...runtime.Object) *cloud.Service[state] {
scheme := runtime.NewScheme()
dyn := dynamicfake.NewSimpleDynamicClientWithCustomListKinds(scheme, map[schema.GroupVersionResource]string{
appsCRGVR: "AppList",
servicesCRGVR: "ServiceCRList",
deploymentsGVR: "DeploymentList",
replicaSetsGVR: "ReplicaSetList",
podsGVR: "PodList",
@@ -128,20 +127,18 @@ func TestSyncStatus(t *testing.T) {
// ── ref parsing ─────────────────────────────────────────────────────────────
func TestParseRef(t *testing.T) {
// forward App kind and transition Service kind both resolve.
// The App CR resolves; the core/v1 Service (a child object) resolves distinctly.
if _, gvr, err := parseRef("hanzo.ai:App:hanzo:iam"); err != nil || gvr != appsCRGVR {
t.Errorf("App ref → (%v, %v), want appsCRGVR", gvr, err)
}
if _, gvr, err := parseRef("hanzo.ai:Service:hanzo:iam"); err != nil || gvr != servicesCRGVR {
t.Errorf("Service ref → (%v, %v), want servicesCRGVR", gvr, err)
}
if _, gvr, err := parseRef("apps:Deployment:hanzo:iam"); err != nil || gvr != deploymentsGVR {
t.Errorf("Deployment ref → (%v, %v), want deploymentsGVR", gvr, err)
}
if _, _, err := parseRef(":Service:hanzo:iam"); err != nil {
t.Errorf("core Service ref err = %v, want nil", err)
}
bad := []string{"", "a:b:c", "unknown/Kind:hanzo:iam:x", "apps:Deployment:evil-ns:iam", "apps:Deployment:hanzo:Bad_Name"}
// hanzo.ai:Service is not a kind this plane reads — the operator CR is App.
bad := []string{"", "a:b:c", "hanzo.ai:Service:hanzo:iam", "unknown/Kind:hanzo:iam:x", "apps:Deployment:evil-ns:iam", "apps:Deployment:hanzo:Bad_Name"}
for _, r := range bad {
if _, _, err := parseRef(r); err == nil {
t.Errorf("parseRef(%q) = nil err, want rejection", r)
@@ -215,49 +212,37 @@ func TestObserveApplication(t *testing.T) {
}
}
// ── CR resolution shim (App forward, Service fallback) ───────────────────────
// ── CR resolution ────────────────────────────────────────────────────────────
func TestGetAppCRAppFirst(t *testing.T) {
s := fakeSvc(
appCR("App", "hanzo", "iam", "u-app", "ghcr.io/hanzoai/iam", "v2.0.0", "Running", 1, 1),
appCR("Service", "hanzo", "iam", "u-svc", "ghcr.io/hanzoai/iam", "v1.0.0", "Running", 1, 1),
)
func TestGetAppCR(t *testing.T) {
s := fakeSvc(appCR("App", "hanzo", "iam", "u-app", "ghcr.io/hanzoai/iam", "v2.0.0", "Running", 1, 1))
obj, gvr, err := getAppCR(s, context.Background(), "hanzo", "iam")
if err != nil {
t.Fatalf("getAppCR: %v", err)
}
if gvr != appsCRGVR {
t.Fatalf("gvr = %v, want appsCRGVR (App must win)", gvr)
t.Fatalf("gvr = %v, want appsCRGVR", gvr)
}
if tag, _, _ := unstructured.NestedString(obj.Object, "spec", "image", "tag"); tag != "v2.0.0" {
t.Fatalf("resolved tag = %q, want v2.0.0 (App copy)", tag)
t.Fatalf("resolved tag = %q, want v2.0.0", tag)
}
}
func TestGetAppCRServiceFallback(t *testing.T) {
s := fakeSvc(appCR("Service", "hanzo", "iam", "u-svc", "ghcr.io/hanzoai/iam", "v1.0.0", "Running", 1, 1))
_, gvr, err := getAppCR(s, context.Background(), "hanzo", "iam")
if err != nil || gvr != servicesCRGVR {
t.Fatalf("getAppCR fallback → (%v, %v), want servicesCRGVR", gvr, err)
}
// Missing everywhere → IsNotFound.
// Missing → IsNotFound.
if _, _, err := getAppCR(s, context.Background(), "hanzo", "ghost"); err == nil {
t.Fatal("getAppCR(ghost) = nil err, want NotFound")
}
}
func TestListAppCRsMergeDedup(t *testing.T) {
func TestListAppCRs(t *testing.T) {
s := fakeSvc(
appCR("App", "hanzo", "iam", "u1", "r", "v2.0.0", "Running", 1, 1), // App copy of iam
appCR("Service", "hanzo", "iam", "u2", "r", "v1.0.0", "Running", 1, 1), // stale Service copy — deduped out
appCR("Service", "hanzo", "cloud", "u3", "r", "v1.799.0", "Running", 1, 1),
appCR("App", "hanzo", "iam", "u1", "r", "v2.0.0", "Running", 1, 1),
appCR("App", "hanzo", "cloud", "u3", "r", "v1.799.0", "Running", 1, 1),
)
crs, err := listAppCRs(s, context.Background(), "hanzo")
if err != nil {
t.Fatalf("listAppCRs: %v", err)
}
if len(crs) != 2 {
t.Fatalf("listAppCRs len = %d, want 2 (iam deduped, cloud)", len(crs))
t.Fatalf("listAppCRs len = %d, want 2 (iam, cloud)", len(crs))
}
byName := map[string]string{}
for i := range crs {
@@ -265,7 +250,7 @@ func TestListAppCRsMergeDedup(t *testing.T) {
byName[crs[i].GetName()] = tag
}
if byName["iam"] != "v2.0.0" {
t.Errorf("iam tag = %q, want v2.0.0 (App wins the dedupe)", byName["iam"])
t.Errorf("iam tag = %q, want v2.0.0", byName["iam"])
}
if byName["cloud"] != "v1.799.0" {
t.Errorf("cloud tag = %q", byName["cloud"])
+17 -27
View File
@@ -26,20 +26,22 @@
// ── THE POLICY PRIMITIVE ─────────────────────────────────────────────────────────
//
// This engine IS Hanzo's runtime decision primitive: (Principal, context) -> verdict,
// evaluated in-process, stateless, hot. Feature flags, rollouts, and the launch
// waitlist (waitlist.go — folded in from the former clients/featuregate: a service's
// mode IS the switch waitlist.<svc>) are its first tenants. The aspirational end-state
// — NOT built here, flagged for the next step — is that the platform's OTHER runtime
// decisions are the SAME shape and could COMPOSE this one engine rather than each
// re-deriving it:
// evaluated in-process, stateless, hot. It knows ONLY flags — definitions, evaluation,
// and the platform-switch registry. It has ZERO knowledge of any specific policy that
// rides on it: no host→service map, no waitlist, no service registry. Those COMPOSE
// this engine from the OUTSIDE, one-way (they import flags; flags imports none of them):
//
// - authz (access policy) — (Principal, resource+action) -> allow/deny
// - entitlements (product-access policy) — (Principal, feature/plan) -> granted/denied
// - the launch waitlist gate (clients/admission) — a service's mode IS the switch
// waitlist.<svc>; admission owns the host→service registry + Enforce and reads the
// mode through flags.Bool. It USED to live in this package; extracting it is the
// PROOF the engine composes its tenants rather than absorbing them.
// - authz (access policy) — (Principal, resource+action) -> allow/deny
// - entitlements (product-access policy) — (Principal, feature/plan) -> granted/denied
//
// Both are (Principal, context) -> verdict. Folding them onto this evaluator would make
// Policy ONE composable primitive with one audit log and one hot-apply path. DO NOT
// touch authz/entitlements now — this note only names the target so the seam is
// visible; the launch waitlist is the first fold, done here.
// Every one is (Principal, context) -> verdict. Folding them onto this evaluator makes
// Policy ONE composable primitive with one audit log and one hot-apply path. authz and
// entitlements are NOT built on it yet — this note only names the target so the seam
// stays visible; the launch waitlist (now external) is the first, proven composition.
package flags
import (
@@ -140,7 +142,6 @@ type snapshot struct {
// cached for one TTL (the hot-apply bound).
type Client struct {
stores *cloud.OrgStore[*Store]
registry *cloud.OrgStore[*waitlistStore] // waitlist host→service map (platform tenant)
distinctID string
ttl time.Duration
@@ -442,7 +443,6 @@ func Mount(app *zip.App, deps cloud.Deps) error {
log := deps.Logger.New("subsystem", "flags")
c := &Client{
stores: cloud.NewOrgStore[*Store](deps.DataDir, "flags", openStore),
registry: cloud.NewOrgStore[*waitlistStore](deps.DataDir, "waitlist", openWaitlistStore),
distinctID: firstNonEmpty(os.Getenv("FLAGS_PLATFORM_DISTINCT_ID"), "hanzo-platform:"+firstNonEmpty(deps.Brand, "hanzo")),
ttl: ttlFromEnv(),
}
@@ -450,26 +450,16 @@ func Mount(app *zip.App, deps cloud.Deps) error {
b := cloud.NewBase(deps, "flags")
svc := &cloud.Service[state]{Base: b, State: state{client: c}}
routes(app, svc)
mountWaitlist(c, deps.Brand, log) // fold: seed the host→service registry + register the waitlist.<svc> switches
log.Info("flags engine ready", "engine", "hanzo-flags", "native", engineAvailable, "ttlSeconds", int(c.ttl.Seconds()), "switches", len(Defs()))
return nil
}
// Shutdown closes every open per-org definitions store and the waitlist registry.
// Shutdown closes every open per-org definitions store.
func Shutdown(_ context.Context) error {
if mounted == nil {
if mounted == nil || mounted.stores == nil {
return nil
}
var first error
if mounted.stores != nil {
first = mounted.stores.CloseAll()
}
if mounted.registry != nil {
if err := mounted.registry.CloseAll(); err != nil && first == nil {
first = err
}
}
return first
return mounted.stores.CloseAll()
}
func ttlFromEnv() time.Duration {
+19 -15
View File
@@ -2,31 +2,35 @@ package flags
// The canonical PLATFORM launch switches — the runtime knobs the SuperAdmin flips from
// admin.hanzo.ai. Each is an Insights feature flag (the engine); this table names it,
// categorizes it, and pins the env var that supplies the fallback default so a
// deployment with no Insights opinion behaves exactly as before. Subsystems may
// Register MORE at init — this seed is the launch set, not a closed list.
// categorizes it, and pins a hardcoded Default. RUNTIME flags resolve from /v1/flags
// (the DB engine) → Default, with NO env gate — /v1/flags is the single source of truth,
// flipped live, no redeploy. Only BOOT-TIME, ReadOnly rows (subsystem activation applied
// via CLOUD_ENABLE/CLOUD_*; network-id display) pin an Env, because that IS their boot
// mechanism, not a redundant runtime override. Subsystems may Register MORE at init —
// this seed is the launch set, not a closed list.
func init() {
for _, d := range []Def{
// ── Launch / Waitlist ──
{Key: "waitlist_open", Category: "Launch", Label: "Waitlist intake open", Desc: "Accept new waitlist signups. Off closes intake platform-wide.", Type: TypeBool, Env: "WAITLIST_OPEN", Default: "true"},
{Key: "waitlist_access_capacity", Category: "Launch", Label: "Access capacity", Desc: "How many waitlisted users are granted product access (Insights flag payload = the number).", Type: TypeInt, Env: "WAITLIST_ACCESS_CAPACITY", Default: "0"},
{Key: "waitlist_points_referral", Category: "Launch", Label: "Points · referral", Desc: "Queue-position points awarded per referral.", Type: TypeInt, Env: "WAITLIST_POINTS_REFERRAL", Default: "0"},
{Key: "waitlist_points_social", Category: "Launch", Label: "Points · social", Desc: "Queue-position points awarded for a social share.", Type: TypeInt, Env: "WAITLIST_POINTS_SOCIAL", Default: "0"},
{Key: "waitlist_points_hanzod", Category: "Launch", Label: "Points · run hanzod", Desc: "Queue-position points awarded for running hanzod.", Type: TypeInt, Env: "WAITLIST_POINTS_HANZOD", Default: "0"},
{Key: "waitlist_turnstile", Category: "Launch", Label: "Turnstile challenge", Desc: "Require the Cloudflare Turnstile challenge on waitlist signup.", Type: TypeBool, Env: "WAITLIST_TURNSTILE_ENABLED", Default: "false"},
{Key: "waitlist_signup_rate_limit", Category: "Launch", Label: "Signup rate limit", Desc: "Max waitlist signups per source per minute (0 = unlimited).", Type: TypeInt, Env: "WAITLIST_SIGNUP_RATE_LIMIT", Default: "0"},
// ── Launch / Waitlist (runtime flags: /v1/flags is the single source of truth; DB → Default, no env gate) ──
{Key: "waitlist_open", Category: "Launch", Label: "Waitlist intake open", Desc: "Accept new waitlist signups. Off closes intake platform-wide.", Type: TypeBool, Default: "true"},
{Key: "waitlist_access_capacity", Category: "Launch", Label: "Access capacity", Desc: "How many waitlisted users are granted product access (Insights flag payload = the number).", Type: TypeInt, Default: "0"},
{Key: "waitlist_points_referral", Category: "Launch", Label: "Points · referral", Desc: "Queue-position points awarded per referral.", Type: TypeInt, Default: "0"},
{Key: "waitlist_points_social", Category: "Launch", Label: "Points · social", Desc: "Queue-position points awarded for a social share.", Type: TypeInt, Default: "0"},
{Key: "waitlist_points_hanzod", Category: "Launch", Label: "Points · run hanzod", Desc: "Queue-position points awarded for running hanzod.", Type: TypeInt, Default: "0"},
{Key: "waitlist_turnstile", Category: "Launch", Label: "Turnstile challenge", Desc: "Require the Cloudflare Turnstile challenge on waitlist signup.", Type: TypeBool, Default: "false"},
{Key: "waitlist_signup_rate_limit", Category: "Launch", Label: "Signup rate limit", Desc: "Max waitlist signups per source per minute (0 = unlimited).", Type: TypeInt, Default: "0"},
// ── Signup ──
{Key: "public_signup", Category: "Signup", Label: "Public open signup", Desc: "Allow anyone to create an account (off = invite / waitlist only).", Type: TypeBool, Env: "PUBLIC_SIGNUP_ENABLED", Default: "false"},
// ── Signup (runtime flag: /v1/flags only) ──
{Key: "public_signup", Category: "Signup", Label: "Public open signup", Desc: "Allow anyone to create an account (off = invite / waitlist only).", Type: TypeBool, Default: "false"},
// ── Subsystem activation (boot-time; applying a flip needs an operator reconcile) ──
{Key: "subsystem_iam_active", Category: "Subsystems", Label: "IAM (canary auth cutover)", Desc: "Serve identity from the embedded IAM. CANARY-GATED staged auth cutover; applied at boot via CLOUD_ENABLE.", Type: TypeBool, Default: "false", ReadOnly: true},
{Key: "subsystem_iam2_active", Category: "Subsystems", Label: "IAM v2 (clean-room, beego-free)", Desc: "Serve identity from the clean-room iam2 instead of the beego/Casdoor embed. Selected at boot via CLOUD_IAM_IMPL=iam2 (one selector, applied on the next reconcile). Gate before flipping: the IAM cutover parity suite (universe e2e/50-iam-cutover-parity) must be green against the iam2 shadow.", Type: TypeBool, Default: "false", ReadOnly: true},
{Key: "subsystem_ingress_active", Category: "Subsystems", Label: "Ingress edge", Desc: "Serve the embedded ingress edge (routes/TLS/ACME). Applied at boot via CLOUD_ENABLE.", Type: TypeBool, Default: "false", ReadOnly: true},
{Key: "subsystem_pubsub_active", Category: "Subsystems", Label: "PubSub (NATS+JetStream)", Desc: "Serve the embedded messaging plane. Applied at boot via CLOUD_PUBSUB_ENABLED.", Type: TypeBool, Env: "CLOUD_PUBSUB_ENABLED", Default: "false", ReadOnly: true},
// ── Gateway (full rate-limit / quota / CORS config lives at /v1/gateway) ──
{Key: "gateway_rate_limit_rpm", Category: "Gateway", Label: "Rate limit (req/min)", Desc: "Default per-key request rate limit. Full CORS / quota config lives at /v1/gateway.", Type: TypeInt, Env: "GATEWAY_RATE_LIMIT_RPM", Default: "0"},
{Key: "gateway_quota_daily", Category: "Gateway", Label: "Daily quota", Desc: "Default per-key daily request quota (0 = unlimited).", Type: TypeInt, Env: "GATEWAY_QUOTA_DAILY", Default: "0"},
{Key: "gateway_rate_limit_rpm", Category: "Gateway", Label: "Rate limit (req/min)", Desc: "Default per-key request rate limit. Full CORS / quota config lives at /v1/gateway.", Type: TypeInt, Default: "0"},
{Key: "gateway_quota_daily", Category: "Gateway", Label: "Daily quota", Desc: "Default per-key daily request quota (0 = unlimited).", Type: TypeInt, Default: "0"},
// ── Network (canonical Lux primary network ids — read-only display + override knob) ──
{Key: "network_id_mainnet", Category: "Network", Label: "Network ID · mainnet", Desc: "Lux primary network id for mainnet (convention-fixed = 1).", Type: TypeInt, Env: "LUX_NETWORK_ID_MAINNET", Default: "1", ReadOnly: true},
-4
View File
@@ -25,10 +25,6 @@ func routes(app *zip.App, s *cloud.Service[state]) {
app.Put("/v1/flags/defs/:key", cloud.Handle(s, putDef))
app.Delete("/v1/flags/defs/:key", cloud.Handle(s, deleteDef))
app.Get("/v1/flags/activity", cloud.Handle(s, listActivity))
// Repointed from the former featuregate: the guard's public runtime mode read,
// now answered by the engine decide (host→service→waitlist.<svc>). Exempt from the
// Enforce gate (/v1/featuregate/ prefix), so a gated user can still resolve mode.
app.Get("/v1/featuregate/mode", cloud.Handle(s, waitlistModeRoute))
}
// tenant resolves the org — the tenant-isolation KEY — from the validated
@@ -1,4 +1,4 @@
// Package gatewaypolicy is the runtime-mutable store for the cloud edge ("gateway
// Package edge is the runtime-mutable store for the cloud edge ("gateway
// role") policy: the CORS allowlist, the pre-auth per-client-IP flood cap, and the
// authenticated per-org rate ceiling. It is a LEAF package (stdlib + the Hanzo
// SQLite driver only, no import of the root cloud package) so BOTH consumers can
@@ -27,7 +27,7 @@
// Fail-soft: every resolver (Platform/OrgRPM/CacheTTL/Methods) returns the static/platform default
// on any store error, so a policy-store outage never takes the edge down. Writes
// fail loud (an unavailable store returns an error to the PUT handler).
package gatewaypolicy
package edge
import (
"context"
@@ -179,14 +179,14 @@ type cacheEntry struct {
func New(dataDir, adminOrg string, static Policy) (*Store, error) {
s := &Store{adminOrg: adminOrg, static: static, cache: map[string]cacheEntry{}}
if dataDir == "" {
return s, fmt.Errorf("gatewaypolicy: empty dataDir; running static-only")
return s, fmt.Errorf("edge: empty dataDir; running static-only")
}
if err := os.MkdirAll(dataDir, 0o750); err != nil {
return s, fmt.Errorf("gatewaypolicy: mkdir %s: %w", dataDir, err)
return s, fmt.Errorf("edge: mkdir %s: %w", dataDir, err)
}
db, err := cek.Open(filepath.Join(dataDir, "gateway.db"))
if err != nil {
return s, fmt.Errorf("gatewaypolicy: open: %w", err)
return s, fmt.Errorf("edge: open: %w", err)
}
db.SetMaxOpenConns(1) // one writer; the file lock serializes.
for _, pragma := range []string{
@@ -194,7 +194,7 @@ func New(dataDir, adminOrg string, static Policy) (*Store, error) {
} {
if _, err := db.Exec(pragma); err != nil {
_ = db.Close()
return s, fmt.Errorf("gatewaypolicy: pragma: %w", err)
return s, fmt.Errorf("edge: pragma: %w", err)
}
}
if _, err := db.Exec(`CREATE TABLE IF NOT EXISTS policy (
@@ -203,7 +203,7 @@ func New(dataDir, adminOrg string, static Policy) (*Store, error) {
updated_at INTEGER NOT NULL DEFAULT 0
)`); err != nil {
_ = db.Close()
return s, fmt.Errorf("gatewaypolicy: migrate: %w", err)
return s, fmt.Errorf("edge: migrate: %w", err)
}
s.db = db
return s, nil
@@ -229,11 +229,11 @@ func (s *Store) Get(ctx context.Context, org string) (Policy, bool, error) {
return Policy{}, false, nil
}
if err != nil {
return Policy{}, false, fmt.Errorf("gatewaypolicy: get %q: %w", org, err)
return Policy{}, false, fmt.Errorf("edge: get %q: %w", org, err)
}
var p Policy
if err := json.Unmarshal([]byte(doc), &p); err != nil {
return Policy{}, false, fmt.Errorf("gatewaypolicy: decode %q: %w", org, err)
return Policy{}, false, fmt.Errorf("edge: decode %q: %w", org, err)
}
return p, true, nil
}
@@ -243,7 +243,7 @@ func (s *Store) Get(ctx context.Context, org string) (Policy, bool, error) {
// a write must never silently vanish.
func (s *Store) Put(ctx context.Context, org string, p Policy) (Policy, error) {
if s == nil || s.db == nil {
return Policy{}, fmt.Errorf("gatewaypolicy: store unavailable")
return Policy{}, fmt.Errorf("edge: store unavailable")
}
cur, _, err := s.Get(ctx, org)
if err != nil {
@@ -254,13 +254,13 @@ func (s *Store) Put(ctx context.Context, org string, p Policy) (Policy, error) {
next.UpdatedBy = p.UpdatedBy
doc, err := json.Marshal(next)
if err != nil {
return Policy{}, fmt.Errorf("gatewaypolicy: encode: %w", err)
return Policy{}, fmt.Errorf("edge: encode: %w", err)
}
if _, err := s.db.ExecContext(ctx,
`INSERT INTO policy (org, doc, updated_at) VALUES (?,?,?)
ON CONFLICT(org) DO UPDATE SET doc=excluded.doc, updated_at=excluded.updated_at`,
org, string(doc), next.UpdatedAt); err != nil {
return Policy{}, fmt.Errorf("gatewaypolicy: put %q: %w", org, err)
return Policy{}, fmt.Errorf("edge: put %q: %w", org, err)
}
s.invalidate()
return next, nil
@@ -274,7 +274,7 @@ func (s *Store) Put(ctx context.Context, org string, p Policy) (Policy, error) {
// platform row regardless of which tenant they are currently viewing.
func (s *Store) PutPlatform(ctx context.Context, p Policy) (Policy, error) {
if s == nil {
return Policy{}, fmt.Errorf("gatewaypolicy: nil store")
return Policy{}, fmt.Errorf("edge: nil store")
}
return s.Put(ctx, s.adminOrg, p)
}
@@ -1,4 +1,4 @@
package gatewaypolicy
package edge
import (
"context"
+5 -5
View File
@@ -1,6 +1,6 @@
// Package gateway is the /v1/gateway subsystem: the RUNTIME config plane for
// the cloud edge ("gateway role"). It serves GET/PUT over the SAME
// gatewaypolicy.Store the EdgeCORS/EdgeRateLimit middleware and ScopeRateLimit
// edge.Store the EdgeCORS/EdgeRateLimit middleware and ScopeRateLimit
// read live, so an operator retunes the CORS allowlist, the pre-auth per-IP flood
// cap, or a tenant's authenticated rate ceiling with NO redeploy — replacing the
// gateway's baked-into-an-image KrakenD config.
@@ -30,14 +30,14 @@ import (
"fmt"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/gatewaypolicy"
"github.com/hanzoai/cloud/clients/gateway/edge"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
)
// state is gatewaysvc's own data; shared deps live in the embedded cloud.Base.
type state struct {
store *gatewaypolicy.Store
store *edge.Store
}
var mounted *cloud.Service[state]
@@ -96,7 +96,7 @@ func put(s *cloud.Service[state], c *zip.Ctx) error {
if !ok {
return zip.ErrForbidden("a validated principal is required")
}
var in gatewaypolicy.Policy
var in edge.Policy
if err := json.Unmarshal(c.Body(), &in); err != nil {
return zip.ErrBadRequest("invalid JSON body")
}
@@ -125,7 +125,7 @@ func put(s *cloud.Service[state], c *zip.Ctx) error {
// method allowlist. Only the per-org fields are forwarded, so a tenant can never
// smuggle a platform knob into its own row. Scoped to the caller's org (never a
// body-supplied org); a SuperAdmin may target a specific tenant with ?org=<slug>.
orgCfg := gatewaypolicy.Policy{
orgCfg := edge.Policy{
OrgRPM: in.OrgRPM,
CacheTTLSec: in.CacheTTLSec,
CachePaths: in.CachePaths,
+6 -6
View File
@@ -9,16 +9,16 @@ import (
"testing"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/gatewaypolicy"
"github.com/hanzoai/cloud/clients/gateway/edge"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
// mountApp wires /v1/gateway over a real temp-dir store and returns both so a test
// can assert the HTTP surface and the persisted state.
func mountApp(t *testing.T) (*zip.App, *gatewaypolicy.Store) {
func mountApp(t *testing.T) (*zip.App, *edge.Store) {
t.Helper()
st, err := gatewaypolicy.New(t.TempDir(), "admin", gatewaypolicy.Policy{PerIPRPM: 100, WindowSec: 1})
st, err := edge.New(t.TempDir(), "admin", edge.Policy{PerIPRPM: 100, WindowSec: 1})
if err != nil {
t.Fatalf("store: %v", err)
}
@@ -75,7 +75,7 @@ func TestOrgAdmin_SetsOwnOrgRPM(t *testing.T) {
if code != 200 {
t.Fatalf("GET: %d", code)
}
var p gatewaypolicy.Policy
var p edge.Policy
if err := json.Unmarshal([]byte(body), &p); err != nil {
t.Fatalf("decode: %v (%s)", err, body)
}
@@ -132,13 +132,13 @@ func TestPut_EmptyBodyRejected(t *testing.T) {
}
// getPolicy GETs the effective config for the given identity and decodes it.
func getPolicy(t *testing.T, app *zip.App, hdr map[string]string) gatewaypolicy.Policy {
func getPolicy(t *testing.T, app *zip.App, hdr map[string]string) edge.Policy {
t.Helper()
code, body := call(t, app, http.MethodGet, "/v1/gateway/config", "", hdr)
if code != 200 {
t.Fatalf("GET: %d (%s)", code, body)
}
var p gatewaypolicy.Policy
var p edge.Policy
if err := json.Unmarshal([]byte(body), &p); err != nil {
t.Fatalf("decode: %v (%s)", err, body)
}
@@ -1,15 +1,16 @@
// Package gojabase is the REUSABLE read-write-Base goja host: it runs a Hanzo
// subsystem's self-contained JS bundle (globalThis.handle) inside dop251/goja and
// gives that bundle PERSISTENCE over per-tenant Base/SQLite, injected as native
// host globals. It is the storage-bearing sibling of clients/goja (which is the
// pure JS engine that plans/pricing use with a read-only catalog).
// base.go is the REUSABLE read-write-Base variant of this goja host: NewBase runs
// a Hanzo subsystem's self-contained JS bundle (globalThis.handle) inside
// dop251/goja and gives that bundle PERSISTENCE over per-tenant Base/SQLite,
// injected as native host globals. It is the storage-bearing constructor of THIS
// package — the Base binding is an OPTION: reach for New (above) for a read-only
// catalog bundle (plans/pricing), NewBase here when the bundle stores data.
//
// ONE-AND-ONLY-ONE-WAY. Any subsystem that wants "run my TS business logic in
// goja, persist to Base per tenant" uses THIS package: pass a Bundle + a per-
// tenant Schema (DDL) + the DataDir, get a Host, and Dispatch(ctx, tenant, req).
// goja, persist to Base per tenant" uses NewBase: pass a Bundle + a per-tenant
// Schema (DDL) + the DataDir, get a *BaseHost, and Dispatch(ctx, tenant, req).
// captable is the pilot; esign (#100) and dataroom (#101) reuse it unchanged —
// the binding carries ZERO domain logic (no cap-table, no signatures, no rooms),
// only the engine + the Base bridge.
// only the engine (goja.Host) + the Base bridge.
//
// # Host contract (what the binding injects onto the runtime per dispatch)
//
@@ -33,7 +34,8 @@
// So a request is all-or-nothing without any JS-visible transaction API — a
// multi-statement mutation (e.g. a share transfer: delete source + insert target)
// is atomic for free, and a validation 400 leaves the DB untouched.
package gojabase
package goja
import (
"context"
@@ -44,34 +46,29 @@ import (
"errors"
"fmt"
"time"
"github.com/hanzoai/cloud/clients/goja"
)
// BlobStore is the ONE object-storage seam a bundle uses to persist large binary
// payloads OUTSIDE its per-tenant SQLite (e.g. sign's PDFs — a 32 MiB base64 blob
// in a TEXT column would bloat the tenant DB and get copied on every read). The
// cloud VFS/S3 data plane (deps.VFS) satisfies it, exactly as clients/dataroom
// already uses it for document bytes. Keys are opaque; gojabase tenant-scopes them.
// already uses it for document bytes. Keys are opaque; the binding tenant-scopes them.
type BlobStore interface {
Put(ctx context.Context, key string, payload []byte) error
Get(ctx context.Context, key string) ([]byte, error)
}
// Response mirrors the JS-side {status, body} (reused from clients/goja).
type Response = goja.Response
// Request is the dispatch envelope. The binding adds the tenant (as orgId) and
// BaseRequest is the dispatch envelope. The binding adds the tenant (as orgId) and
// the Base bridge; the caller supplies route/params/query/body.
type Request struct {
type BaseRequest struct {
Route string
Params map[string]string
Query map[string]string
Body any
}
// Config configures a Host.
type Config struct {
// BaseConfig configures a BaseHost.
type BaseConfig struct {
// Name identifies the subsystem ("captable", "esign", "dataroom"). It names
// the goja host AND the per-tenant data subdir ({DataDir}/{Name}/).
Name string
@@ -82,7 +79,7 @@ type Config struct {
Schema string
// DataDir is the deployment data root; per-tenant files land at
// {DataDir}/{Name}/{TenantSegment(tenant)}.db (injective, traversal-safe
// base32 of the raw org bytes — see gojabase/store.go TenantSegment).
// base32 of the raw org bytes — see basestore.go TenantSegment).
DataDir string
// OnOpen is an optional per-tenant seed hook run ONCE after migration (e.g.
// captable seeds the tenant's company row). It runs outside the per-request
@@ -93,31 +90,32 @@ type Config struct {
// provide that a subsystem implements in Go (e.g. esign injects __pdf =
// { stamp, sign } for PDF rendering + x509/PKCS#7 signing). Values are Go
// funcs or map[string]any of Go funcs (goja exposes them as callable JS). They
// are process-global (set once at New), not per-tenant; the binding stays
// are process-global (set once at NewBase), not per-tenant; the binding stays
// domain-free. May be nil. A key MUST NOT collide with __db/__newId/__now/__blob.
HostFns map[string]any
// Blob is the OPTIONAL object-storage seam (see BlobStore). When set, gojabase
// Blob is the OPTIONAL object-storage seam (see BlobStore). When set, the binding
// injects globalThis.__blob = { put(key, b64), get(key) -> b64 } on every
// Dispatch, bound to the tenant: keys are prefixed with {Name}/{TenantSegment}
// so a bundle can NEVER address another tenant's blob. Payloads cross as base64
// strings (goja-friendly); gojabase decodes/encodes at the boundary so the
// strings (goja-friendly); the binding decodes/encodes at the boundary so the
// bundle never handles raw bytes. nil ⇒ no __blob is injected. This is the ONE
// way a bundle keeps big binaries out of its per-tenant SQLite.
Blob BlobStore
}
// Host is a compiled bundle + its per-tenant Base stores. Safe for concurrent use.
type Host struct {
// BaseHost is a compiled bundle + its per-tenant Base stores. Safe for concurrent use.
type BaseHost struct {
name string
engine *goja.Host
engine *Host
stores *stores
hostFns map[string]any
blob BlobStore
}
// New compiles the bundle (via clients/goja) and prepares the per-tenant store
// manager. It does NOT open any tenant DB — those open lazily on first Dispatch.
func New(cfg Config) (*Host, error) {
// NewBase compiles the bundle (via the goja engine, New) and prepares the
// per-tenant store manager. It does NOT open any tenant DB — those open lazily on
// first Dispatch.
func NewBase(cfg BaseConfig) (*BaseHost, error) {
if cfg.Name == "" {
return nil, errors.New("gojabase: Config.Name required")
}
@@ -127,11 +125,11 @@ func New(cfg Config) (*Host, error) {
if cfg.DataDir == "" {
return nil, fmt.Errorf("gojabase[%s]: Config.DataDir required", cfg.Name)
}
engine, err := goja.New(goja.Config{Name: cfg.Name, Bundle: cfg.Bundle})
engine, err := New(Config{Name: cfg.Name, Bundle: cfg.Bundle})
if err != nil {
return nil, err
}
return &Host{
return &BaseHost{
name: cfg.Name,
engine: engine,
stores: newStores(cfg.Name, cfg.DataDir, cfg.Schema, cfg.OnOpen),
@@ -145,7 +143,7 @@ func New(cfg Config) (*Host, error) {
// globalThis.handle. It commits on a <400 non-throwing response and rolls back
// otherwise. tenant MUST be a validated principal's org (the caller resolves it,
// e.g. via clients/principal.Org) — the binding does not itself authenticate.
func (h *Host) Dispatch(ctx context.Context, tenant string, req Request) (*Response, error) {
func (h *BaseHost) Dispatch(ctx context.Context, tenant string, req BaseRequest) (*Response, error) {
db, release, err := h.stores.acquire(ctx, tenant)
if err != nil {
return nil, err
@@ -178,7 +176,7 @@ func (h *Host) Dispatch(ctx context.Context, tenant string, req Request) (*Respo
for k, v := range h.hostFns {
globals[k] = v
}
resp, err := h.engine.DispatchWith(ctx, goja.Request{
resp, err := h.engine.DispatchWith(ctx, Request{
Route: req.Route,
Params: req.Params,
Query: req.Query,
@@ -198,7 +196,7 @@ func (h *Host) Dispatch(ctx context.Context, tenant string, req Request) (*Respo
}
// Close closes every open tenant DB and drops the goja engine. Idempotent.
func (h *Host) Close() error {
func (h *BaseHost) Close() error {
err := h.stores.closeAll()
if h.engine != nil {
_ = h.engine.Close()
@@ -229,8 +227,8 @@ func newBridge(ctx context.Context, q execQuerier) map[string]any {
// tenant. Keys are namespaced to {name}/{TenantSegment(tenant)}/ so a bundle can
// only ever address its OWN tenant's objects — cross-tenant isolation is a host
// property, using the same injective encoding the per-tenant DB file uses. The
// bundle handles base64 strings only; gojabase decodes/encodes at the boundary.
func (h *Host) blobBridge(ctx context.Context, tenant string) map[string]any {
// bundle handles base64 strings only; the binding decodes/encodes at the boundary.
func (h *BaseHost) blobBridge(ctx context.Context, tenant string) map[string]any {
prefix := h.name + "/" + TenantSegment(tenant) + "/"
return map[string]any{
"put": func(key, b64 string) error {
@@ -1,4 +1,4 @@
package gojabase
package goja
import (
"context"
@@ -42,9 +42,9 @@ const tinySchema = `CREATE TABLE IF NOT EXISTS kv(
org TEXT NOT NULL, k TEXT NOT NULL, v TEXT NOT NULL, at INTEGER NOT NULL,
PRIMARY KEY(org,k));`
func newTinyHost(t *testing.T, onOpen func(context.Context, string, *sql.DB) error) *Host {
func newTinyHost(t *testing.T, onOpen func(context.Context, string, *sql.DB) error) *BaseHost {
t.Helper()
h, err := New(Config{
h, err := NewBase(BaseConfig{
Name: "tiny",
Bundle: []byte(tinyBundle),
Schema: tinySchema,
@@ -63,7 +63,7 @@ func TestRoundTrip(t *testing.T) {
ctx := context.Background()
// write
resp, err := h.Dispatch(ctx, "acme", Request{Route: "put", Body: map[string]any{"k": "founder", "v": "ada"}})
resp, err := h.Dispatch(ctx, "acme", BaseRequest{Route: "put", Body: map[string]any{"k": "founder", "v": "ada"}})
if err != nil {
t.Fatal(err)
}
@@ -72,7 +72,7 @@ func TestRoundTrip(t *testing.T) {
}
// read it back (a fresh dispatch → committed data must be visible)
resp, err = h.Dispatch(ctx, "acme", Request{Route: "get", Params: map[string]string{"k": "founder"}})
resp, err = h.Dispatch(ctx, "acme", BaseRequest{Route: "get", Params: map[string]string{"k": "founder"}})
if err != nil {
t.Fatal(err)
}
@@ -90,14 +90,14 @@ func TestPerRequestRollback(t *testing.T) {
ctx := context.Background()
// (a) caught-error path: write then return 500 → status >= 400 → rollback.
resp, err := h.Dispatch(ctx, "acme", Request{Route: "boom"})
resp, err := h.Dispatch(ctx, "acme", BaseRequest{Route: "boom"})
if err != nil {
t.Fatal(err)
}
if resp.Status != 500 {
t.Fatalf("boom status=%d, want 500", resp.Status)
}
resp, err = h.Dispatch(ctx, "acme", Request{Route: "get", Params: map[string]string{"k": "ghost"}})
resp, err = h.Dispatch(ctx, "acme", BaseRequest{Route: "get", Params: map[string]string{"k": "ghost"}})
if err != nil {
t.Fatal(err)
}
@@ -106,10 +106,10 @@ func TestPerRequestRollback(t *testing.T) {
}
// (b) uncaught-throw path: surfaces as a host error, and still rolls back.
if _, err := h.Dispatch(ctx, "acme", Request{Route: "throw"}); err == nil {
if _, err := h.Dispatch(ctx, "acme", BaseRequest{Route: "throw"}); err == nil {
t.Fatal("expected a host error from an uncaught JS throw")
}
resp, err = h.Dispatch(ctx, "acme", Request{Route: "get", Params: map[string]string{"k": "ghost2"}})
resp, err = h.Dispatch(ctx, "acme", BaseRequest{Route: "get", Params: map[string]string{"k": "ghost2"}})
if err != nil {
t.Fatal(err)
}
@@ -122,11 +122,11 @@ func TestTenantIsolation(t *testing.T) {
h := newTinyHost(t, nil)
ctx := context.Background()
if _, err := h.Dispatch(ctx, "acme", Request{Route: "put", Body: map[string]any{"k": "secret", "v": "acme-only"}}); err != nil {
if _, err := h.Dispatch(ctx, "acme", BaseRequest{Route: "put", Body: map[string]any{"k": "secret", "v": "acme-only"}}); err != nil {
t.Fatal(err)
}
// A different tenant must not see acme's row (separate DB file).
resp, err := h.Dispatch(ctx, "globex", Request{Route: "get", Params: map[string]string{"k": "secret"}})
resp, err := h.Dispatch(ctx, "globex", BaseRequest{Route: "get", Params: map[string]string{"k": "secret"}})
if err != nil {
t.Fatal(err)
}
@@ -143,7 +143,7 @@ func TestOnOpenSeed(t *testing.T) {
return err
})
ctx := context.Background()
resp, err := h.Dispatch(ctx, "acme", Request{Route: "get", Params: map[string]string{"k": "seed"}})
resp, err := h.Dispatch(ctx, "acme", BaseRequest{Route: "get", Params: map[string]string{"k": "seed"}})
if err != nil {
t.Fatal(err)
}
@@ -1,4 +1,4 @@
package gojabase
package goja
import (
"container/list"
@@ -1,4 +1,4 @@
package gojabase
package goja
import (
"context"
@@ -84,7 +84,7 @@ func TestLRUEvictionReopen(t *testing.T) {
orgs := []string{"one", "two", "three", "four", "five"}
for _, org := range orgs {
resp, err := h.Dispatch(ctx, org, Request{Route: "put", Body: map[string]any{"k": "name", "v": org + "-val"}})
resp, err := h.Dispatch(ctx, org, BaseRequest{Route: "put", Body: map[string]any{"k": "name", "v": org + "-val"}})
if err != nil {
t.Fatalf("put %s: %v", org, err)
}
@@ -102,7 +102,7 @@ func TestLRUEvictionReopen(t *testing.T) {
// Every tenant — including the ones long since evicted — still reads back its
// own committed row (data persisted to disk; reopen re-migrated + re-read).
for _, org := range orgs {
resp, err := h.Dispatch(ctx, org, Request{Route: "get", Params: map[string]string{"k": "name"}})
resp, err := h.Dispatch(ctx, org, BaseRequest{Route: "get", Params: map[string]string{"k": "name"}})
if err != nil {
t.Fatalf("get %s: %v", org, err)
}
@@ -147,7 +147,7 @@ func TestConcurrentMultiTenantDispatch(t *testing.T) {
go func(org string, i int) {
defer wg.Done()
k := "k" + strconv.Itoa(i)
resp, err := h.Dispatch(ctx, org, Request{
resp, err := h.Dispatch(ctx, org, BaseRequest{
Route: "put",
Body: map[string]any{"k": k, "v": org + "|" + k},
})
@@ -170,7 +170,7 @@ func TestConcurrentMultiTenantDispatch(t *testing.T) {
// Integrity: each tenant holds EXACTLY its own perTenant rows, values intact,
// none leaked from a sibling (esp. the case/separator-variant pairs).
for _, org := range tenants {
resp, err := h.Dispatch(ctx, org, Request{Route: "list"})
resp, err := h.Dispatch(ctx, org, BaseRequest{Route: "list"})
if err != nil {
t.Fatalf("list %s: %v", org, err)
}
+6 -6
View File
@@ -18,10 +18,10 @@
// READ-WRITE variant: this package hosts bundles with a read-only catalog
// injected once at New (plans/pricing). Subsystems that need PERSISTENCE — a
// bundle that reads AND writes per-tenant Base/SQLite (captable #97, esign #100,
// dataroom #101) — use the sibling clients/gojabase, which builds on THIS engine
// (via DispatchWith) and injects a tenant-bound __db bridge per request. Reach
// for gojabase when your bundle stores data; reach for goja directly only for a
// read-only bundle.
// dataroom #101) — use NewBase (base.go / basestore.go), the Base binding folded
// into THIS package: it builds on this engine (via DispatchWith) and injects a
// tenant-bound __db bridge per request. Reach for NewBase when your bundle stores
// data; reach for New directly only for a read-only bundle.
package goja
import (
@@ -184,7 +184,7 @@ func (h *Host) Dispatch(ctx context.Context, req Request) (*Response, error) {
// DispatchWith is Dispatch plus a set of per-call NATIVE globals installed on the
// runtime immediately before handle() runs (left in place until the next
// dispatch on that slot overwrites them). This is the read-WRITE extension of
// the read-only plan/pricing pattern: clients/gojabase passes a tenant-bound
// the read-only plan/pricing pattern: the NewBase binding passes a tenant-bound
// __db bridge (+ __newId/__now) here so a bundle's SQL calls hit the right
// per-tenant Base. The slot is held exclusively for the whole call (withSlot
// serializes it), so installing globals on the shared runtime is race-free, and
@@ -256,7 +256,7 @@ func (h *Host) DispatchWith(ctx context.Context, req Request, hostGlobals map[st
return fmt.Errorf("gojahost[%s]: handle(%s) returned %T, want object", h.name, req.Route, exported)
}
// Fail closed: a response with no EXPLICIT, valid numeric status must NOT
// be treated as success. gojabase's per-request transaction commits iff
// be treated as success. NewBase's per-request transaction commits iff
// status < 400, so a defaulted 200 would silently PERSIST a bundle that
// forgot to set a status — on a boundary whose safety IS the status gate.
// Default 500 (which rolls the transaction back); only a bundle-provided
-95
View File
@@ -1,95 +0,0 @@
# `clients/gojabase` — the reusable read-write-Base goja host
`gojabase` is the **one-and-only-one-way** to run a Hanzo subsystem's
self-contained JS/TS business logic (a goja bundle exposing `globalThis.handle`)
in-process **with persistence over per-tenant Base/SQLite**. It is the
storage-bearing sibling of [`clients/goja`](../goja) (the pure JS engine that
`plans`/`pricing` use with a read-only catalog).
captable (#97) is the pilot. **esign (#100) and dataroom (#101) reuse this
package unchanged** — it carries ZERO domain logic (no cap table, no signatures,
no rooms), only the engine + the Base bridge.
## What a subsystem provides
```go
host, err := gojabase.New(gojabase.Config{
Name: "captable", // names the goja host AND the data subdir
Bundle: bundleBytes, // the go:embed'd bundle (globalThis.handle)
Schema: schemaDDL, // per-tenant SQLite DDL (CREATE TABLE IF NOT EXISTS …)
DataDir: deps.DataDir, // files land at {DataDir}/{Name}/{tenantSlug}.db
OnOpen: seedRow, // optional per-tenant seed, run once after migrate
})
```
Then, in each zip route handler, resolve the tenant from the **validated**
principal and dispatch:
```go
org, ok := principal.Tenant(c) // gojabase does NOT authenticate; the leaf does
if !ok { return zip.ErrForbidden("X-Org-Id required") }
resp, err := host.Dispatch(c.Context(), org, gojabase.Request{
Route: "stakeholders.add",
Params: map[string]string{"id": c.Param("id")},
Body: decodedJSONBody, // any (map / slice / scalar), or nil for reads
})
c.SetHeader("Content-Type", "application/json")
return c.Bytes(resp.Status, resp.Body) // resp is {Status int, Body json.RawMessage}
```
## What the bundle sees (the host contract)
gojabase injects these native globals onto the runtime **per dispatch**, bound to
the tenant's DB + a per-request transaction:
```
globalThis.__db.query(sql, args) -> row objects (SELECT; TEXT→string)
globalThis.__db.exec(sql, args) -> { changes, lastId } (INSERT/UPDATE/DELETE)
globalThis.__newId() -> collision-resistant id (crypto/rand, 128-bit)
globalThis.__now() -> unix milliseconds
globalThis.__blob.put(key, b64) -> (only when Config.Blob is set) store bytes off-DB
globalThis.__blob.get(key) -> b64 (only when Config.Blob is set) read them back
globalThis.handle({ route, params, query, orgId, body }) -> { status, body }
```
`__blob` is the OPTIONAL object-storage seam (`Config.Blob`, backed by the cloud
VFS/S3). Use it for large binaries that must NOT bloat the per-tenant SQLite — e.g.
sign's PDFs: the bundle stores the bytes with `__blob.put` and keeps only the
returned key in a column. Keys are tenant-scoped by the host (`{Name}/{TenantSegment}`),
so a bundle can never reach another tenant's blobs. Payloads cross as base64.
`orgId` is the tenant passed to `Dispatch` — the bundle uses it to scope rows
(defence in depth on top of the per-tenant file). `args` is a positional array
bound to `?` placeholders. The Go host owns the schema (migrations); the bundle
issues SQL against it — column names are the coupling, so keep them in sync.
## Guarantees
- **Per-tenant isolation** — one SQLite file per org
(`{DataDir}/{Name}/{TenantSegment}.db`), opened lazily, migrated once, pooled
(LRU-capped + idle-evicted). `TenantSegment` is an **injective**, traversal-safe
encoding (lowercased unpadded base32 of the raw org bytes), so DISTINCT orgs —
including case/separator variants like `Acme`/`acme` and `a b`/`a_b` — NEVER
share a file, and the `[a-z2-7]` segment can never traverse the data tree.
- **Atomicity** — each `Dispatch` runs `handle` inside ONE transaction that
**commits iff** the response status `< 400` and `handle` did not throw;
otherwise it **rolls back**. Multi-statement mutations (e.g. a share transfer:
shrink source + insert target) are all-or-nothing for free, and a validation
400 leaves the DB untouched. `MaxOpenConns(1)` serializes writes per tenant.
- **No JS-visible transaction API** — the per-request transaction removes the
need for one; bundles just call `query`/`exec`.
## Leaf wiring (register)
Register the leaf and blank-import it in `apps/apps.go`. It mounts
under the mount-all default (empty `CLOUD_ENABLE`) — the captable/sign/dataroom
folds are NOT staged (their standalone apps are retired/empty, so the one binary
is authoritative from first write):
```go
func init() { cloud.RegisterWithShutdown("captable", 133, cloud.Typed(Mount), shutdown) }
```
See `clients/captable` for the complete reference leaf (schema, seed, routes).
+11 -1
View File
@@ -8,13 +8,18 @@ import (
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/agents"
luxlog "github.com/luxfi/log"
fiber "github.com/zap-proto/fiber/v3"
"github.com/zap-proto/zip"
)
// testTimeout bounds one in-process request. It is generous on purpose — see req().
const testTimeout = 60 * time.Second
func mountLink(t *testing.T) *zip.App {
t.Helper()
app := zip.New(zip.Config{Logger: luxlog.New("test")})
@@ -46,7 +51,12 @@ func req(t *testing.T, app *zip.App, method, path, org, user string, body any) (
if user != "" {
rq.Header.Set("X-User-Id", user)
}
resp, err := app.Fiber().Test(rq)
// The default test timeout is ONE SECOND, which measures the build box's load
// rather than this package: on a busy machine a handler that returns in
// microseconds still reports "i/o timeout", and the suite fails for a reason
// that has nothing to do with the code. The bound stays (a hang must still
// fail) — it is just far enough out to be a real hang and not a busy CPU.
resp, err := app.Fiber().Test(rq, fiber.TestConfig{Timeout: testTimeout, FailOnTimeout: true})
if err != nil {
t.Fatalf("Test %s %s: %v", method, path, err)
}
+3 -1
View File
@@ -16,7 +16,9 @@ var errNotFound = errors.New("link: not found")
// Store is the login-manager database. ONE SQLite file ({DataDir}/link.db) holds
// every org's Links; tenancy is the (org, subject) pair. It holds NO metering
// client — it is structurally incapable of charging commerce.
// client — it is structurally incapable of charging commerce, and no usage series:
// account-usage lives in its own subsystem (clients/usage), so link owns links and
// nothing usage.
type Store struct {
db *sql.DB
}
+5 -107
View File
@@ -20,22 +20,6 @@ func TestAppsGVR(t *testing.T) {
}
}
// TestCRGVRsOrder pins the read order: the fleet's kind first, the kind it
// collapsed from second. Order is load-bearing — when both kinds claim a name,
// the first one wins the row, and App is the kind that owns the Deployment.
func TestCRGVRsOrder(t *testing.T) {
got := crGVRs()
want := []schema.GroupVersionResource{appsGVR, servicesGVR}
if len(got) != len(want) {
t.Fatalf("crGVRs() = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("crGVRs()[%d] = %v, want %v", i, got[i], want[i])
}
}
}
// TestObserveFleetSeesAppCRs is the regression that matters: the live fleet is
// declared as App CRs, and the board rendered none of them. Seed only App CRs —
// the shape of the real `hanzo` namespace — and the board must list them.
@@ -63,100 +47,27 @@ func TestObserveFleetSeesAppCRs(t *testing.T) {
}
}
// TestObserveFleetSeesBothKinds — the fleet is mid-collapse: system workloads are
// App CRs, untransitioned ones are still Service CRs. The board must show every
// workload regardless of which kind declares it.
func TestObserveFleetSeesBothKinds(t *testing.T) {
s := fakeService(
appCRObj("cloud", "hanzo", "ghcr.io/hanzoai/cloud", "v1.801.38"),
serviceCRObj("legacy", "hanzo", "ghcr.io/hanzoai/legacy", "v0.1.0"),
)
views, err := observeFleet(s, context.Background())
if err != nil {
t.Fatalf("observeFleet: %v", err)
}
if len(views) != 2 {
t.Fatalf("got %d rows, want 2 (one App + one Service)", len(views))
}
}
// TestObserveFleetDedupesACollidingName — when an App CR and a Service CR both
// claim one name, there is still only ONE workload: a Deployment has one
// controller ownerRef, and the operator's Claim guard gives it to the App. The
// board reports one row, from the App.
func TestObserveFleetDedupesACollidingName(t *testing.T) {
s := fakeService(
appCRObj("commerce-admin", "hanzo", "ghcr.io/hanzoai/commerce-admin", "0.2.0-amd64"),
serviceCRObj("commerce-admin", "hanzo", "ghcr.io/hanzoai/commerce-admin", "0.1.0-amd64"),
)
views, err := observeFleet(s, context.Background())
if err != nil {
t.Fatalf("observeFleet: %v", err)
}
if len(views) != 1 {
t.Fatalf("got %d rows for one workload, want 1 — a colliding name is one workload, not two", len(views))
}
if views[0].DeclaredTag != "0.2.0-amd64" {
t.Fatalf("declaredTag = %q, want the App's 0.2.0-amd64 — the App owns the Deployment", views[0].DeclaredTag)
}
}
// TestResolveTargetPrefersTheAppKind — resolution reports which kind holds the
// workload, because the deploy path's behavior depends on it.
func TestResolveTargetPrefersTheAppKind(t *testing.T) {
// TestResolveTargetFindsTheApp — resolution reports the namespace an App lives in,
// scanning production first.
func TestResolveTargetFindsTheApp(t *testing.T) {
s := fakeService(appCRObj("cloud", "hanzo", "ghcr.io/hanzoai/cloud", "v1.801.38"))
ns, gvr, err := resolveTarget(s, context.Background(), "cloud")
ns, err := resolveTarget(s, context.Background(), "cloud")
if err != nil {
t.Fatalf("resolveTarget: %v", err)
}
if ns != "hanzo" {
t.Errorf("ns = %q, want hanzo", ns)
}
if gvr != appsGVR {
t.Errorf("gvr = %v, want appsGVR", gvr)
}
}
// TestResolveTargetFindsAServiceKind — a Service-declared workload still resolves,
// and reports the Service kind so the deploy path patches it.
func TestResolveTargetFindsAServiceKind(t *testing.T) {
s := fakeService(serviceCRObj("legacy", "hanzo-devnet", "ghcr.io/hanzoai/legacy", "v0.1.0"))
ns, gvr, err := resolveTarget(s, context.Background(), "legacy")
if err != nil {
t.Fatalf("resolveTarget: %v", err)
}
if ns != "hanzo-devnet" {
t.Errorf("ns = %q, want hanzo-devnet", ns)
}
if gvr != servicesGVR {
t.Errorf("gvr = %v, want servicesGVR", gvr)
}
}
// TestResolveTargetIsNotFoundForAnAbsentWorkload — fail closed, no silent default.
func TestResolveTargetIsNotFoundForAnAbsentWorkload(t *testing.T) {
s := fakeService()
if _, _, err := resolveTarget(s, context.Background(), "ghost"); err == nil {
if _, err := resolveTarget(s, context.Background(), "ghost"); err == nil {
t.Fatal("resolveTarget on an absent workload returned nil error, want not-found")
}
}
// TestKindAtPinsTheNamespace — an explicit namespace must not silently fall back
// to another namespace's workload of the same name.
func TestKindAtPinsTheNamespace(t *testing.T) {
s := fakeService(appCRObj("cloud", "hanzo", "ghcr.io/hanzoai/cloud", "v1.801.38"))
if _, err := kindAt(s, context.Background(), "hanzo-devnet", "cloud"); err == nil {
t.Fatal("kindAt found `cloud` in hanzo-devnet, where it does not exist — an explicit namespace must be honored")
}
gvr, err := kindAt(s, context.Background(), "hanzo", "cloud")
if err != nil {
t.Fatalf("kindAt(hanzo, cloud): %v", err)
}
if gvr != appsGVR {
t.Errorf("gvr = %v, want appsGVR", gvr)
}
}
// TestReleaseRefusesAGitDeclaredWorkload — an App CR is synced from universe by
// Hanzo CD with selfHeal, so patching it here is reverted on the next sync. The
// release must refuse and say where the declaration lives, rather than report a
@@ -186,19 +97,6 @@ func TestReleaseRefusesAGitDeclaredWorkload(t *testing.T) {
}
}
// TestReleaseStillPatchesAServiceDeclaredWorkload — a Service CR has no git
// declarer (cloud and kubectl write them directly), so the release still patches.
func TestReleaseStillPatchesAServiceDeclaredWorkload(t *testing.T) {
s := fakeService(serviceCRObj("legacy", "hanzo-devnet", "ghcr.io/hanzoai/legacy", "v0.1.0"))
ns, tag, changed, err := releaseService(s, context.Background(), "legacy", "ghcr.io/hanzoai/legacy:v0.2.0")
if err != nil {
t.Fatalf("releaseService on a Service-declared workload: %v", err)
}
if !changed || ns != "hanzo-devnet" || tag != "v0.2.0" {
t.Fatalf("got (ns=%q tag=%q changed=%v), want (hanzo-devnet, v0.2.0, true)", ns, tag, changed)
}
}
// TestObserveFleetSkipsAnEmptyNamespace — a namespace with no workloads is not an
// error; the board still renders the reachable ones.
func TestObserveFleetSkipsAnEmptyNamespace(t *testing.T) {
+71 -210
View File
@@ -1,29 +1,28 @@
// paas.go — the cluster-facing half of the native Hanzo PaaS control plane.
//
// It mounts /v1/paas/* on the unified cloud binary and speaks to the SAME
// operator surface the standalone platform's deploy-executor drove: the
// `hanzo.ai/v1` `services` CustomResource. Two responsibilities, both a straight
// port of the Node platform:
// It mounts /v1/paas/* on the unified cloud binary and reads the operator's
// `hanzo.ai/v1` `App` CustomResource the one workload kind the fleet runs on:
//
// GET /v1/paas/apps — the fleet drift board (inventory.ts): list every
// operator Service CR across the platform
// namespaces, read declared vs running tag +
// health from the CR (+ its status), and attach
// the drift verdict (drift.go / apps-drift.ts).
// GET /v1/paas/apps/:app — one service row by CR name.
// POST /v1/paas/apps/:app/deploy— deploy a new image tag by merge-patching the
// Service CR's `.spec.image` (deploy-executor.ts).
// The operator reconciles the rollout; cloud never
// reimplements a deployer.
// GET /v1/paas/health — real k8s reachability + Service CRD presence.
// operator App CR across the platform namespaces,
// read declared vs running tag + health from the CR
// (+ its status), and attach the drift verdict
// (drift.go / apps-drift.ts).
// GET /v1/paas/apps/:app — one app row by CR name.
// POST /v1/paas/apps/:app/deploy— refuse (409): an App CR is declared in universe
// git (infra/k8s/operator/crs/) and reconciled by
// Hanzo CD with selfHeal, so a patch here is
// reverted on the next sync. The response names the
// git path to commit the tag to.
// GET /v1/paas/health — real k8s reachability + App CRD presence.
//
// SECURITY — every route is SUPERADMIN ONLY, fail-closed, gated on the SAME
// predicate the rest of cloud uses: c.IsAdmin() (true only for a JWT-validated
// principal whose org is the admin org, matching the gateway's admin-guard — see
// clients/admin). Unlike clients/ml (per-tenant namespaces), the PaaS control
// plane reads and mutates SYSTEM Service CRs across the whole fleet, so it is
// admin-only: a tenant must never patch another org's — or a platform — service.
// The user-facing PaaS view lives in console; users never call this surface.
// plane reads SYSTEM App CRs across the whole fleet, so it is admin-only: a tenant
// must never observe another org's — or a platform — app. The user-facing PaaS
// view lives in console; users never call this surface.
//
// k8s client: built in-process from the in-cluster service account
// (rest.InClusterConfig) with a KUBECONFIG fallback for local/dev — the identical
@@ -34,7 +33,6 @@ package paas
import (
"context"
"encoding/json"
"fmt"
"net/http"
"regexp"
@@ -48,34 +46,15 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime/schema"
k8stypes "k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/dynamic"
"k8s.io/client-go/rest"
"k8s.io/client-go/tools/clientcmd"
)
// The operator's workload CRs. `App` is the collapsed kind the fleet runs on;
// `Service` is the kind it collapsed from and is still served, still reconciled,
// and still the kind cloud writes for tenant workloads (clients/platform). Both
// carry the same spec, so a reader that knows only one is blind to the other.
//
// Reading `services` alone made this board blind to the fleet: 69 Apps run in the
// scanned namespaces against 7 Service CRs (1 in `hanzo`), so the drift board
// rendered 7 rows for a 69-app fleet and every read of an App-declared service
// 404'd. The read order below mirrors clients/deploy's appCRGVRs(): App first
// (the fleet's kind), Service second (tenant + untransitioned CRs).
//
// Asserted in the tests; a typo here silently breaks the whole board.
var (
appsGVR = schema.GroupVersionResource{Group: "hanzo.ai", Version: "v1", Resource: "apps"}
servicesGVR = schema.GroupVersionResource{Group: "hanzo.ai", Version: "v1", Resource: "services"}
)
// crGVRs is the workload-CR read order: the fleet's kind first, the kind it
// collapsed from second. Every read walks this list.
func crGVRs() []schema.GroupVersionResource {
return []schema.GroupVersionResource{appsGVR, servicesGVR}
}
// appsGVR is the operator App CR — the one workload kind the fleet runs on. The
// drift board reads it across the platform namespaces. Asserted in the tests; a
// typo here silently breaks the whole board.
var appsGVR = schema.GroupVersionResource{Group: "hanzo.ai", Version: "v1", Resource: "apps"}
// deploymentsGVR is the live Deployment behind each Service — the source of the
// RUNNING tag (the operator Service CR status does not surface the running image,
@@ -247,30 +226,22 @@ func getApp(s *cloud.Service[state], c *zip.Ctx) error {
return zip.ErrBadRequest("app must be a DNS-1123 label")
}
for _, ns := range scanOrder() {
for _, gvr := range crGVRs() {
obj, err := s.State.dyn.Resource(gvr).Namespace(ns).Get(c.Context(), name, metav1.GetOptions{})
if err != nil {
if apierrors.IsNotFound(err) {
continue
}
return k8sErr(s, "get", err)
obj, err := s.State.dyn.Resource(appsGVR).Namespace(ns).Get(c.Context(), name, metav1.GetOptions{})
if err != nil {
if apierrors.IsNotFound(err) {
continue
}
repository, _, _ := unstructured.NestedString(obj.Object, "spec", "image", "repository")
return c.JSON(http.StatusOK, observeCR(obj, ns, nsEnv[ns], runningTagOf(s, c.Context(), ns, name, repository)))
return k8sErr(s, "get", err)
}
repository, _, _ := unstructured.NestedString(obj.Object, "spec", "image", "repository")
return c.JSON(http.StatusOK, observeCR(obj, ns, nsEnv[ns], runningTagOf(s, c.Context(), ns, name, repository)))
}
return zip.ErrNotFound("service not found in the platform namespaces")
return zip.ErrNotFound("app not found in the platform namespaces")
}
// observeFleet lists every workload CR across the scanned namespaces and maps
// each to an AppView. A namespace that does not exist / is empty is skipped,
// never fatal (the fleet board must still render the reachable namespaces).
//
// Both kinds are listed, in crGVRs() order, deduped by name within a namespace:
// one workload is one row even when an App CR and a Service CR both claim the
// name. That collision is real — a Deployment can only have one controller
// ownerRef, so the operator's Claim guard makes the App the owner — and the board
// reports the owner rather than rendering the workload twice.
// observeFleet lists every App CR across the scanned namespaces and maps each to
// an AppView. A namespace that does not exist / is empty is skipped, never fatal
// (the fleet board must still render the reachable namespaces).
func observeFleet(s *cloud.Service[state], ctx context.Context) ([]AppView, error) {
var views []AppView
for _, ns := range scanOrder() {
@@ -280,23 +251,16 @@ func observeFleet(s *cloud.Service[state], ctx context.Context) ([]AppView, erro
// whole board, so the declared/health/phase columns still render.
running := runningTagsIn(s, ctx, ns)
env := nsEnv[ns]
seen := make(map[string]bool)
for _, gvr := range crGVRs() {
list, err := s.State.dyn.Resource(gvr).Namespace(ns).List(ctx, metav1.ListOptions{})
if err != nil {
if apierrors.IsNotFound(err) {
continue
}
return nil, k8sErr(s, "list", err)
}
for i := range list.Items {
cr := &list.Items[i]
if seen[cr.GetName()] {
continue
}
seen[cr.GetName()] = true
views = append(views, observeCR(cr, ns, env, running[cr.GetName()]))
list, err := s.State.dyn.Resource(appsGVR).Namespace(ns).List(ctx, metav1.ListOptions{})
if err != nil {
if apierrors.IsNotFound(err) {
continue
}
return nil, k8sErr(s, "list", err)
}
for i := range list.Items {
cr := &list.Items[i]
views = append(views, observeCR(cr, ns, env, running[cr.GetName()]))
}
}
sort.Slice(views, func(i, j int) bool {
@@ -311,13 +275,14 @@ func observeFleet(s *cloud.Service[state], ctx context.Context) ([]AppView, erro
return views, nil
}
// ── deploy: merge-patch the Service CR image (deploy-executor.ts) ─────────────
// ── deploy: refuse a git-declared App ────────────────────────────────────────
// deploy rolls a new image tag onto a service by merge-patching ONLY the Service
// CR's `.spec.image`. The operator reconciles the rollout (Deployment update,
// rolling restart) — this is the exact contract deploy-executor.ts implemented,
// now native. Content-Type is JSON merge-patch (application/merge-patch+json),
// which the operator CRD accepts; the dynamic client's MergePatchType sets it.
// deploy refuses. Every App CR in the platform namespaces is declared in universe
// git (infra/k8s/operator/crs/) and reconciled by Hanzo CD with selfHeal, so a
// direct patch here is reverted on the next sync — the tag would appear to deploy
// and then silently roll back. The endpoint confirms the app exists (404 if not)
// and returns 409 naming the git path to commit the tag to, the one way to deploy
// it.
func deploy(s *cloud.Service[state], c *zip.Ctx) error {
if err := ready(s); err != nil {
return err
@@ -326,139 +291,35 @@ func deploy(s *cloud.Service[state], c *zip.Ctx) error {
if !appNameRE.MatchString(name) {
return zip.ErrBadRequest("app must be a DNS-1123 label")
}
var req struct {
Tag string `json:"tag"` // required — the new image tag (e.g. v1.1.3)
Repository string `json:"repository"` // optional — override image repo; else keep the CR's
Namespace string `json:"namespace"` // optional — target ns; else resolve to where the CR lives
if _, err := resolveTarget(s, c.Context(), name); err != nil {
return err
}
if err := json.Unmarshal(c.Body(), &req); err != nil {
return zip.Errorf(http.StatusBadRequest, "invalid JSON body: %v", err)
}
tag := strings.TrimSpace(req.Tag)
if tag == "" {
return zip.ErrBadRequest("'tag' is required (the image tag to deploy)")
}
if strings.ContainsAny(tag, " \t\n/") || len(tag) > 128 {
return zip.ErrBadRequest("'tag' must be a single image tag (no whitespace or '/')")
}
repo := strings.TrimSpace(req.Repository)
if repo != "" && !imageRepoRE.MatchString(repo) {
return zip.ErrBadRequest("'repository' is not a valid image repository path")
}
ns := strings.TrimSpace(req.Namespace)
var gvr schema.GroupVersionResource
if ns != "" {
if _, ok := nsEnv[ns]; !ok {
return zip.ErrBadRequest("'namespace' must be a platform namespace (hanzo|hanzo-testnet|hanzo-devnet)")
}
found, err := kindAt(s, c.Context(), ns, name)
if err != nil {
return err
}
gvr = found
} else {
resolvedNS, resolvedGVR, err := resolveTarget(s, c.Context(), name)
if err != nil {
return err
}
ns, gvr = resolvedNS, resolvedGVR
}
// A git-declared workload has a declarer already, and it is not this endpoint.
// The App CRs in these namespaces are synced from universe `infra/k8s/operator/
// crs/` by Hanzo CD with selfHeal on, so a patch here is reverted on the next
// sync — the tag would appear to deploy and then silently roll back. Refuse
// instead, and name the one way to deploy it. Service CRs carry no git
// declarer (cloud and kubectl write them directly), so they still patch.
if gvr == appsGVR {
return zip.Errorf(http.StatusConflict,
"%s is declared in git (App/%s, universe infra/k8s/operator/crs/%s.yaml) and reconciled by Hanzo CD with selfHeal — a patch here would be reverted on the next sync. Deploy it by committing the tag to that file.",
name, name, name)
}
// Build the merge-patch. When repository is omitted we patch only the tag +
// pullPolicy so an existing repo is preserved (JSON merge-patch merges keys,
// so omitting `repository` leaves the CR's value intact).
image := map[string]any{"tag": tag, "pullPolicy": "Always"}
if repo != "" {
image["repository"] = repo
}
patch, err := json.Marshal(map[string]any{"spec": map[string]any{"image": image}})
if err != nil {
return zip.Errorf(http.StatusInternalServerError, "encode patch: %v", err)
}
out, err := s.State.dyn.Resource(gvr).Namespace(ns).
Patch(c.Context(), name, k8stypes.MergePatchType, patch, metav1.PatchOptions{})
if err != nil {
switch {
case apierrors.IsNotFound(err):
return zip.ErrNotFound("service not found in namespace " + ns)
case apierrors.IsInvalid(err), apierrors.IsBadRequest(err):
return zip.Errorf(http.StatusUnprocessableEntity, "patch rejected by kubernetes: %v", err)
default:
return k8sErr(s, "patch", err)
}
}
s.Log.Info("deployed via Service CR patch",
"app", name, "namespace", ns, "tag", tag, "repository", repo,
"actor", c.User(), "requestID", c.RequestID())
// Read the effective repo from the patched CR (the caller may have omitted
// repository to keep the CR's existing value) so the running-tag container
// match uses the real declared repo.
effRepo, _, _ := unstructured.NestedString(out.Object, "spec", "image", "repository")
view := observeCR(out, ns, nsEnv[ns], runningTagOf(s, c.Context(), ns, name, effRepo))
return c.JSON(http.StatusOK, map[string]any{
"rolledOut": true,
"target": ns + "/" + name,
"reason": "patched Service/" + name + " image to " + tag,
"app": view,
})
return zip.Errorf(http.StatusConflict,
"%s is declared in git (App/%s, universe infra/k8s/operator/crs/%s.yaml) and reconciled by Hanzo CD with selfHeal — a patch here would be reverted on the next sync. Deploy it by committing the tag to that file.",
name, name, name)
}
// resolveTarget finds the namespace AND the kind a workload CR lives under,
// scanning in env order (main→test→dev) so a bare deploy targets production, and
// in crGVRs() order within a namespace. Returns a clean 404 when the CR exists
// under neither kind in any of them.
func resolveTarget(s *cloud.Service[state], ctx context.Context, name string) (string, schema.GroupVersionResource, error) {
// resolveTarget finds the namespace an App CR lives in, scanning in env order
// (main→test→dev) so a bare lookup targets production. Returns a clean 404 when
// the App exists in none of them.
func resolveTarget(s *cloud.Service[state], ctx context.Context, name string) (string, error) {
for _, ns := range scanOrder() {
for _, gvr := range crGVRs() {
if _, err := s.State.dyn.Resource(gvr).Namespace(ns).Get(ctx, name, metav1.GetOptions{}); err == nil {
return ns, gvr, nil
} else if !apierrors.IsNotFound(err) {
return "", schema.GroupVersionResource{}, k8sErr(s, "get", err)
}
}
}
return "", schema.GroupVersionResource{}, zip.ErrNotFound("service " + name + " not found in the platform namespaces")
}
// kindAt reports the workload kind present at (ns, name), or an empty GVR when
// the workload exists under neither kind there.
func kindAt(s *cloud.Service[state], ctx context.Context, ns, name string) (schema.GroupVersionResource, error) {
for _, gvr := range crGVRs() {
if _, err := s.State.dyn.Resource(gvr).Namespace(ns).Get(ctx, name, metav1.GetOptions{}); err == nil {
return gvr, nil
if _, err := s.State.dyn.Resource(appsGVR).Namespace(ns).Get(ctx, name, metav1.GetOptions{}); err == nil {
return ns, nil
} else if !apierrors.IsNotFound(err) {
return schema.GroupVersionResource{}, k8sErr(s, "get", err)
return "", k8sErr(s, "get", err)
}
}
return schema.GroupVersionResource{}, zip.ErrNotFound("service " + name + " not found in namespace " + ns)
return "", zip.ErrNotFound("app " + name + " not found in the platform namespaces")
}
// ── health ────────────────────────────────────────────────────────────────
// health is a REAL probe: it verifies the API server is reachable and that the
// App CRD — the kind the fleet runs on, and the kind the board is useless
// without — is served, and reports the actual state. 200 only when everything is
// ok; 503 + the real reason otherwise (never status-theater). Probing the
// Service CRD alone reported ok while the board rendered 7 rows for a 69-app
// fleet: the CRD it probed was served, just not the one the fleet uses. Not
// admin-gated — liveness must be probe-able by the platform/operator without a JWT.
// App CRD — the kind the fleet runs on, and the kind the board is useless without
// — is served, and reports the actual state. 200 only when everything is ok; 503
// + the real reason otherwise (never status-theater). Not admin-gated — liveness
// must be probe-able by the platform/operator without a JWT.
func health(s *cloud.Service[state], c *zip.Ctx) error {
res := map[string]any{"service": "paas", "status": "ok"}
if s.State.dyn == nil {
@@ -484,15 +345,15 @@ func ready(s *cloud.Service[state]) error {
// k8sErr maps a raw API error to an honest gateway-level error. RBAC denials name
// the missing access so the operator knows exactly what to grant the cloud service
// account (get/list/patch on services.hanzo.ai). Mirrors ml.k8sErr.
// account (get/list on apps.hanzo.ai). Mirrors ml.k8sErr.
func k8sErr(s *cloud.Service[state], op string, err error) error {
s.Log.Error("k8s op failed", "op", op, "resource", servicesGVR.Resource, "err", err)
s.Log.Error("k8s op failed", "op", op, "resource", appsGVR.Resource, "err", err)
if apierrors.IsForbidden(err) {
return zip.Errorf(http.StatusBadGateway,
"%s services: kubernetes RBAC denied (cloud service account needs %s on services.hanzo.ai): %v",
"%s apps: kubernetes RBAC denied (cloud service account needs %s on apps.hanzo.ai): %v",
op, op, err)
}
return zip.Errorf(http.StatusBadGateway, "%s services failed: %v", op, err)
return zip.Errorf(http.StatusBadGateway, "%s apps failed: %v", op, err)
}
// newDynamic builds the dynamic client from the in-cluster service account,
@@ -588,9 +449,9 @@ func healthFromStatus(status map[string]any) string {
return "red"
}
// observeCR maps one Service CR (+ its operator-reconciled status + the running
// tag observed from the live Deployment) into an AppView, attaching the drift
// verdict. This is inventory.ts observeService fused with apps-api.ts toAppView:
// observeCR maps one App CR (+ its operator-reconciled status + the running tag
// observed from the live Deployment) into an AppView, attaching the drift verdict.
// This is inventory.ts observeService fused with apps-api.ts toAppView:
// declared tag from the CR spec, running tag from the Deployment (passed in),
// health + phase + endpoints from the operator-reconciled CR status.
func observeCR(obj *unstructured.Unstructured, namespace, env, runningTag string) AppView {
+2 -2
View File
@@ -91,7 +91,7 @@ func TestIntegrationIdempotentDeploy(t *testing.T) {
s := itClient(t)
ctx := context.Background()
before, err := s.State.dyn.Resource(servicesGVR).Namespace("hanzo").Get(ctx, itService, metav1.GetOptions{})
before, err := s.State.dyn.Resource(appsGVR).Namespace("hanzo").Get(ctx, itService, metav1.GetOptions{})
if err != nil {
t.Fatalf("get %s before: %v", itService, err)
}
@@ -105,7 +105,7 @@ func TestIntegrationIdempotentDeploy(t *testing.T) {
// Same-image merge-patch (the identical body the deploy handler builds).
patch := []byte(`{"spec":{"image":{"tag":"` + tag + `","repository":"` + repo + `","pullPolicy":"Always"}}}`)
after, err := s.State.dyn.Resource(servicesGVR).Namespace("hanzo").
after, err := s.State.dyn.Resource(appsGVR).Namespace("hanzo").
Patch(ctx, itService, k8stypes.MergePatchType, patch, metav1.PatchOptions{})
if err != nil {
t.Fatalf("idempotent patch: %v", err)
-9
View File
@@ -8,15 +8,6 @@ import (
"k8s.io/apimachinery/pkg/runtime/schema"
)
// TestServicesGVR pins the operator Service CR identity. A typo here silently
// breaks every read/deploy, so it is asserted (matches ml_test's GVR guard).
func TestServicesGVR(t *testing.T) {
want := schema.GroupVersionResource{Group: "hanzo.ai", Version: "v1", Resource: "services"}
if servicesGVR != want {
t.Fatalf("servicesGVR = %v, want %v", servicesGVR, want)
}
}
// TestIsSemverTag is the exact semver policy from apps-drift.ts SEMVER_TAG:
// strictly vMAJOR.MINOR.PATCH; everything else is floating.
func TestIsSemverTag(t *testing.T) {
+27 -78
View File
@@ -1,33 +1,21 @@
// release.go — the native first-party release seam: roll a proven, clean-semver
// image onto its operator hanzo.ai/v1 Service CR by patching spec.image, so the
// operator reconciles the Deployment rollout.
// release.go — the first-party release seam. build.go's RegisterServiceReleaser is
// the inversion that lets a build-completion path (clients/platform/release.go, or
// any package-cloud caller) request a rollout with no cloud⇄paas import cycle.
//
// This is the in-cluster, direct-CR replacement for universe's image-update.yml
// GitOps hop (a service CI fires repository_dispatch → a PR bumps the CR in git →
// ArgoCD syncs → the operator reconciles). Same determinism — clean-semver only,
// resolve the CR by metadata.name — but with NO round-trip through GitHub/ArgoCD:
// cloud patches the live CR directly and the operator does the rest. It is the
// keystone that makes push→build→image→live automatic.
//
// clients/paas already OWNS the first-party Service CR control plane (the drift
// board + the /v1/paas .../deploy patch), so the release primitive lives here;
// build.go's RegisterServiceReleaser is the inversion seam that lets a
// build-completion path (clients/platform/release.go, or any package-cloud caller)
// trigger a rollout with no cloud⇄paas import cycle.
// The App CRs in the platform namespaces are declared in universe git
// (infra/k8s/operator/crs/) and reconciled by Hanzo CD with selfHeal, so a direct
// spec.image patch is reverted on the next sync. releaseService therefore refuses
// and names the one way to roll a tag: commit it to the manifest. The clean-semver
// gate (splitReleaseImage) still validates the request so a caller gets an honest,
// specific error.
package paas
import (
"context"
"encoding/json"
"fmt"
"strings"
"github.com/hanzoai/cloud"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
k8stypes "k8s.io/apimachinery/pkg/types"
)
// splitReleaseImage splits a full image ref into (repository, tag) and enforces
@@ -55,20 +43,15 @@ func splitReleaseImage(image string) (repository, tag string, err error) {
return repository, tag, nil
}
// releaseService rolls image onto the first-party Service CR named `service` by
// merge-patching ONLY spec.image (repository + tag + pullPolicy=Always). It:
//
// - resolves the CR by metadata.name across the platform namespaces in env order
// (main first — a bare release targets production), the SAME resolution
// image-update.yml uses (service name ⇒ CR name, no hardcoded map);
// - enforces the clean-semver gate (splitReleaseImage); and
// - is IDEMPOTENT: a CR already declaring this exact (repository, tag) is a no-op
// (changed=false), so re-firing a release never churns the operator.
//
// The operator reconciles the Deployment rollout; cloud never reimplements a
// deployer. Returns the namespace it patched, the resolved semver tag, and whether
// the CR actually changed. A `service` that is not a DNS-1123 label, an image that
// fails the gate, or a service with no operator CR is an honest error (no patch).
// releaseService refuses. The App CRs in the platform namespaces are declared in
// universe git (infra/k8s/operator/crs/) and reconciled by Hanzo CD with selfHeal,
// so a direct spec.image patch is reverted on the next sync — a release would look
// applied and then silently roll back. It validates the request (DNS-1123 service
// name, clean-semver image via splitReleaseImage), confirms the App exists across
// the platform namespaces in env order (main first — an honest error when it does
// not), and returns the refusal naming the one way to release it: commit the tag
// to the manifest. Returns the resolved namespace and semver tag for the caller's
// log; changed is always false.
func releaseService(s *cloud.Service[state], ctx context.Context, service, image string) (ns, tag string, changed bool, err error) {
if e := ready(s); e != nil {
return "", "", false, e
@@ -77,56 +60,22 @@ func releaseService(s *cloud.Service[state], ctx context.Context, service, image
if !appNameRE.MatchString(service) {
return "", "", false, fmt.Errorf("service %q must be a DNS-1123 label (the CR metadata.name)", service)
}
repository, tag, err := splitReleaseImage(image)
if err != nil {
if _, tag, err = splitReleaseImage(image); err != nil {
return "", "", false, err
}
ns, gvr, err := resolveTarget(s, ctx, service)
if err != nil {
if ns, err = resolveTarget(s, ctx, service); err != nil {
return "", "", false, err
}
// A git-declared workload already has a declarer. Hanzo CD syncs the App CRs
// in these namespaces from universe `infra/k8s/operator/crs/` with selfHeal on,
// so a patch here is reverted on the next sync — the release would look applied
// and then silently roll back. Refuse, and name the one way to release it.
if gvr == appsGVR {
return ns, tag, false, fmt.Errorf(
"service %q is declared in git (App/%s, universe infra/k8s/operator/crs/%s.yaml) and reconciled by Hanzo CD with selfHeal — a patch here would be reverted on the next sync; release it by committing the tag to that file",
service, service, service)
}
// Idempotency: skip the patch (and the operator churn) when the CR already
// declares this exact image. A get error other than the terminal states falls
// through to the patch, which is itself the source of truth.
if cur, gErr := s.State.dyn.Resource(gvr).Namespace(ns).Get(ctx, service, metav1.GetOptions{}); gErr == nil {
curRepo, _, _ := unstructured.NestedString(cur.Object, "spec", "image", "repository")
curTag, _, _ := unstructured.NestedString(cur.Object, "spec", "image", "tag")
if curRepo == repository && curTag == tag {
return ns, tag, false, nil
}
}
patch, _ := json.Marshal(map[string]any{"spec": map[string]any{"image": map[string]any{
"repository": repository,
"tag": tag,
"pullPolicy": "Always",
}}})
if _, e := s.State.dyn.Resource(gvr).Namespace(ns).Patch(ctx, service, k8stypes.MergePatchType, patch, metav1.PatchOptions{}); e != nil {
if apierrors.IsNotFound(e) {
return ns, tag, false, fmt.Errorf("service %q has no operator CR in namespace %s", service, ns)
}
return ns, tag, false, k8sErr(s, "patch", e)
}
s.Log.Info("released via operator Service CR patch",
"service", service, "namespace", ns, "repository", repository, "tag", tag)
return ns, tag, true, nil
return ns, tag, false, fmt.Errorf(
"service %q is declared in git (App/%s, universe infra/k8s/operator/crs/%s.yaml) and reconciled by Hanzo CD with selfHeal — a patch here would be reverted on the next sync; release it by committing the tag to that file",
service, service, service)
}
// registerReleaser wires the native first-party release seam (build.go's
// registerReleaser wires the first-party release seam (build.go's
// RegisterServiceReleaser inversion) to this mounted paas service. Called once
// from routes, so a proven image produced anywhere in the binary (cloud's own
// self-release, or a future in-process first-party builder) rolls live through the
// SAME CR-patch primitive with no cloud⇄paas import cycle.
// from routes, so a release requested anywhere in the binary (cloud's own
// self-release, or a future in-process first-party builder) reaches the SAME
// refusal with no cloud⇄paas import cycle.
func registerReleaser(s *cloud.Service[state]) {
cloud.RegisterServiceReleaser(func(ctx context.Context, ev cloud.ServiceReleaseEvent) error {
_, _, _, err := releaseService(s, ctx, ev.Service, ev.Image)
+40 -61
View File
@@ -2,6 +2,7 @@ package paas
import (
"context"
"strings"
"testing"
"github.com/hanzoai/cloud"
@@ -14,16 +15,6 @@ import (
dynamicfake "k8s.io/client-go/dynamic/fake"
)
// serviceCRObj builds a real-shaped operator Service CR for the fake cluster.
func serviceCRObj(name, ns, repo, tag string) *unstructured.Unstructured {
return &unstructured.Unstructured{Object: map[string]any{
"apiVersion": "hanzo.ai/v1",
"kind": "Service",
"metadata": map[string]any{"name": name, "namespace": ns},
"spec": map[string]any{"image": map[string]any{"repository": repo, "tag": tag, "pullPolicy": "Always"}},
}}
}
// appCRObj builds a real-shaped operator App CR — the kind the fleet runs on —
// for the fake cluster.
func appCRObj(name, ns, repo, tag string) *unstructured.Unstructured {
@@ -36,22 +27,20 @@ func appCRObj(name, ns, repo, tag string) *unstructured.Unstructured {
}
// fakeService builds a hermetic paas Service backed by an in-memory fake dynamic
// client seeded with objs — the release patch is exercised without a real cluster.
// Both workload kinds are registered, so a test can seed either.
// client seeded with objs — the release path is exercised without a real cluster.
func fakeService(objs ...runtime.Object) *cloud.Service[state] {
scheme := runtime.NewScheme()
dyn := dynamicfake.NewSimpleDynamicClientWithCustomListKinds(scheme, map[schema.GroupVersionResource]string{
appsGVR: "AppList",
servicesGVR: "ServiceList",
deploymentsGVR: "DeploymentList",
}, objs...)
return &cloud.Service[state]{Base: cloud.Base{Log: luxlog.New("test")}, State: state{dyn: dyn}}
}
// declaredImage reads spec.image.{repository,tag} off the live CR in the fake.
// declaredImage reads spec.image.{repository,tag} off the live App CR in the fake.
func declaredImage(t *testing.T, s *cloud.Service[state], ns, name string) (repo, tag, pull string) {
t.Helper()
obj, err := s.State.dyn.Resource(servicesGVR).Namespace(ns).Get(context.Background(), name, metav1.GetOptions{})
obj, err := s.State.dyn.Resource(appsGVR).Namespace(ns).Get(context.Background(), name, metav1.GetOptions{})
if err != nil {
t.Fatalf("get CR %s/%s: %v", ns, name, err)
}
@@ -98,40 +87,31 @@ func TestSplitReleaseImage(t *testing.T) {
}
}
// TestReleaseServicePatchesCR proves the keystone: a proven image patches the
// matching Service CR's spec.image.tag so the operator rolls it out.
func TestReleaseServicePatchesCR(t *testing.T) {
s := fakeService(serviceCRObj("cloud", "hanzo", "ghcr.io/hanzoai/cloud", "v1.799.16"))
// TestReleaseServiceRefusesAGitDeclaredApp proves the seam refuses a real App CR
// (git-declared, selfHeal-reverted): a valid semver image yields the refusal that
// names where to commit the tag, and the CR is left untouched.
func TestReleaseServiceRefusesAGitDeclaredApp(t *testing.T) {
s := fakeService(appCRObj("cloud", "hanzo", "ghcr.io/hanzoai/cloud", "v1.799.16"))
ns, tag, changed, err := releaseService(s, context.Background(), "cloud", "ghcr.io/hanzoai/cloud:v1.800.0")
if err != nil {
t.Fatalf("releaseService: %v", err)
if err == nil || changed {
t.Fatalf("git-declared App: got (changed=%v,err=%v), want (false, refusal)", changed, err)
}
if ns != "hanzo" || tag != "v1.800.0" || !changed {
t.Fatalf("releaseService = (ns=%q,tag=%q,changed=%v), want (hanzo,v1.800.0,true)", ns, tag, changed)
if ns != "hanzo" || tag != "v1.800.0" {
t.Errorf("refusal should still report (ns=hanzo,tag=v1.800.0), got (%q,%q)", ns, tag)
}
repo, gotTag, pull := declaredImage(t, s, "hanzo", "cloud")
if repo != "ghcr.io/hanzoai/cloud" || gotTag != "v1.800.0" || pull != "Always" {
t.Errorf("CR image = (%q,%q,%q), want (ghcr.io/hanzoai/cloud,v1.800.0,Always)", repo, gotTag, pull)
if !strings.Contains(err.Error(), "declared in git") || !strings.Contains(err.Error(), "crs/cloud.yaml") {
t.Errorf("refusal must name where to commit; got %q", err)
}
// CR must be unchanged.
if _, gotTag, _ := declaredImage(t, s, "hanzo", "cloud"); gotTag != "v1.799.16" {
t.Errorf("CR tag mutated to %q on a refused release", gotTag)
}
}
// TestReleaseServiceIdempotent proves re-firing the SAME image is a no-op (no
// operator churn): changed=false, no error.
func TestReleaseServiceIdempotent(t *testing.T) {
s := fakeService(serviceCRObj("iam", "hanzo", "ghcr.io/hanzoai/iam", "v1.28.16"))
ns, tag, changed, err := releaseService(s, context.Background(), "iam", "ghcr.io/hanzoai/iam:v1.28.16")
if err != nil {
t.Fatalf("releaseService: %v", err)
}
if ns != "hanzo" || tag != "v1.28.16" || changed {
t.Fatalf("idempotent release = (ns=%q,tag=%q,changed=%v), want (hanzo,v1.28.16,false)", ns, tag, changed)
}
}
// TestReleaseServiceRejectsFloating proves a non-semver image is refused BEFORE
// any patch — the CR is left untouched.
// TestReleaseServiceRejectsFloating proves a non-semver image is refused by the
// clean-semver gate BEFORE the App is even resolved — the CR is left untouched.
func TestReleaseServiceRejectsFloating(t *testing.T) {
s := fakeService(serviceCRObj("cloud", "hanzo", "ghcr.io/hanzoai/cloud", "v1.799.16"))
s := fakeService(appCRObj("cloud", "hanzo", "ghcr.io/hanzoai/cloud", "v1.799.16"))
if _, _, changed, err := releaseService(s, context.Background(), "cloud", "ghcr.io/hanzoai/cloud:latest"); err == nil || changed {
t.Fatalf("floating tag: got (changed=%v,err=%v), want (false, error)", changed, err)
}
@@ -144,30 +124,27 @@ func TestReleaseServiceRejectsFloating(t *testing.T) {
// TestReleaseServiceUnknown proves a service with no operator CR is an honest
// error, never a silent success.
func TestReleaseServiceUnknown(t *testing.T) {
s := fakeService(serviceCRObj("cloud", "hanzo", "ghcr.io/hanzoai/cloud", "v1.799.16"))
s := fakeService(appCRObj("cloud", "hanzo", "ghcr.io/hanzoai/cloud", "v1.799.16"))
if _, _, changed, err := releaseService(s, context.Background(), "ghost", "ghcr.io/hanzoai/ghost:v1.0.0"); err == nil || changed {
t.Fatalf("unknown service: got (changed=%v,err=%v), want (false, error)", changed, err)
}
}
// TestReleaseServiceMainFirst proves a bare release resolves the production
// namespace (hanzo) before test/dev, even when the CR exists in several.
// namespace (hanzo) before test/dev, even when the App exists in several — the
// refusal reports that namespace.
func TestReleaseServiceMainFirst(t *testing.T) {
s := fakeService(
serviceCRObj("cloud", "hanzo", "ghcr.io/hanzoai/cloud", "v1.799.16"),
serviceCRObj("cloud", "hanzo-testnet", "ghcr.io/hanzoai/cloud", "v1.799.16"),
appCRObj("cloud", "hanzo", "ghcr.io/hanzoai/cloud", "v1.799.16"),
appCRObj("cloud", "hanzo-testnet", "ghcr.io/hanzoai/cloud", "v1.799.16"),
)
ns, _, _, err := releaseService(s, context.Background(), "cloud", "ghcr.io/hanzoai/cloud:v1.800.0")
if err != nil {
t.Fatalf("releaseService: %v", err)
ns, _, changed, err := releaseService(s, context.Background(), "cloud", "ghcr.io/hanzoai/cloud:v1.800.0")
if err == nil || changed {
t.Fatalf("git-declared App: got (changed=%v,err=%v), want (false, refusal)", changed, err)
}
if ns != "hanzo" {
t.Fatalf("resolved ns = %q, want hanzo (main first)", ns)
}
// testnet CR must be untouched (bare release targets production only).
if _, tag, _ := declaredImage(t, s, "hanzo-testnet", "cloud"); tag != "v1.799.16" {
t.Errorf("testnet CR mutated to %q by a production release", tag)
}
}
// TestReleaseServiceFailClosed proves that with no cluster client the release
@@ -180,22 +157,24 @@ func TestReleaseServiceFailClosed(t *testing.T) {
}
// TestRegisterReleaserRoundTrip proves the build.go inversion seam: after
// registerReleaser, cloud.OnServiceRelease dispatches to the paas primitive and
// patches the CR — the path clients/platform/release.go drives on a self-release.
// registerReleaser, cloud.OnServiceRelease dispatches to the paas primitive — the
// path clients/platform/release.go drives on a self-release. The App is
// git-declared, so the dispatch surfaces the refusal and the CR is untouched.
func TestRegisterReleaserRoundTrip(t *testing.T) {
s := fakeService(serviceCRObj("cloud", "hanzo", "ghcr.io/hanzoai/cloud", "v1.799.16"))
s := fakeService(appCRObj("cloud", "hanzo", "ghcr.io/hanzoai/cloud", "v1.799.16"))
registerReleaser(s)
t.Cleanup(func() { cloud.RegisterServiceReleaser(nil) })
if !cloud.ServiceReleaserRegistered() {
t.Fatal("ServiceReleaserRegistered() = false after registerReleaser")
}
if err := cloud.OnServiceRelease(context.Background(), cloud.ServiceReleaseEvent{
err := cloud.OnServiceRelease(context.Background(), cloud.ServiceReleaseEvent{
Service: "cloud", Image: "ghcr.io/hanzoai/cloud:v1.801.0", SHA: "deadbeef",
}); err != nil {
t.Fatalf("OnServiceRelease: %v", err)
})
if err == nil {
t.Fatal("OnServiceRelease dispatched to a git-declared App; want the refusal error")
}
if _, tag, _ := declaredImage(t, s, "hanzo", "cloud"); tag != "v1.801.0" {
t.Errorf("CR tag = %q after OnServiceRelease, want v1.801.0", tag)
if _, tag, _ := declaredImage(t, s, "hanzo", "cloud"); tag != "v1.799.16" {
t.Errorf("CR tag = %q after OnServiceRelease, want v1.799.16 (untouched)", tag)
}
}
+166
View File
@@ -0,0 +1,166 @@
// Package payout is the ONE attributed-credit money seam shared by the credit
// programs — referrals, affiliates, authors: read an org's metered spend (the
// qualify / accrual base) and grant a promo credit to its wallet (a payout made
// in credits, landing in commerce's Credit/trial bucket). It was three
// byte-identical commerce.go copies (their own doc-comments said so); extracted
// here so the S2S commerce binding — the HTTP deposit + usage-rollup path — lives
// exactly ONCE.
//
// Every payout lands via the SAME COMMERCE_SERVICE_TOKEN S2S path, the same
// X-Org-Id=<org> namespace + bare-org `user` subject that admin.grantCredit uses,
// so it is indistinguishable from an admin grant except by its ledger tag
// (grant:referral / grant:affiliate / grant:author, all → the commerce Credit/trial
// bucket per DepositKind's grant:* rule). The tag is supplied BY THE CALLER, so
// payout carries zero program domain logic — it is the money seam, nothing else.
//
// Commerce is an INTERFACE so each program's store/handler logic stays testable
// with a fake ledger; Client is the ONE production binding. A program keeps its
// own narrow (unexported-method) seam and a thin adapter delegating to Client —
// Go package-scoped interface methods can't cross packages, and the adapter is
// where a program still names its own grant tag.
package payout
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
"github.com/hanzoai/cloud/clients/commerceinproc"
)
// Commerce is the narrow money seam an attributed-credit program needs: read a
// referred/deploying org's metered spend and grant a credit to a wallet. The HTTP
// impl (Client) below is the ONE production binding.
type Commerce interface {
Configured() bool
// Deposit grants amountCents to org's wallet (Credit/trial bucket via the
// caller-supplied grant:* tag) and returns the ledger transaction id.
Deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (txnID string, err error)
// SpendCents is the org's month-to-date metered consumption — the qualify
// signal / commission accrual base (spend × the program's rate).
SpendCents(ctx context.Context, org, user string) (int64, error)
}
// ErrUnconfigured is returned by a Deposit against an unwired commerce so the
// caller records an honest failure rather than reporting a phantom grant.
var ErrUnconfigured = errors.New("payout: commerce endpoint not configured")
// Client is the production commerce binding (COMMERCE_SERVICE_TOKEN S2S).
type Client struct {
base string
token string
http *http.Client
}
// NewClient builds the production binding. base is the commerce HTTP URL (via
// commerceinproc.BaseURL at the call site); token is COMMERCE_SERVICE_TOKEN.
func NewClient(base, token string) *Client {
return &Client{
base: strings.TrimRight(strings.TrimSpace(base), "/"),
token: strings.TrimSpace(token),
http: commerceinproc.Client(15 * time.Second),
}
}
func (c *Client) Configured() bool { return c != nil && c.base != "" && c.token != "" }
// Deposit posts POST /v1/billing/deposit — the ONE money-in primitive (identical
// to admin.grantCredit's deposit). Commerce's EdgeAuth pins the body `user` to the
// X-Org-Id subject, so a payout can never be mis-targeted to another wallet.
func (c *Client) Deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (string, error) {
if !c.Configured() {
return "", ErrUnconfigured
}
if currency == "" {
currency = "usd"
}
body, err := json.Marshal(map[string]any{
"user": user,
"currency": currency,
"amount": amountCents,
"notes": notes,
"tags": tags,
})
if err != nil {
return "", err
}
raw, err := c.do(ctx, http.MethodPost, "/v1/billing/deposit", nil, org, body)
if err != nil {
return "", err
}
var out struct {
TransactionID string `json:"transactionId"`
}
if err := json.Unmarshal(raw, &out); err != nil {
return "", fmt.Errorf("commerce deposit decode: %w", err)
}
return out.TransactionID, nil
}
// SpendCents reads GET /v1/billing/usage-rollup and returns consumedCents. Zero
// (not an error) when commerce is unconfigured so a partial deploy degrades to
// "no spend to accrue yet" rather than a 5xx.
func (c *Client) SpendCents(ctx context.Context, org, user string) (int64, error) {
if !c.Configured() {
return 0, nil
}
q := url.Values{"user": {user}}
raw, err := c.do(ctx, http.MethodGet, "/v1/billing/usage-rollup", q, org, nil)
if err != nil {
return 0, err
}
var out struct {
ConsumedCents int64 `json:"consumedCents"`
}
if err := json.Unmarshal(raw, &out); err != nil {
return 0, fmt.Errorf("commerce rollup decode: %w", err)
}
return out.ConsumedCents, nil
}
// do performs one admin-S2S commerce request. X-Org-Id=<org> is the per-org
// namespace selector commerce's EdgeAuth trusts only behind the service token.
func (c *Client) do(ctx context.Context, method, path string, q url.Values, org string, body []byte) ([]byte, error) {
u := c.base + path
if enc := q.Encode(); enc != "" {
u += "?" + enc
}
var r io.Reader
if body != nil {
r = bytes.NewReader(body)
}
req, err := http.NewRequestWithContext(ctx, method, u, r)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json")
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
if c.token != "" {
req.Header.Set("Authorization", "Bearer "+c.token)
}
if org != "" {
req.Header.Set("X-Org-Id", org)
}
resp, err := c.http.Do(req)
if err != nil {
return nil, fmt.Errorf("commerce unreachable: %w", err)
}
defer func() { _ = resp.Body.Close() }()
out, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return nil, err
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("commerce status %d", resp.StatusCode)
}
return out, nil
}
+108
View File
@@ -0,0 +1,108 @@
package payout
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"testing"
)
// TestDepositPostsGrant proves Deposit posts POST /v1/billing/deposit with the
// service token + X-Org-Id namespace and the body the commerce ledger expects,
// and returns the transactionId. This is the ONE money-in path the three credit
// programs share.
func TestDepositPostsGrant(t *testing.T) {
var gotOrg, gotAuth, gotPath, gotMethod string
var gotBody map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotMethod, gotPath = r.Method, r.URL.Path
gotOrg = r.Header.Get("X-Org-Id")
gotAuth = r.Header.Get("Authorization")
b, _ := io.ReadAll(r.Body)
_ = json.Unmarshal(b, &gotBody)
_ = json.NewEncoder(w).Encode(map[string]string{"transactionId": "txn_123"})
}))
defer srv.Close()
c := NewClient(srv.URL, "svc-tok")
if !c.Configured() {
t.Fatal("client with base+token must be Configured")
}
txn, err := c.Deposit(context.Background(), "acme", "acme", 500, "", "welcome", "grant:referral")
if err != nil {
t.Fatalf("Deposit: %v", err)
}
if txn != "txn_123" {
t.Fatalf("txn = %q, want txn_123", txn)
}
if gotMethod != http.MethodPost || gotPath != "/v1/billing/deposit" {
t.Fatalf("request = %s %s, want POST /v1/billing/deposit", gotMethod, gotPath)
}
if gotOrg != "acme" {
t.Fatalf("X-Org-Id = %q, want acme", gotOrg)
}
if gotAuth != "Bearer svc-tok" {
t.Fatalf("Authorization = %q, want Bearer svc-tok", gotAuth)
}
// currency defaults to usd; amount is the cents int; tag carries the program's grant class.
if gotBody["currency"] != "usd" || gotBody["amount"].(float64) != 500 || gotBody["tags"] != "grant:referral" {
t.Fatalf("body = %v, want currency=usd amount=500 tags=grant:referral", gotBody)
}
}
// TestSpendCentsReadsRollup proves SpendCents reads GET /v1/billing/usage-rollup
// (the accrual base) and returns consumedCents.
func TestSpendCentsReadsRollup(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet || r.URL.Path != "/v1/billing/usage-rollup" {
t.Errorf("request = %s %s, want GET /v1/billing/usage-rollup", r.Method, r.URL.Path)
}
if r.URL.Query().Get("user") != "acme" {
t.Errorf("user query = %q, want acme", r.URL.Query().Get("user"))
}
_ = json.NewEncoder(w).Encode(map[string]int64{"consumedCents": 4200})
}))
defer srv.Close()
c := NewClient(srv.URL, "svc-tok")
got, err := c.SpendCents(context.Background(), "acme", "acme")
if err != nil {
t.Fatalf("SpendCents: %v", err)
}
if got != 4200 {
t.Fatalf("consumedCents = %d, want 4200", got)
}
}
// TestUnconfigured proves the fail-soft contract: an unwired client returns
// ErrUnconfigured from Deposit (an honest failure, never a phantom grant) and 0
// from SpendCents (degrades to "no spend yet", not a 5xx).
func TestUnconfigured(t *testing.T) {
c := NewClient("", "") // no base, no token
if c.Configured() {
t.Fatal("client with empty base/token must NOT be Configured")
}
if _, err := c.Deposit(context.Background(), "acme", "acme", 100, "usd", "", "grant:author"); err != ErrUnconfigured {
t.Fatalf("Deposit err = %v, want ErrUnconfigured", err)
}
got, err := c.SpendCents(context.Background(), "acme", "acme")
if err != nil || got != 0 {
t.Fatalf("SpendCents = (%d, %v), want (0, nil)", got, err)
}
}
// TestNon2xxIsError proves a commerce 5xx surfaces as an error (the caller then
// records a failed payout / retries), never a silent success.
func TestNon2xxIsError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusBadGateway)
}))
defer srv.Close()
c := NewClient(srv.URL, "svc-tok")
if _, err := c.Deposit(context.Background(), "acme", "acme", 100, "usd", "", "grant:affiliate"); err == nil {
t.Fatal("Deposit against a 502 must return an error")
}
}
-1
View File
@@ -57,7 +57,6 @@ func fakeK8s(objs ...runtime.Object) *k8sClient {
scheme := runtime.NewScheme()
dyn := dynamicfake.NewSimpleDynamicClientWithCustomListKinds(scheme, map[schema.GroupVersionResource]string{
appsGVR: "AppList",
servicesGVR: "ServiceList",
jobsGVR: "JobList",
namespacesGVR: "NamespaceList",
resourceQuotasGVR: "ResourceQuotaList",
+26 -64
View File
@@ -4,7 +4,7 @@
// CR into the caller's OWN tenant namespace; the Hanzo operator reconciles it
// into a Deployment + Service + Ingress (+ HPA/PDB) on DOKS. cloud never
// reimplements a deployer — it writes one CR, exactly like clients/paas
// patches system CRs, but here every object lives in `tenant-<org>` where the
// reads system CRs, but here every object lives in `tenant-<org>` where the
// org is the gateway-minted, IAM-VALIDATED tenant (c.Org()), never a value from
// the request body or path. That derivation is the whole cross-tenant isolation
// boundary: a caller cannot name another org's namespace because the namespace
@@ -36,61 +36,32 @@ import (
"k8s.io/client-go/tools/clientcmd"
)
// appsGVR is the operator App CR — the collapsed workload kind, and the kind a
// tenant app is written as. A role-less App dispatches to the operator's service
// profile (controllers/app.rs `classify("") => Dispatch::Service`), which is the
// same reconcile the Service kind ran, so an App carries a tenant workload
// verbatim — same spec, same Deployment, same core Service.
// appsGVR is the operator App CR — the workload kind a tenant app is written as.
// A role-less App dispatches to the operator's service profile
// (controllers/app.rs `classify("") => Dispatch::Service`), so an App carries a
// tenant workload with the default profile: one Deployment + one core Service.
// A user app is one of these CRs in its tenant namespace.
var appsGVR = schema.GroupVersionResource{Group: "hanzo.ai", Version: "v1", Resource: "apps"}
// servicesGVR is the kind tenant apps were written as before the collapse. Still
// served, still reconciled, and still the sole declarer of the tenant CRs written
// before this cutover — so reads, patches and deletes resolve it as a fallback.
// It is not written to any more: nothing here mints a new Service CR.
//
// REMOVABLE once no Service CR remains in any tenant namespace: drop it from
// crGVRs(), drop the delete-both in deleteService, and this file is App-only.
// A user app is one of these CRs in its tenant namespace.
var servicesGVR = schema.GroupVersionResource{Group: "hanzo.ai", Version: "v1", Resource: "services"}
// crGVRs is the tenant workload-CR resolution order: the kind we write first, the
// kind we used to write second. Every read/patch/delete walks this list.
func crGVRs() []schema.GroupVersionResource {
return []schema.GroupVersionResource{appsGVR, servicesGVR}
}
// resolveCR reports the kind the named tenant workload CR exists under, and
// whether it exists at all. A lookup error other than not-found is returned as-is
// — a tenant write must never proceed on an unknown cluster state.
// resolveCR reports whether the named tenant App CR exists (and, for the caller's
// patch, its GVR). A lookup error other than not-found is returned as-is — a
// tenant write must never proceed on an unknown cluster state.
func (k *k8sClient) resolveCR(ctx context.Context, ns, name string) (schema.GroupVersionResource, bool, error) {
for _, gvr := range crGVRs() {
_, err := k.dyn.Resource(gvr).Namespace(ns).Get(ctx, name, metav1.GetOptions{})
if err == nil {
return gvr, true, nil
}
if !apierrors.IsNotFound(err) {
return schema.GroupVersionResource{}, false, err
}
_, err := k.dyn.Resource(appsGVR).Namespace(ns).Get(ctx, name, metav1.GetOptions{})
if err == nil {
return appsGVR, true, nil
}
if !apierrors.IsNotFound(err) {
return schema.GroupVersionResource{}, false, err
}
return schema.GroupVersionResource{}, false, nil
}
// getCR fetches the named tenant workload CR under whichever kind holds it.
// Callers are read-only observers, so a not-found under both kinds is returned as
// the last error — an honest "unknown", never an invented object.
// getCR fetches the named tenant App CR. Callers are read-only observers, so a
// not-found is returned as the error — an honest "unknown", never an invented
// object.
func (k *k8sClient) getCR(ctx context.Context, ns, name string) (*unstructured.Unstructured, error) {
var err error
for _, gvr := range crGVRs() {
var obj *unstructured.Unstructured
obj, err = k.dyn.Resource(gvr).Namespace(ns).Get(ctx, name, metav1.GetOptions{})
if err == nil {
return obj, nil
}
if !apierrors.IsNotFound(err) {
return nil, err
}
}
return nil, err
return k.dyn.Resource(appsGVR).Namespace(ns).Get(ctx, name, metav1.GetOptions{})
}
// jobsGVR is the batch/v1 Job used to launch an in-cluster BuildKit build.
@@ -567,10 +538,8 @@ func (k *k8sClient) applyService(ctx context.Context, org, project string, a App
_, cErr := k.dyn.Resource(appsGVR).Namespace(ns).Create(ctx, desired, metav1.CreateOptions{})
return cErr
}
// Redeploy: merge-patch only .spec (+ labels), leaving the operator status
// and resourceVersion intact. Patch the kind the CR actually is — an app
// deployed before the collapse is still a Service CR, and re-declaring it as
// an App twin would leave two CRs claiming one Deployment.
// Redeploy: merge-patch only .spec (+ labels), leaving the operator status and
// resourceVersion intact.
patch, _ := json.Marshal(map[string]any{
"metadata": map[string]any{"labels": desired.Object["metadata"].(map[string]any)["labels"]},
"spec": desired.Object["spec"],
@@ -606,23 +575,16 @@ func (k *k8sClient) scaleService(ctx context.Context, org, name string, replicas
return err
}
// deleteService removes an app's workload CR under EVERY kind (best-effort
// teardown on app/project delete). A NotFound is success (already gone).
//
// Both kinds are deleted, not just the one that resolves first. Either kind alone
// re-materializes the Deployment: they are the same reconcile behind two names,
// so a leftover Service CR would rebuild an app the tenant just deleted — the app
// would come back minutes later, still billing.
// deleteService removes an app's App CR (best-effort teardown on app/project
// delete). A NotFound is success (already gone).
func (k *k8sClient) deleteService(ctx context.Context, org, name string) error {
if err := k.ready(); err != nil {
return err
}
ns := tenantNamespace(org)
for _, gvr := range crGVRs() {
err := k.dyn.Resource(gvr).Namespace(ns).Delete(ctx, name, metav1.DeleteOptions{})
if err != nil && !apierrors.IsNotFound(err) {
return err
}
err := k.dyn.Resource(appsGVR).Namespace(ns).Delete(ctx, name, metav1.DeleteOptions{})
if err != nil && !apierrors.IsNotFound(err) {
return err
}
return nil
}
+26 -70
View File
@@ -18,21 +18,6 @@ func TestAppsGVR(t *testing.T) {
}
}
// TestCRGVRsOrder pins the resolution order: the kind we write first, the kind we
// used to write second.
func TestCRGVRsOrder(t *testing.T) {
got := crGVRs()
want := []schema.GroupVersionResource{appsGVR, servicesGVR}
if len(got) != len(want) {
t.Fatalf("crGVRs() = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("crGVRs()[%d] = %v, want %v", i, got[i], want[i])
}
}
}
// TestTenantCRIsWrittenAsApp — a tenant app is declared as an App CR, and no
// Service CR is minted for it.
func TestTenantCRIsWrittenAsApp(t *testing.T) {
@@ -51,64 +36,48 @@ func TestTenantCRIsWrittenAsApp(t *testing.T) {
}
}
// TestRedeployOfALegacyServiceCRDoesNotMintATwin — an app deployed before the
// collapse is still a Service CR. Re-declaring it as an App would leave two CRs
// claiming one Deployment, which the operator resolves only by flipping its
// ownerRef. Patch the kind it actually is.
func TestRedeployOfALegacyServiceCRDoesNotMintATwin(t *testing.T) {
legacy := &unstructured.Unstructured{Object: map[string]any{
// TestRedeployPatchesTheAppInPlace — a redeploy over an existing App CR
// merge-patches it (new image) rather than minting a twin.
func TestRedeployPatchesTheAppInPlace(t *testing.T) {
app := &unstructured.Unstructured{Object: map[string]any{
"apiVersion": "hanzo.ai/v1",
"kind": "Service",
"kind": "App",
"metadata": map[string]any{"name": "api", "namespace": "tenant-maxpower"},
"spec": map[string]any{"image": map[string]any{"repository": "ghcr.io/hanzoai/nginx", "tag": "1.26"}},
}}
k := fakeK8s(legacy)
k := fakeK8s(app)
ctx := context.Background()
if err := k.applyService(ctx, "maxpower", "proj", Application{Slug: "api", Replicas: 1}, "ghcr.io/hanzoai/nginx:1.27"); err != nil {
t.Fatalf("applyService over a legacy Service CR: %v", err)
t.Fatalf("applyService redeploy: %v", err)
}
// No App twin.
if _, err := k.dyn.Resource(appsGVR).Namespace("tenant-maxpower").Get(ctx, "api", metav1.GetOptions{}); err == nil {
t.Fatal("minted an App twin next to the live Service CR — two declarers for one Deployment")
}
// The legacy CR carries the new image.
obj, err := k.dyn.Resource(servicesGVR).Namespace("tenant-maxpower").Get(ctx, "api", metav1.GetOptions{})
obj, err := k.dyn.Resource(appsGVR).Namespace("tenant-maxpower").Get(ctx, "api", metav1.GetOptions{})
if err != nil {
t.Fatalf("legacy Service CR: %v", err)
t.Fatalf("App CR: %v", err)
}
tag, _, _ := unstructured.NestedString(obj.Object, "spec", "image", "tag")
if tag != "1.27" {
t.Fatalf("legacy CR tag = %q, want 1.27 — the redeploy must patch the kind it is", tag)
t.Fatalf("App CR tag = %q, want 1.27 — the redeploy must patch it in place", tag)
}
}
// TestDeleteRemovesBothKinds — either kind alone re-materializes the Deployment,
// so a teardown that removes only one brings the app back minutes later, still
// billing.
func TestDeleteRemovesBothKinds(t *testing.T) {
legacy := &unstructured.Unstructured{Object: map[string]any{
"apiVersion": "hanzo.ai/v1",
"kind": "Service",
"metadata": map[string]any{"name": "api", "namespace": "tenant-maxpower"},
"spec": map[string]any{"image": map[string]any{"repository": "ghcr.io/hanzoai/nginx", "tag": "1.26"}},
}}
// TestDeleteRemovesTheApp — teardown removes the tenant's App CR so it stops
// reconciling (and stops billing).
func TestDeleteRemovesTheApp(t *testing.T) {
app := &unstructured.Unstructured{Object: map[string]any{
"apiVersion": "hanzo.ai/v1",
"kind": "App",
"metadata": map[string]any{"name": "api", "namespace": "tenant-maxpower"},
"spec": map[string]any{"image": map[string]any{"repository": "ghcr.io/hanzoai/nginx", "tag": "1.27"}},
}}
k := fakeK8s(legacy, app)
k := fakeK8s(app)
ctx := context.Background()
if err := k.deleteService(ctx, "maxpower", "api"); err != nil {
t.Fatalf("deleteService: %v", err)
}
for _, gvr := range crGVRs() {
if _, err := k.dyn.Resource(gvr).Namespace("tenant-maxpower").Get(ctx, "api", metav1.GetOptions{}); err == nil {
t.Errorf("%s CR survived the delete — it would rebuild the app the tenant deleted", gvr.Resource)
}
if _, err := k.dyn.Resource(appsGVR).Namespace("tenant-maxpower").Get(ctx, "api", metav1.GetOptions{}); err == nil {
t.Error("App CR survived the delete — it would rebuild the app the tenant deleted")
}
}
@@ -120,36 +89,23 @@ func TestDeleteOfAnAbsentAppIsSuccess(t *testing.T) {
}
}
// TestResolveCRFindsEitherKind — resolution reports the kind that holds the CR,
// and reports absence honestly.
func TestResolveCRFindsEitherKind(t *testing.T) {
// TestResolveCRFindsTheApp — resolution reports the App CR that holds the tenant
// workload, and reports absence honestly.
func TestResolveCRFindsTheApp(t *testing.T) {
app := &unstructured.Unstructured{Object: map[string]any{
"apiVersion": "hanzo.ai/v1", "kind": "App",
"metadata": map[string]any{"name": "new", "namespace": "tenant-maxpower"},
"spec": map[string]any{},
}}
legacy := &unstructured.Unstructured{Object: map[string]any{
"apiVersion": "hanzo.ai/v1", "kind": "Service",
"metadata": map[string]any{"name": "old", "namespace": "tenant-maxpower"},
"spec": map[string]any{},
}}
k := fakeK8s(app, legacy)
k := fakeK8s(app)
ctx := context.Background()
for _, tc := range []struct {
name string
want schema.GroupVersionResource
}{
{"new", appsGVR},
{"old", servicesGVR},
} {
gvr, found, err := k.resolveCR(ctx, "tenant-maxpower", tc.name)
if err != nil || !found {
t.Fatalf("resolveCR(%s): found=%v err=%v", tc.name, found, err)
}
if gvr != tc.want {
t.Errorf("resolveCR(%s) = %v, want %v", tc.name, gvr, tc.want)
}
gvr, found, err := k.resolveCR(ctx, "tenant-maxpower", "new")
if err != nil || !found {
t.Fatalf("resolveCR(new): found=%v err=%v", found, err)
}
if gvr != appsGVR {
t.Errorf("resolveCR(new) = %v, want appsGVR", gvr)
}
if _, found, err := k.resolveCR(ctx, "tenant-maxpower", "ghost"); found || err != nil {
t.Errorf("resolveCR(ghost): found=%v err=%v, want found=false err=nil", found, err)
+13 -10
View File
@@ -260,17 +260,20 @@ func ValidatedProject(c *zip.Ctx) (string, bool) {
return project, Validated(c) && !IsDefaultProject(project)
}
// BillingAccount resolves the caller's funding BillingAccount id — the GCP-style
// account (models/billingaccount) that pays for this request's usage. It mirrors
// Project: a zero-copy read of the gateway-minted X-Billing-Account-Id header (in
// production the gateway mints it from the validated IAM `billing_account` claim;
// off-gateway, SanitizeIdentity re-injects the caller's own value).
// BillingAccount resolves the account that PAYS for this request — the IAM
// `billing_account` claim, as the `<kind>:<subject>` string ai/object.ParseAccount
// reads back. It mirrors Project: a zero-copy read of a SERVER-MINTED header.
//
// It is an ATTRIBUTION hint ONLY. The account that is actually debited is resolved
// SERVER-SIDE by commerce from the org's ProjectBinding (resolveAccountId), never
// from this header — so a mislabelled account can only ever misattribute the
// caller's OWN spend within its own org and can NEVER redirect spend to another
// tenant's account. Empty when no account is in scope (the org-wide default pool).
// It is AUTHORITATIVE, not a hint. Both minters bind it from the validated claim
// and strip any client copy first — the gateway (iamauth.Claims.MintedBillingAccount)
// and, on the in-cluster path, cloud's own SanitizeIdentity (idClaims.mintedBillingAccount)
// — so a surviving value is IAM's signed statement of who pays, never a caller
// naming its own payer. Hand it to Payer as Credential.Account; Payer bounds it to
// the caller's own org and falls back when it is absent.
//
// Empty when IAM minted no account: a token from before the claim shipped, or an
// opaque hk-/sk- key that never carried claims. Payer's legacy rule answers for
// those, so an empty value bills the same account it always did — never nothing.
// The value is CLONED because it is retained past the request for telemetry.
func BillingAccount(c *zip.Ctx) string {
acct := strings.TrimSpace(c.Header("X-Billing-Account-Id"))
+1 -1
View File
@@ -14,7 +14,7 @@ import (
"path"
"strings"
minio "github.com/minio/minio-go/v7"
minio "github.com/hanzoai/s3-go"
"github.com/hanzoai/cloud/clients/s3admin"
"github.com/hanzoai/cloud/clients/sites"
+2 -2
View File
@@ -17,8 +17,8 @@ import (
"strings"
"time"
minio "github.com/minio/minio-go/v7"
"github.com/minio/minio-go/v7/pkg/credentials"
minio "github.com/hanzoai/s3-go"
"github.com/hanzoai/s3-go/pkg/credentials"
)
// errAlreadyExists is returned by a Provisioner when the backend reports the
+24 -133
View File
@@ -1,152 +1,43 @@
package referrals
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
"github.com/hanzoai/cloud/clients/commerceinproc"
"github.com/hanzoai/cloud/clients/payout"
)
// commerce is the narrow money seam the referral loop needs: read a referee's
// metered spend (the qualify signal) and grant a promo credit to a wallet (the
// bonus). It is an INTERFACE so the store/handler logic is testable with a fake
// ledger — the HTTP impl below is the ONE production binding.
// bonus, ledger tag grant:referral). It is an INTERFACE so the store/handler logic
// is testable with a fake ledger; the production binding is clients/payout, reached
// through the thin adapter below.
//
// This mirrors clients/admin/commerce.go's deposit + usage-rollup EXACTLY: the
// same COMMERCE_SERVICE_TOKEN S2S path, the same X-Org-Id=<org> namespace + bare
// org `user` subject that admin.grantCredit uses — so a referral bonus lands in
// precisely the wallet the balance panel reads, indistinguishable from an admin
// grant except by its ledger tag (grant:referral vs grant:admin, both → the
// commerce Credit/trial bucket per DepositKind's grant:* rule).
// The S2S impl (COMMERCE_SERVICE_TOKEN path, X-Org-Id=<org> namespace, bare-org
// `user` subject) was three byte-identical commerce.go copies; it now lives ONCE in
// clients/payout. A referral bonus still lands in precisely the wallet the balance
// panel reads, indistinguishable from an admin grant except by its grant:referral tag.
type commerce interface {
configured() bool
// deposit grants amountCents to org's wallet (Credit/trial bucket via the
// grant:referral tag) and returns the ledger transaction id.
deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (txnID string, err error)
// spendCents is the org's month-to-date metered consumption (the qualify
// signal): the referee has actually USED the product, not just signed up.
spendCents(ctx context.Context, org, user string) (int64, error)
}
// errUnconfigured is returned by a deposit against an unwired commerce so the
// caller records an honest failure rather than reporting a phantom grant.
var errUnconfigured = errors.New("referrals: commerce endpoint not configured")
// errUnconfigured is the shared sentinel a deposit against an unwired commerce
// returns, so the caller records an honest failure rather than a phantom grant.
var errUnconfigured = payout.ErrUnconfigured
// httpCommerce is the production commerce binding (COMMERCE_SERVICE_TOKEN S2S).
type httpCommerce struct {
base string
token string
http *http.Client
// commerceSeam adapts the shared payout.Client onto this program's lowercase seam
// (Go package-scoped interface methods cannot cross packages). Zero logic — pure
// delegation; the money path lives in clients/payout.
type commerceSeam struct{ c *payout.Client }
func (s commerceSeam) configured() bool { return s.c.Configured() }
func (s commerceSeam) deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (string, error) {
return s.c.Deposit(ctx, org, user, amountCents, currency, notes, tags)
}
func (s commerceSeam) spendCents(ctx context.Context, org, user string) (int64, error) {
return s.c.SpendCents(ctx, org, user)
}
func newCommerceClient(base, token string) *httpCommerce {
return &httpCommerce{
base: strings.TrimRight(strings.TrimSpace(base), "/"),
token: strings.TrimSpace(token),
http: commerceinproc.Client(15 * time.Second),
}
}
func (c *httpCommerce) configured() bool { return c != nil && c.base != "" && c.token != "" }
// deposit posts POST /v1/billing/deposit — the ONE money-in primitive (identical
// to admin.commerceClient.deposit). Commerce's EdgeAuth pins the body `user` to
// the X-Org-Id subject, so a bonus can never be mis-targeted to another wallet.
func (c *httpCommerce) deposit(ctx context.Context, org, user string, amountCents int64, currency, notes, tags string) (string, error) {
if !c.configured() {
return "", errUnconfigured
}
if currency == "" {
currency = "usd"
}
body, err := json.Marshal(map[string]any{
"user": user,
"currency": currency,
"amount": amountCents,
"notes": notes,
"tags": tags,
})
if err != nil {
return "", err
}
raw, err := c.do(ctx, http.MethodPost, "/v1/billing/deposit", nil, org, body)
if err != nil {
return "", err
}
var out struct {
TransactionID string `json:"transactionId"`
}
if err := json.Unmarshal(raw, &out); err != nil {
return "", fmt.Errorf("commerce deposit decode: %w", err)
}
return out.TransactionID, nil
}
// spendCents reads GET /v1/billing/usage-rollup and returns consumedCents. Zero
// (not an error) when commerce is unconfigured so a partial deploy degrades to
// "not qualified yet" rather than a 5xx.
func (c *httpCommerce) spendCents(ctx context.Context, org, user string) (int64, error) {
if !c.configured() {
return 0, nil
}
q := url.Values{"user": {user}}
raw, err := c.do(ctx, http.MethodGet, "/v1/billing/usage-rollup", q, org, nil)
if err != nil {
return 0, err
}
var out struct {
ConsumedCents int64 `json:"consumedCents"`
}
if err := json.Unmarshal(raw, &out); err != nil {
return 0, fmt.Errorf("commerce rollup decode: %w", err)
}
return out.ConsumedCents, nil
}
// do performs one admin-S2S commerce request. X-Org-Id=<org> is the per-org
// namespace selector commerce's EdgeAuth trusts only behind the service token.
func (c *httpCommerce) do(ctx context.Context, method, path string, q url.Values, org string, body []byte) ([]byte, error) {
u := c.base + path
if enc := q.Encode(); enc != "" {
u += "?" + enc
}
var r io.Reader
if body != nil {
r = bytes.NewReader(body)
}
req, err := http.NewRequestWithContext(ctx, method, u, r)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json")
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
if c.token != "" {
req.Header.Set("Authorization", "Bearer "+c.token)
}
if org != "" {
req.Header.Set("X-Org-Id", org)
}
resp, err := c.http.Do(req)
if err != nil {
return nil, fmt.Errorf("commerce unreachable: %w", err)
}
defer func() { _ = resp.Body.Close() }()
out, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return nil, err
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("commerce status %d", resp.StatusCode)
}
return out, nil
}
// newCommerceClient builds the production binding, delegating to clients/payout.
func newCommerceClient(base, token string) commerce { return commerceSeam{payout.NewClient(base, token)} }
+2 -2
View File
@@ -23,8 +23,8 @@ import (
"strconv"
"strings"
minio "github.com/minio/minio-go/v7"
"github.com/minio/minio-go/v7/pkg/credentials"
minio "github.com/hanzoai/s3-go"
"github.com/hanzoai/s3-go/pkg/credentials"
)
// Admin holds the shared S3 admin connection parameters, sourced once from the
+1 -1
View File
@@ -24,7 +24,7 @@ import (
"io"
"sync/atomic"
minio "github.com/minio/minio-go/v7"
minio "github.com/hanzoai/s3-go"
"github.com/hanzoai/cloud/clients/s3admin"
"github.com/hanzoai/cloud/types"
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"fmt"
"testing"
minio "github.com/minio/minio-go/v7"
minio "github.com/hanzoai/s3-go"
"github.com/hanzoai/cloud/clients/s3admin"
"github.com/hanzoai/cloud/types"
+296
View File
@@ -0,0 +1,296 @@
package samples
import (
"context"
"fmt"
"net"
"os"
"strings"
"testing"
"time"
aiobject "github.com/hanzoai/ai/object"
)
// live_test.go proves the plane against a REAL datastore: the DDL actually parses
// and creates, the binds actually match the column types, the materialized view
// actually feeds its target, and the reads actually return what was written.
// Nothing else can prove those — a mock only ever re-asserts our own assumptions.
//
// It is OPT-IN (skipped unless DATASTORE_ADDR names one) so CI, which has no
// warehouse, stays green:
//
// docker run -d --name ch -p 19000:9000 clickhouse/clickhouse-server:latest
// DATASTORE_ADDR=127.0.0.1:19000 go test ./clients/samples -run TestLive -v
//
// It WRITES, so it refuses any non-loopback address: pointing it at a real
// warehouse must never inject test rows into a tenant's series.
func liveDatastore(t *testing.T) {
t.Helper()
addr := strings.TrimSpace(os.Getenv("DATASTORE_ADDR"))
if addr == "" {
t.Skip("no DATASTORE_ADDR — set it to a LOCAL datastore to prove the DDL end to end")
}
host, _, err := net.SplitHostPort(addr)
if err != nil {
t.Fatalf("DATASTORE_ADDR %q is not host:port: %v", addr, err)
}
// Fail closed: this test inserts, so it only ever runs against loopback.
if ip := net.ParseIP(host); host != "localhost" && (ip == nil || !ip.IsLoopback()) {
t.Fatalf("refusing to run a WRITING test against non-loopback %q", addr)
}
if aiobject.DatastoreEnabled() {
return
}
aiobject.InitDatastore() // async dial + self-provisions the `hanzo` database
for i := 0; i < 100; i++ {
if aiobject.DatastoreEnabled() {
return
}
time.Sleep(100 * time.Millisecond)
}
t.Fatalf("datastore at %s never became ready", addr)
}
// TestLivePlaneRoundTrip is the whole contract against a real engine: the schema
// creates, a sample records, and both reads return it — org-scoped.
func TestLivePlaneRoundTrip(t *testing.T) {
liveDatastore(t)
ctx := context.Background()
org := fmt.Sprintf("live-%d", time.Now().UnixNano())
in := Sample{
Org: org, Source: SourceAgent, Unit: "tgt-live", Host: "box.local",
Kind: KindGPU, At: time.Now().UTC().Add(-time.Minute),
CPUs: 20, Memory: 128 << 30, MemUsed: 64 << 30, MemFree: 64 << 30,
Load1: 2.5, Load5: 2, Load15: 1.5, GPUUtil: 0.75,
GPUs: 1, GPUModel: "GB10", CostCents: 0,
}
if err := Record(ctx, in); err != nil {
t.Fatalf("Record against a live datastore: %v", err)
}
got, err := Series(ctx, Query{Org: org, Range: "1h"})
if err != nil {
t.Fatalf("Series: %v", err)
}
if len(got) != 1 {
t.Fatalf("want 1 sample back, got %d", len(got))
}
s := got[0]
if s.Org != org || s.Unit != "tgt-live" || s.Kind != KindGPU || s.Source != SourceAgent {
t.Fatalf("identity did not round-trip: %+v", s)
}
if s.CPUs != 20 || s.GPUs != 1 || s.GPUModel != "GB10" || s.Memory != 128<<30 {
t.Fatalf("spec did not round-trip: %+v", s)
}
if s.GPUUtil < 0.74 || s.GPUUtil > 0.76 || s.Load1 < 2.49 || s.Load1 > 2.51 {
t.Fatalf("metrics did not round-trip through Float32: %+v", s)
}
if s.At.IsZero() || time.Since(s.At) > 2*time.Hour {
t.Fatalf("ts did not round-trip: %v", s.At)
}
latest, err := Latest(ctx, org)
if err != nil {
t.Fatalf("Latest: %v", err)
}
if _, ok := latest["tgt-live"]; !ok {
t.Fatalf("the board overlay must carry the unit, got %v", latest)
}
}
// The tenancy boundary against a REAL engine, not just a built statement: org B
// reading with A's unit id gets nothing.
func TestLiveCrossTenantIsolation(t *testing.T) {
liveDatastore(t)
ctx := context.Background()
a := fmt.Sprintf("live-a-%d", time.Now().UnixNano())
b := fmt.Sprintf("live-b-%d", time.Now().UnixNano())
secret := Sample{Org: a, Source: SourceAgent, Unit: "tgt-secret", Kind: KindGPU,
At: time.Now().UTC(), GPUUtil: 0.99, Load1: 9}
if err := Record(ctx, secret); err != nil {
t.Fatalf("Record A: %v", err)
}
// B asks for A's exact unit id.
got, err := Series(ctx, Query{Org: b, Unit: "tgt-secret", Range: "1h"})
if err != nil {
t.Fatalf("Series B: %v", err)
}
if len(got) != 0 {
t.Fatalf("CROSS-TENANT LEAK: org %s read %d of org %s's samples: %+v", b, len(got), a, got)
}
board, err := Latest(ctx, b)
if err != nil {
t.Fatalf("Latest B: %v", err)
}
if len(board) != 0 {
t.Fatalf("CROSS-TENANT LEAK: org %s's board carried %v", b, board)
}
// A still sees its own.
own, err := Series(ctx, Query{Org: a, Unit: "tgt-secret", Range: "1h"})
if err != nil || len(own) != 1 {
t.Fatalf("org A must read its own sample, got %d / %v", len(own), err)
}
}
// A hostile sample must be storable-or-rejected, never a corrupt row: the clamps
// are what let the binds meet UInt16/UInt8/Float32 without the engine erroring.
func TestLiveHostileSampleIsClampedNotRejectedByEngine(t *testing.T) {
liveDatastore(t)
ctx := context.Background()
org := fmt.Sprintf("live-hostile-%d", time.Now().UnixNano())
err := Record(ctx, Sample{
Org: org, Source: SourceAgent, Unit: "tgt-hostile", Kind: KindLaptop,
At: time.Now().UTC(), CPUs: 1 << 20, GPUs: 99999,
Load1: 1e300, Load5: -1, GPUUtil: 42, Memory: -1,
Host: strings.Repeat("h", 500), GPUModel: strings.Repeat("m", 500),
})
if err != nil {
t.Fatalf("a clamped hostile sample must still be a legal row: %v", err)
}
got, err := Series(ctx, Query{Org: org, Range: "1h"})
if err != nil || len(got) != 1 {
t.Fatalf("want the clamped row back, got %d / %v", len(got), err)
}
s := got[0]
if s.CPUs != maxCPUs || s.GPUs != maxGPUs || s.GPUUtil != 1 || s.Load1 != maxLoad {
t.Fatalf("the engine stored an unclamped value: %+v", s)
}
if s.Load5 != 0 || s.Memory != 0 {
t.Fatalf("negatives must have floored to 0: %+v", s)
}
}
// An untenanted row must never reach a real table, even when the datastore IS
// connected — this is the fail-closed half of Record that the no-datastore unit
// test cannot reach.
func TestLiveRecordFailsClosedOnBadTenant(t *testing.T) {
liveDatastore(t)
ctx := context.Background()
for _, tc := range []struct {
name string
s Sample
want error
}{
{"blank org", Sample{Source: SourceAgent, Unit: "u", Kind: KindGPU}, errOrg},
{"blank unit", Sample{Org: "acme", Source: SourceAgent, Kind: KindGPU}, errUnit},
{"unknown source", Sample{Org: "acme", Source: "evil", Unit: "u", Kind: KindGPU}, errSource},
{"unknown kind", Sample{Org: "acme", Source: SourceAgent, Unit: "u", Kind: "evil"}, errKind},
} {
t.Run(tc.name, func(t *testing.T) {
if err := Record(ctx, tc.s); err != tc.want {
t.Fatalf("want %v, got %v", tc.want, err)
}
})
}
}
// The hourly rollup is fed by the view on insert and is itself org-scoped — the
// trend plane must not become a cross-tenant back door around the raw table.
func TestLiveRollupIsFedAndOrgScoped(t *testing.T) {
liveDatastore(t)
ctx := context.Background()
org := fmt.Sprintf("live-mv-%d", time.Now().UnixNano())
at := time.Now().UTC()
for i, util := range []float64{0.2, 0.9, 0.5} {
if err := Record(ctx, Sample{
Org: org, Source: SourceAgent, Unit: "tgt-mv", Kind: KindGPU,
At: at.Add(time.Duration(i) * time.Second), GPUUtil: util, Load1: 1, MemUsed: 1 << 30,
}); err != nil {
t.Fatalf("Record %d: %v", i, err)
}
}
rows, err := aiobject.DatastoreQuery(ctx,
`SELECT unit, maxMerge(gpu_util_max) AS mx, avgMerge(gpu_util_avg) AS av
FROM hanzo.compute_samples_hourly WHERE org = ? GROUP BY unit`, org)
if err != nil {
t.Fatalf("the rollup must be queryable: %v", err)
}
if len(rows) != 1 {
t.Fatalf("the view must have fed exactly one unit's rollup, got %d rows", len(rows))
}
if got := f64(rows[0]["mx"]); got < 0.89 || got > 0.91 {
t.Fatalf("gpu_util_max want ~0.9, got %v", got)
}
if got := f64(rows[0]["av"]); got < 0.52 || got > 0.55 {
t.Fatalf("gpu_util_avg want ~0.533, got %v", got)
}
// Another tenant's rollup read finds nothing of this org's.
other, err := aiobject.DatastoreQuery(ctx,
`SELECT count() AS n FROM hanzo.compute_samples_hourly WHERE org = ?`, org+"-other")
if err != nil {
t.Fatalf("rollup read: %v", err)
}
if n := i64(other[0]["n"]); n != 0 {
t.Fatalf("CROSS-TENANT LEAK in the rollup: %d rows", n)
}
}
// Latest must mean LATEST. `LIMIT 1 BY unit` only picks the newest row if the
// ordering is right, and a board showing a stale sample as current utilization
// would be a silent lie — so the ordering is proven against the engine, not read.
func TestLiveLatestPicksTheNewestPerUnit(t *testing.T) {
liveDatastore(t)
ctx := context.Background()
org := fmt.Sprintf("live-latest-%d", time.Now().UnixNano())
at := time.Now().UTC().Add(-10 * time.Minute)
// Deliberately NOT in time order, and the newest is not the largest value —
// so neither insert order nor max() could pass this by accident.
for _, s := range []struct {
off time.Duration
util float64
}{{2 * time.Minute, 0.5}, {4 * time.Minute, 0.1}, {time.Minute, 0.9}} {
if err := Record(ctx, Sample{
Org: org, Source: SourceAgent, Unit: "tgt-latest", Kind: KindGPU,
At: at.Add(s.off), GPUUtil: s.util,
}); err != nil {
t.Fatalf("Record: %v", err)
}
}
// A second unit proves the "per unit" half.
if err := Record(ctx, Sample{Org: org, Source: SourceAgent, Unit: "tgt-other",
Kind: KindLaptop, At: at, GPUUtil: 0.3}); err != nil {
t.Fatalf("Record other: %v", err)
}
board, err := Latest(ctx, org)
if err != nil {
t.Fatalf("Latest: %v", err)
}
if len(board) != 2 {
t.Fatalf("want one row per unit (2), got %d: %v", len(board), board)
}
got := board["tgt-latest"]
if got.GPUUtil < 0.09 || got.GPUUtil > 0.11 {
t.Fatalf("Latest must be the NEWEST sample (util 0.1 @ +4m), got %v", got.GPUUtil)
}
if !got.At.Equal(at.Add(4 * time.Minute).Truncate(time.Millisecond)) {
t.Fatalf("Latest picked the wrong instant: %v", got.At)
}
if board["tgt-other"].Kind != KindLaptop {
t.Fatalf("each unit keeps its own latest: %+v", board["tgt-other"])
}
}
// The shipped DDL must be idempotent: ensure() runs on every cold write path, so a
// second pass over an existing table/view can never error.
func TestLiveSchemaIsIdempotent(t *testing.T) {
liveDatastore(t)
ctx := context.Background()
for i := 0; i < 3; i++ {
for _, ddl := range schema {
if err := aiobject.DatastoreExec(ctx, ddl); err != nil {
t.Fatalf("pass %d: DDL is not idempotent: %v", i, err)
}
}
}
}
+259
View File
@@ -0,0 +1,259 @@
package samples
import (
"context"
"strconv"
"strings"
"time"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud/clients/principal"
)
// read.go is the series' read face: two questions, two queries.
//
// - Series — "how did this unit look over the last N?" (the chart)
// - Latest — "how does every unit look right now?" (the board's overlay)
//
// Both are built by PURE functions (buildSeries / buildLatest) so the tenancy and
// injection properties are unit-testable without a warehouse. The rules they hold:
// org leads the WHERE as a BOUND parameter and is never optional; every caller
// value is bound; `range` and `source` select from a CLOSED allowlist and are
// never string-built; and the scan is always bounded by both a time predicate and
// a LIMIT.
// Query is a resolved, already-authorized read. Org is the tenant and is REQUIRED
// — the controller fills it from the validated principal, never from a client
// field. Unit/Source narrow within that tenant; Range is an allowlisted label.
type Query struct {
Org string
Unit string
Source string
Range string
}
// ranges is the CLOSED set of windows a caller may ask for. The label is a KEY
// into this map, never text that reaches a statement: an unknown or hostile
// ?range can therefore only ever resolve to the default. 90d is the raw table's
// full TTL horizon.
var ranges = map[string]time.Duration{
"1h": time.Hour,
"6h": 6 * time.Hour,
"24h": 24 * time.Hour,
"7d": 7 * 24 * time.Hour,
"30d": 30 * 24 * time.Hour,
"90d": 90 * 24 * time.Hour,
}
// DefaultRange is the window a read gets when ?range is absent or unrecognized.
const DefaultRange = "24h"
// maxRows bounds a Series read. A SERVER constant (never caller input), so
// rendering it into the statement is injection-safe by construction. 24h of
// per-minute heartbeats is ~1440 rows per unit; this holds a month of them.
const maxRows = 50000
// maxUnits bounds a Latest read — the number of distinct compute units one org's
// board can render. Also a server constant.
const maxUnits = 5000
// latestWindow bounds the Latest scan. A sample older than this is not "current
// utilization" for a live board; the full history stays available through Series.
const latestWindow = 24 * time.Hour
// since resolves a range label to its window start. Unknown → the default.
func since(label string) time.Time {
d, ok := ranges[strings.TrimSpace(label)]
if !ok {
d = ranges[DefaultRange]
}
return time.Now().UTC().Add(-d)
}
// org validates the tenant key for a READ. Fails closed on blank/oversized —
// identical to the write's rule, so neither side can be entered without a tenant.
func org(v string) (string, error) {
v = strings.TrimSpace(v)
if v == "" || len(v) > principal.MaxOrgLen {
return "", errOrg
}
return v, nil
}
// buildSeries assembles the time-series read. Pure, so the isolation and
// injection properties are proven by unit test. org is bound FIRST and is
// unconditional; unit/source are bound narrowers WITHIN that tenant, so no
// combination of them can ever widen the scan past the caller's own org.
func buildSeries(q Query) (string, []any, error) {
o, err := org(q.Org)
if err != nil {
return "", nil, err
}
sql := `SELECT ` + cols + ` FROM ` + table + ` WHERE org = ? AND ts >= ?`
args := []any{o, since(q.Range)}
if u := strings.TrimSpace(q.Unit); u != "" {
if len(u) > maxUnit {
return "", nil, errUnit
}
sql += ` AND unit = ?`
args = append(args, u)
}
if s := strings.ToLower(strings.TrimSpace(q.Source)); s != "" {
if !validSource(s) {
return "", nil, errSource
}
sql += ` AND source = ?`
args = append(args, s)
}
sql += ` ORDER BY ts ASC LIMIT ` + strconv.Itoa(maxRows)
return sql, args, nil
}
// buildLatest assembles the board's overlay: each of the org's units with its most
// recent sample. `LIMIT 1 BY unit` over the (org, unit, ts) ordering is the
// engine's own "latest per key" — one bounded pass, no self-join.
func buildLatest(o string) (string, []any, error) {
v, err := org(o)
if err != nil {
return "", nil, err
}
sql := `SELECT ` + cols + ` FROM ` + table + ` WHERE org = ? AND ts >= ?` +
` ORDER BY unit ASC, ts DESC LIMIT 1 BY unit LIMIT ` + strconv.Itoa(maxUnits)
return sql, []any{v, time.Now().UTC().Add(-latestWindow)}, nil
}
// Series returns the org's samples over a bounded window, oldest first — the data
// behind a utilization chart. A blank org fails closed; an absent datastore is an
// honest empty (never fabricated zeros).
func Series(ctx context.Context, q Query) ([]Sample, error) {
sql, args, err := buildSeries(q)
if err != nil {
return nil, err
}
if !aiobject.DatastoreEnabled() {
return []Sample{}, nil
}
rows, err := aiobject.DatastoreQuery(ctx, sql, args...)
if err != nil {
return nil, err
}
out := make([]Sample, 0, len(rows))
for _, r := range rows {
out = append(out, sampleFrom(r))
}
return out, nil
}
// Latest returns each of the org's units keyed by unit id, with its most recent
// sample inside latestWindow. A blank org fails closed; an absent datastore is an
// honest empty map, which a board renders as "no samples yet" rather than failing.
func Latest(ctx context.Context, o string) (map[string]Sample, error) {
sql, args, err := buildLatest(o)
if err != nil {
return nil, err
}
if !aiobject.DatastoreEnabled() {
return map[string]Sample{}, nil
}
rows, err := aiobject.DatastoreQuery(ctx, sql, args...)
if err != nil {
return nil, err
}
out := make(map[string]Sample, len(rows))
for _, r := range rows {
s := sampleFrom(r)
if s.Unit == "" {
continue
}
out[s.Unit] = s
}
return out, nil
}
// sampleFrom rebuilds a Sample from one DatastoreQuery row. The driver decodes
// each column to its own native Go type (uint16/uint64/float32/time.Time/…); the
// coercers below accept those natives, so a driver or transport change degrades to
// a zero value rather than crashing a read.
func sampleFrom(r map[string]any) Sample {
return Sample{
Org: str(r["org"]),
Source: str(r["source"]),
Unit: str(r["unit"]),
Host: str(r["host"]),
Kind: str(r["kind"]),
At: ts(r["ts"]),
CPUs: int(i64(r["cpus"])),
Memory: i64(r["memory"]),
MemUsed: i64(r["mem_used"]),
MemFree: i64(r["mem_free"]),
Load1: f64(r["load1"]),
Load5: f64(r["load5"]),
Load15: f64(r["load15"]),
GPUUtil: f64(r["gpu_util"]),
GPUs: int(i64(r["gpus"])),
GPUModel: str(r["gpu_model"]),
CostCents: i64(r["cost_cents"]),
}
}
func str(v any) string {
if s, ok := v.(string); ok {
return s
}
return ""
}
func ts(v any) time.Time {
if t, ok := v.(time.Time); ok {
return t.UTC()
}
return time.Time{}
}
func i64(v any) int64 {
switch n := v.(type) {
case int:
return int64(n)
case int8:
return int64(n)
case int16:
return int64(n)
case int32:
return int64(n)
case int64:
return n
case uint:
return int64(n)
case uint8:
return int64(n)
case uint16:
return int64(n)
case uint32:
return int64(n)
case uint64:
return int64(n)
case float32:
return int64(n)
case float64:
return int64(n)
default:
return 0
}
}
func f64(v any) float64 {
switch n := v.(type) {
case float32:
return float64(n)
case float64:
return n
case int:
return float64(n)
case int64:
return float64(n)
case uint64:
return float64(n)
default:
return 0
}
}
+270
View File
@@ -0,0 +1,270 @@
package samples
import (
"errors"
"strings"
"testing"
"time"
"github.com/hanzoai/cloud/clients/principal"
)
// ── (a) cross-tenant isolation ──────────────────────────────────────────────
//
// org is the ONLY tenant key. These prove the three properties that make it one:
// it is ALWAYS present, it is always BOUND (never interpolated), and no other
// input can widen a read past it.
// Every read the package can build carries `org = ?` as its FIRST predicate, with
// the caller's own org as the first bind. There is no code path to a statement
// without it.
func TestEveryReadIsOrgScopedFirstAndBound(t *testing.T) {
series, args, err := buildSeries(Query{Org: "acme", Unit: "u1", Source: SourceAgent, Range: "7d"})
if err != nil {
t.Fatalf("buildSeries: %v", err)
}
latest, largs, err := buildLatest("acme")
if err != nil {
t.Fatalf("buildLatest: %v", err)
}
for name, q := range map[string]struct {
sql string
args []any
}{"series": {series, args}, "latest": {latest, largs}} {
if !strings.Contains(q.sql, "WHERE org = ?") {
t.Fatalf("%s: org must lead the WHERE as a bound param: %s", name, q.sql)
}
if len(q.args) == 0 || q.args[0] != "acme" {
t.Fatalf("%s: first bind must be the caller's org, got %v", name, q.args)
}
if strings.Contains(q.sql, "'acme'") {
t.Fatalf("%s: org must never be interpolated: %s", name, q.sql)
}
}
}
// The heart of it: org B's read is built for org B, no matter what org A's
// identifiers it knows. Asking for another tenant's unit does not widen the scan —
// it narrows it to (B AND A's unit), which is by construction empty.
func TestCrossTenantReadCannotReachAnotherOrg(t *testing.T) {
// org A's data.
_, aArgs, err := buildSeries(Query{Org: "org-a", Unit: "tgt-secret"})
if err != nil {
t.Fatalf("buildSeries A: %v", err)
}
if aArgs[0] != "org-a" {
t.Fatalf("A's read must bind org-a, got %v", aArgs[0])
}
// org B naming A's unit: still bound to org-b, so A's rows are unreachable.
sql, bArgs, err := buildSeries(Query{Org: "org-b", Unit: "tgt-secret"})
if err != nil {
t.Fatalf("buildSeries B: %v", err)
}
if bArgs[0] != "org-b" {
t.Fatalf("B's read must bind org-b, got %v", bArgs[0])
}
for _, a := range bArgs {
if s, ok := a.(string); ok && s == "org-a" {
t.Fatal("org-a must never appear in a read built for org-b")
}
}
if strings.Contains(sql, "org-a") {
t.Fatal("org-a must never appear in B's statement")
}
// The unit narrows WITHIN the tenant — it never replaces the tenant predicate.
if strings.Index(sql, "org = ?") > strings.Index(sql, "unit = ?") {
t.Fatalf("the org predicate must precede the unit narrower: %s", sql)
}
// Latest is per-tenant too — B's board can only ever be built from B's rows.
_, lArgs, err := buildLatest("org-b")
if err != nil {
t.Fatalf("buildLatest B: %v", err)
}
if lArgs[0] != "org-b" {
t.Fatalf("B's board must bind org-b, got %v", lArgs[0])
}
}
// A missing tenant fails CLOSED — a read is never built "for everyone".
func TestBlankOrgFailsClosed(t *testing.T) {
for _, o := range []string{"", " ", "\t", strings.Repeat("o", principal.MaxOrgLen+1)} {
if _, _, err := buildSeries(Query{Org: o}); err != errOrg {
t.Fatalf("buildSeries(%q) must fail closed, got %v", o, err)
}
if _, _, err := buildLatest(o); err != errOrg {
t.Fatalf("buildLatest(%q) must fail closed, got %v", o, err)
}
}
}
// ── injection ───────────────────────────────────────────────────────────────
// Nothing a caller supplies is ever rendered into a statement. A classic payload
// in every user-facing field must appear ONLY as a bind — the statement itself
// stays byte-identical to the benign one.
func TestCallerValuesAreBoundNeverBuilt(t *testing.T) {
const evil = "x' OR 1=1 --"
hostile, hargs, err := buildSeries(Query{Org: evil, Unit: evil, Range: evil})
if err != nil {
t.Fatalf("buildSeries: %v", err)
}
benign, _, err := buildSeries(Query{Org: "acme", Unit: "u1", Range: "24h"})
if err != nil {
t.Fatalf("buildSeries: %v", err)
}
if hostile != benign {
t.Fatalf("a hostile value changed the STATEMENT:\n hostile=%s\n benign =%s", hostile, benign)
}
if strings.Contains(hostile, "OR 1=1") || strings.Contains(hostile, "--") {
t.Fatalf("payload reached the statement: %s", hostile)
}
var found bool
for _, a := range hargs {
if a == evil {
found = true
}
}
if !found {
t.Fatal("the payload must survive as a BOUND value (proving it was bound, not built)")
}
}
// `source` is a CLOSED allowlist: an unknown value is rejected outright rather
// than reaching the statement in any form.
func TestSourceIsAllowlisted(t *testing.T) {
for _, s := range []string{SourceAgent, SourceBYO, SourceCloud, SourceVisor, "AGENT", " byo "} {
if _, _, err := buildSeries(Query{Org: "acme", Source: s}); err != nil {
t.Fatalf("source %q must be accepted, got %v", s, err)
}
}
for _, s := range []string{"wat", "agent'--", "agent OR 1=1", "'"} {
if _, _, err := buildSeries(Query{Org: "acme", Source: s}); err != errSource {
t.Fatalf("source %q must fail closed, got %v", s, err)
}
}
}
// `range` is a KEY into a closed map, never text. An unknown label can only ever
// resolve to the default window — it can never reach the statement.
func TestRangeIsAllowlistedAndDefaults(t *testing.T) {
now := time.Now().UTC()
for label, d := range ranges {
got := now.Sub(since(label))
if got < d-time.Minute || got > d+time.Minute {
t.Fatalf("range %q want ~%v, got %v", label, d, got)
}
}
for _, bad := range []string{"", "wat", "1h; DROP TABLE hanzo.compute_samples", "999d"} {
got := now.Sub(since(bad))
want := ranges[DefaultRange]
if got < want-time.Minute || got > want+time.Minute {
t.Fatalf("unknown range %q must fall back to %s, got %v", bad, DefaultRange, got)
}
}
// The window is a time.Time BIND, never rendered text.
_, args, err := buildSeries(Query{Org: "acme", Range: "1h; DROP TABLE hanzo.compute_samples"})
if err != nil {
t.Fatalf("buildSeries: %v", err)
}
if _, ok := args[1].(time.Time); !ok {
t.Fatalf("the range bind must be a time.Time, got %T", args[1])
}
}
// Every read is bounded twice — by a time predicate and by a LIMIT — so no query
// can walk the whole table.
func TestReadsAreBounded(t *testing.T) {
sql, args, err := buildSeries(Query{Org: "acme"})
if err != nil {
t.Fatalf("buildSeries: %v", err)
}
if !strings.Contains(sql, "ts >= ?") || !strings.Contains(sql, "LIMIT ") {
t.Fatalf("a series read must be time- and LIMIT-bounded: %s", sql)
}
if len(args) != 2 {
t.Fatalf("want (org, since) binds, got %v", args)
}
lsql, _, err := buildLatest("acme")
if err != nil {
t.Fatalf("buildLatest: %v", err)
}
if !strings.Contains(lsql, "LIMIT 1 BY unit") || !strings.Contains(lsql, "ts >= ?") {
t.Fatalf("a board read must be per-unit and time-bounded: %s", lsql)
}
}
// An over-long unit is rejected on the read path too — the same rule as the write,
// so a key can never be silently truncated onto another unit's series.
func TestOversizedUnitFailsClosedOnRead(t *testing.T) {
if _, _, err := buildSeries(Query{Org: "acme", Unit: strings.Repeat("u", maxUnit+1)}); err != errUnit {
t.Fatalf("an over-long unit must fail closed, got %v", err)
}
}
// ── caller error vs infrastructure error ────────────────────────────────────
// Every rejection this package can produce is a CALLER error, so an HTTP face can
// answer 400 with a safe message. Conflating the two is how a warehouse outage
// gets blamed on a tenant — or worse, how its text leaks to one.
func TestEveryRejectionIsMarkedInvalid(t *testing.T) {
for _, err := range []error{errOrg, errUnit, errSource, errKind} {
if !errors.Is(err, ErrInvalid) {
t.Fatalf("%v must be an ErrInvalid so a face can answer 400", err)
}
}
// The reads' rejections carry the marker through.
if _, _, err := buildSeries(Query{Org: ""}); !errors.Is(err, ErrInvalid) {
t.Fatalf("a blank org must be marked invalid, got %v", err)
}
if _, _, err := buildSeries(Query{Org: "acme", Source: "evil"}); !errors.Is(err, ErrInvalid) {
t.Fatalf("an unknown source must be marked invalid, got %v", err)
}
// A caller-safe message names the closed vocabulary and nothing internal.
if !strings.Contains(errSource.Error(), SourceAgent) {
t.Fatalf("the source error must tell the caller the vocabulary: %v", errSource)
}
for _, err := range []error{errOrg, errUnit, errSource, errKind} {
if strings.Contains(err.Error(), table) {
t.Fatalf("a caller-facing error must not name our tables: %v", err)
}
}
}
// ── row decode ──────────────────────────────────────────────────────────────
// The driver hands each column back in its own native type; the decode must accept
// those natives and round-trip the value plane.
func TestSampleFromNativeDriverTypes(t *testing.T) {
at := time.Unix(1700000000, 0).UTC()
s := sampleFrom(map[string]any{
"org": "acme", "source": SourceAgent, "unit": "tgt-1", "host": "box",
"kind": KindGPU, "ts": at,
"cpus": uint16(8), "memory": uint64(32 << 30), "mem_used": uint64(8 << 30),
"mem_free": uint64(24 << 30), "load1": float32(1.5), "load5": float32(1.25),
"load15": float32(0.5), "gpu_util": float32(0.5), "gpus": uint8(2),
"gpu_model": "GB10", "cost_cents": uint64(42),
})
if s.Org != "acme" || s.Unit != "tgt-1" || s.Kind != KindGPU || !s.At.Equal(at) {
t.Fatalf("identity did not round-trip: %+v", s)
}
if s.CPUs != 8 || s.GPUs != 2 || s.Memory != 32<<30 || s.CostCents != 42 {
t.Fatalf("numbers did not round-trip: %+v", s)
}
if s.Load1 != 1.5 || s.GPUUtil != 0.5 {
t.Fatalf("floats did not round-trip: %+v", s)
}
}
// A missing or unexpectedly-typed column degrades to a zero value rather than
// panicking a read — a driver change can never crash the board.
func TestSampleFromToleratesJunk(t *testing.T) {
s := sampleFrom(map[string]any{"org": 42, "cpus": "eight", "ts": "nope"})
if s.Org != "" || s.CPUs != 0 || !s.At.IsZero() {
t.Fatalf("junk must degrade to zero values, got %+v", s)
}
if s2 := sampleFrom(map[string]any{}); s2.Unit != "" {
t.Fatalf("an empty row must decode to the zero Sample, got %+v", s2)
}
}
+406
View File
@@ -0,0 +1,406 @@
// Package samples is the fleet's compute-utilization time series: ONE table, ONE
// writer seam, ONE read face that every compute source feeds.
//
// The fleet already reports lifecycle, liveness and static inventory — the Visor
// machines, the BYO workers that dialed in, the BYO clusters, the agent
// run-targets. None of it answers "how hot is this GPU / how loaded is this
// machine, over time, per org": hanzo.compute_usage is spend/lifecycle-shaped
// (org, app, project, kind, event, machine_id, size, price_cents, ts) and carries
// no cpu/mem/gpu, and o11y's samples_v4 carries no org. An agent run-target does
// carry a real Spec + Metrics, but only the LAST one — a snapshot on the row, not
// a series. This package is the missing plane, and it is the ONLY one: sources
// append here rather than growing private telemetry tables.
//
// It is deliberately a LEAF. It depends on the datastore seam (ai/object) and the
// tenancy vocabulary (clients/principal) and nothing else in cloud, so every
// compute source — clients/agents, clients/visor, ml — can import it without a
// cycle (clients/link imports clients/agents, and clients/agents imports this).
//
// - Record(ctx, Sample) append one utilization sample (samples.go)
// - Series(ctx, Query) an org's samples over a bounded window (read.go)
// - Latest(ctx, org) each unit's most recent sample, for a fleet board
//
// ISOLATION: org is the ONLY tenant key. It leads every statement as a BOUND
// parameter (never interpolated), and a blank or oversized org fails CLOSED on
// both the write and the read — a caller for org A can never write into, or read,
// org B's series. The caller MUST pass an org it already validated server-side
// (principal.Org / a verified claim), never a raw client header.
//
// AVAILABILITY: the warehouse is never in a caller's critical path. With no
// datastore configured, Record is a no-op that returns nil — a heartbeat, a
// session or a board must never fail because the time series is absent.
package samples
import (
"context"
"errors"
"fmt"
"math"
"strings"
"sync"
"time"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud/clients/principal"
)
// Sources — the CLOSED vocabulary of compute planes that feed the series. Closed
// (validated, never coerced) so `source` stays a real dimension: an unknown source
// is a wiring bug, and silently storing it would quietly fragment every rollup.
const (
SourceAgent = "agent" // a linked run-target's heartbeat (clients/agents)
SourceBYO = "byo" // a bring-your-own worker or cluster that dialed in
SourceCloud = "cloud" // a cloud workload
SourceVisor = "visor" // a Visor-provisioned machine
)
// Kinds — the CLOSED vocabulary of compute units. The first five mirror the agent
// run-target kinds (clients/agents.Target*); `worker` is the BYO fleet's own unit,
// which is presence, not a dispatch destination. This is the fleet-wide superset,
// deliberately NOT an import of the agents vocabulary: agents owns "what an agent
// can be dispatched to", this owns "what the fleet can meter". They overlap today
// and are free to diverge without either dragging the other.
const (
KindLaptop = "laptop"
KindCloud = "cloud"
KindGPU = "gpu"
KindCluster = "cluster"
KindMachine = "machine"
KindWorker = "worker"
)
// Bounds. The numeric caps are the COLUMN's own range (see tableDDL) — a value the
// column cannot represent is a bug or an attack, and clamping to the type ceiling
// is the only write that is guaranteed well-formed.
const (
maxUnit = 128 // a unit id (target id / worker id / machine name / cluster name)
maxHost = 253 // a DNS name
maxGPUModel = 64 // matches the agent Spec's field cap
maxVocab = 32 // an over-long source/kind can only be junk; bounds the compare
maxCPUs = 65535 // cpus is UInt16
maxGPUs = 255 // gpus is UInt8
// load* is Float32. A real loadavg is single digits; a million is already
// absurd, and staying far below the float32 ceiling (~3.4e38) means the
// float64→float32 narrowing can never produce +Inf.
maxLoad = 1e6
)
// ErrInvalid marks a CALLER's error — a malformed tenant or an out-of-vocabulary
// narrower — as categorically distinct from an infrastructure failure. The two must
// never be conflated at an HTTP face: an unknown ?source is a 400 whose text is
// safe to echo (it is our own closed vocabulary), whereas a warehouse failure is
// neither the caller's fault nor theirs to read — its text names our tables and
// hosts. Faces branch on errors.Is(err, ErrInvalid).
var ErrInvalid = errors.New("samples: invalid request")
var (
errOrg = fmt.Errorf("%w: org required", ErrInvalid)
errUnit = fmt.Errorf("%w: unit required", ErrInvalid)
errSource = fmt.Errorf("%w: source must be %s|%s|%s|%s", ErrInvalid, SourceAgent, SourceBYO, SourceCloud, SourceVisor)
errKind = fmt.Errorf("%w: kind must be %s|%s|%s|%s|%s|%s", ErrInvalid,
KindLaptop, KindCloud, KindGPU, KindCluster, KindMachine, KindWorker)
)
// Sample is one utilization measurement of one compute unit at one instant — the
// value the whole plane is built around. Org/Source/Unit/Kind identify it, At is
// when it was measured (server-stamped by the caller; a zero value means "now"),
// and the rest is what the unit WAS and was DOING at that instant.
//
// The static capability (CPUs/Memory/GPUs/GPUModel) rides every row on purpose:
// the series then answers "how hot was this GPU" without a join against a registry
// whose row may have since been rewritten or deregistered. A sample is a fact, and
// a fact carries its own context.
type Sample struct {
Org string // the tenant — the ONLY tenancy key
Source string // agent | byo | cloud | visor
Unit string // the source's own id for this unit
Host string // the hostname the unit reports (may be empty)
Kind string // laptop | cloud | gpu | cluster | machine | worker
At time.Time
CPUs int // logical cores
Memory int64 // total RAM, bytes
MemUsed int64 // bytes
MemFree int64 // bytes
Load1 float64
Load5 float64
Load15 float64
GPUUtil float64 // 0..1 aggregate utilization
GPUs int // accelerator count
GPUModel string // the representative accelerator ("GB10"); GPUs carries the count
CostCents int64 // the resale price of this unit for this sample's hour; 0 when unpriced
}
func validSource(s string) bool {
switch s {
case SourceAgent, SourceBYO, SourceCloud, SourceVisor:
return true
}
return false
}
func validKind(k string) bool {
switch k {
case KindLaptop, KindCloud, KindGPU, KindCluster, KindMachine, KindWorker:
return true
}
return false
}
// sanitize returns a Sample every field of which the columns can represent. It is
// total (never errors) and splits on a deliberate line:
//
// - KEYS (org, unit) are only TRIMMED here, never truncated. Truncating a key
// re-attributes the row: a clamped org could collide with another tenant's
// prefix, and a clamped unit merges two units' series. Over-long keys fail
// CLOSED in validate instead.
// - COSMETIC fields (host, gpu_model) are clamped — they label the row, they do
// not key it, so a truncated value is inert.
// - NUMBERS are clamped to their column's range, with non-finite floats coerced
// to 0, so no client can smuggle a NaN/Inf or overflow a column.
func (s Sample) sanitize() Sample {
out := Sample{
Org: strings.TrimSpace(s.Org),
Source: strings.ToLower(clampStr(s.Source, maxVocab)),
Unit: strings.TrimSpace(s.Unit),
Host: clampStr(s.Host, maxHost),
Kind: strings.ToLower(clampStr(s.Kind, maxVocab)),
At: s.At.UTC(),
CPUs: clampInt(s.CPUs, maxCPUs),
Memory: nonNegI64(s.Memory),
MemUsed: nonNegI64(s.MemUsed),
MemFree: nonNegI64(s.MemFree),
Load1: clampF(s.Load1, maxLoad),
Load5: clampF(s.Load5, maxLoad),
Load15: clampF(s.Load15, maxLoad),
GPUUtil: clampF01(s.GPUUtil),
GPUs: clampInt(s.GPUs, maxGPUs),
GPUModel: clampStr(s.GPUModel, maxGPUModel),
CostCents: nonNegI64(s.CostCents),
}
if out.At.IsZero() {
out.At = time.Now().UTC() // the server owns the clock, always
}
return out
}
// validate fails CLOSED on anything that would make the row untenanted,
// unattributable, or silently re-keyed. It judges the SANITIZED value, so it is
// the last word on what reaches the table.
func (s Sample) validate() error {
if s.Org == "" || len(s.Org) > principal.MaxOrgLen {
return errOrg
}
if s.Unit == "" || len(s.Unit) > maxUnit {
return errUnit
}
if !validSource(s.Source) {
return errSource
}
if !validKind(s.Kind) {
return errKind
}
return nil
}
// args renders the sanitized Sample as the INSERT's positional binds, each in the
// column's OWN type. sanitize has already bounded every value to that column's
// range, so these narrowings are lossless by construction (proven in the tests).
func (s Sample) args() []any {
return []any{
s.Org, s.Source, s.Unit, s.Host, s.Kind, s.At,
uint16(s.CPUs), uint64(s.Memory), uint64(s.MemUsed), uint64(s.MemFree),
float32(s.Load1), float32(s.Load5), float32(s.Load15), float32(s.GPUUtil),
uint8(s.GPUs), s.GPUModel, uint64(s.CostCents),
}
}
// ── schema ──────────────────────────────────────────────────────────────────
const table = "hanzo.compute_samples"
// cols is the ONE column list. The INSERT binds it positionally and the SELECTs
// project it, so the write and read shapes can never drift apart.
const cols = `org, source, unit, host, kind, ts, cpus, memory, mem_used, mem_free, load1, load5, load15, gpu_util, gpus, gpu_model, cost_cents`
// tableDDL is the ONE definition of hanzo.compute_samples: the raw series.
// MergeTree (append-only — a sample is a fact, never updated), partitioned by
// month so the TTL expiry drops whole parts instead of mutating, and ordered
// org-FIRST so a tenant's read is a contiguous prefix scan — the same shape
// eval_traces and cloud_usage use.
const tableDDL = `
CREATE TABLE IF NOT EXISTS hanzo.compute_samples (
org LowCardinality(String),
source LowCardinality(String),
unit String,
host String,
kind LowCardinality(String),
ts DateTime64(3),
cpus UInt16,
memory UInt64,
mem_used UInt64,
mem_free UInt64,
load1 Float32,
load5 Float32,
load15 Float32,
gpu_util Float32,
gpus UInt8,
gpu_model String,
cost_cents UInt64
) ENGINE = MergeTree
PARTITION BY toYYYYMM(ts)
ORDER BY (org, unit, ts)
TTL toDateTime(ts) + INTERVAL 90 DAY`
// rollupDDL is the hourly trend target table. Keyed (org, unit, hour) — org first,
// same tenancy prefix as the raw table.
//
// Its TTL matches the raw table's 90 days ON PURPOSE: this rollup exists to make a
// long-window trend CHEAP (a 30d read is ~720 rows per unit instead of ~43k), not
// to retain beyond the raw series. Keeping trends longer than the raw samples is a
// storage-cost decision the operator owns — it is a one-line TTL change here.
const rollupDDL = `
CREATE TABLE IF NOT EXISTS hanzo.compute_samples_hourly (
org LowCardinality(String),
unit String,
source LowCardinality(String),
kind LowCardinality(String),
hour DateTime,
gpu_util_avg AggregateFunction(avg, Float32),
gpu_util_max AggregateFunction(max, Float32),
load1_avg AggregateFunction(avg, Float32),
mem_used_avg AggregateFunction(avg, UInt64),
cost_cents_max AggregateFunction(max, UInt64)
) ENGINE = AggregatingMergeTree
PARTITION BY toYYYYMM(hour)
ORDER BY (org, unit, hour)
TTL hour + INTERVAL 90 DAY`
// rollupViewDDL feeds the rollup on every insert into the raw table. The TO form
// (an explicit target table, not an implicit .inner) so the aggregate survives a
// view rebuild and can be read directly. A TO-view is an INSERT SELECT: its
// columns match the target POSITIONALLY, so this projection is in the target's
// exact declared order — keep the two in lockstep.
const rollupViewDDL = `
CREATE MATERIALIZED VIEW IF NOT EXISTS hanzo.compute_samples_hourly_mv
TO hanzo.compute_samples_hourly
AS SELECT
org,
unit,
source,
kind,
toStartOfHour(ts) AS hour,
avgState(gpu_util) AS gpu_util_avg,
maxState(gpu_util) AS gpu_util_max,
avgState(load1) AS load1_avg,
avgState(mem_used) AS mem_used_avg,
maxState(cost_cents) AS cost_cents_max
FROM hanzo.compute_samples
GROUP BY org, unit, source, kind, hour`
// schema is the whole plane, in dependency order: the raw table, the rollup's
// target, then the view that feeds it. Every statement is IF NOT EXISTS.
var schema = []string{tableDDL, rollupDDL, rollupViewDDL}
const insertStmt = `INSERT INTO ` + table + ` (` + cols + `) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)`
var (
ensureMu sync.Mutex
ensured bool
)
// ensure creates the plane idempotently, serialized so a burst of first writes
// issues one DDL pass rather than a storm. It latches ONLY on success, so a
// transient datastore failure at boot is retried on the next write instead of
// poisoning the process (the eval telemetry pattern).
func ensure(ctx context.Context) error {
ensureMu.Lock()
defer ensureMu.Unlock()
if ensured {
return nil
}
for _, ddl := range schema {
if err := aiobject.DatastoreExec(ctx, ddl); err != nil {
return fmt.Errorf("samples: ensure schema: %w", err)
}
}
ensured = true
return nil
}
// Record appends one utilization sample to the series.
//
// Fail-SOFT on absence, fail-CLOSED on nonsense — the two are different and are
// reported differently:
//
// - No datastore configured (DatastoreEnabled() == false): a no-op returning
// nil. The time series is optional infrastructure; its absence must never
// fail or stall the heartbeat, session or board that emitted the sample.
// - A blank/oversized org, a blank/oversized unit, or an unknown source/kind:
// an error and NO write. An untenanted or misattributed row is worse than no
// row at all.
//
// It is synchronous and honours the ctx it is given — the CALLER owns the
// concurrency policy, exactly as the billing warehouse write does (`go
// zapWriteUsage(...)`). An emitter on a request path should hand it a DETACHED,
// bounded context so neither a slow warehouse nor a client disconnect can touch
// its own contract.
func Record(ctx context.Context, s Sample) error {
if !aiobject.DatastoreEnabled() {
return nil // no warehouse — the sample is dropped, the caller is untouched
}
s = s.sanitize()
if err := s.validate(); err != nil {
return err
}
if err := ensure(ctx); err != nil {
return err
}
if err := aiobject.DatastoreExec(ctx, insertStmt, s.args()...); err != nil {
return fmt.Errorf("samples: record: %w", err)
}
return nil
}
// ── bounds (total functions; mirrors clients/agents.Spec/Metrics Sanitize) ───
func clampStr(s string, n int) string {
s = strings.TrimSpace(s)
if len(s) > n {
return strings.ToValidUTF8(s[:n], "")
}
return s
}
func clampInt(i, hi int) int {
if i < 0 {
return 0
}
if i > hi {
return hi
}
return i
}
func nonNegI64(i int64) int64 {
if i < 0 {
return 0
}
return i
}
// clampF returns a finite, non-negative float bounded by hi (NaN/Inf/negative → 0),
// so no sample can poison a column or overflow the float32 narrowing.
func clampF(f float64, hi float64) float64 {
if math.IsNaN(f) || math.IsInf(f, 0) || f < 0 {
return 0
}
if f > hi {
return hi
}
return f
}
// clampF01 returns a finite float in [0,1] — a utilization ratio.
func clampF01(f float64) float64 { return clampF(f, 1) }
+231
View File
@@ -0,0 +1,231 @@
package samples
import (
"context"
"math"
"strings"
"testing"
"time"
"github.com/hanzoai/cloud/clients/principal"
)
// good is a well-formed sample — the baseline each hostile case mutates from.
func good() Sample {
return Sample{
Org: "acme", Source: SourceAgent, Unit: "tgt-1", Host: "box.local",
Kind: KindGPU, At: time.Unix(1700000000, 0).UTC(),
CPUs: 8, Memory: 32 << 30, MemUsed: 8 << 30, MemFree: 24 << 30,
Load1: 1.5, Load5: 1.2, Load15: 0.9, GPUUtil: 0.42,
GPUs: 1, GPUModel: "GB10", CostCents: 0,
}
}
// ── (b) sanitize bounds ─────────────────────────────────────────────────────
// A hostile or buggy source must never poison a column. Every non-finite float
// collapses to 0, every negative to 0, and every over-range number to its column's
// ceiling — the row is always well-formed no matter what was handed in.
func TestSanitizeBoundsHostileValues(t *testing.T) {
s := Sample{
Org: " acme ", Source: " AGENT ", Unit: " tgt-1 ", Kind: " GPU ",
Host: strings.Repeat("h", 400),
CPUs: 1 << 20, // far past the UInt16 column
Memory: -5,
MemUsed: -1, MemFree: -99,
Load1: math.NaN(), Load5: math.Inf(1), Load15: -3,
GPUUtil: 7.5, // a ratio > 1
GPUs: 9999,
GPUModel: strings.Repeat("m", 200),
CostCents: -100,
}.sanitize()
if s.Org != "acme" || s.Unit != "tgt-1" {
t.Fatalf("keys must be trimmed: org=%q unit=%q", s.Org, s.Unit)
}
if s.Source != SourceAgent || s.Kind != KindGPU {
t.Fatalf("vocab must be lower-cased: source=%q kind=%q", s.Source, s.Kind)
}
if len(s.Host) != maxHost {
t.Fatalf("host must clamp to %d, got %d", maxHost, len(s.Host))
}
if len(s.GPUModel) != maxGPUModel {
t.Fatalf("gpu model must clamp to %d, got %d", maxGPUModel, len(s.GPUModel))
}
if s.CPUs != maxCPUs {
t.Fatalf("cpus must clamp to the UInt16 ceiling %d, got %d", maxCPUs, s.CPUs)
}
if s.GPUs != maxGPUs {
t.Fatalf("gpus must clamp to the UInt8 ceiling %d, got %d", maxGPUs, s.GPUs)
}
if s.Memory != 0 || s.MemUsed != 0 || s.MemFree != 0 || s.CostCents != 0 {
t.Fatalf("negatives must floor at 0: %+v", s)
}
if s.Load1 != 0 || s.Load5 != 0 || s.Load15 != 0 {
t.Fatalf("NaN/Inf/negative loads must collapse to 0: %v %v %v", s.Load1, s.Load5, s.Load15)
}
if s.GPUUtil != 1 {
t.Fatalf("gpu_util must clamp into [0,1], got %v", s.GPUUtil)
}
}
// A huge-but-finite load must clamp to maxLoad rather than survive into the
// Float32 column as +Inf. This is the narrowing the args() binds depend on.
func TestSanitizeClampsLoadBelowFloat32Ceiling(t *testing.T) {
s := good()
s.Load1 = 1e300
s = s.sanitize()
if s.Load1 != maxLoad {
t.Fatalf("load1 want clamp to %v, got %v", maxLoad, s.Load1)
}
if math.IsInf(float64(float32(s.Load1)), 0) {
t.Fatal("clamped load must not narrow to +Inf in the Float32 column")
}
}
// A zero At means "now" — the server owns the clock, so no row is ever written
// with a zero timestamp.
func TestSanitizeStampsZeroTime(t *testing.T) {
s := good()
s.At = time.Time{}
if got := s.sanitize().At; got.IsZero() {
t.Fatal("a zero At must be stamped with the server clock")
}
}
// The binds are lossless for any sanitized sample: every narrowing (int→uint16,
// int→uint8, float64→float32) is inside the column's range by construction.
func TestArgsNarrowingIsLosslessForSanitizedSamples(t *testing.T) {
s := Sample{
Org: "acme", Source: SourceAgent, Unit: "u", Kind: KindGPU,
CPUs: 1 << 30, GPUs: 1 << 20, Load1: 1e300, GPUUtil: 99,
Memory: math.MaxInt64, CostCents: math.MaxInt64,
}.sanitize()
args := s.args()
if len(args) != strings.Count(insertStmt, "?") {
t.Fatalf("args=%d but insert has %d placeholders", len(args), strings.Count(insertStmt, "?"))
}
if got := args[6].(uint16); uint64(got) != uint64(s.CPUs) {
t.Fatalf("cpus bind wrapped: %d != %d", got, s.CPUs)
}
if got := args[14].(uint8); uint64(got) != uint64(s.GPUs) {
t.Fatalf("gpus bind wrapped: %d != %d", got, s.GPUs)
}
for i, a := range args {
if f, ok := a.(float32); ok {
if math.IsNaN(float64(f)) || math.IsInf(float64(f), 0) {
t.Fatalf("arg %d narrowed to a non-finite float32: %v", i, f)
}
}
}
}
// The column list, the INSERT placeholders and the bind count are ONE shape. This
// is the drift that silently corrupts a warehouse write, so it is asserted.
func TestInsertShapeMatchesColumnList(t *testing.T) {
ncols := len(strings.Split(cols, ","))
nph := strings.Count(insertStmt, "?")
nargs := len(good().sanitize().args())
if ncols != nph || nph != nargs {
t.Fatalf("shape drift: %d columns, %d placeholders, %d args", ncols, nph, nargs)
}
}
// ── validate: fail CLOSED on anything untenanted or misattributed ────────────
func TestValidateFailsClosed(t *testing.T) {
cases := []struct {
name string
mut func(*Sample)
want error
}{
{"blank org", func(s *Sample) { s.Org = "" }, errOrg},
{"whitespace org", func(s *Sample) { s.Org = " " }, errOrg},
{"oversized org", func(s *Sample) { s.Org = strings.Repeat("o", principal.MaxOrgLen+1) }, errOrg},
{"blank unit", func(s *Sample) { s.Unit = "" }, errUnit},
{"oversized unit", func(s *Sample) { s.Unit = strings.Repeat("u", maxUnit+1) }, errUnit},
{"unknown source", func(s *Sample) { s.Source = "wat" }, errSource},
{"blank source", func(s *Sample) { s.Source = "" }, errSource},
{"unknown kind", func(s *Sample) { s.Kind = "toaster" }, errKind},
{"blank kind", func(s *Sample) { s.Kind = "" }, errKind},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
s := good()
tc.mut(&s)
if err := s.sanitize().validate(); err != tc.want {
t.Fatalf("want %v, got %v", tc.want, err)
}
})
}
if err := good().sanitize().validate(); err != nil {
t.Fatalf("a well-formed sample must validate, got %v", err)
}
}
// An over-long org must be REJECTED, never truncated: a clamped org would write
// the row under a DIFFERENT tenant (a 129-char org silently becoming its own
// 128-char prefix could collide with a real one). Same for unit, which would
// merge two units' series. This is the sharpest tenancy edge in the package.
func TestOversizedKeysAreRejectedNotTruncated(t *testing.T) {
victim := strings.Repeat("a", principal.MaxOrgLen) // a real 128-char org
attacker := victim + "-extra" // wants to land in victim's series
s := good()
s.Org = attacker
san := s.sanitize()
if san.Org == victim {
t.Fatal("sanitize truncated an over-long org onto another tenant's key")
}
if err := san.validate(); err != errOrg {
t.Fatalf("an over-long org must fail closed, got %v", err)
}
u := good()
u.Unit = strings.Repeat("u", maxUnit) + "-extra"
if err := u.sanitize().validate(); err != errUnit {
t.Fatalf("an over-long unit must fail closed, got %v", err)
}
}
// ── (c) fail-soft: no datastore ⇒ Record is a no-op returning nil ────────────
// The whole plane is optional infrastructure. With no datastore configured every
// emitter must sail straight through — including, deliberately, one carrying a
// sample that would otherwise be rejected: nothing is written either way, so the
// caller is never punished for an absent warehouse.
func TestRecordIsNoOpWithoutDatastore(t *testing.T) {
if err := Record(context.Background(), good()); err != nil {
t.Fatalf("Record must no-op to nil without a datastore, got %v", err)
}
if err := Record(context.Background(), Sample{}); err != nil {
t.Fatalf("even an empty sample must not fail without a datastore, got %v", err)
}
}
// A cancelled context must not turn into a caller-visible failure either.
func TestRecordIgnoresCancelledContextWithoutDatastore(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
cancel()
if err := Record(ctx, good()); err != nil {
t.Fatalf("Record must no-op to nil, got %v", err)
}
}
// The reads are honest-empty (never fabricated) when the warehouse is absent, and
// still fail closed on a blank tenant BEFORE consulting it.
func TestReadsAreHonestEmptyWithoutDatastore(t *testing.T) {
got, err := Series(context.Background(), Query{Org: "acme"})
if err != nil || len(got) != 0 {
t.Fatalf("Series want empty/nil, got %v / %v", got, err)
}
m, err := Latest(context.Background(), "acme")
if err != nil || len(m) != 0 {
t.Fatalf("Latest want empty/nil, got %v / %v", m, err)
}
if _, err := Series(context.Background(), Query{Org: ""}); err != errOrg {
t.Fatalf("blank org must fail closed even with no datastore, got %v", err)
}
if _, err := Latest(context.Background(), ""); err != errOrg {
t.Fatalf("blank org must fail closed even with no datastore, got %v", err)
}
}
+4 -4
View File
@@ -1,16 +1,16 @@
package sign
// schema is the per-tenant SQLite DDL gojabase runs (idempotently) on first open
// schema is the per-tenant SQLite DDL NewBase runs (idempotently) on first open
// of each tenant's {DataDir}/sign/{org}.db. It is a faithful, lean projection of
// the Documenso Envelope/DocumentData/Recipient/Field/Signature/DocumentAuditLog
// models onto one PDF per document — the complete single-document signing flow.
// Ids are application-generated TEXT (via the gojabase __newId() host-fn), so no
// Ids are application-generated TEXT (via the NewBase __newId() host-fn), so no
// autoincrement crosses the host boundary. Tenant isolation is the per-tenant
// FILE (gojabase), so there is no org column — the bundle only ever sees its own
// FILE (NewBase), so there is no org column — the bundle only ever sees its own
// tenant's DB.
// PDF BYTES do NOT live in this DB: document_data.data / initial_data hold the
// opaque object-storage blob KEY (type 'BLOB_KEY'), and the bytes live on the
// gojabase __blob seam (deps.VFS / S3). A 32 MiB base64 PDF in a TEXT column would
// NewBase __blob seam (deps.VFS / S3). A 32 MiB base64 PDF in a TEXT column would
// bloat the per-tenant SQLite and be copied on every read — the same object-store
// seam clients/dataroom uses for document bytes. `data` points at the current
// (sealed once completed) PDF; `initial_data` keeps the original.
+10 -10
View File
@@ -7,13 +7,13 @@
// (the #96 pilot) established: the server-side domain (documents, recipients,
// fields, the signing flow/state machine, audit trail, completion) is ported to
// a self-contained goja bundle in github.com/hanzoai/sign; the REUSABLE
// clients/gojabase binding runs it and gives it PERSISTENCE over per-tenant
// clients/goja binding runs it and gives it PERSISTENCE over per-tenant
// Base/SQLite (__db/__newId/__now, one SQLite file per tenant, ONE transaction
// per request). This leaf adds ZERO storage glue of its own.
//
// THE HARD PART — PDF + PKI — is the one capability goja cannot provide: it is
// implemented as Go host-functions (signer.go: pdfcpu render + digitorus/pdfsign
// x509/PKCS#7 seal) and injected via the additive gojabase Config.HostFns as
// x509/PKCS#7 seal) and injected via the additive goja BaseConfig.HostFns as
// __pdf = { stamp, sign }. The signing-request/recipient/field/audit LOGIC and
// the seal ORCHESTRATION stay in the TS bundle; only the crypto/PDF primitive is
// Go. A real signed PDF comes out.
@@ -22,7 +22,7 @@
// VALIDATED cloud principal (principal.Org), never a client header. Recipient
// token routes (/v1/sign/o/:org/sign/:token) are unauthenticated capability
// links: the :org segment selects the tenant DB and the crypto-random token
// authorizes — a wrong org simply cannot hold a valid token. gojabase pre-routes
// authorizes — a wrong org simply cannot hold a valid token. NewBase pre-routes
// the bundle's db to that tenant, so isolation is a host property.
//
// ACTIVATION: sign is NOT staged — it mounts under the mount-all default (empty
@@ -40,7 +40,7 @@ import (
"net/http"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/gojabase"
"github.com/hanzoai/cloud/clients/goja"
"github.com/hanzoai/cloud/clients/principal"
signbundle "github.com/hanzoai/sign"
"github.com/zap-proto/zip"
@@ -52,7 +52,7 @@ const maxBody = 32 << 20
// state is sign's own data; shared deps live in the embedded cloud.Base.
type state struct {
host *gojabase.Host
host *goja.BaseHost
}
// mounted is the active service so shutdown can release the per-tenant stores.
@@ -79,7 +79,7 @@ func Mount(app *zip.App, deps cloud.Deps) error {
// Sign persists PDF BYTES on the object-storage seam (deps.VFS), NOT inline in
// the per-tenant SQLite — a 32 MiB base64 PDF in a TEXT column would bloat the
// tenant DB and be re-copied on every read. gojabase injects it as __blob,
// tenant DB and be re-copied on every read. NewBase injects it as __blob,
// tenant-scoped. Without VFS sign cannot store documents, so serve health-only
// (cloud stays up) rather than write PDFs into the tenant DB.
if deps.VFS == nil {
@@ -95,7 +95,7 @@ func Mount(app *zip.App, deps cloud.Deps) error {
if err != nil {
return fmt.Errorf("sign.Mount: load bundle: %w", err)
}
host, err := gojabase.New(gojabase.Config{
host, err := goja.NewBase(goja.BaseConfig{
Name: "sign",
Bundle: bundle,
Schema: schema,
@@ -104,7 +104,7 @@ func Mount(app *zip.App, deps cloud.Deps) error {
HostFns: map[string]any{"__pdf": sg.pdfHostObject()},
})
if err != nil {
return fmt.Errorf("sign.Mount: gojabase host: %w", err)
return fmt.Errorf("sign.Mount: goja NewBase host: %w", err)
}
s := &cloud.Service[state]{Base: cloud.NewBase(deps, "sign"), State: state{host: host}}
mounted = s
@@ -179,7 +179,7 @@ func token(s *cloud.Service[state], route string, readBody bool) zip.Handler {
}
// dispatch decodes the body, runs the bundle route on the tenant's Base store
// (one transaction per request via gojabase), and writes {status, body}.
// (one transaction per request via NewBase), and writes {status, body}.
func dispatch(s *cloud.Service[state], c *zip.Ctx, route, tenant string, params map[string]string, readBody bool) error {
var body any
if readBody {
@@ -193,7 +193,7 @@ func dispatch(s *cloud.Service[state], c *zip.Ctx, route, tenant string, params
}
}
}
resp, err := s.State.host.Dispatch(c.Context(), tenant, gojabase.Request{
resp, err := s.State.host.Dispatch(c.Context(), tenant, goja.BaseRequest{
Route: route,
Params: params,
Body: body,
+2 -2
View File
@@ -128,7 +128,7 @@ func decode(t *testing.T, b []byte) map[string]any {
}
// TestFullSigningFlow is the end-to-end wire proof of the COMPLETE e-sign flow on
// the reusable gojabase RW-Base host: create document → add recipient → add
// the reusable NewBase RW-Base host: create document → add recipient → add
// fields → send → recipient signs each field → complete → the document seals to
// COMPLETED with a REAL x509/PKCS#7 signed PDF, and the audit trail records every
// step. All per-tenant Base/SQLite-backed.
@@ -310,7 +310,7 @@ func TestFullSigningFlow(t *testing.T) {
// TestTenantIsolation proves two orgs never see each other's documents (the DB is
// per-tenant; a wrong token/org combination cannot resolve) and that a validation
// error rolls the request transaction back (gojabase atomicity).
// error rolls the request transaction back (NewBase atomicity).
func TestTenantIsolation(t *testing.T) {
app, _ := mountApp(t)
pdf, _ := os.ReadFile("testdata/example.pdf")
+1 -1
View File
@@ -309,7 +309,7 @@ func sanitizeText(s string) string {
// ---- host-function objects injected into the goja runtime -------------------
// pdfHostObject builds the __pdf = { stamp, sign } host object (injected via
// gojabase Config.HostFns) over base64 PDF strings — the PDF/PKI primitive goja
// NewBase Config.HostFns) over base64 PDF strings — the PDF/PKI primitive goja
// cannot do. A returned Go error surfaces in JS as a thrown Error the bundle
// catches. Signing ORCHESTRATION stays in the TS bundle; only stamp+seal is Go.
func (s *signer) pdfHostObject() map[string]any {
+1 -1
View File
@@ -31,7 +31,7 @@ import (
"strings"
"sync"
minio "github.com/minio/minio-go/v7"
minio "github.com/hanzoai/s3-go"
"github.com/zap-proto/zip"
luxlog "github.com/luxfi/log"
+1 -1
View File
@@ -63,7 +63,7 @@ import (
"strings"
"time"
minio "github.com/minio/minio-go/v7"
minio "github.com/hanzoai/s3-go"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/principal"
+10
View File
@@ -35,6 +35,7 @@ import (
"sync"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/cron"
tasksui "github.com/hanzoai/cloud/clients/tasks/ui"
tasksauth "github.com/hanzoai/tasks/pkg/auth"
tasks "github.com/hanzoai/tasks/pkg/tasks"
@@ -66,6 +67,15 @@ func Mount(app *zip.App, deps cloud.Deps) error {
app.All("/tasks/*", ui)
deps.Logger.New("subsystem", "tasks").Info("tasks HTTP+UI surface mounted (shared in-process engine)", "brand", deps.Brand)
// Platform cron is a FACET of tasks, not its own subsystem: it mounts NO routes,
// only registers durable schedules on the SAME shared engine (cloud.EmbeddedTasks)
// this surface fronts. Folded in here as a terminal sub-mount (was a separate Wire
// entry) so there is ONE tasks subsystem. cron.Mount just launches a background
// starter that waits for the engine wired after MountAll — no ordering dependency.
if err := cron.Mount(app, deps); err != nil {
return err
}
return nil
}
+408
View File
@@ -0,0 +1,408 @@
package usage
import (
"fmt"
"net/http"
"sort"
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
)
// account.go is the ACCOUNT-USAGE HTTP layer: the collector's record endpoint and
// the per-provider sample dash. It moved here from clients/link's usage plane so the
// ONE usage subsystem owns ALL usage. It records what a developer's OWN AI accounts
// have consumed of their OWN plans (metered from each provider's own login) into the
// warehouse series (datastore.go); the READ faces are summary (usage.go), this dash
// (samples), and analytics.
//
// POST /v1/usage report samples (the collector) -> {accepted, stored}
// GET /v1/usage/samples ONE provider account's own dash -> {current, windows}
//
// It records usage and NOTHING else: keeping the link REGISTRY current (which
// accounts are signed in, their latest snapshot) is clients/link's own concern,
// refreshed by POST /v1/links. Recording a sample and registering a link are two
// orthogonal operations, each with exactly one home — a usage report no longer
// writes a Link row, so there is one and only one way to set an account's snapshot.
//
// Every route is org+subject scoped through the same caller() gate as summary: a
// validated principal and a non-empty org, else 401. A caller reads and writes only
// their OWN accounts. The org is NEVER a parameter.
// caller resolves the (org, subject) scope for a request: the VALIDATED IAM owner
// claim (principal.Org — the trusted minted X-Org-Id, never a client header) plus
// c.User() (the owning subject, non-empty once principal.Org returns ok, since Org
// composes Validated). Every account-usage handler and the summary's account block
// gate on it — an off-gateway forge with no validated user is refused fail-closed.
func caller(c *zip.Ctx) (org, user string, ok bool) {
org, ok = principal.Org(c)
if !ok {
return "", "", false
}
return org, trim(c.User()), true
}
// Ranges — the closed allowlist for a sample-dash read window. This is the
// FINE-GRAINED account-usage grammar (a live lane dash cares about the 1h/6h
// scale); the coarser cost/analytics grammar (aiobject.ResolveCloudUsageWindow:
// 24h/7d/30d/custom + a bucket interval) drives summary and analytics. Two lanes,
// two grammars, each complete for its read.
const (
Range1h = "1h"
Range24h = "24h"
Range7d = "7d"
Range30d = "30d"
)
// resolveRange maps a range label to an absolute [from, to) window. Pure: `now` is
// injected. An unknown label is an error, never a silent default — a caller who
// asked for a window we do not have must be told, not shown a different one.
func resolveRange(label string, now time.Time) (from, to time.Time, err error) {
now = now.UTC()
switch trim(label) {
case "", Range24h:
return now.Add(-24 * time.Hour), now, nil
case Range1h:
return now.Add(-time.Hour), now, nil
case Range7d:
return now.Add(-7 * 24 * time.Hour), now, nil
case Range30d:
return now.Add(-30 * 24 * time.Hour), now, nil
}
return time.Time{}, time.Time{}, fmt.Errorf("range must be one of 1h, 24h, 7d, 30d")
}
// ── views ────────────────────────────────────────────────────────────────────
// sampleView is one window instance on the wire. Unknown values are OMITTED rather
// than sent as zero, and `confidence` says whether the counters that remain mean
// anything — so a console renders "—" where the meter knew nothing, and never a
// fabricated 0.
type sampleView struct {
Lane string `json:"lane"`
Window string `json:"window"`
WindowMinutes int32 `json:"windowMinutes,omitempty"`
WindowStart string `json:"windowStart,omitempty"`
ResetsAt string `json:"resetsAt,omitempty"`
UsedPct float64 `json:"usedPct"`
Confidence string `json:"confidence"`
Synthetic bool `json:"synthetic,omitempty"`
Requests int64 `json:"requests,omitempty"`
InputTokens int64 `json:"inputTokens,omitempty"`
OutputTokens int64 `json:"outputTokens,omitempty"`
TotalTokens int64 `json:"totalTokens,omitempty"`
CachedInputTokens int64 `json:"cachedInputTokens,omitempty"`
CostCents int64 `json:"costCents,omitempty"`
CostLimitCents int64 `json:"costLimitCents,omitempty"`
Currency string `json:"currency,omitempty"`
Account string `json:"account,omitempty"`
Plan string `json:"plan,omitempty"`
Machine string `json:"machine,omitempty"`
}
func toSampleView(x Sample) sampleView {
return sampleView{
Lane: x.Lane, Window: x.Window, WindowMinutes: x.WindowMinutes,
WindowStart: rfc3339Of(x.WindowStart), ResetsAt: rfc3339Of(x.ResetsAt),
UsedPct: x.UsedPct, Confidence: x.Confidence, Synthetic: x.Synthetic,
Requests: x.Requests, InputTokens: x.InputTokens, OutputTokens: x.OutputTokens,
TotalTokens: x.TotalTokens, CachedInputTokens: x.CachedInputTokens,
CostCents: x.CostCents, CostLimitCents: x.CostLimitCents, Currency: x.Currency,
Account: x.Account, Plan: x.Plan, Machine: x.Machine,
}
}
func rfc3339Of(t time.Time) string {
if t.IsZero() {
return ""
}
return t.UTC().Format(time.RFC3339)
}
// TotalView is one row of the account-usage board (the summary's `accounts.rows`).
// Source and scope are what keep the board honest — see the Source/Scope consts.
type TotalView struct {
Source string `json:"source"` // account | hanzo
Scope string `json:"scope"` // user | org
Provider string `json:"provider"`
Window string `json:"window,omitempty"`
Requests int64 `json:"requests,omitempty"`
Tokens int64 `json:"tokens,omitempty"`
CostCents int64 `json:"costCents,omitempty"`
UsedPct float64 `json:"usedPct,omitempty"`
Confidence string `json:"confidence"`
Windows int64 `json:"windows,omitempty"`
}
func toTotalView(t Total) TotalView {
return TotalView{
Source: t.Source, Scope: t.Scope, Provider: t.Provider, Window: t.Window,
Requests: t.Requests, Tokens: t.Tokens, CostCents: t.CostCents,
UsedPct: t.UsedPct, Confidence: t.Confidence, Windows: t.Windows,
}
}
// ── ingest ───────────────────────────────────────────────────────────────────
// sampleReq is one reported sample.
//
// THERE IS NO `ts`. The server owns the observation clock, always — and that is a
// security property, not a convenience. `ts` is the version that decides which read
// of a window wins; a client that could set it could pin a stale or flattering
// snapshot as newest forever, and no later truthful poll would ever overwrite it.
//
// The historical-window case does not need one. A sample says WHICH window it
// measures with `windowStart` (or `resetsAt` + `windowMinutes`, which the meter
// reports anyway) — all bounded to a sane interval around now. That separates two
// clocks a single `ts` would braid together: WHEN THE WINDOW WAS (the client's fact
// to state) and WHEN WE LEARNED IT (ours). A backfill of a real historical window
// lands at the right instant with an honest observation time.
type sampleReq struct {
Provider string `json:"provider"`
Account string `json:"account"`
Plan string `json:"plan"`
Kind string `json:"kind"`
Machine string `json:"machine"`
Lane string `json:"lane"`
Window string `json:"window"`
WindowMinutes int32 `json:"windowMinutes"`
WindowStart string `json:"windowStart"` // RFC3339; bounded
ResetsAt string `json:"resetsAt"` // RFC3339; bounded
UsedPct float64 `json:"usedPct"`
Confidence string `json:"confidence"`
Synthetic bool `json:"synthetic"`
Requests int64 `json:"requests"`
InputTokens int64 `json:"inputTokens"`
OutputTokens int64 `json:"outputTokens"`
TotalTokens int64 `json:"totalTokens"`
CachedInputTokens int64 `json:"cachedInputTokens"`
CostCents int64 `json:"costCents"`
CostLimitCents int64 `json:"costLimitCents"`
Currency string `json:"currency"`
}
// reportReq accepts one sample or many: a poller reports its lanes in one call.
type reportReq struct {
Samples []sampleReq `json:"samples"`
sampleReq
}
// samplesOf flattens the one-or-many body into the batch.
func (r reportReq) samplesOf() []sampleReq {
if len(r.Samples) > 0 {
return r.Samples
}
if r.Provider == "" && r.Window == "" {
return nil
}
return []sampleReq{r.sampleReq}
}
// parseSample validates a reported sample and turns it into a bounded value. The
// CLOSED vocabularies (provider presence, window, kind) are 400s, never silently
// rewritten — a caller whose window class we did not understand must be told, or
// their dash would quietly fill with a class they never reported. Everything else
// is clamped by Sanitize.
func parseSample(in sampleReq, now time.Time) (Sample, error) {
if trim(in.Provider) == "" {
return Sample{}, zip.ErrBadRequest("provider is required")
}
if trim(in.Machine) == "" {
return Sample{}, zip.ErrBadRequest("machine is required")
}
if !validWindow(trim(in.Window)) {
return Sample{}, zip.ErrBadRequest("window must be one of 6h, day, week, month")
}
if k := trim(in.Kind); k != "" && !validKind(k) {
return Sample{}, zip.ErrBadRequest("kind must be subscription or apikey")
}
ws, err := parseInstant(in.WindowStart)
if err != nil {
return Sample{}, zip.ErrBadRequest("windowStart must be RFC3339")
}
ra, err := parseInstant(in.ResetsAt)
if err != nil {
return Sample{}, zip.ErrBadRequest("resetsAt must be RFC3339")
}
s := Sample{
Provider: in.Provider, Account: in.Account, Plan: in.Plan, Kind: in.Kind,
Machine: in.Machine, Lane: in.Lane, Window: in.Window,
WindowMinutes: in.WindowMinutes, WindowStart: ws, ResetsAt: ra,
UsedPct: in.UsedPct, Confidence: in.Confidence, Synthetic: in.Synthetic,
Requests: in.Requests, InputTokens: in.InputTokens, OutputTokens: in.OutputTokens,
TotalTokens: in.TotalTokens, CachedInputTokens: in.CachedInputTokens,
CostCents: in.CostCents, CostLimitCents: in.CostLimitCents, Currency: in.Currency,
}
return s.Sanitize(now), nil
}
func parseInstant(s string) (time.Time, error) {
if trim(s) == "" {
return time.Time{}, nil
}
return time.Parse(time.RFC3339, trim(s))
}
// reportResp answers a record: how many samples were accepted, and — honestly —
// whether the warehouse stored them. `stored:false` means the datastore was
// unavailable; the samples were validated and accepted but not persisted, so a
// device can retry without being blocked.
type reportResp struct {
Accepted int `json:"accepted"`
Stored bool `json:"stored"`
}
// record ingests a batch of samples and appends them to the warehouse series. It is
// FAIL-SOFT: a datastore outage costs a poll of history (stored:false), never a
// failed request. It records usage ONLY — the link registry is refreshed separately
// via POST /v1/links, so there is one and only one way to update an account row.
func record(s *cloud.Service[state], c *zip.Ctx) error {
org, user, ok := caller(c)
if !ok {
return zip.ErrUnauthorized("sign in to report usage")
}
var body reportReq
if err := c.Bind(&body); err != nil {
return err
}
raw := body.samplesOf()
if len(raw) == 0 {
return zip.ErrBadRequest("at least one sample is required")
}
if len(raw) > maxSamples {
return zip.ErrBadRequest(fmt.Sprintf("at most %d samples per report", maxSamples))
}
now := time.Now()
samples := make([]Sample, 0, len(raw))
for _, in := range raw {
x, err := parseSample(in, now)
if err != nil {
return err
}
samples = append(samples, x)
}
// History is fail-soft: a warehouse outage must never fail a report or block a
// device. `stored` tells the caller which happened — honestly.
stored := true
if err := s.State.warehouse.WriteSamples(c.Context(), org, user, samples, now); err != nil {
s.Log.Debug("account usage write skipped", "org", org, "err", err)
stored = false
}
return c.JSON(http.StatusAccepted, reportResp{Accepted: len(samples), Stored: stored})
}
// ── reads ────────────────────────────────────────────────────────────────────
type dashResp struct {
Provider string `json:"provider"`
Account string `json:"account,omitempty"`
Range string `json:"range"`
From string `json:"from"`
To string `json:"to"`
Source string `json:"source"` // account — the provider's own meter
Scope string `json:"scope"` // user
Available bool `json:"available"` // false = warehouse unavailable, NOT "no usage"
Current []sampleView `json:"current"` // the live state of each lane: the dash headline
Windows []sampleView `json:"windows"` // every instance in range, newest first
}
// samples is the PER-PROVIDER view: one connected account's own consumption of its
// own plan — "my Claude Max plan is 47% through its 6h window, resets at 14:20".
//
// `current` is the newest instance of each lane (the headline); `windows` is the
// history behind it. Both are computed from ONE deduped read — no second query.
func samples(s *cloud.Service[state], c *zip.Ctx) error {
org, user, ok := caller(c)
if !ok {
return zip.ErrUnauthorized("sign in to view usage")
}
provider := trim(c.Query("provider"))
if provider == "" {
return zip.ErrBadRequest("provider is required")
}
if len(provider) > maxProvider {
return zip.ErrBadRequest("provider too long")
}
acct := trim(c.Query("account"))
if len(acct) > maxAccount {
return zip.ErrBadRequest("account too long")
}
window := trim(c.Query("window"))
if window != "" && !validWindow(window) {
return zip.ErrBadRequest("window must be one of 6h, day, week, month")
}
rangeLabel := trim(c.Query("range"))
from, to, err := resolveRange(rangeLabel, time.Now())
if err != nil {
return zip.ErrBadRequest(err.Error())
}
if rangeLabel == "" {
rangeLabel = Range24h
}
out := dashResp{
Provider: provider, Account: acct, Range: rangeLabel,
From: rfc3339Of(from), To: rfc3339Of(to),
Source: SourceAccount, Scope: ScopeUser,
Current: []sampleView{}, Windows: []sampleView{},
}
rows, ok := s.State.warehouse.Series(c.Context(), org, user, provider, acct, window, from, to)
if !ok {
return c.JSON(http.StatusOK, out) // Available=false: honest "unavailable"
}
out.Available = true
for _, x := range rows {
out.Windows = append(out.Windows, toSampleView(x))
}
for _, x := range currentOf(rows) {
out.Current = append(out.Current, toSampleView(x))
}
return c.JSON(http.StatusOK, out)
}
// currentOf picks the newest instance of each lane — the live state. Rows arrive
// newest-first, so the first sighting of a lane is its current instance.
func currentOf(rows []Sample) []Sample {
seen := map[string]bool{}
out := make([]Sample, 0, 4)
for _, x := range rows {
k := x.Account + "\x00" + x.Lane
if seen[k] {
continue
}
seen[k] = true
out = append(out, x)
}
sort.SliceStable(out, func(i, j int) bool {
if out[i].Account != out[j].Account {
return out[i].Account < out[j].Account
}
return windowRank(out[i].Window) < windowRank(out[j].Window)
})
return out
}
// windowRank orders the window classes by breadth, so the dash renders the lanes
// narrowest-first (6h before week) rather than in read order.
func windowRank(w string) int {
switch w {
case Window6h:
return 1
case WindowDay:
return 2
case WindowWeek:
return 3
case WindowMonth:
return 4
}
return 0
}
+739
View File
@@ -0,0 +1,739 @@
package usage
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
fiber "github.com/zap-proto/fiber/v3"
"github.com/zap-proto/zip"
)
// account_test.go is the gate on the account-usage plane, moved here with the plane
// from clients/link. The bar it holds:
//
// ISOLATION org+subject lead EVERY query as BOUND args; a co-tenant is
// unreachable, a blank tenancy fails closed.
// IDEMPOTENCY a re-report of one window collapses to one row — asserted at the
// key (sample_test.go) AND at the engine/read contract here.
// HONESTY unavailable is not zero; a source is labelled; nothing is invented.
//
// LIVE COVERAGE, STATED PLAINLY: no datastore is reachable from this suite, so the
// DDL and the SQL are proven by CONTRACT (the exact statements, and the exact bound
// args, that the warehouse would receive) and not by execution. The dedup itself —
// that ClickHouse's ReplacingMergeTree collapses these keys and that argMax reads
// them back — is UNPROVEN LIVE and needs a warehouse smoke before the plane is
// trusted for anything that bills.
//
// The plane records usage ONLY (POST /v1/usage → the warehouse series); keeping the
// link REGISTRY current is clients/link's own concern (POST /v1/links), tested in
// that package — so the former "a report also upserts a Link" tests do not live here.
const usageTestTimeout = 60 * time.Second
// mountBare mounts the usage subsystem with NO commerce configured — the account
// plane never touches commerce, and the summary degrades to honest zeros.
func mountBare(t *testing.T) *zip.App { return mountApp(t, "", "") }
// drive runs one in-process request with a principal: X-Org-Id (the tenant) +
// X-User-Id (the validated subject the gateway sets ONLY from a verified credential).
// An empty user is the anonymous forge (org header, no validated principal) that
// every route must refuse.
func drive(t *testing.T, app *zip.App, method, path, org, user string, body any) (int, []byte) {
t.Helper()
var r io.Reader
if body != nil {
b, _ := json.Marshal(body)
r = bytes.NewReader(b)
}
rq := httptest.NewRequest(method, path, r)
if body != nil {
rq.Header.Set("Content-Type", "application/json")
}
if org != "" {
rq.Header.Set("X-Org-Id", org)
}
if user != "" {
rq.Header.Set("X-User-Id", user)
}
resp, err := app.Fiber().Test(rq, fiber.TestConfig{Timeout: usageTestTimeout, FailOnTimeout: true})
if err != nil {
t.Fatalf("Test %s %s: %v", method, path, err)
}
defer func() { _ = resp.Body.Close() }()
b, _ := io.ReadAll(resp.Body)
return resp.StatusCode, b
}
// ── isolation ────────────────────────────────────────────────────────────────
// argsLead asserts a query's first bound args are the tenancy, in order.
func argsLead(t *testing.T, what string, args []any, want ...any) {
t.Helper()
if len(args) < len(want) {
t.Fatalf("%s: only %d args, want tenancy to lead with %v", what, len(args), want)
}
for i, w := range want {
if args[i] != w {
t.Fatalf("%s: arg %d = %v, want %v (tenancy MUST lead, bound)", what, i, args[i], w)
}
}
}
// TestQueriesAreTenantScoped: every read leads with org AND subject as BOUND
// parameters. This is the #1 property of the plane — a caller reads only their own
// accounts, within their own org.
func TestQueriesAreTenantScoped(t *testing.T) {
from, to := now.Add(-24*time.Hour), now
q, args := seriesQuery("acme", "alice", "claude", "", "", from, to)
if !strings.Contains(q, "WHERE org = ? AND subject = ? AND provider = ?") {
t.Fatalf("series WHERE must lead org+subject:\n%s", q)
}
argsLead(t, "series", args, "acme", "alice", "claude")
q, args = summaryQuery("acme", "alice", from, to)
if !strings.Contains(q, "WHERE org = ? AND subject = ?") {
t.Fatalf("summary WHERE must lead org+subject:\n%s", q)
}
argsLead(t, "summary", args, "acme", "alice")
q, args = hanzoQuery("acme", from, to)
if !strings.Contains(q, "WHERE organization = ?") {
t.Fatalf("hanzo WHERE must lead organization:\n%s", q)
}
argsLead(t, "hanzo", args, "acme")
}
// TestNoTenantValueIsInterpolated: a hostile org/subject/provider/account can never
// reach the SQL text — it is bound, always. If any of these appear in the statement
// itself, the plane is injectable.
func TestNoTenantValueIsInterpolated(t *testing.T) {
evil := "acme' OR 1=1 --"
from, to := now.Add(-time.Hour), now
for _, tc := range []struct {
name string
q string
}{
{"series/org", first(seriesQuery(evil, "alice", "claude", "", "", from, to))},
{"series/subject", first(seriesQuery("acme", evil, "claude", "", "", from, to))},
{"series/provider", first(seriesQuery("acme", "alice", evil, "", "", from, to))},
{"series/account", first(seriesQuery("acme", "alice", "claude", evil, "", from, to))},
{"series/window", first(seriesQuery("acme", "alice", "claude", "", evil, from, to))},
{"summary/org", first(summaryQuery(evil, "alice", from, to))},
{"summary/subject", first(summaryQuery("acme", evil, from, to))},
{"hanzo/org", first(hanzoQuery(evil, from, to))},
} {
if strings.Contains(tc.q, evil) || strings.Contains(tc.q, "1=1") {
t.Fatalf("%s: a caller value reached the SQL TEXT — injectable:\n%s", tc.name, tc.q)
}
}
}
// TestReadsFailClosedOnBlankTenancy: a blank org or subject must never reach the
// warehouse at all. If it did, the WHERE would match rows with a blank tenant
// rather than none.
func TestReadsFailClosedOnBlankTenancy(t *testing.T) {
w := &warehouse{}
ctx := t.Context()
for _, tc := range []struct{ org, subject string }{
{"", "alice"}, {"acme", ""}, {"", ""},
} {
if _, ok := w.Series(ctx, tc.org, tc.subject, "claude", "", "", now.Add(-time.Hour), now); ok {
t.Fatalf("Series(%q,%q) must fail closed", tc.org, tc.subject)
}
if _, ok := w.AccountTotals(ctx, tc.org, tc.subject, now.Add(-time.Hour), now); ok {
t.Fatalf("AccountTotals(%q,%q) must fail closed", tc.org, tc.subject)
}
if err := w.WriteSamples(ctx, tc.org, tc.subject, []Sample{{Provider: "claude"}}, now); err == nil {
t.Fatalf("WriteSamples(%q,%q) must fail closed", tc.org, tc.subject)
}
}
if _, ok := w.HanzoTotals(ctx, "", now.Add(-time.Hour), now); ok {
t.Fatal("HanzoTotals(\"\") must fail closed")
}
}
// TestFailClosedNoPrincipalOnUsage: an org header with NO validated user (the
// off-gateway forge) is refused on every usage route — read AND write. The usage
// subsystem's contract is 401 (sign in), the same as summary/analytics.
func TestFailClosedNoPrincipalOnUsage(t *testing.T) {
app := mountBare(t)
for _, tc := range []struct {
method, path string
body any
}{
{http.MethodGet, "/v1/usage/samples?provider=claude", nil},
{http.MethodGet, "/v1/usage/summary", nil},
{http.MethodPost, "/v1/usage", map[string]any{
"provider": "claude", "machine": "m1", "window": "6h", "usedPct": 42}},
} {
if code, _ := drive(t, app, tc.method, tc.path, "acme", "", tc.body); code != http.StatusUnauthorized {
t.Fatalf("%s %s with no validated principal want 401, got %d", tc.method, tc.path, code)
}
}
}
// ── idempotency: the engine + read contract ──────────────────────────────────
// TestDedupKeyIsTheWindowInstanceNotTheClock is the schema half of the idempotency
// gate, and it pins the trap: the dedup key must be the WINDOW INSTANCE, and `ts`
// must be the VERSION, never part of the key. Keying on the observation clock would
// give every poll a distinct key, collapse nothing, and multiply every total by the
// poll rate.
func TestDedupKeyIsTheWindowInstanceNotTheClock(t *testing.T) {
ddl := accountUsageDDL[0]
if !strings.Contains(ddl, "ENGINE = ReplacingMergeTree(ts)") {
t.Fatalf("the base table must REPLACE re-reports, versioned by ts:\n%s", ddl)
}
const key = "ORDER BY (org, subject, provider, account, lane, window_start)"
if !strings.Contains(ddl, key) {
t.Fatalf("dedup key must be %s:\n%s", key, ddl)
}
// ts in the key would defeat the engine entirely.
if strings.Contains(ddl, "ORDER BY") && strings.Contains(orderByOf(ddl), "ts") {
t.Fatalf("ts must NOT be in the dedup key — every poll would key a new row:\n%s", orderByOf(ddl))
}
// machine in the key would store one account-wide window once per device.
if strings.Contains(orderByOf(ddl), "machine") {
t.Fatalf("machine must NOT be in the dedup key — a plan's quota is the ACCOUNT's, not the device's:\n%s", orderByOf(ddl))
}
// Partitioning by the observation clock would put a re-report in a different
// partition from the original, where ReplacingMergeTree can never collapse it.
if !strings.Contains(ddl, "PARTITION BY toYYYYMM(window_start)") {
t.Fatalf("must partition by window_start, or re-reports across a month boundary never collapse:\n%s", ddl)
}
}
// orderByOf extracts the ORDER BY clause for precise assertions.
func orderByOf(ddl string) string {
i := strings.Index(ddl, "ORDER BY")
if i < 0 {
return ""
}
rest := ddl[i:]
if j := strings.Index(rest, "\n"); j > 0 {
return rest[:j]
}
return rest
}
// TestReadsDedupExplicitly: ReplacingMergeTree collapses only when parts merge, on
// the server's own schedule — so a read issued before that merge SEES the
// duplicates. Correctness may never be delegated to the engine: the reader must
// dedup with argMax over the same key.
func TestReadsDedupExplicitly(t *testing.T) {
q, _ := seriesQuery("acme", "alice", "claude", "", "", now.Add(-time.Hour), now)
if !strings.Contains(q, "argMax(") {
t.Fatalf("the series read must dedup with argMax — the engine's merge is not load-bearing:\n%s", q)
}
const group = "GROUP BY org, subject, provider, account, lane, window_start"
if !strings.Contains(q, group) {
t.Fatalf("the read must group by the dedup key EXACTLY (%s):\n%s", group, q)
}
}
// TestSummarySumsDedupedRowsNotPolls is the sharpest read gate: the sum must be
// OUTSIDE and the dedup INSIDE. Summing the raw table would multiply every total by
// the poll rate — the same fact counted once per poll.
func TestSummarySumsDedupedRowsNotPolls(t *testing.T) {
q, _ := summaryQuery("acme", "alice", now.Add(-24*time.Hour), now)
inner := strings.Index(q, "argMax(")
outer := strings.Index(q, "sum(")
if inner < 0 || outer < 0 {
t.Fatalf("summary must dedup (argMax) and then sum:\n%s", q)
}
if outer > inner {
t.Fatalf("sum() must wrap the deduped subquery, not the raw table:\n%s", q)
}
if !strings.Contains(q, "GROUP BY provider, window") {
t.Fatalf("summary must group per (provider, window) — window classes NEST, so summing across them double-counts:\n%s", q)
}
}
// TestRollupIsDedupPreserving: a materialized view is an INSERT TRIGGER, not a view
// of the merged table — it sees every poll before ReplacingMergeTree collapses
// anything. So sumState() here would inflate a day's tokens by the poll rate and
// the base table could never undo it. The rollup must use dedup-preserving
// aggregates.
func TestRollupIsDedupPreserving(t *testing.T) {
mv := accountUsageDDL[2]
if strings.Contains(mv, "sumState(") {
t.Fatalf("sumState in the rollup sums every POLL — it must be argMaxState:\n%s", mv)
}
for _, want := range []string{
"argMaxState(total_tokens, ts)", "argMaxState(requests, ts)",
"argMaxState(cost_cents, ts)", "maxState(used_pct)",
} {
if !strings.Contains(mv, want) {
t.Fatalf("the rollup must carry %s:\n%s", want, mv)
}
}
if !strings.Contains(mv, "GROUP BY org, subject, provider, account, day, lane, `window`, window_start") {
t.Fatalf("the rollup must keep window-instance granularity, or its sums cannot be honest:\n%s", mv)
}
// The target's aggregate columns must match the MV's state functions.
if !strings.Contains(accountUsageDDL[1], "AggregateFunction(argMax, UInt64, DateTime64(3, 'UTC'))") ||
!strings.Contains(accountUsageDDL[1], "AggregateFunction(max, Float64)") {
t.Fatalf("the rollup target's types must match the MV's states:\n%s", accountUsageDDL[1])
}
}
// TestReservedWordIsQuoted: `window` is a ClickHouse keyword; every statement that
// names the column must quote it or the DDL and the reads fail at parse.
func TestReservedWordIsQuoted(t *testing.T) {
for i, stmt := range accountUsageDDL {
for _, bad := range []string{"\n window ", "(window,", " window,"} {
if strings.Contains(stmt, bad) {
t.Fatalf("ddl[%d] names `window` unquoted (%q) — it is a reserved word:\n%s", i, bad, stmt)
}
}
}
if !strings.Contains(accountUsageInsert, "`window`") {
t.Fatalf("the insert must quote `window`:\n%s", accountUsageInsert)
}
q, _ := seriesQuery("acme", "alice", "claude", "", "6h", now.Add(-time.Hour), now)
if !strings.Contains(q, "`window` = ?") {
t.Fatalf("the window filter must quote the column:\n%s", q)
}
}
// TestInsertBindsEveryColumn: the placeholder count must equal the column count, or
// values land in the wrong columns.
func TestInsertBindsEveryColumn(t *testing.T) {
cols := strings.Count(accountUsageInsert[:strings.Index(accountUsageInsert, "VALUES")], ",") + 1
vals := strings.Count(accountUsageInsert[strings.Index(accountUsageInsert, "VALUES"):], "?")
if cols != vals {
t.Fatalf("insert has %d columns but %d placeholders", cols, vals)
}
}
// ── fail-soft ────────────────────────────────────────────────────────────────
// TestReportSucceedsWithoutTheWarehouse is the availability gate: with NO datastore
// (this suite's real condition), a record must still be ACCEPTED — losing a poll of
// history may never fail a report or block a device — and it must SAY so honestly
// (stored:false), never claim history it does not have.
func TestReportSucceedsWithoutTheWarehouse(t *testing.T) {
app := mountBare(t)
code, b := drive(t, app, http.MethodPost, "/v1/usage", "acme", "alice", map[string]any{
"provider": "claude", "account": "alice@x", "machine": "m1", "plan": "Claude Max",
"kind": "subscription", "window": "6h", "lane": "five_hour", "windowMinutes": 300,
"usedPct": 47.5, "confidence": "percentOnly",
})
if code != http.StatusAccepted {
t.Fatalf("report want 202 with no warehouse, got %d (%s)", code, b)
}
var got reportResp
_ = json.Unmarshal(b, &got)
if got.Accepted != 1 {
t.Fatalf("accepted = %d, want 1", got.Accepted)
}
if got.Stored {
t.Fatal("stored must be false with no warehouse — the report must not claim history it does not have")
}
}
// TestDashIsUnavailableNotEmpty: with no warehouse the dash must say so. Reporting
// `available:false` is the difference between "we cannot tell you" and the lie "you
// used nothing".
func TestDashIsUnavailableNotEmpty(t *testing.T) {
app := mountBare(t)
code, b := drive(t, app, http.MethodGet, "/v1/usage/samples?provider=claude", "acme", "alice", nil)
if code != http.StatusOK {
t.Fatalf("dash want 200, got %d (%s)", code, b)
}
var got dashResp
_ = json.Unmarshal(b, &got)
if got.Available {
t.Fatal("available must be false with no warehouse")
}
if got.Source != SourceAccount || got.Scope != ScopeUser {
t.Fatalf("the dash must label its source+scope: %+v", got)
}
if got.Windows == nil || got.Current == nil {
t.Fatal("empty collections must serialize as [] , never null")
}
}
// ── the global view (now the summary's account board) ────────────────────────
// TestSummaryLabelsItsSources is the honesty gate on the union: the two ledgers mean
// different things, so every row must say which it came from and whose usage it
// covers — and the response must report each side's availability independently, so
// half a warehouse never turns the other half into zeros. It now rides the ONE
// /v1/usage/summary (accounts block), beside spend + LLM.
func TestSummaryLabelsItsSources(t *testing.T) {
app := mountBare(t)
code, b := drive(t, app, http.MethodGet, "/v1/usage/summary?range=7d", "acme", "alice", nil)
if code != http.StatusOK {
t.Fatalf("summary want 200, got %d (%s)", code, b)
}
got := decodeSummary(t, b)
if got.Range != "7d" {
t.Fatalf("range = %q, want 7d", got.Range)
}
if got.Accounts.Account.Scope != ScopeUser || got.Accounts.Account.Source != accountUsageTable {
t.Fatalf("the account side must be labelled user-scoped: %+v", got.Accounts.Account)
}
if got.Accounts.Hanzo.Scope != ScopeOrg || got.Accounts.Hanzo.Source != llmTable {
t.Fatalf("the hanzo side must be labelled org-scoped: %+v", got.Accounts.Hanzo)
}
if got.Accounts.Account.Available || got.Accounts.Hanzo.Available {
t.Fatal("both sides must report unavailable with no warehouse")
}
if got.Accounts.Rows == nil {
t.Fatal("rows must serialize as [], never null")
}
}
// TestSourcesAreStampedServerSide: the source and scope of a row are facts about
// WHICH ledger answered — never data from it. A row from the account meter can never
// present itself as Hanzo's cost of record, even if the warehouse returns a column
// saying so.
func TestSourcesAreStampedServerSide(t *testing.T) {
hostile := map[string]any{
"provider": "claude", "source": SourceHanzo, "scope": ScopeOrg,
"total_tokens": uint64(10), "cost_cents": uint64(999), "confidence_rank": uint8(0),
}
acct := accountTotalOf(hostile)
if acct.Source != SourceAccount || acct.Scope != ScopeUser {
t.Fatalf("an account row must be stamped account/user, got %s/%s", acct.Source, acct.Scope)
}
hz := hanzoTotalOf(map[string]any{"provider": "anthropic", "requests": uint64(5)})
if hz.Source != SourceHanzo || hz.Scope != ScopeOrg {
t.Fatalf("a hanzo row must be stamped hanzo/org, got %s/%s", hz.Source, hz.Scope)
}
if hz.Confidence != ConfidenceExact {
t.Fatalf("cloud_usage counts calls we billed — it is exact, got %q", hz.Confidence)
}
// A Hanzo row carries NO window: cloud_usage is a per-call ledger, not a window
// meter. Inventing a class would invent a quota.
if hz.Window != "" {
t.Fatalf("a hanzo row must not claim a window class, got %q", hz.Window)
}
}
// TestBothSidesShareOneRange: the union compares one period. Two resolvers could
// drift and silently compare a day of plan usage against a week of spend.
func TestBothSidesShareOneRange(t *testing.T) {
for _, label := range []string{"1h", "24h", "7d", "30d", ""} {
from, to, err := resolveRange(label, now)
if err != nil {
t.Fatalf("range %q: %v", label, err)
}
_, aArgs := summaryQuery("acme", "alice", from, to)
_, hArgs := hanzoQuery("acme", from, to)
// The account side binds instants; the Hanzo side binds the datastore's
// literal form of the SAME instants.
if aArgs[2] != from.UTC() || aArgs[3] != to.UTC() {
t.Fatalf("account side lost the range: %v", aArgs)
}
if hArgs[1] != tsLiteral(from) || hArgs[2] != tsLiteral(to) {
t.Fatalf("hanzo side lost the range: %v", hArgs)
}
}
}
// TestUnknownRangeIsRefused: an unknown range must be an error, never a silent
// default — a caller who asked for a window we do not have must be told, not shown
// a different one and left to believe it.
func TestUnknownRangeIsRefused(t *testing.T) {
for _, bad := range []string{"90d", "1y", "all", "custom", "1h; DROP TABLE", "24H"} {
if _, _, err := resolveRange(bad, now); err == nil {
t.Fatalf("range %q must be refused", bad)
}
}
app := mountBare(t)
if code, _ := drive(t, app, http.MethodGet, "/v1/usage/samples?provider=claude&range=90d", "acme", "alice", nil); code != http.StatusBadRequest {
t.Fatalf("an unknown range want 400, got %d", code)
}
}
// ── the boundary ─────────────────────────────────────────────────────────────
// TestReportRejectsUnknownVocabulary: the closed vocabularies are 400s, never
// silently rewritten — a caller whose window we did not understand must be told, or
// their dash quietly fills with a class they never reported.
func TestReportRejectsUnknownVocabulary(t *testing.T) {
app := mountBare(t)
for _, tc := range []struct {
name string
body map[string]any
}{
{"no provider", map[string]any{"machine": "m1", "window": "6h"}},
{"no machine", map[string]any{"provider": "claude", "window": "6h"}},
{"no window", map[string]any{"provider": "claude", "machine": "m1"}},
{"legacy window", map[string]any{"provider": "claude", "machine": "m1", "window": "weekly"}},
{"cased window", map[string]any{"provider": "claude", "machine": "m1", "window": "Week"}},
{"5h window", map[string]any{"provider": "claude", "machine": "m1", "window": "5h"}},
{"bad kind", map[string]any{"provider": "claude", "machine": "m1", "window": "6h", "kind": "root"}},
{"bad windowStart", map[string]any{"provider": "claude", "machine": "m1", "window": "6h", "windowStart": "yesterday"}},
{"bad resetsAt", map[string]any{"provider": "claude", "machine": "m1", "window": "6h", "resetsAt": "soon"}},
} {
if code, b := drive(t, app, http.MethodPost, "/v1/usage", "acme", "alice", tc.body); code != http.StatusBadRequest {
t.Fatalf("%s: want 400, got %d (%s)", tc.name, code, b)
}
}
if code, _ := drive(t, app, http.MethodGet, "/v1/usage/samples?provider=claude&window=weekly", "acme", "alice", nil); code != http.StatusBadRequest {
t.Fatalf("dash with a non-canonical window want 400, got %d", code)
}
if code, _ := drive(t, app, http.MethodGet, "/v1/usage/samples", "acme", "alice", nil); code != http.StatusBadRequest {
t.Fatalf("dash with no provider want 400, got %d", code)
}
}
// TestBatchIsBounded: one report is capped, so a hostile client cannot bloat the
// warehouse in a single call.
func TestBatchIsBounded(t *testing.T) {
app := mountBare(t)
batch := make([]map[string]any, 0, maxSamples+1)
for i := 0; i <= maxSamples; i++ {
batch = append(batch, map[string]any{
"provider": "claude", "machine": "m1", "window": "6h",
"lane": fmt.Sprintf("lane-%d", i), "usedPct": 1,
})
}
if code, _ := drive(t, app, http.MethodPost, "/v1/usage", "acme", "alice",
map[string]any{"samples": batch}); code != http.StatusBadRequest {
t.Fatalf("an over-cap batch want 400, got %d", code)
}
if code, _ := drive(t, app, http.MethodPost, "/v1/usage", "acme", "alice",
map[string]any{"samples": []map[string]any{}}); code != http.StatusBadRequest {
t.Fatalf("an empty batch want 400, got %d", code)
}
}
// TestReportAcceptsOneOrMany: a poller reports its lanes in ONE call; a simple
// client posts one sample. Both are the same route, and every lane is accepted.
func TestReportAcceptsOneOrMany(t *testing.T) {
app := mountBare(t)
// Claude's real shape: four lanes, three of them the SAME 10080 minutes — they
// must survive as three distinct meters.
code, b := drive(t, app, http.MethodPost, "/v1/usage", "acme", "alice", map[string]any{
"samples": []map[string]any{
{"provider": "claude", "account": "a@x", "machine": "m1", "window": "6h",
"lane": "five_hour", "windowMinutes": 300, "usedPct": 47, "confidence": "percentOnly"},
{"provider": "claude", "account": "a@x", "machine": "m1", "window": "week",
"lane": "seven_day", "windowMinutes": 10080, "usedPct": 12, "confidence": "percentOnly"},
{"provider": "claude", "account": "a@x", "machine": "m1", "window": "week",
"lane": "seven_day_sonnet", "windowMinutes": 10080, "usedPct": 30, "confidence": "percentOnly"},
{"provider": "claude", "account": "a@x", "machine": "m1", "window": "week",
"lane": "seven_day_opus", "windowMinutes": 10080, "usedPct": 80, "confidence": "percentOnly"},
},
})
if code != http.StatusAccepted {
t.Fatalf("batch want 202, got %d (%s)", code, b)
}
var got reportResp
_ = json.Unmarshal(b, &got)
if got.Accepted != 4 {
t.Fatalf("accepted = %d, want 4", got.Accepted)
}
}
// TestLanesSharingAWindowKeepDistinctKeys is the collision gate. Claude's
// seven_day, seven_day_sonnet and seven_day_opus are three DIFFERENT meters at the
// SAME 10080 minutes. Keyed by window class they would collapse onto each other and
// two of the three would be silently overwritten; only `lane` tells them apart.
func TestLanesSharingAWindowKeepDistinctKeys(t *testing.T) {
resets := now.Add(3 * 24 * time.Hour)
keys := map[string]bool{}
for _, lane := range []string{"seven_day", "seven_day_sonnet", "seven_day_opus"} {
s := Sample{Provider: "claude", Account: "a@x", Machine: "m1", Window: WindowWeek,
Lane: lane, WindowMinutes: 10080, ResetsAt: resets}.Sanitize(now)
// The dedup key is (…, lane, window_start): same instant, distinct lanes.
k := s.Lane + "|" + s.WindowStart.String()
if keys[k] {
t.Fatalf("lane %q collided — two of Claude's three weekly meters would be lost", lane)
}
keys[k] = true
}
if len(keys) != 3 {
t.Fatalf("want 3 distinct keys, got %d", len(keys))
}
}
// ── projections ──────────────────────────────────────────────────────────────
// TestCurrentIsTheNewestInstancePerLane: the dash headline is each lane's live
// state — the newest instance, one per lane.
func TestCurrentIsTheNewestInstancePerLane(t *testing.T) {
// Rows arrive newest-first, as the read orders them.
rows := []Sample{
{Account: "a@x", Lane: "five_hour", Window: Window6h, WindowStart: now, UsedPct: 47},
{Account: "a@x", Lane: "seven_day", Window: WindowWeek, WindowStart: now, UsedPct: 12},
{Account: "a@x", Lane: "five_hour", Window: Window6h, WindowStart: now.Add(-6 * time.Hour), UsedPct: 99},
}
got := currentOf(rows)
if len(got) != 2 {
t.Fatalf("current must be one row per lane, got %d", len(got))
}
if got[0].Lane != "five_hour" || got[0].UsedPct != 47 {
t.Fatalf("current must be the NEWEST instance (47%%), got %+v", got[0])
}
// Ordered by window breadth, so a dash renders 6h then week.
if windowRank(got[0].Window) > windowRank(got[1].Window) {
t.Fatalf("current must order by window breadth: %+v", got)
}
}
// TestPercentOnlyKeepsItsZeros: Claude reports percentOnly and NO tokens. Those
// zeros mean UNKNOWN, and the flag is what says so — the view must carry it, and
// must not invent counters.
func TestPercentOnlyKeepsItsZeros(t *testing.T) {
v := toSampleView(Sample{
Provider: "claude", Lane: "five_hour", Window: Window6h, WindowMinutes: 300,
UsedPct: 47.5, Confidence: ConfidencePercentOnly,
})
if v.Confidence != ConfidencePercentOnly {
t.Fatalf("the flag must survive to the wire, got %q", v.Confidence)
}
b, _ := json.Marshal(v)
// omitempty drops the unknown counters entirely rather than asserting 0.
for _, absent := range []string{"totalTokens", "requests", "costCents"} {
if strings.Contains(string(b), absent) {
t.Fatalf("%s must be OMITTED when unknown, not sent as 0: %s", absent, b)
}
}
if !strings.Contains(string(b), `"usedPct":47.5`) {
t.Fatalf("the one real number must be present: %s", b)
}
// windowMinutes carries the TRUTH (300 = 5h), never the class's nominal 360.
if v.WindowMinutes != 300 {
t.Fatalf("windowMinutes must be the meter's own 300, got %d", v.WindowMinutes)
}
}
// ── the absent instant ───────────────────────────────────────────────────────
// TestAbsentInstantIsStorable is the gate on a bug that would have corrupted almost
// every row: Go's zero time is YEAR 1, which a DateTime64 column cannot represent —
// and an absent resets_at is the COMMON case, since most meters report no boundary.
// Binding the zero time would break or silently corrupt the insert. Absence must be
// the epoch sentinel on the way in, and absent again on the way out — never a date.
func TestAbsentInstantIsStorable(t *testing.T) {
if got := dsInstant(time.Time{}); !got.Equal(epoch) {
t.Fatalf("an absent instant must store as the epoch, got %s (year %d)", got, got.Year())
}
if y := dsInstant(time.Time{}).Year(); y < 1900 {
t.Fatalf("year %d is outside DateTime64 — the insert would break", y)
}
real := now.Add(time.Hour)
if got := dsInstant(real); !got.Equal(real.UTC()) {
t.Fatalf("a real instant must pass through, got %s", got)
}
// And it round-trips back to ABSENT, so the wire omits it rather than claiming
// a window reset in 1970.
if got := dsTimeOf(epoch); !got.IsZero() {
t.Fatalf("the epoch must read back as absent, got %s", got)
}
if got := dsTimeOf(real); !got.Equal(real.UTC()) {
t.Fatalf("a real instant must read back, got %s", got)
}
if v := toSampleView(Sample{Lane: "l", Window: Window6h}); v.ResetsAt != "" || v.WindowStart != "" {
t.Fatalf("an absent instant must be omitted on the wire, got %+v", v)
}
}
// TestEveryValidWindowKeysARepresentableInstant: a sample whose meter reported no
// duration and no reset must STILL key a storable instant — via the class's nominal
// length — or it would land on the zero instant and break the insert.
func TestEveryValidWindowKeysARepresentableInstant(t *testing.T) {
for _, w := range []string{Window6h, WindowDay, WindowWeek, WindowMonth} {
got := Sample{Provider: "hanzo", Machine: "m1", Window: w}.Sanitize(now)
if got.WindowStart.IsZero() {
t.Fatalf("window %q keyed the zero instant — the insert would break", w)
}
if got.WindowStart.Year() < 1900 {
t.Fatalf("window %q keyed year %d — outside DateTime64", w, got.WindowStart.Year())
}
// The nominal is a derivation aid ONLY: it must never be reported as the
// meter's own measurement.
if got.WindowMinutes != 0 {
t.Fatalf("window %q invented windowMinutes=%d — the meter never said that",
w, got.WindowMinutes)
}
}
// A meter that DOES report its duration keeps the truth: Claude's 6h-class lane
// really is 300 minutes, and the nominal 360 must never overwrite it.
got := Sample{Provider: "claude", Machine: "m1", Window: Window6h, WindowMinutes: 300}.Sanitize(now)
if got.WindowMinutes != 300 {
t.Fatalf("the meter's 300 was overwritten with the class nominal: %d", got.WindowMinutes)
}
}
// ── confidence ───────────────────────────────────────────────────────────────
// TestWeakestConfidenceWins: an aggregate over lanes may only be as trustworthy as
// its WEAKEST part. A sum whose parts were partly unknown is not exact — it is
// missing whatever those lanes never reported.
func TestWeakestConfidenceWins(t *testing.T) {
// Ranked by belief: exact is best, unknown is worst.
if !(confidenceRank(ConfidenceExact) < confidenceRank(ConfidenceEstimated) &&
confidenceRank(ConfidenceEstimated) < confidenceRank(ConfidencePercentOnly) &&
confidenceRank(ConfidencePercentOnly) < confidenceRank(ConfidenceUnknown)) {
t.Fatal("confidence must rank by how much a reader may believe it")
}
// An unrecognised value is the WORST, never accidentally trusted.
if confidenceRank("totally-legit") != confidenceRank(ConfidenceUnknown) {
t.Fatal("an unrecognised confidence must rank as unknown")
}
for _, c := range []string{ConfidenceExact, ConfidenceEstimated, ConfidencePercentOnly, ConfidenceUnknown} {
if got := confidenceOfRank(int64(confidenceRank(c))); got != c {
t.Fatalf("rank round-trip: %q → %q", c, got)
}
}
if got := confidenceOfRank(99); got != ConfidenceUnknown {
t.Fatalf("an out-of-range rank must decode as unknown, got %q", got)
}
}
// TestConfidenceRankExprIsBuiltFromConstants: the warehouse's ranking is GENERATED
// from the Go one, so the two can never drift — a drift would still run and just
// rank wrongly, invisibly. It also pins that only this package's own constants are
// interpolated: no caller value reaches the expression.
func TestConfidenceRankExprIsBuiltFromConstants(t *testing.T) {
got := confidenceRankExpr
for _, c := range []string{ConfidenceExact, ConfidenceEstimated, ConfidencePercentOnly} {
want := fmt.Sprintf("confidence = '%s', %d", c, confidenceRank(c))
if !strings.Contains(got, want) {
t.Fatalf("the expression must rank %q as %d:\n%s", c, confidenceRank(c), got)
}
}
if !strings.HasPrefix(got, "multiIf(") || !strings.HasSuffix(got, fmt.Sprintf("%d)", confidenceRank(ConfidenceUnknown))) {
t.Fatalf("malformed rank expression: %s", got)
}
// The summary must aggregate on the RANK, never on the string — alphabetical
// order over confidence values is a coincidence, not a semantic.
q, _ := summaryQuery("acme", "alice", now.Add(-time.Hour), now)
if !strings.Contains(q, "max("+confidenceRankExpr+")") {
t.Fatalf("summary must take max() of the RANK:\n%s", q)
}
if strings.Contains(q, "max(confidence)") {
t.Fatalf("summary must not order confidence STRINGS alphabetically:\n%s", q)
}
}
// TestAccountTotalDecodesTheWeakestConfidence closes the loop from the query's rank
// back to a labelled row.
func TestAccountTotalDecodesTheWeakestConfidence(t *testing.T) {
for rank, want := range map[uint8]string{
0: ConfidenceExact, 1: ConfidenceEstimated,
2: ConfidencePercentOnly, 3: ConfidenceUnknown,
} {
got := accountTotalOf(map[string]any{"provider": "claude", "confidence_rank": rank})
if got.Confidence != want {
t.Fatalf("rank %d → %q, want %q", rank, got.Confidence, want)
}
}
}
// first returns a query builder's SQL, dropping its args.
func first(q string, _ []any) string { return q }
+611
View File
@@ -0,0 +1,611 @@
package usage
import (
"context"
"fmt"
"strings"
"sync"
"time"
aiobject "github.com/hanzoai/ai/object"
)
// datastore.go is the WAREHOUSE projection of account usage: the hanzo.account_usage
// time series, its daily rollup, and the org-scoped reads over them. It moved here
// from clients/link with the rest of the usage plane so the ONE usage subsystem owns
// ALL usage.
//
// WHY A SECOND SOURCE. hanzo.cloud_usage already records every HANZO-ROUTED
// inference (our gateway's own ledger, cost of record — the same llmTable the
// summary and analytics faces read). This table records something different in
// kind: what a user's OWN provider account has consumed of its OWN plan, metered by
// @hanzo/usage reading that provider's own login. Nobody else can know that a Claude
// Max plan is 47% through its 6h window — no Hanzo request produced it. The two are
// never conflated: they are separate tables, read separately, and labelled by source
// wherever they appear together.
//
// FAIL-SOFT IS THE CONTRACT. Every function here degrades to a no-op (writes) or an
// honest "unavailable" (reads) when the datastore is absent — never an error that
// reaches a caller, never a fabricated zero that reads as truth.
//
// TENANCY. Every statement leads with `org = ? AND subject = ?` as BOUND
// parameters. No value a client controls is ever concatenated into SQL.
// warehouse is the account-usage datastore projection. It holds ONLY the
// idempotent-DDL latch; the connection itself is aiobject's (a package global), so
// the warehouse owns no closable handle and the usage subsystem needs no Shutdown.
// dsReady latches the DDL on success only, so a datastore still connecting at boot
// is retried on the next call rather than permanently written off.
type warehouse struct {
dsMu sync.Mutex
dsReady bool
}
const (
accountUsageTable = "hanzo.account_usage"
accountUsageDaily = "hanzo.account_usage_daily"
accountUsageMV = "hanzo.account_usage_daily_mv"
)
// accountUsageDDL is the ONE definition of the account-usage schema: the base
// series, the daily rollup target, and the materialized view that fills it.
//
// ENGINE — ReplacingMergeTree(ts), NOT MergeTree. A poller re-reports the SAME
// window as it fills (see windowInstance); those reports are the same fact
// observed twice, so the table must COLLAPSE them, not stack them. `ts` (the
// server's observation clock) is the version: the newest read of an instance wins.
//
// ORDER BY IS THE DEDUP KEY, and every part of it earns its place:
//
// org, subject the tenant and the owner — isolation is a prefix scan, and a
// co-tenant's report can never replace this user's row
// provider,
// account the account being metered
// lane the meter's identity. Claude's seven_day, seven_day_sonnet and
// seven_day_opus are three different meters at the same 10080
// minutes; without `lane` two of the three silently overwrite
// the first
// window_start WHICH instance of the window — so a re-report replaces, and a
// NEW window appends
//
// `ts` is deliberately NOT in the key: it is the version. Keying on it would give
// every poll a distinct key, collapse nothing, and multiply every sum by the poll
// rate — the exact bug this engine is chosen to prevent.
//
// `machine` is deliberately NOT in the key either: a plan's quota belongs to the
// ACCOUNT, not the device. A laptop and a desktop watching the same Claude account
// observe the SAME window; keying by machine would store that one fact twice and
// double every total. The column stays as an attribute — who observed it last.
//
// PARTITION BY window_start, not ts — because ReplacingMergeTree only ever
// collapses rows WITHIN a partition. Partitioning by the observation clock would
// put a re-report of last month's window in a different partition from the
// original, where it could never collapse: the duplicate would be permanent.
// Partitioning by the key's own instant keeps every report of an instance together
// for the life of the row.
var accountUsageDDL = []string{
`CREATE TABLE IF NOT EXISTS ` + accountUsageTable + ` (
org LowCardinality(String),
subject String,
provider LowCardinality(String),
account String,
plan LowCardinality(String),
kind LowCardinality(String),
lane LowCardinality(String),
` + "`window`" + ` LowCardinality(String),
window_minutes UInt32,
window_start DateTime64(3, 'UTC'),
resets_at DateTime64(3, 'UTC'),
used_pct Float64,
requests UInt64,
input_tokens UInt64,
output_tokens UInt64,
total_tokens UInt64,
cached_input_tokens UInt64,
cost_cents UInt64,
cost_limit_cents UInt64,
currency LowCardinality(String),
confidence LowCardinality(String),
synthetic UInt8,
machine String,
ts DateTime64(3, 'UTC')
) ENGINE = ReplacingMergeTree(ts)
PARTITION BY toYYYYMM(window_start)
ORDER BY (org, subject, provider, account, lane, window_start)
TTL toDateTime(window_start) + INTERVAL 1 YEAR`,
// The daily rollup target. It keeps WINDOW-INSTANCE granularity on purpose —
// see the MV below for why a coarser key would be wrong — which is still a
// large rollup: a 30s poller emits ~2880 rows/day per lane and this holds ~4
// (one per 6h instance).
`CREATE TABLE IF NOT EXISTS ` + accountUsageDaily + ` (
org LowCardinality(String),
subject String,
provider LowCardinality(String),
account String,
day Date,
lane LowCardinality(String),
` + "`window`" + ` LowCardinality(String),
window_start DateTime64(3, 'UTC'),
requests AggregateFunction(argMax, UInt64, DateTime64(3, 'UTC')),
input_tokens AggregateFunction(argMax, UInt64, DateTime64(3, 'UTC')),
output_tokens AggregateFunction(argMax, UInt64, DateTime64(3, 'UTC')),
total_tokens AggregateFunction(argMax, UInt64, DateTime64(3, 'UTC')),
cached_input_tokens AggregateFunction(argMax, UInt64, DateTime64(3, 'UTC')),
cost_cents AggregateFunction(argMax, UInt64, DateTime64(3, 'UTC')),
used_pct AggregateFunction(max, Float64)
) ENGINE = AggregatingMergeTree
PARTITION BY toYYYYMM(day)
ORDER BY (org, subject, provider, account, day, lane, window_start)
TTL day + INTERVAL 1 YEAR`,
// The rollup, and the ONE subtlety that decides whether it tells the truth:
//
// A materialized view is an INSERT TRIGGER, not a view of the merged table. It
// sees every raw row as it arrives — including the polls ReplacingMergeTree
// will later collapse. So `sumState(total_tokens)` here would sum EVERY POLL of
// the same window and inflate a day's tokens by the poll rate; the base table's
// dedup could never undo it, because this is a different table. The aggregate
// must therefore be dedup-PRESERVING:
//
// argMaxState(x, ts) keeps the newest read of each window instance — the
// same rule the base table's engine applies, so
// re-reporting is a no-op here too
// maxState(used_pct) is idempotent by nature (re-reading 47% keeps 47), and
// over a day of resetting windows it means "peak
// utilisation that day" — an honest daily metric
//
// Summing then happens at READ time, over instances that are disjoint in time.
`CREATE MATERIALIZED VIEW IF NOT EXISTS ` + accountUsageMV + ` TO ` + accountUsageDaily + ` AS
SELECT
org,
subject,
provider,
account,
toDate(window_start) AS day,
lane,
` + "`window`" + `,
window_start,
argMaxState(requests, ts) AS requests,
argMaxState(input_tokens, ts) AS input_tokens,
argMaxState(output_tokens, ts) AS output_tokens,
argMaxState(total_tokens, ts) AS total_tokens,
argMaxState(cached_input_tokens, ts) AS cached_input_tokens,
argMaxState(cost_cents, ts) AS cost_cents,
maxState(used_pct) AS used_pct
FROM ` + accountUsageTable + `
GROUP BY org, subject, provider, account, day, lane, ` + "`window`" + `, window_start`,
}
// ensureAccountUsage creates the account-usage objects idempotently. It latches
// only on success, so a datastore that is still connecting at boot is retried on
// the next call rather than permanently poisoned.
func (w *warehouse) ensureAccountUsage(ctx context.Context) error {
if !aiobject.DatastoreEnabled() {
return fmt.Errorf("account usage: datastore not connected")
}
w.dsMu.Lock()
defer w.dsMu.Unlock()
if w.dsReady {
return nil
}
for _, stmt := range accountUsageDDL {
if err := aiobject.DatastoreExec(ctx, stmt); err != nil {
return fmt.Errorf("account usage ddl: %w", err)
}
}
w.dsReady = true
return nil
}
// accountUsageInsert is the ONE insert statement; its column order is the argument
// order WriteSamples binds.
const accountUsageInsert = `INSERT INTO ` + accountUsageTable + ` (
org, subject, provider, account, plan, kind, lane, ` + "`window`" + `, window_minutes,
window_start, resets_at, used_pct, requests, input_tokens, output_tokens,
total_tokens, cached_input_tokens, cost_cents, cost_limit_cents, currency,
confidence, synthetic, machine, ts
) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)`
// WriteSamples appends the caller's sanitized samples to the warehouse, stamping
// each with the server's observation clock. Org and subject are the SERVER's values
// (from the validated principal), bound positionally — a sample cannot carry them,
// so it cannot forge them.
//
// It is FAIL-SOFT by contract: an absent or blocked datastore is not an error the
// caller must surface to a device — losing a poll of history must never fail a
// report. The record handler turns this error into an honest `stored:false`.
func (w *warehouse) WriteSamples(ctx context.Context, org, subject string, samples []Sample, now time.Time) error {
if org == "" || subject == "" {
return fmt.Errorf("account usage: blank tenancy")
}
if len(samples) == 0 {
return nil
}
if err := w.ensureAccountUsage(ctx); err != nil {
return err
}
ts := now.UTC()
for _, x := range samples {
if err := aiobject.DatastoreExec(ctx, accountUsageInsert,
org, subject, x.Provider, x.Account, x.Plan, x.Kind, x.Lane, x.Window,
uint32(x.WindowMinutes), dsInstant(x.WindowStart), dsInstant(x.ResetsAt), x.UsedPct,
uint64(x.Requests), uint64(x.InputTokens), uint64(x.OutputTokens),
uint64(x.TotalTokens), uint64(x.CachedInputTokens), uint64(x.CostCents),
uint64(x.CostLimitCents), x.Currency, x.Confidence, boolBit(x.Synthetic),
x.Machine, ts); err != nil {
return fmt.Errorf("account usage insert: %w", err)
}
}
return nil
}
func boolBit(b bool) uint8 {
if b {
return 1
}
return 0
}
// ── reads ────────────────────────────────────────────────────────────────────
// dedupedSelect is the inner query EVERY read is built on, and the reason the reads
// are correct.
//
// ReplacingMergeTree collapses duplicates only when parts merge — which happens
// eventually, on the server's own schedule. A read issued before that merge sees
// every poll. So correctness cannot be delegated to the engine: the reader dedups
// EXPLICITLY, with argMax(col, ts) grouped by the same key the engine uses. The
// engine's collapse then only ever saves space; it is never load-bearing for a
// number a user reads.
//
// The GROUP BY is the dedup key EXACTLY — never a superset. Grouping by an
// attribute that a poll can change (a plan rename, a re-classified window) would
// split one instance into two groups and double-count it.
const dedupedSelect = `SELECT
provider,
account,
lane,
window_start,
argMax(` + "`window`" + `, ts) AS window,
argMax(window_minutes, ts) AS window_minutes,
argMax(plan, ts) AS plan,
argMax(kind, ts) AS kind,
argMax(resets_at, ts) AS resets_at,
argMax(used_pct, ts) AS used_pct,
argMax(requests, ts) AS requests,
argMax(input_tokens, ts) AS input_tokens,
argMax(output_tokens, ts) AS output_tokens,
argMax(total_tokens, ts) AS total_tokens,
argMax(cached_input_tokens, ts) AS cached_input_tokens,
argMax(cost_cents, ts) AS cost_cents,
argMax(cost_limit_cents, ts) AS cost_limit_cents,
argMax(currency, ts) AS currency,
argMax(confidence, ts) AS confidence,
argMax(synthetic, ts) AS synthetic,
argMax(machine, ts) AS machine,
max(ts) AS ts
FROM ` + accountUsageTable + `
WHERE `
const dedupedGroup = `
GROUP BY org, subject, provider, account, lane, window_start`
// maxReadRows caps any one read. A user's lanes over 30 days are a few hundred
// rows; the cap is the backstop that keeps a pathological account from returning
// an unbounded result.
const maxReadRows = 5000
// confidenceRankExpr is the warehouse's spelling of confidenceRank — GENERATED
// from it, so the severity order lives in exactly one place. Hand-writing the
// multiIf would be a second copy of the vocabulary, free to drift from the Go one
// the moment either changed, and the drift would be invisible: the query would
// still run and just rank wrongly.
//
// The interpolated values are this package's own constants, never caller input
// (TestConfidenceRankExprIsBuiltFromConstants holds that).
var confidenceRankExpr = buildConfidenceRankExpr()
func buildConfidenceRankExpr() string {
var b strings.Builder
b.WriteString("multiIf(")
for _, c := range []string{ConfidenceExact, ConfidenceEstimated, ConfidencePercentOnly} {
fmt.Fprintf(&b, "confidence = '%s', %d, ", c, confidenceRank(c))
}
fmt.Fprintf(&b, "%d)", confidenceRank(ConfidenceUnknown))
return b.String()
}
// seriesQuery builds the per-provider dash read: every window instance of one
// account (or of every account on one provider) in [from, to), deduped, newest
// first.
//
// It is a PURE builder returning (sql, args) so the isolation contract is unit
// testable without a warehouse: org and subject ALWAYS lead as bound args, and no
// caller value is ever concatenated. `window` is allowlist-checked by the caller
// before it reaches here.
func seriesQuery(org, subject, provider, account, window string, from, to time.Time) (string, []any) {
where := []string{"org = ?", "subject = ?", "provider = ?", "window_start >= ?", "window_start < ?"}
args := []any{org, subject, provider, from.UTC(), to.UTC()}
if account != "" {
where = append(where, "account = ?")
args = append(args, account)
}
if window != "" {
where = append(where, "`window` = ?")
args = append(args, window)
}
q := dedupedSelect + strings.Join(where, " AND ") + dedupedGroup +
"\nORDER BY window_start DESC, lane ASC\nLIMIT ?"
return q, append(args, maxReadRows)
}
// summaryQuery builds the global view's ACCOUNT side: the caller's own linked
// accounts, totalled per (provider, window).
//
// It sums the DEDUPED instances — the inner query collapses each window's polls to
// one row, and only then does the outer sum add them up. Summing the raw table
// would multiply every total by the poll rate.
//
// It groups by (provider, window) and NEVER across windows, because window classes
// NEST: a 6h lane's consumption is also inside its week lane's. Adding them would
// count the same tokens twice. Instances WITHIN one class are disjoint in time, so
// summing those is sound.
func summaryQuery(org, subject string, from, to time.Time) (string, []any) {
inner := dedupedSelect +
strings.Join([]string{"org = ?", "subject = ?", "window_start >= ?", "window_start < ?"}, " AND ") +
dedupedGroup
// max() over the confidence RANK takes the WEAKEST contributing lane — the only
// honest aggregate. A sum whose parts were partly unknown cannot be called
// exact: it is missing whatever those lanes did not report. (Ranking, not
// max() over the strings: see confidenceRank.)
q := `SELECT
provider,
window,
sum(requests) AS requests,
sum(total_tokens) AS total_tokens,
sum(cost_cents) AS cost_cents,
max(used_pct) AS used_pct,
max(` + confidenceRankExpr + `) AS confidence_rank,
count() AS windows
FROM (` + inner + `)
GROUP BY provider, window
ORDER BY total_tokens DESC, provider ASC
LIMIT ?`
return q, []any{org, subject, from.UTC(), to.UTC(), maxReadRows}
}
// hanzoQuery builds the global view's HANZO side: the org's Hanzo-routed usage per
// provider over the same window, from hanzo.cloud_usage — the same ledger (llmTable)
// and the same `provider` grouping the summary/analytics faces read, so the two can
// never disagree about what Hanzo charged.
//
// It is ORG-scoped, not user-scoped, and every row it produces says so (ScopeOrg).
// cloud_usage identifies a user as the qualified `<org>/<name>` form, which is not
// the bare subject this plane keys on; narrowing on an unproven identity match
// would silently report "no Hanzo usage" — a fabricated zero — rather than the
// truth. The scope label is the honest alternative until that equality is proven.
func hanzoQuery(org string, from, to time.Time) (string, []any) {
q := `SELECT provider, count() AS requests, sum(total_tokens) AS total_tokens, ` +
`sum(cost_cents) AS cost_cents FROM ` + llmTable + ` ` +
`WHERE organization = ? AND timestamp >= ? AND timestamp < ? ` +
`GROUP BY provider ORDER BY total_tokens DESC LIMIT ?`
return q, []any{org, tsLiteral(from), tsLiteral(to), maxReadRows}
}
// Series reads one provider account's window instances. It returns (rows, true) when
// the warehouse answered and (nil, false) when it is unavailable, so the caller can
// say "unavailable" instead of showing zeros that read as "you used nothing".
func (w *warehouse) Series(ctx context.Context, org, subject, provider, account, window string, from, to time.Time) ([]Sample, bool) {
if org == "" || subject == "" || provider == "" {
return nil, false
}
if err := w.ensureAccountUsage(ctx); err != nil {
return nil, false
}
q, args := seriesQuery(org, subject, provider, account, window, from, to)
rows, err := aiobject.DatastoreQuery(ctx, q, args...)
if err != nil {
return nil, false
}
out := make([]Sample, 0, len(rows))
for _, r := range rows {
out = append(out, sampleOf(r))
}
return out, true
}
// sampleOf decodes one warehouse row into a Sample. Pure, so the whole read path's
// shape is testable without a warehouse.
func sampleOf(r map[string]any) Sample {
return Sample{
Provider: dsString(r["provider"]),
Account: dsString(r["account"]),
Plan: dsString(r["plan"]),
Kind: dsString(r["kind"]),
Machine: dsString(r["machine"]),
Lane: dsString(r["lane"]),
Window: dsString(r["window"]),
WindowMinutes: int32(dsInt64(r["window_minutes"])),
WindowStart: dsTimeOf(r["window_start"]),
ResetsAt: dsTimeOf(r["resets_at"]),
UsedPct: dsFloat(r["used_pct"]),
Confidence: dsString(r["confidence"]),
Synthetic: dsInt64(r["synthetic"]) != 0,
Requests: dsInt64(r["requests"]),
InputTokens: dsInt64(r["input_tokens"]),
OutputTokens: dsInt64(r["output_tokens"]),
TotalTokens: dsInt64(r["total_tokens"]),
CachedInputTokens: dsInt64(r["cached_input_tokens"]),
CostCents: dsInt64(r["cost_cents"]),
CostLimitCents: dsInt64(r["cost_limit_cents"]),
Currency: dsString(r["currency"]),
}
}
// Total is one provider's usage in the global view, from ONE source at ONE scope.
type Total struct {
Source string
Scope string
Provider string
Window string
Requests int64
Tokens int64
CostCents int64
UsedPct float64
Confidence string
Windows int64
}
// AccountTotals reads the caller's own account usage per (provider, window).
func (w *warehouse) AccountTotals(ctx context.Context, org, subject string, from, to time.Time) ([]Total, bool) {
if org == "" || subject == "" {
return nil, false
}
if err := w.ensureAccountUsage(ctx); err != nil {
return nil, false
}
q, args := summaryQuery(org, subject, from, to)
rows, err := aiobject.DatastoreQuery(ctx, q, args...)
if err != nil {
return nil, false
}
out := make([]Total, 0, len(rows))
for _, r := range rows {
out = append(out, accountTotalOf(r))
}
return out, true
}
// accountTotalOf decodes one account-side summary row. It STAMPS source+scope
// server-side (they are facts about which ledger answered, never data from it), so
// an account row can never present itself as Hanzo's cost of record.
func accountTotalOf(r map[string]any) Total {
return Total{
Source: SourceAccount, Scope: ScopeUser,
Provider: dsString(r["provider"]),
Window: dsString(r["window"]),
Requests: dsInt64(r["requests"]),
Tokens: dsInt64(r["total_tokens"]),
CostCents: dsInt64(r["cost_cents"]),
UsedPct: dsFloat(r["used_pct"]),
// The WEAKEST contributing lane's confidence — decoded from the rank the
// query aggregated on, so a partly-unknown sum is never labelled exact.
Confidence: confidenceOfRank(dsInt64(r["confidence_rank"])),
Windows: dsInt64(r["windows"]),
}
}
// HanzoTotals reads the org's Hanzo-routed usage per provider over the same window.
// Its rows are always `exact` — cloud_usage counts real calls we billed.
func (w *warehouse) HanzoTotals(ctx context.Context, org string, from, to time.Time) ([]Total, bool) {
if org == "" {
return nil, false
}
if !aiobject.DatastoreEnabled() {
return nil, false
}
if err := aiobject.EnsureCloudUsageTable(ctx); err != nil {
return nil, false
}
q, args := hanzoQuery(org, from, to)
rows, err := aiobject.DatastoreQuery(ctx, q, args...)
if err != nil {
return nil, false
}
out := make([]Total, 0, len(rows))
for _, r := range rows {
out = append(out, hanzoTotalOf(r))
}
return out, true
}
// hanzoTotalOf decodes one Hanzo-side summary row. Source+scope are stamped
// server-side, and confidence is `exact` because cloud_usage counts calls we
// actually billed — it never estimates. It carries NO window: cloud_usage is a
// per-call ledger, not a window meter, so its totals are simply the requested
// range. Fabricating a window class here would invent a quota that does not exist.
func hanzoTotalOf(r map[string]any) Total {
return Total{
Source: SourceHanzo, Scope: ScopeOrg,
Provider: dsString(r["provider"]),
Requests: dsInt64(r["requests"]),
Tokens: dsInt64(r["total_tokens"]),
CostCents: dsInt64(r["cost_cents"]),
Confidence: ConfidenceExact,
}
}
// ── cell coercion ────────────────────────────────────────────────────────────
// DatastoreQuery decodes each column into its native scan type; these accept the
// widths the driver can hand back, so a column's exact type never reaches a caller.
func dsString(v any) string {
if s, ok := v.(string); ok {
return s
}
return ""
}
func dsInt64(v any) int64 {
switch n := v.(type) {
case int64:
return n
case uint64:
if n > 1<<62 {
return 1 << 62
}
return int64(n)
case uint32:
return int64(n)
case int32:
return int64(n)
case uint8:
return int64(n)
case float64:
return int64(n)
}
return 0
}
func dsFloat(v any) float64 {
switch f := v.(type) {
case float64:
return finite(f)
case float32:
return finite(float64(f))
case uint64:
return float64(f)
case int64:
return float64(f)
}
return 0
}
// epoch is the sentinel for an ABSENT instant.
//
// Go's zero time is year 1, which a DateTime64 column cannot represent — binding it
// would break or silently corrupt the row, and an absent resets_at is the COMMON
// case (most meters report no boundary). So absence is stored as the epoch, which
// the column represents natively and reads back unambiguously as "not set": no real
// window resets in 1970. dsInstant applies it on write, dsTimeOf reverses it on
// read, and the wire omits the field entirely — absence never becomes a date.
var epoch = time.Unix(0, 0).UTC()
// dsInstant renders an instant for a DateTime64 column, mapping the absent (zero)
// instant to the epoch sentinel.
func dsInstant(t time.Time) time.Time {
if t.IsZero() {
return epoch
}
return t.UTC()
}
// dsTimeOf decodes a DateTime64 cell, mapping the epoch sentinel back to absent.
func dsTimeOf(v any) time.Time {
t, ok := v.(time.Time)
if !ok || t.Unix() <= 0 {
return time.Time{}
}
return t.UTC()
}
+38 -11
View File
@@ -20,7 +20,8 @@ const llmTable = "hanzo.cloud_usage"
// ── Response shape ──────────────────────────────────────────────────────────
type Scope struct {
Org string `json:"org"`
Org string `json:"org"`
User string `json:"user,omitempty"` // the caller's subject; whose linked-account rows the accounts block carries
}
// CategorySpend is one row of the spend-by-category breakdown: a friendly category
@@ -78,17 +79,43 @@ type Sources struct {
Warehouse bool `json:"warehouse"`
}
// Summary is the whole org-scoped usage footprint roll-up: the cost roll-up
// (spend) + the LLM usage totals, over one window, for one org.
// Accounts is the account-usage board folded into the summary: the caller's OWN
// linked provider accounts (metered from each provider's own login — a Claude Max
// plan's window %) beside the org's Hanzo-routed usage. Every row is labelled by
// source/scope/confidence and the two are NEVER summed together — a plan's percent
// is not money, and a provider's own spend is not a Hanzo charge. Each side reports
// its own availability, so half a warehouse never turns the other half into zeros.
// This is the account-usage plane's global view (moved from clients/link) unified
// under the ONE /v1/usage/summary; the per-account time series is GET /v1/usage/samples.
type Accounts struct {
Rows []TotalView `json:"rows"`
Account SourceState `json:"account"`
Hanzo SourceState `json:"hanzo"`
}
// SourceState is one side of the account board's availability + labelling: which
// ledger answered, at what scope, and a human note.
type SourceState struct {
Available bool `json:"available"`
Scope string `json:"scope"`
Source string `json:"source"` // the table of record
Note string `json:"note"`
}
// Summary is the whole own-scoped usage footprint roll-up over one window: the cost
// roll-up (spend) + the org's LLM usage totals (llm) + the caller's linked-account
// board (accounts). Spend/LLM are org-scoped; the account board is the caller's own
// (org+subject). One screen, one authoritative money source, one window.
type Summary struct {
Range string `json:"range"`
Start string `json:"start"`
End string `json:"end"`
Interval string `json:"interval"`
Scope Scope `json:"scope"`
Spend Spend `json:"spend"`
LLM LLM `json:"llm"`
Sources Sources `json:"sources"`
Range string `json:"range"`
Start string `json:"start"`
End string `json:"end"`
Interval string `json:"interval"`
Scope Scope `json:"scope"`
Spend Spend `json:"spend"`
LLM LLM `json:"llm"`
Accounts Accounts `json:"accounts"`
Sources Sources `json:"sources"`
}
// ── Commerce ledger row (mirrors commerce GET /v1/billing/transactions) ──────
+434
View File
@@ -0,0 +1,434 @@
package usage
import (
"math"
"strings"
"time"
)
// sample.go is the ACCOUNT-USAGE value plane: ONE metering lane of ONE provider
// account, as observed at one instant. It is the atom hanzo.account_usage stores
// and the usage faces read. It moved here from clients/link so the ONE usage
// subsystem owns ALL usage — link owns links, and nothing usage.
//
// IT IS SHAPED TO WHAT @hanzo/usage's METER CAN ACTUALLY REPORT — never more. The
// meter reads each provider's OWN login (no API key) and yields a UsageSnapshot:
// rate-limit lanes carrying a used PERCENT (RateWindow.usedPercent + windowMinutes
// + resetsAt), an OPTIONAL absolute UsageTotals (tokens/requests — present only at
// `exact` confidence), an OPTIONAL ProviderCostSnapshot (used/limit, MONEY), and a
// dataConfidence saying how much of that is real. Claude — the account this plane
// exists for — reports dataConfidence `percentOnly` and NO totals at all: its truth
// is the five_hour / seven_day PERCENT and nothing else. So UsedPct is a
// first-class column here and an absolute token quota is NOT one: no source can
// produce it (neither the meter nor the hanzoai/plans catalog — see the quota gap
// note on UsedPct).
//
// THREE ORTHOGONAL VALUES say "which meter", because they genuinely differ:
//
// - Window the CANONICAL class the platform bills and limits on
// - WindowMinutes the duration the meter ACTUALLY reported — never rounded to fit
// the class, so the console renders the truth ("5h") not the bucket
// - Lane the meter's own lane id — the IDENTITY, and the only value that
// can key a lane
//
// Claude's five_hour lane is 300 minutes and classes as `6h`. Its seven_day,
// seven_day_sonnet and seven_day_opus lanes ALL class as `week` at 10080 minutes
// yet are THREE DIFFERENT METERS — only Lane tells them apart, so only Lane can key
// them. Collapsing them onto the class would silently overwrite two of the three.
//
// NO SECRET LIVES HERE: a sample is counters and percents. The provider token never
// leaves the device.
// The CANONICAL window value: the ONE closed vocabulary that rate limits, quotas,
// and usage rollups all share, platform-wide. Lowercase, no case variants, no
// synonyms — a window is one of exactly these four values.
//
// This declaration is the single source of the vocabulary. (hanzoai/commerce today
// carries an ad-hoc set — weekly/daily/monthly/hourly plus capitalized variants,
// and no sub-day window at all; a later commerce pass adopts THESE values. Do not
// copy the old strings back in.)
const (
Window6h = "6h"
WindowDay = "day"
WindowWeek = "week"
WindowMonth = "month"
)
// Confidence values — how real a sample's numbers are. These mirror @hanzo/usage's
// UsageDataConfidence EXACTLY (they are the meter's own wire values, already
// carried on Usage.Confidence), so a value crosses the whole system unmapped.
//
// This is the ANTI-FABRICATION FLAG, and the reason it is a column: a Claude
// sample is `percentOnly`, so its token counters are 0 because they are UNKNOWN —
// not because the account consumed nothing. Without this value a reader cannot
// tell "0 tokens" from "no idea", and would render a fabricated zero. With it, the
// console renders "—".
const (
ConfidenceExact = "exact"
ConfidenceEstimated = "estimated"
ConfidencePercentOnly = "percentOnly"
ConfidenceUnknown = "unknown"
)
// Sources for the global view: which plane a usage row came from. They are NOT
// interchangeable and are never added together —
//
// - SourceAccount the provider's OWN plan consumption, metered from the user's
// own login. Its cost is what the PROVIDER says it charged (0 for a flat
// subscription); its percent is plan quota. NOT a Hanzo charge.
// - SourceHanzo hanzo.cloud_usage — Hanzo-routed inference. COST OF RECORD.
const (
SourceAccount = "account"
SourceHanzo = "hanzo"
)
// Scopes for a global-view row: the tenancy the row's numbers cover. The two
// sources answer at different scopes and the row says which, so a reader never
// silently compares a user's plan usage against an org's whole spend.
const (
ScopeUser = "user" // the caller's OWN linked accounts (org+subject)
ScopeOrg = "org" // the whole org's Hanzo-routed usage
)
// Account-kind vocabulary — how a metered account is credentialed. It mirrors the
// link registry's Kind (subscription | apikey) so a Sample and its Link agree, but
// each package validates its OWN inputs: the value is a fact about the account, not
// a shared place, so the two are decoupled and neither can force a change on the
// other.
const (
KindSubscription = "subscription"
KindAPIKey = "apikey"
)
// Field bounds. A sample is a handful of numbers plus short identifiers; the caps
// keep a hostile or buggy client from bloating the warehouse.
const (
maxProvider = 64
maxAccount = 256
maxPlan = 128
maxMachine = 256
maxLane = 64
maxCurrency = 8
// maxSamples caps ONE report. A machine's meter yields a few lanes per provider
// (Claude alone has four), and a cost-history backfill adds a row per day, so
// the cap is generous — but closed.
maxSamples = 256
// maxCounter bounds a counter well inside UInt64 and float64's exact-integer
// range, so a garbage 1e30 clamps instead of wrapping the warehouse column.
maxCounter = 1 << 52
// maxWindowMinutes is a year — longer than any real metering window.
maxWindowMinutes = 366 * 24 * 60
)
// Clock bounds for the two client-supplied instants. The SERVER owns the
// observation clock (Sample.ts) absolutely; these bound only the WINDOW the sample
// describes, which the client legitimately knows (Claude reports resets_at).
const (
// maxAhead lets a window reset in the future (that is what a reset IS) but not
// beyond the longest window plus slack — no far-future pinning.
maxAhead = 400 * 24 * time.Hour
// maxBehind lets a backfill report a historical window inside the table's own
// TTL, and no further — no far-past rows that the TTL would drop on arrival.
maxBehind = 400 * 24 * time.Hour
)
// Sample is one metering lane's consumption of one provider account at one
// observation.
//
// TENANCY IS NOT ON THE VALUE. Org and subject are bound by the server from the
// validated principal at the boundary, so a client cannot assert whose usage this
// is — the shape makes cross-tenant writes unrepresentable rather than merely
// refused.
type Sample struct {
Provider string // matches Link.Provider / @hanzo/usage providerRegistry id
Account string // the account label (an identifier, never a secret)
Plan string // the plan name from the provider identity; display only
Kind string // subscription | apikey (validKind) — mirrors Link.Kind
Machine string // the machine that OBSERVED this (an attribute, not a key)
Lane string // the meter's lane id (five_hour|seven_day_opus|…) — the identity
Window string // the canonical class (6h|day|week|month)
WindowMinutes int32 // the duration the meter reported (300 for Claude's 5h)
WindowStart time.Time // WHICH instance of the window this measures — the dedup key
ResetsAt time.Time // when the window resets (RateWindow.resetsAt); zero = unknown
// UsedPct is the lane's used percent, 0..100 — RateWindow.usedPercent. For a
// subscription account this is THE quota signal and often the ONLY one.
//
// QUOTA GAP (deliberate, reported, not faked): there is NO absolute
// used/limit token quota column, because nothing can populate one. The meter
// reports a percent, not a limit (Claude: dataConfidence `percentOnly`). The
// hanzoai/plans catalog expresses `ai.requests_per_min` / `ai.tokens_per_min`
// — per-MINUTE rate limits, a different concept from a window quota — and in
// any case it is HANZO's own plan catalog, which cannot know what Anthropic
// grants a Claude Max plan. An always-zero quota_limit would read as "the
// limit is zero"; absent is honest. It lands as an additive column when a
// source for it exists.
UsedPct float64
Confidence string // exact|estimated|percentOnly|unknown — see the const block
// Synthetic marks a lane the meter FABRICATED (RateWindow.isSyntheticPlaceholder
// — set when a provider returned null for a lane and the adapter filled it in).
// Carried so a made-up lane is never rendered as observed truth.
Synthetic bool
// Absolute counters — UsageTotals. Present only when the source really reports
// them (`exact`); zero otherwise, which Confidence disambiguates.
Requests int64
InputTokens int64
OutputTokens int64
TotalTokens int64
CachedInputTokens int64
// Money — ProviderCostSnapshot, in minor units (cents). CostCents is what the
// PROVIDER says this account spent (Claude's extra_usage overage; 0 for a flat
// subscription — the plan is the charge). CostLimitCents is that meter's budget
// (ProviderCostSnapshot.limit) — a MONEY cap, never a token quota. Neither is a
// Hanzo charge: this plane holds no metering client.
CostCents int64
CostLimitCents int64
Currency string // ProviderCostSnapshot.currencyCode; "" = unknown
}
// validWindow reports whether w is the canonical window vocabulary.
func validWindow(w string) bool {
switch w {
case Window6h, WindowDay, WindowWeek, WindowMonth:
return true
}
return false
}
// validConfidence reports whether c is a known meter confidence.
func validConfidence(c string) bool {
switch c {
case ConfidenceExact, ConfidenceEstimated, ConfidencePercentOnly, ConfidenceUnknown:
return true
}
return false
}
// validKind reports whether k is a known account kind.
func validKind(k string) bool { return k == KindSubscription || k == KindAPIKey }
// confidenceRank orders the confidences by HOW MUCH A READER MAY BELIEVE THEM —
// exact is 0, unknown is the worst. It exists so an aggregate can pick the WEAKEST
// contributing confidence explicitly, by rank.
//
// Ordering strings alphabetically instead would be an accident that happens to
// look right: 'exact' < 'percentOnly' < 'unknown' sorts by severity purely by luck
// of spelling, and the first value that breaks the coincidence (an 'approximate',
// say) would silently start reporting mixed data as trustworthy. Severity is a
// property of the vocabulary, so it is declared once, here.
func confidenceRank(c string) uint8 {
switch c {
case ConfidenceExact:
return 0
case ConfidenceEstimated:
return 1
case ConfidencePercentOnly:
return 2
}
return 3 // unknown, and anything unrecognised
}
// confidenceOfRank is confidenceRank's inverse, for decoding an aggregate.
func confidenceOfRank(r int64) string {
switch r {
case 0:
return ConfidenceExact
case 1:
return ConfidenceEstimated
case 2:
return ConfidencePercentOnly
}
return ConfidenceUnknown
}
// nominalMinutes is a window class's NOMINAL length — a property of the
// vocabulary, not a measurement.
//
// It exists only so that every sample has a derivable instance even when the meter
// reported no duration and no reset: without it such a sample would key the zero
// instant, which is not a representable warehouse timestamp. It is NEVER stored as
// the sample's WindowMinutes, which stays the meter's own value (0 = "the meter did
// not say", rendered as unknown). That keeps the reported truth and the class's
// nominal separate: Claude's 6h-class lane really is 300 minutes, and this function
// must never be allowed to say otherwise.
func nominalMinutes(window string) int32 {
switch window {
case Window6h:
return 6 * 60
case WindowDay:
return 24 * 60
case WindowWeek:
return 7 * 24 * 60
case WindowMonth:
return 30 * 24 * 60
}
return 0
}
// Sanitize bounds every field so a warehouse row stays small, finite, and
// well-formed no matter what a client sends: strings trimmed and length-capped,
// counters non-negative and clamped, percents coerced into [0,100] even for
// NaN/Inf, instants bounded to a sane window around `now`, and the two enums
// defaulted rather than trusted. It is TOTAL (never errors), so the write path
// always has a safe value — validation of the CLOSED vocabularies (window, kind)
// is the boundary's job and is a 400, because silently rewriting a caller's window
// class would corrupt their dash.
//
// It does NOT set the observation clock: the server stamps that, so a client can
// never backdate a sample or pin a stale one as newest.
func (s Sample) Sanitize(now time.Time) Sample {
out := Sample{
Provider: clampStr(s.Provider, maxProvider),
Account: clampStr(s.Account, maxAccount),
Plan: clampStr(s.Plan, maxPlan),
Kind: clampStr(s.Kind, maxProvider),
Machine: clampStr(s.Machine, maxMachine),
Lane: clampStr(s.Lane, maxLane),
Window: clampStr(s.Window, maxLane),
WindowMinutes: int32(clampInt64(int64(s.WindowMinutes), maxWindowMinutes)),
UsedPct: clampPct(finite(s.UsedPct)),
Confidence: clampStr(s.Confidence, maxLane),
Synthetic: s.Synthetic,
Requests: clampInt64(s.Requests, maxCounter),
InputTokens: clampInt64(s.InputTokens, maxCounter),
OutputTokens: clampInt64(s.OutputTokens, maxCounter),
TotalTokens: clampInt64(s.TotalTokens, maxCounter),
CachedInputTokens: clampInt64(s.CachedInputTokens, maxCounter),
CostCents: clampInt64(s.CostCents, maxCounter),
CostLimitCents: clampInt64(s.CostLimitCents, maxCounter),
Currency: clampStr(s.Currency, maxCurrency),
}
if !validKind(out.Kind) {
out.Kind = KindSubscription
}
if !validConfidence(out.Confidence) {
// An unrecognised (or absent) confidence is `unknown` — the honest default.
// Never `exact`: a client must EARN that word, since it is what licenses a
// reader to believe the counters.
out.Confidence = ConfidenceUnknown
}
// A lane defaults to its window class: a simple reporter that knows only "my 6h
// window" still gets a stable, correctly-keyed lane of its own.
if out.Lane == "" {
out.Lane = out.Window
}
out.ResetsAt = boundTime(s.ResetsAt, now)
// The instance derives from the meter's own duration when it reported one, and
// from the class's nominal when it did not — so every sample of a valid window
// has a representable key, while WindowMinutes keeps reporting only what the
// meter actually said.
span := out.WindowMinutes
if span <= 0 {
span = nominalMinutes(out.Window)
}
out.WindowStart = windowInstance(s.WindowStart, out.ResetsAt, span, now)
return out
}
// windowInstance resolves WHICH instance of the window a sample measures. It is
// THE IDEMPOTENCY KEY: two reports of the same live window must land on the same
// instant, or the same consumption is stored twice and every sum double-counts.
//
// A snapshot is CUMULATIVE-TO-DATE within its window, not a delta — a poller
// re-reading its 6h lane every 30s reports "47%… 48%… 51%" OF THE SAME WINDOW. So
// the row is keyed by the window instance and REPLACED as it grows, never appended.
// Keying on the observation time (the obvious move) would defeat this completely:
// every poll carries a fresh instant, so every poll would key a new row and nothing
// would ever collapse.
//
// Precedence, most authoritative first:
//
// 1. an explicit start — the meter's own instance id, already bounded
// 2. resets_at the window's length — the provider's own instance boundary
// (Claude reports resets_at, and every machine watching that account reads the
// SAME value, so independent observers agree without coordination)
// 3. the observation floored to a fixed grid of the window's length — for a meter
// that reports no reset, this is deterministic, so independent observers still
// agree
// 4. the zero instant — a windowless lifetime counter is ONE row per lane,
// forever replaced, never summed across polls
//
// It is pure: `now` is injected, so the test is deterministic.
func windowInstance(start, resetsAt time.Time, windowMinutes int32, now time.Time) time.Time {
if !start.IsZero() {
return boundTime(start, now).Truncate(time.Second)
}
d := time.Duration(windowMinutes) * time.Minute
if !resetsAt.IsZero() && d > 0 {
return resetsAt.Add(-d).Truncate(time.Second)
}
if d > 0 {
return now.UTC().Truncate(d)
}
return time.Time{}
}
// boundTime clamps a client-supplied instant into [now-maxBehind, now+maxAhead] and
// normalises it to UTC, so neither a far-future nor a far-past value can be stored.
// The zero instant means "absent" and stays zero.
func boundTime(t, now time.Time) time.Time {
if t.IsZero() {
return time.Time{}
}
t = t.UTC()
if lo := now.UTC().Add(-maxBehind); t.Before(lo) {
return lo
}
if hi := now.UTC().Add(maxAhead); t.After(hi) {
return hi
}
return t
}
// finite maps NaN/Inf to 0 so a hostile percent can never reach the warehouse or
// break a JSON encode. (clampPct alone cannot: NaN fails every comparison and
// would pass straight through.)
func finite(f float64) float64 {
if math.IsNaN(f) || math.IsInf(f, 0) {
return 0
}
return f
}
// clampPct bounds a percent to [0,100]; a provider that reports a nonsense value
// can never poison a headroom ordering.
func clampPct(v float64) float64 {
if v < 0 {
return 0
}
if v > 100 {
return 100
}
return v
}
// clampInt64 returns a non-negative int64 no larger than hi.
func clampInt64(v, hi int64) int64 {
if v < 0 {
return 0
}
if v > hi {
return hi
}
return v
}
// clampStr trims a string and caps its length, keeping the result valid UTF-8 so a
// multi-byte rune cut at the cap can never yield a broken column value.
func clampStr(s string, n int) string {
s = trim(s)
if len(s) > n {
return strings.ToValidUTF8(s[:n], "")
}
return s
}
// trim is strings.TrimSpace, aliased for terse boundary code.
func trim(s string) string { return strings.TrimSpace(s) }
+195
View File
@@ -0,0 +1,195 @@
package usage
import (
"math"
"strings"
"testing"
"time"
)
var now = time.Date(2026, 7, 15, 12, 0, 0, 0, time.UTC)
// TestWindowInstanceIsStableAcrossRepolls is THE idempotency gate at the value
// layer: a poller re-reading the SAME live window must land on the SAME instance
// every time, because that instant is the dedup key. If it drifted, each poll would
// key a new row, nothing would ever collapse, and every sum would be multiplied by
// the poll rate.
//
// A snapshot is cumulative-to-date, not a delta: these three reports say "42% …
// 47% … 51% OF THE SAME 6h WINDOW", and they must be ONE row.
func TestWindowInstanceIsStableAcrossRepolls(t *testing.T) {
resets := now.Add(90 * time.Minute) // the provider's own boundary
var keys []time.Time
for i, at := range []time.Time{now, now.Add(30 * time.Second), now.Add(4 * time.Minute)} {
s := Sample{
Provider: "claude", Machine: "m1", Window: Window6h, Lane: "five_hour",
WindowMinutes: 300, ResetsAt: resets, UsedPct: float64(42 + i*5),
}.Sanitize(at)
keys = append(keys, s.WindowStart)
}
for i := 1; i < len(keys); i++ {
if !keys[i].Equal(keys[0]) {
t.Fatalf("re-poll %d keyed a DIFFERENT instance (%s vs %s) — every sum would double-count",
i, keys[i], keys[0])
}
}
// And it is the provider's boundary: resets_at the window's length.
if want := resets.Add(-300 * time.Minute); !keys[0].Equal(want) {
t.Fatalf("instance = %s, want resets_atwindow = %s", keys[0], want)
}
}
// TestIndependentObserversAgree: two machines watching the SAME account must key
// the same window instance, or one plan's quota is stored twice and doubles.
// Neither the reset-derived path nor the grid-bucket fallback may depend on which
// machine observed, or on when within the window it looked.
func TestIndependentObserversAgree(t *testing.T) {
resets := now.Add(2 * time.Hour)
laptop := Sample{Provider: "claude", Machine: "laptop", Window: Window6h,
WindowMinutes: 300, ResetsAt: resets}.Sanitize(now)
desktop := Sample{Provider: "claude", Machine: "desktop", Window: Window6h,
WindowMinutes: 300, ResetsAt: resets}.Sanitize(now.Add(3 * time.Minute))
if !laptop.WindowStart.Equal(desktop.WindowStart) {
t.Fatalf("two observers of one account keyed different instances: %s vs %s",
laptop.WindowStart, desktop.WindowStart)
}
// The no-reset fallback buckets on a fixed grid, so observers still agree even
// when the meter reports no boundary at all.
a := Sample{Provider: "codex", Machine: "a", Window: Window6h, WindowMinutes: 360}.Sanitize(now)
b := Sample{Provider: "codex", Machine: "b", Window: Window6h, WindowMinutes: 360}.Sanitize(now.Add(7 * time.Minute))
if !a.WindowStart.Equal(b.WindowStart) {
t.Fatalf("grid fallback disagreed: %s vs %s", a.WindowStart, b.WindowStart)
}
}
// TestNewWindowIsANewRow: when a window RESETS, the next instance must key
// differently — dedup must collapse re-reports without collapsing real history.
func TestNewWindowIsANewRow(t *testing.T) {
first := Sample{Provider: "claude", Machine: "m1", Window: Window6h,
WindowMinutes: 300, ResetsAt: now.Add(time.Hour)}.Sanitize(now)
next := Sample{Provider: "claude", Machine: "m1", Window: Window6h,
WindowMinutes: 300, ResetsAt: now.Add(6 * time.Hour)}.Sanitize(now.Add(2 * time.Hour))
if first.WindowStart.Equal(next.WindowStart) {
t.Fatal("a NEW window instance collapsed onto the previous one — real history would be lost")
}
}
// TestWindowlessCounterIsOneRow: a counter whose meter reported no duration and no
// reset still keys ONE stable instance per lane — its class's nominal bucket — so
// re-polls of it COLLAPSE onto one row (ReplacingMergeTree by ts) instead of
// stacking a new row per poll and being summed. The instance is never the zero
// instant, which every ranged read filters out (window_start ∈ [from,to)) and the
// TTL drops on arrival — see TestEveryValidWindowKeysARepresentableInstant.
func TestWindowlessCounterIsOneRow(t *testing.T) {
a := Sample{Provider: "hanzo", Machine: "m1", Window: WindowMonth, TotalTokens: 10}.Sanitize(now)
b := Sample{Provider: "hanzo", Machine: "m1", Window: WindowMonth, TotalTokens: 20}.Sanitize(now.Add(time.Hour))
if a.WindowStart.IsZero() || b.WindowStart.IsZero() {
t.Fatalf("a windowless counter must key a representable instant, got %s / %s", a.WindowStart, b.WindowStart)
}
if !a.WindowStart.Equal(b.WindowStart) {
t.Fatalf("two polls of one windowless counter must key the SAME instance (one row), got %s vs %s",
a.WindowStart, b.WindowStart)
}
}
// TestSanitizeBounds: every field is bounded, so a hostile or buggy client can
// never bloat a row, wrap a column, or smuggle a non-finite float.
func TestSanitizeBounds(t *testing.T) {
huge := strings.Repeat("x", 5000)
got := Sample{
Provider: huge, Account: huge, Plan: huge, Machine: huge, Lane: huge, Currency: huge,
Kind: "wildcard", Confidence: "totally-legit",
UsedPct: math.Inf(1),
WindowMinutes: 1 << 30,
Requests: -5, InputTokens: math.MaxInt64, OutputTokens: -1,
TotalTokens: math.MaxInt64, CachedInputTokens: -99,
CostCents: math.MaxInt64, CostLimitCents: -1,
}.Sanitize(now)
if len(got.Provider) > maxProvider || len(got.Account) > maxAccount ||
len(got.Plan) > maxPlan || len(got.Machine) > maxMachine ||
len(got.Lane) > maxLane || len(got.Currency) > maxCurrency {
t.Fatalf("a string escaped its cap: %+v", got)
}
if got.UsedPct != 0 {
t.Fatalf("+Inf percent must coerce to 0, got %v", got.UsedPct)
}
if got.WindowMinutes != maxWindowMinutes {
t.Fatalf("windowMinutes must clamp to %d, got %d", maxWindowMinutes, got.WindowMinutes)
}
for name, v := range map[string]int64{
"requests": got.Requests, "inputTokens": got.InputTokens, "outputTokens": got.OutputTokens,
"totalTokens": got.TotalTokens, "cachedInputTokens": got.CachedInputTokens,
"costCents": got.CostCents, "costLimitCents": got.CostLimitCents,
} {
if v < 0 || v > maxCounter {
t.Fatalf("%s escaped [0,%d]: %d", name, maxCounter, v)
}
}
// An unrecognised kind/confidence must DEFAULT, never pass through — `exact`
// especially must be earned, since it is what licenses a reader to believe.
if got.Kind != KindSubscription {
t.Fatalf("an unknown kind must default to subscription, got %q", got.Kind)
}
if got.Confidence != ConfidenceUnknown {
t.Fatalf("an unknown confidence must default to unknown, got %q", got.Confidence)
}
}
// TestSanitizePercentIsClamped covers the percent range specifically: it drives a
// headroom ordering, so a nonsense value must never order candidates.
func TestSanitizePercentIsClamped(t *testing.T) {
for _, tc := range []struct{ in, want float64 }{
{-10, 0}, {0, 0}, {47.5, 47.5}, {100, 100}, {1e9, 100},
{math.NaN(), 0}, {math.Inf(-1), 0},
} {
got := Sample{Provider: "claude", Machine: "m", Window: Window6h, UsedPct: tc.in}.Sanitize(now)
if got.UsedPct != tc.want {
t.Fatalf("usedPct %v → %v, want %v", tc.in, got.UsedPct, tc.want)
}
}
}
// TestClientClockIsBounded: the two instants a client may state are clamped to a
// sane interval, so neither a far-future nor a far-past value is ever stored.
func TestClientClockIsBounded(t *testing.T) {
far := Sample{Provider: "claude", Machine: "m", Window: Window6h,
WindowStart: now.AddDate(50, 0, 0)}.Sanitize(now)
if far.WindowStart.After(now.Add(maxAhead)) {
t.Fatalf("a far-future windowStart was stored: %s", far.WindowStart)
}
old := Sample{Provider: "claude", Machine: "m", Window: Window6h,
WindowStart: now.AddDate(-50, 0, 0)}.Sanitize(now)
if old.WindowStart.Before(now.Add(-maxBehind)) {
t.Fatalf("a far-past windowStart was stored: %s", old.WindowStart)
}
}
// TestLaneDefaultsToWindow: a reporter that knows only its window class still gets
// a stable, correctly-keyed lane.
func TestLaneDefaultsToWindow(t *testing.T) {
got := Sample{Provider: "claude", Machine: "m", Window: WindowWeek}.Sanitize(now)
if got.Lane != WindowWeek {
t.Fatalf("lane must default to the window class, got %q", got.Lane)
}
}
// TestCanonicalWindowVocabulary pins the closed vocabulary: exactly four values,
// lowercase, no synonyms and no case variants. This is the value rate limits,
// quotas and usage rollups all share, so drift here is drift everywhere.
func TestCanonicalWindowVocabulary(t *testing.T) {
for _, ok := range []string{"6h", "day", "week", "month"} {
if !validWindow(ok) {
t.Fatalf("%q must be canonical", ok)
}
}
for _, bad := range []string{
"", "5h", "1h", "hour", "hourly", "daily", "weekly", "monthly", "session", "total",
"6H", "Day", "Week", "WEEK", "Monthly", "7d", "30d", " day", "day ",
} {
if validWindow(bad) {
t.Fatalf("%q must NOT be a window — the vocabulary is closed", bad)
}
}
}
+95 -46
View File
@@ -1,7 +1,16 @@
// Package usage mounts the Hanzo Cloud UNIFIED USAGE surface (GET /v1/usage/summary):
// one org-scoped roll-up answering "what am I running and what does it cost" — the
// flagship footprint view. It is a native-Go composition over the two canonical
// shared cost sources, and exists because NO single endpoint aggregates this today:
// Package usage is the ONE Hanzo Cloud usage surface (/v1/usage). It owns ALL usage
// — clients/link owns links and nothing usage — as one coherent plane over one
// window grammar:
//
// - POST /v1/usage record account-usage samples (the collector's data
// plane; write path over the hanzo.account_usage warehouse series, datastore.go).
// - GET /v1/usage/samples one provider account's own lane dash (the time series).
// - GET /v1/usage/summary the flagship own-scoped footprint roll-up (below).
// - GET /v1/usage/analytics{,/access} the entitlement-gated rich per-org read.
//
// The summary answers "what am I running and what does it cost" by composing THREE
// complementary sources, each degrading independently to honest zeros (a source
// marker says which answered — a partial deploy never fabricates spend or usage):
//
// - Spend (the genuinely-missing categorized cost roll-up): the commerce ledger —
// usage-rollup (authoritative month-to-date consumed + prepaid wallet) plus the
@@ -12,24 +21,26 @@
// - LLM usage totals: hanzo.cloud_usage, the per-org warehouse ledger (the same
// table /v1/analytics/* and the o11y board read). Totals only here — the
// per-model / timeseries detail stays at /v1/analytics/*.
// - The account board: the caller's OWN linked provider accounts (a Claude Max
// plan's window %, metered from the provider's own login) beside the org's
// Hanzo-routed usage, every row labelled by source/scope and NEVER summed. This
// is the account-usage global view, unified here from clients/link.
//
// The console Usage view composes THIS (cost roll-up + LLM totals) with the existing
// org-scoped inventory endpoints (/v1/machines, /v1/gpus, /v1/agents, provisioning
// lists) for the per-kind counts — one screen, this one authoritative money source.
// The console Usage view composes THIS with the existing org-scoped inventory
// endpoints (/v1/machines, /v1/gpus, /v1/agents, provisioning lists) for the
// per-kind counts — one screen, this one authoritative money source.
//
// TENANT ISOLATION (the bar). The org is the VALIDATED IAM owner claim
// (principal.Org — the trusted X-Org-Id the identity middleware minted from the
// caller's verified bearer, HIP-0026; NEVER a client header) AND a validated
// principal is required (c.User() set only for a verified bearer). The commerce
// subject is pinned server-side to that org; the warehouse query binds it
// POSITIONALLY. A caller can only ever read its OWN org. Fail-closed: no principal
// → 401. Every source degrades independently to honest zeros (source markers say
// which answered) — a partial deploy never fabricates spend or usage.
// TENANT ISOLATION (the bar). The org is the VALIDATED IAM owner claim (principal.Org
// — the trusted X-Org-Id the identity middleware minted from the caller's verified
// bearer, HIP-0026; NEVER a client header) AND a validated principal is required
// (c.User() set only for a verified bearer). The commerce subject is pinned
// server-side to that org; every warehouse query binds org (and, for the account
// board, subject) POSITIONALLY. A caller can only ever read/write its OWN org
// fail-closed: no principal → 401.
//
// Registered as "usagesvc" (NOT "usage") + order 131: the name diverges from the
// /v1/usage route so serve.go's generic GET /v1/<name>/health parks at
// /v1/usagesvc/health and never shadows the summary. Order 131 binds /v1/usage/*
// before the ai subsystem's /v1/* catch-all (150).
// Registered as "usage" (order 131, OwnsHealth=false): the generic GET
// /v1/usage/health liveness route is a distinct path and never shadows these; order
// 131 binds /v1/usage/* before the ai subsystem's /v1/* catch-all (150).
package usage
import (
@@ -56,11 +67,13 @@ import (
// unbounded read.
const maxLedgerRows = 1000
// state is the usage subsystem's own data: the commerce S2S reader. The shared
// deps (logger, billing, brand) live in the embedded cloud.Base, reached as
// s.Log / s.Bill — never re-plumbed here.
// state is the usage subsystem's own data: the commerce S2S reader (spend) and the
// account-usage warehouse projection (samples + account board). The shared deps
// (logger, billing, brand) live in the embedded cloud.Base, reached as s.Log /
// s.Bill — never re-plumbed here.
type state struct {
commerce *commerceReader
commerce *commerceReader
warehouse *warehouse
}
// Mount wires the usage surface onto app per HIP-0106 — one line over the generic
@@ -70,21 +83,31 @@ func Mount(app *zip.App, deps cloud.Deps) error {
}
// build constructs the usage state: the commerce S2S reader from its env
// (COMMERCE_SERVICE_TOKEN is a KMS-sourced secret already on the cloud env,
// never hard-coded).
// (COMMERCE_SERVICE_TOKEN is a KMS-sourced secret already on the cloud env, never
// hard-coded) and the account-usage warehouse (a DDL latch over aiobject's shared
// datastore — no handle of its own, so the subsystem needs no Shutdown).
func build(b cloud.Base) (state, error) {
cr := newCommerceReader(commerceinproc.BaseURL(os.Getenv("CLOUD_COMMERCE_HTTP_URL")), os.Getenv("COMMERCE_SERVICE_TOKEN"))
b.Log.Info("usage summary surface", "prefix", "/v1/usage", "commerce", cr.configured())
return state{commerce: cr}, nil
b.Log.Info("usage surface", "prefix", "/v1/usage", "commerce", cr.configured())
return state{commerce: cr, warehouse: &warehouse{}}, nil
}
// routes registers the usage surface.
// - /summary — basic own-org usage roll-up, UNGATED.
// - /analytics/access echoes a plan's resolved AnalyticsAccess (always 200, free
// floor on error) so dashboards self-configure against the live @hanzo/plans
// catalog.
// - /analytics is the rich per-org read, entitlement-gated (access.Datastore).
// routes registers the ONE usage surface — the read faces plus the account-usage
// data plane (moved here from clients/link so usage owns ALL usage).
// - POST /usage record account-usage samples (the collector).
// - GET /usage/samples one provider account's own lane dash (time series).
// - GET /summary the own-scoped footprint roll-up (spend + LLM + the
// account board), UNGATED.
// - GET /analytics/access echoes a plan's resolved AnalyticsAccess (always 200,
// free floor on error) so dashboards self-configure against the live catalog.
// - GET /analytics is the rich per-org read, entitlement-gated (access.Datastore).
//
// No :param routes here, so registration order is irrelevant; the generic
// /v1/usage/health liveness route (OwnsHealth=false) is a distinct path and never
// shadows these.
func routes(app *zip.App, s *cloud.Service[state]) {
app.Post("/v1/usage", cloud.Handle(s, record))
app.Get("/v1/usage/samples", cloud.Handle(s, samples))
app.Get("/v1/usage/summary", cloud.Handle(s, summary))
app.Get("/v1/usage/analytics/access", cloud.Handle(s, analyticsAccess))
app.Get("/v1/usage/analytics", cloud.Handle(s, analytics))
@@ -95,11 +118,11 @@ func routes(app *zip.App, s *cloud.Service[state]) {
// grammar, no drift). Composes commerce spend + warehouse LLM totals, each
// degrading independently to honest zeros.
func summary(s *cloud.Service[state], c *zip.Ctx) error {
// principal.Org requires a VALIDATED principal (c.User() set only for a
// verified bearer) and returns the trusted, minted X-Org-Id refusing a forged
// header on the no-principal path. This is the "org from validated bearer ONLY"
// contract.
org, ok := principal.Org(c)
// caller composes principal.Org (a VALIDATED principal c.User() set only for a
// verified bearer returning the trusted, minted X-Org-Id, refusing a forged
// header) with the subject. This is the "org from validated bearer ONLY" contract;
// user scopes the account board to the caller's OWN linked accounts.
org, user, ok := caller(c)
if !ok {
return zip.ErrUnauthorized("sign in to view usage")
}
@@ -119,6 +142,10 @@ func summary(s *cloud.Service[state], c *zip.Ctx) error {
// ── LLM totals (warehouse) ── honest-empty when the datastore is not connected.
llm, warehouseOK := buildLLMBlock(s, ctx, org, start, end)
// ── Account board ── the caller's own linked provider accounts beside the org's
// Hanzo-routed usage, over the SAME window; each side degrades independently.
accounts := buildAccountsBlock(s, ctx, org, user, start, end)
// Per-tenant money must never be cached by the browser or an intermediary.
c.SetHeader("Cache-Control", "no-store")
return c.JSON(http.StatusOK, Summary{
@@ -126,9 +153,10 @@ func summary(s *cloud.Service[state], c *zip.Ctx) error {
Start: start.UTC().Format(time.RFC3339),
End: end.UTC().Format(time.RFC3339),
Interval: interval,
Scope: Scope{Org: org},
Scope: Scope{Org: org, User: user},
Spend: spend,
LLM: llm,
Accounts: accounts,
Sources: Sources{Commerce: spend.Available, Warehouse: warehouseOK},
})
}
@@ -282,7 +310,7 @@ func buildAnalyticsBlock(s *cloud.Service[state], ctx context.Context, org strin
out := ProviderBreakdown{Available: true, Items: make([]ProviderRow, 0, len(rows)), Source: llmTable}
for _, r := range rows {
out.Items = append(out.Items, ProviderRow{
Provider: aString(r["provider"]),
Provider: dsString(r["provider"]),
Requests: aInt64(r["requests"]),
Tokens: aInt64(r["tokens"]),
CostCents: aInt64(r["cost_cents"]),
@@ -291,13 +319,34 @@ func buildAnalyticsBlock(s *cloud.Service[state], ctx context.Context, org strin
return out
}
// aString coerces a datastore string cell to string across the driver/JSON
// transports (mirrors aInt64 in query.go). Non-string → "".
func aString(v any) string {
if s, ok := v.(string); ok {
return s
// buildAccountsBlock reads the caller's OWN linked-account usage (AccountTotals,
// user-scoped) beside the org's Hanzo-routed usage (HanzoTotals, org-scoped) over
// [start,end), folding both into the labelled account board. Each side degrades
// independently to honest-unavailable (Available=false), so half a warehouse never
// fabricates the other half. The two row sets are concatenated, NEVER summed — a
// plan's percent is not money, a provider's own spend is not a Hanzo charge — and
// each row is stamped with its source/scope server-side.
func buildAccountsBlock(s *cloud.Service[state], ctx context.Context, org, user string, start, end time.Time) Accounts {
a := Accounts{
Rows: []TotalView{},
Account: SourceState{Scope: ScopeUser, Source: accountUsageTable,
Note: "your own linked accounts, metered from each provider's own login; plan consumption, not a Hanzo charge"},
Hanzo: SourceState{Scope: ScopeOrg, Source: llmTable,
Note: "your org's Hanzo-routed inference; cost of record"},
}
return ""
if rows, ok := s.State.warehouse.AccountTotals(ctx, org, user, start, end); ok {
a.Account.Available = true
for _, t := range rows {
a.Rows = append(a.Rows, toTotalView(t))
}
}
if rows, ok := s.State.warehouse.HanzoTotals(ctx, org, start, end); ok {
a.Hanzo.Available = true
for _, t := range rows {
a.Rows = append(a.Rows, toTotalView(t))
}
}
return a
}
// buildSpendBlock reads the commerce rollup + ledger for org and rolls them into

Some files were not shown because too many files have changed in this diff Show More