Compare commits

...
Author SHA1 Message Date
antje 139bd734f0 deps: luxfi/consensus v1.36.2 -> v1.36.3 — the v1.36.2 tag was re-pushed
Cold builds fail sumdb verification against the moved tag (downloaded
eKzasq4O... vs sealed IbeWQF1w...). v1.36.3 is the immutable successor;
never re-tag a published version.
2026-07-17 01:26:35 -07:00
antje 6e24fcfd28 mirror: skip hidden files — AppleDouble forks pass the extension check
._foo.png is a mac resource fork, not a render; 700+ of them poisoned a
library within an hour of the mirror going live.
2026-07-17 01:15:16 -07:00
antje 9e7931e81c render: poll window matches the dispatch cap; engine recycles after each render
The 10m local history poll undercut the 4h startToCloseTimeout the dispatcher
grants — live renders (observed 8-70m) were marked failed while still sampling;
only the mirror later delivered them. renderWindow now matches the cap.

The engine leaks ~58GB per render. The handler signals a recycle after each
COMPLETED render (never on timeout — the engine may still be sampling and the
mirror rescues late finishes); the supervisor restarts on the signal.
2026-07-17 01:09:39 -07:00
494a6582c4 bench(finance): BenchmarkListUsage — the co-resident usage-read path (BUG 2) (#325)
The finance domain had no benchmark; this measures the read that replaced the
commerceinproc self-dispatch — finance.ListUsage over a per-org SQLite ledger,
at 100/1000/5000 seeded debits. Backs the reproducibility claim in the
hanzo-unified-tenant-cloud paper (1.25/10.5/61 ms). Also surfaces a real N+1:
store.Entries fetches postings per row the usage view never uses — a
postings-free read would cut this ~10x (follow-up).

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-17 01:06:20 -07:00
zandhanzo-dev b0254454c9 Money names its unit: Int -> Atto
There were two words for one idea. money.Amount.Minor() returns an integer at the
CURRENCY's decimals — money.USD declares 2, so cents. cloudmoney.Amount.Int()
returns an integer at 18 decimals — atto. Both are "the backing integer", neither
name says which, and they differ by 10^16.

That is not a style complaint. It is how

    Amount: cloudmoney.FromInt(u.Charge.Minor()), // exact 18-dp USD, no floor

got written, reviewed, and shipped. It type-checked (both sides *big.Int), it read
as "make an Amount from the integer", and it billed a $17.376 zen call as
$0.0000000000000017 until v1.801.44. The package documented the trap in prose —
"Take the decimal, never Amount.Minor()" — because a comment was the only place
the unit existed. Prose is not a type.

So name the unit, not the Go type: FromInt -> FromAtto, Int() -> Atto(),
IntString() -> AttoString(). FromCents/Cents already did this and were never
confused with anything. Now the units are visible at the call site, and the
mistake reads as one: FromAtto(x.Cents()) is obviously wrong where
FromInt(x.Minor()) was obviously fine. It no longer type-checks either —
FromAtto takes *big.Int, Cents() returns int64 — so the pairing that cost us the
money is now two independent kinds of error instead of zero.

Values are untouched: Atto/AttoString return exactly what Int/IntString did, so
the treasury ledger hash and every stored 18-decimal string are byte-identical.
No migration, no data change — only the names, and the compiler found every one
of them (45 sites; a regex could not have, because .Int() also belongs to big.Int
and decimal).

Zero regressions: the failing set is identical to origin/main.
2026-07-17 00:52:06 -07:00
zeekayandClaude Opus 4.8 c93ddf92bf build(deps): adopt stable luxfi/keys v1.4.1 (unbreak force-moved v1.4.0 checksum)
luxfi/keys v1.4.0 was force-moved on the remote (tree 5153d639→80a3745a),
producing a go.sum checksum mismatch that broke `go mod tidy`/`go build` and
the hanzoai/cloud image build in CI. Every keys tag v1.1.0..v1.4.0 was
force-moved with content changes; only v1.4.1 is byte-stable (local==remote
tree 17551acd). Adopt v1.4.1 as the single stable version. Its go.mod floors
the unified luxfi stack, so the transitive set moves forward (geth 1.17.12→
1.20.1, consensus 1.35.32→1.36.2, crypto/database/warp/zap/…), all within v1.

go.mod/go.sum only; diff vs origin/main is luxfi/* exclusively. Embedded
iam2 v0.14.0, apps.go, and concurrent commerce/agent work untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 00:50:13 -07:00
74478bb9a2 build: decomplect the cloud image — cloud compiles ONLY Go (#321)
The console SPA, agent-skills catalog, and native flags staticlib are each now
built by their OWN CI as a versioned immutable image and PULLED into the cloud
build, instead of rebuilding node+python+rust from scratch every release. The
console stage (cold npm install + full Next static export, cache-busted every
build) was the ~20-min long pole; it is now a registry pull.

- Dockerfile: console/skills/flagslib stages -> FROM ${CONSOLE_IMAGE}/
  ${SKILLS_IMAGE}/${FLAGS_IMAGE} prebuilt pulls; COPY sources updated
  (/dist, /catalog, /libhanzo_flags.a). Mirror golang+alpine bases, GOPRIVATE,
  and every RED gate (SQLCipher proof, modernc guard, cek frozen-format) are
  unchanged. Pins are ghcr.io so both buildx lanes pull directly; mirrored to
  registry.hanzo.ai (S3) for GET-flow consumers. release.yml still owns the
  cloud image + v* tags (it resolves CONSOLE_IMAGE to a fresh console-embed
  digest, as CONSOLE_CACHEBUST did).
- native/flags/Dockerfile: the cloud-flags artifact (rust -> scratch /libhanzo_flags.a).
- hanzo.yml: images: cloud-flags (distinct artifact, never a v* tag) + zccache
  RUSTC_WRAPPER on the native-flags gate (no-op unless the runner carries it).

Companion artifact publishers: hanzoai/console#(console-embed),
hanzoai/openapi#(agent-skills).

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-17 00:49:57 -07:00
hanzo-dev f76e714d37 refactor(flags): extract the Policy primitive — fold featuregate's waitlist into the flags engine
featureflags -> flags (Hanzo's Unleash-analog / hanzoai/flags): ONE runtime
decision engine, (Principal, context) -> verdict, evaluated in-process, hot.

Fold the launch-control waitlist off its DUPLICATE SQLite mode store
(clients/featuregate) onto the one engine:

  - a service's waitlist mode IS the platform switch waitlist.<svc>,
    evaluated through the same native evaluator as every other flag
  - the host->service registry folds into clients/flags (waitlist_store.go,
    mode column dropped — the mode is the switch)
  - featuregate.Enforce stays but as a CONSUMER of flags.WaitlistModeForHost
    (the decide), via an injectable Gate seam
  - /v1/featuregate/mode -> served by flags (the decide), same wire shape
  - /v1/admin/services -> an admin lens like /v1/admin/flags
  - per-user approval still reuses IAM (featuregate/approval.go, unchanged)
  - featuregate dropped from apps Wire() — it exposes only Enforce now

The flags package doc names the aspirational end-state: authz (access policy)
and entitlements (product-access policy) are the SAME (Principal,context)->
verdict shape and could COMPOSE this one engine. NOT touched here — flagged only.

Tests (green): featuregate RULE acceptance matrix + approvals; folded-registry
store (Seed/ServiceForHost/List/Upsert); flags env/board/parsers;
apps TestWireOrderMatchesFrozen. Native-engine + cek store tests need
CGO+libsqlcipher+libhanzo_flags (CI), unchanged.
2026-07-17 00:48:30 -07:00
hanzo-dev 53cd7c585d billing bridge: ask commerce what it gates, stop keeping a copy
The bridge's mint guard kept its own list of commerce's mint routes under the
comment 'kept in lockstep with api/billing/handlers.go'. A comment cannot hold
two lists together, and it hadn't: 10 paths here against the 16 commerce gates.
Six money-mint routes were outside this guard entirely.

Commerce now DECLARES its gated surface — middleware.Mint records what it gates,
MintRoutes() exports it (commerce v1.49.0) — so the guard reads that declaration
instead of copying it. A mint route added in commerce is covered here with nobody
remembering anything, which is the only version of this that survives contact
with a busy repo.

Registration is what populates the registry, so the guard registers commerce's
billing routes before reading it. The request-level assertion is unchanged and is
the part that matters: an ordinary org user's call must never REACH commerce,
because arriving at all means arriving with the admin service token that
satisfies MayMintMoney.

Proof it bites: allowlisting 'deposit' fails with the escalation itself —
'POST /v1/billing/deposit reached commerce carrying Bearer svc-tok'.

16/16 gated routes refused. All six bridge tests pass.
2026-07-17 00:41:23 -07:00
antjeandGitHub 8479418021 gpu: mirror local studio renders to the library (independent of claims) (#324)
Every 30s (its own ticker in the connect select loop), scan --studio-dir/output
recursively for image files new or changed since the last scan and POST each to
<studio-url>/v1/library/upload with the worker's bearer, tagged ?node=<identity>
and ?subpath=<subfolder>. So EVERY render lands in studio.hanzo.ai — including
ones produced OUTSIDE the job path: a graph hand-run on the node, or a render
that finished after its activity was reaped (the stranded-late-render class). The
mirror's independence from claims/activities is the point.

State is a tiny in-memory map[relpath]size that skips unchanged files; the studio
endpoint dedupes byte-identical uploads, so a re-scan after restart is cheap and
harmless. One log line per newly stored file; upload failures are summarized once
per scan and retried next tick (no 5xx spam). New --studio-url flag (default
https://studio.hanzo.ai, HANZO_STUDIO_UPLOAD_URL honored) so the mirror works
WITHOUT jobs. No new deps. Test: cli/gpu_mirror_test.go (new/changed uploaded,
unchanged skipped, node+subpath+bearer carried).
2026-07-17 00:20:24 -07:00
antje 309c830abb agent: v0.1.2 — pass a completion billing refusal (402) through, not a 502
Bump github.com/hanzoai/agent v0.1.1 -> v0.1.2 and have the in-process completer
return the agent typed hz.UpstreamError{Status,Body} on a non-2xx completion. The
round then passes a caller-facing 4xx (402 insufficient_balance, 429, 403) through
verbatim so a no-credit user sees "add credits", not an opaque "agent: completion"
502. Proven live: POST /v1/agent with a real hk- key ran the round end-to-end and
the completion returned 402. Dropped the now-unused clip() helper. Simplified
commerce_errorscope_test to the real invariant (typed 403 never flattened to 5xx)
now that commerce v1.48.10 honors status itself; the scope stays as the boundary.
2026-07-17 00:01:18 -07:00
antje f4ab7e2bd3 deps: commerce v1.48.5 -> v1.48.10 — free $0 tiers stay self-serve (paid-tier gate keys on price, not includedCreditUsd) 2026-07-16 22:38:59 -07:00
zandhanzo-dev 94873e3e0c MountAll's doc outlived Typed
It still said the MountFunc takes the app as `any` and that in-repo subsystems
recover it via Typed. Neither is true: MountFunc names *zip.App and Typed is
deleted. The app is handed to each mount as itself.
2026-07-16 22:24:17 -07:00
antje 0e377c52ad feat(sync,git): Gitea-native sync provider + webhook reject parity; fix LoadConfig flag panic
CHANGE 1 — the /v1/sync git provider drives GITEA (the one git store), not the
retired cloud embedded store:
- clients/sync/gitea.go: a Gitea REST + go-git client (GIT_ADMIN_TOKEN, GITEA_URL).
  Inbound = fast-forward-only go-git fetch(source) -> push(Gitea), so a diverged ref
  is a conflict, never overwritten (split-brain guard, now on Gitea). Outbound = a
  Gitea push-mirror (sync_on_commit) so Gitea itself propagates every commit. Fails
  closed when GIT_ADMIN_TOKEN is unset.
- git_provider.go Reconcile and sync_api.go patch/delete now compose those Gitea
  primitives; the cloud embedded seams (InboundGitSync/ImportGitRepo/EnsureGitMirror)
  are retired from the sync path. resolve() decision, loop guard, cursor idempotency,
  and hop limit are unchanged.

CHANGE 3 — webhook reject parity + a pre-existing root test panic:
- Wrap /v1/git/webhook and the slack(events,commands)/discord/teams/telegram inbound
  webhooks in cloud.Terminal so a bad-signature 401 / malformed 400 survives the
  commerce /v1 500-flatten (uniform reject codes, matching /v1/sync and
  /v1/connector/github/webhook).
- config.go LoadConfig: guard the process-global flag registration with a sync.Once,
  so a re-entrant LoadConfig (many test callers in one binary) no longer panics
  "flag redefined: enable".

CHANGE 2 (retire the embedded git server) is NOT done: it is still a live dependency.
cloneURL resolves to api.hanzo.ai/v1/git (cloud's OWN embedded server) and the
coding-agent orchestrator clones from uploadPack, pushes to receivePack, and reads the
store via VerifyRef. Deferred — migrate coding to Gitea first.

Tests (CGO_ENABLED=0): clients/sync + clients/integrations green; new
clients/sync/gitea_test.go proves reconcile acts on Gitea and the fast-forward guard.
2026-07-16 22:20:41 -07:00
hanzo-dev 66e4999a44 Merge: Mount takes *zip.App — delete the Typed shim and its 85 wrappers 2026-07-16 21:57:01 -07:00
antje 1cf505ebd1 edge: scope commerce error envelope to its own routes (unblock release smoke — 14 endpoints 500→4xx)
commercemid.ErrorHandlerJSON() was installed as a /v1 GROUP middleware, but fiber
matches group middleware by PREFIX, not by the handle a route registered on. So on
the shared /v1 it wrapped EVERY subsystem mounted after commerce (projects, agents,
wallets, functions, integrations, marketplace, team, s3, analytics, knowledge,
automations, deploy, billing) and flattened their typed zip.HTTPError (403 "X-Org-Id
required", 400, …) into a blanket 500 — the store envelope always renders 500. The
authenticated release smoke (#322) correctly fails on 5xx, so this blocked every
release since it landed.

Fix stays cloud-side (no commerce dep bump, no money-path churn): commerceErrorScope
guards the envelope by commercePrefixes, so it stays on commerce and every other
subsystem renders its own status via zip default. Pre-commerce subsystems (kms,
o11y) already did; this makes the post-commerce ones match. Verified with the REAL
commerce middleware: /v1/projects 403 (was 500), /v1/store/current keeps the 500
envelope. Regression test added.
2026-07-16 21:50:00 -07:00
zandhanzo-dev 25fcaa4351 Mount takes *zip.App: delete Typed and the 85 wrappers
MountFunc took `app any` and cloud.Typed asserted it back to *zip.App on every
mount — a runtime check doing the type system's job, with a failure branch that
could not fire because the only value ever passed is a *zip.App. Every subsystem
paid for it: 85 call sites read cloud.Typed(x.Mount) instead of x.Mount.

The reason given was circular. cloud said `any` was load-bearing because an
external module (licensing) exposed func(any, Deps) error; licensing said it used
`any` to avoid an import cycle in pkg/cloud. Each pointed at the other, and the
cycle cannot exist: this package already imports zip (build.go), and zip does not
import cloud. The `any` was justified by nothing.

So name the type. MountFunc is func(*zip.App, Deps) error — what every subsystem
already exported and what licensing's own doc claimed all along. Typed is deleted,
the 85 wrappers are gone, and the three in-repo mounts that hand-rolled the same
assertion (mountZen, mountMetrics, MountO11y) just take the app.

The compiler immediately found what the `any` had been hiding: four mounts still
shaped func(any, ...), one of them across a module boundary. That is the point —
a signature drift is now a build failure instead of a runtime error nobody would
see until a subsystem mounted.

Also here, because the same rip surfaced them:

  - iam: v1.31.27-0.20260716191958-4400762928a2 -> v1.31.28, and the replace
    pinning a second pseudo-version is dropped. The required pseudo-version named
    a commit that no longer exists (it was rebased away), so `go mod tidy` could
    not resolve it; v1.31.28 is a real tag and a strict superset of what the
    replace pointed at. A version, not a coordinate, and no replace.
  - licensing -> v0.1.5, which is where the typed Mount ships.
  - cloud.OrgConfig is aliased next to LicenseEntitlement. Both are named by
    CommerceClient's methods, but only one was exported, so the exported
    interface could not be implemented from outside without reaching into
    cloud/types — an omission, not a boundary.
  - build_registration_test.go tested Typed and nothing else. "Recovers the
    *zip.App" proved an adapter passed through its argument; "fails closed on a
    wrong type" cannot be compiled now. What is left is the assertion that a
    subsystem signature IS a MountFunc — which the build checks.

No regressions: the failing set is byte-identical to origin/main (11 pre-existing
TestAudit_*).
2026-07-16 21:47:19 -07:00
zeekay f49d283282 build(iam2): bump embedded iam2 v0.1.1 → v0.14.0 (parity-complete surface)
The CLOUD_IAM_IMPL=iam2 identity fold (clients/iam2, identitySpec) was pinned
to iam2 v0.1.1 — an early cut missing the whole console-parity surface. Bump
to v0.14.0 so the embed actually serves what hanzo.id needs:

  - RFC/IETF surface (HIP-0111): OAuth2 code+PKCE/refresh/client_credentials/
    password, RFC 8693 token-exchange, 7662 introspection, 7009 revocation,
    8414 AS-metadata, OIDC UserInfo, SCIM 2.0 Users
  - Casdoor verb-alias compat (transitional cutover bridge) for every verb the
    live console/gateway hard-code: users/orgs/apps/providers/roles/projects
  - operator bootstrap upsert (IAM CR reconciliation), TOTP MFA enrollment,
    organization-scoped projects (ScopeSwitcher)

No wiring change — main's identitySpec + co-mingle iam2server.Mount(app, db)
compile unchanged against the v0.14.0 API. zip already at v1.8.3.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 21:43:03 -07:00
antje f20e7679ef feat(connector,sync): /v1/connector/github/webhook namespace + reject paths survive the commerce /v1 500-flatten
CHANGE A — first-party vs external route naming. Rename /v1/github-webhook ->
/v1/connector/github/webhook, opening the external-platform namespace
/v1/connector/<provider>/webhook (github now; gitlab/others are sibling literal
routes later, each with its own signature scheme). /v1/git/webhook (first-party
Hanzo Git) and /v1/sync (the bridge) are unchanged. No live consumer breaks:
the GitHub App isn't created yet.

CHANGE B — 500 -> real 4xx on the reject paths. mountCommerce registers
commerce's ErrorHandlerJSON on app.Group("/v1"); that filter rewrites ANY error
a downstream /v1 handler PROPAGATES into a hardcoded HTTP 500. Every /v1
subsystem mounted after commerce is wrapped the same way -- git, sync AND
integrations alike. git does NOT escape it: its isolated tests read 401 only
because they don't co-mount commerce (its bad-sig path is never exercised in
prod, so the flatten went unseen). Add cloud.Terminal, which writes a returned
*zip.HTTPError in-band and returns nil, so the filter's c.Next() sees nil and
has nothing to flatten. Wrap /v1/sync (all verbs) and the connector webhook.
sync no-principal is now 401 (was 403) -- an authentication failure, matching
the webhook's bad-sig 401. The commerce filter is left untouched.

Tests reproduce the /v1 flatten filter and assert: sync unauth -> 401, connector
bad-sig -> 401, malformed body -> 400; connector resolves at the new path and
the old /v1/github-webhook 404s.
2026-07-16 21:32:46 -07:00
antje c93fd2c4dd edge: /v1/agent self-meters (fix 503 balance_unavailable on the agent orchestrator)
The global BillingGate priced /v1/agent/* at a flat 1c via a dead legacy
bot-reverse-proxy rule in DefaultPrice, so every agent request was gated: the
unauthenticated preset/conversation reads hit the balance check, failed closed,
and 503d before the orchestrator handler ever ran. The route was mounted and
winning precedence the whole time — the gate short-circuited ahead of it.

/v1/agent is self-metered: the reads are free and POST /v1/agent bills through
the /v1/chat/completions it runs in-process (gated + metered downstream), so the
edge must price it 0 or double-bill. Drop the legacy branch (and its now-unused
cloudEdgePriceCents const); price /v1/agent and /v1/agent/* at 0. Tests updated.
2026-07-16 21:06:09 -07:00
hanzo-dev 2f4095e68a Merge feat/openapi-spec: /v1/openapi.json generated from the live router
The spec IS the router, not a description of it: apps.Wire() -> MountAll ->
app.Fiber().GetRoutes(). 983 operations / 692 paths / 109 products, served beside
/zap so ZAP and OpenAPI are two projections of one route table rather than two
sources that can disagree.

A drift test proves the bijection and was proved to fire; it already caught
/v1/pricing-policy and /v1/pricing/policy collapsing onto one operationId.

# Conflicts:
#	serve.go
2026-07-16 20:42:42 -07:00
6233b3c805 fix(billing,smoke): usage reads the co-resident ledger (not a self-dispatch) + a real authenticated release smoke (#322)
A live authenticated smoke surfaced two production bugs; this fixes both and
adds the durable smoke that now guards every release.

BUG — /v1/billing/usage 500 for a valid caller. usage() proxied
"/v1/billing/usage" through commerceinproc, which re-dispatches BY PATH.
Commerce's own billing routes are behind //go:build cloud and never compiled
here, so the ONLY registration of that path is usage() itself — the S2S hop
re-entered the handler, which self-answered "sign in to view billing". This is
the SAME defect balance() was already fixed for. usage() now reads the usage
ledger DIRECTLY from cloud's own finance ledger (finance.ListUsage → the
wallet→revenue debits RecordUsage wrote), off the self-dispatching hop;
split-deploy falls back to the commerce S2S read, unchanged.

BUG — the balance gate 402'd read-only GETs (fixed in hanzoai/ai, pinned via
the go.mod bump). A $0-balance org could not VIEW its own resources. Fixed in
the ai module's BalanceGateFilter — reads never spend, so GET/HEAD/OPTIONS are
exempt; only writes/metered POSTs gate on balance.

SMOKE — cmd/smoke: a durable, authenticated per-subsystem prober. One
side-effect-free read per core subsystem; a read that 402s (balance gate) or
5xx-es (crash) fails the release. Baked into the image (Dockerfile) and wired
into release.yml as the functional gate after the boot check, so a release can
never ship with chat/billing/projects/kms/... down. The smoke token is
KMS/secret-sourced (never hardcoded); absent → the anonymous matrix still gates
public/authed and catches every 402-on-read / 5xx.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-16 20:35:56 -07:00
hanzo-dev 52acb303c2 cli: multi-identity credential store with active-pointer switch
Store every logged-in identity in ~/.hanzo/identities.json keyed by a stable
<owner>/<name> key (admin/z, hanzo/z), with an Active pointer; mirror the active
identity into credentials.json so every legacy single-file reader is unchanged.
A second login as a different owner for the same email (privilege separation:
hanzo-admin-guard vs hanzo-console) is stored beside the first, not over it.

New: auth list (whoami --all), auth switch <owner|owner/name> (alias use),
logout [<owner>]. Token refresh writes through the store (SaveActive) so the
active identity stays fresh after rotation. All files stay mode 0600.

One credential store, active-pointer switch, backward compatible.
2026-07-16 20:20:53 -07:00
hanzo-dev 951703f486 dedup: fold hanzoai/o11y module wildcard into the one o11y subsystem
apps.Wire() had TWO {Name:"o11y"} entries: the in-repo read plane
(o11y.MountO11y) and the external hanzoai/o11y module wildcard
(cloud.Typed(o11ymod.Mount)). Fold the module wildcard in as the TERMINAL
sub-mount inside o11y.MountO11y (registered after every specific /v1/o11y/*
route, so Fiber in-order match still gives them precedence), delete the 2nd
Wire entry and the now-unused o11ymod import -> ONE o11y spec.

/v1/o11y/health is preserved exactly: the merged entry keeps OwnsHealth=false,
so the generic always-ok route is registered before MountAll (ahead of the
wildcard) — byte-identical to when the module co-entry, also OwnsHealth=false,
triggered it. Frozen wire order updated (two co-owned rows -> one); the
no-duplicate test no longer needs an o11y exemption.
2026-07-16 20:17:18 -07:00
hanzo-dev 7022064723 dedup: delete dead clients/session
Zero importers, no cmd/session, absent from apps.Wire(); its /v1/code/sessions
surface was never mounted (dark). Removes session.go, store.go, session_test.go.
2026-07-16 20:10:06 -07:00
hanzo-dev 8dabc5c469 fix(cloud): serve /v1/iam/* — the org-scoped edge to Hanzo IAM
The one-binary console (console.hanzo.ai — cloud serves console's static export)
reads org members + projects at /v1/iam/*, but cloud 404'd those (IAM isn't
folded in-process yet) and the old /org/iam BFF proxy is pruned from the static
export. So the browser got the SPA shell (HTTP 200 HTML), which the client
surfaced as "Request failed (HTTP 200)" — the Platform page died on "Could not
load".

iam_edge.go adds an org-scoped reverse edge at /v1/iam/* → the standalone IAM,
mounted when IAM is NOT folded in-process (else that subsystem owns the path — no
double-mount). The org is PINNED to the caller's validated, server-minted
X-Org-Id (never a raw client header) — load-bearing, since IAM's own authz is
permissive on the org-keyed routes, so without the pin one tenant could read
another's projects. A super admin may cross; a tenant may not; writes require an
org admin and must carry the caller's own org. Shares the ONE IAM identity
(iamHost/iamCred) with the API-key resolver (DRY).

Verified: 7 gate tests (pin, cross-tenant refuse, super-cross, allow-list, 401,
write-gate, own predicate) + go build clean.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-16 20:07:12 -07:00
zandGitHub ac65cf4ec3 chore(deps): bump hanzoai/tasks → v1.51.1 (fix api.hanzo.ai concurrent-map-writes crash)
Fixes the live recurring tasksd concurrent-map-writes fatal (hanzoai/tasks#18, v1.51.1). Build green; the failing 'Test' check is the pre-existing repo-wide LoadConfig flag-redefine panic (fails on main + all branches), unrelated to this go.mod-only bump.
2026-07-16 20:02:07 -07:00
hanzo-dev 5e06e29de5 Merge: tenant apps are App CRs — the last writer of the Service kind
clients/platform was the only thing still minting Service CRs; every other
declarer is already App. A role-less App dispatches to the operator's service
profile, the same reconcile the Service kind ran, so a tenant workload carries
over verbatim.

Existing tenant CRs still resolve (App first, Service second). A redeploy patches
the kind it IS rather than minting a twin — two CRs on one name is the
commerce-admin ownerRef flap, not a migration. Teardown deletes BOTH kinds:
either alone rebuilds the app the tenant just deleted, still billing.

113 tests pass, 7 new.
2026-07-16 19:40:04 -07:00
hanzo-dev 095e1cce4e platform: tenant apps are App CRs — the last writer of the Service kind
clients/platform was the only thing still minting Service CRs. Every other
declarer is already App: universe git has zero hanzo.ai/v1 kind:Service, the
operator calls App "the sole workload reconciler for the collapsed fleet", and 69
of the 80 live CRs are Apps. A tenant app was the exception for no reason — a
role-less App dispatches to the operator's service profile (controllers/app.rs
`classify("") => Dispatch::Service`), which is the same reconcile the Service kind
ran, so an App carries a tenant workload verbatim.

Nothing mints a new Service CR after this. What already exists still resolves:
reads, patches, scales and deletes walk crGVRs() — App first, Service second — so
the 3 live tenant CRs written before this keep working untouched.

A redeploy of a pre-collapse app patches the kind it IS rather than declaring an
App twin. Two CRs claiming one name is not a migration, it is the commerce-admin
flap: both kinds materialize the same children through the same materializer under
one field manager, so the Deployment's ownerRef just flips between them.

Teardown deletes BOTH kinds. Either kind alone re-materializes the Deployment, so
removing only the one that resolves first would rebuild the app the tenant just
deleted — running, and still billing, minutes after a successful delete.

Sequencing (deploy order matters): operator v0.7.7 carries the Claim guard that
makes App the deterministic owner when both kinds claim a name. This is safe
before it — the no-twin rule means a colliding pair is never created here — but
the guard is what makes an existing collision safe to clean up.

REMOVABLE once no Service CR remains in any tenant namespace: drop servicesGVR
from crGVRs() and the delete-both, and this file is App-only.

Tests: 113 pass; 7 new (kind is App, no twin on legacy redeploy, delete removes
both kinds, idempotent delete, resolution order, absence honest). The 10 tests
that asserted a Service CR is written now assert the kind we write.
TestMigrateOverLegacyPlatformApps fails identically on pristine main (sqlcipher
codec, environmental).
2026-07-16 19:39:56 -07:00
antje bffe597ff0 apps: frozen wire list — add sync entry (main added /v1/sync to Wire; frozen drifted, breaking TestWireOrderMatchesFrozen); fix agent route comment 2026-07-16 19:38:00 -07:00
hanzo-dev 601d5660b7 Merge: paas reads the kind the fleet runs on, and does not fight the git declarer
The PaaS control plane serves platform.hanzo.ai and read services only: 69 App CRs
run in the scanned namespaces against 7 Service CRs, so the SUPERADMIN drift board
rendered 7 rows for a 69-app fleet — 1 in production — and /v1/paas/health probed
the Service CRD, found it served, and reported ok over a blind board.

Reads now walk App first, Service second, deduped by name. Writes refuse a
git-declared App (Hanzo CD syncs it with selfHeal, so a patch is reverted on the
next sync) and name the file to commit to; a Service CR still patches.

35 tests pass, 0 fail.
2026-07-16 19:30:16 -07:00
hanzo-devandz 8bf2dda4d1 feat(shard): in-binary org→owner shard router for horizontal writer scale
Lifts the unified binary off replicas:1 on DigitalOcean (RWO-only block
storage, no RWX) WITHOUT any shared volume. Each org is pinned by rendezvous
hashing (ha.Owner over the static CLOUD_PEERS ring) to exactly one owner pod;
a request whose org this pod does not own is transparently forwarded to the
owner, so every per-org SQLite store (KMS, finance, and every org-keyed store),
audit append, per-org rate ceiling, and prepaid billing debit runs on ONE pod.

THE INVARIANT — no two pods ever write one tenant's SQLite file — is upheld by
two independent guarantees that compose: per-pod RWO PVC (different physical
files per pod) + org→owner routing (all of an org's writes on one pod).

- shardrouter.go: the middleware. Runs immediately after SanitizeIdentity so it
  keys on the VALIDATED, server-minted X-Org-Id (never a client header), hashes
  the SAME injective SanitizeOrg slug the on-disk path uses (routing key ≡ file
  key), and forwards via a fasthttp streaming proxy (SSE/chat pass through — a
  streaming chat still carries a per-org billing debit, so it routes too) with
  dial-only retry across an owner's roll gap and a 421 loop-guard on divergence.
  Static identical membership rules out dual-owner and routing loops.
- config.go: CLOUD_PEERS (id@addr ring) + POD_NAME (self). Sharding auto-enables
  only when >1 peer; Validate fails closed if self is not in the ring or if
  embedded iam (non-shardable process-local sessions) is co-enabled.
- serve.go: wires the middleware after IdentityMiddleware; shard-aware boot log.
- audit_serve.go: per-shard audit is automatic on the per-pod PVC; stamp the
  shard id on the AU-9 checkpoint stream so the tail-truncation monitor tracks
  N heads. writerpin.SingleWriter is correct PER SHARD (each pod sole writer of
  its orgs); the writer lease is pod-local under per-pod PVC and stays off.

No-op (byte-identical to today) when CLOUD_PEERS names ≤1 pod. N pinned at 3;
rebalance-on-N-change (a tenant-file move) is the documented follow-up.

Tests prove: exactly one owner per org (deterministic, total, evenly
distributed); all pods agree (no dual-writer); owned served locally, unowned
forwarded to the owner (local chain never runs, hop header set, response
streamed); org-less served locally; loop-guard 421; boot-gate fail-closed.
2026-07-16 19:28:27 -07:00
hanzo-dev db226b321b Merge: bill zen in the credit unit — FromInt(Minor()) understated every debit by 10^16 2026-07-16 19:26:12 -07:00
hanzo-dev f24596453e Bill zen in the credit unit, not the currency's minor unit
zen prices every SKU as an exact 18-dp value tagged money.USD. money.USD
declares 2 decimals, so Amount.Minor() rescales that value to CENTS, while
cloudmoney.FromInt reads its argument as 18-dp. Composing the two divided
every zen debit by 10^16: a $17.376 charge debited $0.000000000000001738,
and any charge under half a cent folded to a zero that metering.Record drops
before it reaches the ledger — no debit row, call served free.

The spend gate read the same composition. FromInt(est.Minor()).Cents() is
always 0, and AuthorizeVerdict only compares balance against size when
AmountCents > 0, so the size check was dead code and any org with a positive
balance could draw a request of any size. The dust debits never accumulated,
so the cap could not trip either.

Route the seam through one conversion. credit() carries the exact decimal
across and changes only the minor-unit convention — no rescale, no rounding,
no factor — because the decimal is the value and a currency's Decimals is a
rendering convention. cloudmoney.FromDecimal is the typed counterpart of
ParseUSD that makes this expressible without a string round-trip. nano()
takes the typed Amount rather than a bare *big.Int, so a cents integer can no
longer reach the warehouse fold.

The tests asserted Amount.Int() against Charge.Minor() — the same integer on
both sides — so they proved a round-trip and were blind to the unit. They now
assert against known dollar values built through a different constructor: a
$17.376 charge debits $17.376. Reintroducing Minor()-into-FromInt fails all
of them, including a gate test that admits an over-cap request.
2026-07-16 19:24:29 -07:00
hanzo-dev 185cca68f2 Forward only the billing paths the console reads
The /v1/billing/* bridge attaches the commerce service token, and that token
satisfies commerce's MayMintMoney. The subpath was checked for traversal but
not against a set, so POST /v1/billing/deposit forwarded a mint for any
signed-in user — pinned to their own subject, which aims the mint rather than
stopping it. Commerce closed this on its direct path the day after the bridge
reopened it, and wider: the bridge needs only a validated principal, never an
admin bit.

The forwardable set is now a per-method table, consulted before the token is
attached, 404 on anything else. Per-method because payouts is a GET read and a
POST mint on one path — a method-blind set hands the mint to every reader.

Nothing reachable today: commerce's mint is not compiled into this binary and
commerce.hanzo.svc resolves here. But devnet runs a real mint-capable commerce
and points cloud-api at it under a second name for the same thing; unifying
those names would arm it. The gate should exist before that cleanup does.
2026-07-16 18:54:20 -07:00
blue db4801d787 Gate the billing bridge on an allowlist: the service token is authorization
/v1/billing/* forwarded any subpath to commerce carrying the admin
COMMERCE_SERVICE_TOKEN, validated only against path traversal. Forwarding IS
authorization there: commerce gates money-mint on
MayMintMoney = IsServiceToken || IsSuperAdmin, so every forwarded path ran with
platform authority. Commerce 403s an org admin who POSTs /v1/billing/deposit
directly; through this bridge the same person was handed the platform's own
credential, and the subject-pinning aimed the credit at their own account.
Pinning is an IDOR control, not an authority control.

Add billingForwardable: a per-method allowlist of the endpoints the console
actually calls, enforced before the token is attached. An unlisted path 404s.
Allowlist, not denylist — a mint route commerce adds tomorrow is unreachable
with no change here. GET and POST are separate sets because `payouts` is a read
AND a mint-gated write; one method-blind set would hand the mint to every reader.
The POST set holds nothing that creates balance from a client-named amount.

Not exploitable in the current topology: commerce is co-resident in cloud, whose
build never compiles mount.go (//go:build cloud), so no api.Route billing bundle
is reachable and the forward loops back to this bridge. It is one COMMERCE_URL
away from live — devnet already runs a standalone commerce.

Tests: an ordinary org user's deposit/credit/refund/credit-grants/husd/allotment
now never reach commerce; the console's 12 real calls still forward; the store
bridge still cannot tunnel into billing.
2026-07-16 18:36:07 -07:00
hanzo-dev 337c159db0 Embed zen v1.4.0 so the prompt-cache fix reaches production
zen v1.4.0 bills the prompt cache: cache_read was priced in the catalog,
published in /v1/models, and never charged. cloud pinned v1.3.11, so the
fix could not reach api.hanzo.ai, where the traffic lands.

The module graph cannot move: zen's own go.mod is byte-identical between
v1.3.11 and v1.4.0 (same sha256), and cloud holds exactly one zen edge.

NOTE: this is necessary but NOT sufficient. hmoney.Minor() returns cents
while cloudmoney.FromInt expects atto, so apps/zen.go understates every
debit by 1e16 and the spend gate reads AmountCents 0. Correct pricing is
still zeroed downstream. Tracked separately; that fix is what makes cache
billing real.
2026-07-16 18:21:12 -07:00
hanzo-dev 20e7591f53 openapi: serve GET /v1/openapi.json generated from the live router
The route table gets a third projection. /zap replays the /v1 handlers, the
console renders them, and GET /v1/openapi.json now describes them — all read
from the ONE router after MountAll, so none can drift and none holds a second
copy. There is no checked-in spec file and no second registry.

openapi.Live(app) reads app.Fiber().GetRoutes(true) (fiber's own filter drops
Use() middleware); every other function is pure over that []Route. Each
operation is tagged with its product — the first path segment after /v1/ —
so a CLI can build `hanzo <product> <resource> <verb>` with no judgment.

Reading the LIVE router is the only total source: POST /v1/kms/auth/login is
registered as Group("/v1/kms/auth").Post("/login") and no grep can find it,
and the route set is a function of deployment config, so the document varies
per deployment — correctly. Unauthenticated: it grants no capability, every
route it names stays auth-gated, and `hanzo --help` must build its tree
before login.

The drift guard (cmd/cloud/openapi_test.go) is a bijection over the fully
mounted apps.Wire(): 983 operations / 692 paths / 109 products, every live
route present, no operation invented. Shown to fail on a broken translation
(353 routes reported missing) before being restored.

Honest boundaries, asserted rather than papered over:
  - No schemas. The router holds func(*zip.Ctx) error; the request type is a
    local inside the handler (var req secretPutRequest; json.Unmarshal(...)),
    unreachable by reflection. cloud.Handle[S] is generic over the SERVICE,
    not the payload. The path to schemas is zip's typed ops, which today
    number zero — which is why zip's own generator emits nothing here.
  - No responses block. OpenAPI 3.1 makes it optional; fabricating 200/ok on
    ~900 routes would assert what nothing knows.
  - HEAD/CONNECT excluded — forced, not taste. CONNECT has no OpenAPI field;
    fiber auto-generates HEAD in startupProcess(), so including it would make
    the document depend on lifecycle stage.
  - Catch-alls are opaque: POST /v1/billing/deposit is not a route here.

Corrects LLM.md, which the code contradicted: Wire() does exist (apps.go:188),
MountAll does not sort, and byte-identical patterns MERGE rather than panic.
A high handler count is not a collision — app.Post(path, mw1, mw2, mw3, h) is
one registration with four handlers (apps/commerce.go:151), and 34 live routes
are that shape, so the generator never reads the count.
2026-07-16 18:14:23 -07:00
antje 9d5d51936a cloud: mount hanzoai/agent /v1/agent (no-shim); delete dead clients/chat
clients/agent adapter injects the ai completion (in-process, billed) + tools.Default()
into github.com/hanzoai/agent; POST /v1/agent live path. Dispatch resolves the caller
via the ONE canonical tools.PrincipalFrom — no reconstructed principal. Builds green
on the v1.801.35 base. DEPLOY-BLOCKED: go.mod uses a local replace for hanzoai/agent
(CI needs a fetchable release + the local module-cache shallow-clone bug resolved).
2026-07-16 18:00:50 -07:00
hanzo-dev c5cefc39a6 Bill the prompt cache in production: zen v1.3.11 -> v1.4.0
cache_read was priced in the catalog and published in /v1/models, but never
charged: on an openai upstream every cached input token billed at the full
`in` rate, and on an anthropic one it left the bill entirely. zen v1.4.0
normalizes both dialects into one tally over three disjoint classes
(fresh + cached + cacheWrite == the whole prompt) and derives once per
response. cloud embeds zen via apps/zen.go, so the fix is inert at
api.hanzo.ai until this pin moves.

zen v1.4.0's go.mod is byte-identical to v1.3.11's, so the require edit and
the two go.sum lines are the whole change: no transitive pin moves, and
nothing else in the graph names zen.
2026-07-16 17:49:16 -07:00
hanzo-dev 7f6b3a4cfe Merge: fix the zen margin test's decimal + money types (unblocks CI on main) 2026-07-16 17:48:36 -07:00
z e538ae0b7f apps: price the zen test with the decimal money actually uses
The zen margin test never compiled, so CI has been red on main since it landed:
it parsed prices with shopspring/decimal and handed the result to hmoney.New,
which takes hanzoai/decimal. Two identically-named types, one of them wrong.

    vet: cannot use d (struct type "github.com/shopspring/decimal".Decimal)
         as "github.com/hanzoai/decimal".Decimal value in argument to hmoney.New

Money has exactly one decimal. A second one that merely LOOKS like it is how a
price silently becomes a different number, which is why the compiler is right to
refuse. Parse with hanzoai/decimal (decimal.Parse — the same call zen itself
prices with); this was the only file in cloud importing shopspring.

Read the debit through Amount.Int(), not Amount.Minor(). The test held two money
types at once: zen's hanzoai/money.Amount (Charge/Cost) has Minor(), but
meterUsage returns cloud's clients/money.Amount, whose accessor is Int() — it
wraps Minor() and returns the identical *big.Int, so the assertions are unchanged.

The tests themselves are worth keeping: they prove the debit is the retail Charge
and never the upstream COGS, that a 3x-margin tier collects 3x, and that a
sub-cent call does not floor to zero. They just could not run.

Not run locally: ./apps links a prebuilt Rust staticlib (libhanzo_flags.a) that
is built in CI, not here. vet — the step CI actually failed — passes, and the
package builds.
2026-07-16 17:48:26 -07:00
hanzo-dev 2593b1b18e GOPRIVATE names the namespace that is actually private
GOPRIVATE named zap-proto/*, which is public -- all 55 repos, all 6 modules
cloud needs served anonymously from the public proxy. The namespace that is
private went unnamed: github.com/hanzoai/* (467 private repos incl. ai,
account, commerce, orm, xorm, beego, csqlite). It resolved only by falling
through GOPROXY's direct fallback, and passed checksums only because go.sum
already pins everything.

containment.yml then set GOSUMDB=off to compensate. GOSUMDB does not scope to
a namespace: that disabled checksum verification for every module in the
build, public ones included, in the image that handles payments -- breaking
the invariant the Dockerfile three files away states and honors.

Public modules keep proxy and sumdb immutability; private ones go direct and
authenticated. Verified under the CI and Dockerfile env with -mod=readonly and
sumdb on: build 0, go mod verify all verified.
2026-07-16 17:33:16 -07:00
hanzo-devandantje 40ca6a48a8 feat(sync): universal /v1/sync engine — GitHub/GitLab ⇆ native Hanzo Git
One reconcile loop, one place a sync happens. clients/sync owns the Sync
record (source/target/direction/trigger/cursor), a per-org sqlite `sync`
table, /v1/sync CRUD + /v1/sync/:id/run, and a kind→Provider registry with a
single Reconcile(sync,event) contract. Git is the first provider, composing the
existing git object-plane seams (InboundGitSync inbound, ImportGitRepo reconcile,
EnsureGitMirror outbound) — no second copy of any git op.

Triggers resolve to Syncs and enqueue the engine (cloud.Sync), never sync
directly: the GitHub App webhook now serves the path it actually fires at
(/v1/github-webhook, was a prod 404 at /v1/integrations/github/webhook) and hands
the verified push to the engine; the Gitea push webhook gains a loop guard
(skip pusher == GIT_SYNC_ACTOR). Loops break on the engine cursor (identical
fingerprints are a no-op) + actor guard; chained propagation (a sync's target is
another's source) is bounded by a hop limit.

Seams (root cloud): SyncFunc + RegisterSync/Sync, GitMirrorController +
EnsureGitMirror. Fail-closed on unmounted engine / missing secret. Tests
(CGO_ENABLED=0): engine loop-guard/idempotency/chain/hop-limit, git resolve,
CRUD+patch+run, webhook signature/isolation/enqueue, gitea loop guard.
2026-07-16 17:26:06 -07:00
hanzo-dev 2cc52957b4 build: GOPRIVATE names the namespace that is actually private
GOPRIVATE listed github.com/zap-proto/*. Every zap-proto repo is public — all 55
of them — and every zap-proto module this build needs is served by the public
proxy anonymously. It was never the reason anything resolved direct.

The namespace that IS private went unnamed: github.com/hanzoai/* — ai, account,
commerce, orm, xorm, beego, csqlite and ~30 more. Those only ever built by
falling through GOPROXY's `direct` fallback after the proxy 404'd them, and only
kept passing the checksum step because go.sum already pins them, so no sumdb
lookup happens. It worked by accident, one added dependency away from failing.

containment.yml compensated for that unnamed namespace with GOSUMDB=off, which
does not scope to hanzoai — it disables checksum verification for EVERY module in
the build, the public majority included. The Dockerfile's comment reasoned the
same way inverted ("hanzoai/* and luxfi/* are PUBLIC ... only zap-proto/* is
exempt"); hanzoai/* is largely private and luxfi/* (all 37 deps here) is public.

Naming github.com/hanzoai/* is what the off switch was standing in for. GOPRIVATE
implies GONOPROXY+GONOSUMDB for exactly that namespace, so the private modules go
direct+authenticated and skip the sumdb that cannot see them, while zap-proto and
luxfi keep the public proxy + checksum db — the immutable hashes that make a
force-moved tag unable to break or poison the build. GONOSUMDB and GOSUMDB=off
are dropped: scoped by GOPRIVATE, they are redundant, and blanket-off is a
supply-chain regression in a money image.

hanzoiam/* is not listed: a74b7de reverted the scim/saml/ldap embed, so nothing
in go.mod requires it. It goes back when the modules resolve, named as private.

Verified under the exact CI/Dockerfile env (GOPRIVATE=github.com/hanzoai/* only,
GOSUMDB=sum.golang.org, GOFLAGS=-mod=readonly): build exit 0, vet exit 0,
`go mod verify` = all modules verified, and `go mod download` resolves both a
public module (zap-proto/zip) and a private one (hanzoai/ai).
2026-07-16 17:21:31 -07:00
antje 28aaa39d56 deps: ai v1.816.0 -> v1.817.0 — global auto-routing switch is the '*' settings row, not ROUTER_ENABLED env (config-as-Base)
Runtime routing policy now sources from the GlobalDefaultOwner OrgSettings
row (admin.hanzo.ai-editable SQLite), env demoted to deprecated fallback.
Per-org override unchanged.
2026-07-16 17:16:33 -07:00
hanzo-dev dfe30a3883 paas: read the kind the fleet runs on, and don't fight the git declarer
The PaaS control plane serves platform.hanzo.ai and read `services` only. The
fleet collapsed onto `kind: App` and this reader never followed: 69 App CRs run
across the scanned namespaces against 7 Service CRs, so the SUPERADMIN drift
board rendered 7 rows for a 69-app fleet — 1 row in `hanzo`, production, and that
row is a duplicate CR. It was not an error anyone could see. The board looked
plausible and was blind to 68 of 69 services; every read of an App-declared
service 404'd; and /v1/paas/health probed the Service CRD, found it served, and
reported ok — status theater over a blind board. clients/deploy already solved
this with an App-first/Service-second read order; this gives the same order to
the reader that needed it.

Both kinds are listed and deduped by name: one workload is one row even when an
App CR and a Service CR both claim the name, because a Deployment has one
controller ownerRef and the operator's Claim guard gives it to the App.

The write path is the harder half. Hanzo CD syncs 68 of the 69 App CRs from
universe `infra/k8s/operator/crs/` with selfHeal on, so patching an App CR here
is reverted on the next sync — the deploy would report success and silently roll
back. That is worse than refusing, so deploy and release now resolve which kind
holds the workload and refuse a git-declared one, naming the file to commit to.
A Service CR has no git declarer (cloud and kubectl write them directly), so it
still patches exactly as before — the tenant plane and the untransitioned CRs are
untouched. release.go's "no ArgoCD" claim was true when written and is not now.

This does not restore an admin deploy button for the git-declared fleet. Under
GitOps that button belongs in git, and ImageUpdate already names that seam
(registry→git→cluster). Whether /v1/paas/deploy should commit to universe or be
retired is a CTO call, not one to make silently inside a read fix.

Tests: 35 pass, 0 fail, including the pre-existing release suite. The fleet-sees-
App-CRs case is seeded to the shape of the real `hanzo` namespace.
2026-07-16 17:12:40 -07:00
antje a74b7de5c4 revert(iam2): back out scim/saml/ldap embed — hanzoiam repos do not resolve
Reverts e4b3c88f. go.mod pinned github.com/hanzoiam/{ldap,saml,scim}, but none
of the three repos exist: https, ssh, and API all return not found, and ldap
was never fetched by any proxy or cache. Cold machines — including release
runners — fail go mod download, so main could neither build nor release.
Local builds passed only on module caches warmed 21:36-21:48 UTC today.

iam2 restored to v0.1.1, the state prod v1.801.38 ships. Re-land the embed
unchanged once the hanzoiam repos exist and resolve from a cold GOMODCACHE.
2026-07-16 17:01:45 -07:00
hanzo-dev f662973eb2 Merge: one name for the account, one for the org — and the last copy of the payer rule deleted 2026-07-16 16:38:08 -07:00
z e72d77b21b billing: one name for the account, one name for the org
Two functions were called Payer and returned different things: account.Payer
returns the ACCOUNT that pays, principal.Payer returned the ORG whose ledger
holds it. Those are different values on the same request — a person in the shared
signup org pays from account "hanzo/alice" held in ledger "hanzo" — and one name
for both is how the gate came to key the pool while the debit spent the person.
Rename it to what it returns: principal.HomeOrg. An org names a ledger; an
account names a wallet within it.

Fix the last copy of the rule with it. clients/metering is cloud's vendored
metering client, and its IdentityFromGatewayHeaders still hardcoded `user := org`
under the same false premise, while its doc claimed cloud "mirrors" the module
"exactly so every product keys the SAME ledger entry" — a promise two independent
copies cannot keep. Both now call hanzoai/account.Payer, so they agree by
construction and the comment is true for a reason.

Its test asserted the divergence in words — "want hanzo (per-org billing key, not
org/sub)" — which is how a premise outlives the code that disproved it. It now
asserts equality with the rule.
2026-07-16 16:37:43 -07:00
hanzo-devandz b2c5b4f81e build(cloud): mirror base images to ghcr.io/hanzoai/mirror/* (kill public.ecr.aws 429s)
public.ecr.aws (ECR Public) rate-limits anonymous pulls with HTTP 429 on shared
CI runners; a 429 on any base pull aborts the release (a release died pulling
python:3.12-alpine). Repoint all five FROMs to 1:1 linux/amd64 mirrors in our
own GHCR namespace, pinned by digest for immutability:

  node    ghcr.io/hanzoai/mirror/node:24-alpine@sha256:0cb0e7c3195bce740b6c8d8b27432c92360e3b7f1528087f2c50640b177950c6
  python  ghcr.io/hanzoai/mirror/python:3.12-alpine@sha256:aa679aa4eed6eb56c1dc6ad3f1b98b7d2d788fd961596779d188fdedad97fb38
  rust    ghcr.io/hanzoai/mirror/rust:1-alpine3.22@sha256:b348cb409ac0a73de15065997a360063cf87465574a15e3e4469862cb8996f02
  golang  ghcr.io/hanzoai/mirror/golang:1.26-alpine3.22@sha256:47d47cb5cc3c7dac409dcb6c3a98a6263571218046cd02d709527feef804a77c
  alpine  ghcr.io/hanzoai/mirror/alpine:3.22@sha256:7c8cb692ae09657cbc4a3f3cbd0e8d5a2690ba38386aaaf252dbb060bf5eb2e6

The four the task named plus alpine:3.22 (the final-stage base — same registry,
same 429 exposure) so no FROM still hits public.ecr.aws. release.yml already
logs the build into ghcr.io (GH_PAT, docker/login-action) before building, so
buildx resolves these private mirrors today with no new plumbing. Only the five
FROM lines + one rationale comment change; nothing else in the Dockerfile.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-16 15:48:48 -07:00
hanzo-dev 6c77cfccab feat(ai): bump ai v1.814.0 → v1.816.0 — router self-probe, flywheel trainer, casibase metering
Ships the full router flywheel into the cloud binary: the self-probe
(continuous tagged auto traffic → reward ledger), the fit→gate→auto-deploy
→publish trainer, the routing-latency guard (~0.68us heuristic), and
casibase-chat usage/o11y metering. Also carries the zen warehouse+span
wiring already on main.

Migrates the billing-subject callers (balance.go, account/billing.go +
tests) from the ai/object.Payer that v1.816.0 REMOVED to the extracted
github.com/hanzoai/account.Payer — the same rule in its new home (the
concurrent decomplect). Same replace directive for the force-pushed iam
pseudo-version account v0.2.0 pins.

Claude-Session: https://claude.ai/code/session_018PmFAHZvbBSTsuWyebwMra
2026-07-16 15:32:35 -07:00
hanzo-dev 34800f6991 ci(cloud): GOPRIVATE += github.com/hanzoiam/* (enterprise IAM modules)
The iam2 embed pulls github.com/hanzoiam/{scim,saml} (+ ldap under -tags iam2_ldap).
hanzoiam is a distinct org from hanzoai, so hanzoai/* did NOT cover it — the test
phase would route these private modules through the public proxy/sumdb and 404.
Git auth is already handled by the reusable CI's GH_PAT insteadOf (covers any
github.com private repo the PAT reads).
2026-07-16 15:04:30 -07:00
zeekay e4b3c88f82 feat(iam2): wire SCIM + SAML into the cloud embed; LDAP GPL-isolated behind -tags iam2_ldap
Last code step of the iam2 migration. clients/iam2 blank-imports the Apache-2.0
enterprise features so their init() self-registers into iam2's feature registry;
iam2server.Mount -> feature.MountAll auto-mounts them under CLOUD_IAM_IMPL=iam2.
ONE mechanism (database/sql driver pattern) — no feature.Register call in cloud,
which (feature.Register appends with no dedup) would double-mount and collide routes.

  _ github.com/hanzoiam/scim   SCIM 2.0 provisioning  (/scim/*)
  _ github.com/hanzoiam/saml   SAML IdP + SP SSO      (/v1/iam/saml/*, /v1/iam/acs, /v1/iam/get-saml-login)

LDAP is GPL-isolated: hanzoiam/ldap links goldap (GPL-2.0), so it is opt-in only via
clients/iam2/ldap_enabled.go (//go:build iam2_ldap). The DEFAULT cloud binary stays
copyleft-free — proven: `go list -deps ./clients/iam2/` has no goldap; only under
-tags iam2_ldap does it pull github.com/lor00x/goldap + hanzoai/ldapserver.

deps: hanzoai/iam2 v0.1.1->v0.1.4; + hanzoiam/{scim,saml,ldap}; go mod tidy.
2026-07-16 14:59:47 -07:00
hanzo-devandz 6a47436062 feat(kms): one-time legacy ZapDB -> per-org SQLite migration (deploy-safe cutover)
Without this, deploying the per-org store over live data boots an EMPTY store and
orphans every secret — cloud KMS is the authoritative source the kms-operator syncs
out to every service, so that is a cluster-wide outage. New() now runs a one-time,
writer-only, keyed, FATAL-on-error migration BEFORE serving: it opens the legacy
{DataDir}/kms ZapDB (encrypted at rest with the master key), streams every SEALED
secret (kms/secrets/ prefix; the JSON value carries the full coordinate + AES-GCM
ciphertext + ML-KEM wrapped DEK) verbatim into its per-org SQLite file via store.put
— NEVER unsealing, so no plaintext is exposed and the AAD path-binding survives —
then archives the legacy dir to {DataDir}/kms.migrated so the OS lock is released for
good. Idempotent (prior .migrated marker or absent store = no-op; put upserts).

Tests: migrate roundtrip (opens to same plaintext), no-legacy-store no-op, and the
cross-org relocation defense still holds after migration. Full kms suite green (2
pre-existing admin-edge fails only).
2026-07-16 14:55:16 -07:00
hanzo-devandz b31c642959 feat(kms): per-org SQLite store — replaces OS-locked embedded ZapDB, lifts replicas=1
The embedded luxfi/kms ZapDB (a Badger fork) held an exclusive OS lock on ONE dir
for ALL orgs — the hard reason cloud ran replicas=1 behind a cross-process writer
lease. This replaces it with per-org encrypted SQLite ({DataDir}/orgs/{org}/kms.db
via cloud.OrgDB->cek, per-db DEK), which has no exclusive-opener lock, so distinct
tenants never contend and different pods can serve different tenants.

Crypto stays in the client (Seal/Open AES-256-GCM envelope, AAD-bound to the FULL
/orgs/{org} path): plaintext never reaches the file, and a record physically moved
into another org's file still fails to Open (cross-org swap defense preserved).
Adds cloud.PlatformDB for the reserved non-tenant (_platform) partition.

Verified: 54 PASS / 2 FAIL in clients/kms; the 2 fails (admin-edge dualmount/authz)
are PRE-EXISTING — proven failing identically on origin/main. concurrent_open_probe
proves per-org SQLite has no exclusive lock. Build + finance (pure-Go) green.

Branch only — NOT for main/deploy until red review passes.
2026-07-16 14:55:16 -07:00
zandhanzo-dev 2ac7f835e2 billing: the edge gate keys the account the debit spends
The gate keyed `user := home` — the org pool, always — on the premise that
prepaid billing is per-org. That premise is false. A person in the shared signup
org holds their OWN account: its members are strangers, not a team, and a shared
org is not a shared wallet. That is what IAM's signed billing_account claim states
and what ai's meter debits.

So the gate authorized against a balance nobody drained. Fund the pool and a
signup-org person still 402s, because their usage comes out of their own account;
fund the person and an empty pool blocks them anyway. Two layers, two answers, one
request.

Resolve through hanzoai/account.Payer — the same function ai debits with, on the
same credential — so the gate and the debit cannot name different accounts. The
premise is removed rather than restated.

The masquerade split is preserved by construction, not by care: the account is
resolved WITHIN the home org, so Account.Org IS the home org and a SuperAdmin
acting in another org still bills their own ledger. A claim naming a foreign
ledger is refused, so it cannot redirect a debit into the org being acted on.

Tests assert against the rule rather than a constant, so they cannot drift the way
the premise did: the signup-org person keys their own account, a real org pools,
the claim wins for a person and a project, and every case is checked equal to what
the debit computes.
2026-07-16 14:42:34 -07:00
hanzo-dev c563d8a4a6 merge: iam2 embed subsystem — CLOUD_IAM_IMPL selects beego (default) or iam2
clients/iam2 mounts the clean-room zip+orm IAM at /v1/iam when CLOUD_IAM_IMPL=iam2,
else beego, byte-for-byte unchanged. Flag OFF by default → inert. iam2 v0.1.1 seam.
2026-07-16 14:38:03 -07:00
hanzo-dev 724d011417 feat(iam2): select /v1/iam impl via CLOUD_IAM_IMPL (default beego, unchanged)
apps.Wire's identity slot now calls identitySpec(): CLOUD_IAM_IMPL=iam2 mounts the clean-room iam2 (zip+orm), anything else — including unset, the production default — keeps the legacy beego Casdoor embed byte-for-byte. The two impls own the SAME absolute prefixes (/v1/iam/*, /login/oauth/*) and cannot co-mount, so exactly one occupies the slot per boot and mount order is preserved. Off by default => completely inert until a canary flips the flag; selection (this) stays orthogonal to activation (cfg.Enabled).
2026-07-16 14:32:15 -07:00
hanzo-dev 53e22f0875 feat(iam2): clean-room IAM v2 embed subsystem (zip+orm, beego-free)
clients/iam2 folds the beego-free Hanzo IAM v2 into the unified cloud binary as an in-process identity plane — the either/or twin of clients/iam. Matches the cloud.Typed contract func(*zip.App, cloud.Deps) error: cloud hands subsystems a cloud.Deps (not an orm.DB), so Mount opens its OWN embedded SQLite ({DataDir}/iam2/iam.db, mirroring the beego embed's {DataDir}/iam layout), seeds config new-only+idempotent from the SAME init_data.json the beego iam uses (non-fatal, honest degrade), then iam2server.Mount registers the whole surface at the canonical absolute paths.

Fail-closed like clients/iam: a store-open or mount failure serves 503 on the identity prefixes while every co-resident subsystem stays up; iam2server.Mount's only panic path (a registered enterprise feature) is recovered in safeMount so it never crashes the shared binary. A TODO marks where the parallel-lane hanzoiam/{scim,saml,ldap} feature.Register lines land.

Pins github.com/hanzoai/iam2 v0.1.1 (seam held stable across the parallel internals refactor); transitive MVS bumps are all patch-level within v1.x (zip 1.8.3, orm promoted to direct, luxfi/crypto 1.20.1, argon2id 1.0.0, pgx 5.9.2). Inert until wired — see the apps.Wire gating follow-up.
2026-07-16 14:32:06 -07:00
hanzo-dev 7b63b49324 fix(ci): bound the SECOND version scan too (compute step) — same unbounded --paginate
The compute-next-version step had the same full-registry --paginate as the tag
step (fixed in f0abd21). It ran first, so it could hang before the build. Bound
it to one page too. Both version scans are now O(1 page), not O(registry).
2026-07-16 14:28:00 -07:00
hanzo-dev 0494a6d3c7 feat(zen): warehouse + gen_ai span emission with exact margin — ai v1.814.0
zen's commerce Meter now also calls ai's TraceServedUsage (recordTrace
WITHOUT recordUsage — the commerce debit stays the ONE billing source,
never doubled), carrying zen's exact per-tier retail (Charge) and upstream
COGS (Cost) folded atto→nano, so zen* traffic in the unified binary lands
in hanzo.cloud_usage + the o11y span plane with TRUE margin instead of
being warehouse-blind. Rides the ai v1.813.1→v1.814.0 bump; balance.go
(+ its drift-guard test) migrated to the renamed Payer/PayerOf API —
same subjects, one rule.

Claude-Session: https://claude.ai/code/session_018PmFAHZvbBSTsuWyebwMra
2026-07-16 14:26:17 -07:00
zandGitHub 27cc091884 Merge pull request #315 from hanzoai/cloud-zen-family-events
feat(zen): embedded-zen meter writes the family RoutingEvent (last link) + ai v1.813.5/zen v1.3.11
2026-07-16 14:08:50 -07:00
hanzo-dev 64b0d1f8a1 feat(zen): embedded-zen meter writes the family RoutingEvent (the last link)
The embedded zen mount serves the zen catalog in-process and never reaches ai's
pipeToFamily, so zen* calls produced ZERO routing events — starving stats, world,
spark retrain, and /v1/feedback joins. Wire cloud's zen Meter to ALSO write the
RoutingEvent through the ONE shared writer object.RecordFamilyRouting (source="family",
served arm = zen.Usage.Upstream, join key = zen.Usage.ResponseID — the client-visible
response id, new in zen v1.3.11 — tokens + retail cost), fire-and-forget beside the
existing debit. Bumps zen v1.3.7 → v1.3.11 (Usage.ResponseID); ai already v1.813.6
carries object.RecordFamilyRouting.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-16 14:08:28 -07:00
antje 54b8bae797 git: Gitea push-webhook ingest (POST /v1/git/webhook)
The external Hanzo Git server (Gitea fork, git.hanzo.ai) POSTs push events
here so a push landing on it drives the SAME push-to-deploy core the embedded
smart-HTTP receive-pack path drives: fireBranchBuild -> cloud.OnGitPush deploy
trigger + EmitLifecycle. One code path, no duplication.

HMAC auth: X-Gitea-Signature is hex HMAC-SHA256 of the raw body, verified
constant-time against GIT_WEBHOOK_SECRET (KMS-synced hanzo/prod:/git/webhook-secret).
Fail-closed 401 on unset secret or mismatch. Only X-Gitea-Event: push acts;
others 204. Zero-SHA / non-branch refs are no-ops.
2026-07-16 12:05:08 -07:00
zandGitHub 5fb8f50e0a Merge decomplect/account-payer: migrate to ai.Payer — unbreaks main, ends the self-serve 402
main pinned ai v1.813.6 (which deletes billing_subject.go for the Payer/Account
refactor) without migrating the call sites, so main did not compile:
  clients/billing/balance.go:59: undefined: aiobject.BillingSubject

This migrates the call sites to the ONE rule: Payer(Credential)->Account
(owner = Person|Org|Project). Ends the live 402 where a self-serve signup's top-up
minted to the shared org pool 'hanzo' while the gate debited 'hanzo/alice' ($0) --
customer paid, locked out, money in a pool their org-mates could spend.

Green: go build -tags 'libsqlite3 sqlite_fts5' ./clients/... rc=0;
go test ./clients/billing ./clients/account rc=0. go.mod/go.sum identical to main.
2026-07-16 11:51:27 -07:00
hanzo-dev bd520de13f fix(billing): migrate off deleted BillingSubject → ai.Payer/PayerOf — unbreaks main
main pins ai v1.813.6, which contains the Payer/Account refactor (billing_subject.go
and its PERSONAL_BILLING_ORGS/ORG_BILLING_ORGS lying default are deleted). But the
call sites were never migrated, so main does not compile:
  clients/billing/balance.go:59: undefined: aiobject.BillingSubject
  clients/billing/balance.go:61: undefined: aiobject.BillingSubjectFromUserKey

Migrates balance.go + finance.go + billing.go to the ONE rule —
Payer(Credential{Owner,Name}).Subject() / PayerOf(org,key).Subject() — and DRYs the
duplicate subject resolvers. This is the cloud half of the fix that ends the live
self-serve 402 (top-up minted to the shared org pool 'hanzo' while the gate debited
'hanzo/alice' = $0).

go.mod/go.sum untouched vs main. Green: go build -tags 'libsqlite3 sqlite_fts5'
./clients/... rc=0; go test ./clients/billing ./clients/account rc=0.
2026-07-16 11:50:51 -07:00
hanzo-dev 65e782a855 refactor(billing): route top-up + console subject through ai.Payer, one rule
The console top-up (clients/account/billing.go) and the finance read
(clients/billing/finance.go) each re-implemented the billing subject as "always
the org" — the reverted lineage's rule. Against ai's gate, which bills a signup
person per-person, that is the split: money minted to subject "hanzo" (the pool)
while the gate debited "hanzo/alice" ($0) → the paid-up member 402'd.

Delete both twins; resolve the subject through the ONE rule, ai/object.Payer,
keyed on the IAM username (X-User-Name) the gate also keys on. Top-up credits and
console reads now land on the SAME account the gate debits — they cannot drift
because there is one function. The killed PERSONAL_BILLING_ORGS / ORG_BILLING_ORGS
env is inert here too (test proves hostile values change nothing).

Requires ai v1.809.5, which must be tagged FROM ai main after decomplect/account-payer
merges — NOT off a branch. The prior one-rule fix was tagged off an unmerged branch
(v1.806.8/.9), main never got it, and every later tag resurrected the allowlists;
that is why this bug is live. go.sum refreshes via `go mod tidy` once the tag exists.
Verified locally via a replace to the ai branch: clients build + twin tests green.
2026-07-16 11:47:10 -07:00
antje 74d94d1f3e ci(release): retire notify-universe — Hanzo CD owns git→cluster sync
The repository_dispatch deploy hub is gone (universe image-update.yml
removed in universe d07cf945; the dispatches were silently suppressed by
the flagged sender account regardless). Deploys are declared-tag bumps in
universe crs/, synced by Hanzo CD (ArgoCD, ns hanzo-cd) and reconciled by
the operator. [skip ci]
2026-07-16 11:36:10 -07:00
antjeandGitHub ce6c4dc374 deps: hanzoai/ai v1.813.1 -> v1.813.6 (balance-exempt routing config + enso family helper) (#314)
ai#102: /v1/get-routing-defaults + org-settings CRUD + routing-ledger
export are configuration metadata, never wallet-gated — unblocks reading
org routing defaults for $0-balance orgs and the operator platform flip.
Edge auto-routing billing tests green.
2026-07-16 11:28:33 -07:00
antje 2134876826 platform: allow the self-hosted fleet registry in the native build lane
registry.hanzo.ai/{hanzoai,luxfi,zooai}/ join the /v1/runner push allowlist —
Wave 0 of the native CI/CD migration. Until now only release.yml's crane
mirror could reach the fleet registry; the native BuildKit lane was
ghcr-only by policy.
2026-07-16 11:13:07 -07:00
hanzo-dev f0abd2116e fix(ci): version-assignment scanned the WHOLE registry — livelocked releases
The "atomic free-version assignment" step paginated every container version
(`gh api --paginate .../versions`) to find the max release number. As the
registry accumulated tags this grew unbounded and hung the step for 30+ min,
livelocking every cloud release. Container versions are created newest-first and
version tags are monotonic, so the max is always on the newest page — query one
bounded page (?per_page=100) instead of the full history. Fast + correct.
2026-07-16 11:11:39 -07:00
hanzo-dev ebf267908c refactor(ledger): drop redundant "core" — the ledger adapter is apps.ledger
Values, not places: the finance-backed credit-ledger adapter is qualified by its
namespace (apps.ledger), not a braided ledgercoreCredit compound. Rename the type
ledgercoreCredit → ledger, the file commerce_ledger.go → ledger.go, and scrub
"ledgercore" from prose (the ledger IS the core — "core" adds nothing). No behavior
change; admin/core tests green, changed packages build.
2026-07-16 10:18:47 -07:00
antje 4a64ef568b feat(chat): clients/chat — one /v1/chat tool-calling orchestrator
POST /v1/chat runs one LLM tool-calling round that lets a model manage a system
via tools. Composes existing cloud pieces, reinventing nothing:
- LLM routing + per-org reserve/settle billing: the ai subsystem's
  /v1/chat/completions, invoked in-process (Fiber Test) — the only path that
  both returns tool_calls AND carries the billing gate.
- tool plane (clients/tools): the org's registered MCP/registry tools are
  offered to the model and dispatched server-side (activation + price gated).
- capabilities: graph (advisory node-ops -> ops the client applies) and create
  (server-executed, tools = the org's registered MCP render services).
Returns {reply, actions, ops}. chat mounts before ai so /v1/chat resolves here
(the ai /v1/chat alias is shadowed); ai keeps /v1/chat/completions.
2026-07-16 10:02:29 -07:00
hanzo-devandantje 73632723f6 feat(ai): bump ai subsystem v1.813.0 → v1.813.1
Carries the PAID-Enso revert + the family learning loop: per-family-call RoutingEvents
+ shadow A/B (records what the learned engine would have picked), /v1/feedback signal
contract (up/down/regenerate/switch/abandon/accept/revert/rating/dismiss) with the
online reward forward to the engine's /route/observe, ROUTER_ADMIN_TOKEN service-auth
on the training-data exports, and Zen/Enso provider branding. Lights up /v1/router/stats
+ world.hanzo.ai (shadow-vs-served agreement).

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-16 09:53:56 -07:00
hanzo-dev f5d57e35f1 feat(ai): bump ai subsystem v1.812.0 → v1.813.0 — router policy, rewards, margin, DO backfill
Ships the per-org router policy (get/update-router-policy + the org > '*' >
conf fold on every auto route), the routing-reward ledger, the nano margin
ledger (costNano/billedNano/marginNano + unpriced flagging), and the
gaps-only DigitalOcean usage backfill (POST /v1/admin/usage/backfill-do,
dry-run default). The release image also re-embeds console@main
(CONSOLE_REF=main), picking up the console Router page (v8.4.137+).

Claude-Session: https://claude.ai/code/session_018PmFAHZvbBSTsuWyebwMra
2026-07-16 09:28:20 -07:00
hanzo-dev 8f61d57963 feat(credit): ONE ledger seam — commerce credit + admin grant mint into finance.Current()
The last mile of "one way to grant credit": cloud implements commerce's
creditledger.CreditLedger (v1.48.5) over the native finance ledger and injects it
at mountCommerce (EmbedConfig.Ledger). Now commerce's POST /v1/billing/credit AND
the admin.hanzo.ai grant (/v1/admin/customers/:org/credit) both mint into the SAME
per-org finance wallet the ai prepaid gate reads — a granted credit is immediately
spendable, no split ledger. The admin path drops its parallel finance.Deposit for
the one creditledger.Credit call (idempotency key rides through; commerce HTTP
deposit remains the split-deploy fallback).

- apps/commerce_ledger.go: ledgercoreCredit adapter (compile-time asserted against
  commerce's exported interface); org-pool wallet (Subject==Org); fails closed with
  no co-resident finance.
- apps/commerce.go: inject Ledger: ledgercoreCredit{} at mountCommerce.
- clients/admin/core/grant.go: grantDeposit prefers creditledger.Get() (the one
  ledger) over its own finance.Deposit.

Verified: changed pkgs build green on commerce v1.48.5 + ai v1.812.0; admin/core
grant tests pass. Ships with the ai free-tier removal (v1.811.0+) already on main.
2026-07-16 09:14:59 -07:00
antje 5bfe2842a1 ci(release): pass GIT_AUTH_TOKEN build secret — Dockerfile renamed the id from gh_token (46081689) but the workflow still passed gh_token, so the private console clone ran credentialless and the release failed 2026-07-16 01:50:00 -07:00
antje 4608168991 build: consume the standard GIT_AUTH_TOKEN build secret (was gh_token)
One secret id everywhere: BuildKit's gitsource convention (GIT_AUTH_TOKEN,
which the fabric's buildFrontendCmd already attaches) is also what the
Dockerfile's console-embed and go-mod fetch stages mount. gh_token was a
second name for the same credential.
2026-07-16 01:19:03 -07:00
antje a74c95abc2 deps: hanzoai/ai v1.812.0 + commerce v1.48.5 — reward ledger, router policy, connections import reach prod
ai v1.809.4 → v1.812.0: RoutingEvent reward ledger (/v1/add-routing-reward,
/v1/export-routing-rewards), per-org router policy + observability, Enso
public arms + retrain-status, connected-provider usage import (OpenAI/
Anthropic), own-brand god-view gate (TokenIsOwnBrand), Bearer-aware
get-cloud-usages, billing nano margin ledger, unpriced-model flagging.

commerce v1.48.2 → v1.48.5: mint-gated org-keyed credit primitive +
injected-ledger seam; fixes the phantom luxfi/cevm test import that broke
go mod tidy for every consumer (this repo included).
2026-07-16 01:16:12 -07:00
antje b7e4843cd7 feat(fabric): GIT_AUTH_TOKEN build secret — private-repo fetches for the build Jobs
The buildkit Job now surfaces the console-git-token Secret (optional) as the
GIT_AUTH_TOKEN env, and buildctl attaches it as the same-named build secret;
BuildKit's gitsource presents it as the HTTPS credential for the git context.
Fixes the fabric's inability to build PRIVATE repos (github.com/hanzoai/cloud
itself: 'could not read Username' — the rel_HvInGKU failure). Public repos and
Secret-less clusters fetch anonymously exactly as before.
2026-07-16 01:15:07 -07:00
antje 9b97ca9164 feat(git): public repos — anonymous read for the build fabric
A repo gains a visibility bit: PATCH /v1/git/repos/:name {"public":true}
(or create-time "public"). Public grants ANONYMOUS upload-pack/info-refs
only — receive-pack and the whole control plane stay org-authed, and a
private or missing repo answers the same uniform 404 so anonymous probing
cannot enumerate.

This is what lets the credential-less buildkit build Job (launchDirectBuild
git context) fetch from the embedded git server — the same reason public
GitHub repos build with no env. Private-repo builds remain a later
GIT_AUTH_TOKEN feature.

resolvePackRepo grows the allowPublic branch (fetch-side only): anonymous
callers address org-level repos by the orgRE-validated :org path segment;
the path-vs-identity guard for authenticated callers is unchanged.
2026-07-16 01:05:30 -07:00
antjeandGitHub 4b255e8e9a test(platform): end-to-end git push -> buildFromPush enqueue proof (#311)
Wire a REAL smart-HTTP git push through the actual production seam
(cloud.RegisterPushBuilder <-> cloud.OnGitPush) into the real platform
push builder, and assert the matching app's build is enqueued (a
building deployment for the pushed commit). The two halves were covered
in isolation (clients/git TestPushFiresBuildTrigger; platform
TestBuildFromPush_LaunchesMatchingApp) but nothing connected a live push
to the real builder end to end. Pure-Go dev build (CGO_ENABLED=0).
2026-07-16 00:45:07 -07:00
hanzo-dev 41be25c713 Merge land/slack-on-bridge: Slack chat path onto the generalized bridge — one chat code path 2026-07-16 00:15:57 -07:00
hanzo-dev 301706dd54 refactor(integrations): Slack chat path onto the generalized bridge — one chat code path 2026-07-16 00:13:59 -07:00
171 changed files with 12415 additions and 3859 deletions
+14 -4
View File
@@ -89,15 +89,25 @@ jobs:
with:
go-version-file: go.mod
- name: go env for private modules (matches Dockerfile — zap-proto is direct+authenticated)
- name: go env for private modules
env:
GH_PAT: ${{ secrets.GH_PAT }}
# GOPRIVATE names exactly the namespace that is private. github.com/hanzoai/*
# is: ai, account, commerce, orm, xorm, beego, csqlite and ~30 more are
# private repos, so they must resolve direct+authenticated and skip a sumdb
# that cannot see them. Everything else stays on the public proxy + checksum
# db, which is what makes a module hash immutable: zap-proto (all 55 repos)
# and luxfi (all 37 deps here) are public and proxy-served.
#
# This previously named zap-proto — public, and never the reason anything
# here was direct — and then set GOSUMDB=off to compensate for hanzoai/*
# being absent, which disabled checksum verification for EVERY module in the
# build, public ones included. Naming the private namespace is what the off
# switch was standing in for.
run: |
git config --global url."https://x-access-token:${GH_PAT}@github.com/".insteadOf "https://github.com/"
{
echo "GOPRIVATE=github.com/zap-proto/*"
echo "GONOSUMDB=github.com/zap-proto/*"
echo "GOSUMDB=off"
echo "GOPRIVATE=github.com/hanzoai/*"
echo "GOPROXY=https://proxy.golang.org,direct"
} >> "$GITHUB_ENV"
+80 -31
View File
@@ -96,9 +96,9 @@ jobs:
# an image (even from a run that died before tagging) is never reused.
cont_max=""
if command -v gh >/dev/null 2>&1; then
cont_max="$(GH_TOKEN="$GH_PAT" gh api --paginate \
'/orgs/hanzoai/packages/container/cloud/versions' \
--jq '.[].metadata.container.tags[]' 2>/dev/null \
cont_max="$(GH_TOKEN="$GH_PAT" gh api \
'/orgs/hanzoai/packages/container/cloud/versions?per_page=100' \
--jq '.[].metadata.container.tags[]?' 2>/dev/null \
| sed 's/^v//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true)"
fi
@@ -221,10 +221,10 @@ jobs:
# frozen snapshot the persistent BuildKit cache would otherwise serve forever.
build-args: |
CONSOLE_CACHEBUST=${{ steps.ver.outputs.cachebust }}
# gh_token: BuildKit secret the Dockerfile consumes to fetch private
# GIT_AUTH_TOKEN: BuildKit secret the Dockerfile consumes to fetch private
# cross-org Go modules (hanzoai/*, luxfi/*) over authenticated git.
secrets: |
gh_token=${{ secrets.GH_PAT }}
GIT_AUTH_TOKEN=${{ secrets.GH_PAT }}
- name: Smoke test — the binary MUST boot to "listening" with no crash signature
run: |
@@ -293,6 +293,67 @@ jobs:
echo "SMOKE PASS: cloud:smoke booted to \"listening\" with no crash signature"
# ── Functional smoke — authenticated per-subsystem probe (the REAL gate) ─────
# The boot check above proves the process REACHES "listening"; this proves the
# mounted HTTP surface actually WORKS. /smoke (cmd/smoke, baked into the image)
# hits ONE side-effect-free read per core subsystem and FAILS the release on any
# broken code — above all a 402 on a READ (the balance-gate-over-blocks-reads
# regression) or a 5xx (a crash, e.g. the /v1/billing/usage self-dispatch 500).
# So a release can never ship with chat/billing/projects/kms/... down.
- name: Functional smoke — per-subsystem probe (fails the release if a core endpoint is broken)
env:
KMS_CLIENT_ID: ${{ secrets.KMS_CLIENT_ID }}
KMS_CLIENT_SECRET: ${{ secrets.KMS_CLIENT_SECRET }}
KMS_ENDPOINT: ${{ vars.KMS_ENDPOINT || 'https://kms.hanzo.ai' }}
# A KMS-provisioned short-lived smoke bearer, injected as a secret (NEVER
# hardcoded). Absent → the anonymous matrix still gates public/authed and
# catches every 402-on-read / 5xx.
SMOKE_TOKEN: ${{ secrets.SMOKE_TOKEN }}
run: |
set -euo pipefail
IMAGE=cloud:smoke
CID=""
cleanup() { [ -n "$CID" ] && docker rm -f "$CID" >/dev/null 2>&1 || true; }
trap cleanup EXIT
KEY="$(head -c 32 /dev/urandom | base64 | tr -d '\n')"
CID="$(docker run -d \
--tmpfs /data:rw,size=64m \
-e CLOUD_DATA_DIR=/data -e CLOUD_ENV=smoke -e CLOUD_KMS_MASTER_KEY_REF="$KEY" \
"$IMAGE")"
# Wait for the HTTP listener to bind (or the process to die).
up=0
for _ in $(seq 1 60); do
lg="$(docker logs "$CID" 2>&1 || true)"
printf '%s' "$lg" | grep -q '"message":"listening"' && { up=1; break; }
[ "$(docker inspect -f '{{.State.Running}}' "$CID" 2>/dev/null || echo false)" != "true" ] && break
sleep 1
done
if [ "$up" != 1 ]; then
echo "::group::boot logs"; docker logs "$CID" 2>&1 || true; echo "::endgroup::"
echo "FUNCTIONAL SMOKE INFRA FAIL: image never reached \"listening\""
exit 1
fi
# Token: prefer the injected secret; else mint from KMS (a provisioned smoke
# identity); else run the anonymous matrix. Never hardcoded.
if [ -z "${SMOKE_TOKEN:-}" ] && [ -n "${KMS_CLIENT_ID:-}" ]; then
KT=$(curl -sf "$KMS_ENDPOINT/v1/kms/auth/login" -H 'Content-Type: application/json' \
-d "{\"clientId\":\"$KMS_CLIENT_ID\",\"clientSecret\":\"$KMS_CLIENT_SECRET\"}" | jq -r '.accessToken // empty' || true)
[ -n "$KT" ] && SMOKE_TOKEN=$(curl -sf "$KMS_ENDPOINT/v1/kms/orgs/hanzo/secrets/smoke/TOKEN?env=prod" \
-H "Authorization: Bearer $KT" | jq -r '.secret.value // empty' || true)
fi
if [ -n "${SMOKE_TOKEN:-}" ]; then echo "::add-mask::$SMOKE_TOKEN"; echo "smoke: AUTHENTICATED matrix"; else echo "smoke: ANONYMOUS matrix (no SMOKE_TOKEN wired)"; fi
# /smoke is baked into the image (Dockerfile) — exec it INSIDE the container,
# so it probes the real mounted surface at localhost:8080 with no port/network
# plumbing. A non-zero exit here fails the release BEFORE any image is pushed.
docker exec \
-e SMOKE_BASE_URL=http://127.0.0.1:8080 \
-e SMOKE_TOKEN="${SMOKE_TOKEN:-}" \
"$CID" /smoke
# ── Migration smoke — the gate the v1.800.1 crashloop would have tripped ─────
# The plain smoke above boots on a FRESH /data, so every subsystem's migrate()
# takes its CREATE-TABLE path and no forward-migration is exercised — which is
@@ -406,7 +467,7 @@ jobs:
build-args: |
CONSOLE_CACHEBUST=${{ steps.ver.outputs.cachebust }}
secrets: |
gh_token=${{ secrets.GH_PAT }}
GIT_AUTH_TOKEN=${{ secrets.GH_PAT }}
# THE RECEIPT + ATOMIC VERSION ASSIGNMENT (race-safe). Reached only because
# build + smoke + push all succeeded, so a proven image exists under the unique
@@ -434,11 +495,15 @@ jobs:
git fetch --tags --force --quiet
git_max="$(git tag -l 'v[0-9]*.[0-9]*.[0-9]*' \
| sed 's/^v//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true)"
# Newest page only — NOT --paginate. Container versions are created
# newest-first and version tags are monotonic, so the highest version
# is always among the most-recent versions; paginating the WHOLE
# registry history is what livelocked this step as tags accumulated.
cont_max=""
if command -v gh >/dev/null 2>&1; then
cont_max="$(GH_TOKEN="$GH_PAT" gh api --paginate \
'/orgs/hanzoai/packages/container/cloud/versions' \
--jq '.[].metadata.container.tags[]' 2>/dev/null \
cont_max="$(GH_TOKEN="$GH_PAT" gh api \
'/orgs/hanzoai/packages/container/cloud/versions?per_page=100' \
--jq '.[].metadata.container.tags[]?' 2>/dev/null \
| sed 's/^v//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true)"
fi
max="$(printf '%s\n%s\n%s\n' "1.786.0" "$git_max" "$cont_max" \
@@ -487,25 +552,9 @@ jobs:
echo "::error::could not acquire a free version tag after 8 attempts"
exit 1
# Notify universe so the GitOps pipeline rolls the new image to prod — same
# image-update contract every service uses (gateway, iam, …). Runs ONLY after
# build-amd64 succeeds, i.e. only for a version whose image is proven pushed and
# tagged. A failed release never reaches here, so universe is never asked to
# deploy a phantom tag.
notify-universe:
needs: build-amd64
runs-on: [hanzo-build-linux-amd64]
steps:
- name: Repository dispatch (image-update)
uses: peter-evans/repository-dispatch@v3
with:
token: ${{ secrets.UNIVERSE_DISPATCH_TOKEN }}
repository: hanzoai/universe
event-type: image-update
client-payload: |
{
"service": "cloud",
"image": "ghcr.io/hanzoai/cloud:${{ needs.build-amd64.outputs.version_v }}",
"sha": "${{ github.sha }}",
"env": "all"
}
# Deploy = a declared-tag bump in hanzoai/universe crs/cloud.yaml — Hanzo CD
# (the ArgoCD instance in ns hanzo-cd) syncs universe→cluster and the operator
# reconciles the CR. The old notify-universe repository_dispatch hub is retired
# (its flagged-sender dispatches were silently suppressed anyway); the native
# release path (release.go rolloutRelease) and deliberate promote commits own
# the bump.
+66 -139
View File
@@ -2,133 +2,50 @@
#
# This image is a SINGLE artifact that serves BOTH the /v1 API AND the console
# UI from one process: the console is compiled into the Go binary via
# //go:embed (see webui.go). The pipeline is:
# //go:embed (see webui.go). The final `/cloud` binary already carries the UI —
# no separate console Service, no second origin; the embedded console calls /v1
# on its own host.
#
# 1. console stage → build the hanzoai/console static bundle
# 2. (copied) → into webui/dist/ of the Go build context
# 3. build stage → `go build` bakes webui/dist into the binary (go:embed)
#
# so the final `/cloud` binary already carries the UI. No separate console
# Service, no second origin — the embedded console calls /v1 on its own host.
#
# ── console UI stage ─────────────────────────────────────────────────────────
# Builds the console SPA and emits a STATIC bundle at /out. console is fetched
# at a pinned ref (CONSOLE_REF) using the same gh_token BuildKit secret the Go
# build uses for private modules.
#
# console exposes `npm run build:embed` (scripts/build-embed.mjs): it prunes the
# Next server route handlers (BFF proxies — they collapse to the cloud /v1/* the
# SPA calls same-origin), wraps the client catch-all pages for output:'export',
# and neutralizes the root layout's request-time headers() read (the per-host
# <title>, resolved client-side in the embed) so the STATIC export prerenders
# clean — emitting out/. This stage runs it and copies out/ into /out, which the
# Go build drops into webui/dist so //go:embed bakes the FULL @hanzo/gui console
# into the ONE binary. This stage FAILS HARD: the prod image MUST carry the real
# console — a missing/broken build:embed is a build ERROR, never a silent degrade
# to the placeholder shell. The one escape hatch is --build-arg ALLOW_PLACEHOLDER=1
# (pure-Go dev image with no Node console), which is NEVER set for prod.
FROM public.ecr.aws/docker/library/node:24-alpine@sha256:a0b9bf06e4e6193cf7a0f58816cc935ff8c2a908f81e6f1a95432d679c54fbfd AS console
ARG CONSOLE_REPO=https://github.com/hanzoai/console.git
ARG CONSOLE_REF=main
# CONSOLE_CACHEBUST busts this stage's BuildKit layer cache every build. WHY it must
# exist: the clone+build layer's cache key is derived from the RUN text + build args.
# With only a static `git clone --branch main`, the key NEVER changes, so on the
# persistent ARC dind BuildKit cache every cloud image re-embedded the SAME frozen
# console snapshot — new console work (the native Tracker, …) silently never shipped,
# even on a freshly-built+deployed image. release.yml feeds this the cloud commit sha
# (unique per push) so the clone RUN re-runs each build and re-fetches console
# ${CONSOLE_REF} (main HEAD) fresh. Correctness over cache reuse: the console stage
# rebuilds every time, but the embed is never stale.
ARG CONSOLE_CACHEBUST=none
RUN apk add --no-cache git
WORKDIR /console
# The static export prerenders every page (webpack compile + export prerender);
# give the heap headroom so a large @hanzo/gui build never OOMs into the stub.
ENV NEXT_TELEMETRY_DISABLED=1 NODE_OPTIONS=--max-old-space-size=8192
# Hanzo Analytics: the console's <HanzoAnalytics/> (env-gated) renders the one
# native analytics.hanzo.ai tag only when a website-id is baked in. Default to the
# console.hanzo.ai property (7dce54ee, public per-site) so console+team track on
# the next cloud build. GA4/Pixel stay off (unset). Public id, not a KMS secret.
ARG NEXT_PUBLIC_ANALYTICS_WEBSITE_ID=7dce54ee-41f6-4751-96bf-fe005067c7c7
ENV NEXT_PUBLIC_ANALYTICS_WEBSITE_ID=$NEXT_PUBLIC_ANALYTICS_WEBSITE_ID
RUN --mount=type=secret,id=gh_token \
if [ -s /run/secrets/gh_token ]; then \
git config --global url."https://x-access-token:$(cat /run/secrets/gh_token)@github.com/".insteadOf "https://github.com/"; \
fi && \
echo ">> embedding console ${CONSOLE_REF} (cachebust ${CONSOLE_CACHEBUST})" && \
git clone --depth 1 --branch "${CONSOLE_REF}" "${CONSOLE_REPO}" . && \
echo ">> console @ $(git rev-parse HEAD)" && \
npm install --no-audit --no-fund --fetch-retries=5 --fetch-retry-mintimeout=20000 --fetch-timeout=120000
# FAIL-HARD. build:embed MUST emit a REAL bundle — a non-empty out/index.html AND
# an out/_next/ chunk dir — and /out then carries it into the Go embed path. If the
# target is absent, the export fails, or the output is the placeholder shape, this
# is a build ERROR (exit 1): the prod image can NEVER silently ship the committed
# fallback shell. Escape hatch: --build-arg ALLOW_PLACEHOLDER=1 leaves /out empty
# (Go build keeps the committed shell) for a pure-Go dev image — NEVER set in prod.
ARG ALLOW_PLACEHOLDER=0
RUN mkdir -p /out; \
ok=0; \
if npm run 2>/dev/null | grep -q ' build:embed'; then \
if npm run build:embed && [ -s out/index.html ] && [ -d out/_next ]; then \
cp -r out/. /out/; \
echo ">> embedded REAL console static bundle: $(wc -c < out/index.html)-byte index.html, $(du -sh out/_next | cut -f1) _next/"; \
ok=1; \
else \
echo ">> console build:embed produced NO real bundle (missing/empty out/index.html or out/_next)"; \
fi; \
else \
echo ">> console exposes no build:embed target"; \
fi; \
if [ "$ok" != "1" ]; then \
if [ "$ALLOW_PLACEHOLDER" = "1" ]; then \
echo ">> ALLOW_PLACEHOLDER=1 — keeping committed fallback shell (DEV image only; NEVER prod)"; \
else \
echo ">> FATAL: refusing to ship the placeholder console. Fix the console build:embed, or pass --build-arg ALLOW_PLACEHOLDER=1 for a pure-Go dev image."; \
exit 1; \
fi; \
fi
# ── prebuilt decomplection artifacts (cloud compiles ONLY Go) ────────────────
# The console SPA, the agent-skills catalog, and the native flags staticlib are
# each built by THEIR OWN CI as a versioned immutable image and PULLED here,
# instead of rebuilding node + python + rust from scratch every cloud release.
# The heavy one (console: a cold `npm install` + full Next.js static export,
# force-cache-busted every build) used to dominate the ~20-min build; it is now
# a registry pull.
# console-embed (hanzoai/console Dockerfile.embed) → /dist → webui/dist (go:embed)
# agent-skills (hanzoai/openapi Dockerfile.skills) → /catalog → clients/agentskills/catalog (go:embed)
# cloud-flags (native/flags Dockerfile) → /libhanzo_flags.a → CGO link (clients/featureflags)
# Pinned to ghcr.io so BOTH buildx lanes (release.yml + platform arcbuild) pull
# it directly; the SAME tags are mirrored to registry.hanzo.ai (S3-backed) for
# GET-flow consumers (docker/kaniko/crane). Override any pin with
# --build-arg <NAME>_IMAGE=… — release.yml resolves CONSOLE_IMAGE to a fresh
# console-embed digest, exactly as CONSOLE_CACHEBUST re-fetched console before.
ARG CONSOLE_IMAGE=ghcr.io/hanzoai/console-embed:latest
ARG SKILLS_IMAGE=ghcr.io/hanzoai/agent-skills:latest
ARG FLAGS_IMAGE=ghcr.io/hanzoai/cloud-flags:latest
# ── Go build stage (CGO=1 + SQLCipher — REAL at-rest encryption) ─────────────
# The unified binary embeds IAM (clients/iam) whose per-org store is SQLCipher-
# encrypted (orgIsolation=sqlite), and commerce's per-tenant money DBs likewise.
# A CGO=0 modernc build SILENTLY SHIPS PLAINTEXT. So this builds CGO=1 against
# system libsqlcipher — hanzoai/iam's proven recipe: the `libsqlite3` tag + a
# libsqlcipher symlink + -DSQLITE_HAS_CODEC, with the modernc double-registration
# guard, TestEncryptionProof, and the cek.go golden-vector KAT baked in — so a
# build that fails to link REAL SQLCipher, or that would decrypt existing stores
# differently, produces NO image. alpine3.22 MATCHES the runtime base so the
# libsqlcipher soname the binary links is the SAME one present at runtime. ECR
# Public mirror avoids Docker Hub's 429 rate-limit on shared CI runners.
# ---- agent-skills stage: regenerate the FULL /.well-known/agent-skills catalog
# from the hanzoai/openapi SOT (skills.py) and carry it into the Go embed path
# BEFORE `go build`, the SAME way the console bundle is produced. The committed
# catalog is only the tiny `ai` fallback; prod must embed the full set. FAIL-HARD:
# if the clone/generation can't produce the master index, the image is not built.
FROM public.ecr.aws/docker/library/python:3.12-alpine AS skills
ARG OPENAPI_REPO=https://github.com/hanzoai/openapi.git
ARG OPENAPI_REF=main
RUN apk add --no-cache git && pip install --no-cache-dir pyyaml
WORKDIR /openapi
RUN --mount=type=secret,id=gh_token \
if [ -s /run/secrets/gh_token ]; then \
git config --global url."https://x-access-token:$(cat /run/secrets/gh_token)@github.com/".insteadOf "https://github.com/"; \
fi && \
git clone --depth 1 --branch "${OPENAPI_REF}" "${OPENAPI_REPO}" . && \
python3 skills.py --no-services --out /catalog && \
test -s /catalog/hanzo/index.json
# ── toolchain base images: the golang + alpine FROMs below pull from our own
# GHCR mirror (ghcr.io/hanzoai/mirror/*), pinned by digest. WHY: public.ecr.aws
# rate-limits anonymous pulls (HTTP 429) on shared CI runners and a 429 on ANY
# base pull aborts the release. The mirror packages are 1:1 amd64 copies of the
# upstream public images, digest-pinned for immutability; release.yml logs the
# build into ghcr.io (GH_PAT) before building so they resolve. REFRESH on a
# toolchain bump: crane/regctl copy the new upstream into
# ghcr.io/hanzoai/mirror/<name>:<tag> and repoint the digest below. Canonical
# long-term home is registry.hanzo.ai/hanzoai/mirror/* — repoint once the runners
# carry its IAM pull credentials (follow-up).
# ── Native flags evaluator — hanzo-flags (Rust staticlib, FFI'd into the Go
# binary by clients/featureflags). Stateless PostHog-compatible evaluation;
# definitions live in the per-org SQLite stores. musl staticlib links clean
# against the alpine cgo build below.
FROM public.ecr.aws/docker/library/rust:1-alpine3.22 AS flagslib
RUN apk add --no-cache musl-dev
WORKDIR /src/native/flags
COPY native/flags/Cargo.toml native/flags/Cargo.lock ./
COPY native/flags/src ./src
RUN cargo build --release --locked
# ── console SPA static export (prebuilt → /dist) ─────────────────────────────
FROM ${CONSOLE_IMAGE} AS console
FROM public.ecr.aws/docker/library/golang:1.26-alpine3.22@sha256:727cfc3c40be55cd1bc9a4a059406b28a059857e3be752aa9d09531e12c20c56 AS build
# ── agent-skills catalog (prebuilt → /catalog) ──────────────────────────────
FROM ${SKILLS_IMAGE} AS skills
# ── native flags evaluator staticlib (prebuilt → /libhanzo_flags.a) ──────────
FROM ${FLAGS_IMAGE} AS flagslib
FROM ghcr.io/hanzoai/mirror/golang:1.26-alpine3.22@sha256:47d47cb5cc3c7dac409dcb6c3a98a6263571218046cd02d709527feef804a77c AS build
# CIPHER-FORMAT FREEZE (cek depends on this). The data-plane stores are
# SQLCipher pages in a fixed on-disk format (cipher_compatibility 4). An at-open
# compat pin is infeasible (mattn keys via URI before any pragma), so the format
@@ -149,18 +66,21 @@ RUN set -eux; \
ln -sf "$SC" /usr/lib/libsqlite3.so; \
ln -sf "$SC" /usr/lib/libsqlite3.so.0
WORKDIR /src
# hanzoai/* and luxfi/* are PUBLIC and resolve via the IMMUTABLE public proxy +
# sumdb — go.sum pins those canonical hashes, so a force-re-pointed tag can never
# break the build. GOSUMDB stays ON (a money image must not blanket-disable the
# checksum database); only zap-proto/* is exempt (first-party-direct via GOPRIVATE,
# authenticated git over gh_token). -mod=readonly means the committed go.sum is the
# SOLE source of truth: any drift (a needed hash not present) FAILS the build
# instead of being silently re-recorded. CGO_CFLAGS/LDFLAGS enable the SQLCipher
# codec + URI keying.
# zap-proto/* (all 55 repos) and luxfi/* (all 37 deps here) are PUBLIC and resolve
# via the IMMUTABLE public proxy + sumdb — go.sum pins those canonical hashes, so a
# force-re-pointed tag can never break the build. GOSUMDB stays ON (a money image
# must not blanket-disable the checksum database); github.com/hanzoai/* is the
# exempt namespace — ai, account, commerce, orm, xorm, beego, csqlite and ~30 more
# are PRIVATE repos, so they resolve direct+authenticated (git over gh_token) and
# skip a sumdb that cannot see them. GOPRIVATE named zap-proto until now, which is
# public and was never the reason anything was direct; the private namespace it
# stood for went unnamed and worked only on the GOPROXY `direct` fallback.
# -mod=readonly means the committed go.sum is the SOLE source of truth: any drift
# (a needed hash not present) FAILS the build instead of being silently
# re-recorded. CGO_CFLAGS/LDFLAGS enable the SQLCipher codec + URI keying.
ENV CGO_CFLAGS="-DSQLITE_HAS_CODEC -DSQLITE_USE_URI=1 -I/usr/include/sqlcipher" \
CGO_LDFLAGS="-lsqlcipher" \
GOPRIVATE=github.com/zap-proto/* \
GONOSUMDB=github.com/zap-proto/* \
GOPRIVATE=github.com/hanzoai/* \
GOPROXY=https://proxy.golang.org,direct \
GOFLAGS=-mod=readonly
COPY go.mod go.sum ./
@@ -171,22 +91,22 @@ COPY go.mod go.sum ./
# and resolves fine from a clean cache. That is exactly what wedged the release
# on otel-collector v0.144.10. BUMP THE SUFFIX (-v4 -> -v5) to force a cold
# module cache the next time a phantom pin poisons it.
RUN --mount=type=secret,id=gh_token \
RUN --mount=type=secret,id=GIT_AUTH_TOKEN \
--mount=type=cache,id=cloud-gomod-v4,target=/go/pkg/mod,sharing=locked \
if [ -s /run/secrets/gh_token ]; then \
git config --global url."https://x-access-token:$(cat /run/secrets/gh_token)@github.com/".insteadOf "https://github.com/"; \
if [ -s /run/secrets/GIT_AUTH_TOKEN ]; then \
git config --global url."https://x-access-token:$(cat /run/secrets/GIT_AUTH_TOKEN)@github.com/".insteadOf "https://github.com/"; \
fi && \
go mod download
COPY . .
# Drop the console static bundle into the embed path BEFORE `go build`, so
# //go:embed all:webui/dist bakes it into the binary (same-origin console).
COPY --from=console /out/ /src/webui/dist/
COPY --from=console /dist/ /src/webui/dist/
# Overlay the FULL agent-skills catalog before `go build` so //go:embed all:catalog
# bakes the complete set (all services × brands), not the committed `ai` fallback.
COPY --from=skills /catalog/ /src/clients/agentskills/catalog/
# The native flags staticlib at the exact ${SRCDIR}-relative path the cgo
# directive in clients/featureflags/engine.go links.
COPY --from=flagslib /src/native/flags/target/release/libhanzo_flags.a /src/native/flags/target/release/libhanzo_flags.a
COPY --from=flagslib /libhanzo_flags.a /src/native/flags/target/release/libhanzo_flags.a
# RED gate — modernc double-registration guard: 0 modernc under CGO=1, else the
# "sqlite" driver is registered twice (mattn + modernc) → panic at init.
RUN --mount=type=cache,id=cloud-gomod-v4,target=/go/pkg/mod,sharing=locked \
@@ -215,12 +135,18 @@ RUN --mount=type=cache,id=cloud-gomod-v4,target=/go/pkg/mod,sharing=locked \
RUN --mount=type=cache,id=cloud-gomod-v4,target=/go/pkg/mod,sharing=locked \
--mount=type=cache,id=cloud-gobuild-v4,target=/root/.cache/go-build,sharing=locked \
CGO_ENABLED=1 go build -tags "libsqlite3 sqlite_fts5" -ldflags="-s -w" -o /cloud ./cmd/cloud
# The functional smoke prober (cmd/smoke) — a stdlib-only, static binary shipped
# alongside /cloud so the release gate can `docker exec` it against the freshly-built
# image (and any deployment can be smoked via `docker run --entrypoint /smoke ...`).
RUN --mount=type=cache,id=cloud-gomod-v4,target=/go/pkg/mod,sharing=locked \
--mount=type=cache,id=cloud-gobuild-v4,target=/root/.cache/go-build,sharing=locked \
CGO_ENABLED=0 go build -ldflags="-s -w" -o /smoke ./cmd/smoke
# Prove the SHIPPED binary binds sqlite3_* to libsqlcipher, not a plaintext libsqlite3.
RUN readelf -d /cloud | grep -qE 'NEEDED.*(sqlcipher|sqlite3)' || { echo "FATAL: /cloud links no sqlite/sqlcipher .so"; exit 1; }; \
! ldd /cloud 2>/dev/null | grep -E 'libsqlite3' | grep -vq 'libsqlcipher' || { echo "FATAL: /cloud resolves a NON-sqlcipher libsqlite3 (plaintext risk)"; exit 1; }
# ── final image (alpine, NOT scratch — CGO needs libc + libsqlcipher) ─────────
FROM public.ecr.aws/docker/library/alpine:3.22@sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc1f695dce
FROM ghcr.io/hanzoai/mirror/alpine:3.22@sha256:7c8cb692ae09657cbc4a3f3cbd0e8d5a2690ba38386aaaf252dbb060bf5eb2e6
ARG REVISION=unknown
LABEL org.opencontainers.image.revision="${REVISION}" \
org.opencontainers.image.source="https://github.com/hanzoai/cloud"
@@ -246,6 +172,7 @@ COPY --from=build /usr/share/zoneinfo /usr/share/zoneinfo
COPY --from=build /etc/passwd /etc/passwd
COPY --from=build /etc/group /etc/group
COPY --from=build /cloud /cloud
COPY --from=build /smoke /smoke
EXPOSE 8080 9090 9653
USER 65532:65532
ENTRYPOINT ["/cloud"]
+68 -14
View File
@@ -12,25 +12,34 @@ One way to do everything. Composable, orthogonal, DRY. A new subsystem is a
package under `clients/<name>` that obeys these seams — nothing more.
- **Subsystem shape.** A subsystem exposes `func Mount(app *zip.App, deps cloud.Deps) error`
and self-registers at init with `cloud.Register("<name>", <order>, cloud.Typed(Mount))`
(or `RegisterWithShutdown`). `Mount` wires that subsystem's `/v1/<name>/*` routes
onto the shared `*zip.App`; `cloud.Deps` carries the process-wide handles
(Logger, DataDir, the subsystem `Client` seams). No subsystem reaches into
another's internals.
and is listed in `apps.Wire()` as a `cloud.MountSpec{Name, Mount: cloud.Typed(Mount)}`
(plus `Shutdown`/`OwnsHealth` where it owns them). `Mount` wires that subsystem's
`/v1/<name>/*` routes onto the shared `*zip.App`; `cloud.Deps` carries the
process-wide handles (Logger, DataDir, the subsystem `Client` seams). No
subsystem reaches into another's internals. There is no init()-registry and no
`cloud.Register` — subsystems do NOT self-register.
- **Client seams.** Cross-subsystem calls go through a narrow in-process interface
published in `types` and aliased at the provider, e.g. `commerce.Client =
types.CommerceClient` (`GetOrgConfig` + `CheckEntitlement`). Consumers depend on
the interface, never the implementation; the seam rides zap-proto/zip. Keep each
interface minimal — add a method only when a consumer needs it.
- **Composition root.** `apps/apps.go` blank-imports every subsystem
(its init runs `cloud.Register`), populating `cloud.Registry`. `MountAll`
(build.go) sorts the registry by `Order` and calls `Mount` on each ENABLED
subsystem (`cfg.Enabled`). That ordered blank-import set IS the wiring — there
is no separate `Wire()` function; to add a subsystem you add one import line.
- **Route precedence is a framework guarantee.** The router is zap-proto/fiber
(zip v1.3.0). Most-specific route wins regardless of mount order; a genuine
route CONFLICT panics at mount rather than resolving ambiguously. Subsystems may
therefore mount in any order and still compose deterministically.
- **Composition root.** `apps/apps.go:Wire()` returns `[]cloud.MountSpec` — every
linked subsystem, in mount order, as ONE explicit slice read top-to-bottom.
Slice position IS the order: there is no `Order` field and `MountAll`
(build.go) does NOT sort; it iterates as-given and mounts each ENABLED spec
(`cfg.Enabled`). To add a subsystem you add one line to `Wire()`.
`apps/wire_test.go` freezes the sequence, so a reorder/drop/add fails there.
- **Route precedence.** The router is zap-proto/fiber (zip v1.8.3). Most-specific
route wins regardless of mount order, so subsystems may mount in any order and
still compose deterministically. But precedence is NOT a conflict guard: two
registrations of a byte-identical pattern do NOT panic — fiber MERGES them into
ONE route with both handlers chained, resolving by first-registration. That is
invisible to a `GetRoutes()` entry count (see the bots note below), and it is
NOT distinguishable from a legitimate middleware chain: `app.Post(path, mw1,
mw2, mw3, handler)` is one registration with four handlers (apps/commerce.go:151),
and the whole `/v1/store/*` surface is that shape. A high handler count is
therefore evidence of nothing on its own; only a subsystem that never chains
middleware (bots/visor/runtime) can read `len(Handlers) > 1` as a collision.
- **Per-org data.** The ONE way any subsystem opens a per-org SQLite file is
`cloud.OrgDB(dataDir, org, project, sub)` — or the cached `cloud.OrgStore[T]`
(`NewOrgStore` + `For(org, project)`). Path convention:
@@ -42,6 +51,51 @@ package under `clients/<name>` that obeys these seams — nothing more.
the SOLE driver (blank-imported once, in orgdb.go); subsystems never import a
SQLite driver themselves. The caller owns its schema/migration and Close.
## The route table has three projections, and the router is the source
`serve.go` composes ONE route table and projects it three ways, all after
`MountAll` so each sees a complete table: `/zap` REPLAYS the /v1 handlers
(zapface), the console RENDERS them, and `GET /v1/openapi.json` DESCRIBES them
(`openapi.Mount`). None holds a second copy of anything; none can drift.
- **The spec IS the router.** `openapi.Live(app)` reads
`app.Fiber().GetRoutes(true)` — fiber's own filter drops `Use()` middleware —
and every other function in `openapi/` is a pure function of that `[]Route`.
There is NO checked-in spec file to hand-maintain and no second registry. The
drift guard is `cmd/cloud/openapi_test.go`: a BIJECTION over the fully-mounted
`apps.Wire()` (983 operations / 692 paths / 109 products) — every live route
appears as an operation, every operation is backed by a live route. It is the
only test whose failure means the document lies.
- **Reading the LIVE router is the only total source.** `POST /v1/kms/auth/login`
is registered as `Group("/v1/kms/auth").Post("/login")` — no grep can find that
path; only the assembled router knows it. And the route set is a function of
deployment config (`cfg.Enabled`, plus internal gates like kms's `if kc != nil`),
so **the spec VARIES PER DEPLOYMENT** — correctly: a deployment that does not
mount admin does not advertise it. That is why the document is generated
per-process at request time, not built once in CI.
- **The product axis is mechanical.** The first path segment after `/v1/` IS the
product (`openapi.Product`), tagged onto each operation so a CLI can build
`hanzo <product> <resource> <verb>` with no judgment. It is deliberately NOT the
subsystem name: `clients/billing` also serves `/v1/finance/*`.
- **What the router CANNOT tell you — do not try to fix this in the generator.**
Method, path, path params, and product are derivable; request/response schemas,
query/header params, status codes, and auth are NOT. The router holds a
`func(*zip.Ctx) error`; the request type is a LOCAL inside the handler
(`var req secretPutRequest; json.Unmarshal(ctx.Body(), &req)`), and Go cannot
reflect from a func value into its body. `cloud.Handle[S]` does not help — `S`
is the SERVICE (service.go:90), not the payload; `cloud.Typed` is an
`any→*zip.App` mount adapter. The ONE path to schemas is zip's typed ops
(`zip.Get[In,Out]`), which carry the In/Out types and also yield an MCP tool
from the same registry (zip/openapi.go, zip/mcp.go — today `len(a.ops) == 0`,
so zip's own generator emits nothing here). `GetRoutes()` is a superset of
`app.ops`, so migrating a handler to a typed op adds schema without changing
this pipeline.
- **Catch-alls are opaque, by construction.** `app.Post("/v1/billing/*")` proxies
to another service, so `POST /v1/billing/deposit` is NOT a route in this process
and cannot appear. Measured on the live table: 3 products are wholly opaque
(`bot`, `licensing`, `sentry` — the catch-all IS the product) and 12 more mix
concrete ops with a catch-all hiding an unknown remainder.
## Cross-subsystem seams that are values, not places
- **The per-principal MCP plane is callable in-process.** `clients/automations`
+132 -104
View File
@@ -34,6 +34,7 @@ package apps
import (
"context"
"fmt"
"os"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
@@ -45,7 +46,6 @@ import (
"github.com/hanzoai/authz"
"github.com/hanzoai/licensing"
"github.com/hanzoai/metrics"
o11ymod "github.com/hanzoai/o11y"
// In-repo subsystem packages (clients/*). Each exports a Mount (and, where it
// owns process-lifetime resources, a Shutdown); Wire references them directly.
@@ -53,6 +53,7 @@ import (
"github.com/hanzoai/cloud/clients/admin"
"github.com/hanzoai/cloud/clients/ads"
"github.com/hanzoai/cloud/clients/affiliates"
"github.com/hanzoai/cloud/clients/agent"
"github.com/hanzoai/cloud/clients/agents"
"github.com/hanzoai/cloud/clients/agentskills"
"github.com/hanzoai/cloud/clients/analytics"
@@ -76,8 +77,7 @@ import (
"github.com/hanzoai/cloud/clients/entitlements"
"github.com/hanzoai/cloud/clients/eval"
"github.com/hanzoai/cloud/clients/exec"
"github.com/hanzoai/cloud/clients/featureflags"
"github.com/hanzoai/cloud/clients/featuregate"
"github.com/hanzoai/cloud/clients/flags"
"github.com/hanzoai/cloud/clients/framework"
"github.com/hanzoai/cloud/clients/functions"
"github.com/hanzoai/cloud/clients/gateway"
@@ -85,6 +85,7 @@ import (
"github.com/hanzoai/cloud/clients/graph"
"github.com/hanzoai/cloud/clients/guide"
"github.com/hanzoai/cloud/clients/iam"
"github.com/hanzoai/cloud/clients/iam2"
"github.com/hanzoai/cloud/clients/ingress"
"github.com/hanzoai/cloud/clients/integrations"
"github.com/hanzoai/cloud/clients/kafka"
@@ -114,6 +115,7 @@ import (
"github.com/hanzoai/cloud/clients/sign"
"github.com/hanzoai/cloud/clients/social"
"github.com/hanzoai/cloud/clients/storage"
"github.com/hanzoai/cloud/clients/sync"
"github.com/hanzoai/cloud/clients/tasks"
"github.com/hanzoai/cloud/clients/team"
"github.com/hanzoai/cloud/clients/templates"
@@ -161,6 +163,22 @@ func init() {
})
}
// identitySpec selects the ONE identity backend that owns /v1/iam/* (+ /login/oauth/*)
// for this boot. CLOUD_IAM_IMPL=iam2 picks the clean-room iam2 (zip+orm, beego-free);
// anything else — including unset, the production default — keeps the legacy beego
// Casdoor embed, byte-for-byte today's behavior. The two impls register the SAME
// absolute prefixes and therefore cannot co-mount, so selection (this func) stays
// separate from activation (cfg.Enabled): exactly one spec occupies the identity slot
// in Wire, preserving mount order either way. os.Getenv (not the unexported
// cloud.getenv, which is unreachable from package apps) is the read — CLOUD_IAM_IMPL is
// the deliberate, off-by-default opt-in that keeps iam2 inert until a canary flips it.
func identitySpec() cloud.MountSpec {
if os.Getenv("CLOUD_IAM_IMPL") == "iam2" {
return cloud.MountSpec{Name: "iam2", Mount: iam2.Mount}
}
return cloud.MountSpec{Name: "iam", Mount: iam.Mount}
}
// Wire returns every linked subsystem as a cloud.MountSpec, in mount order. The
// slice position IS the order: cloud.MountAll iterates it as-given, registering each
// subsystem's teardown as a zip shutdown hook so teardown runs in reverse (LIFO).
@@ -169,186 +187,200 @@ func init() {
func Wire() []cloud.MountSpec {
return []cloud.MountSpec{
// embedded NATS :4222 + JetStream.
{Name: "pubsub", Mount: cloud.Typed(pubsub.Mount), Shutdown: pubsub.Shutdown},
{Name: "pubsub", Mount: pubsub.Mount, Shutdown: pubsub.Shutdown},
// embedded Kafka adaptor :9092.
{Name: "kafka", Mount: cloud.Typed(kafka.Mount), Shutdown: kafka.Shutdown},
{Name: "kafka", Mount: kafka.Mount, Shutdown: kafka.Shutdown},
// /.well-known/agent-skills/* — before IAM's /.well-known/* wildcard (50).
{Name: "agentskills", Mount: cloud.Typed(agentskills.Mount)},
{Name: "agentskills", Mount: agentskills.Mount},
// Insights feature-flag evaluation seam (no routes; a hot value plane).
{Name: "featureflags", Mount: cloud.Typed(featureflags.Mount), Shutdown: featureflags.Shutdown, OwnsHealth: true},
{Name: "flags", Mount: flags.Mount, Shutdown: flags.Shutdown, OwnsHealth: true},
// Embedded KMS secrets plane /v1/kms/*. OwnsHealth: serves its own fail-closed
// /v1/kms/health (the generic always-ok route must not shadow it). Fails closed
// until the operator injects CLOUD_KMS_MASTER_KEY_REF. (Its in-process client
// factory is registered separately via cloud.RegisterKMSClientFactory.)
{Name: "kms", Mount: cloud.Typed(kms.Mount), OwnsHealth: true},
{Name: "kms", Mount: kms.Mount, OwnsHealth: true},
// hanzoai/metrics — native o11y. It declares its OWN narrow metrics.Deps (no
// hanzoai/cloud import), so Typed cannot adapt it; mountMetrics builds that Deps
// from cloud.Deps and calls metrics.Mount explicitly.
{Name: "metrics", Mount: mountMetrics},
// Embedded runtime edge (/v1/ingress/*). STAGED — edge listeners stay off unless
// the operator names "ingress" in CLOUD_ENABLE.
{Name: "ingress", Mount: cloud.Typed(ingress.Mount), Shutdown: ingress.Shutdown},
{Name: "ingress", Mount: ingress.Mount, Shutdown: ingress.Shutdown},
// SPECIFIC self-service routes (/v1/iam/{keys,onboard}, /v1/csrf, /v1/embed-status,
// /v1/commerce/topup/wallet). MUST mount before the IAM /v1/iam/* wildcard (50) so
// they win Fiber's first-match scan (framework-guaranteed since zip v1.3.0).
{Name: "account", Mount: cloud.Typed(account.MountAccount)},
{Name: "account", Mount: account.MountAccount},
// Embedded IAM identity plane (/v1/iam/*, /.well-known/*, /login/oauth/*, /_/iam/*,
// /cas/*, /scim/*) — the identity authority, mounts before its dependents. STAGED:
// the operator adds "iam" to --enable only after IAM config + the fold are verified.
{Name: "iam", Mount: cloud.Typed(iam.Mount)},
// Which IMPLEMENTATION owns these prefixes is selected by CLOUD_IAM_IMPL
// (identitySpec): the clean-room iam2 (zip+orm, beego-free) when =="iam2", else the
// legacy beego Casdoor embed — the default (unset = today's behavior, byte-for-byte).
// Both register the SAME absolute paths and cannot co-mount, so this is an either/or
// switch at this ONE slot, never a shadow prefix.
identitySpec(),
// Embedded Base app engine + viral waitlist (/v1/waitlist/*). STAGED behind
// CLOUD_BASE_EMBED. OwnsHealth: native /v1/base/health.
{Name: "base", Mount: cloud.Typed(base.Mount), Shutdown: base.Shutdown, OwnsHealth: true},
// In-repo o11y READ plane + the runtime-handler install (o11y.SetHandler). Every
// specific /v1/o11y/* route registers INSIDE this one mount, hence BEFORE the
// hanzoai/o11y module wildcard (70) — Fiber's in-order match gives them precedence.
// OwnsHealth: the module's order-70 co-entry below owns the single /v1/o11y/health.
{Name: "o11y", Mount: o11y.MountO11y, Shutdown: o11y.ShutdownO11y, OwnsHealth: true},
// hanzoai/o11y module wildcard /v1/o11y/* — co-owner of the ONE o11y concept with
// the in-repo entry above (same name), delegated to via o11y.SetHandler.
{Name: "o11y", Mount: cloud.Typed(o11ymod.Mount)},
{Name: "authz", Mount: cloud.Typed(authz.Mount)},
{Name: "base", Mount: base.Mount, Shutdown: base.Shutdown, OwnsHealth: true},
// The ONE observability subsystem: the in-repo o11y READ plane + runtime-handler
// install (o11y.SetHandler), with the hanzoai/o11y module wildcard /v1/o11y/*
// folded in as the TERMINAL sub-mount INSIDE o11y.MountO11y. Every specific
// /v1/o11y/* route registers before that wildcard, so Fiber's in-order match gives
// them precedence. NOT OwnsHealth: /v1/o11y/health stays the generic always-ok
// route (registered before MountAll), exactly as when the former module co-entry —
// which also set OwnsHealth=false — triggered it.
{Name: "o11y", Mount: o11y.MountO11y, Shutdown: o11y.ShutdownO11y},
{Name: "authz", Mount: authz.Mount},
// Embedded commerce plane /v1/commerce/*, /_/commerce/* — the hanzoai/commerce
// MODULE via the adapter in commerce.go (un-forked; the in-process
// CommerceClient is wired directly in pickCommerceClient).
{Name: "commerce", Mount: cloud.Typed(mountCommerce)},
// hanzoai/licensing. Its Mount is func(any, cloud.Deps) error — a MountFunc
// already — so Wire references it DIRECTLY, not through Typed.
{Name: "commerce", Mount: mountCommerce},
{Name: "licensing", Mount: licensing.Mount},
{Name: "plans", Mount: cloud.Typed(plan.Mount), OwnsHealth: true},
{Name: "pricing", Mount: cloud.Typed(pricing.Mount), OwnsHealth: true},
{Name: "plans", Mount: plan.Mount, OwnsHealth: true},
{Name: "pricing", Mount: pricing.Mount, OwnsHealth: true},
// /v1/s3/buckets/* + /v1/s3/health. Mounts BEFORE provisioning (120) so its static
// routes win over provisioning's /v1/s3/:name. OwnsHealth (real fail-closed probe).
{Name: "storage", Mount: cloud.Typed(storage.Mount), OwnsHealth: true},
{Name: "storage", Mount: storage.Mount, OwnsHealth: true},
// Provisioning control plane: /v1/sql,/v1/vector,/v1/datastore,/v1/kv,/v1/search,/v1/s3,/v1/docdb.
{Name: "provisioning", Mount: cloud.Typed(provisioning.Mount)},
{Name: "billing", Mount: cloud.Typed(billing.Mount)},
{Name: "provisioning", Mount: provisioning.Mount},
{Name: "billing", Mount: billing.Mount},
// CATCH-ALL /v1/billing/* + /v1/commerce/* data bridges — AFTER clients/billing
// (121) + the commerce embed (100). Same clients/account package as "account" (48).
{Name: "account-bridge", Mount: cloud.Typed(account.MountBridge)},
{Name: "do", Mount: cloud.Typed(do.Mount)},
{Name: "platform", Mount: cloud.Typed(platform.Mount), OwnsHealth: true},
{Name: "projects", Mount: cloud.Typed(projects.Mount)},
{Name: "prompts", Mount: cloud.Typed(prompts.Mount)},
{Name: "agents", Mount: cloud.Typed(agents.Mount), Shutdown: agents.Shutdown},
{Name: "account-bridge", Mount: account.MountBridge},
{Name: "do", Mount: do.Mount},
{Name: "platform", Mount: platform.Mount, OwnsHealth: true},
{Name: "projects", Mount: projects.Mount},
{Name: "prompts", Mount: prompts.Mount},
{Name: "agents", Mount: agents.Mount, Shutdown: agents.Shutdown},
// The unified AI login manager registry (/v1/links). Mounts AFTER agents so
// a link revoke can stop the affected agent sessions in-process.
{Name: "link", Mount: cloud.Typed(link.Mount), Shutdown: link.Shutdown},
{Name: "wallets", Mount: cloud.Typed(wallets.Mount), Shutdown: ctxShutdown(wallets.Shutdown)},
{Name: "link", Mount: link.Mount, Shutdown: link.Shutdown},
{Name: "wallets", Mount: wallets.Mount, Shutdown: ctxShutdown(wallets.Shutdown)},
// x402 pay-per-use: settles a signed ERC-3009 authorization to a recipient
// wallet through the metering spine. Mounts AFTER wallets (it resolves the
// recipient via wallets.ResolvePaymentTarget) and provides the Enforce
// middleware a marketplace applies to its priced routes.
{Name: "x402", Mount: cloud.Typed(x402.Mount), Shutdown: ctxShutdown(x402.Shutdown)},
{Name: "paas", Mount: cloud.Typed(paas.Mount), OwnsHealth: true},
{Name: "x402", Mount: x402.Mount, Shutdown: ctxShutdown(x402.Shutdown)},
{Name: "paas", Mount: paas.Mount, OwnsHealth: true},
// GitOps deploy dashboard /v1/deploy/* (the ArgoCD-grade fleet view over the
// operator App CRs). After paas so the release seam paas installs is registered
// before a gitops rollback delegates to it; owns its own /v1/deploy/health.
{Name: "deploy", Mount: cloud.Typed(deploy.Mount), OwnsHealth: true},
{Name: "functions", Mount: cloud.Typed(functions.Mount)},
{Name: "tracker", Mount: cloud.Typed(tracker.Mount)},
{Name: "templates", Mount: cloud.Typed(templates.Mount)},
{Name: "framework", Mount: cloud.Typed(framework.Mount), Shutdown: ctxShutdown(framework.Shutdown)},
{Name: "knowledge", Mount: cloud.Typed(knowledge.Mount)},
{Name: "deploy", Mount: deploy.Mount, OwnsHealth: true},
{Name: "functions", Mount: functions.Mount},
{Name: "tracker", Mount: tracker.Mount},
{Name: "templates", Mount: templates.Mount},
{Name: "framework", Mount: framework.Mount, Shutdown: ctxShutdown(framework.Shutdown)},
{Name: "knowledge", Mount: knowledge.Mount},
// Marketing content loop /v1/content/* (generate → CMS → transition → publish).
// After framework (its DocType store the ops read/write) + knowledge (the sibling
// framework lane); before the AI /v1/* catch-all so /v1/content/* resolves here.
// CRUD/tenancy/install are framework's; this adds the board, lifecycle transition,
// and the generate/publish orchestration over the zen5 + studio + social edges.
{Name: "content", Mount: cloud.Typed(content.Mount), Shutdown: ctxShutdown(content.Shutdown)},
{Name: "content", Mount: content.Mount, Shutdown: ctxShutdown(content.Shutdown)},
// Reverse storefront loop: consume the commerce COMMERCE stream (product.created)
// → content.EnsureCatalogAsset (render the new product's ecom asset, design==slug).
// After content (whose EnsureCatalogAsset it drives). Inert until CLOUD_COMMERCE_NATS_URL
// names the NATS carrying commerce catalog events — the reverse of the forward edge.
{Name: "catalogsync", Mount: cloud.Typed(catalogsync.Mount), Shutdown: catalogsync.Shutdown},
{Name: "ml", Mount: cloud.Typed(ml.Mount), OwnsHealth: true},
{Name: "usage", Mount: cloud.Typed(usage.Mount)},
{Name: "crm", Mount: cloud.Typed(crm.Mount)},
{Name: "catalogsync", Mount: catalogsync.Mount, Shutdown: catalogsync.Shutdown},
{Name: "ml", Mount: ml.Mount, OwnsHealth: true},
{Name: "usage", Mount: usage.Mount},
{Name: "crm", Mount: crm.Mount},
// Native /v1/marketing/* — the in-process fold of github.com/hanzoai/marketing
// (per-org campaign store on Base/SQLite), twin of crm. Owns a DB handle, so
// its Shutdown closes it cleanly on SIGTERM (ctxShutdown adapts func() error).
{Name: "marketing", Mount: cloud.Typed(marketing.Mount), Shutdown: ctxShutdown(marketing.Shutdown)},
{Name: "marketing", Mount: marketing.Mount, Shutdown: ctxShutdown(marketing.Shutdown)},
// Native /v1/ads/* — the net-new per-org ad-campaign store on Base/SQLite,
// twin of crm/marketing. Owns a DB handle, so its Shutdown closes it cleanly
// on SIGTERM (ctxShutdown adapts func() error).
{Name: "ads", Mount: cloud.Typed(ads.Mount), Shutdown: ctxShutdown(ads.Shutdown)},
{Name: "ads", Mount: ads.Mount, Shutdown: ctxShutdown(ads.Shutdown)},
// Native /v1/social/* — the in-process fold of the live social stack
// (github.com/hanzoai/social: social-backend/frontend/orchestrator, a Postiz-style
// scheduler), a per-org accounts+posts store on Base/SQLite, twin of crm. Owns a DB
// handle, so its Shutdown closes it cleanly on SIGTERM (ctxShutdown adapts func() error).
{Name: "social", Mount: cloud.Typed(social.Mount), Shutdown: ctxShutdown(social.Shutdown)},
{Name: "analytics", Mount: cloud.Typed(analytics.Mount), OwnsHealth: true},
{Name: "git", Mount: cloud.Typed(git.Mount)},
{Name: "visor", Mount: cloud.Typed(visor.Mount)},
{Name: "social", Mount: social.Mount, Shutdown: ctxShutdown(social.Shutdown)},
{Name: "analytics", Mount: analytics.Mount, OwnsHealth: true},
{Name: "git", Mount: git.Mount},
// Universal sync (/v1/sync/links + engine). Registers the cloud.SyncEngine the
// GitHub/Gitea webhooks enqueue to; git is its first provider. Owns per-org
// DB handles, so its Shutdown closes them on SIGTERM.
{Name: "sync", Mount: sync.Mount, Shutdown: ctxShutdown(sync.Shutdown)},
{Name: "visor", Mount: visor.Mount},
// Cap table on Base via goja. STAGED behind CLOUD_ENABLE.
{Name: "captable", Mount: cloud.Typed(captable.Mount), Shutdown: captable.Shutdown},
{Name: "code", Mount: cloud.Typed(code.Mount), Shutdown: code.Shutdown},
{Name: "zero-trust", Mount: cloud.Typed(zt.Mount)},
{Name: "captable", Mount: captable.Mount, Shutdown: captable.Shutdown},
{Name: "code", Mount: code.Mount, Shutdown: code.Shutdown},
{Name: "zero-trust", Mount: zt.Mount},
// Data rooms via goja + per-tenant Base. STAGED behind CLOUD_ENABLE. OwnsHealth.
{Name: "dataroom", Mount: cloud.Typed(dataroom.Mount), Shutdown: dataroom.Shutdown, OwnsHealth: true},
{Name: "graph", Mount: cloud.Typed(graph.Mount)},
{Name: "security", Mount: cloud.Typed(security.Mount), Shutdown: ctxShutdown(security.Shutdown), OwnsHealth: true},
{Name: "integrations", Mount: cloud.Typed(integrations.Mount), Shutdown: integrations.Shutdown},
{Name: "sbom", Mount: cloud.Typed(sbom.Mount), OwnsHealth: true},
{Name: "team", Mount: cloud.Typed(team.Mount), Shutdown: ctxShutdown(team.Shutdown)},
{Name: "settings", Mount: cloud.Typed(settings.Mount), Shutdown: settings.Shutdown},
{Name: "notify", Mount: cloud.Typed(notify.Mount), OwnsHealth: true},
{Name: "gateway", Mount: cloud.Typed(gateway.Mount)},
{Name: "entitlements", Mount: cloud.Typed(entitlements.Mount), Shutdown: entitlements.Shutdown},
{Name: "exec", Mount: cloud.Typed(exec.Mount)},
{Name: "websearch", Mount: cloud.Typed(websearch.Mount)},
{Name: "world", Mount: cloud.Typed(world.Mount), Shutdown: ctxShutdown(world.Shutdown)},
{Name: "dataroom", Mount: dataroom.Mount, Shutdown: dataroom.Shutdown, OwnsHealth: true},
{Name: "graph", Mount: graph.Mount},
{Name: "security", Mount: security.Mount, Shutdown: ctxShutdown(security.Shutdown), OwnsHealth: true},
{Name: "integrations", Mount: integrations.Mount, Shutdown: integrations.Shutdown},
{Name: "sbom", Mount: sbom.Mount, OwnsHealth: true},
{Name: "team", Mount: team.Mount, Shutdown: ctxShutdown(team.Shutdown)},
{Name: "settings", Mount: settings.Mount, Shutdown: settings.Shutdown},
{Name: "notify", Mount: notify.Mount, OwnsHealth: true},
{Name: "gateway", Mount: gateway.Mount},
{Name: "entitlements", Mount: entitlements.Mount, Shutdown: entitlements.Shutdown},
{Name: "exec", Mount: exec.Mount},
{Name: "websearch", Mount: websearch.Mount},
{Name: "world", Mount: world.Mount, Shutdown: ctxShutdown(world.Shutdown)},
// The bot runtime's ops face (/v1/bot/*). The transport itself is domain-free;
// the run control plane is "bots" below.
{Name: "runtime", Mount: cloud.Typed(runtime.Mount)},
{Name: "authors", Mount: cloud.Typed(authors.Mount), Shutdown: ctxShutdown(authors.Shutdown)},
{Name: "bots", Mount: cloud.Typed(bots.Mount)},
{Name: "audit", Mount: cloud.Typed(auditlog.Mount)},
{Name: "affiliates", Mount: cloud.Typed(affiliates.Mount)},
{Name: "runtime", Mount: runtime.Mount},
{Name: "authors", Mount: authors.Mount, Shutdown: ctxShutdown(authors.Shutdown)},
{Name: "bots", Mount: bots.Mount},
{Name: "audit", Mount: auditlog.Mount},
{Name: "affiliates", Mount: affiliates.Mount},
// Hanzo Sign (e-signature) via goja + per-tenant Base. STAGED behind CLOUD_ENABLE. OwnsHealth.
{Name: "sign", Mount: cloud.Typed(sign.Mount), Shutdown: sign.Shutdown, OwnsHealth: true},
{Name: "product", Mount: cloud.Typed(product.Mount)},
{Name: "evals", Mount: cloud.Typed(eval.Mount)},
{Name: "treasury", Mount: cloud.Typed(treasury.Mount), Shutdown: ctxShutdown(treasury.Shutdown)},
{Name: "admin", Mount: cloud.Typed(admin.Mount)},
// Launch-control plane: per-service waitlist-mode registry (global SQLite)
// + admin board/toggle (/v1/admin/services*) + the guard's runtime mode read
// (/v1/featuregate/mode). Mounts right after admin so its specific routes bind
// ahead of the AI /v1/* catch-all; native Enforce middleware (wired in serve.go)
// reads the SAME store in-process. Owns a global store, so Shutdown closes it.
{Name: "featuregate", Mount: cloud.Typed(featuregate.Mount), Shutdown: ctxShutdown(featuregate.Shutdown)},
{Name: "tasks", Mount: cloud.Typed(tasks.Mount)},
{Name: "sign", Mount: sign.Mount, Shutdown: sign.Shutdown, OwnsHealth: true},
{Name: "product", Mount: product.Mount},
{Name: "evals", Mount: eval.Mount},
{Name: "treasury", Mount: treasury.Mount, Shutdown: ctxShutdown(treasury.Shutdown)},
{Name: "admin", Mount: admin.Mount},
// Launch-control (per-service waitlist mode) folded into the flags engine: the
// mode IS the switch waitlist.<svc>, the board is the /v1/admin/services lens,
// and /v1/featuregate/mode is served by flags. featuregate is no longer a mounted
// subsystem — it exposes only the native Enforce middleware (wired in serve.go),
// a consumer of flags.WaitlistModeForHost.
{Name: "tasks", Mount: tasks.Mount},
// Platform cron: durable schedules on the shared tasks engine replacing
// every k8s CronJob — entries are cron.hanzo.ai ConfigMaps (universe git),
// runs visible in the Tasks console. Mounts no routes; starts after the
// engine is wired.
{Name: "cron", Mount: cloud.Typed(cron.Mount)},
{Name: "automations", Mount: cloud.Typed(automations.Mount), Shutdown: automations.Shutdown},
{Name: "cron", Mount: cron.Mount},
{Name: "automations", Mount: automations.Mount, Shutdown: automations.Shutdown},
// Native single-connector execution (HIP-0126): runs an ActivePieces JS
// connector action in-process via goja (clients/connectorruntime), retiring
// the standalone auto Node engine. Mounts POST /v1/automations/connectors/:id/run,
// paired with the automations catalogue above; STAGED like the rest.
{Name: "connectorruntime", Mount: cloud.Typed(connectorruntime.Mount)},
{Name: "connectorruntime", Mount: connectorruntime.Mount},
// Unified tool plane: /v1/tools/* — the ONE registry (connectors, functions,
// agents, skills, external MCP servers, full-cloud-control /v1 routes), per-org
// activation, and the unified MCP endpoint. Sources register into it from their
// own Mounts, so mount position is not load-bearing (List/Dispatch run at request
// time); placed after automations, before the zen/ai catch-all so /v1/tools wins.
{Name: "tools", Mount: cloud.Typed(tools.Mount), Shutdown: tools.Shutdown},
{Name: "tools", Mount: tools.Mount, Shutdown: tools.Shutdown},
// Marketplace: /v1/marketplace/* — listing/discovery/install over the tool plane,
// with x402-priced monetized listings. Mounts after tools (it fills the price seam).
{Name: "marketplace", Mount: cloud.Typed(marketplace.Mount), Shutdown: marketplace.Shutdown},
{Name: "referrals", Mount: cloud.Typed(referrals.Mount)},
{Name: "marketplace", Mount: marketplace.Mount, Shutdown: marketplace.Shutdown},
{Name: "referrals", Mount: referrals.Mount},
// Business AI Guide /v1/guide/* — the interactive launch checklist engine +
// the agent that executes a step through the per-principal MCP plane. After
// automations (whose InvokeTool it drives) and referrals; before the ai
// catch-all. Owns per-org SQLite, so its Shutdown closes the stores.
{Name: "guide", Mount: cloud.Typed(guide.Mount), Shutdown: ctxShutdown(guide.Shutdown)},
{Name: "guide", Mount: guide.Mount, Shutdown: ctxShutdown(guide.Shutdown)},
// Hanzo Company — the incorporation + fundraising state machine
// (/v1/company/*). Mounts after the seams it composes (integrations for the
// google token custody; captable/dataroom facades) and before the /v1/* AI
// catch-all so its routes resolve here.
{Name: "company", Mount: cloud.Typed(company.Mount), Shutdown: company.Shutdown},
{Name: "company", Mount: company.Mount, Shutdown: company.Shutdown},
// Chat orchestrator — POST /v1/chat: ONE LLM tool-calling round over the tool
// plane. It COMPOSES the ai completion path (in-process, so per-org billing
// runs) + the unified tool registry, and splits the model's tool calls into
// server-executed actions and client-applied ops. Mounts BEFORE the zen/ai
// catch-all so /v1/chat resolves here (Fiber first-match); the ai module's
// beego /v1/chat alias behind its /v1/* glob is thereby shadowed, while ai
// keeps /v1/chat/completions + /v1/completions.
{Name: "agent", Mount: agent.Mount},
// The bare /v1/* AI catch-all — the LAST route position. Every owning subsystem above
// wins its own namespace (Fiber first-match); AI is the fallback for the rest of /v1/*.
// zen mounts as a /v1-scoped Claim middleware BEFORE ai: it routes zen* models
@@ -357,9 +389,9 @@ func Wire() []cloud.MountSpec {
// other model and the /v1/models list. Order is load-bearing — Claim must
// run before ai's catch-all. (See hip-00NN.)
{Name: "zen", Mount: mountZen},
{Name: "ai", Mount: cloud.Typed(ai.Mount)},
{Name: "ai", Mount: ai.Mount},
// Runtime wasm/proxy plugins — mounts dead last.
{Name: "plugins", Mount: cloud.Typed(plugin.Mount)},
{Name: "plugins", Mount: plugin.Mount},
}
}
@@ -386,11 +418,7 @@ func ServeSingle(name string) error {
// externals, metrics declares its OWN narrow Deps (Logger, DataDir, Brand) and does
// not import hanzoai/cloud, so cloud.Typed cannot bridge it: the composition root
// builds metrics.Deps from cloud.Deps and calls metrics.Mount explicitly here.
func mountMetrics(app any, deps cloud.Deps) error {
a, ok := app.(*zip.App)
if !ok {
return fmt.Errorf("metrics.Mount: app is %T, want *zip.App", app)
}
func mountMetrics(a *zip.App, deps cloud.Deps) error {
return metrics.Mount(a, metrics.Deps{Logger: deps.Logger, DataDir: deps.DataDir, Brand: deps.Brand})
}
+40 -1
View File
@@ -20,6 +20,7 @@ import (
"fmt"
"net/http"
"path/filepath"
"strings"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/commerceclient"
@@ -115,6 +116,13 @@ func mountCommerce(app *zip.App, deps cloud.Deps) error {
// THE native co-residence contract: commerce registers its routes on
// cloud's own app — no second engine, no net/http adaptation.
App: app,
// ONE LEDGER: commerce's POST /v1/billing/credit mints into cloud's native
// finance ledger (the SAME per-org account the AI spend-gate reads), so a
// granted credit is immediately spendable. commerce.Embed calls
// creditledger.Set(this) before routes register; nil would leave commerce on
// its own datastore (standalone), but in this unified binary finance is
// co-resident, so we inject the finance-backed ledger adapter.
Ledger: ledger{},
})
if err != nil {
lg.Error("commerce embed failed — serving fail-closed 503 (cloud stays up)", "err", err)
@@ -126,7 +134,7 @@ func mountCommerce(app *zip.App, deps cloud.Deps) error {
// mirrors the standalone /v1 bundle: gated request context, host, IAM
// resolution; store.Route's own tokenRequired arg gates the CRUD.
storeV1 := app.Group("/v1")
storeV1.Use(commercemid.AddHost(), commercemid.RequestContext(), commercemid.ErrorHandlerJSON())
storeV1.Use(commercemid.AddHost(), commercemid.RequestContext(), commerceErrorScope())
// Unconditional, exactly like the standalone bundle: IAMTokenRequired
// no-ops gracefully when IAM is not initialized.
storeV1.Use(iammiddleware.IAMTokenRequired())
@@ -166,6 +174,37 @@ func mountCommerce(app *zip.App, deps cloud.Deps) error {
// mountCommerceFailClosed serves an honest JSON 503 on every commerce prefix when
// the embed cannot boot, so /v1/commerce/* answers "commerce unavailable" instead
// commerceErrorScope confines commerce's JSON error envelope to commerce's OWN
// routes. commercemid.ErrorHandlerJSON is a `/v1` GROUP middleware, and fiber
// matches group middleware by PREFIX, not by the handle a route registered on —
// so on the shared `/v1` it wraps every subsystem mounted AFTER commerce and
// flattens their typed zip.HTTPError (403/400/…) into a blanket 500 (the store
// envelope always renders 500). Guarded by commercePrefixes, the envelope stays on
// commerce and every other subsystem renders its own status via zip's default
// handler — the pre-commerce subsystems (kms, o11y, …) already do; this makes the
// post-commerce ones (projects, agents, wallets, …) match.
func commerceErrorScope() zip.Handler {
envelope := commercemid.ErrorHandlerJSON()
return func(c *zip.Ctx) error {
if hasCommercePrefix(c.Path()) {
return envelope(c)
}
return c.Next()
}
}
// hasCommercePrefix reports whether path is a commerce-owned root (an exact prefix
// or a child of one), the SAME ownership commercePrefixes encodes for the
// fail-closed mount.
func hasCommercePrefix(path string) bool {
for _, p := range commercePrefixes {
if path == p || strings.HasPrefix(path, p+"/") {
return true
}
}
return false
}
// of falling through to another subsystem's catch-all.
func mountCommerceFailClosed(app *zip.App) {
failed := func(c *zip.Ctx) error {
+82
View File
@@ -0,0 +1,82 @@
package apps
import (
"io"
"net/http/httptest"
"strings"
"testing"
commercemid "github.com/hanzoai/commerce/middleware"
"github.com/zap-proto/zip"
)
// TestCommerceErrorScope proves commerceErrorScope() confines commerce's always-500
// JSON envelope to commerce-owned prefixes: a post-commerce subsystem route
// (/v1/projects) that returns a typed 403 renders 403 (zip default), while a
// commerce route (/v1/store/...) still gets commerce's envelope. Mirrors the frozen
// order: kms (before commerce) → commerce /v1 group chain → projects + store (after).
// Regression for the release-smoke failure where 14 post-commerce endpoints 500'd.
func TestCommerceErrorScope(t *testing.T) {
app := zip.New(zip.Config{})
// kms (before commerce) — a clean 403 baseline (never wrapped by commerce).
app.Get("/v1/kms/health", func(c *zip.Ctx) error { return zip.ErrForbidden("kms says no") })
// commerce (position 39): the REAL group chain, now with the scoped envelope.
sv1 := app.Group("/v1")
sv1.Use(commercemid.AddHost(), commercemid.RequestContext(), commerceErrorScope())
// projects (after commerce) — typed 403; must NOT be clobbered to 500.
app.Get("/v1/projects", func(c *zip.Ctx) error { return zip.ErrForbidden("X-Org-Id required") })
// a commerce store route (after commerce) — typed 403; commerce envelope applies.
app.Get("/v1/store/current", func(c *zip.Ctx) error { return zip.ErrForbidden("store needs org") })
probe := func(path string) (int, string) {
req := httptest.NewRequest("GET", path, nil)
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("%s: %v", path, err)
}
defer resp.Body.Close()
b, _ := io.ReadAll(resp.Body)
return resp.StatusCode, string(b)
}
// The invariant the scope guards: a typed zip.HTTPError (403) a subsystem returns
// is NEVER flattened to a 500 — not before commerce, not after it. (commerce
// >=1.48.10 honors the status itself; the scope keeps commerce's error handler off
// other subsystems' routes regardless, so a future commerce regression can't
// re-clobber them.)
for _, tc := range []struct {
path string
wantCode int
}{
{"/v1/kms/health", 403}, // before commerce
{"/v1/projects", 403}, // after commerce — must NOT be clobbered to 500
{"/v1/store/current", 403}, // commerce's own route — its handler still honors 403
} {
code, body := probe(tc.path)
t.Logf("%-20s -> %d %s", tc.path, code, body)
if code != tc.wantCode {
t.Errorf("%s: got %d, want %d (%s)", tc.path, code, tc.wantCode, body)
}
if strings.Contains(body, "\"status\":5") || code >= 500 {
t.Errorf("%s: a typed 403 was flattened to a 5xx (%s)", tc.path, body)
}
}
for _, p := range []struct {
path string
own bool
}{
{"/v1/store/current", true},
{"/v1/commerce/checkout", true},
{"/v1/projects", false},
{"/v1/agent/presets", false},
{"/v1/agents", false},
} {
if got := hasCommercePrefix(p.path); got != p.own {
t.Errorf("hasCommercePrefix(%q) = %v, want %v", p.path, got, p.own)
}
}
}
+81
View File
@@ -0,0 +1,81 @@
// Copyright © 2026 Hanzo AI. MIT License.
package apps
import (
"context"
"fmt"
"github.com/hanzoai/cloud/clients/finance"
"github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/cloud/types"
"github.com/hanzoai/commerce/billing/creditledger"
)
// ledger implements commerce's creditledger.CreditLedger over cloud's native
// finance ledger — the SAME per-org account (finance.Current()) the AI spend-gate
// reads and the edge meter debits. Injected at mountCommerce (EmbedConfig.Ledger),
// it makes commerce's POST /v1/billing/credit mint into the ONE ledger: a granted
// credit is immediately visible to the gate (one ledger, no split). This is the
// cloud half of the one-ledger seam — commerce defines the interface, cloud
// implements it once here, the compiler enforces the match.
//
// Fails closed when no finance ledger is co-resident; in the unified cloud binary
// finance is always published, so Get() != nil ⇒ credit routes here.
type ledger struct{}
// compile-time proof the adapter satisfies commerce's exported seam.
var _ creditledger.CreditLedger = ledger{}
// Credit posts a balanced deposit (funding:platform → wallet) to the org's POOL
// account (Subject == Org, the wallet the gate reads) and returns the ledger entry
// id + the org's new available balance in cents. Idempotent on IdempotencyKey:
// finance dedups on Ref, so the same key credits AT MOST once.
func (ledger) Credit(ctx context.Context, in creditledger.CreditInput) (string, int64, error) {
fin := finance.Current()
if fin == nil {
return "", 0, fmt.Errorf("commerce credit: no finance ledger co-resident")
}
cur := in.Currency
if cur == "" {
cur = "usd"
}
tag := in.Tag
if tag == "" {
tag = "grant:admin" // non-cash grant bucket (finance is a single wallet; Tags is a memo)
}
id, err := fin.Deposit(ctx, types.DepositInput{
Org: in.Org,
Subject: in.Org, // org-pool wallet == the account the AI gate reads
Amount: money.FromCents(in.AmountCents),
Currency: cur,
Notes: in.Reason,
Tags: tag,
Ref: in.IdempotencyKey,
})
if err != nil {
return "", 0, err
}
bal, berr := fin.Balance(ctx, in.Org, in.Org, cur, false)
if berr != nil {
return id, 0, berr
}
return id, bal.Cents(), nil
}
// Balance returns the org pool's available balance in cents for currency — the
// same read the AI gate performs, so GET /v1/billing/balance and the gate agree.
func (ledger) Balance(ctx context.Context, org, currency string) (int64, error) {
fin := finance.Current()
if fin == nil {
return 0, fmt.Errorf("commerce balance: no finance ledger co-resident")
}
if currency == "" {
currency = "usd"
}
bal, err := fin.Balance(ctx, org, org, currency, false)
if err != nil {
return 0, err
}
return bal.Cents(), nil
}
+12 -11
View File
@@ -16,8 +16,8 @@ import "testing"
// origin/main @c504d2b: 68 specs from the live init()-registry, PLUS the two the
// wave-2 external bumps (ai v1.805.2, o11y v1.5.12) stopped self-registering — ai
// (@150 catch-all) and the hanzoai/o11y module wildcard (@70), which main currently
// DROPS and this PR restores at their order-int slots. "o11y" therefore appears
// twice: the in-repo read plane (order 69) and the module wildcard (70), co-owners.
// DROPS and this PR restores. The module wildcard (order 70) is now folded in as the
// terminal sub-mount of the in-repo o11y read plane (order 69), so "o11y" is ONE spec.
var frozen = []struct {
name string
ownsHealth bool
@@ -26,15 +26,14 @@ var frozen = []struct {
{"pubsub", false, true}, // was order 5
{"kafka", false, true}, // was order 6
{"agentskills", false, false}, // was order 8
{"featureflags", true, true}, // was order 9; native engine: /v1/flags health + store shutdown
{"flags", true, true}, // was order 9; native engine: /v1/flags health + store shutdown
{"kms", true, false}, // was order 10
{"metrics", false, false}, // was order 40
{"ingress", false, true}, // was order 42
{"account", false, false}, // was order 48
{"iam", false, false}, // was order 50
{"base", true, true}, // was order 60; per-org embed added Shutdown (#298)
{"o11y", true, true}, // was order 69
{"o11y", false, false}, // was order 70
{"o11y", false, true}, // ONE observability subsystem (was co-owned orders 69+70): read plane + the hanzoai/o11y module wildcard folded in as MountO11y's terminal sub-mount. OwnsHealth=false keeps /v1/o11y/health the generic always-ok route the module co-entry used to trigger.
{"authz", false, false}, // was order 70
{"commerce", false, false}, // was order 100
{"licensing", false, false}, // was order 110
@@ -69,6 +68,7 @@ var frozen = []struct {
{"social", false, true}, // new: /v1/social fold (after crm)
{"analytics", true, false}, // was order 132
{"git", false, false}, // was order 132
{"sync", false, true}, // /v1/sync engine (owns per-org DB handles → Shutdown)
{"visor", false, false}, // was order 133
{"captable", false, true}, // was order 133
{"code", false, true}, // was order 134
@@ -96,7 +96,6 @@ var frozen = []struct {
{"evals", false, false}, // was order 145
{"treasury", false, true}, // was order 146
{"admin", false, false}, // was order 146
{"featuregate", false, true}, // new: launch-control waitlist-mode plane (after admin, was order 147)
{"tasks", false, false}, // was order 147
{"cron", false, false}, // durable platform cron on the shared engine (post-freeze add)
{"automations", false, true}, // was order 148
@@ -106,6 +105,7 @@ var frozen = []struct {
{"referrals", false, false}, // was order 149
{"guide", false, true}, // new: Business AI Guide (after referrals, before ai)
{"company", false, true}, // new: Hanzo Company formation state machine (after guide)
{"agent", false, false}, // new: /v1/agent tool-calling round (before zen/ai catch-all)
{"zen", false, false}, // zen* claim middleware before ai's catch-all (hip-00NN)
{"ai", false, false}, // was order 150
{"plugins", false, false}, // was order 900
@@ -135,17 +135,18 @@ func TestWireOrderMatchesFrozen(t *testing.T) {
}
}
// TestWireNoDuplicateEnablement guards the ONE intentional duplicate: only "o11y"
// may appear twice (the two co-owners of the observability concept). Any other
// duplicate name is a copy-paste bug — two specs would both mount under one enable id.
// TestWireNoDuplicateEnablement guards that every subsystem name is unique: a name
// maps 1:1 to an enable id, so a duplicate would mount two specs under one id. (The
// former o11y co-ownership was collapsed — the module wildcard is now a sub-mount of
// the in-repo o11y read plane — so there is no longer any exempt duplicate.)
func TestWireNoDuplicateEnablement(t *testing.T) {
seen := map[string]int{}
for _, s := range Wire() {
seen[s.Name]++
}
for name, n := range seen {
if n > 1 && name != "o11y" {
t.Errorf("subsystem %q wired %d times (only o11y may be a co-owned duplicate)", name, n)
if n > 1 {
t.Errorf("subsystem %q wired %d times (each enable id must be unique)", name, n)
}
}
}
+111 -25
View File
@@ -5,9 +5,12 @@ package apps
import (
"context"
"fmt"
"math/big"
"os"
"strings"
aicontrollers "github.com/hanzoai/ai/controllers"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/metering"
cloudmoney "github.com/hanzoai/cloud/clients/money"
@@ -53,11 +56,7 @@ import (
//
// It is wired BEFORE ai in Wire() so Claim's c.Next() falls through to ai's
// catch-all. zen's catalog reads its upstream keys from KMS via the Key resolver.
func mountZen(app any, deps cloud.Deps) error {
a, ok := app.(*zip.App)
if !ok {
return fmt.Errorf("zen.Mount: app is %T, want *zip.App", app)
}
func mountZen(a *zip.App, deps cloud.Deps) error {
z, err := zen.New(zen.Config{
Logger: deps.Logger,
Key: zenKeyResolver(deps.KMS),
@@ -110,11 +109,17 @@ func commerceGate(m *metering.Client) zen.Gate {
if t.BillingOrg == "" {
return fmt.Errorf("a billable tenant is required (no anonymous usage)")
}
// zen's estimate is exact 18-dp atto-USD (hanzoai/money). Fold to whole
// cents for the balance check via cloud's typed money.Amount.Cents() — a
// sub-cent estimate gates as 0 (any-positive-balance), matching the edge
// gate's AmountCents contract. The post-serve Meter debits the exact 18-dp.
cents := cloudmoney.FromInt(est.Minor()).Cents()
// zen's estimate is an exact 18-dp USD value. Fold it to whole cents for
// the balance check via cloud's typed money.Amount.Cents() — a sub-cent
// estimate gates as 0 (any-positive-balance), matching the edge gate's
// AmountCents contract. The post-serve Meter debits the exact 18-dp.
//
// The fold is Cents() on the CREDIT amount, never Minor() on the zen one:
// Minor() renders money.USD's 2 decimals, so it returned cents that FromInt
// then read as atto — every estimate came back 0, and AuthorizeVerdict skips
// its `available >= AmountCents` check when AmountCents is 0, admitting a
// request of ANY size against any positive balance.
cents := credit(est).Cents()
v, err := m.AuthorizeVerdict(ctx, metering.AuthInput{
User: t.BillingOrg,
Org: t.BillingOrg,
@@ -158,25 +163,106 @@ func (g commerceMeterImpl) Record(ctx context.Context, u zen.Usage) {
if u.Tenant.BillingOrg == "" {
return // never debit an unattributable request
}
usage := metering.Usage{
User: u.Tenant.BillingOrg,
Org: u.Tenant.BillingOrg,
Actor: u.Tenant.User,
Model: u.Model,
Provider: zenProvider,
Service: zenService,
Project: u.Tenant.Project,
PromptTokens: u.PromptTokens,
// Beside the commerce debit, land the SAME warehouse row + gen_ai span every
// native ai path writes (TraceServedUsage = recordTrace WITHOUT recordUsage —
// the debit below is the one billing source, never doubled). zen knows its
// EXACT per-tier retail (Charge) and upstream COGS (Cost), so the row carries
// true margin (credit → nano). Without this, zen* traffic is
// warehouse/o11y-blind exactly where prod runs (the unified binary).
aicontrollers.TraceServedUsage(context.Background(), aicontrollers.ServedUsage{
Owner: u.Tenant.BillingOrg,
User: u.Tenant.User,
Model: u.Model,
Provider: zenProvider,
RequestID: u.RequestID,
Status: "success",
PromptTokens: u.PromptTokens,
CompletionTokens: u.CompletionTokens,
TotalTokens: u.PromptTokens + u.CompletionTokens,
Amount: cloudmoney.FromInt(u.Cost.Minor()), // exact 18-dp USD, no floor
RequestID: u.RequestID,
Currency: "usd",
Status: "success",
}
BilledNano: nano(credit(u.Charge)),
CostNano: nano(credit(u.Cost)),
})
// Detached: the request context is recycled once the handler returns, so a
// background context carries the debit to commerce without racing the reply.
usage := meterUsage(u)
go func() { _, _ = g.m.Record(context.Background(), usage) }()
// Enso learning ledger: the embedded zen mount serves the zen catalog in-process
// and never reaches ai's pipeToFamily, so ai's family-event writer never runs for
// zen traffic. Write the SAME RoutingEvent here (source="family") through the ONE
// shared writer, keyed on the client-visible response id (zen.Usage.ResponseID), so
// zen* calls land in the same ledger — stats, world, spark retrain, and /v1/feedback
// all read these rows. No prompt text; no shadow (zen.Usage carries no request
// text — that stays the auto/enso-proxy path's job). Fire-and-forget.
owner := u.Tenant.Org
if owner == "" {
owner = u.Tenant.BillingOrg
}
go aiobject.RecordFamilyRouting(aiobject.FamilyRoutingInput{
Owner: owner,
User: u.Tenant.User,
RequestedModel: u.Model,
RoutedModel: u.Upstream,
ResponseId: u.ResponseID,
PromptTokens: u.PromptTokens,
CompletionTokens: u.CompletionTokens,
CostCents: credit(u.Charge).Cents(),
RouterEndpoint: os.Getenv("ROUTER_ENDPOINT"),
})
}
// meterUsage projects a served zen.Usage onto the commerce debit. It is the ONE
// place the debit's amount is chosen, and it is pure — no ledger, no warehouse —
// so the money property is a unit test rather than an integration.
//
// The amount is the RETAIL Charge: what the caller pays. Cost is the upstream
// COGS we pay to serve the call; it is never the debit. It rides only the
// warehouse row (CostNano), where margin = Charge Cost stays exact. Debiting
// Cost would collect our own COGS and book zero margin on every zen call — and
// because the affiliate and OSS payout bases read this debit, it would fund
// their shares out of principal. This mirrors ai, whose debit is likewise the
// customer price (usageBilledCents), never its CostIn/CostOut COGS.
func meterUsage(u zen.Usage) metering.Usage {
return metering.Usage{
User: u.Tenant.BillingOrg,
Org: u.Tenant.BillingOrg,
Actor: u.Tenant.User,
Model: u.Model,
Provider: zenProvider,
Service: zenService,
Project: u.Tenant.Project,
PromptTokens: u.PromptTokens,
CompletionTokens: u.CompletionTokens,
TotalTokens: u.PromptTokens + u.CompletionTokens,
Amount: credit(u.Charge), // exact 18-dp USD, no floor
RequestID: u.RequestID,
Currency: "usd",
Status: "success",
}
}
// credit re-denominates a zen price into cloud's credit unit. It is the ONE
// conversion at this seam — every site below goes through it, so the unit is
// decided once rather than re-derived per call site.
//
// zen prices every SKU as an exact 18-dp value tagged money.USD (meter.go:
// money.New(<18-dp decimal>, money.USD)), and cloud's credit unit is the SAME USD
// value at 18-dp storage scale. So the conversion carries the exact decimal across
// and changes only the minor-unit convention: no rescale, no rounding, no factor.
// It is right by construction because the decimal is the value — the currency's
// Decimals is a rendering convention, not part of it.
//
// It must NEVER go through Amount.Minor(). money.USD declares 2 decimals, so
// Minor() rescales zen's 18-dp value to CENTS; feeding cents to the 18-dp
// FromInt understates the debit by 10^16 (a $17.376 charge debits $0.0000000000000017),
// and folds every sub-cent charge to a zero the ledger drops entirely.
func credit(a hmoney.Amount) cloudmoney.Amount { return cloudmoney.FromDecimal(a.Decimal()) }
// nano folds an exact credit Amount to nano-USD (1e-9) for the warehouse margin
// columns. It takes the typed Amount rather than a bare *big.Int so the unit is
// carried by the type: a cents integer is not a cloudmoney.Amount and can no
// longer be passed here. A single request's cost always fits int64 at nano.
func nano(a cloudmoney.Amount) int64 {
return new(big.Int).Div(a.Atto(), big.NewInt(1_000_000_000)).Int64()
}
// zenService is the commerce service axis zen* spend attributes to. zen serves
+305
View File
@@ -0,0 +1,305 @@
// Copyright 2026 Hanzo AI Inc. All Rights Reserved.
package apps
import (
"context"
"io"
"net/http"
"net/http/httptest"
"strconv"
"sync/atomic"
"testing"
"github.com/hanzoai/cloud/clients/metering"
cloudmoney "github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/decimal"
hmoney "github.com/hanzoai/money"
"github.com/hanzoai/zen"
)
// usd builds an exact USD amount from a decimal string, the way zen prices a
// call (18-dp native, never through float).
//
// The decimal here must be hanzoai/decimal, the one hmoney.New takes and the one
// zen prices with — not shopspring's identically-named type. Money has exactly one
// decimal; a second one that merely LOOKS like it is how a price silently becomes
// a different number.
func usd(t *testing.T, s string) hmoney.Amount {
t.Helper()
d, err := decimal.Parse(s)
if err != nil {
t.Fatalf("parse %q: %v", s, err)
}
return hmoney.New(d, hmoney.USD)
}
// zen5Usage is one served call at zen5's live rates: 1M in + 1M out, priced at
// the family's 3× margin (retail = cost × margin).
//
// in : cost 1.392 → retail 4.176
// out: cost 4.40 → retail 13.20
func zen5Usage(t *testing.T) zen.Usage {
t.Helper()
return zen.Usage{
Tenant: zen.Tenant{BillingOrg: "acme", User: "acme/alice", Project: "p1"},
Model: "zen5",
PromptTokens: 1_000_000,
CompletionTokens: 1_000_000,
Charge: usd(t, "17.376"), // 4.176 + 13.20 — what the caller pays
Cost: usd(t, "5.792"), // 1.392 + 4.40 — what we pay upstream
RequestID: "req-1",
}
}
// dollars is the EXPECTED money, built from a plain dollar literal through cloud's
// OWN ParseUSD — deliberately a DIFFERENT constructor than the code under test uses.
//
// That is what pins the UNIT. These tests once asserted meterUsage(u).Amount.Int()
// against u.Charge.Minor(): both sides re-derived the number through the same
// conversion, so the assertion only proved an integer round-tripped and was blind
// to what the integer MEANT. It passed while every zen debit was 10^16 too small.
// Comparing money to a known dollar amount cannot be blind that way: if the debit
// is off by any factor, it is not $17.376 and the test fails.
func dollars(t *testing.T, s string) cloudmoney.Amount {
t.Helper()
a, err := cloudmoney.ParseUSD(s)
if err != nil {
t.Fatalf("parse %q: %v", s, err)
}
return a
}
// The debit is the RETAIL Charge — what the caller pays — never the upstream
// COGS. This is the money property: it fails if the Amount is built from
// u.Cost. The tier is margin-bearing (retail != cost), so the two values are
// distinguishable and the assertion cannot pass by coincidence.
func TestMeterUsageDebitsRetailNotCost(t *testing.T) {
u := zen5Usage(t)
if u.Charge.Cmp(u.Cost) == 0 {
t.Fatal("fixture is not margin-bearing: retail == cost, so the test could not tell them apart")
}
got := meterUsage(u).Amount
// The known dollar value the fixture charges — a $17.376 call debits $17.376.
if want := dollars(t, "17.376"); got.Cmp(want) != 0 {
t.Errorf("debit = $%s, want the retail Charge $%s", got, want)
}
if cogs := dollars(t, "5.792"); got.Cmp(cogs) == 0 {
t.Errorf("debit = $%s, which is the upstream COGS — the caller must be billed retail, not our cost", got)
}
}
// At the family's 3× margin the debit is exactly 3× the COGS: we collect the
// full retail price, not the wholesale one. Debiting Cost would collect 1/3 —
// our own COGS — and book zero margin.
func TestMeterUsageCollectsTheFullMargin(t *testing.T) {
u := zen5Usage(t)
debit := meterUsage(u).Amount
// 3x the COGS in the SAME unit as the debit — $5.792 + $5.792 + $5.792.
// Summing the credit Amount keeps the comparison in exact dollars; the old
// version multiplied Cost.Minor() (cents, 579 after rounding away 5.792's
// third decimal) and compared it to a debit that was not cents at all.
cogs := dollars(t, "5.792")
thriceCOGS := cogs.Add(cogs).Add(cogs)
if debit.Cmp(thriceCOGS) != 0 {
t.Errorf("debit = $%s, want 3x COGS = $%s (retail = cost x margin, margin 3.0)", debit, thriceCOGS)
}
if want := dollars(t, "17.376"); debit.Cmp(want) != 0 {
t.Errorf("debit = $%s, want $%s", debit, want)
}
}
// The debit carries zen's exact 18-dp value with no floor: a sub-cent call must
// not round to zero on the way to the ledger.
func TestMeterUsageKeepsExactSubCentCharge(t *testing.T) {
u := zen5Usage(t)
u.Charge = usd(t, "0.004176") // 1k input tokens at 4.176/MTok — well under a cent
u.Cost = usd(t, "0.001392")
got := meterUsage(u).Amount
if want := dollars(t, "0.004176"); got.Cmp(want) != 0 {
t.Errorf("debit = $%s, want the exact sub-cent charge $%s", got, want)
}
// metering.Record drops a zero Amount before it ever reaches the ledger
// (`if !c.Enabled() || amt.IsZero() ... return nil`), so a floored sub-cent
// charge is not a small debit — it is NO DEBIT ROW, and the call is free.
if got.IsZero() {
t.Error("sub-cent charge floored to zero — Record drops a zero Amount, so the call is served free with no debit row")
}
}
// The unit trap this seam shipped with, pinned so it cannot come back. zen prices
// an exact 18-dp value but tags it money.USD, whose Currency declares 2 decimals —
// so Charge.Minor() renders CENTS, while cloudmoney.FromInt reads its argument as
// 18-dp. Composing them silently divides every debit by 10^16.
//
// The tests above already fail if credit() regresses to that composition; this one
// names WHY, and proves the two conversions are still distinguishable — an
// assertion that cannot tell right from wrong is worse than no assertion.
func TestCreditIsNotTheMinorUnit(t *testing.T) {
charge := usd(t, "17.376")
if got, want := credit(charge), dollars(t, "17.376"); got.Cmp(want) != 0 {
t.Fatalf("credit($17.376) = $%s, want $%s", got, want)
}
// The conversion that shipped, spelled out.
old := cloudmoney.FromAtto(charge.Minor())
if old.Cmp(credit(charge)) == 0 {
t.Fatal("FromInt(Minor()) agrees with credit() — the fixture can no longer tell the units apart, so these tests prove nothing")
}
if old.Cents() != 0 {
t.Errorf("FromInt(Minor()).Cents() = %d, want 0 — that this was ALWAYS 0 is what made the spend gate admit every request", old.Cents())
}
}
// The identity fields ride with the debit unchanged: the debit lands on the org
// that PAYS (BillingOrg), scoped to its project, with the actor for the audit
// trail.
func TestMeterUsageAttribution(t *testing.T) {
u := zen5Usage(t)
m := meterUsage(u)
for _, c := range []struct{ name, got, want string }{
{"User", m.User, "acme"},
{"Org", m.Org, "acme"},
{"Actor", m.Actor, "acme/alice"},
{"Project", m.Project, "p1"},
{"Model", m.Model, "zen5"},
{"Provider", m.Provider, zenProvider},
{"Service", m.Service, zenService},
{"Currency", m.Currency, "usd"},
} {
if c.got != c.want {
t.Errorf("%s = %q, want %q", c.name, c.got, c.want)
}
}
if m.TotalTokens != 2_000_000 {
t.Errorf("TotalTokens = %d, want 2000000", m.TotalTokens)
}
}
// The spend gate must REFUSE a request whose estimate exceeds the balance.
//
// This is the sharpest edge of the unit bug and the reason it is a security
// finding, not only a revenue one. AuthorizeVerdict gates size like this:
//
// funded := available > 0
// if in.AmountCents > 0 { funded = available >= in.AmountCents }
//
// The estimate reached it as FromInt(est.Minor()).Cents(), which is ALWAYS 0 —
// so the size branch was DEAD and every request rode `available > 0`. Any org
// with a single cent of balance could draw an unbounded call. The debits were
// dust too, so the balance never fell and the cap could never trip.
func TestCommerceGateRefusesAnOverCapRequest(t *testing.T) {
const availableCents = 500 // the org holds $5.00
var authorized atomic.Bool
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v1/billing/balance":
_, _ = io.WriteString(w, `{"available":`+strconv.Itoa(availableCents)+`}`)
case "/v1/billing/spend-alerts/authorize":
authorized.Store(true)
_, _ = io.WriteString(w, `{"allow":true}`)
default:
_, _ = io.WriteString(w, `[]`)
}
}))
defer srv.Close()
m, err := metering.New(metering.Config{BaseURL: srv.URL, Token: "t", Org: "acme"})
if err != nil {
t.Fatal(err)
}
gate := commerceGate(m)
if gate == nil {
t.Fatal("gate is nil — the client must be enabled for this proof to mean anything")
}
tenant := zen.Tenant{BillingOrg: "acme", User: "acme/alice", Project: "p1"}
// $17.376 against a $5.00 balance: over cap, must be refused.
if err := gate(context.Background(), tenant, "zen5", usd(t, "17.376")); err == nil {
t.Error("gate ADMITTED a $17.376 request against a $5.00 balance — the estimate is reaching AuthorizeVerdict as 0 cents, so the size check never runs")
}
// The same balance must still admit a request it can actually cover, or the
// test would pass by refusing everything.
if err := gate(context.Background(), tenant, "zen5", usd(t, "1.00")); err != nil {
t.Errorf("gate refused an affordable $1.00 request against a $5.00 balance: %v", err)
}
if !authorized.Load() {
t.Error("the affordable request never reached the spend-cap authorize step")
}
}
// The estimate must reach the balance check as the RIGHT number of cents. The
// refusal test above proves the gate says no; this proves it says no for the
// right reason — that $17.376 is folded to 1738 cents, not to 0.
func TestCommerceGateFoldsTheEstimateToRealCents(t *testing.T) {
for _, c := range []struct {
charge string
want int64
}{
{"17.376", 1738}, // rounds half-away-from-zero at the cent
{"1000.00", 100000},
{"1.00", 100},
{"0.004176", 0}, // sub-cent gates as "any positive balance"
} {
if got := credit(usd(t, c.charge)).Cents(); got != c.want {
t.Errorf("$%s folds to %d cents, want %d", c.charge, got, c.want)
}
}
}
// A zero-value zen price must convert and fold without panicking. zen leaves
// Charge/Cost as the zero Amount for a free SKU, and that value carries the
// EMPTY currency code, not "USD" — so this also pins that credit() reads the
// decimal rather than dispatching on the currency. The old attoToNano guarded a
// nil *big.Int here; nano() needs no guard because decimal.Coef() returns a real
// zero big.Int, never nil, but the property is worth holding.
func TestCreditAndNanoHandleTheZeroPrice(t *testing.T) {
var free hmoney.Amount // zero value: no currency, no coefficient
got := credit(free)
if !got.IsZero() {
t.Errorf("credit(zero) = $%s, want $0", got)
}
if n := nano(got); n != 0 {
t.Errorf("nano(credit(zero)) = %d, want 0", n)
}
if c := got.Cents(); c != 0 {
t.Errorf("credit(zero).Cents() = %d, want 0", c)
}
// And a free call books no debit row, which is correct — nothing is owed.
u := zen5Usage(t)
u.Charge, u.Cost = free, free
if amt := meterUsage(u).Amount; !amt.IsZero() {
t.Errorf("free call debits $%s, want $0", amt)
}
}
// nano carries real money to the warehouse margin columns. It is the last place
// the unit could silently collapse: attoToNano(cents) divided a cents integer by
// 1e9 and produced 0 for every charge under $10,000,000.
func TestNanoFoldsCreditToRealNano(t *testing.T) {
for _, c := range []struct {
charge string
want int64
}{
{"17.376", 17_376_000_000},
{"5.792", 5_792_000_000},
{"0.004176", 4_176_000},
{"1000.00", 1_000_000_000_000},
} {
if got := nano(credit(usd(t, c.charge))); got != c.want {
t.Errorf("nano($%s) = %d, want %d", c.charge, got, c.want)
}
}
}
+15 -2
View File
@@ -15,6 +15,7 @@ import (
"fmt"
"os"
"path/filepath"
"strings"
"time"
"github.com/hanzoai/cloud/audit"
@@ -57,6 +58,18 @@ func buildAuditRecorder(cfg *Config, logger luxlog.Logger) (*audit.Recorder, err
return nil, fmt.Errorf("open audit store: %w", err)
}
// PER-SHARD audit under horizontal scale. The trail lives at {DataDir}/audit.db on
// THIS pod's own RWO PVC, so under shard routing each pod's chain covers ONLY the
// tenants routed to it (its shard) — and org-scoped audit queries route to the
// owning shard where those records live. Soundness: the chain is a per-FILE hash
// chain whose head is recovered at open; because no two pods share the file, there
// is no cross-pod head to fork (the very failure that pinned cloud to replicas:1 was
// two pods on ONE audit file). Integrity is preserved WITHIN each partition; a
// deployment-wide view is the union of the N per-shard chains. The shard id is
// stamped on the AU-9 checkpoint stream below so the external tail-truncation monitor
// tracks N heads (one per shard) rather than expecting a single global head.
shard := strings.TrimSpace(cfg.ShardSelf) // "" when single-pod — a harmless empty tag
// AU-9 tail-truncation anchor: emit a periodic head-digest checkpoint to the
// append-only observability log (and, when a mirror supports it, an
// independent digest store). An external o11y monitor compares consecutive
@@ -67,7 +80,7 @@ func buildAuditRecorder(cfg *Config, logger luxlog.Logger) (*audit.Recorder, err
if logger != nil {
rec.StartCheckpoints(interval, func(cp audit.Checkpoint) {
logger.Info("audit_head_checkpoint",
"count", cp.Count, "head", cp.Head, "ts", cp.Time.Format(time.RFC3339Nano))
"shard", shard, "count", cp.Count, "head", cp.Head, "ts", cp.Time.Format(time.RFC3339Nano))
})
} else {
rec.StartCheckpoints(interval, nil)
@@ -76,7 +89,7 @@ func buildAuditRecorder(cfg *Config, logger luxlog.Logger) (*audit.Recorder, err
if logger != nil {
count, head := rec.Head()
logger.Info("audit trail ready (tamper-evident, append-only)",
"store", dbPath, "records", count, "head", head,
"store", dbPath, "shard", shard, "records", count, "head", head,
"mirror", mirror != nil, "checkpoint_interval", interval.String())
}
return rec, nil
+18 -9
View File
@@ -47,21 +47,30 @@ type iamKeys struct {
// credential it returns a resolver that resolves nothing (keys stay anonymous —
// never a fabricated principal), so a deployment lacking the credential is safe.
func newIAMKeys() *iamKeys {
base := strings.TrimRight(env("IAM_URL", "IAM_INTERNAL_URL"), "/")
id := strings.TrimSpace(os.Getenv("IAM_MINT_CLIENT_ID"))
secret := strings.TrimSpace(os.Getenv("IAM_MINT_CLIENT_SECRET"))
var auth string
if id != "" && secret != "" {
auth = "Basic " + base64.StdEncoding.EncodeToString([]byte(id+":"+secret))
}
return &iamKeys{
base: base,
auth: auth,
base: iamHost(),
auth: iamCred(),
http: &http.Client{Timeout: 5 * time.Second},
cache: newCache[string, *idClaims](60 * time.Second),
}
}
// iamHost is the standalone IAM origin cloud talks to; iamCred is the service
// credential (client_secret_basic) it presents — the ONE IAM identity, shared by
// the API-key resolver here and the /v1/iam edge (iam_edge.go), so both
// authenticate to IAM the same way. Empty cred → a deployment lacking the
// credential stays safe (the caller treats "" as unconfigured).
func iamHost() string { return strings.TrimRight(env("IAM_URL", "IAM_INTERNAL_URL"), "/") }
func iamCred() string {
id := strings.TrimSpace(os.Getenv("IAM_MINT_CLIENT_ID"))
secret := strings.TrimSpace(os.Getenv("IAM_MINT_CLIENT_SECRET"))
if id == "" || secret == "" {
return ""
}
return "Basic " + base64.StdEncoding.EncodeToString([]byte(id+":"+secret))
}
func env(names ...string) string {
for _, n := range names {
if v := strings.TrimSpace(os.Getenv(n)); v != "" {
+12 -26
View File
@@ -656,30 +656,17 @@ func pickVaultClient(cfg *Config, log luxlog.Logger) VaultClient {
return clients.DisabledVault()
}
// MountFunc is a subsystem's mount contract. app is `any`, not *zip.App, and that
// is load-bearing: some external modules expose Mount as func(any, Deps) error
// (e.g. hanzoai/licensing), which apps.Wire references DIRECTLY — a
// func(any,…) value is not assignable to a func(*zip.App,…) parameter, so
// narrowing the type would break them at compile time. The concrete value is
// always a *zip.App; strongly-typed Mounts (func(*zip.App, Deps) error, what every
// in-repo subsystem exports) are adapted by Typed, which recovers it in ONE place.
type MountFunc func(app any, deps Deps) error
// Typed adapts a strongly-typed subsystem Mount — func(*zip.App, Deps) error,
// the signature every in-repo subsystem already exports — into the registry's
// MountFunc. It performs the *zip.App recovery in ONE place, fail-closed with a
// clear error, so no subsystem repeats the `a, ok := app.(*zip.App)` boilerplate.
// The concrete value MountAll passes is always a *zip.App, so the assertion is
// total in practice; it stays as a defensive, self-documenting guard.
func Typed(mount func(*zip.App, Deps) error) MountFunc {
return func(app any, deps Deps) error {
a, ok := app.(*zip.App)
if !ok {
return fmt.Errorf("cloud.Mount: app is %T, want *zip.App", app)
}
return mount(a, deps)
}
}
// MountFunc is a subsystem's mount contract: register your routes on app, using
// deps for everything shared. Every subsystem in the fleet exports exactly this
// signature, so Wire references each one directly and the compiler checks it.
//
// app was once `any`, on the stated grounds that an external module (licensing)
// exposed func(any, Deps) error and narrowing would break it — while licensing
// said it used `any` to avoid an import cycle in pkg/cloud. Each cited the other,
// and the cycle could not exist: this package already imports zip, and zip does
// not import cloud. The `any` bought nothing and cost every subsystem a Typed()
// wrapper plus a runtime type assertion whose failure branch was unreachable.
type MountFunc func(app *zip.App, deps Deps) error
// ShutdownFunc releases a subsystem's process-lifetime resources (background
// goroutines, open DB handles) on graceful shutdown. It must be idempotent and
@@ -704,8 +691,7 @@ type MountSpec struct {
// MountAll mounts every ENABLED subsystem in specs, in slice order — the order is
// the composition root's (apps.Wire()); MountAll does NOT sort. app is the
// concrete *zip.App from Serve; the MountFunc accepts it as `any` and in-repo
// subsystems recover it via Typed.
// concrete *zip.App from Serve, handed to each MountFunc as itself.
//
// Teardown is wired HERE, at mount time: right after a subsystem mounts, its
// ShutdownFunc (if any) is registered via app.OnShutdown. zip drains those hooks
+1 -1
View File
@@ -16,7 +16,7 @@ import (
// noopMount mounts nothing: the fake specs below carry the behavior under test in
// their Shutdown, not their Mount.
func noopMount(any, cloud.Deps) error { return nil }
func noopMount(*zip.App, cloud.Deps) error { return nil }
// freeAddr reserves an ephemeral loopback port and hands back its address; the
// listener is closed so the app under test can bind it.
+18 -39
View File
@@ -1,49 +1,28 @@
package cloud_test
import (
"strings"
"testing"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
)
// TestTyped_RecoversZipApp verifies cloud.Typed adapts a strongly-typed
// func(*zip.App, Deps) into the registry MountFunc: it hands the concrete
// *zip.App straight through to the wrapped mount.
func TestTyped_RecoversZipApp(t *testing.T) {
app := zip.New(zip.Config{})
var got *zip.App
mf := cloud.Typed(func(a *zip.App, _ cloud.Deps) error {
got = a
return nil
})
if err := mf(app, cloud.Deps{}); err != nil {
t.Fatalf("Typed mount returned error: %v", err)
}
if got != app {
t.Fatalf("Typed did not pass the concrete *zip.App through (got %p, want %p)", got, app)
}
}
// TestTyped_WrongTypeFailsClosed verifies cloud.Typed fails closed with a clear
// error — never a panic — when the registry passes a value that is not a
// *zip.App. This is the single, central replacement for the per-subsystem
// assertion boilerplate.
func TestTyped_WrongTypeFailsClosed(t *testing.T) {
called := false
mf := cloud.Typed(func(*zip.App, cloud.Deps) error {
called = true
return nil
})
err := mf("not-a-zip-app", cloud.Deps{})
if err == nil {
t.Fatal("Typed must return an error on a non-*zip.App value")
}
if called {
t.Fatal("Typed must NOT invoke the wrapped mount on a type mismatch")
}
if !strings.Contains(err.Error(), "*zip.App") {
t.Errorf("error should name the wanted type *zip.App, got: %v", err)
}
// TestMountFunc_IsTheSubsystemSignature pins the registry's mount contract: the
// signature every subsystem exports IS a cloud.MountFunc, checked by the compiler.
//
// This file used to test cloud.Typed, the adapter that took a MountFunc's `any`
// app and asserted it back to *zip.App. Both of its tests went with it, and
// neither is a loss:
//
// - "Typed recovers the *zip.App" only ever proved the adapter handed through
// the value it was given. MountFunc now names *zip.App, so there is no
// recovery step left to get wrong.
// - "Typed fails closed on a wrong type" can no longer be written: passing
// "not-a-zip-app" to a MountFunc is a compile error, so the runtime branch it
// exercised does not exist. A test asserting a wrong type is rejected is
// precisely what a type already is.
//
// What remains is the only claim worth making, and the build enforces it.
func TestMountFunc_IsTheSubsystemSignature(t *testing.T) {
var _ cloud.MountFunc = func(*zip.App, cloud.Deps) error { return nil }
}
+128 -22
View File
@@ -160,24 +160,21 @@ type loginFlags struct {
}
func runLogin(env *Env, lf *loginFlags, cmd *cobra.Command) error {
creds, err := LoadCredentials()
if err != nil {
return err
}
var creds *Credentials
switch {
case lf.token != "":
// Paste an externally-minted token. Decode claims for identity.
tr := &tokenResp{AccessToken: lf.token, TokenType: "Bearer"}
creds = credsFromToken(tr)
creds = credsFromToken(&tokenResp{AccessToken: lf.token, TokenType: "Bearer"})
case lf.username != "" || lf.passwordStdin:
// Password grant — kept for automation (--username/--password-stdin).
username := lf.username
if username == "" {
username, err = prompt(cmd, "Email: ")
u, err := prompt(cmd, "Email: ")
if err != nil {
return err
}
username = u
}
password, err := readPassword(cmd, lf.passwordStdin)
if err != nil {
@@ -192,14 +189,15 @@ func runLogin(env *Env, lf *loginFlags, cmd *cobra.Command) error {
default:
// The ONE interactive way: RFC 8628 device flow — link + QR + code,
// approve from any signed-in browser or phone. Headless-safe.
creds, err = runDeviceLogin(cmd, env, lf.scope)
c, err := runDeviceLogin(cmd, env, lf.scope)
if err != nil {
return err
}
creds = c
}
// Optional machine-to-machine tokens for the platform control plane,
// stored alongside the identity so apps/deploy work post-login.
// stored with this identity so apps/deploy work post-login.
if lf.platformToken != "" {
creds.PlatformToken = lf.platformToken
}
@@ -207,14 +205,23 @@ func runLogin(env *Env, lf *loginFlags, cmd *cobra.Command) error {
creds.BuildToken = lf.buildToken
}
if err := creds.Save(); err != nil {
// Persist under this identity's stable key and make it active. A second
// login as a different owner (e.g. admin vs hanzo for the same email, via a
// different --client-id) is stored beside the first, never over it;
// credentials.json mirrors whichever is active for legacy readers.
store, err := LoadIdentities()
if err != nil {
return err
}
key := store.Put(creds)
if err := store.Save(); err != nil {
return err
}
who := firstNonEmpty(creds.Subject, "(unknown)")
if creds.Owner != "" {
who += " @ " + creds.Owner
}
fmt.Fprintf(cmd.OutOrStdout(), "Logged in as %s (token expires %s)\n", who, shortTime(creds.Expiry))
fmt.Fprintf(cmd.OutOrStdout(), "Logged in as %s [%s] (token expires %s)\n", who, key, shortTime(creds.Expiry))
return nil
}
@@ -249,28 +256,92 @@ func bindLoginFlags(cmd *cobra.Command, lf *loginFlags) {
func newLogoutCmd() *cobra.Command {
return &cobra.Command{
Use: "logout",
Short: "Remove stored credentials",
Args: cobra.NoArgs,
Use: "logout [<owner>]",
Short: "Remove a stored identity (the active one, or the named owner)",
Args: cobra.MaximumNArgs(1),
PersistentPreRunE: func(*cobra.Command, []string) error { return nil },
RunE: func(cmd *cobra.Command, _ []string) error {
if err := DeleteCredentials(); err != nil {
RunE: func(cmd *cobra.Command, args []string) error {
store, err := LoadIdentities()
if err != nil {
return err
}
fmt.Fprintln(cmd.OutOrStdout(), "Logged out.")
if len(store.Identities) == 0 {
fmt.Fprintln(cmd.OutOrStdout(), "Not logged in.")
return nil
}
target := store.Active
if len(args) == 1 {
if target, err = store.resolve(args[0]); err != nil {
return err
}
}
store.Remove(target)
if err := store.Save(); err != nil {
return err
}
msg := "Logged out of " + target + "."
if store.Active != "" {
msg += " Active is now " + store.Active + "."
}
fmt.Fprintln(cmd.OutOrStdout(), msg)
return nil
},
}
}
// identityRow is the JSON/table projection of one stored identity.
type identityRow struct {
Key string `json:"key"`
Owner string `json:"owner"`
Subject string `json:"subject"`
Expiry int64 `json:"expiry,omitempty"`
Active bool `json:"active"`
}
// listIdentities renders every stored identity (active marked with *) — the
// shared body of `hanzo auth list` and `hanzo whoami --all`.
func listIdentities(env *Env, _ *cobra.Command) error {
store, err := LoadIdentities()
if err != nil {
return err
}
rows := make([]identityRow, 0, len(store.Identities))
for _, k := range store.keys() {
c := store.Identities[k]
rows = append(rows, identityRow{
Key: k, Owner: c.Owner, Subject: c.Subject, Expiry: c.Expiry,
Active: k == store.Active,
})
}
return env.emit(rows, func(w io.Writer) {
if len(rows) == 0 {
fmt.Fprintln(w, "No stored identities. Run `hanzo login`.")
return
}
tw := newTab(w)
fmt.Fprintln(tw, "ACTIVE\tKEY\tOWNER\tSUBJECT\tEXPIRES")
for _, r := range rows {
active := ""
if r.Active {
active = "*"
}
fmt.Fprintf(tw, "%s\t%s\t%s\t%s\t%s\n", active, r.Key, r.Owner, r.Subject, shortTime(r.Expiry))
}
tw.Flush()
})
}
func newWhoamiCmd(envOf func() *Env) *cobra.Command {
var verify bool
var verify, all bool
cmd := &cobra.Command{
Use: "whoami",
Short: "Show the current identity from the stored token",
Short: "Show the active identity from the stored token (--all lists every stored identity)",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
env := envOf()
if all {
return listIdentities(env, cmd)
}
tok := env.accessToken()
if tok == "" {
return fmt.Errorf("not logged in: run `hanzo login`")
@@ -300,6 +371,7 @@ func newWhoamiCmd(envOf func() *Env) *cobra.Command {
},
}
cmd.Flags().BoolVar(&verify, "verify", false, "verify the token against the IAM userinfo endpoint")
cmd.Flags().BoolVar(&all, "all", false, "list every stored identity (active marked with *)")
return cmd
}
@@ -329,11 +401,11 @@ func verifyUserInfo(ctx context.Context, env *Env, token string) error {
func newAuthCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
cmd := &cobra.Command{
Use: "auth",
Short: "Manage authentication",
Short: "Manage authentication and stored identities",
}
tokenCmd := &cobra.Command{
Use: "token",
Short: "Print the stored access token",
Short: "Print the active access token",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
tok := envOf().accessToken()
@@ -344,7 +416,41 @@ func newAuthCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
return nil
},
}
cmd.AddCommand(newLoginCmd(envOf, gf), newLogoutCmd(), newWhoamiCmd(envOf), tokenCmd)
listCmd := &cobra.Command{
Use: "list",
Aliases: []string{"ls", "identities"},
Short: "List stored identities (active marked with *)",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error { return listIdentities(envOf(), cmd) },
}
switchCmd := &cobra.Command{
Use: "switch <owner>",
Aliases: []string{"use"},
Short: "Make a stored identity active (accepts an owner, or a full owner/name key)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
store, err := LoadIdentities()
if err != nil {
return err
}
key, err := store.resolve(args[0])
if err != nil {
return err
}
store.Active = key
if err := store.Save(); err != nil {
return err
}
c := store.Identities[key]
who := firstNonEmpty(c.Subject, "(unknown)")
if c.Owner != "" {
who += " @ " + c.Owner
}
fmt.Fprintf(cmd.OutOrStdout(), "Switched to %s [%s] (token expires %s)\n", who, key, shortTime(c.Expiry))
return nil
},
}
cmd.AddCommand(newLoginCmd(envOf, gf), newLogoutCmd(), newWhoamiCmd(envOf), tokenCmd, listCmd, switchCmd)
return cmd
}
+175
View File
@@ -208,3 +208,178 @@ func TestAuthTokenCommand(t *testing.T) {
t.Fatalf("auth token output: %q", out)
}
}
// TestMultiIdentityLoginSwitch is the full multi-identity story: two logins for
// the same email under different owners (admin vs hanzo — the privilege-
// separation case) coexist, `auth list` shows both, `switch` flips the active
// pointer and rewrites credentials.json, and legacy single-file readers always
// see the active identity.
func TestMultiIdentityLoginSwitch(t *testing.T) {
sandbox(t)
adminTok := makeJWT(map[string]any{"email": "z@hanzo.ai", "owner": "admin", "sub": "u-admin", "exp": float64(2000000000)})
hanzoTok := makeJWT(map[string]any{"email": "z@hanzo.ai", "owner": "hanzo", "sub": "u-hanzo", "exp": float64(2000000001)})
// First login → admin/z is stored and active.
out, err := runRoot(t, "", "login", "--token", adminTok)
if err != nil {
t.Fatalf("login admin: %v", err)
}
if !strings.Contains(out, "admin/z") {
t.Fatalf("login should report the key: %q", out)
}
if c, _ := LoadCredentials(); c.Owner != "admin" || c.Subject != "z@hanzo.ai" {
t.Fatalf("active not admin after first login: %+v", c)
}
// Second login (different owner) → added beside admin/z, becomes active,
// does NOT clobber the first.
if _, err := runRoot(t, "", "login", "--token", hanzoTok); err != nil {
t.Fatalf("login hanzo: %v", err)
}
store, err := LoadIdentities()
if err != nil {
t.Fatalf("load identities: %v", err)
}
if len(store.Identities) != 2 {
t.Fatalf("want 2 identities, got %d: %v", len(store.Identities), store.keys())
}
if store.Identities["admin/z"] == nil || store.Identities["hanzo/z"] == nil {
t.Fatalf("both identities must persist, got %v", store.keys())
}
if store.Active != "hanzo/z" {
t.Fatalf("active = %q, want hanzo/z (last login)", store.Active)
}
// Legacy reader sees the active (hanzo) identity.
if c, _ := LoadCredentials(); c.Owner != "hanzo" {
t.Fatalf("credentials.json not mirroring active: %+v", c)
}
// auth list shows both, with the active row marked.
out, err = runRoot(t, "", "auth", "list")
if err != nil {
t.Fatalf("auth list: %v", err)
}
for _, want := range []string{"admin/z", "hanzo/z", "z@hanzo.ai", "*"} {
if !strings.Contains(out, want) {
t.Fatalf("auth list missing %q in:\n%s", want, out)
}
}
// switch admin → active flips + credentials.json is rewritten to admin.
if _, err := runRoot(t, "", "auth", "switch", "admin"); err != nil {
t.Fatalf("auth switch admin: %v", err)
}
if st, _ := LoadIdentities(); st.Active != "admin/z" {
t.Fatalf("active after switch = %q, want admin/z", st.Active)
}
if c, _ := LoadCredentials(); c.Owner != "admin" || c.Subject != "z@hanzo.ai" {
t.Fatalf("switch did not rewrite credentials.json: %+v", c)
}
// whoami (top-level, reads the active token) reflects admin.
out, err = runRoot(t, "", "whoami")
if err != nil {
t.Fatalf("whoami: %v", err)
}
if !strings.Contains(out, "admin") || !strings.Contains(out, "z@hanzo.ai") {
t.Fatalf("whoami not reflecting the switched-to identity: %q", out)
}
// switch by the full owner/name key works too.
if _, err := runRoot(t, "", "auth", "switch", "hanzo/z"); err != nil {
t.Fatalf("auth switch hanzo/z: %v", err)
}
if c, _ := LoadCredentials(); c.Owner != "hanzo" {
t.Fatalf("switch by full key failed: %+v", c)
}
}
// TestAuthListJSON checks the machine-readable projection.
func TestAuthListJSON(t *testing.T) {
sandbox(t)
tok := makeJWT(map[string]any{"email": "z@hanzo.ai", "owner": "admin", "sub": "a"})
if _, err := runRoot(t, "", "login", "--token", tok); err != nil {
t.Fatalf("login: %v", err)
}
out, err := runRoot(t, "", "auth", "list", "-o", "json")
if err != nil {
t.Fatalf("auth list json: %v", err)
}
var rows []identityRow
if err := json.Unmarshal([]byte(out), &rows); err != nil {
t.Fatalf("json unmarshal: %v\n%s", err, out)
}
if len(rows) != 1 || rows[0].Key != "admin/z" || rows[0].Owner != "admin" || !rows[0].Active {
t.Fatalf("json rows wrong: %+v", rows)
}
}
// TestLogoutOneOfMany removes a single identity and, only when the last one is
// gone, clears the store entirely.
func TestLogoutOneOfMany(t *testing.T) {
sandbox(t)
admin := makeJWT(map[string]any{"email": "z@hanzo.ai", "owner": "admin", "sub": "a"})
hanzo := makeJWT(map[string]any{"email": "z@hanzo.ai", "owner": "hanzo", "sub": "h"})
if _, err := runRoot(t, "", "login", "--token", admin); err != nil {
t.Fatal(err)
}
if _, err := runRoot(t, "", "login", "--token", hanzo); err != nil { // active = hanzo/z
t.Fatal(err)
}
// logout of the named owner (admin) leaves hanzo/z active.
if _, err := runRoot(t, "", "logout", "admin"); err != nil {
t.Fatalf("logout admin: %v", err)
}
store, _ := LoadIdentities()
if store.Identities["admin/z"] != nil {
t.Fatalf("admin/z not removed: %v", store.keys())
}
if store.Active != "hanzo/z" {
t.Fatalf("active = %q, want hanzo/z", store.Active)
}
if c, _ := LoadCredentials(); c.Owner != "hanzo" {
t.Fatalf("credentials.json not mirroring survivor: %+v", c)
}
// logout of the active (no arg) removes the last identity → both files gone.
if _, err := runRoot(t, "", "logout"); err != nil {
t.Fatalf("logout active: %v", err)
}
if c, _ := LoadCredentials(); c.AccessToken != "" {
t.Fatalf("credentials.json not cleared: %+v", c)
}
if st, _ := LoadIdentities(); len(st.Identities) != 0 {
t.Fatalf("identity store not cleared: %v", st.keys())
}
}
// TestMigrateLegacyCredentials proves a pre-multi-identity credentials.json is
// adopted into the store and preserved when a new identity is added.
func TestMigrateLegacyCredentials(t *testing.T) {
sandbox(t)
// Simulate an old single-file login: only credentials.json exists.
legacy := credsFromToken(&tokenResp{AccessToken: makeJWT(map[string]any{
"email": "z@hanzo.ai", "owner": "hanzo", "sub": "h",
})})
if err := legacy.Save(); err != nil {
t.Fatal(err)
}
store, err := LoadIdentities()
if err != nil {
t.Fatalf("load identities: %v", err)
}
if store.Identities["hanzo/z"] == nil || store.Active != "hanzo/z" {
t.Fatalf("legacy credentials not migrated: active=%q keys=%v", store.Active, store.keys())
}
// A fresh login as a different owner preserves the migrated identity.
if _, err := runRoot(t, "", "login", "--token", makeJWT(map[string]any{
"email": "z@hanzo.ai", "owner": "admin", "sub": "a",
})); err != nil {
t.Fatal(err)
}
st2, _ := LoadIdentities()
if len(st2.Identities) != 2 || st2.Identities["hanzo/z"] == nil {
t.Fatalf("migrated identity lost after new login: %v", st2.keys())
}
}
+165 -1
View File
@@ -55,7 +55,7 @@ var controlCommands = map[string]string{
"login": "authenticate against Hanzo IAM (hanzo.id) and store a token",
"logout": "remove stored credentials",
"whoami": "show the current identity from the stored token",
"auth": "manage authentication (login, logout, whoami, token)",
"auth": "manage authentication + stored identities (login, logout, whoami, list, switch, token)",
"apps": "list/get the platform apps board (declared/running/drift)",
"deploy": "drive a platform redeploy (rolling restart, zero-downtime)",
"clusters": "provision/list/select dedicated DOKS clusters",
@@ -236,6 +236,170 @@ func DeleteCredentials() error {
return nil
}
// ---------------------------------------------------------------------------
// Identity store — ~/.hanzo/identities.json. Holds EVERY logged-in identity
// keyed by its stable "<owner>/<name>" key, with an Active pointer. On every
// write the active identity is mirrored into credentials.json (above), so every
// legacy single-file reader keeps seeing the current identity unchanged. This
// is the ONE credential store; credentials.json is its active-view mirror.
// ---------------------------------------------------------------------------
// IdentityStore is the on-disk shape of ~/.hanzo/identities.json.
type IdentityStore struct {
Active string `json:"active,omitempty"`
Identities map[string]*Credentials `json:"identities,omitempty"`
}
// key is the stable per-identity store key "<owner>/<name>", where name is the
// email local-part (else the raw subject). The same identity yields the same
// key every login, so re-login updates in place; the same email under a
// different org (privilege separation) yields a distinct key (admin/z vs
// hanzo/z) and is stored side by side rather than clobbering.
func (c *Credentials) key() string {
name := c.Subject
if i := strings.IndexByte(name, '@'); i > 0 {
name = name[:i]
}
return firstNonEmpty(c.Owner, "-") + "/" + firstNonEmpty(name, "-")
}
func identitiesPath() (string, error) {
dir, err := hanzoDir()
if err != nil {
return "", err
}
return filepath.Join(dir, "identities.json"), nil
}
// LoadIdentities reads the store. A pre-existing single-file credentials.json
// with no store yet is migrated in (read-only) as the sole, active identity, so
// upgrades are seamless — the first write persists it into the store.
func LoadIdentities() (*IdentityStore, error) {
p, err := identitiesPath()
if err != nil {
return nil, err
}
s := &IdentityStore{Identities: map[string]*Credentials{}}
if err := loadJSON(p, s); err != nil {
return nil, err
}
if s.Identities == nil {
s.Identities = map[string]*Credentials{}
}
if len(s.Identities) == 0 {
if c, err := LoadCredentials(); err == nil && c.AccessToken != "" {
k := c.key()
s.Identities[k] = c
s.Active = k
}
}
return s, nil
}
// keys returns the identity keys, sorted, for deterministic output.
func (s *IdentityStore) keys() []string {
ks := make([]string, 0, len(s.Identities))
for k := range s.Identities {
ks = append(ks, k)
}
sort.Strings(ks)
return ks
}
// Put stores c under its key and makes it active, returning the key.
func (s *IdentityStore) Put(c *Credentials) string {
if s.Identities == nil {
s.Identities = map[string]*Credentials{}
}
k := c.key()
s.Identities[k] = c
s.Active = k
return k
}
// Remove deletes an identity; Save re-points Active if it was the one removed.
func (s *IdentityStore) Remove(key string) { delete(s.Identities, key) }
// resolve turns a user selector into a stored key: an exact key wins; otherwise
// a bare owner matches iff exactly one identity carries it.
func (s *IdentityStore) resolve(sel string) (string, error) {
if _, ok := s.Identities[sel]; ok {
return sel, nil
}
var match []string
for _, k := range s.keys() {
if s.Identities[k].Owner == sel {
match = append(match, k)
}
}
switch len(match) {
case 1:
return match[0], nil
case 0:
return "", fmt.Errorf("no stored identity for %q (see `hanzo auth list`)", sel)
default:
return "", fmt.Errorf("%q is ambiguous across %s — pass the full owner/name key", sel, strings.Join(match, ", "))
}
}
// Save persists the store (0600) and mirrors the active identity into
// credentials.json for legacy single-file readers. When the store is empty it
// removes both files. Active is normalized to a real key first.
func (s *IdentityStore) Save() error {
if _, ok := s.Identities[s.Active]; !ok {
s.Active = ""
if ks := s.keys(); len(ks) > 0 {
s.Active = ks[0]
}
}
if len(s.Identities) == 0 {
return clearCredentialStore()
}
p, err := identitiesPath()
if err != nil {
return err
}
if err := writeJSON(p, s, 0o600); err != nil {
return err
}
return s.Identities[s.Active].Save() // mirror active → credentials.json (0600)
}
// SaveActive writes c back as the active identity (store + mirror), keeping the
// two consistent after an in-place token refresh. With no store yet it falls
// back to the single-file write.
func SaveActive(c *Credentials) error {
s, err := LoadIdentities()
if err != nil {
return err
}
if len(s.Identities) == 0 {
return c.Save()
}
k := s.Active
if k == "" || s.Identities[k] == nil {
k = c.key()
}
s.Identities[k] = c
s.Active = k
return s.Save()
}
// clearCredentialStore removes the identity store and its credentials.json
// mirror (used by logout when the last identity is removed).
func clearCredentialStore() error {
for _, pathOf := range []func() (string, error){credentialsPath, identitiesPath} {
p, err := pathOf()
if err != nil {
return err
}
if err := os.Remove(p); err != nil && !os.IsNotExist(err) {
return err
}
}
return nil
}
// ---------------------------------------------------------------------------
// Env — the effective, resolved settings a command operates with.
// ---------------------------------------------------------------------------
+150 -2
View File
@@ -54,6 +54,10 @@ const (
heartbeatEvery = 30 * time.Second
claimPoll = 2 * time.Second
claimLeaseSecs = 120
// renderWindow matches the dispatch cap (studio gpu_dispatch sets
// startToCloseTimeout 14400s). The old 10m local poll undercut it and
// marked live renders failed while they kept sampling (observed 8-70m).
renderWindow = 4 * time.Hour
// localComfyUI is the studio render backend the studio.render handler drives.
localComfyUI = "http://127.0.0.1:8188"
// defaultStudioUploadURL is where finished render outputs are POSTed so they
@@ -95,6 +99,7 @@ func newGPUCmd(envOf func() *Env, _ *globalFlags) *cobra.Command {
var engineEndpoint string
var registerProvider bool
var studioDir string
var studioURL string
connect := &cobra.Command{
Use: "connect",
Short: "Register this GPU and run the outbound worker loop",
@@ -107,6 +112,7 @@ func newGPUCmd(envOf func() *Env, _ *globalFlags) *cobra.Command {
engineEndpoint: engineEndpoint,
registerProvider: registerProvider,
studioDir: studioDir,
studioURL: studioURL,
}
if daemon {
return installDaemon(cmd, opts)
@@ -121,6 +127,7 @@ func newGPUCmd(envOf func() *Env, _ *globalFlags) *cobra.Command {
connect.Flags().StringVar(&engineEndpoint, "engine-endpoint", "", "public URL to advertise for gateway routing (defaults to --engine-url; a BYO node needs a reachable URL/tunnel)")
connect.Flags().BoolVar(&registerProvider, "register-provider", false, "auto-register the engine endpoint as an org model provider (POST /v1/add-provider)")
connect.Flags().StringVar(&studioDir, "studio-dir", os.Getenv("HANZO_STUDIO_DIR"), "local Hanzo Studio checkout; when set, connect launches and supervises the render backend on 127.0.0.1:8188")
connect.Flags().StringVar(&studioURL, "studio-url", firstNonEmpty(os.Getenv("HANZO_STUDIO_UPLOAD_URL"), defaultStudioUploadURL), "studio base URL the render mirror uploads finished images to (POST /v1/library/upload)")
status := &cobra.Command{
Use: "status",
@@ -370,6 +377,7 @@ type connectOpts struct {
engineEndpoint string // public URL to advertise (defaults to engineURL)
registerProvider bool // auto POST /v1/add-provider for the engine
studioDir string // local Studio checkout to launch + supervise on :8188
studioURL string // studio base the render mirror uploads finished images to
}
func runConnect(cmd *cobra.Command, env *Env, opts connectOpts) error {
@@ -422,6 +430,25 @@ func runConnect(cmd *cobra.Command, env *Env, opts connectOpts) error {
poll := time.NewTicker(claimPoll)
defer poll.Stop()
// Render mirror — independent of claims by design. It scans the local studio
// output tree every heartbeatEvery and uploads every image to the org's library
// (POST /v1/library/upload), so EVERY render lands in studio.hanzo.ai even when
// it was produced outside the job path — a graph hand-run on this node, or a
// render that finished after its activity was reaped (the stranded-late-render
// class). Active only when a studio checkout is named (there is local output to
// mirror); a nil channel case never fires when it is not.
w.studioUploadURL = firstNonEmpty(opts.studioURL, w.studioUploadURL)
mirrorBase := w.studioUploadURL
mirrorDir := ""
seen := map[string]int64{}
var mirC <-chan time.Time
if opts.studioDir != "" {
mirrorDir = filepath.Join(opts.studioDir, "output")
mir := time.NewTicker(heartbeatEvery)
defer mir.Stop()
mirC = mir.C
}
// Heartbeat once immediately so the machine reports online without waiting a
// full interval.
_ = w.heartbeat(ctx)
@@ -449,6 +476,8 @@ func runConnect(cmd *cobra.Command, env *Env, opts connectOpts) error {
if err := w.claimAndRun(ctx, out); err != nil {
fmt.Fprintf(cmd.ErrOrStderr(), "claim: %v\n", err)
}
case <-mirC:
w.mirrorRenders(ctx, out, mirrorDir, mirrorBase, seen)
}
}
}
@@ -732,7 +761,7 @@ func (e *Env) ensureToken(ctx context.Context) (string, error) {
nc.RefreshToken = e.creds.RefreshToken
}
*e.creds = *nc
_ = e.creds.Save()
_ = SaveActive(e.creds) // refresh the active identity in the store + mirror
}
// On refresh failure fall through: the current token may still be valid
// (clock skew) and the server is the authority.
@@ -823,7 +852,7 @@ func (w *worker) studioRenderHandler(ctx context.Context, input json.RawMessage)
return nil, fmt.Errorf("studio.render: no prompt_id in /prompt response")
}
// Poll history until the prompt shows up (completed).
deadline := time.Now().Add(10 * time.Minute)
deadline := time.Now().Add(renderWindow)
for time.Now().Before(deadline) {
select {
case <-ctx.Done():
@@ -847,6 +876,11 @@ func (w *worker) studioRenderHandler(ctx context.Context, input json.RawMessage)
if uerr != nil {
return nil, fmt.Errorf("studio.render: prompt %s rendered but gallery upload failed: %w", pr.PromptID, uerr)
}
// The engine leaks ~58GB per render; recycling after each completed
// render caps it at one render's worth. Boot (~20s) is noise next to
// 8-70m renders. Never recycle on the timeout path — the engine may
// still be sampling and the mirror rescues late finishes.
requestStudioRecycle()
return map[string]any{"promptId": pr.PromptID, "outputs": outputs, "gallery": gallery}, nil
}
}
@@ -961,6 +995,120 @@ func (w *worker) postGalleryOutput(ctx context.Context, base, tok, org, name, su
return filepath.Join(out.Subfolder, out.Name), nil
}
// isImageFile reports whether name carries a render image extension the library accepts.
func isImageFile(name string) bool {
switch strings.ToLower(filepath.Ext(name)) {
case ".png", ".jpg", ".jpeg", ".webp":
return true
}
return false
}
// mirrorRenders scans dir (the local studio output tree) for image files new or
// changed since the last scan and POSTs each to base/v1/library/upload with the
// worker's bearer, tagged with this node's identity, so EVERY render lands in the
// org's studio library — including ones produced OUTSIDE the job path. seen (rel
// path -> size) skips unchanged files; the endpoint dedupes, so a re-scan after a
// restart is cheap and harmless. One log line per newly stored file; upload
// failures are summarized once per scan and retried next tick (no 5xx log spam).
func (w *worker) mirrorRenders(ctx context.Context, out io.Writer, dir, base string, seen map[string]int64) {
tok, err := w.env.ensureToken(ctx)
if err != nil {
return
}
base = strings.TrimRight(base, "/")
failed := 0
var firstErr error
_ = filepath.Walk(dir, func(p string, info os.FileInfo, werr error) error {
if werr != nil || info == nil || info.IsDir() || !isImageFile(p) {
return nil
}
// Hidden files and AppleDouble forks (`._*`, `.DS_Store`) ride along with
// mac scp and are not renders — `._foo.png` passes the extension check
// but is a 4KB resource fork that poisons the library.
if strings.HasPrefix(filepath.Base(p), ".") {
return nil
}
rel, rerr := filepath.Rel(dir, p)
if rerr != nil {
return nil
}
rel = filepath.ToSlash(rel)
if seen[rel] == info.Size() {
return nil
}
data, derr := os.ReadFile(p)
if derr != nil || len(data) == 0 {
return nil
}
sub, name := "", rel
if i := strings.LastIndex(rel, "/"); i >= 0 {
sub, name = rel[:i], rel[i+1:]
}
existed, perr := w.postLibraryUpload(ctx, base, tok, sub, name, data)
if perr != nil {
failed++
if firstErr == nil {
firstErr = perr
}
return nil
}
seen[rel] = info.Size()
if !existed {
fmt.Fprintf(out, "mirrored %s (%d bytes) -> %s\n", rel, len(data), base)
}
return nil
})
if failed > 0 {
fmt.Fprintf(out, "mirror: %d file(s) failed to upload, will retry: %v\n", failed, firstErr)
}
}
// postLibraryUpload multipart-POSTs one image to base/v1/library/upload with the
// worker's IAM bearer, landing it in the org's library (orgs/{org}/output). The
// file's subfolder rides as ?subpath and this node's identity as ?node so the
// render is filterable by its source in Queue & History. Returns whether the
// endpoint already had a byte-identical copy (dedup).
func (w *worker) postLibraryUpload(ctx context.Context, base, tok, sub, name string, data []byte) (bool, error) {
var buf bytes.Buffer
mw := multipart.NewWriter(&buf)
part, err := mw.CreateFormFile("image", name)
if err != nil {
return false, err
}
if _, err := part.Write(data); err != nil {
return false, err
}
if err := mw.Close(); err != nil {
return false, err
}
q := url.Values{"node": {w.identity}}
if sub != "" {
q.Set("subpath", sub)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, base+"/v1/library/upload?"+q.Encode(), &buf)
if err != nil {
return false, err
}
req.Header.Set("Authorization", "Bearer "+tok)
req.Header.Set("Content-Type", mw.FormDataContentType())
req.Header.Set("Accept", "application/json")
resp, err := w.http.Do(req)
if err != nil {
return false, err
}
defer resp.Body.Close()
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if resp.StatusCode/100 != 2 {
return false, fmt.Errorf("POST /v1/library/upload HTTP %d: %s", resp.StatusCode, serverMessage(raw))
}
var out struct {
Existed bool `json:"existed"`
}
_ = json.Unmarshal(raw, &out)
return out.Existed, nil
}
// inputImage is one uploaded input shipped with the job: a base64 blob plus the
// input-dir-relative location it must occupy on this worker so LoadImage finds it.
type inputImage struct {
+92
View File
@@ -0,0 +1,92 @@
package cli
import (
"bytes"
"context"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"time"
)
// TestMirrorRenders verifies the render mirror: it scans the local studio output
// tree and POSTs every image (new or changed) to /v1/library/upload with the node's
// identity + subfolder + bearer, skips unchanged files across scans, and re-uploads
// a changed file. This is the path that lands EVERY render in studio.hanzo.ai even
// when it was produced outside the job/claim path.
func TestMirrorRenders(t *testing.T) {
t.Setenv("HANZO_TOKEN", "test-bearer")
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "renders"), 0o755); err != nil {
t.Fatal(err)
}
write := func(rel, body string) {
if err := os.WriteFile(filepath.Join(dir, filepath.FromSlash(rel)), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
write("renders/a.png", "\x89PNG-a")
write("top.jpg", "jpg-top")
write("notes.txt", "not an image")
type up struct{ name, node, subpath, auth string }
var got []up
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/library/upload" {
http.Error(w, "not found", http.StatusNotFound)
return
}
_ = r.ParseMultipartForm(1 << 20)
name := ""
if r.MultipartForm != nil {
for _, fh := range r.MultipartForm.File["image"] {
name = fh.Filename
}
}
got = append(got, up{
name: name,
node: r.URL.Query().Get("node"),
subpath: r.URL.Query().Get("subpath"),
auth: r.Header.Get("Authorization"),
})
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"ok":true,"path":"x"}`))
}))
defer srv.Close()
w := &worker{env: &Env{}, http: &http.Client{Timeout: 10 * time.Second}, identity: "spark"}
seen := map[string]int64{}
var buf bytes.Buffer
w.mirrorRenders(context.Background(), &buf, dir, srv.URL, seen)
if len(got) != 2 {
t.Fatalf("uploaded %d files, want 2 images (the .txt is skipped): %+v", len(got), got)
}
var a *up
for i := range got {
if got[i].name == "a.png" {
a = &got[i]
}
}
if a == nil || a.subpath != "renders" || a.node != "spark" || a.auth != "Bearer test-bearer" {
t.Fatalf("renders/a.png upload = %+v, want subpath=renders node=spark bearer set", a)
}
// A second scan re-uploads nothing (seen matches every size).
got = nil
w.mirrorRenders(context.Background(), &buf, dir, srv.URL, seen)
if len(got) != 0 {
t.Fatalf("second scan uploaded %d files, want 0 (all unchanged): %+v", len(got), got)
}
// A changed file is re-uploaded on the next scan.
write("renders/a.png", "\x89PNG-a-grew")
got = nil
w.mirrorRenders(context.Background(), &buf, dir, srv.URL, seen)
if len(got) != 1 || got[0].name != "a.png" {
t.Fatalf("after change, uploaded %+v, want just renders/a.png", got)
}
}
+13
View File
@@ -107,6 +107,17 @@ func stopStudio(cmd *exec.Cmd) {
}
}
// studioRecycle carries at most one pending recycle request; the render
// handler signals it after each completed render (see gpu.go).
var studioRecycle = make(chan struct{}, 1)
func requestStudioRecycle() {
select {
case studioRecycle <- struct{}{}:
default:
}
}
// superviseStudio keeps the local render backend on :8188 alive until ctx
// ends. Quiet by design: one line per restart event, not a probe firehose.
func superviseStudio(ctx context.Context, dir string, out io.Writer) {
@@ -145,6 +156,8 @@ func superviseStudio(ctx context.Context, dir string, out io.Writer) {
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGTERM)
}
return
case <-studioRecycle:
restart("recycle")
case <-tick.C:
if studioHealthy(ctx) {
continue
+4
View File
@@ -161,6 +161,10 @@ func routesBridge(s *cloud.Service[state], app *zip.App) {
// console calls, forwarded to commerce with the admin service token and SCOPED to the
// validated caller's own subject (billing.go). Registered AFTER clients/billing's
// specific routes (121 < 122) so those win and this catches the rest. GET+POST only.
// The wildcard is what the ROUTER matches; it is NOT the forwardable set — billing.go's
// billingForwardable allowlist decides that, per method, and 404s everything else
// BEFORE the admin service token is attached. Widening this pattern grants nothing on
// its own; adding a line to that table is the only way to expose an endpoint.
app.Get("/v1/billing/*", cloud.Handle(s, billingData))
app.Post("/v1/billing/*", requireCSRF(s, cloud.Handle(s, billingData)))
// Per-tenant STORE DATA bridge — the canonical /v1/commerce/* the console calls,
+126 -13
View File
@@ -7,6 +7,14 @@
// read/act on its OWN ledger (balance / usage / invoices / subscriptions /
// payment-methods / spend-alerts / …), never another's.
//
// TWO INDEPENDENT BOUNDS, because the token makes this a privileged forwarder:
// 1. WHICH ENDPOINT — billingForwardable, the per-method allowlist below. It is the
// authorization gate: an unlisted path is 404'd before the token is ever attached, so
// no money-MINT route (deposit/credit/refund/…) can be reached through this bridge.
// 2. WHOSE DATA — the subject-pinning below. It aims a permitted call at the caller's own
// ledger. It is an IDOR control and NOT an authority control: on a mint route it would
// have pinned the CREDIT to the attacker's own account. (1) is what stops that.
//
// WHY A SERVER HANDLER (not a same-origin passthrough). Commerce's billing surface is
// service-token-gated and filters DIFFERENT endpoints on DIFFERENT subject params —
// subscriptions on ?userId, payment-methods on ?customerId, usage on ?user. Pinning
@@ -28,10 +36,115 @@ import (
"strings"
"unicode"
"github.com/hanzoai/account"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
)
// billingForwardable — THE allowlist of billing endpoints this bridge may forward, keyed
// by method. It is the whole authorization story of the bridge, because forwarding IS
// authorization here: every forwarded request carries the admin COMMERCE_SERVICE_TOKEN,
// and commerce's money gate is MayMintMoney(c) = IsServiceToken(c) || IsSuperAdmin(c)
// (middleware/platformonly.go). The token satisfies IsServiceToken, so ANY subpath that
// reaches commerce is executed with PLATFORM authority — not the caller's. Commerce 403s
// an org admin who calls POST /v1/billing/deposit directly; without this table the bridge
// handed that same person the platform's own credential and minted it for them, scoped —
// by the subject-pinning below — to their OWN account. That is the escalation, and
// subject-pinning is what AIMS it, not what stops it. Only a path gate stops it.
//
// It is an ALLOWLIST, never a denylist: a denylist must enumerate every mint route
// (deposit/credit/refund/credit-grants/payouts/husd/allotment…) and stays correct only
// until commerce adds the next one — a route this file has never heard of is then
// forwarded by default. Here the default is REFUSE, so a new commerce mint route is
// unreachable the day it lands, with no change on this side. One table, one place; a path
// not in it cannot reach commerce, by construction.
//
// GET and POST are SEPARATE sets because a read bridge and a write bridge are different
// concerns: `payouts` is a legitimate read and a money-MINT write (api/billing/handlers.go
// `api.Get("/payouts", ListPayouts)` vs `api.Post("/payouts", mintRequired, CreatePayout)`),
// so one method-blind set would hand the mint to every reader. The POST set is therefore
// deliberately tiny and holds NOTHING that creates spendable balance from a client-named
// amount: cancel/reactivate a subscription, vault a card, create a budget, and a top-up
// that CHARGES a real card (money in, not minted). Every entry is a call the console
// actually makes; `{}` matches exactly one opaque id segment.
//
// EVIDENCE — each entry is a live console call (repo hanzoai/console):
//
// GET balance src/lib/api/billing.ts:397 sidebar wallet + billing overview
// GET usage src/lib/api/billing.ts:415 cost reports / AI metrics
// GET invoices src/lib/api/billing.ts:419 invoice history table
// GET invoices/{}/pdf src/components/products/billing/BillingInvoices.tsx:31
// GET subscriptions src/lib/api/billing.ts:423 subscriptions list
// GET payment-methods src/lib/api/billing.ts:450 saved cards (masked)
// GET spend-alerts src/lib/api/billing.ts:482 budgets / spend caps
// GET payment-config src/lib/api/billing.ts:552 public Square app/location id
// GET plans src/lib/api/plans.ts:126 published tiers
// GET payouts src/components/products/SettlementModule.tsx:61 settlement view
// POST subscriptions/{}/cancel src/lib/api/billing.ts:434
// POST subscriptions/{}/reactivate src/lib/api/billing.ts:444
// POST payment-methods src/lib/api/billing.ts:461 vault a Square nonce (no PAN)
// POST spend-alerts src/lib/api/billing.ts:500 create a budget
// POST topup/token src/lib/api/billing.ts:565 charge a card → credit
//
// balance/usage/payment-methods are ALSO served natively by clients/billing (order 121),
// which wins over this catch-all (122), so those entries are reached only on a deploy
// where that subsystem is disabled. They are listed because they are legitimate reads of
// the caller's own ledger, not because this bridge is their primary route.
//
// NOT LISTED, deliberately: `me/welcome` and `grant-starter` (console calls the first at
// billing.ts:407 and the second server-side at src/lib/server/billing-grant.ts:35) exist
// in NEITHER the pinned commerce (v1.48.5) route table — both 404 today whether or not
// this bridge forwards them, and grant-starter is mint-gated and browser-unreachable by
// design. The console's PATCH/DELETE calls (spend-alerts/{}, payment-methods/{}) are absent
// because routesBridge mounts GET+POST only, so they never reached this handler.
var billingForwardable = map[string][]string{
http.MethodGet: {
"balance",
"usage",
"invoices",
"invoices/{}/pdf",
"subscriptions",
"payment-methods",
"spend-alerts",
"payment-config",
"plans",
"payouts",
},
http.MethodPost: {
"subscriptions/{}/cancel",
"subscriptions/{}/reactivate",
"payment-methods",
"spend-alerts",
"topup/token",
},
}
// isForwardableBilling reports whether method+sub is in billingForwardable. sub has
// already passed isSafeSegment, so no segment can contain a slash, a percent-escape, or a
// traversal — a pattern segment therefore matches exactly one real segment and `{}` cannot
// swallow a path. Fail-closed: an unknown method or an unlisted path is false.
func isForwardableBilling(method, sub string) bool {
got := strings.Split(sub, "/")
for _, pattern := range billingForwardable[method] {
want := strings.Split(pattern, "/")
if len(want) != len(got) {
continue
}
match := true
for i, seg := range want {
if seg != "{}" && seg != got[i] {
match = false
break
}
}
if match {
return true
}
}
return false
}
// billingSubjectKeys — every query/body param through which a commerce billing endpoint
// identifies its subject. Kept identical to commerce's edge-auth billingSubjectKeys
// {user,userId,customerId} AND console's billing-scope.ts BILLING_SUBJECT_KEYS. Change
@@ -48,17 +161,6 @@ func isSubjectKey(k string) bool {
return false
}
// billingSubject — the commerce billing subject for an org+user: ALWAYS the org
// (`org`), lowercased. Every member of an org reads/scopes to the ONE org billing
// account — the same subject the gateway gate reads and debits. `name` is recorded
// for metrics, never for the billing key. This is the ONE rule; the former
// PERSONAL_BILLING_ORGS / ORG_BILLING_ORGS allowlists are gone. Keep in lockstep
// with ai/object.BillingSubject so the console view and the gate never disagree.
func billingSubject(org, name string) string {
_ = name
return strings.ToLower(strings.TrimSpace(org))
}
// scopedBillingSearch — pin every billingSubjectKey to subject (OVERWRITING any client
// value — the browser cannot widen scope) and DROP org. Every OTHER param (currency,
// status, date range) passes through untouched. Mirrors billing-scope.ts.
@@ -165,10 +267,21 @@ func billingData(s *cloud.Service[state], c *zip.Ctx) error {
return zip.ErrBadRequest("invalid billing path")
}
}
// THE authorization gate. Forwarding is authorization: the request below carries the
// admin service token, which satisfies commerce's MayMintMoney. So refuse anything the
// console does not actually call — BEFORE the token is attached. Fail closed (404, the
// same answer an unrouted path gives, so this leaks no map of the money surface).
if !isForwardableBilling(method, sub) {
return zip.Errorf(http.StatusNotFound, "not a forwardable billing endpoint")
}
// Scope EVERY request to the caller's OWN subject — query AND write body — so
// commerce's per-tenant isolation can never be crossed from the browser.
subject := billingSubject(cr.owner, cr.name)
// commerce's per-tenant isolation can never be crossed from the browser. The
// subject comes from the ONE rule (ai/object.Payer), keyed on the IAM username
// (cr.username = X-User-Name) the gate also keys on — so a top-up credits the
// SAME account the gate debits. Keying on cr.name (X-User-Id, a UUID on the
// direct-bearer path) would fund an account the gate never reads: the split.
subject := account.Payer(account.Credential{Owner: cr.owner, Name: cr.username}).Subject()
inQuery, _ := url.ParseQuery(string(c.Fiber().Request().URI().QueryString()))
q := scopedBillingSearch(inQuery, subject)
+26 -18
View File
@@ -2,6 +2,7 @@ package account
import (
"encoding/json"
"github.com/hanzoai/account"
"io"
"net/http"
"net/http/httptest"
@@ -16,37 +17,44 @@ import (
// ── pure scoping ─────────────────────────────────────────────────────────────
// TestBillingSubject proves the top-up subject is resolved through the ONE rule
// (ai/object.Payer) — so a top-up credits the SAME account the ai gate debits and
// the console reads. The signup org bills per-person (matching the gate), which is
// the whole fix: money and gate land on one account.
func TestBillingSubject(t *testing.T) {
cases := []struct{ org, name, want string }{
{"acme", "alice", "acme"}, // any member bills the ONE org account
{"hanzo", "Dave", "hanzo"}, // no per-user wallet; org, lowercased
{"hanzo", "z", "hanzo"}, // another member — same org account
{"hanzo", "", "hanzo"}, // no name → org
{"Hanzo", "z", "hanzo"}, // lowercased
{"", "x", ""}, // no org → empty subject
{"acme", "alice", "acme"}, // real org: any member bills the ONE org account
{"hanzo", "Dave", "hanzo/dave"}, // signup org: each person bills their OWN account
{"hanzo", "z", "hanzo/z"}, // another signup person — their own account
{"hanzo", "", "hanzo"}, // no name (org-owned principal) → org pool
{"Hanzo", "Z", "hanzo/z"}, // folded
{"", "x", ""}, // no org → empty subject (cannot bill)
}
for _, c := range cases {
if got := billingSubject(c.org, c.name); got != c.want {
t.Fatalf("billingSubject(%q,%q): want %q, got %q", c.org, c.name, c.want, got)
got := account.Payer(account.Credential{Owner: c.org, Name: c.name}).Subject()
if got != c.want {
t.Fatalf("Payer(%q,%q).Subject(): want %q, got %q", c.org, c.name, c.want, got)
}
}
}
// TestBillingSubject_IgnoresLegacyEnv locks that the killed allowlist envs have NO
// effect: the subject is ALWAYS the org, whether or not the old PERSONAL_BILLING_ORGS
// / ORG_BILLING_ORGS knobs are set. This mirrors ai/object.BillingSubject (one rule,
// no config) so the console view and the gateway gate can never disagree.
// effect: nothing reads them. Set to values that WOULD have flipped every
// resolution — the subject is unchanged. This is the console/top-up half of the
// same proof ai carries (one rule, no config), so the view and the gate can never
// disagree, and the deleted CR env is a genuine no-op.
func TestBillingSubject_IgnoresLegacyEnv(t *testing.T) {
t.Setenv("PERSONAL_BILLING_ORGS", "hanzo,acme")
t.Setenv("ORG_BILLING_ORGS", "hanzo")
t.Setenv("PERSONAL_BILLING_ORGS", "hanzo,acme") // would have split acme per-user
t.Setenv("ORG_BILLING_ORGS", "hanzo") // would have pooled the signup org
cases := []struct{ org, name, want string }{
{"hanzo", "z", "hanzo"},
{"acme", "alice", "acme"},
{"maxpower", "dave", "maxpower"},
{"hanzo", "z", "hanzo/z"}, // env cannot pool the signup org
{"acme", "alice", "acme"}, // env cannot split a real org per-user
{"maxpower", "dave", "maxpower"}, // untouched
}
for _, c := range cases {
if got := billingSubject(c.org, c.name); got != c.want {
t.Fatalf("legacy env must be ignored: billingSubject(%q,%q) want %q, got %q", c.org, c.name, c.want, got)
got := account.Payer(account.Credential{Owner: c.org, Name: c.name}).Subject()
if got != c.want {
t.Fatalf("legacy env must be ignored: Payer(%q,%q).Subject() want %q, got %q", c.org, c.name, c.want, got)
}
}
}
+257
View File
@@ -0,0 +1,257 @@
package account
import (
"net/http"
"strings"
"testing"
commercebilling "github.com/hanzoai/commerce/api/billing"
commercemid "github.com/hanzoai/commerce/middleware"
"github.com/zap-proto/zip"
)
// bridge_mint_test.go — the privilege-escalation boundary of the /v1/billing/*
// bridge: an ordinary signed-in ORG user must never reach commerce's money-MINT
// surface.
//
// THE ESCALATION THIS LOCKS OUT. The bridge forwards to commerce with the admin
// COMMERCE_SERVICE_TOKEN. Commerce gates every mint on
// MayMintMoney(c) = IsServiceToken(c) || IsSuperAdmin(c) (middleware/platformonly.go)
// — and the bridge's service token satisfies IsServiceToken. So ANY subpath the
// bridge forwards is executed by commerce as the PLATFORM, not as the caller.
// billingData scopes the SUBJECT to the caller's own account, which is exactly the
// attack rather than a defense: an org user mints to THEMSELVES. Commerce's own
// gate comment names this: "let ANY org owner self-credit unlimited balance (POST
// /v1/billing/deposit &c.) → unlimited free inference."
//
// Commerce 403s that same org admin when they call it DIRECTLY
// (TestC1_OrgAdminDeniedOnEveryMintRoute) and mints 201 for the service token
// (TestC1_ServiceTokenMintsDeposit). The bridge is what converts the former into
// the latter. The gate therefore has to live HERE, at the point that hands out the
// token: forwardable subpaths are an ALLOWLIST, and a mint path is not on it.
//
// alice is an ordinary org user — X-Org-Id "acme", owner != "admin", NOT a
// SuperAdmin — i.e. precisely the principal commerce refuses at the front door.
// TestBridge_OrgUserCannotReachMint is the reproduction. Each of these commerce
// subpaths is PlatformOnly-gated (api/billing/handlers.go: `mintRequired`), meaning
// possession of the service token IS authority to create spendable balance. None
// may leave cloud. A request that never reaches commerce cannot mint, so the
// assertion is twofold: the caller is refused AND upstream saw nothing.
// mintSurface asks COMMERCE which routes it gates, rather than keeping a copy.
//
// The list used to live here by hand under "kept in lockstep with
// api/billing/handlers.go" — and it had already drifted: 10 paths here against
// 16 commerce actually gates. A comment cannot hold two lists together. Now
// commerce DECLARES its gated surface (middleware.Mint records what it gates)
// and we read that declaration, so a mint route added there is covered here with
// nobody remembering to do anything.
//
// Registration is what populates the registry, so register first, then read.
func mintSurface(t *testing.T) []commercemid.MintRoute {
t.Helper()
commercebilling.Route(zip.New(zip.Config{DisableStartupMessage: true}).Group("/v1"))
var out []commercemid.MintRoute
for _, r := range commercemid.MintRoutes() {
// Only what THIS bridge can address: it forwards /v1/billing/* alone.
if !strings.HasPrefix(r.Path, "/v1/billing/") {
continue
}
// A wildcard segment needs some concrete value to be requestable; which
// one is irrelevant, since a refused call never reaches an id.
parts := strings.Split(r.Path, "/")
for i, seg := range parts {
if strings.HasPrefix(seg, ":") || seg == "{}" {
parts[i] = "probe"
}
}
r.Path = strings.Join(parts, "/")
out = append(out, r)
}
if len(out) == 0 {
t.Fatal("commerce declared no /v1/billing mint routes — the registry is not being populated")
}
return out
}
func TestBridge_OrgUserCannotReachMint(t *testing.T) {
mintPaths := mintSurface(t)
t.Logf("commerce declares %d gated /v1/billing mint routes", len(mintPaths))
for _, m := range mintPaths {
t.Run(m.Method+" "+m.Path, func(t *testing.T) {
f := &fakeBilling{}
t.Setenv("COMMERCE_URL", f.server(t).URL)
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-tok")
app := mountApp(t, "http://iam.invalid", "", "")
code, body := callH(t, app, m.Method, m.Path, alice, `{}`)
// The mint request must NEVER reach commerce: arriving there at all means
// it arrived bearing the admin service token, which IS the authority to
// mint (MayMintMoney → AuthorizeMint → the ledger write).
if f.path != "" {
t.Fatalf("ESCALATION: an ordinary org user's %s %s reached commerce at %q "+
"carrying %q — the service token that satisfies MayMintMoney. "+
"Minted subject=%v amount=%v in org=%q.",
m.Method, m.Path, f.path, f.auth, f.body["user"], f.body["amount"], f.org)
}
if code != http.StatusNotFound {
t.Fatalf("%s %s: want 404 (not a forwardable billing endpoint), got %d (%s)",
m.Method, m.Path, code, body)
}
})
}
}
// TestBridge_ConsoleCallsStillForward is the other half of the allowlist: the calls the
// console ACTUALLY makes must still reach commerce. An allowlist that blocks the product
// is not a fix, so each entry here is a live console call (cited in billing.go), and this
// test fails if a future edit narrows the table below the console's real needs.
func TestBridge_ConsoleCallsStillForward(t *testing.T) {
calls := []struct{ method, path, want string }{
{http.MethodGet, "/v1/billing/invoices", "/v1/billing/invoices"},
{http.MethodGet, "/v1/billing/invoices/inv_123/pdf", "/v1/billing/invoices/inv_123/pdf"},
{http.MethodGet, "/v1/billing/subscriptions", "/v1/billing/subscriptions"},
{http.MethodGet, "/v1/billing/spend-alerts", "/v1/billing/spend-alerts"},
{http.MethodGet, "/v1/billing/payment-config", "/v1/billing/payment-config"},
{http.MethodGet, "/v1/billing/plans", "/v1/billing/plans"},
{http.MethodGet, "/v1/billing/payouts", "/v1/billing/payouts"},
{http.MethodPost, "/v1/billing/subscriptions/sub_1/cancel", "/v1/billing/subscriptions/sub_1/cancel"},
{http.MethodPost, "/v1/billing/subscriptions/sub_1/reactivate", "/v1/billing/subscriptions/sub_1/reactivate"},
{http.MethodPost, "/v1/billing/payment-methods", "/v1/billing/payment-methods"},
{http.MethodPost, "/v1/billing/spend-alerts", "/v1/billing/spend-alerts"},
{http.MethodPost, "/v1/billing/topup/token", "/v1/billing/topup/token"},
}
for _, call := range calls {
t.Run(call.method+" "+call.path, func(t *testing.T) {
f := &fakeBilling{}
t.Setenv("COMMERCE_URL", f.server(t).URL)
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-tok")
app := mountApp(t, "http://iam.invalid", "", "")
code, body := callH(t, app, call.method, call.path, alice, "{}")
if code != http.StatusOK {
t.Fatalf("%s %s: want 200 (the console needs this), got %d (%s)",
call.method, call.path, code, body)
}
if f.path != call.want {
t.Fatalf("%s %s must forward to %q, got %q", call.method, call.path, call.want, f.path)
}
})
}
}
// TestBridge_UnlistedPathsAreRefused covers the rest of the money surface — routes that
// are NOT mint-gated but that the console never calls. The bridge is not a general
// commerce proxy; least privilege means "only what the product needs", so these 404
// even though commerce would have served them to a service token.
func TestBridge_UnlistedPathsAreRefused(t *testing.T) {
unlisted := []struct{ method, path string }{
{http.MethodPost, "/v1/billing/invoices"}, // CreateInvoice (admin group)
{http.MethodPost, "/v1/billing/invoices/i1/pay"}, // PayInvoice
{http.MethodPost, "/v1/billing/invoices/i1/void"}, // VoidInvoice
{http.MethodPost, "/v1/billing/meters"}, // CreateMeter
{http.MethodPost, "/v1/billing/pricing-rules"}, // CreatePricingRule
{http.MethodPost, "/v1/billing/withdraw"}, // money OUT
{http.MethodPost, "/v1/billing/usage"}, // RecordUsage — the meter itself
{http.MethodGet, "/v1/billing/balance/all"}, // every subject's balance
{http.MethodGet, "/v1/billing/sbom"}, // OSS payout surface
{http.MethodGet, "/v1/billing/oss-payout/summary"}, // OSS payout rollup
{http.MethodPost, "/v1/billing/subscriptions"}, // CreateBillingSubscription
}
for _, u := range unlisted {
t.Run(u.method+" "+u.path, func(t *testing.T) {
f := &fakeBilling{}
t.Setenv("COMMERCE_URL", f.server(t).URL)
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-tok")
app := mountApp(t, "http://iam.invalid", "", "")
code, _ := callH(t, app, u.method, u.path, alice, "{}")
if f.path != "" {
t.Fatalf("%s %s is not a console call and must not reach commerce, but upstream saw %q",
u.method, u.path, f.path)
}
if code != http.StatusNotFound {
t.Fatalf("%s %s: want 404, got %d", u.method, u.path, code)
}
})
}
}
// TestBridge_ReadAllowlistIsNotAWriteAllowlist pins the method split. `payouts` is the
// proof that one method-blind set would be a hole: GET /payouts is a plain read, POST
// /payouts is `mintRequired` (api/billing/handlers.go). The same string must resolve
// differently by method, or reading the settlement view would grant minting a payout.
func TestBridge_ReadAllowlistIsNotAWriteAllowlist(t *testing.T) {
if !isForwardableBilling(http.MethodGet, "payouts") {
t.Fatal("GET payouts is a live console read and must be forwardable")
}
if isForwardableBilling(http.MethodPost, "payouts") {
t.Fatal("POST payouts is mint-gated in commerce and must NEVER be forwardable")
}
// A GET-only entry must not leak into POST, and vice-versa.
if isForwardableBilling(http.MethodPost, "invoices") {
t.Fatal("POST invoices must not inherit the GET entry")
}
if isForwardableBilling(http.MethodGet, "topup/token") {
t.Fatal("GET topup/token must not inherit the POST entry")
}
// An unknown method fails closed (the router mounts GET+POST only; defense in depth).
for _, m := range []string{http.MethodPut, http.MethodPatch, http.MethodDelete, ""} {
if isForwardableBilling(m, "balance") {
t.Fatalf("method %q must fail closed", m)
}
}
// `{}` matches exactly ONE segment — it can never swallow a path into a mint route.
if isForwardableBilling(http.MethodPost, "subscriptions/a/b/cancel") {
t.Fatal("{} must match exactly one segment")
}
}
// TestBridge_StoreBridgeCannotReachBilling is the sibling lock. /v1/commerce/* carries the
// SAME admin token with FULL CRUD, and its own allowlist (commerceStoreHeads) is what keeps
// it a store proxy. Prove it cannot tunnel into the money surface — a store head that
// resolved to `billing` would reopen this hole from the other bridge.
func TestBridge_StoreBridgeCannotReachBilling(t *testing.T) {
for _, p := range []string{
"/v1/commerce/billing/deposit",
"/v1/commerce/billing",
"/v1/commerce/checkout",
"/v1/commerce/_/commerce/tenants",
} {
f := &fakeBilling{}
t.Setenv("COMMERCE_URL", f.server(t).URL)
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-tok")
app := mountApp(t, "http://iam.invalid", "", "")
code, _ := callH(t, app, http.MethodPost, p, alice, `{"amount":100000000}`)
if f.path != "" {
t.Fatalf("store bridge %q must never reach commerce, but upstream saw %q", p, f.path)
}
if code != http.StatusNotFound {
t.Fatalf("store bridge %q: want 404, got %d", p, code)
}
}
}
// TestBridge_MintIsRefusedEvenWithForgedSubject proves the refusal does not depend
// on the subject-pinning. Pinning is an IDOR control, not an authority control: it
// makes the mint land on the CALLER's own account, which is the attack, not a
// defense. The path gate must refuse before any of that logic runs.
func TestBridge_MintIsRefusedEvenWithForgedSubject(t *testing.T) {
f := &fakeBilling{}
t.Setenv("COMMERCE_URL", f.server(t).URL)
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-tok")
app := mountApp(t, "http://iam.invalid", "", "")
code, _ := callH(t, app, http.MethodPost, "/v1/billing/deposit", alice,
`{"user":"victim","userId":"victim","amount":100000000}`)
if f.path != "" {
t.Fatalf("ESCALATION: deposit reached commerce at %q with the service token", f.path)
}
if code != http.StatusNotFound {
t.Fatalf("forged-subject deposit: want 404, got %d", code)
}
}
+6 -1
View File
@@ -108,8 +108,13 @@ func routes(app *zip.App, s *cloud.Service[core.State]) {
app.Get("/v1/admin/bases", core.GuardScoped(s, bases))
// ── Platform control plane — SuperAdmin ONLY (launch/release/flags + access). ──
app.Get("/v1/admin/flags", core.Guard(s, flags))
app.Get("/v1/admin/flags", core.Guard(s, flagsBoard))
app.Put("/v1/admin/flags/:key", core.Guard(s, setFlag))
// Launch-control services board — the waitlist-mode lens on the flag engine (twin
// of /v1/admin/flags), folded in from the former featuregate control plane.
app.Get("/v1/admin/services", core.Guard(s, services))
app.Post("/v1/admin/services", core.Guard(s, upsertService))
app.Post("/v1/admin/services/:service/mode", core.Guard(s, setServiceMode))
app.Get("/v1/admin/waitlist", core.Guard(s, waitlist))
app.Post("/v1/admin/waitlist/boost", core.Guard(s, waitlistBoost))
+28 -12
View File
@@ -18,9 +18,8 @@ import (
"github.com/hanzoai/cloud/audit"
"github.com/hanzoai/cloud/clients/admin/money"
"github.com/hanzoai/cloud/clients/finance"
finmoney "github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/cloud/clients/principal"
"github.com/hanzoai/cloud/types"
"github.com/hanzoai/commerce/billing/creditledger"
"github.com/zap-proto/zip"
)
@@ -183,22 +182,39 @@ func ApplyGrant(s *cloud.Service[State], c *zip.Ctx, org string, req CreditReque
// one shape regardless of which path moved the money.
func grantDeposit(s *cloud.Service[State], c *zip.Ctx, org, currency, notes, tag, source string, amountCents int64) (before int64, txID string, after int64, afterExact string, err error) {
ctx := c.Context()
if fin := finance.Current(); fin != nil {
if bal, berr := fin.Balance(ctx, org, org, currency, false); berr == nil {
before = bal.Cents()
// ONE credit path: prefer the in-proc commerce credit ledger (creditledger) — the
// SAME injected ledger adapter commerce's POST /v1/billing/credit mints through
// and the ai prepaid gate reads. An admin grant and a self-serve credit thus move
// money the ONE way, into the ONE ledger; the admin path no longer carries its own
// parallel finance.Deposit. The operator-nonce idempotency key rides through so a
// retried grant dedupes (finance dedups on Ref). Before/after balances are read from
// the SAME co-resident finance ledger for the audit trail (exact, sub-cent visible).
if led := creditledger.Get(); led != nil {
if fin := finance.Current(); fin != nil {
if bal, berr := fin.Balance(ctx, org, org, currency, false); berr == nil {
before = bal.Cents()
}
}
id, derr := fin.Deposit(ctx, types.DepositInput{
Org: org, Subject: org, Amount: finmoney.FromCents(amountCents), Currency: currency, Notes: notes, Tags: tag,
id, balCents, cerr := led.Credit(ctx, creditledger.CreditInput{
Org: org,
Currency: currency,
Reason: notes,
Tag: tag,
IdempotencyKey: grantIdempotencyKey(c, org, currency, source, amountCents),
AmountCents: amountCents,
})
if derr != nil {
return before, "", before, "", derr
if cerr != nil {
return before, "", before, "", cerr
}
if bal, berr := fin.Balance(ctx, org, org, currency, false); berr == nil {
after, afterExact = bal.Cents(), bal.IntString() // afterExact = the EXACT balance (sub-cent visible)
after = balCents
if fin := finance.Current(); fin != nil {
if bal, berr := fin.Balance(ctx, org, org, currency, false); berr == nil {
afterExact = bal.AttoString() // afterExact = the EXACT balance (sub-cent visible)
}
}
return before, id, after, afterExact, nil
}
// Split deploy: no co-resident finance ledger → the commerce billing HTTP deposit, with
// Split deploy: no co-resident credit ledger → the commerce billing HTTP deposit, with
// its operator-nonce idempotency key so a retried grant dedupes at commerce.
beforeC, _ := s.State.Commerce.Credits(ctx, org)
idem := grantIdempotencyKey(c, org, currency, source, amountCents)
+8 -8
View File
@@ -3,11 +3,11 @@ package admin
// The PLATFORM CONTROL PLANE board (/v1/admin/flags) — every runtime LAUNCH / RELEASE
// switch (waitlist, public signup, subsystem activation, gateway limits, network ids)
// with its LIVE value, evaluated through the embedded native flag engine
// (clients/featureflags → native/flags, SQLite-per-project definitions + Rust FFI
// (clients/flags → native/flags, SQLite-per-project definitions + Rust FFI
// evaluation). SuperAdmin only (mounted behind core.Guard, like every /v1/admin/*).
//
// ONE flag engine, TWO verbs. GET reads the board; PUT writes a switch's definition
// through featureflags.SetPlatformSwitch — the ONE write path, audited in the store's
// through flags.SetPlatformSwitch — the ONE write path, audited in the store's
// activity log. A flip is hot: this pod applies immediately, peers converge within one
// evaluation TTL (default 15s), no redeploy. Org/project product flags are managed on
// /v1/flags (org-scoped); this surface is the platform's own switchboard.
@@ -18,13 +18,13 @@ import (
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/featureflags"
"github.com/hanzoai/cloud/clients/flags"
"github.com/zap-proto/zip"
)
// flags answers GET /v1/admin/flags — the platform control-plane read board.
func flags(s *cloud.Service[core.State], c *zip.Ctx) error {
return core.OK(c, featureflags.Board())
// flagsBoard answers GET /v1/admin/flags — the platform control-plane read board.
func flagsBoard(s *cloud.Service[core.State], c *zip.Ctx) error {
return core.OK(c, flags.Board())
}
// setFlag answers PUT /v1/admin/flags/:key — store/overwrite one platform switch's
@@ -42,8 +42,8 @@ func setFlag(s *cloud.Service[core.State], c *zip.Ctx) error {
if len(body) == 0 || !json.Valid(body) {
return zip.ErrBadRequest("body must be the flag definition JSON")
}
if err := featureflags.SetPlatformSwitch(key, json.RawMessage(body), c.UserEmail()); err != nil {
if err := flags.SetPlatformSwitch(key, json.RawMessage(body), c.UserEmail()); err != nil {
return zip.ErrBadRequest(err.Error())
}
return core.OK(c, featureflags.Board())
return core.OK(c, flags.Board())
}
+73
View File
@@ -0,0 +1,73 @@
package admin
// The /v1/admin/services board — the launch-control LENS on the ONE flag engine, twin
// of /v1/admin/flags. Every hosted service (studio/chat/console/app/api/team + runtime
// onboards) with its LIVE waitlist mode — the switch waitlist.<svc> evaluated through
// clients/flags. This is the "remove the waitlist one service at a time" toggle.
// SuperAdmin only (core.Guard), like every platform /v1/admin/*.
//
// Formerly clients/featuregate owned its OWN SQLite mode store + this control plane;
// both folded onto the flag engine so the platform has ONE decision plane. featuregate
// now owns only the native Enforce middleware — a consumer of flags.WaitlistModeForHost.
// Per-user approval (the second, orthogonal axis) stays IAM's, reached via the existing
// admin IAM proxy — not re-served here.
import (
"errors"
"net/http"
"strings"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/flags"
"github.com/zap-proto/zip"
)
// services answers GET /v1/admin/services — the launch board (every service + live mode).
func services(s *cloud.Service[core.State], c *zip.Ctx) error {
rows, err := flags.ListWaitlistServices(c.Context())
if err != nil {
return zip.Errorf(http.StatusInternalServerError, "list services: %v", err)
}
return core.OK(c, map[string]any{"services": rows})
}
// upsertService answers POST /v1/admin/services — onboard or edit a hosted service so a
// new host is governed WITHOUT a redeploy. A re-register PRESERVES the live switch.
func upsertService(s *cloud.Service[core.State], c *zip.Ctx) error {
var in flags.ServiceInput
if err := c.Bind(&in); err != nil {
return err
}
if strings.TrimSpace(in.Service) == "" {
return zip.ErrBadRequest("service slug is required")
}
view, err := flags.UpsertWaitlistService(c.Context(), in, c.UserEmail())
if err != nil {
return zip.ErrBadRequest(err.Error())
}
return core.OK(c, map[string]any{"service": view})
}
// setServiceMode answers POST /v1/admin/services/:service/mode — flip one service's
// waitlist switch {waitlistMode:bool}. The launch lever; hot, no redeploy.
func setServiceMode(s *cloud.Service[core.State], c *zip.Ctx) error {
service := strings.TrimSpace(c.Param("service"))
if service == "" {
return zip.ErrBadRequest("service is required")
}
var body struct {
WaitlistMode bool `json:"waitlistMode"`
}
if err := c.Bind(&body); err != nil {
return err
}
view, err := flags.SetWaitlistMode(c.Context(), service, body.WaitlistMode, c.UserEmail())
if err != nil {
if errors.Is(err, flags.ErrServiceNotFound) {
return zip.ErrNotFound("service not found: " + service)
}
return zip.Errorf(http.StatusInternalServerError, "set mode: %v", err)
}
return core.OK(c, map[string]any{"service": view})
}
+141
View File
@@ -0,0 +1,141 @@
// Package agent mounts the hanzoai/agent orchestrator into cloud: POST /v1/agent
// (+ /v1/agent/presets, /v1/agent/conversations). The orchestrator logic and its
// per-org conversation history live in github.com/hanzoai/agent, which imports
// NEITHER cloud NOR ai. Cloud is the composition root: it injects the two seams —
// - Completer: the ai subsystem's /v1/chat/completions, replayed in-process (the
// one path that returns tool_calls AND carries per-org reserve/settle billing);
// - ToolPlane: the unified tool registry (tools.Default()), so /v1/agent's
// server-executed tools are the org's activated MCP/registry tools.
// /v1/agent is a DISTINCT path (not /v1/chat, which ai owns as completions), so a
// specific route wins over ai's /v1/* glob — no collision.
package agent
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
hz "github.com/hanzoai/agent"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/tools"
openai "github.com/sashabaranov/go-openai"
fiber "github.com/zap-proto/fiber/v3"
"github.com/zap-proto/zip"
)
// maxCompletionResponse bounds the in-process completion body read so a hostile or
// broken upstream cannot balloon memory.
const maxCompletionResponse = 8 << 20
// Mount wires POST /v1/agent (+ reads) into cloud, injecting the ai completion and
// the tool plane. The caller identity comes from cloud's validated principal.
func Mount(app *zip.App, deps cloud.Deps) error {
if app == nil {
return fmt.Errorf("agent.Mount: nil zip.App")
}
_, err := hz.Mount(app, hz.Deps{
Logger: deps.Logger,
DataDir: deps.DataDir,
Brand: deps.Brand,
Model: deps.AIDefaultModel,
Principal: func(c *zip.Ctx) (hz.Principal, bool) {
p, ok := tools.PrincipalFrom(c)
if !ok {
return hz.Principal{}, false
}
return hz.Principal{Org: p.Org, Project: p.Project, User: p.User, Cred: credential(c)}, true
},
}, aiCompleter{app: app}, toolPlane{})
return err
}
// ── Completer: replay /v1/chat/completions in-process ─────────────────────────────
type aiCompleter struct{ app *zip.App }
// Complete replays the request against the SAME app at /v1/chat/completions, so it
// flows the whole middleware chain (per-org reserve/settle billing) and returns
// tool_calls. Non-streaming. Mirrors the tool plane's in-process dispatch contract:
// the caller's OWN credential headers are replayed; no minted authority header.
func (a aiCompleter) Complete(ctx context.Context, cred map[string]string, req openai.ChatCompletionRequest) (openai.ChatCompletionResponse, error) {
req.Stream = false
b, err := json.Marshal(req)
if err != nil {
return openai.ChatCompletionResponse{}, err
}
hreq := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", bytes.NewReader(b)).WithContext(ctx)
hreq.Header.Set("Content-Type", "application/json")
for k, v := range cred {
hreq.Header.Set(k, v)
}
resp, err := a.app.Fiber().Test(hreq, fiber.TestConfig{Timeout: 0})
if err != nil {
return openai.ChatCompletionResponse{}, err
}
defer func() { _ = resp.Body.Close() }()
raw, _ := io.ReadAll(io.LimitReader(resp.Body, maxCompletionResponse))
if resp.StatusCode/100 != 2 {
// Carry the completion's OWN status + body so the round can pass a
// caller-facing refusal (402 insufficient_balance, 429, 403) straight
// through instead of masking it as a gateway 502. hz.UpstreamError is the
// agent's typed seam for exactly this.
return openai.ChatCompletionResponse{}, &hz.UpstreamError{Status: resp.StatusCode, Body: raw}
}
var out openai.ChatCompletionResponse
if err := json.Unmarshal(raw, &out); err != nil {
return openai.ChatCompletionResponse{}, fmt.Errorf("decode completion: %w", err)
}
return out, nil
}
// ── ToolPlane: adapter over the unified registry ──────────────────────────────────
type toolPlane struct{}
func (toolPlane) List(ctx context.Context, scope hz.Scope) []hz.Tool {
ts := tools.Default().List(ctx, tools.Scope{Org: scope.Org, Project: scope.Project})
out := make([]hz.Tool, 0, len(ts))
for _, t := range ts {
out = append(out, hz.Tool{
Name: t.Name,
Description: t.Description,
Schema: t.Schema,
Activated: t.Activated,
Dispatchable: t.Dispatchable,
})
}
return out
}
func (toolPlane) Exists(ctx context.Context, scope hz.Scope, name string) bool {
return tools.Default().Exists(ctx, tools.Scope{Org: scope.Org, Project: scope.Project}, name)
}
// Dispatch resolves the caller the ONE canonical way — tools.PrincipalFrom(c) — so
// the tool runs under the SAME validated identity + credential as a direct call.
// No reconstruction: the credential is only ever read from the live request.
func (toolPlane) Dispatch(c *zip.Ctx, name string, args map[string]any) (any, error) {
p, ok := tools.PrincipalFrom(c)
if !ok {
return nil, zip.ErrForbidden("a validated principal is required")
}
return tools.Default().Dispatch(c.Context(), p, name, args)
}
// ── helpers ───────────────────────────────────────────────────────────────────────
// credential extracts the caller's replayable credential headers (the same set the
// tool plane replays) so the in-process completion runs as the caller.
func credential(c *zip.Ctx) map[string]string {
cred := map[string]string{}
for _, h := range []string{"Authorization", "X-Authorization", "Cookie", "Accept-Language", "X-Forwarded-For"} {
if v := c.Header(h); v != "" {
cred[h] = v
}
}
return cred
}
+1 -1
View File
@@ -676,7 +676,7 @@ func recordRunEnd(s *cloud.Service[state], ctx context.Context, in RunEndInput)
// meterUnit records one metered unit for an HTTP caller's org. Nil/disabled meter → no-op.
func meterUnit(s *cloud.Service[state], org string, c *zip.Ctx) {
s.Bill.Meter(principal.Payer(c), principal.Project(c), meterKind, cloud.ResourceFeeCents(feeEnvPrefix, meterKind), c.RequestID(), cloud.ClientIP(c))
s.Bill.Meter(principal.HomeOrg(c), principal.Project(c), meterKind, cloud.ResourceFeeCents(feeEnvPrefix, meterKind), c.RequestID(), cloud.ClientIP(c))
}
// meterRun records one metered unit for a flow run from the durable path (no HTTP
+17 -16
View File
@@ -4,7 +4,7 @@ import (
"context"
"strings"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/account"
"github.com/hanzoai/cloud/clients/finance"
"github.com/zap-proto/zip"
)
@@ -30,12 +30,14 @@ import (
// self-dispatch, and the number shown is the number that admits or refuses a request.
// The commerce S2S read stays as the split-deploy fallback, unchanged.
// balanceSubject resolves the wallet key for the caller by CALLING ai/object.BillingSubject
// — the same function the ai prepaid gate (routers/filter_balance.go resolveBillingKey) and
// the usage debit resolve. It is deliberately not re-implemented here: cloud and ai each
// keeping their own copy of this rule is what let them drift apart (cloud's console view
// scoping to the org while the gate scoped to "org/user"), so the view showed a funded org
// while the gate refused the member. One function, one rule, one wallet.
// subjectFor resolves the billing subject for the caller by CALLING the ONE rule,
// ai/object.Payer — the same function the ai prepaid gate (routers/filter_balance.go
// resolveBillingKey) and the usage debit resolve. It is the SHARED resolver for every
// commerce-projected read in this package (the balance read AND the finance reads), so
// there is exactly one copy of the rule. cloud and ai each keeping their own copy is
// what let them drift apart (cloud's console view scoping to the org while the gate
// scoped to "org/user"), so the view showed a funded org while the gate refused the
// member. One function, one rule, one wallet.
//
// org is the VALIDATED principal org. The name half uses the SAME precedence as
// clients/account.resolveCaller: X-User-Name (the IAM username the identity boundary mints
@@ -43,22 +45,21 @@ import (
// X-User-Id. Both are authorityHeaders — stripped on ingress and re-injected only from
// verified claims — so neither is a client value.
//
// The X-User-Id fallback goes through ai's own BillingSubjectFromUserKey because that
// header's shape is path-dependent: the gateway historically minted X-User-Id == the
// username, while the in-binary direct-Bearer path mints the UUID subject, and callers
// hold it as an "<owner>/<name>" key. FromUserKey is the function ai already uses to fold
// that key form back to a subject, so the two agree by construction instead of by a
// re-implemented split here.
// The X-User-Id fallback goes through the shared account.PayerOf because that header's shape is
// path-dependent: the gateway historically minted X-User-Id == the username, while the
// in-binary direct-Bearer path mints the UUID subject, and callers hold it as an
// "<owner>/<name>" key. PayerOf is the parse ai already uses to fold that key form back
// to the payer, so the two agree by construction instead of by a re-implemented split.
//
// KNOWN RESIDUAL: a validated principal carrying NEITHER X-User-Name NOR an "<owner>/<name>"
// X-User-Id (i.e. a bare username id) folds to the org pool. That requires a JWT with no
// `name` and no `preferred_username`, since the boundary mints X-User-Name from either;
// production tokens carry one. Called out for review rather than papered over.
func balanceSubject(c *zip.Ctx, org string) string {
func subjectFor(c *zip.Ctx, org string) string {
if name := strings.TrimSpace(c.Header("X-User-Name")); name != "" {
return aiobject.BillingSubject(org, name)
return account.Payer(account.Credential{Owner: org, Name: name}).Subject()
}
return aiobject.BillingSubjectFromUserKey(org, strings.TrimSpace(c.User()))
return account.PayerOf(org, strings.TrimSpace(c.User())).Subject()
}
// availableCents returns the caller's spendable prepaid balance from the co-resident
+25 -17
View File
@@ -4,11 +4,11 @@ import (
"context"
"encoding/json"
"fmt"
"github.com/hanzoai/account"
"net/http"
"net/http/httptest"
"testing"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud/clients/finance"
"github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/cloud/types"
@@ -19,11 +19,19 @@ import (
// shows, and can be made to fail so a test can prove an unreadable balance is never
// rendered as zero.
type fakeFinance struct {
wallets map[string]int64 // "org|subject" -> cents
err error
gotOrg string
gotSubj string
calls int
wallets map[string]int64 // "org|subject" -> cents
usageRows []finance.UsageRow
err error
gotOrg string
gotSubj string
calls int
}
// ListUsage satisfies the optional co-resident usage-read capability coResidentUsage
// resolves; returns the seeded rows so a test can prove the usage view answers from the
// ledger instead of the self-dispatching commerce hop.
func (f *fakeFinance) ListUsage(context.Context, string, int) ([]finance.UsageRow, error) {
return f.usageRows, f.err
}
func (f *fakeFinance) Balance(_ context.Context, org, subject, _ string, _ bool) (money.Amount, error) {
@@ -81,21 +89,21 @@ func TestBalance_ReadsFinanceLedgerNotCommerce(t *testing.T) {
// TestBalance_SubjectIsTheGateSubject pins the invariant this incident broke: the wallet
// the console SHOWS must be the wallet the ai prepaid gate READS. Both derive it from the
// one function, ai/object.BillingSubject, so they cannot drift apart again — cloud keeping
// its own copy of this rule is exactly what let the console show a funded org while the
// gate refused the member.
// one function, ai/object.Payer, so they cannot drift apart again — cloud keeping its own
// copy of this rule is exactly what let the console show a funded org while the gate
// refused the member.
func TestBalance_SubjectIsTheGateSubject(t *testing.T) {
// Reproduce the live pod env: both allowlists present-but-empty. Under ai v1.809.2
// PERSONAL_BILLING_ORGS defaults to "hanzo", so "hanzo" is a PERSONAL-billing org and
// the gate reads the per-user wallet.
// The allowlists are gone; set them to values that WOULD have flipped the resolution
// to prove they are inert — nothing reads them, the signup org still bills per-person.
t.Setenv("PERSONAL_BILLING_ORGS", "")
t.Setenv("ORG_BILLING_ORGS", "")
t.Setenv("ORG_BILLING_ORGS", "hanzo")
// The gate's subject for this principal, from ai itself — not a value this test invents.
// This is what routers/filter_balance.go resolveBillingKey computes from the JWT claims.
want := aiobject.BillingSubject("hanzo", "z")
// This is what routers/filter_balance.go resolveBillingKey computes from the JWT claims:
// a person in the signup org bills their OWN account, hanzo/z.
want := account.Payer(account.Credential{Owner: "hanzo", Name: "z"}).Subject()
if want != "hanzo/z" {
t.Fatalf("precondition: ai v1.809.2 resolves the live env to %q, want hanzo/z", want)
t.Fatalf("precondition: ai resolves a signup-org person to %q, want hanzo/z", want)
}
// Every identity shape a validated principal can arrive in must land on that ONE wallet.
@@ -104,7 +112,7 @@ func TestBalance_SubjectIsTheGateSubject(t *testing.T) {
{"gateway mints X-User-Name", "z", "8f14e45f-ea1b-4c2a-9f3d-000000000001"},
// In-binary direct-Bearer: X-User-Id is the UUID subject, X-User-Name carries the name.
{"in-binary direct bearer", "z", "hanzo/z"},
// No X-User-Name: the "<owner>/<name>" key form folds back via FromUserKey.
// No X-User-Name: the "<owner>/<name>" key form folds back via PayerOf.
{"owner/name id, no X-User-Name", "", "hanzo/z"},
} {
t.Run(tc.name, func(t *testing.T) {
+19 -1
View File
@@ -246,6 +246,24 @@ func usage(s *cloud.Service[state], c *zip.Ctx) error {
if !ok {
return zip.ErrUnauthorized("sign in to view billing")
}
// Co-resident, read the usage ledger DIRECTLY from cloud's own finance ledger
// (usage_coresident.go explains why this is NOT a commerce proxy: proxying
// "/v1/billing/usage" re-enters THIS handler — commerce's own /v1/billing/usage
// route is behind //go:build cloud and never compiled here, so the only
// registration of that path is this handler — and the in-proc S2S hop carries no
// validated principal, so usage() self-answered "sign in to view billing"; that
// self-dispatch is the 500 a valid caller saw). This is the exact move balance()
// already makes. Off the co-resident path the commerce S2S proxy is unchanged.
if body, coResident, err := coResidentUsage(c.Context(), org, strings.TrimSpace(c.Query("product")), strings.TrimSpace(c.Query("groupBy"))); err != nil {
s.Log.Warn("finance usage read failed", "org", org, "err", err)
return zip.Errorf(http.StatusBadGateway, "billing upstream unreachable")
} else if coResident {
c.SetHeader("Content-Type", "application/json")
c.SetHeader("Cache-Control", "no-store")
return c.Bytes(http.StatusOK, body)
}
if !s.State.commerce.configured() {
return zip.Errorf(http.StatusNotImplemented, "billing is not configured")
}
@@ -286,7 +304,7 @@ func balance(s *cloud.Service[state], c *zip.Ctx) error {
// true "not signed in" (401), matching usage/gpuCharge.
return zip.ErrUnauthorized("sign in to view billing")
}
cents, coResident, err := availableCents(c.Context(), org, balanceSubject(c, org))
cents, coResident, err := availableCents(c.Context(), org, subjectFor(c, org))
if err != nil {
// A balance that cannot be READ is unknown — surface it as an upstream failure.
// It must never render as a zero balance: unknown is not "broke".
+8 -8
View File
@@ -202,7 +202,7 @@ func financeBalance(s *cloud.Service[state], c *zip.Ctx) error {
// The ONE balance read (balance.go) — the same wallet /v1/billing/balance answers, so
// the two surfaces can never disagree. Co-resident this is the finance ledger; only a
// split deploy falls through to the commerce S2S read below.
if cents, coResident, err := availableCents(c.Context(), org, balanceSubject(c, org)); err != nil {
if cents, coResident, err := availableCents(c.Context(), org, subjectFor(c, org)); err != nil {
s.Log.Warn("finance balance read failed", "org", org, "err", err)
return zip.Errorf(http.StatusBadGateway, "billing upstream unreachable")
} else if coResident {
@@ -354,7 +354,7 @@ func financePaymentMethods(s *cloud.Service[state], c *zip.Ctx) error {
return zip.Errorf(http.StatusNotImplemented, "billing is not configured")
}
// Portal read filters on customerId; the subject is pinned to the caller's own org.
body, status, err := s.State.commerce.get(c.Context(), "/v1/billing/portal/payment-methods", org, financeSubject(org, nil))
body, status, err := s.State.commerce.get(c.Context(), "/v1/billing/portal/payment-methods", org, financeSubject(subjectFor(c, org), nil))
if err != nil {
s.Log.Warn("commerce payment-methods read failed", "org", org, "err", err)
return zip.Errorf(http.StatusBadGateway, "billing upstream unreachable")
@@ -436,12 +436,12 @@ func financeCaller(s *cloud.Service[state], c *zip.Ctx) (string, bool) {
return principal.Org(c)
}
// financeSubject builds the commerce query with every billing-subject key PINNED to org
// (the client can never widen scope), plus any extra passthrough params.
func financeSubject(org string, extra url.Values) url.Values {
// financeSubject builds the commerce query with every billing-subject key PINNED to
// subject (the client can never widen scope), plus any extra passthrough params.
func financeSubject(subject string, extra url.Values) url.Values {
q := url.Values{}
for _, k := range billingSubjectKeys {
q.Set(k, org)
q.Set(k, subject)
}
for k, vs := range extra {
for _, v := range vs {
@@ -454,7 +454,7 @@ func financeSubject(org string, extra url.Values) url.Values {
// financeGet does one org-scoped commerce GET and decodes the 2xx body into out. A
// non-2xx or unreachable upstream is surfaced honestly (never masked as empty data).
func financeGet(s *cloud.Service[state], c *zip.Ctx, path, org string, extra url.Values, out any) error {
body, status, err := s.State.commerce.get(c.Context(), path, org, financeSubject(org, extra))
body, status, err := s.State.commerce.get(c.Context(), path, org, financeSubject(subjectFor(c, org), extra))
if err != nil {
s.Log.Warn("commerce finance read failed", "org", org, "path", path, "err", err)
return zip.Errorf(http.StatusBadGateway, "billing upstream unreachable")
@@ -472,7 +472,7 @@ func financeGet(s *cloud.Service[state], c *zip.Ctx, path, org string, extra url
// credits/usage/ledger projections share). Tolerates the wrapped {transactions:[…]}
// shape and a bare array.
func financeTxns(s *cloud.Service[state], c *zip.Ctx, org string) ([]commerceTxn, error) {
body, status, err := s.State.commerce.get(c.Context(), "/v1/billing/transactions", org, financeSubject(org, url.Values{"limit": {"2000"}}))
body, status, err := s.State.commerce.get(c.Context(), "/v1/billing/transactions", org, financeSubject(subjectFor(c, org), url.Values{"limit": {"2000"}}))
if err != nil {
s.Log.Warn("commerce transactions read failed", "org", org, "err", err)
return nil, zip.Errorf(http.StatusBadGateway, "billing upstream unreachable")
+84
View File
@@ -0,0 +1,84 @@
package billing
// usage_coresident.go — the co-resident source for GET /v1/billing/usage.
//
// WHY IT EXISTS. balance() already reads cloud's OWN finance ledger directly rather
// than proxying "/v1/billing/balance" through commerceinproc, because co-resident the
// ONLY registration of that path is balance() itself (commerce's api.Route() is behind
// //go:build cloud and never compiled here), so the S2S proxy re-dispatches BY PATH
// straight back into the same handler, which self-answers "sign in to view billing"
// (the in-proc hop carries no validated principal). usage() had the SAME defect on
// "/v1/billing/usage" — a valid caller's usage read re-entered usage() and failed. This
// file gives usage() the co-resident answer balance() already has: the usage ledger read
// straight from finance (the wallet→revenue debits RecordUsage wrote), off the
// self-dispatching hop. Split deploy (no co-resident finance) falls back to the commerce
// S2S read, unchanged.
import (
"context"
"encoding/json"
"time"
"github.com/hanzoai/cloud/clients/finance"
)
// coResidentUsage builds the customer usage envelope from cloud's OWN finance ledger
// when the money plane is co-resident (finance.Current() published AND exposes the
// usage read). It returns (body, true, nil) with the commerce-shaped
// {user,count,usage:[...]} envelope — enriched + optionally ?product=filtered /
// ?groupBy=product-reduced exactly like the proxied path — or (nil, false, nil) when
// finance is not co-resident (split deploy), so the caller falls back to the commerce
// S2S read. This is what keeps /v1/billing/usage off the self-dispatching commerceinproc
// hop; a real read failure surfaces as a non-nil error (never a masked-empty ledger).
func coResidentUsage(ctx context.Context, org, product, groupBy string) ([]byte, bool, error) {
fin := finance.Current()
if fin == nil {
return nil, false, nil // split deploy → commerce S2S read
}
// The usage read is an OPTIONAL capability (the base FinanceClient is
// Balance+Deposit+RecordUsage); a finance impl without it falls back to the proxy.
lister, ok := fin.(interface {
ListUsage(context.Context, string, int) ([]finance.UsageRow, error)
})
if !ok {
return nil, false, nil
}
rows, err := lister.ListUsage(ctx, org, 2000)
if err != nil {
return nil, false, err
}
env := usageEnvelope(org, rows)
if out, ok := enrichUsageLedger(env, product, groupBy); ok {
return out, true, nil
}
return env, true, nil
}
// usageEnvelope renders finance usage rows as commerce's GetUsage envelope
// ({user,count,usage:[{transactionId,amount,metadata,createdAt}]}) — the exact shape the
// console's normalizeUsageRecords + this package's enrichUsageLedger already parse.
// amount is USD cents; metadata carries the metered unit (model) the debit recorded, so
// enrichUsageLedger can still attribute a product where the unit implies one.
func usageEnvelope(org string, rows []finance.UsageRow) []byte {
type usageRow struct {
TransactionID string `json:"transactionId"`
Amount int64 `json:"amount"`
Metadata map[string]any `json:"metadata"`
CreatedAt string `json:"createdAt"`
}
out := make([]usageRow, 0, len(rows))
for _, r := range rows {
md := map[string]any{}
if r.Model != "" {
md["model"] = r.Model
}
out = append(out, usageRow{
TransactionID: r.ID,
Amount: r.Cents,
Metadata: md,
CreatedAt: time.Unix(r.CreatedAt, 0).UTC().Format(time.RFC3339),
})
}
body, _ := json.Marshal(map[string]any{"user": org, "count": len(out), "usage": out})
return body
}
+89
View File
@@ -0,0 +1,89 @@
package billing
import (
"context"
"encoding/json"
"testing"
"github.com/hanzoai/cloud/clients/finance"
)
// TestCoResidentUsage proves usage() answers from the finance ledger (never the
// self-dispatching commerce hop) and shapes the commerce GetUsage envelope the
// console parses. fakeFinance + publishFinance live in balance_test.go.
func TestCoResidentUsage(t *testing.T) {
publishFinance(t, &fakeFinance{usageRows: []finance.UsageRow{
{ID: "u1", Cents: 150, Model: "gpt-x", CreatedAt: 1_700_000_000},
{ID: "u2", Cents: 75, Model: "embed-y", CreatedAt: 1_700_000_100},
}})
body, coResident, err := coResidentUsage(context.Background(), "acme", "", "")
if err != nil {
t.Fatalf("coResidentUsage: %v", err)
}
if !coResident {
t.Fatal("want coResident=true when finance is published")
}
var env struct {
User string `json:"user"`
Count int `json:"count"`
Usage []struct {
TransactionID string `json:"transactionId"`
Amount int64 `json:"amount"`
Metadata map[string]any `json:"metadata"`
CreatedAt string `json:"createdAt"`
} `json:"usage"`
}
if err := json.Unmarshal(body, &env); err != nil {
t.Fatalf("envelope not valid JSON: %v\n%s", err, body)
}
if env.User != "acme" || env.Count != 2 || len(env.Usage) != 2 {
t.Fatalf("bad envelope: user=%q count=%d rows=%d", env.User, env.Count, len(env.Usage))
}
if env.Usage[0].TransactionID != "u1" || env.Usage[0].Amount != 150 {
t.Errorf("row0 = %+v", env.Usage[0])
}
if env.Usage[0].Metadata["model"] != "gpt-x" {
t.Errorf("row0 metadata missing model: %+v", env.Usage[0].Metadata)
}
if env.Usage[0].CreatedAt == "" {
t.Error("row0 createdAt should be RFC3339, got empty")
}
}
// TestCoResidentUsageSplitDeploy proves that with no co-resident finance, usage()
// falls through to the commerce S2S proxy (coResident=false), unchanged.
func TestCoResidentUsageSplitDeploy(t *testing.T) {
finance.Publish(nil)
body, coResident, err := coResidentUsage(context.Background(), "acme", "", "")
if err != nil {
t.Fatalf("coResidentUsage: %v", err)
}
if coResident || body != nil {
t.Fatalf("want fall-through (coResident=false, nil body); got coResident=%v body=%s", coResident, body)
}
}
// TestCoResidentUsageGroupBy proves the ?groupBy=product reduction still runs on the
// co-resident path (a token-metered row attributes to product "inference").
func TestCoResidentUsageGroupBy(t *testing.T) {
publishFinance(t, &fakeFinance{usageRows: []finance.UsageRow{{ID: "u1", Cents: 150, Model: "gpt-x", CreatedAt: 1_700_000_000}}})
body, ok, err := coResidentUsage(context.Background(), "acme", "", "product")
if err != nil || !ok {
t.Fatalf("coResidentUsage groupBy: ok=%v err=%v", ok, err)
}
var grouped struct {
GroupBy string `json:"groupBy"`
Groups []struct {
Product string `json:"product"`
Requests int `json:"requests"`
AmountCents int64 `json:"amountCents"`
} `json:"groups"`
}
if err := json.Unmarshal(body, &grouped); err != nil {
t.Fatalf("grouped not valid JSON: %v\n%s", err, body)
}
if grouped.GroupBy != "product" || len(grouped.Groups) != 1 {
t.Fatalf("bad grouped envelope: %s", body)
}
}
+4 -4
View File
@@ -198,7 +198,7 @@ func (s *service) handleSearch(c *zip.Ctx) error {
if err != nil {
return err
}
eng, err := s.engineFor(org, principal.Payer(c), principal.Project(c))
eng, err := s.engineFor(org, principal.HomeOrg(c), principal.Project(c))
if err != nil {
return zip.ErrInternal("open index")
}
@@ -238,7 +238,7 @@ func (s *service) handleContext(c *zip.Ctx) error {
if err != nil {
return err
}
eng, err := s.engineFor(org, principal.Payer(c), principal.Project(c))
eng, err := s.engineFor(org, principal.HomeOrg(c), principal.Project(c))
if err != nil {
return zip.ErrInternal("open index")
}
@@ -343,7 +343,7 @@ func (s *service) handleAsk(c *zip.Ctx) error {
if err != nil {
return err
}
eng, err := s.engineFor(org, principal.Payer(c), principal.Project(c))
eng, err := s.engineFor(org, principal.HomeOrg(c), principal.Project(c))
if err != nil {
return zip.ErrInternal("open index")
}
@@ -393,7 +393,7 @@ func (s *service) handleIndex(c *zip.Ctx) error {
if err != nil {
return zip.ErrInternal("open index")
}
res, err := s.indexRepo(c.Context(), org, principal.Payer(c), principal.Project(c), store, repo, body.Files, body.Prune)
res, err := s.indexRepo(c.Context(), org, principal.HomeOrg(c), principal.Project(c), store, repo, body.Files, body.Prune)
if err != nil {
s.log.Warn("code index failed", "org", org, "repo", repo, "err", err)
return zip.ErrInternal("index failed")
-268
View File
@@ -1,268 +0,0 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package featuregate is the launch-control plane for Hanzo's hosted services:
// the ONE source of truth for whether each service (studio.hanzo.ai, hanzo.chat,
// console.hanzo.ai, hanzo.app, api.hanzo.ai, hanzo.team, …) is in WAITLIST MODE.
// It owns a small global SQLite registry {service, hosts, waitlistMode} and serves
// the control-plane admin.hanzo.ai calls to list services and flip a service's
// mode — the "remove the waitlist one service at a time" toggle.
//
// TWO knobs, ONE rule (decomplected):
//
// - PER-SERVICE waitlistMode on|off — this registry (a row per hosted service).
// OFF = open to any signed-up user. This is the admin's launch lever.
// - PER-USER approvalStatus pending|approved — owned by IAM (iam#104:
// get-pending-users / approve-user / reject-user). REUSED, not rebuilt.
//
// THE RULE, applied at ONE native enforcement point (Enforce, middleware.go) and
// mirrored by the interim @file waitlist-guard (which reads the SAME registry over
// GET /v1/featuregate/mode so an admin toggle governs it WITHOUT an ingress edit):
//
// if waitlistMode[host] AND NOT user.approved → bounce to the waitlist
// if approved OR mode=off → allow
// unauthenticated → login first
//
// Surface (all /v1/, never /api/):
//
// GET /v1/admin/services (global-admin) list every service + mode — the board
// POST /v1/admin/services/:service/mode (global-admin) flip waitlistMode {waitlistMode:bool}
// GET /v1/featuregate/mode?host=<h> (public read) the guard's runtime mode lookup
//
// The Pending-Users QUEUE (approve/reject/pending) is NOT re-served here — it is
// IAM's iam#104, reached by admin.hanzo.ai through its existing /admin/iam gated
// proxy. One approval API, one registry, one enforcement rule — no per-app copy.
//
// serve.go auto-registers GET /v1/featuregate/health.
package featuregate
import (
"context"
"fmt"
"net/http"
"os"
"path/filepath"
"strings"
"sync"
"time"
"github.com/hanzoai/cloud"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
type svc struct {
store *Store
log luxlog.Logger
}
// moduleStoreRef / moduleMu back moduleStore() — the lazy handle the native
// Enforce middleware (which serve.go wires BEFORE MountAll) reads per request.
var (
moduleMu sync.RWMutex
moduleStoreRef *Store
mounted *svc
)
// Mount wires the feature-gate surface onto app per HIP-0106.
func Mount(app *zip.App, deps cloud.Deps) error {
if app == nil {
return fmt.Errorf("featuregate.Mount: nil zip.App")
}
log := deps.Logger
if log == nil {
return fmt.Errorf("featuregate.Mount: nil deps.Logger")
}
log = log.New("subsystem", "featuregate")
if deps.DataDir == "" {
return fmt.Errorf("featuregate.Mount: empty DataDir")
}
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
return fmt.Errorf("featuregate.Mount: data dir: %w", err)
}
store, err := openStore(filepath.Join(deps.DataDir, "featuregate.db"))
if err != nil {
return fmt.Errorf("featuregate.Mount: open store: %w", err)
}
// Seed the live hosted services idempotently (INSERT OR IGNORE — never
// clobbers a live admin toggle). Brand-scoped so a Lux/Zoo deployment seeds
// its OWN hosts, not hanzo.ai's.
created, serr := store.Seed(context.Background(), seedFor(deps.Brand), time.Now().Unix())
if serr != nil {
_ = store.Close()
return fmt.Errorf("featuregate.Mount: seed: %w", serr)
}
s := &svc{store: store, log: log}
mounted = s
setModuleStore(store)
// Control-plane — admin board + toggle + onboard (global-admin, fail-closed).
app.Get("/v1/admin/services", s.guard(s.listServices))
app.Post("/v1/admin/services", s.guard(s.upsertService))
app.Post("/v1/admin/services/:service/mode", s.guard(s.setMode))
// Runtime mode read for the @file waitlist-guard (cheap, cacheable, no auth —
// it returns only a boolean for the queried host, never an enumeration).
app.Get("/v1/featuregate/mode", s.modeForHost)
log.Info("featuregate mounted", "brand", deps.Brand, "seeded", created)
return nil
}
// Mount order is fixed by the apps.Wire() composition root (right after admin,
// before the AI /v1/* catch-all). All routes are specific (/v1/admin/services*,
// /v1/featuregate/*), so they bind ahead of the catch-all regardless; the static
// /v1/featuregate/mode and the /v1/admin/services list bind before the
// /:service/mode param route. This package no longer self-registers via an init().
// guard wraps a handler with the global-admin gate — the SAME predicate the rest
// of cloud uses (c.IsAdmin(), true only for a JWT-validated owner==AdminOrg after
// SanitizeIdentity). Fail-closed: a non-admin never reaches the handler.
func (s *svc) guard(h func(*zip.Ctx) error) zip.Handler {
return func(c *zip.Ctx) error {
if !c.IsAdmin() {
return zip.ErrForbidden("global admin required")
}
return h(c)
}
}
// ── control-plane handlers ─────────────────────────────────────────────────────
// listServices answers GET /v1/admin/services — the launch dashboard's board.
// Global-admin only. Envelope = { status:"ok", data:{ services:[...] } } (the
// console admin proxy unwraps { status, msg, data }).
func (s *svc) listServices(c *zip.Ctx) error {
rows, err := s.store.List(c.Context())
if err != nil {
return zip.Errorf(http.StatusInternalServerError, "list services: %v", err)
}
return adminOK(c, map[string]any{"services": rows})
}
// upsertRequest is the POST /v1/admin/services body — onboard or edit a service.
type upsertRequest struct {
Service string `json:"service"`
DisplayName string `json:"displayName"`
Description string `json:"description"`
Hosts []string `json:"hosts"`
WaitlistMode bool `json:"waitlistMode"`
}
// upsertService registers or edits a hosted service so a new host is governed
// WITHOUT a redeploy. Global-admin only. A re-register preserves the live mode of
// an existing service (see Store.Upsert) — never silently re-gating an opened one.
func (s *svc) upsertService(c *zip.Ctx) error {
var body upsertRequest
if err := c.Bind(&body); err != nil {
return err
}
if strings.TrimSpace(body.Service) == "" {
return zip.ErrBadRequest("service slug is required")
}
updated, err := s.store.Upsert(c.Context(), Service{
Service: body.Service,
DisplayName: body.DisplayName,
Description: body.Description,
Hosts: body.Hosts,
WaitlistMode: body.WaitlistMode,
}, strings.TrimSpace(c.User()), time.Now().Unix())
if err != nil {
return zip.Errorf(http.StatusInternalServerError, "upsert service: %v", err)
}
s.log.Info("service registered", "service", updated.Service, "hosts", updated.Hosts, "by", c.User())
return adminOK(c, map[string]any{"service": updated})
}
// modeRequest is the POST /v1/admin/services/:service/mode body.
type modeRequest struct {
WaitlistMode bool `json:"waitlistMode"`
}
// setMode flips one service's waitlist mode. Global-admin only. The toggle takes
// effect immediately (the native middleware reads the store per request; the guard
// re-reads within its short TTL) — no redeploy, no ingress edit. UpdatedBy records
// the admin who flipped it (the toggle's own audit).
func (s *svc) setMode(c *zip.Ctx) error {
service := strings.TrimSpace(c.Param("service"))
if service == "" {
return zip.ErrBadRequest("service is required")
}
var body modeRequest
if err := c.Bind(&body); err != nil {
return err
}
by := strings.TrimSpace(c.User())
updated, err := s.store.SetMode(c.Context(), service, body.WaitlistMode, by, time.Now().Unix())
if err != nil {
if err == errNotFound {
return zip.ErrNotFound("service not found: " + service)
}
return zip.Errorf(http.StatusInternalServerError, "set mode: %v", err)
}
s.log.Info("waitlist mode toggled", "service", updated.Service, "waitlistMode", updated.WaitlistMode, "by", by)
return adminOK(c, map[string]any{"service": updated})
}
// modeForHost answers GET /v1/featuregate/mode?host=<h> — the runtime lookup the
// @file waitlist-guard caches. It returns ONLY the boolean mode for the ONE queried
// host (no enumeration of the registry), so it is safe to serve without auth to an
// in-cluster caller. An un-governed host returns waitlistMode:false, known:false —
// the guard, attached only to gated hosts, fails SAFE (treats unknown as gated) on
// its own side; the native middleware treats unknown as pass.
func (s *svc) modeForHost(c *zip.Ctx) error {
host := strings.TrimSpace(c.Query("host"))
if host == "" {
host = c.Fiber().Hostname()
}
mode, service, known, err := s.store.ModeForHost(c.Context(), host)
if err != nil {
return zip.Errorf(http.StatusInternalServerError, "mode for host: %v", err)
}
return c.JSON(http.StatusOK, map[string]any{
"host": NormalizeHost(host),
"service": service,
"waitlistMode": mode,
"known": known,
})
}
// adminOK writes the { status:"ok", msg, data } envelope the console admin surface
// (originGet/originPost via app/admin/aggregate) unwraps — identical to
// clients/admin's ok() and clients/authors' adminOK.
func adminOK(c *zip.Ctx, data any) error {
return c.JSON(http.StatusOK, map[string]any{"status": "ok", "msg": "", "data": data})
}
// ── module store + shutdown ────────────────────────────────────────────────────
func setModuleStore(st *Store) {
moduleMu.Lock()
defer moduleMu.Unlock()
moduleStoreRef = st
}
// Shutdown closes the featuregate store. Idempotent.
func Shutdown() error {
moduleMu.Lock()
st := moduleStoreRef
moduleStoreRef = nil
moduleMu.Unlock()
mounted = nil
if st == nil {
return nil
}
return st.Close()
}
-162
View File
@@ -1,162 +0,0 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
// Licensed under the Apache License, Version 2.0.
package featuregate
import (
"bytes"
"context"
"encoding/json"
"io"
"net/http/httptest"
"testing"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
func newControlPlane(t *testing.T) (*zip.App, *Store) {
t.Helper()
st := newTestStore(t)
if _, err := st.Seed(context.Background(), []SeedService{
{Service: "chat", DisplayName: "Chat", Hosts: []string{"hanzo.chat"}, WaitlistMode: true},
{Service: "api", DisplayName: "API", Hosts: []string{"api.hanzo.ai"}, WaitlistMode: true},
}, 100); err != nil {
t.Fatalf("seed: %v", err)
}
s := &svc{store: st, log: luxlog.New("test")}
app := zip.New(zip.Config{Logger: luxlog.New("test")})
app.Get("/v1/admin/services", s.guard(s.listServices))
app.Post("/v1/admin/services", s.guard(s.upsertService))
app.Post("/v1/admin/services/:service/mode", s.guard(s.setMode))
app.Get("/v1/featuregate/mode", s.modeForHost)
return app, st
}
func call(t *testing.T, app *zip.App, method, path string, admin bool, body any) (int, []byte) {
t.Helper()
var r io.Reader
if body != nil {
b, _ := json.Marshal(body)
r = bytes.NewReader(b)
}
hr := httptest.NewRequest(method, path, r)
if body != nil {
hr.Header.Set("Content-Type", "application/json")
}
if admin {
hr.Header.Set("X-User-Id", "z")
hr.Header.Set("X-Org-Id", "admin")
hr.Header.Set("X-User-IsAdmin", "true")
}
resp, err := app.Fiber().Test(hr)
if err != nil {
t.Fatalf("Test %s %s: %v", method, path, err)
}
defer func() { _ = resp.Body.Close() }()
b, _ := io.ReadAll(resp.Body)
return resp.StatusCode, b
}
func TestControlPlane_ListRequiresGlobalAdmin(t *testing.T) {
app, _ := newControlPlane(t)
// Non-admin → 403.
if code, _ := call(t, app, "GET", "/v1/admin/services", false, nil); code != 403 {
t.Fatalf("non-admin list = %d, want 403", code)
}
// Admin → 200 with both services.
code, body := call(t, app, "GET", "/v1/admin/services", true, nil)
if code != 200 {
t.Fatalf("admin list = %d, want 200", code)
}
var env struct {
Data struct {
Services []Service `json:"services"`
} `json:"data"`
}
if err := json.Unmarshal(body, &env); err != nil {
t.Fatalf("decode: %v (%s)", err, body)
}
if len(env.Data.Services) != 2 {
t.Fatalf("services = %d, want 2", len(env.Data.Services))
}
}
func TestControlPlane_ToggleTakesEffectImmediately(t *testing.T) {
app, _ := newControlPlane(t)
// Before: hanzo.chat reads mode ON.
code, body := call(t, app, "GET", "/v1/featuregate/mode?host=hanzo.chat", false, nil)
if code != 200 {
t.Fatalf("mode read = %d", code)
}
if got := modeBit(t, body); !got {
t.Fatalf("before toggle waitlistMode=%v, want true", got)
}
// Admin opens chat (mode OFF).
if code, _ := call(t, app, "POST", "/v1/admin/services/chat/mode", true, modeRequest{WaitlistMode: false}); code != 200 {
t.Fatalf("toggle = %d, want 200", code)
}
// After: the SAME mode read reflects it immediately (no redeploy, no cache).
_, body = call(t, app, "GET", "/v1/featuregate/mode?host=hanzo.chat", false, nil)
if got := modeBit(t, body); got {
t.Fatalf("after toggle waitlistMode=%v, want false", got)
}
// A non-admin cannot toggle.
if code, _ := call(t, app, "POST", "/v1/admin/services/api/mode", false, modeRequest{WaitlistMode: false}); code != 403 {
t.Fatalf("non-admin toggle = %d, want 403", code)
}
// Unknown service → 404.
if code, _ := call(t, app, "POST", "/v1/admin/services/nope/mode", true, modeRequest{WaitlistMode: false}); code != 404 {
t.Fatalf("toggle unknown = %d, want 404", code)
}
}
func TestControlPlane_UpsertOnboardsHost(t *testing.T) {
app, _ := newControlPlane(t)
// Onboard a new gated service at runtime.
code, _ := call(t, app, "POST", "/v1/admin/services", true, upsertRequest{
Service: "studio", DisplayName: "Studio", Hosts: []string{"studio.hanzo.ai"}, WaitlistMode: true,
})
if code != 200 {
t.Fatalf("upsert = %d, want 200", code)
}
// Its host now resolves to mode ON.
_, body := call(t, app, "GET", "/v1/featuregate/mode?host=studio.hanzo.ai", false, nil)
if !modeBit(t, body) {
t.Fatalf("onboarded host not gated")
}
// Non-admin cannot onboard.
if code, _ := call(t, app, "POST", "/v1/admin/services", false, upsertRequest{Service: "x", Hosts: []string{"x.hanzo.ai"}}); code != 403 {
t.Fatalf("non-admin upsert = %d, want 403", code)
}
}
func TestControlPlane_ModeRead_UnknownHostHonest(t *testing.T) {
app, _ := newControlPlane(t)
_, body := call(t, app, "GET", "/v1/featuregate/mode?host=example.com", false, nil)
var out struct {
WaitlistMode bool `json:"waitlistMode"`
Known bool `json:"known"`
}
if err := json.Unmarshal(body, &out); err != nil {
t.Fatalf("decode: %v", err)
}
if out.Known || out.WaitlistMode {
t.Fatalf("unknown host = %+v, want known:false waitlistMode:false", out)
}
}
func modeBit(t *testing.T, body []byte) bool {
t.Helper()
var out struct {
WaitlistMode bool `json:"waitlistMode"`
}
if err := json.Unmarshal(body, &out); err != nil {
t.Fatalf("decode mode: %v (%s)", err, body)
}
return out.WaitlistMode
}
+44 -29
View File
@@ -12,12 +12,31 @@
// See the License for the specific language governing permissions and
// limitations under the License.
// Package featuregate is the launch-control ENFORCEMENT for Hanzo's hosted services:
// the native middleware (Enforce) + the per-user approval predicate (Approvals, reused
// from IAM). It is a CONSUMER of the ONE policy engine — the per-service waitlist MODE
// and the host→service registry live in clients/flags (a service's mode IS the
// switch waitlist.<svc>, evaluated through the native engine); the admin board is the
// /v1/admin/services lens and the guard's runtime mode read is /v1/featuregate/mode,
// both served there. This package owns only enforcement, decomplected into two axes:
//
// - PER-SERVICE waitlist mode on|off — the flags switch waitlist.<svc>,
// resolved for a request host via flags.WaitlistModeForHost (the decide).
// - PER-USER approvalStatus pending|approved — owned by IAM (approval.go), REUSED.
//
// THE RULE, applied at ONE native enforcement point (Enforce):
//
// if waitlistMode[host] AND NOT user.approved → bounce to the waitlist
// if approved OR mode=off → allow
// unauthenticated → login first
package featuregate
import (
"context"
"net/http"
"strings"
"github.com/hanzoai/cloud/clients/flags"
"github.com/zap-proto/zip"
)
@@ -45,9 +64,10 @@ import (
// It reads the sanitized X-User-Id / X-User-IsAdmin / X-User-Approved that
// IdentityMiddleware minted, so it MUST run after it and (like BillingGate) before
// the subsystem handlers. It is deliberately NOT wired here — the unified-binary
// agent owns serve.go's boot chain; this package exposes Enforce + the store so the
// one-line app.Use lands without a merge collision. The store is resolved lazily
// (moduleStore()) so Enforce can be constructed before Mount runs.
// agent owns serve.go's boot chain; this package exposes Enforce so the one-line
// app.Use lands without a merge collision. The decide (flags.WaitlistModeForHost) is
// resolved PER REQUEST and fail-opens until the flags engine has mounted, so Enforce
// can be constructed before Mount runs.
//
// WHY NATIVE IS CANONICAL (in-cluster-bypass). The @file edge guard only gates
// traffic arriving THROUGH the ingress — a pod reaching another service's pod
@@ -80,6 +100,12 @@ type EnforceConfig struct {
// ExemptPrefixes are request-path prefixes never gated (health/metrics/auth).
// A sensible default set is used when empty.
ExemptPrefixes []string
// Gate is THE decide: it resolves whether a request host is in waitlist mode,
// via the ONE policy engine. When nil it is flags.WaitlistModeForHost —
// host→service→waitlist.<svc>. Injected only in tests. Fail-open by contract:
// known=false (unmounted / registry error / un-governed host) → not gated.
Gate func(ctx context.Context, host string) (mode bool, service string, known bool)
}
// defaultExemptPrefixes are the paths enforcement must never touch — HIP-0106
@@ -94,14 +120,19 @@ var defaultExemptPrefixes = []string{
"/__guard/", // the @file guard's own callback surface (defense in depth)
}
// Enforce builds the native enforcement middleware. It is a no-op passthrough when
// no registry store is resolved yet (moduleStore() nil, i.e. Mount hasn't run) —
// so a request before boot completes is never wrongly gated.
// Enforce builds the native enforcement middleware. It is a no-op passthrough when the
// decide reports the host is not governed (gate known=false — the flags registry not
// mounted yet, a store error, or an un-governed host), so a request before boot
// completes is never wrongly gated.
func Enforce(cfg EnforceConfig) zip.Handler {
approvals := cfg.Approvals
if approvals == nil {
approvals = NewApprovals(cfg.IAMBase, 0)
}
gate := cfg.Gate
if gate == nil {
gate = flags.WaitlistModeForHost // the ONE decide: host→service→waitlist.<svc>
}
exempt := cfg.ExemptPrefixes
if len(exempt) == 0 {
exempt = defaultExemptPrefixes
@@ -109,10 +140,6 @@ func Enforce(cfg EnforceConfig) zip.Handler {
waitlistURL := strings.TrimRight(strings.TrimSpace(cfg.WaitlistURL), "/")
return func(c *zip.Ctx) error {
store := moduleStore()
if store == nil {
return c.Next() // registry not mounted yet — never gate pre-boot
}
path := c.Path()
for _, p := range exempt {
if strings.HasPrefix(path, p) {
@@ -134,14 +161,13 @@ func Enforce(cfg EnforceConfig) zip.Handler {
return c.Next()
}
host := c.Fiber().Hostname()
mode, _, known, err := store.ModeForHost(c.Context(), host)
if err != nil || !known || !mode {
// Un-governed host, mode OFF, or a registry read error → allow. A
// governed host is opened by flipping mode OFF; an unknown host is
// not ours to gate at the shared cloud edge (fail-open on a read
// error keeps the API available — the guard is the belt-and-braces
// gate for the hosts that must stay closed).
mode, _, known := gate(c.Context(), c.Fiber().Hostname())
if !known || !mode {
// Un-governed host, mode OFF, or a registry read error (the decide folds
// all three into known=false) → allow. A governed host is opened by
// flipping its waitlist.<svc> switch OFF; an unknown host is not ours to
// gate at the shared cloud edge — the @file guard is the belt-and-braces
// gate for the hosts that must stay closed.
return c.Next()
}
@@ -231,14 +257,3 @@ func isAPIClient(c *zip.Ctx) bool {
}
return false
}
// ── package-level store resolution (lazy, for Enforce constructed before Mount) ──
// moduleStore returns the registry store once Mount has opened it, else nil. The
// Enforce closure calls it PER REQUEST, so serve.go can construct Enforce before
// MountAll runs (the store is set during Mount, well before the first request).
func moduleStore() *Store {
moduleMu.RLock()
defer moduleMu.RUnlock()
return moduleStoreRef
}
+31 -24
View File
@@ -14,27 +14,31 @@ import (
"github.com/zap-proto/zip"
)
// gateApp mounts Enforce over a seeded registry and a catch-all "ok" handler. The
// injected approval status decides whether the caller is off the waitlist. Returns
// a driver that issues one request (host + path + identity + Accept).
// testGate is the injected decide (the flags engine's WaitlistModeForHost seam):
// hanzo.chat is gated, api.hanzo.ai is open, everything else is un-governed. This is
// exactly what flags.WaitlistModeForHost returns for the equivalent registry, without
// standing up the native flag engine (cgo) in a middleware unit test.
func testGate(_ context.Context, host string) (mode bool, service string, known bool) {
switch host {
case "hanzo.chat":
return true, "chat", true // gated
case "api.hanzo.ai":
return false, "api", true // open
default:
return false, "", false // un-governed
}
}
// gateApp mounts Enforce over the injected decide and a catch-all "ok" handler. The
// injected approval status decides whether the caller is off the waitlist.
func gateApp(t *testing.T, approvalStatus string) *zip.App {
t.Helper()
st := newTestStore(t)
if _, err := st.Seed(context.Background(), []SeedService{
{Service: "chat", Hosts: []string{"hanzo.chat"}, WaitlistMode: true}, // gated
{Service: "api", Hosts: []string{"api.hanzo.ai"}, WaitlistMode: false}, // open
}, 100); err != nil {
t.Fatalf("seed: %v", err)
}
setModuleStore(st)
t.Cleanup(func() { setModuleStore(nil) })
approvals := newApprovalsWithLookup(func(context.Context, string, string) (string, bool) {
return approvalStatus, true
}, time.Minute)
app := zip.New(zip.Config{Logger: luxlog.New("test")})
app.Use(Enforce(EnforceConfig{WaitlistURL: "https://waitlist.hanzo.ai", Approvals: approvals}))
app.Use(Enforce(EnforceConfig{WaitlistURL: "https://waitlist.hanzo.ai", Approvals: approvals, Gate: testGate}))
app.Get("/*", func(c *zip.Ctx) error { return c.String(200, "ok") })
return app
}
@@ -141,16 +145,17 @@ func TestRule_UnauthenticatedBrowser_BouncedToWaitlist(t *testing.T) {
}
// MONEY-CRITICAL: a paid inference request with a Hanzo API key MUST flow through
// Enforce even on the seeded waitlist-ON `api` host — it is possession-gated + billed
// downstream, never waitlist-gated. Without the exemption THE RULE would 401 it and
// break inference cluster-wide.
// Enforce even on a waitlist-ON host — it is possession-gated + billed downstream,
// never waitlist-gated. Without the exemption THE RULE would 401 it and break
// inference cluster-wide.
func TestRule_APIKeyInference_NeverGated(t *testing.T) {
app := gateApp(t, "pending") // api.hanzo.ai is seeded waitlist ON
app := gateApp(t, "pending")
for _, key := range []string{"hk-43f50b6b", "sk-hz-abc", "pk-hz-obs", "fw_live_x", "hz_secret"} {
// The exact paid-inference shape: Bearer key, JSON accept, NO session/user.
// The exact paid-inference shape: Bearer key, JSON accept, NO session/user, on a
// GATED host — the exemption, not mode, must carry it through.
for _, p := range []string{"/v1/chat/completions", "/v1/models", "/v1/embeddings"} {
code, _ := drive(t, app, greq{
host: "api.hanzo.ai", path: p, accept: "application/json",
host: "hanzo.chat", path: p, accept: "application/json",
authorization: "Bearer " + key,
})
if code != 200 {
@@ -159,7 +164,7 @@ func TestRule_APIKeyInference_NeverGated(t *testing.T) {
}
}
// The api-key / x-api-key header form is exempt too.
code, _ := drive(t, app, greq{host: "api.hanzo.ai", path: "/v1/chat/completions", accept: "application/json", apiKeyHeader: "hk-headerform"})
code, _ := drive(t, app, greq{host: "hanzo.chat", path: "/v1/chat/completions", accept: "application/json", apiKeyHeader: "hk-headerform"})
if code != 200 {
t.Fatalf("api-key header inference = %d, want 200", code)
}
@@ -200,14 +205,16 @@ func TestRule_ForwardHeaderApproved_ThroughWithoutLookup(t *testing.T) {
}
}
func TestEnforce_NoStore_IsPassthrough(t *testing.T) {
setModuleStore(nil)
// The DEFAULT gate (nil Gate → flags.WaitlistModeForHost) fail-opens before the flags
// engine has mounted: with no engine, WaitlistModeForHost returns known=false for every
// host, so Enforce never gates pre-boot.
func TestEnforce_DefaultGate_FailsOpenPreBoot(t *testing.T) {
app := zip.New(zip.Config{Logger: luxlog.New("test")})
app.Use(Enforce(EnforceConfig{WaitlistURL: "https://waitlist.hanzo.ai",
Approvals: newApprovalsWithLookup(func(context.Context, string, string) (string, bool) { return "pending", true }, time.Minute)}))
app.Get("/*", func(c *zip.Ctx) error { return c.String(200, "ok") })
code, _ := drive(t, app, greq{host: "hanzo.chat", path: "/dashboard", user: "u", org: "acme", accept: html})
if code != 200 {
t.Fatalf("no registry mounted = %d, want 200 (never gate pre-boot)", code)
t.Fatalf("default gate pre-boot = %d, want 200 (never gate before flags mounts)", code)
}
}
-74
View File
@@ -1,74 +0,0 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package featuregate
import "strings"
// seedFor returns the initial registry for a brand. LAUNCH POSTURE: every hosted
// product is seeded waitlistMode=ON (gated) — the platform opens in waitlist mode,
// and admin.hanzo.ai flips services OFF one at a time. Seed is idempotent
// (INSERT OR IGNORE), so this is only ever the FIRST-BOOT default; a live toggle is
// never overwritten. admin.<brand> is deliberately NOT seeded (it is admin-only via
// admin-guard, not a waitlist surface).
//
// White-labeled by brand so a Lux/Zoo/Pars deployment governs its OWN hosts. New
// hosted services onboard at runtime via POST /v1/admin/services (no redeploy).
func seedFor(brand string) []SeedService {
d := domainFor(brand)
// The apex product hosts each brand ships, plus their <label>.<domain> alias.
return []SeedService{
{Service: "studio", DisplayName: "Studio", Description: "AI app studio",
WaitlistMode: true, Hosts: []string{"studio." + d}},
{Service: "chat", DisplayName: "Chat", Description: "AI chat",
WaitlistMode: true, Hosts: hostsFor(brand, "chat", "chat."+d)},
{Service: "console", DisplayName: "Console", Description: "Cloud console",
WaitlistMode: true, Hosts: []string{"console." + d}},
{Service: "app", DisplayName: "App", Description: "App builder",
WaitlistMode: true, Hosts: hostsFor(brand, "app", "app."+d)},
{Service: "api", DisplayName: "API", Description: "Inference API gateway",
WaitlistMode: true, Hosts: []string{"api." + d}},
{Service: "team", DisplayName: "Team", Description: "Team workspace",
WaitlistMode: true, Hosts: hostsFor(brand, "team", "team."+d)},
}
}
// domainFor maps a brand to its primary domain. Defaults to hanzo.ai.
func domainFor(brand string) string {
switch strings.ToLower(strings.TrimSpace(brand)) {
case "lux":
return "lux.network"
case "zoo":
return "zoo.ngo"
case "pars":
return "pars.network"
default:
return "hanzo.ai"
}
}
// hostsFor returns the apex-brand host (<brand>.<tld> for the label, e.g.
// hanzo.chat / hanzo.app / hanzo.team) plus the <label>.<domain> alias, when the
// brand ships an apex-label domain; else just the alias. Hanzo/Zoo ship
// hanzo.chat / zoo.chat style apex hosts; the generic alias always applies.
func hostsFor(brand, label, alias string) []string {
switch strings.ToLower(strings.TrimSpace(brand)) {
case "", "hanzo":
return []string{"hanzo." + label, alias}
case "zoo":
return []string{"zoo." + label, alias}
default:
return []string{alias}
}
}
-374
View File
@@ -1,374 +0,0 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package featuregate
import (
"context"
"database/sql"
"errors"
"fmt"
"sort"
"strings"
// The ONE Hanzo SQLite driver (registers "sqlite" under both build tags).
// Same storage pattern as clients/authors / clients/affiliates.
_ "github.com/hanzoai/sqlite"
)
// errNotFound is returned when a service slug (or a host) is not in the registry.
var errNotFound = errors.New("featuregate: service not found")
// Service is one hosted Hanzo service in the launch-control registry — the ONE
// source of truth for whether that service is in waitlist mode. Hosts are the
// public hostnames the service answers on (the key the guard / native middleware
// look a request up by). WaitlistMode ON = gated (only APPROVED users past the
// waitlist); OFF = open to any signed-up user. UpdatedBy records the admin who
// last flipped the mode (audit trail on the toggle itself).
type Service struct {
Service string `json:"service"`
DisplayName string `json:"displayName"`
Hosts []string `json:"hosts"`
WaitlistMode bool `json:"waitlistMode"`
Description string `json:"description"`
CreatedAt int64 `json:"createdAt"`
UpdatedAt int64 `json:"updatedAt"`
UpdatedBy string `json:"updatedBy"`
}
// SeedService is one row of the initial registry (the live hosted services). Seed
// is idempotent (INSERT OR IGNORE), so a restart NEVER overwrites an admin's live
// toggle — the seed only ever CREATES a missing row.
type SeedService struct {
Service string
DisplayName string
Hosts []string
Description string
WaitlistMode bool // the launch default for a freshly-seeded service
}
// Store is the feature-gate registry. ONE global SQLite file holds every hosted
// service's waitlist mode + its hostnames. It is a PLATFORM-WIDE config store (not
// per-tenant): the waitlist mode of hanzo.chat is one global value, toggled from
// admin.hanzo.ai, read by every enforcement point. Two tables, normalized:
//
// feature_services(service PK, display_name, waitlist_mode, description, …)
// feature_hosts(host PK, service FK) -- host → service, the hot lookup index
type Store struct {
db *sql.DB
}
func openStore(path string) (*Store, error) {
db, err := sql.Open("sqlite", path)
if err != nil {
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
}
db.SetMaxOpenConns(1)
for _, pragma := range []string{
"PRAGMA busy_timeout=5000",
"PRAGMA journal_mode=WAL",
"PRAGMA foreign_keys=ON",
} {
if _, err := db.Exec(pragma); err != nil {
_ = db.Close()
return nil, fmt.Errorf("pragma %q: %w", pragma, err)
}
}
s := &Store{db: db}
if err := s.migrate(); err != nil {
_ = db.Close()
return nil, err
}
return s, nil
}
func (s *Store) migrate() error {
const ddl = `
CREATE TABLE IF NOT EXISTS feature_services (
service TEXT PRIMARY KEY,
display_name TEXT NOT NULL DEFAULT '',
waitlist_mode INTEGER NOT NULL DEFAULT 0,
description TEXT NOT NULL DEFAULT '',
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
updated_by TEXT NOT NULL DEFAULT ''
);
CREATE TABLE IF NOT EXISTS feature_hosts (
host TEXT PRIMARY KEY,
service TEXT NOT NULL,
FOREIGN KEY(service) REFERENCES feature_services(service) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS ix_feature_hosts_service ON feature_hosts(service);
`
if _, err := s.db.Exec(ddl); err != nil {
return fmt.Errorf("featuregate migrate: %w", err)
}
return nil
}
// Close closes the underlying database. Idempotent-safe via sql.DB.
func (s *Store) Close() error { return s.db.Close() }
// NormalizeHost reduces a request Host to the registry key: lowercased, trimmed,
// port stripped. ONE canonicalization for the seed, the toggle, and every lookup,
// so "Hanzo.Chat:443" and "hanzo.chat" resolve to the same service.
func NormalizeHost(host string) string {
h := strings.ToLower(strings.TrimSpace(host))
if i := strings.IndexByte(h, ':'); i >= 0 {
h = h[:i]
}
return h
}
// Seed inserts the initial registry idempotently (INSERT OR IGNORE on both tables),
// so a boot never clobbers a live admin toggle. Returns the number of services
// created (0 on a warm store).
func (s *Store) Seed(ctx context.Context, rows []SeedService, now int64) (int, error) {
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return 0, fmt.Errorf("seed tx: %w", err)
}
created := 0
for _, r := range rows {
svc := strings.ToLower(strings.TrimSpace(r.Service))
if svc == "" {
continue
}
mode := 0
if r.WaitlistMode {
mode = 1
}
res, err := tx.ExecContext(ctx,
`INSERT OR IGNORE INTO feature_services
(service, display_name, waitlist_mode, description, created_at, updated_at, updated_by)
VALUES (?,?,?,?,?,?,?)`,
svc, r.DisplayName, mode, r.Description, now, now, "seed")
if err != nil {
_ = tx.Rollback()
return 0, fmt.Errorf("seed service %q: %w", svc, err)
}
if n, _ := res.RowsAffected(); n > 0 {
created++
}
// Hosts map to the service; INSERT OR IGNORE so a host already claimed by
// ANY service is never re-pointed by a re-seed (first claim wins).
for _, h := range r.Hosts {
host := NormalizeHost(h)
if host == "" {
continue
}
if _, err := tx.ExecContext(ctx,
`INSERT OR IGNORE INTO feature_hosts (host, service) VALUES (?,?)`, host, svc); err != nil {
_ = tx.Rollback()
return 0, fmt.Errorf("seed host %q: %w", host, err)
}
}
}
if err := tx.Commit(); err != nil {
return 0, fmt.Errorf("seed commit: %w", err)
}
return created, nil
}
// List returns every registered service (with its hosts), sorted by slug. This is
// the admin Services board.
func (s *Store) List(ctx context.Context) ([]Service, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT service, display_name, waitlist_mode, description, created_at, updated_at, updated_by
FROM feature_services`)
if err != nil {
return nil, fmt.Errorf("list services: %w", err)
}
defer func() { _ = rows.Close() }()
byService := map[string]*Service{}
out := make([]Service, 0, 16)
for rows.Next() {
var svc Service
var mode int
if err := rows.Scan(&svc.Service, &svc.DisplayName, &mode, &svc.Description,
&svc.CreatedAt, &svc.UpdatedAt, &svc.UpdatedBy); err != nil {
return nil, fmt.Errorf("scan service: %w", err)
}
svc.WaitlistMode = mode != 0
svc.Hosts = []string{}
out = append(out, svc)
}
if err := rows.Err(); err != nil {
return nil, err
}
for i := range out {
byService[out[i].Service] = &out[i]
}
hostRows, err := s.db.QueryContext(ctx, `SELECT host, service FROM feature_hosts`)
if err != nil {
return nil, fmt.Errorf("list hosts: %w", err)
}
defer func() { _ = hostRows.Close() }()
for hostRows.Next() {
var host, svc string
if err := hostRows.Scan(&host, &svc); err != nil {
return nil, fmt.Errorf("scan host: %w", err)
}
if s := byService[svc]; s != nil {
s.Hosts = append(s.Hosts, host)
}
}
if err := hostRows.Err(); err != nil {
return nil, err
}
for i := range out {
sort.Strings(out[i].Hosts)
}
sort.Slice(out, func(i, j int) bool { return out[i].Service < out[j].Service })
return out, nil
}
// Get returns one service by slug, or errNotFound.
func (s *Store) Get(ctx context.Context, service string) (Service, error) {
svc := strings.ToLower(strings.TrimSpace(service))
row := s.db.QueryRowContext(ctx,
`SELECT service, display_name, waitlist_mode, description, created_at, updated_at, updated_by
FROM feature_services WHERE service=?`, svc)
var out Service
var mode int
err := row.Scan(&out.Service, &out.DisplayName, &mode, &out.Description,
&out.CreatedAt, &out.UpdatedAt, &out.UpdatedBy)
if errors.Is(err, sql.ErrNoRows) {
return Service{}, errNotFound
}
if err != nil {
return Service{}, fmt.Errorf("get service: %w", err)
}
out.WaitlistMode = mode != 0
out.Hosts = []string{}
hostRows, err := s.db.QueryContext(ctx, `SELECT host FROM feature_hosts WHERE service=? ORDER BY host`, svc)
if err != nil {
return Service{}, fmt.Errorf("get hosts: %w", err)
}
defer func() { _ = hostRows.Close() }()
for hostRows.Next() {
var h string
if err := hostRows.Scan(&h); err != nil {
return Service{}, fmt.Errorf("scan host: %w", err)
}
out.Hosts = append(out.Hosts, h)
}
return out, hostRows.Err()
}
// Upsert creates or updates a service (display name, description, hosts, initial
// mode) so a new hosted service can be onboarded from admin.<brand> WITHOUT a
// redeploy. On an existing service it updates the metadata + REPLACES the host set
// but PRESERVES the live waitlist_mode (a re-register never silently re-gates an
// opened service); on a NEW service it sets the given mode. Hosts already claimed
// by ANOTHER service are skipped (first-claim wins — a host maps to one service).
func (s *Store) Upsert(ctx context.Context, in Service, by string, now int64) (Service, error) {
svc := strings.ToLower(strings.TrimSpace(in.Service))
if svc == "" {
return Service{}, fmt.Errorf("featuregate: service slug required")
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return Service{}, fmt.Errorf("upsert tx: %w", err)
}
mode := 0
if in.WaitlistMode {
mode = 1
}
// INSERT the new row (mode = requested); on conflict keep the LIVE mode and
// refresh only the metadata.
if _, err := tx.ExecContext(ctx,
`INSERT INTO feature_services
(service, display_name, waitlist_mode, description, created_at, updated_at, updated_by)
VALUES (?,?,?,?,?,?,?)
ON CONFLICT(service) DO UPDATE SET
display_name=excluded.display_name,
description=excluded.description,
updated_at=excluded.updated_at,
updated_by=excluded.updated_by`,
svc, in.DisplayName, mode, in.Description, now, now, strings.TrimSpace(by)); err != nil {
_ = tx.Rollback()
return Service{}, fmt.Errorf("upsert service: %w", err)
}
// Replace THIS service's hosts (delete its own, re-add), leaving other
// services' host claims untouched; a host owned by another service is skipped.
if _, err := tx.ExecContext(ctx, `DELETE FROM feature_hosts WHERE service=?`, svc); err != nil {
_ = tx.Rollback()
return Service{}, fmt.Errorf("clear hosts: %w", err)
}
for _, h := range in.Hosts {
host := NormalizeHost(h)
if host == "" {
continue
}
if _, err := tx.ExecContext(ctx,
`INSERT OR IGNORE INTO feature_hosts (host, service) VALUES (?,?)`, host, svc); err != nil {
_ = tx.Rollback()
return Service{}, fmt.Errorf("add host %q: %w", host, err)
}
}
if err := tx.Commit(); err != nil {
return Service{}, fmt.Errorf("upsert commit: %w", err)
}
return s.Get(ctx, svc)
}
// SetMode flips one service's waitlist mode and stamps who/when. errNotFound if the
// slug is unknown. Returns the updated service.
func (s *Store) SetMode(ctx context.Context, service string, mode bool, by string, now int64) (Service, error) {
svc := strings.ToLower(strings.TrimSpace(service))
m := 0
if mode {
m = 1
}
res, err := s.db.ExecContext(ctx,
`UPDATE feature_services SET waitlist_mode=?, updated_at=?, updated_by=? WHERE service=?`,
m, now, strings.TrimSpace(by), svc)
if err != nil {
return Service{}, fmt.Errorf("set mode: %w", err)
}
if n, _ := res.RowsAffected(); n == 0 {
return Service{}, errNotFound
}
return s.Get(ctx, svc)
}
// ModeForHost is the HOT lookup the enforcement points call once per request: it
// resolves a request host to its service and that service's waitlist mode. `known`
// is false when the host is not in the registry (an UN-GOVERNED host — the native
// middleware passes it through; the guard, attached only to gated hosts, fails
// safe to gated). host is normalized here so the caller passes the raw Host.
func (s *Store) ModeForHost(ctx context.Context, host string) (mode bool, service string, known bool, err error) {
h := NormalizeHost(host)
if h == "" {
return false, "", false, nil
}
row := s.db.QueryRowContext(ctx,
`SELECT s.service, s.waitlist_mode
FROM feature_hosts h JOIN feature_services s ON s.service = h.service
WHERE h.host = ?`, h)
var svc string
var m int
scanErr := row.Scan(&svc, &m)
if errors.Is(scanErr, sql.ErrNoRows) {
return false, "", false, nil
}
if scanErr != nil {
return false, "", false, fmt.Errorf("mode for host: %w", scanErr)
}
return m != 0, svc, true, nil
}
-168
View File
@@ -1,168 +0,0 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
// Licensed under the Apache License, Version 2.0.
package featuregate
import (
"context"
"path/filepath"
"testing"
)
func newTestStore(t *testing.T) *Store {
t.Helper()
st, err := openStore(filepath.Join(t.TempDir(), "featuregate.db"))
if err != nil {
t.Fatalf("openStore: %v", err)
}
t.Cleanup(func() { _ = st.Close() })
return st
}
func TestSeed_Idempotent_NeverClobbersLiveToggle(t *testing.T) {
st := newTestStore(t)
ctx := context.Background()
seed := []SeedService{
{Service: "chat", DisplayName: "Chat", Hosts: []string{"hanzo.chat", "chat.hanzo.ai"}, WaitlistMode: true},
{Service: "api", DisplayName: "API", Hosts: []string{"api.hanzo.ai"}, WaitlistMode: true},
}
created, err := st.Seed(ctx, seed, 100)
if err != nil {
t.Fatalf("seed: %v", err)
}
if created != 2 {
t.Fatalf("first seed created = %d, want 2", created)
}
// Admin opens chat (mode OFF).
if _, err := st.SetMode(ctx, "chat", false, "z@hanzo.ai", 200); err != nil {
t.Fatalf("SetMode: %v", err)
}
// A re-seed (restart) must NOT re-gate chat.
created, err = st.Seed(ctx, seed, 300)
if err != nil {
t.Fatalf("re-seed: %v", err)
}
if created != 0 {
t.Fatalf("re-seed created = %d, want 0 (idempotent)", created)
}
got, err := st.Get(ctx, "chat")
if err != nil {
t.Fatalf("Get chat: %v", err)
}
if got.WaitlistMode {
t.Fatalf("re-seed clobbered the live toggle: chat re-gated")
}
}
func TestModeForHost_NormalizationAndUnknown(t *testing.T) {
st := newTestStore(t)
ctx := context.Background()
if _, err := st.Seed(ctx, []SeedService{
{Service: "chat", Hosts: []string{"hanzo.chat"}, WaitlistMode: true},
}, 100); err != nil {
t.Fatalf("seed: %v", err)
}
// Case-insensitive + port-stripped resolution to the same service.
for _, h := range []string{"hanzo.chat", "Hanzo.Chat", "hanzo.chat:443", " HANZO.CHAT "} {
mode, svc, known, err := st.ModeForHost(ctx, h)
if err != nil {
t.Fatalf("ModeForHost(%q): %v", h, err)
}
if !known || svc != "chat" || !mode {
t.Fatalf("ModeForHost(%q) = mode=%v svc=%q known=%v, want true/chat/true", h, mode, svc, known)
}
}
// An un-governed host is honestly unknown.
_, _, known, err := st.ModeForHost(ctx, "example.com")
if err != nil {
t.Fatalf("ModeForHost(unknown): %v", err)
}
if known {
t.Fatalf("example.com reported known; want unknown")
}
}
func TestSetMode_TogglesAndStamps(t *testing.T) {
st := newTestStore(t)
ctx := context.Background()
if _, err := st.Seed(ctx, []SeedService{{Service: "api", Hosts: []string{"api.hanzo.ai"}, WaitlistMode: true}}, 100); err != nil {
t.Fatalf("seed: %v", err)
}
updated, err := st.SetMode(ctx, "api", false, "z@hanzo.ai", 500)
if err != nil {
t.Fatalf("SetMode: %v", err)
}
if updated.WaitlistMode || updated.UpdatedBy != "z@hanzo.ai" || updated.UpdatedAt != 500 {
t.Fatalf("SetMode result = %+v, want mode off, by z@hanzo.ai, at 500", updated)
}
// The host now reads mode OFF.
mode, _, known, _ := st.ModeForHost(ctx, "api.hanzo.ai")
if !known || mode {
t.Fatalf("after open: mode=%v known=%v, want false/true", mode, known)
}
// Unknown service → errNotFound.
if _, err := st.SetMode(ctx, "nope", true, "z", 600); err != errNotFound {
t.Fatalf("SetMode(nope) err = %v, want errNotFound", err)
}
}
func TestUpsert_OnboardsAndPreservesLiveMode(t *testing.T) {
st := newTestStore(t)
ctx := context.Background()
// Onboard a brand-new service at mode ON, no redeploy.
created, err := st.Upsert(ctx, Service{
Service: "search", DisplayName: "Search", Hosts: []string{"search.hanzo.ai"}, WaitlistMode: true,
}, "z@hanzo.ai", 100)
if err != nil {
t.Fatalf("Upsert new: %v", err)
}
if !created.WaitlistMode || len(created.Hosts) != 1 || created.Hosts[0] != "search.hanzo.ai" {
t.Fatalf("Upsert new = %+v", created)
}
// Admin opens it.
if _, err := st.SetMode(ctx, "search", false, "z@hanzo.ai", 150); err != nil {
t.Fatalf("SetMode: %v", err)
}
// A metadata edit (add a host) must PRESERVE the opened mode, even though the
// request body carries waitlistMode=true (a re-register never re-gates).
edited, err := st.Upsert(ctx, Service{
Service: "search", DisplayName: "Search v2", Hosts: []string{"search.hanzo.ai", "find.hanzo.ai"}, WaitlistMode: true,
}, "z@hanzo.ai", 200)
if err != nil {
t.Fatalf("Upsert edit: %v", err)
}
if edited.WaitlistMode {
t.Fatalf("Upsert edit re-gated an opened service")
}
if edited.DisplayName != "Search v2" || len(edited.Hosts) != 2 {
t.Fatalf("Upsert edit = %+v, want display Search v2 + 2 hosts", edited)
}
}
func TestList_SortedWithHosts(t *testing.T) {
st := newTestStore(t)
ctx := context.Background()
if _, err := st.Seed(ctx, []SeedService{
{Service: "chat", Hosts: []string{"chat.hanzo.ai", "hanzo.chat"}, WaitlistMode: true},
{Service: "api", Hosts: []string{"api.hanzo.ai"}, WaitlistMode: false},
}, 100); err != nil {
t.Fatalf("seed: %v", err)
}
list, err := st.List(ctx)
if err != nil {
t.Fatalf("List: %v", err)
}
if len(list) != 2 || list[0].Service != "api" || list[1].Service != "chat" {
t.Fatalf("List order = %v, want [api chat]", []string{list[0].Service, list[1].Service})
}
if len(list[1].Hosts) != 2 || list[1].Hosts[0] != "chat.hanzo.ai" {
t.Fatalf("chat hosts = %v, want sorted [chat.hanzo.ai hanzo.chat]", list[1].Hosts)
}
}
+49
View File
@@ -0,0 +1,49 @@
package finance
import "context"
// UsageRow is one recorded usage debit — the READ twin of RecordUsage. The SAME
// wallet→revenue posting a metered call wrote is read back here, so the usage a
// customer SEES is exactly what drained their wallet. Cents is the debit magnitude
// (USD minor units); Model is the metered-unit label the debit carried (Entry.Memo);
// CreatedAt is unix seconds.
type UsageRow struct {
ID string `json:"id"`
Cents int64 `json:"cents"`
Model string `json:"model"`
CreatedAt int64 `json:"createdAt"`
}
// ListUsage returns org's recorded usage debits, most-recent-first, up to limit
// (limit <= 0 lists all). It reads the org's OWN finance file — the file IS the
// tenant boundary, so this can only ever return the caller's org's usage — and keeps
// ONLY the usage-debit entries (a deposit/grant is not usage).
//
// It is the CO-RESIDENT read the customer billing surface uses INSTEAD of the S2S
// HTTP hop: co-resident, commerce's own /v1/billing/usage route is not compiled into
// this binary, so proxying that path self-dispatches straight back into the customer
// handler. Reading the ledger here is the same move balance() already makes, so the
// usage view can never self-answer "sign in to view billing".
func (f *ledgerFinance) ListUsage(ctx context.Context, org string, limit int) ([]UsageRow, error) {
store, err := f.storeFor(org, false)
if err != nil {
return nil, err
}
entries, err := store.Entries(ctx, limit)
if err != nil {
return nil, err
}
rows := make([]UsageRow, 0, len(entries))
for _, e := range entries {
if e.Kind != kindUsage {
continue // deposits/grants are credits, not usage
}
rows = append(rows, UsageRow{
ID: e.ID,
Cents: e.Amount.Cents(),
Model: e.Memo,
CreatedAt: e.CreatedAt,
})
}
return rows, nil
}
+42
View File
@@ -0,0 +1,42 @@
package finance
import (
"context"
"fmt"
"testing"
"github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/cloud/types"
)
// BenchmarkListUsage measures the co-resident usage read that replaced the
// commerceinproc self-dispatch (BUG 2): a single per-org SQLite query over the
// finance ledger. Seeds N usage debits, then reads them back — the exact path
// GET /v1/billing/usage now takes co-resident.
func BenchmarkListUsage(b *testing.B) {
for _, n := range []int{100, 1000, 5000} {
b.Run(fmt.Sprintf("entries=%d", n), func(b *testing.B) {
b.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
f := New(b.TempDir())
ctx := context.Background()
for i := 0; i < n; i++ {
if err := f.RecordUsage(ctx, types.UsageInput{
Org: "acme", Subject: "acme",
Amount: money.FromCents(int64(i + 1)), Model: "zen-1", RequestID: fmt.Sprintf("r%d", i),
}); err != nil {
b.Fatalf("seed: %v", err)
}
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
rows, err := f.ListUsage(ctx, "acme", n+1)
if err != nil {
b.Fatalf("ListUsage: %v", err)
}
if len(rows) != n {
b.Fatalf("want %d rows, got %d", n, len(rows))
}
}
})
}
}
+75
View File
@@ -0,0 +1,75 @@
package finance
import (
"context"
"testing"
"github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/cloud/types"
)
// TestListUsage proves the co-resident usage read returns the SAME debits
// RecordUsage wrote (magnitude + model), most-recent-first, and excludes deposits —
// so /v1/billing/usage can answer from the ledger instead of self-dispatching.
func TestListUsage(t *testing.T) {
t.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
f := New(t.TempDir())
ctx := context.Background()
const org = "acme"
// A grant (deposit) must NOT appear as usage.
if _, err := f.Deposit(ctx, types.DepositInput{Org: org, Subject: org, Amount: money.FromCents(10_000), Ref: "grant1"}); err != nil {
t.Fatalf("deposit: %v", err)
}
// Two usage debits.
if err := f.RecordUsage(ctx, types.UsageInput{Org: org, Subject: org, Amount: money.FromCents(150), Model: "gpt-x", RequestID: "r1"}); err != nil {
t.Fatalf("usage r1: %v", err)
}
if err := f.RecordUsage(ctx, types.UsageInput{Org: org, Subject: org, Amount: money.FromCents(75), Model: "embed-y", RequestID: "r2"}); err != nil {
t.Fatalf("usage r2: %v", err)
}
rows, err := f.ListUsage(ctx, org, 100)
if err != nil {
t.Fatalf("ListUsage: %v", err)
}
if len(rows) != 2 {
t.Fatalf("want 2 usage rows (deposit excluded), got %d: %+v", len(rows), rows)
}
var total int64
sawModel := map[string]int64{}
for _, r := range rows {
if r.ID == "" || r.CreatedAt == 0 {
t.Errorf("row missing id/createdAt: %+v", r)
}
total += r.Cents
sawModel[r.Model] = r.Cents
}
if total != 225 {
t.Errorf("want total 225 cents, got %d", total)
}
if sawModel["gpt-x"] != 150 || sawModel["embed-y"] != 75 {
t.Errorf("model→cents mismatch: %+v", sawModel)
}
// Idempotent replay of a recorded request must not double-count.
if err := f.RecordUsage(ctx, types.UsageInput{Org: org, Subject: org, Amount: money.FromCents(150), Model: "gpt-x", RequestID: "r1"}); err != nil {
t.Fatalf("usage r1 replay: %v", err)
}
rows2, err := f.ListUsage(ctx, org, 100)
if err != nil {
t.Fatalf("ListUsage after replay: %v", err)
}
if len(rows2) != 2 {
t.Errorf("idempotent replay must not add a row; want 2, got %d", len(rows2))
}
// An org with no file yet reads empty, not an error.
empty, err := f.ListUsage(ctx, "neverused", 100)
if err != nil {
t.Fatalf("ListUsage empty org: %v", err)
}
if len(empty) != 0 {
t.Errorf("want 0 rows for unused org, got %d", len(empty))
}
}
@@ -1,6 +1,6 @@
//go:build cgo
package featureflags
package flags
// The native evaluator — hanzo-flags (native/flags), a stateless Rust staticlib
// with PostHog-compatible semantics (rollout hash, property operators, variants,
@@ -41,7 +41,7 @@ func engineEvaluate(defsJSON, ctxJSON []byte) (json.RawMessage, error) {
out := C.hanzo_flags_evaluate(cDefs, cCtx)
if out == nil {
return nil, fmt.Errorf("featureflags: native evaluator returned nil")
return nil, fmt.Errorf("flags: native evaluator returned nil")
}
defer C.hanzo_flags_free(out)
res := []byte(C.GoString(out))
@@ -51,7 +51,7 @@ func engineEvaluate(defsJSON, ctxJSON []byte) (json.RawMessage, error) {
Error string `json:"error"`
}
if err := json.Unmarshal(res, &probe); err == nil && probe.Error != "" {
return nil, fmt.Errorf("featureflags: %s", probe.Error)
return nil, fmt.Errorf("flags: %s", probe.Error)
}
return json.RawMessage(res), nil
}
@@ -1,6 +1,6 @@
//go:build !cgo
package featureflags
package flags
// Pure-Go builds carry no native evaluator (the same posture as the SQLCipher
// at-rest layer: cgo builds get the real engine, !cgo builds degrade loudly).
@@ -15,5 +15,5 @@ import (
const engineAvailable = false
func engineEvaluate(_, _ []byte) (json.RawMessage, error) {
return nil, fmt.Errorf("featureflags: native evaluator not built (cgo disabled)")
return nil, fmt.Errorf("flags: native evaluator not built (cgo disabled)")
}
@@ -1,4 +1,4 @@
// Package featureflags is cloud's NATIVE feature-flag engine: definitions live in
// Package flags is cloud's NATIVE feature-flag engine: definitions live in
// per-(org, project) SQLite (cloud.OrgDB — {DataDir}/orgs/{org}/projects/{project}/
// flags.db, encrypted at rest via cek) and evaluation runs in-process through the
// embedded hanzo-flags Rust evaluator (native/flags, FFI) with PostHog-compatible
@@ -22,7 +22,25 @@
// FAIL-SAFE. When the native engine is absent (!cgo) or a store read fails,
// evaluation degrades to env fallback -> literal default and the HTTP surface says
// so honestly — never fail-wrong.
package featureflags
//
// ── THE POLICY PRIMITIVE ─────────────────────────────────────────────────────────
//
// This engine IS Hanzo's runtime decision primitive: (Principal, context) -> verdict,
// evaluated in-process, stateless, hot. Feature flags, rollouts, and the launch
// waitlist (waitlist.go — folded in from the former clients/featuregate: a service's
// mode IS the switch waitlist.<svc>) are its first tenants. The aspirational end-state
// — NOT built here, flagged for the next step — is that the platform's OTHER runtime
// decisions are the SAME shape and could COMPOSE this one engine rather than each
// re-deriving it:
//
// - authz (access policy) — (Principal, resource+action) -> allow/deny
// - entitlements (product-access policy) — (Principal, feature/plan) -> granted/denied
//
// Both are (Principal, context) -> verdict. Folding them onto this evaluator would make
// Policy ONE composable primitive with one audit log and one hot-apply path. DO NOT
// touch authz/entitlements now — this note only names the target so the seam is
// visible; the launch waitlist is the first fold, done here.
package flags
import (
"context"
@@ -122,6 +140,7 @@ type snapshot struct {
// cached for one TTL (the hot-apply bound).
type Client struct {
stores *cloud.OrgStore[*Store]
registry *cloud.OrgStore[*waitlistStore] // waitlist host→service map (platform tenant)
distinctID string
ttl time.Duration
@@ -138,7 +157,7 @@ func (c *Client) configured() bool { return c != nil && c.stores != nil && engin
// is a pure in-memory FFI call.
func (c *Client) evaluateProject(org, project string, ctx []byte) (json.RawMessage, error) {
if !c.configured() {
return nil, fmt.Errorf("featureflags: engine not configured")
return nil, fmt.Errorf("flags: engine not configured")
}
st, err := c.stores.For(org, project)
if err != nil {
@@ -251,7 +270,7 @@ func (c *Client) invalidate() {
func SetPlatformSwitch(key string, definition json.RawMessage, actor string) error {
c := mounted
if c == nil || c.stores == nil {
return fmt.Errorf("featureflags: not mounted")
return fmt.Errorf("flags: not mounted")
}
st, err := c.stores.For(platformOrg, platformProject)
if err != nil {
@@ -415,31 +434,42 @@ type state struct {
// error at boot.
func Mount(app *zip.App, deps cloud.Deps) error {
if deps.Logger == nil {
return fmt.Errorf("featureflags.Mount: nil deps.Logger")
return fmt.Errorf("flags.Mount: nil deps.Logger")
}
if deps.DataDir == "" {
return fmt.Errorf("featureflags.Mount: empty deps.DataDir")
return fmt.Errorf("flags.Mount: empty deps.DataDir")
}
log := deps.Logger.New("subsystem", "featureflags")
log := deps.Logger.New("subsystem", "flags")
c := &Client{
stores: cloud.NewOrgStore[*Store](deps.DataDir, "flags", openStore),
registry: cloud.NewOrgStore[*waitlistStore](deps.DataDir, "waitlist", openWaitlistStore),
distinctID: firstNonEmpty(os.Getenv("FLAGS_PLATFORM_DISTINCT_ID"), "hanzo-platform:"+firstNonEmpty(deps.Brand, "hanzo")),
ttl: ttlFromEnv(),
}
mounted = c
b := cloud.NewBase(deps, "featureflags")
b := cloud.NewBase(deps, "flags")
svc := &cloud.Service[state]{Base: b, State: state{client: c}}
routes(app, svc)
log.Info("featureflags engine ready", "engine", "hanzo-flags", "native", engineAvailable, "ttlSeconds", int(c.ttl.Seconds()), "switches", len(Defs()))
mountWaitlist(c, deps.Brand, log) // fold: seed the host→service registry + register the waitlist.<svc> switches
log.Info("flags engine ready", "engine", "hanzo-flags", "native", engineAvailable, "ttlSeconds", int(c.ttl.Seconds()), "switches", len(Defs()))
return nil
}
// Shutdown closes every open per-org definitions store.
// Shutdown closes every open per-org definitions store and the waitlist registry.
func Shutdown(_ context.Context) error {
if mounted == nil || mounted.stores == nil {
if mounted == nil {
return nil
}
return mounted.stores.CloseAll()
var first error
if mounted.stores != nil {
first = mounted.stores.CloseAll()
}
if mounted.registry != nil {
if err := mounted.registry.CloseAll(); err != nil && first == nil {
first = err
}
}
return first
}
func ttlFromEnv() time.Duration {
@@ -1,4 +1,4 @@
package featureflags
package flags
import (
"crypto/rand"
@@ -1,4 +1,4 @@
package featureflags
package flags
// The canonical PLATFORM launch switches — the runtime knobs the SuperAdmin flips from
// admin.hanzo.ai. Each is an Insights feature flag (the engine); this table names it,
@@ -1,4 +1,4 @@
package featureflags
package flags
// /v1/flags — the product flag API, org-scoped through the gateway principal
// (HIP-0026) and project-scoped through the principal's project. Evaluation is
@@ -25,6 +25,10 @@ func routes(app *zip.App, s *cloud.Service[state]) {
app.Put("/v1/flags/defs/:key", cloud.Handle(s, putDef))
app.Delete("/v1/flags/defs/:key", cloud.Handle(s, deleteDef))
app.Get("/v1/flags/activity", cloud.Handle(s, listActivity))
// Repointed from the former featuregate: the guard's public runtime mode read,
// now answered by the engine decide (host→service→waitlist.<svc>). Exempt from the
// Enforce gate (/v1/featuregate/ prefix), so a gated user can still resolve mode.
app.Get("/v1/featuregate/mode", cloud.Handle(s, waitlistModeRoute))
}
// tenant resolves the org — the tenant-isolation KEY — from the validated
@@ -1,4 +1,4 @@
package featureflags
package flags
// The definitions store — SQLite per (org, project) via cloud.OrgDB (HIP-0302
// physical isolation: {DataDir}/orgs/{org}/projects/{project}/flags.db). Flag
@@ -37,7 +37,7 @@ CREATE TABLE IF NOT EXISTS flag_activity (
CREATE INDEX IF NOT EXISTS idx_flag_activity_key ON flag_activity(key, id);
`
if _, err := db.Exec(schema); err != nil {
return nil, fmt.Errorf("featureflags: migrate: %w", err)
return nil, fmt.Errorf("flags: migrate: %w", err)
}
return &Store{db: db}, nil
}
@@ -116,7 +116,7 @@ func (s *Store) Get(key string) (DefRow, bool, error) {
func (s *Store) Upsert(key string, definition json.RawMessage, actor string) error {
var def map[string]any
if err := json.Unmarshal(definition, &def); err != nil {
return fmt.Errorf("featureflags: definition not an object: %w", err)
return fmt.Errorf("flags: definition not an object: %w", err)
}
def["key"] = key
norm, err := json.Marshal(def)
+288
View File
@@ -0,0 +1,288 @@
package flags
// The waitlist LENS on the ONE flag engine — the launch-control plane folded in from
// the former clients/featuregate. Decomplected into the two orthogonal axes it always
// was, now with a single decision plane:
//
// - MODE (per service): waitlist.<svc> IS a platform switch, evaluated through the
// SAME native engine as every other platform flag. There is no second mode store.
// - HOST MAP + metadata: the registry (waitlist_store.go) resolves a request host
// to the service whose switch governs it, and carries display metadata.
//
// The decide is WaitlistModeForHost(host) → (mode, service, known): resolve host→svc,
// then read waitlist.<svc>. featuregate.Enforce is now a CONSUMER of this decide, and
// /v1/featuregate/mode + the /v1/admin/services board read it too. Per-user approval
// (pending|approved) stays IAM's (featuregate/approval.go) — the second, orthogonal
// axis, unchanged.
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"time"
"github.com/hanzoai/cloud"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
// SeedService is one row of the launch registry (a hosted service + its hosts). Mode
// is intentionally absent — the launch posture (gated) is waitlistDef's Default "true".
type SeedService struct {
Service string
DisplayName string
Description string
Hosts []string
}
// ServiceInput is the admin onboard/edit payload for /v1/admin/services. WaitlistMode
// sets the launch switch for a NEW service; a re-register PRESERVES the live switch.
type ServiceInput struct {
Service string `json:"service"`
DisplayName string `json:"displayName"`
Description string `json:"description"`
Hosts []string `json:"hosts"`
WaitlistMode bool `json:"waitlistMode"`
}
// ServiceView is one service as the admin board renders it: the registry row plus its
// LIVE waitlist mode (the waitlist.<svc> switch evaluated through the engine).
type ServiceView struct {
ServiceRow
WaitlistMode bool `json:"waitlistMode"`
}
// waitlistKey is the ONE naming rule: a service's mode is the switch waitlist.<svc>.
func waitlistKey(svc string) string { return "waitlist." + strings.ToLower(strings.TrimSpace(svc)) }
// waitlistDef is the platform switch for one service's mode. Default "true" = the
// launch posture (gated until an admin opens it), so a deployment with no stored flag
// behaves exactly as the old featuregate seed (waitlistMode ON).
func waitlistDef(svc, display string) Def {
if strings.TrimSpace(display) == "" {
display = svc
}
return Def{
Key: waitlistKey(svc),
Category: "Launch",
Label: "Waitlist · " + display,
Desc: "Waitlist mode for " + display + ": ON gates the service to APPROVED users; OFF opens it.",
Type: TypeBool,
Default: "true",
}
}
// ensureWaitlistDef registers a service's switch if it is not already registered
// (Mount registers the seed set with nicer labels; this covers runtime onboards).
func ensureWaitlistDef(svc, display string) {
if _, ok := lookupDef(waitlistKey(svc)); !ok {
Register(waitlistDef(svc, display))
}
}
// boolDef is the minimal PostHog flag definition for a boolean switch value.
func boolDef(on bool) json.RawMessage {
if on {
return json.RawMessage(`{"active":true}`)
}
return json.RawMessage(`{"active":false}`)
}
// requireRegistry resolves the platform-tenant registry store, or an error when the
// engine is not mounted (writes need it; the decide fail-opens instead).
func requireRegistry() (*waitlistStore, error) {
c := mounted
if c == nil || c.registry == nil {
return nil, fmt.Errorf("flags: waitlist registry not mounted")
}
return c.registry.For(platformOrg, platformProject)
}
// WaitlistModeForHost is THE decide the Enforce consumer, /v1/featuregate/mode, and
// the admin board call: resolve host→service, then read the waitlist.<svc> switch
// through the engine. FAIL-OPEN by construction — an unmounted registry, a store
// error, or an un-governed host all return known=false, so a request is NEVER gated
// pre-boot or on a registry fault (availability over a hard gate, matching the guard).
func WaitlistModeForHost(ctx context.Context, host string) (mode bool, service string, known bool) {
c := mounted
if c == nil || c.registry == nil {
return false, "", false
}
st, err := c.registry.For(platformOrg, platformProject)
if err != nil {
return false, "", false
}
svc, known, err := st.ServiceForHost(ctx, host)
if err != nil || !known {
return false, "", false
}
return Bool(waitlistKey(svc)), svc, true
}
// ListWaitlistServices returns the admin board: every registered service with its LIVE
// mode (the waitlist.<svc> switch). SuperAdmin surface (the caller gates).
func ListWaitlistServices(ctx context.Context) ([]ServiceView, error) {
st, err := requireRegistry()
if err != nil {
return nil, err
}
rows, err := st.List(ctx)
if err != nil {
return nil, err
}
out := make([]ServiceView, 0, len(rows))
for _, r := range rows {
out = append(out, ServiceView{ServiceRow: r, WaitlistMode: Bool(waitlistKey(r.Service))})
}
return out, nil
}
// SetWaitlistMode flips one service's waitlist switch — the launch lever — and returns
// the updated view. It is the ONE write path (through SetPlatformSwitch, audited in the
// flag activity log); the flip is hot (this pod applies immediately, peers converge
// within the eval TTL). ErrServiceNotFound when the slug is unknown.
func SetWaitlistMode(ctx context.Context, service string, mode bool, actor string) (ServiceView, error) {
service = strings.ToLower(strings.TrimSpace(service))
if service == "" {
return ServiceView{}, fmt.Errorf("flags: service is required")
}
st, err := requireRegistry()
if err != nil {
return ServiceView{}, err
}
row, err := st.Get(ctx, service) // ErrServiceNotFound → 404 upstream
if err != nil {
return ServiceView{}, err
}
ensureWaitlistDef(service, row.DisplayName)
if err := SetPlatformSwitch(waitlistKey(service), boolDef(mode), actor); err != nil {
return ServiceView{}, err
}
return ServiceView{ServiceRow: row, WaitlistMode: Bool(waitlistKey(service))}, nil
}
// UpsertWaitlistService onboards or edits a hosted service so a new host is governed
// WITHOUT a redeploy. A NEW service takes in.WaitlistMode as its launch mode; a
// re-register PRESERVES the live switch (never silently re-gating an opened service).
func UpsertWaitlistService(ctx context.Context, in ServiceInput, actor string) (ServiceView, error) {
svc := strings.ToLower(strings.TrimSpace(in.Service))
if svc == "" {
return ServiceView{}, fmt.Errorf("flags: service slug is required")
}
st, err := requireRegistry()
if err != nil {
return ServiceView{}, err
}
_, getErr := st.Get(ctx, svc)
isNew := errors.Is(getErr, ErrServiceNotFound)
if getErr != nil && !isNew {
return ServiceView{}, getErr
}
row, err := st.Upsert(ctx, ServiceRow{
Service: svc,
DisplayName: in.DisplayName,
Description: in.Description,
Hosts: in.Hosts,
}, actor, time.Now().Unix())
if err != nil {
return ServiceView{}, err
}
ensureWaitlistDef(svc, row.DisplayName)
if isNew {
if err := SetPlatformSwitch(waitlistKey(svc), boolDef(in.WaitlistMode), actor); err != nil {
return ServiceView{}, err
}
}
return ServiceView{ServiceRow: row, WaitlistMode: Bool(waitlistKey(svc))}, nil
}
// mountWaitlist seeds the registry and registers a waitlist.<svc> switch per known
// service. Best-effort + fail-safe: a registry error (e.g. cek master key not yet
// injected) degrades to the in-memory seed switches — the decide then fail-opens,
// exactly the flag engine's own boot posture. Called from Mount.
func mountWaitlist(c *Client, brand string, log luxlog.Logger) {
seed := seedWaitlist(brand)
for _, sv := range seed { // in-memory switches — always succeeds
Register(waitlistDef(sv.Service, sv.DisplayName))
}
st, err := c.registry.For(platformOrg, platformProject)
if err != nil {
log.Warn("waitlist registry unavailable — modes degrade to seed defaults", "err", err)
return
}
if _, err := st.Seed(context.Background(), seed, time.Now().Unix()); err != nil {
log.Warn("waitlist registry seed failed", "err", err)
return
}
if rows, err := st.List(context.Background()); err == nil {
for _, r := range rows { // register any persisted onboard beyond the seed
ensureWaitlistDef(r.Service, r.DisplayName)
}
}
}
// waitlistModeRoute answers GET /v1/featuregate/mode?host=<h> — the runtime lookup the
// @file waitlist-guard caches. Public (in-cluster) read: it returns ONLY the boolean
// mode for the ONE queried host, never an enumeration. Same wire shape as the former
// featuregate route, so the interim guard ports 1:1.
func waitlistModeRoute(_ *cloud.Service[state], c *zip.Ctx) error {
host := strings.TrimSpace(c.Query("host"))
if host == "" {
host = c.Fiber().Hostname()
}
mode, service, known := WaitlistModeForHost(c.Context(), host)
return c.JSON(http.StatusOK, map[string]any{
"host": NormalizeHost(host),
"service": service,
"waitlistMode": mode,
"known": known,
})
}
// ── brand seed (moved verbatim from the former featuregate/seed.go) ──────────────
// seedWaitlist returns the launch registry for a brand. White-labeled so a Lux/Zoo/Pars
// deployment governs its OWN hosts. New hosted services onboard at runtime via
// POST /v1/admin/services (no redeploy). admin.<brand> is deliberately NOT seeded (it
// is admin-guarded, not a waitlist surface).
func seedWaitlist(brand string) []SeedService {
d := domainFor(brand)
return []SeedService{
{Service: "studio", DisplayName: "Studio", Description: "AI app studio", Hosts: []string{"studio." + d}},
{Service: "chat", DisplayName: "Chat", Description: "AI chat", Hosts: hostsFor(brand, "chat", "chat."+d)},
{Service: "console", DisplayName: "Console", Description: "Cloud console", Hosts: []string{"console." + d}},
{Service: "app", DisplayName: "App", Description: "App builder", Hosts: hostsFor(brand, "app", "app."+d)},
{Service: "api", DisplayName: "API", Description: "Inference API gateway", Hosts: []string{"api." + d}},
{Service: "team", DisplayName: "Team", Description: "Team workspace", Hosts: hostsFor(brand, "team", "team."+d)},
}
}
// domainFor maps a brand to its primary domain. Defaults to hanzo.ai.
func domainFor(brand string) string {
switch strings.ToLower(strings.TrimSpace(brand)) {
case "lux":
return "lux.network"
case "zoo":
return "zoo.ngo"
case "pars":
return "pars.network"
default:
return "hanzo.ai"
}
}
// hostsFor returns the apex-brand host (hanzo.chat / zoo.chat style) plus the
// <label>.<domain> alias when the brand ships an apex-label domain; else just the alias.
func hostsFor(brand, label, alias string) []string {
switch strings.ToLower(strings.TrimSpace(brand)) {
case "", "hanzo":
return []string{"hanzo." + label, alias}
case "zoo":
return []string{"zoo." + label, alias}
default:
return []string{alias}
}
}
+270
View File
@@ -0,0 +1,270 @@
package flags
// The waitlist REGISTRY — the host→service map + service metadata folded in from
// the former clients/featuregate SQLite store. It is deliberately MODE-FREE: a
// service's waitlist mode is NOT a column here, it is the platform switch
// waitlist.<svc> evaluated through the ONE native engine (waitlist.go). This store
// answers only "which service owns this host, and what is its display metadata" —
// the config the decide needs, with the decision itself owned by the flag engine.
//
// It rides the SAME per-(org,project) OrgDB machinery as the flag defs (opened via
// cloud.OrgStore, encrypted at rest via cek); the registry is PLATFORM-global, so it
// lives in the reserved platform/platform tenant — one waitlist.db for the deployment.
import (
"context"
"database/sql"
"errors"
"fmt"
"sort"
"strings"
)
// ErrServiceNotFound is returned when a service slug is not in the registry.
var ErrServiceNotFound = errors.New("flags: waitlist service not found")
// ServiceRow is one hosted service in the registry (host→service + metadata). The
// waitlist MODE is intentionally absent — it is the platform switch waitlist.<svc>,
// read through the engine; ListWaitlistServices composes the two into a ServiceView.
type ServiceRow struct {
Service string `json:"service"`
DisplayName string `json:"displayName"`
Description string `json:"description"`
Hosts []string `json:"hosts"`
CreatedAt int64 `json:"createdAt"`
UpdatedAt int64 `json:"updatedAt"`
UpdatedBy string `json:"updatedBy"`
}
// waitlistStore is the registry over one OrgDB handle. Two tables, normalized:
//
// wl_services(service PK, display_name, description, …)
// wl_hosts(host PK, service FK) -- host → service, the hot lookup index
type waitlistStore struct {
db *sql.DB
}
// openWaitlistStore migrates the registry schema over an already-opened (pragma'd,
// cek-wrapped) OrgDB handle — the same open contract as openStore for flag defs.
func openWaitlistStore(db *sql.DB) (*waitlistStore, error) {
const schema = `
CREATE TABLE IF NOT EXISTS wl_services (
service TEXT PRIMARY KEY,
display_name TEXT NOT NULL DEFAULT '',
description TEXT NOT NULL DEFAULT '',
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
updated_by TEXT NOT NULL DEFAULT ''
);
CREATE TABLE IF NOT EXISTS wl_hosts (
host TEXT PRIMARY KEY,
service TEXT NOT NULL,
FOREIGN KEY(service) REFERENCES wl_services(service) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS ix_wl_hosts_service ON wl_hosts(service);
`
if _, err := db.Exec(schema); err != nil {
return nil, fmt.Errorf("flags: waitlist migrate: %w", err)
}
return &waitlistStore{db: db}, nil
}
func (s *waitlistStore) Close() error { return s.db.Close() }
// NormalizeHost reduces a request Host to the registry key: lowercased, trimmed,
// port stripped. ONE canonicalization for the seed, onboard, and every lookup, so
// "Hanzo.Chat:443" and "hanzo.chat" resolve to the same service.
func NormalizeHost(host string) string {
h := strings.ToLower(strings.TrimSpace(host))
if i := strings.IndexByte(h, ':'); i >= 0 {
h = h[:i]
}
return h
}
// Seed inserts the initial registry idempotently (INSERT OR IGNORE on both tables),
// so a boot never clobbers a runtime onboard. Returns the number of services created
// (0 on a warm store).
func (s *waitlistStore) Seed(ctx context.Context, rows []SeedService, now int64) (int, error) {
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return 0, fmt.Errorf("waitlist seed tx: %w", err)
}
created := 0
for _, r := range rows {
svc := strings.ToLower(strings.TrimSpace(r.Service))
if svc == "" {
continue
}
res, err := tx.ExecContext(ctx,
`INSERT OR IGNORE INTO wl_services (service, display_name, description, created_at, updated_at, updated_by)
VALUES (?,?,?,?,?,?)`,
svc, r.DisplayName, r.Description, now, now, "seed")
if err != nil {
_ = tx.Rollback()
return 0, fmt.Errorf("waitlist seed service %q: %w", svc, err)
}
if n, _ := res.RowsAffected(); n > 0 {
created++
}
for _, h := range r.Hosts {
host := NormalizeHost(h)
if host == "" {
continue
}
if _, err := tx.ExecContext(ctx,
`INSERT OR IGNORE INTO wl_hosts (host, service) VALUES (?,?)`, host, svc); err != nil {
_ = tx.Rollback()
return 0, fmt.Errorf("waitlist seed host %q: %w", host, err)
}
}
}
if err := tx.Commit(); err != nil {
return 0, fmt.Errorf("waitlist seed commit: %w", err)
}
return created, nil
}
// List returns every registered service (with its hosts), sorted by slug.
func (s *waitlistStore) List(ctx context.Context) ([]ServiceRow, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT service, display_name, description, created_at, updated_at, updated_by FROM wl_services`)
if err != nil {
return nil, fmt.Errorf("list waitlist services: %w", err)
}
defer func() { _ = rows.Close() }()
byService := map[string]*ServiceRow{}
out := make([]ServiceRow, 0, 16)
for rows.Next() {
var r ServiceRow
if err := rows.Scan(&r.Service, &r.DisplayName, &r.Description, &r.CreatedAt, &r.UpdatedAt, &r.UpdatedBy); err != nil {
return nil, fmt.Errorf("scan waitlist service: %w", err)
}
r.Hosts = []string{}
out = append(out, r)
}
if err := rows.Err(); err != nil {
return nil, err
}
for i := range out {
byService[out[i].Service] = &out[i]
}
hostRows, err := s.db.QueryContext(ctx, `SELECT host, service FROM wl_hosts`)
if err != nil {
return nil, fmt.Errorf("list waitlist hosts: %w", err)
}
defer func() { _ = hostRows.Close() }()
for hostRows.Next() {
var host, svc string
if err := hostRows.Scan(&host, &svc); err != nil {
return nil, fmt.Errorf("scan waitlist host: %w", err)
}
if r := byService[svc]; r != nil {
r.Hosts = append(r.Hosts, host)
}
}
if err := hostRows.Err(); err != nil {
return nil, err
}
for i := range out {
sort.Strings(out[i].Hosts)
}
sort.Slice(out, func(i, j int) bool { return out[i].Service < out[j].Service })
return out, nil
}
// Get returns one service by slug, or ErrServiceNotFound.
func (s *waitlistStore) Get(ctx context.Context, service string) (ServiceRow, error) {
svc := strings.ToLower(strings.TrimSpace(service))
row := s.db.QueryRowContext(ctx,
`SELECT service, display_name, description, created_at, updated_at, updated_by FROM wl_services WHERE service=?`, svc)
var out ServiceRow
err := row.Scan(&out.Service, &out.DisplayName, &out.Description, &out.CreatedAt, &out.UpdatedAt, &out.UpdatedBy)
if errors.Is(err, sql.ErrNoRows) {
return ServiceRow{}, ErrServiceNotFound
}
if err != nil {
return ServiceRow{}, fmt.Errorf("get waitlist service: %w", err)
}
out.Hosts = []string{}
hostRows, err := s.db.QueryContext(ctx, `SELECT host FROM wl_hosts WHERE service=? ORDER BY host`, svc)
if err != nil {
return ServiceRow{}, fmt.Errorf("get waitlist hosts: %w", err)
}
defer func() { _ = hostRows.Close() }()
for hostRows.Next() {
var h string
if err := hostRows.Scan(&h); err != nil {
return ServiceRow{}, fmt.Errorf("scan waitlist host: %w", err)
}
out.Hosts = append(out.Hosts, h)
}
return out, hostRows.Err()
}
// Upsert creates or updates a service's metadata + REPLACES its host set (a host
// already claimed by ANOTHER service is skipped — first claim wins). It never
// touches the mode: the mode is the waitlist.<svc> switch, flipped through
// SetWaitlistMode. Returns the stored row.
func (s *waitlistStore) Upsert(ctx context.Context, in ServiceRow, by string, now int64) (ServiceRow, error) {
svc := strings.ToLower(strings.TrimSpace(in.Service))
if svc == "" {
return ServiceRow{}, fmt.Errorf("flags: waitlist service slug required")
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return ServiceRow{}, fmt.Errorf("waitlist upsert tx: %w", err)
}
if _, err := tx.ExecContext(ctx,
`INSERT INTO wl_services (service, display_name, description, created_at, updated_at, updated_by)
VALUES (?,?,?,?,?,?)
ON CONFLICT(service) DO UPDATE SET
display_name=excluded.display_name,
description=excluded.description,
updated_at=excluded.updated_at,
updated_by=excluded.updated_by`,
svc, in.DisplayName, in.Description, now, now, strings.TrimSpace(by)); err != nil {
_ = tx.Rollback()
return ServiceRow{}, fmt.Errorf("upsert waitlist service: %w", err)
}
if _, err := tx.ExecContext(ctx, `DELETE FROM wl_hosts WHERE service=?`, svc); err != nil {
_ = tx.Rollback()
return ServiceRow{}, fmt.Errorf("clear waitlist hosts: %w", err)
}
for _, h := range in.Hosts {
host := NormalizeHost(h)
if host == "" {
continue
}
if _, err := tx.ExecContext(ctx,
`INSERT OR IGNORE INTO wl_hosts (host, service) VALUES (?,?)`, host, svc); err != nil {
_ = tx.Rollback()
return ServiceRow{}, fmt.Errorf("add waitlist host %q: %w", host, err)
}
}
if err := tx.Commit(); err != nil {
return ServiceRow{}, fmt.Errorf("waitlist upsert commit: %w", err)
}
return s.Get(ctx, svc)
}
// ServiceForHost is the HOT lookup the decide calls once per request: it resolves a
// request host to its owning service. known is false for an un-governed host (the
// caller treats it as pass-through). host is normalized here so the caller passes
// the raw Host.
func (s *waitlistStore) ServiceForHost(ctx context.Context, host string) (service string, known bool, err error) {
h := NormalizeHost(host)
if h == "" {
return "", false, nil
}
var svc string
scanErr := s.db.QueryRowContext(ctx, `SELECT service FROM wl_hosts WHERE host=?`, h).Scan(&svc)
if errors.Is(scanErr, sql.ErrNoRows) {
return "", false, nil
}
if scanErr != nil {
return "", false, fmt.Errorf("waitlist service for host: %w", scanErr)
}
return svc, true, nil
}
+100
View File
@@ -0,0 +1,100 @@
package flags
// Registry coverage for the folded host→service store. It drives the store over a raw
// sqlite handle (the same driver OrgDB uses), so it exercises the fold WITHOUT the cek
// at-rest layer — runnable under CGO=0. The MODE is out of scope here by design (it is
// the waitlist.<svc> switch, evaluated by the native engine, covered separately).
import (
"context"
"database/sql"
"errors"
"path/filepath"
"testing"
_ "github.com/hanzoai/sqlite" // registers "sqlite" under both build tags
)
func newWaitlistStore(t *testing.T) *waitlistStore {
t.Helper()
db, err := sql.Open("sqlite", filepath.Join(t.TempDir(), "waitlist.db"))
if err != nil {
t.Fatalf("open: %v", err)
}
st, err := openWaitlistStore(db)
if err != nil {
t.Fatalf("openWaitlistStore: %v", err)
}
t.Cleanup(func() { _ = st.Close() })
return st
}
func TestWaitlistStore_SeedIdempotentAndServiceForHost(t *testing.T) {
st := newWaitlistStore(t)
ctx := context.Background()
seed := []SeedService{
{Service: "chat", DisplayName: "Chat", Hosts: []string{"hanzo.chat", "chat.hanzo.ai"}},
{Service: "api", DisplayName: "API", Hosts: []string{"api.hanzo.ai"}},
}
created, err := st.Seed(ctx, seed, 100)
if err != nil || created != 2 {
t.Fatalf("seed = %d, %v; want 2, nil", created, err)
}
if created, _ := st.Seed(ctx, seed, 200); created != 0 {
t.Fatalf("re-seed created = %d, want 0 (idempotent)", created)
}
// Host resolution is case-insensitive + port-stripped → the same service.
for _, h := range []string{"hanzo.chat", "Hanzo.Chat", "hanzo.chat:443", " HANZO.CHAT "} {
svc, known, err := st.ServiceForHost(ctx, h)
if err != nil || !known || svc != "chat" {
t.Fatalf("ServiceForHost(%q) = %q,%v,%v; want chat,true,nil", h, svc, known, err)
}
}
// An un-governed host is honestly unknown (the decide fail-opens on it).
if _, known, _ := st.ServiceForHost(ctx, "example.com"); known {
t.Fatal("example.com reported known; want unknown")
}
}
func TestWaitlistStore_ListSortedWithHosts(t *testing.T) {
st := newWaitlistStore(t)
ctx := context.Background()
if _, err := st.Seed(ctx, []SeedService{
{Service: "chat", Hosts: []string{"chat.hanzo.ai", "hanzo.chat"}},
{Service: "api", Hosts: []string{"api.hanzo.ai"}},
}, 100); err != nil {
t.Fatalf("seed: %v", err)
}
list, err := st.List(ctx)
if err != nil || len(list) != 2 {
t.Fatalf("List = %d, %v; want 2", len(list), err)
}
if list[0].Service != "api" || list[1].Service != "chat" {
t.Fatalf("List order = [%s %s], want [api chat]", list[0].Service, list[1].Service)
}
if len(list[1].Hosts) != 2 || list[1].Hosts[0] != "chat.hanzo.ai" {
t.Fatalf("chat hosts = %v, want sorted [chat.hanzo.ai hanzo.chat]", list[1].Hosts)
}
}
func TestWaitlistStore_UpsertOnboardsAndReplacesHosts(t *testing.T) {
st := newWaitlistStore(t)
ctx := context.Background()
row, err := st.Upsert(ctx, ServiceRow{Service: "search", DisplayName: "Search", Hosts: []string{"search.hanzo.ai"}}, "z@hanzo.ai", 100)
if err != nil || len(row.Hosts) != 1 || row.Hosts[0] != "search.hanzo.ai" {
t.Fatalf("Upsert new = %+v, %v", row, err)
}
// A metadata edit REPLACES the host set and updates the display name.
row, err = st.Upsert(ctx, ServiceRow{Service: "search", DisplayName: "Search v2", Hosts: []string{"search.hanzo.ai", "find.hanzo.ai"}}, "z@hanzo.ai", 200)
if err != nil || row.DisplayName != "Search v2" || len(row.Hosts) != 2 {
t.Fatalf("Upsert edit = %+v, %v", row, err)
}
svc, known, _ := st.ServiceForHost(ctx, "find.hanzo.ai")
if !known || svc != "search" {
t.Fatalf("onboarded host find.hanzo.ai = %q,%v; want search,true", svc, known)
}
// An unknown slug is ErrServiceNotFound (the admin lens maps it to 404).
if _, err := st.Get(ctx, "nope"); !errors.Is(err, ErrServiceNotFound) {
t.Fatalf("Get(nope) err = %v, want ErrServiceNotFound", err)
}
}
+3 -3
View File
@@ -181,7 +181,7 @@ func invoke(s *cloud.Service[state], c *zip.Ctx) error {
// the post-success debit; fee==0 or unconfigured billing makes this a no-op.
fee := cloud.ResourceFeeCents(invokeFeeEnvPrefix, "invoke")
project, projectValidated := principal.ValidatedProject(c)
if err := s.Bill.Gate(c.Context(), principal.Payer(c), project, projectValidated, "invoke", fee); err != nil {
if err := s.Bill.Gate(c.Context(), principal.HomeOrg(c), project, projectValidated, "invoke", fee); err != nil {
return cloud.DenyResource(c, err)
}
@@ -226,9 +226,9 @@ func invoke(s *cloud.Service[state], c *zip.Ctx) error {
// Either is independently free (fee 0 → no-op), so an operator can bill by
// request alone, compute alone, or both.
if runErr == nil {
s.Bill.Meter(principal.Payer(c), project, "invoke", fee, c.RequestID(), cloud.ClientIP(c))
s.Bill.Meter(principal.HomeOrg(c), project, "invoke", fee, c.RequestID(), cloud.ClientIP(c))
gbSecCents := gbSecondsCents(dur, memLimitMB(f.MemoryLimit), cloud.ResourceFeeCents(gbSecFeeEnvPrefix, "gbsec"))
s.Bill.MeterUsage(principal.Payer(c), "gbsec", metering.Usage{
s.Bill.MeterUsage(principal.HomeOrg(c), "gbsec", metering.Usage{
Model: "gbsec", // the billed unit: GB-seconds of compute.
AmountCents: gbSecCents,
Project: project,
+24
View File
@@ -74,6 +74,7 @@ func coreCreate(s *cloud.Service[state], ctx context.Context, org, headerProject
r := Repo{
ID: id, Org: org, Project: project, Name: name,
Description: strings.TrimSpace(in.Description), DefaultBranch: defaultBranchName,
Public: in.Public,
CreatedAt: now, UpdatedAt: now,
}
if err := provision(s, ctx, store, r); err != nil {
@@ -86,6 +87,29 @@ func coreCreate(s *cloud.Service[state], ctx context.Context, org, headerProject
return toView(s, r, nil, ""), nil
}
// coreSetVisibility flips a repo's public bit — the ONE mutation behind
// PATCH /v1/git/repos/:name. Public grants anonymous READ (upload-pack) only;
// receive-pack and the whole control plane stay org-authed. Returns the
// updated view, or errNotFound.
func coreSetVisibility(s *cloud.Service[state], ctx context.Context, org, project, name string, public bool) (repoView, error) {
store, err := storeFor(s, org)
if err != nil {
return repoView{}, fmt.Errorf("open store: %w", err)
}
name = normalizeName(name)
if err := store.SetPublic(ctx, org, project, name, public, time.Now().Unix()); err != nil {
if errors.Is(err, errNotFound) {
return repoView{}, errNotFound
}
return repoView{}, fmt.Errorf("set visibility: %w", err)
}
r, err := store.Get(ctx, org, project, name)
if err != nil {
return repoView{}, fmt.Errorf("get: %w", err)
}
return toView(s, r, nil, ""), nil
}
// coreList returns the repos for (org, project), most-recently-updated first.
func coreList(s *cloud.Service[state], ctx context.Context, org, project string) ([]repoView, error) {
store, err := storeFor(s, org)
+46 -1
View File
@@ -98,6 +98,7 @@ type repoView struct {
Name string `json:"name"`
Description string `json:"description,omitempty"`
DefaultBranch string `json:"defaultBranch"`
Public bool `json:"public"`
Branches []string `json:"branches,omitempty"`
Head string `json:"head,omitempty"`
CloneURL string `json:"cloneUrl"`
@@ -136,7 +137,7 @@ func sshURL(s *cloud.Service[state], org, name string) string {
func toView(s *cloud.Service[state], r Repo, branches []string, head string) repoView {
return repoView{
ID: r.ID, Org: r.Org, Project: r.Project, Name: r.Name, Description: r.Description,
DefaultBranch: r.DefaultBranch, Branches: branches, Head: head,
DefaultBranch: r.DefaultBranch, Public: r.Public, Branches: branches, Head: head,
CloneURL: cloneURL(s, r.Org, r.Name),
SSHURL: sshURL(s, r.Org, r.Name),
SizeBytes: r.SizeBytes, CreatedAt: rfc3339(r.CreatedAt), UpdatedAt: rfc3339(r.UpdatedAt),
@@ -180,6 +181,10 @@ func Mount(app *zip.App, deps cloud.Deps) error {
// Install the git object-plane importer so the integrations plane (GitHub App)
// can create + mirror-in + fast-forward-sync repos with no integrations⇄git cycle.
cloud.RegisterGitImporter(githubImporter{})
// Install the outbound-mirror controller so the universal sync engine's git
// provider can ensure/remove a repo's mirror target through the SAME store the
// mirror_out reactor pushes from — no syncsvc⇆git cycle (mirror_control.go).
cloud.RegisterGitMirrorController(gitMirrorController{})
// SSH transport: `git clone git@<sshHost>:<org>/<repo>.git`. The listener is
// a per-process goroutine started here and stopped by Shutdown. The host key
@@ -209,10 +214,18 @@ func routes(app *zip.App, s *cloud.Service[state]) {
app.Get("/v1/git/repos", cloud.Handle(s, list))
app.Get("/v1/git/usage", cloud.Handle(s, usage))
app.Get("/v1/git/repos/:name", cloud.Handle(s, get))
app.Patch("/v1/git/repos/:name", cloud.Handle(s, patchRepo))
app.Delete("/v1/git/repos/:name", cloud.Handle(s, del))
// Push generated files without a local git client (hanzo.app builder).
// A distinct trailing segment, so it never shadows the :org/:repo routes.
app.Post("/v1/git/repos/:name/push", cloud.Handle(s, pushFiles))
// Gitea push-webhook ingest (git.hanzo.ai). A static segment that never
// shadows the :org/:repo smart-HTTP routes; HMAC-authed, drives the same
// fireBranchBuild core a native push does (webhook.go). cloud.Terminal writes
// the handler's bad-signature 401 / malformed-body 400 in-band so the commerce
// /v1 ErrorHandlerJSON (co-mounted ahead) cannot flatten it to 500 — the same
// reject-parity /v1/sync + /v1/connector/github/webhook carry.
app.Post("/v1/git/webhook", cloud.Terminal(cloud.Handle(s, webhook)))
// SSH public-key registry (per-user keys for `git clone git@…`).
app.Post("/v1/git/keys", cloud.Handle(s, registerKey))
app.Get("/v1/git/keys", cloud.Handle(s, listKeys))
@@ -327,6 +340,7 @@ type createReq struct {
Name string `json:"name"`
Project string `json:"project"`
Description string `json:"description"`
Public bool `json:"public"`
}
func create(s *cloud.Service[state], c *zip.Ctx) error {
@@ -345,6 +359,37 @@ func create(s *cloud.Service[state], c *zip.Ctx) error {
return c.JSON(http.StatusCreated, view)
}
// patchReq carries the mutable repo settings. Pointer fields distinguish
// "absent" from "zero" so a PATCH changes exactly what the caller sent.
type patchReq struct {
Public *bool `json:"public"`
}
// patchRepo serves PATCH /v1/git/repos/:name — today that is the visibility
// bit. Org-authed like every control-plane op; a public repo grants anonymous
// READ only (smart_http.go), never write.
func patchRepo(s *cloud.Service[state], c *zip.Ctx) error {
org, ok := org(c)
if !ok {
return zip.ErrForbidden("X-Org-Id required")
}
var body patchReq
if err := c.Bind(&body); err != nil {
return err
}
if body.Public == nil {
return zip.ErrBadRequest("nothing to update (supported: public)")
}
view, err := coreSetVisibility(s, c.Context(), org, projectScope(c), c.Param("name"), *body.Public)
if errors.Is(err, errNotFound) {
return zip.ErrNotFound("repo not found")
}
if err != nil {
return zip.Errorf(http.StatusInternalServerError, "%v", err)
}
return c.JSON(http.StatusOK, view)
}
// createErr maps a coreCreate error to its HTTP status. The ONE mapping the REST
// adapter applies; the ZAP adapter (zapErr) maps the SAME sentinels to its own
// wire shape.
+11 -2
View File
@@ -33,8 +33,17 @@ func TestOrgTraversalRejected(t *testing.T) {
if code, _ := do(t, app, http.MethodPost, "/v1/git/repos/x/push", o, map[string]any{"files": []map[string]any{{"path": "a", "content": "x"}}}); code != http.StatusForbidden {
t.Fatalf("push org=%q want 403, got %d", o, code)
}
if code, _ := do(t, app, http.MethodGet, "/v1/git/x/x.git/info/refs?service=git-upload-pack", o, nil); code != http.StatusForbidden {
t.Fatalf("info/refs org=%q want 403, got %d", o, code)
// Push advertisement: a malformed org is never authenticated → 403.
if code, _ := do(t, app, http.MethodGet, "/v1/git/x/x.git/info/refs?service=git-receive-pack", o, nil); code != http.StatusForbidden {
t.Fatalf("receive info/refs org=%q want 403, got %d", o, code)
}
// Fetch advertisement: a malformed org degrades to the ANONYMOUS public-
// read path, where the traversal string is DISCARDED in favor of the
// orgRE-validated :org path segment ("x") — the repo doesn't exist (and
// would be private), so the uniform 404. The traversal never reaches
// storage on either branch.
if code, _ := do(t, app, http.MethodGet, "/v1/git/x/x.git/info/refs?service=git-upload-pack", o, nil); code != http.StatusNotFound {
t.Fatalf("fetch info/refs org=%q want 404, got %d", o, code)
}
}
// A valid org still works — the gate rejects only unsafe segments.
+6 -5
View File
@@ -13,10 +13,11 @@ import (
func TestRootSmartHTTP_HostGuard(t *testing.T) {
app := mountApp(t) // Domain api.hanzo.test → gitHost git.hanzo.test
// On the git host the root route reaches infoRefs, whose first gate rejects
// a missing X-Org-Id with 403 — proving the route matched and the handler
// ran (not a routing miss).
req := httptest.NewRequest(http.MethodGet, "/acme/repo.git/info/refs?service=git-upload-pack", nil)
// On the git host the root route reaches infoRefs; the PUSH advertisement
// rejects a missing X-Org-Id with 403 (anonymous read exists only for
// upload-pack on public repos) — proving the route matched and the handler
// ran (not a routing miss, which would be 404).
req := httptest.NewRequest(http.MethodGet, "/acme/repo.git/info/refs?service=git-receive-pack", nil)
req.Host = "git.hanzo.test"
resp, err := app.Fiber().Test(req, testCfg)
if err != nil {
@@ -29,7 +30,7 @@ func TestRootSmartHTTP_HostGuard(t *testing.T) {
// On the api host the identical path has no /v1/git prefix, so the guard
// falls through (c.Next()) and nothing matches → 404. Proves the root route
// never serves off the git host.
req = httptest.NewRequest(http.MethodGet, "/acme/repo.git/info/refs?service=git-upload-pack", nil)
req = httptest.NewRequest(http.MethodGet, "/acme/repo.git/info/refs?service=git-receive-pack", nil)
req.Host = "api.hanzo.test"
resp, err = app.Fiber().Test(req, testCfg)
if err != nil {
+72
View File
@@ -0,0 +1,72 @@
package git
import (
"context"
"errors"
"fmt"
"strings"
"time"
)
// mirror_control.go implements the cloud.GitMirrorController seam: the universal
// sync engine's git provider ENSURES or REMOVES a native repo's outbound mirror
// target through it, reusing the SAME repo_mirrors store + validateMirrorTarget
// gate the /mirror endpoint and mirror_out reactor use — one outbound target list,
// no second path. Registered in Mount via cloud.RegisterGitMirrorController, so the
// engine drives it with no syncsvc⇆git import cycle (the pushBuilder / GitImporter
// idiom).
type gitMirrorController struct{}
// EnsureMirror registers (enabled) or removes (disabled) the outbound mirror to url
// for (org, project, repo). Idempotent: enabling an already-present target is a
// no-op, disabling an absent one is a no-op. url is validated + canonicalized
// through validateMirrorTarget (https, no userinfo, outbound-target allowlist), so
// the engine can never register native pushes to an untrusted or internal host.
func (gitMirrorController) EnsureMirror(ctx context.Context, org, project, repo, url string, enabled bool) error {
s := mounted.Load()
if s == nil {
return fmt.Errorf("git: not mounted")
}
name := normalizeName(repo)
if !nameRE.MatchString(name) {
return fmt.Errorf("git: invalid repo name")
}
canonical, host, err := validateMirrorTarget(url)
if err != nil {
return err
}
store, err := storeFor(s, org)
if err != nil {
return err
}
mirrors, err := store.ListMirrors(ctx, org, project, name)
if err != nil {
return err
}
var existing *MirrorTarget
for i := range mirrors {
if strings.EqualFold(mirrors[i].Host, host) {
existing = &mirrors[i]
break
}
}
switch {
case enabled && existing == nil:
id, err := genID("mir")
if err != nil {
return err
}
if err := store.CreateMirror(ctx, MirrorTarget{
ID: id, Org: org, Project: project, Repo: name,
Host: host, URL: canonical, CreatedAt: time.Now().Unix(),
}); err != nil && !errors.Is(err, errConflict) {
return err
}
case !enabled && existing != nil:
if _, err := store.DeleteMirror(ctx, org, project, name, existing.ID); err != nil {
return err
}
}
return nil
}
+130
View File
@@ -0,0 +1,130 @@
package git
import (
"crypto/rand"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"github.com/hanzoai/cloud/cek"
)
// The cek data plane fail-closes without a master key on encryption-capable
// builds; supply one process-wide so the per-org git.db opens in local runs
// exactly as it does in CI (mirrors clients/flags).
func TestMain(m *testing.M) {
k := make([]byte, 32)
if _, err := rand.Read(k); err != nil {
panic(err)
}
cek.SetMasterKey(k)
os.Exit(m.Run())
}
// TestPublicRepo_AnonymousRead proves the visibility model end to end:
// - a repo defaults PRIVATE: anonymous fetch advertisement is a uniform 404
// - PATCH {"public":true} (org-authed) flips it
// - anonymous upload-pack advertisement then serves 200 with the git
// advertisement content type — credential-less clone works (the build
// fabric's requirement)
// - anonymous receive-pack advertisement stays 403 — public is READ ONLY
// - PATCH without auth is 403
func TestPublicRepo_AnonymousRead(t *testing.T) {
app := mountApp(t)
if code, _ := do(t, app, http.MethodPost, "/v1/git/repos", "acme", map[string]any{"name": "site"}); code != http.StatusCreated {
t.Fatalf("create: want 201, got %d", code)
}
anonGet := func(path string) *http.Response {
req := httptest.NewRequest(http.MethodGet, path, nil)
resp, err := app.Fiber().Test(req, testCfg)
if err != nil {
t.Fatalf("anon %s: %v", path, err)
}
return resp
}
advPath := "/v1/git/acme/site.git/info/refs?service=git-upload-pack"
// Private by default: anonymous read is a uniform not-found (no existence
// oracle — a missing repo answers identically).
if resp := anonGet(advPath); resp.StatusCode != http.StatusNotFound {
t.Fatalf("anon fetch on private repo: want 404, got %d", resp.StatusCode)
}
if resp := anonGet("/v1/git/acme/ghost.git/info/refs?service=git-upload-pack"); resp.StatusCode != http.StatusNotFound {
t.Fatalf("anon fetch on missing repo: want 404, got %d", resp.StatusCode)
}
// Unauthenticated PATCH may not flip visibility.
if code, _ := do(t, app, http.MethodPatch, "/v1/git/repos/site", "", map[string]any{"public": true}); code != http.StatusForbidden {
t.Fatalf("anon patch: want 403, got %d", code)
}
// Org-authed PATCH flips it and the view reflects it.
code, body := do(t, app, http.MethodPatch, "/v1/git/repos/site", "acme", map[string]any{"public": true})
if code != http.StatusOK {
t.Fatalf("patch public: want 200, got %d (%s)", code, body)
}
if !strings.Contains(string(body), `"public":true`) {
t.Fatalf("patch view missing public flag: %s", body)
}
// Anonymous fetch advertisement now serves.
resp := anonGet(advPath)
if resp.StatusCode != http.StatusOK {
t.Fatalf("anon fetch on public repo: want 200, got %d", resp.StatusCode)
}
if ct := resp.Header.Get("Content-Type"); ct != "application/x-git-upload-pack-advertisement" {
t.Fatalf("anon fetch content-type: got %q", ct)
}
// Push advertisement stays authenticated even on a public repo.
if resp := anonGet("/v1/git/acme/site.git/info/refs?service=git-receive-pack"); resp.StatusCode != http.StatusForbidden {
t.Fatalf("anon push advert on public repo: want 403, got %d", resp.StatusCode)
}
// Flip back: anonymous read closes again.
if code, _ := do(t, app, http.MethodPatch, "/v1/git/repos/site", "acme", map[string]any{"public": false}); code != http.StatusOK {
t.Fatalf("patch private: want 200, got %d", code)
}
if resp := anonGet(advPath); resp.StatusCode != http.StatusNotFound {
t.Fatalf("anon fetch after re-privating: want 404, got %d", resp.StatusCode)
}
}
// TestPublicRepo_CreatePublic proves the create-time flag is honored, and that
// an authed caller of ANOTHER org still cannot address the repo through the
// path-vs-identity guard (public read is for the anonymous fetch path; an
// authenticated caller keeps strict org scoping).
func TestPublicRepo_CreatePublic(t *testing.T) {
app := mountApp(t)
code, body := do(t, app, http.MethodPost, "/v1/git/repos", "acme", map[string]any{"name": "oss", "public": true})
if code != http.StatusCreated || !strings.Contains(string(body), `"public":true`) {
t.Fatalf("create public: code=%d body=%s", code, body)
}
req := httptest.NewRequest(http.MethodGet, "/v1/git/acme/oss.git/info/refs?service=git-upload-pack", nil)
resp, err := app.Fiber().Test(req, testCfg)
if err != nil {
t.Fatalf("anon fetch: %v", err)
}
if resp.StatusCode != http.StatusOK {
t.Fatalf("anon fetch on create-time public repo: want 200, got %d", resp.StatusCode)
}
// A DIFFERENT authenticated org addressing acme's repo path is refused by
// the path-vs-identity guard (unchanged by the public bit).
req = httptest.NewRequest(http.MethodGet, "/v1/git/acme/oss.git/info/refs?service=git-upload-pack", nil)
req.Header.Set("X-Org-Id", "rival")
req.Header.Set("X-User-Id", "u_rival")
resp, err = app.Fiber().Test(req, testCfg)
if err != nil {
t.Fatalf("cross-org fetch: %v", err)
}
if resp.StatusCode != http.StatusForbidden {
t.Fatalf("cross-org fetch: want 403 (path-vs-identity), got %d", resp.StatusCode)
}
}
+31 -32
View File
@@ -49,33 +49,15 @@ func setGitNoCache(c *zip.Ctx) {
// selected by the ?service= query param; both upload-pack (fetch) and
// receive-pack (push) advertise here.
func infoRefs(s *cloud.Service[state], c *zip.Ctx) error {
org, ok := org(c)
if !ok {
return zip.ErrForbidden("X-Org-Id required")
}
name, err := repoNameParam(c)
if err != nil {
return err
}
project := projectScope(c)
// The org path segment must match the authenticated org. A caller may only
// reach their own org's namespace, never another's, even if they craft a
// different :org in the URL (Red: path-vs-identity confusion).
if p := c.Param("org"); p != "" && p != org {
return zip.ErrForbidden("org path does not match authenticated org")
}
service := c.Query("service")
if service != svcUploadPack && service != svcReceivePack {
return zip.ErrBadRequest("service must be git-upload-pack or git-receive-pack")
}
store, err := storeFor(s, org)
// Anonymous read is allowed ONLY for the fetch advertisement of a PUBLIC
// repo; the push advertisement (receive-pack) always requires the org.
org, project, name, err := resolvePackRepo(s, c, service == svcUploadPack)
if err != nil {
return zip.Errorf(http.StatusInternalServerError, "open store: %v", err)
}
if _, err := store.Get(c.Context(), org, project, name); err != nil {
return zip.ErrNotFound("repo not found")
return err
}
// Advertisement is bounded by ref count (not pack size) — safe to buffer.
@@ -102,7 +84,7 @@ func infoRefs(s *cloud.Service[state], c *zip.Ctx) error {
// `git upload-pack --stateless-rpc` stdin and git's stdout is handed to fasthttp
// as the response body — no pack bytes are buffered in this process.
func uploadPack(s *cloud.Service[state], c *zip.Ctx) error {
org, project, name, err := resolvePackRepo(s, c)
org, project, name, err := resolvePackRepo(s, c, true) // public repos fetch anonymously
if err != nil {
return err
}
@@ -129,7 +111,7 @@ func uploadPack(s *cloud.Service[state], c *zip.Ctx) error {
// push returns. Memory stays bounded: the pack streams to disk, only the tiny
// report is buffered.
func receivePack(s *cloud.Service[state], c *zip.Ctx) error {
org, project, name, err := resolvePackRepo(s, c)
org, project, name, err := resolvePackRepo(s, c, false) // push is NEVER anonymous
if err != nil {
return err
}
@@ -186,19 +168,35 @@ func packRequestBody(c *zip.Ctx) io.Reader {
}
// resolvePackRepo is the shared front-half of every smart-HTTP pack handler:
// resolve the org from X-Org-Id, validate the repo name, enforce the URL org
// segment matches the authenticated org (path-vs-identity guard), and confirm
// the repo exists. Returns the (org, project, name) the pack driver operates on.
func resolvePackRepo(s *cloud.Service[state], c *zip.Ctx) (string, string, string, error) {
orgID, ok := org(c)
if !ok {
return "", "", "", zip.ErrForbidden("X-Org-Id required")
// resolve the org, validate the repo name, enforce the URL org segment matches
// the authenticated org (path-vs-identity guard), and confirm the repo exists.
// Returns the (org, project, name) the pack driver operates on.
//
// allowPublic is the READ concession: with no authenticated org, a fetch-side
// caller (upload-pack) may still resolve a repo that is (a) addressed by an
// explicit, orgRE-safe :org path segment, (b) org-level (no project sub-scope —
// anonymous callers have no validated project identity), and (c) marked Public.
// A private or missing repo answers the SAME 404, so anonymous probing cannot
// distinguish existence. Push (receive-pack) never passes allowPublic.
func resolvePackRepo(s *cloud.Service[state], c *zip.Ctx, allowPublic bool) (string, string, string, error) {
orgID, authed := org(c)
if !authed {
if !allowPublic {
return "", "", "", zip.ErrForbidden("X-Org-Id required")
}
orgID = c.Param("org")
if orgID == "" || !orgRE.MatchString(orgID) {
return "", "", "", zip.ErrForbidden("X-Org-Id required")
}
}
name, err := repoNameParam(c)
if err != nil {
return "", "", "", err
}
project := projectScope(c)
if !authed {
project = "" // anonymous has no validated sub-scope; public repos are org-level
}
if p := c.Param("org"); p != "" && p != orgID {
return "", "", "", zip.ErrForbidden("org path does not match authenticated org")
}
@@ -206,7 +204,8 @@ func resolvePackRepo(s *cloud.Service[state], c *zip.Ctx) (string, string, strin
if serr != nil {
return "", "", "", zip.Errorf(http.StatusInternalServerError, "open store: %v", serr)
}
if _, gerr := store.Get(c.Context(), orgID, project, name); gerr != nil {
r, gerr := store.Get(c.Context(), orgID, project, name)
if gerr != nil || (!authed && !r.Public) {
return "", "", "", zip.ErrNotFound("repo not found")
}
return orgID, project, name, nil
+28 -4
View File
@@ -28,6 +28,7 @@ type Repo struct {
Name string
Description string
DefaultBranch string
Public bool // public repos allow ANONYMOUS read (upload-pack); writes stay org-authed
SizeBytes int64
CreatedAt int64
UpdatedAt int64
@@ -63,6 +64,7 @@ CREATE TABLE IF NOT EXISTS repos (
name TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
default_branch TEXT NOT NULL DEFAULT 'main',
public INTEGER NOT NULL DEFAULT 0,
size_bytes INTEGER NOT NULL DEFAULT 0,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
@@ -120,18 +122,25 @@ CREATE INDEX IF NOT EXISTS ix_inbound_conflicts_repo ON inbound_conflicts(org, p
if _, err := s.db.Exec(ddl); err != nil {
return fmt.Errorf("migrate: %w", err)
}
// public: visibility bit added after the initial schema. Fresh DBs get it
// from the CREATE TABLE above; pre-existing DBs gain it here. The duplicate-
// column error on fresh DBs is the expected no-op.
if _, err := s.db.Exec(`ALTER TABLE repos ADD COLUMN public INTEGER NOT NULL DEFAULT 0`); err != nil &&
!strings.Contains(err.Error(), "duplicate column") {
return fmt.Errorf("migrate public column: %w", err)
}
return nil
}
// Close closes the underlying database.
func (s *Store) Close() error { return s.db.Close() }
const repoCols = `id,org,project,name,description,default_branch,size_bytes,created_at,updated_at`
const repoCols = `id,org,project,name,description,default_branch,public,size_bytes,created_at,updated_at`
func scanRepo(sc interface{ Scan(...any) error }) (Repo, error) {
var r Repo
err := sc.Scan(&r.ID, &r.Org, &r.Project, &r.Name, &r.Description,
&r.DefaultBranch, &r.SizeBytes, &r.CreatedAt, &r.UpdatedAt)
&r.DefaultBranch, &r.Public, &r.SizeBytes, &r.CreatedAt, &r.UpdatedAt)
return r, err
}
@@ -139,9 +148,9 @@ func scanRepo(sc interface{ Scan(...any) error }) (Repo, error) {
// already exists in the org.
func (s *Store) Create(ctx context.Context, r Repo) error {
_, err := s.db.ExecContext(ctx,
`INSERT INTO repos (`+repoCols+`) VALUES (?,?,?,?,?,?,?,?,?)`,
`INSERT INTO repos (`+repoCols+`) VALUES (?,?,?,?,?,?,?,?,?,?)`,
r.ID, r.Org, r.Project, r.Name, r.Description, r.DefaultBranch,
r.SizeBytes, r.CreatedAt, r.UpdatedAt)
r.Public, r.SizeBytes, r.CreatedAt, r.UpdatedAt)
if err != nil {
if isUnique(err) {
return errConflict
@@ -220,6 +229,21 @@ func (s *Store) SetSize(ctx context.Context, org, project, name string, sizeByte
return nil
}
// SetPublic flips a repo's visibility and bumps updated_at. Public grants
// ANONYMOUS READ (upload-pack) only — receive-pack stays org-authed always.
func (s *Store) SetPublic(ctx context.Context, org, project, name string, public bool, updatedAt int64) error {
res, err := s.db.ExecContext(ctx,
`UPDATE repos SET public=?, updated_at=? WHERE org=? AND project=? AND name=?`,
public, updatedAt, org, project, name)
if err != nil {
return fmt.Errorf("set public: %w", err)
}
if n, _ := res.RowsAffected(); n == 0 {
return errNotFound
}
return nil
}
// Delete removes a repo row AND cascade-deletes its lifecycle config
// (subscriptions + mirror targets) in one transaction, so a deleted repo can
// never leave an orphaned external mirror target that a re-created repo of the
+141
View File
@@ -0,0 +1,141 @@
package git
import (
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"net/http"
"os"
"strings"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
)
// Gitea push-webhook ingest. The external Hanzo Git server (a Gitea fork,
// service hanzo-git.hanzo.svc, host git.hanzo.ai) POSTs here on every push so a
// push that lands on it drives the SAME push-to-deploy core the embedded
// smart-HTTP receive-pack path drives: fireBranchBuild → cloud.OnGitPush (deploy
// trigger) + EmitLifecycle (mirror-out / Slack). One deploy trigger, one
// lifecycle stream, regardless of which git server the push landed on — no
// second code path.
//
// Auth is Gitea's HMAC: X-Gitea-Signature is the hex HMAC-SHA256 of the raw
// request body under a shared secret. The secret is KMS-synced
// (hanzo/prod:/git/webhook-secret) into the cloud CR as env GIT_WEBHOOK_SECRET;
// it is NEVER hardcoded. Fail-closed: an unset secret or a mismatched signature
// is 401, so a misconfigured deployment refuses webhooks rather than trusting
// them.
const (
webhookSecretEnv = "GIT_WEBHOOK_SECRET"
giteaEventHeader = "X-Gitea-Event"
giteaSigHeader = "X-Gitea-Signature"
// syncActorEnv names the login the universal sync engine's inbound relay pushes
// AS when it lands an upstream push into native git. A native push webhook whose
// pusher equals it is the ECHO of our own relay — re-driving the build/mirror
// would ping-pong straight back to the upstream it came from. Unset ⇒ no login is
// treated as the sync bot (no push is suppressed), so a deployment without the
// relay keeps today's behavior; set it to the relay's Gitea login to arm the
// guard. Idempotent SHAs already make the echo a no-op downstream; this skips it
// early and explicitly (loop guard, engine-level twin in syncsvc).
syncActorEnv = "GIT_SYNC_ACTOR"
// zeroSHA is git's all-zero object id — the `after` of a branch delete and
// the `before` of a branch create.
zeroSHA = "0000000000000000000000000000000000000000"
)
// giteaPush is the subset of Gitea's push payload the deploy core needs. Gitea
// varies the actor field name across versions (login vs username) for both the
// repo owner and the pusher, so each accepts both; the first non-empty wins.
type giteaPush struct {
Ref string `json:"ref"`
Before string `json:"before"`
After string `json:"after"`
Repository struct {
Name string `json:"name"`
Owner struct {
Username string `json:"username"`
Login string `json:"login"`
} `json:"owner"`
} `json:"repository"`
Pusher struct {
Login string `json:"login"`
Username string `json:"username"`
} `json:"pusher"`
}
// webhook ingests a Gitea push webhook and funnels it through the shared
// push-to-deploy core (fireBranchBuild). Non-push events and no-op pushes
// (branch delete, non-branch ref) are acknowledged 204 so Gitea does not retry.
func webhook(s *cloud.Service[state], c *zip.Ctx) error {
// Only push drives a build; every other Gitea event is an acknowledged no-op.
if c.Header(giteaEventHeader) != "push" {
return c.NoContent(http.StatusNoContent)
}
// Verify BEFORE parse so an unauthenticated body is never decoded. An unset
// secret is a 401 (fail-closed), not an open door.
body := c.Body()
if !validSignature(os.Getenv(webhookSecretEnv), c.Header(giteaSigHeader), body) {
return zip.ErrUnauthorized("invalid webhook signature")
}
var ev giteaPush
if err := json.Unmarshal(body, &ev); err != nil {
return zip.ErrBadRequest("invalid push payload")
}
// Branch pushes only: refs/heads/<branch>. Tags and other refs are ignored.
branch, ok := strings.CutPrefix(ev.Ref, "refs/heads/")
if !ok || branch == "" {
return c.NoContent(http.StatusNoContent)
}
// A zero (or empty) `after` is a branch delete / empty push — no build.
if ev.After == "" || ev.After == zeroSHA {
return c.NoContent(http.StatusNoContent)
}
owner := firstNonEmptyStr(ev.Repository.Owner.Username, ev.Repository.Owner.Login)
name := ev.Repository.Name
if owner == "" || name == "" {
return zip.ErrBadRequest("missing repository owner or name")
}
pusher := firstNonEmptyStr(ev.Pusher.Login, ev.Pusher.Username)
// Loop guard: a push made by the sync service account is the ECHO of an inbound
// relay this cloud just performed (it landed the upstream push into native AS
// this login). Re-driving the build/mirror for it would ping-pong back to the
// upstream. Skip it — acknowledged 204, no build. Unset GIT_SYNC_ACTOR ⇒ nothing
// is suppressed (today's behavior); the SHA is already identical so the outbound
// mirror would no-op regardless, but the skip is explicit + early.
if actor := strings.TrimSpace(os.Getenv(syncActorEnv)); actor != "" && strings.EqualFold(pusher, actor) {
return c.NoContent(http.StatusNoContent)
}
// The SAME funnel receive-pack drives (fireBranchBuilds → fireBranchBuild):
// cloud.OnGitPush deploy trigger + EmitLifecycle. project is "" — Gitea repos
// are org-level, the scope the smart-HTTP pack handlers resolve for a native
// push. Detached from the request (WithoutCancel) so the build outlives the
// 204, matching receivePack.
fireBranchBuild(s, context.WithoutCancel(c.Context()), owner, "", name, branch, ev.Before, ev.After, pusher)
return c.NoContent(http.StatusNoContent)
}
// validSignature reports whether sigHex is the hex HMAC-SHA256 of body under
// secret, compared in constant time. An empty secret or malformed hex is false
// (fail-closed) — never a bypass.
func validSignature(secret, sigHex string, body []byte) bool {
if secret == "" || sigHex == "" {
return false
}
sig, err := hex.DecodeString(sigHex)
if err != nil {
return false
}
mac := hmac.New(sha256.New, []byte(secret))
mac.Write(body)
return hmac.Equal(sig, mac.Sum(nil))
}
+234
View File
@@ -0,0 +1,234 @@
package git
import (
"bytes"
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"net/http"
"net/http/httptest"
"sync"
"testing"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
)
const testWebhookSecret = "test-webhook-secret"
// pushCapture records every push event the registered builder receives.
type pushCapture struct {
sync.Mutex
events []cloud.GitPushEvent
}
// signHook returns Gitea's hex HMAC-SHA256 of body under secret.
func signHook(secret string, body []byte) string {
mac := hmac.New(sha256.New, []byte(secret))
mac.Write(body)
return hex.EncodeToString(mac.Sum(nil))
}
// postHook posts a raw body to /v1/git/webhook with the given event + signature
// headers, returning the status code. The body is sent verbatim (the signed
// message), never re-marshalled by the harness.
func postHook(t *testing.T, app *zip.App, event, sig string, body []byte) int {
t.Helper()
req := httptest.NewRequest(http.MethodPost, "/v1/git/webhook", bytes.NewReader(body))
req.Header.Set("Content-Type", "application/json")
if event != "" {
req.Header.Set(giteaEventHeader, event)
}
if sig != "" {
req.Header.Set(giteaSigHeader, sig)
}
resp, err := app.Fiber().Test(req, testCfg)
if err != nil {
t.Fatalf("Test POST /v1/git/webhook: %v", err)
}
_ = resp.Body.Close()
return resp.StatusCode
}
// captureBuilder installs a push-builder that records every event, and cleans up.
func captureBuilder(t *testing.T) *pushCapture {
t.Helper()
got := &pushCapture{}
cloud.RegisterPushBuilder(func(_ context.Context, ev cloud.GitPushEvent) error {
got.Lock()
got.events = append(got.events, ev)
got.Unlock()
return nil
})
t.Cleanup(func() { cloud.RegisterPushBuilder(nil) })
return got
}
func pushPayload(owner, name, ref, before, after, pusher string) []byte {
var p giteaPush
p.Ref = ref
p.Before = before
p.After = after
p.Repository.Name = name
p.Repository.Owner.Username = owner
p.Pusher.Login = pusher
b, _ := json.Marshal(p)
return b
}
// TestWebhookFiresBuild proves a signed push webhook drives the SAME
// cloud.OnGitPush core a native push does, with the parsed owner/repo/branch/tip.
func TestWebhookFiresBuild(t *testing.T) {
t.Setenv(webhookSecretEnv, testWebhookSecret)
got := captureBuilder(t)
app := mountApp(t)
after := "1111111111111111111111111111111111111111"
body := pushPayload("acme", "code", "refs/heads/main",
"0000000000000000000000000000000000000000", after, "hanzo-dev")
if code := postHook(t, app, "push", signHook(testWebhookSecret, body), body); code != http.StatusNoContent {
t.Fatalf("signed push want 204, got %d", code)
}
got.Lock()
defer got.Unlock()
if len(got.events) != 1 {
t.Fatalf("want exactly 1 push event, got %d: %+v", len(got.events), got.events)
}
ev := got.events[0]
if ev.Org != "acme" || ev.Repo != "code" || ev.Branch != "main" || ev.Commit != after {
t.Fatalf("unexpected event: %+v", ev)
}
if ev.CloneURL != "https://api.hanzo.test/v1/git/acme/code.git" {
t.Fatalf("unexpected clone URL: %q", ev.CloneURL)
}
}
// TestWebhookRejectsBadSignature proves a wrong signature is 401 and fires no build.
func TestWebhookRejectsBadSignature(t *testing.T) {
t.Setenv(webhookSecretEnv, testWebhookSecret)
got := captureBuilder(t)
app := mountApp(t)
body := pushPayload("acme", "code", "refs/heads/main", zeroSHA,
"1111111111111111111111111111111111111111", "hanzo-dev")
if code := postHook(t, app, "push", signHook("wrong-secret", body), body); code != http.StatusUnauthorized {
t.Fatalf("wrong-signature push want 401, got %d", code)
}
if code := postHook(t, app, "push", "not-hex", body); code != http.StatusUnauthorized {
t.Fatalf("malformed-signature push want 401, got %d", code)
}
if code := postHook(t, app, "push", "", body); code != http.StatusUnauthorized {
t.Fatalf("missing-signature push want 401, got %d", code)
}
got.Lock()
defer got.Unlock()
if len(got.events) != 0 {
t.Fatalf("rejected webhooks must fire no build, got %+v", got.events)
}
}
// TestWebhookMissingSecretFailsClosed proves that with no GIT_WEBHOOK_SECRET set,
// even a body signed under the empty secret is refused — the seam is fail-closed.
func TestWebhookMissingSecretFailsClosed(t *testing.T) {
t.Setenv(webhookSecretEnv, "")
got := captureBuilder(t)
app := mountApp(t)
body := pushPayload("acme", "code", "refs/heads/main", zeroSHA,
"1111111111111111111111111111111111111111", "hanzo-dev")
if code := postHook(t, app, "push", signHook("", body), body); code != http.StatusUnauthorized {
t.Fatalf("unset-secret push want 401, got %d", code)
}
got.Lock()
defer got.Unlock()
if len(got.events) != 0 {
t.Fatalf("unset secret must fire no build, got %+v", got.events)
}
}
// TestWebhookLoopGuardSkipsSyncActor proves the loop guard: a push BY the
// configured sync service account (GIT_SYNC_ACTOR) is the echo of an inbound relay
// and is acknowledged 204 with NO build, while a push by any other actor still
// drives the build — so a relay can never ping-pong back to the upstream it came
// from.
func TestWebhookLoopGuardSkipsSyncActor(t *testing.T) {
t.Setenv(webhookSecretEnv, testWebhookSecret)
t.Setenv(syncActorEnv, "hanzo-sync")
got := captureBuilder(t)
app := mountApp(t)
after := "1111111111111111111111111111111111111111"
// A push BY the sync actor is our own relay's echo → skipped, no build.
bot := pushPayload("acme", "code", "refs/heads/main", zeroSHA, after, "hanzo-sync")
if code := postHook(t, app, "push", signHook(testWebhookSecret, bot), bot); code != http.StatusNoContent {
t.Fatalf("sync-actor push want 204, got %d", code)
}
// A push by anyone else still drives the build.
human := pushPayload("acme", "code", "refs/heads/main", zeroSHA, after, "alice")
if code := postHook(t, app, "push", signHook(testWebhookSecret, human), human); code != http.StatusNoContent {
t.Fatalf("human push want 204, got %d", code)
}
got.Lock()
defer got.Unlock()
if len(got.events) != 1 {
t.Fatalf("only the non-sync-actor push must build, got %d: %+v", len(got.events), got.events)
}
if ev := got.events[0]; ev.Org != "acme" || ev.Branch != "main" {
t.Fatalf("unexpected build event: %+v", got.events[0])
}
}
// TestWebhookEventFilter proves a non-push Gitea event is an acknowledged 204
// no-op that fires no build (even with a valid signature).
func TestWebhookEventFilter(t *testing.T) {
t.Setenv(webhookSecretEnv, testWebhookSecret)
got := captureBuilder(t)
app := mountApp(t)
body := pushPayload("acme", "code", "refs/heads/main", zeroSHA,
"1111111111111111111111111111111111111111", "hanzo-dev")
if code := postHook(t, app, "issues", signHook(testWebhookSecret, body), body); code != http.StatusNoContent {
t.Fatalf("non-push event want 204, got %d", code)
}
if code := postHook(t, app, "", signHook(testWebhookSecret, body), body); code != http.StatusNoContent {
t.Fatalf("absent event header want 204, got %d", code)
}
got.Lock()
defer got.Unlock()
if len(got.events) != 0 {
t.Fatalf("non-push events must fire no build, got %+v", got.events)
}
}
// TestWebhookZeroShaNoOp proves a branch-delete push (after=000..0) and a
// non-branch ref are acknowledged 204 no-ops that fire no build.
func TestWebhookZeroShaNoOp(t *testing.T) {
t.Setenv(webhookSecretEnv, testWebhookSecret)
got := captureBuilder(t)
app := mountApp(t)
del := pushPayload("acme", "code", "refs/heads/main",
"1111111111111111111111111111111111111111", zeroSHA, "hanzo-dev")
if code := postHook(t, app, "push", signHook(testWebhookSecret, del), del); code != http.StatusNoContent {
t.Fatalf("branch-delete push want 204, got %d", code)
}
tag := pushPayload("acme", "code", "refs/tags/v1.0.0", zeroSHA,
"1111111111111111111111111111111111111111", "hanzo-dev")
if code := postHook(t, app, "push", signHook(testWebhookSecret, tag), tag); code != http.StatusNoContent {
t.Fatalf("tag push want 204, got %d", code)
}
got.Lock()
defer got.Unlock()
if len(got.events) != 0 {
t.Fatalf("no-op pushes must fire no build, got %+v", got.events)
}
}
+1 -1
View File
@@ -359,7 +359,7 @@ func doStep(s *cloud.Service[state], c *zip.Ctx) error {
if !ok {
return zip.ErrForbidden("X-Org-Id required")
}
payer := principal.Payer(c)
payer := principal.HomeOrg(c)
id := idParam(c)
store, cur, _, rows, err := snapshotFor(s, c.Context(), org)
if err != nil {
+152
View File
@@ -0,0 +1,152 @@
// Package iam2 mounts the clean-room Hanzo IAM v2 (zip-native, beego-free) into the
// unified hanzoai/cloud binary as the identity plane, selected OVER the legacy beego
// Casdoor embed (clients/iam) by CLOUD_IAM_IMPL=iam2. It is the either/or twin of
// clients/iam: both own the SAME absolute prefixes (/v1/iam/*, /login/oauth/*), so
// they CANNOT co-mount — EXACTLY ONE is wired per boot (apps.Wire → identitySpec).
// Default (CLOUD_IAM_IMPL unset) keeps the beego embed, so this package is completely
// inert until the flag flips — safe to land in a churning main.
//
// WHY iam2 is NOT staged like iam. "iam" is staged because iamserver.InitEmbed boots
// the WHOLE Beego runtime and mutates process-global Beego state (web.BeeApp / the
// shared AppConfig), which corrupts the sibling `ai` casdoor fork under mount-all.
// iam2 carries NO such process-global: it opens its OWN orm.DB and registers
// zip-native routes, so the shared-global hazard that pins iam to staged does not
// exist here. The deliberate CLOUD_IAM_IMPL=iam2 opt-in is itself the gate.
//
// FAIL-CLOSED, NOT FAIL-LOUD. A store-open or mount failure degrades THIS subsystem
// to a 503 on the identity prefixes (mountFailClosed) while every co-resident
// subsystem (KMS, o11y, ...) stays up — the fold's blast-radius isolation. It never
// panics the shared binary (iam2server.Mount's only panic path — a registered
// enterprise feature failing to mount — is recovered in safeMount).
package iam2
import (
"context"
"fmt"
"net/http"
"os"
"path/filepath"
iam2server "github.com/hanzoai/iam2/server"
"github.com/hanzoai/orm"
"github.com/zap-proto/zip"
"github.com/hanzoai/cloud"
)
// iam2Prefixes are the canonical absolute prefixes the iam2 identity surface owns,
// used ONLY for the fail-closed 503 — on success iam2server.Mount registers the real
// routes itself. Mirrors the identity subset of clients/iam.iamPrefixes; iam2's bare
// /healthz is deliberately excluded — it is a shared-liveness path, not an auth
// surface, so 503-ing it would mask the binary's own health rather than an identity
// outage.
var iam2Prefixes = []string{
"/v1/iam", // OIDC/OAuth2 + entity CRUD + the Casdoor verb-alias compat layer
"/login/oauth", // browser authorize surface (the /v1/iam/oauth/authorize 302 target)
}
// Mount opens iam2's embedded store, seeds config from the same init_data.json the
// beego iam uses (non-fatal), and registers the whole iam2 surface onto cloud's shared
// zip.App. It matches the cloud.Typed contract (func(*zip.App, cloud.Deps) error) so
// apps.Wire references it via cloud.Typed exactly like clients/iam.Mount — cloud hands
// subsystems a cloud.Deps, not an orm.DB, so iam2 opens its own store here.
func Mount(app *zip.App, deps cloud.Deps) error {
log := deps.Logger.New("subsystem", "iam2")
dbPath, initDataPath := paths(deps)
// SQLite does not create parent dirs; ensure it exists (0700 — identity data).
if err := os.MkdirAll(filepath.Dir(dbPath), 0o700); err != nil {
log.Error("iam2 data dir create failed — serving fail-closed 503 (cloud stays up)", "err", err, "dir", filepath.Dir(dbPath))
mountFailClosed(app)
return nil
}
db, err := iam2server.OpenSQLite(dbPath)
if err != nil {
log.Error("iam2 store open failed — serving fail-closed 503 (cloud stays up)", "err", err, "path", dbPath)
mountFailClosed(app)
return nil
}
// Seed is NON-FATAL: new-only + idempotent config bootstrap (orgs/apps/providers/
// certs) from the SAME init_data.json the beego iam seeds from. A missing or partial
// file leaves iam2 mounted-but-unseeded (honest degrade) rather than blocking the
// identity plane; an already-seeded store simply skips everything.
if sum, serr := iam2server.Seed(context.Background(), db, initDataPath); serr != nil {
log.Warn("iam2 seed skipped (non-fatal)", "err", serr, "init_data", initDataPath)
} else {
log.Info("iam2 seed applied", "created", sum.Created, "skipped", sum.Skipped, "init_data", initDataPath)
}
// TODO(iam2 enterprise): feature.Register(scim.New()); feature.Register(saml.New());
// feature.Register(ldap.New()) — the hanzoiam/{scim,saml,ldap} modules land in a
// parallel lane. iam2server.Mount already calls feature.MountAll, so enabling them is
// these 3 register lines + a go.mod bump; do NOT add the imports until the modules
// are pushed (an unresolved import breaks the build).
// iam2server.Mount registers the whole surface at the canonical absolute paths. It
// PANICS only if a registered enterprise feature fails to mount (none today); recover
// so a future boot-misconfig degrades to fail-closed 503 instead of crashing the
// shared binary — the same blast-radius isolation clients/iam gives.
if err := safeMount(app, db); err != nil {
log.Error("iam2 mount failed — serving fail-closed 503 (cloud stays up)", "err", err)
mountFailClosed(app)
return nil
}
log.Info("iam2 embedded in-process (zip-native, beego-free)", "db", dbPath, "prefixes", iam2Prefixes)
return nil
}
// paths derives iam2's SQLite file and init_data.json path from cloud.Deps, mirroring
// how clients/iam sources them. The store lives under {DataDir}/iam2 — its OWN dir,
// parallel to the beego embed's {DataDir}/iam, so the two impls (either/or, never
// co-resident) keep independent stores and iam2 never opens beego's casdoor-schema
// file. DataDir empty falls back to CWD exactly as the beego default does.
// init_data.json is CWD-relative "init_data.json" — the beego iam's conf default —
// honoring the same `initDataFile` env override, so a deployment points BOTH impls at
// one file (DRY, one source of seed truth).
func paths(deps cloud.Deps) (dbPath, initDataPath string) {
root := deps.DataDir
if root == "" {
root = "."
}
dbPath = filepath.Join(root, "iam2", "iam.db")
initDataPath = os.Getenv("initDataFile")
if initDataPath == "" {
initDataPath = "init_data.json"
}
return dbPath, initDataPath
}
// safeMount runs iam2server.Mount under a recover so its only panic path — a registered
// enterprise feature failing to mount — becomes an error the caller fail-closes on,
// never a crash of the shared cloud binary. With zero features registered today it
// always returns nil.
func safeMount(app *zip.App, db orm.DB) (err error) {
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("iam2 mount panicked: %v", r)
}
}()
iam2server.Mount(app, db)
return nil
}
// mountFailClosed serves an honest JSON 503 on every identity prefix when iam2 cannot
// boot, so /v1/iam/* answers "iam unavailable" instead of falling through to the
// console SPA catch-all (which would 200 an auth path). cloud and every other
// subsystem stay up — the fold's blast-radius isolation. Byte-identical error contract
// to clients/iam.mountFailClosed so clients see one shape regardless of impl.
func mountFailClosed(app *zip.App) {
failed := zip.AdaptNetHTTP(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusServiceUnavailable)
_, _ = w.Write([]byte(`{"error":"iam unavailable","code":503}`))
}))
for _, p := range iam2Prefixes {
app.All(p+"/*", failed)
}
}
+61 -6
View File
@@ -101,8 +101,8 @@ func bridgeReady() {
// binary this is non-negotiable: middleware.Recover() wraps only the sync request
// goroutine, so an unrecovered panic here would crash EVERY tenant and subsystem
// (Red M-2). The recover defer is registered LAST so it runs FIRST (LIFO); release
// still runs after it — a panicking turn frees its slot. This is the generalized
// twin of the shipped slackSpawn: ONE spawn+recover for every platform.
// still runs after it — a panicking turn frees its slot. ONE spawn+recover for
// every platform (Slack included).
func bridgeSpawn(s *cloud.Service[state], org string, run func()) {
go func() {
defer bridgeLim.release(org)
@@ -120,10 +120,65 @@ func bridgeRecover(s *cloud.Service[state], org string) {
}
}
// The bounded agent-turn pool (orgLimiter: GLOBAL cap + PER-ORG sub-limit) and the
// single-use link seen-set (seenSet) are the SHARED primitives defined once in
// slack_events.go / slack_verify.go — the bridge reuses them (bridgeLim / bridgeSeen
// above) rather than redeclaring, so every platform bounds against the SAME types.
// ── the bounded per-org agent-turn pool (the ONE limiter every adapter binds on) ─
// orgLimiter bounds concurrent agent turns two ways: a GLOBAL cap (total in-flight
// across all orgs) AND a PER-ORG cap (max in-flight for any single org). Data /
// token / billing isolation already holds via the resolved org; this adds the
// AVAILABILITY isolation that stops one tenant exhausting the shared worker pool. It
// lives here (provider-agnostic): bridgeLim (the shared chat pool) above, the Slack
// coding pool (codingLim), and every adapter bound against the SAME type.
type orgLimiter struct {
mu sync.Mutex
inflight map[string]int
perOrg int
global chan struct{}
}
func newOrgLimiter(global, perOrg int) *orgLimiter {
if global < 1 {
global = 1
}
if perOrg < 1 {
perOrg = 1
}
if perOrg > global {
perOrg = global
}
return &orgLimiter{inflight: make(map[string]int), perOrg: perOrg, global: make(chan struct{}, global)}
}
// acquire takes one global + one per-org slot for org, non-blocking. It returns
// false (nothing acquired, no slot leaked) when the org is at its per-org cap OR
// the global pool is full — the per-org check precedes the global take, and the
// global take only bumps the per-org count on a successful send.
func (l *orgLimiter) acquire(org string) bool {
l.mu.Lock()
defer l.mu.Unlock()
if l.inflight[org] >= l.perOrg {
return false
}
select {
case l.global <- struct{}{}:
l.inflight[org]++
return true
default:
return false
}
}
// release returns the org's slot and the global slot. Called exactly once per
// successful acquire.
func (l *orgLimiter) release(org string) {
l.mu.Lock()
if n := l.inflight[org]; n > 1 {
l.inflight[org] = n - 1
} else {
delete(l.inflight, org)
}
l.mu.Unlock()
<-l.global
}
// ── the ONE agent brain (shared by every platform + its @mention/DM/slash) ──
+89 -3
View File
@@ -2,9 +2,11 @@ package integrations
import (
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"strconv"
"strings"
"sync"
"time"
)
@@ -16,9 +18,9 @@ import (
// the SAME s.State.stateKey — one signing key, distinct named subjects.
//
// The signing/verifying algebra is the generalized twin of slack_verify.go's
// signSlackSubject/verifySlackSubject; the shared low-level primitives it composes
// the constant-time hmacB64URL, the single-use seenSet, abs64 — live ONCE in
// slack_verify.go and are reused here (no redeclaration).
// signSlackSubject/verifySlackSubject. The shared low-level primitives both compose —
// the constant-time hmacB64URL, the single-use seenSet, abs64 — live ONCE HERE
// (provider-agnostic) and are reused by the Slack primitives too (no redeclaration).
// linkStateTTLSec is the account-link state lifetime — the browser legs must
// complete within it. It is ALSO the single-use seen-set TTL (bridgeSeen), so a
@@ -81,3 +83,87 @@ func verifySubject(key []byte, state string, now int64) (subject, nonce string,
}
return parts[0], parts[2], true
}
// ── in-process single-use seen-set (link-state nonces) ──────────────────────
// seenSet is an age-based single-use / seen-set with an atomic test-and-set. It
// backs every platform's per-user link single-use guarantee (bridgeSeen for the
// generalized adapters, slackUsedStates for Slack): a signed link state is redeemed
// exactly once within its TTL.
//
// SCOPE: per-PROCESS. In a multi-replica deployment the single-use guarantee here
// is DEFENSE-IN-DEPTH: the PRIMARY single-use guarantee is the platform's own
// server-side single-use OAuth `code` (a second exchange of the same code fails at
// the platform) AND hanzo.id's single-use OIDC `code`, plus the state's HMAC +
// browser-bound cookie + short TTL. Eviction is strictly age-based (a within-TTL
// entry is NEVER evicted), which is what forbids an evict-then-replay attack; memory
// is bounded temporally, not by count.
type seenSet struct {
mu sync.Mutex
ttl time.Duration
at map[string]time.Time
order []string // insertion order, for age-based pruning
}
func newSeenSet(ttl time.Duration) *seenSet {
return &seenSet{ttl: ttl, at: make(map[string]time.Time)}
}
// prune drops expired entries oldest-first, stopping at the first still-fresh one.
// Caller holds the lock.
func (s *seenSet) prune(now time.Time) {
i := 0
for ; i < len(s.order); i++ {
k := s.order[i]
t, ok := s.at[k]
if !ok {
continue // already removed via a re-insert
}
if now.Sub(t) > s.ttl {
delete(s.at, k)
} else {
break
}
}
if i > 0 {
s.order = append(s.order[:0], s.order[i:]...)
}
}
// seenAndAdd atomically tests-and-sets: returns true if k was already seen (a
// duplicate/replay); otherwise records it and returns false. The empty key is
// non-dedupable (always unique). `now` zero → time.Now.
func (s *seenSet) seenAndAdd(k string, now time.Time) bool {
if k == "" {
return false
}
if now.IsZero() {
now = time.Now()
}
s.mu.Lock()
defer s.mu.Unlock()
s.prune(now)
if _, ok := s.at[k]; ok {
return true
}
s.at[k] = now
s.order = append(s.order, k)
return false
}
// ── shared low-level helpers ─────────────────────────────────────────────────
// hmacB64URL is the constant-time-composable HMAC-SHA256 of payload under key,
// base64url-encoded. Composed by every signed-state primitive (bridge + Slack).
func hmacB64URL(key []byte, payload string) string {
mac := hmac.New(sha256.New, key)
mac.Write([]byte(payload))
return base64.RawURLEncoding.EncodeToString(mac.Sum(nil))
}
func abs64(x int64) int64 {
if x < 0 {
return -x
}
return x
}
+92
View File
@@ -0,0 +1,92 @@
package integrations
import (
"bytes"
"context"
"net/http"
"net/http/httptest"
"testing"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/kms"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
// flatten_test.go guards the connector webhook against the 500→4xx bug: integrations
// mounts AFTER the commerce embed, whose /v1 ErrorHandlerJSON rewrites ANY propagated
// downstream error into a hardcoded 500. cloud.Terminal writes the reject status
// in-band so it survives. It also pins the route move to the /v1/connector namespace.
// installV1Flatten reproduces apps.mountCommerce's /v1 ErrorHandlerJSON (see the sync
// twin): a /v1 group middleware that turns any propagated downstream error into 500.
func installV1Flatten(app *zip.App) {
app.Group("/v1").Use(func(c *zip.Ctx) error {
if err := c.Next(); err != nil {
return c.Bytes(http.StatusInternalServerError, []byte(`{"error":"flattened"}`))
}
return nil
})
}
// newAppUnderFlatten mounts integrations BEHIND the /v1 flatten filter (production
// mount order: commerce before integrations in apps.Wire).
func newAppUnderFlatten(t *testing.T, kc *kms.Client) *zip.App {
t.Helper()
app := zip.New(zip.Config{Logger: luxlog.New("test")})
installV1Flatten(app)
deps := cloud.Deps{Logger: luxlog.New("test"), DataDir: t.TempDir(), Domain: "api.hanzo.ai"}
if kc != nil {
deps.KMS = kc
}
if err := Mount(app, deps); err != nil {
t.Fatalf("Mount: %v", err)
}
t.Cleanup(func() { _ = Shutdown(context.Background()) })
return app
}
// TestConnectorWebhookRejectsSurviveCommerceFlatten: under the commerce /v1 flatten
// filter a bad signature stays 401 and a malformed body stays 400 — never 500.
func TestConnectorWebhookRejectsSurviveCommerceFlatten(t *testing.T) {
const secret = "wh_secret_flat"
t.Setenv(githubWebhookSecretEnv, secret)
app := newAppUnderFlatten(t, newKMS(t))
p := pushPayload(t, 111, "widgets", "refs/heads/main")
if r := webhookPost(t, app, "push", "sha256=deadbeef", "", p); r.Code != http.StatusUnauthorized {
t.Fatalf("bad-sig under flatten want 401, got %d (%s)", r.Code, r.Body)
}
// Well-signed over the raw bytes, but the body is not valid JSON → 400.
bad := []byte("{not-json")
if r := webhookPost(t, app, "push", ghSign(secret, bad), "", bad); r.Code != http.StatusBadRequest {
t.Fatalf("malformed body under flatten want 400, got %d (%s)", r.Code, r.Body)
}
}
// TestConnectorWebhookRouteMoved pins the /v1/connector/github/webhook namespace: the
// new path resolves to the handler (a bad signature is rejected, not 404'd) and the
// old top-level /v1/github-webhook is gone (404).
func TestConnectorWebhookRouteMoved(t *testing.T) {
const secret = "wh_secret_moved"
t.Setenv(githubWebhookSecretEnv, secret)
app := newApp(t, newKMS(t))
p := pushPayload(t, 111, "widgets", "refs/heads/main")
// webhookPost targets the new path; a bad signature resolving to 401 proves it.
if r := webhookPost(t, app, "push", "sha256=deadbeef", "", p); r.Code != http.StatusUnauthorized {
t.Fatalf("new /v1/connector/github/webhook must resolve (bad-sig 401), got %d (%s)", r.Code, r.Body)
}
// The old path no longer exists.
rq := httptest.NewRequest(http.MethodPost, "/v1/github-webhook", bytes.NewReader(p))
rq.Header.Set("X-GitHub-Event", "push")
rq.Header.Set("X-Hub-Signature-256", ghSign(secret, p))
resp, err := app.Fiber().Test(rq)
if err != nil {
t.Fatalf("old path: %v", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusNotFound {
t.Fatalf("old /v1/github-webhook must 404 after the move, got %d", resp.StatusCode)
}
}
+43 -25
View File
@@ -5,8 +5,8 @@ import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"net/http"
"net/url"
"os"
"strconv"
"strings"
@@ -15,16 +15,19 @@ import (
"github.com/zap-proto/zip"
)
// github_webhook.go is the INBOUND half of the bidirectional sync: GitHub POSTs a
// push event to /v1/integrations/github/webhook, we HMAC-verify it, resolve the org
// from the (signed) installation id, and fast-forward-only advance the native
// branch. PUBLIC at the JWT layer (GitHub has no Hanzo session) — auth is the
// signature, verified fail-closed here, exactly like the Slack events webhook.
// github_webhook.go is the GitHub trigger of the universal sync engine: the org's
// GitHub App POSTs a push event to /v1/connector/github/webhook, we HMAC-verify it, resolve
// the org from the (signed) installation id, mint the installation token, and hand
// the push to cloud.Sync. The engine resolves the SyncLink(s) for the repo and the
// git provider fast-forward-only advances native. PUBLIC at the JWT layer (GitHub
// has no Hanzo session) — auth is the signature, verified fail-closed here, exactly
// like the Slack events webhook.
//
// SPLIT-BRAIN GUARD lives in the git object plane (cloud.InboundGitSync →
// fast-forward-only fetch): a divergence never overwrites native. LOOP PREVENTION
// is the Origin stamp: an inbound advance re-emits push.landed with Origin == the
// source host, so the outbound mirror skips that target and no ping-pong occurs.
// SPLIT-BRAIN GUARD lives in the git object plane (the fast-forward-only fetch): a
// divergence never overwrites native. LOOP PREVENTION is the engine's cursor
// (identical SHAs are a no-op) + actor guard (a push our own mirror made is
// skipped), so no ping-pong occurs. This handler stays thin: verify, resolve, mint,
// enqueue — the engine is the one place the sync happens.
// githubMaxWebhookBody bounds the payload we read + sign over. GitHub caps webhook
// bodies well under this; a hostile/oversized body can neither exhaust memory nor
@@ -63,6 +66,14 @@ type githubPushEvent struct {
Installation struct {
ID int64 `json:"id"`
} `json:"installation"`
// Sender/Pusher identify who pushed — the App/bot login for a push our own
// outbound mirror made, which the engine's loop guard fast-skips.
Sender struct {
Login string `json:"login"`
} `json:"sender"`
Pusher struct {
Name string `json:"name"`
} `json:"pusher"`
}
// githubWebhook verifies + processes an inbound GitHub webhook. It ALWAYS answers a
@@ -112,6 +123,8 @@ func githubWebhook(s *cloud.Service[state], c *zip.Ctx) error {
}
branch := strings.TrimPrefix(ev.Ref, "refs/heads/")
// Mint the installation token HERE (the App plane owns token custody) and pass
// it THROUGH the event, so the engine's git provider fetches without re-minting.
tok, err := InstallationToken(c.Context(), org)
if err != nil {
return zip.Errorf(http.StatusBadGateway, "mint github installation token: %v", err)
@@ -120,25 +133,30 @@ func githubWebhook(s *cloud.Service[state], c *zip.Ctx) error {
if clone == "" && ev.Repository.FullName != "" {
clone = "https://github.com/" + ev.Repository.FullName + ".git"
}
res, err := cloud.InboundGitSync(c.Context(), cloud.GitInboundReq{
Org: org, Repo: ev.Repository.Name, Branch: branch,
CloneURL: clone, Token: tok, Origin: hostOf(clone),
// The universal sync engine is the ONE place a sync happens: it resolves the
// SyncLink(s) whose source is this GitHub repo and applies each (the git
// provider fast-forward-advances native, honoring the link's direction + loop
// guard + cursor). Fail-closed when the engine is unmounted — never a fake OK.
res, err := cloud.Sync(c.Context(), cloud.SyncEvent{
Kind: "git", Provider: "github", Org: org,
Locator: clone, Repo: ev.Repository.Name, Branch: branch,
Before: ev.Before, After: ev.After, Actor: actorOf(ev), Token: tok,
})
if err != nil {
return zip.Errorf(http.StatusBadGateway, "inbound sync: %v", err)
if errors.Is(err, cloud.ErrSyncUnavailable) {
return zip.Errorf(http.StatusServiceUnavailable, "sync engine not available")
}
return zip.Errorf(http.StatusBadGateway, "sync: %v", err)
}
return c.JSON(http.StatusOK, map[string]any{
"applied": res.Applied, "conflict": res.Conflict, "noop": res.NoOp, "detail": res.Detail,
})
return c.JSON(http.StatusOK, map[string]any{"ran": res.Ran, "skipped": res.Skipped})
}
// hostOf returns the lowercased host of a URL (for the loop-prevention Origin
// stamp, which must match the outbound mirror target's stored host). "" on a
// parse miss — the git plane then never suppresses a mirror it can't attribute.
func hostOf(raw string) string {
u, err := url.Parse(strings.TrimSpace(raw))
if err != nil {
return ""
// actorOf returns the login that pushed on GitHub — the App/bot for a push our own
// outbound mirror made (the loop-guard fast-path; cursor idempotency is the real
// guarantee).
func actorOf(ev githubPushEvent) string {
if ev.Sender.Login != "" {
return ev.Sender.Login
}
return strings.ToLower(u.Hostname())
return ev.Pusher.Name
}
+56 -21
View File
@@ -76,6 +76,38 @@ func useImporter(t *testing.T) *recordingImporter {
return f
}
// syncCapture records the events the webhook hands cloud.Sync: the webhook now
// enqueues every verified push to the universal engine (the one place a sync
// happens), so the test asserts the ENQUEUED event rather than a direct git call.
type syncCapture struct {
mu sync.Mutex
events []cloud.SyncEvent
}
func (e *syncCapture) count() int {
e.mu.Lock()
defer e.mu.Unlock()
return len(e.events)
}
func (e *syncCapture) last() cloud.SyncEvent {
e.mu.Lock()
defer e.mu.Unlock()
return e.events[len(e.events)-1]
}
func useSyncEngine(t *testing.T) *syncCapture {
t.Helper()
e := &syncCapture{}
cloud.RegisterSync(func(_ context.Context, ev cloud.SyncEvent) (cloud.SyncResult, error) {
e.mu.Lock()
e.events = append(e.events, ev)
e.mu.Unlock()
return cloud.SyncResult{Ran: 1}, nil
})
t.Cleanup(func() { cloud.RegisterSync(nil) })
return e
}
// pushPayload builds a GitHub push event body.
func pushPayload(t *testing.T, inst int64, repo, ref string) []byte {
t.Helper()
@@ -97,7 +129,7 @@ func pushPayload(t *testing.T, inst int64, repo, ref string) []byte {
// non-empty; org, when non-empty, sets a (would-be-spoofed) X-Org-Id header.
func webhookPost(t *testing.T, app *zip.App, event, sig, org string, payload []byte) httpResult {
t.Helper()
rq := httptest.NewRequest(http.MethodPost, "/v1/integrations/github/webhook", bytes.NewReader(payload))
rq := httptest.NewRequest(http.MethodPost, "/v1/connector/github/webhook", bytes.NewReader(payload))
rq.Header.Set("Content-Type", "application/json")
rq.Header.Set("X-GitHub-Event", event)
if sig != "" {
@@ -148,7 +180,7 @@ func TestVerifyGitHubSignature(t *testing.T) {
// installation id (a spoofed X-Org-Id is ignored), each installation maps to its
// OWN org, and ping/non-branch/delete/unknown-install are benign no-ops.
func TestWebhookOrgIsolationAndFailClosed(t *testing.T) {
fake := useImporter(t)
eng := useSyncEngine(t)
srv := mockGitHub(t, nil)
withGithubApp(t, srv)
const secret = "wh_secret_xyz"
@@ -160,35 +192,38 @@ func TestWebhookOrgIsolationAndFailClosed(t *testing.T) {
_ = mounted.State.store.Upsert(ctx, Connection{Org: "acme", Provider: "github", ExternalID: "111"})
_ = mounted.State.store.Upsert(ctx, Connection{Org: "beta", Provider: "github", ExternalID: "222"})
// 1) Valid push for installation 111 → org acme, delegated to the git plane.
// 1) Valid push for installation 111 → org acme, enqueued to the sync engine.
p := pushPayload(t, 111, "widgets", "refs/heads/main")
if r := webhookPost(t, app, "push", ghSign(secret, p), "", p); r.Code != http.StatusOK {
t.Fatalf("valid push want 200, got %d (%s)", r.Code, r.Body)
}
if fake.inboundN() != 1 {
t.Fatalf("valid push must call InboundSync once, got %d", fake.inboundN())
if eng.count() != 1 {
t.Fatalf("valid push must enqueue one sync event, got %d", eng.count())
}
in := fake.lastInbound()
if in.Org != "acme" || in.Repo != "widgets" || in.Branch != "main" || in.Origin != "github.com" {
t.Fatalf("wrong inbound req: %+v", in)
in := eng.last()
if in.Kind != "git" || in.Provider != "github" || in.Org != "acme" || in.Repo != "widgets" || in.Branch != "main" {
t.Fatalf("wrong sync event: %+v", in)
}
if in.Locator != "https://github.com/acme-gh/widgets.git" {
t.Fatalf("event must carry the source clone URL, got %q", in.Locator)
}
// 2) A SPOOFED X-Org-Id must NOT override the installation-derived org.
p = pushPayload(t, 111, "widgets", "refs/heads/main")
webhookPost(t, app, "push", ghSign(secret, p), "beta", p) // attacker claims beta
if got := fake.lastInbound().Org; got != "acme" {
if got := eng.last().Org; got != "acme" {
t.Fatalf("spoofed X-Org-Id must be ignored; org resolved to %q (want acme)", got)
}
// 3) Installation 222 → org beta (each install isolated to its own org).
p = pushPayload(t, 222, "secret-svc", "refs/heads/main")
webhookPost(t, app, "push", ghSign(secret, p), "", p)
if got := fake.lastInbound(); got.Org != "beta" || got.Repo != "secret-svc" {
if got := eng.last(); got.Org != "beta" || got.Repo != "secret-svc" {
t.Fatalf("installation 222 must resolve to beta, got %+v", got)
}
// 4) BAD signature → 401, git plane never touched.
before := fake.inboundN()
// 4) BAD signature → 401, engine never touched.
before := eng.count()
p = pushPayload(t, 111, "widgets", "refs/heads/main")
if r := webhookPost(t, app, "push", "sha256=deadbeef", "", p); r.Code != http.StatusUnauthorized {
t.Fatalf("bad signature want 401, got %d", r.Code)
@@ -197,25 +232,25 @@ func TestWebhookOrgIsolationAndFailClosed(t *testing.T) {
if r := webhookPost(t, app, "push", "", "", p); r.Code != http.StatusUnauthorized {
t.Fatalf("absent signature want 401, got %d", r.Code)
}
if fake.inboundN() != before {
t.Fatalf("an unsigned/bad-signed push must NEVER reach the git plane")
if eng.count() != before {
t.Fatalf("an unsigned/bad-signed push must NEVER reach the sync engine")
}
// 6) Unknown installation → 200 ignored, not delegated.
before = fake.inboundN()
// 6) Unknown installation → 200 ignored, not enqueued.
before = eng.count()
p = pushPayload(t, 999, "widgets", "refs/heads/main")
if r := webhookPost(t, app, "push", ghSign(secret, p), "", p); r.Code != http.StatusOK {
t.Fatalf("unknown install want 200, got %d", r.Code)
}
if fake.inboundN() != before {
t.Fatal("unknown installation must not delegate to the git plane")
if eng.count() != before {
t.Fatal("unknown installation must not enqueue a sync event")
}
// 7) ping → 200. 8) tag (non-branch) → 200 ignored. 9) branch delete → 200 ignored.
if r := webhookPost(t, app, "ping", ghSign(secret, []byte(`{"zen":"x"}`)), "", []byte(`{"zen":"x"}`)); r.Code != http.StatusOK {
t.Fatalf("ping want 200, got %d", r.Code)
}
before = fake.inboundN()
before = eng.count()
tag := pushPayload(t, 111, "widgets", "refs/tags/v1.0.0")
webhookPost(t, app, "push", ghSign(secret, tag), "", tag)
del, _ := json.Marshal(map[string]any{
@@ -224,8 +259,8 @@ func TestWebhookOrgIsolationAndFailClosed(t *testing.T) {
"installation": map[string]any{"id": 111},
})
webhookPost(t, app, "push", ghSign(secret, del), "", del)
if fake.inboundN() != before {
t.Fatal("tag push and branch delete must NOT drive an inbound sync")
if eng.count() != before {
t.Fatal("tag push and branch delete must NOT enqueue a sync event")
}
}
+21 -10
View File
@@ -270,18 +270,29 @@ func Mount(app *zip.App, deps cloud.Deps) error {
// passes through) — because their auth is done INSIDE the handler: HMAC-SHA256 over
// the raw body for the events/commands webhooks, and signed __Host- cookie + state
// for the link legs. They must NOT be placed behind any principal/tenant gate.
//
// Every verify-inside inbound webhook (slack events/commands, discord interactions,
// teams events, telegram webhook — like /v1/connector/github/webhook) is
// cloud.Terminal-wrapped so its bad-signature 401 / malformed-body 400 is written
// in-band and survives the commerce /v1 ErrorHandlerJSON (co-mounted ahead of us),
// which would otherwise flatten a propagated 4xx to 500. Uniform reject codes.
func routes(app *zip.App, s *cloud.Service[state]) {
app.Get("/v1/integrations", cloud.Handle(s, list))
app.Post("/v1/integrations/slack/events", cloud.Handle(s, slackEvents))
app.Post("/v1/integrations/slack/commands", cloud.Handle(s, slackCommands))
app.Post("/v1/integrations/slack/events", cloud.Terminal(cloud.Handle(s, slackEvents)))
app.Post("/v1/integrations/slack/commands", cloud.Terminal(cloud.Handle(s, slackCommands)))
app.Get("/v1/integrations/slack/link", cloud.Handle(s, slackLink))
app.Get("/v1/integrations/slack/link/slack", cloud.Handle(s, slackLinkSlack))
app.Get("/v1/integrations/slack/link/callback", cloud.Handle(s, slackLinkCallback))
// GitHub App sync (github_app.go / github_webhook.go). The literal github paths
// register BEFORE the /:provider wildcards so they win under registration-order
// matching (same discipline as the slack bridge). The webhook is PUBLIC at the
// JWT layer — HMAC-verified inside; repos/import are org-authed via the principal.
app.Post("/v1/integrations/github/webhook", cloud.Handle(s, githubWebhook))
// GitHub App sync (github_app.go / github_webhook.go). The App POSTs push events
// to /v1/connector/github/webhook — the EXTERNAL-platform namespace
// /v1/connector/<provider>/webhook (github now; gitlab/others are sibling literal
// routes later, each with its own signature scheme + handler). It is PUBLIC at the
// JWT layer, HMAC-verified inside, and hands the push to the universal sync engine
// (cloud.Sync). cloud.Terminal writes the handler's reject status in-band so the
// commerce /v1 ErrorHandlerJSON (co-mounted ahead of us) cannot flatten a bad-sig
// 401 / malformed-body 400 to 500. repos/import register BEFORE the /:provider
// wildcards (registration-order matching) and are org-authed via the principal.
app.Post("/v1/connector/github/webhook", cloud.Terminal(cloud.Handle(s, githubWebhook)))
app.Get("/v1/integrations/github/repos", cloud.Handle(s, githubRepos))
app.Post("/v1/integrations/github/repos/import", cloud.Handle(s, githubImport))
// ChatBridge adapters (bridge.go + discord/teams/telegram). Same discipline as
@@ -291,16 +302,16 @@ func routes(app *zip.App, s *cloud.Service[state]) {
// Framework JWT, Telegram secret-token; the link legs use signed __Host- cookies +
// state. Telegram's /connect is org-authed via the principal (like the framework
// connect). They must NOT sit behind any principal/tenant gate.
app.Post("/v1/integrations/discord/interactions", cloud.Handle(s, discordInteractions))
app.Post("/v1/integrations/discord/interactions", cloud.Terminal(cloud.Handle(s, discordInteractions)))
app.Get("/v1/integrations/discord/link", cloud.Handle(s, discordLink))
app.Get("/v1/integrations/discord/link/discord", cloud.Handle(s, discordLinkDiscord))
app.Get("/v1/integrations/discord/link/callback", cloud.Handle(s, discordLinkCallback))
app.Post("/v1/integrations/teams/events", cloud.Handle(s, teamsEvents))
app.Post("/v1/integrations/teams/events", cloud.Terminal(cloud.Handle(s, teamsEvents)))
app.Get("/v1/integrations/teams/link", cloud.Handle(s, teamsLink))
app.Get("/v1/integrations/teams/link/aad", cloud.Handle(s, teamsLinkAAD))
app.Get("/v1/integrations/teams/link/callback", cloud.Handle(s, teamsLinkCallback))
app.Post("/v1/integrations/telegram/connect", cloud.Handle(s, telegramConnect))
app.Post("/v1/integrations/telegram/webhook", cloud.Handle(s, telegramWebhook))
app.Post("/v1/integrations/telegram/webhook", cloud.Terminal(cloud.Handle(s, telegramWebhook)))
app.Get("/v1/integrations/telegram/link", cloud.Handle(s, telegramLink))
app.Get("/v1/integrations/telegram/link/auth", cloud.Handle(s, telegramLinkAuth))
app.Get("/v1/integrations/telegram/link/callback", cloud.Handle(s, telegramLinkCallback))
+19 -18
View File
@@ -203,19 +203,19 @@ func TestSlackDedupeIdempotency(t *testing.T) {
newApp(t, newKMS(t)) // mounts the store (table created in migrate); sets `mounted`
ctx := context.Background()
fresh, err := mounted.State.store.MarkSlackEvent(ctx, "Ev-1")
fresh, err := mounted.State.store.MarkEvent(ctx, "slack", "Ev-1")
if err != nil || !fresh {
t.Fatalf("first sighting must be fresh (err=%v fresh=%v)", err, fresh)
}
again, err := mounted.State.store.MarkSlackEvent(ctx, "Ev-1")
again, err := mounted.State.store.MarkEvent(ctx, "slack", "Ev-1")
if err != nil || again {
t.Fatalf("a Slack retry of the same event_id must be a duplicate (err=%v again=%v)", err, again)
}
// A different id is fresh; an empty key is non-dedupable (always fresh).
if f, _ := mounted.State.store.MarkSlackEvent(ctx, "Ev-2"); !f {
if f, _ := mounted.State.store.MarkEvent(ctx, "slack", "Ev-2"); !f {
t.Fatal("a distinct event_id must be fresh")
}
if f, _ := mounted.State.store.MarkSlackEvent(ctx, ""); !f {
if f, _ := mounted.State.store.MarkEvent(ctx, "slack", ""); !f {
t.Fatal("an empty key must be non-dedupable (fresh)")
}
}
@@ -591,15 +591,16 @@ func TestSlackShedReturnsNon2xxAndDoesNotRecord(t *testing.T) {
app := newApp(t, newKMS(t))
slackBridgeReady(mounted)
// Swap in a cap-1 limiter, then saturate it so the next handler acquire sheds.
saved := slackLim
slackLim = newOrgLimiter(1, 1)
t.Cleanup(func() { slackLim = saved })
// Swap in a cap-1 limiter on the SHARED bridge pool, then saturate it so the next
// handler acquire sheds.
saved := bridgeLim
bridgeLim = newOrgLimiter(1, 1)
t.Cleanup(func() { bridgeLim = saved })
if cb := connectSlack(t, app, "shedorg", "acmecode"); cb.Code != http.StatusFound {
t.Fatalf("connect: %d (%s)", cb.Code, cb.Body)
}
if !slackLim.acquire("shedorg") {
if !bridgeLim.acquire("shedorg") {
t.Fatal("precondition: fill the cap-1 pool")
}
@@ -610,7 +611,7 @@ func TestSlackShedReturnsNon2xxAndDoesNotRecord(t *testing.T) {
}
// The dedupe key was NOT burned: marking it now must be FRESH — else a later
// retry would be deduped away and the message lost forever.
if fresh, err := mounted.State.store.MarkSlackEvent(context.Background(), "EvShed"); err != nil || !fresh {
if fresh, err := mounted.State.store.MarkEvent(context.Background(), "slack", "EvShed"); err != nil || !fresh {
t.Fatalf("shed must NOT record the event_id (fresh=%v err=%v)", fresh, err)
}
// And no reply was posted (the turn never ran).
@@ -630,24 +631,24 @@ func TestSlackTurnPanicRecoveredAndSlotReleased(t *testing.T) {
newApp(t, newKMS(t))
slackBridgeReady(mounted)
saved := slackLim
slackLim = newOrgLimiter(1, 1)
t.Cleanup(func() { slackLim = saved })
saved := bridgeLim
bridgeLim = newOrgLimiter(1, 1)
t.Cleanup(func() { bridgeLim = saved })
const org = "panicorg"
// Simulate the handler acquiring the single slot, then hand a PANICKING turn to
// slackSpawn (which owns the release).
if !slackLim.acquire(org) {
// bridgeSpawn (which owns the release).
if !bridgeLim.acquire(org) {
t.Fatal("precondition: acquire the single slot")
}
slackSpawn(mounted, org, func() { panic("boom in a slack turn") })
bridgeSpawn(mounted, org, func() { panic("boom in a slack turn") })
// The recovered goroutine must release its slot; poll until a fresh acquire
// succeeds. Reaching here at all proves the panic did not crash the process.
released := false
for i := 0; i < 400; i++ {
if slackLim.acquire(org) {
slackLim.release(org)
if bridgeLim.acquire(org) {
bridgeLim.release(org)
released = true
break
}
+4 -4
View File
@@ -15,7 +15,7 @@ import (
// slack_coding.go turns the @hanzo Slack front-door into an ENGINEER: a message
// `@hanzo code: <repo> <task>` (or `/hanzo code: <repo> <task>`) branches off the
// chat-only reply path (slack_events.go slackAgentReply) into a durable coding
// chat reply path (slack_events.go → the shared bridge brain) into a durable coding
// run — a fresh agent works a NATIVE /v1/git repo in a sandbox, pushes a branch,
// opens a native PR work item, and reports back IN THE SAME THREAD. Everything
// that is NOT the `code:` trigger stays on the existing chat path, unchanged.
@@ -33,7 +33,7 @@ import (
const (
// codingTaskTimeout bounds one detached coding run end to end. It is far longer
// than slackAgentTimeout (a chat turn) because a real coding run clones, runs a
// than a chat turn (bridgeAgentTimeout) because a real coding run clones, runs a
// model-driven edit loop, and pushes. Overridable via SLACK_CODING_TIMEOUT_SEC.
codingTaskDefaultTimeout = 25 * time.Minute
// codingDefaultConcurrency / codingDefaultOrgConcurrency bound simultaneous
@@ -57,8 +57,8 @@ const (
var codingRepoRE = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`)
// codingLim bounds detached coding runs (global + per-org), separate from the
// chat-turn pool (slackLim) because a coding run is long-lived. Initialized once
// alongside slackLim in slackBridgeReady.
// shared chat-turn pool (bridgeLim) because a coding run is long-lived. Initialized
// once in slackBridgeReady.
var codingLim *orgLimiter
// codingDispatcher is the assembled coding orchestrator, injected by the
-69
View File
@@ -1,69 +0,0 @@
package integrations
import (
"context"
"fmt"
"time"
)
// Durable Slack event de-duplication, on the existing integrations Store (same
// {DataDir}/integrations.db file). The table is created in the store's migrate()
// (store.go) — fail-loud at Mount, one place. It exists because an agent turn is
// BILLED: a Slack retry (Slack re-delivers an event_id when it does NOT get a 2xx
// within ~3s) must never trigger a SECOND billed run, and because it is DB-backed
// the guarantee survives a process restart.
//
// REPLICA SCOPE (Red M2): this table lives in the per-PROCESS embedded SQLite file
// (Base/SQLite is single-writer per HIP-0302). It therefore dedupes WITHIN one
// process only — the guarantee does NOT span replicas. The unified cloud binary
// runs the integrations store as a single embedded-SQLite writer, so the billed
// Slack webhook path MUST run single-replica; that is the shipping invariant.
// Multi-replica is a follow-up (back this with a shared SETNX store — Valkey /
// Postgres — keyed on event_id). Note the residual is already small: every event
// we actually run or dedup is 2xx-acked (only a capacity SHED returns a retriable
// non-2xx, and a shed never ran, so it is not a double-process), and Slack only
// retries on a non-2xx / timeout — so a duplicate delivery to a second replica
// requires our ack to first exceed Slack's ~3s budget.
// slackEventTTL bounds how long a dedupe row is retained. Slack's event retry
// horizon is minutes; a day is a wide margin, after which a row can no longer
// correspond to a live retry and is safe to reap.
const slackEventTTL = 24 * time.Hour
// MarkSlackEvent is the atomic durable dedupe test-and-set: it inserts event_key
// and returns fresh=true only on the FIRST sighting. A duplicate (a Slack retry of
// the same event_id / slash trigger_id) hits the PRIMARY KEY, the insert affects
// zero rows, and fresh=false. An empty key is non-dedupable (fresh) — callers only
// dedupe non-empty keys onto the billed path. The single INSERT ... ON CONFLICT DO
// NOTHING + RowsAffected IS the single-use proof (no read-then-write race),
// mirroring the store's ConsumeNonce. A genuine DB error is surfaced so the caller
// fails CLOSED (skips the run) rather than risk a double charge.
func (s *Store) MarkSlackEvent(ctx context.Context, key string) (bool, error) {
if key == "" {
return true, nil
}
res, err := s.db.ExecContext(ctx,
`INSERT INTO slack_events (event_key, created_at) VALUES (?, ?)
ON CONFLICT(event_key) DO NOTHING`, key, time.Now().Unix())
if err != nil {
return false, fmt.Errorf("mark slack event: %w", err)
}
n, _ := res.RowsAffected()
return n == 1, nil
}
// GCSlackEvents reaps dedupe rows created before `before` (unix seconds), bounding
// the table's growth. Returns how many rows were removed. Called opportunistically
// from the webhook path so the table cannot accrete without bound.
func (s *Store) GCSlackEvents(ctx context.Context, before int64) (int64, error) {
res, err := s.db.ExecContext(ctx, `DELETE FROM slack_events WHERE created_at < ?`, before)
if err != nil {
return 0, fmt.Errorf("gc slack events: %w", err)
}
n, _ := res.RowsAffected()
return n, nil
}
// staleSlackEventCutoff is the GC horizon: an event row older than the retry
// horizon can never match a live Slack retry, so it is safe to reap.
func staleSlackEventCutoff() int64 { return time.Now().Add(-slackEventTTL).Unix() }
+127 -258
View File
@@ -8,28 +8,37 @@ import (
"net/http"
"net/url"
"os"
"strconv"
"strings"
"sync"
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/agents"
"github.com/zap-proto/zip"
)
// slack_events.go is the @hanzo agent FRONT-DOOR inside the integrations plane:
// the Slack Events webhook + slash command, routed to an on-behalf-of agent run
// and answered in-thread. It CONSUMES the existing Slack OAuth provider
// (slack.go) — the per-org bot token it sealed — and the framework seams
// (OrgForExternalID / TokenFor / ConnectionFor); it never re-implements custody
// or org resolution.
// slack_events.go is the SLACK ADAPTER of the ChatBridge core (bridge.go): the Slack
// Events webhook + slash command. Like every adapter (discord_events.go /
// telegram_events.go / teams_events.go) its three edges are —
//
// ISOLATION BAR: a workspace's events reach ONLY the org that connected that
// Slack team. The org comes ONLY from OrgForExternalID(team_id) — never the
// payload — the reply uses THAT org's bot token (TokenFor), and the run is THAT
// org's agent on behalf of THAT org's linked user. team_id itself is trustworthy
// only because the whole payload is HMAC-verified with SLACK_SIGNING_SECRET first.
// inbound-auth : verifySlackSignature (HMAC-SHA256 over the raw body + replay window)
// parse : routeSlackEvent / parseSlashCommand (pure)
// reply : slackReplier (bot-token chat.post{Message,Ephemeral}) / response_url
//
// — and it delegates the shared middle to the core: the bounded per-org pool
// (bridgeLim), the durable dedupe (store.MarkEvent), org resolution
// (OrgForExternalID — the ISOLATION ROOT), the ONE agent brain (bridgeReply, run
// ON-BEHALF-OF the linked user), and the per-user link. The @hanzo CHAT turn is now
// ONE code path across all four platforms.
//
// ONE Slack-SPECIFIC branch stays, deliberately NOT folded into the chat bridge: a
// `code:` prefix routes to the durable CODING agent (slack_coding.go) — its own pool
// (codingLim) and detached run. The coding agent is a distinct flow from a chat turn.
//
// ISOLATION BAR: a workspace's events reach ONLY the org that connected that Slack
// team. The org comes ONLY from OrgForExternalID("slack", team_id) — never a payload
// field — and team_id is trustworthy only because the whole payload is HMAC-verified
// with SLACK_SIGNING_SECRET first. The reply uses THAT org's bot token (TokenFor);
// the run is THAT org's agent on behalf of THAT org's linked user.
//
// MOUNT HANDOFF (registered in integrations.go's routes(); this file deliberately
// does NOT edit Mount/routes — clean separation):
@@ -40,136 +49,36 @@ import (
// app.Get("/v1/integrations/slack/link/slack", cloud.Handle(s, slackLinkSlack))
// app.Get("/v1/integrations/slack/link/callback", cloud.Handle(s, slackLinkCallback))
// ── bridge process state (bounded pool + single-use link nonces) ────────────
const (
// slackAgentTimeout bounds one async agent turn end to end (org resolve +
// run + Slack post). Generous: a run executes a real model completion.
slackAgentTimeout = 110 * time.Second
// slackDefaultConcurrency caps simultaneous agent turns ACROSS ALL orgs so a
// workspace insider cannot exhaust goroutines/FDs by bursting @hanzo.
// Overridable via SLACK_AGENT_CONCURRENCY.
slackDefaultConcurrency = 32
// slackDefaultOrgConcurrency caps simultaneous agent turns for a SINGLE org (a
// fraction of the global pool) so one tenant cannot starve the others.
// Overridable via SLACK_AGENT_ORG_CONCURRENCY.
slackDefaultOrgConcurrency = 8
// slackMaxBody bounds the webhook body we read + sign over. Slack payloads are
// small; a hostile/oversized body can neither exhaust memory nor slip past the
// HMAC (we sign exactly what we read).
slackMaxBody = 1 << 20 // 1 MiB
)
// slackMaxBody bounds the webhook body we read + sign over, AND every Slack Web API
// response we read. Slack payloads are small; a hostile/oversized body can neither
// exhaust memory nor slip past the HMAC (we sign exactly what we read).
const slackMaxBody = 1 << 20 // 1 MiB
var (
slackBridgeOnce sync.Once
slackLim *orgLimiter // bounded agent-turn pool: total cap + PER-ORG sub-limit
slackUsedStates *seenSet // single-use link-state nonces (process-lifetime)
slackUsedStates *seenSet // single-use link-state nonces (process-lifetime)
)
// slackBridgeReady lazily initializes the bridge's process state: the bounded
// agent-turn pool (global cap + per-org sub-limit) and the single-use link
// seen-set. The durable dedupe table is created in the store's migrate() at Mount,
// so nothing store-scoped happens here. Cheap + idempotent; every Slack handler
// calls it first.
// slackBridgeReady lazily initializes the Slack adapter's OWN process state the
// coding pool (codingLim; the coding agent is a DISTINCT flow) and the single-use
// link-state seen-set — plus the shared bridge state (bridgeReady: the bounded chat
// pool + link seen-set). The durable dedupe table is created in the store's migrate()
// at Mount. Cheap + idempotent; every Slack handler calls it first.
func slackBridgeReady(s *cloud.Service[state]) {
bridgeReady()
slackBridgeOnce.Do(func() {
slackLim = newOrgLimiter(slackAgentConcurrency(), slackOrgConcurrency())
codingLim = newOrgLimiter(codingConcurrency(), codingOrgConcurrency())
slackUsedStates = newSeenSet(time.Duration(slackLinkTTLSec) * time.Second)
})
}
// slackSpawn runs an ALREADY-SLOTTED agent turn in a recovered goroutine. The
// webhook handler acquires the pool slot SYNCHRONOUSLY (slackLim.acquire) BEFORE
// recording the dedupe key, so a capacity shed can return a retriable non-2xx
// without burning the event_id (Red M-1); the slotted turn is then handed here.
// Two guarantees: (1) the slot is released on every exit, and (2) a panic anywhere
// in the turn (handleSlack* → slackAgentReply → agents.RunOnBehalf — a large
// surface over UNTRUSTED Slack input) is CONTAINED. On the SHARED multi-tenant
// cloud binary this is non-negotiable: middleware.Recover() wraps only the sync
// request goroutine, so an unrecovered panic here would crash EVERY tenant and
// subsystem (Red M-2). The recover defer is registered LAST so it runs FIRST
// (LIFO), and release still runs after it — a panicking turn frees its slot.
func slackSpawn(s *cloud.Service[state], org string, run func()) {
go func() {
defer slackLim.release(org)
defer slackRecover(s, org)
run()
}()
}
// slackRecover contains a panic in an async agent turn so it can never crash the
// shared cloud process. Called ONLY as a deferred func (recover must be a direct
// call in the deferred function).
func slackRecover(s *cloud.Service[state], org string) {
if r := recover(); r != nil {
s.Log.Error("slack: agent turn panic (recovered)", "org", org, "err", r)
}
}
// orgLimiter bounds concurrent agent turns two ways: a GLOBAL cap (total in-flight
// across all orgs) AND a PER-ORG cap (max in-flight for any single org). Data /
// token / billing isolation already holds via the resolved org; this adds the
// AVAILABILITY isolation that stops one tenant exhausting the shared worker pool.
type orgLimiter struct {
mu sync.Mutex
inflight map[string]int
perOrg int
global chan struct{}
}
func newOrgLimiter(global, perOrg int) *orgLimiter {
if global < 1 {
global = 1
}
if perOrg < 1 {
perOrg = 1
}
if perOrg > global {
perOrg = global
}
return &orgLimiter{inflight: make(map[string]int), perOrg: perOrg, global: make(chan struct{}, global)}
}
// acquire takes one global + one per-org slot for org, non-blocking. It returns
// false (nothing acquired, no slot leaked) when the org is at its per-org cap OR
// the global pool is full — the per-org check precedes the global take, and the
// global take only bumps the per-org count on a successful send.
func (l *orgLimiter) acquire(org string) bool {
l.mu.Lock()
defer l.mu.Unlock()
if l.inflight[org] >= l.perOrg {
return false
}
select {
case l.global <- struct{}{}:
l.inflight[org]++
return true
default:
return false
}
}
// release returns the org's slot and the global slot. Called exactly once per
// successful acquire.
func (l *orgLimiter) release(org string) {
l.mu.Lock()
if n := l.inflight[org]; n > 1 {
l.inflight[org] = n - 1
} else {
delete(l.inflight, org)
}
l.mu.Unlock()
<-l.global
}
// ── Events webhook ──────────────────────────────────────────────────────────
// slackEvents is the Slack Events API webhook (the app's request_url:
// https://{domain}/v1/integrations/slack/events). It HMAC-verifies the raw body,
// answers the url_verification challenge, and routes @mentions / DMs to an
// on-behalf-of agent run — acking FAST (empty 200) and doing the billed work
// async under the bounded pool, deduped durably on event_id.
// answers the url_verification challenge, and routes @mentions / DMs — acking FAST
// (empty 200) and doing the billed work async on the bridge under the bounded pool,
// deduped durably on event_id. A `code:` prompt branches to the coding flow.
func slackEvents(s *cloud.Service[state], c *zip.Ctx) error {
slackBridgeReady(s)
secret := slackSigningSecret()
@@ -187,42 +96,61 @@ func slackEvents(s *cloud.Service[state], c *zip.Ctx) error {
return c.String(http.StatusOK, d.Challenge)
case slackRouteAgent:
// ISOLATION ROOT, resolved SYNC (so the per-org limiter keys on the real
// tenant and a shed happens BEFORE anything is recorded): org comes ONLY
// from the install→org map for the Slack-verified team_id. An event for a
// team no org connected is dropped (nothing to do — not a shed).
// tenant and a shed happens BEFORE anything is recorded): org comes ONLY from
// the install→org map for the Slack-verified team_id. An event for a team no
// org connected is dropped (nothing to do — not a shed).
org, ok := OrgForExternalID("slack", d.TeamID)
if !ok {
s.Log.Warn("slack: event for unconnected team", "team", d.TeamID)
return c.NoContent(http.StatusOK)
}
// SHED BEFORE the dedupe write (Red M-1). Try to acquire a pool slot first;
// if the pool is full, record NOTHING and return a retriable NON-2xx — the
// turn never ran, so no event_id is burned and Slack re-delivers when a slot
// frees (no lost @mention, no double-run).
if !slackLim.acquire(org) {
// Echo-loop guard (belt-and-suspenders with the route-level bot_id drop): drop
// the bot's OWN message using THIS org's recorded bot user id, before a pool
// slot or a dedupe row is spent. A bot echo never triggers a run either way.
if conn, ok := ConnectionFor(org, "slack"); ok && conn.BotUserID != "" && d.User == conn.BotUserID {
return c.NoContent(http.StatusOK)
}
// SHED BEFORE the dedupe write (Red M-1). Try to acquire a pool slot first; if
// the pool is full, record NOTHING and return a retriable NON-2xx — the turn
// never ran, so no event_id is burned and Slack re-delivers when a slot frees
// (no lost @mention, no double-run).
if !bridgeLim.acquire(org) {
s.Log.Warn("slack: at capacity, shedding for retry", "org", org)
return zip.Errorf(http.StatusTooManyRequests, "slack agent pool at capacity")
}
// Slot held. DURABLE dedupe (BILLED path): a Slack retry of an event that
// already ran must never double-run. Release the slot on every path that
// does NOT dispatch. Fail CLOSED on a dedupe error (skip) rather than risk a
// double charge.
fresh, err := s.State.store.MarkSlackEvent(c.Context(), slackEventKey(raw))
// already ran must never double-run. Release the slot on every path that does
// NOT dispatch. Fail CLOSED on a dedupe error (skip) rather than risk a double
// charge.
key := slackEventKey(raw)
fresh, err := s.State.store.MarkEvent(c.Context(), "slack", key)
if err != nil {
slackLim.release(org)
bridgeLim.release(org)
s.Log.Warn("slack: event dedupe error, skipping", "err", err)
return c.NoContent(http.StatusOK)
}
if !fresh {
slackLim.release(org)
bridgeLim.release(org)
return c.NoContent(http.StatusOK)
}
// Opportunistic GC so the dedupe table cannot grow without bound.
if _, gerr := s.State.store.GCSlackEvents(c.Context(), staleSlackEventCutoff()); gerr != nil {
if _, gerr := s.State.store.GCEvents(c.Context(), staleEventCutoff()); gerr != nil {
s.Log.Warn("slack: dedupe gc", "err", gerr)
}
route := d
slackSpawn(s, org, func() { handleSlackAgent(s, org, route) })
// CODING is a DISTINCT flow (its own pool + detached run); everything else is a
// chat turn on the shared bridge. Either way the pool slot is held only for the
// short synchronous hand-off (the coding run detaches under codingLim).
if codingText, isCoding := codingIntent(route.Text); isCoding {
bridgeSpawn(s, org, func() { slackCodingEvent(s, org, route, codingText) })
return c.NoContent(http.StatusOK)
}
in := Inbound{
Provider: "slack", ExternalID: route.TeamID, User: route.User,
Channel: route.Channel, ThreadID: route.ThreadTS, Text: route.Text, DedupeKey: key,
}
reply := slackReplier(s, org, route.Channel, route.ThreadTS, route.User)
bridgeSpawn(s, org, func() { runBridgeTurn(s, org, in, reply) })
return c.NoContent(http.StatusOK)
default: // slackRouteAck / slackRouteIgnore — valid but nothing to act on
return c.NoContent(http.StatusOK)
@@ -234,7 +162,7 @@ func slackEvents(s *cloud.Service[state], c *zip.Ctx) error {
// slackCommands handles a Slack slash command (application/x-www-form-urlencoded)
// at https://{domain}/v1/integrations/slack/commands. Same HMAC gate; deduped on
// trigger_id; acks within Slack's 3s budget (empty 200) and posts the answer
// asynchronously via the command's response_url.
// asynchronously via the command's response_url on the bridge.
func slackCommands(s *cloud.Service[state], c *zip.Ctx) error {
slackBridgeReady(s)
secret := slackSigningSecret()
@@ -251,25 +179,25 @@ func slackCommands(s *cloud.Service[state], c *zip.Ctx) error {
return zip.ErrBadRequest("missing team_id or user_id")
}
// Resolve the org SYNC (may be "" — a workspace whose Hanzo connection was
// removed; bounded under the same limiter, handled in handleSlackSlash), then
// SHED before the dedupe write (Red M-1), same order as the events path.
// removed; handled in slackSlashTurn), then SHED before the dedupe write (Red
// M-1), same order as the events path.
org, _ := OrgForExternalID("slack", team)
if !slackLim.acquire(org) {
if !bridgeLim.acquire(org) {
s.Log.Warn("slack: at capacity, shedding slash", "org", org)
return zip.Errorf(http.StatusTooManyRequests, "slack agent pool at capacity")
}
fresh, err := s.State.store.MarkSlackEvent(c.Context(), triggerID)
fresh, err := s.State.store.MarkEvent(c.Context(), "slack", triggerID)
if err != nil {
slackLim.release(org)
bridgeLim.release(org)
s.Log.Warn("slack: slash dedupe error, skipping", "err", err)
return c.NoContent(http.StatusOK)
}
if !fresh {
slackLim.release(org)
bridgeLim.release(org)
return c.NoContent(http.StatusOK)
}
route := slackRoute{Kind: slackRouteAgent, TeamID: team, Channel: channel, User: user, Text: text}
slackSpawn(s, org, func() { handleSlackSlash(s, org, route, responseURL) })
in := Inbound{Provider: "slack", ExternalID: team, User: user, Channel: channel, Text: text}
bridgeSpawn(s, org, func() { slackSlashTurn(s, org, in, responseURL) })
return c.NoContent(http.StatusOK)
}
@@ -290,117 +218,72 @@ func parseSlashCommand(raw []byte) (team, channel, user, text, responseURL, trig
return
}
// ── the bridge (per-org isolated) ───────────────────────────────────────────
// ── Slack dispatch: chat via the bridge, coding via its own flow ────────────
// handleSlackAgent answers an @mention / DM for a PRE-RESOLVED org (the ISOLATION
// ROOT, resolved in the sync webhook path via OrgForExternalID on the
// Slack-verified team_id — never a client value). It skips the bot's own echo,
// fetches THAT org's bot token (the reply sink), and posts the agent's answer — or,
// when the user is unlinked, the account-link prompt EPHEMERALLY (so a link URL
// never reaches a whole channel) — into the SAME thread.
func handleSlackAgent(s *cloud.Service[state], org string, d slackRoute) {
ctx, cancel := context.WithTimeout(context.Background(), slackAgentTimeout)
defer cancel()
// Echo-loop guard: drop the bot's OWN messages using THIS org's recorded bot
// user id (the CTO seam), belt-and-suspenders with the route-level bot_id drop.
if conn, ok := ConnectionFor(org, "slack"); ok && conn.BotUserID != "" && d.User == conn.BotUserID {
return
// slackReplier is the Slack Events reply seam handed to runBridgeTurn: it fetches
// THIS org's bot token (the isolation-scoped reply sink) and posts the agent's
// answer in-thread, or — for the (sensitive) account-link prompt — EPHEMERALLY so a
// link URL never reaches a whole channel. Bound to the resolved org (never a payload
// field), so the reply can only ever use the connecting org's token.
func slackReplier(s *cloud.Service[state], org, channel, threadTS, user string) replyFunc {
return func(ctx context.Context, text string, ephemeral bool) error {
tok, err := TokenFor(ctx, org, "slack", slackBotTokenSecret)
if err != nil {
return err
}
if ephemeral {
return slackPostEphemeral(ctx, string(tok), channel, user, text)
}
return slackPostThread(ctx, string(tok), channel, threadTS, text)
}
}
// slackCodingEvent runs the @mention/DM CODING path for a PRE-RESOLVED org: it
// fetches THIS org's bot token (the reply sink) and hands off to slack_coding.go,
// which owns the parse, the link check, its own bounded pool (codingLim), and the
// detached run. Coding is deliberately NOT folded into the chat bridge.
func slackCodingEvent(s *cloud.Service[state], org string, d slackRoute, codingText string) {
ctx, cancel := context.WithTimeout(context.Background(), bridgeAgentTimeout)
defer cancel()
// Echo-loop guard already applied in the sync path; the bot token is the reply
// sink both the ack and the result card post through.
tok, err := TokenFor(ctx, org, "slack", slackBotTokenSecret)
if err != nil {
s.Log.Warn("slack: bot token fetch", "team", d.TeamID, "err", err)
return
}
// A `code:` prefix branches off the chat-only reply into a durable coding run
// (fresh sandbox agent → native git branch → PR, reported back in-thread).
// Everything else stays on the existing chat path below, unchanged.
if codingText, isCoding := codingIntent(d.Text); isCoding {
handleSlackCoding(s, ctx, org, string(tok), d.TeamID, d.Channel, d.ThreadTS, d.User, codingText)
return
}
reply, linkPrompt := slackAgentReply(s, ctx, org, d.TeamID, d.User, d.Text)
if reply == "" {
return
}
if linkPrompt {
if err := slackPostEphemeral(ctx, string(tok), d.Channel, d.User, reply); err != nil {
s.Log.Warn("slack: ephemeral post", "team", d.TeamID, "err", err)
}
return
}
if err := slackPostThread(ctx, string(tok), d.Channel, d.ThreadTS, reply); err != nil {
s.Log.Warn("slack: thread post", "team", d.TeamID, "err", err)
}
handleSlackCoding(s, ctx, org, string(tok), d.TeamID, d.Channel, d.ThreadTS, d.User, codingText)
}
// handleSlackSlash runs the agent for a slash command (org PRE-RESOLVED in the
// sync path) and delivers the reply via the (host-pinned) response_url. An answer
// goes in_channel; a link prompt goes ephemeral (only the invoker sees it). An
// empty org means the workspace's Hanzo connection was removed.
func handleSlackSlash(s *cloud.Service[state], org string, d slackRoute, responseURL string) {
ctx, cancel := context.WithTimeout(context.Background(), slackAgentTimeout)
// slackSlashTurn is the async slash body dispatched on the bridge. An empty org means
// the workspace's Hanzo connection was removed. A `code:` prompt branches to the
// coding flow (slack_coding.go). Otherwise it runs the ONE agent brain (bridgeReply)
// and delivers via the (host-pinned) response_url: an answer goes in_channel; the
// account-link prompt goes ephemeral (only the invoker sees it).
func slackSlashTurn(s *cloud.Service[state], org string, in Inbound, responseURL string) {
ctx, cancel := context.WithTimeout(context.Background(), bridgeAgentTimeout)
defer cancel()
if org == "" {
_ = slackPostResponseURL(ctx, responseURL, "ephemeral", "This Slack workspace isn't connected to Hanzo yet.")
return
}
// A `code:` prefix branches into a durable coding run (see slack_coding.go).
if codingText, isCoding := codingIntent(d.Text); isCoding {
handleSlackSlashCoding(s, ctx, org, d.TeamID, d.Channel, d.User, codingText, responseURL)
if codingText, isCoding := codingIntent(in.Text); isCoding {
handleSlackSlashCoding(s, ctx, org, in.ExternalID, in.Channel, in.User, codingText, responseURL)
return
}
reply, linkPrompt := slackAgentReply(s, ctx, org, d.TeamID, d.User, d.Text)
if reply == "" {
text, ephemeral := bridgeReply(s, ctx, org, in.Provider, in.ExternalID, in.User, in.Text)
if text == "" {
return
}
responseType := "in_channel"
if linkPrompt {
if ephemeral {
responseType = "ephemeral"
}
if err := slackPostResponseURL(ctx, responseURL, responseType, reply); err != nil {
s.Log.Warn("slack: slash reply", "team", d.TeamID, "err", err)
if err := slackPostResponseURL(ctx, responseURL, responseType, text); err != nil {
s.Log.Warn("slack: slash reply", "team", in.ExternalID, "err", err)
}
}
// slackAgentReply is the ONE agent brain, shared by the @mention/DM and slash
// paths. It resolves the caller's linked Hanzo identity and either runs the agent
// ON BEHALF OF them IN-PROCESS (RunOnBehalf — no gateway hop) returning the
// model's answer, or, when unlinked, returns a short prompt carrying the link URL.
// linkPrompt reports whether the reply is the (sensitive) link prompt — the caller
// MUST deliver those ephemerally. Every returned string is safe to post; internal
// errors are logged (never a token) and surfaced as a terse message.
func slackAgentReply(s *cloud.Service[state], ctx context.Context, org, teamID, slackUser, text string) (reply string, linkPrompt bool) {
link, linked, err := getSlackUserLink(s, org, slackUser)
if err != nil {
s.Log.Warn("slack: user link lookup", "team", teamID, "err", err)
return "Sorry — I couldn't reach your Hanzo account just now. Please try again shortly.", false
}
if !linked {
u, serr := slackLinkURL(s, teamID, slackUser)
if serr != nil {
s.Log.Error("slack: link url", "team", teamID, "err", serr)
return "Connect your Hanzo account to use @hanzo.", true
}
return "Connect your Hanzo account to use @hanzo: " + u, true
}
// IN-PROCESS on-behalf-of run: org (isolation gate) + the linked user's Hanzo
// subject drive billing/attribution; org is the tenant + balance. No bearer, no
// gateway hop. The agent ref is env-set (SLACK_AGENT_REF, default "hanzo").
run, rerr := agents.RunOnBehalf(ctx, org, link.Subject, slackAgentRef(), text)
if rerr != nil {
s.Log.Warn("slack: agent run", "team", teamID, "err", rerr) // never logs a token
return "Sorry — the agent hit an error handling that. Please try again.", false
}
if run.Status != "ok" {
return "Sorry — the agent hit an error handling that. Please try again.", false
}
if strings.TrimSpace(run.Output) == "" {
return "(the agent returned an empty response)", false
}
return run.Output, false
}
// ── event routing (pure) ────────────────────────────────────────────────────
type slackRouteKind int
@@ -658,6 +541,9 @@ func slackPostResponseURL(ctx context.Context, responseURL, responseType, text s
func slackSigningSecret() string { return strings.TrimSpace(os.Getenv("SLACK_SIGNING_SECRET")) }
// slackAgentRef resolves the agent the Slack CODING flow runs (slack_coding.go). The
// CHAT path resolves its agent through the bridge (bridgeAgentRef("slack")); this
// stays for the coding path, unchanged: SLACK_AGENT_REF, default "hanzo".
func slackAgentRef() string {
if v := strings.TrimSpace(os.Getenv("SLACK_AGENT_REF")); v != "" {
return v
@@ -665,27 +551,10 @@ func slackAgentRef() string {
return "hanzo"
}
func slackAgentConcurrency() int {
if v, err := strconv.Atoi(strings.TrimSpace(os.Getenv("SLACK_AGENT_CONCURRENCY"))); err == nil && v > 0 {
return v
}
return slackDefaultConcurrency
}
// slackOrgConcurrency caps how many agent turns a SINGLE org may run concurrently
// (Red M1 availability isolation) — a fraction of the global pool, so one tenant
// can't monopolize it. Overridable via SLACK_AGENT_ORG_CONCURRENCY; clamped to the
// global cap in newOrgLimiter.
func slackOrgConcurrency() int {
if v, err := strconv.Atoi(strings.TrimSpace(os.Getenv("SLACK_AGENT_ORG_CONCURRENCY"))); err == nil && v > 0 {
return v
}
return slackDefaultOrgConcurrency
}
// slackReadBody returns the exact raw request body the HMAC must be computed over,
// bounded to slackMaxBody. fiber's transport already bounds the body; this is the
// local belt-and-suspenders at the crypto boundary.
// local belt-and-suspenders at the crypto boundary. Shared: every adapter's webhook
// reads its raw body through here.
func slackReadBody(c *zip.Ctx) []byte {
b := c.Body()
if len(b) > slackMaxBody {
-82
View File
@@ -7,7 +7,6 @@ import (
"encoding/hex"
"strconv"
"strings"
"sync"
"time"
)
@@ -153,84 +152,3 @@ func verifySlackLink(key []byte, state string, now int64) (teamID, slackUserID,
}
return parts[0], parts[1], n, true
}
// ── in-process single-use seen-set (link-state nonces) ──────────────────────
// seenSet is an age-based single-use / seen-set with an atomic test-and-set. It
// backs the per-user link's single-use guarantee: a signed link state is redeemed
// exactly once within its TTL.
//
// SCOPE: per-PROCESS. In a multi-replica deployment the single-use guarantee here
// is DEFENSE-IN-DEPTH: the PRIMARY single-use guarantee is Slack's own server-side
// single-use OAuth `code` (a second exchange of the same code fails at Slack) AND
// hanzo.id's single-use OIDC `code`, plus the state's HMAC + browser-bound cookie
// + short TTL. Eviction is strictly age-based (a within-TTL entry is NEVER
// evicted), which is what forbids an evict-then-replay attack; memory is bounded
// temporally, not by count.
type seenSet struct {
mu sync.Mutex
ttl time.Duration
at map[string]time.Time
order []string // insertion order, for age-based pruning
}
func newSeenSet(ttl time.Duration) *seenSet {
return &seenSet{ttl: ttl, at: make(map[string]time.Time)}
}
// prune drops expired entries oldest-first, stopping at the first still-fresh one.
// Caller holds the lock.
func (s *seenSet) prune(now time.Time) {
i := 0
for ; i < len(s.order); i++ {
k := s.order[i]
t, ok := s.at[k]
if !ok {
continue // already removed via a re-insert
}
if now.Sub(t) > s.ttl {
delete(s.at, k)
} else {
break
}
}
if i > 0 {
s.order = append(s.order[:0], s.order[i:]...)
}
}
// seenAndAdd atomically tests-and-sets: returns true if k was already seen (a
// duplicate/replay); otherwise records it and returns false. The empty key is
// non-dedupable (always unique). `now` zero → time.Now.
func (s *seenSet) seenAndAdd(k string, now time.Time) bool {
if k == "" {
return false
}
if now.IsZero() {
now = time.Now()
}
s.mu.Lock()
defer s.mu.Unlock()
s.prune(now)
if _, ok := s.at[k]; ok {
return true
}
s.at[k] = now
s.order = append(s.order, k)
return false
}
// ── helpers ──────────────────────────────────────────────────────────────────
func hmacB64URL(key []byte, payload string) string {
mac := hmac.New(sha256.New, key)
mac.Write([]byte(payload))
return base64.RawURLEncoding.EncodeToString(mac.Sum(nil))
}
func abs64(x int64) int64 {
if x < 0 {
return -x
}
return x
}
+1 -11
View File
@@ -84,18 +84,8 @@ CREATE TABLE IF NOT EXISTS oauth_nonces (
);
CREATE INDEX IF NOT EXISTS ix_nonces_created ON oauth_nonces(created_at);
-- slack_events is the Slack agent bridge's durable event-dedupe table (see
-- slack_dedupe.go). Created here in migrate() fail-loud at Mount, one place
-- so the billed webhook path never runs against a missing table (a lazy first-use
-- ensure could half-init and permanently disable the path).
CREATE TABLE IF NOT EXISTS slack_events (
event_key TEXT PRIMARY KEY,
created_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS ix_slack_events_created ON slack_events(created_at);
-- bridge_events is the ChatBridge's durable, provider-keyed event-dedupe table
-- (see bridge_dedupe.go) shared by every non-Slack platform (Teams/Discord/Telegram).
-- (see bridge_dedupe.go) shared by EVERY chat platform (Slack/Teams/Discord/Telegram).
-- Created here in migrate() fail-loud at Mount, one place so the billed webhook
-- path never runs against a missing table (a lazy first-use ensure could half-init
-- and permanently disable the path). PK is (provider, event_key) so platform id
+7 -6
View File
@@ -6,10 +6,11 @@ import (
luxlog "github.com/luxfi/log"
)
// TestReaderReadOnlyRoundTrip proves the reader HA path: a store written by the
// writer (writable, keyed) can be reopened READ-ONLY with the lock guard bypassed
// and its secrets read back — without taking the exclusive write lock. This is
// the mechanism that lets read replicas serve KMS reads off a restored store.
// TestReaderReadOnlyRoundTrip proves the reader HA path: a per-org file written by
// the writer (keyed) can be reopened in READ-ONLY mode and its secrets read back,
// while mutations fail closed. Per-org SQLite is WAL-shareable (no exclusive
// opener lock), so a reader can serve KMS reads off the shared/restored files
// locally — it no longer needs to reverse-proxy KMS to the writer.
func TestReaderReadOnlyRoundTrip(t *testing.T) {
dir := t.TempDir()
log := luxlog.NewNoOpLogger()
@@ -27,7 +28,7 @@ func TestReaderReadOnlyRoundTrip(t *testing.T) {
t.Fatalf("writer Close: %v", err)
}
// Reader: reopen the SAME store read-only (BypassLockGuard), same key.
// Reader: reopen the SAME per-org store read-only, same key.
r, err := New(Config{DataDir: dir, MasterKeyB64: key, ReadOnly: true}, log)
if err != nil {
t.Fatalf("reader New (read-only): %v", err)
@@ -65,7 +66,7 @@ func TestReaderWithoutRestoredStoreFailsClosed(t *testing.T) {
func TestReaderWithoutKeyFailsClosed(t *testing.T) {
dir := t.TempDir()
log := luxlog.NewNoOpLogger()
// Seed an encrypted store so storeExistsOnDisk is true.
// Seed a per-org store so a restored store exists on disk.
w, err := New(Config{DataDir: dir, MasterKeyB64: b64key(t, 0x5A)}, log)
if err != nil {
t.Fatalf("seed: %v", err)
+55 -60
View File
@@ -1,94 +1,89 @@
package kms
// Concurrent-open invariant (HA carve evidence — a REGRESSION GUARD, not a feature).
// Concurrent-open invariant (HORIZONTAL-SCALE evidence — a REGRESSION GUARD).
//
// blue_readonly_test.go proves the SEQUENTIAL reader path (write, CLOSE the
// writer, THEN reopen RO). That is a red herring for HA: in a shared-PVC
// same-node topology the writer pod holds the ZapDB store open for WRITE (an
// actively-growing memtable WAL) while a reader pod would open the SAME on-disk
// files. This test proves that scenario is NOT supported and, deliberately,
// asserts the FAILURE so the constraint is enforced in CI:
// This test replaces the former ZapDB probe, which ASSERTED that a second opener
// of the single embedded ZapDB (Badger fork) store FAILED — the exclusive OS lock
// that pinned cloud to replicas=1 and forced the reader tier to reverse-proxy KMS.
// The KMS store is now PER-ORG SQLite ({DataDir}/orgs/{org}/kms.db via cloud.OrgDB
// → cek), which has NO single-opener lock. This test pins the property that makes
// cloud horizontally scalable OUT OF THE BOX:
//
// Opening a live ZapDB (Badger fork) store READ-ONLY while the writer is
// mid-write fails with "Log truncate required to run DB" — Badger's RO open
// replays the current memtable WAL, finds it partially written
// (end offset < preallocated size), and REFUSES to truncate it (truncation
// is a write, forbidden in RO mode). There is no torn read; there is no open.
// 1. Two independent Clients open the SAME data dir CONCURRENTLY and both work —
// there is no exclusive lock, so a second pod can open the store (impossible
// with the old ZapDB store).
// 2. Distinct orgs land in distinct files, so two "pods" writing DIFFERENT
// tenants never contend — the file IS the tenant boundary.
// 3. The SAME org's file is openable by a second handle (SQLite is WAL-shareable
// for reads), which is what lets a reader serve KMS locally instead of
// proxying. Concurrent WRITERS to one org still require the consistent-hash
// org→pod routing documented in the package report; this guard is about the
// absence of the hard OS lock, not a license for uncoordinated writes.
//
// CONSEQUENCE (the design this guards): a reader-role pod must NOT open the KMS
// ZapDB store off the live writer's PVC. The KMS store is the ONE cloud store
// that is NOT concurrently shareable (unlike the audit SQLite store — see
// audit/shareability_probe_test.go — which shares cleanly over WAL). Therefore a
// reader serves KMS by REVERSE-PROXYING /v1/kms/* (and every mutation +
// /v1/admin/*) to the writer, never by opening the store locally. If a future
// zapdb release makes live concurrent RO-open work, THIS TEST WILL FAIL — that is
// the signal to revisit the reader-serves-KMS-locally option.
// If a future change reintroduces a single global, OS-locked KMS store, THIS TEST
// WILL FAIL — the signal that the replicas=1 constraint has crept back in.
//
// Run:
// CGO_ENABLED=0 GOWORK=off GOFLAGS=-mod=mod go test ./clients/kms/ -run ConcurrentOpen -v
import (
"fmt"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
luxlog "github.com/luxfi/log"
)
func TestConcurrentOpen_LiveWriterStoreIsNotROShareable(t *testing.T) {
func TestConcurrentOpen_PerOrgSQLiteHasNoExclusiveLock(t *testing.T) {
dir := t.TempDir()
log := luxlog.NewNoOpLogger()
key := b64key(t, 0x5A)
// Writer: create the encrypted store and seed baseline secrets, then keep
// writing so the current memtable WAL is genuinely mid-flight (not flushed,
// not closed) when the reader attempts to open.
w, err := New(Config{DataDir: dir, MasterKeyB64: key}, log)
// Two independent Clients over the SAME data dir — the two-pods-one-PVC shape.
// With the old ZapDB store the second open would fail on the exclusive lock;
// per-org SQLite has none, so both open.
a, err := New(Config{DataDir: dir, MasterKeyB64: key}, log)
if err != nil {
t.Fatalf("writer New: %v", err)
t.Fatalf("client A New: %v", err)
}
defer w.Close()
for i := 0; i < 200; i++ {
if err := w.Put("/orgs/acme", fmt.Sprintf("K%d", i), "default", []byte(fmt.Sprintf("v%d", i))); err != nil {
t.Fatalf("writer seed Put %d: %v", i, err)
}
defer a.Close()
b, err := New(Config{DataDir: dir, MasterKeyB64: key}, log)
if err != nil {
t.Fatalf("client B New (second opener must succeed — no exclusive lock): %v", err)
}
defer b.Close()
var stop atomic.Bool
// Distinct orgs → distinct files → no cross-tenant contention. Each client
// writes a different org concurrently; both succeed.
var wg sync.WaitGroup
wg.Add(1)
errs := make(chan error, 2)
wg.Add(2)
go func() {
defer wg.Done()
for i := 200; !stop.Load(); i++ {
if err := w.Put("/orgs/acme", fmt.Sprintf("K%d", i), "default", []byte(fmt.Sprintf("v%d", i))); err != nil {
return // writer wound down; not the subject of this assertion
}
time.Sleep(200 * time.Microsecond)
if err := a.Put("/orgs/acme", "K", "prod", []byte("acme-secret")); err != nil {
errs <- fmt.Errorf("A Put acme: %w", err)
}
}()
time.Sleep(50 * time.Millisecond) // ensure the WAL is actively mid-write
// Reader: attempt to open the SAME files READ-ONLY (BypassLockGuard) while the
// writer is live. INVARIANT: this must fail (no torn read, no silent success).
r, err := New(Config{DataDir: dir, MasterKeyB64: key, ReadOnly: true}, log)
stop.Store(true)
wg.Wait()
if err == nil {
if r != nil {
_ = r.Close()
go func() {
defer wg.Done()
if err := b.Put("/orgs/globex", "K", "prod", []byte("globex-secret")); err != nil {
errs <- fmt.Errorf("B Put globex: %w", err)
}
t.Fatal("EXPECTED concurrent RO-open of a live ZapDB writer to FAIL, but it " +
"succeeded. If zapdb now supports live concurrent RO-open, the reader " +
"tier may serve KMS locally instead of proxying — revisit the design.")
}()
wg.Wait()
close(errs)
for e := range errs {
t.Fatalf("concurrent per-org write failed: %v", e)
}
// The failure is the WAL-truncation refusal, confirming Badger's RO open cannot
// coexist with a live writer's unflushed memtable.
if !strings.Contains(err.Error(), "truncate") && !strings.Contains(err.Error(), "Log truncate") {
t.Logf("concurrent RO-open failed (as required) with a different error: %v", err)
// Cross-read: A reads globex (written by B, same shared dir) — the files are
// shared on disk, only the WRITER should be one pod per org.
if got, err := a.Get("/orgs/globex", "K", "prod"); err != nil || string(got) != "globex-secret" {
t.Fatalf("A cross-read globex = %q err=%v, want globex-secret", got, err)
}
t.Logf("INVARIANT HELD: live ZapDB store is NOT RO-shareable (%v) — reader must proxy /v1/kms/*", err)
if got, err := b.Get("/orgs/acme", "K", "prod"); err != nil || string(got) != "acme-secret" {
t.Fatalf("B cross-read acme = %q err=%v, want acme-secret", got, err)
}
t.Log("INVARIANT HELD: per-org SQLite has no exclusive-opener lock — two clients " +
"open the same data dir concurrently; distinct orgs never contend. replicas=1 lifted.")
}
+57 -104
View File
@@ -11,15 +11,18 @@
// calls, mounted onto cloud's Fiber app: JWT-gated, org-scoped
// secrets CRUD + a real health probe + the SPA admin config (mount.go).
//
// STORAGE — luxfi/kms's SecretStore is an embedded ZapDB (github.com/luxfi/zapdb)
// KV opened UNDER CLOUD_DATA_DIR/kms (the RWO PVC where per-tenant SQLite lives),
// so there is no PostgreSQL and no external DB. cloud runs replicas=1/Recreate, so
// the single-writer KV is safe. Secrets are sealed with AES-256-GCM envelope
// encryption (store.Seal: a fresh per-secret DEK sealed under the 32-byte master
// key) BEFORE they hit the store — plaintext never touches disk. The KV itself is
// ALSO opened with ZapDB block-level encryption under the same key (defense in
// depth). See New for the fail-secure open strategy across the health-only↔keyed
// transition.
// STORAGE — sealed secrets persist to PER-ORG SQLite (store.go): each org's
// secrets live in ITS OWN encrypted file {CLOUD_DATA_DIR}/orgs/{org}/kms.db via
// the canonical cloud.OrgDB → cek seam, mirroring clients/finance. This REPLACES
// the single embedded ZapDB KV, whose exclusive OS lock pinned cloud to
// replicas=1: a per-org SQLite file has no single-opener lock, so different pods
// can serve different tenants and cloud scales horizontally (consistent-hash
// org→pod; see the package report). Two layers of at-rest protection, both rooted
// in the SAME env-only master key: (1) each secret is sealed with an AES-256-GCM
// envelope (store.Seal: a fresh per-secret DEK wrapped by the master key) BEFORE
// it reaches SQLite — plaintext never touches disk; (2) cek opens each file
// SQLCipher-encrypted under a per-db DEK wrapped by the master key (defense in
// depth). No PostgreSQL, no external DB, no ZapDB.
//
// BOOTSTRAP — cloud hosting the secret store is a chicken-and-egg: cloud cannot
// fetch its OWN master key from the KMS it hosts. The 32-byte master key is
@@ -55,18 +58,12 @@ import (
"encoding/base64"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/hanzoai/cloud/types"
kmsstore "github.com/luxfi/kms/pkg/store"
luxlog "github.com/luxfi/log"
// zapdb is the canonical Lux embedded KV engine (github.com/luxfi/zapdb). Its
// Go package is still named `badger` (it is a hardened Badger fork), so we
// alias it to zapdb to keep call sites self-documenting: this is ZapDB, NOT
// raw dgraph-io/badger. There is no dgraph-io/badger anywhere in cloud.
zapdb "github.com/luxfi/zapdb"
)
// masterKeyLen is the AES-256 KEK size store.Seal/Open require (32 bytes).
@@ -109,11 +106,10 @@ const (
//
// The zero value is not usable; construct with New.
type Client struct {
db *zapdb.DB // held so Close can release the KV (SecretStore does not expose it)
store *kmsstore.SecretStore
masterKey []byte // 32-byte KEK; nil ⇒ health-only fail-closed mode
mpcAddr string // MPC daemon address; "" ⇒ Sign fails closed
vaultID string // MPC vault id; "" ⇒ Sign fails closed
store *secretStore // per-org SQLite persistence (store.go); no OS lock
masterKey []byte // 32-byte KEK; nil ⇒ health-only fail-closed mode
mpcAddr string // MPC daemon address; "" ⇒ Sign fails closed
vaultID string // MPC vault id; "" ⇒ Sign fails closed
log luxlog.Logger
}
@@ -128,17 +124,12 @@ type Config struct {
MPCAddr string // MPC daemon host:port(,...) — CLOUD_KMS_MPC_ADDR
MPCVaultID string // MPC vault id — CLOUD_KMS_MPC_VAULT_ID
// ReadOnly opens the store in reader mode: READ-ONLY, with the ZapDB lock
// guard BYPASSED so it coexists with the writer's replication stream without
// ever taking the exclusive write lock. Set by the reader HA role. The store
// must already be hydrated on disk (restored from S3/vfs via zapdb-replicate);
// a reader with no restored store fails closed rather than serving nothing.
//
// FRESHNESS CAVEAT: ZapDB (a Badger fork) is single-process for live writes —
// a read-only handle sees the snapshot present at open time, not increments a
// separate restore process applies afterward. Readers therefore periodically
// re-hydrate + reopen (or proxy writes-sensitive reads to the writer). This is
// acceptable for slowly-changing secrets; see the reader deployment notes.
// ReadOnly opens the store in reader mode: mutations fail closed so a replica
// never forks the authoritative writer's state, and a reader with no restored
// store under {DataDir}/orgs fails closed at New rather than serving nothing.
// Set by the reader HA role. Unlike the former ZapDB store, per-org SQLite is
// RO-shareable over WAL, so a reader CAN open the files locally — the reader
// no longer needs to reverse-proxy KMS to the writer (see the package report).
ReadOnly bool
}
@@ -158,71 +149,34 @@ func New(cfg Config, log luxlog.Logger) (*Client, error) {
if dir == "" {
return nil, fmt.Errorf("kms.New: empty DataDir")
}
dbDir := filepath.Join(dir, "kms")
masterKey, keyErr := decodeMasterKey(cfg.MasterKeyB64)
// Store-open strategy, fail-SECURE across the health-only↔keyed transition:
// Fail-SECURE across the health-only↔keyed transition. There is no single store
// to open here: per-org files open LAZILY on first access (store.go), each via
// cloud.OrgDB → cek. So the health-only↔keyed distinction is purely the presence
// of a valid master key (Ready()): with a key, secret ops seal/open and cek
// encrypts each file at rest; without one, every secret op fails closed with
// ErrMasterKeyMissing and NO org file is created (nothing to persist, nothing to
// brick). The former ZapDB KEYREGISTRY brick foot-gun is gone: cek rotates by
// re-wrapping a per-file sidecar (no page is rewritten), and a wrong/absent key
// makes cek.Open FAIL at first access — never a silent plaintext downgrade.
//
// keyed → open the on-disk store ENCRYPTED at rest with the master
// key (WithEncryptionKey), on top of the per-secret Seal
// envelope. zapdb rejects a WRONG key at open (rotation
// without re-encrypt fails closed, not a silent downgrade).
// no key, no store → open an EPHEMERAL IN-MEMORY store, never touching disk.
// no key, store → FAIL: an encrypted store already exists but its key is
// present absent (the operator dropped CLOUD_KMS_MASTER_KEY_REF).
// Refuse loudly rather than silently ignore encrypted data.
//
// Why in-memory for the fresh health-only case (no store yet): a disk-backed
// zapdb opened WITHOUT a key writes a PLAINTEXT KEYREGISTRY. If the operator
// then injects the real key on the next boot, zapdb's registry sanity check
// rejects the now-mismatched registry and zapdb.Open fails PERMANENTLY —
// bricking KMS until the data dir is wiped. Health-only mode can serve no
// secret op anyway (every Get/Put fails closed without the key), so there is
// nothing to persist; an ephemeral in-memory KV lets health/metadata work while
// leaving the on-disk dir untouched, so the first KEYED boot opens a clean
// encrypted store. No unencrypted secret store is ever written to disk.
var opts zapdb.Options
switch {
case keyErr == nil && cfg.ReadOnly:
// Reader HA role: open the hydrated on-disk store READ-ONLY, bypassing the
// lock guard so it never contends with the writer (which owns the RWO PVC)
// or a co-located restore process. No exclusive write lock is ever taken —
// this is what lets N readers run without violating the single-writer
// invariant. The store must already exist (restored from the replication
// stream); refuse to serve if it does not, rather than open an empty store.
if !storeExistsOnDisk(dbDir) {
return nil, fmt.Errorf("kms.New: reader mode but no restored store at %s — hydrate via zapdb-replicate restore before serving KMS reads", dbDir)
// Reader HA role fails CLOSED at New (not at first read) so a mis-provisioned
// reader never boots "healthy" over nothing:
// no master key → cannot decrypt any file at rest → refuse.
// no restored store under {DataDir}/orgs → nothing to serve → refuse.
if cfg.ReadOnly {
if keyErr != nil {
return nil, fmt.Errorf("kms.New: reader mode requires a master key to open the encrypted store: %w", keyErr)
}
opts = zapdb.DefaultOptions(dbDir).WithLogger(nil).
WithEncryptionKey(masterKey).WithIndexCacheSize(16 << 20).
WithReadOnly(true)
opts.BypassLockGuard = true
case keyErr == nil:
if err := os.MkdirAll(dbDir, 0o700); err != nil {
return nil, fmt.Errorf("kms.New: create store dir: %w", err)
if !hasRestoredStore(dir) {
return nil, fmt.Errorf("kms.New: reader mode but no restored store under %s — hydrate before serving KMS reads", filepath.Join(dir, "orgs"))
}
opts = zapdb.DefaultOptions(dbDir).WithLogger(nil).
WithEncryptionKey(masterKey).WithIndexCacheSize(16 << 20)
case cfg.ReadOnly:
// Reader with no master key: cannot decrypt the store at rest. Fail closed
// (health-only) rather than pretend to serve secrets.
return nil, fmt.Errorf("kms.New: reader mode requires a master key to open the encrypted store: %w", keyErr)
case storeExistsOnDisk(dbDir):
// An encrypted store is present but no key was supplied: do not silently
// open a fresh in-memory store and pretend the on-disk secrets are gone.
return nil, fmt.Errorf("kms.New: encrypted store present at %s but no master key configured: %w", dbDir, keyErr)
default:
opts = zapdb.DefaultOptions("").WithLogger(nil).WithInMemory(true)
}
db, err := zapdb.Open(opts)
if err != nil {
return nil, fmt.Errorf("kms.New: open store %s: %w", dbDir, err)
}
c := &Client{
db: db,
store: kmsstore.NewSecretStore(db),
store: newSecretStore(dir, cfg.ReadOnly),
masterKey: masterKey, // nil when keyErr != nil
mpcAddr: strings.TrimSpace(cfg.MPCAddr),
vaultID: strings.TrimSpace(cfg.MPCVaultID),
@@ -231,18 +185,17 @@ func New(cfg Config, log luxlog.Logger) (*Client, error) {
if keyErr != nil {
c.log.Warn("kms master key not configured; secret ops fail closed (health-only mode)", "err", keyErr)
}
return c, nil
}
// storeExistsOnDisk reports whether a zapdb store was already initialized under
// dir. zapdb writes a MANIFEST at the store root on first open, so its presence
// marks an existing (encrypted) store — used to refuse a keyless boot over real
// data rather than silently shadow it with an in-memory KV.
func storeExistsOnDisk(dir string) bool {
if _, err := os.Stat(filepath.Join(dir, "MANIFEST")); err == nil {
return true
// One-time cutover from the legacy embedded ZapDB store to per-org SQLite. Writer
// + keyed only: a reader must not migrate, and the encrypted-at-rest legacy store
// can only be read with the master key. FATAL on error — booting the empty per-org
// store while legacy secrets sit unmigrated would orphan every secret (cloud KMS is
// the source the kms-operator syncs out), so refuse to serve rather than lose them.
if !cfg.ReadOnly && keyErr == nil {
if err := migrateLegacyZapDB(dir, masterKey, c.store, c.log); err != nil {
return nil, fmt.Errorf("kms.New: legacy ZapDB migration: %w", err)
}
}
return false
return c, nil
}
// decodeMasterKey decodes and validates the base64 master key. A key that is
@@ -336,7 +289,7 @@ func (c *Client) Get(path, name, env string) ([]byte, error) {
if err := validCoords(path, name, env); err != nil {
return nil, err
}
sec, err := c.store.Get(path, name, env)
sec, err := c.store.get(path, name, env)
if err != nil {
if errors.Is(err, kmsstore.ErrSecretNotFound) {
return nil, kmsstore.ErrSecretNotFound
@@ -364,7 +317,7 @@ func (c *Client) Put(path, name, env string, value []byte) error {
if err != nil {
return fmt.Errorf("kms: seal secret: %w", err)
}
if err := c.store.Put(sec); err != nil {
if err := c.store.put(sec); err != nil {
return fmt.Errorf("kms: write secret: %w", err)
}
return nil
@@ -376,7 +329,7 @@ func (c *Client) List(path, env string) ([]SecretMeta, error) {
if !ValidSegment(env, maxEnvLen) {
return nil, ErrInvalidKey
}
secs, err := c.store.List(path, env)
secs, err := c.store.list(path, env)
if err != nil {
return nil, fmt.Errorf("kms: list secrets: %w", err)
}
@@ -392,7 +345,7 @@ func (c *Client) Delete(path, name, env string) error {
if err := validCoords(path, name, env); err != nil {
return err
}
if err := c.store.Delete(path, name, env); err != nil {
if err := c.store.del(path, name, env); err != nil {
if errors.Is(err, kmsstore.ErrSecretNotFound) {
return kmsstore.ErrSecretNotFound
}
@@ -468,10 +421,10 @@ const (
// Close releases the embedded store. Safe to call once at shutdown.
func (c *Client) Close() error {
if c == nil || c.db == nil {
if c == nil || c.store == nil {
return nil
}
return c.db.Close()
return c.store.close()
}
// ── ref parsing ──────────────────────────────────────────────────────────────
+5 -3
View File
@@ -50,7 +50,7 @@ func masterKeyB64(t *testing.T) string {
// tests mount exactly kms (the same spec apps.Wire() carries) without linking
// the whole bundle. cfg.Enable still gates it, exactly as in production.
func mountSpecs() []cloud.MountSpec {
return []cloud.MountSpec{{Name: "kms", Mount: cloud.Typed(kms.Mount), OwnsHealth: true}}
return []cloud.MountSpec{{Name: "kms", Mount: kms.Mount, OwnsHealth: true}}
}
// newApp wires BuildDeps + the canonical middleware + MountAll for the kms
@@ -167,7 +167,9 @@ func TestSecretNotStoredInPlaintext(t *testing.T) {
if err := deps.KMS.PutSecret(context.Background(), "svc/DB_URL@main", marker); err != nil {
t.Fatalf("PutSecret: %v", err)
}
// Close so the KV flushes to disk, then scan the store files for the marker.
// Close so the per-org SQLite files flush to disk, then scan every store file
// under the data dir for the marker (per-org files live at {dir}/orgs/*/kms.db;
// an org-less facade ref lands in {dir}/orgs/_platform/kms.db).
if c, ok := deps.KMS.(*kms.Client); ok {
if err := c.Close(); err != nil {
t.Fatalf("close: %v", err)
@@ -175,7 +177,7 @@ func TestSecretNotStoredInPlaintext(t *testing.T) {
}
found := false
root := filepath.Join(dir, "kms")
root := dir
err := filepath.Walk(root, func(p string, info os.FileInfo, err error) error {
if err != nil || info.IsDir() {
return err
+111
View File
@@ -0,0 +1,111 @@
package kms
// migrate.go is the ONE-TIME cutover from the legacy embedded ZapDB KMS store to
// the per-org SQLite store (store.go). The old store held EVERY org's sealed
// secrets in ONE ZapDB KV under {DataDir}/kms, whose exclusive OS lock pinned cloud
// to replicas=1. Deploying the per-org store over live data WITHOUT this migration
// would boot an EMPTY store and orphan every secret — a cluster-wide outage, since
// cloud's KMS is the authoritative source the kms-operator syncs OUT to every
// service's k8s Secret. So New() runs this on the WRITER, keyed, BEFORE serving,
// and treats any error as FATAL (refuse to boot rather than silently lose secrets).
//
// SAFETY. Secrets move SEALED: the legacy VALUE is the JSON-marshaled store.Secret
// carrying the AES-256-GCM ciphertext + ML-KEM wrapped DEK, and it is re-stored
// verbatim — NEVER unsealed here — so no plaintext is exposed and the per-secret
// Seal envelope (AAD-bound to the full /orgs/{org} path) is preserved byte-for-byte;
// a migrated record Opens exactly as before and still fails Open if relocated across
// orgs. On success the legacy dir is RENAMED to {DataDir}/kms.migrated (not deleted)
// so it is never reopened — releasing the ZapDB OS lock for good — while the old
// ciphertext stays recoverable. Idempotent: a prior .migrated marker or an absent
// legacy store is a no-op; dst.put upserts, so a partially-copied prior run resumes.
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
kmsstore "github.com/luxfi/kms/pkg/store"
luxlog "github.com/luxfi/log"
zapdb "github.com/luxfi/zapdb"
)
// legacySecretPrefix mirrors the (unexported) prefix the legacy SecretStore wrote
// every record under — kms/secrets/{path}/{env}/{name}. The stored VALUE is the
// JSON store.Secret (full coordinate + sealed bytes), so migration decodes the
// VALUE and never has to parse the ambiguous key (path itself contains '/').
var legacySecretPrefix = []byte("kms/secrets/")
// migrateLegacyZapDB performs the one-time ZapDB -> per-org SQLite cutover. Called
// by New() for the WRITER, keyed, before serving. Returns nil (no-op) when there is
// nothing to migrate. Any real error is returned so New treats it as FATAL.
func migrateLegacyZapDB(dataDir string, masterKey []byte, dst *secretStore, log luxlog.Logger) error {
legacyDir := filepath.Join(dataDir, "kms")
migratedDir := legacyDir + ".migrated"
// Already cut over (or a fresh deploy that never had a legacy store): no-op.
if _, err := os.Stat(migratedDir); err == nil {
return nil
}
// No legacy ZapDB present — its MANIFEST marks an initialized store: no-op.
if _, err := os.Stat(filepath.Join(legacyDir, "MANIFEST")); err != nil {
return nil
}
if len(masterKey) == 0 {
// The legacy store is encrypted at rest; without the key it cannot be read.
// Never destroy unreadable secrets — refuse so a keyed boot can migrate.
return fmt.Errorf("legacy store present at %s but master key absent — refusing to skip", legacyDir)
}
opts := zapdb.DefaultOptions(legacyDir).WithLogger(nil).
WithEncryptionKey(masterKey).WithIndexCacheSize(16 << 20)
db, err := zapdb.Open(opts)
if err != nil {
return fmt.Errorf("open legacy store %s: %w", legacyDir, err)
}
n, copyErr := copyLegacySecrets(db, dst)
// Close BEFORE renaming so the OS lock is released and the dir is movable.
closeErr := db.Close()
if copyErr != nil {
return fmt.Errorf("copy legacy secrets: %w", copyErr)
}
if closeErr != nil {
return fmt.Errorf("close legacy store: %w", closeErr)
}
// Cutover marker — rename ONLY after every secret landed in SQLite. The old store
// is now archived (never reopened, lock gone) yet the sealed ciphertext survives.
if err := os.Rename(legacyDir, migratedDir); err != nil {
return fmt.Errorf("mark migrated (rename %s): %w", legacyDir, err)
}
log.Info("kms: migrated legacy ZapDB store to per-org SQLite (replicas=1 lock released)",
"secrets", n, "archived", migratedDir)
return nil
}
// copyLegacySecrets streams every sealed secret under the kms/secrets/ prefix from
// the legacy ZapDB into its per-org SQLite file via dst.put. It decodes the VALUE
// (the JSON store.Secret: full coordinate + sealed ciphertext + wrapped DEK), so no
// key parsing and no unsealing occurs. Returns the count copied.
func copyLegacySecrets(db *zapdb.DB, dst *secretStore) (int, error) {
n := 0
err := db.View(func(txn *zapdb.Txn) error {
iopts := zapdb.DefaultIteratorOptions
iopts.Prefix = legacySecretPrefix
it := txn.NewIterator(iopts)
defer it.Close()
for it.Rewind(); it.Valid(); it.Next() {
var sec kmsstore.Secret
if err := it.Item().Value(func(val []byte) error {
return json.Unmarshal(val, &sec)
}); err != nil {
return fmt.Errorf("decode secret %q: %w", it.Item().Key(), err)
}
if err := dst.put(&sec); err != nil {
return fmt.Errorf("write secret %s/%s/%s: %w", sec.Path, sec.Env, sec.Name, err)
}
n++
}
return nil
})
return n, err
}
+132
View File
@@ -0,0 +1,132 @@
package kms
import (
"os"
"path/filepath"
"testing"
kmsstore "github.com/luxfi/kms/pkg/store"
luxlog "github.com/luxfi/log"
zapdb "github.com/luxfi/zapdb"
)
// TestMigrateLegacyZapDB proves the one-time cutover: a SEALED secret in the legacy
// embedded ZapDB lands in its per-org SQLite file, still Opens to the SAME plaintext
// (envelope preserved byte-for-byte), the legacy store is archived (never reopened),
// and a second run is an idempotent no-op.
func TestMigrateLegacyZapDB(t *testing.T) {
dir := t.TempDir()
key := testKey()
const path, name, env, plaintext = "/orgs/acme/ci", "TOKEN", "main", "s3cr3t-value"
seedLegacyZapDB(t, dir, key, path, name, env, plaintext)
dst := newSecretStore(dir, false)
if err := migrateLegacyZapDB(dir, key, dst, luxlog.NewNoOpLogger()); err != nil {
t.Fatalf("migrate: %v", err)
}
// The secret is now in per-org SQLite and still opens to the original plaintext.
sec, err := dst.get(path, name, env)
if err != nil {
t.Fatalf("get after migrate: %v", err)
}
pt, err := kmsstore.Open(key, sec)
if err != nil {
t.Fatalf("open migrated secret: %v", err)
}
if string(pt) != plaintext {
t.Fatalf("plaintext mismatch after migrate: got %q want %q", pt, plaintext)
}
// Legacy store archived; original dir renamed away.
if _, err := os.Stat(filepath.Join(dir, "kms.migrated")); err != nil {
t.Fatalf("legacy store not archived to kms.migrated: %v", err)
}
if _, err := os.Stat(filepath.Join(dir, "kms", "MANIFEST")); err == nil {
t.Fatal("legacy dir should have been renamed away, still present")
}
// Idempotent: a second run is a clean no-op.
if err := migrateLegacyZapDB(dir, key, dst, luxlog.NewNoOpLogger()); err != nil {
t.Fatalf("second migrate (idempotent) failed: %v", err)
}
}
// TestMigrateLegacyZapDB_NoLegacyStore: a fresh deploy with no legacy store is a
// silent no-op (never creates the archive marker on nothing).
func TestMigrateLegacyZapDB_NoLegacyStore(t *testing.T) {
dir := t.TempDir()
dst := newSecretStore(dir, false)
if err := migrateLegacyZapDB(dir, testKey(), dst, luxlog.NewNoOpLogger()); err != nil {
t.Fatalf("migrate over empty dir must be a no-op, got: %v", err)
}
if _, err := os.Stat(filepath.Join(dir, "kms.migrated")); err == nil {
t.Fatal("no legacy store — must not create a .migrated marker")
}
}
// TestMigrateLegacyZapDB_CrossOrgRelocationStillDefended: after migration, the
// AAD-bound envelope invariant holds — a migrated record's sealed bytes, physically
// placed under a DIFFERENT org's coordinate, fail to Open (the swap defense survives
// the cutover).
func TestMigrateLegacyZapDB_CrossOrgRelocationStillDefended(t *testing.T) {
dir := t.TempDir()
key := testKey()
const path, name, env, plaintext = "/orgs/acme/ci", "TOKEN", "main", "acme-only"
seedLegacyZapDB(t, dir, key, path, name, env, plaintext)
dst := newSecretStore(dir, false)
if err := migrateLegacyZapDB(dir, key, dst, luxlog.NewNoOpLogger()); err != nil {
t.Fatalf("migrate: %v", err)
}
sec, err := dst.get(path, name, env)
if err != nil {
t.Fatalf("get: %v", err)
}
// Forge the same sealed bytes onto another org's coordinate; Open must reject it.
relocated := &kmsstore.Secret{
Path: "/orgs/evil/ci", Name: name, Env: env,
Scheme: sec.Scheme, Ciphertext: sec.Ciphertext, WrappedDEK: sec.WrappedDEK,
}
if _, err := kmsstore.Open(key, relocated); err == nil {
t.Fatal("relocated secret Opened — AAD path binding broke across migration")
}
}
// seedLegacyZapDB writes one sealed secret into a legacy ZapDB at {dir}/kms, exactly
// as the pre-per-org KMS did.
func seedLegacyZapDB(t *testing.T, dir string, key []byte, path, name, env, plaintext string) {
t.Helper()
legacyDir := filepath.Join(dir, "kms")
if err := os.MkdirAll(legacyDir, 0o700); err != nil {
t.Fatal(err)
}
opts := zapdb.DefaultOptions(legacyDir).WithLogger(nil).
WithEncryptionKey(key).WithIndexCacheSize(16 << 20)
db, err := zapdb.Open(opts)
if err != nil {
t.Fatalf("open legacy zapdb: %v", err)
}
sec, err := kmsstore.Seal(key, path, name, env, []byte(plaintext))
if err != nil {
_ = db.Close()
t.Fatalf("seal: %v", err)
}
if err := kmsstore.NewSecretStore(db).Put(sec); err != nil {
_ = db.Close()
t.Fatalf("put: %v", err)
}
if err := db.Close(); err != nil {
t.Fatalf("close legacy zapdb: %v", err)
}
}
// testKey is a deterministic 32-byte master key for tests.
func testKey() []byte {
k := make([]byte, masterKeyLen)
for i := range k {
k[i] = byte(i + 1)
}
return k
}
+3 -3
View File
@@ -165,9 +165,9 @@ func newEmbeddedClient(cfg *cloud.Config, log luxlog.Logger) (cloud.KMSClient, e
MasterKeyB64: cfg.KMSMasterKeyRef,
MPCAddr: cfg.KMSMPCAddr,
MPCVaultID: cfg.KMSMPCVaultID,
// Reader HA role opens the KMS store READ-ONLY (BypassLockGuard) off a
// restored replica — never the exclusive write lock. Writer (default) opens
// writable exactly as before.
// Reader HA role opens the per-org KMS files READ-ONLY (mutations fail
// closed); per-org SQLite is WAL-shareable, so no exclusive lock is taken.
// Writer (default) opens writable exactly as before.
ReadOnly: cfg.Role.IsReader(),
}, log)
if err != nil {
+2 -2
View File
@@ -37,8 +37,8 @@ func newDualApp(t *testing.T, mk string) *zip.App {
app.Use(middleware.RequestID())
app.Use(middleware.Logger(deps.Logger))
specs := []cloud.MountSpec{
{Name: "kms", Mount: cloud.Typed(kms.Mount), OwnsHealth: true},
{Name: "admin", Mount: cloud.Typed(admin.Mount)},
{Name: "kms", Mount: kms.Mount, OwnsHealth: true},
{Name: "admin", Mount: admin.Mount},
}
if err := cloud.MountAll(app, specs, cfg, deps); err != nil {
t.Fatalf("MountAll: %v", err)
+143 -198
View File
@@ -1,11 +1,24 @@
package kms
// RED RE-REVIEW — attacking the new 3-way store-open switch in New() and the
// hoisted validCoords. Goal: defeat the fail-secure switch via any
// {key present|absent|wrong} × {store absent|present|MANIFEST-missing|plaintext}
// combination, or find a silent downgrade / brick the switch was meant to close.
// RED RE-REVIEW v2 — attacking the PER-ORG SQLite store-open model that replaces
// the former ZapDB 3-way open switch. The old switch (keyed / no-key-in-memory /
// no-key-fail) and its KEYREGISTRY/MANIFEST sentinels are GONE: secrets now persist
// to {DataDir}/orgs/{org}/kms.db (cloud.OrgDB → cek), opened LAZILY per org, and
// the confidentiality boundary is the per-secret AES-256-GCM Seal envelope. This
// re-review re-attacks the invariants that still matter under the new model:
// - health-only writes NOTHING to disk and does not brick the next keyed boot;
// - a keyless / wrong-key process cannot read, AND does not destroy, on-disk
// sealed secrets (no silent shadow, no data loss);
// - the facade enforces the same coord validation as the HTTP boundary;
// - a legitimate root path stays valid;
// - distinct orgs are PHYSICALLY isolated (separate files), on top of the REST
// authz gate.
// The ZapDB-mechanism attacks (RR4/RR4b MANIFEST-sentinel, RR5 plaintext-brick)
// are retired: that mechanism no longer exists and its foot-gun is eliminated
// (proven positively by TestRR1).
import (
"bytes"
"context"
"crypto/rand"
"encoding/base64"
@@ -14,7 +27,6 @@ import (
"testing"
luxlog "github.com/luxfi/log"
zapdb "github.com/luxfi/zapdb"
)
func rrKey(t *testing.T, fill byte) string {
@@ -35,16 +47,13 @@ func rrRandKey(t *testing.T) string {
return base64.StdEncoding.EncodeToString(k)
}
// RR-1: the fix's headline claim — fresh health-only boot (no key, no store) must
// NOT write ANYTHING to disk (no plaintext KEYREGISTRY, no MANIFEST), so the first
// keyed boot opens a clean encrypted store. Verify the dir stays empty AND the
// subsequent keyed boot succeeds + roundtrips.
// RR-1 (former 8c foot-gun, now ELIMINATED): a fresh health-only boot (no key)
// must write NOTHING to disk — every secret op fails closed — so the first keyed
// boot opens a clean store and roundtrips. No plaintext store, no brick.
func TestRR1_FreshHealthOnlyWritesNothing(t *testing.T) {
root := t.TempDir()
log := luxlog.NewNoOpLogger()
dbDir := filepath.Join(root, "kms")
// Boot 1: no key, no existing store → ephemeral in-memory.
c1, err := New(Config{DataDir: root, MasterKeyB64: ""}, log)
if err != nil {
t.Fatalf("fresh health-only New: %v", err)
@@ -52,28 +61,21 @@ func TestRR1_FreshHealthOnlyWritesNothing(t *testing.T) {
if c1.Ready() {
t.Fatal("health-only must not be Ready")
}
// A write in health-only mode is refused, so nothing lands on disk.
if err := c1.Put("/orgs/x", "K", "default", []byte("v")); err == nil {
t.Fatal("BREACH: health-only Put succeeded (would write plaintext-less but a store file nonetheless)")
}
c1.Close()
// The on-disk kms dir must contain NO store files (ideally not exist at all).
if entries, statErr := os.ReadDir(dbDir); statErr == nil {
for _, e := range entries {
t.Errorf("BREACH: fresh health-only wrote to disk: %s/%s", dbDir, e.Name())
}
// A MANIFEST or KEYREGISTRY here would re-introduce the brick.
if _, e := os.Stat(filepath.Join(dbDir, "MANIFEST")); e == nil {
t.Fatal("BRICK-REGRESSION: MANIFEST written in health-only mode")
}
if _, e := os.Stat(filepath.Join(dbDir, "KEYREGISTRY")); e == nil {
t.Fatal("BRICK-REGRESSION: plaintext KEYREGISTRY written in health-only mode")
}
} else {
t.Logf("clean: %s does not exist after health-only boot", dbDir)
// No org store file may have been created by the health-only boot.
if any := anyStoreFile(root); any != "" {
t.Fatalf("BREACH: fresh health-only wrote a store file to disk: %s", any)
}
// Boot 2: real key → must open a CLEAN encrypted store and roundtrip (8c fixed).
// Boot 2: real key → clean store + roundtrip (the brick is gone).
c2, err := New(Config{DataDir: root, MasterKeyB64: rrKey(t, 0xA1)}, log)
if err != nil {
t.Fatalf("BRICK: keyed boot after health-only failed: %v", err)
t.Fatalf("BRICK-REGRESSION: keyed boot after health-only failed: %v", err)
}
defer c2.Close()
if err := c2.Put("/orgs/x", "K", "default", []byte("v")); err != nil {
@@ -82,176 +84,118 @@ func TestRR1_FreshHealthOnlyWritesNothing(t *testing.T) {
if got, err := c2.Get("/orgs/x", "K", "default"); err != nil || string(got) != "v" {
t.Fatalf("roundtrip after health-only→key: got=%q err=%v", got, err)
}
t.Logf("8c FIXED: health-only wrote nothing; keyed boot opened clean encrypted store + roundtrips")
t.Logf("foot-gun ELIMINATED: health-only wrote nothing; keyed boot opened clean + roundtrips")
}
// RR-2 (8b re-verify): encrypted store present + NO key → MUST fail loudly, NOT
// silently shadow with an in-memory store (Blue's first attempt regressed this).
func TestRR2_EncryptedStoreNoKeyFailsLoud(t *testing.T) {
// RR-2 (8b re-verify): an existing store + NO key must fail CLOSED (health-only),
// never a silent shadow — AND the on-disk sealed secret must NOT be lost: a keyed
// reboot recovers it. Proves no silent downgrade and no data loss.
func TestRR2_ExistingStoreNoKeyFailsClosedNoDataLoss(t *testing.T) {
root := t.TempDir()
log := luxlog.NewNoOpLogger()
// Create an encrypted store with a key + a secret.
c1, err := New(Config{DataDir: root, MasterKeyB64: rrKey(t, 0xB2)}, log)
if err != nil {
t.Fatalf("boot1: %v", err)
}
_ = c1.Put("/orgs/x", "SECRET", "default", []byte("must-not-vanish"))
if err := c1.Put("/orgs/x", "SECRET", "default", []byte("must-not-vanish")); err != nil {
t.Fatalf("seed put: %v", err)
}
c1.Close()
// Boot 2: no key. MANIFEST exists → must ERROR, not open in-memory.
// Boot 2: no key → health-only; every secret op fails closed.
c2, err := New(Config{DataDir: root, MasterKeyB64: ""}, log)
if err == nil {
defer c2.Close()
t.Fatalf("SILENT-DOWNGRADE: encrypted store + no key opened without error (Ready=%v). "+
"On-disk secret would be shadowed by an empty in-mem KV.", c2.Ready())
if err != nil {
t.Fatalf("boot2 (no key) New: %v", err)
}
t.Logf("8b holds: encrypted+no-key fails loud: %v", err)
if c2.Ready() {
t.Fatal("BREACH: keyless client reports Ready over an existing store")
}
if _, gErr := c2.Get("/orgs/x", "SECRET", "default"); gErr == nil {
t.Fatal("BREACH: keyless Get read the secret")
}
c2.Close()
// Boot 3: the key returns → the secret is intact (no silent shadow destroyed it).
c3, err := New(Config{DataDir: root, MasterKeyB64: rrKey(t, 0xB2)}, log)
if err != nil {
t.Fatalf("boot3 (key returns): %v", err)
}
defer c3.Close()
got, err := c3.Get("/orgs/x", "SECRET", "default")
if err != nil || string(got) != "must-not-vanish" {
t.Fatalf("DATA LOSS: secret gone after a keyless boot: got=%q err=%v", got, err)
}
t.Logf("8b holds: keyless boot is health-only fail-closed AND non-destructive (secret recovered)")
}
// RR-3 (8a re-verify): wrong key over an encrypted store → fail closed (zapdb
// registry sanity), never silent.
func TestRR3_WrongKeyFailsClosed(t *testing.T) {
// RR-3 (8a re-verify): a wrong key cannot read the sealed secret (the Seal
// envelope is bound to the master key) AND does not destroy it — the right key
// still recovers it. Fail closed, non-destructive.
func TestRR3_WrongKeyFailsClosedNonDestructive(t *testing.T) {
root := t.TempDir()
log := luxlog.NewNoOpLogger()
c1, _ := New(Config{DataDir: root, MasterKeyB64: rrKey(t, 0xC3)}, log)
_ = c1.Put("/orgs/x", "K", "default", []byte("v"))
c1.Close()
c2, err := New(Config{DataDir: root, MasterKeyB64: rrKey(t, 0xD4)}, log)
if err == nil {
defer c2.Close()
_, gErr := c2.Get("/orgs/x", "K", "default")
t.Fatalf("SILENT-DOWNGRADE: wrong key opened store without error (Get=%v)", gErr)
}
t.Logf("8a holds: wrong key fails closed: %v", err)
}
// RR-4: THE MANIFEST-SENTINEL BLIND SPOT. storeExistsOnDisk keys ONLY on MANIFEST.
// If a real encrypted store loses its MANIFEST but keeps KEYREGISTRY + SST/vlog
// (botched backup restore, partial rsync, fs corruption), a keyless boot sees no
// MANIFEST → goes IN-MEMORY, silently shadowing the on-disk encrypted data. Prove
// the sentinel's false-negative → silent downgrade. Then prove the follow-on: a
// keyed boot over the MANIFEST-less-but-KEYREGISTRY-present dir.
func TestRR4_ManifestSentinelBlindSpot(t *testing.T) {
root := t.TempDir()
log := luxlog.NewNoOpLogger()
dbDir := filepath.Join(root, "kms")
// Create a real encrypted store.
c1, err := New(Config{DataDir: root, MasterKeyB64: rrKey(t, 0xE5)}, log)
c1, err := New(Config{DataDir: root, MasterKeyB64: rrKey(t, 0xC3)}, log)
if err != nil {
t.Fatalf("boot1: %v", err)
}
_ = c1.Put("/orgs/x", "K", "default", []byte("on-disk-encrypted"))
if err := c1.Put("/orgs/x", "K", "default", []byte("v")); err != nil {
t.Fatalf("seed: %v", err)
}
c1.Close()
// List the on-disk artifacts.
before, _ := os.ReadDir(dbDir)
var names []string
for _, e := range before {
names = append(names, e.Name())
// Wrong key: read must fail closed (no plaintext).
c2, err := New(Config{DataDir: root, MasterKeyB64: rrKey(t, 0xD4)}, log)
if err != nil {
t.Fatalf("boot2 (wrong key) New: %v", err)
}
t.Logf("store artifacts: %v", names)
if pt, gErr := c2.Get("/orgs/x", "K", "default"); gErr == nil {
t.Fatalf("BREACH: wrong key read the secret: %q", pt)
}
c2.Close()
// Simulate MANIFEST loss (keep KEYREGISTRY + data files).
keyReg := filepath.Join(dbDir, "KEYREGISTRY")
if _, e := os.Stat(keyReg); e != nil {
t.Fatalf("expected KEYREGISTRY present: %v", e)
// Right key: still recovers (wrong-key access was non-destructive).
c3, err := New(Config{DataDir: root, MasterKeyB64: rrKey(t, 0xC3)}, log)
if err != nil {
t.Fatalf("boot3 (right key): %v", err)
}
if e := os.Remove(filepath.Join(dbDir, "MANIFEST")); e != nil {
t.Fatalf("remove MANIFEST: %v", e)
}
// Boot 2: NO key, MANIFEST gone but KEYREGISTRY + data remain.
c2, err := New(Config{DataDir: root, MasterKeyB64: ""}, log)
if err == nil {
// It opened. Is it in-memory (silent downgrade) while encrypted data sits
// on disk? storeExistsOnDisk returned false because MANIFEST is gone.
defer c2.Close()
t.Logf("SENTINEL BLIND SPOT: no-key boot with MANIFEST removed but KEYREGISTRY present "+
"→ New succeeded (Ready=%v). storeExistsOnDisk keys only on MANIFEST, so a store that "+
"lost its MANIFEST is treated as absent → in-memory shadow of on-disk encrypted data. "+
"KEYREGISTRY still on disk: %v", c2.Ready(), fileExists(keyReg))
// Not a confidentiality breach (data still encrypted at rest), but a silent
// data-availability downgrade the fix's own goal ("do not silently ignore
// on-disk secrets") does not fully achieve when the sentinel file is the one lost.
} else {
t.Logf("no-key boot with MANIFEST-less store failed (conservative): %v", err)
defer c3.Close()
if got, err := c3.Get("/orgs/x", "K", "default"); err != nil || string(got) != "v" {
t.Fatalf("secret lost after wrong-key access: got=%q err=%v", got, err)
}
t.Logf("8a holds: wrong key fails closed (Seal envelope) and is non-destructive")
}
// RR-5: PLAINTEXT-STORE MIGRATION BRICK. An operator upgrading from the OLD blue
// binary (which wrote a plaintext store) to the new keyed binary: a KEYED boot
// with WithEncryptionKey against a PRE-EXISTING PLAINTEXT store. zapdb's registry
// sanity rejects it → brick. Prove the migration path fails (documented risk).
func TestRR5_PlaintextStoreThenKeyedBrick(t *testing.T) {
// RR-6: health-only mode creates NO store file and fails every secret op closed —
// no disk write leaks from the no-key path.
func TestRR6_HealthOnlyNeverTouchesDisk(t *testing.T) {
root := t.TempDir()
log := luxlog.NewNoOpLogger()
dbDir := filepath.Join(root, "kms")
// Simulate the OLD behavior: open a PLAINTEXT disk store directly (no key).
if err := os.MkdirAll(dbDir, 0o700); err != nil {
t.Fatal(err)
}
pdb, err := zapdb.Open(zapdb.DefaultOptions(dbDir).WithLogger(nil))
if err != nil {
t.Fatalf("open plaintext store: %v", err)
}
// write something so it's a real store with a MANIFEST + KEYREGISTRY.
_ = pdb.Update(func(txn *zapdb.Txn) error { return txn.Set([]byte("k"), []byte("v")) })
pdb.Close()
if !fileExists(filepath.Join(dbDir, "MANIFEST")) {
t.Fatal("expected plaintext store MANIFEST")
}
// Now the NEW binary boots WITH a key → keyed branch, WithEncryptionKey.
c, err := New(Config{DataDir: root, MasterKeyB64: rrKey(t, 0xF6)}, log)
if err != nil {
t.Logf("MIGRATION BRICK CONFIRMED: keyed boot over a pre-existing PLAINTEXT store fails "+
"(New → %v). An operator upgrading from a binary that wrote a plaintext store must wipe "+
"{DataDir}/kms first. Not a confidentiality issue (no secret was in the plaintext store "+
"in the fixed flow), but a migration foot-gun if a plaintext store ever reached disk.", err)
return
}
defer c.Close()
t.Logf("keyed boot over plaintext store SUCCEEDED (Ready=%v) — zapdb tolerated it?", c.Ready())
}
// RR-6: does the in-memory branch leak ANY disk write? Drive a full lifecycle in
// health-only (in-mem) — List/health-metadata — and confirm the dir is pristine.
func TestRR6_InMemoryNeverTouchesDisk(t *testing.T) {
root := t.TempDir()
log := luxlog.NewNoOpLogger()
dbDir := filepath.Join(root, "kms")
c, err := New(Config{DataDir: root, MasterKeyB64: ""}, log)
if err != nil {
t.Fatalf("in-mem New: %v", err)
t.Fatalf("health-only New: %v", err)
}
defer c.Close()
// List works (no key needed) but must not create disk files.
// List is metadata-only (no key needed) but must not create a store file.
if _, err := c.List("/orgs/x", "default"); err != nil {
t.Fatalf("list in health-only: %v", err)
}
// Secret ops fail closed.
if err := c.Put("/orgs/x", "K", "default", []byte("v")); err == nil {
t.Fatal("BREACH: Put succeeded in health-only in-memory mode")
t.Fatal("BREACH: Put succeeded in health-only mode")
}
if _, err := c.Get("/orgs/x", "K", "default"); err == nil {
t.Fatal("BREACH: Get succeeded in health-only in-memory mode")
t.Fatal("BREACH: Get succeeded in health-only mode")
}
if _, err := os.Stat(dbDir); err == nil {
if entries, _ := os.ReadDir(dbDir); len(entries) > 0 {
t.Fatalf("BREACH: in-memory mode created disk files under %s", dbDir)
}
if any := anyStoreFile(root); any != "" {
t.Fatalf("BREACH: health-only mode created a store file: %s", any)
}
t.Logf("in-memory health-only: List OK, Put/Get fail closed, zero disk writes")
t.Logf("health-only: List OK, Put/Get fail closed, zero store files")
}
// RR-7: validator dedup — the facade (parseRef→Get/Put) must reject the SAME bad
// coords the HTTP boundary rejects, now that validation is hoisted into kms.
// Re-run the facade-asymmetry attack; it must now be BLOCKED.
// RR-7: the facade (parseRef→Get/Put) must reject the SAME bad coords the HTTP
// boundary rejects — the hoisted validCoords closes the facade asymmetry.
func TestRR7_FacadeValidationNowEnforced(t *testing.T) {
log := luxlog.NewNoOpLogger()
c, err := New(Config{DataDir: t.TempDir(), MasterKeyB64: rrRandKey(t)}, log)
@@ -261,22 +205,16 @@ func TestRR7_FacadeValidationNowEnforced(t *testing.T) {
defer c.Close()
ctx := context.Background()
// NUL + control chars in the ref — previously roundtripped; must now error.
badRef := "svc/EVIL\x00NAME@pr\x01od"
if err := c.PutSecret(ctx, badRef, []byte("x")); err == nil {
t.Fatalf("REGRESSION: facade still accepts NUL+ctrl ref %q — validation not enforced on Put", badRef)
} else {
t.Logf("facade Put rejects bad ref: %v", err)
t.Fatalf("REGRESSION: facade still accepts NUL+ctrl ref %q", badRef)
}
// Empty ref → name "" — must error now.
if err := c.PutSecret(ctx, "", []byte("x")); err == nil {
t.Fatalf("REGRESSION: facade still stores a nameless secret (empty ref)")
}
// Direct Get with bad coords must also reject (not just Put).
if _, err := c.Get("/orgs/x", "bad\x00name", "default"); err == nil {
t.Fatalf("REGRESSION: Get accepts NUL in name")
}
// A CLEAN ref must still work.
if err := c.PutSecret(ctx, "svc/GOOD@prod", []byte("ok")); err != nil {
t.Fatalf("clean ref rejected: %v", err)
}
@@ -286,70 +224,77 @@ func TestRR7_FacadeValidationNowEnforced(t *testing.T) {
t.Logf("facade asymmetry CLOSED: bad coords rejected on Put+Get+empty, clean ref works")
}
// RR-8: validCoords must NOT reject a legitimate path='/' (org/collection root).
// A too-strict validator would break every root-level secret. Guard against an
// over-correction that fails closed on valid input.
// RR-8: validCoords must NOT reject a legitimate path='/' (the org/collection root
// the parseRef("NAME") case). Guard against an over-correction failing closed on
// valid input.
func TestRR8_RootPathStillValid(t *testing.T) {
log := luxlog.NewNoOpLogger()
c, _ := New(Config{DataDir: t.TempDir(), MasterKeyB64: rrRandKey(t)}, log)
defer c.Close()
// path "/" (root) with a bare name — the parseRef("NAME") case.
if err := c.Put("/", "ROOT_SECRET", "default", []byte("v")); err != nil {
t.Fatalf("root-path Put rejected (over-correction): %v", err)
}
if got, err := c.Get("/", "ROOT_SECRET", "default"); err != nil || string(got) != "v" {
t.Fatalf("root-path roundtrip: got=%q err=%v", got, err)
}
// List at root must also pass validation.
if _, err := c.List("/", "default"); err != nil {
t.Fatalf("root-path List rejected: %v", err)
}
t.Logf("root path '/' remains valid for Put/Get/List")
}
func fileExists(p string) bool {
_, err := os.Stat(p)
return err == nil
}
// RR-4b: blast radius of the MANIFEST blind spot. After a store loses MANIFEST,
// does supplying the KEY on a later boot (keyed branch → WithEncryptionKey over
// the KEYREGISTRY-present-but-MANIFEST-absent dir) RECOVER the secret, or
// corrupt/brick? Determines whether RR4 is a transient blip or data loss.
func TestRR4b_ManifestLossThenKeyedRecovery(t *testing.T) {
// RR-9 (NEW): PHYSICAL org isolation. Two orgs' secrets must land in SEPARATE
// on-disk files — defense in depth UNDER the REST authz gate — and a record
// physically relocated into another org's file must still fail to Open (the Seal
// AAD binds the full /orgs/{org} path). This is the new-model isolation the ZapDB
// single-store never had.
func TestRR9_PerOrgPhysicalIsolation(t *testing.T) {
root := t.TempDir()
log := luxlog.NewNoOpLogger()
dbDir := filepath.Join(root, "kms")
key := rrKey(t, 0xAB)
key := rrKey(t, 0x9E)
c1, err := New(Config{DataDir: root, MasterKeyB64: key}, log)
c, err := New(Config{DataDir: root, MasterKeyB64: key}, log)
if err != nil {
t.Fatalf("boot1: %v", err)
t.Fatalf("new: %v", err)
}
if err := c1.Put("/orgs/x", "K", "default", []byte("recover-me")); err != nil {
t.Fatalf("put: %v", err)
if err := c.Put("/orgs/acme", "K", "prod", []byte("acme-only")); err != nil {
t.Fatalf("put acme: %v", err)
}
c1.Close()
if err := c.Put("/orgs/globex", "K", "prod", []byte("globex-only")); err != nil {
t.Fatalf("put globex: %v", err)
}
c.Close()
// Lose MANIFEST.
if e := os.Remove(filepath.Join(dbDir, "MANIFEST")); e != nil {
t.Fatalf("rm MANIFEST: %v", e)
// Distinct files exist for the two orgs.
acme := filepath.Join(root, "orgs", "acme", "kms.db")
globex := filepath.Join(root, "orgs", "globex", "kms.db")
if _, err := os.Stat(acme); err != nil {
t.Fatalf("acme file missing: %v", err)
}
if _, err := os.Stat(globex); err != nil {
t.Fatalf("globex file missing: %v", err)
}
// Boot with the SAME key (keyed branch). What does zapdb do with no MANIFEST?
c2, err := New(Config{DataDir: root, MasterKeyB64: key}, log)
if err != nil {
t.Logf("BLAST RADIUS: after MANIFEST loss, keyed boot FAILS (New → %v) — the store is "+
"bricked until wiped; the on-disk secret is UNRECOVERABLE via this path.", err)
return
}
defer c2.Close()
got, gErr := c2.Get("/orgs/x", "K", "default")
if gErr == nil && string(got) == "recover-me" {
t.Logf("BLAST RADIUS: MANIFEST loss is RECOVERABLE — keyed boot rebuilt the manifest and "+
"the secret survives (Get=%q). RR4 is a transient availability blip, not data loss.", got)
} else {
t.Logf("BLAST RADIUS: keyed boot after MANIFEST loss opened but the secret is GONE "+
"(Get=%q err=%v) — silent data loss.", got, gErr)
// acme's file must NOT contain globex's plaintext (nor vice versa) — sealed.
ab, _ := os.ReadFile(acme)
if bytes.Contains(ab, []byte("globex-only")) || bytes.Contains(ab, []byte("acme-only")) {
t.Fatal("BREACH: plaintext secret found in the per-org SQLite file")
}
t.Logf("per-org physical isolation holds: separate files, no cross-org plaintext")
}
// anyStoreFile returns the first per-org kms.db found under {root}/orgs, or "" if
// none — the "did a boot create a store file?" probe for the health-only tests.
func anyStoreFile(root string) string {
var found string
_ = filepath.Walk(filepath.Join(root, "orgs"), func(p string, info os.FileInfo, err error) error {
if err != nil || info.IsDir() {
return nil
}
if filepath.Base(p) == "kms.db" {
found = p
}
return nil
})
return found
}
+65 -49
View File
@@ -1,13 +1,19 @@
package kms
// RED tests for the embedded KMS client internals: encryption-key restart
// semantics (rotation / downgrade), Sign fail-closed, and master-key non-leakage.
// White-box (same package) so we can drive New/Get/Put with explicit keys.
// RED tests for the embedded KMS client internals: master-key restart semantics
// over the PER-ORG SQLite backend (wrong-key confidentiality, health-only fail
// closed, the clean health-only→key transition), Sign fail-closed, and master-key
// non-leakage. White-box (same package) so we can drive New/Get/Put with explicit
// keys. The confidentiality boundary is now the per-secret AES-256-GCM Seal
// envelope (bound to the master key), not a store-level key registry — so these
// vectors assert the REAL boundary and confirm the former ZapDB KEYREGISTRY brick
// foot-gun is gone.
import (
"context"
"crypto/rand"
"encoding/base64"
"errors"
"strings"
"testing"
@@ -32,12 +38,13 @@ func randB64Key(t *testing.T) string {
return base64.StdEncoding.EncodeToString(k)
}
// VECTOR 8a: restart with a DIFFERENT master key (rotation without re-encrypt).
// zapdb's KEYREGISTRY sanity-text check must make badger.Open FAIL — so the
// store does NOT silently open with a wrong key (which would corrupt/lose data
// or, worse, appear to work). We assert New returns an error → pickKMSClient
// falls back to DisabledKMS (fail-closed). Proof it is not a silent downgrade.
func TestVector8a_RestartWrongKeyFailsClosed(t *testing.T) {
// VECTOR 8a: restart with a DIFFERENT master key. Per-org SQLite has no KEYREGISTRY,
// so the store REOPENS without a brick — but the per-secret AES-256-GCM Seal is
// bound to key A, so key B CANNOT Open the record: Get fails closed, returning no
// plaintext. Confidentiality holds on the REAL boundary (the Seal envelope), and
// in production cek additionally refuses to unwrap the per-file DEK under the wrong
// key at open. Proof it is not a silent downgrade AND not an availability brick.
func TestVector8a_WrongKeyCannotReadSealedSecret(t *testing.T) {
dir := t.TempDir()
log := luxlog.NewNoOpLogger()
@@ -53,24 +60,29 @@ func TestVector8a_RestartWrongKeyFailsClosed(t *testing.T) {
t.Fatalf("close1: %v", err)
}
// Boot 2: key B (rotation, store still encrypted with A's datakey registry).
// zapdb sanity check should reject B → New errors.
// Boot 2: key B. New/open succeeds (no registry to brick); the read must fail
// closed because the Seal envelope is bound to key A.
c2, err := New(Config{DataDir: dir, MasterKeyB64: b64key(t, 0xBB)}, log)
if err == nil {
// If it did NOT error, prove it at least cannot read A's secret with B.
defer c2.Close()
_, gErr := c2.Get("/orgs/x", "K", "default")
t.Fatalf("SILENT-DOWNGRADE RISK: reopened store with WRONG key without error "+
"(Get err=%v). Expected badger.Open to fail on KEYREGISTRY sanity mismatch.", gErr)
if err != nil {
t.Fatalf("boot2 New should succeed (no brick): %v", err)
}
t.Logf("rotation w/o re-encrypt fails closed: New(wrong key) → %v", err)
defer c2.Close()
pt, gErr := c2.Get("/orgs/x", "K", "default")
if gErr == nil {
t.Fatalf("BREACH: wrong key READ the sealed secret: got %q", pt)
}
if pt != nil {
t.Fatalf("BREACH: wrong-key Get returned %d plaintext bytes — must be nil", len(pt))
}
t.Logf("wrong-key read fails closed via the Seal envelope (no plaintext), no brick: %v", gErr)
}
// VECTOR 8b: store first created ENCRYPTED (key A), then restarted with NO key
// (health-only). zapdb must reject the plaintext-open of an encrypted registry —
// so you cannot silently DOWNGRADE an encrypted store to unencrypted. New should
// error (→ DisabledKMS), never open an encrypted store as plaintext.
func TestVector8b_EncryptedThenNoKeyFailsClosed(t *testing.T) {
// VECTOR 8b: store created with a key, then restarted with NO key. The keyless
// process runs HEALTH-ONLY (not Ready): every secret op fails closed with
// ErrMasterKeyMissing BEFORE the store is touched, so it can never read or shadow
// the encrypted secrets. (In production cek would also refuse to open the
// encrypted file without the key.) No silent plaintext downgrade.
func TestVector8b_NoKeyIsHealthOnlyFailClosed(t *testing.T) {
dir := t.TempDir()
log := luxlog.NewNoOpLogger()
@@ -81,29 +93,34 @@ func TestVector8b_EncryptedThenNoKeyFailsClosed(t *testing.T) {
_ = c1.Put("/orgs/x", "K", "default", []byte("v"))
c1.Close()
// Boot 2: NO key. Store dir already has an encrypted KEYREGISTRY.
// Boot 2: NO key → health-only.
c2, err := New(Config{DataDir: dir, MasterKeyB64: ""}, log)
if err == nil {
defer c2.Close()
if c2.Ready() {
t.Fatalf("BREACH: encrypted store reopened in READY mode with no key")
}
// It opened but is health-only. Is that a silent downgrade of the encrypted KV?
t.Fatalf("SILENT-DOWNGRADE RISK: encrypted store reopened WITHOUT key did not error "+
"(Ready=%v). Expected badger.Open to fail decrypting the KEYREGISTRY.", c2.Ready())
if err != nil {
t.Fatalf("boot2 (no key) New: %v", err)
}
t.Logf("encrypted→no-key fails closed: New(no key on encrypted dir) → %v", err)
defer c2.Close()
if c2.Ready() {
t.Fatal("BREACH: keyless client reports Ready")
}
if _, gErr := c2.Get("/orgs/x", "K", "default"); !errors.Is(gErr, ErrMasterKeyMissing) {
t.Fatalf("keyless Get = %v, want ErrMasterKeyMissing (fail closed before store)", gErr)
}
if pErr := c2.Put("/orgs/x", "K2", "default", []byte("v")); !errors.Is(pErr, ErrMasterKeyMissing) {
t.Fatalf("keyless Put = %v, want ErrMasterKeyMissing (fail closed before store)", pErr)
}
t.Logf("no-key boot is health-only; every secret op fails closed before the store")
}
// VECTOR 8c: the FOOT-GUN — store first created in HEALTH-ONLY (no key, plaintext
// KEYREGISTRY), then the operator injects the real key on the next boot. Does
// badger.Open reject the now-mismatched (plaintext) registry, bricking the store
// until wiped? This is the availability trap Blue flagged as untested.
func TestVector8c_HealthOnlyThenKeyBricks(t *testing.T) {
// VECTOR 8c: the former FOOT-GUN, now ELIMINATED. A health-only boot (no key)
// writes NOTHING (Put fails closed), so it leaves no half-initialized/plaintext
// store to poison the next boot. When the operator injects the real key, the store
// opens CLEAN and roundtrips — no brick, no data-dir wipe required. The ZapDB
// health-only→key brick is gone.
func TestVector8c_HealthOnlyThenKeyIsClean(t *testing.T) {
dir := t.TempDir()
log := luxlog.NewNoOpLogger()
// Boot 1: NO key → health-only, plaintext KEYREGISTRY created.
// Boot 1: NO key → health-only; writes are refused, so nothing lands on disk.
c1, err := New(Config{DataDir: dir, MasterKeyB64: ""}, log)
if err != nil {
t.Fatalf("boot1 (health-only): %v", err)
@@ -111,29 +128,28 @@ func TestVector8c_HealthOnlyThenKeyBricks(t *testing.T) {
if c1.Ready() {
t.Fatal("health-only client should not be Ready")
}
if err := c1.Put("/orgs/x", "K", "default", []byte("v")); !errors.Is(err, ErrMasterKeyMissing) {
t.Fatalf("health-only Put = %v, want ErrMasterKeyMissing (writes nothing)", err)
}
c1.Close()
// Boot 2: operator injects the real key. Plaintext registry now mismatches.
// Boot 2: operator injects the real key. Clean open + roundtrip, no brick.
c2, err := New(Config{DataDir: dir, MasterKeyB64: b64key(t, 0xDD)}, log)
if err != nil {
t.Logf("FOOT-GUN CONFIRMED: after a health-only boot, injecting the real key BRICKS "+
"the store (New → %v). Operator must wipe {DataDir}/kms before first real boot, "+
"or the KMS never comes up. Availability trap, not a data-confidentiality breach.", err)
return
t.Fatalf("health-only→key New: %v", err)
}
// If it opened, is it usable (Ready + roundtrip)?
defer c2.Close()
if !c2.Ready() {
t.Fatalf("after health-only→key boot: client not Ready (err=nil but unusable)")
t.Fatalf("after health-only→key boot: client not Ready")
}
if err := c2.Put("/orgs/x", "K", "default", []byte("v")); err != nil {
t.Fatalf("FOOT-GUN: Ready but Put fails after health-only→key transition: %v", err)
t.Fatalf("Put after health-only→key transition: %v", err)
}
got, err := c2.Get("/orgs/x", "K", "default")
if err != nil || string(got) != "v" {
t.Fatalf("FOOT-GUN: roundtrip broken after health-only→key transition: got=%q err=%v", got, err)
t.Fatalf("roundtrip broken after health-only→key transition: got=%q err=%v", got, err)
}
t.Logf("health-only→key transition is CLEAN: store upgrades to encrypted and roundtrips OK")
t.Logf("health-only→key transition is CLEAN: no brick (foot-gun eliminated), roundtrips OK")
}
// VECTOR 8d: clean restart with the SAME key must preserve + decrypt secrets
+325
View File
@@ -0,0 +1,325 @@
package kms
// store.go is the per-ORG SQLite persistence for sealed KMS secrets — the
// backend that REPLACES the single OS-locked ZapDB store, so cloud is no longer
// pinned to replicas=1.
//
// WHY. luxfi/kms's SecretStore is one embedded ZapDB (a Badger fork) opened under
// a SINGLE directory for ALL orgs. Badger takes an exclusive OS lock on its live
// files, so exactly one process may open it — the hard reason cloud ran
// replicas=1 with a cross-process writer lease. This store drops ZapDB entirely:
// each org's sealed secrets live in ITS OWN encrypted SQLite file
// ({DataDir}/orgs/{org}/kms.db via the canonical cloud.OrgDB → cek seam), which
// has NO exclusive-opener lock. Two tenants never share a file, so different pods
// can serve different tenants (consistent-hash org→pod); within a tenant, WAL +
// busy_timeout serialize writers and the horizontal-scale model pins each tenant
// to one writer pod. See the Red Handoff in the package report.
//
// THE FILE IS THE TENANT BOUNDARY. fileOrg() shards a store path to a file by its
// leading "/orgs/{org}" segment (the shape BOTH faces already use: the REST
// handler folds :org into /orgs/{org}, and the in-process facade's refs carry it
// too). A path with no such segment is genuinely deployment-wide and routes to
// the reserved {DataDir}/orgs/_platform/kms.db partition. cloud.OrgDB folds the
// org through the INJECTIVE SanitizeOrg slugger, so two distinct orgs can never
// collide on one file — physical isolation ON TOP OF the REST authz gate.
//
// CRYPTO STAYS IN THE CLIENT. This layer persists ALREADY-SEALED records: Seal /
// Open (the AES-256-GCM envelope, luxfi/kms/pkg/store) run in kms.go, so plaintext
// never reaches this file. The stored `path` column is the FULL store path
// (including /orgs/{org}); Seal's AAD binds that full path, so a record physically
// moved into another org's file still fails to Open — the cross-org record-swap
// defense is preserved, the store merely shards WHERE the row lives.
import (
"database/sql"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"sync"
"time"
"github.com/hanzoai/cloud"
kmsstore "github.com/luxfi/kms/pkg/store"
)
// reservedPlatformSlug mirrors cloud.PlatformDB's reserved partition: a path that
// names no tenant org routes here. It carries '_', which SanitizeOrg never emits,
// so it can never alias a real org's file.
const reservedPlatformSlug = "_platform"
// errReadOnly is returned by a reader-mode store when a mutation is attempted. A
// reader must never fork the authoritative writer's state.
var errReadOnly = errors.New("kms: store is read-only (reader HA role)")
// secretStore holds the lazily-opened, cached per-org SQLite handles. Each file
// is opened + migrated once on first touch and cached by org slug. Opens are
// serialized so a concurrent first touch opens exactly once — the same shape as
// clients/finance.
type secretStore struct {
dataDir string
readOnly bool
mu sync.Mutex
dbs map[string]*sql.DB // key: fileOrg(path) → open handle for that org's kms.db
}
func newSecretStore(dataDir string, readOnly bool) *secretStore {
return &secretStore{dataDir: dataDir, readOnly: readOnly, dbs: map[string]*sql.DB{}}
}
// fileOrg extracts the org whose file holds a secret path. A path shaped
// "/orgs/{org}[/…]" belongs to tenant {org}; anything else is deployment-wide and
// lands in the reserved platform partition. The returned value is the RAW org (or
// the reserved sentinel); cloud.OrgDB folds a raw org through SanitizeOrg, so
// distinct raw orgs stay on distinct files.
func fileOrg(path string) string {
p := strings.Trim(strings.TrimSpace(path), "/")
if p == "" {
return reservedPlatformSlug
}
segs := strings.SplitN(p, "/", 3)
if segs[0] == "orgs" && len(segs) >= 2 && segs[1] != "" {
return segs[1]
}
return reservedPlatformSlug
}
// dbFor resolves (opening + migrating + caching on first use) the SQLite handle
// for the org that owns path. When the file does not yet exist:
// - create=true (the Put path) → create it.
// - create=false (read/list/delete) → return (nil, nil); the caller treats
// absence as "no such secret" and NEVER litters an empty store shell for an
// org that only had a read attempted.
// A reader (s.readOnly) never creates: create is forced false, and it performs no
// DDL (the writer already migrated). The org is folded through the injective
// slugger inside cloud.OrgDB, so a path can never traverse out of {DataDir}/orgs
// or reach another tenant.
func (s *secretStore) dbFor(path string, create bool) (*sql.DB, error) {
if s.readOnly {
create = false
}
org := fileOrg(path)
s.mu.Lock()
defer s.mu.Unlock()
if db, ok := s.dbs[org]; ok {
return db, nil
}
if !create && !s.orgFileExists(org) {
return nil, nil // nothing to open; caller returns not-found / empty
}
var (
db *sql.DB
err error
)
if org == reservedPlatformSlug {
db, err = cloud.PlatformDB(s.dataDir, "kms")
} else {
// OrgDB SanitizeOrg-slugs the org, creates {DataDir}/orgs/{slug} 0700, opens
// via cek (encrypted at rest, per-db DEK) with the single-writer + WAL pragmas.
db, err = cloud.OrgDB(s.dataDir, org, "", "kms")
}
if err != nil {
return nil, fmt.Errorf("kms: open org store %q: %w", org, err)
}
if !s.readOnly {
if err := migrateSecrets(db); err != nil {
_ = db.Close()
return nil, fmt.Errorf("kms: migrate org store %q: %w", org, err)
}
}
s.dbs[org] = db
return db, nil
}
// orgFileExists reports whether the on-disk kms.db for org already exists — the
// reader's "is this store hydrated?" check. Best-effort: an unreadable path is
// treated as absent (fail closed).
func (s *secretStore) orgFileExists(org string) bool {
var path string
if org == reservedPlatformSlug {
path = filepath.Join(s.dataDir, "orgs", reservedPlatformSlug, "kms.db")
} else {
slug := cloud.SanitizeOrg(org)
if slug == "" {
return false
}
path = filepath.Join(s.dataDir, "orgs", slug, "kms.db")
}
_, err := os.Stat(path)
return err == nil
}
// migrateSecrets creates the sealed-secret table. Idempotent (IF NOT EXISTS), so
// a reopen or a reader (opening an already-migrated file) is a no-op. The primary
// key (path, env, name) is the exact coordinate the ZapDB key encoded
// (kms/secrets/{path}/{env}/{name}), so listing semantics carry over unchanged.
func migrateSecrets(db *sql.DB) error {
const ddl = `
CREATE TABLE IF NOT EXISTS kms_secrets (
path TEXT NOT NULL,
env TEXT NOT NULL,
name TEXT NOT NULL,
scheme TEXT NOT NULL DEFAULT 'aead+mlkem',
ciphertext BLOB NOT NULL,
wrapped_dek BLOB NOT NULL,
key_handle TEXT NOT NULL DEFAULT '',
policy_id TEXT NOT NULL DEFAULT '',
created_at INTEGER NOT NULL DEFAULT 0,
updated_at INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (path, env, name)
);`
if _, err := db.Exec(ddl); err != nil {
return err
}
return nil
}
// put upserts a SEALED secret into its org's file. On conflict it rewrites the
// ciphertext/wrapped-DEK (a re-seal always mints a fresh per-secret DEK) and bumps
// updated_at while preserving created_at.
func (s *secretStore) put(sec *kmsstore.Secret) error {
if s.readOnly {
return errReadOnly
}
db, err := s.dbFor(sec.Path, true)
if err != nil {
return err
}
scheme := sec.Scheme
if scheme == "" {
scheme = kmsstore.ModeStandard
}
now := time.Now().Unix()
_, err = db.Exec(
`INSERT INTO kms_secrets (path,env,name,scheme,ciphertext,wrapped_dek,key_handle,policy_id,created_at,updated_at)
VALUES (?,?,?,?,?,?,?,?,?,?)
ON CONFLICT(path,env,name) DO UPDATE SET
scheme=excluded.scheme, ciphertext=excluded.ciphertext, wrapped_dek=excluded.wrapped_dek,
key_handle=excluded.key_handle, policy_id=excluded.policy_id, updated_at=excluded.updated_at`,
sec.Path, sec.Env, sec.Name, scheme, sec.Ciphertext, sec.WrappedDEK, sec.KeyHandle, sec.PolicyID, now, now)
if err != nil {
return fmt.Errorf("kms: write secret: %w", err)
}
return nil
}
// get reads a sealed secret by its full coordinate. Returns ErrSecretNotFound
// (verbatim, for a 404 mapping) when absent. The reconstructed Secret carries the
// FULL stored path, so the Client's Open reproduces Seal's AAD exactly.
func (s *secretStore) get(path, name, env string) (*kmsstore.Secret, error) {
db, err := s.dbFor(path, false)
if err != nil {
return nil, err
}
if db == nil {
return nil, kmsstore.ErrSecretNotFound // org has no store file yet
}
sec := &kmsstore.Secret{Path: path, Name: name, Env: env}
err = db.QueryRow(
`SELECT scheme, ciphertext, wrapped_dek, key_handle, policy_id FROM kms_secrets
WHERE path=? AND env=? AND name=?`, path, env, name).
Scan(&sec.Scheme, &sec.Ciphertext, &sec.WrappedDEK, &sec.KeyHandle, &sec.PolicyID)
if errors.Is(err, sql.ErrNoRows) {
return nil, kmsstore.ErrSecretNotFound
}
if err != nil {
return nil, fmt.Errorf("kms: read secret: %w", err)
}
return sec, nil
}
// list returns the metadata (never ciphertext) of the secrets at EXACTLY (path,
// env) — the same exact-coordinate listing the ZapDB prefix scan did (it never
// recursed into sub-paths). Nothing sensitive is decrypted.
func (s *secretStore) list(path, env string) ([]*kmsstore.Secret, error) {
db, err := s.dbFor(path, false)
if err != nil {
return nil, err
}
if db == nil {
return nil, nil // org has no store file yet → no secrets
}
rows, err := db.Query(
`SELECT name, scheme, key_handle, policy_id FROM kms_secrets
WHERE path=? AND env=? ORDER BY name`, path, env)
if err != nil {
return nil, fmt.Errorf("kms: list secrets: %w", err)
}
defer func() { _ = rows.Close() }()
var out []*kmsstore.Secret
for rows.Next() {
sec := &kmsstore.Secret{Path: path, Env: env}
if err := rows.Scan(&sec.Name, &sec.Scheme, &sec.KeyHandle, &sec.PolicyID); err != nil {
return nil, fmt.Errorf("kms: scan secret: %w", err)
}
out = append(out, sec)
}
return out, rows.Err()
}
// del removes a secret, returning ErrSecretNotFound (verbatim) when it was absent
// so the REST layer maps a missing delete to 404 rather than 200.
func (s *secretStore) del(path, name, env string) error {
if s.readOnly {
return errReadOnly
}
db, err := s.dbFor(path, false)
if err != nil {
return err
}
if db == nil {
return kmsstore.ErrSecretNotFound // org has no store file yet
}
res, err := db.Exec(`DELETE FROM kms_secrets WHERE path=? AND env=? AND name=?`, path, env, name)
if err != nil {
return fmt.Errorf("kms: delete secret: %w", err)
}
n, err := res.RowsAffected()
if err != nil {
return fmt.Errorf("kms: delete secret: %w", err)
}
if n == 0 {
return kmsstore.ErrSecretNotFound
}
return nil
}
// hasRestoredStore reports whether any per-org kms.db already exists under
// {dataDir}/orgs — the reader's boot-time "is there anything to serve?" check, so
// a reader with an empty data dir fails closed at New rather than opening nothing
// and answering as if healthy.
func hasRestoredStore(dataDir string) bool {
root := filepath.Join(dataDir, "orgs")
entries, err := os.ReadDir(root)
if err != nil {
return false
}
for _, e := range entries {
if !e.IsDir() {
continue
}
if _, err := os.Stat(filepath.Join(root, e.Name(), "kms.db")); err == nil {
return true
}
}
return false
}
// close closes every cached per-org handle (best-effort), returning the first
// error. Safe to call once at shutdown.
func (s *secretStore) close() error {
s.mu.Lock()
defer s.mu.Unlock()
var first error
for org, db := range s.dbs {
if err := db.Close(); err != nil && first == nil {
first = err
}
delete(s.dbs, org)
}
return first
}
+7 -7
View File
@@ -15,22 +15,22 @@ func TestUsageAmountMoney(t *testing.T) {
if !ok {
t.Fatal("parse atto")
}
typed := money.FromInt(atto)
typed := money.FromAtto(atto)
u := Usage{User: "x", Amount: typed}
got := u.amountMoney()
if got.Cents() != 0 {
t.Errorf("typed 0.00132 USD: cents=%d want 0 (sub-cent, not floored up)", got.Cents())
}
if got.Int().Cmp(atto) != 0 {
t.Errorf("typed 0.00132 USD: atto=%s want 1320000000000000 (exact, no floor)", got.Int())
if got.Atto().Cmp(atto) != 0 {
t.Errorf("typed 0.00132 USD: atto=%s want 1320000000000000 (exact, no floor)", got.Atto())
}
// Legacy micros: 2000 micros = 0.002 USD → reconstructed 18-dp exact.
u2 := Usage{User: "x", AmountMicros: 2000}
got2 := u2.amountMoney()
want2 := new(big.Int).Mul(big.NewInt(2000), big.NewInt(1_000_000_000_000)) // 2000e12 atto
if got2.Int().Cmp(want2) != 0 {
t.Errorf("micros 2000: atto=%s want %s", got2.Int(), want2)
if got2.Atto().Cmp(want2) != 0 {
t.Errorf("micros 2000: atto=%s want %s", got2.Atto(), want2)
}
// Legacy cents: 50 cents.
@@ -47,7 +47,7 @@ func TestUsageAmountMoney(t *testing.T) {
// Typed wins over legacy fields when both set.
u4 := Usage{User: "x", Amount: typed, AmountCents: 999, AmountMicros: 999}
if u4.amountMoney().Int().Cmp(atto) != 0 {
t.Errorf("typed should win over legacy fields: got %s", u4.amountMoney().Int())
if u4.amountMoney().Atto().Cmp(atto) != 0 {
t.Errorf("typed should win over legacy fields: got %s", u4.amountMoney().Atto())
}
}
+1 -1
View File
@@ -503,7 +503,7 @@ func (u Usage) amountMoney() money.Amount {
}
if u.AmountMicros > 0 {
// micro-USD (1e6 = $1) → 18-dp USD: scale the integer micros by 1e12.
return money.FromInt(new(big.Int).Mul(big.NewInt(u.AmountMicros), big.NewInt(1_000_000_000_000)))
return money.FromAtto(new(big.Int).Mul(big.NewInt(u.AmountMicros), big.NewInt(1_000_000_000_000)))
}
if u.AmountCents > 0 {
return money.FromCents(u.AmountCents)
+22 -7
View File
@@ -4,6 +4,8 @@ import (
"context"
"net/http"
"strings"
"github.com/hanzoai/account"
)
// Identity headers minted by the Hanzo gateway (the trust boundary). A product
@@ -11,8 +13,9 @@ import (
// client-supplied value (the gateway strips them on ingress). See
// commerce/CLAUDE.md "Gateway Trust Headers".
const (
HeaderUserID = "X-User-Id"
HeaderOrgID = "X-Org-Id"
HeaderUserID = "X-User-Id"
HeaderOrgID = "X-Org-Id"
HeaderAccount = "X-Billing-Account-Id"
)
// PriceFunc computes the cost (in cents) to record for a completed request.
@@ -132,9 +135,12 @@ func (c *Client) recordAsync(r *http.Request, u Usage, onErr func(*http.Request,
}
// IdentityFromGatewayHeaders builds an AuthInput from the gateway-minted
// identity headers. Prepaid billing is per-org, so the billing key (User) is
// the org slug (X-Org-Id) — the same key the LLM gate uses (user.Owner). The
// full "{org}/{sub}" identity is recorded as Actor for the audit trail only.
// identity headers. User — the account this request pays from — is resolved by
// the ONE rule every layer that touches money shares (hanzoai/account.Payer), so
// this client cannot key a different account than the gate that authorizes the
// request or the ledger that records it. The full "{org}/{sub}" identity is
// recorded as Actor for the audit trail only; it never decides which balance is
// gated.
//
// When there is no org (anonymous / org-less token) User falls back to the bare
// sub so a per-user balance can still gate; without either, User is empty and
@@ -143,9 +149,18 @@ func IdentityFromGatewayHeaders(r *http.Request) AuthInput {
org := strings.TrimSpace(r.Header.Get(HeaderOrgID))
sub := strings.TrimSpace(r.Header.Get(HeaderUserID))
user := org // per-org billing key
// Which ACCOUNT pays is resolved by the ONE rule every layer that touches money
// shares (hanzoai/account.Payer), reading the gateway-minted billing_account
// claim. This was `user := org` — the pool, always — on the premise that prepaid
// billing is per-org. A person in the shared signup org holds their OWN account,
// so that premise gated a balance nobody drained.
user := account.Payer(account.Credential{
Owner: org,
Name: sub,
Account: strings.TrimSpace(r.Header.Get(HeaderAccount)),
}).Subject()
if user == "" {
user = sub // org-less fallback
user = sub // org-less fallback: no org names no account, but a sub can still gate
}
actor := sub
if org != "" && sub != "" {
+9 -2
View File
@@ -9,6 +9,7 @@ import (
"testing"
"time"
"github.com/hanzoai/account"
"github.com/hanzoai/cloud/clients/metering"
)
@@ -93,8 +94,14 @@ func TestMiddleware_GatesAndRecords(t *testing.T) {
if n != 1 || amt != 7 {
t.Fatalf("recorded (count=%d amount=%d), want (1, 7)", n, amt)
}
if usr != "hanzo" {
t.Errorf("recorded user = %q, want hanzo (per-org billing key, not org/sub)", usr)
// gatewayReq is a person in the SHARED SIGNUP org, who holds their own account:
// its members are strangers, not a team, so a shared org is not a shared wallet.
// This asserted "hanzo" — the org pool — while ai's gate debited "hanzo/alice",
// which is the funded-pool-then-402 split. The account is whatever the one rule
// says, so assert against the rule rather than restating a premise it disproved.
want := account.Payer(account.Credential{Owner: "hanzo", Name: "alice"}).Subject()
if usr != want {
t.Errorf("recorded user = %q, want %q (the account the shared rule resolves)", usr, want)
}
}
+2 -2
View File
@@ -268,7 +268,7 @@ func create(s *cloud.Service[state], k resourceKind) zip.Handler {
// post-success debit; fee==0 or unconfigured billing makes this a no-op.
fee := cloud.ResourceFeeCents(computeFeeEnvPrefix, k.kind)
_, projectValidated := principal.ValidatedProject(c)
if err := s.State.bill.Gate(c.Context(), principal.Payer(c), project, projectValidated, k.kind, fee); err != nil {
if err := s.State.bill.Gate(c.Context(), principal.HomeOrg(c), project, projectValidated, k.kind, fee); err != nil {
return cloud.DenyResource(c, err)
}
@@ -299,7 +299,7 @@ func create(s *cloud.Service[state], k resourceKind) zip.Handler {
// Resource created — debit the caller's org ledger for the compute
// submission (per-org, env-attributed, async best-effort). Ongoing
// GPU-hour cost reuses s.State.bill.Meter from a future runtime usage watcher.
s.State.bill.Meter(principal.Payer(c), project, k.kind, fee, c.RequestID(), cloud.ClientIP(c))
s.State.bill.Meter(principal.HomeOrg(c), project, k.kind, fee, c.RequestID(), cloud.ClientIP(c))
return c.JSON(http.StatusCreated, view(out, true))
}
}

Some files were not shown because too many files have changed in this diff Show More