Compare commits

...
Author SHA1 Message Date
hanzo-dev 18749656a2 refactor: MountSpec is AppSpec — they are apps, mount is the verb
Renamed across 18 files. The composition root lists apps; "mount" is what you do
to one, not what one is.

Also carried in, because both were blocking a build:
  - PluginSpec takes every prefix an app owns, not one. o11y owns /v1/o11y AND
    /v1/sentry (the Sentry-protocol ingest), and a prefix left out is not an
    error — it is a silent 404 on that subtree, which for Sentry ingest means
    quietly dropping every error event in the fleet.
  - clients/websearch used zip.AdaptNetHTTPFunc, removed in zip v1.12.0. main
    does not compile without this. Now AdaptNetHTTP(http.HandlerFunc(f)) —
    http.HandlerFunc already IS an http.Handler, which is why the func-shaped
    adapter was redundant.

NOT DONE, deliberately: currying Mount on Deps. Measured first — it costs 102
app signature changes across 103 files, and the payoff does not arrive. The
point of currying was that a cloud app would become a zip.Service and the
adapters would vanish; but cloud's Mount takes Router, not *zip.App, and that
distinction is load-bearing — newScope bounds which prefixes an app's middleware
may touch, and apps.TestWireFrozen fails on any new Global grant (only 6 of ~108
apps hold one). Curried, PluginSpec STILL needs its type assertion. 102
signatures for an adapter that stays is not a trade worth making; unifying the
router types first would be, and that is a different change.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 16:04:21 -07:00
hanzo-dev 7fcf19709a cd+site: delete the duplicate lifecycle; keep what it found
clients/projects already owns the versioned-release model for sites, and
does it better than the packages this removes:

  <org>/.releases/<slug>/rel_<manifest digest>/  immutable, content-addressed,
  a SIBLING of the mutable prefix so a purge cannot shred a live release;
  ActivateRelease is one atomic UPDATE ... WHERE EXISTS (release row);
  servePrefix re-validates the id before it can widen a prefix and falls back
  to the legacy prefix, so there is no flag day; rollback is activating an
  older id, already routed.

clients/cd + clients/site re-implemented that with a WEAKER pointer: a v<N>
counter instead of a content digest, and a plain PUT of a CURRENT object that
could name a release whose row does not exist. Shipping it would have been a
second way to do a solved thing.

The premise it was built on was also wrong: hanzoai/static was never going to
need to resolve CURRENT, because it does not resolve the pointer at all —
clients/sites does, through the store. There was no ordering constraint and no
flag day to sequence.

What it genuinely found is kept in LLM.md as a named gap: releases are never
garbage-collected, and once retention exists activate must verify the BYTES,
not just the row, before it flips. clients/platform/handoff.go goes with it —
its digest-over-tag insight is real but it was scaffolding for this
architecture and never became load-bearing.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 15:47:04 -07:00
hanzo-dev d3f91115c5 platform: one writer for "what is live"
rolloutRelease wrote twice — the operator Service CR patch plus a
repository_dispatch mirror at hanzoai/universe — composed best-effort so
the step passed if EITHER landed. Two writers for one fact, and the
composition hid their disagreement: the patch fails, the mirror succeeds,
and the cluster and git describe different production states with nothing
reporting a problem.

The mirror was never actually running. It reads UNIVERSE_DISPATCH_TOKEN,
which is not set on the cloud deployment (197 env vars; only GH_PAT is
there), so it failed closed on every release. Every rollout in production
was already the CR patch alone. This deletes a phantom second writer, not
a second path.

With one writer the remaining failure is reported instead of tolerated: a
release that cannot patch the CR has built, smoke-passed and tagged an
image that is NOT live, and saying otherwise is worse than failing.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 15:42:00 -07:00
hanzo-dev d699a0e297 cd: the registry — one name, one owner
Kinds register themselves; cd imports none of them and never branches on Kind.
Adding "function" or "worker" later is a Register call, not an edit to the
lifecycle — the property that stops the per-kind pipelines from growing back.

A duplicate name is an error rather than a silent overwrite. Two registrations
for one name mean two owners for one deployment, which is the same two-writer
confusion that let the CR patch and the universe mirror disagree about what is
live. Failing at mount is loud and early; overwriting would surface much later
as a rollout that mysteriously went somewhere else.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 15:24:59 -07:00
hanzo-dev be883e4659 platform: name the CI->CD seam, and the two writers it hides
releaseFor is build -> smoke -> tag -> notify, but only the first three answer
CI's question ("is this good, and what did it produce?"). notify answers CD's
("what is live?") — and answers it TWICE: it patches the Service CR directly AND
mirrors an image update into universe, with a fallback between them. So "what is
live" has two writers that can disagree, and the fallback hides it: the CR patch
fails, the mirror succeeds, and the cluster and git describe different production
states with nothing reporting a problem.

That is also why a release cannot be rolled back. Neither writer records WHICH
artifact was placed, only that it was pushed somewhere; rollback needs the set of
placements and no one was keeping it.

A release run produces exactly one durable fact: a verified, immutable artifact
at a known reference. That is where CI ends. Artifact() expresses it in cd's
vocabulary and is deliberately the ONLY thing handed over — a seam this small
means the runner cannot regrow deployment opinions.

It is built only after smoke passes, because an unverified image handed to cd
would become a Placement and therefore a rollback target someone selects during
an incident. When a digest is known the reference uses it: a tag can be moved, a
digest cannot, and a rollback resolving a moved tag deploys different bytes than
the ones that were verified.

NOT LOAD-BEARING YET: rolloutRelease still runs. This makes the switch a one-line
change once cd is mounted and a workload Target is registered.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 13:40:53 -07:00
hanzo-dev 73f66218b5 cd+site: one deploy lifecycle, and a live pointer that makes rollback possible
"Get code to production" was five mechanisms — platform/v1/runner, cloud/projects,
hanzocd, universe-git-sync, and hand-rolled Deployments — each re-deriving how to
build, where to put the result, how to version it, and how to undo it. Braiding
those four independent facts is why a static site could not be rolled back, did
not appear in the platform UI, and was not tracked by CD at all.

clients/cd is the lifecycle, and it is ignorant on purpose: an Artifact being an
image or a bundle is a property of the BUILD OUTPUT, not a different lifecycle —
a container and a site differ in how bytes are produced and served, never in what
"promote" or "roll back" mean. So there is one Engine and a Target interface, and
cd holds no compiled dependency on any kind; adding one never edits the package.
Deploy and Rollback are the SAME function with a different Placement, which falls
out of releases being immutable and removes the code path that only runs during
an incident and is therefore only tested during one. Place always completes
before Activate flips, so a broken build can fail to replace production but can
never take it down.

clients/site implements the origin kind. The old publishSite wrote every deploy
to the SAME prefix "<org>/<slug>/", and BindHost mapped a host to (org, slug) —
so "which release is live" was never a stored fact, just whatever bytes were
written last. That absence, not a missing feature, is the structural reason
rollback did not exist. Bundles now land at "<org>/<slug>/v<N>/" and one object,
"<org>/<slug>/CURRENT", names the live one: a single atomic PUT, in the same
failure domain as the bytes it names, readable with a plain object GET.

NOT YET WIRED: no routes are mounted and no writer is switched. Cutting live
sites over requires the static server to resolve CURRENT first, or a site would
404 on its next deploy.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 13:23:48 -07:00
hanzo-dev 2f683d0625 feat(o11y): the first cloud app that is not linked in
o11y builds as its own binary and the composition root loads it at run time
(cloud.PluginSpec -> zip.Load) instead of linking it. The subsystem's code did
not move and did not fork: cmd/o11y calls the same o11y.MountO11y the Wire
entry used to. Only the process changed, which is the point — where a
subsystem runs becomes a deployment decision rather than a property of the
source.

Deleting the clients/o11y IMPORT is what does the work; dropping the mount
alone would have kept the graph linked.

Measured on this tree (interleaved A/B, 3 rounds, on a contended box):

  cmd/cloud packages   4336 -> 3250   (-1086, -25.0%)
  cmd/cloud binary     496.6 -> 401.3 MiB (-95.3 MiB, -19.2%)
  cmd/cloud link RSS   5.50 -> 4.48 GiB   (-18.6%, no overlap across rounds)
  cmd/cloud relink     21.58s -> 17.36s min (noisy; still a 3250-pkg link)

  o11y plugin          2734 packages, 165.1 MiB, 7.8s, 2.49 GiB peak

The real win is not cloud's link time, it is that an o11y change no longer
relinks cloud AT ALL: 7.8s/2.49 GiB against 21.6s/5.50 GiB, and the host is
untouched.

Proven end to end: cloud starts the child on a private unix socket and
/v1/o11y/* is answered by it — /v1/o11y/status returns o11y's own 403 while
the host binary contains ZERO clients/o11y symbols (nm: 450 -> 0).

Tests: cmd/cloud gains a TestMain that builds the plugin, because MountAll can
no longer mount o11y without it, and newTestApp/fullyMountedApp now shut the
app down — a mounted plugin is a child process, and leaking it holds the test's
stdout open until `go test` fails a suite that passed.

The frozen wire golden flips o11y's hasShutdown true->false: teardown moved
into the child with the resources. Name/OwnsHealth/Global stay pinned.

KNOWN GAPS, not merge-ready (see branch report):
  - /v1/sentry/* is a second prefix o11y owns; zip.Load takes one, so it 404s.
  - the host loses its OTel tracer provider: clients/o11y's init() registered
    it (nm: installTraceProvider 24 -> 0). The bootstrap is a HOST concern
    braided into the o11y package and must be split out before merge.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 12:48:55 -07:00
zeekay 9cff401caa fix(o11y): the zap receiver serves the logs pipeline too
The rendered ingest config declares receivers: [zap] for BOTH the traces and
the logs pipeline, but the factory only registered WithTraces. A collector
asked to build a logs pipeline from a receiver that cannot produce logs fails
at construction — so the previous commit would have taken the embedded
collector down at boot, not degraded it.

WithLogs now wraps o11y/pkg/zaplogreceiver, the MsgLogBatch = 3 end of the
same wire.

Severity text rides separately from the numeric level on purpose: an emitter
may use its own vocabulary (WARN vs Warning), and re-deriving the text from
the number would silently rewrite what the service actually said.
2026-07-27 12:06:38 -07:00
zeekayandhanzo-dev d2a04e5d0e fix(deps): oxy from upstream, not a replace — every published cloud was unresolvable
`hanzoai/cloud` could not be used as a dependency by anything. Its go.mod
carried

    require github.com/vulcand/oxy/v2 v2.0.0-00010101000000-000000000000
    replace github.com/vulcand/oxy/v2 => github.com/traefik/oxy/v2 v2.0.0-2026...

and a `replace` in a *dependency* is ignored — Go honours it only in the
main module. Consumers therefore resolved the literal all-zero
pseudo-version and stopped:

    github.com/hanzoai/cloud@v1.801.218 requires
      github.com/vulcand/oxy/v2@v2.0.0-00010101000000-000000000000: invalid version

That is every module requiring cloud — ai, amqp, auto, commerce, cloud-oss,
gateway, ingress — plus hanzoai/base transitively, which is how this
surfaced. It built here only because ~/work/hanzo/go.work supplies the
replace that no consumer ever sees.

traefik's fork exists because it kept `module github.com/vulcand/oxy/v2`,
so it is reachable *only* through a replace. Upstream publishes real
versions through v2.2.0, and cloud uses exactly four symbols from it —
`forward.New`, `roundrobin.New`, `UpsertServer`, `roundrobin.Weight` —
all present with identical signatures. So the replace buys nothing here
and costs consumability. Require upstream directly; no code change.

This is the standing rule, not a new one: a fork owns its module path and
is required directly. Never a replace.

Verified: `go build ./...` resolves with zero module errors and every Go
package compiles. The five packages that still fail do so on
`ld: library 'resolv' not found` / `'stdlib.h' file not found` and build
once SDKROOT is set — the known macOS toolchain issue, unrelated.

Marker worth grepping for elsewhere: `v0.0.0-00010101000000-000000000000`
in a published go.mod means a replace resolved it, and reliably marks a
module that cannot be consumed.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 12:04:32 -07:00
hanzo-dev a5536422c3 docs: the index is /v1/index, and the four search-ish things are four things
Records the namespace collision that ate two routes and the naming boundary
that resolves it: hanzoai/search is the product, websearch queries outside,
crawl fetches, clients/index stores. Plus the .dek rule a store rename must
follow.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 11:54:15 -07:00
1df70dc5a5 perf(identity): verify an IAM token once, not once per ZAP frame (#369)
A ZAP socket authenticates at upgrade and then replays its credential on every
frame, because each frame is dispatched as an in-process HTTP request through
the same identity boundary. So the SAME token was signature-verified again for
every call on an already-authenticated connection — a chatty client paid an RSA
verification per frame.

The boundary keeps its shape. What changes is that the pure part of it —
token to claims — is memoized, so the parse happens once while a token is valid
and every transport benefits, not just ZAP.

Why this is a memo and not a second way to be trusted:
  - The KEY is the token. A hit is impossible without presenting the bytes that
    already validated, so the cache can hold no identity the one validator did
    not produce.
  - Only successes are stored. A rejected token is re-verified every time.
  - Entries die at the TOKEN'S OWN exp, never later, so expiry semantics are
    exactly what they were.
  - Revocation is not weakened: JWT validation never consulted a revocation
    list, so a revoked-but-unexpired token was accepted before and is accepted
    now, for the same bounded window.

Keyed by SHA-256 of the token rather than the token, so a credential never sits
in a map key where a dump or a stray log could surface it. Bounded at 8192 with
an expired-first sweep; overflow costs re-verification, never admission.

Seven tests pin the properties that make it safe rather than that it remembers
things: a different token never hits, a token past its exp misses, an
already-expired or exp-less token is never stored, and the map stays bounded.
Full gate 163 ok / 0 fail; the kms red-team, principal and zapface suites are
unchanged.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-27 11:53:19 -07:00
hanzo-dev 5cac08c06a Merge remote-tracking branch 'origin/main' into HEAD
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 11:52:02 -07:00
hanzo-dev 335bb094f8 Merge remote-tracking branch 'origin/main' into HEAD
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 11:51:27 -07:00
hanzo-dev 816f5a0495 cloud: an app system can run as its own binary — one line at the composition root
zip v1.10.0 makes a loaded plugin and a linked-in service the same type:
zip.Load(prefix, Plugin) returns a zip.Service, exactly what a compiled-in service
is. PluginSpec carries that through to Wire(), so a subsystem's LOCATION stops
being a property of its source. Moving one out of the binary is editing which
MountSpec the composition root lists — not a rewrite, and nothing downstream
(routing, health, shutdown ordering) can tell which kind it got.

The plugin names one of Addr (already listening), Bin (go:embed'd) or Path; for
the latter two zip runs it as a child on a private unix socket.

Global is set deliberately and the mount refuses a scoped Router rather than
working by accident: zip.Load registers the prefix itself, so handing it the
per-subsystem scope would nest that prefix under the subsystem name and the routes
would answer at a path nobody requests. That is a wiring mistake worth failing on.

Nothing is moved out of process here. This is the seam; which subsystems use it is
a deployment decision, made one at a time.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 11:51:21 -07:00
hanzo-dev b36ded7040 refactor(index): the in-binary index is /v1/index, not /v1/search
/v1/search already belongs to the hanzoai/ai RAG plane, which registers
/v1/search and a /v1/search/{name} pattern. That pattern swallowed the two
single-segment routes on this subsystem — GET /v1/search/health and
/v1/search/version answered 404 in production while every deeper route worked,
because the deeper paths do not match it. Two subsystems answering one path is
how a route silently disappears; the openapi projection is what finally showed
both owners.

The name is also the honest one. Hanzo's SEARCH product is hanzoai/search — our
own Meilisearch build, serving search.hanzo.ai and the docs corpus.
clients/websearch queries the outside world, hanzoai/crawl fetches it. This
subsystem is none of those: it is the thing an application writes documents into
and queries back, the storage primitive underneath. It is an index, so it is
called index and it lives at /v1/index.

The store moves {DataDir}/search.db -> index.db, and migrateStore carries the
WHOLE family, not just the database. cek keeps the wrapped data key beside the
file as "<path>.dek": moving the .db alone would strand the key, cek would mint
a fresh one, and every existing document would be undecryptable — data loss that
presents as an empty index. The -wal and -shm sidecars hold committed pages that
may not be checkpointed yet, so they move too. Idempotent; when both stores
exist it leaves them alone rather than guessing which encrypted one wins.

Tables are named for what they hold now that the file is dedicated: indexes,
docs, terms.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 11:50:01 -07:00
hanzo-dev f174bd45af Merge remote-tracking branch 'origin/main' into marketing-iam-audience
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 11:42:48 -07:00
zeekayandhanzo-dev 32f5e52112 commerce: hand it the master key cloud already holds
This is the line that unblocks billing.

commerce encrypts its per-tenant money stores under a 32-byte KEK and refuses to
boot without one on a libsqlcipher build — correctly; it will not open money data
unencrypted. But it reached for its OWN env var, so the same process needed a
SECOND key, provisioned through a second KMS path and a second sync object. That
key was never provisioned, and the refusal was silent from out here: Mount never
reached transport.SetApp, so every S2S billing read fell through to the network
and DNS-resolved the in-process placeholder. Fleet-wide, funded accounts read
$0 and every paid call 402'd "Insufficient balance". The ledger held the money
the whole time.

cloud already resolves a 32-byte master (CLOUD_KMS_MASTER_KEY_REF — the same one
durableCipher and cek derive from). It now decodes it once into Deps.MasterKey
and hands it to commerce. One process, one key, and a dependency the type system
can see instead of an ambient env read two packages away.

nil is preserved as "unset or malformed" — never a partial or wrong-length key,
because a wrong key encrypts against a store no other key can open — and commerce
then falls back to its own env, unchanged, which is the standalone and pure-Go
dev path.

deps: commerce v1.49.25 (carries EmbedConfig.MasterKey + migrate-on-open), which
pulls luxfi/consensus v1.36.11 — the clean tag cut to replace the force-moved
v1.36.x line.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 11:41:32 -07:00
hanzo-dev 960cad7c51 feat(search): list, stats and delete complete the dialect
Three Meilisearch endpoints were missing, and their absence was not cosmetic:
without GET /indexes an index whose uid nobody remembers is unreachable, and
without DELETE /indexes/:uid a mistaken uid is permanent. GET /stats reports
per-index document counts. The listing carries counts in one query rather than
one per index.

EnsureIndex now REFUSES an empty uid instead of storing one. A blank uid is not
addressable through the API, so documents written under it are invisible and its
registry row cannot be dropped by name — a row that can only be created by a
caller inside the process getting it wrong, and that nothing could then clean up.
The handlers already rejected an empty uid; the store now does too, so the
invariant holds wherever the call comes from.

DropIndex removes terms, documents and the registry row in ONE transaction, so
an index can never survive as a row with no documents or as documents with no
row.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 11:30:31 -07:00
hanzo-dev bdc1cfadd0 test(marketing): prove the roster seam against a REALLY mounted IAM
Every other test stubs rosterFn, so the cross-repo read — the point of the
change — never actually ran. This mounts the co-resident IAM subsystem the way
cloud does at boot, writes users through IAM's own model, and reads them back
through the production reader: deleted excluded, other tenant excluded,
credentials masked.

It also takes ownership of the fail-closed assertion. clients/iam.DB() is a
process-global with no un-mount, so "IAM unavailable" is only observable before
this test mounts one; asserting it here, first, replaces a sibling test that was
silently depending on running earlier (and did not, once this file existed).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 11:21:00 -07:00
hanzo-dev 08e1c4988e feat(marketing): resolve an audience to real customers through IAM
You could enroll an address you already knew. You could not say "email
everyone in my org" or "everyone who used model X" — audience members were
cohort identifiers with no way to become mailboxes. This closes that.

A customer is not a CRM row; it is a user in Hanzo IAM, owned by its org. So
the roster is READ in-process from the embedded IAM (iam/pkg/store.
GetMailableUsers over clients/iam.DB()) — the same seam clients/platform uses
for the IAM-owned Project — read-only, masked, IAM's model.User verbatim. There
is no marketing-local contact table to drift and no HTTP hop inside the binary.

An audience with no event filter is every mailable customer; with one, the
warehouse distinct_ids are joined to that roster and whatever matched nobody is
COUNTED, never invented into an address. One resolution path serves both the
preview and the send, so what an operator is shown is what would be mailed.

An announcement is therefore not a new engine: it is a one-step sequence with an
audience enrolled into it — the existing enroll route takes an audienceId where
it takes an address. Every message still walks the drip engine and state.deliver,
so claimed-once delivery, the suppression gate, and the signed unsubscribe footer
all hold unchanged. The test proves it: an opted-out customer is enrolled with
everyone else and still never reaches the rail.

The org is principal.Org, which IS IAM's Owner, so an audience only ever resolves
its own tenant. No IAM co-mounted is a 503 — a send reported successful to nobody
is the worse failure.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 11:11:30 -07:00
6f99219571 test(automations): stop racing for a port — retry the bind instead (#368)
TestTriggerPayloadThreadsThroughDurableRun failed in a full-tree run and passed
on its own. The cause is not the test's subject: it picks a port with
Listen(":0"), CLOSES the listener, then hands the bare number to Embed. Between
the close and the bind the port is anybody's, and `go test ./...` has several
packages playing the same trick at once.

Holding the listener cannot fix it, because Embed binds the port itself. Retrying
does: a second draw lands on a different ephemeral port, so a collision costs a
retry instead of a red build. embedEngine wraps pick-close-embed in five
attempts, and the three sites in this package now share it.

Left deliberately: 18 other test files use the same idiom. They are worth the
same treatment, but each needs its own read — this changes only the package whose
flake we actually observed.

Verified: five consecutive package runs clean, and the full widened gate is
162 ok / 0 fail.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-27 11:03:19 -07:00
zeekay cee0d96772 feat(o11y): cloud probes the fleet — availability without a scrape stack
The 20 health checks lived in a separate metrics stack as blackbox scrape jobs,
for one reason: o11y received pushed telemetry and had no way to ask a question.
o11y/pkg/prober closed that, so the probes live here now and emit
hanzo_service_up through the same meter as every other metric — one store, one
place alert rules live.

Each target keeps its OWN health path. /healthz, /health, /ping and
/api/public/health are all in use across the fleet, and probing the wrong one
reports a healthy service as down; these were carried over verbatim from the
scrape config they replace.

Opt-out, not opt-in (O11Y_PROBES=false). This is the signal availability
alerting depends on, and a signal that defaults to off is off in exactly the
deployment that needed it. Failure to start is non-fatal — losing probes must
not take the API plane down with them.
2026-07-27 10:59:30 -07:00
zooqueen a1cee2be24 Merge github/main
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 10:51:15 -07:00
hanzo-dev 95f56c3dcf feat(answer): the answer engine gets one home, behind one door
The web-grounded loop lived scattered across clients/ask as relevance.go,
stream.go and web.go — three files that together were an answer engine nobody
had named. It is now clients/answer: plan → search → read → rank → synthesize →
cite → follow-up, each of the five values with exactly one home.

ONE DOOR. answer registers no routes. /v1/ask stays the only entrance and
delegates when a `mode` (search|news|research|deep) selects web grounding —
"deep research" is a VALUE handed to that door, never a second route.

BOUNDED, not an agent loop: ≤3 LLM calls, ≤maxQueries search passes, ≤maxRead
fetches, a 90s wall clock, and a token ceiling past which the follow-up is
skipped. Metered once, against the resolved payer.

CLEAN-ROOM. scira, the obvious reference for this shape, is AGPL-3.0. Nothing
here is derived from it — copying would put AGPL obligations on the whole cloud
binary. The package says so at the top so the next person does not "helpfully"
port a snippet across.

Landed on current main rather than the branch it was written on: that branch is
~1660 commits behind, and merging it would have dragged a stale tree over
everything shipped since. Only clients/ask/ask.go had drifted (Mount now takes
cloud.Router; BillingOrg reads principal.Ledger), and both changes are carried
forward here — answer.go needed the same HomeOrg → Ledger rename.

go build ./clients/... clean; clients/answer and clients/ask both green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 10:51:06 -07:00
hanzo-dev 0e924d70dc o11y v1.5.31 -> v1.5.32: DELETE /v1/sentry/projects/{id}
o11y is embedded IN this binary (buildEmbeddedHandler runs it in-process; the
standalone o11y Deployment is only the fail-soft fallback), so /v1/sentry/* ships
by bumping this pin — there is no separate service to release.

v1.5.32 adds DELETE /v1/sentry/projects/{id}. Projects were create-and-read only
(PUT/PATCH/DELETE all 405), so a naming mistake was permanent — which is how an
off-convention project became unfixable. Org-scoped: a cross-tenant delete and a
nonexistent id return the SAME not-found so ids cannot be probed; deleting
revokes the DSN with no separate step (ResolveIngest fails closed); retained
events are untouched.

Fresh forward tag, never a repoint — a moved tag is a go.sum checksum mismatch
for everyone who already fetched it.

go vet ./clients/o11y/... clean; go.mod diff is the single version line.
2026-07-27 10:49:30 -07:00
hanzo-dev 47e2b053ad deps: zap-proto/zip v1.10.0 (+ hanzoai/orm v0.6.16)
zip v1.10.0 carries zap-proto/http v0.3.0, where wire headers are length-prefixed
pairs instead of JSON. That breaks against v1.9.x, so every ZAP service moves
together. orm v0.6.14 called zaphttp.NewTransport, which v0.3.0 removed, so it
moves in the same commit.

Builds clean.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 10:33:54 -07:00
hanzo-dev 3c4f3e446c docs: record the search plane and release-on-merge in LLM.md
Two things a reader cannot infer from the code and will otherwise re-break:
why the search index is a term table and not FTS5 (the module is absent from
the SQLite this binary links, and the build tag that looks like it enables it
is inert), and that releases are cut by a merge to main through the GitHub App
rather than by any workflow file.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 10:23:24 -07:00
hanzo-dev 6d8c95d9f1 fix(search): health and version answer in production
Declared on the subsystem's group, GET /v1/search/health and /v1/search/version
did not survive the real mount composition: they answered on a bare app in
tests and 404'd in production, while every deeper route on the SAME group
worked. They are now absolute paths on app, the idiom every other OwnsHealth
subsystem already uses (clients/esign, clients/kms), which is what those
subsystems' working health routes in prod demonstrate.

This mattered more than a missing probe: a Meilisearch client checks /health
before it will use a server at all, so the whole surface read as absent.

The route test that should have caught it could not: a missing route and a real
"that index does not exist" are both 404, and it only looked at the status. It
now checks for the Meilisearch error shape, which the router's own not-found
does not carry — so an unregistered route fails and a genuine index_not_found
does not.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 10:20:55 -07:00
f2ba09f373 fix(paywall,kms): unpaywall sign-in, retire a dead auth model in the red tests, gate the whole tree (#367)
The go-unit gate named twelve packages out of 266. Everything else could go red
on main and never produce a red build, so two real defects had been sitting
there. Fixing both, then widening the gate so the next one cannot hide.

Sign-in was one flag away from an outage. Reachable() exempted /v1/ai/signin,
/v1/ai/signout and /v1/ai/account on the belief that the auth surface had been
namespaced. It had not: production answers /v1/signin, /v1/signout and
/v1/get-account, and all three /v1/ai/* spellings are 404. The never-gate list
therefore exempted three paths that do not exist and gated the three that do —
flip CLOUD_ENTITLEMENTS_ENFORCE and every unpaid user is locked out of logging
in. The file's own comment warned about "a 402 in front of SIGN-IN"; it was
pointed the wrong way. Both spellings are listed now, per this file's rule that
the list is deliberately generous.

The five KMS red-team failures were the opposite: the code is right and the
tests were stale. SuperAdmin moved to home-org membership — homeOrg == adminOrg
and human — because `owner` names the APP's org, not the user's, and isAdmin was
never consulted. The mints carried owner + isAdmin and no orgs claim, so they
asked for admin as nobody and got a correct 403. They now mint the orgs claim
IAM actually issues. The machine-denial assertions are untouched and still pass:
a machine principal is refused SuperAdmin whatever its audience.

The gate takes ./... with two named skips, each env-dependent rather than
broken — the cek master key, and a test asserting a hostname does NOT resolve,
which a NXDOMAIN-hijacking resolver fails for reasons unrelated to this code.
162 packages, 0 failures.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-27 10:12:15 -07:00
hanzo-dev 19853d8ad2 fix(integrations): a GitHub App could never be connected
The connect flow gated every provider's authorize leg on Creds().ClientID,
which assumes that leg is OAuth2. A GitHub App install URL is built from the
app slug and has no client id at all — githubCreds deliberately leaves it empty
and says so — so a fully configured App (slug + id + private key all present in
prod) was refused with "OAuth is not configured" for a credential its protocol
never uses.

Nothing could complete the install callback, so no connection row was ever
written, so OrgForExternalID could not map an installation to an org, so the
inbound webhook ignored every delivery. The App has been installed on hanzoai
with push events the whole time; cloud has been acking those pushes 200 and
doing nothing with them.

Readiness is now a question the provider answers: AuthorizeReady, optional, and
when nil the leg is standard OAuth2 and readiness is still ClientID. The
dual-path reasoning the old check protected is unchanged — a provider with only
apikey creds still fails closed here rather than offering a dead consent URL
with an empty client_id.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 10:04:09 -07:00
hanzo-dev 5cd1170225 feat(admin/infra): full DigitalOcean management behind one safety discipline
Extends the infra board from read-mostly to droplet delete/resize, load
balancer delete and node pool scaling. Every mutation runs the discipline
deleteVolume established — a FRESH forced cross-cluster re-scan, a verdict
the server derives from it, and an audit record for success, failure AND
refusal — because that discipline is now written once:

  - infra.go: mutation[T] + run() hold the whole shape. A handler supplies
    only WHAT to change; it never decides WHETHER. deleteVolume moves onto
    it unchanged in behaviour.
  - analyze.go: Snapshot.verdict folds the shared completeness gate with
    one rule function per resource, so every (Deletable|Mutable|Scalable,
    BlockedReason) pair on the board is derived in one place.

Routes (SuperAdmin only, as the rest of the board):
  DELETE /v1/admin/infra/droplets/:id
  POST   /v1/admin/infra/droplets/:id/resize          {"size","disk"}
  DELETE /v1/admin/infra/loadbalancers/:id
  POST   /v1/admin/infra/clusters/:id/nodepools/:pool/scale  {"count"}

Refusals, all proven server-side:
  - a droplet DOKS owns is neither deletable nor resizable — the node pool
    recreates it, so the operator would buy an outage and no change
  - a load balancer a live type=LoadBalancer Service claims in ANY cluster,
    matched on the DOKS load-balancer-id annotation or on address; also one
    forwarding to droplets outside every cluster, which no Service can vouch
    for. Member droplets alone are NOT a liveness signal — a leaked DOKS
    load balancer still lists every node in its cluster.
  - a node pool scaled below 1, or a shrink leaving the cluster no
    schedulable node. What is NOT provable is stated rather than faked:
    DOKS picks which nodes go, so PDBs/taints/affinity are the cluster's to
    enforce, and the response says so.
  - anything at all when the scan is incomplete. The fail-closed rule that
    protected volumes now covers every mutation.

Services join the cluster scan the way PVs already did, so a Services list
failure fails the completeness gate and freezes the whole board.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 09:49:41 -07:00
hanzo-dev cc4c5fe488 Merge origin/main
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 09:41:50 -07:00
hanzo-dev e71d1a8300 fix(integrations): the build trigger must not depend on the mirror
The deploy trigger fired after cloud.Sync, so a mirror that was unavailable or
erroring returned 503/502 before the trigger was ever reached — a sync outage
would silently stop the whole fleet from releasing, with a verified push and
nothing to show for it.

A build reads from GitHub, not from native, so it has no dependency on
mirroring. It now fires as soon as the org and branch are resolved, before the
installation token is minted and before the sync runs.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 09:41:38 -07:00
hanzo-dev b366413cbc Merge origin/main
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 09:40:50 -07:00
hanzo-dev 8e2212f37b chore(deps): zip v1.8.3 -> v1.9.0, http v0.2.0 -> v0.2.2
zip v1.9.0 brings Service/Add/Load/Reload/Unload, Mount(prefix, addr),
unix-socket ZAP, and Transport{Serve, Dial}.

The two breaking changes in the v1.8.4..v1.9.0 window cost nothing here:
zip.RegisterTransport is not called in this repo (clients/plugin has its own
same-named registry over http.RoundTripper, which is unrelated), and the only
zaphttp.NewTransport(addr) call site is in hanzoai/orm, whose v0.6.15 already
carries the rename to zaphttp.Dial("tcp", addr) and requires http v0.2.2. So
orm goes 0.6.14 -> 0.6.15 rather than this repo pinning an older http.

go mod tidy also promotes golang.org/x/text to direct.

Verified: all 173 non-main packages typecheck, and the named packages build and
test green. The 94 main packages are not linked here on purpose.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 09:40:38 -07:00
hanzo-dev 718b58cebb feat(platform): a merge to main publishes the next version
A push to hanzoai/cloud main built nothing. The image and its v* tags have ONE
owner — release.go — reached only by POST /v1/runner {release:true}, and
nothing anywhere called it: no cron, no CLI verb, no workflow. Releases were
manual curls by whoever held the machine token, which is why main could sit
ahead of production indefinitely with no image behind it.

The GitHub App already delivers pushes to /v1/connector/github/webhook, where
they are HMAC-verified and handed to the sync engine. That handler now also
fires cloud.OnGitPush — the SAME single-registrant deploy trigger the embedded
git server fires — so an upstream merge and a native push travel one seam
rather than two CIs. Platform decides what a push means: an app that tracks the
repo rebuilds, and cloud's own upstream cuts a release. Cloud is the machine, so
it calls the release in-process; the build token is never handed to a caller.

isReleasePush is deliberately narrow, since it publishes the image the whole
fleet runs: the release repo by URL (not by org, which does not identify a
repo), the main branch only, and a pinned commit only — a push with no commit
is ignored rather than re-resolved, which could otherwise build something newer
than the event describes. Bot-authored pushes are excluded by the guard that
already exists for automations, so the release's own tag and mirror pushes
cannot retrigger it.

launchRelease is now the ONE release entry point, shared by the HTTP path and
the push trigger, so a release cut either way is the same pipeline with the
same guards. It is single-flight: the next version is computed from the tags
that already exist, so two overlapping runs would compute the SAME version and
race to publish it. A second trigger is refused rather than queued — by the
time the first finishes, its commit is already the newer one — and a failed
start always clears the guard, which would otherwise wedge releases forever.

Also adds the TestMain harness clients/platform never had. cek fails closed
rather than opening the data plane unencrypted, so TestSecretEnvEndToEndDeploy
failed at openStore on pristine main; verified there before fixing it here.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 09:40:04 -07:00
hanzo-dev 44e1718ed4 Merge origin/main into refactor/invert-ai-object
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 09:35:39 -07:00
hanzo-dev 809ca44a3f refactor: delete the k8s membership file the move left behind
The previous commit built the new home for pod-listing membership
(clients/membership.K8s, installed into cloud.Peers from apps/) and added
membership_writers.go, but never removed the file it superseded. Both declare
membershipSource and httpPortOf in package cloud, so the root package did not
compile:

  ./membership_writers.go:19:6: membershipSource redeclared in this block
  ./membership_k8s.go:42:6: other declaration of membershipSource

Deleting it drops no behaviour: every symbol in it has a live counterpart in
clients/membership/k8s.go, whose test moved along with it, and apps/install.go
already wires cloud.Peers = membership.K8s.

  root package   995 -> 676 packages
  k8s.io         263 -> 0

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 09:35:10 -07:00
47370e10da fix(git): carry the verb in git's ZAP methods — clients/git was red on main (#366)
zapface stopped inferring the HTTP verb from a method name (98d5a7c1) because a
RESTful surface gives no signal to infer from: one path answers GET, PATCH and
DELETE, and a guess could send a delete as a post. Correct change. It left one
caller behind, and clients/git has been failing on main ever since.

git's three ZAP calls now carry POST, and the doc block in zap.go stops
describing the old verb-less mapping.

Worth noting where this hid: hanzo.yml's go-unit gate lists packages one by one
and clients/git is not among them, so a red package sat on main without a single
red build. That is the more expensive bug and it is not fixed here.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-27 09:34:02 -07:00
fa010e7ccf chore(git): one push path — delete the /v1/git/webhook door (#365)
A push landing on git.hanzo.ai already fans out in-process: receive-pack over
HTTP/SSH and the client-less /push all funnel through fireBranchBuild, which
fires the deploy trigger and the lifecycle stream. /v1/git/webhook was a second
door into that same funnel, by its own comment -- "the SAME funnel receive-pack
drives" -- for an external git server posting push events.

There is no external git server. git.hanzo.ai IS this binary's /v1/git plane;
the cluster runs no git deployment, only the CI runner. The route logged zero
hits across six hours on every pod. It was the last thing keeping GIT_WEBHOOK_SECRET
and a signature verifier alive.

Its pusher-identity loop guard goes with it, and nothing is lost: echo
suppression is the Origin seam, where it belongs. An inbound sync stamps the
source host on the lifecycle event and mirror_out skips the target that matches,
so a GitHub push cannot bounce back to GitHub. That guard covers every transport
rather than one.

GIT_SYNC_ACTOR stays -- clients/sync still uses it as a Sync's default actor.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-27 09:29:08 -07:00
hanzo-dev 4c1ec90d87 refactor: cut ai/object and the k8s client out of the root package
Every subsystem imports this package for Deps, so its import graph is the floor
under all 107 of them. Two imports put 1122 packages into that floor that no
subsystem needs:

  github.com/hanzoai/ai/object   1480 pkgs, used for four Set* calls
  k8s.io/client-go               263 pkgs, used by one file

Both only ever pushed outward — nothing here read a value back — so the values
stay where they are built and the registration moves to apps/, which already
links both. ai/object's four callbacks become plain function types declared
here; the pod-listing half of membership moves to clients/membership and is
installed into cloud.Peers.

  root package   1798 -> 676 packages
  k8s.io            263 -> 0
  aws                96 -> 0
  grpc               65 -> 0
  go-git             56 -> 0
  cloud.google.com   18 -> 0
  sigs.k8s.io        10 -> 0

Behaviour is unchanged. A nil callback is left uninstalled rather than
installed as nil, which is what already happened when a subsystem was not
co-resident, and Peers being nil gives the same static peer set that an
out-of-cluster process already fell back to. UsageEvent is duplicated on
purpose: sharing the ai module's type would reintroduce the import.

Link time is 96% of a one-line rebuild here and scales with reachable packages,
so shrinking the shared base is worth more than extracting any single app —
every app pays the base.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 09:27:12 -07:00
787969ab77 chore(git): drop the X-Gitea-* webhook fallback — one spelling, X-Git-* (#364)
The fallback existed so cloud and a separate git image could roll in either
order during a rename. There is no separate image: git.hanzo.ai is this
binary's own /v1/git plane, and the cluster has no git server deployment at all,
only the CI runner. Nothing is left that sends the old names.

This is what the constants themselves asked for -- "delete both the moment the
fork ships the new names, this is a rename in flight, not a compatibility layer
to keep."

The both-spellings test becomes a one-spelling test rather than being deleted,
so the header contract stays pinned.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-27 09:26:20 -07:00
b27d84f802 chore: call our forge Hanzo Git, not Gitea (#363)
git.hanzo.ai is the native /v1/git plane in this binary. It is not Gitea — that
host does not even answer Gitea's API — but nineteen comments still called it
that, and one of them sent me looking for a mirror-sync endpoint that does not
exist. A wrong name in a comment costs real time.

Renamed where we were describing OUR system, and dropped the upstream project's
name from the porting notes: what matters is that the pattern came from the
forge we seeded this plane from, not which forge that was.

Two categories are deliberately left alone. X-Gitea-Event and X-Gitea-Signature
are bytes on the wire, not branding — they are the pre-rename spellings the git
image still sends, already documented here as a rename in flight behind
X-Git-*, and renaming the constants would simply stop webhooks parsing.
And gitea.com stays in the provider lists beside github.com, gitlab.com,
bitbucket.org and codeberg.org: that is somebody else's host that a customer may
sync FROM, which is exactly the kind of thing we should keep supporting.

Comments and one yaml comment only; no code path changes.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-27 09:17:50 -07:00
hanzo-dev fe73340046 build(console): pin the embed to sha-147ecd3 — ships the live error plane
cloud.hanzo.ai (and console.hanzo.ai, which 301s there) serves the console
in-process via //go:embed, so the console error plane can only reach production
by moving this pin. The standalone console Deployment is already on v8.5.31, but
it backs only /auth/signup and the brand /v1 proxies — not the main UI.

sha-147ecd3 is console main and carries @hanzo/event 0.3.4, which resolves the
Sentry DSN from `product: 'console'`. Until it ships, the console reports ZERO
errors: <=0.3.1 had no envelope code, and no DSN could reach the bundle because
NEXT_PUBLIC_* inlines at BUILD time while the native builder passes only VERSION.

Moving the pin is what this file's own header asks for — the previous `:latest`
behaviour is exactly how cloud v1.801.215 silently baked the wrong console.

Built via POST /v1/runner (dockerfile: Dockerfile.embed, arch amd64); job
pf-runner-hufgh5wloutp Complete. The image needs a cloud release to reach prod.
2026-07-27 09:13:05 -07:00
7226ef5fb4 ci: drop the inert GitHub->forge sync workflow (#362)
This file has never done anything, and says so in its own header: "Inert until
hanzoai/cloud stops being a Gitea pull mirror." It also describes a system we do
not run — it calls git.hanzo.ai "this Gitea", but the forge is our own native
/v1/git plane in the cloud binary, not Gitea.

The direction it was meant to cover already works, through the sync engine
rather than a cron. Verified on the live fleet just now: the forge tracks GitHub
within minutes with nothing of its own outstanding, and a branch pushed to the
forge appeared on GitHub in under twenty seconds. Inbound is fast-forward only,
so a divergence is a conflict rather than a silent overwrite — the guard this
workflow was written to provide, enforced where the bytes actually move.

Keeping a scheduled job that duplicates a working path only creates a second
source of truth for "is main synced", and this one would answer wrongly.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-27 09:11:50 -07:00
hanzo-dev 65e8c5bd75 feat(search): fold the full-text index into the cloud binary
/v1/search is a per-org inverted index on Base/SQLite that speaks the
Meilisearch REST dialect, so a Meilisearch client repoints at it by changing
one host and needs no code change. It replaces the standalone Meilisearch
containers with a subsystem of the one binary, which is what gives it per-org
tenancy, encryption at rest via cek, and the platform's auth and o11y instead
of a separate process with its own master key and its own volume.

Tenancy is the reason this belongs in the binary. A standalone Meilisearch has
one global keyspace behind a master key, so every consumer sharing an instance
shares its indexes. Here the tenant is principal.Org — the value minted from
the VALIDATED bearer owner claim (HIP-0026), never a client-supplied header —
and every query filters WHERE org=?. Two orgs may both hold an index named
"messages" without either seeing the other's documents. The JS client already
sends `Authorization: Bearer`, so an org's cloud API key drops straight in
where the Meilisearch master key was.

The index is an ordinary term table, NOT FTS5. FTS5 is a compile-time module,
and cloud links the SYSTEM SQLite so the SQLCipher codec is real: that library
ships ENABLE_FTS3 and HAS_CODEC with no fts5 module, and the `sqlite_fts5`
build tag only affects the vendored amalgamation, so it is inert. An index
built on FTS5 opens fine on a pure-Go build, passes its tests, and then cannot
create a single table in the shipped binary. search_terms is keyed
(org, uid, term, pk) so a prefix query is an index range scan, behaves
identically in every build lane, and costs nothing but rows. Query terms are
OR-joined prefixes ranked by how many of them a document matches.

Verified in all three build lanes — pure-Go, the default cgo amalgamation, and
the production lane the Dockerfile uses (cgo + libsqlcipher + SQLITE_REQUIRE_CODEC).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 09:04:55 -07:00
hanzo-dev 358982636e refactor(esign): rename the sign subsystem to esign
The e-signature subsystem is named esign end to end: package esign in
clients/esign, the /v1/esign/* surface, the "esign" mount name, and the
matching hanzoai/esign repository. Sub-paths that name the ACT of signing
(/o/:org/sign/:token, the sign.view/field/complete/reject token scopes) keep
"sign" — those are the verb, not the subsystem.

Storage follows the subsystem name in two places, so the rename moves data:

  - Per-tenant document stores live at {DataDir}/{subsystem}/{tenant}.db, and
    the development signer's key material sits in the same directory.
    migrateDataDir carries the directory over on first boot, before anything
    opens a store under the new name. It is idempotent, and when BOTH
    directories exist it leaves them alone and says so rather than guessing
    which copy of a tenant's documents wins. A failure aborts the boot: serving
    an empty document store while signed documents sit orphaned under the old
    name would look like data loss to every tenant.

  - Blob keys are namespaced {subsystem}/{tenant}/ and derived per call, so PDF
    bytes written under the previous name are not addressable under the new one.
    Production holds no signed documents (its data directory has zero tenant
    stores), so nothing is stranded there; a self-hosted deployment carrying
    documents needs its object-store prefix copied alongside this upgrade.

Also adds the TestMain harness the suite never had. cek fails closed rather
than writing legal documents to disk in the clear, so without a dev master key
TestFullSigningFlow, TestTenantIsolation and TestSequentialSigningOrder all
failed at the first document write — on main as well as here. The whole suite
passes now.

The apps package doc claimed six subsystems were staged; only ingress is.
It now points at config.go's stagedSubsystems as the single source of that set
instead of keeping a second, drifting copy.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 09:04:55 -07:00
zeekay 871acf5964 feat(metrics): request metrics push to o11y over ZAP, not Prometheus scrape
The instruments lived in a private Prometheus registry behind an exposition
handler. That is a scrape model — something has to come and collect — and o11y
already receives metrics over ZAP, so cloud now pushes them the way it pushes
traces and logs. One transport for all three signals.

Two things this turned up, both worth stating plainly:

MetricsHandler had no caller. It was defined and never wired into the serving
path, so the exposition endpoint it describes was not actually being served by
this binary.

More importantly, cloud installed NO meter provider. The only one in the process
was a noop for the collector's internal telemetry. Converting the instruments to
OTel without fixing that would have bound every one of them to the global no-op
and discarded every measurement while the code still read as instrumented — the
exact failure this commit exists to avoid. The composition root now installs a
meter provider alongside the tracer provider, exporting through
luxfi/metric's OTel-to-ZAP adapter.

Cardinality is unchanged: product from the finite route table, status folded to
four classes, org from the validated IAM owner claim and never a client-chosen
string.

⚠️ Deployment: universe carries VictoriaMetrics scrape config
(infra/k8s/monitoring/vmsingle-scrapeconfig.yaml). Anything pointed at cloud's
/metrics should move to reading these series from o11y instead; a scrape target
that no longer exists goes quiet rather than loud.
2026-07-27 08:59:43 -07:00
antje 6f747339bf identity: a machine credential resolves its org from the subject
v1.801.244 403'd every existing customer API key on every ORG-SCOPED route
(/v1/agents "X-Org-Id required", /v1/gpus, /v1/billing/balance 401) while unscoped
/v1/models kept answering 200. Caught in the post-pin gate and rolled back.

The cause is the fail-closed rule applied to the wrong principal kind. Failing closed
on a missing `orgs` claim is right for a HUMAN token — a stale session must degrade to
nothing rather than to the minting app's org. But an hk-/sk- key is not a member of
anything, so IAM mints it no `orgs` claim at all, by design. homeOrg read the empty
membership set, resolved nothing, and no X-Org-Id was minted for a credential that was
perfectly valid.

A machine credential's org comes from the token SUBJECT — the rule IAM states for
itself in internal/authz/authz.go. iamKeys.lookup ALREADY does that resolution
(get-user?accessKey -> the user row's owner) and already caches it, so this needs no
new round-trip on the hot path: it records the resolved owner in an unexported
subjectOrg, and homeOrg prefers it. Unexported and untagged, so encoding/json can
never populate it from a token and no caller can forge it — the same rule the identity
headers follow: never decoded from the request, only minted from something verified.

isMachinePrincipal is NOT the discriminator for this. It is `type == "application" ||
KMS-audience`, and a key-resolved principal carries neither (lookup sets no Type and a
key has no audience), so it reports false for exactly the population that broke. The
two are different sets and are now treated as such.

Three kinds, three sources, deliberately not one fallback chain:

  - API key      -> subjectOrg (from the subject; no application involved)
  - machine JWT  -> owner. A client_credentials principal IS the application, cannot
                    choose which app it is, and needs that app's client secret to
                    exist at all, so there is no app-selection hazard. Omitting this
                    would fail closed on the "<org>-platform-kms" sync identity, whose
                    org-scoped access runs through this boundary (isKMSMachinePrincipal
                    gates ONLY the admin grant, precisely so that access keeps working).
  - human JWT    -> orgs[0], else fail closed. Unchanged, and the whole point: for a
                    human, `owner` is the app they logged in through.

The escalation stays closed: the SuperAdmin arm excludes machine principals, so the
machine branch reading `owner` cannot reach it, and a key principal resolves the same
org it resolved before this series began.

Tests drive a REAL key through the whole boundary into an ORG-SCOPED handler, which is
the shape that broke — verified non-vacuous by reverting the fix and watching them
reproduce the exact production symptom (403 on /v1/agents, 200 on nothing). Plus: pk-
still authenticates nothing, a legacy HUMAN token still fails closed so the two paths
cannot collapse, a machine JWT in the admin org is still not SuperAdmin, and a token
cannot smuggle subjectOrg.

Baseline: 15 pre-existing failures on clean main (SQLCipher wants a tmpfs this host
lacks), 15 after, empty diff — none introduced.
2026-07-27 08:56:08 -07:00
8b8c90c943 ci: retire the GitHub workflows — nothing can run them any more (#361)
arc-system was deleted today, which finishes what "remove GitHub CI — native
git.hanzo.ai + cd.hanzo.ai only" started. GitHub now has no runner of any kind:
hosted minutes are billing-blocked for this org, and the self-hosted pool these
files targeted no longer exists.

That is worse than it sounds. A job with no runner does not fail — it QUEUES,
forever. Every push leaves a permanently pending CI/CD and Sync run, and a
pending required check is a merge blocker, so the whole repo ends up needing
--admin to land anything.

Both files are gone rather than repointed because there is nowhere to point
them. The gate they provided still exists where CI actually lives:
.hanzo/workflows/cicd.yml runs on the git-runner StatefulSet (4/4). Mirroring
still happens too — the mirror pulls on its own interval and the App webhook
also drives it; the nudge only ever bought latency.

This also reverts my own change from earlier today, which moved the sync job
onto hanzo-build-linux-amd64 at the moment that pool was being torn down. It
traded a fast failure for an infinite queue, which is the wrong direction.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-27 08:47:28 -07:00
zeekay b3d3ce9e2b feat(o11y): the embedded collector ingests ZAP, not OTLP
otlpreceiver bound OTLP gRPC to 0.0.0.0:4317 and OTLP HTTP to :4318. 4317 is the
canonical ZAP port — the one luxfi/trace emits to and o11y/pkg/zapreceiver
listens on — so with the fleet migrated, cloud had an OTLP listener sitting on
the port receiving ZAP frames it cannot parse. This was a live protocol
collision, not only a dependency to shed.

zapingest.go is a collector receiver.Factory wrapping o11y/pkg/zapreceiver: it
decodes the JSON SpanBatch out of the ZAP envelope and hands pdata to the
pipeline's next consumer, so memory_limiter/resource/batch and the datastore
exporters are untouched. One receiver, one port.

Two conversion details worth knowing. Scope is not on the wire — SpanBatch
carries app name and resource attributes, not instrumentation scope — so spans
land in a single unnamed ScopeSpans rather than inventing one the emitter never
sent. And JSON has one number type, so an integer attribute arrives as float64
and is written back as an int when it is one, which keeps http.status_code
queryable as 502 rather than 502.0.

cloud's own source now imports no otlp package at all. What remains in go.mod is
indirect, pulled by the otelcol framework and prometheus/otlptranslator.

The factory-resolution test still passes, which is the load-bearing one: it
proves every component key in the rendered pipeline maps to a real factory, so
a config/factory drift fails at construction rather than at runtime.
2026-07-27 08:46:03 -07:00
hanzo-dev f138aeea61 refactor(cloud): follow the ai surface, and re-fix the gate main relocated
The ai routes moved to /v1/ai/<resource>; these are cloud's callers and the two
gates that key on their paths.

THE ONE THAT MATTERS. main had already consolidated the paywall into spend.go
("one spend predicate, and billable is not price") — and carried the stale paths
across with it. `Reachable` still listed /v1/signin, /v1/signout, /v1/get-account,
which are no longer routes. That list is what keeps the gate from refusing the
path to payment, so the old spellings put a 402 in FRONT OF SIGN-IN. The file
already documents this exact outage happening twice — once from a trailing slash,
once from casing — and it was live again, in the new location, before this commit.

account_principal.go is the second one, and it fails silently rather than loudly.
It fronts the account read with the validated principal and intercepts by PATH
MATCH, so a stale literal does not error — the middleware just stops firing, falls
through to the casibase surface, and hands the SPA the anonymous owner again. Its
own docstring explains that this is what bounced operators to login while the same
session got 200 from every /v1/admin route. The path is a named constant now, with
that reasoning attached to it.

Rebasing the original branch was not possible and should not have been: main
DELETED routers/paywall.go and paywall_test.go outright. Replaying a fix onto a
deleted file would have resurrected a dead gate beside the live one. The fix is
re-applied where the logic actually lives now.

Also here: the DO fleet board and the zapface verb contract, cherry-picked clean.
infra.Routes takes cloud.Router rather than *zip.App — main moved every sibling to
that interface while this was on a branch.

vet clean across the touched packages; the root package tests pass. The remaining
failures are environmental and reproduce on pure origin/main: cmd/admin and two
test binaries need native/flags/target/release/libhanzo_flags.a (a Rust artifact
CI builds and no worktree checks in), and clients/admin/audit needs
CLOUD_KMS_MASTER_KEY_REF.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 08:38:05 -07:00
hanzo-dev 98d5a7c15e fix(zapface): carry the HTTP verb, stop inferring it from the method name
The ZAP dispatcher chose its HTTP method by looking at the method NAME: a "get-"
prefix meant GET, everything else meant POST. That was sound only because the /v1
surface encoded its verb in every route name — /v1/get-store and /v1/update-store
are different names, so a name WAS a complete request description.

The RESTful surface removes that signal entirely. /v1/ai/providers/{owner}/{name}
answers GET, PATCH and DELETE at one path, and no prefix distinguishes them. Under
the old rule every one of those would have been sent as a POST: a read as a write,
and a DELETE landing on the create route instead of destroying anything. Silently,
with a 2xx.

So the method is now an HTTP request line — "<VERB> <path>":

    GET    ai/providers/acme/openai
    PATCH  ai/providers/acme/openai
    DELETE ai/providers/acme/openai
    POST   ai/providers

A method with no verb is REFUSED, not defaulted. Defaulting would convert a
caller's omission into a wrong-but-plausible request, and on a surface where the
verb decides between reading a resource and destroying it, a plausible wrong guess
is the worst available outcome. The integration test asserts the refusal names the
missing verb.

The fixture in the integration test now switches on METHOD AND PATH, which is what
made the old design's problem concrete: it is the same pair the dispatcher has to
carry, and the reason a name alone can no longer produce it.

Worth recording: this face has no first-party consumer left — console's src/lib/zap
is gone and nothing imports it — but /zap is still mounted in serve.go, so it is
fixed rather than left silently guessing. Whether to retire the mount is a separate
call. The cloud.event checkout carrying the same code is a WORKTREE of this repo on
another branch; it inherits this fix on merge and was deliberately not touched.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 08:38:05 -07:00
hanzo-dev eb6b93c09f feat(admin): DigitalOcean fleet board — inventory, safety analysis, controls
admin.hanzo.ai could show what we BILL but not what we RUN. The 13 TiB of block
storage, the 64 nodes, the 295 volumes were only visible in the DO console, with
no cross-reference to what Kubernetes actually mounts — so "is this orphaned?"
had no answer a human could act on.

/v1/admin/infra is that answer: the DO account inventory joined against EVERY
cluster's PersistentVolumes, folded into one classification per volume.

The safety rule is the whole point, and it is the mistake this was written after
nearly making. A volume is deletable ONLY when no PersistentVolume in ANY cluster
names it in spec.csi.volumeHandle. Specifically:

  - Reference beats absence. One PV anywhere protects the volume, no matter what
    the tags, the attachment state, or the other seven clusters say.
  - DOKS `k8s:<uuid>` tags are ADVISORY. They survive cluster deletion and they
    lie in both directions, so they are displayed and never trusted.
  - An incomplete scan freezes everything. If a single cluster is unreachable,
    the cross-reference is unsound, so NOTHING is deletable — not degraded, not
    best-effort. TestIncompleteScanBlocksEveryDeletion and
    TestUnreachableClusterFreezesEverything hold that line.
  - Idle is not orphaned. A Bound PVC no pod mounts is flagged for review and is
    never counted as reclaimable.

The server never trusts the client's verdict: DELETE re-runs a full fresh
cross-cluster scan and refuses anything that verdict does not clear, so a volume
that went live between page-load and button-press is protected. Deletes snapshot
first by default; the snapshot is the undo. Every mutation is audited, including
the denials.

Verified against the live account: 8 clusters, 64 nodes, 295 volumes, $8610.30/mo.
Of 13.19 TiB, exactly 3 volumes / 500 GiB / $50/mo are unreferenced — matching a
hand audit. The naive "no k8s tag" test would have proposed deleting 4.39 TiB of
live cluster data.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 08:38:05 -07:00
006b567852 ci(sync): run the mirror nudge on our pool — a hosted job never starts (#360)
The nudge has failed on every push since it landed, and not for any reason the
job's own fail-soft could catch: GitHub-hosted minutes are billing-blocked for
this org, so the run dies before step 1 with "The job was not started because
recent account payments have failed". continue-on-error protects a STEP, and
there is no step to protect.

Moving back to hanzo-build-linux-amd64 — the pool the previous revision of this
file used, for exactly this reason. Nothing else changes: the direction, the
mirror-sync call, the idempotency and the fail-soft behaviour are all as written.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-27 08:31:39 -07:00
antje 37adfe297c money: the starter grant follows the person, on the signed membership set
The abuse gate this seam needs is "only the caller's OWN org is funded".
The payer of record is the SELECTED org, and IAM's signed `orgs` claim lets
a person act in any org they belong to, so funding whichever wallet happens
to be paying mints $5 per org to anyone who can create them — unlimited, and
personal slugs auto-suffix.

Membership alone cannot express it: a founder is a member of every org they
create, so `selected ∈ orgs` admits all of them. POSITION can. IAM builds
the set home-first from the authoritative user row (store.MemberOrgRefs
seeds it with user.Owner before appending memberships), so orgs[0] is the
caller's own org and everything they merely joined or created follows it.

principal.Owner is now exactly that value: SanitizeIdentity mints
X-User-Owner from idClaims.homeOrg() — orgs[0].org — instead of the `owner`
claim (d14219415). That fix is what makes this gate reachable at all. `owner`
carried the minting APPLICATION's org, so for an onboarded account it never
equalled the ledger and this comparison could not match; the grant was
structurally unreachable, which is why v1.801.242 is inert. Reading the same
accessor SanitizeIdentity reads keeps one authority instead of a second
opinion about who the caller is.

Comparison is byte-exact, and safely so: isMember admits a selected org only
on `o.Org == org`, so X-Org-Id can only be a byte-identical entry of the
signed set and X-User-Owner is entry zero of it. A case difference would mean
the two did not come from the same claim — a reason to refuse, not to
normalise. An absent home refuses outright: a pre-v1.33.0 or machine token
carries no membership set, and the field it would otherwise fall back to is
the one just removed for being wrong.

Everything else stands unchanged: new-not-unseen (zero balance AND zero
lifetime usage, so first contact after a deploy never pays a retroactive $5
to the existing customer base), idempotent on the address, one map load on
the hot path, never rejects. 12 middleware tests + 9 money proofs green under
-race against the shipped engine. Enforcement stays off.
2026-07-27 08:19:53 -07:00
81142c8561 refactor(git): read through a Repository, not through go-git (#359)
/v1/git had no model. The JSON browse handlers took *gogit.Repository and
*object.Commit as arguments, the HTML twin took the same, and the code
intelligence feeder opened go-git itself to walk a tree. Git was not an
implementation of version control here; it WAS the thing, spread across eight
files.

This puts the value back in the middle. A Repository is a named, versioned
content tree: refs name revisions, a revision has a tree of paths, a path holds
bytes. That is the whole model, and it is all any reader ever wanted — resolve a
ref, then read Tree, Blob, Log or WalkText. Revision is opaque on purpose; it
holds a commit sha only because the git backend put one there.

go-git now lives in three files instead of eight: the backend that implements
the model, the write path that builds commit objects, and repository
initialisation. browse.go, ui.go, index_on_push.go, index_on_import.go and
build_on_push.go no longer import it at all. Adding hg or svn becomes a backend
beside gitbackend.go — it does not touch a single reader.

Two things came out in the wash. Failures now carry sentinel errors, so a
handler can tell "no such ref" from "this repository is broken"; browse.go used
to answer 404 "unknown ref" for both. And WalkText takes the per-file size cap
rather than leaving it to the callback, because the caller has to skip an
oversize blob BEFORE reading it — that skip is what keeps a multi-GB file out of
a 1 Gi pod, and it only works where the size is known cheaply.

Behaviour is unchanged and deliberately so: the ref-resolution fallback, the
dirs-before-files ordering, the binary check, the README candidate list and the
seven-character shortSha are all carried over verbatim. The HTML surface keeps
its eight-character short form, which has always differed from the JSON one —
worth fixing, not worth hiding inside a refactor.

The boundary is enforced by a test that parses the package's imports, and it was
verified to fail when a go-git import is reintroduced into browse.go. A boundary
that is not enforced is a comment.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-27 08:10:27 -07:00
Antje Worring 94066ef9c6 money: a balance says WHICH account it is
GET /v1/billing/balance answered {balance,holds,available} — three numbers
with no subject. So "which account am I about to fund?" had no answer a
client could trust. A browser could only guess by decoding its own token,
and a guess that disagrees with the server is exactly how money lands in an
account the spend gate never reads — the same class of split that once
showed a funded org while the gate refused the member.

Echo the subject instead. balance() already resolves it to READ the wallet;
that one value is now also REPORTED, so display and read cannot drift. It is
called once and used twice rather than resolved a second time for display.

`account` is cloud-resolved via the one rule (account.Payer) and is never
decoded from upstream — commerce does not send it — so it is empty on the
split-deploy proxy path and omitted there rather than rendered blank.

This is what lets a checkout name the payer BEFORE the customer pays, from
the same resolution that will actually be credited.

The test asserts the reported account equals the subject the ledger was read
with, across all three identity shapes a validated principal arrives in
(gateway-minted X-User-Name, in-binary direct bearer, and the owner/name id
that folds back via PayerOf). Verified to FAIL without the change.

Read-only: no money semantics change, no new route, one field.
2026-07-27 08:07:30 -07:00
zeekay 6a537e6c14 feat(o11y): cloud's spans reach the sink as pdata, not OTLP proto
The in-process trace sink was described as Cost-0 — the batch delivered by
value, never serialized. It wasn't. Reaching dstraces.ConsumeTraces, which takes
ptrace.Traces, went SDK spans -> otlptrace.Exporter -> OTLP proto ->
proto.Marshal -> ptrace.UnmarshalTraces. Every batch was marshalled and
immediately unmarshalled in memory, and that round-trip is what pulled
go.opentelemetry.io/proto/otlp and otlptrace into the binary.

spanconv.go converts SDK spans to pdata once, in place: grouped by resource then
scope so a batch from one service yields one ResourceSpans, with identity,
timing, status, events and links carried across. The sink's payload type changes
from []*tracepb.ResourceSpans to ptrace.Traces on both sides of the contract, and
the router now speaks sdktrace.SpanExporter instead of otlptrace.Client.

zaptrace is deleted. It existed only to implement otlptrace.Client, an interface
nothing here implements anymore; luxfi/trace owns the ZAP span wire and is now
the fallback when the in-process sink is not mounted.

otlptrace and proto/otlp drop to // indirect — cloud's own source imports
neither. What holds them is the embedded collector's OTLP receiver, a separate
surface.

Tests: the converter is pinned on grouping, parent linkage, attributes and
timestamps, because a converter that silently drops a parent link still looks
like it works. Two test bugs were fixed rather than worked around — one depended
on a package-level slice another test happened to fill first, and one passed a
nil batch to exercise routing, which now short-circuits by design since the SDK
may legitimately export nothing.
2026-07-27 08:03:42 -07:00
hanzo-dev be3b6e97ea fix(apps): name the engine that actually owns "sqlite" without cgo
The one-engine allowlist listed github.com/hanzoai/sqlite/internal/engine as the
cgo-free entry, "keyed through the hanzoai/sqlcipher codec VFS". No such package
exists: the fork has no internal/ directory, and driver_nocgo.go blank-imports
modernc, which registers "sqlite" in its own init. So on a pure-Go build modernc
owns the driver, the allowlist did not admit it, and the guard failed on the
build `make test` actually runs.

Added with the reason the guard asks for. The invariant it protects is unchanged
and still enforced: exactly ONE engine, reached only through the fork — nothing
here imports modernc directly, which is what would make this a real finding
rather than a naming gap. Nor is at-rest encryption lost by that engine being
unkeyed: cek wraps it in the pure-Go SQLCipher envelope, which reads and writes
the same page format as the C codec.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 08:02:22 -07:00
hanzo-dev 610d68c9be ci: nudge git.hanzo.ai to pull on push
git.hanzo.ai mirrors this repo by PULL on a ~10-minute interval, and arcd runs
CI/CD there — so every push waited out that interval before anything built.
This asks Gitea to pull HEAD immediately.

Latency only: the repo already mirrors via the App webhook, so a missing
HANZO_GIT_TOKEN or a failed curl is non-fatal and never fails the push.
Idempotent (mirror-sync just pulls HEAD) and concurrency-coalesced.
2026-07-27 08:01:43 -07:00
antje d142194156 identity: the home org is the USER's, never the minting app's
The `owner` claim has never carried the user's organization. IAM stamps the
APPLICATION's org into it (oidc/jwt.go Sign: `Owner: app.Organization`), so the same
person authenticating through two apps presented two different orgs. It read as
correct for years only because, pre-onboarding, the app org and the user org were
both "hanzo"; onboarding broke the coincidence, not the claim.

Cloud consumed exactly that field, and fed it to two different gates:

  - the billing anchor (owner -> effOrg -> X-Org-Id -> BillingOrg), which made the
    paying tenant CALLER-SELECTABLE: a hanzo user authenticating through lux-cloud
    spent lux's ledger. lux-cloud/zoo-cloud/pars-cloud are all live accepted
    audiences, so this crossed brands, not merely accounts.
  - the SuperAdmin predicate (owner == adminOrg), which made platform sudo a
    property of the app you logged in through. admin-console and hanzo-admin-guard
    are both org=admin with the tenant gate (orgChoiceMode) OFF.

One poisoned value, two defects, one accessor: idClaims.homeOrg reads orgs[0].org
from the signed membership set, which IAM builds home-first from the authoritative
user row (store.MemberOrgRefs). SanitizeIdentity reads that everywhere it used to
read `owner`, so both close together.

IAM knew: internal/authz/authz.go refuses these claims internally and says the org
"comes from the token SUBJECT ... never from the token's `owner`/`organization`
claims", and the Sign/SignUserToken pair documents the divergence while naming
cloud's SanitizeIdentity as the consumer. No IAM change is needed or wanted —
rewriting `owner` would move every personal-account actor out of the shared hanzo
ledger and silently change the claim for every relying party.

FAIL CLOSED on a token that names no home org (pre-v1.33.0, or a machine token, for
which IAM omits the claim). It must never fall back to `owner`: that is the
app-selected value being removed. An unresolvable home org grants no scoping at all
and is logged at WARN with the audience, so a real legacy principal is visible
rather than silently denied. Bounded by token TTL.

Membership of the admin org remains the WHOLE SuperAdmin predicate; the isAdmin bit
is deliberately not added as a second term. Reading the user's own org is what
closes the escalation, and requiring isAdmin would deny every operator whose row
lacks the bit — a lockout, not a hardening. That contract is pinned by
TestSuperAdminGate_IsAdminOrgMembership ("ONE predicate, no second signal") and
relied on by TestMasqueradeSpendsOwnBooks, whose SuperAdmin carries isAdmin=false.
The summary comment claiming `claims.isAdmin && owner == adminOrg` was wrong twice
over and is corrected rather than implemented.

Fixtures: tokenClaims now seeds orgs[0] == owner, the ordinary case where the app
org and the user org agree. Every fixture previously set `owner` alone — a token
shape production has not emitted since IAM v1.33.0 — and because the two values were
always equal in tests, no assertion could tell them apart. That is precisely how a
caller-selectable tenant and a caller-selectable admin gate survived a green suite.
Tests needing the two to disagree, or modelling a pre-claim token, now say so.

TestLegacyTokenCannotSwitch changes contract deliberately: it asserted a claim-less
token "stays pinned to home" and expected the app's org. Pinning to it IS the
vulnerability. It now resolves nothing — the original intent ("no token gains
reach") strengthened from one org to none.

Baseline verified: 15 pre-existing failures on clean main (SQLCipher needs a tmpfs
this host lacks), 15 after, empty diff — no failure introduced.
2026-07-27 07:57:48 -07:00
antjeandhanzo-dev 69ca63a2ca ci: one live gate per ref, and a bound on how long a wedged job holds a runner
The build fleet jammed twice last night, ~90 minutes each time, and both times
we did it to ourselves. Six pushes to main between 06:40 and 06:51 started six
CI/CD runs at once (30243491724, 30243825902, 30243830673, 30243919868,
30243927270, 30244050708); each holds a hanzo-build-linux-amd64 runner for its
whole length, 13-26 minutes observed. The nodes starved, runners died mid-build
with "the self-hosted runner lost communication with the server", and ARC left
the EphemeralRunner objects Running against runs that had already finished — so
the listener scaled correctly over corrupt state and never recovered.

Nothing kept the copies apart. .github/workflows/cicd.yml is now the only
workflow GitHub runs for this repo, and it carried no concurrency group at all.
The one that used to, containment.yml, got that group in 2b1c37a7d and then was
folded into .hanzo/workflows/cicd.yml, which kept the group and lost the
timeout. Both halves are back, each where it belongs.

Cancelling is safe for this caller specifically: it never fires on a tag ref, so
it cannot kill a release — the cloud image and its v* tags belong to
clients/platform/release.go — and the one image hanzo.yml declares, cloud-flags,
is consumed through an explicitly pinned FLAGS_IMAGE ARG, never "whatever this
run just published". A publisher pairing an image with a tag must keep
cancel-in-progress: false; a guard must not.

timeout-minutes cannot go on the caller: GitHub rejects it on a job that `uses:`
a reusable workflow. So the gate's own bound has to live in hanzoai/ci's
reusable, which still inherits the six-hour default — designed, not shipped
here, because that means moving the shared v1 tag while that repo is mid-cutover
(cloud's .hanzo caller already points at a @v2 tag that does not exist yet).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 07:53:07 -07:00
hanzo-dev f5efdd83de fix(tasks): the embedded SPA is served at /tasks, not /_/tasks
The embed test required index.html to reference /_/tasks/assets/. Nothing serves
that prefix: tasks.go mounts the UI with StripPrefix("/tasks", …) behind /tasks
and /tasks/*, and embed.go documents the same. The committed bundle is built with
base /tasks/ and is correct; the assertion was left behind by the move.

Kept as a real check rather than deleted — it is the guard that the bundle's base
matches the mount, and a mismatch loads tasks.hanzo.ai as a blank shell with every
hashed chunk 404ing. It now names the path that actually serves it, and says what
breaks when the two disagree.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 07:50:44 -07:00
hanzo-dev cce5d79800 fix(migration): verify the output after the writer has finished, and through cek
The round-trip test read its destination files while the dst pool was still open
and with a bare sql.Open. Both are wrong for what the migration actually writes:
a dst handle holds an open transaction until it closes, and on the pure-Go
backend the ciphertext at the real path is only written when the handle seals, so
the verification was reading an unfinished file — and the files are encrypted, so
a plain open reports "file is not a database" even once they are complete.

It now closes the pool before reading, which is what the migration itself does
before its output counts as done, and reads the destinations through cek. The
deferred Close stays as the error-path net (it is safe twice). The source is
left as a plain file on purpose and now says so: it stands in for POSTGRES, not
for a cek store.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 07:49:16 -07:00
hanzo-dev cea2b7d7ba ci: remove GitHub CI — native git.hanzo.ai + cd.hanzo.ai only
This workflow ran on a self-hosted actions-runner-controller pool. ARC is
being decommissioned: CI executes inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.

Removed rather than repointed — there is no GitHub-side pool to move to, and
leaving it would keep ARC load-bearing.
2026-07-27 07:20:23 -07:00
hanzo-dev f2684575af ci: remove GitHub CI — native git.hanzo.ai + cd.hanzo.ai only
This workflow ran on a self-hosted actions-runner-controller pool. ARC is
being decommissioned: CI executes inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.

Removed rather than repointed — there is no GitHub-side pool to move to, and
leaving it would keep ARC load-bearing.
2026-07-27 07:20:20 -07:00
hanzo-dev a7ca7036dc ci: remove GitHub CI — native git.hanzo.ai + cd.hanzo.ai only
This workflow ran on a self-hosted actions-runner-controller pool. ARC is
being decommissioned: CI executes inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.

Removed rather than repointed — there is no GitHub-side pool to move to, and
leaving it would keep ARC load-bearing.
2026-07-27 07:20:18 -07:00
zeekayandhanzo-dev 9ee5b8cc62 commerce/transport: refuse the placeholder instead of dialing it
CI/CD / containment (push) Successful in 2m28s
Hanzo CI/CD / cicd (push) Canceled after 9m10s
CI/CD / gate (push) Canceled after 9m10s
PlaceholderBase names no host by construction, so when no in-process handler is
published, falling through to http.DefaultTransport can only ever fail — and it
fails as `dial tcp: lookup commerce.inproc: no such host`, three layers away from
the real cause.

That is what happened on 2026-07-27, and it cost most of a day to find.
commerce.Embed refused to boot (COMMERCE_KMS_MASTER_KEY unset on a
libsqlcipher-linked build — a CORRECT refusal, the store is plaintext), so Mount
never reached SetApp. Metering had already pinned PlaceholderBase because
Enabled("commerce") was true, so every balance read DNS-failed, every wallet read
0, and every paid call 402'd "Insufficient balance" against funded accounts. The
ledger really did hold the money; nothing could reach it. The error blamed DNS.

The braid: Enabled("commerce") is a CONFIG INTENT, a published handler is a
RUNTIME FACT. Metering treats the first as proof of the second, and they can
disagree. This does not un-braid them — that belongs where Mount handles Embed's
failure — but it stops the disagreement from being reported as a network problem.
A seam that cannot serve should say so, not guess.

Scoped to the placeholder only: a split-deploy against a REAL commerce URL still
falls back to plain HTTP, unchanged, and there is a test pinning each direction
(refusal names the path and never says "no such host"; a real host still dials).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-27 07:11:23 -07:00
2657ad4525 feat(admin): fail-closed white-label tenant admission for the operator cockpit (#343)
admin.hanzo.ai (apps/operator) is served DIRECTLY off cloud-api /v1 (the
hanzo_iam_token/cloud_session_id path SanitizeIdentity validates), so the
/v1/admin/* gate is the only boundary. GuardScoped previously admitted ANY
org-admin (principal.IsOrgAdmin) — so any customer's own-org admin could open
the cockpit. Tighten it to the WL admission tier:

  super (owner==AdminOrg, cross-tenant)  OR
  org-admin of an org in State.WLTenants (subtree-scoped via ResolveScope)

- core.State.WLTenants: fail-closed allowlist (nil/empty => SuperAdmin-only),
  seeded ONCE from ADMIN_WL_TENANT_ORGS at Mount. IsWhiteLabelTenant is the one
  read (verbatim, trimmed, no fold).
- core.GuardScoped now requires super OR (IsOrgAdmin && Org in WLTenants);
  a non-enabled org-admin gets the same 403 as a member/forge. Fleet god-views
  stay core.Guard (super-only) — a WL tenant never reaches a fleet number.
- /v1/admin/me returns isWhiteLabel + scopeOrgs so the SPA renders the subtree
  cockpit server-authoritatively (super=fleet, WL=own subtree).

Tests: WL tenant admitted+own-subtree; non-WL org-admin refused on every scoped
panel; WL tenant 403 on every god-view; fleet-only default (empty set) fail-closed;
IsWhiteLabelTenant unit-pinned. Harness enables 'maxpower' as the WL test tenant.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-27 07:11:23 -07:00
antje dab3067771 money: fund a new account at first contact, not at a handler
CI/CD / containment (push) Successful in 3m0s
Hanzo CI/CD / cicd (push) Successful in 9m9s
CI/CD / gate (push) Successful in 9m9s
v1.801.241 shipped the grant wired into cloud's /v1/iam/onboard first-run
branch. It was present in the binary and NEVER FIRED, twice over:

  - api.hanzo.ai/v1/iam/* is routed to the IAM SERVICE at the edge, so
    cloud's onboard handler is not on that path at all;
  - on console.hanzo.ai, where cloud does serve it, `additional :=
    cr.owner != ""` reads the EFFECTIVE org. A fresh IAM-v2 signup lands
    in the shared signup org, so every signup takes the "additional"
    branch and first-run is unreachable.

Both failures share one shape: the trigger depended on WHICH HOST served
the request and WHICH COMPONENT created the org, and both vary. This seam
depends on neither. It asks the only question with a stable answer — does
the wallet this credential spends from exist yet — in the binary that owns
the ledger, so it cannot be routed around. The onboard wiring is removed
rather than left beside it: one way, not two, and the dead one was dead.

Address unchanged and still the gate's own: principal.WalletOf, passed
through as CreditInput.Subject, idempotent on `starter:<subject>` — no
timestamp, no nonce, no request id — so a retry, a re-login, a restart
that empties the cache and a concurrent burst all derive one key and
finance dedups inside the insert transaction.

Two guards this seam needs that a handler did not:

  ELIGIBILITY IS "NEW", NOT "UNSEEN". On the first request after a deploy
  every existing org is unseen; granting on that alone would hand a
  retroactive $5 to the whole customer base. A wallet qualifies only with
  a zero balance AND zero lifetime usage — a funded org is skipped, and so
  is one that spent down to exactly zero, which balance alone cannot tell
  from a new account.

  HOME ORG ONLY. The payer of record is the SELECTED org (9bafa850c), and
  IAM's signed `orgs` claim lets a person act in any org they belong to —
  so funding whichever wallet is paying would mint $5 per org to anyone who
  can create them. Requiring selected == home (X-User-Owner, unforgeable)
  makes the grant follow the person. It costs nothing legitimate: IAM's
  first-run provision moves a founder into the new org, while additional
  orgs are created without moving anyone and are therefore never funded.

Hot path: mounted app-wide ahead of both gates so a new account is funded
before anything asks whether it can pay. Steady state is one map load;
only a wallet's first request in a process reads the ledger. It never
rejects — funding is not an authorization decision, and a failed grant
leaves $0, which the gate refuses on its own.

Proven under -race against the shipped engine (cgo + real libsqlcipher):
package apps re-derives the gate's address via account.Payer and reads
THAT, never the grant's return value, with a negative control showing a
pool grant is invisible to a signup-org member; package cloud pins the
cost (50 requests, 1 ledger look) and every principal that must not be
funded. Enforcement stays off.
2026-07-27 00:55:28 -07:00
antje 7ddad4087f money: a new account is funded at the address the gate reads
CI/CD / containment (push) Successful in 1m59s
Hanzo CI/CD / cicd (push) Successful in 9m24s
CI/CD / gate (push) Successful in 9m26s
SpendGate ships dark for one stated reason (middleware_spend.go): "no
ensureStarterCredit runs anywhere". So a brand-new signup's wallet is $0
with no self-service way to fund it, and enforcing on that state trades a
revenue leak for a total signup outage. This is that funding path. It does
NOT flip the switch — that stays a separate, deliberate act.

The grant is only real if it lands where the gate reads, and that has gone
wrong twice before (clients/principal/wallet.go names both). The gate
resolves its address as principal.WalletOf -> account.Payer(...).Subject().
IAM v2 mints no `billing_account` claim, so Payer takes its documented
fallback, and for an owner that is not the shared signup org that fallback
is Org(owner) — whose Subject() IS the bare slug, the pool. An empty
CreditInput.Subject addresses exactly that account. Grant and gate meet by
construction, not by coincidence.

It fires in cloud's onboarding handler because that is the one place that
holds both halves. IAM owns provisioning but has no ledger and forbids the
write (schema/user.go: "authoritative in Commerce, not here"); granting
there would put a commerce dependency inside the provisioning transaction,
where a blip becomes a failed signup. Commerce holds the ledger but not the
credential, and who pays is a property of the credential. Onboarding is
also the account-creation event for money: federated and email-code signup
create a USER but no org, so every real path converges here.

Idempotent on the ADDRESS — no timestamp, no nonce, no request id — so a
retry, a double-submit, a re-login and two concurrent onboards all derive
one key; finance dedups on it inside the insert transaction. Additional
orgs get nothing: creating them is unlimited, so granting per-create would
be a scriptable $5 mint. Amount is the server's shared constant; no client
field reaches it. A failed grant leaves the account at $0, which the gate
refuses — failure is never mistaken for funding.

Proven against the shipped engine (cgo + real libsqlcipher, -race): the
balance is read back through the gate's own address rule, never the grant's
return value, plus a negative control showing a pool grant is INVISIBLE to
a signup-org member's person wallet — the trap this addressing avoids.
2026-07-27 00:06:49 -07:00
hanzo-dev 96347e35ff fix(sites): the purge tests never built a usable Purger
They constructed it as a bare struct literal, which left two fields only NewPurger
supplies. pending was nil, so admit panicked on its first write — the whole
package aborted there. ceiling was zero, and takeToken refuses when
inMinute >= ceiling, so even past the panic every call was rate-limited away and
no request reached the server.

One helper now supplies both, keeping the zero coalescing window the tests want so
each call is admitted and the assertions stay about the request rather than the
debounce. NewPurger itself is still not used here: it reads the real environment
and arms a 10s window.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 23:50:34 -07:00
hanzo-dev 8bebb1a2c8 fix(link): a month is a window class, not a lifetime counter
The windowless-counter test drove a WindowMonth sample and required it to key the
zero instant. Month is one of the four window classes and carries a nominal
duration, so Sanitize resolves it to that month's instance — by design, since
every sample of a valid window needs a representable idempotency key or two
reports of the same window would store the same consumption twice.

Windowless means no class at all: a meter reporting a lifetime total names no
window, nominalMinutes yields zero, and the sample keys the zero instant — one
row per lane, forever replaced. The test now drives that, and also pins the other
direction, so a future change that let a real class collapse onto the zero
instant (silently merging every month into one row) fails here.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 23:48:15 -07:00
zooqueen fc03abaf2a Merge github/main
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 23:48:05 -07:00
hanzo-dev 76fc7f3cb6 docs(account): the per-user key is not an hk- key any more
Eleven comments across the account key surface called it "the user's `hk-` key".
Cloud does not choose that prefix — mintUserKey just returns whatever IAM's
mint-user-keys hands back, and IAM's newAccessKey() has returned keys.Mint("sk")
since the key seam was unified. The only "hk-" literal in this package is in a
test fixture.

So the surface is described by a prefix it no longer issues, which makes the
retirement look like a breaking change to a live shape when it is really a
re-key of a fixed, shrinking set of stored values. Named for what it is — the
per-user Cloud API key — so the prefix is IAM's business, where it is decided.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 23:47:54 -07:00
hanzo-dev 934599fdbf fix(gpu): stop killing a studio that is still loading its model
A booting studio and a dead one look identical to the supervisor. While the
41GB Qwen-Edit model pages in, the HTTP server is not listening yet, so
studioHealthy() is false and studioBusy() returns ok=false — which also
disqualifies the "alive-busy" guard that exists to tolerate slow probes. The
death counter therefore ran during startup: studioStartWindow (180s) plus three
45s ticks meant the process was killed ~5 minutes into a load that needs 5-11
minutes on a loaded box, and relaunched into exactly the same fate.

The result was a silent non-converging loop — 5 restarts in 30 minutes, zero
renders, every claimed job failing "engine not up: connection refused" — that
looks like a hung GPU rather than a supervisor bug.

The fact that separates the two cases is whether the process still exists, so
liveness is now deferred for a live child inside studioBootGrace. A child that
actually exited is restarted immediately, so a real crash is not papered over.
This is the startup-probe/liveness-probe split, which this supervisor lacked.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 23:47:46 -07:00
hanzo-dev bf6eeef277 fix(research): wire the ship checkpoint the composition root wires
Hanzo CI/CD / cicd (push) Canceled after 1m3s
CI/CD / gate (push) Canceled after 1m3s
CI/CD / containment (push) Canceled after 1m8s
The durable-record tests built durability with no WithCheckpoint, while build.go
passes WithCheckpoint(durableCheckpoint) precisely so ship-before-ack folds the
WAL into the real path before reading it. Without it the snapshot reads a file
the backend has not written: the pure-Go envelope keeps the plaintext on tmpfs
and re-encrypts to the real path only on Checkpoint or Close, so the ship either
finds stale ciphertext or, on a store never yet closed, no file at all — which is
what the failure said.

The harness now wires the same seam through sqlitedrv.Checkpoint, a no-op on the
write-time-encrypting backends, so the tests exercise the shipping path the
composition root actually builds rather than one missing its checkpoint.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 23:46:24 -07:00
hanzo-dev 2b1c37a7d9 ci(containment): one live check per ref, and a bound on how long it can hold a runner
Hanzo CI/CD / cicd (push) Canceled after 2m0s
CI/CD / gate (push) Canceled after 2m0s
CI/CD / containment (push) Canceled after 2m4s
This guard runs a full `go build ./...` on every push to main. Nothing kept the
copies apart, so a busy hour had eight of them in flight at once, each holding a
hanzo-build runner for tens of minutes to answer a question about a commit that
main had already moved past. The pool is not short of capacity — it is allowed to
100 and was running fourteen — it was being spent re-checking superseded trees.

Cancelling is safe here because this is a guard, not a publisher. It answers
whether the tree at a ref violates containment, and the only tree that can still
violate anything is the newest one, which the surviving run checks. release.yml
sets cancel-in-progress: false for the opposite reason: killing it between pushing
an image and writing its tag would lose work. Nothing here is lost by stopping
early.

The timeout is the other half. Runs were dying at exit 130 with "runner has
received a shutdown signal" — an ephemeral runner reclaimed mid-build — and with
no timeout-minutes the job otherwise inherits GitHub's six-hour default, so a
wedged one sits on a runner for the rest of the morning. 75 minutes is generous
against a cold module cache; observed runs are 26-42.

The three checks are untouched.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 23:46:19 -07:00
hanzo-dev 896ee9acc5 fix(zt): the unconfigured contract is empty projections, not 503 everywhere
The test required every read to 503 when ZT has no credential. Two of them
deliberately stopped: an unconfigured deployment genuinely has no networks and no
edge nodes, so those lists return an honest-empty 200 rather than turning a clean
"nothing here yet" console into an error on every page load. An empty list
discloses nothing, so that is presentation, not a relaxed gate.

The gate is unchanged and the test now says which side of it each route is on:
networks and edge nodes answer empty; mesh services — a real inventory, not a
per-org projection — and a specific network lookup, which cannot honestly be
"empty", both still 503. Renamed to stop promising 503 everywhere, and it now
also asserts the empty responses disclose nothing.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 23:40:12 -07:00
hanzo-dev 89ff76e436 fix(test): two more suites that assumed a plaintext, shareable store
prompts built its legacy-schema fixture with a bare sql.Open, leaving a plaintext
file. Converting one to encrypted is a production operation needing the live
codec, so the fixture failed the build the suite runs on before reaching the
migration it exists to test. It goes through cek now, as the agents fixture
already does.

team's persistence case reopens the workspace store on a fresh handle mid-test
and expects the first handle's committed writes. On the pure-Go envelope each
handle holds its own RAM copy and seals on close, so the second sees the last
sealed state — envelope.go documents that. The property is real on the codec-
linked build the image ships, so the case is scoped to it like the audit and kms
guards, and `make test-codec` runs it.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 23:29:28 -07:00
antje 013fb34a91 money: one address for the gate, the debit and the grant
CI/CD / containment (push) Successful in 4m44s
Hanzo CI/CD / cicd (push) Successful in 10m12s
CI/CD / gate (push) Successful in 10m14s
The money's address is (ledger, account). The zen path carried only the
ledger and let the gate read the org POOL, so for a member of the shared
signup org — where members are strangers to each other and each holds
their own account — zen gated a pool while the ai path debited the
person. Both halves of that broke at once: a brand-new $0 signup read the
signup org's funded pool and served for free, and a member who had bought
credit was refused because the purchase had landed in that same pool.

This is the THIRD recurrence of one bug, and clients/principal/wallet.go
already catalogues the other two. Every time, two layers derived one
address two ways. So nothing here re-derives:

  - cloudTenantResolver calls principal.WalletOf, the same resolver the
    edge gate calls, and carries BOTH halves in zen.Tenant (zen v1.4.4
    adds Wallet + the Payer method, where the default lives so a gate and
    a meter cannot pick it differently). The zen gate and the zen debit
    read that ONE expression. Tenant.User stays the actor: for a machine
    key the payer is the org while the actor is the key, so an address
    read off the actor bills the wrong account.
  - an unresolvable principal now yields the ZERO tenant, which zen
    refuses, instead of a half-filled one. The ok-bit is the answer.

Fixing only that direction would have made the other symptom worse — the
gate would read a member wallet no funding path could reach. So the
credit side moves with it. creditledger.CreditInput gains Subject
(commerce v1.49.24; empty = the pool, so every tenant org is unchanged),
the admin grant resolves its target through principal.WalletFor — the
same account.Payer the spend gate asks, which is why naming a member of a
POOLED org still credits that org's one balance — and every balance read
around the grant uses the credited account rather than the pool, so the
audit trail stops reporting a before/after that never moved. The
idempotency key now hashes the SUBJECT: two members, one nonce, one
amount are two grants, and hashing the org made the second dedupe away.

POST /v1/admin/finance/deposit is deleted. It existed only because the
grant could not name a member, and it was the unsafe way to do it — no
cap, no audit row, no idempotency ref, so a double-click credited twice.
ONE credit-write path: core.ApplyGrant.

metering's AuthInput said the gated balance is "always the org via User".
That contract is what zen.go was obeying, so it is retired here in favour
of the address doctrine; leaving both is how a fourth recurrence starts.
2026-07-26 23:16:38 -07:00
hanzo-dev 43ff6f1638 build: add the target that runs against the engine the image ships
Hanzo CI/CD / cicd (push) Canceled after 2m25s
CI/CD / gate (push) Canceled after 2m25s
CI/CD / containment (push) Canceled after 2m37s
Neither existing test target links a codec, so cek falls back to the pure-Go
envelope and every test pinning the shipped storage posture skips. The suite can
be green while nothing has exercised what production runs.

test-codec builds with the image's tags. The tag alone is not sufficient: it
selects the C engine, but the codec is a runtime probe of the libsqlcipher that
engine links, and a csqlite built against plain SQLite compiles and satisfies the
one-engine guard while CodecLinked() stays false — the storage tests would go on
quietly skipping. So it carries SQLITE_REQUIRE_CODEC=1, the same assertion the
Dockerfile makes before building /cloud, and either exercises the real engine or
fails saying it cannot.

It therefore fails on a machine without SQLCipher, which is the honest result and
the reason it is a separate target rather than the default.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 23:14:10 -07:00
hanzo-dev 42e5895107 fix(admission): stop asserting two retired key prefixes are credentials
CI/CD / containment (push) Successful in 2m26s
Hanzo CI/CD / cicd (push) Canceled after 4m34s
CI/CD / gate (push) Canceled after 4m34s
The waitlist-exemption test drove fw_ and hz_ keys and required them to be
admitted as paid inference. Both were removed from APIKeyPrefixes in
auth_identity.go — the single source this package mirrors — as families nothing
ever minted, which only widened what counts as a credential. The middleware
tracks that list correctly; the test did not, so it failed while demanding the
surface be reopened.

It now drives the three families cloud actually mints. The money-critical
property it exists for is unchanged: a real API key still flows through a
waitlist-gated host.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 23:09:41 -07:00
hanzo-dev 4e3d2f55c0 mount: a subsystem's middleware reaches its own routes, not the binary
CI/CD / containment (push) Successful in 2m26s
Hanzo CI/CD / cicd (push) Canceled after 2m37s
CI/CD / gate (push) Canceled after 2m37s
Mounting the embedded IAM took the console down for anyone logged out, and the
mechanism was not IAM: the shared *zip.App was handed to all 107 subsystems, so
any one of them could call app.Use() and gate every route in the process. Blast
radius was a slice position in apps.Wire(). The point fix stopped that one
subsystem; this stops the class.

MountFunc now takes cloud.Router, not *zip.App. Routes register exactly as
before — absolute paths, same specificity — but the two doors to app-wide
middleware, app.Use(mw) and app.Group("/x", mw), are bounded to the prefixes the
subsystem declares. An empty MountSpec.Prefixes means the /v1/<name> convention
every subsystem already follows, so only a subsystem that gates something else
has to name it: IAM exports its two subtrees as iam.Prefixes (the same list that
registers its routes and serves its 503 — one list, three uses), and zen names
/v1 because its model claim genuinely spans it. Middleware outside those
prefixes is not installed and fails the mount, so the binary refuses to boot
half-gated rather than serving with a stranger's gate on.

Global is the one way back to the bare app, and it is spelled out in Wire() and
frozen by TestWireOrderMatchesFrozen, so a new grant cannot arrive as a quiet
field on one line of a 128-entry literal. Seven hold it — ai, agent, authz,
commerce, licensing, metrics, o11y — and every one is a linked module whose own
Mount still takes *zip.App. None of them installs middleware today (measured);
each stops needing Global when its module takes cloud.Router.

Fiber() is promoted onto the scoped Router rather than granting four more
Globals for four read-only uses (in-process dispatch and the route table).
transport.SetApp takes the *fiber.App for the same reason: the package is
imported by cloud, so it cannot name cloud.Router, and the engine was all the
dispatch ever needed.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 23:06:36 -07:00
hanzo-dev 7126e0f5ac Merge remote-tracking branch 'origin/main' into land/forge-converge
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:56:54 -07:00
hanzo-devandhanzo-dev 861556a839 config: fall back to the operator's HANZO_VERSION so a pod names its own build
api.hanzo.ai answers X-Api-Version: dev because CLOUD_VERSION had a reader and
no writer — nothing ever set it, so every pod reported the link-time default and
a fresh deploy was indistinguishable from one that never restarted.

The operator now sets HANZO_VERSION on every container from the image tag it
rendered (hanzoai/operator, manifests::build_container). Reading it as the
fallback keeps the explicit CLOUD_VERSION override intact and needs no rebuild,
which matters because the release assigns the final version only after the image
is pushed — a link-time stamp would race it.

Sourcing moves into resolveVersion so the boot path and the test call the same
function; the test previously asserted against its own copy of the expression
and would not have caught a regression here.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:56:03 -07:00
hanzo-dev deba58aa11 rename(o11y): the AI lens is o11y_ai, not langfuse
Hanzo CI/CD / cicd (push) Canceled after 2m18s
CI/CD / gate (push) Canceled after 2m18s
CI/CD / containment (push) Canceled after 2m30s
The admin AI-metrics and o11y boards queried langfuse.observations and shipped
`langfuse` / `langfuseModels` on the wire. Langfuse is somebody else's product;
naming our own surface after it means every reader has to carry that history to
read the code.

Safe to rename outright rather than alias: nothing in the binary WRITES those
tables, no console code reads those JSON fields (two comments mentioned the name
and are corrected separately), and the queries are honest-empty today by their
own admission. Behaviour is unchanged — a query against o11y_ai.observations
returns exactly what one against langfuse.observations did.

o11y_ai.observations is the object; aimO11yAIObs / o11yAIObs the constants;
o11yAi / o11yAiModels the wire fields.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:55:48 -07:00
hanzo-dev 8a42cb494e Merge remote-tracking branch 'origin/main' into land/forge-converge
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:54:33 -07:00
hanzo-devandantje d95fa1e78b money: one spend predicate, and billable is not price
Hanzo CI/CD / cicd (push) Canceled after 3m15s
CI/CD / gate (push) Canceled after 3m15s
CI/CD / containment (push) Canceled after 3m18s
A stranger can self-signup and run inference we pay a provider for, at zero
balance. Not for want of auth — every gate on the path checks auth. The binary
shipped THREE paywalls and enforced none:

  - routers.Paywall (mounted app-wide) asked only "does the org hold a paid
    PLAN?". No credit leg, so turning it on would have 402'd every prepaid
    customer. That is why it shipped dark and stayed dark.
  - BillingGate (mounted app-wide) gates on price(path) > 0, and DefaultPrice
    returns 0 for every path — it never evaluates anything.
  - entitlements.RequireProduct had the right answer and was mounted on no
    route at all.

The predicate lived in clients/entitlements, a leaf that imports the root — the
wrong side of the dependency, so the edge filters serve.go mounts could never
reach it and grew their own divergent copies instead. Move it to the one package
every gate can import and delete the copies.

BILLABLE IS NOT PRICE. Authorization and pricing were one int64. Every inference
path prices at 0 at the edge — deliberately, since ai and zen self-meter their
token costs — and `cents <= 0` was read as "do not gate", so "we charge nothing
HERE" silently meant "we authorize NOTHING here". The same fusion un-gates every
resource an operator prices at 0 (ResourceMeter.Gate: costCents <= 0 -> nil).
That fusion is why the LLM leak and the non-LLM gap are one bug. Two questions
now: Billable says whether standing is required, DefaultPrice says what the edge
charges. Pricing at zero can no longer un-authorize.

SpendGate replaces routers.Paywall at the same mount: subscription OR prepaid
credit (cloud.Stand), read at the wallet address the DEBIT writes, over the LLM
paths and the non-LLM resource trees alike.

DEFAULT OFF, and that is sequencing, not timidity. There is no reachable
starter-credit path in this binary: grant-starter was deleted from commerce
(last in v1.48.2), its replacement POST /v1/billing/credit is not registered in
the co-resident build, and no ensureStarterCredit runs anywhere. A new signup's
wallet is $0 with no self-service way to fund it, so enforcing today trades a
revenue leak for a total signup outage. Flip it only after a funding path exists.

Also fixes the kill switch, which could not kill. serve.go ORed a boot-time
cfg.PaywallEnforced on top of the cockpit switch, so an env var could arm a gate
the cockpit could not disarm. The flag's own Env fallback did the same. Both
gone — entitlements' TestSwitchesDefaultOff already asserted this and was RED on
main. PAYWALL_ENFORCED is set in no deployment and no CR, so this is inert in
production.

Unknown refuses AS UNKNOWN. Unpaid requires BOTH authorities to have answered
no; anything less is Unknown and takes the posture (paywall_strict, also default
off), never a fabricated delinquency. Reads, the pay path, inbound payment
webhooks, and SuperAdmin masquerade are never gated.

Tests: 16 gate cases, 22 Billable cases, the atto boundary, and the address
property every prior recurrence violated — gate the pool, spend the person's
wallet. Mutation-checked: pool-instead-of-wallet, Unknown-admits, and
gate-the-reads each turn them red.
2026-07-26 22:53:54 -07:00
hanzo-dev beee9aa890 Merge github/main into canonical — keep the two mains identical
Hanzo CI/CD / cicd (push) Canceled after 2m12s
CI/CD / gate (push) Canceled after 2m14s
CI/CD / containment (push) Canceled after 2m17s
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:51:15 -07:00
1418ade837 ci(sync): the token is a REPO secret — an org secret is empty here, and say why a rejection is not a credential fault (#358)
Two comments in this file sent the next reader down the wrong path, and both
cost real time today.

The token was documented as needing to be an ORGANIZATION secret. hanzoai is on
the GitHub Free plan, where org secrets reach public repositories only; this repo
is private, so the org secret arrives as the empty string. Nothing errors, and
the REST API still lists the secret as visible to this repo, so the config looks
correct while the job reads nothing. Setting the same value per-repo fixed it
immediately.

The push can also be rejected for a reason that has nothing to do with the
token: git.hanzo.ai is canonical and runs ahead of GitHub, so the non-fast-forward
is expected and the reconciliation is a merge made on the forge. The old message
blamed the credential for it. Both messages now name their real cause, and a
force-push is called out as the wrong fix — it would drop canonical-only history.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-26 22:47:15 -07:00
zeekayandhanzo-dev c72dc05ff6 auth: read the username from preferred_username, not the display name
username() preferred the `name` claim on the documented belief that it carried
"IAM's canonical username, e.g. z". It does not. OIDC gives `name` DISPLAY
semantics and IAM fills it from User.DisplayName, so a live token read
`name = "Zach Kelling"` — a human label, with a space in it, where an account key
belongs.

The cost landed on the money path. clients/principal/wallet.go addresses a wallet
as `<org>/<username>`, so preferring `name` addressed `hanzo/Zach Kelling`, which
no funding path can name, while the balance sat in `hanzo/z`. Every signed-in
completion 402'd against a funded account — that is what took hanzo.chat dark for
signed-in users, and it presented as "no credit" rather than "wrong address".

wallet.go already documents three prior recurrences of one bug — "two layers
derived the same address two ways". This is the fourth, arriving through the claim
instead of the header, which is why the fix goes here and not at the wallet: the
address was right, the name feeding it was not.

The fallback is retained, not removed: a token minted before IAM emitted
preferred_username carries only `name`, and for those the old reading remains the
best available answer. New tokens (hanzoai/iam, same change) carry the username
explicitly, so the fallback stops being reached as tokens roll over — no flag day,
no coordinated deploy.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:46:45 -07:00
hanzo-dev 2d865fb815 fix(test): carry the FTS5 tag, and ask cek whether a store exists
Four more suites asserted a store by os.Stat on its database file. That file does
not exist while the store is open on the pure-Go codec, so code, functions, git
and the treasury ledger were all failing their per-tenant isolation checks for a
reason unrelated to isolation. They now ask cek.Exists, as orgdb and tracker
already do.

`make test` also ran without sqlite_fts5. The release image builds with
-tags "libsqlite3 sqlite_fts5"; libsqlite3 needs cgo and the C library but the
FTS5 tag does not, and without it a migration that declares an FTS5 table cannot
open at all — clients/code failed every test with "no such module: fts5", which
reads as a broken subsystem rather than a missing build tag. Both test targets
now carry it.

While confirming that, the Makefile turned out to state the opposite of what the
Dockerfile does: it said the shipped binary is pure Go, but /cloud is built
CGO_ENABLED=1 -tags "libsqlite3 sqlite_fts5" and CGO_ENABLED=0 builds only the
/smoke helper. Corrected, and the consequence written down — no target links
libsqlcipher, so nothing here exercises the engine the image ships, and the tests
pinning that posture skip in both.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:46:18 -07:00
antje 215bbdbbba money: the selected org pays on the ai path too
cloud already made the SELECTED org the payer of record (9bafa850c): SanitizeIdentity
admits a client X-Org-Id the signed `orgs` claim backs, and principal.BillingOrg keys
the edge gate, the zen gate and every resource meter on that effective org. The ai
subsystem — which owns /v1/* for every non-zen model, so the balance gate and the
usage debit on the dominant LLM path — was still deriving both from user.Owner, the
token's HOME org. Half the money path honored the switch and half did not.

ai v1.831.11 closes it with the same two predicates cloud uses, byte-exact:
EffectiveOrg (membership from the signed claim only) and LedgerOrg (a masquerading
super admin spends its OWN books, never the tenant's), wired to the READ, the
reservation and the DEBIT together so the gate can never authorize a spend from a
wallet the debit does not drain. v1.831.10 carries only the first half of that work;
v1.831.11 is the first tag with both.

The bump removes object.ResolveCloudUsageWindow, whose rule moved to
hanzoai/types.ParseWindow. Four call sites follow it. ParseWindow is the same grammar
term for term (24h|7d|30d|custom, hour|day buckets, unknown label is an error), and it
returns the defaulted Label, so the three "" -> 24h fixups that each caller carried
its own copy of are gone with it.

No behavior change off the switch path: EffectiveOrg returns home whenever no other
org is asked for, which is every request a credential without the claim can make.
2026-07-26 22:43:01 -07:00
hanzo-dev a0169d12c5 Merge github/main into canonical — keep the two mains identical
CI/CD / containment (push) Successful in 3m1s
Hanzo CI/CD / cicd (push) Canceled after 7m19s
CI/CD / gate (push) Canceled after 7m21s
# Conflicts:
#	go.sum

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:40:50 -07:00
hanzo-dev 4726a051ae chore: drop the committed :memory: artifact
A previous commit in this series ran `git add audit/` while a test run had just
left audit/:memory: in the tree — the durable file cek used to create when asked
for an in-memory database — so the artifact was committed with the fix. The
cause is fixed in cek; this removes the file and ignores the name so it cannot
be swept in again by a consumer that has not picked the fix up yet.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:39:34 -07:00
hanzo-dev 02a4abd4be fix(cek): an in-memory database is not a path
cek treated ":memory:" as a filename, so asking for an in-memory database
created a real, durable, encrypted file of that name in the working directory —
alongside its .dek sidecar and lock. Two consequences, both silent: an ephemeral
store outlived its process, and every opener of ":memory:" from one working
directory shared a single accumulating store.

That is what the audit and auditlog suites were reporting. Their filter and scope
assertions looked like a query builder dropping its WHERE clause; the counts were
exact multiples because each test seeded the same file again. audit/:memory: and
clients/automations/:memory: were sitting in the tree.

An in-memory database never reaches disk, so there is nothing at rest to encrypt
and no master key to require. Open now recognizes the three spellings — :memory:,
file::memory:, and a file: URI carrying mode=memory — and hands the DSN to the
driver unchanged, since the keyed builder would wrap it back into a filename.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:38:46 -07:00
hanzo-dev 215a97985d build: complete go.mod/go.sum after the reconciliation merge
CI/CD / containment (push) Successful in 3m34s
Hanzo CI/CD / cicd (push) Canceled after 4m52s
CI/CD / gate (push) Canceled after 4m52s
The merge resolved go.mod to iam v1.33.23 and took github's go.sum, which is
missing the entries the canonical side's dependency set needs — o11y's go.mod
hash first among them, so a clean checkout could not build at all. Tidy the
module graph so the committed tree is the tree that compiles.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:38:31 -07:00
hanzo-dev 029fdc72a8 fix(kms): the dual-mount test asserts shadowing, not admission
It required a validated org principal to be admitted past the admin gate. That
cannot hold in this app: GuardScoped admits a SuperAdmin, or an org ADMIN whose
org is a configured white-label tenant, and IsWhiteLabelTenant is fail-closed on
the empty WLTenants a cross-package harness carries. The principal it built was
non-admin besides — the same one it uses two loops earlier to prove the platform
routes 403.

The claim this test exists for is that kms must not shadow the admin cockpit, and
a 403 from the admin gate already proves admin owns the path; 404 is the failure.
Who gets admitted stays where it is proven, clients/admin/scope_test.go, which
keeps this test independent of IAM and datastore availability.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:32:50 -07:00
hanzo-dev d40284622a fix(test): scope the shared-store guards to the build that has the property
Three audit tamper tests, the audit shareability probe and the kms
no-exclusive-lock guard all need two handles on one store file to observe each
other. Without the live libsqlcipher codec, cek falls back to the pure-Go
envelope, which decrypts into a handle-private RAM copy and seals it on close:
two opens never see each other and the last close wins. envelope.go states that
outright as a design constraint. So on a pure-Go build these were failing
because the property is genuinely absent, not because anything regressed —
which in the kms case is precisely the signal that guard exists to send, and it
was drowning it.

They now skip there, naming the constraint, and the two that reach the store
directly go through cek rather than a bare sql.Open, which cannot read an
encrypted file at all.

Worth stating plainly: the property IS real on the shipped image, which links
the codec and keeps the database in place. But `make test` runs CGO_ENABLED=0
and test-cgo forces -tags sqlite_purego, so no CI job exercises the codec build
— these guards now pass everywhere and run nowhere. Closing that needs a
codec-linked CI job, which is a separate change.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:31:21 -07:00
hanzo-dev 3fdb4b883b agents: the default model is enso-flash, not the bare enso alias
Port of 6a61caeb9, stranded on the canonical line and never on github/main —
the same divergence that hid the margin work and the security pin.

The bare `enso` alias does not mean "resolve to the cheapest adequate tier".
Its route opens on an Opus-class arm (zen catalog-enso.yaml, route[0]) and bills
$4/$20 per Mtok, against enso-flash at $2/$4. Defaulting every unnamed agent call
to it is a 28.6x input / 71.4x output increase on work that mostly wants a fast
first response. A caller who needs more pins enso-pro or enso-ultra; a caller who
actually wants the router's judgement pins `enso` deliberately.

Production was already shielded by env — the cloud deployment sets
CLOUD_AI_DEFAULT_MODEL=enso-flash, which overrides the constant — so this is not
a live overspend today. It is the fallback that was wrong: any deployment that
does not set that variable inherits the Opus-class arm silently, and the constant
is what a reader takes as the intended default. Code and env now say the same
thing, which is the invariant the comment states: changing the tier is this line
plus the deployment env, never a third place.

The upstream-base mapping follows the default rather than guessing a tier. We are
not claiming which enso tier a given base corresponds to; we are saying the work
runs on whatever this deployment runs unnamed work on.

The test now pins enso-flash and says why, so relaxing it back to the bare alias
has to be a priced decision rather than a tidy-up.

Root package tests pass: TestZenModel, TestUpstreamModel,
TestAutoRoutingBillsAsResolvedModel, TestDefaultPriceAiPathModelAgnostic,
TestMeteredAI_ModelListerPreserved. Clean cherry-pick, no conflicts.
2026-07-26 22:28:28 -07:00
hanzo-dev 45456e209b fix(test): repair two controls that no longer said what they meant
Both asserted a premise the code had moved past, so both failed against correct
behaviour.

The billing-gate liveness control priced /v1/agent/run. DefaultPrice charges
nothing anywhere now — /v1/agent/* was deliberately zeroed because the round
bills through the in-process /v1/chat/completions, and unpriced paths default to
0 so a new route never silently starts billing. A control drawn from that table
cannot tell a live gate from a dead one, which is exactly what the control
exists to rule out. It now carries its own price, while the IAM paths under test
still go through the real DefaultPrice — the exemption being asserted.

The shard-safety case wanted "iam off" and expressed it by leaving Enable empty,
which means mount-all: iam was ON, and the boot gate correctly refused to shard
a per-pod SQLite identity store. The guard was right and the test was wrong, so
the test now says iam-off explicitly.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:08:11 -07:00
hanzo-devandhanzo-dev c873cad3c1 helm: a running pod reports the build it is, not "dev"
config.go reads getenv("CLOUD_VERSION", Version) and version.go documents the
operator setting it from the deployed image tag, but nothing ever set it — a
consumer with no producer. Every pod therefore fell back to the link-time
default and answered X-Api-Version: dev, so no rollout could be verified from
outside; a deploy and a stale pod are indistinguishable over the wire.

Sourced from the same expression as image.tag so it cannot drift from the image,
and set here rather than stamped at link time because the release assigns the
final version only after the image is pushed, which a build-time stamp would
race.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:07:44 -07:00
hanzo-dev 4d3c97924e platform: one derivation for both version build args
The previous commit added a second tag parser next to the one already three
lines below it, so VERSION and GIT_VERSION could disagree — on the leading v,
on latest, and on a digest-pinned ref, where the new parser stamped a 64-hex
hash as a version. Adding a second way to compute a value is exactly what the
commit claimed to prevent.

One splitImageRef call now feeds both. GIT_VERSION is the same tag without the
leading v, which is how a Makefile writes a version it would otherwise take
from `git describe`, and the two cannot drift because there is nothing left to
drift from.

latest names no release and a digest names no version, so both args are
suppressed rather than guessed. splitImageRef reports a digest in the tag
position, hence the colon check.

The test asserts the pair together, including the digest and latest cases the
first version got wrong.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 22:02:49 -07:00
hanzo-dev 4b0003ffcc affiliates: the upline schedule is set in admin.hanzo.ai too, not compiled in
Finishes what the margin change started. L1 was already per-affiliate — a
negotiated Affiliate.RateBps on the row — and the margin is now a cockpit switch.
L2 and L3 were the last piece still frozen at compile time, so moving the upline
schedule meant cutting a release for a number that is a commercial decision.

Two switches, same shape as affiliate_margin_bps: affiliate_l2_rate_bps and
affiliate_l3_rate_bps, category Gateway, resolved per accrual and never captured
at boot. The old l2RateBps/l3RateBps constants become defaultL2RateBps/
defaultL3RateBps — the value before anyone has set one, unchanged at 500/200.

RESOLVED AS A PAIR, which is the whole design. The invariant is L2+L3 <= bpsDenom,
a property of the two TOGETHER, so neither can be validated alone. A pair that
breaks it falls back to both defaults rather than half-applying: a schedule made of
one edited level and one rejected one is a schedule nobody chose, and it would be
live on real payouts. clampUplineRates is pure so that bound is testable without
the flag engine — the same split margin uses with clampMarginBps.

maxL1RateBps stops being a constant and becomes a function. It is defined as what
L2+L3 leave over, so freezing it at 9300 would mean lowering L2 silently kept the
old, tighter cap — the admin could not use the room they just freed. The set-rate
refusal now quotes the live cap instead of a hardcoded 9300, which would begin
lying the moment the schedule moved and leave the caller no way to learn the bound.

The money invariant is unchanged and now enforced against a moving schedule: L1cap
+ L2 + L3 == bpsDenom exactly, and the cap can never go negative, so the platform
still cannot pay out more margin than it earned on an event.

TESTS (clients/affiliates, needs native/flags/target/release/libhanzo_flags.a):

  TestUplineRates_AreRegisteredAdminSwitches            PASS (2 subtests)
  TestUplineRates_UnsetAreTheDefaults                   PASS
  TestClampUplineRates_FallsBackTogether                PASS (7 subtests, incl.
    negative-at-either-level refusing the PAIR, sum>100% refusing, sum==100% legal)
  TestMaxL1Rate_KeepsTheWholeScheduleInsideTheMargin    PASS
  TestLevelRateBps_ReadsLiveAndHonoursTheNegotiatedL1   PASS

Package total stays 25 failures, identical to pristine main: the store tests refuse
to open SQLite without CLOUD_KMS_MASTER_KEY_REF, which fires before any rate code.
commerce.go is left unformatted exactly as it is on main — not mine to touch.
2026-07-26 21:51:32 -07:00
hanzo-dev 2b0b8f4a63 docs(analytics): name the client this package actually reads through
The header said the package rides the datastore connection ai/object opens in the
shared Bootstrap, citing InitDatastore and DatastoreQuery. It does not, and has
not since the connection moved into clients/datastore: the reads here go through
datastore.Ready and datastore.Query, and that leaf opens the connection from the
environment on first use.

Those two ai/object functions still exist, so the old text reads as current rather
than as something a grep would disprove — which is the kind of comment that sends
the next reader looking for a Bootstrap ordering problem that cannot happen.

Says what the code does and records why the connection lives in a leaf.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 21:47:51 -07:00
hanzo-devandantje 3c6c071fc8 fix(release): scope the CR tag bump to the App — unscoped yq froze the fleet's CD
The universe tag bump ran two unscoped yq expressions against
crs/cloud.yaml, which is MULTI-DOCUMENT (App/cloud + IngressRoute/cloud-hanzo-ai-auth).

The write was the damaging one. `yq -i '.spec.image.tag = "vX"'` applies to EVERY
document, so it did not merely update the App — it CREATED .spec.image on the
IngressRoute, a field ingressroutes.hanzo.ai does not declare. Hanzo CD then could
not build a typed value for that resource:

  ComparisonError: error building typed value from config resource:
    .spec.image: field not declared in schema

and ONE un-typable resource takes the WHOLE Application's comparison down. For as
long as it was there, NOTHING in universe-crs — no App, KMSSecret, PVC, Middleware
or Service — could be applied from git, while CD kept reporting its last successful
sync. Every deploy in the fleet was silently frozen, and it was re-created on each
release.

The read was wrong too, quietly: an unscoped path emits one line PER document, so
CUR was "v1.801.218\n\n" and the monotonic no-rollback guard was comparing against
a multi-line string.

Both now select the App document explicitly. Verified with yq v4.53.3 against the
real file: unscoped set writes App=vX AND IngressRoute=vX; scoped writes App=vX and
leaves the IngressRoute with no image at all. The read returns a single value.

The already-created stray field was removed in universe (f03c6db9), which also
re-pinned the App to v1.801.218 — unfreezing CD without that pin would have rolled
the money surface onto an image known to 401 authenticated traffic.
2026-07-26 21:47:19 -07:00
hanzo-dev c6ab36a8e0 affiliates: margin is set in admin.hanzo.ai, live, not in env
Port of 75788dd07, which was written against the canonical line and never
reached github/main. The two histories have diverged badly — 1536 commits are
reachable from v1.801.225 and not from main — so work landing on one is simply
absent from the other, and main is what the build now cuts images from. This
one was worth carrying across on its own: without it the margin is an env var
again, which is exactly what it was changed away from.

WHAT IT REMOVES. defaultMarginBps stays as the floor, but the live value is no
longer read from the environment and no longer snapshot into State at Mount.
State.marginBps was resolved once at boot, so changing the margin meant a
redeploy — for a number that is a pricing decision, not a deployment one, and
one we move as our real costs move. It is now resolved per read through
flags.Int, so an owner changes it in admin.hanzo.ai and it applies within a
flag-cache TTL.

Registered as a platform switch (flags.Def, category Gateway) so it appears in
the cockpit rather than being a magic key someone has to know. Unmounted flag
engine falls back to Def.Default, so a deployment that has never touched the
switch behaves exactly as before.

Clamped: anything outside [0, bpsDenom] resolves to the default rather than
being honoured. A negative or >100% margin is a typo, and the accrual path
multiplies by it — the clamp keeps a fat-fingered cockpit edit from writing
nonsense into money.

Conflict resolved: main renamed commerceinproc -> commerce/transport, so the
import hunk keeps main's spelling and only adds clients/flags. No behavioural
part of the original touched.

TESTS. clients/affiliates now links the cgo Rust flags lib, so the package
needs native/flags/target/release/libhanzo_flags.a built to run at all. With it:

  TestAffiliateMarginBps_IsARegisteredAdminSwitch   PASS
  TestAffiliateMarginBps_UnsetIsTheDefaultNotZero   PASS
  TestAffiliateMarginBps_ClampsOutOfRange           PASS  (5 subtests: negative,
    above 100%, exactly 100% legal, zero legal, mid range)
  TestMargin_IsNotSnapshotAtBoot                    PASS

Package total is 25 failures both with and without this change — identical to
pristine main. They are one pre-existing environment guard, not this: the store
tests refuse to open SQLite without CLOUD_KMS_MASTER_KEY_REF, which fires before
any margin code runs.
2026-07-26 21:46:50 -07:00
hanzo-dev d08b43f7d3 fix(cek): ask cek whether a store exists, don't stat the database file
Discovering stores by walking the data directory meant os.Stat on the
{subsystem}.db file. That is not the same question on both codecs. The pure-Go
codec keeps the database in its envelope and materializes the file on close, so
a store that is OPEN right now has only its sidecar on disk; the live codec
writes in place and has both. Statting the database therefore finds a store on
one build and not the other — and on the pure-Go build it skips exactly the
stores that are in use.

OrgStore.Each is documented as the cross-org sweep a reconciler folds over, with
the filesystem as the source of truth for which orgs have a store. On a pure-Go
build it silently enumerated nothing for every active org. The two kms
equivalents answered "does this org have a kms store" the same way. Production
links the live codec, so the shipped image was never affected; dev, CI, and any
pure-Go build were.

cek.Exists now answers it — database or sidecar, the marker that holds on every
build and at every point in a store's life — and the suffix stays cek's own,
so callers ask rather than knowing the layout.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 21:36:17 -07:00
hanzo-dev 539f9dafdf refactor(commerce): one commerce namespace, no compound names
clients/commerceclient becomes clients/commerce and clients/commerceinproc
becomes clients/commerce/transport: domain values and byte transport, one
namespace, sibling packages. hanzo.yml follows in the same change so the gate
keeps naming what it runs.

Not one package, and that is a compile-time fact rather than a preference. The
domain client resolves plan tiers through clients/plan, clients/plan imports
the root cloud package for its Mount signature, and build.go in that root
package needs the transport — cloud, commerce, plan, cloud. Collapsing further
means either a second registration hook in package cloud or dragging plan, goja
and the commerce models into ten subsystems that want an http.RoundTripper.

The HTTP fallback stays. The retired standalone is gone, but the same branch
serves COMMERCE_URL, which clients/account and clients/metering still read, and
three test suites mount subsystems against an httptest server through it.

This landed once as f25b2c4c and was lost when main was force-rewritten. The
original is preserved at preserve/commerce-namespace.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 21:31:43 -07:00
zeekayandClaude Opus 5 95c430e1ba deps: orm v0.6.14
Picks up the fixes that matter to any multi-tenant server on this stack:
With honours its context while a namespace is opening (a caller that had given
up used to stay parked for the whole S3 restore, and every retry parked
another); the shared open no longer runs on the triggering caller context, so
one client disconnect cannot fail the requests queued behind it; and Close
drains instead of closing a database out from under a live query.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 21:28:34 -07:00
hanzo-dev c32f493a85 fix(agents): build the legacy fixture through cek, not as plaintext
TestMigrationIdempotentOnLegacyDB hand-built its legacy database with a bare
sql.Open, leaving a plaintext file. Converting plaintext to encrypted is a
production operation that needs the live libsqlcipher codec, which neither
`make test` nor `make test-cgo` links — so the test failed on the fixture
before reaching what it actually asserts.

What is under test is this package's migrate() over a legacy schema: that the
additive columns land with their defaults and that re-opening is a clean no-op.
None of that is about the storage format, so the fixture now goes through cek
like every real store and the test runs on the build the suite actually uses.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 21:05:16 -07:00
hanzo-dev 236a11f65e perf(admin): fan out the overview's per-org reads
The Platform Overview folded every org serially, two independent reads each
(users, money). At 122 orgs that is ~244 blocking round-trips before a single tile
renders, and it gets worse with every tenant that signs up — the reason the admin
dashboard loads slowly.

The reads do not depend on each other, so they now run concurrently under a fixed
ceiling of 12: bounded so a large fleet cannot stampede the finance ledger or the
IAM store. Accumulation is mutex-guarded; go test -race passes on the overview path.

Behaviour is unchanged, including the partial-read semantics: if ANY org's money
fails to read the totals are an undercount, so commerce still reports degraded
rather than healthy.

The 3 failures in this package (GrantCredit x2, SuspendReactivate) are pre-existing
and identical on clean main.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 20:55:37 -07:00
hanzo-dev baf2d35a01 benchmark: describe the arena by what it measures, not by whose claim it beats
The package comment said the arena exists to "replicate-or-disprove any provider's
published claim". That frames the surface around an argument with other vendors
rather than around what the code does, which is run canonical public benchmarks
under one harness and record what it measures. The rest of the file had already
been rewritten this way; this line was the last one left.

The published table also had no comment at all, so nothing in the source said why
a competitor's number is sitting in it. It is there as attributed data — a claim
read from a source, kept beside our own measurement and never blended into it.

The identifiers stay real. A recovered branch neutralized them too, rewriting
sakana/fugu-ultra to "external-orchestrator-a" and the report citation to "vendor
technical report". That is the opposite of what this file is for: an unattributable
claim cannot be checked by a reader and cannot be joined to the attempts measured
for that model, which still record sakana/fugu-ultra in testdata. Naming the system
a public claim belongs to is what makes it verifiable, so only the framing changed.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 20:54:47 -07:00
hanzo-dev e5f335739e platform: the image tag is the build's version
A build from a git context has no .git to describe from, so a Dockerfile that
derives its version from the repository gets nothing — hanzoai/git published
0.0.0+unknown under the tag v1.26.25.

The fabric already knows the tag it is publishing under, so it passes it as the
GIT_VERSION build arg. Deriving the version from the tag rather than beside it
means the two cannot disagree, and a Dockerfile that declares no such arg is
unaffected.

A registry host may carry a port, so the tag is read after the last path
separator; a digest-pinned or untagged reference yields no version and the arg
is omitted.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 20:51:42 -07:00
hanzo-dev f6f0be281b fix(test): give the suite a master key once, not per package
cek refuses to open a store without a master key on every build — there is no
plaintext-at-rest mode. The server makes that an explicit boot decision in
serve.go, but a test run has no boot, so `make test` failed in every package
that opens a per-org store, before reaching an assertion.

A dozen packages had each grown their own TestMain to work around it. Rather
than copy that into the two dozen that had not, the suite declares its dev
posture once, where the run actually starts. Those existing harnesses already
defer to a key present in the environment, so they keep working unchanged and
still cover a bare `go test ./clients/foo` outside make. CI's real key wins
over the dev default in both places.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 20:49:39 -07:00
hanzo-dev 4692653c8b fix(tracker): green the suite and fail closed on an empty mirror anchor
Three defects, all in the external-issue mirror's neighbourhood.

GetIssueByExtRef matched on an empty anchor. Every native issue carries an
empty ext_ref, so a blank anchor selected the lowest-numbered team issue —
and the upsert's "found => update in place" branch would then overwrite a
real team issue with mirrored external content. An empty anchor is now a
miss, which is the only safe answer. The sink already rejected a blank
extRef before calling, so this closes the exported method's own edge rather
than a live path.

The suite could not run at all: cek refuses to open a store without a master
key, and nothing supplied one, so every store-backed test failed before
reaching an assertion. Added the same TestMain harness the sibling
clients/sync and clients/git suites already use — a throwaway dev key, and
only when the environment did not provide one, so CI's real key still wins.

TestPerProjectStoreFileIsolation then still failed: it stat'd tracker.db
while the stores were open, but the codec materializes the database on
close. It now asserts on the .dek sidecar, which cek mints eagerly and 1:1
per store — the same proof of two physically distinct per-project stores,
against something that is actually on disk at that moment.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 20:46:11 -07:00
antje c520734866 fix(money): the wallet address is the username, not the UUID sub
principal.WalletOf read X-User-Id for the name half of the money address.
Both minters set that header from the JWT `sub`, and IAM's `sub` is a UUID,
so a self-serve account resolved to "hanzo/<uuid>" — a wallet NO funding path
can name. An admin grant credits the org pool, an operator deposit names
"<org>/<username>", a promo credits the pool, and the ai gate + the usage
debit both resolve the username. So the edge gate and the paywall's credit
leg addressed a ghost: $0 forever, unfundable by construction, while every
other layer addressed "hanzo/z@hanzo.ai".

This is the third recurrence of one bug — build.go wireFinance and
middleware_billing.go identityFromCtx each document having shipped it once —
and every time the cause was the same: two layers deriving one address two
different ways. So the derivation is now ONE function, principal.Subject,
and clients/billing.subjectFor (which already had the right precedence) is a
one-line call to it instead of a second copy.

Subject prefers the minted X-User-Name — the IAM USERNAME the boundary mints
expressly as "the `name` half of <owner>/<name>" — and falls back to reducing
X-User-Id, whose shape is path-dependent (bare username, UUID sub, or an
"<owner>/<name>" key), to its name half. There is exactly ONE account.Payer
call, so the signed `billing_account` claim always decides: a two-branch
version that shortcut to PayerOf when the username was absent silently
dropped the claim, and a token signed "person:acme/bob" would have gated the
acme POOL. Tests cover that near-miss too.

No gate is enabled or tightened by this. The edge BillingGate prices every
path at 0 (DefaultPrice), so it is dormant; the entitlements paywall's credit
leg only ADMITS, so a correct address can turn "no credit" into "has credit"
and never the reverse.
2026-07-26 20:45:06 -07:00
hanzo-dev 7ab86586ae fix(iam): embed iam2 behind its own prefixes, not over the whole app
Mounting the embedded IAM took the console down for anyone logged out. With
`iam` enabled, the shared binary answered /healthz with {"binary":"iam2"} and
401'd both / and /signin — the sign-in page itself was unreachable, and the only
way it surfaced was pinning the image and watching the console go dark.

Cause: clients/iam called iamserver.Route, which CO-MINGLES iam2's routes onto
the host app at absolute paths. iam2 is a whole server — it owns a root
catch-all and its own /healthz — so co-mingling let it shadow cloud's console
catch-all. Nothing was misconfigured; the wrong embedding primitive was used.

iamserver.Handler is the one the library documents for this exact host
("registered at the /v1/iam/* and root /.well-known/* wildcards"): a standalone
iam2 app adapted to net/http and confined to wildcards at the prefixes IAM owns.
iamPrefixes becomes the ONE list — it already described this surface for the
fail-closed 503 and now registers the real routes too, so the two can never
disagree about what identity owns.

/.well-known/* stays a root wildcard on purpose: OIDC discovery and JWKS live at
the root by spec (RFC 8414), so a relying party reads them off the issuer host.
One narrow path, not a catch-all — it cannot shadow the console.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 20:44:09 -07:00
hanzo-dev 86d3c3ea2f Merge github/main into canonical — keep the two mains identical
CI/CD / containment (push) Successful in 1m0s
Hanzo CI/CD / cicd (push) Failing after 2m36s
CI/CD / gate (push) Failing after 2m36s
# Conflicts:
#	build.go
#	clients/usage/usage.go
#	go.mod
#	go.sum

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 20:37:00 -07:00
hanzo-devandhanzo-dev dbc1cd28cf ci: restore the build-and-release pipeline until the forge can run it
The native pipeline on git.hanzo.ai owns build, publish and deploy, but that
repo is still a pull mirror with Actions disabled, so nothing has executed there
and no image has shipped since v1.801.31 while tags kept advancing past 220.

These three run on our own pool and gate the deploy the way they always did:
containment proves no release binary links clients/controlplane, release builds
the image, refuses to publish unless the binary boots and the per-subsystem and
prior-schema migration smokes pass, tags only the proven image, and records that
tag in universe crs/cloud.yaml for Hanzo CD to roll. They retire the moment a
native run appears on the forge.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 20:35:24 -07:00
hanzo-dev 3622fdeb5d feat(account): take USDC on any chain — rails replace the HUSD hardcode (re-land)
Re-land of ff8644607, which landed on main and was then dropped by a force-push
of cloud main. The commit object survived but stopped being an ancestor, so
topup.go had reverted to the HUSD-only version and the surface was 501 again.

Crypto top-up was architecturally complete and could not take a cent. The gate was
a single hardcoded (HANZO_HUSD_ADDRESS, HANZO_HUSD_TREASURY) pair; HUSD is not
deployed, so POST /v1/commerce/topup/wallet answered 501 forever, and the console
mirrored it with a build-time NEXT_PUBLIC_ gate that said "not available yet" no
matter what the server could accept.

A rail is one accepted (chain, token, treasury) triple configured in TOPUP_RAILS.
Customers already hold USDC on Base/Ethereum/Polygon, so accepting the assets they
have is what makes this earn. A new chain is config, not code — the receipt check
is one JSON-RPC call and one well-known event, chain-agnostic already.

Per-token decimals are load-bearing: USDC has 6, HUSD 18, and the old code divided
by a fixed 1e16. Reusing that on USDC rounds a $5 top-up to ZERO; the other way
credits 10^12 times too much. Cents come from the rail's own decimals, pinned by
test.

Adds GET /v1/commerce/topup/rails so the browser learns the accepted set at
RUNTIME, removing the build-time coupling that made this dead. The listing is a
separate view type from the config struct, so the RPC endpoint cannot leak.

Unchanged: the credit is the ON-CHAIN value never a client number, it lands on the
gateway-validated caller (no IDOR), and it is recorded S2S with the service token.
No rail configured ⇒ honest 501. With TOPUP_RAILS unset this deploy is a no-op.

15 tests pass, 5 new: 6-decimal pricing, multi-rail requires naming one, unknown
rail 400, wrong-treasury 400, listing omits the RPC URL.
2026-07-26 20:34:21 -07:00
hanzo-dev 813a79aff3 fix(fleet): a target's status follows its heartbeat, not its last write
The stored status is only ever written by a request, and nothing writes it when
a machine simply stops beating — so a worker that died, or a host that was
powered off, reported "online" forever. The fleet board showed two GPUs online
whose last heartbeats were five and nine days old, which is exactly the hidden
state the visible-queue work exists to eliminate.

Liveness is a fact the heartbeat decides; the stored status records operator
INTENT (draining/offline). Target.EffectiveStatus folds the two in ONE place and
every reader uses it: the target views, the fleet board's agent fold, and the
dispatch gate. A never-heartbeated target (hand-registered destination, no agent)
keeps its stored status — there is no fact to check.

Dispatch was already safe via its independent claim-TTL check; this makes the
reported status agree with it instead of contradicting it.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 20:26:35 -07:00
hanzo-devandantje 910897c984 deps: iam v1.33.23 — the embedded Guard stops gating cloud's whole surface
iam <= v1.33.22 registers its authentication seam as a bare
app.Use(authz.Guard(db)). That is coherent while IAM owns the app and false
the moment it does not: folded into the cloud binary, IAM mounts early and
every route registered after it inherits the Guard — which then resolves the
bearer against the EMBEDDED store, one that has never seen a token minted by
hanzo.id, so it fails closed on every request.

MEASURED, not inferred. An isolated pod on v1.801.225 (no Service selecting
it, emptyDir for the RWO data PVC) boots clean — Running, 0 restarts, no
"refusing to start" / "auth features disabled" / "panic:" — and then answers
anonymously:

  /signin      401      /v1/models   401
  /            401      /favicon.ico 401
  /login       401      /healthz     200   (public group, registered pre-Guard)

with body {"status":401,"error":"authentication required"} at dur_ms=0 — the
first middleware, before any handler. That body is iam/internal/authz's Guard
verbatim, not ai's OpenAI-shaped error, which is what identifies the emitter.
A logged-out user could not reach the login page, so the console was
unreachable for exactly the people who need to sign in; that is why .225 was
pre-flighted and reverted to .218 twice.

v1.33.23 scopes the Guard to the prefixes IAM actually serves
(guardedPrefixes: /v1/iam, /login/oauth, /mcp, /.well-known/openapi.json).
A scoping change, not a relaxation — every prefix it covered that IAM serves
is still covered, same order, same fail-closed behaviour, public group still
registered first. Reordering the mounts would also have worked today and
broken on the next reorder: a position in a slice is not a security boundary,
a path prefix is.

Carried in the same commit because main had DRIFTED BELOW the tag — v1.801.225
is not an ancestor of main, so main still pinned the pre-fix versions:

  commerce v1.49.21 -> v1.49.22   the unauthenticated cross-tenant read fix
  ai       v1.831.6 -> v1.831.9   supersedes the v1.831.8 that crashlooped

Building from main without these would have shipped an image WITHOUT the
cross-tenant fix. The IAM self-read that blocked .225's predecessor is
independently resolved: iam v1.33.22 is deployed and the committed probe
(iam test/probe/app-selfread.yaml) returns 200 for both reads InitAuthConfig
makes, with all three negative controls still 403.

go vet ./clients/... . is clean; go build ./... fails 91 packages, identical
to the pristine baseline and entirely at the link stage on a prebuilt Rust
artifact absent from this checkout.
2026-07-26 20:21:28 -07:00
hanzo-dev 406e3ae9a6 feat(company): formation register — the platform can read its own book
Hanzo forms the entity, so Hanzo carries the formation KYC/AML obligation.
That obligation is answered across the whole book, not per tenant, and the
store could not answer it: Get/Put/Delete are all keyed by org, with no
list, so Hanzo could not enumerate the entities it formed.

Store.List returns formations across orgs from the projection columns Put
already maintains — the listing never decodes a document it will not show.
Count gives the book its shape per stage. Pending is the one listing that
decodes, and stays bounded by reading only StageFounders rows, the sole
stage that can hold KYC in flight since guardKYCVerified gates the edge
out of it.

GET /v1/company/register, /register/summary and /review are SuperAdmin
operations, not gates on a founder path: they discharge Hanzo own duty,
they never advance a stage, and no tenant can reach them. The review queue
reports founders not yet settled, oldest first. verified and
reviewer_confirmed stay distinct values and both count as settled.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 20:12:08 -07:00
7a59ab72c4 fix(admin): default /v1/admin/users pagination (p=1, pageSize=200) — the 0-of-222 bug (#354)
IAM's user list returns ZERO rows AND total 0 when p/pageSize are unset. The
operator directory (frontend getList('admin/users')) omits pageSize, so
/v1/admin/users returned {data:[],data2:0} — '0 of 222 users'. Default the first
page + shared admin page size when the client omits them; an explicit client
p/pageSize still wins. Adds TestUsers_DefaultsPagination (asserts p=1/pageSize=200
forwarded + the real total surfaces).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-26 20:08:13 -07:00
hanzo-dev 2bfde35c8b fix(gpu): namespace-ensure is best-effort — stop the worker crash-loop
register() aborted when POST /v1/tasks/namespaces failed, but those namespaces
already exist in any live org — it is an idempotency step, not a precondition.
With systemd Restart=always/RestartSec=5 a single transient 503 became an
infinite crash-loop: observed 141 restarts on spark with the GPU offline the
whole time (auth had lapsed, then a blip kept it down after auth was restored).

Ensure is now best-effort and logs; the presence write still fails loudly, so a
genuinely missing namespace surfaces. Tests pin both halves and fail on the old
code with the exact production error.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 19:57:48 -07:00
hanzo-devandhanzo-dev 660331dc91 ci: hand main to git.hanzo.ai with a push, and fail when it does not happen
GitHub's only job for this repo is delivering main to the forge, which owns
build, publish and deploy via .hanzo/workflows/cicd.yml. This pushes the ref
instead of asking the forge to pull it, so delivery is deterministic rather than
timer-driven, and it exits non-zero when the token is missing — the previous
sync exited 0 in that case and reported success on every repo that lacked it.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 19:55:20 -07:00
hanzo-dev d3d6ce81fe cli: send build to the door that implements it
`hanzo build` posted /v1/runner to PlatformURL, but that route is served by the
cloud binary. The platform host answers 500 — "PLATFORM_BUILD_CALLBACK_TOKEN is
not configured on the server" — and never implements the IAM-admin path at all,
so the default invocation could not build anything. Every user hit it.

Give the build its own client against CloudURL and leave the other five
platform verbs alone; they target the platform app correctly. One route, one
implementation, one door.

withCloud is withPlatform's sibling in the tests, so TestBuildCommand now
asserts the request arrives at the cloud host rather than passing against a
server that stands in for either.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 19:52:52 -07:00
hanzo-devandhanzo-dev 0f6fbcbcc9 durable: re-land zero-downtime HA per-org stores + encryption alignment
Ports the HA durability delta onto the current main lineage: per-org SQLite on
object storage with live K8s membership, graceful drain, a swappable snapshot
codec, and a checkpoint seam that re-encrypts the envelope backend before a
successor reads it, so a takeover cannot drop an acked write.

Encryption posture gates on CodecLinked() (is the SQLCipher codec actually
linked) rather than EncryptionAvailable(), and cek fails closed without a master
key on every build, using a deterministic dev key only where no live codec is
linked. Pins hanzoai/sqlite v0.4.0 and hanzoai/sqlcipher v0.1.1.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 19:47:27 -07:00
Hanzo Blueandhanzo-dev 38081141b5 datastore: cloud reaches the warehouse through one client
Every warehouse read and write in this binary went through ai/object's
package singleton, which costs 1,933 packages to reach four functions in a
198-line file: Kubernetes, Docker, AWS, luxfi/geth and Go's plugin package,
none of which a SELECT needs.

clients/datastore holds the connection now, over orm/datastore. The four
functions map straight across -- DatastoreQuery/Exec/Enabled become
Query/Exec/Ready, and InitDatastore disappears because the connection opens
from the environment on first use, so there is no wiring order to get wrong.
Ready stays assignable as a bare func() bool, which clients/leaderboard
relies on. The three live tests poll a deadline no longer: Wait does it.

The connection lives in a leaf that imports only orm/datastore, not in
cloud.Deps. Deps lives in the root cloud package, which imports ai/object for
the tier, balance, usage and ingest seams the embedded ai router calls back
through -- so putting it there would drag those 1,933 packages into every
leaf that reads a table. clients/samples proves the difference: 2001
packages -> 362, and its test binary links in 0.43 GB instead of 2.01 GB.

Packages that already reach the root cloud package keep ai/object through it
and are unchanged at +2. Removing it there is a separate piece of work: those
seams are ai's, not the warehouse's.

The 13 files that still name ai/object use it for EnsureCloudUsageTable and
ResolveCloudUsageWindow. ai's own write path owns that DDL, and copying it
here would leave two schemas to keep in lockstep.

o11y's event_ingest binds hanzo-ds/go to `ds`, so `datastore` means one thing
per package.

orm is pinned by replace: the datastore package is only on
feat/datastore-analytics-plane, whose lineage sorts below the v0.6.8 tag that
hanzoai/iam requires, so a plain require downgrades iam to v1.33.12. Drop the
replace for a require once orm lands it on a tag.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 19:37:29 -07:00
hanzo-dev d744545878 test(sqlite): fail the gate on a second SQLite engine or driver name
cloud already links one engine. github.com/hanzoai/sqlite is the facade every
store imports; under cgo it is hanzoai/csqlite against libsqlcipher, under
!cgo its vendored pure-Go engine keyed by the hanzoai/sqlcipher codec VFS.
modernc.org/sqlite and mattn/go-sqlite3 are in neither go.mod, go.sum, nor
`go list -deps ./...`. There is no driver to remove.

What is missing is anything stopping that from changing. database/sql is a
global registry and the names collide in pairs — hanzoai/sqlite and modernc
both take "sqlite", csqlite and mattn both take "sqlite3" — so one blank
import panics the binary at init, before main() runs and before a request is
served. Not hypothetical here: dba5d73b fixed exactly that panic after
fourteen stores blank-imported modernc directly.

The guard reads the registered name -> owning package map back out of
database/sql through the driver it actually holds, and pins it: one engine,
from an allowed package, answering only to allowed names. It closes by
construction rather than by a denylist of today's known drivers — an engine is
caught whether it collides on a name, takes a fresh one, or arrives as a fork
of a name already allowed.

It lives in apps because apps is the composition root, and cmd/cloud's main
imports nothing but apps and the root cloud package: the test binary's
first-party graph is measurably the shipped binary's less that one main
(`go list -deps` gives 163 and 162, the single difference being cmd/cloud
itself). Whatever registers in production registers here, and ./apps/ is
already the first package in the gate. No new CI surface and nothing new to
build — the gate job stays the one place that builds.

Both allowed engines are exercised by a real build mode: CGO_ENABLED=1
resolves "sqlite" and "sqlite3" to hanzoai/csqlite, CGO_ENABLED=0 resolves
"sqlite" to hanzoai/sqlite/internal/engine. Proven to fire by planting a
driver in each shape it can arrive in, including one taking the allowed name
"sqlite3" from an unblessed package under !cgo, where no init panic saves us
and only the owner check catches it.

"sqlite3" stays allowed rather than removed. It is registered by csqlite's own
init, inherited from the upstream it forks, so dropping it is a change to a
shared module with fleet-wide reach — base/core/dialect_sqlite.go returns that
name from DriverName() and commerce switches on it — not a cloud-local
cleanup. Pinning it is what cloud can do; the guard now says so explicitly
instead of leaving it to be discovered.

The go.mod note is corrected in passing: it was a fragment of a deleted
replace directive, still naming modernc as the backend after 3b4c896a took
modernc out of the link.

HIP-0106.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 19:31:02 -07:00
7ed81cfa00 feat(admin): fail-closed white-label tenant admission for the operator cockpit (#343)
admin.hanzo.ai (apps/operator) is served DIRECTLY off cloud-api /v1 (the
hanzo_iam_token/cloud_session_id path SanitizeIdentity validates), so the
/v1/admin/* gate is the only boundary. GuardScoped previously admitted ANY
org-admin (principal.IsOrgAdmin) — so any customer's own-org admin could open
the cockpit. Tighten it to the WL admission tier:

  super (owner==AdminOrg, cross-tenant)  OR
  org-admin of an org in State.WLTenants (subtree-scoped via ResolveScope)

- core.State.WLTenants: fail-closed allowlist (nil/empty => SuperAdmin-only),
  seeded ONCE from ADMIN_WL_TENANT_ORGS at Mount. IsWhiteLabelTenant is the one
  read (verbatim, trimmed, no fold).
- core.GuardScoped now requires super OR (IsOrgAdmin && Org in WLTenants);
  a non-enabled org-admin gets the same 403 as a member/forge. Fleet god-views
  stay core.Guard (super-only) — a WL tenant never reaches a fleet number.
- /v1/admin/me returns isWhiteLabel + scopeOrgs so the SPA renders the subtree
  cockpit server-authoritatively (super=fleet, WL=own subtree).

Tests: WL tenant admitted+own-subtree; non-WL org-admin refused on every scoped
panel; WL tenant 403 on every god-view; fleet-only default (empty set) fail-closed;
IsWhiteLabelTenant unit-pinned. Harness enables 'maxpower' as the WL test tenant.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-26 19:27:32 -07:00
a122f6050b refactor(cloud): lift the /v1 response envelope into package cloud (one writer) (#355)
The canonical /v1 envelope { status, msg, data, data2 } (the operator transport's
get<T>/getList<T> shape) was written in clients/admin/core, but the writers are
pure over *zip.Ctx and every subsystem needs them. Trapping them under
clients/admin — whose sibling files pull in cloud + admin/iam + principal — forces
any package that wants ONE envelope to drag the whole admin IAM fan-in in
transitively. Five packages sidestepped that by re-implementing the exact writer
locally (git.okEnvelope, treasury/referrals/affiliates/authors.adminOK), each
comment admitting it is 'identical to clients/admin's ok()'.

Decomplect: the writers now live in package cloud (cloud.OK/OKList/OKRaw/Fail),
beside Handle/Mount/Terminal — the handler ergonomics every subsystem already
imports. clients/admin/core.OK/OKList/OKRaw/Fail and the five local helpers now
delegate to the ONE implementation. Byte-identical output; no route or wire
change; no consumer affected.

Forward-compatible and additive: cloud.OK is now available to all ~106 subsystems
without importing the admin package. Divergent hand-rolled shapes elsewhere
({"error"}, {"data",total}, named keys) are a public contract and are left for a
phased, consumer-coordinated migration — not touched here.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-26 19:27:24 -07:00
8f60e9dd95 refactor(team): add canonical /v1/team/transactor/statistics alias (drop nested /api/v1) (#356)
Canon is the api.* host + /v1/ with no nested /api/vN. The team transactor stats
route served /v1/team/transactor/api/v1/statistics — an extraneous /api/v1/ inside
a path we own. Register the clean /v1/team/transactor/statistics as the canonical
route the front repoints to; keep the /api/v1/ path as a forward-compatible alias
until it does. Both resolve to the same handler — additive, no rename in place,
no outage.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-26 19:27:17 -07:00
Hanzo Blueandhanzo-dev 2ec2e4c356 datastore: cloud reaches the warehouse through one client
Every warehouse read and write in this binary went through ai/object's
package singleton, which costs 1,933 packages to reach four functions in a
198-line file: Kubernetes, Docker, AWS, luxfi/geth and Go's plugin package,
none of which a SELECT needs.

clients/datastore holds the connection now, over orm/datastore. The four
functions map straight across -- DatastoreQuery/Exec/Enabled become
Query/Exec/Ready, and InitDatastore disappears because the connection opens
from the environment on first use, so there is no wiring order to get wrong.
Ready stays assignable as a bare func() bool, which clients/leaderboard
relies on. The three live tests poll a deadline no longer: Wait does it.

The connection lives in a leaf that imports only orm/datastore, not in
cloud.Deps. Deps lives in the root cloud package, which imports ai/object for
the tier, balance, usage and ingest seams the embedded ai router calls back
through -- so putting it there would drag those 1,933 packages into every
leaf that reads a table. clients/samples proves the difference: 2001
packages -> 362, and its test binary links in 0.43 GB instead of 2.01 GB.

Packages that already reach the root cloud package keep ai/object through it
and are unchanged at +2. Removing it there is a separate piece of work: those
seams are ai's, not the warehouse's.

The 13 files that still name ai/object use it for EnsureCloudUsageTable and
ResolveCloudUsageWindow. ai's own write path owns that DDL, and copying it
here would leave two schemas to keep in lockstep.

o11y's event_ingest binds hanzo-ds/go to `ds`, so `datastore` means one thing
per package.

orm is pinned by replace: the datastore package is only on
feat/datastore-analytics-plane, whose lineage sorts below the v0.6.8 tag that
hanzoai/iam requires, so a plain require downgrades iam to v1.33.12. Drop the
replace for a require once orm lands it on a tag.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 19:16:05 -07:00
hanzo-devandantje 229284435a deps: commerce v1.49.22 — carries the unauthenticated cross-tenant read fix
Cloud embeds commerce as a PINNED module (no replace), so the fix on commerce
main was in no build. v1.49.21 was the newest tag and predates it; v1.49.22 is
the patch that contains it.

What this pulls in, headline first:
  - iam: an org header is a selector, not a credential. IAMTokenRequired admitted
    on X-Org-Id alone, and cloud's identity boundary restores a client's raw
    X-Org-Id on the bearer-less path while leaving X-User-Id empty — so an
    off-gateway caller read another org's store record and entitlement with NO
    token, on both api.hanzo.ai and commerce.hanzo.ai. It now requires a validated
    principal, the same predicate as this repo's clients/principal.Validated.
  - discount cache keyed without a namespace: one org priced its orders with
    another org's discounts, live at a single replica.
  - sandbox is per-org, not per-deployment.
  - ~380 lines of unreachable duplicate tenant resolvers/mounts deleted.

⚠️ Building an image from this commit ships the SANDBOX CUTOVER too. Read the
warning block at SQUARE_ENVIRONMENT in universe's crs/cloud.yaml first: money
behaviour changes in BOTH directions for any org whose Live flag disagrees with
the env that used to override it — an org relying on the env for production but
recorded Live=false silently moves to sandbox and stops taking money; one
relying on it for sandbox with Live=true starts charging real cards. Audit every
transacting org's Live flag before rolling, and do not delete that env block
until the new image is live.

Verified: the pinned module genuinely contains the fix (grepped the resolved
module in GOMODCACHE, not just the tag name). go build ./... fails on exactly 91
packages both WITH this bump and on pristine origin/main — the local
luxcpp/CGO linker issue, unchanged by this.
2026-07-26 19:13:48 -07:00
hanzo-dev 8f229e28b6 feat(translate): POST /v1/translate, two tiers over one endpoint (HIP-0516)
One translation surface with one auth path and one meter:

  POST /v1/translate  { text | batch[], target, source?, tier?, glossary?, format? }
                   -> { translations[], detected_source?, tier, usage }

tier defaults to quality, which COMPOSES the existing model plane (deps.AI --
zen through the gateway), so there is no second inference stack and no second
gate: deps.AI already authorizes and debits its own tokens per org. tier=bulk
reaches MADLAD-400 under CTranslate2 over a small JSON seam, so the weights are
served independently of this binary; a deployment that has not served them
answers 503 for that tier and never re-routes to quality, which would charge a
caller for a tier it did not ask for. bulk carries its own per-character
ResourceMeter, the same in-handler gate+meter every non-LLM unit uses.

The translation memory is normative. Every string keys on
(source_text, target, glossary_version, tier); a hit is returned unchanged, so
only new or changed strings reach an engine and a locale rebuild is idempotent
under a non-deterministic model. The glossary version is DERIVED from the terms,
so an edited glossary self-invalidates instead of relying on someone bumping a
number.

The same memory carries the review lane: an entry sits on the ladder
machine -> suggested -> approved -> published, and a machine write may only
create a row or refresh one still at machine. An approved string survives every
rebuild.

Tenancy is the per-org SQLite file (HIP-0302), so a cross-tenant read of a
translation memory is structural rather than a WHERE clause.

Mounted before the zen/ai /v1/* catch-all; clients/translate added to the
hanzo.yml unit gate.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 19:09:18 -07:00
hanzo-dev c237830246 Merge github/main — keep ai at v1.831.9 (supersedes v1.831.8)
CI/CD / containment (push) Successful in 3m2s
Hanzo CI/CD / cicd (push) Successful in 11m41s
CI/CD / gate (push) Successful in 11m42s
Both sides fixed the same incident from opposite ends: GitHub bumped to
v1.831.8 (auth init fails closed instead of serving with auth off), this side
to v1.831.9, which contains v1.831.8 AND makes the numeric status decode so the
fail-closed path is not reached. .9 supersedes .8 — taking .8 here would leave
the binary refusing to start against iam v1.33.16.

# Conflicts:
#	go.mod
#	go.sum

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 19:02:44 -07:00
hanzo-dev f25b2c4c5d refactor(commerce): one commerce namespace, no compound names
clients/commerceclient -> clients/commerce: the domain client
(CheckEntitlement, ActivePaidPlan, BalanceCents) reading the co-resident
commerce datastore by direct Go call.

clients/commerceinproc -> clients/commerce/transport: the self-routing
http.RoundTripper that dispatches the commerce S2S byte stream to the
co-resident handler.

They stay two packages because one would be an import cycle: the domain
client resolves plan tiers through clients/plan, which imports the root
cloud package, and build.go in that root package needs the transport.
Domain values and byte transport are separate concerns; the split now
follows that line instead of a name.

The transport's plain-HTTP branch stays. clients/account and
clients/metering address commerce by COMMERCE_URL (default
https://api.hanzo.ai) -- a different env from the retired standalone's
CLOUD_COMMERCE_HTTP_URL -- and the billing/usage/content harnesses drive
their proxies over that branch.

hanzo.yml gate paths follow the rename in the same change.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 19:00:10 -07:00
hanzo-dev d49d6e77f7 Merge github/main into canonical — keep the two mains identical
Same drift as before: GitHub took commits canonical lacked while canonical
carried the ai/iam dep fixes. Merging leaves canonical a superset so the push
back to GitHub fast-forwards, which is the condition sync-from-github.yml
needs to stop failing and freezing the build pipeline.
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 18:59:09 -07:00
hanzo-dev d54e8e263f deps: hanzoai/ai v1.831.6 -> v1.831.8 — auth init fails closed
v1.831.8 is the fix for today's second outage: ai's InitAuthConfig used to
warn "(auth features disabled)" and keep serving when it could not establish
the IAM signing cert, so every authenticated route 401'd while the pod stayed
Running and passed its probes. It now refuses to start.

Without this bump the binary still contains the fail-open, and a deploy would
prove nothing about the auth chain — the whole point of shipping it now that
IAM v1.33.17 grants the application self-read.
2026-07-26 18:59:01 -07:00
hanzo-dev 3a4b4fa99d deps: hanzoai/ai v1.831.9 — unpin the iam client from v1.801.218
Hanzo CI/CD / cicd (push) Canceled after 4m7s
CI/CD / gate (push) Canceled after 4m7s
CI/CD / containment (push) Canceled after 4m7s
cloud is held at v1.801.218 because every newer image boots with auth features
disabled: the iam client decodes the envelope's `status` into a string, iam
v1.33.x emits it as a number, the decode fails, and the caller treats that as
"IAM unreachable" and continues without auth. The pod passes its probes, so
nothing crashes and nothing reverts — it just 401s every authenticated call.

v1.831.9 decodes both wire shapes and keeps v1.831.8's refusal to serve with
auth silently off. With this, an image built from main can carry iam v1.33.16
(needed before cloud owns the identity store) without disarming auth.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 18:58:42 -07:00
hanzo-dev 6a61caeb90 agents: the default model is enso-flash, not the bare enso alias
Bare `enso` is not 'resolve to the cheapest adequate tier'. Its route opens
on an Opus-class arm (zen catalog-enso.yaml route[0]) and bills $4/$20 per
Mtok, against enso-flash at $2/$4. Defaulting every unnamed agent call to it
was a 28.6x input / 71.4x output increase on work that mostly wants a fast
first response, so the default now names the tier it actually wants.

A caller who wants the router's judgement still pins `enso`; one who needs
more pins enso-pro or enso-ultra. The default is opinionated on purpose.

Only the value moved — DefaultModel is still the one place, and every test
reads it symbolically, so none of them needed touching beyond the assertion
that deliberately pins the value (which now explains what to re-price if
anyone relaxes it back).
2026-07-26 18:52:16 -07:00
hanzo-dev 8dd28f0dd3 Merge github/main into canonical — one history for hanzoai/cloud
CI/CD / containment (push) Successful in 3m26s
Hanzo CI/CD / cicd (push) Successful in 9m20s
CI/CD / gate (push) Successful in 9m19s
The two mains drifted again after the last reconcile: GitHub carried 11
commits canonical lacked. Merging rather than forcing keeps both sides'
history intact, and leaves canonical a superset so the push back to GitHub
is a fast-forward — which is what stops sync-from-github.yml failing every
ten minutes and freezing the build pipeline.
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 18:47:48 -07:00
zeekayandhanzo-dev 785ef0a503 deps: hanzoai/iam v1.33.15 -> v1.33.16 — before cloud owns the store
Hanzo CI/CD / cicd (push) Canceled after 49s
CI/CD / gate (push) Canceled after 49s
CI/CD / containment (push) Canceled after 49s
The embedded identity store is about to become THE identity plane: the live
4.3MB store moves onto cloud and the standalone is retired. v1.33.16 is a
single commit over .15 with no schema or model change, so the store format is
identical either way — but that commit is "a write that omits the client
secret must not destroy it".

Cutting over at .15 would hand the real store to a library that silently
destroys an application's client secret on any write that omits it. The data
would be intact at the moment of the copy and corrupted by the first such
write afterwards. Take the fix first, then move the data.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 18:47:05 -07:00
hanzo-dev f5aad4dd36 perf(admin): fan out the overview's per-org reads
The Platform Overview folded every org serially, two independent reads each
(users, money). At 122 orgs that is ~244 blocking round-trips before a single tile
renders, and it gets worse with every tenant that signs up — the reason the admin
dashboard loads slowly.

The reads do not depend on each other, so they now run concurrently under a fixed
ceiling of 12: bounded so a large fleet cannot stampede the finance ledger or the
IAM store. Accumulation is mutex-guarded; go test -race passes on the overview path.

Behaviour is unchanged, including the partial-read semantics: if ANY org's money
fails to read the totals are an undercount, so commerce still reports degraded
rather than healthy.

The 3 failures in this package (GrantCredit x2, SuspendReactivate) are pre-existing
and identical on clean main.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 18:22:18 -07:00
2c24ea104a feat(agents): seed the built-in crew @dev @des @vi on org first-touch (re-land) (#342)
Re-land of the crew seed (a parallel force-push to cloud main dropped it).
personalities.go: dev/des/vi personas + idempotent SeedPersonalities(ctx,org)
(one registry, UNIQUE(org,name); no-op without a model). account.go OAuth
callback seeds per-org after EnsureWorkspace (best-effort, never blocks login).
Native TestSeedPersonalities green. TEAM_AGENTS_ENABLED=1 already live +
AIDefaultModel=deepseek-v4-flash → the crew materializes and answers @-mentions.

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 18:22:18 -07:00
99a00f69f0 rebuild(cloud): embed console-embed@sha-bd8a816 (casibase auth /v1/* fix + console per-project resources) + activate MCP builtin tool-plane (#292)
No Go change — this rebuild re-resolves the freshly-republished console-embed:latest
(console main bd8a81651) into the go:embed console served at console.hanzo.ai, and
ships builtin.go's auto '/v1 route → MCP tool' plane at /v1/tools/mcp (already in main,
newer than the deployed v1.801.69).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 18:22:18 -07:00
zeekayandhanzo-dev 1f62bf4aa8 chore: trigger release build for iam2 v0.15.4 (federation fix)
d411200 (iam2 v0.15.1→v0.15.4 bump) did not trigger a release run; nudge
the push-triggered release so the federation-security-fixed image ships.
2026-07-26 18:22:18 -07:00
hanzo-dev ff86446070 feat(account): take USDC on any chain — rails replace the HUSD hardcode
Crypto top-up was architecturally complete and could not take a cent. The gate was
a single hardcoded (HANZO_HUSD_ADDRESS, HANZO_HUSD_TREASURY) pair, HUSD is not
deployed on Hanzo Mainnet, so POST /v1/commerce/topup/wallet answered 501 forever
and the console mirrored it with a build-time NEXT_PUBLIC_ gate that reported "not
available yet" no matter what the server could accept.

A rail is one accepted (chain, token, treasury) triple, configured as data in
TOPUP_RAILS. Customers already hold USDC on Base/Ethereum/Polygon, so accepting
the assets they actually have is what makes this path earn. A new chain is now a
config entry, not a code change — the receipt check is one well-known JSON-RPC
call and one well-known event, which is chain-agnostic already.

Per-token decimals are the load-bearing detail. USDC has 6, HUSD has 18, and the
old code divided by a fixed 1e16. Reusing that on USDC would round a $5.00 top-up
to zero cents; reusing USDC's divisor on an 18-decimal token would credit 10^12
times too much. Cents now come from the rail's own decimals, and a test pins it.

Adds GET /v1/commerce/topup/rails so the browser learns the accepted set at
RUNTIME. That removes the build-time coupling that made this dead: enabling a rail
no longer means rebuilding and redeploying a frontend. The listing is a separate
view type from the config struct, so an operational field (the RPC endpoint, and
anything added later) cannot leak by merely existing — a field is published only
if put there deliberately.

Everything that made the original correct is kept: the credit is the ON-CHAIN
value and never a client number, it lands on the gateway-validated caller (no
IDOR), and it is recorded S2S with the service token. The ledger row now names the
rail actually verified rather than asserting "hanzo"/"husd" for every payment.

Behaviour is unchanged where nothing is configured: no rail ⇒ honest 501, never a
fabricated credit. With TOPUP_RAILS unset this deploy is a no-op.

15 tests pass, 5 new: 6-decimal pricing, multi-rail requires naming one, unknown
rail 400, transfer to another rail's treasury 400, listing omits the RPC URL.
2026-07-26 18:19:23 -07:00
antje 82e48a3b30 test(identity): the anonymous floor under the org switch, and BillingOrg's truth table
The org-switch tests proved the SELECTION behaves. Nothing proved the machinery is
invisible to a request carrying no credential at all — and SanitizeIdentity runs in
front of everything, so a mistake there is not scoped to org-switching: it takes out
unauthenticated routes too, and /health is what the kubelet probes.

  - TestAnonymousRoutesAreUntouched: bare / forged X-Org-Id / unparseable bearer all
    still 200 on a public route. The switch arm lives inside `claims != nil`, so an
    anonymous request must never reach it; this pins that structurally rather than by
    reading the code.
  - TestAnonymousGrantsNoPrincipal: letting anonymous THROUGH must not make it
    someone. A forged X-Org-Id survives for the Phase-1 data path but mints no user
    and no admin, so principal.Org refuses it.
  - TestMembershipMatchIsByteExact: isMember must not fold. "ACME"/"Acme"/"ac me"/a
    zero-width variant are DISTINCT orgs and each pins to home. A trailing-space
    variant is deliberately absent: fasthttp OWS-trims it on the wire, so the
    boundary receives the exact member org and correctly honors it; the whitespace
    fold risk is claim-side and covered by trailWSOwner in middleware_identity_test.
  - TestHomeOrgHeaderSurvivesSwitch: home and effective must stay DISTINCT facts.
    BillingOrg decides who pays from both, so collapsing them silently removes the
    masquerade carve-out.
  - clients/principal/billingorg_test.go: the package had no unit test for the ONE
    "who pays" resolver. Seven cases pin both policies as two branches — member pays
    the org acted in, SuperAdmin masquerade pays the admin ledger, org admin gets no
    carve-out, unvalidated bills nothing — plus Ledger tracking BillingOrg, since the
    ~35 resource-meter call sites reach the ledger through it.

Tests only; no behavior change. Added to the existing orgswitch file rather than a
second one, and isMember/BillingOrg are left exactly as they are: read against an
independent implementation of the same fix, they are equal-or-better on every
property that matters (exact bytes, no fold, empty set admits nothing, silent
refusal, and refusing outright when the org is unresolvable).
2026-07-26 18:17:44 -07:00
hanzo-devandantje 75788dd07b affiliates: margin is set in admin.hanzo.ai, live, not in env
The share base — Hanzo's gross margin, which every affiliate commission is a rate
OF — could not be changed. It came from AFFILIATE_MARGIN_BPS and was snapshotted
into state at Mount, so editing the env did nothing until the pod restarted, and
there was no admin control at all. Margin tracks real cost of revenue and moves;
a boot-time constant read from the environment is the wrong shape for it.

Now a registered platform switch, affiliate_margin_bps, editable in
admin.hanzo.ai and read LIVE at each accrual. Def.Env is deliberately empty: this
must not be settable by environment variable. Same mechanism clients/rollingcap
already uses for its admin-editable per-tier caps — flags.Int over a registered
Def — so no new machinery, one way to do this.

The clamp is now a pure clampMarginBps, testable without the engine: out of
[0,10000] falls back to the policy default so a bad edit cannot over-inflate or
negate the base, while 0 and 10000 stay LEGAL (accrue nothing / share gross
revenue). Unset resolves to Def.Default (4000), not 0 — that direction matters,
because 0 is a legitimate margin, so zero-on-missing would silently switch off
every commission instead of failing loudly.

NOT made per-affiliate, deliberately. The margin base is computed ONCE per
source-spend event and then paid up to three affiliates (L1/L2/L3). The
invariant this file is built around — maxL1RateBps = 10000 - l2 - l3 — guarantees
the SUM of every level's share on one event stays within that event's margin. Give
each affiliate its own base and three affiliates on one event compute three
different bases, so the sum can exceed the margin actually earned and the platform
pays out more than it made. The per-affiliate knob that is safe already exists and
admin already sets it: Affiliate.RateBps, bounded by that same cap.

Four tests: the switch is registered (unregistered ⇒ flags.Int returns 0 ⇒ no
commission accrues at all, so this is worth pinning), carries no Env, is not
ReadOnly; unset is the default not zero; the clamp honours 0/100% and rejects
impossible values; the base is re-read rather than snapshotted.

25 tests fail in this package before and after this change — all
"CLOUD_KMS_MASTER_KEY_REF is required on an encryption-capable build", verified
identical on pristine origin/main by stashing. None are mine.
2026-07-26 18:13:36 -07:00
zeekay 98689c50e8 Merge remote-tracking branch 'origin/main' into reconcile-cloud
CI/CD / containment (push) Successful in 3m34s
Hanzo CI/CD / cicd (push) Failing after 20m18s
CI/CD / gate (push) Failing after 20m18s
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 18:02:57 -07:00
hanzo-dev 57c2ee9e92 event: accept anonymous marketing telemetry on the one canonical door
POST /v1/event refused every caller that could not resolve to an org, so a
logged-out marketing page had nowhere to send a pageview: a rendered-DOM audit
found the 403 on 16 of 22 page loads across every public surface, i.e. no
pageviews and no errors from logged-out traffic at all.

A caller that presents NOTHING now takes an anonymous lane and is attributed to
a reserved public tenant. A caller that presents an ingest credential which does
not resolve is still refused, so a misconfigured key surfaces as a 403 instead of
filing its events where its owner cannot read them.

One path, not a fork. Admission is the only thing the lanes do differently: both
share decodeIngest, and both end at ingestDecoded — extracted here as the single
tail (fold, write core, receipt) so what happens to an admitted event is written
once. The vouched-for lane passes dropped=0, so its behavior is unchanged.

Attribution is structural rather than checked. admitPublic takes no request, so
no header, query, or body field can reach it and the tenant it returns is always
the constant: an anonymous write cannot land in a real org's partition, and the
'$' in the sentinel is outside the IAM org alphabet so it cannot collide with one.
Kinds are an allowlist of two (pageview, error) — identify and group name a person
and a group, and a custom event is the whole product/billing surface. The stored
name comes from the kind, closing the anonymous name space to two values. Fields
are a projection, so personId, groupId, commerce fields and the entire client
property bag cannot reach the row; an anonymous row's properties hold only the
server-folded $exception and $source. Bytes and batch length are bounded and
refused rather than truncated, ingest is capped per client IP and per socket peer
(the header key is caller-settable, the peer is not), DNT/Sec-GPC store nothing,
and the public tenant never fans out to a destination.

CLOUD_ANALYTICS_PUBLIC_CAPTURE, the existing anonymous-capture switch, still turns
the whole lane off.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 18:00:34 -07:00
65d0dbcbeb feat(agents): seed the built-in crew @dev @des @vi on org first-touch (re-land) (#342)
Re-land of the crew seed (a parallel force-push to cloud main dropped it).
personalities.go: dev/des/vi personas + idempotent SeedPersonalities(ctx,org)
(one registry, UNIQUE(org,name); no-op without a model). account.go OAuth
callback seeds per-org after EnsureWorkspace (best-effort, never blocks login).
Native TestSeedPersonalities green. TEAM_AGENTS_ENABLED=1 already live +
AIDefaultModel=deepseek-v4-flash → the crew materializes and answers @-mentions.

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 18:00:20 -07:00
5f518ae493 rebuild(cloud): embed console-embed@sha-bd8a816 (casibase auth /v1/* fix + console per-project resources) + activate MCP builtin tool-plane (#292)
No Go change — this rebuild re-resolves the freshly-republished console-embed:latest
(console main bd8a81651) into the go:embed console served at console.hanzo.ai, and
ships builtin.go's auto '/v1 route → MCP tool' plane at /v1/tools/mcp (already in main,
newer than the deployed v1.801.69).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 18:00:07 -07:00
zeekayandhanzo-dev 2dfce35465 chore: trigger release build for iam2 v0.15.4 (federation fix)
d411200 (iam2 v0.15.1→v0.15.4 bump) did not trigger a release run; nudge
the push-triggered release so the federation-security-fixed image ships.
2026-07-26 18:00:07 -07:00
hanzo-dev be8651ae31 Merge remote-tracking branch 'origin/main' into fix/agent-default-enso
# Conflicts:
#	go.mod
2026-07-26 17:54:12 -07:00
hanzo-dev 6058aa3532 deps: repin hanzoai/o11y to v1.5.31 — the tag, not a GC'd pseudo-version
main was unbuildable. go.mod pinned
o11y v1.5.31-0.20260726155004-2b66f3201d03 and that revision no longer
exists upstream, so every container build died at `go mod download` with
"invalid version: unknown revision 2b66f3201d03".

v1.5.31 is cut from o11y 6c1f0135b, which is the commit the pseudo-version
was reaching for: it carries the kv-go cache refactor that drops upstream
go-redis, and it still pins hanzoai/sqlite v0.3.2. Tagging HEAD instead
would have dragged in sqlite v0.4.0 and forced a data-plane driver bump
through MVS; falling back to v1.5.30 would have re-added the go-redis
dependency the refactor just removed. This is the one commit that fixes
the build without doing either.
2026-07-26 17:53:24 -07:00
zeekayandhanzo-dev 8eff71c937 deps: hanzoai/iam v1.33.8 -> v1.33.15 (CORS preflight)
cloud imports the IAM server at a published semver and serves it in-process, so
the embedded identity surface was seven patches behind the standalone one. That
mattered: v1.33.15 is the release where the Guard stops authenticating CORS
preflights. Standalone deploy/iam is already on it, so leaving cloud on v1.33.8
meant flipping hanzo.id to the embedded IAM would have silently REGRESSED the
fix — the browser would go back to a 401 preflight and an empty org switcher.

v1.33.15 renames the entrypoint (its "Route, not Mount" change), so the call and
the comments around it move with it.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 17:43:28 -07:00
antje e6a1af1a53 build: pin hanzoai/o11y to the released v1.5.30, not a revision that never shipped
main was unbuildable. go.mod pinned o11y at the pseudo-version
v1.5.31-0.20260726155004-2b66f3201d03, and that revision exists in exactly one
place: somebody's module cache. It is not on origin and not reachable from any
tag, so `go mod download` in the on-cluster build resolved "unknown revision"
and every image build of cloud failed before it compiled a line.

v1.5.30 is the last released patch and its tree is byte-identical to the cached
pseudo-version (diff -rq over both module directories is empty), so this is the
same code under a name the builder can actually fetch — not a version bump, and
not a jump forward past anything.
2026-07-26 17:37:02 -07:00
antje 9bafa850cd money: the SELECTED org is the payer of record
The @hanzo/iam switcher moved data and nothing else. A person picks an org,
every surface stamps X-Org-Id from that choice, and the money still came out
of their HOME org: the boundary re-minted X-Org-Id from the `owner` claim
unconditionally, and principal.BillingOrg keyed the debit on home anyway. Both
halves are fixed here, and they are one fix — one org, for data and money alike.

  - SanitizeIdentity honors a selection the token's signed `orgs` claim says the
    caller belongs to (isMember). IAM already mints that set, home first;
    nothing else can widen it. A selection OUTSIDE the set is discarded, not
    refused — a stale localStorage choice after a revoked membership reads and
    bills the caller's OWN org, never someone else's. A legacy token, an opaque
    key and a machine principal carry no set, so they cannot switch at all.
  - principal.BillingOrg returns that effective org. Platform sudo is the one
    exception and is not a selection: a masquerading SuperAdmin still spends
    from the admin ledger, keyed on the header the boundary already mints.
  - HomeOrg -> Ledger. A name that states the wrong fact is how a gate ends up
    keying one wallet while the debit spends another.

And an unresolvable org now REFUSES instead of charging someone else:

  - WalletOf propagates ok=false. It used to discard BillingOrg's ok-bit and
    return an EMPTY ledger, which is not "no ledger" but "whatever the next
    layer substitutes".
  - metering.AuthorizeVerdict and Record refuse an empty org. orgFor fell back
    to the deployment's BRAND org, so an org-less principal was gated against
    Hanzo's balance and its debit posted under Hanzo's header. orgFor stays for
    CONFIG reads (spend rules, caps, plan tier), where the platform's own org IS
    the right default, and is no longer reachable from the money path.

Tests fail without the change: TestSelectedOrgIsThePayer bills "hanzo" instead
of the selected "acme"; TestUnresolvableOrgRefuses resolves a wallet with no
payer; TestUnresolvableOrgRefuses_NoBrandSubstitute posts a real debit to the
brand's ledger.
2026-07-26 17:37:02 -07:00
hanzo-dev c1a36548e1 test(guide): prove the eight Guide growth seams end to end
Mounts the real wire — framework + content + automations + guide + company on one
zip.App — and walks a fresh org through the agentic-company journey, asserting each
seam against the in-process subsystems instead of a stub. The only fakes are the AI
completion (a deterministic draft) and the growth-observe seams, which are bound to
org-scoped reads exactly as apps/wire_seams.go binds them in prod.

What it proves:

  1. formation begins at structure and is idempotent, and the founder-KYC gate holds:
     an unattributed founder cannot cross founders to payment (422), a non-SuperAdmin
     decision is refused (403), and only an attributed SuperAdmin reviewer
     confirmation opens it.
  2. the marketing module and its Campaign DocType resolve for an org that never ran
     an install, while an unknown module still resolves false.
  3. the observe layer is org-scoped: seams true only for alpha leave beta at formed
     with zeroed metrics and no signal set.
  4. suggest ranks the journey root first, reports it automatable, and fails closed
     without a principal.
  5. the tactics corpus filters by observed stage; an explicit stage query lifts the
     stage floor but can never unlock a has:<capability> tactic whose signal is
     absent; a category query is a strict subset of the unfiltered read.
  6. a content_generate step runs the real invoke to content to framework path: a
     Campaign lands, the step auto-marks done, and the acted detector independently
     re-marks it done after a reset.
  7. the blueprint admin plane is SuperAdmin-gated, and a disable takes effect on the
     next org resolve.
  8. the loop advances autonomously over four iterations: the observed stage climbs
     formed to launched to activated to scaling as each milestone's real effect comes
     online, the next-step pointer walks the chain, and three real work-product docs
     land.

Seam 5 asserts the category filter server-side against a category that actually
surfaces at the observed stage, so a non-empty result is required before any
no-leak claim is made — an empty read would prove nothing.

LLM.md records why these tests looked slow: they are fsync-bound, and t.TempDir()
under /tmp puts every SQLite commit behind the ext4 journal.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 17:35:16 -07:00
zeekay 25c1a1d3f3 deps: hanzoai/orm v0.6.10 — one version across the fleet
v0.6.10 is the current published tag and main matches it exactly. Every Go
backend that depends on orm now pins the same one, so "the orm" means a single
thing everywhere rather than a spread of v0.6.8 / v0.6.9 / pseudo-versions.

v0.6.10 carries db.Registry as a generic Registry[T io.Closer]. That generic is
the point: the registry calls exactly one method on a handle — Close — because
releasing it is the whole of its job, and what a handle IS stays the caller
business. Pinning T to orm own DB interface would have forced every owner of
per-tenant files to adopt orm entity API just to get lifecycle management, which
is exactly the toll that made commerce write its own unbounded userDBs/orgDBs
maps instead of reusing this. commerce now runs RegistryConfig[DB] with its own
DB type.

Verified per repo rather than assumed: commerce ./db builds and its tests pass,
git ./models/db builds, base orm consumer builds.
2026-07-26 17:34:28 -07:00
hanzo-dev 84a7f7b9a1 fix(agents): default to enso, and stop upstream names reaching a customer
An agent created without a model was stored with the deployment default,
whose literal was an UPSTREAM name — so GET /v1/agents answered
"deepseek-v4-flash" for des, dev, vi and verify-run. That publishes which
base sits behind a Hanzo model, which is exactly what the enso name exists
to abstract. It is also a margin leak: the branded SKUs carry retail
pricing while the raw upstream ids are served near cost.

ONE place decides the default. cloud.DefaultModel is the only literal;
CLOUD_AI_DEFAULT_MODEL defaults to it and every subsystem reads that. The
duplicate hardcode in clients/code/ask.go is gone.

ONE function decides the brand boundary. cloud.ZenModel maps an upstream
family name to the Hanzo name; cloud.UpstreamModel is the predicate behind
it, matching a family as a word so "qwen3.5-397b" matches and "zen5-coder"
does not. Applied at every write (config at Mount, caller at create and
update) and guarded at every read (toView, toRunView, the activity feed),
so an upstream name can neither enter the registry nor be served from it.

Store.migrateModel rewrites the rows that predate this. Idempotent: it
selects by the same predicate that decides where rows land, so a second
pass moves nothing. Reversible: the pre-migration value goes to
model_snapshot first (INSERT OR IGNORE, so the first value seen is kept),
and one UPDATE restores it. agent_runs is deliberately NOT rewritten — a
run records what actually happened; toRunView guards the presentation.

Two further leaks found by sweeping the customer-visible surfaces:

  - GET /v1/benchmark/presets shipped the enso-ultra arm list as "shipped
    enso-ultra blend" — a direct enso-to-upstream disclosure. The reference
    preset is now a worked example in models we name.
  - hanzo code --help advertised upstream ids, and the Zen5 Pro picker
    description named the base it maps to.

The guard is clients/agents/brand_test.go: it drives every read of the
registry against a deployment configured with an upstream default and
scans the raw response bytes for any upstream family. Reverting any one of
the write or read guards makes it fail.

That package could not run at all — since hanzoai/sqlite v0.3 the pure-Go
build is encryption-capable, so every store open wanted a master key and
all 104 tests errored out. It now has the same TestMain every other
store-backed package already uses.

BLAST RADIUS: an agent with no explicit model moves from a near-cost
upstream SKU to the branded family, and enso is priced $4/$20 per Mtok
against deepseek-v4-flash at $0.14/$0.28. The four live rows carry 21 runs
total, so absolute exposure today is small, but the rate change is real
and the knob is one constant plus CLOUD_AI_DEFAULT_MODEL.
2026-07-26 17:30:34 -07:00
zeekay 6f6e03ff81 deps: repin hanzoai/ai to v1.831.6 — the tag, not a pseudo-version
Was v1.831.5-0.20260726065328-5420f6ba9987 while tag v1.831.6 exists and is
newer. A pseudo-version shadowing a real, later tag is the drift that had this
fleet disagreeing about what a dependency means — the same defect fixed in cloud
and commerce for hanzoai/orm today.

Checked the rest rather than bulk-repinning: cloud sendgrid-go and o11y, and
commerce sendgrid-go, are pseudo-versions AHEAD of their latest tag. Those are
the honest case for an unreleased commit, and "fixing" them would be a
downgrade. Only the two genuinely shadowing pins moved.
2026-07-26 17:00:33 -07:00
antje 716217670f fix(auth): a publishable key must not authenticate — then one pk-
pk- is the published key and sk- is the secret one. Cloud did not hold that line:
IdentityFromRequest resolved ANY isAPIKey token into "the same principal a JWT
yields", and pk- is in APIKeyPrefixes. So a key IAM documents as "stored
verbatim, safe to show" — the one you put in a browser bundle — authenticated
like a secret and could READ everything its owning org could.

The codebase had three different answers on this and could not all be right:
auth_identity.go called pk- write-only, build.go called it read-only, and
analytics/publishable.go said the IAM family "mint a FULL principal that can
READ" and therefore built a SECOND publishable-key family to avoid it — pk_
(underscore), HMAC'd under CLOUD_INGEST_KEY_SECRET, with its own mint endpoint.
The underscore was load-bearing: it kept that key out of isAPIKey.

Fixed at the boundary rather than routed around it. IdentityFromRequest refuses a
publishable key outright, so publishable means publishable at every door, not
just the one that remembered to dodge. pk- deliberately STAYS in APIKeyPrefixes:
OrgForKey must still resolve it to learn which tenant a browser beacon belongs
to. Resolvable, not authenticating.

That makes the collapse safe, so it happens in the same commit: the pk_ family,
its HMAC, its secret and POST /v1/ingest/keys are gone, and /v1/event resolves a
pk- through the SAME IAM seam as every other key. One publishable key, issued by
the service that owns identity.

Order matters and is the whole point — collapsing first would have shipped a
public read credential.

Verified: full build; the analytics suite green with the auth tests rewritten
against the IAM seam (a stubbed resolveKeyOrg) so they still assert what they
always did — admitted, key's org beats a forged one, unresolvable fails closed.
TestIAMEmbedBehindMiddlewareChain and TestConfigValidate_ShardSafety fail
identically with this change stashed; they are pre-existing.
2026-07-26 16:58:53 -07:00
zeekayandhanzo-dev 514c4f8e2f ci: resolve the reusable from .hanzo/workflows
The forge resolves `uses:` only under WORKFLOW_DIRS (.hanzo/workflows), so
pointing at hanzoai/ci/.github/workflows/build.yml@v1 failed outright:

  path ".github/workflows/build.yml" must be under a configured workflow directory

This repo has built NOTHING since that took effect. hanzoai/ci now publishes the
reusable from .hanzo/workflows/build.yml and tags it v2; a new tag rather than a
force-moved v1, because moving a floating tag is what desynced that repo's two
heads earlier today.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 16:57:09 -07:00
hanzo-dev 5116fbf6fc git(webhook): read X-Git-* headers, accept the pre-rename pair
Our webhook header says X-Git, not X-Gitea. Cloud is the RECEIVER, so it moves
first: it now prefers X-Git-Event / X-Git-Signature and still reads the
X-Gitea-* pair the git image sends today.

Order is the whole point. The two images roll independently, and if the fork
started sending X-Git-* while cloud only understood X-Gitea-*, every push
webhook would fall through the event check and 204 — no signature error, no log,
deploys just silently stop firing. Receiver-accepts-both makes either order
safe. The fallback is a rename in flight, not a compatibility layer: it comes
out in the same change that makes the fork send the new names.

Also drops the upstream name from this file
type giteaPush -> pushEvent, and the doc comments now describe the ingest
without it. The two remaining mentions are the pre-rename header literals,
which are wire values, not our branding.

TestWebhookAcceptsBothHeaderSpellings covers it with one app and one capture,
asserting the build count climbs 1 then 2 so neither spelling can pass on the
others build. Proven non-vacuous: with the fallback reads removed it fails
X-Gitea-Event/X-Gitea-Signature not honored: builds = 1, want 2. Polls for the
count rather than sleeping, because fireBranchBuild is detached from the request
(context.WithoutCancel) - a fixed sleep is what made the first version flake on
TempDir cleanup.

go test ./clients/git/ -run Webhook passes; go build ./clients/git/ clean;
both files gofmt-clean. (A fresh worktree cannot link ./cmd/account - clients/flags
wants native/flags/target/release/libhanzo_flags.a, a Rust artifact only the main
checkout has built. Pre-existing, unrelated to this change.)
2026-07-26 16:52:32 -07:00
zeekayandhanzo-dev 1558023ec6 rip beego out of cloud
The last real use was sessionAccessToken, which mapped a first-party session
cookie to a server-stored JWT by reading web.GlobalSessions — the process-global
the retired Casdoor iam-v1 embed populated.

That embed is gone. IAM v2 (github.com/hanzoai/iam) is zip-native on
hanzoai/orm + hanzoai/sqlite and mounts directly on cloud app, so NOTHING in
this binary ever set that global. The function could only return "". It was dead
code keeping an entire web framework, and its process-global config, in the
dependency graph.

Removed rather than kept as a fallback: a bridge to a session manager that is
never initialised is not a fallback, it is a lie about where sessions come from.
If a first-party session needs to resolve to a token again it resolves through
IAM v2.

The guarantee it existed to provide is unchanged and now stronger. The test no
longer skips when a session manager happens to be present and no longer reaches
into a framework global — a session cookie without a bearer resolves ANONYMOUS,
unconditionally. Green.

beego is out of go.mod. One HTTP framework (zip), one data layer (orm).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 16:41:23 -07:00
hanzo-dev efaeedb82c fix(kms): serve the operator's list contract so the standalone KMS can retire
The embedded /v1/kms plane and the standalone KMS answered the same request
differently, which is what blocked pointing the KMS operator at cloud:
  operator sends ?environment=&secretPath=   cloud read ?env=&path=
  operator reads  {"names":[...]}            cloud returned {"secrets":[],"total":0}
Repointing KMS_API_BASE would therefore have parsed as 'no secrets' for all 108
KMSSecret syncs — every service silently starved rather than erroring.

listSecrets now accepts either spelling and returns a superset carrying BOTH
`secrets`/`total` (this plane's clients) and `names` (the operator). One endpoint
serves both, so the standalone can be retired without a flag day or a lockstep
operator release.

firstNonEmpty is split out as a plain function so the precedence rule (primary
before alias, empty treated as absent) is tested without a request context: 5/5.
The remaining clients/kms failures are pre-existing and environmental — the suite
wants CLOUD_KMS_MASTER_KEY_REF (105 failures on untouched main); my test passes
standalone and go vet is clean.

Parity must still be proven against a POPULATED path before the operator is
repointed and the standalone retired.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 16:29:36 -07:00
8913ece0a6 feat(agents): seed the built-in crew @dev @des @vi on org first-touch (re-land) (#342)
Re-land of the crew seed (a parallel force-push to cloud main dropped it).
personalities.go: dev/des/vi personas + idempotent SeedPersonalities(ctx,org)
(one registry, UNIQUE(org,name); no-op without a model). account.go OAuth
callback seeds per-org after EnsureWorkspace (best-effort, never blocks login).
Native TestSeedPersonalities green. TEAM_AGENTS_ENABLED=1 already live +
AIDefaultModel=deepseek-v4-flash → the crew materializes and answers @-mentions.

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 16:28:59 -07:00
zeekayandhanzo-dev 94130dca61 rebuild(cloud): embed console-embed@sha-bd8a816 (casibase auth /v1/* fix + console per-project resources) + activate MCP builtin tool-plane (#292)
No Go change — this rebuild re-resolves the freshly-republished console-embed:latest
(console main bd8a81651) into the go:embed console served at console.hanzo.ai, and
ships builtin.go's auto '/v1 route → MCP tool' plane at /v1/tools/mcp (already in main,
newer than the deployed v1.801.69).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 16:28:59 -07:00
zeekayandhanzo-dev d9a99e6a54 chore: trigger release build for iam2 v0.15.4 (federation fix)
8e61a92 (iam2 v0.15.1→v0.15.4 bump) did not trigger a release run; nudge
the push-triggered release so the federation-security-fixed image ships.
2026-07-26 16:28:59 -07:00
hanzo-dev 4ea8b06f79 fix(code): normalize embedder base to /v1 — deployed CLOUD_AI_BASE_URL lacks /v1, so /v1/code semantic tier posted to /embeddings (405) → vectors:0. Append /v1 when absent; ask/RAG now works. (#238) 2026-07-26 16:28:59 -07:00
hanzo-dev 829cdf72cc refactor(auto): decomplect — remove the /v1/auto reverse-proxy + clients/auto (#176)
Kill the second automation surface. /v1/auto was a per-org reverse proxy
(clients/auto + clients/auto/proxy) to the standalone hanzoai/auto engine
(auto.hanzo.svc) — a duplicate of the native, in-process /v1/automations
Connectors+Automations engine (clients/automations, cloud.EmbeddedTasks,
706-piece catalogue). One engine, one surface: /v1/automations is the ONE
native automation engine. The external engine + its console link-out are
retired (console + universe in paired PRs).

- Remove the order-140 blank import of clients/auto from subsystems.
- Delete clients/auto/ (auto.go + proxy/).

No functional loss: /v1/automations already serves flows/versions/runs/
pieces/MCP natively. clients/kb keeps its own AUTO_UPSTREAM piece-runner
coupling (a separate, pre-existing bridge to a never-implemented engine
endpoint) — reported for a follow-up, not touched here.

go build ./... green, go vet green, go test ./clients/automations + root ok.
2026-07-26 16:28:59 -07:00
hanzo-dev 802bb6ad29 cloud: follow the Kafka adaptor to its canonical name
hanzoai/stream is retired as a product -- it is the Kafka interface to Hanzo
PubSub, not a thing of its own -- so the module is now github.com/hanzoai/kafka.
The repo had in fact been redirecting hanzoai/kafka -> hanzoai/stream, so the
import path and the canonical URL disagreed.

Pinned v1.2.1, not the newer tag on the adaptor's main. main had already dropped
Broker.Serve() for Startup() + caller-driven HandleConnection(), and this
package calls Serve(): taking main would have broken the build for a reason
unrelated to the rename. v1.2.1 is v1.2.0 plus the rename, same API. Migrating
onto Startup/HandleConnection is a separate deliberate step.

clients/kafka tests pass.
2026-07-26 16:23:03 -07:00
zeekay 8a38817b5e deps: hanzoai/orm v0.6.8 — the published tag, not a pseudo-version
cloud was pinned to v0.6.8-0.20260726065619-7b3c62da906d: a commit, not a
release. A pseudo-version pins a moment rather than a contract, so MVS can
resolve a tree nobody reviewed and nothing in the fleet agrees on what "the orm"
is.

Now the same published v0.6.8 as hanzoai/git and hanzoai/visor. That tag carries
db.Registry — per-tenant resolve/bound/evict/materialise, the primitive behind
one SQLite per object with S3 as the source of truth.
2026-07-26 16:12:38 -07:00
zeekayandhanzo-dev c4047ee492 docs: the CD plane is /v1/deploy/*, not /v1/deploy/api/*
/v1/ only, never /api/. Two package comments claimed an /api/ segment
this code has never served:

  dashboard.go:1   "projection API at /v1/deploy/api/*"
  projection.go:14 "ArgoCD-UI-compatible /v1/deploy/api/v1/* surface"

The registered prefix is one constant — dashboard.go:45
`const dashPrefix = "/v1/deploy"` — and every route is dashPrefix+"/...".
The same file's own line 6 already says "no /api/, no inner /v1", and
e2e/tests/100-cd-deploy.spec.ts:68 asserts /v1/deploy/api/v1/applications
404s. So the comments contradicted the code, the sibling comment, and the
test. Comment-only: a reader copying the documented path would have built
a client against a URL that 404s.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 16:01:22 -07:00
zeekay e72addafca fix: point forge API calls at /v1 — /api/v1 is gone
The fork moved its API off /api to /v1, so every call built against
${{ github.server_url }}/api/v1/... now 404s. Verified live with a control:
/v1/version 200, /api/v1/version 404, a nonsense path 404.

This is the build-dispatch in sync-from-github, so a fast-forward from GitHub
was landing commits and then silently failing to trigger the build.
2026-07-26 15:51:34 -07:00
zeekay 08f2554969 fix(provisioning): hand customers a kv:// DSN, not redis://
The DBaaS provisioner returned connection strings as redis://…, and that string
goes to the customer over the JSON API as connectionString. Our own client
parses kv://, kvs:// and unix:// only, so a provisioned KV addon shipped a DSN
that hanzokv/go rejects.

The Datastore spec.type stays "valkey". That one is the operator's contract —
Engine::Valkey in operator/src/crd.rs — not our vocabulary to rename. Same for
the requirepass config directive the server reads.

Pre-existing test failures in this package are unchanged: the failing set is
identical before and after (verified by diff), and includes cases like
TestStore_InstanceColumnRoundTrips that never touch a DSN.
2026-07-26 13:20:19 -07:00
zeekay d2ad33b065 refactor(kb): one prefix — /v1/kb
The knowledge surface mounted every handler twice, at /v1/knowledge and /v1/kb.
Two paths to one handler is two answers to "where is this", and the smoke test
had already picked the other one from the package doc.

/v1/kb is the prefix. The package doc and the smoke check follow it.
2026-07-26 13:16:37 -07:00
zeekay 5876cecb7a refactor(kb): one prefix — /v1/kb
The knowledge surface mounted every handler twice, at /v1/knowledge and /v1/kb.
Two paths to one handler is two answers to "where is this", and the smoke test
had already picked the other one from the package doc.

/v1/kb is the prefix. The package doc and the smoke check follow it.
2026-07-26 13:16:37 -07:00
zeekayandhanzo-dev 00f36a85af chore: commit outstanding working-tree changes
10 files changed, 102 insertions(+), 102 deletions(-)

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 13:16:37 -07:00
zeekayandhanzo-dev 6e82c09897 chore: commit outstanding working-tree changes
10 files changed, 102 insertions(+), 102 deletions(-)

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 13:16:37 -07:00
b1ed2a668e feat(sync/tracker/git): GitHub App → native tracker mirror + index-on-import + .hanzo/workflows orchestrator (#357)
The native side of the org-wide GitHub App sync. The push→reconcile path already
existed (githubWebhook → cloud.Sync); this adds the two missing halves the owner
asked for and layers /v1/code over every mirrored repo.

TRACKER MIRROR (issues → native tracker, the one way, no second store):
- tracker_seam.go: cloud.UpsertIssue inversion seam (twin of cloud.Sync) — feeders
  never import clients/tracker.
- clients/tracker/github_sink.go: registers the sink; idempotent-by-ExtRef upsert
  into one "GH" team (Source=git, Repo discriminator), open/closed→todo/done.
- clients/tracker/store.go: GetIssueByExtRef + (org,project,ext_ref) index.
- clients/integrations/github_webhook.go: `issues`/`issue_comment` → mirror the
  parent issue (same signed-installation→org resolution as push).
- clients/integrations/github_issues.go: mirror mapping + POST
  /v1/integrations/github/issues/backfill (bounded, idempotent, returns counts).

INDEX-ON-IMPORT (/v1/code covers EVERY mirrored repo, not just pushed-since-import):
- clients/git/index_on_import.go + ImportRepo/mirror: emit the SAME
  cloud.LifecyclePushLanded the push/inbound paths emit, so index_on_push indexes
  the default-branch tip on import too. Origin = source host → mirror_out suppresses
  the echo. Detached + best-effort (never blocks/fails the writer).

.hanzo/workflows NATIVE CI/CD (design + dormant MVP):
- clients/git/build_on_push.go: a lifecycle reactor that reads .hanzo/workflows/*.yml
  (or root hanzo.yml, same images: schema) at the pushed tip and enqueues each image
  to platform /v1/arcd/enqueue (BuildKit→operator), NO GitHub Actions. Ships DORMANT
  (no-op unless CLOUD_NATIVE_CICD_ENABLED + the enqueue token) so linking is inert.

Tests: tracker sink (idempotent upsert + status map + isolation), integrations
compile+run, git import emits push.landed, orchestrator parse/body/tag/owner map.
All green (CGO_ENABLED=0, pure-Go as prod ships).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-26 12:30:54 -07:00
3baf677016 feat(sync/tracker/git): GitHub App → native tracker mirror + index-on-import + .hanzo/workflows orchestrator (#357)
The native side of the org-wide GitHub App sync. The push→reconcile path already
existed (githubWebhook → cloud.Sync); this adds the two missing halves the owner
asked for and layers /v1/code over every mirrored repo.

TRACKER MIRROR (issues → native tracker, the one way, no second store):
- tracker_seam.go: cloud.UpsertIssue inversion seam (twin of cloud.Sync) — feeders
  never import clients/tracker.
- clients/tracker/github_sink.go: registers the sink; idempotent-by-ExtRef upsert
  into one "GH" team (Source=git, Repo discriminator), open/closed→todo/done.
- clients/tracker/store.go: GetIssueByExtRef + (org,project,ext_ref) index.
- clients/integrations/github_webhook.go: `issues`/`issue_comment` → mirror the
  parent issue (same signed-installation→org resolution as push).
- clients/integrations/github_issues.go: mirror mapping + POST
  /v1/integrations/github/issues/backfill (bounded, idempotent, returns counts).

INDEX-ON-IMPORT (/v1/code covers EVERY mirrored repo, not just pushed-since-import):
- clients/git/index_on_import.go + ImportRepo/mirror: emit the SAME
  cloud.LifecyclePushLanded the push/inbound paths emit, so index_on_push indexes
  the default-branch tip on import too. Origin = source host → mirror_out suppresses
  the echo. Detached + best-effort (never blocks/fails the writer).

.hanzo/workflows NATIVE CI/CD (design + dormant MVP):
- clients/git/build_on_push.go: a lifecycle reactor that reads .hanzo/workflows/*.yml
  (or root hanzo.yml, same images: schema) at the pushed tip and enqueues each image
  to platform /v1/arcd/enqueue (BuildKit→operator), NO GitHub Actions. Ships DORMANT
  (no-op unless CLOUD_NATIVE_CICD_ENABLED + the enqueue token) so linking is inert.

Tests: tracker sink (idempotent upsert + status map + isolation), integrations
compile+run, git import emits push.landed, orchestrator parse/body/tag/owner map.
All green (CGO_ENABLED=0, pure-Go as prod ships).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-26 12:30:54 -07:00
antje 4746b9759c docs: name our engine Hanzo Datastore in comments; fix stale driver claims
Comment/prose only (plus one test's own t.Fatal diagnostic string).
go build + go vet green on every touched package.

clients/o11y/LLM.md also carried two claims the code disproves: it said the
query plane READS "via clickhouse-go v2.44.0 (upstream)" and pins "upstream
ch-go v0.71.0". Neither module is in the tree —

  grep -cE 'ClickHouse/ch-go|ClickHouse/clickhouse-go' go.sum go.mod -> 0 0
  clients/o11y/event_ingest.go:59:  datastore "github.com/hanzo-ds/go"
  go.mod: github.com/hanzo-ds/go v1.0.1 · github.com/hanzo-ds/native v0.72.0
  gh api /repos/hanzo-ds/native -> "de-branded fork of ch-go"

Deliberately NOT renamed, each verified third-party:
  - clients/blueprint/estimate.go dbHints — sniffs a USER's project deps
  - clients/guide/base.yaml — factual prose about ClickHouse Inc's public
    benchmarks; rebranding it would make a true statement false
  - clients/samples/live_test.go — `docker run clickhouse/clickhouse-server`
  - o11y/LLM.md `clickhousetraces`/`clickhouselogsexporter` — OTel exporter
    factory type strings, must match the compiled collector
  - o11y/LLM.md `ch-go` where it describes the external dd-sketch exporter
2026-07-26 11:54:31 -07:00
antje bb50b2aeef docs: name our engine Hanzo Datastore in comments; fix stale driver claims
Comment/prose only (plus one test's own t.Fatal diagnostic string).
go build + go vet green on every touched package.

clients/o11y/LLM.md also carried two claims the code disproves: it said the
query plane READS "via clickhouse-go v2.44.0 (upstream)" and pins "upstream
ch-go v0.71.0". Neither module is in the tree —

  grep -cE 'ClickHouse/ch-go|ClickHouse/clickhouse-go' go.sum go.mod -> 0 0
  clients/o11y/event_ingest.go:59:  datastore "github.com/hanzo-ds/go"
  go.mod: github.com/hanzo-ds/go v1.0.1 · github.com/hanzo-ds/native v0.72.0
  gh api /repos/hanzo-ds/native -> "de-branded fork of ch-go"

Deliberately NOT renamed, each verified third-party:
  - clients/blueprint/estimate.go dbHints — sniffs a USER's project deps
  - clients/guide/base.yaml — factual prose about ClickHouse Inc's public
    benchmarks; rebranding it would make a true statement false
  - clients/samples/live_test.go — `docker run clickhouse/clickhouse-server`
  - o11y/LLM.md `clickhousetraces`/`clickhouselogsexporter` — OTel exporter
    factory type strings, must match the compiled collector
  - o11y/LLM.md `ch-go` where it describes the external dd-sketch exporter
2026-07-26 11:54:31 -07:00
zeekayandClaude Opus 5 9cf85146fb ci(containment): prove containment from the import graph, not from a link
CI/CD / containment (push) Successful in 3m45s
Hanzo CI/CD / cicd (push) Successful in 12m15s
CI/CD / gate (push) Successful in 12m15s
The containment job led with `go build ./...`, which links every cmd/ main —
and linking needs native/flags/target/release/libhanzo_flags.a, the Rust
staticlib clients/flags pulls in under cgo. Nothing in this job builds it:
`make native` runs in the gate job, in that job's own workspace. On GitHub it
passed only because the arc runners reuse a workspace and target/ is
gitignored, so an earlier gate job's leftover .a was still on disk. On a
container runner with a fresh volume there is no leftover, and every cmd/ main
died at `ld: cannot find .../libhanzo_flags.a` — a guard job failing on an
artifact it does not own, telling us nothing about containment.

Containment is an import-graph property and `go list -deps` — already the
check doing the real work here — reads the same untagged file set without
linking. The compile added no proof, only a second place that has to build.
The gate job stays the ONE place that builds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 11:22:30 -07:00
zeekayandhanzo-dev 517b8a0650 ci(containment): prove containment from the import graph, not from a link
The containment job led with `go build ./...`, which links every cmd/ main —
and linking needs native/flags/target/release/libhanzo_flags.a, the Rust
staticlib clients/flags pulls in under cgo. Nothing in this job builds it:
`make native` runs in the gate job, in that job's own workspace. On GitHub it
passed only because the arc runners reuse a workspace and target/ is
gitignored, so an earlier gate job's leftover .a was still on disk. On a
container runner with a fresh volume there is no leftover, and every cmd/ main
died at `ld: cannot find .../libhanzo_flags.a` — a guard job failing on an
artifact it does not own, telling us nothing about containment.

Containment is an import-graph property and `go list -deps` — already the
check doing the real work here — reads the same untagged file set without
linking. The compile added no proof, only a second place that has to build.
The gate job stays the ONE place that builds.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 11:22:30 -07:00
zeekayandClaude Opus 5 12c35ea045 test(apps): add the prefs frozen row — wire golden was latent-red on main
Wire() mounts prefs (63fc3daf) between settings and notify; the frozen
sequence never got the row, so TestWireOrderMatchesFrozen has failed on main
ever since with "Wire() has 107 specs, frozen sequence has 106". Same shape as
af16ada7 (destinations).

The row records what Wire actually does — OwnsHealth false, Shutdown non-nil
(prefs.Shutdown closes the store) — so the golden still fails the moment the
mount ORDER or those flags change, which is the only thing it is for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 11:22:22 -07:00
zeekayandhanzo-dev 526172f200 test(apps): add the prefs frozen row — wire golden was latent-red on main
Wire() mounts prefs (dd20b05f) between settings and notify; the frozen
sequence never got the row, so TestWireOrderMatchesFrozen has failed on main
ever since with "Wire() has 107 specs, frozen sequence has 106". Same shape as
fe63e9bb (destinations).

The row records what Wire actually does — OwnsHealth false, Shutdown non-nil
(prefs.Shutdown closes the store) — so the golden still fails the moment the
mount ORDER or those flags change, which is the only thing it is for.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 11:22:22 -07:00
hanzo-dev 8146cd3be3 paywall: flip it from admin.hanzo.ai — no redeploy
The switch was already in the cockpit and already registered; it just governed
nothing. clients/flags imports the root package, and the root package imports
routers to mount the edge middleware, so root could never import flags and no
filter mounted from serve.go could read a switch. `paywall_enforced` therefore
drove clients/entitlements.RequireProduct — a leaf, free to import flags — while
the paywall serve.go actually mounts stayed env-gated. And RequireProduct is
mounted nowhere, so flipping that switch changed nothing at all.

Inverted at the direction the dependency already runs, rather than fighting it:
the root package holds the seam (switch.go — SetSwitchReader in, Switch out),
flags fills it on Mount, serve.go reads it. No new import in either direction, and
the key is a constant in the root package so the registry entry, the evaluator and
the edge cannot drift onto different strings.

Enforcement is now `cfg.PaywallEnforced || Switch(...)`, so PAYWALL_ENFORCED stays
the floor: a deployment that already sets it keeps enforcing exactly as before,
and Switch reports false until an engine mounts — an unmounted switch never
enforces a paywall nobody turned on.

Tests cover dark-before-mount, that a flip reaches the edge, that flipping back
OFF restores access (the kill switch is the point), and detach.
2026-07-26 10:32:14 -07:00
hanzo-dev 71bead9cea paywall: flip it from admin.hanzo.ai — no redeploy
The switch was already in the cockpit and already registered; it just governed
nothing. clients/flags imports the root package, and the root package imports
routers to mount the edge middleware, so root could never import flags and no
filter mounted from serve.go could read a switch. `paywall_enforced` therefore
drove clients/entitlements.RequireProduct — a leaf, free to import flags — while
the paywall serve.go actually mounts stayed env-gated. And RequireProduct is
mounted nowhere, so flipping that switch changed nothing at all.

Inverted at the direction the dependency already runs, rather than fighting it:
the root package holds the seam (switch.go — SetSwitchReader in, Switch out),
flags fills it on Mount, serve.go reads it. No new import in either direction, and
the key is a constant in the root package so the registry entry, the evaluator and
the edge cannot drift onto different strings.

Enforcement is now `cfg.PaywallEnforced || Switch(...)`, so PAYWALL_ENFORCED stays
the floor: a deployment that already sets it keeps enforcing exactly as before,
and Switch reports false until an engine mounts — an unmounted switch never
enforces a paywall nobody turned on.

Tests cover dark-before-mount, that a flip reaches the edge, that flipping back
OFF restores access (the kill switch is the point), and detach.
2026-07-26 10:32:14 -07:00
zeekay 806a60f147 fix(api): /api/ returns a real 404 instead of the console shell
We version at /v1/ and never carry an /api/ prefix. But nothing enforced it:
an /api/ request matched no route, fell through to the SPA mount, and came
back 200 text/html. So api.hanzo.ai/api/v1/user rendered the console page,
and api.hanzo.ai/api/totally-made-up-nonsense answered 200 as well.

A caller on the wrong prefix got a page that looked like it worked instead of
an error telling them the prefix does not exist. Adding /api/ to apiPrefixes
puts it in the same class as /v1/, /zap and /healthz: an unmatched path there
is a real 404 in JSON, never the shell.

Pinned with a regression test, since the failure mode is a 200 that looks fine
rather than a crash.
2026-07-26 10:24:38 -07:00
zeekay 70aae48c86 fix(api): /api/ returns a real 404 instead of the console shell
We version at /v1/ and never carry an /api/ prefix. But nothing enforced it:
an /api/ request matched no route, fell through to the SPA mount, and came
back 200 text/html. So api.hanzo.ai/api/v1/user rendered the console page,
and api.hanzo.ai/api/totally-made-up-nonsense answered 200 as well.

A caller on the wrong prefix got a page that looked like it worked instead of
an error telling them the prefix does not exist. Adding /api/ to apiPrefixes
puts it in the same class as /v1/, /zap and /healthz: an unmatched path there
is a real 404 in JSON, never the shell.

Pinned with a regression test, since the failure mode is a 200 that looks fine
rather than a crash.
2026-07-26 10:24:38 -07:00
zeekay 3b33171632 chore(deps): luxfi/kms v1.11.8 -> v1.12.9
KMS is the embedded luxfi/kms SecretStore in this binary serving /v1/kms/*,
and it was eight minor versions behind. The standalone image in-cluster is
already v1.12.10, so the library the API runs on was older than the daemon it
replaced. clients/kms builds on it.
2026-07-26 10:16:53 -07:00
zeekay 2c887dba3b chore(deps): luxfi/kms v1.11.8 -> v1.12.9
KMS is the embedded luxfi/kms SecretStore in this binary serving /v1/kms/*,
and it was eight minor versions behind. The standalone image in-cluster is
already v1.12.10, so the library the API runs on was older than the daemon it
replaced. clients/kms builds on it.
2026-07-26 10:16:53 -07:00
hanzo-dev 6dbb59fe54 paywall: read enforcement per request, and name why the cockpit cannot reach it
You asked to flip the paywall from admin.hanzo.ai. Two things stood in the way,
and only one of them is code I can move.

Fixed: enforcement was evaluated ONCE at mount — Paywall(bool) chose a
passthrough closure at startup — so the value could not change without
restarting the binary. It is now a func read per request, so the moment a
reader is wired the switch hot-applies. A nil reader is the dark default.
serve.go passes cfg.PaywallEnforced for now, byte-identical behaviour.

Also gave the cockpit switch the env fallback it was missing: the
`paywall_enforced` Def registered no Env, so it resolved store -> "false" and
ignored PAYWALL_ENFORCED entirely. Anyone reading that switch got a different
answer than the running gate. It now falls back to the same env var, and
entitlements exports Enforced() as the ONE reader of the key.

NOT fixed, because it is structural and worth stating rather than hacking
around: clients/flags imports the ROOT package (cloud.Deps/Handle/OrgStore),
and the root package imports routers to mount this middleware. So root can
never import flags, and NO edge filter mounted from serve.go can read a
switch. That is why `paywall_enforced` governs
clients/entitlements.RequireProduct — a leaf, free to import flags — while the
middleware serve.go actually mounts is env-gated. Worse, RequireProduct is
mounted NOWHERE (every call site is a commented-out example), so flipping that
switch in admin.hanzo.ai today changes nothing at all. Closing it means either
inverting flags off the root package, or moving enforcement onto RequireProduct
at the gated route groups. Both are real changes; neither is a config nudge.

Tests: enforcement flips on an already-mounted app (on, off, and back — the
kill switch must restore access), and a nil reader never gates.
2026-07-26 09:11:11 -07:00
hanzo-dev 704011d389 paywall: read enforcement per request, and name why the cockpit cannot reach it
You asked to flip the paywall from admin.hanzo.ai. Two things stood in the way,
and only one of them is code I can move.

Fixed: enforcement was evaluated ONCE at mount — Paywall(bool) chose a
passthrough closure at startup — so the value could not change without
restarting the binary. It is now a func read per request, so the moment a
reader is wired the switch hot-applies. A nil reader is the dark default.
serve.go passes cfg.PaywallEnforced for now, byte-identical behaviour.

Also gave the cockpit switch the env fallback it was missing: the
`paywall_enforced` Def registered no Env, so it resolved store -> "false" and
ignored PAYWALL_ENFORCED entirely. Anyone reading that switch got a different
answer than the running gate. It now falls back to the same env var, and
entitlements exports Enforced() as the ONE reader of the key.

NOT fixed, because it is structural and worth stating rather than hacking
around: clients/flags imports the ROOT package (cloud.Deps/Handle/OrgStore),
and the root package imports routers to mount this middleware. So root can
never import flags, and NO edge filter mounted from serve.go can read a
switch. That is why `paywall_enforced` governs
clients/entitlements.RequireProduct — a leaf, free to import flags — while the
middleware serve.go actually mounts is env-gated. Worse, RequireProduct is
mounted NOWHERE (every call site is a commented-out example), so flipping that
switch in admin.hanzo.ai today changes nothing at all. Closing it means either
inverting flags off the root package, or moving enforcement onto RequireProduct
at the gated route groups. Both are real changes; neither is a config nudge.

Tests: enforcement flips on an already-mounted app (on, off, and back — the
kill switch must restore access), and a nil reader never gates.
2026-07-26 09:11:11 -07:00
hanzo-dev d0baefc484 deps: pin commerce to a real tag — main did not build
go.mod pinned github.com/hanzoai/commerce v1.49.20-0.20260726065453-1f6a1e10775c.
A vX.Y.Z-0.<ts>-<hash> pseudo-version means "a commit after vX.Y.(Z-1)", so Go
validates it against v1.49.19 — and that tag is NOT on the remote. commerce jumps
v1.49.12 -> v1.49.20 there, while v1.49.19 exists only in a local clone, so it was
either never pushed or later removed. Every consumer of the module graph therefore
failed resolution, not just one package:

  invalid pseudo-version: preceding tag (v1.49.19) not found

Reproduced on pristine origin/main, so this is not from any local edit. It is the
same shape as the iam v1.33.6 re-tag: a moved or missing tag breaking everyone
downstream, and it blocks building a cloud image at all.

Fixed by pinning the real immutable tag v1.49.21, which CONTAINS the commit the
pseudo-version was reaching for (verified: git merge-base --is-ancestor
1f6a1e10775c v1.49.21). That also satisfies the standing rule that pins are
immutable semver, never floating or derived.

Verified: ./clients/commerceclient builds, and the root package, ./routers and
./clients/entitlements all vet clean. The remaining local link errors are a
missing prebuilt native/flags/target/release/libhanzo_flags.a, which CI builds —
unrelated to this and present before.
2026-07-26 09:10:55 -07:00
hanzo-dev 21cd20589e deps: pin commerce to a real tag — main did not build
go.mod pinned github.com/hanzoai/commerce v1.49.20-0.20260726065453-1f6a1e10775c.
A vX.Y.Z-0.<ts>-<hash> pseudo-version means "a commit after vX.Y.(Z-1)", so Go
validates it against v1.49.19 — and that tag is NOT on the remote. commerce jumps
v1.49.12 -> v1.49.20 there, while v1.49.19 exists only in a local clone, so it was
either never pushed or later removed. Every consumer of the module graph therefore
failed resolution, not just one package:

  invalid pseudo-version: preceding tag (v1.49.19) not found

Reproduced on pristine origin/main, so this is not from any local edit. It is the
same shape as the iam v1.33.6 re-tag: a moved or missing tag breaking everyone
downstream, and it blocks building a cloud image at all.

Fixed by pinning the real immutable tag v1.49.21, which CONTAINS the commit the
pseudo-version was reaching for (verified: git merge-base --is-ancestor
1f6a1e10775c v1.49.21). That also satisfies the standing rule that pins are
immutable semver, never floating or derived.

Verified: ./clients/commerceclient builds, and the root package, ./routers and
./clients/entitlements all vet clean. The remaining local link errors are a
missing prebuilt native/flags/target/release/libhanzo_flags.a, which CI builds —
unrelated to this and present before.
2026-07-26 09:10:55 -07:00
zeekay 1f3314afea chore(deps): hanzokv/go v9.22.0, via hanzoai/o11y
v9.22.0 renames the package identifier from redis to kv, and its companion
modules moved path with it: extra/redisotel -> extra/kvotel and extra/rediscmd
-> extra/kvcmd, which stop at v9.21.1 under the old names.

Nothing in cloud imports any of them, but hanzoai/o11y's rediscache does, so
cloud cannot bump the client alone: extra/rediscmd v9.21.1 still says redis.X
and does not compile against a package now called kv. Taking o11y at
2b66f3201, which moved to the kv-named companions, resolves all three
consistently.
2026-07-26 08:59:57 -07:00
zeekay 6c720908f4 chore(deps): hanzokv/go v9.22.0, via hanzoai/o11y
v9.22.0 renames the package identifier from redis to kv, and its companion
modules moved path with it: extra/redisotel -> extra/kvotel and extra/rediscmd
-> extra/kvcmd, which stop at v9.21.1 under the old names.

Nothing in cloud imports any of them, but hanzoai/o11y's rediscache does, so
cloud cannot bump the client alone: extra/rediscmd v9.21.1 still says redis.X
and does not compile against a package now called kv. Taking o11y at
2b66f3201, which moved to the kv-named companions, resolves all three
consistently.
2026-07-26 08:59:57 -07:00
hanzo-dev ae9c334e69 paywall: decide on a canonical path, so casing and a slash cannot flip the gate
Two bugs, one cause: gated() and allowlisted() compared the raw request path
against lists written in lower case with no trailing slash. The same route in
another form missed both, and which list it missed decided which way it broke.

  /V1/chat/completions   missed the "/v1/" prefix    -> skipped the paywall
  /v1/IAM/callback       missed the "/v1/iam/" allow -> 402 on an OAuth callback
  /v1/signin/            matched no exact case       -> 402 in front of sign-in

So a shift key was a bypass, and a trailing slash was a lockout. Both are now
one canonical() applied once at the top of gated(), with allowlisted() reading
the folded value — the gate decides on a single form and cannot disagree with
itself. /v1/ itself is untouched: nothing is trimmed below the prefix.

Tests assert both directions, including that folding does not open a hole
(/V1/chat/completions/ still gates) and that the non-/v1 surface is unaffected.
Verified they catch the bug rather than restate the code: with canonical()
stubbed to fold nothing, all four fail and name each path above; with the fix,
the package is green.

Enforcement is still off (PAYWALL_ENFORCED=false), so this changes nothing in
production today — it removes two of the reasons the flag could not be flipped.
2026-07-26 08:39:21 -07:00
hanzo-dev 2b3f5f6f58 paywall: decide on a canonical path, so casing and a slash cannot flip the gate
Two bugs, one cause: gated() and allowlisted() compared the raw request path
against lists written in lower case with no trailing slash. The same route in
another form missed both, and which list it missed decided which way it broke.

  /V1/chat/completions   missed the "/v1/" prefix    -> skipped the paywall
  /v1/IAM/callback       missed the "/v1/iam/" allow -> 402 on an OAuth callback
  /v1/signin/            matched no exact case       -> 402 in front of sign-in

So a shift key was a bypass, and a trailing slash was a lockout. Both are now
one canonical() applied once at the top of gated(), with allowlisted() reading
the folded value — the gate decides on a single form and cannot disagree with
itself. /v1/ itself is untouched: nothing is trimmed below the prefix.

Tests assert both directions, including that folding does not open a hole
(/V1/chat/completions/ still gates) and that the non-/v1 surface is unaffected.
Verified they catch the bug rather than restate the code: with canonical()
stubbed to fold nothing, all four fail and name each path above; with the fix,
the package is green.

Enforcement is still off (PAYWALL_ENFORCED=false), so this changes nothing in
production today — it removes two of the reasons the flag could not be flipped.
2026-07-26 08:39:21 -07:00
zeekay 0f334eb808 fix(docker): run tini as PID 1 so orphaned git children get reaped
/cloud is PID 1 in its container (ENTRYPOINT ["/cloud"], no init wrapper,
shareProcessNamespace unset), and PID 1 inherits every orphaned descendant.

git is not one process. fetch/clone fan out to git-upload-pack,
git-index-pack, git-rev-list and git-pack-objects. When cloud kills a wedged
direct child — gitPackStream.Close does exactly that, and correctly, to avoid
hanging on a disconnected client — those grandchildren orphan and reparent to
PID 1. A Go program never reaps adopted orphans and there is no SIGCHLD
handler anywhere in this codebase, so each one becomes a permanent zombie
holding a PID slot.

Measured on worker-xl-37bw71 via a hostPID probe:

  == total procs: 18741
    18553 git
  == zombie count: 18553
  == zombie git PPIDs: 18553 -> ppid 4141630
  --- ppid 4141630: /cloud

18,553 of 18,741 processes on that node were zombie git, every one parented
to /cloud. The node crossed kubelet's pid.available<10% threshold and evicted
pods — including cloud itself, which is replicas:1/Recreate, so it was a full
/v1 outage until reschedule. insights-web went with it. A zombie consumes no
CPU and no memory, so nothing but the eviction ever surfaced it.

Every gitCmd() call site was audited first: all 17 reap correctly via Run,
Output, withPackSlot(ctx, cmd.Run) or Start+Wait. The leak was never a missing
Wait in this code — it is the grandchildren, which only an init can collect.

`--` keeps cloud's own args untouched, and tini forwards signals unchanged so
SIGTERM still drains normally. `test -x /sbin/tini` makes a wrong path fail
the BUILD rather than producing an image that cannot start, matching the
existing `test -n "$SC"` guard beside it.
2026-07-26 06:53:17 -07:00
zeekay 26d5e0f433 fix(docker): run tini as PID 1 so orphaned git children get reaped
/cloud is PID 1 in its container (ENTRYPOINT ["/cloud"], no init wrapper,
shareProcessNamespace unset), and PID 1 inherits every orphaned descendant.

git is not one process. fetch/clone fan out to git-upload-pack,
git-index-pack, git-rev-list and git-pack-objects. When cloud kills a wedged
direct child — gitPackStream.Close does exactly that, and correctly, to avoid
hanging on a disconnected client — those grandchildren orphan and reparent to
PID 1. A Go program never reaps adopted orphans and there is no SIGCHLD
handler anywhere in this codebase, so each one becomes a permanent zombie
holding a PID slot.

Measured on worker-xl-37bw71 via a hostPID probe:

  == total procs: 18741
    18553 git
  == zombie count: 18553
  == zombie git PPIDs: 18553 -> ppid 4141630
  --- ppid 4141630: /cloud

18,553 of 18,741 processes on that node were zombie git, every one parented
to /cloud. The node crossed kubelet's pid.available<10% threshold and evicted
pods — including cloud itself, which is replicas:1/Recreate, so it was a full
/v1 outage until reschedule. insights-web went with it. A zombie consumes no
CPU and no memory, so nothing but the eviction ever surfaced it.

Every gitCmd() call site was audited first: all 17 reap correctly via Run,
Output, withPackSlot(ctx, cmd.Run) or Start+Wait. The leak was never a missing
Wait in this code — it is the grandchildren, which only an init can collect.

`--` keeps cloud's own args untouched, and tini forwards signals unchanged so
SIGTERM still drains normally. `test -x /sbin/tini` makes a wrong path fail
the BUILD rather than producing an image that cannot start, matching the
existing `test -n "$SC"` guard beside it.
2026-07-26 06:53:17 -07:00
zandClaude Opus 4.8 6133c9f2ac feat(build): stamp VERSION from the image tag
Pass the image tag through to buildkit as build-arg:VERSION, so a binary reports
the version it was actually published under (cloud links it into cloud.Version →
the X-Api-Version header) instead of the "dev" default.

The tag is already the authority for what is being built, so deriving VERSION
from it means the reported version can never drift from the ref that was pushed.
Skips "latest" (not a version) and is a no-op for any Dockerfile without an
ARG VERSION.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-26 01:06:53 -07:00
zandhanzo-dev 06e399491b feat(build): stamp VERSION from the image tag
Pass the image tag through to buildkit as build-arg:VERSION, so a binary reports
the version it was actually published under (cloud links it into cloud.Version →
the X-Api-Version header) instead of the "dev" default.

The tag is already the authority for what is being built, so deriving VERSION
from it means the reported version can never drift from the ref that was pushed.
Skips "latest" (not a version) and is a no-op for any Dockerfile without an
ARG VERSION.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-26 01:06:53 -07:00
zeekay 5d878adfdf chore(deps): drop upstream go-redis — one KV client, at github.com/hanzokv/go
cloud imported NEITHER redis client in its own source; both were `// indirect`,
reached through four dependencies. Each has now moved, so tidy drops upstream:

  o11y/pkg/cache/rediscache   (also carried extra/redisotel + the redismock test)
  hanzoai/ai/object
  hanzoai/commerce/infra
  hanzoai/orm

github.com/redis/* count in this go.mod is now 0. Editing cloud directly at any
point would have been a fix at the wrong layer — the duplication lived upstream
of it.
2026-07-25 23:56:56 -07:00
zeekay f0ab9e7fb9 chore(deps): drop upstream go-redis — one KV client, at github.com/hanzokv/go
cloud imported NEITHER redis client in its own source; both were `// indirect`,
reached through four dependencies. Each has now moved, so tidy drops upstream:

  o11y/pkg/cache/rediscache   (also carried extra/redisotel + the redismock test)
  hanzoai/ai/object
  hanzoai/commerce/infra
  hanzoai/orm

github.com/redis/* count in this go.mod is now 0. Editing cloud directly at any
point would have been a fix at the wrong layer — the duplication lived upstream
of it.
2026-07-25 23:56:56 -07:00
antje 63fc3dafdc feat(prefs): one per-user preference plane, so a person is the same person in every product
Today each surface keeps its own copy of "who you are and how you like things" in
its own localStorage, so the same person reads as two different users depending
on which tab they are in — the console remembers a theme insights has never heard
of, and neither survives a new device.

`GET/PATCH /v1/prefs` is the one place that answers it. Two decisions carry the
design:

PATCH, not PUT. A surface saves the keys it owns and nothing else. Under a PUT
every client would be responsible for preserving every other client's keys, and
the first one to forget would silently delete them — so the merge lives on the
server, inside the store transaction. Read-modify-write across two calls is a
lost update, and multi-tab saving is the case a preferences surface actually
sees, not an edge case.

The key is the canonical `<owner>/<name>`, never the bare X-User-Id. A bare name
is not unique across orgs: `hanzo/z` and `admin/z` are two different people, and
keying on `z` would hand one of them the other's document.

Isolation is server-side on every statement, and there is deliberately no path to
read someone else's preferences — not for an org admin, not for a platform
SuperAdmin. Nothing here is a credential, so unlike settings there is no KMS split
to maintain; if a preference ever needs custody it does not belong in this table.

Tests: 8 merge/decode cases (a partial save preserves a foreign surface's keys; a
null value deletes; a nested object is REPLACED, not deep-merged, so a value can
actually be cleared; a corrupt row starts the user fresh instead of failing every
future write; the bound holds on the merged result, not just the patch) plus 3
real-SQLite cases (missing reads as empty, two writers' distinct keys both
survive, two subjects never share a document).

The store tests inject a throwaway cek master key rather than skipping without
one — they skipped on an encryption-capable build at first, which meant the
isolation invariant was untested on exactly the configuration production ships.
2026-07-25 23:09:06 -07:00
antje dd20b05f41 feat(prefs): one per-user preference plane, so a person is the same person in every product
Today each surface keeps its own copy of "who you are and how you like things" in
its own localStorage, so the same person reads as two different users depending
on which tab they are in — the console remembers a theme insights has never heard
of, and neither survives a new device.

`GET/PATCH /v1/prefs` is the one place that answers it. Two decisions carry the
design:

PATCH, not PUT. A surface saves the keys it owns and nothing else. Under a PUT
every client would be responsible for preserving every other client's keys, and
the first one to forget would silently delete them — so the merge lives on the
server, inside the store transaction. Read-modify-write across two calls is a
lost update, and multi-tab saving is the case a preferences surface actually
sees, not an edge case.

The key is the canonical `<owner>/<name>`, never the bare X-User-Id. A bare name
is not unique across orgs: `hanzo/z` and `admin/z` are two different people, and
keying on `z` would hand one of them the other's document.

Isolation is server-side on every statement, and there is deliberately no path to
read someone else's preferences — not for an org admin, not for a platform
SuperAdmin. Nothing here is a credential, so unlike settings there is no KMS split
to maintain; if a preference ever needs custody it does not belong in this table.

Tests: 8 merge/decode cases (a partial save preserves a foreign surface's keys; a
null value deletes; a nested object is REPLACED, not deep-merged, so a value can
actually be cleared; a corrupt row starts the user fresh instead of failing every
future write; the bound holds on the merged result, not just the patch) plus 3
real-SQLite cases (missing reads as empty, two writers' distinct keys both
survive, two subjects never share a document).

The store tests inject a throwaway cek master key rather than skipping without
one — they skipped on an encryption-capable build at first, which meant the
isolation invariant was untested on exactly the configuration production ships.
2026-07-25 23:09:06 -07:00
antje e4b4654d8d git: delete the unwired import trio — one import path, not two
importrepo.go added POST /v1/git/repos/:name/import but the route was never
registered, so its three tests 404'd and go test ./clients/git was red. It
re-composed the exact primitives github_import.go already wires (importFetch,
ensureMirrorTarget, RecordConflict/ClearConflict) — a duplicate import path,
not a second concern. ci.go committed .hanzo/workflows via that dead path only.

Delete all three (719 lines). github_import.go remains the one import path;
the git suite is green.
2026-07-25 22:13:18 -07:00
antje 98e05b6aa4 git: delete the unwired import trio — one import path, not two
importrepo.go added POST /v1/git/repos/:name/import but the route was never
registered, so its three tests 404'd and go test ./clients/git was red. It
re-composed the exact primitives github_import.go already wires (importFetch,
ensureMirrorTarget, RecordConflict/ClearConflict) — a duplicate import path,
not a second concern. ci.go committed .hanzo/workflows via that dead path only.

Delete all three (719 lines). github_import.go remains the one import path;
the git suite is green.
2026-07-25 22:13:18 -07:00
e4dbf48639 feat(deploy): GET /v1/deploy/gitops — the CD plane read, distinct from the workload board
Every other read in clients/deploy projects operator App CRs: one row per
workload, declared vs running tag. A CD Application (apps.hanzo.ai/v1alpha1) is
the layer ABOVE that — the git source CD polls, the commit it last applied, the
deploys it performed.

They disagree in exactly the case an operator most needs to see: main carries a
new image pin, CD has not applied that commit yet, so every App CR still declares
the old tag and the drift board reads "Synced" while the deploy is stuck. Reading
the CD plane makes that visible instead of invisible.

k8s.CDApplications is the shared coordinate (clients/k8s), not a private copy.
Build + clients/deploy tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 17:37:14 -07:00
a1bf850c96 feat(deploy): GET /v1/deploy/gitops — the CD plane read, distinct from the workload board
Every other read in clients/deploy projects operator App CRs: one row per
workload, declared vs running tag. A CD Application (apps.hanzo.ai/v1alpha1) is
the layer ABOVE that — the git source CD polls, the commit it last applied, the
deploys it performed.

They disagree in exactly the case an operator most needs to see: main carries a
new image pin, CD has not applied that commit yet, so every App CR still declares
the old tag and the drift board reads "Synced" while the deploy is stuck. Reading
the CD plane makes that visible instead of invisible.

k8s.CDApplications is the shared coordinate (clients/k8s), not a private copy.
Build + clients/deploy tests green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 17:37:14 -07:00
21f01caef8 chore(sync): delete the per-repo sync.yml — superseded by the org webhook
A single GitHub ORG-level push webhook now posts to git.hanzo.ai/v1/sync for every
repo in the org (verified: real pushes to universe + gateway synced their mirrors
instantly), so a per-repo nudge is redundant.

It was also mostly theatre: HANZO_GIT_TOKEN exists on exactly ONE repo in the org
(hanzoai/app) and is not an org secret, so in every other repo this workflow hit its
fail-soft branch and did nothing while still reporting success.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 17:37:14 -07:00
f0931cab5f chore(sync): delete the per-repo sync.yml — superseded by the org webhook
A single GitHub ORG-level push webhook now posts to git.hanzo.ai/v1/sync for every
repo in the org (verified: real pushes to universe + gateway synced their mirrors
instantly), so a per-repo nudge is redundant.

It was also mostly theatre: HANZO_GIT_TOKEN exists on exactly ONE repo in the org
(hanzoai/app) and is not an org secret, so in every other repo this workflow hit its
fail-soft branch and did nothing while still reporting success.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 17:37:14 -07:00
antje 928615f919 feat(entitlements): standing, git import, principal wallet
Lands the in-flight work alongside the two auth/royalty commits already on
main: an entitlements standing check with its tests, the git CI + repo-import
clients, and the principal wallet. Builds clean (go build ./... exit 0).
2026-07-25 15:35:07 -07:00
antje fbe41ec92b feat(entitlements): standing, git import, principal wallet
Lands the in-flight work alongside the two auth/royalty commits already on
main: an entitlements standing check with its tests, the git CI + repo-import
clients, and the principal wallet. Builds clean (go build ./... exit 0).
2026-07-25 15:35:07 -07:00
antje f1761f12fc cloud auth: exclude ALL machine principals from SuperAdmin (red M1 fix)
Follow-up to the audience decomplection (6f371409c). Red found that once the aud is
no longer a gate, owner==adminOrg becomes the sole key to cloud SuperAdmin, reachable
by ANY admin-org client_credentials app — and the existing exclusion
(isKMSMachinePrincipal) only caught the one -platform-kms audience. A generic admin-org
machine app (or IAM's blank-Organization→adminOrg default) would inherit platform-admin.

Fix: idClaims now parses IAM's `type` claim; isMachinePrincipal(claims) =
type=="application" OR the owner-bound KMS-sync audience. The SuperAdmin grant and the
X-User-IsOrgAdmin mint both gate on !isMachinePrincipal. type=="application" catches
EVERY machine app (object/token_oauth.go stamps it); the KMS-aud union is the
defensive fallback for a token missing `type`. Fail-closed: unknown/empty type is
treated as NON-machine so a real human admin (owner==adminOrg) is never locked out —
human admin behavior is UNCHANGED (a human admin token, any audience, still SuperAdmin).

Proven: new TestSanitizeIdentity case 'admin-org application-type token is denied
SuperAdmin (M1)' passes; the human 'arbitrary audience still gets SuperAdmin' case and
both machine-denial cases pass; the full clients/kms red machine-token suite
(AdminSlip, ForeignMachineAud, MultiValueAud, AdminOrgMachineToken, PaaSSyncEndToEnd)
green. Also scrubs the dead jwtAudiencesFromEnv reference in clients/deploy/login.go
(L2). HELD (unpushed) pending the IAM cutover settling + red re-review.
2026-07-25 15:22:36 -07:00
antje f50f29a8bb cloud auth: exclude ALL machine principals from SuperAdmin (red M1 fix)
Follow-up to the audience decomplection (f429d7228). Red found that once the aud is
no longer a gate, owner==adminOrg becomes the sole key to cloud SuperAdmin, reachable
by ANY admin-org client_credentials app — and the existing exclusion
(isKMSMachinePrincipal) only caught the one -platform-kms audience. A generic admin-org
machine app (or IAM's blank-Organization→adminOrg default) would inherit platform-admin.

Fix: idClaims now parses IAM's `type` claim; isMachinePrincipal(claims) =
type=="application" OR the owner-bound KMS-sync audience. The SuperAdmin grant and the
X-User-IsOrgAdmin mint both gate on !isMachinePrincipal. type=="application" catches
EVERY machine app (object/token_oauth.go stamps it); the KMS-aud union is the
defensive fallback for a token missing `type`. Fail-closed: unknown/empty type is
treated as NON-machine so a real human admin (owner==adminOrg) is never locked out —
human admin behavior is UNCHANGED (a human admin token, any audience, still SuperAdmin).

Proven: new TestSanitizeIdentity case 'admin-org application-type token is denied
SuperAdmin (M1)' passes; the human 'arbitrary audience still gets SuperAdmin' case and
both machine-denial cases pass; the full clients/kms red machine-token suite
(AdminSlip, ForeignMachineAud, MultiValueAud, AdminOrgMachineToken, PaaSSyncEndToEnd)
green. Also scrubs the dead jwtAudiencesFromEnv reference in clients/deploy/login.go
(L2). HELD (unpushed) pending the IAM cutover settling + red re-review.
2026-07-25 15:22:36 -07:00
zeekayandClaude Opus 5 c4623f8dd2 ci(cloud): CI gates, it does not deploy — drop the builder that could not build
The `deploy` job I carried over from .hanzo/workflows/deploy.yml claimed to
build the cloud image and roll it out, and could do neither. Measured, not
assumed:

- `buildctl-daemonless.sh` is not in the image this fleet serves for
  `hanzo-build-linux-amd64` — every label in that pool maps to
  catthehacker/ubuntu:act-24.04 (universe:infra/k8s/git-runner/statefulset.yaml).
- Its `secrets.GIT_CLONE_TOKEN` exists on neither the repo nor the org; the
  forge org carries GH_PAT, GHCR_USER, GHCR_TOKEN and nothing else.
- `kubectl patch app` is undone on the next poll by cd.hanzo.ai's selfHeal,
  which restores the CR from the universe pin.

Fixing it would have been worse than deleting it: the cloud image and its v*
tags already have exactly ONE owner, clients/platform/release.go (POST
/v1/runner {release:true}). A second builder on the same commit is the
double-build hanzo.yml's images: block already refuses by name. Rollout is,
and stays, a reviewed tag pin in hanzoai/universe.

So the native pipeline is what CI is for: gate + containment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 15:21:51 -07:00
zeekayandhanzo-dev cd5cbd9345 ci(cloud): CI gates, it does not deploy — drop the builder that could not build
The `deploy` job I carried over from .hanzo/workflows/deploy.yml claimed to
build the cloud image and roll it out, and could do neither. Measured, not
assumed:

- `buildctl-daemonless.sh` is not in the image this fleet serves for
  `hanzo-build-linux-amd64` — every label in that pool maps to
  catthehacker/ubuntu:act-24.04 (universe:infra/k8s/git-runner/statefulset.yaml).
- Its `secrets.GIT_CLONE_TOKEN` exists on neither the repo nor the org; the
  forge org carries GH_PAT, GHCR_USER, GHCR_TOKEN and nothing else.
- `kubectl patch app` is undone on the next poll by cd.hanzo.ai's selfHeal,
  which restores the CR from the universe pin.

Fixing it would have been worse than deleting it: the cloud image and its v*
tags already have exactly ONE owner, clients/platform/release.go (POST
/v1/runner {release:true}). A second builder on the same commit is the
double-build hanzo.yml's images: block already refuses by name. Rollout is,
and stays, a reviewed tag pin in hanzoai/universe.

So the native pipeline is what CI is for: gate + containment.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 15:21:51 -07:00
zeekayandantje 31559562b5 auth: two key families and OAuth2, not six shapes
The API recognised six credential shapes — pk-, sk-, hk-, pk_, fw_, hz_.
Two of them, fw_ and hz_, were listed as API-key prefixes and minted by
nothing at all: dead entries that widened what counts as a credential
for no reason. Removed from both the authority and admission's mirror,
and from the redaction regex that enumerated the same set.

Note fw_ is still the framework module's TABLE prefix (fw_doctypes,
fw_documents, fw_roles) — unrelated to keys, untouched.

isAPIKey now reads the one list instead of repeating it as a chain of
prefix tests, so the authority is a value rather than a shape spread
across two functions.

hk- stays accepted for now, deliberately: IAM mints it (mintKey delegates
to iam.mintUserKey — cloud only validates), so dropping it here before
IAM renames the family would reject every key IAM hands out. The order is
IAM mints sk-, holders re-key, then the entry goes.
2026-07-25 15:19:04 -07:00
zeekayandantje 796ba9df68 auth: two key families and OAuth2, not six shapes
The API recognised six credential shapes — pk-, sk-, hk-, pk_, fw_, hz_.
Two of them, fw_ and hz_, were listed as API-key prefixes and minted by
nothing at all: dead entries that widened what counts as a credential
for no reason. Removed from both the authority and admission's mirror,
and from the redaction regex that enumerated the same set.

Note fw_ is still the framework module's TABLE prefix (fw_doctypes,
fw_documents, fw_roles) — unrelated to keys, untouched.

isAPIKey now reads the one list instead of repeating it as a chain of
prefix tests, so the authority is a value rather than a shape spread
across two functions.

hk- stays accepted for now, deliberately: IAM mints it (mintKey delegates
to iam.mintUserKey — cloud only validates), so dropping it here before
IAM renames the family would reject every key IAM hands out. The order is
IAM mints sk-, holders re-key, then the entry goes.
2026-07-25 15:19:04 -07:00
zeekayandantje f016e6710a feat(authors): /v1/authors/basis — an author can audit their own royalty
An author saw the AMOUNT and never the BASIS. GET /v1/authors/basis serves the
arithmetic behind the number: the share as applied, the CURRENT cost model (rate
card + class sizing, read from clients/blueprint), the immutable author_ledger
rows verbatim, the deploy edges that made each deploying org count, and a
reconciliation of ledger to balance.

The value is captured at accrual and already stored — LatchAccrual writes
share_bps/spend_cents/earning_cents in the SAME transaction as the balance
increment — so a row explains itself forever and is served verbatim, never
recomputed. The rate card is deliberately NOT stamped per row: spend_cents is
commerce's all-provider rollup, priced hour-by-hour across the month, so a single
per-row card would be a fabrication by construction. It is served separately and
labelled current via asOf. Zero schema change, zero migration.

- compute_proof is served exactly as stored: null when absent, never a hash, a
  txn id, or a word standing in for an attestation that does not exist
- PURE READ: never sweeps, accrues or pays, unlike GET /v1/authors which accrues
  lazily — an audit must not move the money it is auditing
- org-scoped from the principal; no id/org accepted from URL, query or body; an
  org with no author record gets {isAuthor:false}, never a 404 that leaks
- a row keeps the share AS APPLIED while the top level shows the current one, and
  shareSource distinguishes a negotiated override from the platform default
- reconciliation is account-wide (LedgerTotals), never the returned window, and
  window.truncated says so when rows were cut
- /v1/admin/authors/:id/basis mirrors the same builder for support
- blueprint gains Rates()/Sizing() so authors never re-declares a price or a
  footprint; the cost model keeps one owner
- authors tests gain the repo's standard TestMain dev-key shim (as in the root
  package) so the suite runs on an encryption-capable build
2026-07-25 15:19:04 -07:00
zeekayandantje 0b46a4ff8d feat(authors): /v1/authors/basis — an author can audit their own royalty
An author saw the AMOUNT and never the BASIS. GET /v1/authors/basis serves the
arithmetic behind the number: the share as applied, the CURRENT cost model (rate
card + class sizing, read from clients/blueprint), the immutable author_ledger
rows verbatim, the deploy edges that made each deploying org count, and a
reconciliation of ledger to balance.

The value is captured at accrual and already stored — LatchAccrual writes
share_bps/spend_cents/earning_cents in the SAME transaction as the balance
increment — so a row explains itself forever and is served verbatim, never
recomputed. The rate card is deliberately NOT stamped per row: spend_cents is
commerce's all-provider rollup, priced hour-by-hour across the month, so a single
per-row card would be a fabrication by construction. It is served separately and
labelled current via asOf. Zero schema change, zero migration.

- compute_proof is served exactly as stored: null when absent, never a hash, a
  txn id, or a word standing in for an attestation that does not exist
- PURE READ: never sweeps, accrues or pays, unlike GET /v1/authors which accrues
  lazily — an audit must not move the money it is auditing
- org-scoped from the principal; no id/org accepted from URL, query or body; an
  org with no author record gets {isAuthor:false}, never a 404 that leaks
- a row keeps the share AS APPLIED while the top level shows the current one, and
  shareSource distinguishes a negotiated override from the platform default
- reconciliation is account-wide (LedgerTotals), never the returned window, and
  window.truncated says so when rows were cut
- /v1/admin/authors/:id/basis mirrors the same builder for support
- blueprint gains Rates()/Sizing() so authors never re-declares a price or a
  footprint; the cost model keeps one owner
- authors tests gain the repo's standard TestMain dev-key shim (as in the root
  package) so the suite runs on an encryption-capable build
2026-07-25 15:19:04 -07:00
zeekayandClaude Opus 5 ac92d8ee2c deps: ai v1.831.3 — hk- key resolution over client_secret_basic
Unblocks customer API keys. IAM gates get-user?accessKey= on a confidential-APP
principal (p.App != "" holding CapKeyResolve), and it derives p.App ONLY from
`Authorization: Basic <clientId>:<clientSecret>`. ai v1.831.2 sent a
client_credentials bearer, whose principal is org-scoped with an EMPTY App, so
every hk- key got 401 "API key validation failed: IAM error: auth:Unauthorized
operation". v1.831.3 sends Basic — probed live against prod IAM v1.33.8 with
the real hanzo-cloud credential: Basic → 200 ok, bearer → auth:Unauthorized
operation, query-params → 401.

Also collapses the balance gate's duplicate resolver onto the same one, so the
prepaid gate stops silently fail-opening on hk- keys.

The application row needed no change: it is already admin/hanzo-cloud, and
admin IS a reserved signing owner, so the owner-pin was never the blocker.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 15:16:59 -07:00
zeekayandhanzo-dev b5192af1e1 deps: ai v1.831.3 — hk- key resolution over client_secret_basic
Unblocks customer API keys. IAM gates get-user?accessKey= on a confidential-APP
principal (p.App != "" holding CapKeyResolve), and it derives p.App ONLY from
`Authorization: Basic <clientId>:<clientSecret>`. ai v1.831.2 sent a
client_credentials bearer, whose principal is org-scoped with an EMPTY App, so
every hk- key got 401 "API key validation failed: IAM error: auth:Unauthorized
operation". v1.831.3 sends Basic — probed live against prod IAM v1.33.8 with
the real hanzo-cloud credential: Basic → 200 ok, bearer → auth:Unauthorized
operation, query-params → 401.

Also collapses the balance gate's duplicate resolver onto the same one, so the
prepaid gate stops silently fail-opening on hk- keys.

The application row needed no change: it is already admin/hanzo-cloud, and
admin IS a reserved signing owner, so the owner-pin was never the blocker.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 15:16:59 -07:00
antje 27ec4aba9b fix(build): pin the prebuilt inputs — a cloud image was not reproducible
`ARG CONSOLE_IMAGE/SKILLS_IMAGE/FLAGS_IMAGE` all defaulted to `:latest`, and those
defaults are LOAD-BEARING: the builder that actually ships our releases is the native
one (POST /v1/runner -> launchDirectBuild -> BuildKit) and it passes no --build-arg.
`release.yml`, which the old comment said would "resolve CONSOLE_IMAGE to a fresh
digest", is a stub and resolves nothing. So the console baked into a cloud image was
decided by WHEN the build ran, not by what we shipped — build the same cloud sha twice,
get two different products.

It bit today: cloud v1.801.215 was built ~12 minutes before console CI finished
publishing the console-embed carrying v8.5.26, so a release whose entire purpose was
that console change silently baked the previous console and went green. On the live
site `document.fonts.size` stayed 0 and every customer kept reading the product in a
fallback face. Nothing failed; the version number just stopped meaning anything.

Pinned to the immutable per-commit tags each input's own CI publishes. A console,
skills or flags change now reaches production through a reviewable cloud commit that
names it — which is the same "floating tags never reach a cluster" rule we already
apply to cluster pins, applied one layer down where it was quietly missing.
2026-07-25 15:02:21 -07:00
antje e12a781599 fix(build): pin the prebuilt inputs — a cloud image was not reproducible
`ARG CONSOLE_IMAGE/SKILLS_IMAGE/FLAGS_IMAGE` all defaulted to `:latest`, and those
defaults are LOAD-BEARING: the builder that actually ships our releases is the native
one (POST /v1/runner -> launchDirectBuild -> BuildKit) and it passes no --build-arg.
`release.yml`, which the old comment said would "resolve CONSOLE_IMAGE to a fresh
digest", is a stub and resolves nothing. So the console baked into a cloud image was
decided by WHEN the build ran, not by what we shipped — build the same cloud sha twice,
get two different products.

It bit today: cloud v1.801.215 was built ~12 minutes before console CI finished
publishing the console-embed carrying v8.5.26, so a release whose entire purpose was
that console change silently baked the previous console and went green. On the live
site `document.fonts.size` stayed 0 and every customer kept reading the product in a
fallback face. Nothing failed; the version number just stopped meaning anything.

Pinned to the immutable per-commit tags each input's own CI publishes. A console,
skills or flags change now reaches production through a reviewable cloud commit that
names it — which is the same "floating tags never reach a cluster" rule we already
apply to cluster pins, applied one layer down where it was quietly missing.
2026-07-25 15:02:21 -07:00
zeekayandClaude Opus 5 0f12f8f587 ci: one native pipeline — the gates now gate the deploy
.github/workflows holds exactly one file: the sync nudge, which runs zero CI.
The three pipelines that used to run beside each other are one graph in
.hanzo/workflows/cicd.yml:

  .github/workflows/cicd.yml       -> job `gate`         (unchanged caller;
      every real detail still lives in hanzo.yml, which platform.hanzo.ai
      reads too, so no build logic moved)
  .github/workflows/containment.yml-> job `containment`  (verbatim; only its
      own self-exclusion path follows the file)
  .hanzo/workflows/deploy.yml      -> job `deploy`, needs: [gate, containment]

That needs: edge is the whole point. deploy.yml fired on `push: main`
independently of both gates, so a red suite or a containment breach still cut
an image and patched three operator apps — the gates could only report a fact
the deploy had already ignored. One graph is the only way one job blocks
another.

Two defects fixed on the way:

- deploy.yml declared no `workflow_dispatch`, but sync-from-github.yml
  dispatches the pipeline by name after every fast-forward (a push made with
  the workflow token triggers nothing). That curl could only 404, so synced
  commits built nothing. cicd.yml declares the trigger; the sync now names it.
- hanzo.yml still credited a release.yml that was deleted. The v* tags and the
  main image are owned by clients/platform/release.go (POST /v1/runner
  {release:true}) — the in-cloud port of that workflow.

Not yet live: hanzoai/cloud on git.hanzo.ai is still a Gitea PULL MIRROR with
Actions off (measured: mirror=true, has_actions=false), so nothing under
.hanzo/workflows runs there until it is converted to canonical
(hanzoai/.github: scripts/forge-migrate.sh convert).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:57:57 -07:00
zeekayandhanzo-dev b6042582b8 ci: one native pipeline — the gates now gate the deploy
.github/workflows holds exactly one file: the sync nudge, which runs zero CI.
The three pipelines that used to run beside each other are one graph in
.hanzo/workflows/cicd.yml:

  .github/workflows/cicd.yml       -> job `gate`         (unchanged caller;
      every real detail still lives in hanzo.yml, which platform.hanzo.ai
      reads too, so no build logic moved)
  .github/workflows/containment.yml-> job `containment`  (verbatim; only its
      own self-exclusion path follows the file)
  .hanzo/workflows/deploy.yml      -> job `deploy`, needs: [gate, containment]

That needs: edge is the whole point. deploy.yml fired on `push: main`
independently of both gates, so a red suite or a containment breach still cut
an image and patched three operator apps — the gates could only report a fact
the deploy had already ignored. One graph is the only way one job blocks
another.

Two defects fixed on the way:

- deploy.yml declared no `workflow_dispatch`, but sync-from-github.yml
  dispatches the pipeline by name after every fast-forward (a push made with
  the workflow token triggers nothing). That curl could only 404, so synced
  commits built nothing. cicd.yml declares the trigger; the sync now names it.
- hanzo.yml still credited a release.yml that was deleted. The v* tags and the
  main image are owned by clients/platform/release.go (POST /v1/runner
  {release:true}) — the in-cloud port of that workflow.

Not yet live: hanzoai/cloud on git.hanzo.ai is still a Gitea PULL MIRROR with
Actions off (measured: mirror=true, has_actions=false), so nothing under
.hanzo/workflows runs there until it is converted to canonical
(hanzoai/.github: scripts/forge-migrate.sh convert).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 14:57:57 -07:00
zeekayandClaude Opus 5 c056a1d75d o11y(vmproxy): admit the absolute fleet signals and live alert state
The Lux board's allowlist described the fleet only in relative terms — who is up,
who is at what height, who has peers. A fleet that has stopped agrees with itself
perfectly on all of it: same height, same hash, five up, four peers each. The board
paints a full row of green over a frozen chain, which is how two days of freeze
looked like a healthy network.

Four series that cannot be satisfied by agreement:

  lux_network_c_tip_age_seconds     wall clock minus the freshest tip's own
                                    timestamp — climbs while the chain sits still
  lux_network_block_height_spread   a lagging or wedged validator
  lux_network_tip_hash_variants     distinct hashes at one height; >1 is a fork
  lux_network_ready_but_rpc_dead    pods reporting Ready while their RPC is dead

And the firing alert set itself, aggregated by name/network/severity. vmalert
already remote-writes its state into the same VictoriaMetrics this proxy reads, so
the board can show what is ACTUALLY firing over the path that already exists — no
second data path, no second ingress, no second access boundary to get wrong, and
no UI re-derivation of the rules that could quietly disagree with them.

The aggregation is deliberate: alertname/network/severity and nothing finer, so the
series stays small and no instance-level detail crosses the boundary.

Contract unchanged otherwise — exact-string allowlist, SuperAdmin-only, fail-closed.
All five allowlist and gate tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:48:15 -07:00
zeekayandhanzo-dev 249efbee86 o11y(vmproxy): admit the absolute fleet signals and live alert state
The Lux board's allowlist described the fleet only in relative terms — who is up,
who is at what height, who has peers. A fleet that has stopped agrees with itself
perfectly on all of it: same height, same hash, five up, four peers each. The board
paints a full row of green over a frozen chain, which is how two days of freeze
looked like a healthy network.

Four series that cannot be satisfied by agreement:

  lux_network_c_tip_age_seconds     wall clock minus the freshest tip's own
                                    timestamp — climbs while the chain sits still
  lux_network_block_height_spread   a lagging or wedged validator
  lux_network_tip_hash_variants     distinct hashes at one height; >1 is a fork
  lux_network_ready_but_rpc_dead    pods reporting Ready while their RPC is dead

And the firing alert set itself, aggregated by name/network/severity. vmalert
already remote-writes its state into the same VictoriaMetrics this proxy reads, so
the board can show what is ACTUALLY firing over the path that already exists — no
second data path, no second ingress, no second access boundary to get wrong, and
no UI re-derivation of the rules that could quietly disagree with them.

The aggregation is deliberate: alertname/network/severity and nothing finer, so the
series stays small and no instance-level detail crosses the boundary.

Contract unchanged otherwise — exact-string allowlist, SuperAdmin-only, fail-closed.
All five allowlist and gate tests pass.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 14:48:15 -07:00
antje e1ca02a11c fix(release): a partial tag list has no maximum — ask the registry, and fail if it can't answer
The release computed a version BELOW one already published, aimed at overwriting it.
Measured today: `POST /v1/runner {release:true}` on a tree whose live image was
v1.801.213 answered `image: ghcr.io/hanzoai/cloud:v1.801.210`.

Three defects, each fixed in one place:

1. The published-tag read asked GitHub's package-metadata API, which needs a
   `read:packages` scope our PAT does not carry. It answered 403.

2. That failure was logged as a warning and swallowed, so the version was computed
   from git tags alone. Git tags had stalled at v1.801.209 while the registry had
   reached v1.801.213 — hence 210. Folding in the published tags IS the phantom-tag
   prevention the doc comment promises; with it silently disabled, the promise was
   not kept. A partial view has no maximum, so the compute now FAILS rather than
   guesses. Fail-closed, no silent cap.

3. githubJSON decoded the body regardless of status. GitHub answers a non-2xx with
   an error object, never the success shape, so decoding it produced a bogus
   unmarshal error that MASKED the 403 — which is why this read as a decode bug and
   survived four releases. It now decodes only a 2xx and returns the status, so every
   caller's existing `if code != …` policy is reachable for the first time. The
   422-collision and 204 policies are untouched (those callers pass out=nil).

The registry, not GitHub's metadata API, is now the authority on which tags exist:
it is what `docker pull` resolves against and what an overwrite would clobber, and
its pull scope is anonymous — one less credential in the path. Pagination is
mandatory, not incidental: GHCR caps a page at 1000 tags in insertion order, so the
newest versions are on the LAST page; reading page one alone reports a stale max, the
same unsound answer by another route. Exceeding the page bound is an error, never a
truncated list.

Tests: the regression test fails on the old behaviour with the exact signature
("compute returned 1.786.43 — a version below a pushed image") and passes on the fix;
pagination proves the last page's max wins; githubJSON proves 403 surfaces as 403.
2026-07-25 13:59:35 -07:00
antje 69c5fca283 fix(release): a partial tag list has no maximum — ask the registry, and fail if it can't answer
The release computed a version BELOW one already published, aimed at overwriting it.
Measured today: `POST /v1/runner {release:true}` on a tree whose live image was
v1.801.213 answered `image: ghcr.io/hanzoai/cloud:v1.801.210`.

Three defects, each fixed in one place:

1. The published-tag read asked GitHub's package-metadata API, which needs a
   `read:packages` scope our PAT does not carry. It answered 403.

2. That failure was logged as a warning and swallowed, so the version was computed
   from git tags alone. Git tags had stalled at v1.801.209 while the registry had
   reached v1.801.213 — hence 210. Folding in the published tags IS the phantom-tag
   prevention the doc comment promises; with it silently disabled, the promise was
   not kept. A partial view has no maximum, so the compute now FAILS rather than
   guesses. Fail-closed, no silent cap.

3. githubJSON decoded the body regardless of status. GitHub answers a non-2xx with
   an error object, never the success shape, so decoding it produced a bogus
   unmarshal error that MASKED the 403 — which is why this read as a decode bug and
   survived four releases. It now decodes only a 2xx and returns the status, so every
   caller's existing `if code != …` policy is reachable for the first time. The
   422-collision and 204 policies are untouched (those callers pass out=nil).

The registry, not GitHub's metadata API, is now the authority on which tags exist:
it is what `docker pull` resolves against and what an overwrite would clobber, and
its pull scope is anonymous — one less credential in the path. Pagination is
mandatory, not incidental: GHCR caps a page at 1000 tags in insertion order, so the
newest versions are on the LAST page; reading page one alone reports a stale max, the
same unsound answer by another route. Exceeding the page bound is an error, never a
truncated list.

Tests: the regression test fails on the old behaviour with the exact signature
("compute returned 1.786.43 — a version below a pushed image") and passes on the fix;
pagination proves the last page's max wins; githubJSON proves 403 surfaces as 403.
2026-07-25 13:59:35 -07:00
zeekayandClaude Opus 5 513c1a357d fix(sync): the nudge runs on OUR pool — GitHub-hosted is billing-blocked
The first Sync to Hanzo Git run failed before executing a step: "The job was
not started because recent account payments have failed or your spending limit
needs to be increased." GitHub-hosted minutes are refused for this org, so a
nudge on ubuntu-latest is a coin flip on an invoice. Our ARC pool is free,
in-cluster, and demonstrably running this org's CI right now — and the nudge is
one bounded curl.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 13:46:51 -07:00
zeekayandhanzo-dev 0c805b9367 fix(sync): the nudge runs on OUR pool — GitHub-hosted is billing-blocked
The first Sync to Hanzo Git run failed before executing a step: "The job was
not started because recent account payments have failed or your spending limit
needs to be increased." GitHub-hosted minutes are refused for this org, so a
nudge on ubuntu-latest is a coin flip on an invoice. Our ARC pool is free,
in-cluster, and demonstrably running this org's CI right now — and the nudge is
one bounded curl.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 13:46:51 -07:00
zandClaude Opus 4.8 0975379d09 refactor(k8s): one declaration per cluster coordinate, six copies deleted
The App CR coordinate was declared three times under two names (paas appsGVR,
deploy appsCRGVR, platform appsGVR); Deployments twice; Namespaces three times
(platform namespacesGVR, provisioning nsGVR, ml nsGVR). Same value, six extra
places to disagree.

That is not hypothetical here. The whole point of the App/Service kind migration
is that this coordinate CHANGES, and a subsystem holding a private copy silently
reads the wrong resource and reports an honest-looking empty board — which paas
already shipped once (it scanned services while the fleet had moved to apps).

clients/k8s now holds Apps, Deployments and Namespaces; 18 files reference them.
There is deliberately no Services coordinate: the fleet holds zero of that kind,
so offering it would invite the same bug back.

Build + vet clean across paas/deploy/platform/provisioning/ml/k8s; paas and apps
tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 13:46:31 -07:00
zandhanzo-dev 6158f6b111 refactor(k8s): one declaration per cluster coordinate, six copies deleted
The App CR coordinate was declared three times under two names (paas appsGVR,
deploy appsCRGVR, platform appsGVR); Deployments twice; Namespaces three times
(platform namespacesGVR, provisioning nsGVR, ml nsGVR). Same value, six extra
places to disagree.

That is not hypothetical here. The whole point of the App/Service kind migration
is that this coordinate CHANGES, and a subsystem holding a private copy silently
reads the wrong resource and reports an honest-looking empty board — which paas
already shipped once (it scanned services while the fleet had moved to apps).

clients/k8s now holds Apps, Deployments and Namespaces; 18 files reference them.
There is deliberately no Services coordinate: the fleet holds zero of that kind,
so offering it would invite the same bug back.

Build + vet clean across paas/deploy/platform/provisioning/ml/k8s; paas and apps
tests green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 13:46:31 -07:00
zeekayandClaude Opus 5 70235a59b6 ci: wire cloud's sync, and say plainly why GitHub still runs two gates
Adds the two halves of the sync this repo never had, and deletes the tombstone:

  .github/workflows/sync.yml            nudge git.hanzo.ai on a GitHub push
  .hanzo/workflows/sync-from-github.yml 10-min fast-forward, fail-loud
  .github/workflows/release.yml         DELETED — since 4e6ec52a it existed
                                        only to echo "the native pipeline is
                                        deploy.yml". A workflow whose whole job
                                        is to announce that it is not the
                                        pipeline is the tombstone the law
                                        abolishes; sync.yml's header carries
                                        that sentence now.

cicd.yml and containment.yml STAY, and the reason is measured, not deferred:
against the live forge, hanzoai/cloud reports `mirror: true` and
`has_actions: false` — it is a Gitea pull mirror with Actions disabled, and
`/api/v1/repos/hanzoai/cloud/actions/tasks` returns ZERO runs, so
.hanzo/workflows/deploy.yml has never executed. Moving the GitHub gates now
would not relocate CI, it would end it: cicd.yml is the only running test gate
plus the cloud-flags image, and containment.yml is a SECURITY control (no
release binary may link clients/controlplane or spoof testing.Testing()).
Compare hanzoai/app, which reports `mirror: false, has_actions: true` and whose
native ff-main run succeeds every 10 minutes — that is what this repo needs.

The unblock is one act on the forge, not a code change: convert hanzoai/cloud
from pull mirror to canonical, which is also what enables Actions
(hanzoai/.github scripts/forge-migrate.sh convert --repo hanzoai/cloud). The
files are staged so the flip is the only remaining step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 13:40:30 -07:00
zeekayandhanzo-dev f0a1d510bf ci: wire cloud's sync, and say plainly why GitHub still runs two gates
Adds the two halves of the sync this repo never had, and deletes the tombstone:

  .github/workflows/sync.yml            nudge git.hanzo.ai on a GitHub push
  .hanzo/workflows/sync-from-github.yml 10-min fast-forward, fail-loud
  .github/workflows/release.yml         DELETED — since a877eda2 it existed
                                        only to echo "the native pipeline is
                                        deploy.yml". A workflow whose whole job
                                        is to announce that it is not the
                                        pipeline is the tombstone the law
                                        abolishes; sync.yml's header carries
                                        that sentence now.

cicd.yml and containment.yml STAY, and the reason is measured, not deferred:
against the live forge, hanzoai/cloud reports `mirror: true` and
`has_actions: false` — it is a Gitea pull mirror with Actions disabled, and
`/api/v1/repos/hanzoai/cloud/actions/tasks` returns ZERO runs, so
.hanzo/workflows/deploy.yml has never executed. Moving the GitHub gates now
would not relocate CI, it would end it: cicd.yml is the only running test gate
plus the cloud-flags image, and containment.yml is a SECURITY control (no
release binary may link clients/controlplane or spoof testing.Testing()).
Compare hanzoai/app, which reports `mirror: false, has_actions: true` and whose
native ff-main run succeeds every 10 minutes — that is what this repo needs.

The unblock is one act on the forge, not a code change: convert hanzoai/cloud
from pull mirror to canonical, which is also what enables Actions
(hanzoai/.github scripts/forge-migrate.sh convert --repo hanzoai/cloud). The
files are staged so the flip is the only remaining step.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 13:40:30 -07:00
antje 33ebd3931b ci(native): runs-on hanzo-build-linux-amd64 — the label runners declare
Runners declare [ubuntu-latest ubuntu-22.04 ubuntu-24.04
hanzo-build-linux-amd64]. 'hanzo-linux-amd64' is not among them, so this
workflow's jobs queue forever unclaimed — which is why this repo has never
produced a native build (actions/runs total_count was 0).
2026-07-25 13:27:48 -07:00
antje a4e4d551af ci(native): runs-on hanzo-build-linux-amd64 — the label runners declare
Runners declare [ubuntu-latest ubuntu-22.04 ubuntu-24.04
hanzo-build-linux-amd64]. 'hanzo-linux-amd64' is not among them, so this
workflow's jobs queue forever unclaimed — which is why this repo has never
produced a native build (actions/runs total_count was 0).
2026-07-25 13:27:48 -07:00
z ba1a184309 fix(auth): ai v1.831.2 — hk- API keys resolve via bearer (GA blocker)
Picks up the fix for: every customer hk- key 401'd at inference because
getUserByAccessKey authenticated to IAM with query-param clientId/clientSecret,
which yields no principal and so can never satisfy authz.CapKeyResolve.

Pairs with IAM_KEY_RESOLVE_APPS=hanzo-cloud on the cloud CR — the allowlist is
fail-secure, so both halves are required.
2026-07-25 13:06:33 -07:00
z e6d7c23237 fix(auth): ai v1.831.2 — hk- API keys resolve via bearer (GA blocker)
Picks up the fix for: every customer hk- key 401'd at inference because
getUserByAccessKey authenticated to IAM with query-param clientId/clientSecret,
which yields no principal and so can never satisfy authz.CapKeyResolve.

Pairs with IAM_KEY_RESOLVE_APPS=hanzo-cloud on the cloud CR — the allowlist is
fail-secure, so both halves are required.
2026-07-25 13:06:33 -07:00
zandClaude Opus 4.8 399f4d350a feat(paas): discover the scan set so tenant-<org> workloads are visible
The scan set was a literal list, so a tenant namespace was invisible however
correctly it classified. discoverNamespaces asks the cluster instead — listing
NAMESPACES (the honest question: 'which namespaces are ours?') rather than
deriving the set from a cluster-wide CR list, which would answer a different
question and pull every tenant's objects through this process to do it. No new
RBAC: the cloud SA already holds namespaces get,list.

Safety properties, both pinned by tests:
  * discovery only ADDS to the first-party set, so an empty or failed listing
    degrades to previous behavior instead of blanking the board. (Caught by
    TestDeploy_SuperAdmin_EnvSelectsNamespace: an early version returned only
    what it discovered, and a fake with no Namespace objects produced an empty
    scan set -> 404.)
  * nsClass still filters, so a namespace that is not ours can never enter the
    set however discovery goes.

Tenancy is unchanged and still computed BEFORE any List: scopedNamespaces
confines a non-super OrgAdmin to namespaces whose tenant equals their validated
org; a tenant-<org> namespace authorizes to that org.

The cache sits behind a POINTER because cloud.Service stores State by value — a
mutex in  is copied and guards nothing (go vet: 'assignment copies lock
value'). vet is clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 12:50:33 -07:00
zandhanzo-dev de5782e48b feat(paas): discover the scan set so tenant-<org> workloads are visible
The scan set was a literal list, so a tenant namespace was invisible however
correctly it classified. discoverNamespaces asks the cluster instead — listing
NAMESPACES (the honest question: 'which namespaces are ours?') rather than
deriving the set from a cluster-wide CR list, which would answer a different
question and pull every tenant's objects through this process to do it. No new
RBAC: the cloud SA already holds namespaces get,list.

Safety properties, both pinned by tests:
  * discovery only ADDS to the first-party set, so an empty or failed listing
    degrades to previous behavior instead of blanking the board. (Caught by
    TestDeploy_SuperAdmin_EnvSelectsNamespace: an early version returned only
    what it discovered, and a fake with no Namespace objects produced an empty
    scan set -> 404.)
  * nsClass still filters, so a namespace that is not ours can never enter the
    set however discovery goes.

Tenancy is unchanged and still computed BEFORE any List: scopedNamespaces
confines a non-super OrgAdmin to namespaces whose tenant equals their validated
org; a tenant-<org> namespace authorizes to that org.

The cache sits behind a POINTER because cloud.Service stores State by value — a
mutex in  is copied and guards nothing (go vet: 'assignment copies lock
value'). vet is clean.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 12:50:33 -07:00
zandClaude Opus 4.8 96ac62283a refactor(paas): one total namespace classifier; unhide hanzo-mainnet
Three places encoded one fact and had drifted apart: the nsEnv map (3 namespaces),
nsOrg's suffix-strip (knew only -devnet/-testnet), and scanOrder's literal list
(the same 3). A tenant-<org> namespace matched none of them, so it classified as
its OWN org and was never scanned — tenant workloads were invisible by
construction, and hanzo-mainnet's CR was invisible too.

nsClass(ns) -> (tenant, env, ok) is now the single source of that truth, and it is
TOTAL: every input is decided, an unrecognised namespace is classified OUT rather
than guessed at, so the reader still cannot reach beyond the platform tier. nsOrg,
envOf and nsForEnv are projections of it; the nsEnv map is deleted. There is no
second place left to disagree.

Tenancy is unchanged and still computed BEFORE any List: scopedNamespaces confines
a non-super OrgAdmin to namespaces whose tenant equals their validated org, so a
tenant-<org> namespace now authorizes to that org instead of to hanzo.

scanOrder gains hanzo-mainnet (first-party, was missing). Discovering tenant-*
needs the dynamic client threaded into the scan set — the follow-up; the
authorization axis it needs already exists.

Tests: scanOrder now asserts the invariant that matters (every scanned namespace
classifies to a real tenant, else it would render rows no OrgAdmin could be
confined to), plus a totality/confinement table incl. tenant-maxpower -> maxpower,
tenant- -> out, kube-system -> out, hanzo-evil -> out.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 12:28:07 -07:00
zandhanzo-dev c0b285183a refactor(paas): one total namespace classifier; unhide hanzo-mainnet
Three places encoded one fact and had drifted apart: the nsEnv map (3 namespaces),
nsOrg's suffix-strip (knew only -devnet/-testnet), and scanOrder's literal list
(the same 3). A tenant-<org> namespace matched none of them, so it classified as
its OWN org and was never scanned — tenant workloads were invisible by
construction, and hanzo-mainnet's CR was invisible too.

nsClass(ns) -> (tenant, env, ok) is now the single source of that truth, and it is
TOTAL: every input is decided, an unrecognised namespace is classified OUT rather
than guessed at, so the reader still cannot reach beyond the platform tier. nsOrg,
envOf and nsForEnv are projections of it; the nsEnv map is deleted. There is no
second place left to disagree.

Tenancy is unchanged and still computed BEFORE any List: scopedNamespaces confines
a non-super OrgAdmin to namespaces whose tenant equals their validated org, so a
tenant-<org> namespace now authorizes to that org instead of to hanzo.

scanOrder gains hanzo-mainnet (first-party, was missing). Discovering tenant-*
needs the dynamic client threaded into the scan set — the follow-up; the
authorization axis it needs already exists.

Tests: scanOrder now asserts the invariant that matters (every scanned namespace
classifies to a real tenant, else it would render rows no OrgAdmin could be
confined to), plus a totality/confinement table incl. tenant-maxpower -> maxpower,
tenant- -> out, kube-system -> out, hanzo-evil -> out.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 12:28:07 -07:00
853a2807f3 fix(apps): wire the orphaned platform + projects Shutdown (SIGTERM resource leak)
Both subsystems define Shutdown and neither was wired, so the serve layer never
called them: platform.Shutdown (clients/platform/platform.go:892) cancels the
build-reconciler goroutine and the compute meter and closes the store;
projects.Shutdown closes its store. On SIGTERM both were orphaned.

wire_test encoded the leak as expected behavior (hasShutdown: false) — it now
asserts the fix. platform.Shutdown/projects.Shutdown are func() error, so they
take the same ctxShutdown wrapper as wallets/x402/framework/content.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 11:53:35 -07:00
06029d645f fix(apps): wire the orphaned platform + projects Shutdown (SIGTERM resource leak)
Both subsystems define Shutdown and neither was wired, so the serve layer never
called them: platform.Shutdown (clients/platform/platform.go:892) cancels the
build-reconciler goroutine and the compute meter and closes the store;
projects.Shutdown closes its store. On SIGTERM both were orphaned.

wire_test encoded the leak as expected behavior (hasShutdown: false) — it now
asserts the fix. platform.Shutdown/projects.Shutdown are func() error, so they
take the same ctxShutdown wrapper as wallets/x402/framework/content.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 11:53:35 -07:00
hanzo-dev d48861d98c destinations(insights): the project token is ours (hi_), not an upstream phc_
The adapter documented its credential as 'e.g. phc_…' — an upstream brand that
does not exist in our fork. Hanzo Insights already issues Hanzo-branded tokens
(insights/models/utils.py: hi_ project, hix_ personal, his_ secret, hia_ oauth),
and capture's validate_token takes any ASCII<=64 string — it only REJECTS the
personal prefixes. So the insights sink needs a hi_ project token from our OWN
instance: one auth system, no upstream credential, nothing to invent.
2026-07-25 11:34:25 -07:00
hanzo-dev 75e5615b0b destinations(insights): the project token is ours (hi_), not an upstream phc_
The adapter documented its credential as 'e.g. phc_…' — an upstream brand that
does not exist in our fork. Hanzo Insights already issues Hanzo-branded tokens
(insights/models/utils.py: hi_ project, hix_ personal, his_ secret, hia_ oauth),
and capture's validate_token takes any ASCII<=64 string — it only REJECTS the
personal prefixes. So the insights sink needs a hi_ project token from our OWN
instance: one auth system, no upstream credential, nothing to invent.
2026-07-25 11:34:25 -07:00
antje f515148e1d refactor(o11y): rip the CLOUD_OTLP_INGEST_ENABLED gate — capability, not a flag
The embedded OTLP ingest was gated behind a boolean env on top of its real
precondition (a datastore DSN). That extra switch is what let the fleet's ONLY
ingest path be silently turned off: the standalone collector it was meant to
fall back to is retired, all 24 otel-agents target cloud:4318, and every one of
them sat in connection-refused / queue-full / 'Rejecting data' — logs from every
service discarded, with nothing surfacing the loss.

Ingest now runs exactly when a DSN is configured, because the DSN is what it
writes to. One condition, derived from actual capability. Fail-soft on
construction error is unchanged, so a bad telemetry config still cannot take
cloud down. Dropped TestIngestEnabled with the function it covered.
2026-07-25 11:09:43 -07:00
antje ffaa319c6e refactor(o11y): rip the CLOUD_OTLP_INGEST_ENABLED gate — capability, not a flag
The embedded OTLP ingest was gated behind a boolean env on top of its real
precondition (a datastore DSN). That extra switch is what let the fleet's ONLY
ingest path be silently turned off: the standalone collector it was meant to
fall back to is retired, all 24 otel-agents target cloud:4318, and every one of
them sat in connection-refused / queue-full / 'Rejecting data' — logs from every
service discarded, with nothing surfacing the loss.

Ingest now runs exactly when a DSN is configured, because the DSN is what it
writes to. One condition, derived from actual capability. Fail-soft on
construction error is unchanged, so a bad telemetry config still cannot take
cloud down. Dropped TestIngestEnabled with the function it covered.
2026-07-25 11:09:43 -07:00
hanzo-dev 1d0a984ca4 destinations: our first-party sinks are Hanzo-branded; upstream names live only in NOTICE
The two FIRST-PARTY sinks leaked their upstream fork names into the public API:
Name() returned 'Hanzo Analytics (Umami)' / 'Hanzo Insights (PostHog)' and the
platform IDs — which are the API path (/v1/destinations/<id>), the DB platform
column, and the KMS secretsPath — were literally 'umami' / 'posthog'. These are
OUR products, so they are now named for Hanzo, one way:

  id 'umami'   -> 'analytics'   Name 'Hanzo Analytics'   (analytics.hanzo.ai)
  id 'posthog' -> 'insights'    Name 'Hanzo Insights'    (insights.hanzo.ai)

Error strings follow ('analytics: websiteId is required', 'insights: api_key is
required'). Upstream attribution moves to a NOTICE file — the ONE place it
belongs — covering Umami, PostHog, Casibase, and Casdoor.

Left alone deliberately: integrations/analytics.go's 'posthog' provider is a
GENUINE third-party connector (PostHog Cloud, the user's own Personal API Key),
like ga4/meta — naming a real external service is correct, not a brand leak.
Source comments that explain which upstream a fork derives from are engineering
attribution, not user-visible branding.

Build green. The 2 store/fanout test failures are the pre-existing KMS
fail-closed guard (CLOUD_KMS_MASTER_KEY_REF required) — identical on clean main,
verified by stashing this change.
2026-07-25 10:59:45 -07:00
hanzo-dev d3a640bae8 destinations: our first-party sinks are Hanzo-branded; upstream names live only in NOTICE
The two FIRST-PARTY sinks leaked their upstream fork names into the public API:
Name() returned 'Hanzo Analytics (Umami)' / 'Hanzo Insights (PostHog)' and the
platform IDs — which are the API path (/v1/destinations/<id>), the DB platform
column, and the KMS secretsPath — were literally 'umami' / 'posthog'. These are
OUR products, so they are now named for Hanzo, one way:

  id 'umami'   -> 'analytics'   Name 'Hanzo Analytics'   (analytics.hanzo.ai)
  id 'posthog' -> 'insights'    Name 'Hanzo Insights'    (insights.hanzo.ai)

Error strings follow ('analytics: websiteId is required', 'insights: api_key is
required'). Upstream attribution moves to a NOTICE file — the ONE place it
belongs — covering Umami, PostHog, Casibase, and Casdoor.

Left alone deliberately: integrations/analytics.go's 'posthog' provider is a
GENUINE third-party connector (PostHog Cloud, the user's own Personal API Key),
like ga4/meta — naming a real external service is correct, not a brand leak.
Source comments that explain which upstream a fork derives from are engineering
attribution, not user-visible branding.

Build green. The 2 store/fanout test failures are the pre-existing KMS
fail-closed guard (CLOUD_KMS_MASTER_KEY_REF required) — identical on clean main,
verified by stashing this change.
2026-07-25 10:59:45 -07:00
antje f7a15f970f ask: route research synthesis to an available capable model + fallback
/v1/ask web synthesis routed to deps.AIDefaultModel (bare deepseek-v4-flash),
a paid third-party flash model. When it is throttled, out of balance on the
M2M identity, or its provider is down, synthesize() emitted the degraded
"model is unavailable" note even though sources were grounded — a weak answer
for research/deep, the mode that most needs a strong writer.

Route synthesis per mode with a fallback chain (synthModels): research/deep
lead with zen5 (a capable Hanzo model that streams FREE on the binary's M2M
identity — strong + always reachable), search/news with zen5-flash (fast
tier). Caller in.Model still wins; CLOUD_ASK_MODEL_<MODE>/CLOUD_ASK_MODEL
override; deps.AIDefaultModel is appended as a final backstop. synthesize()
now tries the chain in order and only degrades honestly when EVERY model is
down — a single model outage advances to the next capable model instead of
emitting an empty/weak answer. Additive: books-advisor (empty mode) and the
per-answer metering are untouched.

Proven on the live M2M path: model=zen5 -> strong, clean, 12-source Clojure
research report (deepseek-v4-flash intermittently 402s; zen5 200s free).
2026-07-25 10:23:26 -07:00
antje 4ffb8852b9 ask: route research synthesis to an available capable model + fallback
/v1/ask web synthesis routed to deps.AIDefaultModel (bare deepseek-v4-flash),
a paid third-party flash model. When it is throttled, out of balance on the
M2M identity, or its provider is down, synthesize() emitted the degraded
"model is unavailable" note even though sources were grounded — a weak answer
for research/deep, the mode that most needs a strong writer.

Route synthesis per mode with a fallback chain (synthModels): research/deep
lead with zen5 (a capable Hanzo model that streams FREE on the binary's M2M
identity — strong + always reachable), search/news with zen5-flash (fast
tier). Caller in.Model still wins; CLOUD_ASK_MODEL_<MODE>/CLOUD_ASK_MODEL
override; deps.AIDefaultModel is appended as a final backstop. synthesize()
now tries the chain in order and only degrades honestly when EVERY model is
down — a single model outage advances to the next capable model instead of
emitting an empty/weak answer. Additive: books-advisor (empty mode) and the
per-answer metering are untouched.

Proven on the live M2M path: model=zen5 -> strong, clean, 12-source Clojure
research report (deepseek-v4-flash intermittently 402s; zen5 200s free).
2026-07-25 10:23:26 -07:00
zeekayandClaude Opus 4.8 18933da584 feat(team): Inbox activity-notification feed (mentions, DMs, comments, assignments)
The workbench Inbox activity feed was always empty: the write path materialized
DocNotifyContext (seed.go projectNotifyContexts) but never generated the
InboxNotification docs the Inbox reads. The classic Inbox
(notification-resources/InboxNotificationsClientImpl) reads notifications via
findAll — findAll(notification.class.ActivityInboxNotification, {user,
archived:false}) and CommonInboxNotification — the SAME query path as
DocNotifyContext, NOT domainRequest (that stub serves the separate, newer
@hcengineering/communication operation-domain, a different wire shape the classic
Inbox never queries).

notify.go adds projectNotifications, the trigger the write path (txCreate/txUpdate)
runs after applying a content Tx. It generates ActivityInboxNotification docs
(idempotent, read-flag-preserving) that findAll returns:
  - @-mention in a channel message      -> notify each mentioned member
  - direct message                      -> notify every other participant
  - comment (ChatMessage on a doc)      -> notify the doc's subscribers + mentioned
  - assignee set on a doc (Issue, ...)   -> notify the assignee (auto-subscribed),
                                           attached to a synthesized DocUpdateMessage

Recipients are auto-subscribed (ensureNotifyContext) so the Inbox can group the
notification under a context; dncID/personSpaceOf are shared with
projectNotifyContexts so contexts never diverge or duplicate. Read/unread + the
badge count need no extra machinery — the client writes isViewed:true through the
ordinary tx path and findAll returns the updated flag.

domainRequest's communication stub is left well-formed empty and its comment
corrected to reflect that it is a separate subsystem, not the Inbox path.

Tests: mention/DM/assignment/comment each produce exactly one unread
ActivityInboxNotification returned by the Inbox findAll query; author never
self-notified; idempotent replay preserves read state; unread count shrinks on read.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 08:40:40 -07:00
zeekayandhanzo-dev 4efffd4944 feat(team): Inbox activity-notification feed (mentions, DMs, comments, assignments)
The workbench Inbox activity feed was always empty: the write path materialized
DocNotifyContext (seed.go projectNotifyContexts) but never generated the
InboxNotification docs the Inbox reads. The classic Inbox
(notification-resources/InboxNotificationsClientImpl) reads notifications via
findAll — findAll(notification.class.ActivityInboxNotification, {user,
archived:false}) and CommonInboxNotification — the SAME query path as
DocNotifyContext, NOT domainRequest (that stub serves the separate, newer
@hcengineering/communication operation-domain, a different wire shape the classic
Inbox never queries).

notify.go adds projectNotifications, the trigger the write path (txCreate/txUpdate)
runs after applying a content Tx. It generates ActivityInboxNotification docs
(idempotent, read-flag-preserving) that findAll returns:
  - @-mention in a channel message      -> notify each mentioned member
  - direct message                      -> notify every other participant
  - comment (ChatMessage on a doc)      -> notify the doc's subscribers + mentioned
  - assignee set on a doc (Issue, ...)   -> notify the assignee (auto-subscribed),
                                           attached to a synthesized DocUpdateMessage

Recipients are auto-subscribed (ensureNotifyContext) so the Inbox can group the
notification under a context; dncID/personSpaceOf are shared with
projectNotifyContexts so contexts never diverge or duplicate. Read/unread + the
badge count need no extra machinery — the client writes isViewed:true through the
ordinary tx path and findAll returns the updated flag.

domainRequest's communication stub is left well-formed empty and its comment
corrected to reflect that it is a separate subsystem, not the Inbox path.

Tests: mention/DM/assignment/comment each produce exactly one unread
ActivityInboxNotification returned by the Inbox findAll query; author never
self-notified; idempotent replay preserves read state; unread count shrinks on read.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 08:40:40 -07:00
antje 73950a2eac cloud: pin iam v1.33.8 — fresh forward tag, resolve v1.33.6 force-retag go.sum landmine
Supersedes the interim v1.33.7@3af15c79 pin (missing F1a/F1b/login.go — the
login code that FAILED the real-browser gate) with v1.33.8@71f7ee47 (the
real-browser-gate-GREEN cutover HEAD: /access_token alias + F1a + F1b +
login.go PKCE fix). Forward-only; go.sum reconciled to the fresh v1.33.8
hash (stale mismatched v1.33.6 + interim v1.33.7 lines swept). The /v1/ask
grounding code already on main rides along.
2026-07-25 07:04:36 -07:00
antje a664c138b5 cloud: pin iam v1.33.8 — fresh forward tag, resolve v1.33.6 force-retag go.sum landmine
Supersedes the interim v1.33.7@3af15c79 pin (missing F1a/F1b/login.go — the
login code that FAILED the real-browser gate) with v1.33.8@71f7ee47 (the
real-browser-gate-GREEN cutover HEAD: /access_token alias + F1a + F1b +
login.go PKCE fix). Forward-only; go.sum reconciled to the fresh v1.33.8
hash (stale mismatched v1.33.6 + interim v1.33.7 lines swept). The /v1/ask
grounding code already on main rides along.
2026-07-25 07:04:36 -07:00
zeekayandClaude Opus 4.8 5d2ecfdcbf feat(integrations): Chrome connector — local browser-extension pairing
Add Chrome as an org-plane apikey connector on /v1/integrations (the
/connectors surface both hanzo.app and the console render). It represents
pairing the Hanzo browser extension — a LOCAL install that drives Chrome
over an MCP bridge for agent tasks.

Honest model: no third-party OAuth (there is no external IdP for a local
extension). It is the framework's apikey shape with a LOCAL PAIRING token
as the credential, sealed to KMS under api_key. Verification is STRUCTURAL
(offline: non-empty + length + base64url/JWT charset, fail-closed) because
the extension bridge is localhost-bound and unreachable from cloud — no
faked remote call. AdminOnly, seal-before-row, token-free errors, parity
with the cloudflare/warpcast/whatsapp org apikey connectors.

Tests: registers + lists on /v1/integrations, verify-before-store, KMS
custody round-trip, tenant isolation, org-admin gate, no token leak.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 06:54:58 -07:00
zeekayandhanzo-dev 891f0c6dfa feat(integrations): Chrome connector — local browser-extension pairing
Add Chrome as an org-plane apikey connector on /v1/integrations (the
/connectors surface both hanzo.app and the console render). It represents
pairing the Hanzo browser extension — a LOCAL install that drives Chrome
over an MCP bridge for agent tasks.

Honest model: no third-party OAuth (there is no external IdP for a local
extension). It is the framework's apikey shape with a LOCAL PAIRING token
as the credential, sealed to KMS under api_key. Verification is STRUCTURAL
(offline: non-empty + length + base64url/JWT charset, fail-closed) because
the extension bridge is localhost-bound and unreachable from cloud — no
faked remote call. AdminOnly, seal-before-row, token-free errors, parity
with the cloudflare/warpcast/whatsapp org apikey connectors.

Tests: registers + lists on /v1/integrations, verify-before-store, KMS
custody round-trip, tenant isolation, org-admin gate, no token leak.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-25 06:54:58 -07:00
hanzo-dev 9c1b995338 deps: bump hanzoai/iam v1.33.6 → v1.33.7 (unjam cloud CI)
v1.33.6 was force-re-tagged on GitHub after this go.sum recorded it → the Go
proxy holds the immutable original (h1:79DN2h1R) while GitHub serves the moved
tag → 'go mod verify' checksum mismatch fails EVERY cloud build (the containment
gate), blocking all cloud deploys (exception-scrub, paywall dark-ship, console
platform-shell fix). v1.33.7 is an IMMUTABLE re-cut of the intended content
(commit 3af15c79 'serve the deployed fleet's legacy token/userinfo path aliases'
= the current v1.33.6 tip the fleet runs). Bumping to it (x.x.x+1) pins a stable,
integrity-verifiable hash. Builds green. Stop force-moving published tags.
2026-07-25 06:44:06 -07:00
hanzo-dev ed1e789634 deps: bump hanzoai/iam v1.33.6 → v1.33.7 (unjam cloud CI)
v1.33.6 was force-re-tagged on GitHub after this go.sum recorded it → the Go
proxy holds the immutable original (h1:79DN2h1R) while GitHub serves the moved
tag → 'go mod verify' checksum mismatch fails EVERY cloud build (the containment
gate), blocking all cloud deploys (exception-scrub, paywall dark-ship, console
platform-shell fix). v1.33.7 is an IMMUTABLE re-cut of the intended content
(commit 3af15c79 'serve the deployed fleet's legacy token/userinfo path aliases'
= the current v1.33.6 tip the fleet runs). Bumping to it (x.x.x+1) pins a stable,
integrity-verifiable hash. Builds green. Stop force-moving published tags.
2026-07-25 06:44:06 -07:00
antje 6c179f6d03 release: embed console@02e5478417 — platform.hanzo.ai platform-mode + OSS App Store
console-embed:latest (07:31Z) carries the native platform/deploy home + the
1030-app OSS marketplace. Cut a cloud release to embed it so platform.hanzo.ai
serves the deploy platform, not the generic console home.
2026-07-25 00:33:47 -07:00
antje a1d4235fc8 release: embed console@02e5478417 — platform.hanzo.ai platform-mode + OSS App Store
console-embed:latest (07:31Z) carries the native platform/deploy home + the
1030-app OSS marketplace. Cut a cloud release to embed it so platform.hanzo.ai
serves the deploy platform, not the generic console home.
2026-07-25 00:33:47 -07:00
antjeandClaude Opus 4.8 4351d4d4ca paywall: 402 gate requiring an active paid plan before product access (dark-shipped)
A request filter (routers.Paywall) that, when PAYWALL_ENFORCED=true, refuses a gated
/v1 product route from a validated org with NO active paid plan — 402
{error:"subscription_required", plan:"pro", price:"$20/mo",
url:"https://cloud.hanzo.ai/plans"} — then PAYG. Default FALSE: a DARK SHIP, zero
behavior change until an owner flips the flag. Not deployed.

- routers.Paywall (serve.go: app.Use AFTER IdentityMiddleware/BillingGate, BEFORE
  MountAll) keys on the VALIDATED principal.Org / owner claim, NEVER a client X-Org-Id.
  Decision order: dark-ship passthrough -> non-/v1 & allow-listed sell/service routes ->
  anonymous (the route's own auth answers) -> super-admin bypass -> fail-OPEN (org
  unresolved / commerce nil / read error) -> live paid plan admits -> else 402. A
  paywall in front of the pay button is catastrophic, so the allow-list exempts
  /v1/signin,/v1/signout,/v1/get-account, the WHOLE /v1/billing/* money surface (the
  console PlansModule.tsx -> PlansApi subscribe flow reads GET /v1/billing/plans; +
  subscribe/balance/payment-methods/usage/subscriptions), /v1/plans*, the /v1/iam/*
  auth/OAuth/OIDC callbacks, /v1/models*, /v1/entitlements, and every /v1/*/health.

- "active paid plan" = commerceclient.ActivePaidPlan, a NEW optional capability on the
  co-resident commerce client resolved by type-assertion (mirrors types.ModelLister) so
  the narrow CommerceClient interface, disabled.go and rpc.go are untouched; a
  split-deploy/disabled client can't answer -> nil PlanChecker -> fail open. It reads the
  org's ACTIVE or TRIALING (never canceled/past_due/unpaid, per
  commerce/models/subscription.Status), unexpired subscriptions and classifies the tier
  via clients/plan.Paid. Distinct from CheckEntitlement (per-product, Status=Active only
  -> would mis-read a trial as unentitled).

- clients/plan.Paid is catalog-driven off the embedded @hanzo/plans subscription.json
  (the money-truth): a PAID cloud tier = an account category (personal/team/enterprise)
  that costs money (priceMonthly>0 or contactSales). Realizes the owner's HARD CUT at
  "Pro ($20/mo, the EXISTING Pro plan) and above" -> {pro,plus,max,team,team-max,
  enterprise,custom} with NO hardcoded slug list; the world-*/social-* product lines and
  the free developer tier are excluded.

Tests -- go build ./... compiles clean (whole-module go vet exit 0; the only go build
./... errors are pre-existing CGO final-links of cmd/* binaries against the absent local
luxcpp lib, CI-built); go test ./routers/... GREEN 9/9: active-plan passes, no-plan 402
(exact body), allow-listed paths pass with no plan AND never consult commerce, admin
bypass, PAYWALL_ENFORCED=false everything passes, + fail-open/nil-checker/anonymous/
non-v1. clients/plan paid_test.go pins the cut to the real catalog.

Drive-by: clients/plan TestPlans_Vocab was pre-existing RED on origin/main (brittle
`!= 10` vs the v1.4.4 catalog's 12 namespaces) -> converted to a `>=10` lower bound
matching the sibling `Keys < 40` assertion, so the package is green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 23:44:01 -07:00
antjeandhanzo-dev f074b4ad29 paywall: 402 gate requiring an active paid plan before product access (dark-shipped)
A request filter (routers.Paywall) that, when PAYWALL_ENFORCED=true, refuses a gated
/v1 product route from a validated org with NO active paid plan — 402
{error:"subscription_required", plan:"pro", price:"$20/mo",
url:"https://cloud.hanzo.ai/plans"} — then PAYG. Default FALSE: a DARK SHIP, zero
behavior change until an owner flips the flag. Not deployed.

- routers.Paywall (serve.go: app.Use AFTER IdentityMiddleware/BillingGate, BEFORE
  MountAll) keys on the VALIDATED principal.Org / owner claim, NEVER a client X-Org-Id.
  Decision order: dark-ship passthrough -> non-/v1 & allow-listed sell/service routes ->
  anonymous (the route's own auth answers) -> super-admin bypass -> fail-OPEN (org
  unresolved / commerce nil / read error) -> live paid plan admits -> else 402. A
  paywall in front of the pay button is catastrophic, so the allow-list exempts
  /v1/signin,/v1/signout,/v1/get-account, the WHOLE /v1/billing/* money surface (the
  console PlansModule.tsx -> PlansApi subscribe flow reads GET /v1/billing/plans; +
  subscribe/balance/payment-methods/usage/subscriptions), /v1/plans*, the /v1/iam/*
  auth/OAuth/OIDC callbacks, /v1/models*, /v1/entitlements, and every /v1/*/health.

- "active paid plan" = commerceclient.ActivePaidPlan, a NEW optional capability on the
  co-resident commerce client resolved by type-assertion (mirrors types.ModelLister) so
  the narrow CommerceClient interface, disabled.go and rpc.go are untouched; a
  split-deploy/disabled client can't answer -> nil PlanChecker -> fail open. It reads the
  org's ACTIVE or TRIALING (never canceled/past_due/unpaid, per
  commerce/models/subscription.Status), unexpired subscriptions and classifies the tier
  via clients/plan.Paid. Distinct from CheckEntitlement (per-product, Status=Active only
  -> would mis-read a trial as unentitled).

- clients/plan.Paid is catalog-driven off the embedded @hanzo/plans subscription.json
  (the money-truth): a PAID cloud tier = an account category (personal/team/enterprise)
  that costs money (priceMonthly>0 or contactSales). Realizes the owner's HARD CUT at
  "Pro ($20/mo, the EXISTING Pro plan) and above" -> {pro,plus,max,team,team-max,
  enterprise,custom} with NO hardcoded slug list; the world-*/social-* product lines and
  the free developer tier are excluded.

Tests -- go build ./... compiles clean (whole-module go vet exit 0; the only go build
./... errors are pre-existing CGO final-links of cmd/* binaries against the absent local
luxcpp lib, CI-built); go test ./routers/... GREEN 9/9: active-plan passes, no-plan 402
(exact body), allow-listed paths pass with no plan AND never consult commerce, admin
bypass, PAYWALL_ENFORCED=false everything passes, + fail-open/nil-checker/anonymous/
non-v1. clients/plan paid_test.go pins the cut to the real catalog.

Drive-by: clients/plan TestPlans_Vocab was pre-existing RED on origin/main (brittle
`!= 10` vs the v1.4.4 catalog's 12 namespaces) -> converted to a `>=10` lower bound
matching the sibling `Keys < 40` assertion, so the package is green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-24 23:44:01 -07:00
antje bce3d9e43c ask: native agentic web-search/deep-research grounding domain
Fold the answer-engine capability into the existing /v1/ask advisor as a new
WEB grounding domain, selected by mode (search|news|research|deep) — ONE
endpoint, not a second /v1/answer. Additive: an empty mode runs the figure
advisor (books) exactly as before.

Server-side bounded agentic loop (clean-room, not AGPL-derived): plan →
in-process native meta-search (reuses websearch.metaSearch via a new in-process
Search seam, no HTTP loopback) → server-side relevance rank/dedup → synthesize a
cited answer via the in-process AI path → follow-ups. Streams the @hanzo/ai
SearchEvent envelope (sources → status → text → follow_ups → done) over SSE, or
returns one JSON object.

Money: the web path GATES the caller's balance and METERS the resolved payer via
the advisor's own per-org ResourceMeter (Base.Bill) — the single revenue debit
(the in-process M2M AI calls are gateway-exempt). Bounded to <=3 LLM calls + a
capped set of search passes; a model outage degrades honestly and is not billed.

- clients/websearch/compose.go: exported in-process Search(ctx,query,lang) seam
- clients/ask/{web,relevance,stream}.go: the web domain, mode registry, ranking, envelope
- clients/ask/ask.go: AskRequest gains mode/q + web params; mode dispatch
2026-07-24 23:25:07 -07:00
antje 6a0d254028 ask: native agentic web-search/deep-research grounding domain
Fold the answer-engine capability into the existing /v1/ask advisor as a new
WEB grounding domain, selected by mode (search|news|research|deep) — ONE
endpoint, not a second /v1/answer. Additive: an empty mode runs the figure
advisor (books) exactly as before.

Server-side bounded agentic loop (clean-room, not AGPL-derived): plan →
in-process native meta-search (reuses websearch.metaSearch via a new in-process
Search seam, no HTTP loopback) → server-side relevance rank/dedup → synthesize a
cited answer via the in-process AI path → follow-ups. Streams the @hanzo/ai
SearchEvent envelope (sources → status → text → follow_ups → done) over SSE, or
returns one JSON object.

Money: the web path GATES the caller's balance and METERS the resolved payer via
the advisor's own per-org ResourceMeter (Base.Bill) — the single revenue debit
(the in-process M2M AI calls are gateway-exempt). Bounded to <=3 LLM calls + a
capped set of search passes; a model outage degrades honestly and is not billed.

- clients/websearch/compose.go: exported in-process Search(ctx,query,lang) seam
- clients/ask/{web,relevance,stream}.go: the web domain, mode registry, ranking, envelope
- clients/ask/ask.go: AskRequest gains mode/q + web params; mode dispatch
2026-07-24 23:25:07 -07:00
antje 35d94d13c9 chore(cloud): bump hanzoai/commerce v1.49.18 → v1.49.19 (money-path fixes)
Ships the two adversarially-found commerce money bugs live (commerce is embedded
in the cloud binary — the tag is the ship artifact):
- coupon redeem double-mint (HIGH): the once-per-user guard filtered Tags= on an
  exact string that never matched the compound written tag, so the same user
  re-minted a credit grant on every submit. Fixed with a deterministic-id
  couponredemption record (fail-closed, per-(code,user) locked). Proven + negative-controlled.
- inventory AdjustStock oversell (MEDIUM): raw deltas drove AvailableQuantity
  negative and still 200'd. Now 409 'insufficient available stock', level untouched.
commerce v1.49.19: gofmt clean, go test ./api/coupon ./api/inventory green, go build ./... 0.
2026-07-24 23:20:44 -07:00
antje e4f663aff8 chore(cloud): bump hanzoai/commerce v1.49.18 → v1.49.19 (money-path fixes)
Ships the two adversarially-found commerce money bugs live (commerce is embedded
in the cloud binary — the tag is the ship artifact):
- coupon redeem double-mint (HIGH): the once-per-user guard filtered Tags= on an
  exact string that never matched the compound written tag, so the same user
  re-minted a credit grant on every submit. Fixed with a deterministic-id
  couponredemption record (fail-closed, per-(code,user) locked). Proven + negative-controlled.
- inventory AdjustStock oversell (MEDIUM): raw deltas drove AvailableQuantity
  negative and still 200'd. Now 409 'insufficient available stock', level untouched.
commerce v1.49.19: gofmt clean, go test ./api/coupon ./api/inventory green, go build ./... 0.
2026-07-24 23:20:44 -07:00
hanzo-dev a7c42d04cc analytics: scrub secrets+PII from exception stacks/messages before store + fan-out
RED (o11y dogfood review) found error $exception (a *Exception struct) bypassed
scrubValue (only string/map/slice cases) — raw err.stack/message stored in
hanzo.events AND forwarded RAW to 3rd-party destinations (forward.go), leaking
tokens/API-URLs-with-query-secrets/PII, esp. now that @hanzo/event captureErrors
routes control-plane stacks (platform/console/hanzo.app all on 0.3.1).

Fix, one place: scrubException() redacts Message+Stack at the foldException fold
point (protects both the stored row and the pre-scrub fan-out copy) + a
*Exception case in scrubValue (defense in depth) + secretRe extends the string
scrubber beyond email to bearer/hk-/sk-/pk-/pk_/fw_/hz_ keys and
?token=/access_token=/api_key= query secrets. Copy semantics — never mutates the
caller. Test: email+bearer+sk-+access_token in a stack all redacted, original
unmutated. go build + package tests green.
2026-07-24 22:53:19 -07:00
hanzo-dev fad100a90d analytics: scrub secrets+PII from exception stacks/messages before store + fan-out
RED (o11y dogfood review) found error $exception (a *Exception struct) bypassed
scrubValue (only string/map/slice cases) — raw err.stack/message stored in
hanzo.events AND forwarded RAW to 3rd-party destinations (forward.go), leaking
tokens/API-URLs-with-query-secrets/PII, esp. now that @hanzo/event captureErrors
routes control-plane stacks (platform/console/hanzo.app all on 0.3.1).

Fix, one place: scrubException() redacts Message+Stack at the foldException fold
point (protects both the stored row and the pre-scrub fan-out copy) + a
*Exception case in scrubValue (defense in depth) + secretRe extends the string
scrubber beyond email to bearer/hk-/sk-/pk-/pk_/fw_/hz_ keys and
?token=/access_token=/api_key= query secrets. Copy semantics — never mutates the
caller. Test: email+bearer+sk-+access_token in a stack all redacted, original
unmutated. go build + package tests green.
2026-07-24 22:53:19 -07:00
zeekayandClaude Opus 4.8 ac55b83214 test(admin): spend-caps write-path brand negative (red I7)
Closes the highest-value cross-tenant vector the read-path negatives don't cover: a
Lux admin POST /v1/admin/spend-caps?org=zoo must forward X-Org-Id=lux downstream
(targetOrg ignores ?org= for a non-super caller), never zoo. -race green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 18:42:38 -07:00
zeekayandhanzo-dev 1e344c0bf4 test(admin): spend-caps write-path brand negative (red I7)
Closes the highest-value cross-tenant vector the read-path negatives don't cover: a
Lux admin POST /v1/admin/spend-caps?org=zoo must forward X-Org-Id=lux downstream
(targetOrg ignores ?org= for a non-super caller), never zoo. -race green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-24 18:42:38 -07:00
zeekayandClaude Opus 4.8 e91f2441c3 feat(o11y): Lux Network dashboard VM-proxy allowlist
Add 12 fixed, parameter-free, cluster="lux-k8s"-pinned queries to the
SuperAdmin-gated VM proxy (node/pod memory, deployment+statefulset health,
lux_validator_* + lux_network_*). Raw-query rejection + range-arg validation
intact. 5/5 TestVMProxy* green. Feeds the console 'Lux Network' board.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 17:53:03 -07:00
zeekayandhanzo-dev 64911105ff feat(o11y): Lux Network dashboard VM-proxy allowlist
Add 12 fixed, parameter-free, cluster="lux-k8s"-pinned queries to the
SuperAdmin-gated VM proxy (node/pod memory, deployment+statefulset health,
lux_validator_* + lux_network_*). Raw-query rejection + range-arg validation
intact. 5/5 TestVMProxy* green. Feeds the console 'Lux Network' board.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-24 17:53:03 -07:00
zeekayandClaude Opus 4.8 c089d28d8d test(admin): brand-named tenant-scope negatives — Lux admin cannot reach Zoo/Hanzo/DO
Makes the generic escalation-line invariant concrete for the real brands the ONE
shared cockpit binary serves. An admitted Lux white-label tenant admin (z@lux.network,
org=lux) is hard-pinned to the Lux subtree:
  - cannot read Zoo org data (?org=zoo ignored → sees only lux)
  - cannot list Hanzo users (IAM read hard-pinned to owner=lux)
  - cannot reach the DigitalOcean god-views (compute/finance are SuperAdmin-only → 403)

All -race green alongside a7583e75's white-label admission suite.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 17:43:17 -07:00
zeekayandhanzo-dev cd78c5b36a test(admin): brand-named tenant-scope negatives — Lux admin cannot reach Zoo/Hanzo/DO
Makes the generic escalation-line invariant concrete for the real brands the ONE
shared cockpit binary serves. An admitted Lux white-label tenant admin (z@lux.network,
org=lux) is hard-pinned to the Lux subtree:
  - cannot read Zoo org data (?org=zoo ignored → sees only lux)
  - cannot list Hanzo users (IAM read hard-pinned to owner=lux)
  - cannot reach the DigitalOcean god-views (compute/finance are SuperAdmin-only → 403)

All -race green alongside a7583e75's white-label admission suite.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-24 17:43:17 -07:00
hanzo-devandzeekay 55007fb1af feat(admin): fail-closed white-label tenant admission for the operator cockpit
admin.hanzo.ai (apps/operator) is served DIRECTLY off cloud-api /v1 (the
hanzo_iam_token/cloud_session_id path SanitizeIdentity validates), so the
/v1/admin/* gate is the only boundary. GuardScoped previously admitted ANY
org-admin (principal.IsOrgAdmin) — so any customer's own-org admin could open
the cockpit. Tighten it to the WL admission tier:

  super (owner==AdminOrg, cross-tenant)  OR
  org-admin of an org in State.WLTenants (subtree-scoped via ResolveScope)

- core.State.WLTenants: fail-closed allowlist (nil/empty => SuperAdmin-only),
  seeded ONCE from ADMIN_WL_TENANT_ORGS at Mount. IsWhiteLabelTenant is the one
  read (verbatim, trimmed, no fold).
- core.GuardScoped now requires super OR (IsOrgAdmin && Org in WLTenants);
  a non-enabled org-admin gets the same 403 as a member/forge. Fleet god-views
  stay core.Guard (super-only) — a WL tenant never reaches a fleet number.
- /v1/admin/me returns isWhiteLabel + scopeOrgs so the SPA renders the subtree
  cockpit server-authoritatively (super=fleet, WL=own subtree).

Tests: WL tenant admitted+own-subtree; non-WL org-admin refused on every scoped
panel; WL tenant 403 on every god-view; fleet-only default (empty set) fail-closed;
IsWhiteLabelTenant unit-pinned. Harness enables 'maxpower' as the WL test tenant.
2026-07-24 17:43:17 -07:00
hanzo-devandzeekay 743fc40b5d feat(admin): fail-closed white-label tenant admission for the operator cockpit
admin.hanzo.ai (apps/operator) is served DIRECTLY off cloud-api /v1 (the
hanzo_iam_token/cloud_session_id path SanitizeIdentity validates), so the
/v1/admin/* gate is the only boundary. GuardScoped previously admitted ANY
org-admin (principal.IsOrgAdmin) — so any customer's own-org admin could open
the cockpit. Tighten it to the WL admission tier:

  super (owner==AdminOrg, cross-tenant)  OR
  org-admin of an org in State.WLTenants (subtree-scoped via ResolveScope)

- core.State.WLTenants: fail-closed allowlist (nil/empty => SuperAdmin-only),
  seeded ONCE from ADMIN_WL_TENANT_ORGS at Mount. IsWhiteLabelTenant is the one
  read (verbatim, trimmed, no fold).
- core.GuardScoped now requires super OR (IsOrgAdmin && Org in WLTenants);
  a non-enabled org-admin gets the same 403 as a member/forge. Fleet god-views
  stay core.Guard (super-only) — a WL tenant never reaches a fleet number.
- /v1/admin/me returns isWhiteLabel + scopeOrgs so the SPA renders the subtree
  cockpit server-authoritatively (super=fleet, WL=own subtree).

Tests: WL tenant admitted+own-subtree; non-WL org-admin refused on every scoped
panel; WL tenant 403 on every god-view; fleet-only default (empty set) fail-closed;
IsWhiteLabelTenant unit-pinned. Harness enables 'maxpower' as the WL test tenant.
2026-07-24 17:43:17 -07:00
antje f599d9beed feat(authors): accept a GitHub/GitLab ORG url in Verify a repo — owner-wide claim
Live bug: pasting github.com/luxfi (an ORG, no repo name) was rejected as malformed.
Now an org url is a first-class target — ownership proven EXACTLY like a repo claim
(the same proveOwnership: OAuth admin/push OR hanzo.json verify-code), against the
owner's canonical <owner>/.github control repo. Controlling that repo == controlling
the owner; an un-owned org fails both proofs → 422, identical to an un-owned repo.
Zero new forge methods. Deploy attribution tries the per-repo claim first, then an
owner-wide org claim — both coexist. New author_orgs table (IF NOT EXISTS, backward-
compatible), first-verify-wins. Softened the defaultShareBps comment (public rate is
presented by the FE, not promised here); the 2000 bps value + all math are byte-identical.

gofmt clean; go build/vet green; go test ./clients/authors ✓ 17/17 (14 pre-existing + 3 new:
org-url accepted+covers-owner, hanzo.json org proof, un-owned org refused).
2026-07-24 17:13:34 -07:00
antje dfaa359511 feat(authors): accept a GitHub/GitLab ORG url in Verify a repo — owner-wide claim
Live bug: pasting github.com/luxfi (an ORG, no repo name) was rejected as malformed.
Now an org url is a first-class target — ownership proven EXACTLY like a repo claim
(the same proveOwnership: OAuth admin/push OR hanzo.json verify-code), against the
owner's canonical <owner>/.github control repo. Controlling that repo == controlling
the owner; an un-owned org fails both proofs → 422, identical to an un-owned repo.
Zero new forge methods. Deploy attribution tries the per-repo claim first, then an
owner-wide org claim — both coexist. New author_orgs table (IF NOT EXISTS, backward-
compatible), first-verify-wins. Softened the defaultShareBps comment (public rate is
presented by the FE, not promised here); the 2000 bps value + all math are byte-identical.

gofmt clean; go build/vet green; go test ./clients/authors ✓ 17/17 (14 pre-existing + 3 new:
org-url accepted+covers-owner, hanzo.json org proof, un-owned org refused).
2026-07-24 17:13:34 -07:00
antje da4bec146b platform: meter running-deployment compute → org spend (closes OSS royalty loop)
The author-royalty sweep (clients/authors) accrues 20% of a deploying org's
METERED spend, but a running template deployment added nothing to that spend —
only build minutes were metered, once (buildmeter.go). EstimateTemplate priced
the SBOM compute rate yet had no consumer. This wires the last stitch.

A periodic single-writer meter (computemeter.go), started next to the build
reconciler and stopped on Shutdown, charges every `live` app's own org for the
compute it consumed since the last tick, at its SBOM rate
(blueprint.EstimateService → the SAME rate card the console shows), through the
shared ResourceMeter → commerce ledger. The org's metered spend now carries real
running compute, so the shipped 20% sweep automatically takes its cut —
creator/treasury paid, end to end: deploy → SBOM-rate compute meter → org spend
→ 20% → creator.

Money-path correctness:
- Idempotent / at-most-once per (app, span): a per-app watermark
  (platform_apps.compute_metered_at) is compare-and-set advanced in the same
  store before the debit, so a double-tick or restart never double-charges.
- Watermark reset to now on every transition INTO live (FinalizeLive + start),
  so a stopped/redeployed app is never billed for time it was not running.
- First sight (watermark 0) starts the clock with no back-charge.
- Metered in microdollars (µ$/hr rate card) — no sub-cent rounding loss.
- Self-deploys are metered like any other app (their compute is theirs); the
  self-royalty exclusion stays entirely in the authors sweep.

blueprint.EstimateService is the single-container analogue of EstimateTemplate,
sharing one cost formula (priceFootprint) so a bare deployment and a one-service
template of the same image price identically.

Tests (CGO_ENABLED=0, -race): compute arithmetic; a live app meters its org at
the SBOM rate per hour; double-tick = single charge; first-sight/stopped never
charge; and the metered spend yields the expected 20% creator cut at monthly
scale.
2026-07-24 16:42:45 -07:00
antje 1ed8f3b37d platform: meter running-deployment compute → org spend (closes OSS royalty loop)
The author-royalty sweep (clients/authors) accrues 20% of a deploying org's
METERED spend, but a running template deployment added nothing to that spend —
only build minutes were metered, once (buildmeter.go). EstimateTemplate priced
the SBOM compute rate yet had no consumer. This wires the last stitch.

A periodic single-writer meter (computemeter.go), started next to the build
reconciler and stopped on Shutdown, charges every `live` app's own org for the
compute it consumed since the last tick, at its SBOM rate
(blueprint.EstimateService → the SAME rate card the console shows), through the
shared ResourceMeter → commerce ledger. The org's metered spend now carries real
running compute, so the shipped 20% sweep automatically takes its cut —
creator/treasury paid, end to end: deploy → SBOM-rate compute meter → org spend
→ 20% → creator.

Money-path correctness:
- Idempotent / at-most-once per (app, span): a per-app watermark
  (platform_apps.compute_metered_at) is compare-and-set advanced in the same
  store before the debit, so a double-tick or restart never double-charges.
- Watermark reset to now on every transition INTO live (FinalizeLive + start),
  so a stopped/redeployed app is never billed for time it was not running.
- First sight (watermark 0) starts the clock with no back-charge.
- Metered in microdollars (µ$/hr rate card) — no sub-cent rounding loss.
- Self-deploys are metered like any other app (their compute is theirs); the
  self-royalty exclusion stays entirely in the authors sweep.

blueprint.EstimateService is the single-container analogue of EstimateTemplate,
sharing one cost formula (priceFootprint) so a bare deployment and a one-service
template of the same image price identically.

Tests (CGO_ENABLED=0, -race): compute arithmetic; a live app meters its org at
the SBOM rate per hour; double-tick = single charge; first-sight/stopped never
charge; and the metered spend yields the expected 20% creator cut at monthly
scale.
2026-07-24 16:42:45 -07:00
zeekayandClaude Opus 4.8 99ad199fea entitlements: server-side unified paywall (RequireProduct + /v1/entitlements)
Extend clients/entitlements with the enforcement authority behind the
@hanzogui/shell client gate — no new subsystem (frozen wire spec intact).

- RequireProduct(commerce, product): group middleware. Defers to the ONE
  authority commerce.CheckEntitlement (plan tier -> @hanzo/plans license
  features). Unvalidated -> 403; nil/error/nil-result -> ALLOW (fail open, an
  outage never locks out a paying user); definitive Active==false -> 402.
  Mirrors clients/team/entitle.go, not the fail-closed enable gate.
- GET /v1/entitlements: authoritative {tier, apps:{studio,bot,world,platform,
  team,admin}} projection the shell reads. Fails SAFE-TO-LOCKED (200, never 500)
  on commerce error; admin = c.IsAdmin().

CATALOG FLAG: @hanzo/plans v1.4.4 licensing.product_ids licenses only team/
engine/engine-rocm. studio, bot, world, platform are ABSENT, so enforcing them
now would 402 every org. Enforcement is therefore wired LIVE nowhere: world/bots/
platform carry commented RequireProduct markers (flip on once the catalog adds
the product); team keeps its deliberate observe-only gate; admin stays super-
admin gated. Projection reports the real per-org bool (studio/bot/world/platform
= false until the catalog is updated).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 16:32:04 -07:00
zeekayandhanzo-dev 9fa690ad18 entitlements: server-side unified paywall (RequireProduct + /v1/entitlements)
Extend clients/entitlements with the enforcement authority behind the
@hanzogui/shell client gate — no new subsystem (frozen wire spec intact).

- RequireProduct(commerce, product): group middleware. Defers to the ONE
  authority commerce.CheckEntitlement (plan tier -> @hanzo/plans license
  features). Unvalidated -> 403; nil/error/nil-result -> ALLOW (fail open, an
  outage never locks out a paying user); definitive Active==false -> 402.
  Mirrors clients/team/entitle.go, not the fail-closed enable gate.
- GET /v1/entitlements: authoritative {tier, apps:{studio,bot,world,platform,
  team,admin}} projection the shell reads. Fails SAFE-TO-LOCKED (200, never 500)
  on commerce error; admin = c.IsAdmin().

CATALOG FLAG: @hanzo/plans v1.4.4 licensing.product_ids licenses only team/
engine/engine-rocm. studio, bot, world, platform are ABSENT, so enforcing them
now would 402 every org. Enforcement is therefore wired LIVE nowhere: world/bots/
platform carry commented RequireProduct markers (flip on once the catalog adds
the product); team keeps its deliberate observe-only gate; admin stays super-
admin gated. Projection reports the real per-org bool (studio/bot/world/platform
= false until the catalog is updated).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-24 16:32:04 -07:00
antje 61deaaee0f docs(iam): supervised Casdoor→embedded-IAM cutover runbook
The code blocker is shipped (iam v1.33.6 aliases + cloud pin). IAM_CUTOVER.md is
the one-session supervised flip: migrate-v1 --dry-run drift gate → real migrate
into /var/lib/cloud/iam/iam.db (before seed) → CLOUD_ENABLE_STAGED=iam → repoint
routes.yaml iam-hanzo-ai iam.hanzo.svc:80→cloud.hanzo.svc:8000 (hot, no ingress
restart) → Playwright parity → retire the Casdoor iam CR. Each step: verify +
rollback. Money/auth-critical; explicitly NOT fired here.
2026-07-24 16:19:22 -07:00
antje 5309dd09d3 docs(iam): supervised Casdoor→embedded-IAM cutover runbook
The code blocker is shipped (iam v1.33.6 aliases + cloud pin). IAM_CUTOVER.md is
the one-session supervised flip: migrate-v1 --dry-run drift gate → real migrate
into /var/lib/cloud/iam/iam.db (before seed) → CLOUD_ENABLE_STAGED=iam → repoint
routes.yaml iam-hanzo-ai iam.hanzo.svc:80→cloud.hanzo.svc:8000 (hot, no ingress
restart) → Playwright parity → retire the Casdoor iam CR. Each step: verify +
rollback. Money/auth-critical; explicitly NOT fired here.
2026-07-24 16:19:22 -07:00
antje 3e3c05883c chore(iam): bump embedded IAM pin v1.33.0 → v1.33.6
Picks up the legacy token/userinfo path aliases (hanzoai/iam v1.33.6) the
deployed fleet hard-codes — /v1/iam/oauth/access_token, /oauth/refresh_token,
/v1/iam/userinfo — so the Casdoor→embedded-IAM cutover serves every hard-coded
caller (hanzo CLI, KMS bridge, gateway admin/waitlist guards, commerce) instead
of 404ing them. Embedded-IAM surface (root pkg + clients/iam) compiles clean and
iamserver still exposes Mount/OpenSQLite/Seed unchanged.
2026-07-24 16:13:54 -07:00
antje 1286c41f16 chore(iam): bump embedded IAM pin v1.33.0 → v1.33.6
Picks up the legacy token/userinfo path aliases (hanzoai/iam v1.33.6) the
deployed fleet hard-codes — /v1/iam/oauth/access_token, /oauth/refresh_token,
/v1/iam/userinfo — so the Casdoor→embedded-IAM cutover serves every hard-coded
caller (hanzo CLI, KMS bridge, gateway admin/waitlist guards, commerce) instead
of 404ing them. Embedded-IAM surface (root pkg + clients/iam) compiles clean and
iamserver still exposes Mount/OpenSQLite/Seed unchanged.
2026-07-24 16:13:54 -07:00
antje fea568c89c authors: automatic creator-payout loop — 20% everywhere, scheduler-driven, Hanzo forks → treasury
Close the OSS creator-payout loop so accrual AND payout run with no human.

20% EVERYWHERE: the treasury revenue-share policy now defaults to 20% (2000 bps)
when unset (ledger.DefaultRevenueShareBps — one place; SetPolicy still accepts any
0–10000). The 25%-era payout test is rewritten share-derived so it never drifts.
(defaultShareBps=2000 already landed @2c47c8f5d.)

AUTOMATE: clients/authors/scheduler.go mirrors clients/social/scheduler.go — a
single-writer ticker (CLOUD_AUTHORS_SCHEDULER_INTERVAL, default 1h) drives
sweepAndPayout: accrue every approved author's period royalty, then AUTO-PAY the
full pending balance. issuePayout is the ONE payout path shared by the scheduler
and the /v1/admin/authors/:id/payout operator override. Idempotent: RecordPayout's
atomic pending guard makes auto-payout at-most-once — never a double-pay, never
exceeds accrued−paid.

HANZO FORKS → TREASURY ("pay ourselves"): a repo owned by a brand org (owner ∈
{hanzoai, hanzo-*}) is auto-attributed on first deploy to the treasury SYSTEM
author (org = brand slug), so Hanzo earns 20% on its own templates when other orgs
deploy them. That royalty is realized into the treasury reserve via the shared
treasury client (new treasury.Credit → ledger Seed, idempotent by ref) — no
external wallet, no new ledger. White-label by brand (lux→luxfi, zoo→zooai).

TDD (clients/authors + clients/treasury green): 20% accrual, auto-payout
idempotency, Hanzo-fork→treasury routing (royalty never hits an external wallet),
maintainer detection.
2026-07-24 16:06:36 -07:00
antje bc04b675fa authors: automatic creator-payout loop — 20% everywhere, scheduler-driven, Hanzo forks → treasury
Close the OSS creator-payout loop so accrual AND payout run with no human.

20% EVERYWHERE: the treasury revenue-share policy now defaults to 20% (2000 bps)
when unset (ledger.DefaultRevenueShareBps — one place; SetPolicy still accepts any
0–10000). The 25%-era payout test is rewritten share-derived so it never drifts.
(defaultShareBps=2000 already landed @ea5ae54b4.)

AUTOMATE: clients/authors/scheduler.go mirrors clients/social/scheduler.go — a
single-writer ticker (CLOUD_AUTHORS_SCHEDULER_INTERVAL, default 1h) drives
sweepAndPayout: accrue every approved author's period royalty, then AUTO-PAY the
full pending balance. issuePayout is the ONE payout path shared by the scheduler
and the /v1/admin/authors/:id/payout operator override. Idempotent: RecordPayout's
atomic pending guard makes auto-payout at-most-once — never a double-pay, never
exceeds accrued−paid.

HANZO FORKS → TREASURY ("pay ourselves"): a repo owned by a brand org (owner ∈
{hanzoai, hanzo-*}) is auto-attributed on first deploy to the treasury SYSTEM
author (org = brand slug), so Hanzo earns 20% on its own templates when other orgs
deploy them. That royalty is realized into the treasury reserve via the shared
treasury client (new treasury.Credit → ledger Seed, idempotent by ref) — no
external wallet, no new ledger. White-label by brand (lux→luxfi, zoo→zooai).

TDD (clients/authors + clients/treasury green): 20% accrual, auto-payout
idempotency, Hanzo-fork→treasury routing (royalty never hits an external wallet),
maintainer detection.
2026-07-24 16:06:36 -07:00
antje 6b15c79eb3 blueprint: OSS-template SBOM → compute-cost model (/v1/blueprint)
Parse a blueprint's docker-compose into its SBOM (bill of container
images) and price the stack's CPU/memory footprint through a documented
rate card (microdollars per vCPU-/GB-hour, DigitalOcean-droplet-derived +
platform margin; tunable via CLOUD_BLUEPRINT_UCPU_HR/_UGB_HR). Sizing is
the declared deploy.resources reservations/limits (or legacy cpus/mem_*)
else a default footprint per inferred class (db/cache/web/worker/other);
deploy.replicas scales it.

EstimateTemplate(id) -> {sbom, vcpuHr, gbHr, microUsdPerHour,
estCentsPerMonth} is the in-process seam the deploy path meters an org on
and the "~$X/mo to run" the console shows — the compute cost the 20%
author royalty (clients/authors, defaultShareBps=2000) is taken from.
Distinct from clients/sbom (CycloneDX packages inside one image, keyed by
digest); this is the bill of IMAGES a stack runs, keyed by template.

Surface: GET /v1/blueprint (index), /v1/blueprint/sbom?template=<id> (one),
/v1/blueprint/sbom (batch), /v1/blueprint/health. 5 embedded OSS
blueprints; pure I/O-free core; 11 tests green; frozen wire-order refrozen.
2026-07-24 15:58:38 -07:00
antje 4ef2510254 blueprint: OSS-template SBOM → compute-cost model (/v1/blueprint)
Parse a blueprint's docker-compose into its SBOM (bill of container
images) and price the stack's CPU/memory footprint through a documented
rate card (microdollars per vCPU-/GB-hour, DigitalOcean-droplet-derived +
platform margin; tunable via CLOUD_BLUEPRINT_UCPU_HR/_UGB_HR). Sizing is
the declared deploy.resources reservations/limits (or legacy cpus/mem_*)
else a default footprint per inferred class (db/cache/web/worker/other);
deploy.replicas scales it.

EstimateTemplate(id) -> {sbom, vcpuHr, gbHr, microUsdPerHour,
estCentsPerMonth} is the in-process seam the deploy path meters an org on
and the "~$X/mo to run" the console shows — the compute cost the 20%
author royalty (clients/authors, defaultShareBps=2000) is taken from.
Distinct from clients/sbom (CycloneDX packages inside one image, keyed by
digest); this is the bill of IMAGES a stack runs, keyed by template.

Surface: GET /v1/blueprint (index), /v1/blueprint/sbom?template=<id> (one),
/v1/blueprint/sbom (batch), /v1/blueprint/health. 5 embedded OSS
blueprints; pure I/O-free core; 11 tests green; frozen wire-order refrozen.
2026-07-24 15:58:38 -07:00
antje 8960655774 ask: unified grounded /v1/ask advisor (books contributor + pluggable registry) 2026-07-24 15:35:55 -07:00
antje 6865c4c5a6 ask: unified grounded /v1/ask advisor (books contributor + pluggable registry) 2026-07-24 15:35:55 -07:00
antje 2c47c8f5dc authors: OSS creator royalty = 20% (was contradictory 25% const / 5% comment)
defaultShareBps 2500→2000. 20% is the ONE canonical creator share across the OSS
marketplace (oss.hanzo.ai, platform templates, 'Earn 20%' CTA). Reconciled the
25%-vs-5% contradiction in the code+comments; test expectations track the const.
2026-07-24 15:28:24 -07:00
antje ea5ae54b46 authors: OSS creator royalty = 20% (was contradictory 25% const / 5% comment)
defaultShareBps 2500→2000. 20% is the ONE canonical creator share across the OSS
marketplace (oss.hanzo.ai, platform templates, 'Earn 20%' CTA). Reconciled the
25%-vs-5% contradiction in the code+comments; test expectations track the const.
2026-07-24 15:28:24 -07:00
hanzo-dev 4e6ec52afd ci(release): neutralize → sync-notice; native build/deploy is .hanzo/workflows/deploy.yml (GitHub is a mirror) 2026-07-24 15:13:32 -07:00
hanzo-dev a877eda205 ci(release): neutralize → sync-notice; native build/deploy is .hanzo/workflows/deploy.yml (GitHub is a mirror) 2026-07-24 15:13:32 -07:00
hanzo-dev 911969ede6 ci(deploy): native Hanzo pipeline — BuildKit ./Dockerfile → ghcr.io/hanzoai/cloud:<sha> → patch app {cloud,cloud-iam2-canary,cloud-reader} 2026-07-24 15:13:25 -07:00
hanzo-dev 01ee372a67 ci(deploy): native Hanzo pipeline — BuildKit ./Dockerfile → ghcr.io/hanzoai/cloud:<sha> → patch app {cloud,cloud-iam2-canary,cloud-reader} 2026-07-24 15:13:25 -07:00
antje 8792a7a1f3 trace-pipeline bump (ai v1.831.1 + o11y v1.5.30) + admin infra visibility + commerce revenue-read fix
- go.mod/go.sum: ai v1.831.0->v1.831.1 (gen_ai spans nest under the request span;
  always stamp org_id + X-Session-Id + deployment.environment), o11y v1.5.28->v1.5.30
  (querier resolves $N trace_id before the trace-summary short-circuit)
- clients/admin: infra visibility — products fleet view, revenue read (fanin), types
- clients/paas: fleet observer published to admin (products/health rollup)

Money/trace commerce co-resident source fix already live via 4500f609e (v1.801.203);
this ships the trace-pipeline module bump + admin/paas visibility.
2026-07-24 12:43:32 -07:00
antje b8658dc543 trace-pipeline bump (ai v1.831.1 + o11y v1.5.30) + admin infra visibility + commerce revenue-read fix
- go.mod/go.sum: ai v1.831.0->v1.831.1 (gen_ai spans nest under the request span;
  always stamp org_id + X-Session-Id + deployment.environment), o11y v1.5.28->v1.5.30
  (querier resolves $N trace_id before the trace-summary short-circuit)
- clients/admin: infra visibility — products fleet view, revenue read (fanin), types
- clients/paas: fleet observer published to admin (products/health rollup)

Money/trace commerce co-resident source fix already live via 9feef2acf (v1.801.203);
this ships the trace-pipeline module bump + admin/paas visibility.
2026-07-24 12:43:32 -07:00
antje 56c1721091 books: address scanner red findings 2026-07-24 12:37:55 -07:00
antje bc3e9e52de books: address scanner red findings 2026-07-24 12:37:55 -07:00
antje 0022b4e255 books: scanner + inbox + vendors/rules + transactions 2026-07-24 12:37:55 -07:00
antje 93f20d40d9 books: scanner + inbox + vendors/rules + transactions 2026-07-24 12:37:55 -07:00
z 95ba3bcd86 docs: modernize + LLM.md + cross-links (one-way SDK model) 2026-07-24 12:29:49 -07:00
z 8b961f3273 docs: modernize + LLM.md + cross-links (one-way SDK model) 2026-07-24 12:29:49 -07:00
z f1ef2f94ba docs: modernize + LLM.md + cross-links (one-way SDK model) 2026-07-24 12:29:42 -07:00
z 96833fde9a docs: modernize + LLM.md + cross-links (one-way SDK model) 2026-07-24 12:29:42 -07:00
zeekayandClaude Fable 5 e87e2dd356 feat(webhooks): delivery log + test-send + secret rotation — the console product surface
Extend the global /v1/webhooks subsystem with the surface a console Webhooks
product needs — persisted delivery logs, inline test-send, secret rotation, and
cheap usage counters — all riding the ONE existing delivery path.

- store: `delivery` table (per-org, endpoint_id/delivery_id/subject/attempt/
  status/http_status/error/duration_ms/created) with opportunistic retention
  (newest 500/endpoint, pruned on insert); recordDelivery/deliveries/usage/
  setSecret methods.
- dispatch: attempt() now returns a rich attemptResult (ok/retryable/httpStatus/
  err/duration); deliver() records ONE best-effort row per attempt from the
  worker goroutine (off the ack path) — "retrying" mid-ladder, terminal "ok"/
  "failed". Same attempt() is the single sign+POST the test-send reuses.
- api: GET /:id/deliveries (org-scoped, newest-first, ?limit default 50/max 200,
  ?status=), POST /:id/test (synchronous single-attempt send + row + inline
  result, works while disabled), POST /:id/rotate-secret (reveal-once, old
  secret invalid immediately). list/get carry deliveries7d + failures7d from one
  grouped aggregate.

Tests (+6): per-attempt logging incl. retries, retention prune, deliveries list
org-isolation/filter/limit, test-send signature+row+disabled, rotate-secret
old-invalid/new-verifies, usage counters. 21/21 green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 12:28:12 -07:00
zeekayandhanzo-dev cfd1c5bf00 feat(webhooks): delivery log + test-send + secret rotation — the console product surface
Extend the global /v1/webhooks subsystem with the surface a console Webhooks
product needs — persisted delivery logs, inline test-send, secret rotation, and
cheap usage counters — all riding the ONE existing delivery path.

- store: `delivery` table (per-org, endpoint_id/delivery_id/subject/attempt/
  status/http_status/error/duration_ms/created) with opportunistic retention
  (newest 500/endpoint, pruned on insert); recordDelivery/deliveries/usage/
  setSecret methods.
- dispatch: attempt() now returns a rich attemptResult (ok/retryable/httpStatus/
  err/duration); deliver() records ONE best-effort row per attempt from the
  worker goroutine (off the ack path) — "retrying" mid-ladder, terminal "ok"/
  "failed". Same attempt() is the single sign+POST the test-send reuses.
- api: GET /:id/deliveries (org-scoped, newest-first, ?limit default 50/max 200,
  ?status=), POST /:id/test (synchronous single-attempt send + row + inline
  result, works while disabled), POST /:id/rotate-secret (reveal-once, old
  secret invalid immediately). list/get carry deliveries7d + failures7d from one
  grouped aggregate.

Tests (+6): per-attempt logging incl. retries, retention prune, deliveries list
org-isolation/filter/limit, test-send signature+row+disabled, rotate-secret
old-invalid/new-verifies, usage counters. 21/21 green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-24 12:28:12 -07:00
zeekayandClaude Fable 5 0b2c54aa24 build(deps): bump hanzoai/commerce v1.49.17 → v1.49.18 (local webhook engine removed; delivery = the global /v1/webhooks dispatcher)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 12:03:41 -07:00
zeekayandhanzo-dev 4aa2ea79cf build(deps): bump hanzoai/commerce v1.49.17 → v1.49.18 (local webhook engine removed; delivery = the global /v1/webhooks dispatcher)
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-24 12:03:41 -07:00
zeekayandClaude Fable 5 39c0d0f9a4 feat(webhooks): global /v1/webhooks — one registry + bus-driven dispatcher for all platform events
Owner directives: webhooks are a first-class top-level resource (/v1/webhooks,
never nested under billing) and EVERY global event on the platform bus must be
webhook-deliverable.

- clients/webhooks: org-scoped registry CRUD at /v1/webhooks (validated principal,
  org from the gateway-minted identity; per-org SQLite store; server-generated
  secret returned only on create) + ONE durable JetStream consumer
  (webhooks-dispatch on stream COMMERCE today; streams are a config list) that
  fans events to subscribed endpoints with NATS-wildcard filters (*, >).
- Delivery = the canonical semantics: X-Webhook-Signature (t=,v1= HMAC-SHA256),
  X-Webhook-Event, X-Webhook-Delivery (stable per attempt-group), fresh sig per
  attempt, 3 attempts 1s/5s/25s+jitter, 10s timeout, retry only network/5xx/429.
  Org-isolated: an event only ever reaches its own org's endpoints; org-less
  events deliver to nobody. Ack-on-queue — a slow subscriber never stalls the bus.
- Fail-soft Mount (registry serves with the bus down; consumer retries) +
  Shutdown drains stores and the worker pool.
- apps: wired after catalogsync; frozen golden updated — including the share and
  books rows the parallel stream added to Wire() without updating the golden
  (the golden was already red on HEAD; reconciled to reality).

15 tests green (fresh run): CRUD auth + org isolation, subject matcher, header/
signature recompute, retry ladder, delivery org isolation, disabled-endpoint
skip, in-process NATS end-to-end.

Supersedes commerce's local delivery engine (v1.49.15) — rip tracked separately.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 11:50:21 -07:00
zeekayandhanzo-dev 2d77642de2 feat(webhooks): global /v1/webhooks — one registry + bus-driven dispatcher for all platform events
Owner directives: webhooks are a first-class top-level resource (/v1/webhooks,
never nested under billing) and EVERY global event on the platform bus must be
webhook-deliverable.

- clients/webhooks: org-scoped registry CRUD at /v1/webhooks (validated principal,
  org from the gateway-minted identity; per-org SQLite store; server-generated
  secret returned only on create) + ONE durable JetStream consumer
  (webhooks-dispatch on stream COMMERCE today; streams are a config list) that
  fans events to subscribed endpoints with NATS-wildcard filters (*, >).
- Delivery = the canonical semantics: X-Webhook-Signature (t=,v1= HMAC-SHA256),
  X-Webhook-Event, X-Webhook-Delivery (stable per attempt-group), fresh sig per
  attempt, 3 attempts 1s/5s/25s+jitter, 10s timeout, retry only network/5xx/429.
  Org-isolated: an event only ever reaches its own org's endpoints; org-less
  events deliver to nobody. Ack-on-queue — a slow subscriber never stalls the bus.
- Fail-soft Mount (registry serves with the bus down; consumer retries) +
  Shutdown drains stores and the worker pool.
- apps: wired after catalogsync; frozen golden updated — including the share and
  books rows the parallel stream added to Wire() without updating the golden
  (the golden was already red on HEAD; reconciled to reality).

15 tests green (fresh run): CRUD auth + org isolation, subject matcher, header/
signature recompute, retry ladder, delivery org isolation, disabled-endpoint
skip, in-process NATS end-to-end.

Supersedes commerce's local delivery engine (v1.49.15) — rip tracked separately.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-24 11:50:21 -07:00
antje c194d4a510 deps(cloud): commerce v1.49.16 → v1.49.17 — activate self-serve create-store + onboarding + paywall
commerce runs co-resident in cloud (commerceinproc). Bumping the dep brings the
Shopify-parity backend live: org-admins can create a store in their OWN org (home==
effective bound), onboarding's trial starts at store creation, and the subscription
paywall (billing/paywall) enforces the $20/mo gate — SuperAdmins exempt, trial/sub/
invite pass. Cloud builds clean with v1.49.17 (verified).
2026-07-24 09:47:04 -07:00
antje 0fd3d62f71 deps(cloud): commerce v1.49.16 → v1.49.17 — activate self-serve create-store + onboarding + paywall
commerce runs co-resident in cloud (commerceinproc). Bumping the dep brings the
Shopify-parity backend live: org-admins can create a store in their OWN org (home==
effective bound), onboarding's trial starts at store creation, and the subscription
paywall (billing/paywall) enforces the $20/mo gate — SuperAdmins exempt, trial/sub/
invite pass. Cloud builds clean with v1.49.17 (verified).
2026-07-24 09:47:04 -07:00
antje 9f9f5c7e75 books: land the AI bookkeeper on /v1/books (native double-entry, per-org SQLite)
A native 'books' domain in the cloud one-binary — the ERPNext Accounts SEMANTICS
ported to Go, NO Postgres/Formance/ERPNext-Python. Reads commerce
/v1/billing/transactions (sole posting source, read-only) and books the
double-entry twin; GAAP rev-rec (Customer Wallet = deferred-revenue liability,
recognized at usage); accrual P&L + Balance Sheet + export; AI Ask brain (SaaS
metrics: MRR/ARR/burn/runway) + clarifying questions; bank engine with PDF
(rsc.io/pdf + foot-check reconciliation), OFX/CSV, Plaid, Teller + reconciliation.
Exact int64 cents, per-org Base/SQLite, ~35 tests green (built blue/red/cto).
Registered in apps.go beside treasury; adds rsc.io/pdf direct dep.
2026-07-24 02:04:15 -07:00
antje 62c58040a6 books: land the AI bookkeeper on /v1/books (native double-entry, per-org SQLite)
A native 'books' domain in the cloud one-binary — the ERPNext Accounts SEMANTICS
ported to Go, NO Postgres/Formance/ERPNext-Python. Reads commerce
/v1/billing/transactions (sole posting source, read-only) and books the
double-entry twin; GAAP rev-rec (Customer Wallet = deferred-revenue liability,
recognized at usage); accrual P&L + Balance Sheet + export; AI Ask brain (SaaS
metrics: MRR/ARR/burn/runway) + clarifying questions; bank engine with PDF
(rsc.io/pdf + foot-check reconciliation), OFX/CSV, Plaid, Teller + reconciliation.
Exact int64 cents, per-org Base/SQLite, ~35 tests green (built blue/red/cto).
Registered in apps.go beside treasury; adds rsc.io/pdf direct dep.
2026-07-24 02:04:15 -07:00
antje bee8d71d6a merge: analytics destination adapters (Umami + PostHog + GA4/Meta ecommerce)
Fan POST /v1/analytics out to analytics.hanzo.ai (Umami /api/send) and
insights.hanzo.ai (PostHog /v1/e); ecommerce -> GA4/Meta conversion mapping.
Fail-soft fanout; adapters inert until per-site KMS keys are set (additive).
2026-07-24 01:51:26 -07:00
antje 84ecd7b6f2 merge: analytics destination adapters (Umami + PostHog + GA4/Meta ecommerce)
Fan POST /v1/analytics out to analytics.hanzo.ai (Umami /api/send) and
insights.hanzo.ai (PostHog /v1/e); ecommerce -> GA4/Meta conversion mapping.
Fail-soft fanout; adapters inert until per-site KMS keys are set (additive).
2026-07-24 01:51:26 -07:00
antje 4d71c12a6b fix(destinations): PostHog → /v1/e, Umami visitor id → id (red must-fix)
Two silent-drop contract bugs from adversarial (red) review of the Umami +
PostHog analytics sinks — both fail soft, so a paid product ingested zero.

- posthog [critical]: Send POSTed to <host>/batch, but Hanzo Insights
  capture-rs registers ONLY /v1/e|/v1/s|/v1/ai (rust/capture/src/router.rs;
  legacy /batch REMOVED, Django APPEND_SLASH=False) → every event 404'd and
  was dropped (postJSON logs, never wedges). Repoint to /v1/e; the
  {api_key,batch:[…]} body is unchanged — capture-rs's untagged
  RawRequest::Batch variant accepts it verbatim.

- umami [medium]: umamiPayload.DistinctID serialized as `distinctId`, a field
  absent from the fork's /api/send zod schema (src/app/api/send/route.ts) →
  silently stripped, so the forwarded visitor id never keyed the session
  (stitching fell back to IP+UA). Rename the JSON tag to `id`, the schema's
  real session key: sessionId = id ? uuid(website,id) : uuid(website,ip,ua,salt).

Tests now assert the real WIRE contract (a struct-field assert cannot catch a
JSON-tag bug — the source of the false confidence red flagged):
TestPostHogSendEndToEnd → /v1/e; TestUmamiBuild + TestUmamiSendEndToEnd → the
visitor id marshals under `id`, never `distinctId`.

go build + go vet + go test ./clients/destinations/... green.
2026-07-24 01:44:56 -07:00
antje 2a1d76ae4d fix(destinations): PostHog → /v1/e, Umami visitor id → id (red must-fix)
Two silent-drop contract bugs from adversarial (red) review of the Umami +
PostHog analytics sinks — both fail soft, so a paid product ingested zero.

- posthog [critical]: Send POSTed to <host>/batch, but Hanzo Insights
  capture-rs registers ONLY /v1/e|/v1/s|/v1/ai (rust/capture/src/router.rs;
  legacy /batch REMOVED, Django APPEND_SLASH=False) → every event 404'd and
  was dropped (postJSON logs, never wedges). Repoint to /v1/e; the
  {api_key,batch:[…]} body is unchanged — capture-rs's untagged
  RawRequest::Batch variant accepts it verbatim.

- umami [medium]: umamiPayload.DistinctID serialized as `distinctId`, a field
  absent from the fork's /api/send zod schema (src/app/api/send/route.ts) →
  silently stripped, so the forwarded visitor id never keyed the session
  (stitching fell back to IP+UA). Rename the JSON tag to `id`, the schema's
  real session key: sessionId = id ? uuid(website,id) : uuid(website,ip,ua,salt).

Tests now assert the real WIRE contract (a struct-field assert cannot catch a
JSON-tag bug — the source of the false confidence red flagged):
TestPostHogSendEndToEnd → /v1/e; TestUmamiBuild + TestUmamiSendEndToEnd → the
visitor id marshals under `id`, never `distinctId`.

go build + go vet + go test ./clients/destinations/... green.
2026-07-24 01:44:56 -07:00
antje 70dff81db5 refactor(share): login-first provisioning, fresh deterministic email, env base
Three hardening fixes from live testing against the zrok controller:
- LOGIN-FIRST: token(org, create) tries login before create — an existing
  account (the common path) costs ONE login and never touches admin; create
  only on a real miss. Slims the controller interface to token()+overview()
  (org-centric — the controller owns all credential derivation, the handler
  just passes org; DRY).
- FRESH EMAIL: zrok SOFT-deletes accounts (a deleted email stays in the unique
  index, un-recreatable forever), so a plain share-<org>@ that was ever touched
  is burned. accountEmail now carries an HMAC freshness suffix
  (share-<org>-<hash8>@hanzo.ai) — deterministic, unguessable, and distinct
  from any prior scheme, so no burn can strand an org.
- ENV BASE: ZROK_API_BASE (default /api/v2) makes the /api/v2 -> /v1 cutover a
  CR edit, not a rebuild — flip it the moment the /v1 zrok image deploys.
Builds + unit tests green.
2026-07-24 01:24:27 -07:00
antje 734c52b93f refactor(share): login-first provisioning, fresh deterministic email, env base
Three hardening fixes from live testing against the zrok controller:
- LOGIN-FIRST: token(org, create) tries login before create — an existing
  account (the common path) costs ONE login and never touches admin; create
  only on a real miss. Slims the controller interface to token()+overview()
  (org-centric — the controller owns all credential derivation, the handler
  just passes org; DRY).
- FRESH EMAIL: zrok SOFT-deletes accounts (a deleted email stays in the unique
  index, un-recreatable forever), so a plain share-<org>@ that was ever touched
  is burned. accountEmail now carries an HMAC freshness suffix
  (share-<org>-<hash8>@hanzo.ai) — deterministic, unguessable, and distinct
  from any prior scheme, so no burn can strand an org.
- ENV BASE: ZROK_API_BASE (default /api/v2) makes the /api/v2 -> /v1 cutover a
  CR edit, not a rebuild — flip it the moment the /v1 zrok image deploys.
Builds + unit tests green.
2026-07-24 01:24:27 -07:00
antje 7d5123dd3d chore(commerce): ship store access billing 2026-07-24 01:10:51 -07:00
antje 1e8ec3e11a chore(commerce): ship store access billing 2026-07-24 01:10:51 -07:00
antje 1dccca6b2f fix(share): per-org account email uses a hyphen, not plus
share-<org>@hanzo.ai. A plus local part trips some validators, and zrok
soft-deletes accounts (its unique email index then blocks a recreate of
the same address) — a hyphenated, stable identity sidesteps both. Verified
live against the controller: create 201 + login 200 for share-hanzo.
2026-07-24 00:57:25 -07:00
antje 7f9ba10b16 fix(share): per-org account email uses a hyphen, not plus
share-<org>@hanzo.ai. A plus local part trips some validators, and zrok
soft-deletes accounts (its unique email index then blocks a recreate of
the same address) — a hyphenated, stable identity sidesteps both. Verified
live against the controller: create 201 + login 200 for share-hanzo.
2026-07-24 00:57:25 -07:00
antje ea06f77e53 feat(destinations): add Umami + PostHog sinks and native ecommerce mapping
Fan the /v1 analytics event stream out to two first-party analytics sinks
alongside the existing ad platforms, and map schema.org/GA4 ecommerce actions
onto GA4 and Meta's native conversion schemas.

- umami.go: POST /api/send (Hanzo Analytics fork). Credential-less public
  beacon keyed by the non-secret website id; forwards the end-user UA+IP so
  Umami attributes session/geo. Pageviews sent without a name (Umami's rule).
- posthog.go: POST /batch (Hanzo Insights fork). Project api_key is the KMS
  secret, carried in the body (never URL/log); distinct_id keyed, empty dropped.
- fanout.go: resolveSecret treats a Secret-less, Fallback-less destination as
  credential-less (public ingest) instead of failing closed.
- translate.go: map product_viewed/product_added/begin_checkout/purchase (+
  schema.org aliases) onto the commerce standard events; lift ecommerce line
  items (items/products array, or a first-class product id) into Conversion.Items.
- ga4.go: render GA4 items[] + purchase transaction_id.
- meta.go: render Meta content_ids/contents/content_type/num_items + order_id.
- destination.go: normalized Item type + Conversion.Items/Referrer.
- send.go: shared non-PII first-party analytics data helper.

Secrets stay in KMS; no credential in any endpoint, error, or log line.
Tests: adapter payload + ecommerce mapping (mock HTTP), translate vocabulary +
item lift, credential-less resolution. go test ./clients/destinations/... green.
2026-07-24 00:52:49 -07:00
antje 5824068aa1 feat(destinations): add Umami + PostHog sinks and native ecommerce mapping
Fan the /v1 analytics event stream out to two first-party analytics sinks
alongside the existing ad platforms, and map schema.org/GA4 ecommerce actions
onto GA4 and Meta's native conversion schemas.

- umami.go: POST /api/send (Hanzo Analytics fork). Credential-less public
  beacon keyed by the non-secret website id; forwards the end-user UA+IP so
  Umami attributes session/geo. Pageviews sent without a name (Umami's rule).
- posthog.go: POST /batch (Hanzo Insights fork). Project api_key is the KMS
  secret, carried in the body (never URL/log); distinct_id keyed, empty dropped.
- fanout.go: resolveSecret treats a Secret-less, Fallback-less destination as
  credential-less (public ingest) instead of failing closed.
- translate.go: map product_viewed/product_added/begin_checkout/purchase (+
  schema.org aliases) onto the commerce standard events; lift ecommerce line
  items (items/products array, or a first-class product id) into Conversion.Items.
- ga4.go: render GA4 items[] + purchase transaction_id.
- meta.go: render Meta content_ids/contents/content_type/num_items + order_id.
- destination.go: normalized Item type + Conversion.Items/Referrer.
- send.go: shared non-PII first-party analytics data helper.

Secrets stay in KMS; no credential in any endpoint, error, or log line.
Tests: adapter payload + ecommerce mapping (mock HTTP), translate vocabulary +
item lift, credential-less resolution. go test ./clients/destinations/... green.
2026-07-24 00:52:49 -07:00
antje 6f371409c7 cloud auth: IAM-native trust — drop the per-app audience allowlist
Trust becomes exactly what IAM asserts: a valid SIGNATURE from a trusted ISSUER
(the brand set) plus EXPIRY. The audience (a minting app's client_id) is now
INFORMATIONAL, not an access gate — cloud no longer keeps a hand-maintained mirror
of IAM's app registry. That mirror was the lone non-IAM-native gate and it drifted:
every new first-party app (most recently hanzo-commerce, breaking the commerce-admin
AI assistant) silently 401'd until someone edited GATEWAY_ALLOWED_AUDIENCES. A new
first-party app now 'just works' with zero cloud change.

Removed: identityValidator.audiences, Config.JWTAudiences, jwtAudiencesFromEnv,
defaultJWTAudiences, BrandAudiences, unionStrings. validate() enforces issuer + expiry
via jwt.Expected{} (empty AnyAudience skips ONLY the audience match; go-jose still
checks exp/nbf against time.Now). Fail-secure on an empty ISSUER set is preserved.

PRESERVED (unchanged): owner-claim org scoping on every guard; SuperAdmin =
owner==adminOrg AND !isKMSMachinePrincipal; the KMS-machine SuperAdmin-denial
(isKMSMachinePrincipal reads claims.Audience directly, not the removed allowlist);
OrgHasUnsafeRune. Tests reframed to the new invariant and all green, incl. the KMS
red adversarial suite (multi-value aud, admin-slip, owner-bound machine-aud, trim/
unsafe owner). One new test proves any aud from a trusted issuer validates while a
wrong issuer / expired token still rejects.

The user-facing behavior change: a real admin (owner==adminOrg, isAdmin) is now
SuperAdmin from ANY first-party app, not only allowlisted ones — owner is the
authority, which is correct for an internal IAM where every app is first-party.
2026-07-24 00:32:47 -07:00
antje f429d7228d cloud auth: IAM-native trust — drop the per-app audience allowlist
Trust becomes exactly what IAM asserts: a valid SIGNATURE from a trusted ISSUER
(the brand set) plus EXPIRY. The audience (a minting app's client_id) is now
INFORMATIONAL, not an access gate — cloud no longer keeps a hand-maintained mirror
of IAM's app registry. That mirror was the lone non-IAM-native gate and it drifted:
every new first-party app (most recently hanzo-commerce, breaking the commerce-admin
AI assistant) silently 401'd until someone edited GATEWAY_ALLOWED_AUDIENCES. A new
first-party app now 'just works' with zero cloud change.

Removed: identityValidator.audiences, Config.JWTAudiences, jwtAudiencesFromEnv,
defaultJWTAudiences, BrandAudiences, unionStrings. validate() enforces issuer + expiry
via jwt.Expected{} (empty AnyAudience skips ONLY the audience match; go-jose still
checks exp/nbf against time.Now). Fail-secure on an empty ISSUER set is preserved.

PRESERVED (unchanged): owner-claim org scoping on every guard; SuperAdmin =
owner==adminOrg AND !isKMSMachinePrincipal; the KMS-machine SuperAdmin-denial
(isKMSMachinePrincipal reads claims.Audience directly, not the removed allowlist);
OrgHasUnsafeRune. Tests reframed to the new invariant and all green, incl. the KMS
red adversarial suite (multi-value aud, admin-slip, owner-bound machine-aud, trim/
unsafe owner). One new test proves any aud from a trusted issuer validates while a
wrong issuer / expired token still rejects.

The user-facing behavior change: a real admin (owner==adminOrg, isAdmin) is now
SuperAdmin from ANY first-party app, not only allowlisted ones — owner is the
authority, which is correct for an internal IAM where every app is first-party.
2026-07-24 00:32:47 -07:00
antje e6499ac02a fix(share): zrok API is /api/v2 + application/zrok.v1+json media type
The controller's go-swagger API mounts at basePath /api/v2 and consumes/
produces ONLY application/zrok.v1+json (application/json → 415/500, and a
v1 path → the SPA's 202 HTML). ensureAccount/login/overview now hit the
right base with the right content type. Verified: /api/v2/login returns
the account token 200.
2026-07-24 00:29:37 -07:00
antje c549074bb8 fix(share): zrok API is /api/v2 + application/zrok.v1+json media type
The controller's go-swagger API mounts at basePath /api/v2 and consumes/
produces ONLY application/zrok.v1+json (application/json → 415/500, and a
v1 path → the SPA's 202 HTML). ensureAccount/login/overview now hit the
right base with the right content type. Verified: /api/v2/login returns
the account token 200.
2026-07-24 00:29:37 -07:00
zeekayandClaude Fable 5 1eb2d288f0 build(deps): bump hanzoai/commerce v1.49.14 → v1.49.15 (billing webhooks live: X-Webhook-* delivery, lifecycle emission, bounded retry)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 00:18:45 -07:00
zeekayandhanzo-dev 3ecbcfe1c8 build(deps): bump hanzoai/commerce v1.49.14 → v1.49.15 (billing webhooks live: X-Webhook-* delivery, lifecycle emission, bounded retry)
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-24 00:18:45 -07:00
antje 438ff9641b feat(share): /v1/share/* — ngrok-native public sharing in the one cloud binary
Provisions a per-org zrok account from the caller's validated IAM identity
so the Rust CLI's `hanzo share <port>` publishes a local service to a public
https://<token>.share.hanzo.ai URL with zero manual setup.

- POST /v1/share/enable: stateless, deterministic per-org account (email
  share+<org>@hanzo.ai, password HMAC(secret,org)) — ensure-account + login
  reconstruct the same credential every time; the zrok controller IS the
  store, no local persistence. Returns {accountToken, controller, namespace,
  urlTemplate} for the tunnel client. Idempotent.
- GET /v1/share: the org's active shares (CLI + console Shares view), honest-
  empty when the controller is unreachable.
- Fail-closed 503 until ZROK_ADMIN_TOKEN; org from principal (never a client
  field) so a caller can only ever provision/read its OWN org. .Group routes,
  mockable controller client, unit-tested.
2026-07-24 00:10:33 -07:00
antje 803267b226 feat(share): /v1/share/* — ngrok-native public sharing in the one cloud binary
Provisions a per-org zrok account from the caller's validated IAM identity
so the Rust CLI's `hanzo share <port>` publishes a local service to a public
https://<token>.share.hanzo.ai URL with zero manual setup.

- POST /v1/share/enable: stateless, deterministic per-org account (email
  share+<org>@hanzo.ai, password HMAC(secret,org)) — ensure-account + login
  reconstruct the same credential every time; the zrok controller IS the
  store, no local persistence. Returns {accountToken, controller, namespace,
  urlTemplate} for the tunnel client. Idempotent.
- GET /v1/share: the org's active shares (CLI + console Shares view), honest-
  empty when the controller is unreachable.
- Fail-closed 503 until ZROK_ADMIN_TOKEN; org from principal (never a client
  field) so a caller can only ever provision/read its OWN org. .Group routes,
  mockable controller client, unit-tested.
2026-07-24 00:10:33 -07:00
zeekayandClaude Opus 4.8 83bd636f0f git: public explore/landing — OSS-first, no sign-in to browse
git.hanzo.ai (served by cloud's embedded git forge, clients/git) returned a raw
403 'sign in to view Hanzo Git' for signed-out visitors. Most Hanzo projects are
OSS, so the default face is now open, GitHub-style:

- uiExplore + /explore: lists every PUBLIC repo across ALL orgs, searchable (?q=),
  no auth. Cross-org via {DataDir}/orgs enumeration (per-org stores have no global
  index), capped at maxExploreScan.
- uiHome: signed out -> public explore/landing; signed in -> your org's repos.
- uiRepoAccess: repo/tree/blob/commits views serve PUBLIC repos anonymously;
  private repos stay org-authed and answer the SAME 404 (no existence leak),
  mirroring smart-HTTP's resolvePackRepo(allowPublic).
- Store.ListPublic: per-org public-repo query.
- state.dataDir threaded through Mount for the org enumeration.

Anonymous clone of public repos already worked (smart_http); this opens the
browser surface to match. Updated TestRootUI_HostGuard to the new public-landing
contract (/ -> 200 explore, missing repo -> native 404).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 15:17:57 -07:00
zeekayandhanzo-dev 031388bd1a git: public explore/landing — OSS-first, no sign-in to browse
git.hanzo.ai (served by cloud's embedded git forge, clients/git) returned a raw
403 'sign in to view Hanzo Git' for signed-out visitors. Most Hanzo projects are
OSS, so the default face is now open, GitHub-style:

- uiExplore + /explore: lists every PUBLIC repo across ALL orgs, searchable (?q=),
  no auth. Cross-org via {DataDir}/orgs enumeration (per-org stores have no global
  index), capped at maxExploreScan.
- uiHome: signed out -> public explore/landing; signed in -> your org's repos.
- uiRepoAccess: repo/tree/blob/commits views serve PUBLIC repos anonymously;
  private repos stay org-authed and answer the SAME 404 (no existence leak),
  mirroring smart-HTTP's resolvePackRepo(allowPublic).
- Store.ListPublic: per-org public-repo query.
- state.dataDir threaded through Mount for the org enumeration.

Anonymous clone of public repos already worked (smart_http); this opens the
browser surface to match. Updated TestRootUI_HostGuard to the new public-landing
contract (/ -> 200 explore, missing repo -> native 404).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-23 15:17:57 -07:00
hanzo-dev 5589d05bfd guide: fold the full Zen of Hanzo genome into a version-aware seed
Seed the DB-backed blueprint with the complete corpus — the 64-principle
spine plus 888 modern + 114 heritage strategies (1002 total), each filed
under a spine principle and tagged by era. base.yaml is the assembled
genome, kept byte-synced with the GitOps origin.

Schema: Blueprint.Principles (the 64 archetypes); Strategy gains
principle/source/era/blog. Validate enforces referential integrity — every
strategy's principle resolves to a spine slug, no duplicate id across the
1002 — plus unique principle slugs.

Version-aware seed (SeedOrUpgrade): a monotonic seedVersion is stamped on
the seed row (source="seed"); an admin write flips it to source="admin".
On Mount a brand with no row is seeded, an UNEDITED seed at an older
generation is upgraded in place, and a brand ever admin-edited is never
touched again. A self-healing backfill marks legacy version>=2 rows as
admin.

Fix two review LOWs:
- authoringBlueprint returns a clone of the embedded fixture in its
  pre-seed fallbacks, so an in-place PATCH can't corrupt the shared value.
- putBlueprint / listBlueprintVersions fetch the write key via
  LatestResolved without parsing, so a corrupt stored row is still
  replaceable by a valid PUT.

Tests: assembled seed parses+validates; /v1/guide/strategies surfaces the
888; a v2 seed upgrades an unedited v1 row and never clobbers an admin
edit; legacy backfill; the two LOW PoCs. Frozen wire green.
2026-07-23 13:28:33 -07:00
hanzo-dev d7f7077484 guide: fold the full Zen of Hanzo genome into a version-aware seed
Seed the DB-backed blueprint with the complete corpus — the 64-principle
spine plus 888 modern + 114 heritage strategies (1002 total), each filed
under a spine principle and tagged by era. base.yaml is the assembled
genome, kept byte-synced with the GitOps origin.

Schema: Blueprint.Principles (the 64 archetypes); Strategy gains
principle/source/era/blog. Validate enforces referential integrity — every
strategy's principle resolves to a spine slug, no duplicate id across the
1002 — plus unique principle slugs.

Version-aware seed (SeedOrUpgrade): a monotonic seedVersion is stamped on
the seed row (source="seed"); an admin write flips it to source="admin".
On Mount a brand with no row is seeded, an UNEDITED seed at an older
generation is upgraded in place, and a brand ever admin-edited is never
touched again. A self-healing backfill marks legacy version>=2 rows as
admin.

Fix two review LOWs:
- authoringBlueprint returns a clone of the embedded fixture in its
  pre-seed fallbacks, so an in-place PATCH can't corrupt the shared value.
- putBlueprint / listBlueprintVersions fetch the write key via
  LatestResolved without parsing, so a corrupt stored row is still
  replaceable by a valid PUT.

Tests: assembled seed parses+validates; /v1/guide/strategies surfaces the
888; a v2 seed upgrades an unedited v1 row and never clobbers an admin
edit; legacy backfill; the two LOW PoCs. Frozen wire green.
2026-07-23 13:28:33 -07:00
hanzo-dev 124a2a2ffb guide: DB-backed blueprint — seeded fixture, 3-tier resolution, SuperAdmin authoring, strategies corpus
Make the whole Guide™ playbook DB-backed and SuperAdmin-editable — nothing static.

Schema: extend the engine into a full Blueprint (Section/Strategy/Template + an
Enabled lever on every item, default-on). The engine stays pure — Blueprint.Curriculum()
projects the ENABLED journey (disabled section drops its steps; disabled step drops and
its dependents lose the edge), so reconcile/Next/Available/Counts are unchanged. Parse is
fail-closed; Validate enforces DAG-acyclic + no-dangling over the authored graph AND the
enabled projection.

Seed: embed base.yaml (12 sections · 67 steps · 114 strategies · 6 templates, a
byte-identical synced copy of the universe fixture) as the blueprint; supersede
default.yaml. On Mount, seed-if-absent into a shared versioned store (guide_blueprint,
brand-keyed) — a redeploy never clobbers admin edits.

Resolve (three tiers): org override (per-org) → brand blueprint (shared DB, seeded,
SuperAdmin-authored) → embedded fixture (fail-safe). A disabled/unparseable tier is
skipped.

Author: /v1/guide/blueprint/* gated on IsSuperAdmin (owner==admin) — a normal org member
or per-org admin gets 403. GET/PUT the whole blueprint, GET versions (PITR), PATCH an
item (edit + the enable/disable lever). Every write is validated fail-closed and appended
as a new version; edits take effect on the next resolve.

Filter: GET /v1/guide/strategies?category=&stage=&workload= — the ENABLED corpus,
org-scoped, joined to the observe layer (stage:* is a monotone readiness floor,
research==formed; has:* maps to module:*/connected:*/analytics/revenue/deployed/
funnel:signups; extend standardConnectors so the join is real).

Green: go build ./clients/guide/... ./apps/...; go vet; go test ./clients/guide/... (50
tests incl SuperAdmin-gate 403, idempotent-seed no-clobber, org>brand>fixture resolution,
disabled section/step drop, corpus filter). apps TestWireOrderMatchesFrozen green (routes
added to the existing guide subsystem).
2026-07-23 12:21:56 -07:00
hanzo-dev c5b69e2139 guide: DB-backed blueprint — seeded fixture, 3-tier resolution, SuperAdmin authoring, strategies corpus
Make the whole Guide™ playbook DB-backed and SuperAdmin-editable — nothing static.

Schema: extend the engine into a full Blueprint (Section/Strategy/Template + an
Enabled lever on every item, default-on). The engine stays pure — Blueprint.Curriculum()
projects the ENABLED journey (disabled section drops its steps; disabled step drops and
its dependents lose the edge), so reconcile/Next/Available/Counts are unchanged. Parse is
fail-closed; Validate enforces DAG-acyclic + no-dangling over the authored graph AND the
enabled projection.

Seed: embed base.yaml (12 sections · 67 steps · 114 strategies · 6 templates, a
byte-identical synced copy of the universe fixture) as the blueprint; supersede
default.yaml. On Mount, seed-if-absent into a shared versioned store (guide_blueprint,
brand-keyed) — a redeploy never clobbers admin edits.

Resolve (three tiers): org override (per-org) → brand blueprint (shared DB, seeded,
SuperAdmin-authored) → embedded fixture (fail-safe). A disabled/unparseable tier is
skipped.

Author: /v1/guide/blueprint/* gated on IsSuperAdmin (owner==admin) — a normal org member
or per-org admin gets 403. GET/PUT the whole blueprint, GET versions (PITR), PATCH an
item (edit + the enable/disable lever). Every write is validated fail-closed and appended
as a new version; edits take effect on the next resolve.

Filter: GET /v1/guide/strategies?category=&stage=&workload= — the ENABLED corpus,
org-scoped, joined to the observe layer (stage:* is a monotone readiness floor,
research==formed; has:* maps to module:*/connected:*/analytics/revenue/deployed/
funnel:signups; extend standardConnectors so the join is real).

Green: go build ./clients/guide/... ./apps/...; go vet; go test ./clients/guide/... (50
tests incl SuperAdmin-gate 403, idempotent-seed no-clobber, org>brand>fixture resolution,
disabled section/step drop, corpus filter). apps TestWireOrderMatchesFrozen green (routes
added to the existing guide subsystem).
2026-07-23 12:21:56 -07:00
hanzo-dev 7038607250 guide: real-time growth-observe layer — signals, stage classifier, /v1/guide/profile
Add the Guide's OBSERVE layer so the Business AI grounds on an org's REAL
platform truth, org-scoped and honest-degrading.

Growth signals extend the Detector vocabulary (detect.go stays THE coupling
point): module:<name>, connected:<provider>, funnel:<stage>, deployed, revenue,
customers:>=<N>. Each reads a sibling subsystem through an injected Signals seam
bound at the composition root (apps/wire_seams.go) — guide imports none of them,
the coding-dispatcher injected-function pattern. A nil seam field honest-degrades
its signal to not-present; the vocabulary is the contract, a read fills in when a
provably org-scoped seam lands. lookupDetector resolves parameterized <kind>:<param>
signals so a curriculum step can auto-detect on any of them.

classifyStage folds a SignalSet into a first-principles growth stage
(formed->launched->activated->scaling) — pure, total, exhaustively tested; the
ladder is data, driven by OUTCOMES (presence, engagement, money), not setup.

GET /v1/guide/profile exposes {stage, signals, keyMetrics}, org-scoped on the
validated principal, READ-ONLY, recomputed each request (real-time by pull) via the
existing reconcile path + the growth probes.

Bound reads (provably org-scoped, nil-safe): framework.ModuleInstalled (new,
module-granularity sibling of Installed), integrations.Connected (new, boolean,
never the token), the in-package analytics funnel. deployed/revenue/customers stay
unbound seams (deploy is cluster-scoped; commerce/crm have no clean per-org
in-process read) — honest-degrading until a provably org-scoped read exists.

Tenant isolation: every signal read keys on the caller's own org; TestHTTPProfile
CrossTenantIsolation proves org A's profile never reflects org B's signals and each
request's probes read only the caller's org. framework/integrations seam reads have
their own org-scoping tests.
2026-07-23 10:13:15 -07:00
hanzo-dev a0dbfc75a0 guide: real-time growth-observe layer — signals, stage classifier, /v1/guide/profile
Add the Guide's OBSERVE layer so the Business AI grounds on an org's REAL
platform truth, org-scoped and honest-degrading.

Growth signals extend the Detector vocabulary (detect.go stays THE coupling
point): module:<name>, connected:<provider>, funnel:<stage>, deployed, revenue,
customers:>=<N>. Each reads a sibling subsystem through an injected Signals seam
bound at the composition root (apps/wire_seams.go) — guide imports none of them,
the coding-dispatcher injected-function pattern. A nil seam field honest-degrades
its signal to not-present; the vocabulary is the contract, a read fills in when a
provably org-scoped seam lands. lookupDetector resolves parameterized <kind>:<param>
signals so a curriculum step can auto-detect on any of them.

classifyStage folds a SignalSet into a first-principles growth stage
(formed->launched->activated->scaling) — pure, total, exhaustively tested; the
ladder is data, driven by OUTCOMES (presence, engagement, money), not setup.

GET /v1/guide/profile exposes {stage, signals, keyMetrics}, org-scoped on the
validated principal, READ-ONLY, recomputed each request (real-time by pull) via the
existing reconcile path + the growth probes.

Bound reads (provably org-scoped, nil-safe): framework.ModuleInstalled (new,
module-granularity sibling of Installed), integrations.Connected (new, boolean,
never the token), the in-package analytics funnel. deployed/revenue/customers stay
unbound seams (deploy is cluster-scoped; commerce/crm have no clean per-org
in-process read) — honest-degrading until a provably org-scoped read exists.

Tenant isolation: every signal read keys on the caller's own org; TestHTTPProfile
CrossTenantIsolation proves org A's profile never reflects org B's signals and each
request's probes read only the caller's org. framework/integrations seam reads have
their own org-scoping tests.
2026-07-23 10:13:15 -07:00
hanzo-dev 83b59c509b research: gate HA object-store durability behind CLOUD_RESEARCH_DURABLE opt-in
The unified store's durability plane auto-activated whenever S3_ADMIN_* was
configured (prod sets it), so deploying it would begin fencing real tenant
data on the object store's conditional-PUT (If-Match) atomicity before that
atomicity is validated against the deployed SeaweedFS version (Red's takeover-
fence staging gate H2). Gate it behind CLOUD_RESEARCH_DURABLE (off by default):
the store runs local-only until the flag is set, and the shard router already
pins each org to one writer (ha.Owner) — so this is NOT the rolling-deploy
outage, and the DDL-drift-proof + evidence-preserving record layer is fully
active. The object-store snapshot/fence turns on deliberately after the gate.
2026-07-23 09:41:46 -07:00
hanzo-dev dba4666173 research: gate HA object-store durability behind CLOUD_RESEARCH_DURABLE opt-in
The unified store's durability plane auto-activated whenever S3_ADMIN_* was
configured (prod sets it), so deploying it would begin fencing real tenant
data on the object store's conditional-PUT (If-Match) atomicity before that
atomicity is validated against the deployed SeaweedFS version (Red's takeover-
fence staging gate H2). Gate it behind CLOUD_RESEARCH_DURABLE (off by default):
the store runs local-only until the flag is set, and the shard router already
pins each org to one writer (ha.Owner) — so this is NOT the rolling-deploy
outage, and the DDL-drift-proof + evidence-preserving record layer is fully
active. The object-store snapshot/fence turns on deliberately after the gate.
2026-07-23 09:41:46 -07:00
hanzo-dev 803db8811c test(framework): guard stampFixture clone — fail if DocField/DocPerm gain a non-scalar field (red LOW-1: shallow-clone would corrupt the always-on registry cross-org) 2026-07-23 07:40:15 -07:00
hanzo-dev df1de096ff test(framework): guard stampFixture clone — fail if DocField/DocPerm gain a non-scalar field (red LOW-1: shallow-clone would corrupt the always-on registry cross-org) 2026-07-23 07:40:15 -07:00
hanzo-dev 75395adfa4 guide: always-on standard modules + AI suggest + per-brand journeys
Complete the stranded Guide feature so a fresh org's journey works with no
per-org module install.

framework: MarkAlwaysOn/AlwaysOnModules mark a registered module always-on —
its DocType fixtures resolve for every org via GetDocType/ListDocTypes and the
Installed predicate, with no per-org install row. This defaults only the SCHEMA
on; every document row stays physically org-scoped. A per-org stored DocType
still overrides the fixture. always_on_isolation_test.go proves org A's records
are unreadable by org B even though the DocType resolves for both.

content: marketing is always-on (doctypes.go init), so content_generate
completes for a fresh org — the fix for "content: marketing module not installed
for org". Drop the now-unreachable not-installed guard in EnsureCatalogAsset (a
fresh org without a studio still quietly skips at Generate).

guide: builtin-2 journey rooted at "Form your company", then positioning →
launch steps and the daily agentic growth loop; positioning gates on company.
Add /v1/guide/suggest + /v1/guide/chat — read-only AI "what to do next", grounded
in the org's real progress + funnel, never running an action. Resolve the
white-label brand journey (brands/<brand>.yaml; zoo ships zoo-1) at Mount.

cek: TestMain seeds an ephemeral master key so the encrypted-at-rest stores open
on an encryption-capable test build (framework, content, guide) — mirrors
compliance/integrations/flags/venue.
2026-07-23 07:37:19 -07:00
hanzo-dev 84281e3ddc guide: always-on standard modules + AI suggest + per-brand journeys
Complete the stranded Guide feature so a fresh org's journey works with no
per-org module install.

framework: MarkAlwaysOn/AlwaysOnModules mark a registered module always-on —
its DocType fixtures resolve for every org via GetDocType/ListDocTypes and the
Installed predicate, with no per-org install row. This defaults only the SCHEMA
on; every document row stays physically org-scoped. A per-org stored DocType
still overrides the fixture. always_on_isolation_test.go proves org A's records
are unreadable by org B even though the DocType resolves for both.

content: marketing is always-on (doctypes.go init), so content_generate
completes for a fresh org — the fix for "content: marketing module not installed
for org". Drop the now-unreachable not-installed guard in EnsureCatalogAsset (a
fresh org without a studio still quietly skips at Generate).

guide: builtin-2 journey rooted at "Form your company", then positioning →
launch steps and the daily agentic growth loop; positioning gates on company.
Add /v1/guide/suggest + /v1/guide/chat — read-only AI "what to do next", grounded
in the org's real progress + funnel, never running an action. Resolve the
white-label brand journey (brands/<brand>.yaml; zoo ships zoo-1) at Mount.

cek: TestMain seeds an ephemeral master key so the encrypted-at-rest stores open
on an encryption-capable test build (framework, content, guide) — mirrors
compliance/integrations/flags/venue.
2026-07-23 07:37:19 -07:00
antje 7a236a2c3a agents: CLI-facing control drain for locally-started sessions
The dashboard already POSTs pause/resume/stop/message, recorded as durable
KindControl events. A cloud-dispatched routed run gets them forwarded to the
tasks engine, but a LOCALLY-started `hanzo code` session is not task-backed —
so the running surface must pull them itself.

Add GET /v1/agents/sessions/:id/control?after=<seq>: an owner-scoped, cursor-
driven drain of a session's control commands (ListControlAfter filters to
control, oldest first). Read-only, org is the only tenant key, foreign id 404s.
Test covers filter/order/cursor/tenant-isolation.
2026-07-23 07:01:31 -07:00
antje e2cf2e978b agents: CLI-facing control drain for locally-started sessions
The dashboard already POSTs pause/resume/stop/message, recorded as durable
KindControl events. A cloud-dispatched routed run gets them forwarded to the
tasks engine, but a LOCALLY-started `hanzo code` session is not task-backed —
so the running surface must pull them itself.

Add GET /v1/agents/sessions/:id/control?after=<seq>: an owner-scoped, cursor-
driven drain of a session's control commands (ListControlAfter filters to
control, oldest first). Read-only, org is the only tenant key, foreign id 404s.
Test covers filter/order/cursor/tenant-isolation.
2026-07-23 07:01:31 -07:00
hanzo-dev 4941e0839e Merge branch 'blue/research-orm' into blue/research-unified
Unify the research record layer (hanzoai/orm typed records, DDL-drift-proof
+ migrateLegacy evidence carry-forward) onto the HA file-durability layer
(ha-elected single-writer + fenced ship-before-ack). Reconcile the three
files both branches touch:
- store.go: ORM's typed-orm rewrite — supersedes the raw-SQL ALTER-migrate
  path (schema migration is structurally impossible under orm).
- migrate_test.go: ORM's legacy-DATA migration test (ALTER-convergence is
  obsolete under orm).
- research.go: both — HA's durability wiring (WithDurable + shipDurable
  ship-before-ack in every write handler) plus ORM's postGrant scope comment.

deps: orm v0.6.1 -> v0.6.7 (CreateIfAbsent typed-record ingest the orm store
needs) and commerce v1.49.13 -> v1.49.14 (DatastoreAdapter satisfies orm
v0.6.7's orm.DB CreateIfAbsent, keeping ./apps/ building).
2026-07-23 04:38:14 -07:00
hanzo-dev d5c1a5b676 Merge branch 'blue/research-orm' into blue/research-unified
Unify the research record layer (hanzoai/orm typed records, DDL-drift-proof
+ migrateLegacy evidence carry-forward) onto the HA file-durability layer
(ha-elected single-writer + fenced ship-before-ack). Reconcile the three
files both branches touch:
- store.go: ORM's typed-orm rewrite — supersedes the raw-SQL ALTER-migrate
  path (schema migration is structurally impossible under orm).
- migrate_test.go: ORM's legacy-DATA migration test (ALTER-convergence is
  obsolete under orm).
- research.go: both — HA's durability wiring (WithDurable + shipDurable
  ship-before-ack in every write handler) plus ORM's postGrant scope comment.

deps: orm v0.6.1 -> v0.6.7 (CreateIfAbsent typed-record ingest the orm store
needs) and commerce v1.49.13 -> v1.49.14 (DatastoreAdapter satisfies orm
v0.6.7's orm.DB CreateIfAbsent, keeping ./apps/ building).
2026-07-23 04:38:14 -07:00
hanzo-dev ac31716ce4 Merge branch 'blue/cloud-ha-sqlite' into blue/research-unified 2026-07-23 04:13:09 -07:00
hanzo-dev eed81e6c8e Merge branch 'blue/cloud-ha-sqlite' into blue/research-unified 2026-07-23 04:13:09 -07:00
hanzo-dev 0b1ec329ff test(durable): gate the encrypted takeover proof on a real cek round-trip (R1)
TestRecordShipsSoTakeoverKeepsIt reopens the shipped store through cek. On a CGO
build whose SQLite lacks SQLCipher, PRAGMA key is silently ignored so cek writes a
plaintext file it then cannot migrate on reopen (sqlcipher_export absent) — and the
suite TestMain still injects a dev key because sqlitedrv.EncryptionAvailable() reports
a false-positive yes, so the test hard-failed under the default CGO build.

cekCanReopen probes the actual round-trip: it holds under pure-Go (plaintext) and
real libsqlcipher (encrypted, exercising the .dek sidecar cross-pod restore), and
skips only on the broken-capability build. go test ./... stays green everywhere;
ship-before-return is proven under CGO_ENABLED=0 and by TestRecordOnNonOwnerFailsClosed.
2026-07-23 04:01:16 -07:00
hanzo-dev 51eb4f281c test(durable): gate the encrypted takeover proof on a real cek round-trip (R1)
TestRecordShipsSoTakeoverKeepsIt reopens the shipped store through cek. On a CGO
build whose SQLite lacks SQLCipher, PRAGMA key is silently ignored so cek writes a
plaintext file it then cannot migrate on reopen (sqlcipher_export absent) — and the
suite TestMain still injects a dev key because sqlitedrv.EncryptionAvailable() reports
a false-positive yes, so the test hard-failed under the default CGO build.

cekCanReopen probes the actual round-trip: it holds under pure-Go (plaintext) and
real libsqlcipher (encrypted, exercising the .dek sidecar cross-pod restore), and
skips only on the broken-capability build. go test ./... stays green everywhere;
ship-before-return is proven under CGO_ENABLED=0 and by TestRecordOnNonOwnerFailsClosed.
2026-07-23 04:01:16 -07:00
hanzo-dev 362515eead durable: loud gate when HA is disabled on multi-replica (L2) + document M3/M4/M5
Red L2: durability disabled now routes through disabledDurability(), which logs at
ERROR on a MULTI-REPLICA deployment (>1 CLOUD_PEERS) — per-org stores then survive
only via shard routing + per-pod RWO PVC, so a lost PVC loses data — and at INFO for
single-replica/dev where local-only is expected. The encryption-capable-build-without-
cipher case is treated as a misconfig (fail closed + loud), never a silent plaintext
ship or a silent drop back to the non-HA outage.

Red M3: documented degraded-open recovery — a degraded pod stays read-only for the
cached store's life (in-place re-acquire would CarryForward-restore under the live
handle = stale reads); recovery is a fresh open (pod restart / shard reroute), with
quiesce-close-reopen as the future enhancement.

Red M4/M5: documented the SeaweedFS operational requirements the fence rests on —
object versioning + no-expiry lifecycle on the org-db prefix (lease round is the
system of record; a dropped/rolled-back lease can un-fence a zombie), and RWO
per-writer PVCs for DataDir, never RWX.
2026-07-23 03:40:32 -07:00
hanzo-dev 945b2d5aba durable: loud gate when HA is disabled on multi-replica (L2) + document M3/M4/M5
Red L2: durability disabled now routes through disabledDurability(), which logs at
ERROR on a MULTI-REPLICA deployment (>1 CLOUD_PEERS) — per-org stores then survive
only via shard routing + per-pod RWO PVC, so a lost PVC loses data — and at INFO for
single-replica/dev where local-only is expected. The encryption-capable-build-without-
cipher case is treated as a misconfig (fail closed + loud), never a silent plaintext
ship or a silent drop back to the non-HA outage.

Red M3: documented degraded-open recovery — a degraded pod stays read-only for the
cached store's life (in-place re-acquire would CarryForward-restore under the live
handle = stale reads); recovery is a fresh open (pod restart / shard reroute), with
quiesce-close-reopen as the future enhancement.

Red M4/M5: documented the SeaweedFS operational requirements the fence rests on —
object versioning + no-expiry lifecycle on the org-db prefix (lease round is the
system of record; a dropped/rolled-back lease can un-fence a zombie), and RWO
per-writer PVCs for DataDir, never RWX.
2026-07-23 03:40:32 -07:00
hanzo-dev c6a121f096 perf(research): one-time marker so a restart skips the legacy re-scan
The legacy migration was idempotent but re-read the raw-SQL tables on every open. A
completion marker (a kindMeta record under a distinct id from the seq clock, written
in the SAME tx as the migration so it commits iff the migration commits) short-circuits
every later open. Greenfield stores mark done immediately. Test asserts the marker is
set post-migration and that a re-open stays correct (no duplication).
2026-07-23 03:39:04 -07:00
hanzo-dev 5b5c95fea9 perf(research): one-time marker so a restart skips the legacy re-scan
The legacy migration was idempotent but re-read the raw-SQL tables on every open. A
completion marker (a kindMeta record under a distinct id from the seq clock, written
in the SAME tx as the migration so it commits iff the migration commits) short-circuits
every later open. Greenfield stores mark done immediately. Test asserts the marker is
set post-migration and that a re-open stays correct (no duplication).
2026-07-23 03:39:04 -07:00
hanzo-dev 1b54ba50d5 test(durable): H2 staging gate — SeaweedFS If-Match/If-None-Match atomicity
The single-writer fence rests on the gateway evaluating If-None-Match:* (create-only)
and If-Match (version-conditioned) preconditions ATOMICALLY server-side — the one
property the in-process fakes cannot prove. This env-gated integration test races 12
concurrent writers against the REAL S3ConditionalStore and asserts EXACTLY ONE wins
each precondition; two winners = split-brain possible, do not ship durability against
that gateway. Skipped in unit runs; run in staging with CLOUD_DURABLE_IT=1 + S3_ADMIN_*
before durability fences real tenant data (or on a SeaweedFS version bump).
2026-07-23 03:38:20 -07:00
hanzo-dev 41c2b3edb2 test(durable): H2 staging gate — SeaweedFS If-Match/If-None-Match atomicity
The single-writer fence rests on the gateway evaluating If-None-Match:* (create-only)
and If-Match (version-conditioned) preconditions ATOMICALLY server-side — the one
property the in-process fakes cannot prove. This env-gated integration test races 12
concurrent writers against the REAL S3ConditionalStore and asserts EXACTLY ONE wins
each precondition; two winners = split-brain possible, do not ship durability against
that gateway. Skipped in unit runs; run in staging with CLOUD_DURABLE_IT=1 + S3_ADMIN_*
before durability fences real tenant data (or on a SeaweedFS version bump).
2026-07-23 03:38:20 -07:00
hanzo-dev 4d5083e7b6 durable: never hold the store lock across object-store I/O + timeouts + overflow guard
Red M1: forPath held the store-wide c.mu through openDurable→Hydrate (Acquire +
CarryForward round-trips, untimed) — a hung SeaweedFS froze the whole subsystem, a
cache-hit For() included. Now the durable open runs with c.mu RELEASED, deduped by
an in-flight record so concurrent For() for one org opens exactly once (cek cannot
open a file twice); every object-store round-trip (hydrate, Sync, close) is bounded
by durableOpTimeout (30s) so a slow store degrades to bounded latency, never a
deadlock. The local-only path is unchanged (disk I/O under c.mu as before).

Red L3: CloseAll ships each final state with c.mu released and time-bounded (was a
background-ctx ship under the lock) — Durable.Close takes a ctx now.

Red L1: unframe checked m+sl > len(b), which a max-uint64 length wraps past, then
panics the slice. Now checked as sl > len(b)-m (subtraction, no wrap) → fails closed.

Tests: TestDurableForDedupsConcurrentOpens (8 concurrent For → one store), the L1
oversized-length frame fails closed, all durable + research + root OrgStore green.
2026-07-23 03:35:24 -07:00
hanzo-dev 8938a22cde durable: never hold the store lock across object-store I/O + timeouts + overflow guard
Red M1: forPath held the store-wide c.mu through openDurable→Hydrate (Acquire +
CarryForward round-trips, untimed) — a hung SeaweedFS froze the whole subsystem, a
cache-hit For() included. Now the durable open runs with c.mu RELEASED, deduped by
an in-flight record so concurrent For() for one org opens exactly once (cek cannot
open a file twice); every object-store round-trip (hydrate, Sync, close) is bounded
by durableOpTimeout (30s) so a slow store degrades to bounded latency, never a
deadlock. The local-only path is unchanged (disk I/O under c.mu as before).

Red L3: CloseAll ships each final state with c.mu released and time-bounded (was a
background-ctx ship under the lock) — Durable.Close takes a ctx now.

Red L1: unframe checked m+sl > len(b), which a max-uint64 length wraps past, then
panics the slice. Now checked as sl > len(b)-m (subtraction, no wrap) → fails closed.

Tests: TestDurableForDedupsConcurrentOpens (8 concurrent For → one store), the L1
oversized-length frame fails closed, all durable + research + root OrgStore green.
2026-07-23 03:35:24 -07:00
hanzo-dev a182ed755d fix(research): carry pre-orm raw-SQL evidence forward on open (was orphaning it)
CRITICAL (red): the orm store read only _entities, so an existing per-org file whose
evidence lived in the old raw-SQL experiment/attempt/artifact tables read as EMPTY —
every logged run silently vanished the instant orm shipped (data intact in-file, but
all reads returned 0). migrateLegacy now runs in openStore: gated on old-table
existence (greenfield skips), idempotent via CreateIfAbsent, fail-secure (a migration
error fails the open rather than serving empty over real data).

- Each version keeps its OLD seq value → canonical/supersession preserved exactly (a
  corrected run stays canonical); the append clock is advanced past every migrated seq
  so a later ingest still supersedes (proven with a ts=0 correction across the boundary).
- Rows read by COLUMN NAME (SELECT *) → a table left by any older schema (even the
  outage-era one missing provenance columns) migrates without a 'no such column'.
- content_hash/revision/status/visibility/consent/provenance + artifact blobs verbatim.
- Regression TestLegacyDataMigration: seed old tables incl a correction pair → open →
  assert pre-migration truth (corrected canonical, retained intact, artifact bytes),
  a later ingest supersedes, an unrelated ingest doesn't flip it, re-open is a no-op.

Also (red): setArtifactVisibility now does its read-modify-write in a transaction like
setGrant (LOW); a scale note at the loaders documents the in-memory-fold tradeoff +
indexed-Filter escalation (MEDIUM, org-bounded); postGrant's client-supplied project
is confirmed intentional (org is the tenant boundary, project an org-internal target
label) with a clarifying comment (LOW).
2026-07-23 03:33:45 -07:00
hanzo-dev 1bd58f0fc0 fix(research): carry pre-orm raw-SQL evidence forward on open (was orphaning it)
CRITICAL (red): the orm store read only _entities, so an existing per-org file whose
evidence lived in the old raw-SQL experiment/attempt/artifact tables read as EMPTY —
every logged run silently vanished the instant orm shipped (data intact in-file, but
all reads returned 0). migrateLegacy now runs in openStore: gated on old-table
existence (greenfield skips), idempotent via CreateIfAbsent, fail-secure (a migration
error fails the open rather than serving empty over real data).

- Each version keeps its OLD seq value → canonical/supersession preserved exactly (a
  corrected run stays canonical); the append clock is advanced past every migrated seq
  so a later ingest still supersedes (proven with a ts=0 correction across the boundary).
- Rows read by COLUMN NAME (SELECT *) → a table left by any older schema (even the
  outage-era one missing provenance columns) migrates without a 'no such column'.
- content_hash/revision/status/visibility/consent/provenance + artifact blobs verbatim.
- Regression TestLegacyDataMigration: seed old tables incl a correction pair → open →
  assert pre-migration truth (corrected canonical, retained intact, artifact bytes),
  a later ingest supersedes, an unrelated ingest doesn't flip it, re-open is a no-op.

Also (red): setArtifactVisibility now does its read-modify-write in a transaction like
setGrant (LOW); a scale note at the loaders documents the in-memory-fold tradeoff +
indexed-Filter escalation (MEDIUM, org-bounded); postGrant's client-supplied project
is confirmed intentional (org is the tenant boundary, project an org-internal target
label) with a clarifying comment (LOW).
2026-07-23 03:33:45 -07:00
hanzo-dev 174e7bcd17 durable: fix live-path lost-write — Record ships (H1), restore dir-order, checkpoint result (M2)
Red H1: research.Record (in-process A/B evidence, experiments.go:425) committed
locally but NEVER shipped — an acked write lost on takeover, and unfenced on a
non-owner. Record now Syncs before returning and propagates a not-acked ship as an
error (mirrors the HTTP shipDurable), so a takeover keeps it and a non-owner fails
closed instead of persisting a stale divergent local copy.

Writing the H1 takeover regression surfaced a real restore bug: restore() wrote the
.dek key sidecar BEFORE RestoreFile created the parent dir, so a fresh successor
(orgs/<slug>/ absent) failed the sidecar write → hydrate degraded → EMPTY store =
the lost write. RestoreFile (which MkdirAll's) now runs first, then the sidecar.
The flat-tempdir unit test masked it; added TestDurableRestoreIntoFreshNestedDir.

Red M2: wal_checkpoint(TRUNCATE) result was discarded — busy!=0 ships a partial
snapshot (committed frames still in WAL) as acked = silent lost write. Now
QueryRow'd; fail closed on busy!=0.

Tests: TestRecordShipsSoTakeoverKeepsIt (takeover keeps the evidence, real cek
key-sidecar cross-pod restore), TestRecordOnNonOwnerFailsClosed, and the nested-dir
restore regression — all green.
2026-07-23 03:30:20 -07:00
hanzo-dev 7140168680 durable: fix live-path lost-write — Record ships (H1), restore dir-order, checkpoint result (M2)
Red H1: research.Record (in-process A/B evidence, experiments.go:425) committed
locally but NEVER shipped — an acked write lost on takeover, and unfenced on a
non-owner. Record now Syncs before returning and propagates a not-acked ship as an
error (mirrors the HTTP shipDurable), so a takeover keeps it and a non-owner fails
closed instead of persisting a stale divergent local copy.

Writing the H1 takeover regression surfaced a real restore bug: restore() wrote the
.dek key sidecar BEFORE RestoreFile created the parent dir, so a fresh successor
(orgs/<slug>/ absent) failed the sidecar write → hydrate degraded → EMPTY store =
the lost write. RestoreFile (which MkdirAll's) now runs first, then the sidecar.
The flat-tempdir unit test masked it; added TestDurableRestoreIntoFreshNestedDir.

Red M2: wal_checkpoint(TRUNCATE) result was discarded — busy!=0 ships a partial
snapshot (committed frames still in WAL) as acked = silent lost write. Now
QueryRow'd; fail closed on busy!=0.

Tests: TestRecordShipsSoTakeoverKeepsIt (takeover keeps the evidence, real cek
key-sidecar cross-pod restore), TestRecordOnNonOwnerFailsClosed, and the nested-dir
restore regression — all green.
2026-07-23 03:30:20 -07:00
hanzo-dev b0222e0c28 compliance,legal: close the 2 red LOWs — role-gate decideAccreditation; catch {{index . "k"}}/{{$.k}} undeclared-field refs 2026-07-23 03:10:46 -07:00
hanzo-dev dc48ee02c5 compliance,legal: close the 2 red LOWs — role-gate decideAccreditation; catch {{index . "k"}}/{{$.k}} undeclared-field refs 2026-07-23 03:10:46 -07:00
hanzo-dev 8293e9de23 Merge remote-tracking branch 'origin/main' into blue/cloud-adopt-provision 2026-07-23 03:04:14 -07:00
hanzo-dev 5ee3853c2d Merge remote-tracking branch 'origin/main' into blue/cloud-adopt-provision 2026-07-23 03:04:14 -07:00
hanzo-dev af16ada769 test(apps): add destinations frozen row — wire golden was latent-red on main (Wire mounted it, frozen omitted it) 2026-07-23 03:02:04 -07:00
hanzo-dev fe63e9bb4e test(apps): add destinations frozen row — wire golden was latent-red on main (Wire mounted it, frozen omitted it) 2026-07-23 03:02:04 -07:00
hanzo-dev d7e579ff79 test(research): concurrent-ingest seq monotonicity under -race
24 concurrent ingests of distinct content must all land (no lost write, no dup) and
the server-assigned append clock (seq) must be unique + gapless in [1,N] — the
concurrency proof for the per-store seq that replaced SQL AUTOINCREMENT, backed by
OrgDB single-writer + orm writeMu + the ingest tx.
2026-07-23 02:59:34 -07:00
hanzo-dev 5770ae8720 test(research): concurrent-ingest seq monotonicity under -race
24 concurrent ingests of distinct content must all land (no lost write, no dup) and
the server-assigned append clock (seq) must be unique + gapless in [1,N] — the
concurrency proof for the per-store seq that replaced SQL AUTOINCREMENT, backed by
OrgDB single-writer + orm writeMu + the ingest tx.
2026-07-23 02:59:34 -07:00
hanzo-dev ec764e44f9 company/compliance/idv/legal: decomplect the KYC/verification decision from the callback
A KYC/verification terminal status is now reached by three orthogonal paths, never a
client-asserted status:

- provider reconcile (pull): company kyc/refresh + compliance verifications/:id/refresh
  consult the wired provider for the settled status; Manual stays pending.
- provider webhook (push): compliance verifications/webhook authenticates by HMAC
  signature (idv.Webhook, KMS-sealed secret, disabled by default) and reconciles the
  referenced check from the provider API, so the request body cannot dictate a status.
- reviewer decision: company kyc/decision (a platform reviewer) and compliance
  verifications/:id/decision (an org admin or platform reviewer) are role-gated and
  attributed (DecidedBy = the acting user), and produce a DISTINCT reviewer_confirmed,
  never a provider_verified.

guardKYCVerified and the compliance decision accept only an attributed provider pass or
reviewer_confirmed. A founder/check records who decided; an unattributed "verified"
fails closed. startKYC clamps a provider's inquiry-time status to pending.

legal: the counsel-review notice is coupled to the category — formation and equity
templates always carry it, forced on override and emitted at render — and an override
body may reference only declared fields.

compliance createAccreditation records only an asserted state; a provider_verified
state routes through the attributed decision endpoint.

Remove the committed cek key sidecars and gitignore the pattern; tests seal their
stores under t.TempDir().
2026-07-23 02:57:21 -07:00
hanzo-dev b5f1e2dcad company/compliance/idv/legal: decomplect the KYC/verification decision from the callback
A KYC/verification terminal status is now reached by three orthogonal paths, never a
client-asserted status:

- provider reconcile (pull): company kyc/refresh + compliance verifications/:id/refresh
  consult the wired provider for the settled status; Manual stays pending.
- provider webhook (push): compliance verifications/webhook authenticates by HMAC
  signature (idv.Webhook, KMS-sealed secret, disabled by default) and reconciles the
  referenced check from the provider API, so the request body cannot dictate a status.
- reviewer decision: company kyc/decision (a platform reviewer) and compliance
  verifications/:id/decision (an org admin or platform reviewer) are role-gated and
  attributed (DecidedBy = the acting user), and produce a DISTINCT reviewer_confirmed,
  never a provider_verified.

guardKYCVerified and the compliance decision accept only an attributed provider pass or
reviewer_confirmed. A founder/check records who decided; an unattributed "verified"
fails closed. startKYC clamps a provider's inquiry-time status to pending.

legal: the counsel-review notice is coupled to the category — formation and equity
templates always carry it, forced on override and emitted at render — and an override
body may reference only declared fields.

compliance createAccreditation records only an asserted state; a provider_verified
state routes through the attributed decision endpoint.

Remove the committed cek key sidecars and gitignore the pattern; tests seal their
stores under t.TempDir().
2026-07-23 02:57:21 -07:00
hanzo-dev 6cb979bbb9 feat(compliance,legal): corporate back-office — KYC/KYB orchestration + legal template engine
Hanzo Compliance (/v1/compliance): org-scoped KYC/KYB verification through a
provider-agnostic seam, accreditation-state tracking, and a compliance-scoped read
of the shared tamper-evident audit plane (SOC 2 posture). Subject PII is sealed at
rest (cek) and referenced by opaque id everywhere else — never in logs, audit, or
URLs. No path yields a verified status on create; a terminal decision comes only
from the provider (refresh) or an authenticated, audited callback.

Hanzo Legal (/v1/legal): a versioned, org-overridable standardized template library
plus a pure, deterministic merge-field generation engine, a sealed document store,
and e-sign + filing seams. The counsel-review notice is a non-droppable invariant on
formation and securities documents.

clients/idv: the ONE identity/business verification seam — honest Manual default,
config-driven Persona/Onfido/Stripe adapter with a KMS-sealed key and a strict,
fail-closed status classifier. Consumed by BOTH compliance onboarding KYB and company
formation KYC; company now resolves a real provider from config (fail-closed),
replacing the manual-only default.

Boundary invariant, enforced in the data model and on the wire: platform tooling with
licensed providers and professionals in the loop — provider-reported or tracked states
only, never a platform assertion of "compliant" or "legally valid".
2026-07-23 02:57:21 -07:00
hanzo-dev 874c328ca5 feat(compliance,legal): corporate back-office — KYC/KYB orchestration + legal template engine
Hanzo Compliance (/v1/compliance): org-scoped KYC/KYB verification through a
provider-agnostic seam, accreditation-state tracking, and a compliance-scoped read
of the shared tamper-evident audit plane (SOC 2 posture). Subject PII is sealed at
rest (cek) and referenced by opaque id everywhere else — never in logs, audit, or
URLs. No path yields a verified status on create; a terminal decision comes only
from the provider (refresh) or an authenticated, audited callback.

Hanzo Legal (/v1/legal): a versioned, org-overridable standardized template library
plus a pure, deterministic merge-field generation engine, a sealed document store,
and e-sign + filing seams. The counsel-review notice is a non-droppable invariant on
formation and securities documents.

clients/idv: the ONE identity/business verification seam — honest Manual default,
config-driven Persona/Onfido/Stripe adapter with a KMS-sealed key and a strict,
fail-closed status classifier. Consumed by BOTH compliance onboarding KYB and company
formation KYC; company now resolves a real provider from config (fail-closed),
replacing the manual-only default.

Boundary invariant, enforced in the data model and on the wire: platform tooling with
licensed providers and professionals in the loop — provider-reported or tracked states
only, never a platform assertion of "compliant" or "legally valid".
2026-07-23 02:57:21 -07:00
hanzo-dev f822b772a4 test(research): give App.Test the repo-standard 30s timeout, not the 1s default
The research HTTP tests used the bare app.Fiber().Test(req) whose default client
timeout (1s) is too tight for a cold cek-encrypted per-org SQLite open, so they
flaked as i/o timeouts. Match the rest of the repo (base/exec/world tests):
fiber.TestConfig{Timeout: 30 * time.Second}. Assertions unchanged.
2026-07-23 02:56:49 -07:00
hanzo-dev a832e1f3ba test(research): give App.Test the repo-standard 30s timeout, not the 1s default
The research HTTP tests used the bare app.Fiber().Test(req) whose default client
timeout (1s) is too tight for a cold cek-encrypted per-org SQLite open, so they
flaked as i/o timeouts. Match the rest of the repo (base/exec/world tests):
fiber.TestConfig{Timeout: 30 * time.Second}. Assertions unchanged.
2026-07-23 02:56:49 -07:00
hanzo-dev e0e16cb319 cloud: wire the per-org store through the HA-durable path (research)
OrgStore gains an opt-in Durability (WithDurable): when set, forPath hydrates each
org's SQLite from the object store BEFORE opening (elected owner CarryForward-seals
to its lease round; non-owner refreshes read-only), binds the handle, and Sync ships
it fenced after a write. A degraded hydrate never blocks the open — the store always
opens (reads local, writes fail closed) so a second replica can never break it. With
no option it is byte-identical to the local-only cache the other 12 subsystems use.

BuildDeps constructs the deployment Durability (buildDurability): the SeaweedFS S3
If-Match ConditionalStore (same s3admin identity as deps.VFS), membership over
CLOUD_PEERS (the SAME set the shard router elects on), and the per-org envelope
Cipher rooted at the KMS master. No object store ⇒ nil ⇒ local-only. An
encryption-capable build with no cipher is refused (never ship plaintext snapshots).

research wires WithDurable(b.Durable) and calls Sync after each write commits
(ship-before-ack): a not-acked ship (deposed/degraded) returns 503 so the client
retries on the org's current owner — no acknowledged write is lost on failover. On a
local deployment the ship is a successful no-op. Scoped to research as the proof; the
seam (WithDurable) is the one other subsystems adopt next.

durable_test.go adds the cek-encrypting-build coverage: the .dek key sidecar ships
with the database bytes and is restored on hydrate, plus the (sidecar,db) frame
round-trip.
2026-07-23 02:56:25 -07:00
hanzo-dev 44d237a66a cloud: wire the per-org store through the HA-durable path (research)
OrgStore gains an opt-in Durability (WithDurable): when set, forPath hydrates each
org's SQLite from the object store BEFORE opening (elected owner CarryForward-seals
to its lease round; non-owner refreshes read-only), binds the handle, and Sync ships
it fenced after a write. A degraded hydrate never blocks the open — the store always
opens (reads local, writes fail closed) so a second replica can never break it. With
no option it is byte-identical to the local-only cache the other 12 subsystems use.

BuildDeps constructs the deployment Durability (buildDurability): the SeaweedFS S3
If-Match ConditionalStore (same s3admin identity as deps.VFS), membership over
CLOUD_PEERS (the SAME set the shard router elects on), and the per-org envelope
Cipher rooted at the KMS master. No object store ⇒ nil ⇒ local-only. An
encryption-capable build with no cipher is refused (never ship plaintext snapshots).

research wires WithDurable(b.Durable) and calls Sync after each write commits
(ship-before-ack): a not-acked ship (deposed/degraded) returns 503 so the client
retries on the org's current owner — no acknowledged write is lost on failover. On a
local deployment the ship is a successful no-op. Scoped to research as the proof; the
seam (WithDurable) is the one other subsystems adopt next.

durable_test.go adds the cek-encrypting-build coverage: the .dek key sidecar ships
with the database bytes and is restored on hydrate, plus the (sidecar,db) frame
round-trip.
2026-07-23 02:56:25 -07:00
hanzo-dev 5b503ab749 deps: bump hanzoai/vfs v0.6.4 → v0.6.6 (modernc out of the graph)
v0.6.4's replica/sqlite.go blank-imports modernc.org/sqlite directly, which
registers the "sqlite" database/sql driver a SECOND time alongside cek's
hanzoai/sqlite — a double-register panic at init the moment a binary links both
(which wiring internal/org's durable path into cloud is the first to do). v0.6.6
(469995c + ffd32ca) routes replica through the ONE hanzoai/sqlite driver, so a
single registration stands. Patch bump, FencedStore API unchanged.
2026-07-23 02:56:07 -07:00
hanzo-dev 0c9cc4d46b deps: bump hanzoai/vfs v0.6.4 → v0.6.6 (modernc out of the graph)
v0.6.4's replica/sqlite.go blank-imports modernc.org/sqlite directly, which
registers the "sqlite" database/sql driver a SECOND time alongside cek's
hanzoai/sqlite — a double-register panic at init the moment a binary links both
(which wiring internal/org's durable path into cloud is the first to do). v0.6.6
(469995c + ffd32ca) routes replica through the ONE hanzoai/sqlite driver, so a
single registration stands. Patch bump, FencedStore API unchanged.
2026-07-23 02:56:07 -07:00
hanzo-dev 51a35fdcc4 refactor(research): migrate store from raw SQL to hanzoai/orm
Kills the 'no such column' DDL-drift bug class permanently: records are typed Go
values stored as JSON in orm's fixed _entities table, so adding a field is a struct
change with ZERO DDL. orm layers over the per-org *sql.DB cloud.OrgDB already opens
(cek-encrypted, single-writer, WAL) via orm.AdaptSQLite — orm manages the records,
the caller owns the file — so encryption at rest and the HA durability plane are
preserved and the openStore(*sql.DB) seam is unchanged.

Behaviors preserved over the orm model:
- dedup/idempotency: CreateIfAbsent keyed by <project>:<id>:<content_hash>
- versioned/canonical: a per-store monotone seq stamped inside the ingest tx
  (never client ts); canonical = latest-appended non-retracted per stable id
- queries (counts, list, totals, projects, artifacts) reimplemented as in-memory
  folds over orm Query (per-org stores are small)
- evidence model intact: ingest forces private/non-trainable/non-publishable,
  faulted runs retained, artifact sha256 server-derived, provenance first-class

Artifact bytes split into a distinct blob keyspace so the diary feed never loads
blobs, and because orm's _entities id is a global PRIMARY KEY (kind-prefixed ids
keep each kind in its own keyspace). Regression test proves a new provenance field
needs no migration; the 11 evidence-semantics tests pass against the orm store.
2026-07-23 02:53:29 -07:00
hanzo-dev df447d3470 refactor(research): migrate store from raw SQL to hanzoai/orm
Kills the 'no such column' DDL-drift bug class permanently: records are typed Go
values stored as JSON in orm's fixed _entities table, so adding a field is a struct
change with ZERO DDL. orm layers over the per-org *sql.DB cloud.OrgDB already opens
(cek-encrypted, single-writer, WAL) via orm.AdaptSQLite — orm manages the records,
the caller owns the file — so encryption at rest and the HA durability plane are
preserved and the openStore(*sql.DB) seam is unchanged.

Behaviors preserved over the orm model:
- dedup/idempotency: CreateIfAbsent keyed by <project>:<id>:<content_hash>
- versioned/canonical: a per-store monotone seq stamped inside the ingest tx
  (never client ts); canonical = latest-appended non-retracted per stable id
- queries (counts, list, totals, projects, artifacts) reimplemented as in-memory
  folds over orm Query (per-org stores are small)
- evidence model intact: ingest forces private/non-trainable/non-publishable,
  faulted runs retained, artifact sha256 server-derived, provenance first-class

Artifact bytes split into a distinct blob keyspace so the diary feed never loads
blobs, and because orm's _entities id is a global PRIMARY KEY (kind-prefixed ids
keep each kind in its own keyspace). Regression test proves a new provenance field
needs no migration; the 11 evidence-semantics tests pass against the orm store.
2026-07-23 02:53:29 -07:00
hanzo-dev 4f7fcfe9a4 org: Durable — the reusable single-writer + hydrate + fenced-ship gate
Packages the handoff_test.go discipline into one value a per-org SQLite store
wires at open/write/close. Composes the three lanes: ha election (CASFencer
lease = WHO writes), replica.FencedStore (CarryForward-on-takeover + round-fenced
ship = HOW it ships, no acknowledged write lost, deposed writer rejected as
ErrStaleRound), and the org envelope Cipher (durable object at rest).

Snapshot is a raw file copy — checkpoint the WAL on the store's sole connection,
read the actual local file bytes, and ship them framed with the cek .dek key
sidecar. Backend-agnostic: identical whether the local file is cek-encrypted
(production) or plaintext (dev), so an existing on-disk store needs no migration
and cek stays the local at-rest gate.

Hydrate never makes a store unopenable (degrades read-only, writes fail closed) —
the outage was a second replica breaking the store.

durable_test.go proves it over one object store: two replicas contend → one
writes, the other defers, both open, no split-brain; a rolling takeover hydrates
the shipped snapshot with no lost write; a deposed writer is fenced; an
unreachable store degrades read-only, never 'unavailable'.
2026-07-23 02:36:15 -07:00
hanzo-dev 046beb5ae8 org: Durable — the reusable single-writer + hydrate + fenced-ship gate
Packages the handoff_test.go discipline into one value a per-org SQLite store
wires at open/write/close. Composes the three lanes: ha election (CASFencer
lease = WHO writes), replica.FencedStore (CarryForward-on-takeover + round-fenced
ship = HOW it ships, no acknowledged write lost, deposed writer rejected as
ErrStaleRound), and the org envelope Cipher (durable object at rest).

Snapshot is a raw file copy — checkpoint the WAL on the store's sole connection,
read the actual local file bytes, and ship them framed with the cek .dek key
sidecar. Backend-agnostic: identical whether the local file is cek-encrypted
(production) or plaintext (dev), so an existing on-disk store needs no migration
and cek stays the local at-rest gate.

Hydrate never makes a store unopenable (degrades read-only, writes fail closed) —
the outage was a second replica breaking the store.

durable_test.go proves it over one object store: two replicas contend → one
writes, the other defers, both open, no split-brain; a rolling takeover hydrates
the shipped snapshot with no lost write; a deposed writer is fenced; an
unreachable store degrades read-only, never 'unavailable'.
2026-07-23 02:36:15 -07:00
hanzo-dev 02b090f414 scrub(s3): rename minio→s3/SeaweedFS across object-store code
The object store is the SeaweedFS S3 gateway reached via github.com/hanzoai/s3-go
(package name minio, aliased s3). Rename the import alias minio→s3 and every
minio.X reference to s3.X; MinioConditionalStore→S3ConditionalStore,
NewMinioConditionalStore→NewS3ConditionalStore; fix comments/docs that named the
old MinIO fork. Upstream transitive module names (github.com/minio/*) in
go.mod/go.sum are external deps of hanzoai/s3-go and are left as-is.

No behavior change: pure rename + comments.
2026-07-23 02:19:31 -07:00
hanzo-dev a8b52baf9f scrub(s3): rename minio→s3/SeaweedFS across object-store code
The object store is the SeaweedFS S3 gateway reached via github.com/hanzoai/s3-go
(package name minio, aliased s3). Rename the import alias minio→s3 and every
minio.X reference to s3.X; MinioConditionalStore→S3ConditionalStore,
NewMinioConditionalStore→NewS3ConditionalStore; fix comments/docs that named the
old MinIO fork. Upstream transitive module names (github.com/minio/*) in
go.mod/go.sum are external deps of hanzoai/s3-go and are left as-is.

No behavior change: pure rename + comments.
2026-07-23 02:19:31 -07:00
hanzo-dev 6c08ad7339 test(metering): fix pre-existing TestRecord_DebitsFinanceInProcess (2 causes)
Pre-existing failure on clean main, unrelated to the pre-pay work — two causes,
both surfaced now that the metering suite runs:

1) Encryption gate: on an encryption-capable (cgo) build, cek refuses to open
   the finance store without CLOUD_KMS_MASTER_KEY_REF. The root package's
   TestMain supplies a throwaway dev key; the clients/metering test package had
   no TestMain, so the finance-in-process test couldn't open a store. Add the
   same dev-key TestMain (mirrors the root; only when the build can encrypt and
   no key is provided — CI's real key still wins).

2) Stale ceil-based assertions vs the exact 18-decimal ledger. Since the
   Money Int->Atto migration, finance debits the EXACT sub-cent amount: a 1.5c
   micros debit lands as 1.5c, leaving 98.5c (/usr/bin/zsh.985) — not a ceiled 98c. The
   RecordResult still reports the ceiled 2c (Cents() rounds up for whole-cent
   contexts), which is unchanged. Assert the exact ledger balance (money.Cmp
   against ParseUSD) instead of bal.Cents(), which ceils 98.5->99. The anti-leak
   invariant holds: the sub-cent debit is recorded exactly, never dropped to 0.

Full metering suite green.
2026-07-23 02:09:39 -07:00
hanzo-dev 8f0d633d24 test(metering): fix pre-existing TestRecord_DebitsFinanceInProcess (2 causes)
Pre-existing failure on clean main, unrelated to the pre-pay work — two causes,
both surfaced now that the metering suite runs:

1) Encryption gate: on an encryption-capable (cgo) build, cek refuses to open
   the finance store without CLOUD_KMS_MASTER_KEY_REF. The root package's
   TestMain supplies a throwaway dev key; the clients/metering test package had
   no TestMain, so the finance-in-process test couldn't open a store. Add the
   same dev-key TestMain (mirrors the root; only when the build can encrypt and
   no key is provided — CI's real key still wins).

2) Stale ceil-based assertions vs the exact 18-decimal ledger. Since the
   Money Int->Atto migration, finance debits the EXACT sub-cent amount: a 1.5c
   micros debit lands as 1.5c, leaving 98.5c (/usr/bin/zsh.985) — not a ceiled 98c. The
   RecordResult still reports the ceiled 2c (Cents() rounds up for whole-cent
   contexts), which is unchanged. Assert the exact ledger balance (money.Cmp
   against ParseUSD) instead of bal.Cents(), which ceils 98.5->99. The anti-leak
   invariant holds: the sub-cent debit is recorded exactly, never dropped to 0.

Full metering suite green.
2026-07-23 02:09:39 -07:00
hanzo-dev ab21d87764 test(metering): pre-pay lifecycle — zero balance refused, funded served+debited
The CTO pre-pay acceptance scenario end to end: a freshly provisioned org
starts at a ZERO balance, so a metered request is REFUSED (402, no free floor,
nothing recorded); after a pre-pay deposit lands, the same request is SERVED
and DEBITS the balance; when the balance is exhausted, it is refused again. The
gate (Client.Authorize: funded := available > 0) already enforced this; this is
the explicit acceptance test for the pre-pay model. Stub balance made
thread-safe (setAvailable). Green under -race.
2026-07-23 01:59:22 -07:00
hanzo-dev 41c73dd177 test(metering): pre-pay lifecycle — zero balance refused, funded served+debited
The CTO pre-pay acceptance scenario end to end: a freshly provisioned org
starts at a ZERO balance, so a metered request is REFUSED (402, no free floor,
nothing recorded); after a pre-pay deposit lands, the same request is SERVED
and DEBITS the balance; when the balance is exhausted, it is refused again. The
gate (Client.Authorize: funded := available > 0) already enforced this; this is
the explicit acceptance test for the pre-pay model. Stub balance made
thread-safe (setAvailable). Green under -race.
2026-07-23 01:59:22 -07:00
hanzo-dev e16fd7a237 merge main into CDP merge (concurrent advance) 2026-07-23 01:42:14 -07:00
hanzo-dev c3d28bd278 merge main into CDP merge (concurrent advance) 2026-07-23 01:42:14 -07:00
hanzo-dev bc6d0a046b research: idempotent migrate — converge old-schema stores, fix outage
The store returned 500 'research store unavailable' because migrate() ran
CREATE TABLE IF NOT EXISTS (a no-op on an existing table) then CREATE INDEX
ix_exp_git ON experiment(git_sha) — which failed 'no such column: git_sha'
on any org store created before the provenance columns landed. CREATE TABLE
IF NOT EXISTS never adds a column to an existing table.

Fix: after creating the tables, ALTER TABLE ADD COLUMN every current non-key
column (tolerating 'duplicate column name' when already present), THEN the
indexes. migrate() now converges any older schema to the current column set
and is idempotent across re-opens. Adding a column = add it to the CREATE
plus the ensure list. Regression tests reproduce the outage (old table
without git_sha) + idempotent re-open + fresh-db all-duplicates.
2026-07-23 01:41:43 -07:00
hanzo-dev 76a7b184b3 research: idempotent migrate — converge old-schema stores, fix outage
The store returned 500 'research store unavailable' because migrate() ran
CREATE TABLE IF NOT EXISTS (a no-op on an existing table) then CREATE INDEX
ix_exp_git ON experiment(git_sha) — which failed 'no such column: git_sha'
on any org store created before the provenance columns landed. CREATE TABLE
IF NOT EXISTS never adds a column to an existing table.

Fix: after creating the tables, ALTER TABLE ADD COLUMN every current non-key
column (tolerating 'duplicate column name' when already present), THEN the
indexes. migrate() now converges any older schema to the current column set
and is idempotent across re-opens. Adding a column = add it to the CREATE
plus the ensure list. Regression tests reproduce the outage (old table
without git_sha) + idempotent re-open + fresh-db all-duplicates.
2026-07-23 01:41:43 -07:00
hanzo-dev 072b4e1fde research: idempotent migrate — converge old-schema stores, fix outage
The store returned 500 'research store unavailable' because migrate() ran
CREATE TABLE IF NOT EXISTS (a no-op on an existing table) then CREATE INDEX
ix_exp_git ON experiment(git_sha) — which failed 'no such column: git_sha'
on any org store created before the provenance columns landed. CREATE TABLE
IF NOT EXISTS never adds a column to an existing table.

Fix: after creating the tables, ALTER TABLE ADD COLUMN every current non-key
column (tolerating 'duplicate column name' when already present), THEN the
indexes. migrate() now converges any older schema to the current column set
and is idempotent across re-opens. Adding a column = add it to the CREATE
plus the ensure list. Regression tests reproduce the outage (old table
without git_sha) + idempotent re-open + fresh-db all-duplicates.
2026-07-23 01:41:03 -07:00
hanzo-dev 411865bcea research: idempotent migrate — converge old-schema stores, fix outage
The store returned 500 'research store unavailable' because migrate() ran
CREATE TABLE IF NOT EXISTS (a no-op on an existing table) then CREATE INDEX
ix_exp_git ON experiment(git_sha) — which failed 'no such column: git_sha'
on any org store created before the provenance columns landed. CREATE TABLE
IF NOT EXISTS never adds a column to an existing table.

Fix: after creating the tables, ALTER TABLE ADD COLUMN every current non-key
column (tolerating 'duplicate column name' when already present), THEN the
indexes. migrate() now converges any older schema to the current column set
and is idempotent across re-opens. Adding a column = add it to the CREATE
plus the ensure list. Regression tests reproduce the outage (old table
without git_sha) + idempotent re-open + fresh-db all-duplicates.
2026-07-23 01:41:03 -07:00
hanzo-dev 09be0200fb merge(cloud): CDP destinations (GA4/Meta/TikTok/Reddit/LinkedIn/X) + AI GTM
Server-side fan-out behind /v1/event: clients/destinations (6 adapters, per-org
KMS-custodied secrets, org-isolated store, bounded fail-soft fan-out), the RAW
pre-scrub forward sink in clients/analytics, and the AI-GTM analytics lens +
destinations_connect tool in clients/guide. Additive; no go.mod change.
2026-07-23 01:40:29 -07:00
hanzo-dev f87bb69f57 merge(cloud): CDP destinations (GA4/Meta/TikTok/Reddit/LinkedIn/X) + AI GTM
Server-side fan-out behind /v1/event: clients/destinations (6 adapters, per-org
KMS-custodied secrets, org-isolated store, bounded fail-soft fan-out), the RAW
pre-scrub forward sink in clients/analytics, and the AI-GTM analytics lens +
destinations_connect tool in clients/guide. Additive; no go.mod change.
2026-07-23 01:40:29 -07:00
hanzo-dev 4d25ca740d fix(help): ingress-owned edge limit, gated categories, opaque ticket ref, capped intake
Red-review fixes on the /v1/help public plane:

- Drop the app per-IP rate limiter. Behind hanzoai/ingress the socket peer IS the
  ingress, so a per-IP limiter keys every customer to ONE shared bucket (a global
  throttle + trivial DoS) and X-Forwarded-For is client-settable (a fresh value per
  request evades the limit and grows the bucket map without bound). The ingress owns
  the per-client edge limit; the plane bounds each request instead.

- GET /v1/help/categories now returns only sections that front a Published + public
  article, so an internal (agent-only) category name or description never leaks.

- Intake returns an opaque random public_ref in place of the monotonic ticket name,
  so an anonymous submitter cannot read the org's ticket volume. The sequential name
  stays internal to the agent plane.

- Cap the whole intake body at 64 KiB before parsing.

Tests: clients/help green (adds no-shared-bucket-throttle, category-gating, oversized-
body, opaque-ref assertions).
2026-07-23 01:38:36 -07:00
hanzo-dev 7b2da28c5b fix(help): ingress-owned edge limit, gated categories, opaque ticket ref, capped intake
Red-review fixes on the /v1/help public plane:

- Drop the app per-IP rate limiter. Behind hanzoai/ingress the socket peer IS the
  ingress, so a per-IP limiter keys every customer to ONE shared bucket (a global
  throttle + trivial DoS) and X-Forwarded-For is client-settable (a fresh value per
  request evades the limit and grows the bucket map without bound). The ingress owns
  the per-client edge limit; the plane bounds each request instead.

- GET /v1/help/categories now returns only sections that front a Published + public
  article, so an internal (agent-only) category name or description never leaks.

- Intake returns an opaque random public_ref in place of the monotonic ticket name,
  so an anonymous submitter cannot read the org's ticket volume. The sequential name
  stays internal to the agent plane.

- Cap the whole intake body at 64 KiB before parsing.

Tests: clients/help green (adds no-shared-bucket-throttle, category-gating, oversized-
body, opaque-ref assertions).
2026-07-23 01:38:36 -07:00
blueandhanzo-dev 7a8a1c35e0 feat(help): native /v1/help support product on the framework engine
Complete the Hanzo Support model as DocType fixtures and add the thin
/v1/help public plane — the native-Go replacement for the Frappe Helpdesk
(Vue + Python Frappe on Werkzeug), on Base in the one cloud binary.

Model (clients/help/help.go): add hd-article + hd-article-category (the KB)
and hd-communication (the ticket conversation thread); add a source field to
hd-ticket for inbound-connector provenance. Agents author and triage all of
it on the generic role-gated /v1/framework/hd-* surface — no new agent code.

Public plane (clients/help/subsystem.go): the anonymous face the secure-by-
default engine cannot serve — GET /v1/help/articles + /articles/:slug (only
status=Published AND is_public=1, re-checked on direct fetch), /categories,
and POST /v1/help/tickets (rate-limited customer intake creating the ticket
plus its opening conversation message). The served org is fixed server-side
(CLOUD_HELP_PUBLIC_ORG, else the deployment brand), never client-chosen, and
fails closed when unset. Storage delegates entirely to the framework
in-process API — one engine, no duplicated CRUD.

Wire help as a mount subsystem (apps.go + frozen wire order), alongside
knowledge, the other framework lane with a companion subsystem.
2026-07-23 01:38:36 -07:00
blueandhanzo-dev 3a8391235e feat(help): native /v1/help support product on the framework engine
Complete the Hanzo Support model as DocType fixtures and add the thin
/v1/help public plane — the native-Go replacement for the Frappe Helpdesk
(Vue + Python Frappe on Werkzeug), on Base in the one cloud binary.

Model (clients/help/help.go): add hd-article + hd-article-category (the KB)
and hd-communication (the ticket conversation thread); add a source field to
hd-ticket for inbound-connector provenance. Agents author and triage all of
it on the generic role-gated /v1/framework/hd-* surface — no new agent code.

Public plane (clients/help/subsystem.go): the anonymous face the secure-by-
default engine cannot serve — GET /v1/help/articles + /articles/:slug (only
status=Published AND is_public=1, re-checked on direct fetch), /categories,
and POST /v1/help/tickets (rate-limited customer intake creating the ticket
plus its opening conversation message). The served org is fixed server-side
(CLOUD_HELP_PUBLIC_ORG, else the deployment brand), never client-chosen, and
fails closed when unset. Storage delegates entirely to the framework
in-process API — one engine, no duplicated CRUD.

Wire help as a mount subsystem (apps.go + frozen wire order), alongside
knowledge, the other framework lane with a companion subsystem.
2026-07-23 01:38:36 -07:00
hanzo-dev cccf8f60dd feat(fleet): GB10-class unified SoC inventory — machine RAM, sm arch, CUDA/driver
nvidia-smi reports memory.total as [N/A] on a Grace-Blackwell SoC (no
dedicated VRAM counter) so the board showed nothing for spark. A unified
NVIDIA SoC now reports the machine RAM snapped to hardware capacity
(snap bound widened to 8 GiB — GB10 firmware reserves ~6.3 GiB), its sm
arch from compute_cap (12.1 -> sm_121), and the host CUDA toolkit +
driver versions ride the registration, mirroring the AMD rocm/hip pair.
2026-07-23 01:27:09 -07:00
hanzo-dev 87aa341e20 feat(fleet): GB10-class unified SoC inventory — machine RAM, sm arch, CUDA/driver
nvidia-smi reports memory.total as [N/A] on a Grace-Blackwell SoC (no
dedicated VRAM counter) so the board showed nothing for spark. A unified
NVIDIA SoC now reports the machine RAM snapped to hardware capacity
(snap bound widened to 8 GiB — GB10 firmware reserves ~6.3 GiB), its sm
arch from compute_cap (12.1 -> sm_121), and the host CUDA toolkit +
driver versions ride the registration, mirroring the AMD rocm/hip pair.
2026-07-23 01:27:09 -07:00
hanzo-dev c0e06e5cbd fix(fleet): unified APU reports the machine RAM, snapped to hardware capacity
An APU board figure now matches the Apple convention: the unified pool is
the MACHINE memory (128 GiB Strix Halo says 128 GiB), not the GTT tunable.
Kernel-visible MemTotal snaps up to the next 16 GiB DIMM capacity only when
the gap is a plausible firmware reservation (<= 6 GiB); a real carve-out
stays honest. evo: 118 GiB GTT / 124.4 GiB visible -> 131072 MiB.
2026-07-23 01:19:23 -07:00
hanzo-dev c18e20d625 fix(fleet): unified APU reports the machine RAM, snapped to hardware capacity
An APU board figure now matches the Apple convention: the unified pool is
the MACHINE memory (128 GiB Strix Halo says 128 GiB), not the GTT tunable.
Kernel-visible MemTotal snaps up to the next 16 GiB DIMM capacity only when
the gap is a plausible firmware reservation (<= 6 GiB); a real carve-out
stays honest. evo: 118 GiB GTT / 124.4 GiB visible -> 131072 MiB.
2026-07-23 01:19:23 -07:00
antje 9a51bffbcb refactor(admin): rename /v1/admin/storage → /v1/admin/block-storage
Free /v1/admin/storage for the operator's S3 object-buckets view (a distinct
storage concern); the DO block-volume + datastore-fill fleet is /block-storage.
No consumers yet, so the rename is clean. Handler storage→blockStorage, file
storage.go→block_storage.go (history preserved). Pairs with the operator SPA
Block Storage page + console repoint.
2026-07-22 23:44:21 -07:00
antje 43871c446a refactor(admin): rename /v1/admin/storage → /v1/admin/block-storage
Free /v1/admin/storage for the operator's S3 object-buckets view (a distinct
storage concern); the DO block-volume + datastore-fill fleet is /block-storage.
No consumers yet, so the rename is clean. Handler storage→blockStorage, file
storage.go→block_storage.go (history preserved). Pairs with the operator SPA
Block Storage page + console repoint.
2026-07-22 23:44:21 -07:00
hanzo-dev b80bc9c37a feat(functions): target=fleet — run a function on the org GPU fleet
A python function created with target=fleet invokes as an fn.run job on
the org gpu-jobs queue through the ONE embedded tasks engine (new
View.StartActivity/DescribeActivity seam, tasks v1.51.4): same queue,
same claim loop, same result a direct tasks-API submit gets. invoke
blocks bounded by TimeoutSec (900s ceiling, matching the sandbox);
longer jobs belong on the tasks API. Job outcome maps onto the sandbox
execResult contract so billing gate + metering + invocation recording
are identical across executors; fail-closed when the engine is not
ready. Also: functions suite gains the sibling TestMain encryption
harness (suite now runs on an encryption-capable build without CI env).
2026-07-22 23:37:53 -07:00
hanzo-dev 93dc54977c feat(functions): target=fleet — run a function on the org GPU fleet
A python function created with target=fleet invokes as an fn.run job on
the org gpu-jobs queue through the ONE embedded tasks engine (new
View.StartActivity/DescribeActivity seam, tasks v1.51.4): same queue,
same claim loop, same result a direct tasks-API submit gets. invoke
blocks bounded by TimeoutSec (900s ceiling, matching the sandbox);
longer jobs belong on the tasks API. Job outcome maps onto the sandbox
execResult contract so billing gate + metering + invocation recording
are identical across executors; fail-closed when the engine is not
ready. Also: functions suite gains the sibling TestMain encryption
harness (suite now runs on an encryption-capable build without CI env).
2026-07-22 23:37:53 -07:00
hanzo-dev 5540308d26 feat(destinations): fan the /v1/event stream out to ad + analytics platforms
Translate the canonical /v1/event stream to each connected platform's
conversion schema and forward it server-side. GA4 (Measurement Protocol)
and Meta (Pixel + Conversions API) work end-to-end; X, LinkedIn, TikTok,
and Reddit are scaffolded against the same Destination interface.

- clients/destinations: the Destination interface + per-platform adapters
  + translator (canonical EVENTS -> normalized conversion) + per-org
  registry (Base/SQLite, API secrets KMS-sealed) + the analytics fan-out
  consumer. A destination may reuse an integrations OAuth token
  (Meta CAPI <- meta_ads) or seal its own; PII match keys are SHA-256
  hashed before send.
- clients/analytics: a one-way fan-out sink after the write core, handed
  the RAW (pre-scrub) batch so a Conversions-API forwarder can hash the
  match keys the warehouse drops; detached + fail-soft, no-op when unset.
- clients/guide: read the analytics funnel (GET /v1/guide/analytics + an
  overview fold) and expose destinations_connect as an MCP tool.

Tests: translator mapping, GA4 + Meta send end-to-end (mock + PII
hashing), store tenant isolation, fan-out, and the connect seam.
2026-07-22 23:11:51 -07:00
hanzo-dev 3010dccef2 feat(destinations): fan the /v1/event stream out to ad + analytics platforms
Translate the canonical /v1/event stream to each connected platform's
conversion schema and forward it server-side. GA4 (Measurement Protocol)
and Meta (Pixel + Conversions API) work end-to-end; X, LinkedIn, TikTok,
and Reddit are scaffolded against the same Destination interface.

- clients/destinations: the Destination interface + per-platform adapters
  + translator (canonical EVENTS -> normalized conversion) + per-org
  registry (Base/SQLite, API secrets KMS-sealed) + the analytics fan-out
  consumer. A destination may reuse an integrations OAuth token
  (Meta CAPI <- meta_ads) or seal its own; PII match keys are SHA-256
  hashed before send.
- clients/analytics: a one-way fan-out sink after the write core, handed
  the RAW (pre-scrub) batch so a Conversions-API forwarder can hash the
  match keys the warehouse drops; detached + fail-soft, no-op when unset.
- clients/guide: read the analytics funnel (GET /v1/guide/analytics + an
  overview fold) and expose destinations_connect as an MCP tool.

Tests: translator mapping, GA4 + Meta send end-to-end (mock + PII
hashing), store tenant isolation, fan-out, and the connect seam.
2026-07-22 23:11:51 -07:00
hanzo-dev ddffb21a3b benchmark: keep fugu as arena DATA (a benchmarked/routable model), not marketing
Restore the fugu-ultra published_claim rows + the test fixture — fugu is just
another model we verify in the arena and can route to, like grok/gpt/opus. The
comments stay neutral (no 'disprove the rival' narration). The rule is only: no
fugu on the Enso MARKETING pages (hanzo.ai/enso — already clean). Reverses the
over-broad purge in 8b3aec0.
2026-07-22 23:07:28 -07:00
hanzo-dev e7ab2ee968 benchmark: keep fugu as arena DATA (a benchmarked/routable model), not marketing
Restore the fugu-ultra published_claim rows + the test fixture — fugu is just
another model we verify in the arena and can route to, like grok/gpt/opus. The
comments stay neutral (no 'disprove the rival' narration). The rule is only: no
fugu on the Enso MARKETING pages (hanzo.ai/enso — already clean). Reverses the
over-broad purge in a80f92c.
2026-07-22 23:07:28 -07:00
hanzo-dev 8b3aec0e95 benchmark: purge competitor from the public /v1/benchmark arena
Remove the 4 sakana/fugu-ultra published_claim rows + scrub the two comments that
named the competitor (house rule: no competitor names in source). The public
leaderboard (unauth GET /v1/benchmark/leaderboard) no longer exposes it. Legit
arena models (grok/gpt/opus/fable etc., provider-reported) are unchanged. Removed
the fugu-ultra test fixture (594 attempts remain, test still >=500 green).
2026-07-22 23:03:12 -07:00
hanzo-dev a80f92cca5 benchmark: purge competitor from the public /v1/benchmark arena
Remove the 4 sakana/fugu-ultra published_claim rows + scrub the two comments that
named the competitor (house rule: no competitor names in source). The public
leaderboard (unauth GET /v1/benchmark/leaderboard) no longer exposes it. Legit
arena models (grok/gpt/opus/fable etc., provider-reported) are unchanged. Removed
the fugu-ultra test fixture (594 attempts remain, test still >=500 green).
2026-07-22 23:03:12 -07:00
antje e0466a63bf feat(admin): GET /v1/admin/storage — DO block-storage fleet + datastore fill
The data source for admin.hanzo.ai's realtime Block Storage board, so we can
watch the analytics datastore fill and scale DO storage before it runs out.
SuperAdmin only (the s.guard wrap). Two REAL sources, each degrading independently:

- Fleet inventory (count · total capacity · monthly cost · per-volume region +
  attachment) from the DigitalOcean API — a new Volumes() on the existing
  DO_API_TOKEN client (paginated). DO gives capacity + attachment but NOT fill %,
  so a volume's used/pct stay ABSENT (the console renders "—", never a fabricated
  number).
- The analytics datastore's own fill from ClickHouse system.disks (the 200Gi PVC)
  over the SAME shared aiobject.DatastoreQuery the analytics + compute lenses use —
  no second connection. This is THE number the operator scales on. Near-full
  volumes raise an alert (warn ≥ 80%, critical ≥ 90%).

Honest by construction: DO unconfigured → empty fleet; datastore not connected →
no datastore card. admin creates no table and holds no storage state — it only reads.

Pure buildStorageSnapshot / alertLevel / datastoreFillFromRow are unit-tested
(fleet totals, cost, absent per-volume fill, the datastore card + alert, the
system.disks bytes→GiB→pct math, zero-capacity → nil). Pairs with console 8.4.151.
2026-07-22 22:36:12 -07:00
antje 1198254ffd feat(admin): GET /v1/admin/storage — DO block-storage fleet + datastore fill
The data source for admin.hanzo.ai's realtime Block Storage board, so we can
watch the analytics datastore fill and scale DO storage before it runs out.
SuperAdmin only (the s.guard wrap). Two REAL sources, each degrading independently:

- Fleet inventory (count · total capacity · monthly cost · per-volume region +
  attachment) from the DigitalOcean API — a new Volumes() on the existing
  DO_API_TOKEN client (paginated). DO gives capacity + attachment but NOT fill %,
  so a volume's used/pct stay ABSENT (the console renders "—", never a fabricated
  number).
- The analytics datastore's own fill from ClickHouse system.disks (the 200Gi PVC)
  over the SAME shared aiobject.DatastoreQuery the analytics + compute lenses use —
  no second connection. This is THE number the operator scales on. Near-full
  volumes raise an alert (warn ≥ 80%, critical ≥ 90%).

Honest by construction: DO unconfigured → empty fleet; datastore not connected →
no datastore card. admin creates no table and holds no storage state — it only reads.

Pure buildStorageSnapshot / alertLevel / datastoreFillFromRow are unit-tested
(fleet totals, cost, absent per-volume fill, the datastore card + alert, the
system.disks bytes→GiB→pct math, zero-capacity → nil). Pairs with console 8.4.151.
2026-07-22 22:36:12 -07:00
hanzo-dev 7e63dbad36 merge(main): integrate main (experiments+catalog) into campaign-gtm 2026-07-22 22:20:48 -07:00
hanzo-dev ea65ba9e48 merge(main): integrate main (experiments+catalog) into campaign-gtm 2026-07-22 22:20:48 -07:00
hanzo-dev f77706ea52 merge(main): integrate latest main into connectors-catalog-100 2026-07-22 22:18:54 -07:00
hanzo-dev 73d51910a8 merge(main): integrate latest main into connectors-catalog-100 2026-07-22 22:18:54 -07:00
hanzo-dev d5c05c8e51 fix(connectors): pin salesforce instance_url to SF hosts (reject IP/IMDS/port), invert recaptcha to allowlist, twitter confidential-client guard (red) 2026-07-22 22:18:29 -07:00
hanzo-dev bcc84015a8 fix(connectors): pin salesforce instance_url to SF hosts (reject IP/IMDS/port), invert recaptcha to allowlist, twitter confidential-client guard (red) 2026-07-22 22:18:29 -07:00
hanzo-dev 48c8e965f0 feat(fleet): fn.run — the functions-runner lane on a linked node
A linked node with uv claims fn.run jobs from the org gpu-jobs queue: an
inline Python script executed in an ephemeral uv run env on the node own
GPU stack, result = output tail + exit code + duration; nonzero exit
fails the activity with the traceback. Poison-loop guard becomes
per-lane, so a render-less node (evo) still claims its own lanes and
declines renders back to the queue. E2E-proven: submit via tasks API ->
evo claim -> uv run (rocm-smi visible) -> result GET.
2026-07-22 22:14:04 -07:00
hanzo-dev 8baac66b43 feat(fleet): fn.run — the functions-runner lane on a linked node
A linked node with uv claims fn.run jobs from the org gpu-jobs queue: an
inline Python script executed in an ephemeral uv run env on the node own
GPU stack, result = output tail + exit code + duration; nonzero exit
fails the activity with the traceback. Poison-loop guard becomes
per-lane, so a render-less node (evo) still claims its own lanes and
declines renders back to the queue. E2E-proven: submit via tasks API ->
evo claim -> uv run (rocm-smi visible) -> result GET.
2026-07-22 22:14:04 -07:00
hanzo-dev 898bc0aa7f feat(campaign): compose the merged experiments primitive for creative A/B
Rebased onto origin/main (experiments primitive landed @541f1b4). Campaign A/B now
composes clients/experiments instead of a nil-seam placeholder:

- experiment.go: AssignFunc composes experiments.Assign (bucketing); AnalyzeFunc
  composes experiments.Analyze (pull-model — it reads the metric from analytics
  itself, no duplicate measurement). Both nil-safe: single-creative honest default.
- metrics.go: GET /v1/campaign/:id/metrics embeds the A/B analysis (abTest) when a
  campaign runs >1 creative and an experiment is wired.
- apps/wire_seams.go: SetExperiment wires Assign+Analyze at the composition root
  (campaign-linked experiments live in the org's default project).

Tests: assigned variant flows to the executor plan (utm_content); single-creative
never assigns; assign-error fails soft to the default creative.
2026-07-22 22:00:10 -07:00
hanzo-dev 4164cd33b0 feat(campaign): compose the merged experiments primitive for creative A/B
Rebased onto origin/main (experiments primitive landed @52decbb). Campaign A/B now
composes clients/experiments instead of a nil-seam placeholder:

- experiment.go: AssignFunc composes experiments.Assign (bucketing); AnalyzeFunc
  composes experiments.Analyze (pull-model — it reads the metric from analytics
  itself, no duplicate measurement). Both nil-safe: single-creative honest default.
- metrics.go: GET /v1/campaign/:id/metrics embeds the A/B analysis (abTest) when a
  campaign runs >1 creative and an experiment is wired.
- apps/wire_seams.go: SetExperiment wires Assign+Analyze at the composition root
  (campaign-linked experiments live in the org's default project).

Tests: assigned variant flows to the executor plan (utm_content); single-creative
never assigns; assign-error fails soft to the default creative.
2026-07-22 22:00:10 -07:00
hanzo-dev 725f2f808e feat(campaign): /v1/campaign GTM orchestration + paid channel consumes connectors
Top-level go-to-market plane: a Campaign is a VALUE that spans channels; a
Channel is an orthogonal EXECUTOR it fans out to. Each channel consumes the
connector plane via integrations.TokenFor — the campaign object never touches a
credential.

- clients/campaign: Campaign model + store (per-org SQLite, cek at rest), the
  Channel interface + injected-func registry, /v1/campaign surface (CRUD +
  launch/pause fan-out + channels add/remove + metrics), experiment seam for
  creative A/B (nil-safe).
- clients/ads/provider.go: the ad-network execution edge (LaunchPaid/PaidSpend/
  PausePaid) — resolves the org's ad token via TokenFor, Meta executed for real,
  fail-closed on a disabled connector. Standalone POST /v1/ads/campaigns/:id/launch.
- clients/analytics/campaign.go: CampaignMetrics — the ONE campaign-metrics query
  over hanzo.events, org + utm_campaign bound positionally (tenancy invariant).
- apps/wire_seams.go: paid channel wired at the composition root (campaign never
  imports ads, ads never imports campaign).

Tests: fan-out, connector-consumes-token (httptest Meta), tenant isolation,
connector-disabled-blocks-send, spend fan-in, analytics tenancy predicate.
2026-07-22 21:54:30 -07:00
hanzo-dev 2f482b1ce4 feat(campaign): /v1/campaign GTM orchestration + paid channel consumes connectors
Top-level go-to-market plane: a Campaign is a VALUE that spans channels; a
Channel is an orthogonal EXECUTOR it fans out to. Each channel consumes the
connector plane via integrations.TokenFor — the campaign object never touches a
credential.

- clients/campaign: Campaign model + store (per-org SQLite, cek at rest), the
  Channel interface + injected-func registry, /v1/campaign surface (CRUD +
  launch/pause fan-out + channels add/remove + metrics), experiment seam for
  creative A/B (nil-safe).
- clients/ads/provider.go: the ad-network execution edge (LaunchPaid/PaidSpend/
  PausePaid) — resolves the org's ad token via TokenFor, Meta executed for real,
  fail-closed on a disabled connector. Standalone POST /v1/ads/campaigns/:id/launch.
- clients/analytics/campaign.go: CampaignMetrics — the ONE campaign-metrics query
  over hanzo.events, org + utm_campaign bound positionally (tenancy invariant).
- apps/wire_seams.go: paid channel wired at the composition root (campaign never
  imports ads, ads never imports campaign).

Tests: fan-out, connector-consumes-token (httptest Meta), tenant isolation,
connector-disabled-blocks-send, spend fan-in, analytics tenancy predicate.
2026-07-22 21:54:30 -07:00
blueandhanzo-dev 1c9df53bb1 connectors: complete third-party catalog to 100% of the historical list
Add 20 connectors on the established declarative planes (no plane changes):

Key connectors (userScope, keyVerify) — CRM/support, analytics, commerce, content:
  zendesk, pipedrive, intercom, reamaze, optimizely, amplitude, mixpanel,
  posthog, coinbase_commerce, shipstation, shipwire, contentful, netlify.
Bespoke body-inspecting Verify (200-with-error-in-body) over a shared, redirect-
refusing transport (verifypost.go): authorizenet, recaptcha.
Org OAuth (AdminOnly, reuse the callback/state plane): salesforce (instance_url
  custody + SSRF guard), adroll, twitter (S256 PKCE derived from the client secret
  so no shared Authorize/Exchange contract changes), google_bigquery, google_cloud
  (reuse google.go plumbing, read-only scopes).

One shared subdomain-origin normalizer (zendesk/reamaze); shopify keeps its own.
Least-privilege scopes throughout; fail-closed + token-free verified per connector.
Real httptest tests: wiring/placement, KMS seal, fail-closed, tenant isolation,
token-free, least-privilege scope, PKCE challenge!=verifier, instance_url SSRF.
2026-07-22 21:53:26 -07:00
blueandhanzo-dev c7376993b9 connectors: complete third-party catalog to 100% of the historical list
Add 20 connectors on the established declarative planes (no plane changes):

Key connectors (userScope, keyVerify) — CRM/support, analytics, commerce, content:
  zendesk, pipedrive, intercom, reamaze, optimizely, amplitude, mixpanel,
  posthog, coinbase_commerce, shipstation, shipwire, contentful, netlify.
Bespoke body-inspecting Verify (200-with-error-in-body) over a shared, redirect-
refusing transport (verifypost.go): authorizenet, recaptcha.
Org OAuth (AdminOnly, reuse the callback/state plane): salesforce (instance_url
  custody + SSRF guard), adroll, twitter (S256 PKCE derived from the client secret
  so no shared Authorize/Exchange contract changes), google_bigquery, google_cloud
  (reuse google.go plumbing, read-only scopes).

One shared subdomain-origin normalizer (zendesk/reamaze); shopify keeps its own.
Least-privilege scopes throughout; fail-closed + token-free verified per connector.
Real httptest tests: wiring/placement, KMS seal, fail-closed, tenant isolation,
token-free, least-privilege scope, PKCE challenge!=verifier, instance_url SSRF.
2026-07-22 21:53:26 -07:00
hanzo-dev 541f1b4cf8 merge(main): integrate latest main into feat/experiments 2026-07-22 21:40:22 -07:00
hanzo-dev 52decbbc04 merge(main): integrate latest main into feat/experiments 2026-07-22 21:40:22 -07:00
hanzo-dev c47252ee73 fix(experiments): gate winner-promotion on org admin (red finding — a prod flag write) 2026-07-22 21:39:39 -07:00
hanzo-dev bb147ac06d fix(experiments): gate winner-promotion on org admin (red finding — a prod flag write) 2026-07-22 21:39:39 -07:00
hanzo-dev ba98244759 feat(fleet): render an AMD APU as the processor it is
A unified-memory APU now reports the cpuinfo marketing name, normalized
from BIOS shouting: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S ->
"AMD Ryzen AI Max+ 395 w/ Radeon 8060S (gfx1151)". Model numbers keep
their casing; discrete cards and non-Ryzen hosts keep their names.
2026-07-22 20:54:20 -07:00
hanzo-dev d965a68885 feat(fleet): render an AMD APU as the processor it is
A unified-memory APU now reports the cpuinfo marketing name, normalized
from BIOS shouting: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S ->
"AMD Ryzen AI Max+ 395 w/ Radeon 8060S (gfx1151)". Model numbers keep
their casing; discrete cards and non-Ryzen hosts keep their names.
2026-07-22 20:54:20 -07:00
hanzo-dev 9343cbf63f feat(account): remove trial funding — usage is pre-paid, org starts at zero
Business model: no trial credits. Onboarding no longer grants any starter
credit — onboardFirstRun drops the commerce GrantCredit call, the account
state drops the commerce client, and provision's result drops trialGranted.
The first-run path still drives the ONE atomic IAM provision (org + admin
move + hashed org-scoped credential); the org just starts at a zero balance.

Removed payout.Client.GrantCredit (its only caller was this funding path —
no dead code); the additive Deposit + the referral/affiliate/author programs
are untouched.

Test: first-run drives provision ONCE (owner/name from get-user, resolved
slug), no funding call; a retry converges.
2026-07-22 20:48:37 -07:00
hanzo-dev aca6db042a feat(account): remove trial funding — usage is pre-paid, org starts at zero
Business model: no trial credits. Onboarding no longer grants any starter
credit — onboardFirstRun drops the commerce GrantCredit call, the account
state drops the commerce client, and provision's result drops trialGranted.
The first-run path still drives the ONE atomic IAM provision (org + admin
move + hashed org-scoped credential); the org just starts at a zero balance.

Removed payout.Client.GrantCredit (its only caller was this funding path —
no dead code); the additive Deposit + the referral/affiliate/author programs
are untouched.

Test: first-run drives provision ONCE (owner/name from get-user, resolved
slug), no funding call; a retry converges.
2026-07-22 20:48:37 -07:00
hanzo-dev e2684eafad feat(experiments): unified A/B EXPERIMENT primitive composing flags+analytics+research
/v1/experiments is the VALUE that composes three existing planes, never a fourth
engine. ASSIGNMENT = flags (deterministic subject->variant via engineEvaluate; no
second bucketing, no assignment store). MEASUREMENT = analytics (one org-scoped
hanzo.events query; the eventsWhere isolation invariant). EVIDENCE = research
(per-variant samples as kind=ab rows; two-proportion z significance is a pure
function over them). Lifecycle: create writes the multivariate flag def; assign is
a flags evaluation; analyze folds analytics outcomes per variant by joining each
subject to its flags assignment; decide promotes a winner by rewriting the flag
rollout to 100%.

Seams added (additive, one-way): flags.Assign/PutDef/GetDef, analytics.Outcomes,
research.Record/List. clients/campaign composes experiments.Assign/Analyze to run a
creative A/B without reinventing assignment or evidence.

Variant KIND is orthogonal: variant.payload can be a feature config, ad-creative
id, email subject, or model id.
2026-07-22 20:42:07 -07:00
hanzo-dev 9c56954ce6 feat(experiments): unified A/B EXPERIMENT primitive composing flags+analytics+research
/v1/experiments is the VALUE that composes three existing planes, never a fourth
engine. ASSIGNMENT = flags (deterministic subject->variant via engineEvaluate; no
second bucketing, no assignment store). MEASUREMENT = analytics (one org-scoped
hanzo.events query; the eventsWhere isolation invariant). EVIDENCE = research
(per-variant samples as kind=ab rows; two-proportion z significance is a pure
function over them). Lifecycle: create writes the multivariate flag def; assign is
a flags evaluation; analyze folds analytics outcomes per variant by joining each
subject to its flags assignment; decide promotes a winner by rewriting the flag
rollout to 100%.

Seams added (additive, one-way): flags.Assign/PutDef/GetDef, analytics.Outcomes,
research.Record/List. clients/campaign composes experiments.Assign/Analyze to run a
creative A/B without reinventing assignment or evidence.

Variant KIND is orthogonal: variant.payload can be a feature config, ad-creative
id, email subject, or model id.
2026-07-22 20:42:07 -07:00
hanzo-dev db33e46737 feat(fleet): honest AMD APU inventory — unified memory, family name, arch + ROCm/HIP
- An APU whose VRAM is a token carve-out now reports its GTT unified pool
  (Strix Halo: 1 GiB VRAM carve-out vs ~118 GiB usable) with unified=true;
  discrete cards keep dedicated VRAM. Fixes evo showing 1 GB.
- gfx family naming: gfx1151 -> AMD Strix Halo (also Strix Point, Phoenix),
  so the board says what the silicon is, not just the iGPU series.
- Inventory records the native gfx arch per GPU and the host ROCm + HIP
  versions (/opt/rocm/.info/version, hipconfig), surfaced on /v1/fleet/workers.
- docs: bring-your-gpu.md converged on hanzo link (gpu connect is gone).
2026-07-22 20:40:35 -07:00
hanzo-dev 27adcc80ea feat(fleet): honest AMD APU inventory — unified memory, family name, arch + ROCm/HIP
- An APU whose VRAM is a token carve-out now reports its GTT unified pool
  (Strix Halo: 1 GiB VRAM carve-out vs ~118 GiB usable) with unified=true;
  discrete cards keep dedicated VRAM. Fixes evo showing 1 GB.
- gfx family naming: gfx1151 -> AMD Strix Halo (also Strix Point, Phoenix),
  so the board says what the silicon is, not just the iGPU series.
- Inventory records the native gfx arch per GPU and the host ROCm + HIP
  versions (/opt/rocm/.info/version, hipconfig), surfaced on /v1/fleet/workers.
- docs: bring-your-gpu.md converged on hanzo link (gpu connect is gone).
2026-07-22 20:40:35 -07:00
hanzo-dev 3d1b9828b5 feat(account): onboard drives ONE atomic IAM provision + funds the trial
The production signup path (POST /v1/iam/onboard) did create-organization THEN
move-user as two non-atomic S2S calls — a fault between them orphaned the org
(the exact bug IAM's provision() fixed), and it minted no credential and
granted no trial. First-run onboarding now calls the ONE atomic
POST /v1/iam/admin/provision (service-token) instead: org + admin move +
hashed credential + trial claim converge, and a mid-flight retry resumes on
the founder's own org rather than orphaning it.

On the one-time trial grant (the once-per-verified-identity claim IAM
returns), it funds the identity's CANONICAL org via commerce
POST /v1/billing/credit with a per-identity idempotency key
(starter:<verified-email>) — so the per-org dedup is a per-identity dedup, the
anti-farm gate (one trial per identity, not per org). Funding is best-effort +
idempotent: a hiccup never fails an onboarding that already provisioned.

The additional-org flow (caller already has a home org) is unchanged
(create-without-move). A legacy fallback keeps the old pair when the service
token is unset, so a partial deploy still onboards.

Test: first-run drives provision ONCE (owner/name from get-user, resolved
slug), funds with the per-identity key + bounded 500c to the canonical org,
and a retry grants no second trial → never double-funds.
2026-07-22 20:33:20 -07:00
hanzo-dev 787664527c feat(account): onboard drives ONE atomic IAM provision + funds the trial
The production signup path (POST /v1/iam/onboard) did create-organization THEN
move-user as two non-atomic S2S calls — a fault between them orphaned the org
(the exact bug IAM's provision() fixed), and it minted no credential and
granted no trial. First-run onboarding now calls the ONE atomic
POST /v1/iam/admin/provision (service-token) instead: org + admin move +
hashed credential + trial claim converge, and a mid-flight retry resumes on
the founder's own org rather than orphaning it.

On the one-time trial grant (the once-per-verified-identity claim IAM
returns), it funds the identity's CANONICAL org via commerce
POST /v1/billing/credit with a per-identity idempotency key
(starter:<verified-email>) — so the per-org dedup is a per-identity dedup, the
anti-farm gate (one trial per identity, not per org). Funding is best-effort +
idempotent: a hiccup never fails an onboarding that already provisioned.

The additional-org flow (caller already has a home org) is unchanged
(create-without-move). A legacy fallback keeps the old pair when the service
token is unset, so a partial deploy still onboards.

Test: first-run drives provision ONCE (owner/name from get-user, resolved
slug), funds with the per-identity key + bounded 500c to the canonical org,
and a retry grants no second trial → never double-funds.
2026-07-22 20:33:20 -07:00
hanzo-dev c11058ed05 Merge remote-tracking branch 'origin/main' into blue/research
# Conflicts:
#	apps/apps.go
#	apps/wire_test.go
#	clients/research/datastore.go
#	clients/research/research.go
#	clients/research/research_test.go
#	clients/research/ssrf.go
#	clients/research/store.go
2026-07-22 20:32:44 -07:00
hanzo-dev fe04a60187 Merge remote-tracking branch 'origin/main' into blue/research
# Conflicts:
#	apps/apps.go
#	apps/wire_test.go
#	clients/research/datastore.go
#	clients/research/research.go
#	clients/research/research_test.go
#	clients/research/ssrf.go
#	clients/research/store.go
2026-07-22 20:32:44 -07:00
hanzo-dev 68b1a17903 feat(payout): idempotent GrantCredit (POST /v1/billing/credit) for on-signup funding
Deposit (POST /v1/billing/deposit) is additive — a retried onboard would
double-fund. GrantCredit posts the idempotent /v1/billing/credit: commerce
dedups on idempotencyKey, so the same key credits at most once. The starter
grant uses a per-identity key against the identity's canonical org, so the
per-org dedup is a per-identity dedup — the trial-abuse gate (one trial per
verified identity, not per org).

A *Client method only (not the shared Commerce interface), so the authors/
affiliates seams are unaffected.

Tests: posts /v1/billing/credit with the per-identity idempotencyKey +
bounded 500c + starter-credit tag + org namespace; unconfigured is an honest
error.
2026-07-22 20:26:04 -07:00
hanzo-dev 61019879f4 feat(payout): idempotent GrantCredit (POST /v1/billing/credit) for on-signup funding
Deposit (POST /v1/billing/deposit) is additive — a retried onboard would
double-fund. GrantCredit posts the idempotent /v1/billing/credit: commerce
dedups on idempotencyKey, so the same key credits at most once. The starter
grant uses a per-identity key against the identity's canonical org, so the
per-org dedup is a per-identity dedup — the trial-abuse gate (one trial per
verified identity, not per org).

A *Client method only (not the shared Commerce interface), so the authors/
affiliates seams are unaffected.

Tests: posts /v1/billing/credit with the per-identity idempotencyKey +
bounded 500c + starter-credit tag + org namespace; unconfigured is an honest
error.
2026-07-22 20:26:04 -07:00
hanzo-dev 2c88a30872 research: N2 — supersession is seq-authoritative, not client ts
Canonical ordering used client ts first. But the backfill importer stamps a wall-clock ts
(~1.7e9) while the SDK live path sends none (ts=0), so an SDK correction/retraction (ts=0)
of a backfilled experiment (ts=big) sorted BEFORE it and NEVER superseded — the stale
value stayed canonical while the producer saw added=1 (the exact 91.4→69.7 stale-correction
case in the intended backfill+live-SDK workflow). canonicalExpWhere/AttWhere now order by
the server's monotonic append clock seq (latest-APPEND-wins, un-forgeable by a client); ts
is display-only (measured-at). datastore.go notes the deferred canonical OLAP read must be
seq-authoritative too (never argMax(ts)).

Cross-tool PoC (TestSupersessionIsSeqNotClientTS): ts=big backfill → ts=0 SDK correction
wins (69.7) AND ts=0 SDK retraction withdraws (canonical=0); attempt correction likewise;
all 17 tests green.
2026-07-22 20:23:18 -07:00
hanzo-dev d9f2735cf8 research: N2 — supersession is seq-authoritative, not client ts
Canonical ordering used client ts first. But the backfill importer stamps a wall-clock ts
(~1.7e9) while the SDK live path sends none (ts=0), so an SDK correction/retraction (ts=0)
of a backfilled experiment (ts=big) sorted BEFORE it and NEVER superseded — the stale
value stayed canonical while the producer saw added=1 (the exact 91.4→69.7 stale-correction
case in the intended backfill+live-SDK workflow). canonicalExpWhere/AttWhere now order by
the server's monotonic append clock seq (latest-APPEND-wins, un-forgeable by a client); ts
is display-only (measured-at). datastore.go notes the deferred canonical OLAP read must be
seq-authoritative too (never argMax(ts)).

Cross-tool PoC (TestSupersessionIsSeqNotClientTS): ts=big backfill → ts=0 SDK correction
wins (69.7) AND ts=0 SDK retraction withdraws (canonical=0); attempt correction likewise;
all 17 tests green.
2026-07-22 20:23:18 -07:00
hanzo-dev 12e3b857ca feat(integrations): Guide marketing/ad connectors — meta/google-ads/analytics/microsoft/tiktok/reddit/linkedin/warpcast/whatsapp (red GO, least-privilege) 2026-07-22 19:59:43 -07:00
hanzo-dev 68b0e1a0dc feat(integrations): Guide marketing/ad connectors — meta/google-ads/analytics/microsoft/tiktok/reddit/linkedin/warpcast/whatsapp (red GO, least-privilege) 2026-07-22 19:59:43 -07:00
hanzo-dev c30eb383db fix(integrations): drop write-capable business_management from meta_ads scopes
Red MEDIUM: meta_ads is documented read-only, but metaScopes carried
business_management — read+WRITE to Business Manager assets (users, ad
accounts, system users). Ad-account enumeration uses /me/adaccounts under
ads_read, so no read path needs it; the custodied ~60-day token no longer
carries business-asset write authority.

Harden the forbidden-scope guard to reject business_management and any
*_management / rw_* grant (not just ads_management), so a write scope cannot
regress. Add two tests: the short-token fallback (fb_exchange_token leg fails →
seal the short token with its real ~now+3600 expiry, no refresh) and a
cross-provider state-confusion negative (a meta_ads state replayed at
google_ads/callback is rejected as invalid state).
2026-07-22 19:49:58 -07:00
hanzo-dev aef466ee2a fix(integrations): drop write-capable business_management from meta_ads scopes
Red MEDIUM: meta_ads is documented read-only, but metaScopes carried
business_management — read+WRITE to Business Manager assets (users, ad
accounts, system users). Ad-account enumeration uses /me/adaccounts under
ads_read, so no read path needs it; the custodied ~60-day token no longer
carries business-asset write authority.

Harden the forbidden-scope guard to reject business_management and any
*_management / rw_* grant (not just ads_management), so a write scope cannot
regress. Add two tests: the short-token fallback (fb_exchange_token leg fails →
seal the short token with its real ~now+3600 expiry, no refresh) and a
cross-provider state-confusion negative (a meta_ads state replayed at
google_ads/callback is rejected as invalid state).
2026-07-22 19:49:58 -07:00
hanzo-dev 30175e876a research: M1/M2 content-address artifacts server-side; M3 close SSRF denylist gaps
M1: the artifact sha256 was client-asserted + never verified (poisonable, hash-addressed
unearned). Now the caller submits the BYTES (base64 content); the SERVER hashes them and
that sha256 is the identity + ref (sha256:<hash>). A client-asserted sha256 must MATCH
(else 422); content is required; poisoning needs a preimage. Bytes are stored + retrievable
by hash (GET /v1/research/artifacts/:sha256, org-scoped). First-writer-wins resolved.

M2: Artifact.ref was an ungated file:// twin of the SSRF-gated endpoint. The server now
DERIVES ref = sha256:<hash> and ignores any client ref — no file:///etc/passwd can be
stored or served back.

M3: ssrfSafe comment corrected to best-effort INGEST HYGIENE (not the dial-time control);
added the ranges the stdlib predicates miss — CGNAT 100.64/10 (Alibaba IMDS
100.100.100.200), IANA protocol 192.0.0/24 (Oracle IMDS 192.0.0.192), 0/8, TEST-NETs,
198.18/15, 240/4, NAT64, v6 doc; named dialTimeGuardTODO as the greppable pin required
before any endpoint dialer ships.

17 tests pass incl content-address poisoning-rejected + blob round-trip + new SSRF ranges.
2026-07-22 19:48:50 -07:00
hanzo-dev b93e7320b1 research: M1/M2 content-address artifacts server-side; M3 close SSRF denylist gaps
M1: the artifact sha256 was client-asserted + never verified (poisonable, hash-addressed
unearned). Now the caller submits the BYTES (base64 content); the SERVER hashes them and
that sha256 is the identity + ref (sha256:<hash>). A client-asserted sha256 must MATCH
(else 422); content is required; poisoning needs a preimage. Bytes are stored + retrievable
by hash (GET /v1/research/artifacts/:sha256, org-scoped). First-writer-wins resolved.

M2: Artifact.ref was an ungated file:// twin of the SSRF-gated endpoint. The server now
DERIVES ref = sha256:<hash> and ignores any client ref — no file:///etc/passwd can be
stored or served back.

M3: ssrfSafe comment corrected to best-effort INGEST HYGIENE (not the dial-time control);
added the ranges the stdlib predicates miss — CGNAT 100.64/10 (Alibaba IMDS
100.100.100.200), IANA protocol 192.0.0/24 (Oracle IMDS 192.0.0.192), 0/8, TEST-NETs,
198.18/15, 240/4, NAT64, v6 doc; named dialTimeGuardTODO as the greppable pin required
before any endpoint dialer ships.

17 tests pass incl content-address poisoning-rejected + blob round-trip + new SSRF ranges.
2026-07-22 19:48:50 -07:00
hanzo-dev 756f995d81 research: H1 — honor retraction (was a silent no-op)
revision is now part of content identity (hashExp/hashAtt) so a retraction of an
otherwise-identical run is a DISTINCT retained version, not an INSERT-OR-IGNORE no-op.
Canonical redefined: THE single latest version by (ts,seq); if that latest is retracted
the id has NO canonical (withdrawn) — the NOT EXISTS no longer filters revision, so a
newest-retraction wins and no earlier version resurfaces. Also canonicalize hashed JSON
(sorted-key, whitespace-free) so a re-serialization never mints a spurious version (L3).

Red PoC: retract → withdrawn from canonical (0), retained keeps both (2); a later
correction restores canonical. 16 tests pass.
2026-07-22 19:41:24 -07:00
hanzo-dev 299dde6848 research: H1 — honor retraction (was a silent no-op)
revision is now part of content identity (hashExp/hashAtt) so a retraction of an
otherwise-identical run is a DISTINCT retained version, not an INSERT-OR-IGNORE no-op.
Canonical redefined: THE single latest version by (ts,seq); if that latest is retracted
the id has NO canonical (withdrawn) — the NOT EXISTS no longer filters revision, so a
newest-retraction wins and no earlier version resurfaces. Also canonicalize hashed JSON
(sorted-key, whitespace-free) so a re-serialization never mints a spurious version (L3).

Red PoC: retract → withdrawn from canonical (0), retained keeps both (2); a later
correction restores canonical. 16 tests pass.
2026-07-22 19:41:24 -07:00
hanzo-dev 4ab24f8884 feat(integrations): Guide marketing/ad/analytics OAuth connectors
Add nine org-scoped connectors for the Business AI Guide's Account-Linking
step, on the same registry + KMS-sealed custody as google/github/cloudflare:

  meta_ads          OAuth  Advertising  Facebook/Instagram Ads + Pages (long-lived token)
  google_ads        OAuth  Advertising  reuses google.go plumbing, adwords scope
  google_analytics  OAuth  Analytics    reuses google.go plumbing, analytics.readonly
  microsoft_ads     OAuth  Advertising  Microsoft Advertising (Bing), refresh token
  tiktok_ads        OAuth  Advertising  TikTok for Business, durable token
  reddit_ads        OAuth  Advertising  Basic-auth client, refresh token
  linkedin_ads      OAuth  Advertising  read + reporting, optional refresh
  warpcast          apikey Social       Farcaster via Neynar key (keyVerify)
  whatsapp          apikey Messaging    WhatsApp Cloud API token + phone id (keyVerify)

Each is one declarative provider file over the existing Provider contract;
oauth_http.go is the shared bounded, token-free transport the form/JSON
exchanges reuse. Scopes are least-privilege (read/reporting; write scopes
added when a write path is wired). All AdminOnly — linking a client's
marketing account is an org-admin action. Tokens are read by the ads /
analytics / marketing readers via integrations.TokenFor(org, provider, name).

Tests: authorize/exchange against httptest stand-ins, KMS seal-before-row,
verify-before-store fail-closed, tenant isolation, org-admin gate, and
token-free errors/responses per provider.
2026-07-22 19:31:20 -07:00
hanzo-dev ae7330a539 feat(integrations): Guide marketing/ad/analytics OAuth connectors
Add nine org-scoped connectors for the Business AI Guide's Account-Linking
step, on the same registry + KMS-sealed custody as google/github/cloudflare:

  meta_ads          OAuth  Advertising  Facebook/Instagram Ads + Pages (long-lived token)
  google_ads        OAuth  Advertising  reuses google.go plumbing, adwords scope
  google_analytics  OAuth  Analytics    reuses google.go plumbing, analytics.readonly
  microsoft_ads     OAuth  Advertising  Microsoft Advertising (Bing), refresh token
  tiktok_ads        OAuth  Advertising  TikTok for Business, durable token
  reddit_ads        OAuth  Advertising  Basic-auth client, refresh token
  linkedin_ads      OAuth  Advertising  read + reporting, optional refresh
  warpcast          apikey Social       Farcaster via Neynar key (keyVerify)
  whatsapp          apikey Messaging    WhatsApp Cloud API token + phone id (keyVerify)

Each is one declarative provider file over the existing Provider contract;
oauth_http.go is the shared bounded, token-free transport the form/JSON
exchanges reuse. Scopes are least-privilege (read/reporting; write scopes
added when a write path is wired). All AdminOnly — linking a client's
marketing account is an org-admin action. Tokens are read by the ads /
analytics / marketing readers via integrations.TokenFor(org, provider, name).

Tests: authorize/exchange against httptest stand-ins, KMS seal-before-row,
verify-before-store fail-closed, tenant isolation, org-admin gate, and
token-free errors/responses per provider.
2026-07-22 19:31:20 -07:00
hanzo-dev 7cc3d501fe feat(venue): /v1/cloud DO/AWS/GCP/Azure account connectors → fleet fold (red GO all 4; SafeRESTConfig gate) 2026-07-22 19:28:45 -07:00
hanzo-dev 7ad8940061 feat(venue): /v1/cloud DO/AWS/GCP/Azure account connectors → fleet fold (red GO all 4; SafeRESTConfig gate) 2026-07-22 19:28:45 -07:00
hanzo-dev b47a01e4e1 research: wire /v1/research + R&D Ops Board into the cloud binary
The evidence plane (HIP-0512) was built + committed but orphaned — absent from
apps.Wire(), so it 404'd in prod and the board showed only its seed snapshot.
Wire it as a non-staged subsystem (mounts under the production mount-all default),
the arena's sibling right after benchmark, and mount its embedded R&D Ops Board at
/research (same-origin with the plane it reads).

- apps.go: import + {Name:"research", Mount, Shutdown} after benchmark
- wire_test.go: frozen sequence updated (research, non-staged, has Shutdown)
- research.go: mount the embedded board UI at /research (like tasks/ui)
- clients/research/ui: the board embed + serving test

go build ./cmd/cloud links (617MB); apps wire/staged guards + full research
suite green. Goes live at api.hanzo.ai/v1/research + /research on image deploy.
2026-07-22 19:25:34 -07:00
hanzo-dev aa119d6108 research: wire /v1/research + R&D Ops Board into the cloud binary
The evidence plane (HIP-0512) was built + committed but orphaned — absent from
apps.Wire(), so it 404'd in prod and the board showed only its seed snapshot.
Wire it as a non-staged subsystem (mounts under the production mount-all default),
the arena's sibling right after benchmark, and mount its embedded R&D Ops Board at
/research (same-origin with the plane it reads).

- apps.go: import + {Name:"research", Mount, Shutdown} after benchmark
- wire_test.go: frozen sequence updated (research, non-staged, has Shutdown)
- research.go: mount the embedded board UI at /research (like tasks/ui)
- clients/research/ui: the board embed + serving test

go build ./cmd/cloud links (617MB); apps wire/staged guards + full research
suite green. Goes live at api.hanzo.ai/v1/research + /research on image deploy.
2026-07-22 19:25:34 -07:00
hanzo-dev 0bee60a961 fix(venue,fleet): close Red's 3 HIGH + 1 MED in the discovery/fold + keyless-verify path
HIGH-2 (exec-kubeconfig RCE) + HIGH-3 (SSRF guarded the wrong field): one hardened
fold entrypoint. fleet.SafeRESTConfig — which dynFromKubeconfig/Register now funnel
through (so it also closes the shared visor.attachCluster BYO path) — REJECTS a
kubeconfig that carries an exec or auth-provider credential plugin (client-go would
run a local binary with the pod's full env) and SSRF-guards restCfg.Host, the ACTUAL
dial target the node-list hits (https-only; no loopback/private/link-local/IMDS/
unspecified/multicast). The guard now covers the kubeconfig SERVER, not the
discovery-reported endpoint (which diverges for DO/Azure). Bypass env
FLEET_ALLOW_PRIVATE_HOSTS for loopback test apiservers only.

HIGH-1 (GCP credentialJson SSRF/LFI/exfil): validateGoogleCredential fail-closes the
customer credentials JSON BEFORE google.CredentialsFromJSON — only a service_account
key is accepted (token_uri pinned to a google host); an external_account (WIF) config,
whose credential_source.url/file and token_url are attacker-chosen (pod SSRF /
arbitrary file read / token exfiltration, firing during verify), is rejected. Keyless
WIF becomes a Hanzo-owned config (follow-on), never customer-authored.

MED-4 (AWS region SSRF): a region is validated against ^[a-z]{2}-[a-z]+-\d+$ before
it is interpolated into the STS/EKS host, so "evil.com/" can never redirect the
assumed-role token off-account.

Venue's wrong-field endpoint pre-guard is removed (fleet is the one gate). Tests:
fleet SafeRESTConfig (exec/auth-provider/non-routable/non-https reject, public allow,
bypass); venue exec-kubeconfig-not-folded, server-diverges-from-endpoint-guarded,
hostile-external_account-rejected, validateGoogleCredential, validRegion. Fold-consumer
suites (visor/ml/fleet) green; cmd/cloud links. (Harness: fiber v3 Test timeout raised
off its 1s default so CI-box load can't flake the fold's TLS dial.)
2026-07-22 19:19:57 -07:00
hanzo-dev e1158dc6eb fix(venue,fleet): close Red's 3 HIGH + 1 MED in the discovery/fold + keyless-verify path
HIGH-2 (exec-kubeconfig RCE) + HIGH-3 (SSRF guarded the wrong field): one hardened
fold entrypoint. fleet.SafeRESTConfig — which dynFromKubeconfig/Register now funnel
through (so it also closes the shared visor.attachCluster BYO path) — REJECTS a
kubeconfig that carries an exec or auth-provider credential plugin (client-go would
run a local binary with the pod's full env) and SSRF-guards restCfg.Host, the ACTUAL
dial target the node-list hits (https-only; no loopback/private/link-local/IMDS/
unspecified/multicast). The guard now covers the kubeconfig SERVER, not the
discovery-reported endpoint (which diverges for DO/Azure). Bypass env
FLEET_ALLOW_PRIVATE_HOSTS for loopback test apiservers only.

HIGH-1 (GCP credentialJson SSRF/LFI/exfil): validateGoogleCredential fail-closes the
customer credentials JSON BEFORE google.CredentialsFromJSON — only a service_account
key is accepted (token_uri pinned to a google host); an external_account (WIF) config,
whose credential_source.url/file and token_url are attacker-chosen (pod SSRF /
arbitrary file read / token exfiltration, firing during verify), is rejected. Keyless
WIF becomes a Hanzo-owned config (follow-on), never customer-authored.

MED-4 (AWS region SSRF): a region is validated against ^[a-z]{2}-[a-z]+-\d+$ before
it is interpolated into the STS/EKS host, so "evil.com/" can never redirect the
assumed-role token off-account.

Venue's wrong-field endpoint pre-guard is removed (fleet is the one gate). Tests:
fleet SafeRESTConfig (exec/auth-provider/non-routable/non-https reject, public allow,
bypass); venue exec-kubeconfig-not-folded, server-diverges-from-endpoint-guarded,
hostile-external_account-rejected, validateGoogleCredential, validRegion. Fold-consumer
suites (visor/ml/fleet) green; cmd/cloud links. (Harness: fiber v3 Test timeout raised
off its 1s default so CI-box load can't flake the fold's TLS dial.)
2026-07-22 19:19:57 -07:00
hanzo-dev c9702cb422 docs(LLM): add /v1/cloud venue plane to the Open Cloud planes map 2026-07-22 19:19:57 -07:00
hanzo-dev 30f4f4cd40 docs(LLM): add /v1/cloud venue plane to the Open Cloud planes map 2026-07-22 19:19:57 -07:00
hanzo-dev 7d47001c55 feat(venue): add Azure (AKS) as the 4th provider — AAD + ARM REST, keyless WIF
Same discovery interface as DO/AWS/GCP, folding into the ONE fleet. An org
registers an Azure AD app (tenant + client): a client_secret drives the
service-principal flow; its absence selects KEYLESS Workload Identity Federation
(Hanzo presents its own federated OIDC token as a client assertion — no customer
secret stored). Discovery is plain ARM REST (net/http, no azure-sdk-for-go): GET
managedClusters per subscription, POST listClusterUserCredential for each
kubeconfig, fold. Tests cover both the SP and keyless-WIF paths (httptest AAD +
ARM stubs) plus token-free errors and keyless-seal.
2026-07-22 19:19:57 -07:00
hanzo-dev 68d0399fbb feat(venue): add Azure (AKS) as the 4th provider — AAD + ARM REST, keyless WIF
Same discovery interface as DO/AWS/GCP, folding into the ONE fleet. An org
registers an Azure AD app (tenant + client): a client_secret drives the
service-principal flow; its absence selects KEYLESS Workload Identity Federation
(Hanzo presents its own federated OIDC token as a client assertion — no customer
secret stored). Discovery is plain ARM REST (net/http, no azure-sdk-for-go): GET
managedClusters per subscription, POST listClusterUserCredential for each
kubeconfig, fold. Tests cover both the SP and keyless-WIF paths (httptest AAD +
ARM stubs) plus token-free errors and keyless-seal.
2026-07-22 19:19:57 -07:00
hanzo-dev e9dfb16b5e feat(base): forward /v1/collections/* to the managed Base orchestrator
The go:embed console (served BY the cloud binary) drives the Base product's
data plane at /v1/collections/* but has no Next BFF (pruned by build:embed), so
the calls hit cloud's native /v1 router — which had no such route → 404, and the
whole Base product (Bases manager + Records) went dark on console.hanzo.ai.

Add a principal-gated reverse proxy (clients/base/collections.go, the clients/o11y
pattern) that forwards /v1/collections[/*] 1:1 to the managed Base (base.hanzo.ai,
in-cluster Service; overridable via CLOUD_BASE_ORCHESTRATOR_URL/HOST) — the
in-binary replacement for the retired console BFF. cloud can't mint a hanzo.id JWT
(JWKS-only), so it forwards the caller's OWN Bearer; the managed Base validates it
and scopes tenants by the token subject. allowCollections reproduces the BFF's
least-privilege allowBaseSurface (collections data plane only; no Base admin tunnel).
Always on, before the embed gate. Tests: allow-list, director (Host+path+bearer),
principal gate (403 fail-closed).
2026-07-22 19:16:20 -07:00
hanzo-dev 910e18609c feat(base): forward /v1/collections/* to the managed Base orchestrator
The go:embed console (served BY the cloud binary) drives the Base product's
data plane at /v1/collections/* but has no Next BFF (pruned by build:embed), so
the calls hit cloud's native /v1 router — which had no such route → 404, and the
whole Base product (Bases manager + Records) went dark on console.hanzo.ai.

Add a principal-gated reverse proxy (clients/base/collections.go, the clients/o11y
pattern) that forwards /v1/collections[/*] 1:1 to the managed Base (base.hanzo.ai,
in-cluster Service; overridable via CLOUD_BASE_ORCHESTRATOR_URL/HOST) — the
in-binary replacement for the retired console BFF. cloud can't mint a hanzo.id JWT
(JWKS-only), so it forwards the caller's OWN Bearer; the managed Base validates it
and scopes tenants by the token subject. allowCollections reproduces the BFF's
least-privilege allowBaseSurface (collections data plane only; no Base admin tunnel).
Always on, before the embed gate. Tests: allow-list, director (Host+path+bearer),
principal gate (403 fail-closed).
2026-07-22 19:16:20 -07:00
hanzo-dev 7b7c53dcb4 research: COALESCE empty-store totals (SUM over zero rows is NULL) + regression test 2026-07-22 18:24:29 -07:00
hanzo-dev 9d71c41059 research: COALESCE empty-store totals (SUM over zero rows is NULL) + regression test 2026-07-22 18:24:29 -07:00
hanzo-dev 6ba7b03d2e feat(venue): /v1/cloud connect-a-cloud-account plane — discover org clusters, fold into the ONE fleet
An org links labeled DigitalOcean / AWS / GCP accounts (verified live, KMS-sealed,
keyless where possible); venue discovers each account's native Kubernetes clusters
and folds them into clients/fleet.Register — the same registry visor surfaces at
/v1/clusters and ml federates onto. No second cluster registry.

- DigitalOcean: PAT -> GET /v2/account, list DOKS clusters, pull each kubeconfig (godo).
- AWS: cross-account STS AssumeRole pinned by external id (confused-deputy), eks
  List/DescribeCluster, k8s-aws-v1 presigned token. Hand-rolled REST + owned SigV4
  (pinned by the canonical get-vanilla vector); no aws-sdk-go-v2 service trees.
- GCP: Workload Identity Federation or SA key via oauth2/google, container REST
  clusters.list; no google container SDK / gRPC.

Multi-credential per org (labeled, /orgs/{org}/cloud/{provider}/{label}), org-scoped
and tenant-isolated, admin-gated mutations, SSRF guard on discovered endpoints,
per-new-cluster billing meter through the shared compute fee. One discovery
interface, thin providers behind it. Zero new module deps (godo + oauth2 present).
2026-07-22 18:24:22 -07:00
hanzo-dev a6dbbdf9f3 feat(venue): /v1/cloud connect-a-cloud-account plane — discover org clusters, fold into the ONE fleet
An org links labeled DigitalOcean / AWS / GCP accounts (verified live, KMS-sealed,
keyless where possible); venue discovers each account's native Kubernetes clusters
and folds them into clients/fleet.Register — the same registry visor surfaces at
/v1/clusters and ml federates onto. No second cluster registry.

- DigitalOcean: PAT -> GET /v2/account, list DOKS clusters, pull each kubeconfig (godo).
- AWS: cross-account STS AssumeRole pinned by external id (confused-deputy), eks
  List/DescribeCluster, k8s-aws-v1 presigned token. Hand-rolled REST + owned SigV4
  (pinned by the canonical get-vanilla vector); no aws-sdk-go-v2 service trees.
- GCP: Workload Identity Federation or SA key via oauth2/google, container REST
  clusters.list; no google container SDK / gRPC.

Multi-credential per org (labeled, /orgs/{org}/cloud/{provider}/{label}), org-scoped
and tenant-isolated, admin-gated mutations, SSRF guard on discovered endpoints,
per-new-cluster billing meter through the shared compute fee. One discovery
interface, thin providers behind it. Zero new module deps (godo + oauth2 present).
2026-07-22 18:24:22 -07:00
hanzo-dev 1b8d6f1d50 merge(main): integrate parallel main into the 3 drive-home plane merges 2026-07-22 18:24:06 -07:00
hanzo-dev b5e2237937 merge(main): integrate parallel main into the 3 drive-home plane merges 2026-07-22 18:24:06 -07:00
zeekayandClaude Opus 4.8 6c9de45895 feat(analytics): behavior read lenses on /v1/analytics/top (pages/referrers/sources)
Extend the ONE breakdown endpoint with three org-scoped behavior lenses over
hanzo.events — answering WHERE people go, WHAT they look at, and WHERE they come
from — as ranked {key, pageviews, visitors, pct} lists alongside models/products:

- topPages       GROUP BY path (WHERE event='$pageview')
- topReferrers   GROUP BY referrer_domain, self/empty referrer bucketed "(direct)"
- topSources     GROUP BY utm_source, empty utm bucketed "(none)"

Read-side only: no ingest/write-core/schema change. Each lens rides eventsWhere
so the org is bound POSITIONALLY (never interpolated) — same tenant boundary as
every other query. pct is share of the TRUE in-window pageview total via
`sum(pageviews) OVER ()`, so a top-N list honestly shows the long tail. Every
lens degrades to honest-empty (available:false, empty items, Debug log) when the
events table is absent/errored — never a 500 (mirrors the overview web lens).

visitors=uniqExact(distinct_id), pageviews=count of $pageview — identical to the
web overview lens.

Tests: breakdownSQL binds org positionally + filters $pageview + buckets
direct/none + window-fn total; buildBreakdown pct is share of in-window total
(not returned-rows sum); honest-empty on datastore error. gofmt+vet clean, no
go.sum drift.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 18:23:27 -07:00
zeekayandhanzo-dev f50161a2b8 feat(analytics): behavior read lenses on /v1/analytics/top (pages/referrers/sources)
Extend the ONE breakdown endpoint with three org-scoped behavior lenses over
hanzo.events — answering WHERE people go, WHAT they look at, and WHERE they come
from — as ranked {key, pageviews, visitors, pct} lists alongside models/products:

- topPages       GROUP BY path (WHERE event='$pageview')
- topReferrers   GROUP BY referrer_domain, self/empty referrer bucketed "(direct)"
- topSources     GROUP BY utm_source, empty utm bucketed "(none)"

Read-side only: no ingest/write-core/schema change. Each lens rides eventsWhere
so the org is bound POSITIONALLY (never interpolated) — same tenant boundary as
every other query. pct is share of the TRUE in-window pageview total via
`sum(pageviews) OVER ()`, so a top-N list honestly shows the long tail. Every
lens degrades to honest-empty (available:false, empty items, Debug log) when the
events table is absent/errored — never a 500 (mirrors the overview web lens).

visitors=uniqExact(distinct_id), pageviews=count of $pageview — identical to the
web overview lens.

Tests: breakdownSQL binds org positionally + filters $pageview + buckets
direct/none + window-fn total; buildBreakdown pct is share of in-window total
(not returned-rows sum); honest-empty on datastore error. gofmt+vet clean, no
go.sum drift.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-22 18:23:27 -07:00
hanzo-dev 6355a05279 feat(automations): /v1/automations IFTTT trigger plane on the one engine (red GO: rate-budget bounded) 2026-07-22 18:22:47 -07:00
hanzo-dev 4e6950c77d feat(automations): /v1/automations IFTTT trigger plane on the one engine (red GO: rate-budget bounded) 2026-07-22 18:22:47 -07:00
hanzo-dev 78f832a681 feat(cloudflare): /v1/cloudflare asset plane (Pages/Workers/Workers-AI/R2/KV/D1), Workers-AI on the unified spine (red GO)
# Conflicts:
#	apps/wire_test.go
2026-07-22 18:22:33 -07:00
hanzo-dev 5829869811 feat(cloudflare): /v1/cloudflare asset plane (Pages/Workers/Workers-AI/R2/KV/D1), Workers-AI on the unified spine (red GO)
# Conflicts:
#	apps/wire_test.go
2026-07-22 18:22:33 -07:00
hanzo-dev f0525629ac feat(integrations): GitHub Pages on the App token (red-SHIP: grant-cached, token-safe, 403-split) 2026-07-22 18:19:38 -07:00
hanzo-dev 0a08cc4c96 feat(integrations): GitHub Pages on the App token (red-SHIP: grant-cached, token-safe, 403-split) 2026-07-22 18:19:38 -07:00
hanzo-dev ba93b55717 research: diary artifacts — POST/GET /v1/research/artifacts (hash-addressed, private-by-default)
Adds the research-diary artifact record (raw-artifact retention class): a
dashboard-snapshot (PNG of the canonical board) or ai-report (generated page),
addressed by sha256 content hash — a re-POST of the same bytes is a no-op (the
hash-addressed idempotency). The bytes live at ref (blob store); this stores the
verifiable manifest with the same provenance (project, git_sha/branch/dirty,
lib_versions) as a run.

  POST /v1/research/artifacts        record one artifact (idempotent by sha256)
  GET  /v1/research/artifacts?run=&project=&since=   diary feed, newest-first

Private by default; public only via the separate visibility grant (POST
/v1/research/grants with sha256), the same rule as runs; no implicit training/commons
rights. Rolls up to hanzo.research_artifact (append-only, keyed by sha256). 14 tests
pass.
2026-07-22 18:12:12 -07:00
hanzo-dev b9a9f856a4 research: diary artifacts — POST/GET /v1/research/artifacts (hash-addressed, private-by-default)
Adds the research-diary artifact record (raw-artifact retention class): a
dashboard-snapshot (PNG of the canonical board) or ai-report (generated page),
addressed by sha256 content hash — a re-POST of the same bytes is a no-op (the
hash-addressed idempotency). The bytes live at ref (blob store); this stores the
verifiable manifest with the same provenance (project, git_sha/branch/dirty,
lib_versions) as a run.

  POST /v1/research/artifacts        record one artifact (idempotent by sha256)
  GET  /v1/research/artifacts?run=&project=&since=   diary feed, newest-first

Private by default; public only via the separate visibility grant (POST
/v1/research/grants with sha256), the same rule as runs; no implicit training/commons
rights. Rolls up to hanzo.research_artifact (append-only, keyed by sha256). 14 tests
pass.
2026-07-22 18:12:12 -07:00
hanzo-dev 0ebddafaab research: versioned append-only model + first-class provenance + consent (CTO legal model)
Reshapes the AttemptStore per the updated ToS/DPA/Research-Supplemental terms:

- VERSIONED, not write-once. A correction APPENDS a new version under the same
  stable id; the prior is RETAINED (superseded), never mutated. Version identity is
  content_hash over the measurement AND its provenance, so an idempotent re-ingest is
  a no-op while a corrected number or a run on a new commit/lib version is a new
  retained version. revision ∈ {original,corrected,retracted}; canonical/superseded
  are derived (latest non-retracted per stable id).
- RETAINED vs CANONICAL exposed as distinct counts everywhere (ingest response,
  /totals, /projects). retained is the truth; canonical is the deduped view, so dedup
  never reads as loss. faulted/failed runs are retained (negative results are evidence).
- PROVENANCE first-class + queryable: project (column), git_sha/git_branch (indexed),
  git_dirty, lib_versions (structured JSON) — the longitudinal 'which lib version
  regressed X' record.
- PRIVATE by default; visibility (private/org/public) + trainable + publishable are
  each a SEPARATE authorized grant (POST /v1/research/grants), never implied by upload.
- Retention classes separable (nonpersonal metadata/provenance permanent; raw
  artifacts own retention; secrets never stored). Honest durability label: versioned ·
  append-only; cloud mirroring rolling out — NOT yet immutable/replicated/recovery-tested.

12 tests pass (pure-Go): idempotent-by-content, correction-appends-version,
provenance-distinct-versions, faulted-retained, private-by-default+grant,
projects/totals canonical+retained, SSRF, HTTP round-trip+provenance+tenant-isolation,
grant-separate-from-upload.
2026-07-22 18:01:14 -07:00
hanzo-dev 5c8338c0be research: versioned append-only model + first-class provenance + consent (CTO legal model)
Reshapes the AttemptStore per the updated ToS/DPA/Research-Supplemental terms:

- VERSIONED, not write-once. A correction APPENDS a new version under the same
  stable id; the prior is RETAINED (superseded), never mutated. Version identity is
  content_hash over the measurement AND its provenance, so an idempotent re-ingest is
  a no-op while a corrected number or a run on a new commit/lib version is a new
  retained version. revision ∈ {original,corrected,retracted}; canonical/superseded
  are derived (latest non-retracted per stable id).
- RETAINED vs CANONICAL exposed as distinct counts everywhere (ingest response,
  /totals, /projects). retained is the truth; canonical is the deduped view, so dedup
  never reads as loss. faulted/failed runs are retained (negative results are evidence).
- PROVENANCE first-class + queryable: project (column), git_sha/git_branch (indexed),
  git_dirty, lib_versions (structured JSON) — the longitudinal 'which lib version
  regressed X' record.
- PRIVATE by default; visibility (private/org/public) + trainable + publishable are
  each a SEPARATE authorized grant (POST /v1/research/grants), never implied by upload.
- Retention classes separable (nonpersonal metadata/provenance permanent; raw
  artifacts own retention; secrets never stored). Honest durability label: versioned ·
  append-only; cloud mirroring rolling out — NOT yet immutable/replicated/recovery-tested.

12 tests pass (pure-Go): idempotent-by-content, correction-appends-version,
provenance-distinct-versions, faulted-retained, private-by-default+grant,
projects/totals canonical+retained, SSRF, HTTP round-trip+provenance+tenant-isolation,
grant-separate-from-upload.
2026-07-22 18:01:14 -07:00
hanzo-dev b048bfcdf1 chore(deps): bump hanzoai/ai v1.831.0 (12-model BYOK catalog) + refreeze wire order (benchmark) 2026-07-22 17:49:49 -07:00
hanzo-dev 12f2afc69c chore(deps): bump hanzoai/ai v1.831.0 (12-model BYOK catalog) + refreeze wire order (benchmark) 2026-07-22 17:49:49 -07:00
hanzo-dev ef454a24cf research: /v1/research evidence plane — per-org SQLite source of truth + hanzoai/datastore roll-up (HIP-0512)
Adds the Hanzo Research surface (HIP-0512 §"Hanzo Research"): every experiment
across every product accrues as immutable, queryable evidence under one
discriminator (kind ∈ benchmark|kernel-perf|training|ablation|policy-eval).

Two planes: each org's transactional SQLite (per HIP-0302, physical file
isolation) is the durable source of truth; it rolls up best-effort into
hanzoai/datastore (ClickHouse) via the account-usage warehouse pattern for the
cross-project OLAP surface. A datastore outage degrades to rolled_up:false, never
a failed ingest.

Ingest is idempotent by stable id — attempt by (project,benchmark,item,model),
experiment by (project,<kind>:<subject>:<task>) with a ts-guarded upsert for
latest-run-canonical — so a corpus imports exactly once however many times it is
replayed (the no-data-loss guarantee) and an out-of-order replay of an older run
cannot regress a newer number.

Surface:
  POST /v1/research/experiments   ingest a batch → SQLite → roll up
  GET  /v1/research/experiments   list, latest-run-canonical (?project= ?kind=)
  GET  /v1/research/projects      every project + real totals (ops board)
  GET  /v1/research/totals        headline aggregate + per-kind (?project=)

Org is the physical tenant boundary (validated principal, never a client field);
project is a first-class column so the org's ops board aggregates across its
projects in one query. BYO endpoint is SSRF-gated at ingest (loopback/private/
link-local/metadata denylist, https required). status is a stored run-state field
(default complete); the live leasing state machine is the durable-execution
increment.

Tests (9, pure-Go): idempotent double-ingest, latest-run-canonical + no-regress,
attempt immutability, project/totals aggregates, SSRF denylist, HTTP round-trip +
tenant isolation + fail-soft roll-up.
2026-07-22 17:49:49 -07:00
hanzo-dev 819b6e1677 research: /v1/research evidence plane — per-org SQLite source of truth + hanzoai/datastore roll-up (HIP-0512)
Adds the Hanzo Research surface (HIP-0512 §"Hanzo Research"): every experiment
across every product accrues as immutable, queryable evidence under one
discriminator (kind ∈ benchmark|kernel-perf|training|ablation|policy-eval).

Two planes: each org's transactional SQLite (per HIP-0302, physical file
isolation) is the durable source of truth; it rolls up best-effort into
hanzoai/datastore (ClickHouse) via the account-usage warehouse pattern for the
cross-project OLAP surface. A datastore outage degrades to rolled_up:false, never
a failed ingest.

Ingest is idempotent by stable id — attempt by (project,benchmark,item,model),
experiment by (project,<kind>:<subject>:<task>) with a ts-guarded upsert for
latest-run-canonical — so a corpus imports exactly once however many times it is
replayed (the no-data-loss guarantee) and an out-of-order replay of an older run
cannot regress a newer number.

Surface:
  POST /v1/research/experiments   ingest a batch → SQLite → roll up
  GET  /v1/research/experiments   list, latest-run-canonical (?project= ?kind=)
  GET  /v1/research/projects      every project + real totals (ops board)
  GET  /v1/research/totals        headline aggregate + per-kind (?project=)

Org is the physical tenant boundary (validated principal, never a client field);
project is a first-class column so the org's ops board aggregates across its
projects in one query. BYO endpoint is SSRF-gated at ingest (loopback/private/
link-local/metadata denylist, https required). status is a stored run-state field
(default complete); the live leasing state machine is the durable-execution
increment.

Tests (9, pure-Go): idempotent double-ingest, latest-run-canonical + no-regress,
attempt immutability, project/totals aggregates, SSRF denylist, HTTP round-trip +
tenant isolation + fail-soft roll-up.
2026-07-22 17:49:49 -07:00
hanzo-dev d0529648b9 fix(cloudflare): floor the per-call BYO inference fee (red F-1)
A non-text Workers AI modality (whisper audio, vision/classification image
bytes) yields no prompt text, so aiPromptText → "" → EstTokens → 0 → the
token-denominated BYO fee → 0. ResourceMeter.Gate short-circuits on a
0-cent cost BEFORE calling commerce Authorize, so a frozen / broke /
over-cap org got ungated, unbilled Hanzo-proxied inference — no balance
check, no debit row.

Root cause: a token-denominated fee cannot price a non-text modality. Fix:
floor the per-CALL fee. BYOInferenceFeeMicros now returns max(token fee,
BYOFloorMicros) with a non-zero default floor ($0.0001), so:
  - the gate reservation is ALWAYS >= 1 cent → the gate ALWAYS runs → a
    frozen/broke/over-cap org is REFUSED, never proxied;
  - every /ai/run leaves a debit row >= the floor → no silent proxied call.
This closes F-1 and the F-2 unbilled-call gap (same root cause) together.

Also reorder aiRun: the balance/freeze gate now runs BEFORE account
resolution, so a refused org makes ZERO Cloudflare contact (no discovery,
no run). The floor is one shared, generic knob in the inference-billing
spine (CLOUD_AI_BYO_FLOOR_UUSD) — not a Cloudflare-specific path.

Tests: a broke org + a non-text model is refused 402 with zero CF contact
and zero debits; a funded text run whose model reports no usage still bills
the floor. BYO fee/floor math unit-tested with the floor isolated.
2026-07-22 17:47:21 -07:00
hanzo-dev 86a0607a51 fix(cloudflare): floor the per-call BYO inference fee (red F-1)
A non-text Workers AI modality (whisper audio, vision/classification image
bytes) yields no prompt text, so aiPromptText → "" → EstTokens → 0 → the
token-denominated BYO fee → 0. ResourceMeter.Gate short-circuits on a
0-cent cost BEFORE calling commerce Authorize, so a frozen / broke /
over-cap org got ungated, unbilled Hanzo-proxied inference — no balance
check, no debit row.

Root cause: a token-denominated fee cannot price a non-text modality. Fix:
floor the per-CALL fee. BYOInferenceFeeMicros now returns max(token fee,
BYOFloorMicros) with a non-zero default floor ($0.0001), so:
  - the gate reservation is ALWAYS >= 1 cent → the gate ALWAYS runs → a
    frozen/broke/over-cap org is REFUSED, never proxied;
  - every /ai/run leaves a debit row >= the floor → no silent proxied call.
This closes F-1 and the F-2 unbilled-call gap (same root cause) together.

Also reorder aiRun: the balance/freeze gate now runs BEFORE account
resolution, so a refused org makes ZERO Cloudflare contact (no discovery,
no run). The floor is one shared, generic knob in the inference-billing
spine (CLOUD_AI_BYO_FLOOR_UUSD) — not a Cloudflare-specific path.

Tests: a broke org + a non-text model is refused 402 with zero CF contact
and zero debits; a funded text run whose model reports no usage still bills
the floor. BYO fee/floor math unit-tested with the floor isolated.
2026-07-22 17:47:21 -07:00
blueandhanzo-dev 48a14de490 fix(automations): bound the trigger run-start surface (RED HIGH-1/MED-1/MED-2/LOW-1)
Deliver was the only run-start path with no bound. Add three orthogonal per-org
bounds in the ONE shared startRun, plus the loop/redelivery hardening:

HIGH-1 amplification (self-trigger loop / fan-out / hook-hammer):
- concurrency: orgRunLimiter now guards EVERY run-start (moved into startRun; runFlow
  parity), full → 429.
- durable rate budget: CountRunsSince (persisted rows, survives restart) caps run-starts
  per rolling minute (CLOUD_AUTOMATIONS_RUNS_PER_MIN, default 300), enforced BEFORE the
  insert → a fan-out/loop hits the ceiling and stops.
- causation depth: TriggerEvent.Depth → FlowRunInput.Depth; Deliver refuses at
  maxCausationDepth so an in-platform cycle terminates. /hooks reads X-Causation-Depth;
  the seam carries depth.

MED-1: the github push webhook skips a bot/App-authored push (isBotActor "[bot]") — our
own outbound mirror pushes AS the App bot, so "on push → our push → push" cannot loop.
MED-2: engine readiness is checked BEFORE the run-row insert, and a transient start
failure DELETEs the row (not FAILED) — a not-ready/crash no longer burns the DedupeKey,
so redelivery retries instead of dropping the event.
LOW-1: /hooks with no X-Idempotency-Key content-hashes the body, so a hammer of identical
POSTs collapses to one run.

Tests (cgo + pure-Go, race-clean): TestDeliverLoopBounded (in-platform cycle terminates),
TestDeliverRateCapped (durable budget), TestDeliverEngineNotReadyRetryable (no key burn),
TestInboundHookContentHashDedupe, TestBotActorGuard.
2026-07-22 17:46:14 -07:00
blueandhanzo-dev 3d7ef2f0c1 fix(automations): bound the trigger run-start surface (RED HIGH-1/MED-1/MED-2/LOW-1)
Deliver was the only run-start path with no bound. Add three orthogonal per-org
bounds in the ONE shared startRun, plus the loop/redelivery hardening:

HIGH-1 amplification (self-trigger loop / fan-out / hook-hammer):
- concurrency: orgRunLimiter now guards EVERY run-start (moved into startRun; runFlow
  parity), full → 429.
- durable rate budget: CountRunsSince (persisted rows, survives restart) caps run-starts
  per rolling minute (CLOUD_AUTOMATIONS_RUNS_PER_MIN, default 300), enforced BEFORE the
  insert → a fan-out/loop hits the ceiling and stops.
- causation depth: TriggerEvent.Depth → FlowRunInput.Depth; Deliver refuses at
  maxCausationDepth so an in-platform cycle terminates. /hooks reads X-Causation-Depth;
  the seam carries depth.

MED-1: the github push webhook skips a bot/App-authored push (isBotActor "[bot]") — our
own outbound mirror pushes AS the App bot, so "on push → our push → push" cannot loop.
MED-2: engine readiness is checked BEFORE the run-row insert, and a transient start
failure DELETEs the row (not FAILED) — a not-ready/crash no longer burns the DedupeKey,
so redelivery retries instead of dropping the event.
LOW-1: /hooks with no X-Idempotency-Key content-hashes the body, so a hammer of identical
POSTs collapses to one run.

Tests (cgo + pure-Go, race-clean): TestDeliverLoopBounded (in-platform cycle terminates),
TestDeliverRateCapped (durable budget), TestDeliverEngineNotReadyRetryable (no key burn),
TestInboundHookContentHashDedupe, TestBotActorGuard.
2026-07-22 17:46:14 -07:00
blueandhanzo-dev a77a938df9 refactor(automations): decomplect trigger-arrival + unify run-start (one way)
Polish toward one-way + compose, no behavior change:

- armTrigger: the ONE trigger-SOURCE arrival plane (cron schedule for POLLING,
  routing-index subscription for WEBHOOK/APP_WEBHOOK, nothing for MANUAL),
  extracted out of setEnabled so status-flip and trigger-arrival are separate
  concerns. The action chain is untouched here — any trigger source pairs with any
  actions (a product, not a switch).
- startRun: the ONE way a firing turns (rule, run id, event) into a durable run —
  persist-gate then dispatch, threading the event payload. Shared by the manual
  /run and event Deliver paths; a cron tick still starts through the engine
  schedule. Deliver collapses to pure match + dispatch; runFlow to a lean handler.

Tests green (cgo + pure-Go), race-clean.
2026-07-22 17:46:14 -07:00
blueandhanzo-dev f39205b729 refactor(automations): decomplect trigger-arrival + unify run-start (one way)
Polish toward one-way + compose, no behavior change:

- armTrigger: the ONE trigger-SOURCE arrival plane (cron schedule for POLLING,
  routing-index subscription for WEBHOOK/APP_WEBHOOK, nothing for MANUAL),
  extracted out of setEnabled so status-flip and trigger-arrival are separate
  concerns. The action chain is untouched here — any trigger source pairs with any
  actions (a product, not a switch).
- startRun: the ONE way a firing turns (rule, run id, event) into a durable run —
  persist-gate then dispatch, threading the event payload. Shared by the manual
  /run and event Deliver paths; a cron tick still starts through the engine
  schedule. Deliver collapses to pure match + dispatch; runFlow to a lean handler.

Tests green (cgo + pure-Go), race-clean.
2026-07-22 17:46:14 -07:00
blueandhanzo-dev 133e4a73c7 feat(automations): compose inbound webhooks into the trigger engine (github seam)
Wires the existing verified provider webhooks into the automations Deliver plane
via a composition-root seam, so a real external event fires subscribed flows —
without integrations importing automations (which imports it for credential
custody).

- integrations.SetAutomationTrigger: a primitive-typed seam (nil-safe no-op when
  automations is disabled), set once at the apps composition root to
  automations.Deliver.
- github push webhook now also fires github/push to subscribed flows for the
  signature-verified installation org (best-effort; ev.After is the dedupe key).
- stripe/channels reach the SAME Deliver with a one-line fireTrigger call.

Test: the seam passes the verified event verbatim, fail-closes on empty org, and
no-ops unwired.
2026-07-22 17:46:14 -07:00
blueandhanzo-dev 7fbe7999cd feat(automations): compose inbound webhooks into the trigger engine (github seam)
Wires the existing verified provider webhooks into the automations Deliver plane
via a composition-root seam, so a real external event fires subscribed flows —
without integrations importing automations (which imports it for credential
custody).

- integrations.SetAutomationTrigger: a primitive-typed seam (nil-safe no-op when
  automations is disabled), set once at the apps composition root to
  automations.Deliver.
- github push webhook now also fires github/push to subscribed flows for the
  signature-verified installation org (best-effort; ev.After is the dedupe key).
- stripe/channels reach the SAME Deliver with a one-line fireTrigger call.

Test: the seam passes the verified event verbatim, fail-closes on empty org, and
no-ops unwired.
2026-07-22 17:46:14 -07:00
blueandhanzo-dev 87f9e08c9a feat(automations): inbound event triggers — fire flows on webhooks/messages/cron
Adds the IFTTT inbound plane to the ONE automations engine: an external event
(provider webhook, inbound message, scheduler tick) starts every enabled flow
whose WEBHOOK/APP_WEBHOOK trigger matches (source, event), threading the event
payload into the run as {{trigger.*}}.

- trigger.go: Deliver(ctx, org, TriggerEvent) — org-scoped match over the routing
  index, per-org durable start via the shared engine (runStarter -> executeFlow),
  at-most-once by DedupeKey (CreateRunIfAbsent gate), fail-closed on a missing org.
- store: automations_triggers routing index (org, provider, event -> flow+version),
  maintained by setEnabled — the inbound analog of the POLLING cron schedule.
- engine: seed outputs["trigger"] so actions read the firing event.
- POST /v1/automations/hooks/:source/:event — authenticated inbound sink.
- each run emits ONE o11y event beside the exactly-once meter + audit.
- main_test.go: cek dev-key TestMain (mirrors clients/sync) so the suite runs under
  cgo and pure-Go alike.

Tests: trigger->action fire + payload threading (real embedded engine), tenant
isolation, idempotency, fail-closed, HTTP org gate, enable/disable lifecycle.
2026-07-22 17:46:14 -07:00
blueandhanzo-dev 3b405ffa7a feat(automations): inbound event triggers — fire flows on webhooks/messages/cron
Adds the IFTTT inbound plane to the ONE automations engine: an external event
(provider webhook, inbound message, scheduler tick) starts every enabled flow
whose WEBHOOK/APP_WEBHOOK trigger matches (source, event), threading the event
payload into the run as {{trigger.*}}.

- trigger.go: Deliver(ctx, org, TriggerEvent) — org-scoped match over the routing
  index, per-org durable start via the shared engine (runStarter -> executeFlow),
  at-most-once by DedupeKey (CreateRunIfAbsent gate), fail-closed on a missing org.
- store: automations_triggers routing index (org, provider, event -> flow+version),
  maintained by setEnabled — the inbound analog of the POLLING cron schedule.
- engine: seed outputs["trigger"] so actions read the firing event.
- POST /v1/automations/hooks/:source/:event — authenticated inbound sink.
- each run emits ONE o11y event beside the exactly-once meter + audit.
- main_test.go: cek dev-key TestMain (mirrors clients/sync) so the suite runs under
  cgo and pure-Go alike.

Tests: trigger->action fire + payload threading (real embedded engine), tenant
isolation, idempotency, fail-closed, HTTP org gate, enable/disable lifecycle.
2026-07-22 17:46:14 -07:00
hanzo-dev 0e076a61b4 fix(integrations): address Red review on GitHub Pages (grant cache, 403 disambiguation, secret scrub)
LOW-1: cache the installation grant set per-installation-id (45s TTL) so a
status-polling console no longer re-enumerates up to 100 upstream repo pages
on every Pages request. Keyed strictly by installation id (never org name) and
TTL-bounded, so a reconnect gets a fresh key and a revoked grant cannot outlive
the TTL — the cache can never widen cross-tenant scope.

LOW-2: pagesErr now distinguishes a rate-limit 403/429 (x-ratelimit-remaining:0,
Retry-After, or a rate-limit body) — surfaced as 429 with the retry hint — from a
genuine permission 403 that keeps the actionable re-authorize message.

INFO-1: truncateBody redacts credential-shaped substrings (gh*_ / github_pat_ /
Bearer <value>) from any surfaced provider error body — defense in depth.

Adopts Red's github_pages_adversarial_test.go (two-installation cross-tenant +
injection/cname/token/confused-deputy matrices) and adds fast-follow tests for
the grant-cache keying/collapse, the 403/429 split, and the secret scrub. Full
integrations suite green under -race.
2026-07-22 17:42:03 -07:00
hanzo-dev bf3975de81 fix(integrations): address Red review on GitHub Pages (grant cache, 403 disambiguation, secret scrub)
LOW-1: cache the installation grant set per-installation-id (45s TTL) so a
status-polling console no longer re-enumerates up to 100 upstream repo pages
on every Pages request. Keyed strictly by installation id (never org name) and
TTL-bounded, so a reconnect gets a fresh key and a revoked grant cannot outlive
the TTL — the cache can never widen cross-tenant scope.

LOW-2: pagesErr now distinguishes a rate-limit 403/429 (x-ratelimit-remaining:0,
Retry-After, or a rate-limit body) — surfaced as 429 with the retry hint — from a
genuine permission 403 that keeps the actionable re-authorize message.

INFO-1: truncateBody redacts credential-shaped substrings (gh*_ / github_pat_ /
Bearer <value>) from any surfaced provider error body — defense in depth.

Adopts Red's github_pages_adversarial_test.go (two-installation cross-tenant +
injection/cname/token/confused-deputy matrices) and adds fast-follow tests for
the grant-cache keying/collapse, the 403/429 split, and the secret scrub. Full
integrations suite green under -race.
2026-07-22 17:42:03 -07:00
hanzo-dev 7d59226a3b feat(integrations): GitHub Pages management on the App installation token
Five org-authed routes, siblings of the repo list/import, addressing one
repo as a resource on the same short-lived installation token:

  GET    /v1/integrations/github/repos/:repo/pages        status + live URL + custom domain
  POST   /v1/integrations/github/repos/:repo/pages        enable/configure (branch source or Actions)
  PUT    /v1/integrations/github/repos/:repo/pages        set/clear custom domain, HTTPS, source
  DELETE /v1/integrations/github/repos/:repo/pages        disable
  POST   /v1/integrations/github/repos/:repo/pages/builds request a build

Org comes from the validated principal; the repo name is intersected with
the installation's granted set and the owner is taken from GitHub's own
full_name, so a caller can neither address an ungranted repo nor inject an
owner into the GitHub API path. The token rides only the Authorization
header. Fail-closed: unconfigured 503, unconnected 409, ungranted 404.
2026-07-22 17:42:03 -07:00
hanzo-dev 9911d9970c feat(integrations): GitHub Pages management on the App installation token
Five org-authed routes, siblings of the repo list/import, addressing one
repo as a resource on the same short-lived installation token:

  GET    /v1/integrations/github/repos/:repo/pages        status + live URL + custom domain
  POST   /v1/integrations/github/repos/:repo/pages        enable/configure (branch source or Actions)
  PUT    /v1/integrations/github/repos/:repo/pages        set/clear custom domain, HTTPS, source
  DELETE /v1/integrations/github/repos/:repo/pages        disable
  POST   /v1/integrations/github/repos/:repo/pages/builds request a build

Org comes from the validated principal; the repo name is intersected with
the installation's granted set and the owner is taken from GitHub's own
full_name, so a caller can neither address an ungranted repo nor inject an
owner into the GitHub API path. The token rides only the Authorization
header. Fail-closed: unconfigured 503, unconnected 409, ungranted 404.
2026-07-22 17:42:03 -07:00
hanzo-dev 3f7254468e test(apps): refreeze wire golden — add benchmark subsystem
The benchmark subsystem was added to Wire() (native /v1/benchmark arena)
without updating the frozen mount-order golden, so TestWireOrderMatchesFrozen
fails on main (93 Wire specs vs 92 frozen). Add the missing entry in its
Wire() position (after evals). Mechanical refreeze, independent of the
Cloudflare asset plane in the parent commit.
2026-07-22 17:41:07 -07:00
hanzo-dev 42d533d616 test(apps): refreeze wire golden — add benchmark subsystem
The benchmark subsystem was added to Wire() (native /v1/benchmark arena)
without updating the frozen mount-order golden, so TestWireOrderMatchesFrozen
fails on main (93 Wire specs vs 92 frozen). Add the missing entry in its
Wire() position (after evals). Mechanical refreeze, independent of the
Cloudflare asset plane in the parent commit.
2026-07-22 17:41:07 -07:00
hanzo-dev 10d31b6ff5 feat(cloudflare): first-class /v1/cloudflare/* asset plane
Consolidate the per-org Cloudflare asset plane under the first-class
/v1/cloudflare/* prefix (sibling of /v1/dns, /v1/domain): connecting the
provider stays on the integrations plane, managing resources moves here.

Surface + core-manage each resource against the CF v4 API with the org's
own KMS-sealed token (fail-closed, tenant-isolated on the validated
principal):
- Zones + Analytics (read): list/get zones, zone traffic dashboard
- Pages, Workers (handlers unchanged, prefix relocated)
- Workers AI: POST /ai/run/{model} metered inference
- R2, KV (+ key value get/put/delete), D1 (+ query): wired for real
  (were Phase-2 501 stubs)

Thin hand-rolled REST over the CF v4 API on net/http, decomplected to one
seam: client.send is the ONE authed request/read primitive (Bearer-only,
bounded, token-free); envelope unwrap (exec/cfDo/cfUpload), verbatim relay
(pass), raw value (getRaw) and the AI run (runAI) all compose over it, so
the request/auth/error dance is written once. acctClient/acctWrite are the
ONE auth+account preamble every account-scoped handler shares — each
resource handler is a thin, orthogonal body over those two seams.

Workers AI meters through the ONE unified usage/billing spine
(cloud.AIMeterProvider "ai") and emits the ONE gen_ai o11y span
(clients.StartGenAISpan), at the thin BYO fee since the org's own token
already paid Cloudflare for the compute. New shared, generic (not
CF-specific) inference-billing surface in metered_ai.go: BYOFeeBps /
BYOInferenceFeeMicros / MicrosToGateCents + exported AIMeterProvider and
EstTokens. The gen_ai span constructor is extracted from aihttp.go so
chat, embed and workers-ai share one.

Least-privilege connector scopes grown to match the now-wired surfaces
(R2/KV/D1/Workers-AI/Analytics). Model, key and id path segments are
validated (anchored charset + no-traversal) so no caller value can
smuggle path structure or a host into an upstream URL.

Tests (httptest CF stub + fake commerce): tenant isolation, the model
SSRF guard, the org-admin mutation gate, raw KV value relay, D1 query,
and the Workers AI debit landing on the unified "ai" spine.
2026-07-22 17:41:07 -07:00
hanzo-dev d930c60dce feat(cloudflare): first-class /v1/cloudflare/* asset plane
Consolidate the per-org Cloudflare asset plane under the first-class
/v1/cloudflare/* prefix (sibling of /v1/dns, /v1/domain): connecting the
provider stays on the integrations plane, managing resources moves here.

Surface + core-manage each resource against the CF v4 API with the org's
own KMS-sealed token (fail-closed, tenant-isolated on the validated
principal):
- Zones + Analytics (read): list/get zones, zone traffic dashboard
- Pages, Workers (handlers unchanged, prefix relocated)
- Workers AI: POST /ai/run/{model} metered inference
- R2, KV (+ key value get/put/delete), D1 (+ query): wired for real
  (were Phase-2 501 stubs)

Thin hand-rolled REST over the CF v4 API on net/http, decomplected to one
seam: client.send is the ONE authed request/read primitive (Bearer-only,
bounded, token-free); envelope unwrap (exec/cfDo/cfUpload), verbatim relay
(pass), raw value (getRaw) and the AI run (runAI) all compose over it, so
the request/auth/error dance is written once. acctClient/acctWrite are the
ONE auth+account preamble every account-scoped handler shares — each
resource handler is a thin, orthogonal body over those two seams.

Workers AI meters through the ONE unified usage/billing spine
(cloud.AIMeterProvider "ai") and emits the ONE gen_ai o11y span
(clients.StartGenAISpan), at the thin BYO fee since the org's own token
already paid Cloudflare for the compute. New shared, generic (not
CF-specific) inference-billing surface in metered_ai.go: BYOFeeBps /
BYOInferenceFeeMicros / MicrosToGateCents + exported AIMeterProvider and
EstTokens. The gen_ai span constructor is extracted from aihttp.go so
chat, embed and workers-ai share one.

Least-privilege connector scopes grown to match the now-wired surfaces
(R2/KV/D1/Workers-AI/Analytics). Model, key and id path segments are
validated (anchored charset + no-traversal) so no caller value can
smuggle path structure or a host into an upstream URL.

Tests (httptest CF stub + fake commerce): tenant isolation, the model
SSRF guard, the org-admin mutation gate, raw KV value relay, D1 query,
and the Workers AI debit landing on the unified "ai" spine.
2026-07-22 17:41:07 -07:00
hanzo-dev df1e2908ee research: mount /v1/research in the composition root + freeze wire order
Registers the research subsystem in apps.Wire() (after benchmark, before
treasury) with its Shutdown, and freezes its position — and benchmark's, which
closes a pre-existing Wire/frozen gap on main — in TestWireOrderMatchesFrozen.
2026-07-22 17:35:37 -07:00
hanzo-dev 2b61d83ea8 research: mount /v1/research in the composition root + freeze wire order
Registers the research subsystem in apps.Wire() (after benchmark, before
treasury) with its Shutdown, and freezes its position — and benchmark's, which
closes a pre-existing Wire/frozen gap on main — in TestWireOrderMatchesFrozen.
2026-07-22 17:35:37 -07:00
zeekayandClaude Opus 4.8 120a74b4b0 refactor(analytics): unify event ingest onto ONE canonical door /v1/event
Make POST /v1/event the single canonical ingest door: one handler
implementation, one write core (ingestEvents), serving every wire shape
and every auth context. The other five routes become thin aliases/shims
delegating to it.

- decodeIngest: ONE wire-tolerant decoder accepting a bare Event object,
  a bare [Event] array, AND the {batch:[…]} | {events:[…]} envelope the
  Segment/beacon/publishable paths speak — all yielding the SAME
  []CaptureEvent the write core consumes.
- eventTenant: auth is the orthogonal pluggable concern on the one door —
  (1) IAM bearer, (2) write-only publishable key (pk_, folded in from the
  old /v1/ingest), (3) out-of-band IAM access key; fail-closed 403 with NO
  brand-host fallback on the canonical door.
- ingestBody: the ONE ingest core (tolerant decode → foldException → the
  ONE write core). eventIngest, the site-host carve (eventWithOrg), and the
  /v1/ingest alias all funnel through it; auth is the only per-door
  difference.
- Site-host carve repointed to /v1/event (eventWithOrg forces org from the
  resolved Site); isAnalyticsPath adds /v1/event so a published-site beacon
  POSTing <host>/v1/event lands host-forced. The org-forced-from-host
  adversarial invariant holds for /v1/event too.
- /v1/ingest is now a thin DEPRECATED alias of eventHandle (one-shot
  deprecation log, $source=ingest tag); /v1/ingest/keys minting unchanged.
- /v1/analytics{,/batch}, /v1/tracker, /v1/insights/e kept as thin
  foreign-protocol shims (external Segment/PostHog compat only) funnelling
  through the SAME write core; captureWithOrg now delegates to ingestBody.

Write core and hanzo.events schema unchanged. Tests prove the three wire
shapes land identically, pk_/bearer/host auth contexts resolve the same
tenant into a byte-identical warehouse row, and the exact app beacon
{batch:[ev]} body is accepted on /v1/event via the site-host carve.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 17:35:24 -07:00
zeekayandhanzo-dev 32091f488c refactor(analytics): unify event ingest onto ONE canonical door /v1/event
Make POST /v1/event the single canonical ingest door: one handler
implementation, one write core (ingestEvents), serving every wire shape
and every auth context. The other five routes become thin aliases/shims
delegating to it.

- decodeIngest: ONE wire-tolerant decoder accepting a bare Event object,
  a bare [Event] array, AND the {batch:[…]} | {events:[…]} envelope the
  Segment/beacon/publishable paths speak — all yielding the SAME
  []CaptureEvent the write core consumes.
- eventTenant: auth is the orthogonal pluggable concern on the one door —
  (1) IAM bearer, (2) write-only publishable key (pk_, folded in from the
  old /v1/ingest), (3) out-of-band IAM access key; fail-closed 403 with NO
  brand-host fallback on the canonical door.
- ingestBody: the ONE ingest core (tolerant decode → foldException → the
  ONE write core). eventIngest, the site-host carve (eventWithOrg), and the
  /v1/ingest alias all funnel through it; auth is the only per-door
  difference.
- Site-host carve repointed to /v1/event (eventWithOrg forces org from the
  resolved Site); isAnalyticsPath adds /v1/event so a published-site beacon
  POSTing <host>/v1/event lands host-forced. The org-forced-from-host
  adversarial invariant holds for /v1/event too.
- /v1/ingest is now a thin DEPRECATED alias of eventHandle (one-shot
  deprecation log, $source=ingest tag); /v1/ingest/keys minting unchanged.
- /v1/analytics{,/batch}, /v1/tracker, /v1/insights/e kept as thin
  foreign-protocol shims (external Segment/PostHog compat only) funnelling
  through the SAME write core; captureWithOrg now delegates to ingestBody.

Write core and hanzo.events schema unchanged. Tests prove the three wire
shapes land identically, pk_/bearer/host auth contexts resolve the same
tenant into a byte-identical warehouse row, and the exact app beacon
{batch:[ev]} body is accepted on /v1/event via the site-host carve.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-22 17:35:24 -07:00
hanzo-dev 8724f6231e research: /v1/research evidence plane — per-org SQLite source of truth + hanzoai/datastore roll-up (HIP-0512)
Adds the Hanzo Research surface (HIP-0512 §"Hanzo Research"): every experiment
across every product accrues as immutable, queryable evidence under one
discriminator (kind ∈ benchmark|kernel-perf|training|ablation|policy-eval).

Two planes: each org's transactional SQLite (per HIP-0302, physical file
isolation) is the durable source of truth; it rolls up best-effort into
hanzoai/datastore (ClickHouse) via the account-usage warehouse pattern for the
cross-project OLAP surface. A datastore outage degrades to rolled_up:false, never
a failed ingest.

Ingest is idempotent by stable id — attempt by (project,benchmark,item,model),
experiment by (project,<kind>:<subject>:<task>) with a ts-guarded upsert for
latest-run-canonical — so a corpus imports exactly once however many times it is
replayed (the no-data-loss guarantee) and an out-of-order replay of an older run
cannot regress a newer number.

Surface:
  POST /v1/research/experiments   ingest a batch → SQLite → roll up
  GET  /v1/research/experiments   list, latest-run-canonical (?project= ?kind=)
  GET  /v1/research/projects      every project + real totals (ops board)
  GET  /v1/research/totals        headline aggregate + per-kind (?project=)

Org is the physical tenant boundary (validated principal, never a client field);
project is a first-class column so the org's ops board aggregates across its
projects in one query. BYO endpoint is SSRF-gated at ingest (loopback/private/
link-local/metadata denylist, https required). status is a stored run-state field
(default complete); the live leasing state machine is the durable-execution
increment.

Tests (9, pure-Go): idempotent double-ingest, latest-run-canonical + no-regress,
attempt immutability, project/totals aggregates, SSRF denylist, HTTP round-trip +
tenant isolation + fail-soft roll-up.
2026-07-22 17:28:09 -07:00
hanzo-dev d6ca4ba692 research: /v1/research evidence plane — per-org SQLite source of truth + hanzoai/datastore roll-up (HIP-0512)
Adds the Hanzo Research surface (HIP-0512 §"Hanzo Research"): every experiment
across every product accrues as immutable, queryable evidence under one
discriminator (kind ∈ benchmark|kernel-perf|training|ablation|policy-eval).

Two planes: each org's transactional SQLite (per HIP-0302, physical file
isolation) is the durable source of truth; it rolls up best-effort into
hanzoai/datastore (ClickHouse) via the account-usage warehouse pattern for the
cross-project OLAP surface. A datastore outage degrades to rolled_up:false, never
a failed ingest.

Ingest is idempotent by stable id — attempt by (project,benchmark,item,model),
experiment by (project,<kind>:<subject>:<task>) with a ts-guarded upsert for
latest-run-canonical — so a corpus imports exactly once however many times it is
replayed (the no-data-loss guarantee) and an out-of-order replay of an older run
cannot regress a newer number.

Surface:
  POST /v1/research/experiments   ingest a batch → SQLite → roll up
  GET  /v1/research/experiments   list, latest-run-canonical (?project= ?kind=)
  GET  /v1/research/projects      every project + real totals (ops board)
  GET  /v1/research/totals        headline aggregate + per-kind (?project=)

Org is the physical tenant boundary (validated principal, never a client field);
project is a first-class column so the org's ops board aggregates across its
projects in one query. BYO endpoint is SSRF-gated at ingest (loopback/private/
link-local/metadata denylist, https required). status is a stored run-state field
(default complete); the live leasing state machine is the durable-execution
increment.

Tests (9, pure-Go): idempotent double-ingest, latest-run-canonical + no-regress,
attempt immutability, project/totals aggregates, SSRF denylist, HTTP round-trip +
tenant isolation + fail-soft roll-up.
2026-07-22 17:28:09 -07:00
antje 4672e8b89d feat(commerce): re-expose plan authority CRUD at /v1/plans/* + commerce v1.49.13
Re-ship increment 3a (the subscription/DNS plan CHARGE authority). Restores the
mount reverted in 43f5656d7 after the v1.49.12 seed bug, now on the RED-approved
fix commerce v1.49.13 (fix/plans-seed-roundtrip):
  - models/plan Metadata_ datastore:",noindex" (envelope round-trips; team.minSeats
    no longer served null)
  - bundle expansion builds a LOCAL zero-price childSnapshotPlan — never writes a
    $0 partial row into the plan authority (fixes world-pro/world-team $0 under-charge)
  - corrective boot seed force-corrects unmanaged partial rows to the embed while
    leaving admin edits (Managed) authoritative

Mount mirrors the 2b catalog mount: planapi.AdminRoute(storeV1, commercebilling.
SeedRows) on the /v1 bundle + own /v1/plans in commercePrefixes so it reaches
commerce (requireSuperAdmin-gated), not the AI /v1/* 402 catch-all. PUBLIC read
stays GET /v1/billing/plans. Money-path suites green (TestProdPath_SeedReadbackAllFields,
TestProdPath_CorrectsPreexistingBadRows); cloud prefix tests green.
2026-07-22 17:16:03 -07:00
antje 39e472a88c feat(commerce): re-expose plan authority CRUD at /v1/plans/* + commerce v1.49.13
Re-ship increment 3a (the subscription/DNS plan CHARGE authority). Restores the
mount reverted in 8c87a38fa after the v1.49.12 seed bug, now on the RED-approved
fix commerce v1.49.13 (fix/plans-seed-roundtrip):
  - models/plan Metadata_ datastore:",noindex" (envelope round-trips; team.minSeats
    no longer served null)
  - bundle expansion builds a LOCAL zero-price childSnapshotPlan — never writes a
    $0 partial row into the plan authority (fixes world-pro/world-team $0 under-charge)
  - corrective boot seed force-corrects unmanaged partial rows to the embed while
    leaving admin edits (Managed) authoritative

Mount mirrors the 2b catalog mount: planapi.AdminRoute(storeV1, commercebilling.
SeedRows) on the /v1 bundle + own /v1/plans in commercePrefixes so it reaches
commerce (requireSuperAdmin-gated), not the AI /v1/* 402 catch-all. PUBLIC read
stays GET /v1/billing/plans. Money-path suites green (TestProdPath_SeedReadbackAllFields,
TestProdPath_CorrectsPreexistingBadRows); cloud prefix tests green.
2026-07-22 17:16:03 -07:00
zeekayandClaude Opus 4.8 0c153f9c36 feat(analytics): route /v1/analytics beacon ingest on published site hosts
Published sites POST analytics beacons to <site-host>/v1/analytics (and
/v1/insights/e), but that hit the static site server and 405'd — while
/v1/base already routed via the sites base-carve. Mirror the base carve
exactly for analytics ingest, so a page on yadota.hanzo.app (or a bound
custom domain) can send its own beacons into hanzo.events.

sites: add analyticsHostHandler + SetAnalyticsHostHandler + isAnalyticsPath
(prefix /v1/analytics OR ==/v1/insights/e). In Middleware, for both the slug
host and the custom-domain branch, carve the ingest POST to the handler with
org = resolved Site.Org (server-supplied, host-derived — never the caller/
body), gated on method POST so the authenticated GET read lenses
(/v1/analytics/overview|timeseries|top on api.hanzo.ai) are never hijacked.
The base carve stays first and independent.

analytics: extract captureWithOrg / insightsWithOrg — the Segment/beacon and
PostHog decode+ingest cores with an EXPLICIT org — so the deprecated aliases
(org via captureTenant) and the site-host carve (org forced from the Site)
share the ONE write core (ingestEvents), no copy. build() installs the carve
via sites.SetAnalyticsHostHandler, gated by the same already-existing flag the
anonymous ingest path uses (CLOUD_ANALYTICS_PUBLIC_CAPTURE, default ON); off
⇒ a site host 405s a beacon POST, unchanged.

Tests: sites carve routing (slug + custom host force Site.Org, not body/header
claim; GET serves static; /v1/base still routes to base; self host Continues;
non-live 405; no-handler 405; isAnalyticsPath). analytics end-to-end through
Mount+sites.Middleware (forced site org, 503-not-403 discriminator; empty
batch 200; custom domain; GET not hijacked; non-site host uses the normal
gate; disabled when public capture off).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 17:02:10 -07:00
zeekayandhanzo-dev 11e08b17aa feat(analytics): route /v1/analytics beacon ingest on published site hosts
Published sites POST analytics beacons to <site-host>/v1/analytics (and
/v1/insights/e), but that hit the static site server and 405'd — while
/v1/base already routed via the sites base-carve. Mirror the base carve
exactly for analytics ingest, so a page on yadota.hanzo.app (or a bound
custom domain) can send its own beacons into hanzo.events.

sites: add analyticsHostHandler + SetAnalyticsHostHandler + isAnalyticsPath
(prefix /v1/analytics OR ==/v1/insights/e). In Middleware, for both the slug
host and the custom-domain branch, carve the ingest POST to the handler with
org = resolved Site.Org (server-supplied, host-derived — never the caller/
body), gated on method POST so the authenticated GET read lenses
(/v1/analytics/overview|timeseries|top on api.hanzo.ai) are never hijacked.
The base carve stays first and independent.

analytics: extract captureWithOrg / insightsWithOrg — the Segment/beacon and
PostHog decode+ingest cores with an EXPLICIT org — so the deprecated aliases
(org via captureTenant) and the site-host carve (org forced from the Site)
share the ONE write core (ingestEvents), no copy. build() installs the carve
via sites.SetAnalyticsHostHandler, gated by the same already-existing flag the
anonymous ingest path uses (CLOUD_ANALYTICS_PUBLIC_CAPTURE, default ON); off
⇒ a site host 405s a beacon POST, unchanged.

Tests: sites carve routing (slug + custom host force Site.Org, not body/header
claim; GET serves static; /v1/base still routes to base; self host Continues;
non-live 405; no-handler 405; isAnalyticsPath). analytics end-to-end through
Mount+sites.Middleware (forced site org, 503-not-403 discriminator; empty
batch 200; custom domain; GET not hijacked; non-site host uses the normal
gate; disabled when public capture off).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-22 17:02:10 -07:00
hanzo-dev 6fbdfe88d8 feat(link): --mirror flag — explicit off-switch for the render mirror sweep
The render mirror sweeps every local studio output into the org library
with the WORKER token, so on a box whose login org differs from the
content org it misroutes files; until now the only disable was pointing
--studio-url at a dead port. --mirror=false is the honest switch; default
true keeps current behavior. Ported from feat/per-gpu-queue 9468354
(superseded branch — the gpu command family there became link on main).
2026-07-22 16:53:17 -07:00
hanzo-dev 18f74ea7e6 feat(link): --mirror flag — explicit off-switch for the render mirror sweep
The render mirror sweeps every local studio output into the org library
with the WORKER token, so on a box whose login org differs from the
content org it misroutes files; until now the only disable was pointing
--studio-url at a dead port. --mirror=false is the honest switch; default
true keeps current behavior. Ported from feat/per-gpu-queue 9468354
(superseded branch — the gpu command family there became link on main).
2026-07-22 16:53:17 -07:00
antje 8ec6d62d16 chore(deps): bump hanzoai/iam v1.32.2 → v1.33.0
Clean-room IAM cutover-parity release. v1.33.0 closes the gaps cloud's
consumers need for a casdoor→clean-room identity cutover:
- orgs membership claim on tokens (multi-org tenancy the validator reads)
- casdoor membership verb aliases (clients/team invite consumes)
- get-user?accessKey= key resolution (hk-/pk-/sk-, CapKeyResolve-gated)
- Docker Registry v2 token + JWKS endpoints
All exported surface cloud imports (pkg/model, pkg/store, server) unchanged
→ compile-safe. NO deploy, NO CLOUD_IAM_IMPL flip (retired); casdoor remains
the live authority until the staged shadow-parity cutover.
2026-07-22 16:14:28 -07:00
antje 00a9fdc2b4 chore(deps): bump hanzoai/iam v1.32.2 → v1.33.0
Clean-room IAM cutover-parity release. v1.33.0 closes the gaps cloud's
consumers need for a casdoor→clean-room identity cutover:
- orgs membership claim on tokens (multi-org tenancy the validator reads)
- casdoor membership verb aliases (clients/team invite consumes)
- get-user?accessKey= key resolution (hk-/pk-/sk-, CapKeyResolve-gated)
- Docker Registry v2 token + JWKS endpoints
All exported surface cloud imports (pkg/model, pkg/store, server) unchanged
→ compile-safe. NO deploy, NO CLOUD_IAM_IMPL flip (retired); casdoor remains
the live authority until the staged shadow-parity cutover.
2026-07-22 16:14:28 -07:00
hanzo-dev fc8337ebb5 sites: drop reserved labels from the first-party allowlist (RED F-2)
Belt-and-suspenders on the brand apex: New() now filters IsReserved labels out of the
first-party allowlist (logging the drop), so an operator setting CLOUD_SITES_FIRSTPARTY=
login (or api/wallet/console/…) can never turn an auth-sensitive host into a publishable
site. Default cd,flow,gallery are clean and unaffected. TestFirstPartyDropsReserved pins it
(cd/flow/gallery serve; api/login/wallet dropped). Closes RED's last actionable finding —
F-1 (org-pin) already fixed in ba9330c; host-parsing + Base-path RED-confirmed sound.
2026-07-22 15:28:36 -07:00
hanzo-dev 9ad57a957c sites: drop reserved labels from the first-party allowlist (RED F-2)
Belt-and-suspenders on the brand apex: New() now filters IsReserved labels out of the
first-party allowlist (logging the drop), so an operator setting CLOUD_SITES_FIRSTPARTY=
login (or api/wallet/console/…) can never turn an auth-sensitive host into a publishable
site. Default cd,flow,gallery are clean and unaffected. TestFirstPartyDropsReserved pins it
(cd/flow/gallery serve; api/login/wallet dropped). Closes RED's last actionable finding —
F-1 (org-pin) already fixed in 541c63f; host-parsing + Base-path RED-confirmed sound.
2026-07-22 15:28:36 -07:00
hanzo-dev ba9330c089 sites: org-PIN first-party hosts (RED #4 ship-blocker) — never shadow-able
RED review of the first-party allowlist found the real risk: siteSlug returns a bare
slug and serve() resolved it via ResolveUniqueLiveSlug (the unique-LIVE-slug-across-ALL-
orgs fallback). So cd.hanzo.ai → Resolve('cd') → a CUSTOMER's project named 'cd' could be
served on our internal brand host — and via the OAuth-redirect path in reserved.go that is
account takeover. The allowlist gates WHICH labels serve; it did NOT pin WHOSE project.

Fix: first-party resolution is PINNED to the owning org (hanzo). siteSlug now returns a
firstParty flag; serve()/resolveLivePinned route a first-party host through the new
Resolver.ResolveOrg → store.ResolveOrgLiveSlug (WHERE org=? AND slug=? AND status='live'),
which can NEVER return another org's project. Multi-tenant hanzo.app is unchanged (still
unique-across-orgs). Fail-closed: no owning org (CLOUD_SITES_FIRSTPARTY_ORG, default hanzo)
⇒ first-party disabled entirely.

TestFirstPartyOrgPinned proves it: a first-party host calls ResolveOrg(hanzo,slug) and NEVER
the unique-slug Resolve; a multi-tenant host does the opposite. tsc/build green.
2026-07-22 15:25:06 -07:00
hanzo-dev 541c63fedb sites: org-PIN first-party hosts (RED #4 ship-blocker) — never shadow-able
RED review of the first-party allowlist found the real risk: siteSlug returns a bare
slug and serve() resolved it via ResolveUniqueLiveSlug (the unique-LIVE-slug-across-ALL-
orgs fallback). So cd.hanzo.ai → Resolve('cd') → a CUSTOMER's project named 'cd' could be
served on our internal brand host — and via the OAuth-redirect path in reserved.go that is
account takeover. The allowlist gates WHICH labels serve; it did NOT pin WHOSE project.

Fix: first-party resolution is PINNED to the owning org (hanzo). siteSlug now returns a
firstParty flag; serve()/resolveLivePinned route a first-party host through the new
Resolver.ResolveOrg → store.ResolveOrgLiveSlug (WHERE org=? AND slug=? AND status='live'),
which can NEVER return another org's project. Multi-tenant hanzo.app is unchanged (still
unique-across-orgs). Fail-closed: no owning org (CLOUD_SITES_FIRSTPARTY_ORG, default hanzo)
⇒ first-party disabled entirely.

TestFirstPartyOrgPinned proves it: a first-party host calls ResolveOrg(hanzo,slug) and NEVER
the unique-slug Resolve; a multi-tenant host does the opposite. tsc/build green.
2026-07-22 15:25:06 -07:00
hanzo-dev 33db8b7c2a build(cloud): build ONLY the cloud binary — retire the Go hanzo CLI target
The shipped hanzo is the Rust CLI (~/work/hanzo/cli, curl hanzo.sh); the Go
cmd/hanzo was only ever a local 'make hanzo' control-plane build (CI/release
already build only cloud). Remove the make target so cloud builds exactly the
one stateless unified-API binary. cmd/hanzo + cli/ stay as reference for the
still-to-port client-side tools (GPU fleet worker link, runner, engine,
security); the code wrapper + zen-tier 1M mechanism are now in the Rust CLI.

Claude-Session: https://claude.ai/code/session_01QSN1woYbvENByMbGUqQ9Me
2026-07-22 15:24:39 -07:00
hanzo-dev 4a916c7f0e build(cloud): build ONLY the cloud binary — retire the Go hanzo CLI target
The shipped hanzo is the Rust CLI (~/work/hanzo/cli, curl hanzo.sh); the Go
cmd/hanzo was only ever a local 'make hanzo' control-plane build (CI/release
already build only cloud). Remove the make target so cloud builds exactly the
one stateless unified-API binary. cmd/hanzo + cli/ stay as reference for the
still-to-port client-side tools (GPU fleet worker link, runner, engine,
security); the code wrapper + zen-tier 1M mechanism are now in the Rust CLI.
2026-07-22 15:24:39 -07:00
hanzo-dev ce5696c182 sites: first-party apex (hanzo.ai) — OPT-IN allowlist for OUR internal sites
Converge cd/flow/gallery.hanzo.ai off the legacy s3://cdn staticFiles plane onto the
Projects PaaS plane (clients/sites → hanzo-sites/hanzo/<slug>). The site router served
only <slug>.hanzo.app (user sites); hanzo.ai is a SelfDomain it never resolved.

hanzo.ai is INTERNAL-ONLY (users get <slug>.hanzo.app), so it uses the OPPOSITE security
model to the multi-tenant apex: sites are OPT-IN via an explicit allowlist
(CLOUD_SITES_FIRSTPARTY=cd,flow,gallery on CLOUD_SITES_FIRSTPARTY_APEX=hanzo.ai). ONLY an
allow-listed label serves; EVERY other hanzo.ai host (api/console/iam/kms/world/chat/…,
listed in reserved.go or not) falls through PROTECTED by default — so a first-come project
can never shadow a real internal host (the reserved.go OAuth account-takeover). No
denylist-completeness burden on the brand's own domain. hanzo.app (user sites) unchanged.

Tests: TestSiteSlugFirstParty pins the boundary (allow-listed serve; api/iam/kms/world/
unlisted all protected; dotted key can't match; hanzo.app unaffected). tsc/build green.
2026-07-22 15:14:21 -07:00
hanzo-dev ef53a03ba7 sites: first-party apex (hanzo.ai) — OPT-IN allowlist for OUR internal sites
Converge cd/flow/gallery.hanzo.ai off the legacy s3://cdn staticFiles plane onto the
Projects PaaS plane (clients/sites → hanzo-sites/hanzo/<slug>). The site router served
only <slug>.hanzo.app (user sites); hanzo.ai is a SelfDomain it never resolved.

hanzo.ai is INTERNAL-ONLY (users get <slug>.hanzo.app), so it uses the OPPOSITE security
model to the multi-tenant apex: sites are OPT-IN via an explicit allowlist
(CLOUD_SITES_FIRSTPARTY=cd,flow,gallery on CLOUD_SITES_FIRSTPARTY_APEX=hanzo.ai). ONLY an
allow-listed label serves; EVERY other hanzo.ai host (api/console/iam/kms/world/chat/…,
listed in reserved.go or not) falls through PROTECTED by default — so a first-come project
can never shadow a real internal host (the reserved.go OAuth account-takeover). No
denylist-completeness burden on the brand's own domain. hanzo.app (user sites) unchanged.

Tests: TestSiteSlugFirstParty pins the boundary (allow-listed serve; api/iam/kms/world/
unlisted all protected; dotted key can't match; hanzo.app unaffected). tsc/build green.
2026-07-22 15:14:21 -07:00
hanzo-dev eef518ecdb benchmark: AttemptStore seam + runner + presets + proof tests on real data
Architecture per CTO directive: read/leaderboard/compare/worker go through the
AttemptStore interface — fileStore is LOCAL-DEV (DataDir JSONL); the cloud backend
(relational + object store, stateless API) swaps in behind the same interface, no
handler change. NEVER pod-local writes in prod.

- store.go: AttemptStore interface + fileStore (append-only, idempotent by item×model)
- runner.go: execution worker (OpenAI-compatible call → extract → score → Append,
  cache-before-spend via Has); items from the evaluator plane
- presets.go: POST/GET /v1/benchmark/presets — design-your-own router blend (enso-<name>)
- benchmark_test.go: PROVES on real attempts — fable-5 measured 81.3 vs published 94.6
  (gap +13.3), grok vs gpt-5.6-sol paired n=198 net+5 McNemar p=0.30, exact-McNemar unit

Fugu Ultra GPQA claim 95.5 vs our harness 94.4% (answered) is the flagship
replicate-or-disprove. Builds clean; unified binary links.
2026-07-22 14:36:10 -07:00
hanzo-dev 57740cfc50 benchmark: AttemptStore seam + runner + presets + proof tests on real data
Architecture per CTO directive: read/leaderboard/compare/worker go through the
AttemptStore interface — fileStore is LOCAL-DEV (DataDir JSONL); the cloud backend
(relational + object store, stateless API) swaps in behind the same interface, no
handler change. NEVER pod-local writes in prod.

- store.go: AttemptStore interface + fileStore (append-only, idempotent by item×model)
- runner.go: execution worker (OpenAI-compatible call → extract → score → Append,
  cache-before-spend via Has); items from the evaluator plane
- presets.go: POST/GET /v1/benchmark/presets — design-your-own router blend (enso-<name>)
- benchmark_test.go: PROVES on real attempts — fable-5 measured 81.3 vs published 94.6
  (gap +13.3), grok vs gpt-5.6-sol paired n=198 net+5 McNemar p=0.30, exact-McNemar unit

Fugu Ultra GPQA claim 95.5 vs our harness 94.4% (answered) is the flagship
replicate-or-disprove. Builds clean; unified binary links.
2026-07-22 14:36:10 -07:00
hanzo-dev 6122473c55 benchmark: native /v1/benchmark arena — replicate-or-disprove any provider
Sibling to /v1/eval, mounted into the unified cloud binary (apps.go). The benchmark
ARENA: run the top-14 canonical public benchmarks against any model/endpoint under ONE
standardized harness, and replicate-or-disprove a published claim. Provenance-first,
never blended: published_claim (vendor-reported) vs hanzo-measured (our harness) are
separate planes — the gap is the signal (Fugu claims GPQA 95.5; our harness measures
~92-95 by fault accounting; fable-5 claims 94.6, we measure 81.3).

Routes: GET /catalog (top-14, native flags), GET /leaderboard?benchmark= (per-model
measured ∥ published, coverage-aware), GET /compare?a=&b= (paired common-set +
rescue/damage + exact McNemar — the only valid arm-vs-arm test), POST /runs (BYO model
or endpoint; cache-before-spend; execution worker follow-on). Builds clean; unified
binary links.
2026-07-22 14:36:10 -07:00
hanzo-dev 1decfcdfc2 benchmark: native /v1/benchmark arena — replicate-or-disprove any provider
Sibling to /v1/eval, mounted into the unified cloud binary (apps.go). The benchmark
ARENA: run the top-14 canonical public benchmarks against any model/endpoint under ONE
standardized harness, and replicate-or-disprove a published claim. Provenance-first,
never blended: published_claim (vendor-reported) vs hanzo-measured (our harness) are
separate planes — the gap is the signal (Fugu claims GPQA 95.5; our harness measures
~92-95 by fault accounting; fable-5 claims 94.6, we measure 81.3).

Routes: GET /catalog (top-14, native flags), GET /leaderboard?benchmark= (per-model
measured ∥ published, coverage-aware), GET /compare?a=&b= (paired common-set +
rescue/damage + exact McNemar — the only valid arm-vs-arm test), POST /runs (BYO model
or endpoint; cache-before-spend; execution worker follow-on). Builds clean; unified
binary links.
2026-07-22 14:36:10 -07:00
antje 43f5656d78 Revert "feat(commerce): expose plan authority CRUD at /v1/plans/* + commerce v1.49.12"
This reverts commit f05b9a6441.
2026-07-22 14:27:28 -07:00
antje 8c87a38fa5 Revert "feat(commerce): expose plan authority CRUD at /v1/plans/* + commerce v1.49.12"
This reverts commit cb977a9633.
2026-07-22 14:27:28 -07:00
zeekayandClaude Opus 4.8 6a7333e5d8 feat(projects): wire analytics + Base data space ON by default for new projects
Every new project now gets analytics collection and a Base data space
(form/forum/data submissions) wired by default — no opt-in — through the ONE
create path.

- createProject applies setProjectDefaults(analytics ON unless the caller passes
  analytics:false; space_id = "<org>/<slug>") then best-effort provisions the
  Base space. /v1/sites (ensureProject) routes through the same helpers, so
  defaults live in exactly one place.
- Fail-soft: a Base provisioning error (incl. embed disabled) is logged and
  swallowed — it never fails project creation, mirroring the CF edge-purge policy.
- clients/base.EnsureSpace: idempotent, in-process (no HTTP) provisioner that
  opens the org's per-org Base app and ensures the public-create "submissions"
  collection exists.
- store: additive analytics/space_id columns (migration matches the existing
  cache_control/last_purge_at ALTER pattern); analytics defaults ON for existing
  rows too. Exposed on the project view as analytics/space.

Tests: analytics default-ON, analytics:false opt-out, space provisioned on
create, and provisioning failure is fail-soft (create still 201, persists).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:09:12 -07:00
zeekayandhanzo-dev 0a6b51a721 feat(projects): wire analytics + Base data space ON by default for new projects
Every new project now gets analytics collection and a Base data space
(form/forum/data submissions) wired by default — no opt-in — through the ONE
create path.

- createProject applies setProjectDefaults(analytics ON unless the caller passes
  analytics:false; space_id = "<org>/<slug>") then best-effort provisions the
  Base space. /v1/sites (ensureProject) routes through the same helpers, so
  defaults live in exactly one place.
- Fail-soft: a Base provisioning error (incl. embed disabled) is logged and
  swallowed — it never fails project creation, mirroring the CF edge-purge policy.
- clients/base.EnsureSpace: idempotent, in-process (no HTTP) provisioner that
  opens the org's per-org Base app and ensures the public-create "submissions"
  collection exists.
- store: additive analytics/space_id columns (migration matches the existing
  cache_control/last_purge_at ALTER pattern); analytics defaults ON for existing
  rows too. Exposed on the project view as analytics/space.

Tests: analytics default-ON, analytics:false opt-out, space provisioned on
create, and provisioning failure is fail-soft (create still 201, persists).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-22 14:09:12 -07:00
antje c3938fed27 fix(team): flatten model HTML in bot replies — a bot answering '<p>yes.</p>' now renders 'yes.', not the literal tag (double-escape bug on hanzo.team DMs/channels) 2026-07-22 14:06:26 -07:00
antje e6fd762b25 fix(team): flatten model HTML in bot replies — a bot answering '<p>yes.</p>' now renders 'yes.', not the literal tag (double-escape bug on hanzo.team DMs/channels) 2026-07-22 14:06:26 -07:00
antje f05b9a6441 feat(commerce): expose plan authority CRUD at /v1/plans/* + commerce v1.49.12
Increment 3a — the subscription/DNS plan CHARGE authority. Mirrors the 2b catalog
mount: register commerce's plan SuperAdmin CRUD (api/plan -> planApi.AdminRoute)
on cloud's /v1 bundle group, injecting the embed seed source
(commercebilling.SeedRows — the SAME @hanzo/plans embed the boot seed +
resolveSubscriptionPlan read), and own /v1/plans in commercePrefixes so it reaches
commerce, not the AI /v1/* 402 catch-all.

commerce v1.49.12 (feat/plans-sot cherry-picked onto v1.49.11): models/plan
authority + runPlansSeed (env COMMERCE_PLANS_SEED, idempotent/count-gated) +
resolveSubscriptionPlan reads the editable authority with embed fallback. The mint
gates score the IMMUTABLE embed (paidTier/IncludedMonthlyCents), NOT the DB price —
an admin edit never moves a charge gate. Seed == embed (TestSeededPricesEqualEmbed),
so first boot changes NO charge. Each handler requireSuperAdmin-gated (anon -> 403).
2026-07-22 13:55:06 -07:00
antje cb977a9633 feat(commerce): expose plan authority CRUD at /v1/plans/* + commerce v1.49.12
Increment 3a — the subscription/DNS plan CHARGE authority. Mirrors the 2b catalog
mount: register commerce's plan SuperAdmin CRUD (api/plan -> planApi.AdminRoute)
on cloud's /v1 bundle group, injecting the embed seed source
(commercebilling.SeedRows — the SAME @hanzo/plans embed the boot seed +
resolveSubscriptionPlan read), and own /v1/plans in commercePrefixes so it reaches
commerce, not the AI /v1/* 402 catch-all.

commerce v1.49.12 (feat/plans-sot cherry-picked onto v1.49.11): models/plan
authority + runPlansSeed (env COMMERCE_PLANS_SEED, idempotent/count-gated) +
resolveSubscriptionPlan reads the editable authority with embed fallback. The mint
gates score the IMMUTABLE embed (paidTier/IncludedMonthlyCents), NOT the DB price —
an admin edit never moves a charge gate. Seed == embed (TestSeededPricesEqualEmbed),
so first boot changes NO charge. Each handler requireSuperAdmin-gated (anon -> 403).
2026-07-22 13:55:06 -07:00
zeekayandClaude Opus 4.8 c829daeb45 feat(projects): dedicated POST /v1/projects/:slug/purge + emit Cache-Tag; DRY the edge purge
- Add POST /v1/projects/:slug/purge (mirrored at /v1/platform/sites/:slug/purge):
  resolve (org,slug), flush the edge cache-tag site-<org>-<slug>, stamp
  LastPurgeAt, persist, return 200 Project. Never touches the S3 origin. Org-scoped
  exactly like deploy (403 no org, 404 unknown slug). CF-unconfigured/failing purge
  is non-fatal (stamps + 200), matching deploy's behavior.
- DRY: extract purgeTag (the ONE cf.PurgeTags + cache-tag derivation + failure
  policy) and purgeEdge (purge + stamp). deploy(onPublish), setDomains, del, and the
  new purge handler all route through purgeTag — no copy-pasted CF call.
- Cache-Tag on served responses already emitted in sites.streamSite (unchanged);
  tightened its comment + CacheTag doc to state the emit/purge pairing invariant.
- Scrub stale svc suffix from prose (6→0): projectsvc→projects, tasksvc→tasks,
  cloud-mlsvc comment reworded; zapsvc test fixture → zap. msvc (third-party) left.
- Enforce one vocabulary in touched comments (project / deployment / S3 origin /
  edge cache-tag / purge); no behavior change.
- Tests: purge_test.go — stamps LastPurgeAt, CF-unconfigured→200, 403/404 scope,
  S3 origin untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 13:48:20 -07:00
zeekayandhanzo-dev d9fbc6af96 feat(projects): dedicated POST /v1/projects/:slug/purge + emit Cache-Tag; DRY the edge purge
- Add POST /v1/projects/:slug/purge (mirrored at /v1/platform/sites/:slug/purge):
  resolve (org,slug), flush the edge cache-tag site-<org>-<slug>, stamp
  LastPurgeAt, persist, return 200 Project. Never touches the S3 origin. Org-scoped
  exactly like deploy (403 no org, 404 unknown slug). CF-unconfigured/failing purge
  is non-fatal (stamps + 200), matching deploy's behavior.
- DRY: extract purgeTag (the ONE cf.PurgeTags + cache-tag derivation + failure
  policy) and purgeEdge (purge + stamp). deploy(onPublish), setDomains, del, and the
  new purge handler all route through purgeTag — no copy-pasted CF call.
- Cache-Tag on served responses already emitted in sites.streamSite (unchanged);
  tightened its comment + CacheTag doc to state the emit/purge pairing invariant.
- Scrub stale svc suffix from prose (6→0): projectsvc→projects, tasksvc→tasks,
  cloud-mlsvc comment reworded; zapsvc test fixture → zap. msvc (third-party) left.
- Enforce one vocabulary in touched comments (project / deployment / S3 origin /
  edge cache-tag / purge); no behavior change.
- Tests: purge_test.go — stamps LastPurgeAt, CF-unconfigured→200, 403/404 scope,
  S3 origin untouched.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-22 13:48:20 -07:00
hanzo-dev 2fb9c2e095 feat(integrations): ecommerce + AI-startup key connectors
One key-verify mechanism (keyVerify) plus the payment, model, messaging, and
SaaS connectors an online store or AI startup needs, on the user-scoped
/v1/connectors plane. Each provider is declarative data — origin, path,
credential placement, offline checks — over a single fail-closed, token-free
verify; a new connector is a registration, not another hand-rolled HTTP dance.

Providers: stripe, paypal, square, shopify; gemini, groq, mistral, cohere,
together, replicate, huggingface, openrouter, xai, fireworks, deepseek,
pinecone; sendgrid, resend, postmark, mailchimp, klaviyo, twilio; hubspot,
notion, linear, airtable.

A customer key seals to KMS only on a 2xx from the provider's cheapest
authenticated read; non-2xx and transport errors store nothing and never echo
the credential. authClient no longer follows redirects, so a 3xx fails closed
and custom credential headers are not forwarded across a hop.
2026-07-22 13:26:14 -07:00
hanzo-dev ebb773fe94 feat(integrations): ecommerce + AI-startup key connectors
One key-verify mechanism (keyVerify) plus the payment, model, messaging, and
SaaS connectors an online store or AI startup needs, on the user-scoped
/v1/connectors plane. Each provider is declarative data — origin, path,
credential placement, offline checks — over a single fail-closed, token-free
verify; a new connector is a registration, not another hand-rolled HTTP dance.

Providers: stripe, paypal, square, shopify; gemini, groq, mistral, cohere,
together, replicate, huggingface, openrouter, xai, fireworks, deepseek,
pinecone; sendgrid, resend, postmark, mailchimp, klaviyo, twilio; hubspot,
notion, linear, airtable.

A customer key seals to KMS only on a 2xx from the provider's cheapest
authenticated read; non-2xx and transport errors store nothing and never echo
the credential. authClient no longer follows redirects, so a 3xx fails closed
and custom credential headers are not forwarded across a hop.
2026-07-22 13:26:14 -07:00
hanzo-dev 1f1c5569d5 merge(main): final catch-up before connectors ship 2026-07-22 09:07:13 -07:00
hanzo-dev ee1435d146 merge(main): final catch-up before connectors ship 2026-07-22 09:07:13 -07:00
hanzo-dev 0a1ebbc0b8 merge(main): catch up before shipping connectors 2026-07-22 08:42:21 -07:00
hanzo-dev 230058ed9b merge(main): catch up before shipping connectors 2026-07-22 08:42:21 -07:00
a36b8d1ba7 build(deps): bump hanzoai/ai v1.829.7 → v1.830.0 (free-tier flash cap #143) (#351)
Pulls the free-tier auto-routing flash cap (ai#111): a CONFIDENT free-tier org's
`auto` is confined to the flash pool (blended-price ceiling), so a non-paying
caller is never routed to a premium model. Enso stays the default; paid/trial,
explicit model, and X-Max-Cost still win. Single-commit jump; cloud builds clean
against v1.830.0 (CGO_ENABLED=0).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-22 08:39:46 -07:00
01c6fc407b build(deps): bump hanzoai/ai v1.829.7 → v1.830.0 (free-tier flash cap #143) (#351)
Pulls the free-tier auto-routing flash cap (ai#111): a CONFIDENT free-tier org's
`auto` is confined to the flash pool (blended-price ceiling), so a non-paying
caller is never routed to a premium model. Enso stays the default; paid/trial,
explicit model, and X-Max-Cost still win. Single-commit jump; cloud builds clean
against v1.830.0 (CGO_ENABLED=0).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-22 08:39:46 -07:00
zandGitHub 6caf1826d7 sync: freshness reconcile scheduler + gitToken graceful fallback (git.hanzo.ai native cutover)
Periodic reconcile scheduler (clients/sync) that keeps every poll sync current (env-gated CLOUD_SYNC_RECONCILE_INTERVAL, leader-safe on the Recreate singleton writer, bounded by the shared reconcileSem), plus OrgStore.Each cross-org sweep primitive and store.ListAll. gitToken now falls back App-token -> GIT_MIRROR_TOKEN -> anonymous instead of hard-failing, so github reconciles (public immediately, private once GIT_MIRROR_TOKEN is set) stop freezing. Tests: scheduler_test, orgdb_test TestOrgStoreEach, git_provider_test TestGitTokenFallback; go vet ./... clean.
2026-07-22 03:52:39 -07:00
zandGitHub 0c49314de2 sync: freshness reconcile scheduler + gitToken graceful fallback (git.hanzo.ai native cutover)
Periodic reconcile scheduler (clients/sync) that keeps every poll sync current (env-gated CLOUD_SYNC_RECONCILE_INTERVAL, leader-safe on the Recreate singleton writer, bounded by the shared reconcileSem), plus OrgStore.Each cross-org sweep primitive and store.ListAll. gitToken now falls back App-token -> GIT_MIRROR_TOKEN -> anonymous instead of hard-failing, so github reconciles (public immediately, private once GIT_MIRROR_TOKEN is set) stop freezing. Tests: scheduler_test, orgdb_test TestOrgStoreEach, git_provider_test TestGitTokenFallback; go vet ./... clean.
2026-07-22 03:52:39 -07:00
antje b5167139d0 feat(commerce): expose SuperAdmin catalog CMS at /v1/catalog/* in the embed
admin.hanzo.ai's editor (console v8.4.154) CRUDs /v1/catalog/entries, but
commerce's setupRoutes wires only /v1/commerce/* — the /v1/catalog CMS
(api.Route -> catalogApi.AdminRoute, standalone-only) 404'd in the co-resident
binary. mountCommerce is the adapter for exactly these setupRoutes-skipped
families: mount catalogApi.AdminRoute on the existing /v1 bundle group (storeV1:
AddHost+RequestContext+errorScope+IAMTokenRequired — the same chain the
standalone /v1 bundle uses), and own /v1/catalog in commercePrefixes so it
reaches commerce, not the AI /v1/* balance catch-all.

No commerce module change: catalogApi.AdminRoute already ships in v1.49.9. Each
handler is requireSuperAdmin-gated (anon -> 401/403, cross-tenant data — never
org-scoped). Editor unchanged; closes the admin-edit half of the loop.
2026-07-22 03:30:36 -07:00
antje 320ff539c5 feat(commerce): expose SuperAdmin catalog CMS at /v1/catalog/* in the embed
admin.hanzo.ai's editor (console v8.4.154) CRUDs /v1/catalog/entries, but
commerce's setupRoutes wires only /v1/commerce/* — the /v1/catalog CMS
(api.Route -> catalogApi.AdminRoute, standalone-only) 404'd in the co-resident
binary. mountCommerce is the adapter for exactly these setupRoutes-skipped
families: mount catalogApi.AdminRoute on the existing /v1 bundle group (storeV1:
AddHost+RequestContext+errorScope+IAMTokenRequired — the same chain the
standalone /v1 bundle uses), and own /v1/catalog in commercePrefixes so it
reaches commerce, not the AI /v1/* balance catch-all.

No commerce module change: catalogApi.AdminRoute already ships in v1.49.9. Each
handler is requireSuperAdmin-gated (anon -> 401/403, cross-tenant data — never
org-scoped). Editor unchanged; closes the admin-edit half of the loop.
2026-07-22 03:30:36 -07:00
hanzo-dev 3337ddcdd5 deploy: bump embedded commerce v1.49.10 → v1.49.11 (billing-reads 502 class fix)
Picks up the #146 sibling fix: the co-resident billing READS (ListInvoices,
ListBillingSubscriptions, ListPayouts, GetPaymentConfig, DownloadInvoicePDF) no longer
nil-deref-panic (→502) on the embed path when no org is in Locals — they resolve the org
via nil-safe GetOrganizationOK. Compile-verified clean:
go build ./apps/... ./clients/account/... ./clients/commerceclient/...
2026-07-22 03:29:52 -07:00
hanzo-dev 470e12644b deploy: bump embedded commerce v1.49.10 → v1.49.11 (billing-reads 502 class fix)
Picks up the #146 sibling fix: the co-resident billing READS (ListInvoices,
ListBillingSubscriptions, ListPayouts, GetPaymentConfig, DownloadInvoicePDF) no longer
nil-deref-panic (→502) on the embed path when no org is in Locals — they resolve the org
via nil-safe GetOrganizationOK. Compile-verified clean:
go build ./apps/... ./clients/account/... ./clients/commerceclient/...
2026-07-22 03:29:52 -07:00
hanzo-dev 00dc99d834 deploy: bump embedded commerce v1.49.9 → v1.49.10 (spend-alerts 502 fix)
Picks up the #146 fix: GET /v1/billing/spend-alerts (the console Budgets page) no
longer panics (→ 502) on the co-resident embed path when no org is in Locals.
commerce ListSpendAlerts + its CRUD siblings + billingSubject now resolve the org
via the nil-safe GetOrganizationOK. Compile-verified clean:
go build ./apps/... ./clients/account/... ./clients/commerceclient/...
2026-07-22 03:09:37 -07:00
hanzo-dev 296fe6261a deploy: bump embedded commerce v1.49.9 → v1.49.10 (spend-alerts 502 fix)
Picks up the #146 fix: GET /v1/billing/spend-alerts (the console Budgets page) no
longer panics (→ 502) on the co-resident embed path when no org is in Locals.
commerce ListSpendAlerts + its CRUD siblings + billingSubject now resolve the org
via the nil-safe GetOrganizationOK. Compile-verified clean:
go build ./apps/... ./clients/account/... ./clients/commerceclient/...
2026-07-22 03:09:37 -07:00
hanzo-dev 41b992e7fd Merge remote-tracking branch 'origin/main' into consolidate/held-work 2026-07-22 03:03:17 -07:00
hanzo-dev 21626ec7b8 Merge remote-tracking branch 'origin/main' into consolidate/held-work 2026-07-22 03:03:17 -07:00
hanzo-dev 8b1ba262aa merge(connectors): OAuth/apikey connector plane (anthropic, openai, copilot, device, refresh)
# Conflicts:
#	clients/integrations/integrations.go
2026-07-22 02:56:27 -07:00
hanzo-dev f0d7dc07be merge(connectors): OAuth/apikey connector plane (anthropic, openai, copilot, device, refresh)
# Conflicts:
#	clients/integrations/integrations.go
2026-07-22 02:56:27 -07:00
antje 2898ae3670 deploy: bump embedded commerce v1.49.8 → v1.49.9
Increment-1 catalog SoT: commerce Bootstrap now seeds the 17 infra-tier
catalogentry rows (11 cloud + 3 gpu + 3 datastore) via runInfraCatalogSeed
(count-gated, idempotent, non-fatal) and projects an infra brand scope +
Metadata at GET /v1/commerce/catalog?brand=infra. Purely additive; billing/
orders/Stripe untouched.
2026-07-22 02:52:44 -07:00
antje 00b6293a65 deploy: bump embedded commerce v1.49.8 → v1.49.9
Increment-1 catalog SoT: commerce Bootstrap now seeds the 17 infra-tier
catalogentry rows (11 cloud + 3 gpu + 3 datastore) via runInfraCatalogSeed
(count-gated, idempotent, non-fatal) and projects an infra brand scope +
Metadata at GET /v1/commerce/catalog?brand=infra. Purely additive; billing/
orders/Stripe untouched.
2026-07-22 02:52:44 -07:00
hanzo-dev 6fd497221d merge(deps): commerce v1.49.3 bump + flags/metering test follow-through
# Conflicts:
#	go.mod
#	go.sum
#	hanzo.yml
2026-07-22 02:46:33 -07:00
hanzo-dev 3766f6ba0c merge(deps): commerce v1.49.3 bump + flags/metering test follow-through
# Conflicts:
#	go.mod
#	go.sum
#	hanzo.yml
2026-07-22 02:46:33 -07:00
antje 6d93966ad8 fix(team): run the entitlement/guest-cap gate at sendInvite too
The team.guests cap ran only at selectWorkspace (the guest's later login),
never at sendInvite — the actual guest add. A workspace admin could invite
guests past the plan cap and the add itself was ungated. Call the SAME
entitle seam at the add point, OBSERVE-MODE consistent with the login gate:
it logs the over-cap denial and admits today, and the 402 arms the day
enforcement returns. Commerce/plans errors admit inside entitle, so a
licensing outage never bricks an invite. Tests prove the gate runs at the
add point (commerce + plan cap consulted, genuine over-cap rank) yet admits,
and that an infra error still records the invite.
2026-07-22 01:53:55 -07:00
antje 67f6111918 fix(team): run the entitlement/guest-cap gate at sendInvite too
The team.guests cap ran only at selectWorkspace (the guest's later login),
never at sendInvite — the actual guest add. A workspace admin could invite
guests past the plan cap and the add itself was ungated. Call the SAME
entitle seam at the add point, OBSERVE-MODE consistent with the login gate:
it logs the over-cap denial and admits today, and the 402 arms the day
enforcement returns. Commerce/plans errors admit inside entitle, so a
licensing outage never bricks an invite. Tests prove the gate runs at the
add point (commerce + plan cap consulted, genuine over-cap rank) yet admits,
and that an infra error still records the invite.
2026-07-22 01:53:55 -07:00
antje a8570e0bd5 fix(team/collab): close seedYLog TOCTOU — route the seed through the hub
createContent seeded a brand-new doc's live update log with a bare
Get(miss)-then-Put. In the gap between the miss and the Put, a concurrent
first edit on the live WS lane could land and then be overwritten by the
seed (lost update), or the seed could overwrite a log that just went live.
Seed THROUGH the collab hub (seedIfAbsent): it reuses the room's flushMu/mu,
so the empty-check and the set are atomic under the room lock and the seed
and the field's live editor serialize on ONE room. The seed applies only
when the log is still empty — never clobbers a live edit. Deterministic +
concurrent (-race) tests prove the live edit always survives.
2026-07-22 01:48:48 -07:00
antje 9187dcdd6e fix(team/collab): close seedYLog TOCTOU — route the seed through the hub
createContent seeded a brand-new doc's live update log with a bare
Get(miss)-then-Put. In the gap between the miss and the Put, a concurrent
first edit on the live WS lane could land and then be overwritten by the
seed (lost update), or the seed could overwrite a log that just went live.
Seed THROUGH the collab hub (seedIfAbsent): it reuses the room's flushMu/mu,
so the empty-check and the set are atomic under the room lock and the seed
and the field's live editor serialize on ONE room. The seed applies only
when the log is still empty — never clobbers a live edit. Deterministic +
concurrent (-race) tests prove the live edit always survives.
2026-07-22 01:48:48 -07:00
antje e4538bcc89 fix(team): converge concurrent logins to one personal workspace
EnsureWorkspace was check-then-insert (WorkspacesOf -> INSERT) with no
uniqueness on the personal-workspace identity, so two concurrent logins for
the same (org, account) could each see 'none' and mint a duplicate personal
workspace (reproduced: 32 concurrent logins -> 6 rows). Add a partial-unique
index on (owner_org, owner) — EnsureWorkspace is the sole creator and always
writes owner=account — with a dedup that converges any pre-existing duplicates
to the earliest row, and make the create an idempotent upsert (ON CONFLICT DO
NOTHING then adopt the winner). Concurrent EnsureWorkspace now yields exactly
one row. Heal logic extracted to adoptExisting, reused on both paths.
2026-07-22 01:44:56 -07:00
antje 701c1f794a fix(team): converge concurrent logins to one personal workspace
EnsureWorkspace was check-then-insert (WorkspacesOf -> INSERT) with no
uniqueness on the personal-workspace identity, so two concurrent logins for
the same (org, account) could each see 'none' and mint a duplicate personal
workspace (reproduced: 32 concurrent logins -> 6 rows). Add a partial-unique
index on (owner_org, owner) — EnsureWorkspace is the sole creator and always
writes owner=account — with a dedup that converges any pre-existing duplicates
to the earliest row, and make the create an idempotent upsert (ON CONFLICT DO
NOTHING then adopt the winner). Concurrent EnsureWorkspace now yields exactly
one row. Heal logic extracted to adoptExisting, reused on both paths.
2026-07-22 01:44:56 -07:00
antje 082cec000c fix(team): surface a real seat-read error instead of a false 0 members
Seats swallowed the row Scan error, so a genuine DB failure reported (0, 0)
— indistinguishable from an org with no members, under-counting billed
seats. Propagate the error (the aggregate COUNT always returns one row, so
any Scan error is a real failure); the wallet's /billing/plan read now
502s honestly and retries instead of rendering a truthful-looking 0.
2026-07-22 01:40:23 -07:00
antje 87cc4db15e fix(team): surface a real seat-read error instead of a false 0 members
Seats swallowed the row Scan error, so a genuine DB failure reported (0, 0)
— indistinguishable from an org with no members, under-counting billed
seats. Propagate the error (the aggregate COUNT always returns one row, so
any Scan error is a real failure); the wallet's /billing/plan read now
502s honestly and retries instead of rendering a truthful-looking 0.
2026-07-22 01:40:23 -07:00
antje 599e063871 fix(auth): console identity from the ONE validated principal, not casibase
DECOMPLECTION. The operator SPA (admin.hanzo.ai) authenticates via /v1/signin (a
cloud PKCE session → the X-User-* principal the gateway/middleware mints) but read
its IDENTITY from /v1/get-account, which the embedded IAM (casibase) answers from ITS
OWN session cookie. A PKCE session is not a casibase session, so get-account returned
owner:"hanzo" (anonymous) or "Unauthorized operation" — and the SPA SuperAdmin gate
(owner==admin && isAdmin), reading that, bounced the operator UI to login even though
the SAME session got 200 from every /v1/admin/* route. Identity source and auth source
disagreed: two session models for one surface.

Now identity IS the principal. AccountFromPrincipal (registered after IdentityMiddleware,
before MountAll's casibase mount) answers /v1/get-account from X-User-Owner/Name/Email/
IsAdmin when a VALIDATED principal is present (owner = HOME org, so a SuperAdmin
org-switched into a tenant stays one); with no principal it falls through to casibase
unchanged (anonymous sign-in + legacy casibase-session callers untouched). One truth,
additive, fail-open. Fixes the operator console UI being unusable via browser login.
2026-07-22 01:36:35 -07:00
antje b8253bf75e fix(auth): console identity from the ONE validated principal, not casibase
DECOMPLECTION. The operator SPA (admin.hanzo.ai) authenticates via /v1/signin (a
cloud PKCE session → the X-User-* principal the gateway/middleware mints) but read
its IDENTITY from /v1/get-account, which the embedded IAM (casibase) answers from ITS
OWN session cookie. A PKCE session is not a casibase session, so get-account returned
owner:"hanzo" (anonymous) or "Unauthorized operation" — and the SPA SuperAdmin gate
(owner==admin && isAdmin), reading that, bounced the operator UI to login even though
the SAME session got 200 from every /v1/admin/* route. Identity source and auth source
disagreed: two session models for one surface.

Now identity IS the principal. AccountFromPrincipal (registered after IdentityMiddleware,
before MountAll's casibase mount) answers /v1/get-account from X-User-Owner/Name/Email/
IsAdmin when a VALIDATED principal is present (owner = HOME org, so a SuperAdmin
org-switched into a tenant stays one); with no principal it falls through to casibase
unchanged (anonymous sign-in + legacy casibase-session callers untouched). One truth,
additive, fail-open. Fixes the operator console UI being unusable via browser login.
2026-07-22 01:36:35 -07:00
antje 062e0d4374 fix(finance): never show held funds as spendable in the balance projection
The commerce S2S fallback set available = balance whenever the reported
available was 0, so a fully-held wallet (balance == holds, available 0)
rendered its entire balance as spendable — money the prepaid gate would
refuse. Derive available as balance NET OF holds when commerce reports only
a balance, clamped at 0 (holds beyond balance never go negative). Extracted
as spendableCents with a table test; a fully-held org proves the handler.
2026-07-22 01:36:03 -07:00
antje 0bbbf3c358 fix(finance): never show held funds as spendable in the balance projection
The commerce S2S fallback set available = balance whenever the reported
available was 0, so a fully-held wallet (balance == holds, available 0)
rendered its entire balance as spendable — money the prepaid gate would
refuse. Derive available as balance NET OF holds when commerce reports only
a balance, clamped at 0 (holds beyond balance never go negative). Extracted
as spendableCents with a table test; a fully-held org proves the handler.
2026-07-22 01:36:03 -07:00
antje f88405f6a6 fix(team): cap workspace-invite org grant at member — no org-admin escalation
A workspace invite passed the body role straight into the org-level IAM
grant, so a single-workspace admin could make an invitee an org IAM admin/
owner (trusted elsewhere via IsAdmin). Cap the org grant at member; the rich
role stays workspace-scoped on the local roster row. Test proves the cap.
2026-07-22 01:01:50 -07:00
antje 01cd52930e fix(team): cap workspace-invite org grant at member — no org-admin escalation
A workspace invite passed the body role straight into the org-level IAM
grant, so a single-workspace admin could make an invitee an org IAM admin/
owner (trusted elsewhere via IsAdmin). Cap the org grant at member; the rich
role stays workspace-scoped on the local roster row. Test proves the cap.
2026-07-22 01:01:50 -07:00
antje 35dbdbc589 fix(team/collab): close room-resurrection race (panic) + stop lost updates on flush
Two defects in the single-replica collab hub (collabws.go):

1) BLOCKING race → 'panic: close of closed channel' + registry corruption. join
   registered its peer in rm.peers AFTER releasing h.mu, so in that gap a concurrent
   leave of the last old peer would GC the room (delete h.rooms[key] + close(rm.stop),
   ending the flusher). The resurrected room then panicked on its next leave
   (close of an already-closed channel) and could evict a live room (split-brain).
   This is the ordinary 'reconnect by the sole editor' shape. Fix: register the peer
   while STILL holding h.mu, before the slow VFS load — a concurrent leave can no
   longer see the room as empty. Lock order stays h->rm. A failed first-open now
   undoes the join via leave() so it leaks neither the peer nor the room+flusher.

2) Silent lost updates on flush. flush cleared rm.dirty BEFORE the Put and discarded
   the Put error, so a transient VFS failure dropped the buffered window (dirty already
   false => neither the ticker nor last-leave retried). Fix: clear dirty only on a
   SUCCESSFUL Put, re-arm it on failure, and serialize the persist (flushMu) so two
   overlapping flushes cannot reorder their Puts.

TDD: TestCollabHubConcurrentJoinLeaveNoPanicNoLeak hammers join+leave on one doc from
500 goroutines under -race (reproduces the panic + a room leak without the fix, clean
with it). TestCollabFlushRetriesAfterPutError proves a failed Put keeps the room dirty
and nothing is persisted, then a later Put persists and clears dirty.
2026-07-22 00:53:51 -07:00
antje 478f50760b fix(team/collab): close room-resurrection race (panic) + stop lost updates on flush
Two defects in the single-replica collab hub (collabws.go):

1) BLOCKING race → 'panic: close of closed channel' + registry corruption. join
   registered its peer in rm.peers AFTER releasing h.mu, so in that gap a concurrent
   leave of the last old peer would GC the room (delete h.rooms[key] + close(rm.stop),
   ending the flusher). The resurrected room then panicked on its next leave
   (close of an already-closed channel) and could evict a live room (split-brain).
   This is the ordinary 'reconnect by the sole editor' shape. Fix: register the peer
   while STILL holding h.mu, before the slow VFS load — a concurrent leave can no
   longer see the room as empty. Lock order stays h->rm. A failed first-open now
   undoes the join via leave() so it leaks neither the peer nor the room+flusher.

2) Silent lost updates on flush. flush cleared rm.dirty BEFORE the Put and discarded
   the Put error, so a transient VFS failure dropped the buffered window (dirty already
   false => neither the ticker nor last-leave retried). Fix: clear dirty only on a
   SUCCESSFUL Put, re-arm it on failure, and serialize the persist (flushMu) so two
   overlapping flushes cannot reorder their Puts.

TDD: TestCollabHubConcurrentJoinLeaveNoPanicNoLeak hammers join+leave on one doc from
500 goroutines under -race (reproduces the panic + a room leak without the fix, clean
with it). TestCollabFlushRetriesAfterPutError proves a failed Put keeps the room dirty
and nothing is persisted, then a later Put persists and clears dirty.
2026-07-22 00:53:51 -07:00
antje 5c8b5c9a08 fix(finance): surface a real balance-read error instead of rendering it as $0
ledgerFinance.Balance discarded store.Balance's error (bal, _ := ...), so a real
read failure — a DB Scan error or a corrupt/garbled stored-balance ParseInt failure —
returned (zero, nil), indistinguishable from a genuine zero. That defeated the
documented 'a balance that cannot be read is unknown, and unknown is not broke'
invariant every caller relies on: /v1/billing/balance and /v1/finance/balance showed
a FUNDED customer $0.00, and the AI prepaid gate (metering.fetchAvailable) refused a
funded org's request. Propagate the error; the callers' guards already render 503 on
a non-nil read error. Direction was already safe (zero only under-reports), so this is
correctness + observability, not an over-charge.

TDD: TestBalanceReadErrorSurfaces closes the store's DB out from under the cached
handle and asserts Balance now returns an error, not (0,nil). Fails before, passes after.
2026-07-22 00:53:51 -07:00
antje 26b2f13f8a fix(finance): surface a real balance-read error instead of rendering it as $0
ledgerFinance.Balance discarded store.Balance's error (bal, _ := ...), so a real
read failure — a DB Scan error or a corrupt/garbled stored-balance ParseInt failure —
returned (zero, nil), indistinguishable from a genuine zero. That defeated the
documented 'a balance that cannot be read is unknown, and unknown is not broke'
invariant every caller relies on: /v1/billing/balance and /v1/finance/balance showed
a FUNDED customer $0.00, and the AI prepaid gate (metering.fetchAvailable) refused a
funded org's request. Propagate the error; the callers' guards already render 503 on
a non-nil read error. Direction was already safe (zero only under-reports), so this is
correctness + observability, not an over-charge.

TDD: TestBalanceReadErrorSurfaces closes the store's DB out from under the cached
handle and asserts Balance now returns an error, not (0,nil). Fails before, passes after.
2026-07-22 00:53:51 -07:00
hanzo-dev 46c0d0e11f merge(main): bring the integration up to current main before shipping 2026-07-22 00:10:59 -07:00
hanzo-dev 6dda723198 merge(main): bring the integration up to current main before shipping 2026-07-22 00:10:59 -07:00
hanzo-dev 86b0b79ecc fix(cli/code): size Codex context window to the served model, not a flat 262144
The codexLike provider hardcoded '-c model_context_window=262144' (+auto_compact
235929) for EVERY model, so enso / zen5 — which the gateway serves at 1M
(ai flagshipWindow pin, live) — were capped at 256K in the 'hanzo code' Codex
wrapper. That surfaced to the user as 'maximum context exceeded' at 262144 even
after switching to zen5-pro.

Make the window model-aware: codeContextWindow(model) → 1M for the enso/zen5
flagship tiers, 131072 for flash; auto-compact at 90%. Threads the served model
into provider(base, model). Only codexLike sets provider (codex has no carrier,
so its model IS the served zen id). Tests updated + a sizing test added.

Claude-Session: https://claude.ai/code/session_01QSN1woYbvENByMbGUqQ9Me
2026-07-21 23:58:19 -07:00
hanzo-dev 47764c02a8 fix(cli/code): size Codex context window to the served model, not a flat 262144
The codexLike provider hardcoded '-c model_context_window=262144' (+auto_compact
235929) for EVERY model, so enso / zen5 — which the gateway serves at 1M
(ai flagshipWindow pin, live) — were capped at 256K in the 'hanzo code' Codex
wrapper. That surfaced to the user as 'maximum context exceeded' at 262144 even
after switching to zen5-pro.

Make the window model-aware: codeContextWindow(model) → 1M for the enso/zen5
flagship tiers, 131072 for flash; auto-compact at 90%. Threads the served model
into provider(base, model). Only codexLike sets provider (codex has no carrier,
so its model IS the served zen id). Tests updated + a sizing test added.
2026-07-21 23:58:19 -07:00
antje 12d0081e3b fix(analytics): stop /v1/analytics 500 — replace panicking sync.Map with mutex+map
The deprecated-alias 'log once per path' used a package sync.Map (Go's HashTrieMap),
which entered a 'ran out of hash bits while inserting' panic state under the hot
ingest path — so the Recover middleware turned EVERY POST /v1/analytics (and /batch,
/tracker) into a 500. Events stopped landing in hanzo.events once the map degraded.
The alias set is tiny (3 paths); swap the sync.Map for a mutex-guarded map — no trie
state to corrupt, cannot panic. Behavior unchanged (still logs each alias once).
go build ./... green.
2026-07-21 23:29:10 -07:00
antje 20d0e9465d fix(analytics): stop /v1/analytics 500 — replace panicking sync.Map with mutex+map
The deprecated-alias 'log once per path' used a package sync.Map (Go's HashTrieMap),
which entered a 'ran out of hash bits while inserting' panic state under the hot
ingest path — so the Recover middleware turned EVERY POST /v1/analytics (and /batch,
/tracker) into a 500. Events stopped landing in hanzo.events once the map degraded.
The alias set is tiny (3 paths); swap the sync.Map for a mutex-guarded map — no trie
state to corrupt, cannot panic. Behavior unchanged (still logs each alias once).
go build ./... green.
2026-07-21 23:29:10 -07:00
antje 1256cda741 fix(agents): retry + model-failover on transient upstream overload so bot replies stop dropping
The agent-run path made ONE completion call; when the default agent model
(deepseek-v4-flash) returned a transient upstream 429 'Platform overloaded'
(~1 in 3 under load), the run recorded an error and the bot reply was dropped.

- types: add ErrUpstreamBusy sentinel — the shared vocabulary for a transient,
  safely-retryable upstream failure (429/5xx/empty-choices/'overloaded').
- aihttp: classify + tag transient chat-completion failures with ErrUpstreamBusy
  (errors.Is-detectable); permanent errors (400/auth/unserved model) stay
  untagged and fail fast. Message preserved; no control-flow change for
  non-agent callers (interactive chat untouched).
- agents/executeRun: bounded retry (3 attempts, equal-jittered backoff, ctx-aware)
  on ErrUpstreamBusy, then ONE failover to the reliable model (CLOUD_AI_FALLBACK_MODEL,
  default 'best') if the agent's own model stays throttled. Retrying a completion
  is side-effect-free, so metering still debits EXACTLY once on the eventual
  success (runAgent meters only r.Status==ok) and never on failed attempts.
- Bill the model ACTUALLY used (r.Model) — a failover run bills 'best', not the
  throttled model it started on.
- Scoped to the autonomous agent/bot run path ONLY; interactive user-facing
  chat/completions behavior is unchanged.

TDD: 429-twice-then-200 -> one ok run + one debit; persistent 429 + failover
exhausted -> clean error run + no debit; failover-success bills the used model;
transient-classifier unit test (429/503/500/empty-choices busy, 400 not).
2026-07-21 23:17:25 -07:00
antje 347b96171e fix(agents): retry + model-failover on transient upstream overload so bot replies stop dropping
The agent-run path made ONE completion call; when the default agent model
(deepseek-v4-flash) returned a transient upstream 429 'Platform overloaded'
(~1 in 3 under load), the run recorded an error and the bot reply was dropped.

- types: add ErrUpstreamBusy sentinel — the shared vocabulary for a transient,
  safely-retryable upstream failure (429/5xx/empty-choices/'overloaded').
- aihttp: classify + tag transient chat-completion failures with ErrUpstreamBusy
  (errors.Is-detectable); permanent errors (400/auth/unserved model) stay
  untagged and fail fast. Message preserved; no control-flow change for
  non-agent callers (interactive chat untouched).
- agents/executeRun: bounded retry (3 attempts, equal-jittered backoff, ctx-aware)
  on ErrUpstreamBusy, then ONE failover to the reliable model (CLOUD_AI_FALLBACK_MODEL,
  default 'best') if the agent's own model stays throttled. Retrying a completion
  is side-effect-free, so metering still debits EXACTLY once on the eventual
  success (runAgent meters only r.Status==ok) and never on failed attempts.
- Bill the model ACTUALLY used (r.Model) — a failover run bills 'best', not the
  throttled model it started on.
- Scoped to the autonomous agent/bot run path ONLY; interactive user-facing
  chat/completions behavior is unchanged.

TDD: 429-twice-then-200 -> one ok run + one debit; persistent 429 + failover
exhausted -> clean error run + no debit; failover-success bills the used model;
transient-classifier unit test (429/503/500/empty-choices busy, 400 not).
2026-07-21 23:17:25 -07:00
hanzo-dev cf5d807cf0 fix(sites): complete the coalesce+ceiling WIP — add sync/strconv imports + fixed-window takeToken() 2026-07-21 22:39:55 -07:00
hanzo-dev d486e3f108 fix(sites): complete the coalesce+ceiling WIP — add sync/strconv imports + fixed-window takeToken() 2026-07-21 22:39:55 -07:00
hanzo-dev 3c4e05632c merge(site-releases): consolidate onto main 2026-07-21 22:38:35 -07:00
hanzo-dev 57b673bfe0 merge(site-releases): consolidate onto main 2026-07-21 22:38:35 -07:00
hanzo-dev 93e23b541d merge(channels): consolidate onto main
# Conflicts:
#	apps/apps.go
#	apps/wire_test.go
#	clients/channels/routes.go
2026-07-21 22:38:22 -07:00
hanzo-dev 007cccfd03 merge(channels): consolidate onto main
# Conflicts:
#	apps/apps.go
#	apps/wire_test.go
#	clients/channels/routes.go
2026-07-21 22:38:22 -07:00
hanzo-dev 1d8de1dda7 merge(cloudflare-connector): consolidate onto main
# Conflicts:
#	clients/integrations/cloudflare.go
#	clients/integrations/cloudflare_test.go
#	clients/integrations/integrations.go
2026-07-21 22:37:57 -07:00
hanzo-dev 318047fc59 merge(cloudflare-connector): consolidate onto main
# Conflicts:
#	clients/integrations/cloudflare.go
#	clients/integrations/cloudflare_test.go
#	clients/integrations/integrations.go
2026-07-21 22:37:57 -07:00
hanzo-dev 56dd542879 merge(account-usage): consolidate onto main
# Conflicts:
#	clients/link/http.go
#	clients/link/store.go
2026-07-21 22:30:47 -07:00
hanzo-dev 0177b19bbd merge(account-usage): consolidate onto main
# Conflicts:
#	clients/link/http.go
#	clients/link/store.go
2026-07-21 22:30:47 -07:00
hanzo-dev 78a8aa3310 merge(link-router): consolidate onto main
# Conflicts:
#	clients/link/http.go
2026-07-21 22:30:05 -07:00
hanzo-dev e7cb198c06 merge(link-router): consolidate onto main
# Conflicts:
#	clients/link/http.go
2026-07-21 22:30:05 -07:00
hanzo-dev e06eeb18da merge(leaderboard): consolidate onto main
# Conflicts:
#	apps/apps.go
2026-07-21 22:16:01 -07:00
hanzo-dev 5d1da72b38 merge(leaderboard): consolidate onto main
# Conflicts:
#	apps/apps.go
2026-07-21 22:16:01 -07:00
hanzo-dev 7f2deef78c merge(cd-projection-clusters-projects-stream): consolidate onto main
# Conflicts:
#	clients/deploy/dashboard.go
#	clients/deploy/dashboard_endpoints_test.go
#	clients/deploy/deploy_test.go
#	clients/deploy/projection.go
#	clients/deploy/stream.go
2026-07-21 22:12:33 -07:00
hanzo-dev 65f7106436 merge(cd-projection-clusters-projects-stream): consolidate onto main
# Conflicts:
#	clients/deploy/dashboard.go
#	clients/deploy/dashboard_endpoints_test.go
#	clients/deploy/deploy_test.go
#	clients/deploy/projection.go
#	clients/deploy/stream.go
2026-07-21 22:12:33 -07:00
hanzo-dev cf24f0cbfc merge(admin-billing-credit): consolidate onto main 2026-07-21 22:09:28 -07:00
hanzo-dev 88d987c61e merge(admin-billing-credit): consolidate onto main 2026-07-21 22:09:28 -07:00
hanzo-dev df250b65db merge(admin-credit-grant): consolidate onto main
# Conflicts:
#	clients/admin/commerce/commerce.go
2026-07-21 22:09:12 -07:00
hanzo-dev 8390b5cab9 merge(admin-credit-grant): consolidate onto main
# Conflicts:
#	clients/admin/commerce/commerce.go
2026-07-21 22:09:12 -07:00
hanzo-dev 8bcfb8eff3 merge(analytics-capture-v2): consolidate onto main
# Conflicts:
#	clients/analytics/analytics.go
#	clients/analytics/capture.go
2026-07-21 22:07:28 -07:00
hanzo-dev 2eaa0f3df4 merge(analytics-capture-v2): consolidate onto main
# Conflicts:
#	clients/analytics/analytics.go
#	clients/analytics/capture.go
2026-07-21 22:07:28 -07:00
hanzo-dev 7f3c381570 merge(zen-upstream-key-env-fallback): consolidate onto main
# Conflicts:
#	apps/zen.go
#	apps/zen_key_test.go
2026-07-21 22:03:03 -07:00
hanzo-dev b26389ad15 merge(zen-upstream-key-env-fallback): consolidate onto main
# Conflicts:
#	apps/zen.go
#	apps/zen_key_test.go
2026-07-21 22:03:03 -07:00
hanzo-dev 78a9c1f02e merge(agents-typed-ops): consolidate onto main 2026-07-21 21:55:48 -07:00
hanzo-dev d75a341f9f merge(agents-typed-ops): consolidate onto main 2026-07-21 21:55:48 -07:00
hanzo-dev abcbbb155b feat(gpu): HANZO_STUDIO_VRAM env overrides studio vram mode
Default stays --normalvram (safe for small BYO GPUs); big-memory boxes (GB10
128G unified) can set HANZO_STUDIO_VRAM=--highvram so the render backend keeps
models GPU-resident. Additive + opt-in; other workers unchanged.
2026-07-21 21:44:51 -07:00
hanzo-dev fab042bbc5 feat(gpu): HANZO_STUDIO_VRAM env overrides studio vram mode
Default stays --normalvram (safe for small BYO GPUs); big-memory boxes (GB10
128G unified) can set HANZO_STUDIO_VRAM=--highvram so the render backend keeps
models GPU-resident. Additive + opt-in; other workers unchanged.
2026-07-21 21:44:51 -07:00
48d3b35e67 refactor(iam): drop Casdoor iam-v1 entirely — cloud embeds the clean hanzoai/iam (#349)
cloud's last tie to the retired Casdoor/Beego fork (hanzoai/iam-v1) is gone:

- clients/iam: embeds the clean hanzoai/iam (zip-native + hanzoai/orm) via
  iamserver.Mount over its own SQLite store; fail-closed 503 on boot failure.
  Removes the dual-impl identitySpec (CLOUD_IAM_IMPL=iam2 branch) + clients/iam2.
- clients/platform + clients/deploy: read the IAM-owned Project resource via
  iam/pkg/store over the embedded DB() instead of iam-v1's object-store ormer.
- cmd/hanzo: retires the `hanzo iam` Casdoor daemon subcommand.
- go.mod: the last transitive iam-v1 edge was hanzoai/ai/object; bump ai
  v1.829.3 -> v1.829.4 (Casdoor-free cut). go mod tidy drops iam-v1 entirely.

Remaining iam-v1 references are accurate "retired/gone" doc comments only.
Tests green: clients/iam, clients/platform, clients/deploy. Wire frozen-order
golden consistent (iam2->iam collapse).

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 20:19:11 -07:00
5e60fb5ab9 refactor(iam): drop Casdoor iam-v1 entirely — cloud embeds the clean hanzoai/iam (#349)
cloud's last tie to the retired Casdoor/Beego fork (hanzoai/iam-v1) is gone:

- clients/iam: embeds the clean hanzoai/iam (zip-native + hanzoai/orm) via
  iamserver.Mount over its own SQLite store; fail-closed 503 on boot failure.
  Removes the dual-impl identitySpec (CLOUD_IAM_IMPL=iam2 branch) + clients/iam2.
- clients/platform + clients/deploy: read the IAM-owned Project resource via
  iam/pkg/store over the embedded DB() instead of iam-v1's object-store ormer.
- cmd/hanzo: retires the `hanzo iam` Casdoor daemon subcommand.
- go.mod: the last transitive iam-v1 edge was hanzoai/ai/object; bump ai
  v1.829.3 -> v1.829.4 (Casdoor-free cut). go mod tidy drops iam-v1 entirely.

Remaining iam-v1 references are accurate "retired/gone" doc comments only.
Tests green: clients/iam, clients/platform, clients/deploy. Wire frozen-order
golden consistent (iam2->iam collapse).

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-21 20:19:11 -07:00
antje 3fef8bccce fix(webui): serve each route's OWN exported shell — ends the OAuth login loop
The embedded console served index.html for EVERY non-asset path, so a deep
load of /auth/callback hydrated '/' instead; AuthGate discarded the ?code and
bounced to /signin — sign-in could never complete on the embedded console.
Now an extensionless path tries the static-export route shell (<route>.html)
first, with the index treatment (no-cache + white-label title rewrite, one
shared brandTitle); direct .html requests are also no-cache.
2026-07-21 19:40:14 -07:00
antje c2a9e8e6c6 fix(webui): serve each route's OWN exported shell — ends the OAuth login loop
The embedded console served index.html for EVERY non-asset path, so a deep
load of /auth/callback hydrated '/' instead; AuthGate discarded the ?code and
bounced to /signin — sign-in could never complete on the embedded console.
Now an extensionless path tries the static-export route shell (<route>.html)
first, with the index treatment (no-cache + white-label title rewrite, one
shared brandTitle); direct .html requests are also no-cache.
2026-07-21 19:40:14 -07:00
edd5f16375 chore(deps): bump hanzoai/ai → v1.829.7 (admin.* signin redeems as admin-console; admin-login P0) (#350)
Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-21 18:50:56 -07:00
hanzo-dev c593b50252 chore(deps): bump hanzoai/ai v1.829.5 → v1.829.6
Router-config routes (/v1/router/{policy,defaults,ledger,rewards,artifact-meta}
and /v1/org/settings) now serve over api.hanzo.ai. The v1.829.5 refactor moved them
to ZAP-native handlers and deleted their beego routes, but :8000 is the beego
web.Router — the ZAP registry backs a separate transport — so they 404'd in prod
(console Router->Policy, chat/app routing defaults). v1.829.6 adds RouterConfigBridge
(one beego adapter dispatching in-process through the SAME gateway registry, so the
ZAP handler stays the sole impl) and restores their isBalanceExempt entries.
2026-07-21 18:18:26 -07:00
hanzo-dev 3e2e1d84be merge: detect AMD GPUs in node inventory (evo gfx1151 was invisible) 2026-07-21 18:11:28 -07:00
hanzo-dev ec777cc40e link: detect AMD GPUs in the node inventory (rocm-smi / kfd topology / vulkaninfo)
detectGPUs now reports AMD accelerators as first-class resources alongside NVIDIA
and Apple Metal — discrete Radeon cards and gfx APUs alike (evo's gfx1151 Radeon
8060S on the RYZEN AI MAX+ 395). Resolution order: rocm-smi --showproductname (name
+ gfx target), then the kfd topology under /sys (GPU nodes by simd_count, gfx from
gfx_target_version), then a vulkaninfo summary; VRAM filled from amdgpu sysfs
mem_info_vram_total. Pure parsers are unit-tested against evo's real rocm-smi CSV
and kfd properties; the gfx_target_version decode (110501 → gfx1151) is covered.
2026-07-21 18:06:33 -07:00
hanzo-dev 1faadab30e merge: hanzo link|unlink|status — bring a machine into the fleet as a node
The unified Go binary is hanzo dev — the unified Hanzo Go binary

Usage:
  hanzo <command> [flags]

Control commands (gcloud/doctl-style):
  agent        invoke a managed Hanzo agent to run a task (headless)
  apps         list/get the platform apps board (declared/running/drift)
  auth         manage authentication + stored identities (login, logout, whoami, list, switch, token)
  bot          launch a computer-using agent (booted desktop or terminal)
  build        enqueue a platform-native build (runner fabric)
  clusters     provision/list/select dedicated DOKS clusters
  code         launch a coding agent (claude, codex, dev) on a Hanzo cloud model
  config       view/edit ~/.hanzo/config preferences
  deploy       drive a platform redeploy (rolling restart, zero-downtime)
  engine       run a local hanzo-engine (OpenAI + Anthropic model server)
  k8s          deploy-target helpers (current target)
  link         bring this machine into the Hanzo cloud fleet as a node (fabric + compute worker)
  login        authenticate against Hanzo IAM (hanzo.id) and store a token
  logout       remove stored credentials
  run          launch a workload on Hanzo compute (container or function)
  runner       run this machine as a JIT CI runner for your org (GitHub Actions)
  security     scan files for hardcoded secrets (local guardrail; no server/auth)
  status       show the org's fleet — every node with each of its GPUs
  unlink       take this machine out of the fleet (deregister + stop hanzod)
  whoami       show the current identity from the stored token

Service subcommands (server mode):
  account      serve the account subsystem standalone
  account-bridge serve the account-bridge subsystem standalone
  admin        serve the admin subsystem standalone
  admission    serve the admission subsystem standalone
  ads          serve the ads subsystem standalone
  affiliates   serve the affiliates subsystem standalone
  agent        serve the agent subsystem standalone
  agents       serve the agents subsystem standalone
  agentskills  serve the agentskills subsystem standalone
  ai           serve the ai subsystem standalone
  analytics    serve the analytics subsystem standalone
  audit        serve the audit subsystem standalone
  authors      serve the authors subsystem standalone
  authz        serve the authz subsystem standalone
  automations  serve the automations subsystem standalone
  base         serve the base subsystem standalone
  billing      serve the billing subsystem standalone
  bots         serve the bots subsystem standalone
  captable     serve the captable subsystem standalone
  catalogsync  serve the catalogsync subsystem standalone
  channels     serve the channels subsystem standalone
  cloud        serve the full unified surface (all enabled subsystems, one listener)
  cloudflare   serve the cloudflare subsystem standalone
  code         serve the code subsystem standalone
  commerce     serve the commerce subsystem standalone
  company      serve the company subsystem standalone
  content      serve the content subsystem standalone
  crm          serve the crm subsystem standalone
  dataroom     serve the dataroom subsystem standalone
  datastore    datastore-fork analytics DB — not a Go serve target (see help text)
  deploy       serve the deploy subsystem standalone
  dns          serve the dns subsystem standalone
  do           serve the do subsystem standalone
  domain       serve the domain subsystem standalone
  entitlements serve the entitlements subsystem standalone
  evals        serve the evals subsystem standalone
  exec         serve the exec subsystem standalone
  flags        serve the flags subsystem standalone
  framework    serve the framework subsystem standalone
  functions    serve the functions subsystem standalone
  gateway      serve the gateway subsystem standalone
  git          serve the git subsystem standalone
  graph        serve the graph subsystem standalone
  guide        serve the guide subsystem standalone
  iam          serve standalone Hanzo IAM (full Beego server: login UI, OAuth2/OIDC, LDAP/RADIUS)
  ingress      serve the ingress subsystem standalone
  integrations serve the integrations subsystem standalone
  kafka        serve the kafka subsystem standalone
  kms          serve the kms subsystem standalone
  knowledge    serve the knowledge subsystem standalone
  licensing    serve the licensing subsystem standalone
  link         serve the link subsystem standalone
  marketing    serve the marketing subsystem standalone
  marketplace  serve the marketplace subsystem standalone
  metrics      serve the metrics subsystem standalone
  ml           serve the ml subsystem standalone
  notify       serve the notify subsystem standalone
  o11y         serve the o11y subsystem standalone
  paas         serve the paas subsystem standalone
  plan         serve the plan subsystem standalone
  platform     serve the platform subsystem standalone
  plugins      serve the plugins subsystem standalone
  pricing      serve the pricing subsystem standalone
  product      serve the product subsystem standalone
  projects     serve the projects subsystem standalone
  prompts      serve the prompts subsystem standalone
  provisioning serve the provisioning subsystem standalone
  pubsub       serve the pubsub subsystem standalone
  referrals    serve the referrals subsystem standalone
  rollingcap   serve the rollingcap subsystem standalone
  runtime      serve the runtime subsystem standalone
  sbom         serve the sbom subsystem standalone
  security     serve the security subsystem standalone
  settings     serve the settings subsystem standalone
  sign         serve the sign subsystem standalone
  social       serve the social subsystem standalone
  storage      serve the storage subsystem standalone
  sync         serve the sync subsystem standalone
  tasks        serve the tasks subsystem standalone
  team         serve the team subsystem standalone
  templates    serve the templates subsystem standalone
  tools        serve the tools subsystem standalone
  tracker      serve the tracker subsystem standalone
  treasury     serve the treasury subsystem standalone
  usage        serve the usage subsystem standalone
  validators   serve the validators subsystem standalone
  visor        serve the visor subsystem standalone
  wallets      serve the wallets subsystem standalone
  websearch    serve the websearch subsystem standalone
  world        serve the world subsystem standalone
  x402         serve the x402 subsystem standalone
  zen          serve the zen subsystem standalone
  zero-trust   serve the zero-trust subsystem standalone

Meta:
  help         show this message
  version      print version and exit

Flags are per-subcommand (e.g. `hanzo cloud --enable=iam,kms --brand=hanzo`,
`hanzo kms --listen=:8443`). Run a subcommand to see its config via env/flags. (HIP-0106): link composes fabric join (hanzo dev — the unified Hanzo Go binary

Usage:
  hanzo <command> [flags]

Control commands (gcloud/doctl-style):
  agent        invoke a managed Hanzo agent to run a task (headless)
  apps         list/get the platform apps board (declared/running/drift)
  auth         manage authentication + stored identities (login, logout, whoami, list, switch, token)
  bot          launch a computer-using agent (booted desktop or terminal)
  build        enqueue a platform-native build (runner fabric)
  clusters     provision/list/select dedicated DOKS clusters
  code         launch a coding agent (claude, codex, dev) on a Hanzo cloud model
  config       view/edit ~/.hanzo/config preferences
  deploy       drive a platform redeploy (rolling restart, zero-downtime)
  engine       run a local hanzo-engine (OpenAI + Anthropic model server)
  k8s          deploy-target helpers (current target)
  link         bring this machine into the Hanzo cloud fleet as a node (fabric + compute worker)
  login        authenticate against Hanzo IAM (hanzo.id) and store a token
  logout       remove stored credentials
  run          launch a workload on Hanzo compute (container or function)
  runner       run this machine as a JIT CI runner for your org (GitHub Actions)
  security     scan files for hardcoded secrets (local guardrail; no server/auth)
  status       show the org's fleet — every node with each of its GPUs
  unlink       take this machine out of the fleet (deregister + stop hanzod)
  whoami       show the current identity from the stored token

Service subcommands (server mode):
  account      serve the account subsystem standalone
  account-bridge serve the account-bridge subsystem standalone
  admin        serve the admin subsystem standalone
  admission    serve the admission subsystem standalone
  ads          serve the ads subsystem standalone
  affiliates   serve the affiliates subsystem standalone
  agent        serve the agent subsystem standalone
  agents       serve the agents subsystem standalone
  agentskills  serve the agentskills subsystem standalone
  ai           serve the ai subsystem standalone
  analytics    serve the analytics subsystem standalone
  audit        serve the audit subsystem standalone
  authors      serve the authors subsystem standalone
  authz        serve the authz subsystem standalone
  automations  serve the automations subsystem standalone
  base         serve the base subsystem standalone
  billing      serve the billing subsystem standalone
  bots         serve the bots subsystem standalone
  captable     serve the captable subsystem standalone
  catalogsync  serve the catalogsync subsystem standalone
  channels     serve the channels subsystem standalone
  cloud        serve the full unified surface (all enabled subsystems, one listener)
  cloudflare   serve the cloudflare subsystem standalone
  code         serve the code subsystem standalone
  commerce     serve the commerce subsystem standalone
  company      serve the company subsystem standalone
  content      serve the content subsystem standalone
  crm          serve the crm subsystem standalone
  dataroom     serve the dataroom subsystem standalone
  datastore    datastore-fork analytics DB — not a Go serve target (see help text)
  deploy       serve the deploy subsystem standalone
  dns          serve the dns subsystem standalone
  do           serve the do subsystem standalone
  domain       serve the domain subsystem standalone
  entitlements serve the entitlements subsystem standalone
  evals        serve the evals subsystem standalone
  exec         serve the exec subsystem standalone
  flags        serve the flags subsystem standalone
  framework    serve the framework subsystem standalone
  functions    serve the functions subsystem standalone
  gateway      serve the gateway subsystem standalone
  git          serve the git subsystem standalone
  graph        serve the graph subsystem standalone
  guide        serve the guide subsystem standalone
  iam          serve standalone Hanzo IAM (full Beego server: login UI, OAuth2/OIDC, LDAP/RADIUS)
  ingress      serve the ingress subsystem standalone
  integrations serve the integrations subsystem standalone
  kafka        serve the kafka subsystem standalone
  kms          serve the kms subsystem standalone
  knowledge    serve the knowledge subsystem standalone
  licensing    serve the licensing subsystem standalone
  link         serve the link subsystem standalone
  marketing    serve the marketing subsystem standalone
  marketplace  serve the marketplace subsystem standalone
  metrics      serve the metrics subsystem standalone
  ml           serve the ml subsystem standalone
  notify       serve the notify subsystem standalone
  o11y         serve the o11y subsystem standalone
  paas         serve the paas subsystem standalone
  plan         serve the plan subsystem standalone
  platform     serve the platform subsystem standalone
  plugins      serve the plugins subsystem standalone
  pricing      serve the pricing subsystem standalone
  product      serve the product subsystem standalone
  projects     serve the projects subsystem standalone
  prompts      serve the prompts subsystem standalone
  provisioning serve the provisioning subsystem standalone
  pubsub       serve the pubsub subsystem standalone
  referrals    serve the referrals subsystem standalone
  rollingcap   serve the rollingcap subsystem standalone
  runtime      serve the runtime subsystem standalone
  sbom         serve the sbom subsystem standalone
  security     serve the security subsystem standalone
  settings     serve the settings subsystem standalone
  sign         serve the sign subsystem standalone
  social       serve the social subsystem standalone
  storage      serve the storage subsystem standalone
  sync         serve the sync subsystem standalone
  tasks        serve the tasks subsystem standalone
  team         serve the team subsystem standalone
  templates    serve the templates subsystem standalone
  tools        serve the tools subsystem standalone
  tracker      serve the tracker subsystem standalone
  treasury     serve the treasury subsystem standalone
  usage        serve the usage subsystem standalone
  validators   serve the validators subsystem standalone
  visor        serve the visor subsystem standalone
  wallets      serve the wallets subsystem standalone
  websearch    serve the websearch subsystem standalone
  world        serve the world subsystem standalone
  x402         serve the x402 subsystem standalone
  zen          serve the zen subsystem standalone
  zero-trust   serve the zero-trust subsystem standalone

Meta:
  help         show this message
  version      print version and exit

Flags are per-subcommand (e.g. `hanzo cloud --enable=iam,kms --brand=hanzo`,
`hanzo kms --listen=:8443`). Run a subcommand to see its config via env/flags., delegated to the Rust CLI installed as hanzo code claude · /home/z/work/hanzo/cloud · start
  model routing: on → api.hanzo.ai (prompts + code go here; usage metered to your org)
  session stream: on → https://hanzo.bot/sessions/sess_4729b3a0a623c55d0d3b7b08fe8ebe35
resume: hanzo --resume 4729b3a0a623c55d0d3b7b08fe8ebe35) with compute-worker
registration (CPU cores+model, memory, each GPU), heartbeats, claims gpu-jobs.
unlink is idempotent; status renders the fleet with each GPU distinct. hanzo dev — the unified Hanzo Go binary

Usage:
  hanzo <command> [flags]

Control commands (gcloud/doctl-style):
  agent        invoke a managed Hanzo agent to run a task (headless)
  apps         list/get the platform apps board (declared/running/drift)
  auth         manage authentication + stored identities (login, logout, whoami, list, switch, token)
  bot          launch a computer-using agent (booted desktop or terminal)
  build        enqueue a platform-native build (runner fabric)
  clusters     provision/list/select dedicated DOKS clusters
  code         launch a coding agent (claude, codex, dev) on a Hanzo cloud model
  config       view/edit ~/.hanzo/config preferences
  deploy       drive a platform redeploy (rolling restart, zero-downtime)
  engine       run a local hanzo-engine (OpenAI + Anthropic model server)
  k8s          deploy-target helpers (current target)
  link         bring this machine into the Hanzo cloud fleet as a node (fabric + compute worker)
  login        authenticate against Hanzo IAM (hanzo.id) and store a token
  logout       remove stored credentials
  run          launch a workload on Hanzo compute (container or function)
  runner       run this machine as a JIT CI runner for your org (GitHub Actions)
  security     scan files for hardcoded secrets (local guardrail; no server/auth)
  status       show the org's fleet — every node with each of its GPUs
  unlink       take this machine out of the fleet (deregister + stop hanzod)
  whoami       show the current identity from the stored token

Service subcommands (server mode):
  account      serve the account subsystem standalone
  account-bridge serve the account-bridge subsystem standalone
  admin        serve the admin subsystem standalone
  admission    serve the admission subsystem standalone
  ads          serve the ads subsystem standalone
  affiliates   serve the affiliates subsystem standalone
  agent        serve the agent subsystem standalone
  agents       serve the agents subsystem standalone
  agentskills  serve the agentskills subsystem standalone
  ai           serve the ai subsystem standalone
  analytics    serve the analytics subsystem standalone
  audit        serve the audit subsystem standalone
  authors      serve the authors subsystem standalone
  authz        serve the authz subsystem standalone
  automations  serve the automations subsystem standalone
  base         serve the base subsystem standalone
  billing      serve the billing subsystem standalone
  bots         serve the bots subsystem standalone
  captable     serve the captable subsystem standalone
  catalogsync  serve the catalogsync subsystem standalone
  channels     serve the channels subsystem standalone
  cloud        serve the full unified surface (all enabled subsystems, one listener)
  cloudflare   serve the cloudflare subsystem standalone
  code         serve the code subsystem standalone
  commerce     serve the commerce subsystem standalone
  company      serve the company subsystem standalone
  content      serve the content subsystem standalone
  crm          serve the crm subsystem standalone
  dataroom     serve the dataroom subsystem standalone
  datastore    datastore-fork analytics DB — not a Go serve target (see help text)
  deploy       serve the deploy subsystem standalone
  dns          serve the dns subsystem standalone
  do           serve the do subsystem standalone
  domain       serve the domain subsystem standalone
  entitlements serve the entitlements subsystem standalone
  evals        serve the evals subsystem standalone
  exec         serve the exec subsystem standalone
  flags        serve the flags subsystem standalone
  framework    serve the framework subsystem standalone
  functions    serve the functions subsystem standalone
  gateway      serve the gateway subsystem standalone
  git          serve the git subsystem standalone
  graph        serve the graph subsystem standalone
  guide        serve the guide subsystem standalone
  iam          serve standalone Hanzo IAM (full Beego server: login UI, OAuth2/OIDC, LDAP/RADIUS)
  ingress      serve the ingress subsystem standalone
  integrations serve the integrations subsystem standalone
  kafka        serve the kafka subsystem standalone
  kms          serve the kms subsystem standalone
  knowledge    serve the knowledge subsystem standalone
  licensing    serve the licensing subsystem standalone
  link         serve the link subsystem standalone
  marketing    serve the marketing subsystem standalone
  marketplace  serve the marketplace subsystem standalone
  metrics      serve the metrics subsystem standalone
  ml           serve the ml subsystem standalone
  notify       serve the notify subsystem standalone
  o11y         serve the o11y subsystem standalone
  paas         serve the paas subsystem standalone
  plan         serve the plan subsystem standalone
  platform     serve the platform subsystem standalone
  plugins      serve the plugins subsystem standalone
  pricing      serve the pricing subsystem standalone
  product      serve the product subsystem standalone
  projects     serve the projects subsystem standalone
  prompts      serve the prompts subsystem standalone
  provisioning serve the provisioning subsystem standalone
  pubsub       serve the pubsub subsystem standalone
  referrals    serve the referrals subsystem standalone
  rollingcap   serve the rollingcap subsystem standalone
  runtime      serve the runtime subsystem standalone
  sbom         serve the sbom subsystem standalone
  security     serve the security subsystem standalone
  settings     serve the settings subsystem standalone
  sign         serve the sign subsystem standalone
  social       serve the social subsystem standalone
  storage      serve the storage subsystem standalone
  sync         serve the sync subsystem standalone
  tasks        serve the tasks subsystem standalone
  team         serve the team subsystem standalone
  templates    serve the templates subsystem standalone
  tools        serve the tools subsystem standalone
  tracker      serve the tracker subsystem standalone
  treasury     serve the treasury subsystem standalone
  usage        serve the usage subsystem standalone
  validators   serve the validators subsystem standalone
  visor        serve the visor subsystem standalone
  wallets      serve the wallets subsystem standalone
  websearch    serve the websearch subsystem standalone
  world        serve the world subsystem standalone
  x402         serve the x402 subsystem standalone
  zen          serve the zen subsystem standalone
  zero-trust   serve the zero-trust subsystem standalone

Meta:
  help         show this message
  version      print version and exit

Flags are per-subcommand (e.g. `hanzo cloud --enable=iam,kms --brand=hanzo`,
`hanzo kms --listen=:8443`). Run a subcommand to see its config via env/flags. is a
superset — non-Go verbs pass through to hanzo-node so nothing breaks.
2026-07-21 17:56:10 -07:00
hanzo-dev 0a24aa76e3 link: hanzo is a superset — delegate non-Go verbs to the Rust CLI (hanzo-node)
The Go unified binary takes the `hanzo` name (HIP-0106). fabricCLI now resolves the
Rust fabric/dev CLI as `hanzo-node` (then a self-guarded `hanzo`), and cmd/hanzo
delegates any verb that is neither a Go control verb nor a served subsystem —
node, dev, wallet, network, … — to it via cli.Passthrough. So one `hanzo` name
serves both: link/unlink/status + the whole Go surface native, and `hanzo node up`
(the fabric that link composes) plus the Rust dev verbs handed through unchanged.
2026-07-21 17:22:39 -07:00
hanzo-dev 6dfd242fba Merge remote-tracking branch 'origin/main' into feat/hanzo-link 2026-07-21 17:22:26 -07:00
hanzo-dev c696d1a4e5 Merge remote-tracking branch 'origin/main' into feat/hanzo-link
# Conflicts:
#	cli/gpu.go
2026-07-21 16:58:53 -07:00
hanzo-dev 42c98780a8 link: RED-review fixes — unlink idempotency + comment sweep
unlink is now idempotent: runDisconnect treats an already-terminal (409) or absent
(404) fleet row as the desired end state (no-op with a clear notice), and unlink
runs stopFabric unconditionally so a deregister error never leaves hanzod running —
the deregister error is reported, not short-circuited. Tests cover the 409/404
idempotent path and the 500 error-surfacing path.

Sweeps the remaining `gpu connect` prose in comments to `link` (engine, runner,
visor fleet/board/visor, and the gpu/fleet spec + engine test headers).
2026-07-21 16:56:58 -07:00
hanzo-dev c591636a8b chore(deps): bump hanzoai/ai v1.829.3 → v1.829.5 (RESTful ZAP-native router routes; beego split-brain killed) 2026-07-21 16:54:53 -07:00
antje e380d0737c fix(team): heal the authenticating caller's own member row (real Seats:0 fix)
The live wallet Seats:0 for maxpower was NOT the orgs claim (Dave's claim already
lists maxpower/admin). Dave OWNS a maxpower workspace, but a team-go migration left
his own member row is_bot=1/active=0, so Seats (which filters active=1 AND is_bot=0)
excluded him while getUserWorkspaces still listed the workspace (no flag filter).
EnsureWorkspace early-returned on the existing workspace without ever correcting
the row, so every re-login kept 0.

A user who just authenticated through IAM is by definition an active, non-bot member
of their own workspace: EnsureWorkspace now forces the caller's OWN row (never
anyone else's) to active=1/is_bot=0 on the existing-workspace path. Idempotent.
TestEnsureWorkspaceHealsMigratedMember reproduces Dave's exact shape red→green.
2026-07-21 16:26:18 -07:00
antje 47e39d78b7 fix(team): home org always ensured a seat; createContent seeds the ydoc log
Two live hanzo.team defects:

1) Wallet Seats:0 for the caller's own org. orgsClaim dropped the HOME org (the
   org the wallet, Seats, and every account-store surface scope to via extra.org)
   whenever the IAM orgs claim was non-empty but did not itself list home — the
   fallback only fired for an EMPTY claim. So establishSession never ensured a
   home-org workspace, and Seats(home) returned 0 for an org that has the caller
   as a member. Home is now unconditionally in the set, so its workspace (hence a
   seat) is ensured at every login. Reproduced in TestOrgsClaimAlwaysIncludesHome.

2) New-Issue dialog description dropped on create. createContent stored only the
   markup SNAPSHOT blob; the collaborative editor replays the Y.js update log the
   WS lane serves (ydoc-<id>-<field>), a different blob, so the description showed
   empty. createContent now also seeds that log from the front-supplied Y.js
   update (never clobbering an existing/live log; scoped to createContent).
   TestCollabCreateContentSeedsYLog covers it.
2026-07-21 16:05:22 -07:00
hanzo-dev d43479712b link: hanzo link|unlink|status — bring a machine into the fleet as a node
link composes the two node memberships under one verb: it starts hanzod via the
canonical `hanzo node up` (best-effort; --no-fabric skips it) and runs the
compute-worker loop that registers this host's CPU (cores + model), memory, and
each GPU as its own resource, then heartbeats and claims jobs from the org queue.
unlink deregisters the node and stops hanzod; status renders the fleet with each
GPU shown distinctly and this box highlighted. Works on a CPU-only node.

Replaces the gpu connect|status|disconnect surface (one way, no alias). Adds a
CPU model field to the advertised inventory — CLI registration and the visor
fleet record in lockstep. The worker machinery (register/heartbeat/claim, studio,
engine advertise) is unchanged; only the command layer and inventory grow.
2026-07-21 16:02:16 -07:00
antje 92910c173d fix(admin): commerce cost god-view uses the in-process transport (fixes commerce.inproc DNS fail)
The admin cockpit's commerce reader (/v1/admin/finance COGS, /v1/admin/usage,
costs) built a PLAIN http.Client but its base is commerceinproc.BaseURL() — which
returns the 'http://commerce.inproc' placeholder when commerce is co-resident. A
plain client DNS-resolves that host → 'lookup commerce.inproc: no such host', so
the admin finance/cost god-view silently errored ('commerce unreachable'). Swap to
commerceinproc.Client() — the self-routing transport metering already uses:
in-process dispatch for the placeholder host, plain HTTP for a split-deploy URL.
Only this admin client hit it (the others use real env URLs).
2026-07-21 15:52:30 -07:00
antje 65b45b1aa7 feat(billing): default rolling-cap fallback — protect pay-as-you-go too
The per-tier rolling cap only governed SEEDED subscription tiers (free/pro/…).
Pay-as-you-go / empty / unknown tiers fell through to uncapped — a burst-spend
hole for the entire non-subscription user base (every current org is
pay-as-you-go). Add ai_rolling_cap_cents_default: when a caller's tier has no
specific cap, the reader falls back to it, so EVERY caller gets a rolling ceiling
once set. Default 0 = opt-in (no behavior change until an admin sets it in the
cockpit); tier-specific caps still win. Fail-open on tier/sum error unchanged.
Test covers the fallback (pay-as-you-go over default → deny; tier-specific beats
default; empty+no-default still admits).
2026-07-21 15:23:02 -07:00
hanzo-dev fb8d30dc14 test(apps): refreeze wire golden — add validators (91st subsystem)
A parallel merge (feat(validators): NFT-gated node provisioning) added the
validators subsystem to Wire() at position 45 (after ads) without updating the
frozen golden, so TestWireOrderMatchesFrozen fails 91 vs 90 — the failing test
behind main's red CI once the go.sum compile error is fixed. Refroze to match
Wire() order + flags (ownsHealth=false, hasShutdown=true).
2026-07-21 15:17:23 -07:00
hanzo-dev 56d1036c52 fix(deps): go mod tidy — complete go.sum, unblock main CI
Main CI/CD has been red for 8+ commits: go.sum was missing transitive
entries (mongo-driver/bson via golang-set, btcd/chainhash/v2 via btcec,
hanzos3/go-sdk via zapdb, go-json-experiment/json + luxfi/filesystem via
luxfi/node) so go vet/test/build all fail before any test runs. A parallel
dep bump landed without tidy. Pure require-list + go.sum reconcile; versions
unchanged (luxfi/node stays v1.36.15).
2026-07-21 14:52:15 -07:00
zeekayandhanzo-dev 79ebb519e9 fix(validators): register /v1/validators collection root flat (avoid trailing-slash 404)
Group("/v1/validators").Post("") registers "/v1/validators/", which the
portal's bare POST /v1/validators would miss. Register the list+provision
collection root via app.Get/app.Post like clients/wallets et al.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-21 14:24:55 -07:00
43a8099d36 feat(fleet): per-GPU render queue — targeted lanes, visibility, cancel, usage (#346)
* test(apps): refreeze wire golden — add rollingcap subsystem

Wire() mounts 90 subsystems but the frozen sequence listed 89: the rollingcap
gate (mounted after billing) was added to Wire() without refreezing the golden,
so TestWireOrderMatchesFrozen has been red on main. Add the missing entry at its
Wire() position — the same maintenance the earlier dns+cloudflare refreeze did.
Pre-existing drift, unrelated to the per-GPU queue work; folded in so cloud CI's
go-unit gate (which runs ./apps/) is green again.

* feat(fleet): per-GPU render queue — targeted lanes, visibility, cancel, usage

Make all BYO-GPU render work flow through the org's gpu-jobs queue, visible and
manageable per GPU, and close the hidden direct-submit hole.

cli/gpu.go
- Two-lane claim: the worker claims its OWN lane ("gpu:<identity>") FIRST, then
  the shared "gpu-jobs" lane — targeting is the taskQueue VALUE within the one
  gpu-jobs namespace, so a job pinned to spark is never starved and no worker
  steals another GPU's targeted job.
- Render submit moves from the open POST /prompt to the gated
  POST /v1/worker/execute with X-Worker-Token (KMS STUDIO_WORKER_TOKEN).
- Worker reports live GPU utilization (nvidia-smi) each heartbeat via
  POST /v1/fleet/samples.

cli/studio.go
- Launch the local ComfyUI with --listen 127.0.0.1 --worker-mode (was 0.0.0.0,
  unauthenticated) — the worker dials loopback, so binding wider only exposed an
  open /prompt; worker-mode gates the submit seam.

clients/visor
- GET /v1/fleet/jobs?gpu=&status= — the org's queue, each row tagged with the
  GPU it targets (''=shared lane) + the claiming worker; ?gpu=X matches target OR
  claimant; status normalized to queued|running|completed|failed|canceled; the
  full ComfyUI graph is omitted (cheap SaveImage label instead).
- POST /v1/fleet/jobs/:id/cancel {run,reason} — org-scoped cancel via the tasks
  CancelActivityForOrg wrapper (tasks v1.51.2).
- POST /v1/fleet/samples — BYO util ingest into the existing samples warehouse
  the board already overlays; fleetUnit gains Queued/Running per-GPU depth.

Reuses ActivitiesForOrg (one engine, one tenant key), fail-soft by source.
TDD: claim precedence, gpuTarget, status normalize, filter, per-node counts,
sample build, route tenancy — ./cli/ + ./clients/visor/ added to the CI go-unit
gate so they run every push.

* fix(gpu): detach + bound the util sampler so a hung nvidia-smi can't wedge the worker

sampleGPUs shelled nvidia-smi with no timeout and reportSample ran SYNCHRONOUSLY in
the worker's select loop (heartbeat tick + render-progress tick). Under GPU/driver
pressure nvidia-smi can hang, blocking the whole loop → no heartbeats, no claims →
the machine flaps offline and stops rendering mid-run.

- sampleGPUs takes a ctx and runs the probe via exec.CommandContext under a 5s cap
  (self-cancels instead of hanging); the probe is an injectable package var.
- reportSample detaches probe+POST onto its own goroutine under a 20s budget and
  returns immediately, so the select loop is never blocked. At most one report in
  flight per ticker site (interval >> budget); self-cancels on worker shutdown.

Test: a hung sampler (blocks until its bounded ctx fires) — reportSample still
returns to the caller at once, proving claim/heartbeat can't wedge (-race clean).

* fix(fleet): adversarial batch — pagination, render preflight, filter/stall/token hardening

F1 (MAJOR): the queue + fleet reads no longer truncate at 100 rows. gpuJobs and
byoWorkers cursor-walk the org's namespace to completion via the new paginated
tasks read (ActivitiesPageForOrg, tasks v1.51.3); gpuJobs then recency-sorts and
bounds terminal history (all live jobs kept + last 50 terminal). Past ~100 lifetime
renders a busy org no longer hides live jobs or drops online workers.

F2 (MAJOR): worker render preflight. A node advertises studioCap + claims render
lanes ONLY when it can serve — STUDIO_WORKER_TOKEN present AND a studio reachable
(or launched via --studio-dir). Otherwise it heartbeats as present but claims
nothing (no poison loop of 403→FAILED→reclaim), with a loud one-time operator
warning. Re-evaluated each heartbeat so a studio dying/recovering flips claiming.

F3: ?gpu= filter is case-insensitive (node ids are lower-case).
F4: a running job past its lease surfaces as 'stalled' (worker died, not yet reaped)
    instead of 'running' forever.
F5: every loopback studio call (execute/history/view/upload/queue) sends
    X-Worker-Token, robust to the worker-mode gate widening scope.
F7: corrected the '/prompt' → '/v1/worker/execute' error string.

Tests: >100-row ordering/bound + stall + case-insensitive filter + cancel err→HTTP
(404/409) mapping; terminal complete/fail hit the exact ns+wf+run path (catches a
routing regression a 200-everything stub would miss); SharePolicy.reject fallback;
not-ready node claims nothing; studioCap gating. tasks v1.51.3 adds
ActivitiesPageForOrg with a >100 pagination test.

---------

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-21 14:13:55 -07:00
antje 22f261f02b security: reserve stg slug — close shared-client redirect takeover vector
The shared hanzo-app OAuth client trusts https://stg.hanzo.app/callback, but `stg` was not a reserved subdomain — an attacker could first-come-claim stg.hanzo.app, run authorize(client_id=hanzo-app, redirect=stg.hanzo.app/callback) (IAM exact-matches), and harvest a logged-in user code minted with aud=hanzo-app that api.hanzo.ai trusts → account takeover. stg.hanzo.app is 404/unbound today, so reserving closes it safely.

Adds `stg` to sites.baseReserved + reserved-superset test (reserved ⊇ {www,stg}) + storage-layer BindHost(stg)-rejected test. Cherry-picked ONLY the isolated clients/sites files from 5ac8f89; the larger per-app IAM client changes (appauth.go/projects.go/iam) stay OUT of main pending red re-review.

Verified: go build ./... ok; CGO=0 go test ./clients/sites/... ok; BindHost(stg)→errReservedHost, stg does not resolve.
2026-07-21 13:48:33 -07:00
zeekayandhanzo-dev a9258b54e9 feat(validators): POST /v1/validators — NFT-gated node provisioning + owner-gated registration
Phase-1 of GDA/SDM validator onboarding on lux.cloud. New /v1/validators/*
subsystem: a caller proves wallet control (EIP-191 personal_sign challenge,
address recovered server-side) AND on-chain ownership of a Validator-tier
GenesisNFT on Ethereum mainnet (ownerOf against 0x31e0F919C67ceDd2Bc3E294340Dc900735810311,
reusing the luxfi/geth read path), then the endpoint:
  - generates a luxd staking identity (TLS+BLS+ML-DSA-65 -> strict-PQ NodeID)
    via luxfi/node/staking — byte-identical to genesis/cmd/venuekeygen — and
    SEALS it into KMS (never plaintext; fail-closed);
  - persists the org -> tokenId -> slot entitlement (per-org SQLite);
  - writes a NEW-node LuxNetwork CR (group node.lux.cloud, ns lux-validators)
    + KMSSecret sync — three hard guards make it structurally incapable of
    touching the live hand-managed luxd StatefulSets;
  - ENQUEUES an owner-gated registration (pending_owner_approval, NEVER
    auto-submitted to any P-Chain).

Adds github.com/luxfi/node v1.36.15 (requires cloud's exact pinned
crypto/ids/geth — zero version skew). 13 tests pass incl. a live ETH-mainnet
ownerOf read; go vet clean.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-21 13:38:14 -07:00
hanzo-dev 8278466fc0 cap+router RED fixes: spend-cap writes require org-admin + bump ai v1.829.3 / commerce v1.49.8
- F2-1: gate the co-resident spend-alert CRUD WRITES (POST/PATCH/DELETE) to an ORG ADMIN
  / SuperAdmin / trusted S2S token (requireSpendCapAdmin) — commerce's user group admitted
  any authenticated member, so a compromised member key could DELETE the org's cap
  (unbounded spend) or POST a 1c enforce cap (org-wide 402 DoS). Reads (list/authorize)
  stay member/S2S-open. Exports accountclient.IsServiceToken for the gate.
- bump github.com/hanzoai/ai v1.829.0 → v1.829.3 (router allowlist HARD floor + no
  unowned-OrgSettings clobber across all writers: beego/ZAP/trainer/generic-setter).
- bump github.com/hanzoai/commerce v1.49.7 → v1.49.8 (spend-cap fails OPEN on unknown
  spend, not closed — no 402-storm on a finance-read blip).
2026-07-21 12:53:44 -07:00
b7f0e069b2 sync: repoint the git provider from Gitea to the native /v1/git plane (#347)
The universal sync engine's git provider drove an EXTERNAL Gitea store
(giteaFromEnv → gitea.mirrorIn / ensurePushMirror). Repoint it to the native git
object-plane seams already registered by clients/git at Mount, so the ONE git
store IS the in-binary /v1/git plane and no byte transits an external git host:

  - inbound (source push)  → cloud.InboundGitSync  (fast-forward-only advance;
                             a diverged native ref is a Conflict, native preserved)
  - reconcile pull/both    → cloud.ImportGitRepo   (ff mirror every branch in;
                             MirrorURL=source registers the native→source push-back)
  - reconcile push-only    → cloud.EnsureGitMirror (declare the outbound target;
                             the native mirror_out lifecycle does the pushing)

sync_api.reconcileOutboundMirror likewise moves onto cloud.EnsureGitMirror — the
ONE outbound-target registrar — so a sync's mirror target is never split across
two stores. gitea.go + gitea_test.go (the entire external-Gitea client) are now
dead and removed: forwards-only, no dead code, DRY.

This also FIXES the provider being inert in prod: giteaFromEnv fails closed
without GITEA_TOKEN/URL (unset on cloud), so no git sync could reconcile; the
native seams are in-process and need no external config. resolve() (the pure
decision core) is unchanged — TestGitResolve green; build + vet clean. The
KMS-gated store tests (TestSyncValidation) fail identically on origin/main
(CLOUD_KMS_MASTER_KEY_REF test-env requirement), orthogonal to this change.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-21 11:51:59 -07:00
5f5aa1b876 git: serve native git UI at git.hanzo.ai root (host-routed, GitHub-style URLs) (#345)
The native /v1/git UI (ui.go) was reachable only under /git/* on every host, so
git.hanzo.ai/ and git.hanzo.ai/<org>/<repo> fell through to the console SPA
catch-all (webui.go app.All("/*")) — the console shadowed the git host root.

Extend the existing onGitHost host-routing (already guarding the root smart-HTTP
/:org/:repo/* clone paths) to the UI: register the SAME handlers at the root
("/", "/:org/:repo", tree/blob/commits) gated to the git host. On api/console
they fall through (c.Next()) to the console catch-all, so a bare /:org/:repo
never shadows it there; on git.hanzo.ai they serve the native browser.

URLs are now canonical per host — one and only one way: base "" on the git host
(git.hanzo.ai/<org>/<repo>, matching the clone URL) and "/git" where the console
embeds the browser. Thread that base through render/templates/href-builders; the
UI clone box shows the clean git-host form (git.hanzo.ai/<org>/<repo>.git).

Non-destructive: additive routes behind a host guard; smart-HTTP clone routing
(distinct /info/refs|/git-*-pack tail) and console/api hosts are unchanged.
TestRootUI_HostGuard covers git-host serve + api-host fall-through.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-21 11:44:11 -07:00
hanzo-dev 68c40c86e7 chore(deps): bump github.com/hanzoai/ai v1.829.0 → v1.829.2 (per-org router config surface)
Picks up the per-org enabled-models allowlist + savings-vs-quality dial: OrgSettings
RouterEnabledModels/RouterQualityBias, the /v1/get-router-policy + /v1/update-router-policy
carry them (GET also returns the servable-model catalog), and resolveAutoModel enforces
the allowlist (both heuristic + engine paths) and the dial (cost-budget narrow + SLO
tighten). Opt-in: an org that sets neither routes exactly as before.
2026-07-21 11:40:42 -07:00
hanzo-dev 4bfc5ecec6 docs(projects/CONTRACT): converge the two site planes into one — static crs/ retires into Projects
Decision + per-site migration runbook: every first-party static site (cd/flow/
gallery/yadota) becomes a Project served by the ONE host-router (clients/sites),
bundle moved to the canonical <bucket>/<org>/<slug> layout (no external-prefix
special case — resolver keeps one code path). <slug>.hanzo.ai becomes a bound host
routed through cloud, mirroring the *.hanzo.app wildcard edge. Steps 1-2 additive
(no live-routing change); the cutover (route *.hanzo.ai via cloud, delete the
staticFiles Middleware+IngressRoute) is a reviewed per-host flip, cd.hanzo.ai LAST.
End state: static-sites.yaml holds zero first-party sites — one router, one S3
layout, one store; sites sourced from hanzo-apps.
2026-07-21 10:59:00 -07:00
6c95c5ec99 refactor(iam): drop cloud's DIRECT iam-v1 dep — use the clean v2 iam OrgRef (#344)
'iam-v1 is dead; use the new clean iam for all things.' cloud imported the dead
Casdoor fork github.com/hanzoai/iam-v1 in 6 files for exactly ONE type: OrgRef
{Org,Role} (a JWT-claim membership ref). The v2 clean-room iam (github.com/
hanzoai/iam) now EXPORTS it at pkg/model.OrgRef (= schema.OrgRef, byte-identical
JSON) as of v1.32.1. Repoint all 6 (auth_identity, token_validator,
clients/team/{invite,account}, + 2 tests) to model.OrgRef and bump iam→v1.32.1.

cloud's own code now has ZERO direct iam-v1 imports. iam-v1 remains ONLY as a
TRANSITIVE dep via hanzoai/ai/object, which still couples to the full Casdoor
IAM API (Claims/GetUser/GetOrganization/MFA…) — a major separate migration
(ai's domain), not an OrgRef swap. Builds clean; model.OrgRef resolves.

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-21 10:55:54 -07:00
hanzo-dev 0863e74a4e cap: mount spend-alerts CRUD writes co-resident — customers could not set a cap
The GET list + GET /authorize are served co-resident, but POST/PATCH/DELETE
/v1/billing/spend-alerts were NOT — so a customer creating/editing/removing a usage
cap fell through the account bridge's /v1/billing/* wildcard (billingForwardable
includes POST spend-alerts), forwarded to COMMERCE_URL (= this binary), and
self-dispatched into the SAME 502 loop authorize hit. Net: self-service cap
management was impossible in the unified binary (every write 502'd).

Register CreateSpendAlert / UpdateSpendAlert / DeleteSpendAlert co-resident with the
exact chain commerce's own route table gates them (api/billing/handlers.go:322-325,
user group userRequired = TokenRequired) + the global RequestContext: an IAM JWT OR
the COMMERCE_SERVICE_TOKEN, org from the gateway-pinned X-Org-Id. Org-scoped by
namespace (a caller writes only their OWN org's caps; a foreign :id misses in their
namespace), so no PinBillingSubject — spend-alerts are org-level. Specific routes
shadow the bridge wildcard (order 100 < 122). Completes the self-service cap CRUD:
list + authorize (already co-resident) + create/edit/delete (this).
2026-07-21 10:00:45 -07:00
79f9ce256c feat(agents): seed the built-in crew @dev @des @vi on org first-touch (re-land) (#342)
Re-land of the crew seed (a parallel force-push to cloud main dropped it).
personalities.go: dev/des/vi personas + idempotent SeedPersonalities(ctx,org)
(one registry, UNIQUE(org,name); no-op without a model). account.go OAuth
callback seeds per-org after EnsureWorkspace (best-effort, never blocks login).
Native TestSeedPersonalities green. TEAM_AGENTS_ENABLED=1 already live +
AIDefaultModel=deepseek-v4-flash → the crew materializes and answers @-mentions.

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-21 09:51:39 -07:00
zeekayandhanzo-dev e4ed9b1760 feat(agents): seed the built-in crew — @dev @des @vi — on org first-touch
The named personas from the old hanzo.ai site, brought into Hanzo Team as
ordinary rows in the ONE agents registry: nothing special-cased downstream —
they list, project into the Team roster as bot members (bots.go), and answer
@-mentions through the SAME agents.RunOnBehalf path every agent uses.

- clients/agents/personalities.go: the canonical crew (dev=builder, des=designer,
  vi=visionary) + SeedPersonalities(ctx, org) — idempotent via the registry's
  UNIQUE(org,name); no-ops without a default model (never a half-seeded org) or
  an unmounted subsystem (safe to call best-effort on the login path).
- clients/team/account.go: the OAuth callback seeds the crew per-org right after
  EnsureWorkspace — a new org gets its default office AND its default crew
  together. Best-effort: a seed hiccup NEVER blocks login.

Native test green: TestSeedPersonalities — creates the crew, ListForOrg returns
the @dev/@des/@vi handles with model+prompt, re-seed is a 0-create no-op (no
dup), no-model is a clean no-op. To make them TALK, the Chunter responder flips
on with TEAM_AGENTS_ENABLED=1 (the deploy env) — the pipeline is already built.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-21 09:46:22 -07:00
hanzo-dev f559c34a91 fix(commerce): serve spend-alerts/authorize co-resident, breaking the 502 self-dispatch loop
The request-edge metering gate (clients/metering scopeAuthorize) reads the per-scope
spend-cap verdict at GET /v1/billing/spend-alerts/authorize over commerceinproc. With no
co-resident handler it fell through to the account bridge's /v1/billing/* wildcard, which
re-forwarded it to COMMERCE_URL (the public api.hanzo.ai edge = this binary) BY PATH through
the same transport, re-entering the wildcard until the depth-8 guard refused -> 502. The live
pod logged ~135x/30m of this (org "maxpower"). The cap is a policy overlay so the gate FAILS
OPEN (outer status 200, no traffic blocked), but every authorize call burned 8 full-app
dispatches and the spend cap never actually evaluated (always fail-open) — a silent policy hole.

Register commerce's own AuthorizeSpendCap co-resident in mountCommerce (order 100, ahead of the
bridge at 122) so the specific route shadows the wildcard and the gate hits the real handler at
depth 1 — the same co-resident move already made for plans/spend-alerts/invoices/etc. Service-
token chain (RequestContext + TokenRequired), mirroring commerce's OWN gate on this route, not
the IAM/PinBillingSubject console chain (a service token is not an IAM JWT, so IAMTokenRequired
would leave GetOrganization unset and AuthorizeSpendCap would 500).

Regression test in clients/commerceinproc/selfdispatch_test.go pins both arrangements: without
the specific route the wildcard self-loops to the depth-8 refusal (the prod 502 signature); with
it the handler serves once at depth 1 and the wildcard never fires.
2026-07-21 09:41:12 -07:00
hanzo-dev 6c7bb25e46 feat(deploy): project static-plane SITES into the fleet list — CD dashboard shows ALL
GET /v1/deploy/applications listed only App CRs (the ~72 pod-backed services), so
every static-plane SITE — cd.hanzo.ai itself, flow, gallery, yadota, … — was
invisible on the CD dashboard. A site has no App CR / Deployment: it is a
`staticFiles` Middleware (S3 origin `s3://cdn/<slug>`) + an IngressRoute (its host),
served straight from S3 with zero pods.

clients/deploy/sites.go: listSiteApplications enumerates the staticFiles Middlewares
per namespace and joins each to its IngressRoute host, projecting one Application row
per site with Role:"site", Repository=the S3 origin, Endpoints=[https://<host>], and
— since a static site is served from exactly its declared prefix — always Synced
(Version==RunningVersion=="static"). Health = routed (Healthy) vs defined-but-unrouted
(Missing). Best-effort (mirrors runningVersions): a missing-CRD/RBAC list error logs
and yields nothing, so the services half of the board always renders.

deploy.go: middlewaresGVR + ingressRoutesGVR (hanzo.ai/v1alpha1). applications.go:
fold site rows into the per-namespace scan + the summary. deploy_test.go:
TestListSiteApplications (a staticFiles Middleware + IngressRoute → one role:"site"
row; an unrouted Middleware → Missing, no endpoints).

Aligns with "delivery is the cloud deploy engine": the CD dashboard now renders the
WHOLE delivery surface — every service AND every site.
2026-07-21 09:40:12 -07:00
zeekayandhanzo-dev 32902f69ec fix(deps): bump hanzoai/ai → v1.829.0 (iam-v1 repoint) so cloud graph drops old iam root
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-21 09:39:14 -07:00
zeekayandhanzo-dev 6ed1e45ae6 refactor(iam): unify on hanzoai/iam@v1.32.0 (former iam2) + retire fork as iam-v1
Clean-room rewrite is now github.com/hanzoai/iam@v1.32.0 (continues the version line
so MVS selects it over the fork's v1.31.x); cloud embeds it via server.Mount. The
fork's object/iamserver/root usages repoint to github.com/hanzoai/iam-v1@v1.31.37.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-21 09:30:35 -07:00
antje 1d0c2f274f fix(ai): split completions (M2M) from embeddings (pk-) credential — stop bot replies 403ing on the read-only publishable key
deps.AI (chat completions, a WRITE endpoint: agents/guide/crm/content/
sitegen/code-ask) and deps.Embed (embeddings, READ-ONLY: code-index + KB)
were ONE shared client authenticated by CLOUD_AI_API_KEY — a read-only
publishable (pk-) key from secret cloud-ai-embed-key. The gateway 403s a
pk- key on any write endpoint ('Publishable keys can only access read-only
endpoints ... use a secret key'), so bot replies intermittently failed.

Split the credential by concern:
- pickCompletionsClient (deps.AI) REFUSES any pk- key (publishableKey guard)
  and authenticates with the binary's IAM M2M identity (IAM_CLIENT_ID/SECRET,
  already deployed) — the durable no-static-key path. A secret sk-/hk- static
  key is still honored as an operator override.
- pickEmbedClient (deps.Embed) keeps the pk- key UNCHANGED (correct
  least-privilege for a read-only call); falls back to the M2M resolution when
  no static embed key is set.

Metering unchanged: both clients wrap the ONE meteredAIClient path. No CR/KMS
change needed — the M2M identity is already in the cloud CR.

Tests (pick_test.go): completions refuse pk- and present the M2M bearer;
completions honor an sk- key; pk--only + no M2M fails closed (never sends the
pk- key to the write endpoint); embeddings present the pk- key; embed falls
back to M2M with no key; BuildDeps splits both credentials end to end.
2026-07-21 09:06:13 -07:00
antje 62d83460ce team(wallet): seats count every verified org + honest Free plan display
- establishSession EnsureWorkspace's the FULL verified membership set, not just
  the home org, so a non-home org's wallet counts the caller as a seat instead
  of "0 Members in your org". The multi-org lane unioned getUserWorkspaces across
  every org but left the seat/member projection seeded only for the home org; one
  membership set now drives the token, the workspace union, AND the seat count.
- wallet page: an empty commerce plan renders "Free" + an Upgrade CTA (the login
  gate admits a no-subscription org on the effective Free tier) instead of a bare
  "—" that read as a data failure. Never fabricates a tier.
- wallet header reflows at narrow widths: the Top up button holds its intrinsic
  size (flexShrink 0) under a wrapping row, so it stays whole at 390px instead of
  clipping to "Top".
- test: Seats counts distinct active non-bot members (owner + guest; bot and
  inactive excluded; tenant-scoped).
2026-07-21 09:01:09 -07:00
antje e265ccc78e analytics: publishable-key direct ingest (pk_) — fastest capture path, no Kafka hop
Adds a write-only publishable key and a direct-to-ClickHouse ingest ALONGSIDE
the existing /v1/event + Kafka-tier pipeline (nothing removed).

- pk_<b64url(org)>.<b64url(hmac)> — org sealed under HMAC-SHA256(CLOUD_INGEST_KEY_SECRET,
  org). Ingest-only BY CONSTRUCTION: the pk_ underscore prefix is outside
  isAPIKey's set, so SanitizeIdentity/OrgForKey refuse it — it can never become a
  bearer principal, so it can never read. Verify is one HMAC compute (no IAM/DB
  hop) — the lowest-latency path. Fails closed when the secret is unset.
- POST /v1/ingest — {batch:[WireEvent]} authed by pk_, org stamped from the SIGNED
  key (never the body), funneled through the ONE write core (ingestEvents) into
  hanzo.events, tagged source=ingest.
- POST /v1/ingest/keys — an org owner (validated principal) mints a pk_ for its
  OWN org.
- GET /v1/errors — type:'error' read lens (validated principal; reads never accept
  the write-only key). error is now first-class in canonicalType/resolveEventName
  ('error'/$error); the WireEvent error object folds into properties.$exception.

Tests: mint↔verify round trip, fail-closed matrix (forged org, wrong secret,
malformed), exception folding, canonicalType.
2026-07-20 23:45:24 -07:00
antje c37d4ba5b6 build(deps): @hanzo/plans v1.4.3 → v1.4.4 (goja bundle NAMESPACES synced)
Picks up the sites.*/base.* namespace grouping in the embedded plans bundle so
/v1/plans/vocab matches the canonical entitlements vocabulary. Subscription caps
(ai.rolling_cap_usd/_window_hours, sites/base included) already flowed via the raw
__PLANS_DATA__ injection; this closes the last display-path drift.
2026-07-20 23:25:44 -07:00
hanzo-dev 9feef2acf5 fix(billing): serve the console's billing READS co-resident, breaking the 502 self-dispatch loop
commerce's api.Route() billing bundle is never compiled into the cloud binary, so
GET /v1/billing/{invoices,subscriptions,spend-alerts,payouts,payment-config} had no
handler here and fell through to the account bridge's /v1/billing/* wildcard. The
bridge forwards to COMMERCE_URL, which defaults to the public api.hanzo.ai edge —
i.e. THIS binary — re-entering the same bridge in an unbounded self-dispatch loop
that surfaces as a 502. In prod there is no separate commerce backend to point
COMMERCE_URL at (the in-cluster commerce Service selects the cloud pods), so
co-residence is the only way to break the loop.

Register commerce's own read handlers on the shared app (order 100, shadowing the
bridge wildcard at 122), behind RequestContext + IAMTokenRequired (org namespace from
the gateway-validated X-Org-Id) + a new PinBillingSubject middleware that carries the
SAME subject-pinning the bridge applies (reusing resolveCaller/scopedBillingSearch/
account.Payer) so a co-resident read can never widen past the caller and an
unvalidated caller is refused before the handler runs. This is the same co-resident
move already made for plans/usage/balance.

Bump the embedded hanzoai/commerce dep to v1.49.7 (catalog SOT + public/admin catalog API).
2026-07-20 18:35:25 -07:00
hanzo-dev cc09ef7c76 merge: serve GET /v1/billing/plans in-process (break the 502 self-dispatch loop) 2026-07-20 17:54:00 -07:00
hanzo-dev 1e5924fd20 fix(commerce): serve GET /v1/billing/plans in-process, breaking the 502 self-dispatch loop
commerce's legacy api.Route() billing bundle (ListPlans, invoices, subscriptions)
is NOT registered by the co-resident embed — setupRoutes wires only /v1/commerce/*
— so /v1/billing/plans had no handler in the cloud binary. The account bridge's
/v1/billing/* wildcard (order 122) then forwarded the read back to commerce at
COMMERCE_URL, which defaults to the public api.hanzo.ai edge, re-entering the same
bridge in an unbounded self-dispatch loop that surfaced as
'commerce unreachable: Get https://api.hanzo.ai/v1/billing/plans' -> 502.

Register commerce's static ListPlans on the shared app in mountCommerce (order 100,
ahead of the bridge) so the specific route shadows the wildcard and plans serve
in-process — the same co-resident move billing.go already makes for usage/balance.
2026-07-20 17:31:49 -07:00
antje 4be228e223 team: collab room flusher — burst-then-idle edits persist on the debounce clock
append() only flushed when the NEXT append found the debounce due, so a
typing burst followed by idle sat dirty in memory until the last peer
left. The per-room flusher ticks the same debounce; GC closes it.
2026-07-20 17:12:55 -07:00
hanzo-devandantje d204d99620 build(deps): commerce v1.49.6 — RED residual fixes (dunning, re-subscribe, books integrity) 2026-07-20 17:12:33 -07:00
antje 07c47cb585 team: collab WS keepalive — server pings keep throttled tabs off the 1006 path
Backgrounded tabs throttle the provider's awareness renewals; without
server pings the idle read deadline fires an abrupt close the provider
surfaces as 1006 — the 'cannot connect to collaboration service' banner.
The browser's network stack auto-pongs even when throttled; each pong
extends the read deadline.
2026-07-20 17:09:33 -07:00
antje 2aa440dba0 team: live collaborative editing — hocuspocus WS lane at /collaborator
The front's @hocuspocus/provider (2.15) speaks its protocol at the bare
/collaborator path with the doc id IN-BAND, which no ingress rewrite can
bridge to the collab relay's /v1/collab/<id> mux — so the live lane joins
the snapshot RPC lane in clients/team: collabws.go serves the hocuspocus
wire (Auth in-band with the SAME HS256 session token + workspace pin +
membership gate as collab.go, SyncStep1 -> replay + empty-diff Step2 +
server Step1, SyncStatus acks, awareness echo keepalive) over zip/wsx,
persisting the Y.js update log per doc on deps.VFS under the tenant-scoped
blob key. The server never parses update payloads: Y.js updates are
commutative + idempotent, so log replay converges; a lone peer's
full-state SyncStep2 (the reply to the server's empty-SV Step1) replaces
the log — compaction without a server-side CRDT. Rooms are in-process
(cloud pins replicas=1, single writer).
2026-07-20 17:07:35 -07:00
antje ea3b885bf0 team: Slack-model multi-org — orgs claim → workspace union + explicit select + invite
Bumps iam to v1.31.34 and carries the verified `orgs` membership-set claim end
to end, so a user's team workspaces union across every org they belong to.

- cloud.VerifiedIdentity gains Orgs []iam.OrgRef, copied from the verified
  claims (idClaims parses the signed `orgs`); empty on legacy tokens.
- establishSession folds the full membership set into the session token
  (extra.orgs), fallback [{owner, admin}] for a legacy token; extra.user carries
  the IAM id for a mid-session refresh. Still fails closed on empty owner. The
  short workspace token (rides the transactor URL) stays minimal (extra.org only).
- getUserWorkspaces unions WorkspacesOf across every session org, each
  WorkspaceInfo tagged with its owning org for the client switcher.
- selectWorkspace resolves an EXPLICIT (org∈session, slug) — clean BadRequest on
  absent, WorkspaceAmbiguous on a slug in two orgs, never a silent default.
  getWorkspaceInfo resolves the token's workspace claim, killing the wss[0]
  default the same way. Single-workspace fast path unchanged (front selects by
  URL). Cross-tenant isolation preserved (every lookup owner_org-scoped).
- Invite plane (clients/team/invite.go): sendInvite resolves the invitee in IAM
  (get-user), POSTs add-membership as the confidential hanzo-team app
  (client_secret_basic, CapMembershipAdmin), and writes the local member row;
  owner/admin only. getMemberships is the mid-session refresh (live get-memberships,
  session-set fallback). AddMember upserts the roster row idempotently.

Tests: table tests for the orgs round-trip (+legacy fallback), the workspace
union, cross-org select, no-default/ambiguous refusals, getWorkspaceInfo
no-default, guest-cap unaffected, invite writes membership+row (mock IAM) +
admin gate, refresh live/fallback, and VerifiedIdentity.Orgs claim flow.
2026-07-20 16:48:25 -07:00
antje 4ce57f4686 team: collaborator RPC plane + chat notify-context projection
The Team front was losing two core lanes against the native backend:

- Issue/doc rich-text creation dead-ended: the front's collaborator-client
  POSTs createContent/updateContent/getContent to /collaborator/rpc/:documentId,
  which only had the Y.js WS relay behind it — every RPC 404'd, so
  createMarkup threw and tracker issues/documents could not be created.
  collab.go now serves that contract on deps.VFS (same tenant-scoped blob
  keys as files.go): snapshots at makeCollabJsonId ids, membership-gated,
  no-oracle 404s. Ingress path-splits /collaborator/rpc → cloud (universe).

- Channels/DMs vanished from the chat navigator on reload: the nav lists
  notification:class:DocNotifyContext per {user} (upstream server triggers
  materialize them; we never did). seed.go's trigger now projects contexts
  from chunter Channel/DirectMessage membership — create on member add,
  remove on leave, lastUpdateTimestamp bump per message (heals pre-existing
  channels on first message) — and mirrors every write as a derived tx that
  tx() broadcasts so live sessions refresh.

Tests: collab RPC round-trip + tenancy red bars; channel→context projection
(create/touch/leave). chat_test's local clChannel const moved to seed.go.
2026-07-20 16:18:33 -07:00
antje 413b41f310 build(deps): plans v1.4.3 — team-max/enterprise license the team product 2026-07-20 16:04:57 -07:00
antje 9153932095 team: native login — IAM password RPC, provider_hint federation, platform severities
The hanzo.team login page goes native: the SPA form now authenticates
straight against Hanzo IAM (there are no local accounts) and the social
buttons land directly in the provider OAuth flow.

- account RPC "login": server-side IAM password grant — the SAME two-step
  the platform e2e auth helper locks (POST /v1/iam/login responseType=code,
  then the confidential code exchange) — followed by the EXACT session
  establishment the OAuth callback runs (now ONE shared establishSession:
  userinfo → verified owner claim → workspace ensure → HS256 token). The
  password rides only in the body of the one IAM login call, is never
  logged or persisted, and bad credentials answer a clean 401 with the
  platform status the form already translates.
- /auth/google and /auth/github: same authorize hop as /auth/openid (the
  one registered callback) carrying provider_hint=provider-google/github,
  so hanzo.id auto-federates straight into the provider (console-proven,
  id >= 0.2.6); explicit ?provider_hint= passes through verbatim.
- /providers now surfaces Google + GitHub + Hanzo so the SPA renders the
  three buttons with zero client wire changes.
- Status.Severity is the platform's STRING enum ("ERROR"), not an int the
  SPA compares against nothing — error styling and retry now behave.

Tests: password login mints a verifying session (mock IAM), bad creds 401
with no password in logs or response, provider_hint mapping, providers
surface. TestPersistenceCRUD remains the known pre-existing host red.
2026-07-20 16:04:12 -07:00
antje d98bcfc1dc build(deps): commerce v1.49.5 — card-on-file self-serve subscribe 2026-07-20 16:03:47 -07:00
antje afae891bee feat(billing): wire the rolling-window AI-spend cap (admin-configurable)
Installs the ai gate's per-tier rolling AI-spend cap — the Anthropic-style burst
limit that resets continuously (usage older than the window drops out of the
trailing sum; no reset job). Composes three co-resident globals and owns no state:

  aiobject.TierReader()  — the caller's commerce plan tier
  finance.Current()      — the ledger's windowed usage sum (SumUsageSince)
  flags.Int(key)         — the admin-editable per-tier caps

The two knobs (window hours + per-tier cap cents) are platform switches, so
admin.hanzo.ai renders and edits them LIVE via the existing /v1/admin/flags
cockpit — zero bespoke admin UI. Seed defaults mirror @hanzo/plans subscription.json
(developer $0.75 / pro $2.50 / plus $12 / max $25 per 3h window).

- clients/rollingcap: the new subsystem (own package — it imports clients/flags,
  which imports root cloud, so the wiring lives above that edge). Registers the
  switches (init) + installs the reader (Mount, no routes). FAILS OPEN on any
  tier/finance error — a commerce blip must never 429 a paying caller.
- types.FinanceClient: expose SumUsageSince (the trailing-window source) on the
  interface (was only on the concrete ledger); billing/marketing test mocks updated.
- apps.go: mount rollingcap after commerce/plan (its globals are wired by then).
- go.mod: hanzoai/ai v1.827.1 → v1.828.1 (the SetRollingCapReader hook).

Inert until deployed on the unified binary; the hook is nil elsewhere. Tests cover
the decision table (over/under-cap, window-off, unknown/uncapped tier, fail-open on
tier+sum errors) + Mount no-op when globals unwired + seed coverage.
2026-07-20 15:53:17 -07:00
antje f6c18f82a0 fix(team): entitle gate observes, never blocks — the 402 bricked all logins
No org has subscription rows yet and no self-serve checkout exists, so the
definitive-no 402 on selectWorkspace locked every user out (live 2026-07-20,
front rendered it as NoLoaderForStrings). Log the denial, admit, and bring
enforcement back with the card-on-file subscribe path.
2026-07-20 15:09:40 -07:00
zeekayandhanzo-dev 8a846ead55 build: force GOWORK=off so cloud builds standalone, not via the parent go.work
Root cause of the "broken module graph" (make test / go build ./... failing with
oxy invalid-version, ugorji/koanf ambiguous imports, k8s.io/kubernetes staging
referencing removed API groups): `~/work/hanzo/go.work` auto-shadows this tree but
does NOT list ./cloud. In that workspace mode Go drops cloud's own go.mod
directives — the oxy replace, the ugorji monolith exclude, and the k8s.io/*
staging pins — so the graph that those directives keep consistent falls apart.
cloud is a standalone deploy unit (own go.mod/Dockerfile/binary) and must not join
that workspace (merging its k8s/otel tree with o11y's reintroduces the koanf split
ambiguity; the parent workspace is independently red on koanf).

Fix: the Makefile forces GOWORK=off for all go targets — exactly how CI and the
Dockerfile build (fresh checkout, no parent go.work). No go.mod change was needed;
the existing directives are correct for module mode. A committed go.work was
rejected: it would flip the Dockerfile into workspace mode after its -mod=readonly
`go mod download` step.

Proof (GOWORK=off, == what make now runs): `go build ./...` exit 0, `go vet ./...`
exit 0, `go mod tidy` stable (no go.mod change). `make build` exit 0. `go test
./...` runs (was fully blocked before): 128 ok / 103 no-test / 10 fail, every
failure runtime not module-graph — encrypted-OrgDB tests that need CGO+libsqlcipher
(the Dockerfile's -tags libsqlite3 stage), a bundle-embed test needing make
deploy-ui, and pre-existing behavior tests (metering/zt/o11y). See LLM.md.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-20 14:59:19 -07:00
antje 3ad9b14fba build(deps): commerce v1.49.4 — per-seat billing release 2026-07-20 13:55:46 -07:00
antje dbe8227442 team: usage/wallet page — @hanzo/ui@8 static embed at /v1/team/billing/ui/ + org-scoped plan read
- clients/team/wallet: small Vite/React page on @hanzo/ui@8 (balance
  three-bucket split, current-period usage, plan + seats, top-up link to
  billing.hanzo.ai), mobile-first monochrome; committed dist go:embed'd
  (the console/tasks one-binary precedent).
- clients/team/billing.go: session-gated serve of the embed + GET
  /v1/team/billing/plan (seats/guests from the org's member rows, plan +
  team.guests cap through the same commerce/plans seams entitle uses);
  orgPrincipal is the ONE token→tenant resolution (files plane rebased on it).
- money reads stay on cloud's own /v1/billing/balance + /v1/usage/summary:
  the hanzo_iam_token cookie the team callback sets is now a validated
  principal (aud hanzo-team appended to defaultJWTAudiences, forwards-only),
  so the org is pinned server-side from the verified claim — no second
  auth mechanism.
- tests: billing 401 unauth (through real Mount), embedded shell + bundle
  served authed, plan org-scoped across two tenants, audience pin.
2026-07-20 13:48:15 -07:00
zeekayandhanzo-dev 10a143b9a0 refactor(routes): group clients/team under app.Group("/v1/team")
Finishes the one convertible subsystem the group sweep skipped: team's routes
were spread across 5 register funcs whose receiver was named `g` (colliding
with the group var). Resolved by passing the group as a zip.Router param
(register(r zip.Router, ...)) instead of *zip.App — one `tg := app.Group(
"/v1/team")` in Mount, threaded to acct/bridge/files.register + the two inline
transactor routes, all rewritten to relative paths. (Note *zip.App does NOT
satisfy zip.Router — App.Fiber() returns *fiber.App vs the interface's
fiber.Router — so the two test harnesses now pass app.Group("/v1/team") too.)

Route table preserved (13 team routes byte-identical); team test suite green
(exercises the real /v1/team/* paths end-to-end); combined build +
TestWireOrderMatchesFrozen pass. deploy stays flat by design — it already DRYs
via const dashPrefix and its loginPath/callbackPath vars are reused for
redirects (scope.go), so grouping would risk redirect paths for no real gain.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-20 13:12:48 -07:00
antje 6338b892e4 refactor(cloud): drop svc from stale comment prose (finish the suffix cleanup)
Follow-up to de05be1d: the doc comments still named packages by their old
svc-suffix (iamsvc.Mount, pricingsvc, provisioningsvc, mlsvc, evalsvc, plansvc,
productsvc, syncsvc, gatewaysvc) — stale references to symbols that are now bare.
Corrected to the real names across config/middleware_identity/eval/gateway/git/
ml/pricing/projects. Comment-only; no code change.

Deliberately kept: cloud-mlsvc at clients/ml/ml.go (a real ClusterRoleBinding
name in ml-rbac.yaml, correctly referenced) and the "zapsvc" test-fixture repo
strings (test data, not an identifier).

Also gofmt'd clients/pricing/admin_http_test.go (pre-existing import-order drift).
2026-07-20 13:01:31 -07:00
antje d6a65e1a97 harden(team): bounded tokens, verified tenant, WS origin gate, OAuth state, billing gate
- token: every session token carries exp (30d; workspace 12h); Decode enforces
  exp/nbf with 60s skew; pre-rollout no-exp tokens honored until a fixed
  legacy cutoff (constant, no env). The "secret" fallback literal is GONE —
  empty secret is a hard ErrNoSecret and the TEAM_DEV_INSECURE hatch is dead.
- account: OAuth state is a random nonce bound to a short-lived cookie
  (navigateUrl rides in the cookie), verified one-shot on callback; the tenant
  comes ONLY from the RS256/JWKS-verified IAM token owner (cloud.NewTokenValidator)
  — fail closed, no default org.
- transactor: WS upgrade enforces an Origin allow-list (same host, team
  surfaces, *.hanzo.ai, absent Origin for non-browser); serves the front's
  /api/v1/statistics poll target (own-workspace sessions only).
- entitle: selectWorkspace requires the org's 'team' license — definitive no
  → 402 + upgradeUrl billing.hanzo.ai; guest role capped by the plan's
  team.guests entitlement (join order); infra errors ALWAYS admit so the gate
  can never brick login mid-rollout.
2026-07-20 12:57:56 -07:00
antje de05be1da9 refactor(cloud): drop the svc suffix — bare package names + spelled-out test helpers
One name per thing, no compound-word cruft. The `svc` suffix was never a real
package (zero `package *svc`) — only import aliases and abbreviated test helpers.

- Import aliases → bare package names: plansvc→plan (commerceclient),
  captablesvc→captable + dataroomsvc→dataroom (company/adapters). No stutter, no
  alias where the bare name is unambiguous.
- Test helpers spelled out: fakeSvc→fakeService, testSvc→testService,
  newSvc→newService — across admin/agents/deploy/domain/functions/ingress/
  integrations/ml/platform/provisioning/storage/wallets tests, callers updated
  in-package.
- Stale `// Package …svc` doc-comment prose corrected to the real package name
  (exec/iam/plugin/pricing/product/provisioning/sync/tasks).

Naming only — no logic change. go build + test-compile green on all 21 packages.

Note: the clients/team package (filesSvc/fsvc rename) is excluded here — it has
concurrent in-progress work; its svc cleanup lands with that change.
2026-07-20 12:56:12 -07:00
antje 7484b3a0c2 plans v1.4.1: hanzo.team commercial model — $20/$100/$200 ladder + $25/user team
Bump github.com/hanzoai/plans v1.4.0 -> v1.4.1 (catalog: pro repriced $20
on hanzo_pro_20, new plus $100, max $200, team $25/user per-seat minSeats 2,
team.guests entitlement, team namespace).

Pin the contract in clients/plan tests: TestPlans_Ladder freezes the
subscription ladder prices + stripe lookup keys + team per-seat/minSeats;
TestLicenseEntitlement_TeamProduct freezes the hanzo.team entitlement gate —
licensing.product:team emitted for pro, plus, max AND team, engine on max,
never on developer. Vocab namespaces 9 -> 10 (team).

Smoke-booted: /v1/plans/subscriptions serves the new ladder,
/v1/plans/entitlements/team carries licensing.product:team.
2026-07-20 10:51:13 -07:00
hanzo-dev e96505d3c7 feat(base): host-as-project-ref — serve /v1/base + /v1/realtime + /_/ on the app host
A published site host now serves its org own Base data plane (HIP-0014). The
sites middleware, on a /v1/base|/v1/realtime|/_/ path, calls an injected per-org
Base handler with the org the SUBDOMAIN resolves to (Site.Org) — never the
caller — so an anon page reaches its own Base, authz by Base collection rules.
One seam (sites.SetBaseHostHandler, mirroring SetResolver; no import cycle),
gated by CLOUD_BASE_PUBLIC_HOST (default OFF): absent the flag a site host serves
only static files, unchanged. This is what makes maxpower.hanzo.app/_/ (admin) +
the public contact form + anon realtime chat work — the token supersedes the
key/host for signed-in users (org from IAM), keys/host are the tokenless path.
2026-07-20 09:00:38 -07:00
hanzo-dev 39b6b12823 wip(sites): MEDIUM-1 — coalesce+ceiling the shared Cloudflare purge (INCOMPLETE)
Partial fix for red MEDIUM-1 (unbounded shared purge = cross-tenant blast
radius). Per-tag coalescing + process-wide per-minute ceiling in the Purger.
NOT finished: MEDIUM-2 (release retention GC), LOW-1 (reject rel=="."),
LOW-2 (empty dest ETag = fail). Do not merge until complete + red re-review.
2026-07-19 23:22:19 -07:00
hanzo-dev 79e936a7f2 wip(sites): MEDIUM-1 — coalesce+ceiling the shared Cloudflare purge (INCOMPLETE)
Partial fix for red MEDIUM-1 (unbounded shared purge = cross-tenant blast
radius). Per-tag coalescing + process-wide per-minute ceiling in the Purger.
NOT finished: MEDIUM-2 (release retention GC), LOW-1 (reject rel=="."),
LOW-2 (empty dest ETag = fail). Do not merge until complete + red re-review.
2026-07-19 23:22:19 -07:00
antje 0c06f9b78d sites: bare <slug>.hanzo.app is the ONE servable host — publish binds + advertises it
The org-scoped two-label design (<slug>.<org>.hanzo.app) was never servable: a
k8s wildcard Ingress host and a Let's Encrypt wildcard cert each match exactly
ONE label, so the two-label host neither routes nor gets TLS. Publish still
stamped it as liveUrl and bound it, so every 'Visit' link and the console/app
cards pointed at a dead host, and the sites edge served nothing.

One host, one way:
- siteURL → https://<slug>.<apex> (bare); siteHost → bare <slug> (the global
  first-come binding key — matches TestSiteHostBindingIsFirstComeAndTenantSafe,
  which already asserted bare-host first-come). A second org publishing the same
  slug is refused the subdomain and serves at its S3 URL only.
- siteSlug parses ONLY the bare host; a dotted key falls through to the API
  pipeline. unique-live-slug resolve (added earlier) keeps pre-binding publishes
  servable with no backfill.
- tests updated to the bare-host contract throughout.
2026-07-19 22:08:55 -07:00
hanzo-dev ebf96d9851 feat(sites): publish by server-side promote into immutable releases
Static sites had no way to put content at a site's prefix through the API.
Add one: a release plane on the existing site engine.

A release is an immutable prefix whose id is a digest of the object manifest
it was promoted from; the site record holds a pointer to the release it
serves, and siteResolver resolves through it. Publishing copies server-side
within the object store, so no bytes traverse the API and no client holds an
S3 credential. Rollback is the same pointer flip aimed at an older release.

Isolation: the source is a path relative to the caller's own org space. The
org segment comes from the validated principal (the one org rule this package
already uses for site prefixes) and the bucket never comes from the request,
so a caller has no syntax for naming another tenant's data. safeRel roots and
cleans both the source and every object key.

Atomicity: the release row is written only after every object lands, and
activation is one statement whose WHERE requires that row in the same tenant,
so a partially-copied release cannot be pointed at. ActivateRelease is the
sole writer of the pointer on the activate path.

Releases live in <org>/.releases/<slug>/<id>/, a sibling of the mutable
serving prefix, so a full-artifact deploy reclaims the pointer without
destroying retained releases. Caps reuse the artifact budget.

POST   /v1/sites/:slug/publish
POST   /v1/sites/:slug/releases
GET    /v1/sites/:slug/releases
POST   /v1/sites/:slug/releases/:release/activate

Mirrored under /v1/platform/sites. Inert for existing sites: an empty pointer
serves the legacy prefix.
2026-07-19 21:48:52 -07:00
hanzo-dev 6fa8e70435 feat(sites): publish by server-side promote into immutable releases
Static sites had no way to put content at a site's prefix through the API.
Add one: a release plane on the existing site engine.

A release is an immutable prefix whose id is a digest of the object manifest
it was promoted from; the site record holds a pointer to the release it
serves, and siteResolver resolves through it. Publishing copies server-side
within the object store, so no bytes traverse the API and no client holds an
S3 credential. Rollback is the same pointer flip aimed at an older release.

Isolation: the source is a path relative to the caller's own org space. The
org segment comes from the validated principal (the one org rule this package
already uses for site prefixes) and the bucket never comes from the request,
so a caller has no syntax for naming another tenant's data. safeRel roots and
cleans both the source and every object key.

Atomicity: the release row is written only after every object lands, and
activation is one statement whose WHERE requires that row in the same tenant,
so a partially-copied release cannot be pointed at. ActivateRelease is the
sole writer of the pointer on the activate path.

Releases live in <org>/.releases/<slug>/<id>/, a sibling of the mutable
serving prefix, so a full-artifact deploy reclaims the pointer without
destroying retained releases. Caps reuse the artifact budget.

POST   /v1/sites/:slug/publish
POST   /v1/sites/:slug/releases
GET    /v1/sites/:slug/releases
POST   /v1/sites/:slug/releases/:release/activate

Mirrored under /v1/platform/sites. Inert for existing sites: an empty pointer
serves the legacy prefix.
2026-07-19 21:48:52 -07:00
zeekayandhanzo-dev 834987d2f4 refactor(routes): group single-prefix subsystems under app.Group("/v1/<x>")
45 subsystems converted from flat full-path registration to the idiomatic zip
app.Group("/v1/<prefix>") + relative-path pattern — DRY the prefix, one and
only one way. Route-preserving: proved Group(p).<M>("/rel") == flat
app.<M>("p/rel") byte-for-byte; bare-prefix root routes kept FLAT (Group(p).
Get("") would add a trailing slash). Multi-prefix / dynamic-path / cross-
function-collision subsystems deliberately left flat.

Every converted subsystem gated on route-table preservation + go build + go vet;
combined ./clients/... + ./apps/... compiles clean; TestWireOrderMatchesFrozen
passes (Wire()/composition root untouched — grouping is inside each Mount).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-19 21:13:40 -07:00
antje 51107c55cc merge: shared admin core/warehouse ClickHouse helpers (data-platform-warehouse-helpers)
feat/data-platform-warehouse-helpers: clients/admin/core/warehouse.go + invoices/
metrics/subscriptions refactored onto it (aimetrics portion already landed via
#10). Conflict resolved keeping main's CreateCreditGrant, dropping the SaaS-metrics
helpers moved to warehouse.go. Tests: clients/admin (all subpkgs) ok, apps ok.
2026-07-19 20:45:00 -07:00
hanzo-devandantje c483b5868b admin(data-platform): shared core/warehouse ClickHouse helpers + invoices/metrics/subscriptions using them + test coverage + aimetrics 2026-07-19 20:43:08 -07:00
antje 4a63b00add merge: fleet gpu-connect CPU arch/cores/RAM reporting (fleet-byo-cpu-spec)
feat/fleet-byo-cpu-spec: gpu-connect nodes report CPU arch + cores + RAM on
/v1/fleet (cli/gpu.go, clients/visor board+fleet). Redundant #331 flags-test
tweak dropped in favor of main's. Tests: visor ok, cli ok, apps ok (CGO-off);
flags green under CGO-on (its one CGO-off failure is pre-existing, matches main).
2026-07-19 20:40:58 -07:00
hanzo-devandantje 41fe52d694 feat(fleet): gpu-connect nodes report CPU arch + cores + RAM on /v1/fleet
BYO nodes that dial in via `hanzo gpu connect` reported only their GPUs, so the
/v1/fleet board (and the world Fleet panel) showed NO CPU arch or system memory
for them — unlike code-linked run-targets, whose Spec already carries
arch/cpus/memory. evo-2 (Strix Halo, x86_64) and spark (GB10, aarch64), both
128 GB, appear on the board as BOTH a run-target AND a gpu-connect worker; the
BYO rows surfaced blank arch/memory.

Add the host's static CPU spec to the fleet presence record, read from the real
machine (never hardcoded), in the SAME convention the fleet already uses for
code-linked nodes so a machine shows ONE arch string across both rows:
  - reporter (cli/gpu.go): registration gains arch, cpus, memory. detectArch is
    `uname -m` (aarch64 | x86_64 | arm64) to match the existing fleet convention
    (NOT runtime.GOARCH's arm64/amd64). detectMemTotal reads /proc/meminfo
    MemTotal on Linux (evo-2, spark) / sysctl hw.memsize on Darwin; cpus =
    runtime.NumCPU. 0/"" when unknown, never faked.
  - decoder (clients/visor/fleet.go): fleetRegistration + byoWorker mirror the
    three fields (lockstep with the CLI) and byoWorkers populates them.
  - board (clients/visor/board.go): workerUnits -> byoUnit fills
    fleetSpec.Arch/CPUs/Memory (the fields agentUnits already sets), so a
    gpu-connect node and a code-linked node describe themselves identically.

Verified on a real GB10 (spark-class): detectArch=aarch64 (uname -m), nproc 20,
/proc/meminfo 127600528 kB -> 130662940672 bytes — byte-identical to how the SAME
box already reports as a code-linked run-target (arch=aarch64 cpus=20
memory=130662940672).

Tests: parseMemTotalKB, detectMemTotal (real host >0), detectArch (uname -m
convention, not GOARCH), buildRegistration host spec; fleetRegistration decode +
byoUnit projection + unknown-spec omitted.
2026-07-19 20:37:43 -07:00
antje 32664d746d merge: admin AI-metrics read view (aimetrics-router-verify)
feat/aimetrics-router-verify: clients/admin/aimetrics.go — /v1/admin/aimetrics
read view. Tests: clients/admin (+subpkgs) ok, apps ok.
2026-07-19 20:37:05 -07:00
hanzo-devandantje 5a50f1ee6a admin(aimetrics): AI-metrics read view (clients/admin/aimetrics) + test 2026-07-19 20:36:47 -07:00
antje 6fa3a2df68 merge: thin audited admin credit-grant relay (admin-credit-grant)
feat/admin-credit-grant: clients/admin/creditgrant.go — audited relay to the
commerce credit-grant. Tests: clients/admin (+subpkgs) ok, apps ok.
2026-07-19 20:35:35 -07:00
hanzo-devandantje 34956f0fc2 feat(admin): thin audited credit-grant relay at POST /v1/admin/credit-grants
The one admin mint surface. SuperAdmin-only (core.Guard); forwards verbatim to
commerce's already-mint-gated POST /v1/billing/credit-grants (middleware.Mint →
PlatformOnly) via COMMERCE_SERVICE_TOKEN, scoped to the target org, and writes one
tamper-evident audit record. Commerce stays the sole credit-grant ledger — no
in-process mint. NOT deployed; for red review (mint surface).
2026-07-19 20:35:15 -07:00
antje c50a5646b5 merge: unified inbound channel ingest plane (channels)
feat/channels: clients/channels/{slack,teams,telegram,store}.go + integrations
ingress + cmd/channels — envelope, pairing, policy, per-platform adapters.
Tests: clients/channels ok, clients/integrations ok, apps ok.
2026-07-19 20:34:57 -07:00
hanzo-devandantje 782627658a feat(channels): unified inbound channel ingest plane (envelope, pairing, policy, per-platform adapters)
Preserve divergent channel-ingest work: clients/channels package (envelope
normalization, pairing, delivery policy, Slack/Discord/Teams/Telegram
adapters, store), cmd/channels entrypoint, integration event-emit hooks,
and apps wiring.
2026-07-19 20:30:04 -07:00
antje cbd91a4d1e sites: serve bare <slug>.hanzo.app again — unique-live-slug resolve
The org-scoped host redesign (<slug>.<org>.hanzo.app) left the edge unservable:
a k8s Ingress host and a Let's Encrypt wildcard each match exactly ONE label, so
the two-label shape never routes nor gets TLS, while the one-label product URL
every surface advertises (palette, share copy, publish toast) was rejected by
siteSlug and fell through to the console pipeline. Net: no published site
resolved at all.

Fix, preserving the org-scoped design:
- siteSlug accepts a bare non-reserved <slug>.<apex> label again (org-scoped
  two-label parsing unchanged, ready for per-org certs later)
- siteResolver falls back for bare keys: explicit site_hosts binding first,
  else ResolveUniqueLiveSlug — serve iff EXACTLY ONE live project owns the
  slug across orgs; ambiguous or draft ⇒ honest 404. Deterministic,
  hijack-safe (reserved labels rejected at the host boundary), and
  migration-free for publishes that predate host binding.
- tests: bare-host parse cases + unique/ambiguous/draft resolve proofs
2026-07-19 20:01:41 -07:00
antje 5acb6b5acd chore(gitignore): ignore local .worktrees/ container
The .worktrees/ directory holds local git worktrees (dev infra), never part of
the tree — mirrors the existing .claude/ rule so a working checkout stays clean.
2026-07-19 19:54:27 -07:00
antje b99ee45898 merge: kmsreseal dual-face auth + owner-claim assertion
feat/kms-reseal-migration: split reseal tokenFunc into src/dst faces (CR app-name
credential vs per-org <org>-platform-kms), assert minted-token owner==target org
(refuse admin), flag empty source folders as seeding-wedge risk. Tests: apps ok;
cmd/kmsreseal ok (full suite green under CGO; new auth/owner tests green under CGO-off).
2026-07-19 19:52:29 -07:00
hanzo-devandantje f2ffc0d770 feat(kmsreseal): dual-face auth + owner-claim assertion for the reseal migration
The reseal migration reads from the standalone KMS and writes into cloud KMS —
two faces with DIFFERENT identities. Split the single tokenFunc into srcAuth/dstAuth:
src uses the CR app-name credentialsRef (the standalone accepts it); dst uses the
per-org <org>-platform-kms credential (cloud accepts it dynamically, admin-denied,
no static audience widening).

Defense-in-depth (LOW-1): decodeJWTOwner reads the minted token owner/isAdmin claims
locally and asserts owner == target org (refusing admin tokens) before any read/write,
so a misscoped credential fails its target instead of acting on the wrong org. An
empty source folder is flagged as a seeding-wedge risk instead of silently skipped.

Tests (auth_test.go): decodeJWTOwner, owner-mismatch + admin-refusal gates,
dual-face token brokering.
2026-07-19 19:50:16 -07:00
antje 2a4267fcfb merge: CD per-app detail endpoints (syncwindows, revision metadata, resource-tree SSE), tenant-scoped
feat/cd-detail-endpoints: serve the three per-app endpoints the ArgoCD SPA
detail view calls, scoped by the same resolveScope/findNamespace path as
dashApp. Tests green: clients/deploy, apps.
2026-07-19 19:48:39 -07:00
hanzo-devandantje d0682cd978 deploy: serve the three per-app CD detail endpoints, tenant-scoped
The ArgoCD SPA's application-detail view calls three per-app endpoints the
projection did not serve, spamming "404 page not found" toasts. Add them,
scoped by the same resolveScope/findNamespace path as dashApp — a SuperAdmin
sees the whole fleet, a validated org member sees only its own apps, a
cross-tenant name is a clean 404 (no oracle), an unvalidated caller fails
closed:

- GET /applications/:name/syncwindows -> the permissive-empty
  ApplicationSyncWindowState (no sync windows run; canSync true).
- GET /applications/:name/revisions/:revision/metadata -> honest minimal
  RevisionMetadata (message = the revision, HEAD resolves to the declared
  image tag; date = the CR creation time; author empty). Image-based deploys
  carry no git commit and the manifest repo is not the app's source, so no
  author is fabricated and it never 404s.
- GET /stream/applications/:name/resource-tree -> the live ApplicationTree
  as SSE (data: {"result": tree}), the scope gate before any emission,
  emitted once then refreshed on the keep-alive interval, honoring ctx cancel.
2026-07-19 19:48:16 -07:00
antje 564e6c4386 merge: Hanzo Domains registrar (name.com) + session store + routed-dispatch reach 2026-07-19 18:31:14 -07:00
hanzo-devandantje 3edbbe2753 feat(cloud): domain registrar (name.com) + session store — routed-dispatch reach + CD promote job
clients/domain: registrar layer — name.com client, pricing, register,
per-org store, /v1 mount. clients/session: session store backing routed
runs. Agents: mailbox + routing reach the dispatch targets; release.yml
gains the declared-tag promote job (universe CR bump, Hanzo CD syncs).
2026-07-19 18:31:12 -07:00
hanzo-devandantje bda1a1681c coding: verify + PR + close the session when a routed run completes
A routed run's machine pushes with its own credential and streams into the
session, but cloud still owns the completion — the integrity gate, the PR row, and
the session's terminal state (the machine never closes the session, so it was
staying "running" forever). Give a routed run the SAME cloud-side completion the
local keystone path runs after a sandbox push.

- completeChanged: the shared terminal for a run that reported changes — VerifyRef
  the pushed branch LANDED (fail-closed to a session error + no PR if absent), file
  the native PR, mirror done, close the session done. The local path (Run) now calls
  it too, so the two paths cannot drift.
- finalizeRouted: maps a machine's terminal report onto that completion — reported
  failure closes the session error (no PR), no-changes closes done (no PR), a changed
  push runs completeChanged. No secret crosses; cloud only reads the ref it can see.
- DeliverRoutedRunActivity runs the completion once, after a real report, on a
  cancel-immune bounded context, so a completed run is never re-executed by a retry.
  The completion seam is injected at the composition root (NewDispatcher), the same
  injected-seam shape index_on_push uses, so the free-function activity reaches the
  dispatcher's git/tracker/session seams without a global Dispatcher.
- RoutedRun carries Actor + AgentRef (cloud-side only, never sent to the machine) so
  the completion attributes the session close and files the PR with the right
  assignee.

Also document the mailbox's single-replica dependency at its definition (accepted,
inherited from cloud's KMS-lock replicas:1) with a future replica-aware note.

Tests: routed changed+verify -> PR filed + session done; verify fails -> no PR +
session error; no changes -> done no PR; reported error -> error no PR (verify never
runs); NewDispatcher wires the seam; the durable type bridge preserves attribution.
2026-07-19 18:17:08 -07:00
hanzo-dev bc762e3704 chore(deps): bump hanzoai/ai v1.827.0 -> v1.827.1 (NULL-safe OrgSettings scan) 2026-07-19 13:19:34 -07:00
hanzo-dev 4fd6c8d427 deploy: debrand the projection instance label argocd.argoproj.io -> hanzo.ai
The CD projection synthesized an argocd.argoproj.io/instance label on every app
(visible on every card). It is Hanzo-native CD, not ArgoCD — the App CRs carry
hanzo.ai/* labels. Emit hanzo.ai/instance instead; env + org labels unchanged.
(The argoproj.io/v1alpha1 response SHAPE stays until the @hanzo/gui FE that reads
@hanzo/ui/cd native types replaces the ArgoCD SPA.)
2026-07-19 12:44:19 -07:00
hanzo-dev 0c24ee1322 deploy: tenant-scope the CD projection to IAM orgs and projects
Resolve each /v1/deploy read request's scope from the validated identity —
the same boundary clients/platform.tenant uses (validated principal +
injective provisioning.SanitizeOrg + the c.IsAdmin SuperAdmin predicate).
A SuperAdmin sees the whole fleet; a validated org member sees only its own
org's apps (hanzo.ai/org label, tenant-<org> namespace); anyone else is
refused. Scoped reads: applications list/detail/resource-tree, clusters,
projects, and the SSE stream. sync/rollback + the argocd bootstrap stay
SuperAdmin-only.

projectApp reads app.kubernetes.io/part-of into spec.project (default when
absent) and surfaces hanzo.ai/org. The projects endpoint reflects the
IAM-owned (org,name) Project resource in-process — org-scoped for a normal
org, all orgs for a SuperAdmin — with a synthesized default so every app's
spec.project resolves. IAM stays the single source; no CD-side project row.
2026-07-19 12:44:19 -07:00
hanzo-dev e9d470d089 chore(deps): bump hanzoai/ai v1.826.7 -> v1.827.0
Integrates the last two router branches now on ai main:
- per-org RoutingPolicy on the hot path (decomplected per-org routing)
- context_window surfaced in /v1/models

(do-ai premium routes + judge/MFJP + mean-field + RouterCostCeiling slider
+ the judge-panel deadlock fix already shipped via v1.826.7, already live.)
clients/... compiles clean against v1.827.0.
2026-07-19 12:32:04 -07:00
hanzo-dev 7b073f334f deploy(stream): guard typed-nil watch object + recover on watch goroutines
A malformed watch event carrying a typed-nil *unstructured.Unstructured would
nil-deref on GetName() in forwardWatch. The read plane installs no panic
recovery around detached goroutines, so that crash would take down the whole
process. Guard the typed-nil, and recover at the spawn site so no future
malformed event can crash the plane. Adds a regression test.

Red review: SHIP (this closes the sole LOW finding).
2026-07-19 10:15:48 -07:00
hanzo-dev 00a31dd644 feat(deploy): project /clusters, /projects, /stream/applications for the CD dashboard
The ArgoCD-UI-compatible surface returned nothing at three endpoints the
applications view calls, so the SPA error-toasted on load:

  GET /v1/deploy/clusters            -> 404
  GET /v1/deploy/projects            -> 404
  GET /v1/deploy/stream/applications -> 404

Add all three as read-only projections over the SAME App-CR source dashAppList
reads (listAppCRs + runningVersions + projectApp: one source, one projection),
SuperAdmin-gated by guard(), safe on cloud-reader (no writer/commerce imports):

- /clusters -> ClusterList of the destinations the fleet reconciles into,
  deduped, always including the in-cluster destination, with a per-cluster
  application count. argoCluster has no config field, so a cluster credential
  cannot be surfaced by construction.
- /projects -> AppProjectList: prefers real argoproj.io/v1alpha1 AppProject CRs
  when that CRD is served (reshaped to only the intended spec fields), otherwise
  synthesizes one permissive project per distinct App-CR project name (default
  always present).
- /stream/applications -> the applications watch as SSE: one ADDED event per
  current App CR, then live ADDED/MODIFIED/DELETED from a per-namespace watch,
  held open with keep-alives. Every watch + goroutine is bound to the request and
  torn down on disconnect; degrades to keep-alive only if the watch verb is not
  granted; fails closed (503) with no cluster client.

Tests (go test -race green): cluster dedupe + always-in-cluster + never-emits-
credentials; project distinct/default + synth-permissive + real-CR-only-intended-
fields; stream ADDED-per-app + zero-app-no-panic + honors-ctx-cancel + SSE-headers;
all three routes 403 without SuperAdmin.
2026-07-19 10:15:48 -07:00
zandGitHub dae694eebf fix(commerce): stop the in-process self-dispatch recursion that crash-loops the writer (#341)
scopeRateLimiter reads its own rules via a co-resident commerce self-dispatch (GET /v1/billing/spend-alerts) on every authed request; the rule cache fills only after the fetch returns, so the self-dispatch re-enters scopeRateLimiter with a cold cache → unbounded in-process recursion → writer stack-overflow (single request) / OOM (concurrent). Dump-attributed (goroutine 4423, 21,823 setRequestCancel) and real-binary A/B verified on current main+fix (GET returns, POST 402s, 40-concurrent peaks 228 goroutines, 0 pileup). Exempt the commerce config surface from its own gate + an on-path depth backstop.
2026-07-19 09:56:11 -07:00
zeekayandhanzo-dev 7107d3c43d harden(team): Chunter responder OFF by default + bounded/lazy (anti-storm)
Post-mortem containment for the v1.801.104 writer crash. To be unambiguous on
root cause: the fatal was the commerce co-resident dispatch reentrancy
(stack: apps.mountCommerce.IAMTokenRequired.func5 → commerce@v1.49.3
iammiddleware.go:157 → unbounded net/http.setRequestCancel goroutines), the bug
PR #341 fixes, introduced by 697b89e (enso per-tier gate) in the .99→.104 range —
NOT this responder. The responder makes ZERO outbound calls at boot (it fires only
from session.tx, the live client-WS write path; never from reconcile/replay). Any
build of current main still crashes until #341 lands, independent of this change.

That said, an unbounded per-message responder IS a foot-gun, so this makes it
safe-by-default and bounded regardless:

- OFF by default: Mount wires the LLM seam ONLY when TEAM_AGENTS_ENABLED=1. A nil
  runAgent makes maybeAgentReply return at the top → NO outbound model call can
  fire. An un/mis-configured binary is provably inert.
- Fresh-only: a message created before this process booted (>60s grace) is a
  replay/backfill and is NEVER answered — kills the "replayed backlog fans out into
  thousands of HTTP calls" failure mode.
- Single-flight per (workspace, space, bot): a burst to one conversation collapses
  to one turn; duplicates dropped, not queued.
- Hard concurrency cap: a global semaphore (TEAM_AGENTS_MAX_CONCURRENCY, default 4,
  clamped 1..64) bounds in-flight turns; over the cap, DROP.
- Circuit breaker per agent: after 3 consecutive failures skip the agent for 60s —
  the backoff that turns a publishable-key 403 storm into a quiet trickle. No
  retries, ever.

TDD (all -race green): TestNoReplyToBacklogAtBoot boots against a 500-message
backlog and asserts ZERO runner calls (then one fresh post IS answered);
TestConcurrencyCapBounded (cap=2, 8 msgs → exactly 2 in-flight, rest dropped);
TestSingleFlightPerConversation (5 msgs, 1 conversation → 1 turn);
TestCircuitBreakerBacksOff (persistent failure → runner called exactly threshold
times, circuit opens). Existing responder + roster tests unchanged.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-19 09:47:44 -07:00
hanzo-dev 6625b5d41b feat(deploy): project /clusters, /projects, /stream/applications for the CD dashboard
The ArgoCD-UI-compatible surface returned nothing at three endpoints the
applications view calls, so the SPA error-toasted on load:

  GET /v1/deploy/clusters            -> 404
  GET /v1/deploy/projects            -> 404
  GET /v1/deploy/stream/applications -> 404

Add all three as read-only projections over the SAME App-CR source dashAppList
reads (listAppCRs + runningVersions + projectApp: one source, one projection),
SuperAdmin-gated by guard(), safe on cloud-reader (no writer/commerce imports):

- /clusters -> ClusterList of the destinations the fleet reconciles into,
  deduped, always including the in-cluster destination, with a per-cluster
  application count. argoCluster has no config field, so a cluster credential
  cannot be surfaced by construction.
- /projects -> AppProjectList: prefers real argoproj.io/v1alpha1 AppProject CRs
  when that CRD is served (reshaped to only the intended spec fields), otherwise
  synthesizes one permissive project per distinct App-CR project name (default
  always present).
- /stream/applications -> the applications watch as SSE: one ADDED event per
  current App CR, then live ADDED/MODIFIED/DELETED from a per-namespace watch,
  held open with keep-alives. Every watch + goroutine is bound to the request and
  torn down on disconnect; degrades to keep-alive only if the watch verb is not
  granted; fails closed (503) with no cluster client.

Tests (go test -race green): cluster dedupe + always-in-cluster + never-emits-
credentials; project distinct/default + synth-permissive + real-CR-only-intended-
fields; stream ADDED-per-app + zero-app-no-panic + honors-ctx-cancel + SSE-headers;
all three routes 403 without SuperAdmin.
2026-07-19 09:25:40 -07:00
hanzo-dev eb48905ecc feat(deploy): project /clusters, /projects, /stream/applications for the CD dashboard
The ArgoCD-UI-compatible surface returned nothing at three endpoints the
applications view calls, so the SPA error-toasted on load:

  GET /v1/deploy/clusters            -> 404
  GET /v1/deploy/projects            -> 404
  GET /v1/deploy/stream/applications -> 404

Add all three as read-only projections over the SAME App-CR source dashAppList
reads (listAppCRs + runningVersions + projectApp: one source, one projection),
SuperAdmin-gated by guard(), safe on cloud-reader (no writer/commerce imports):

- /clusters -> ClusterList of the destinations the fleet reconciles into,
  deduped, always including the in-cluster destination, with a per-cluster
  application count. argoCluster has no config field, so a cluster credential
  cannot be surfaced by construction.
- /projects -> AppProjectList: prefers real argoproj.io/v1alpha1 AppProject CRs
  when that CRD is served (reshaped to only the intended spec fields), otherwise
  synthesizes one permissive project per distinct App-CR project name (default
  always present).
- /stream/applications -> the applications watch as SSE: one ADDED event per
  current App CR, then live ADDED/MODIFIED/DELETED from a per-namespace watch,
  held open with keep-alives. Every watch + goroutine is bound to the request and
  torn down on disconnect; degrades to keep-alive only if the watch verb is not
  granted; fails closed (503) with no cluster client.

Tests (go test -race green): cluster dedupe + always-in-cluster + never-emits-
credentials; project distinct/default + synth-permissive + real-CR-only-intended-
fields; stream ADDED-per-app + zero-app-no-panic + honors-ctx-cancel + SSE-headers;
all three routes 403 without SuperAdmin.
2026-07-19 09:25:40 -07:00
zeekayandhanzo-dev d4c435b17c feat(team): Chunter agent responder — org agents become talkable in chat
Bots-as-members (bots.go/roster reconcile) already projects each org agent as a
workspace Employee, but a message to a bot did nothing — the AI was present and
mute. This adds the WRITE/response half: when a human posts a Chunter ChatMessage
addressed to an active bot member — a DirectMessage whose participants include the
bot, or a channel message that @-mentions it — the transactor runs that agent
through agents.RunOnBehalf (the ONE billed/metered/recorded in-process run path)
and posts the model's answer back into the SAME conversation as that bot, via the
SAME applyTx + hub.broadcast write the SPA and roster projection use.

- chat.go: parseChatMessage, replyTargets (DM-member OR @mention addressing),
  maybeAgentReply (cheap gate → agents list → per-bot async turn), replyAsBot
  (recovered + 90s-bounded goroutine; never blocks the WS loop). plainText/
  htmlMarkup bridge stored markup ↔ LLM text. Loop guard: a bot-authored message
  never triggers a reply.
- transactor.go: transServer gains runAgent (the LLM seam) + log; session.tx fires
  maybeAgentReply on the client write path only (roster/sync call applyTx directly,
  so a projection can never trigger a reply).
- bots.go: agentReplyRunner adapts agents.RunOnBehalf (error-status run → post
  nothing, never an empty bubble).
- team.go Mount wires runAgent=agentReplyRunner. Responder is off (nil runner) when
  unwired, so the path is fully additive.

TDD: 16 tests — parse/ignore, plainText/htmlMarkup, DM + mention addressing, the
full DM reply loop with a fake runner (asserts on-behalf-of user, agent id, plain
prompt, reply authored by the bot in the same conversation), loop-guard, and
disabled-when-no-runner. All clients/team tests green (CGO_ENABLED=0), go vet clean.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-19 09:06:50 -07:00
hanzo-dev 5694426431 chore(deps): bump ai v1.826.6 -> v1.826.7 (judge self-call pileup fix; unblocks commerce v1.49.3) 2026-07-19 07:16:48 -07:00
hanzo-dev f1c663c1bc feat(deploy): OAuth sign-in for the CD dashboard (PKCE, verify-before-mint)
cd.hanzo.ai rendered but every API call 403'd with no way to sign in: the IAM
session cookie is host-scoped to hanzo.id, so no other host's session can ever
authorize cd, and clients/deploy had no login route.

Adds GET /v1/deploy/login + /callback + POST /logout against the admin-console
app (org=admin; hanzo-cloud's org is hanzo, which is why admin users were never
found). Public PKCE client, no secret. The callback verifies the exchanged token
through cloud's own JWKS validator and decides SuperAdmin on VERIFIED claims
before minting; the unverified decode is gone. session/userinfo becomes a public
bootstrap so the SPA can discover it is signed out. guard() is unchanged.

DEPLOY_PUBLIC_URL is now REQUIRED (login/callback 503 without it).

blue built, red cleared: gate stronger than before, zero regressions.
2026-07-19 04:06:40 -07:00
hanzo-dev 2044c2684c deploy: verify the token before minting the session; make sign-in reachable
Red review of the sign-in round trip found no security defect, and four ways it
would fail in practice. All four are the same shape: correct in the happy path,
unhelpful or unreachable in the real one.

MINT ONLY WHAT THIS DEPLOYMENT WILL ACCEPT. The callback now runs the exchanged
token through cloud's OWN validator before writing the cookie, and decides the
admin-org question on VERIFIED claims. The audience allowlist is env-overridable
and jwtAudiencesFromEnv REPLACES the baked default, so a deployment whose
CLOUD_JWT_AUDIENCES / GATEWAY_ALLOWED_AUDIENCES omits this console's client_id
minted a cookie the boundary refused on the next request — 403, document-bounce
to sign-in, IAM session still live, instant code, mint, 403, forever. It now
fails once, with the reason and the knob to turn. The unverified claim decode is
gone with it.

That validator is exported from cloud (NewTokenValidator) rather than rebuilt
here: SanitizeIdentity validates on the way in, a subsystem minting a session
needs the same verdict a moment earlier, and two copies of it could drift into
exactly the mint-then-refuse loop above. jwksURLFor is now the one derivation
both share.

SIGN-IN HAS TO BE REACHABLE FROM WHERE THE USER IS. The dashboard is an XHR
client, so guard()'s document bounce never fires for it — it got a 403 and dead
-ended with no route to sign-in. /v1/deploy/session/userinfo is now the one
public bootstrap route: {loggedIn:false} plus the sign-in URL for an anonymous
caller, the real identity for a SuperAdmin. It discloses no identity, no cluster
state, no configuration, and gates nothing; every route that returns fleet data
or mutates a CR stays guarded.

The OAuth origin is now configuration, not the Host header. Deriving a redirect
from caller-controlled input is only ever saved by the registry's exact-match
check — a second lock covering for a broken first one. With no DEPLOY_PUBLIC_URL
sign-in fails closed naming the knob.

Logout is POST: as a GET any site could sign a SuperAdmin out by navigation,
which a SameSite=Lax cookie still rides. The session lifetime comes from the
verified expiry, clamped — an already-expired token no longer becomes an
eight-hour cookie, and a bogus far-future exp no longer becomes a decade-long
one. Both cookies take the __Host- prefix, which the browser only honours for
Secure, Path=/, Domain-less cookies, so a sibling *.hanzo.ai host cannot shadow
them.
2026-07-19 03:45:48 -07:00
hanzo-dev c35cdc72c0 deploy: sign in to the cd console with IAM, so its SuperAdmin gate is reachable
Every /v1/deploy route gates on c.IsAdmin(), which SanitizeIdentity mints only
from a validated IAM principal whose org is the reserved admin org. The console
had no way to establish one: the IAM session cookie is host-only on hanzo.id, so
a session from hanzo.id or admin.hanzo.ai is never presented to cd.hanzo.ai, and
the whole surface 403'd with no sign-in anywhere. Add the round trip.

  GET /v1/deploy/login    redirect into IAM authorize (PKCE S256, CSRF state)
  GET /v1/deploy/callback exchange the code, mint the session, land on returnTo
  GET /v1/deploy/logout   clear the session for this host

The session is cloud's EXISTING one: the callback writes the IAM access token to
hanzo_iam_token, the first name in cookieTokenNames, which SanitizeIdentity
already reads and independently verifies (signature, issuer, audience, expiry)
into the same principal a Bearer yields. No second session mechanism, and the
gate, the validation and the SuperAdmin predicate are unchanged.

Fail closed at every step: no code is redeemed unless the returned state equals
the nonce in the HttpOnly flow cookie this browser started with (login CSRF), a
principal outside the admin org is refused a cookie outright, and a return path
that is not a path on this host collapses to /. The cookie is HttpOnly, Secure,
SameSite=Lax and host-only, so page JS cannot read the token and no cross-site
POST carries it.

guard() keeps c.IsAdmin() as the only gate; only the shape of the refusal is
negotiated. A browser navigation is sent to the sign-in page instead of a dead-end
403; every API call keeps its 403, decided by Sec-Fetch-Dest/Mode when present and
never inferred from Accept alone, and a non-GET is never redirected.

admin-console is the client because its IAM organization is the admin org;
hanzo-cloud is owned by admin but organized under hanzo, so it resolves
admin-org users in the wrong org and never finds them.
2026-07-19 02:49:20 -07:00
zandGitHub 412b367ad6 chore(deps): bump commerce v1.49.3 — bounded org-resolution cache on the auth path (#340)
Auth-path org resolution hit the datastore on every request, allocating the
Organization before the blocking store call, so requests stalled on the
connection pool each pinned one and the heap tracked the backlog. Confirmed
from a live goroutine profile: 46 waiters in sql.(*DB).conn under
org.Resolve <- IAMTokenRequired, organization.New at 26.4% of a 1301MB heap.

Carries three fixes uncovered while landing it:
- follow commerce's resolver consolidation (middleware/svcorg -> pkg/org)
- point the go-unit test list at clients/flags; the stale clients/featureflags
  path failed setup on a missing directory and had CI/CD red on main
- assert the post-#331 flags contract: runtime flags ignore env, boot-time
  ReadOnly rows still read it. That test asserted the override #331 removed
  and never ran because of the stale path above.
2026-07-19 02:47:02 -07:00
hanzo-dev 86b908e7f5 test(flags): assert the post-#331 contract — runtime flags ignore env
#331 stripped Env from waitlist_*/public_signup/gateway_* so /v1/flags is
the single source of truth, but left this test asserting the env override
it had just removed. The stale clients/featureflags path meant the package
never ran in CI, so it stayed green.

Now pins both halves: a runtime flag holds its default against an env var,
and a boot-time ReadOnly row still reads env.
2026-07-19 02:30:45 -07:00
hanzo-dev 0e8086f88d test(flags): assert the post-#331 contract — runtime flags ignore env
#331 stripped Env from waitlist_*/public_signup/gateway_* so /v1/flags is
the single source of truth, but left this test asserting the env override
it had just removed. The stale clients/featureflags path meant the package
never ran in CI, so it stayed green.

Now pins both halves: a runtime flag holds its default against an env var,
and a boot-time ReadOnly row still reads env.
2026-07-19 02:30:45 -07:00
hanzo-dev a16db40e16 fix(ci): point the go-unit test list at clients/flags
clients/featureflags was renamed to clients/flags; the stale path made
'go test' fail setup on a directory that does not exist, which has been
failing CI/CD on main.
2026-07-19 02:13:29 -07:00
hanzo-dev 9f4cc62ca6 fix(ci): point the go-unit test list at clients/flags
clients/featureflags was renamed to clients/flags; the stale path made
'go test' fail setup on a directory that does not exist, which has been
failing CI/CD on main.
2026-07-19 02:13:29 -07:00
hanzo-dev ffe8c05047 test(metering): follow commerce's resolver consolidation to pkg/org
commerce v1.49.3 folds middleware/svcorg into pkg/org so one resolver
serves every caller; Invalidate moves with it.
2026-07-19 01:47:45 -07:00
hanzo-dev 24119a53b6 test(metering): follow commerce's resolver consolidation to pkg/org
commerce v1.49.3 folds middleware/svcorg into pkg/org so one resolver
serves every caller; Invalidate moves with it.
2026-07-19 01:47:45 -07:00
hanzo-dev 86100a79c1 chore(deps): bump commerce v1.49.3 — bounded org-resolution cache on the auth path
Auth-path org resolution hit the datastore on every request and allocated
the Organization before the blocking store call, so requests stalled on the
connection pool pinned one each and the heap tracked the backlog. v1.49.3
serves request-owned copies from a bounded LRU and refuses credential-named
orgs at the model layer.
2026-07-19 01:32:15 -07:00
hanzo-dev eb9bee21ef chore(deps): bump commerce v1.49.3 — bounded org-resolution cache on the auth path
Auth-path org resolution hit the datastore on every request and allocated
the Organization before the blocking store call, so requests stalled on the
connection pool pinned one each and the heap tracked the backlog. v1.49.3
serves request-owned copies from a bounded LRU and refuses credential-named
orgs at the model layer.
2026-07-19 01:32:15 -07:00
0515c77120 analytics(ingest): PostHog-wire uuid->idempotent MessageID + utm_* attribution mapping (#338)
Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-19 00:05:43 -07:00
hanzo-dev 53c17fb3cc chore(cloud): vendor hanzoai/ai v1.826.6 — DO model catalog + mean-field + judge-panel
Brings the full run into the deployed service: 55-model DO GenAI catalog (Claude
opus-4.8/sonnet-5/fable-5/haiku, GPT-5.6/5.5/4o/o3, deepseek-v4-pro, llama-4, qwen,
glm, kimi — capabilities declared per live probe), the mean-field congestion router
(gated), the live /v1/router/judge-panel endpoint, the Mean-Field Judge Panel, and
geo-aware consent. Prod model ConfigMap (universe) syncs the catalog data separately.
2026-07-18 23:45:24 -07:00
hanzo-dev c71dce4116 chore(deps): bump commerce v1.49.2 — legacy numeric org-id resolves by name (SEV1)
Replaces the pseudo-version pin (v1.49.2-0.20260719024505-24ff20a68f52, the
Bug-A iterator-leak fix only) with the released v1.49.2, which also carries the
Bug-B guard: org.Resolve skips the doomed GetById for a legacy all-digit cached
id (IAM Valkey's stale 1772587477 for 'hanzo') and resolves by name, so the
(*Query).ById legacy-numeric path that hot-looped in v1.801.95 is never taken.
Keeps ai v1.826.4 (in-proc TierReader). go.mod+go.sum only.
2026-07-18 21:43:01 -07:00
z defdf84aea metering: SEV1 fix — cap authorize HARD-timeouts + fails open, never hangs completions
The auth fix let the metering cap check actually reach commerce AuthorizeSpendCap; a
legacy-org GetById hot-loop there then HUNG every completion (no timeout on the
in-proc authorize) — a cap that can block/hang the completion path is worse than one
that does not enforce. scopeAuthorize now runs the authorize under a strict 1.5s
deadline AND a select-based hard timeout that returns even if the in-proc handler
goroutine is STUCK (an unresponsive hot-loop cannot be interrupted, so ctx alone would
not unblock). On timeout OR any error -> AuthorizeVerdict fails OPEN (allow) — a slow,
broken, or hot-looping commerce ALWAYS allows, never waits. OnCapError logs each
fail-open so a degraded cap is observable. Regression test: a 10s-hanging authorize
returns an ALLOW in ~1.5s (completion never hangs).

The commerce hot-loop itself (the root cause) is fixed separately; this timeout is the
non-negotiable safety net that makes the cap path unable to hang regardless.
2026-07-18 21:17:47 -07:00
hanzo-dev 48a7814d3f chore(cloud): vendor hanzoai/ai v1.826.4 — Mean-Field Judge Panel + geo-consent live
v1.826.4 ships the LLM-as-judge dense-reward loop fully activated: the Mean-Field
Judge Panel (diverse calibrated judges, reputation-weighted consensus), geo-aware
consent (EU/UK/EEA explicit opt-in via CF-IPCountry, non-EU opt-out default), judge
config dynamic at admin.hanzo.ai (OrgSettings "*" row, no env), MFJP enabled by
default on a diverse cheap panel, and internal dev orgs seeded on. Judge scoring uses
the existing probe service bearer (no new secret). Also carries the MFJP + scientific
proof from v1.826.3.
2026-07-18 21:08:23 -07:00
hanzo-dev 94a24fce47 chore(deps): bump commerce to datastore iterator conn-leak fix
commerce 24ff20a6 closes single-row query iterators (Query.First). This
stops the Postgres pool leak that starved org.Resolve on the co-resident
balance + per-tier gate path — the 'context deadline exceeded' that made
the Enso per-tier SKU gate fail open and spiked chat latency to 10-40s.
2026-07-18 19:47:55 -07:00
zeekayandhanzo-dev 53761f7e73 test(apps): refreeze wire golden — add dns + cloudflare subsystems
Wire() gained the /v1/dns zone plane (after projects) and /v1/cloudflare edge
plane (after integrations) but the frozen golden in wire_test.go was not
updated, so TestWireOrderMatchesFrozen failed (87 specs vs 85 frozen) — which
red-lit cloud's CI/CD and blocked the auto-release image build. Refreeze the
golden to the exact runtime sequence (verified position-by-position, 87==87).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-18 18:41:01 -07:00
hanzo-dev c5b34364e8 feat(k8s): unified /v1/k8s surface on api.hanzo.ai — DOKS clusters + nodes
The ONE Kubernetes noun, proxied to Visor (clients/visor/k8s.go): list the
org's DOKS clusters, one cluster's detail (node pools + worker nodes), DEPLOY
(create) / delete clusters, and the fleet-wide worker nodes. Reads are org-scoped
by the validated IAM owner; mutations (create/delete) are admin-gated
(principal.IsSuperAdmin || IsOrgAdmin) — real house-account infra spend.

Consolidates the worker-node consumption: managedMachines now reads
/v1/k8s/nodes (was /v1/kubernetes-nodes), matching Visor's consolidated path —
no parallel kubernetes-* surface remains.

- k8s.go: listK8sClusters / getK8sCluster / createK8sCluster (admin) /
  deleteK8sCluster (admin) / listK8sNodes; wire structs + view mappers.
- visor.go: mount the /v1/k8s/* group; managedMachines -> /v1/k8s/nodes.
- tests: proxy + tenant-scoping, detail shape, nodes, and the admin gate
  (a non-admin create/delete is refused BEFORE reaching Visor); the fleet
  DOKS-node fake tracks the new /v1/k8s/nodes path.
2026-07-18 17:38:13 -07:00
67594d3341 chore(deps): bump hanzoai/ai v1.826.0 → v1.826.2 (dense auto-reward + exploration floor) (#336)
Brings the flywheel-turning fixes into the deployed binary: v1.826.1 LLM-judge dense
quality rewards + v1.826.2 dense implicit auto-reward (quality×cost) + epsilon
exploration floor (#109). Enables ROUTER_AUTOREWARD_ENABLED / ROUTER_EXPLORE_EPSILON.
./apps (ai.Mount) compiles clean against v1.826.2.

Co-authored-by: zeekay <zeekay@hanzo.ai>
2026-07-18 17:32:15 -07:00
zeekayandhanzo-dev dc923e566c docs(platform): correct buildJobSpec doc-drift (RED INFO)
- buildJobSpec doc claimed the REVERTED over-hardening (allowPrivilegeEscalation=
  false, all caps dropped); correct it to the actual documented rootless posture
  (defaults left for rootlesskit newuidmap) + point at the securityContext.
- tenantPullSecretName comment overstated "cloud-api holds no secrets grant";
  clarify cloud's only Secrets write is the per-tenant KMS-auth creds in a TENANT
  ns, and that the isolated build ns must stay OFF the tenant-RBAC selector so no
  secrets grant is projected there (R6).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-18 17:23:20 -07:00
zandGitHub 2b4150638c Merge pull request #335 from hanzoai/fix/platform-projects-500
fix(platform): GET /v1/platform/projects 200-empties on unavailable IAM store (console-init 500)
2026-07-18 17:19:51 -07:00
hanzo-dev b238d809e1 fix(platform): list projects degrades to 200 empty when IAM store unavailable
GET /v1/platform/projects 500'd on console dashboard init. The iamStore guard
converts a nil co-resident IAM object store into a typed 503, but listProjects
re-stamped ANY store error as a 500 (zip.Errorf(500, "list: %v", err)),
discarding the status — so a signed-in session's first read broke dashboard
init with {"status":500,"error":"list: platform requires the co-resident IAM
store, which is not initialized"}.

The dashboard's first authenticated read now degrades any store failure to an
empty project set (200 []) — a new org genuinely has zero projects — logging the
real cause for operators (never swallowed), written in-band so no outer error
filter can reflatten it. Also guards a stray nil row from nil-derefing into a
500. The store-level 503 guard + its three unit tests are unchanged.

Repro + regression gate: TestListProjects_NilIAMStore_ServesEmpty200 (real
iamProjects over a nil in-process IAM engine — the deployed condition) and
TestListProjects_StoreError_ServesEmpty200.
2026-07-18 17:18:39 -07:00
zeekayandhanzo-dev 21c171bef2 fix(paas): RED H1/L1 — deploy is superadmin-only + explicit-env (close the platform-restart DoS)
RED found the /v1/paas auth broadening handed every brand-org ("hanzo") OrgAdmin
fleet-wide rolling-restart of the platform's OWN tier (the only namespaces the board
scans are hanzo{,-testnet,-devnet}, where iam/kms/gateway/cloud/… run) — a live DoS
lever, partially re-opening the 2026-07-08 admin-org P0.

H1: the MUTATING POST /v1/paas/apps/:app/deploy now uses operatorGuard (principal.
IsSuperAdmin ONLY), not the broad read guard. Restarting a shared platform service is
a platform-operator action; a customer-org admin — even of the brand org — is refused
403. The READ board (list/get) stays SuperAdmin||OrgAdmin (observe, audit-logged,
bounded). Confinement (scopedNamespaces) unchanged.

L1: deploy REQUIRES ?env=main|test|dev (nsForEnv-validated) — a bare deploy no longer
silently targets production; the CLI requires --env before the call.

Tests: TestDeploy_OrgAdmin_403_Platform (the H1 regression), _NonAdmin_403,
_SuperAdmin_RollingRestart, _RequiresExplicitEnv, _SuperAdmin_EnvSelectsNamespace;
CLI TestDeployRequiresEnv. All green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-18 17:14:42 -07:00
hanzo-dev ed5d5be5dd fix(deps): bump ai v1.825.2 -> v1.826.0 (enso per-tier gate enforcement + real router-stats model ids) 2026-07-18 17:07:21 -07:00
hanzo-dev 697b89e11e fix(billing): enforce enso per-tier gate — inject co-resident commerce tier into ai
The embedded ai per-tier SKU gate (family_tier.go) was fail-open in-cluster: it
resolved the caller's tier with an authed HTTP self-call to the cloud edge, which
401/403s a service token on /v1/billing/*, so the gate saw "" and admitted every
tier — enso/enso-ultra were open to free callers.

Mirror wireFinance's SetBalanceReader: install aiobject.SetTierReader so ai reads
the subscription tier DIRECTLY over the co-resident commerce client the metering
gate already bills over (commerceinproc in-process, with the service token commerce
itself accepts) — never the cloud edge. Add metering.Client.Tier to decode tier.name
from GET /v1/billing/tier. Fail-safe preserved: a commerce error or unknown tier
folds to "" (allow), so a commerce blip never locks out a paying caller.

Bumps ai v1.824.2 -> v1.825.2 (the object.TierReader seam).
2026-07-18 16:54:47 -07:00
hanzo-dev 7070fffd03 feat(visor): fold DOKS worker nodes into the fleet — 3rd machine source
managedMachines unioned Visor's registry (/v1/get-machines) + live droplet list
(/v1/machines); a DOKS cluster's worker NODES appeared in neither (their droplet
carries a k8s tag, not a hanzo-org droplet tag), so world.hanzo.ai showed
standalone droplets but never cluster nodes.

Add GET /v1/kubernetes-nodes as the THIRD source (Visor unions the house-account
hanzo-org-tagged clusters + BYOC Provider.ClusterID clusters and returns each
worker node as a Machine keyed by droplet id). It is processed after registry and
live, so a DOKS node whose droplet is ALSO in the live list dedupes by droplet id
and never lists twice; a cluster-only node surfaces. Independently resilient like
the other two — a kubernetes-nodes outage is logged and skipped, never hiding the
registry/live/BYO sources.

Test: TestMachinesMergeDOKSNodes — a DOKS-only node appears, and a node whose
droplet is already live collapses BY ID (the node row carries a different name, so
only id-dedup can merge it). Full clients/visor suite green (31 subtests).
2026-07-18 16:52:27 -07:00
zeekayandhanzo-dev 3a9a00f2b2 docs(llm): document the unified hanzo CLI ↔ /v1/paas contract (apps/deploy/clusters off one IAM login)
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-18 16:49:03 -07:00
hanzo-dev 11bf05434f merge(cloudflare): move asset routes under /v1/integrations/cloudflare — unified provider shape, never top-level 2026-07-18 16:46:59 -07:00
zandGitHub 4cc5f87137 Merge pull request #334 from hanzoai/feat/event-canonical
analytics: canonical POST /v1/event + fail-closed key->org convergence
2026-07-18 16:46:45 -07:00
hanzo-dev 6fe0f98a5f analytics: canonical POST /v1/event front door (Event|[]Event), one write core
POST /v1/event is the ONE ingestion door: body is a single Event or a JSON-array
batch (no /v1/event/batch), org resolved IAM-only and fail-closed (eventTenant),
funneled through the ONE write core (ingestEvents) into hanzo.events. The
Segment/beacon (/v1/analytics,/v1/tracker) and PostHog (/v1/insights/e) wires
become thin DEPRECATED adapters over the same core. Org is never read from body.
2026-07-18 16:46:16 -07:00
hanzo-dev fd55a5b5c0 analytics: fail-closed project-key->org via the ONE IAM key seam (cloud.OrgForKey)
capture resolves a presented project/API key to its owner org through the single
IAM key resolver (sharedKeys, 60s cache incl. miss-cache). A presented-but-
unresolvable key is refused (403) and NEVER falls through to the brand-host
fallback, so a keyed request can never cross-tenant write. Anonymous marketing
traffic still resolves to the public brand org server-side from Host.
2026-07-18 16:46:16 -07:00
eb6ab336d5 chore(deps): bump hanzoai/ai v1.824.2 → v1.825.1 (Enso auto-serve + churn-resilient trainer) (#333)
Brings the merged Enso router fixes into the deployed cloud binary:
- #107 (v1.825.0): auto never routes to a family SKU it can't serve + forward the
  resolved model (withModel body rewrite) → model=auto serves 200 (was 404); grant-
  aware known predicate; flywheel boots from the single shared Bootstrap.
- #108 (v1.825.1): trainer fits EARLY (~90s after boot) then cadence → completed
  retrain cycles survive frequent redeploys (churn-resilient).
./apps (ai.Mount site) compiles clean against v1.825.1 (API-compatible).

Co-authored-by: zeekay <zeekay@hanzo.ai>
2026-07-18 16:44:32 -07:00
z 7e38d98551 billing: admit the verified S2S service token past the /v1/billing/* gate (cap authorize) + flag commerce
The metering cap-gate authorize (and the SuperAdmin cap-oversight Forward) call the
in-proc /v1/billing/spend-alerts/authorize with the COMMERCE_SERVICE_TOKEN, but the
customer /v1/billing/* bridge required a validated IAM principal -> "sign in to view
billing" (403) -> the cap fails-open and never enforces. billingData now admits a
trusted S2S caller carrying the verified service token: org from the EdgeAuth-controlled
X-Org-Id, query forwarded VERBATIM (a trusted caller names its own subject), no
subject-pin. A public caller can never present it (the gateway 401s a Bearer that is
not an IAM JWT / hk-|pk-|sk- key), and an unauthenticated caller still gets 403 (tested).
Adds spend-alerts/authorize to the GET allowlist; constant-time token compare.

ATOMIC with the flag: bumps commerce to v1.49.2 (SPEND_CAP_ENFORCE, default OFF), so
the instant the cap can reach the handler the enforcement gate is fail-open in the
binary -> auto-deploy stays safe until an operator flips the flag after the canary proof.

Security invariants tested: public/unauth -> 403; wrong bearer -> 403; verified token +
X-Org-Id -> forwards verbatim; token без X-Org-Id -> 403.
2026-07-18 16:40:05 -07:00
hanzo-dev 044eb14c18 refactor(cloudflare): move asset routes under /v1/integrations/cloudflare
The per-org Cloudflare asset plane (Pages/Workers/R2/KV/D1) is repointed from a
first-level /v1/cloudflare/* surface to /v1/integrations/cloudflare/*, so a
third-party provider is connected AND used under one unified namespace — matching
where the connector's connect/callback/verify/disconnect legs and the KMS token
coordinate already live. Pure path move: no auth, isolation, or handler logic
changes. Route registrations, doc comments, and tests repointed together.

No collision with the connector's parametric routes: the asset paths are all
3+ segments (/cloudflare/{pages,workers,r2,kv,d1}/...) while the connector's
/v1/integrations/:provider and /:provider/{connect,callback,disconnect,verify}
are 1- and 2-segment patterns whose literal second segment never equals an asset
group. Static-under-param co-registration is already proven in the integrations
plane (slack/link static beside :provider). Mount order unchanged: integrations
before cloudflare.
2026-07-18 16:38:59 -07:00
hanzo-dev 606c20bcc5 merge(dns): /v1/dns forward head — path-guarded org-scoped proxy so console.hanzo.ai/dns loads zones
Red-cleared (double-encoding traversal closed, 9 tests green). Bearer-relayed, no standing cred.
2026-07-18 16:28:42 -07:00
zeekayandhanzo-dev 2dab7ddf12 fix(platform): rootless buildkit securityContext — match documented posture
The first rootless spec over-hardened (allowPrivilegeEscalation:false +
capabilities drop ALL), which breaks rootlesskit's setuid newuidmap/newgidmap
sub-uid mapping — proven by an on-cluster canary:
  newuidmap ... failed: operation not permitted
Relax to the documented moby/buildkit k8s rootless posture: privileged:false,
runAsUser/Group 1000, runAsNonRoot, seccomp+AppArmor Unconfined, and leave
allowPrivilegeEscalation / default caps at k8s defaults (newuidmap needs them).
Still user-namespaced, no host root — the decisive win over privileged=true.
Re-canaried: rootless build + scoped push-hanzoai cred pushed to ghcr OK.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-18 16:26:32 -07:00
zeekayandhanzo-dev c99117ace9 feat(cli,paas): unify apps/deploy/clusters on the LIVE Go cloud — one IAM login, org-scoped
`hanzo apps list`, `hanzo deploy`, `hanzo clusters` targeted the OLD TS-Dokploy
contract (/v1/apps, /v1/org/{org}/cluster, /v1/org/.../redeploy) — all 404 on the
live Go cloud (ghcr.io/hanzoai/cloud). Repoint the CLI at the endpoints the Go
cloud actually serves, authorized off the SAME IAM login `hanzo build` now uses
(no --platform-token). Drift confirmed live as z@hanzo.ai:
  /v1/apps            → 404      /v1/paas/apps         → 403 (was SuperAdmin-only)
  /v1/org/*/cluster   → 404      /v1/clusters          → 200 (already org-scoped)
                                 /v1/platform/projects → 500 (co-resident IAM off)

CLI (cli/platform.go, cli/commands.go):
  apps list/get   → GET /v1/paas/apps[/{app}]  (no client org filter — the board is
                    confined to the caller's org SERVER-side by the validated identity)
  deploy <app>    → POST /v1/paas/apps/{app}/deploy  (rolling restart; --env selects
                    the lifecycle namespace; org from identity, not the path)
  clusters list/get → GET /v1/clusters  (Visor-managed + BYO; org from identity)
Removed the TS-contract vestiges with NO Go backend: `apps sync` (the board is
live-computed), `clusters create/select/install-baseline/target` and `k8s target`
(DOKS provisioning + deploy-target selection are not implemented on the Go cloud).
Reshaped the Cluster DTO to the live visor clusterView (dropped dead Phase/Active/
operator/baseline fields). Platform client doc corrected: it CAN validate IAM tokens.

Backend (clients/paas): authorize the fleet board off ONE IAM identity, exactly like
/v1/runner (clients/platform/runner.go). guard now admits a validated principal who
is a SuperAdmin OR an OrgAdmin (principal.Validated + IsSuperAdmin || IsOrgAdmin —
the ONE verifier, unforgeable off-gateway), and each handler CONFINES a non-super
caller to the platform namespaces its own validated org owns (scopedNamespaces, keyed
on principal.Org — never a client header): a SuperAdmin sees the whole fleet, an
OrgAdmin only its own org (empty board / clean 404 otherwise), so a tenant admin can
never observe — or restart — another org's, or a platform, app. `?org=` cannot widen
the view (confinement is at the namespace scan, before the filter).

deploy is now a real zero-downtime ROLLING RESTART (the kubectl-rollout-restart
mechanism: stamp the pod-template hanzo.ai/restartedAt annotation) instead of the
409-refuse. It never changes the declared TAG (that stays a git commit, the one thing
Hanzo CD's selfHeal reconciles), so there is no drift to revert — the honest,
GitOps-compatible "redeploy this app". resolveTarget split so the machine release
path (release.go) keeps the full scan; the identity-scoped deploy/getApp use the
caller's authorized namespaces.

SECURITY: this broadens auth on a control-plane surface (new mutating restart path).
Mirrors blue's IAM-admin pattern; flag for red. TDD: 12 new paas cases (role gate,
tenant confinement on list/get/deploy, forged-org cannot widen, rolling-restart lands
the annotation, foreign-org 404 + no mutation) + the CLI path/DTO tests, all green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-18 16:23:03 -07:00
zeekayandhanzo-dev 20320d27a3 fix(platform): close RED H1/H2/M1 on the /v1/runner build path
RED re-review of the unify-infra PaaS-auth flip found 2 HIGH + 1 MED on the
privileged build endpoint. Fixes:

H1 (borders CRITICAL) — cross-org supply-chain push. imageAllowed() permitted
ghcr.io/{hanzoai,luxfi,zooai}/ regardless of the caller's validated org, so any
org-admin could overwrite another brand's prod image via the shared push cred.
Bind the image's registry-org to the caller's org (orgRegistryNamespaces map);
only a real SuperAdmin may cross. The machine (fabric) token keeps full
owned-registry latitude. Cross-org now 403; same-org 202.

H2 — privileged rootful buildkit in the main platform namespace with the shared
3-org push cred. buildJobSpec is now ROOTLESS (moby/buildkit:*-rootless, uid 1000,
no privileged, no privilege-escalation, caps dropped, --oci-worker-no-process-
sandbox), runs in a DEDICATED isolated namespace (CLOUD_PLATFORM_BUILD_NS default
→ hanzo-build, off the platform ns), and mounts ONLY the target org's push
credential (push-<namespace>), never the shared kaniko-ghcr. Node-pool taint +
automountServiceAccountToken:false retained.

M1 — `image` bypassed validateBuildInputs → buildkit --output attribute
injection (ghcr.io/hanzoai/x,registry.insecure=true). Added validateImageRef
(strict single OCI ref, rejects comma/space/quote/newline/'='), folded into
validateBuildInputs and enforced early at the handler.

I2 — the identity validator now fail-closes on an empty resolved issuer OR
audience set instead of silently disabling that axis; issuerAllowed denies on
an empty trusted set.

TDD: cross-org 403 + same-org 202 + SuperAdmin cross-org + orgless-403 +
image-injection-reject + rootless/scoped-cred spec + empty-trust-set-deny all
green (pure-Go, as prod ships).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-18 16:16:54 -07:00
hanzo-dev 23cce4635b fix(dns): close double-encoded traversal in the /v1/dns path guard
The prior prefix guard checked fasthttp's URI().Path(), which decodes only ONE
layer of percent-encoding. A DOUBLE-encoded traversal survives that one decode as
a literal %2e/%2f that still KEEPS the /v1/dns/ prefix -- so the prefix check
passes, cloud forwards base + /v1/dns/%2e%2e/admin, and the upstream decodes the
second layer to /v1/admin. Proven bypasses: /v1/dns/%252e%252e/admin,
/v1/dns/%252e%252e%252fadmin, /v1/dns/..%252fadmin.

After the prefix check, also refuse any once-decoded path that still carries a `%`
(a still-encoded byte => the client double-encoded) or `..` (residual traversal).
Neither appears in a legitimate DNS-API path -- zone labels are DNS names /
punycode xn--, and the query string (checked separately) is unaffected. Fail
closed 400 before a byte leaves cloud.

Also relay the upstream Location header so a 3xx -- never followed, per
CheckRedirect -- passes back verbatim (status + Location) as the comment claims,
rather than being silently dropped.

Regression: the escaped-path test gains the 3 double-encoded vectors (each refused
400 with 0 upstream bytes), plus a redirect test proving an upstream 302 is not
followed and its Location relays verbatim. 9 tests green.
2026-07-18 15:40:42 -07:00
hanzo-dev 9dfce8dc90 merge(cloud): per-org /v1/cloudflare asset plane
Adds the clients/cloudflare subsystem — Pages+Workers wired, R2/KV/D1 stubbed —
gated by the org-comingling guardrail and org-admin mutation check; wired into
apps/apps.go. Red-reviewed SHIP: comingling guardrail + org-admin mutation gate
verified PASS, 12/12 tests green, isolation core intact.
2026-07-18 15:29:47 -07:00
hanzo-dev fe34402e0b fix(dns): lock the /v1/dns forward head to its own prefix; don't follow upstream 3xx
The forward head built the upstream target from uri.Path(), which is NORMALIZED
and percent-decoded. Fiber matches the /v1/dns/* wildcard on the RAW path, so a
dot-segment or encoded-dot traversal (/v1/dns/../../admin/secrets,
/v1/dns/..%2f..%2fadmin, /v1/dns/../../../metrics) still routed to the handler
while the normalized path escaped the prefix -- letting the caller drive the
WHOLE path on the DNS host. Contained today only because the upstream 404s
unknown paths; a latent path-scope escape the moment :8443 serves anything else.

Guard the normalized path fail-closed BEFORE building the target: require it to
be exactly /v1/dns or under /v1/dns/, else 400 and forward nothing. Because the
path is already normalized, every traversal/encoded-dot escape fails this check.
Correct the comment that wrongly claimed the path was locked by the route match
(the host-pinning claim was, and stays, true).

Also stop the shared http.Client from following upstream 3xx (CheckRedirect =>
http.ErrUseLastResponse) so redirect responses pass through verbatim and a 3xx
can never silently re-target the request onto another host or path.

Regression test proves fail-closed: each escaped path is refused (400) and 0
bytes reach the upstream. Existing 7 tests stay green (8 total).
2026-07-18 15:29:39 -07:00
hanzo-dev e138fa60b5 fix(cloudflare): red fixes — comingling guardrail, org-admin mutation gate, stored-account resolution
Addresses Red's FIX-THEN-SHIP findings:
- HIGH: stamp X-Hanzo-Org (the served org) on every /v1/cloudflare response so a
  per-org caller can detect a pinned/comingled org. The platform Pages client asserts
  it equals the requested org and fails LOUD if a non-org-switch-capable service token
  made the identity boundary pin X-Org-Id to the token's own owner — no silent
  cross-tenant read/write.
- MEDIUM: gate mutations (POST/PUT/DELETE) on principal.IsOrgAdmin via a new authWrite
  front door (reads stay validated-org-only), parity with the AdminOnly connector. A
  non-admin is refused before any KMS token read and never reaches Cloudflare.
- LOW: resolveAccount now prefers the account captured at connect time
  (integrations.ConnectionFor ExternalID), falling back to live /accounts discovery
  only when none is stored — no per-call round-trip, deterministic for multi-account
  tokens.

Tests: +TestResponseStampsActingOrg, +TestMutationRequiresOrgAdmin,
+TestStoredAccountSkipsDiscovery; existing mutation tests drive as org admin. 12/12
pass, -race clean.
2026-07-18 15:28:34 -07:00
hanzo-dev f9b0b0ef64 feat(cloudflare): per-org /v1/cloudflare asset plane (Pages+Workers wired, R2/KV/D1 stubbed)
New cloud subsystem clients/cloudflare exposing /v1/cloudflare/{pages,workers,r2,kv,d1}/*,
sibling to hanzodns's /v1/dns. It reads each org's KMS-sealed Cloudflare token in-process
through the integrations custody seam (integrations.TokenFor) and proxies to the Cloudflare
API v4 with the cfDo shape reused verbatim from hanzodns — no global env token, no
bearer-relay hop (that is only hanzodns's separate-process need).

Tenant isolation: org is derived ONLY from the validated principal (principal.Org); the KMS
token path is keyed on that org, so cross-org token reach is structurally impossible and an
unvalidated request fails closed (403). Pages (project CRUD, deploy, custom-domain add/delete)
and Workers (script put/list/delete via multipart module upload, workers.dev subdomain, zone
route bind/list/delete) are wired; R2/KV/D1 ship typed provider methods + routes that answer
an honest 501 (never a fake success).

Appends the Workers connector scopes (Account:Workers Scripts:Edit, Zone:Workers Routes:Edit)
for the now-callable capabilities and wires the subsystem into apps.Wire after integrations.
2026-07-18 15:28:34 -07:00
hanzo-dev e6d93ee637 deploy: clean API paths — /v1/deploy/<resource>, no /api/ prefix, no inner /v1
House rule: no extraneous /api/, just /v1/. The projection API moves from
/v1/deploy/api/v1/* → /v1/deploy/<resource> (settings, session/userinfo, version,
account/can-i, applications, applications/{name}/resource-tree, .../{sync,rollback}).
This IS the deploy API now — the superseded native /v1/deploy/{applications,:name/*}
routes are removed (their readers stay, reused by the projection). health +
reconcile unchanged. Guard test updated.
2026-07-18 15:05:11 -07:00
hanzo-dev b84bfa1ca3 feat(dns): forward /v1/dns/* to the DNS control plane under the caller's own bearer
console.hanzo.ai serves the DnsModule but cloud held no /v1/dns head, so
console.hanzo.ai/v1/dns/* 404'd and the dashboard showed empty zones. Add a thin
forward head (clients/dns) that relays each /v1/dns/* request to the DNS control
plane (HANZO_DNS_URL, default the in-cluster coredns-hanzodns service), preserving
verb, path, query, body, status codes and error bodies.

Isolation is bearer-relay: the head forwards the caller's OWN validated bearer
(cloud.CallerBearer) plus the server-validated X-Org-Id, substituting NO service
credential, so the DNS plane's own per-org authorization still holds and a caller
in org A can reach only org A's zones. Fail-closed: no validated principal => 403,
before any byte leaves cloud. It builds a fresh upstream request, so no inbound
header is blindly relayed; the upstream host comes only from env (no SSRF).

Decomplect the token resolution the identity boundary and this relay both need
into one callerToken helper (validatedPrincipal now delegates to it) and expose
CallerBearer for the relay; an opaque API key is never relayed as a bearer.
2026-07-18 14:56:20 -07:00
hanzo-dev ec7b68441d merge(cloud): /v1/deploy is API-only — the FE moved to the hanzoai/spa cd-ui App
The monochrome dashboard now serves at cd.hanzo.ai/ (root, base-href /) from the
cd-ui App CR (hanzoai/spa); cloud keeps ONLY the IAM-gated projection API at
/v1/deploy/api/*. Drops the go:embed FE + the deploy-ui-embed Dockerfile stage.
The FE is no longer /v1/-prefixed and no longer baked into the money binary.
2026-07-18 14:26:49 -07:00
hanzo-dev 9102deae8d deploy: drop the FE from the money binary — /v1/deploy is API-only
The monochrome dashboard SPA now ships as the hanzoai/spa-based cd-ui App CR
served at cd.hanzo.ai/ (base-href /); cloud keeps ONLY the IAM-gated projection
API, moved from /v1/deploy/ui/api/* to /v1/deploy/api/* (same-origin with the
SPA). Removes the go:embed dashFS + static serve (dashStatic/serveDashIndex) +
webui/dist + the deploy-ui-embed Dockerfile COPY stage. Guard test updated to the
/v1/deploy/api/* routes (still 403 without SuperAdmin). RED invariant holds — the
API terminates in cloud behind IdentityMiddleware + the guard.
2026-07-18 14:15:07 -07:00
zeekayandhanzo-dev ff7c26c882 feat(cli): hanzo build owner/name shorthand → GitHub https URL
The platform build muscle (launchDirectBuild) clones an https git URL; the CLI
sent the bare `owner/name` positional verbatim, so `hanzo build luxfi/wallet`
failed server-side with "repo.url must use https". normalizeRepoURL expands a
bare owner/name to https://github.com/owner/name (the host for every
hanzoai/luxfi/zooai repo) and passes an explicit URL / scp-style remote through
untouched. Test: TestNormalizeRepoURL. Completes the IAM-login dogfood:
`hanzo build luxfi/wallet ... --sha <full> ...` now 202s + launches the build.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-18 13:51:21 -07:00
zeekayandhanzo-dev 6348531d7d sites: org-scope published host + CI guard against zen streaming regression
Publish routing is now org-scoped: a project publishes to
<slug>.<org>.<apex> (e.g. myapp.maxpower.hanzo.app) instead of the flat
global <slug>.<apex>. The slug namespace becomes per-org — two orgs can
own the same slug and their sites can never collide or shadow one another.

- deploy.go: siteHost(org,slug)=<slug>.<org> is the ONE bound/resolved key;
  onPublish binds it; siteURL renders https://<slug>.<org>.<apex>.
- sites.go siteSlug: accept the two-label host <slug>.<org>.<apex>, validate
  both labels (slug non-reserved), return <slug>.<org> as the resolve key so
  bind and resolve agree. Org isolation is now STRUCTURAL in the hostname.
- store unchanged: site_hosts already keys on arbitrary full host strings
  (custom-domain path proves exact full-host ResolveHost match).

containment.yml: add a required "zen streaming-fix floor" check that fails
any PR/push whose effective github.com/hanzoai/zen is below v1.4.1 (the
first release carrying the SSE body-close fix, commit 50328b8). A stale
branch that reverts go.mod's zen pin to v1.4.0 can no longer silently
re-break streaming (empty SSE completions) — the durable root-cause guard.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-18 13:49:05 -07:00
hanzo-dev 27a627bf35 feat(admin): thin audited credit-grant relay at POST /v1/admin/credit-grants
The one admin mint surface. SuperAdmin-only (core.Guard); forwards verbatim to
commerce's already-mint-gated POST /v1/billing/credit-grants (middleware.Mint →
PlatformOnly) via COMMERCE_SERVICE_TOKEN, scoped to the target org, and writes one
tamper-evident audit record. Commerce stays the sole credit-grant ledger — no
in-process mint. NOT deployed; for red review (mint surface).

Assisted-by: neo:claude-opus-4-8
2026-07-18 13:43:34 -07:00
hanzo-dev f6ab219ba1 feat(admin): thin audited credit-grant relay at POST /v1/admin/credit-grants
The one admin mint surface. SuperAdmin-only (core.Guard); forwards verbatim to
commerce's already-mint-gated POST /v1/billing/credit-grants (middleware.Mint →
PlatformOnly) via COMMERCE_SERVICE_TOKEN, scoped to the target org, and writes one
tamper-evident audit record. Commerce stays the sole credit-grant ledger — no
in-process mint. NOT deployed; for red review (mint surface).
2026-07-18 13:43:34 -07:00
zeekayandhanzo-dev 0d79eeb1e0 feat(cli,platform): unify PaaS auth on IAM — one login authorizes build/deploy/apps
A plain `hanzo login` (IAM) now authorizes every PaaS control-plane op with no
separate --build-token / --platform-token. ONE identity, org+role scoped.

CLI (cli/cli.go): Env.buildToken() and Env.platformToken() fall back to the IAM
access token as the FINAL resort (precedence unchanged above it: flag > env >
credential-store service token > IAM login). So after `hanzo login` the CLI sends
the IAM JWT as the platform bearer. "No token" errors now point at `hanzo login`
and fire only when there is ALSO no IAM login. Tests: added
TestBuildTokenFallsBackToIAM / TestPlatformTokenFallsBackToIAM (precedence
preserved — a dedicated token still wins).

Platform (clients/platform/runner.go): /v1/runner (build-enqueue) — the one
control-plane endpoint that ignored identity — now accepts EITHER the shared
build-callback token (machine path: git-push, self-release, operator; constant-time,
unchanged) OR a validated IAM principal who is an admin (principal.IsSuperAdmin ||
principal.IsOrgAdmin over principal.Validated). Both bounded by the SAME
owned-registry allowlist, so identity never widens the image boundary. Release
self-publish stays machine-token-only. IAM builds are org-attributed to the caller's
VALIDATED org and refuse a foreign organizationId unless SuperAdmin. Reuses the ONE
identity verifier (SanitizeIdentity mints unforgeable X-User-* from the verified JWT)
— no parallel JWT crypto. deploy/apps were already IAM-authorized via the tenant()
boundary. Tests: 7 new IAM cases (admin launches, non-admin 403, forged-no-user 403,
disallowed-image 403, foreign-org 403, release 403) + corrected fail-closed 403.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-18 13:22:37 -07:00
zeekayandhanzo-dev 783e16ebd0 fix(cloud): re-pin zen v1.4.2 — restore the SSE stream body-close fix
The argo/gitops merge db0957b (v1.801.77) resolved the go.mod conflict to its
stale second parent (zen v1.4.0), silently reverting the v1.4.1 bump landed at
v1.801.76. v1.4.0 still carries the serve() `defer resp.Body.Close()` race that
empties every streamed completion (200 with a 0-byte body) — which broke every
hanzo.app builder stream. Re-pin to zen v1.4.2 (the body-close fix + reasoning_content
passthrough regression pin) and thinking v0.1.1 (reasoning default). Streaming
now forwards every chunk, including delta.reasoning_content.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-18 12:55:21 -07:00
hanzo-dev a404cb1f4a merge(cloud): embed the monochrome ArgoCD UI (ghcr.io/hanzoai/deploy-ui-embed) into /v1/deploy/ui
The deploy-ui-embed image is published; cloud's Dockerfile now COPYs its /dist
into clients/deploy/webui/dist (go:embed). release.yml builds a cloud image that
serves the REAL monochrome dashboard at /v1/deploy/ui instead of the fallback.
2026-07-18 12:46:52 -07:00
hanzo-dev 4c0be6271c cloud: embed the monochrome ArgoCD UI bundle (ghcr.io/hanzoai/deploy-ui-embed) into clients/deploy/webui/dist
Mirrors the console-embed stage: FROM the prebuilt deploy-ui-embed image, COPY
/dist -> clients/deploy/webui/dist (go:embed source for /v1/deploy/ui). GATE: do
NOT merge until ghcr.io/hanzoai/deploy-ui-embed:latest is published (else the
build cannot pull the base). Until merged, the money binary serves the fallback.
2026-07-18 12:43:17 -07:00
hanzo-dev 34d97820b3 merge(cloud): deploy dashboard RED fast-follows (guard test + health hardening) 2026-07-18 12:40:51 -07:00
hanzo-dev a8443e1f2a deploy: RED fast-follows — guard table-test (LOW-2) + drop raw error from public health (INFO-1)
LOW-2: TestDeployRoutesRequireAdmin asserts every /v1/deploy(/ui) route 403s
without X-User-IsAdmin and passes with it (health stays public) — guards against
a future unguarded-route refactor.
INFO-1: the unauthenticated /v1/deploy/health path reports booleans only; the raw
k8s error (apiserver/RBAC detail) is logged server-side, not returned.
LOW-1 (CSRF): verified no-op — the IAM session cookie is SameSite=Lax AND the
ambient cookie->JWT bridge is same-origin-gated (sessionBridgeSameOrigin), so a
cross-origin CSRF POST gets no identity and the deploy guard 403s.
2026-07-18 12:40:50 -07:00
hanzo-dev e1e7315457 merge(cloud): ArgoCD monochrome dashboard via App-CR projection at /v1/deploy/ui
Serves the full ArgoCD React UI fed a read-projection of operator App CRs shaped
as v1alpha1 Applications — no argocd api-server/repo-server/redis/stored CRD.
SuperAdmin-gated; IAM owns identity at the edge. Money binary builds green;
projection render tests pass. Real monochrome bundle is a CI artifact (make
deploy-ui / deploy-ui-embed image); fallback shell until that lands.
2026-07-18 12:21:56 -07:00
zandGitHub 32d7c32aea docs(llm): document /v1/deploy GitOps plane (quality pass) 2026-07-18 12:20:20 -07:00
hanzo-dev c196cb9c4a docs(llm): document the /v1/deploy GitOps plane + embedded gitops-engine 2026-07-18 12:20:06 -07:00
hanzo-dev d23f8342c9 deploy: make deploy-ui builds the monochrome bundle into webui/dist (gitignored)
CI story for the dashboard bundle, mirroring make webui: DEPLOY_DIR=<hanzoai/deploy
rebrand/hanzo-monochrome> yarn build -> clients/deploy/webui/dist (go:embed). Only
the fallback index.html + .gitignore are tracked; the real 43MB bundle is
build-time-only. Money binary builds green with the real bundle embedded.
2026-07-18 12:19:40 -07:00
hanzo-dev a58def8a15 deploy: ArgoCD monochrome dashboard via App-CR projection at /v1/deploy/ui
Serves the full ArgoCD React UI fed a READ-PROJECTION of operator App CRs shaped
as v1alpha1 Applications — NO argocd api-server, NO repo-server, NO redis, NO
stored Application/AppProject CRD. projection.go maps App CR -> Application +
resource-tree (reusing the native readers/engine health). dashboard.go
reimplements the UI's api-server subset (settings/userinfo/version/can-i +
applications list/get/resource-tree + sync/rollback->App-CR reconcile) + serves
the go:embed'd monochrome bundle with base-href rewrite. SuperAdmin-gated; argocd
auth disabled (IAM owns identity at the edge). Projection render tests green.
UI bundle is a CI artifact (committed fallback shell; make deploy-ui overwrites).
2026-07-18 12:19:40 -07:00
hanzo-dev b831d76649 chore(cloud): vendor hanzoai/ai v1.824.2 — real model names for super-admin platform view
v1.824.2 unmasks the router-stats model ids (arm-N → real names like zen5-coder /
opus-4.8) for the PLATFORM scope when the caller is a super-admin of the own brand;
every other caller keeps the arm-N privacy masking. So the world.hanzo.ai admin view
(Routing Throughput / Enso arms) shows actual models instead of "Enso arm 2".
2026-07-18 12:15:04 -07:00
hanzo-dev ffc01f28ea fix(cloud/visor): fleet surfaces DO droplets via registry+live-DO union
world's admin fleet (GET /v1/machines -> listMachines) sourced machines only
from Visor's registry (/v1/get-machines), so DigitalOcean droplets that were
provisioned but not (yet) in the registry never appeared -- DO nodes were
entirely missing from the fleet.

Source the managed-machine set as the deduped UNION of the registry AND
Visor's LIVE DO reseller list (GET /v1/machines -> ListComputeMachines ->
service.ListOrgMachines, the live Droplets.ListByTag(orgTag)). Dedup is by
provider id OR name; the registry entry wins a collision so its enrichment/
masking is preserved. One helper (managedMachines) now feeds listMachines,
listGPUs and the /v1/fleet board so all three agree on which machines exist,
not just how they normalize. BYO fold unchanged; only machines Visor actually
returns are surfaced (nothing fabricated).
2026-07-18 12:13:48 -07:00
z fac09ffaa6 billing: metering client honors METERING_TEST (safe test-mode canary/staging)
buildMeteringClient ignored the documented METERING_TEST env, so the metering client
was ALWAYS live (c.test=false) — a staging/canary could not route debits to the
sandbox books, and the usage-cap smoke would have moved real money. Now METERING_TEST=true
sets Config.Test, so fin.RecordUsage writes the TEST finance books and the cap read
(org.TestMode via SQUARE_ENVIRONMENT=sandbox) sees the SAME test books. Unset in prod
= live, unchanged.
2026-07-18 12:13:31 -07:00
z 3e61401ebf cap: enforce + alert on the FINANCE ledger (where the unified binary records usage)
The spend cap read commerce's transaction store, which the co-resident cloud binary
leaves EMPTY (usage is recorded via fin.RecordUsage on the finance ledger) — so in
prod the cap summed 0 and never enforced, and the alert never fired. This wires the
cap onto the ledger prod actually writes, ORG-WIDE (the finance Entry carries no
scope; per-scope is a follow-up):

- sqlstore.SumByKindSince: additive read-only aggregate (kind + created_at index,
  18-decimal TEXT folded in Go) — no Entry schema change.
- finance.SumUsageSince: the org's metered usage (cents) since a cutoff, deposits
  excluded, sandbox books for a test org.
- finance.SetUsageHook: dependency-inverted post-debit seam (finance never imports
  commerce) the cap alert fires through.
- apps/commerce.go: SetPeriodSpendReader(financePeriodSpend) so AuthorizeSpendCap
  reads finance spend since the UTC month start, and SetUsageHook(fireCapAlert) so a
  finance debit fires the org's spend-alerts on the same crossing.

Composes the commerce policy/CRUD/promo/admin/ancestor-fix (commerce
v1.49.1->v1.49.2 injection seam) — a targeted host re-wire, not a redo.
2026-07-18 12:07:10 -07:00
hanzo-dev 8d84734984 deps: commerce v1.49.0->v1.49.1 — real subscription tier derivation
Completes the Enso per-tier gate: ai v1.824.1 already enforces min_tier at the
family pipe + auto-router; this bumps the co-resident commerce so /v1/billing/tier
returns the caller's REAL plan (was stubbed always-Free). Fail-open on uncertainty.
2026-07-18 11:43:21 -07:00
zandGitHub db0957b2b9 merge(cloud): embed argo gitops-engine under /v1/deploy — reconcile + RED HIGH-1 prune fuse (inert: DEPLOY_ENGINE_ENABLED off) 2026-07-18 11:27:44 -07:00
hanzo-dev 8f11c84800 chore(deploy): go mod tidy after rebase onto main (union: main deps + gitops-engine v0.7.2 + k8s 0.35.3 staging) 2026-07-18 11:27:25 -07:00
hanzo-dev 8cb8f6625f chore(cloud): vendor hanzoai/ai v1.824.1 — Enso flywheel boots from Mount
v1.824.1 boots StartRouterTrainer + StartRouterProbe from ai.Mount, so the flywheel
runs in the deployed (embedded-in-cloud) service, not just the standalone aid binary.
Both still self-gate on their env flags; universe sets ROUTER_TRAIN_ENABLED=1 to turn
training on. Also carries the retrain-timeline fix (retrains now count).
2026-07-18 11:27:15 -07:00
hanzo-dev 51f0c1cbd4 deploy: prune-safety fuse (RED HIGH-1) on the engine reconcile
Five guards before any deletion: (i) refuse an empty desired set; (ii) dry-run
sizes the prune set + a count/ratio fuse (DEPLOY_ENGINE_PRUNE_MAX default 10,
_RATIO default 0.20) refuses a mass prune; (iii) WithPruneConfirmed gates prune
on the fuse passing; (iv) PVC + KMSSecret are excluded from prune entirely (data
anchors, irreversible); (v) parseManifestDir walks recursively so a nested
manifest is never silently dropped (which prune would read as a deletion).
prune stays off by default (DEPLOY_ENGINE_PRUNE).
2026-07-18 11:26:02 -07:00
hanzo-dev 0c5dbcaaac deploy: pin gitops-engine to hanzoai/deploy/gitops-engine v0.7.2 (no replace)
Drops the filesystem replace => ../deploy/gitops-engine. The fork's engine module
was renamed to its real repo path (github.com/hanzoai/deploy/gitops-engine, tag
gitops-engine/v0.7.2) so cloud requires it as a normal pinned version — CI builds
the money binary with NO sibling checkout, NO argoproj alias. tidy + scoped build
green over SSH.
2026-07-18 11:26:02 -07:00
hanzo-dev 1180fe77a1 deploy: embed argo gitops-engine in-process under /v1/deploy (reconcile half) 2026-07-18 11:26:02 -07:00
hanzo-dev 7c02008ade ci(release): auto-promote the proven tag into universe crs/cloud.yaml
Every merge to main builds + smoke-tests + tags a proven image, but nothing
recorded that tag as the desired state Hanzo CD deploys, so api.hanzo.ai sat on
a stale pin (v1.801.71) while proven images (…72-…75) never rolled. The old
image-update.yml deploy hub was deleted in the Hanzo CD cutover; a direct CR
patch is reverted by ArgoCD selfHeal.

Add a promote job that, after the tag receipt, bumps spec.image.tag in
hanzoai/universe crs/cloud.yaml and commits deploy(cloud): <tag> — the SAME
yq-bump the hanzoai/ci reusable does for every other service. The universe-crs
ArgoCD Application (automated sync + selfHeal) then reconciles it to the cluster.
No hand-dispatch, no hand-edit.
2026-07-18 11:02:47 -07:00
d9bfa032ea fix(cloud): zen v1.4.1 (SSE body-close race) + thinking v0.1.1 (reasoning default)
Fixes empty streaming completions for zen5* (hanzo.app builder P0):
- zen v1.4.1 stops serve() from closing the upstream body before fiber's lazy
  SendStreamWriter drains it (every SSE completion was truncated to a 200 + empty body).
- thinking v0.1.1 makes glm-5.2/deepseek Off send reasoning_effort:none, so zen5-coder
  streams the answer immediately instead of a long silent content:null reasoning preamble.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-18 10:49:51 -07:00
z e2acfd2a8f admin+metering: SuperAdmin usage-cap + promo control plane; fix net/http spend_cap→402
Adds the /v1/admin control plane admin.hanzo.ai drives, twinning /v1/admin/flags:
  - /v1/admin/promos (GET/PUT, core.Guard SuperAdmin) → commerce /v1/platform/promo:
    configure the admin-controlled plan promo (percentOff/start/end/plans/active).
  - /v1/admin/spend-caps (GET/POST/PATCH/DELETE, core.GuardScoped) → commerce
    /v1/billing/spend-alerts with X-Org-Id: oversee/override ANY org’s usage caps
    (SuperAdmin via ?org=; a scoped admin hard-pinned to their own — the escalation
    line). Reuses the customer’s OWN spend-alert rows, no parallel model.
commerce.Forward is the ONE service-token seam these ride, relaying commerce’s own
status so a 400/403/404 surfaces honestly instead of masking as success.

Fixes clients/metering/middleware.go defaultOnDenied: a FUNDED caller over a
per-scope spend cap now maps to a DISTINCT 402 spend_cap_exceeded (errors.Is), not
the 503 it fell through to — parity with the zip-native denyVerdict, so any product
on the net/http middleware surfaces the same honest verdict.
2026-07-18 09:56:58 -07:00
hanzo-dev 4bc9f99f3d build(deps): bump embedded luxfi/kms v1.11.8 -> v1.12.4
Brings the ACTIVE Hanzo KMS custody plane (api.hanzo.ai/v1/kms/*, served by the
cloud-embedded luxfi/kms per HIP-0106) to the latest v1.x. keys (v1.4.1) and
crypto (v1.20.2) already latest; luxfi/mpc stays out of the graph (threshold
signing is a wire-coupled external daemon, not a linked module). v1.12.4 verifies
against the public sumdb; clients/kms + cmd/cloud build green.
2026-07-18 09:52:56 -07:00
z 88f27d7305 fix(cloud/fleet): never surface a GPU slug's VRAM as system RAM
toMachineView's memGB fallback read the integer before "gb" out of any
size slug. On a DO GPU droplet the slug's gb is VRAM (gpu-h100x8-640gb ->
640 GB VRAM), not system RAM, so a GPU node missing its upstream memSize
would render 640 GB of "system memory" -- a misleading number.

Guard the fallback with the GPU check already needed for v.GPU: reuse the
single gpuSpecOf(slug) call (spec, isGpu) and apply the slug's gb figure
only when !isGpu. Real m.MemSize still takes precedence for every provider,
GPU nodes included, so a GPU machine that reports its true RAM is
unaffected -- only the VRAM-as-RAM fallback is suppressed.

Table tests: a gpu-h100x8-640gb slug with empty MemSize yields Mem=="" (not
"640 GB") while still resolving GPU=="H100", and the same slug with a real
MemSize=="1920gb" reports "1920 GB".
2026-07-18 09:43:47 -07:00
z 618eafc2bd fix(cloud/fleet): map system memory + parse DO size-slug vCPU/RAM
The fleet view (world.hanzo.ai cloud variant) renders machines from
cloud's /v1/machines -> listMachines -> toMachineView. Two honest-data
gaps left system RAM and DigitalOcean vCPU counts blank:

1. System memory was never surfaced: machineView had no memory field and
   toMachineView never read the upstream memSize, so every provider's RAM
   column rendered empty.
2. DO vCPU was dropped: toMachineView filled vcpu only when CpuSize parsed
   as a bare integer, but DigitalOcean reports size SLUGS (s-4vcpu-8gb),
   so strconv.Atoi failed and vCPU showed nothing.

Fix:
- Add MemSize to visorMachine (upstream already sends it; it was simply
  unmapped) and Mem to machineView.
- parseSizeSlug pulls the integer before "vcpu" and the integer before
  "gb" out of a size slug (s-4vcpu-8gb -> 4,8; g-8vcpu-32gb -> 8,32).
- normalizeMem renders "N GB" only for trustworthy inputs (explicit
  gb/gib, explicit mb converted with rounding, a bare integer as MB when
  >=1024 else GB) and returns "" for anything ambiguous -- never a
  fabricated number.
- toMachineView keeps the Atoi(CpuSize) path and falls back to the slug's
  vcpu; sets Mem from normalizeMem(MemSize) and falls back to the slug's
  gb figure. The GPU-spec logic is unchanged.

Table tests cover the slug parser, the mem rounding, and the mapper
precedence (explicit values win, honest omission when neither yields one).
2026-07-18 09:37:45 -07:00
zandGitHub cf216f193f Merge feat/code-normalized-continue into main 2026-07-18 09:30:51 -07:00
antje d8c258e94d gpu worker: collect SaveGLB outputs — 3D meshes travel back to the library
collectOutputs read only the 'images' key, but SaveGLB publishes under '3d', so a
generated .glb never mirrored to the org library. Gather every saver's outputs
(images + 3d), so the studio 3D lane's mesh lands like an image or video render.
2026-07-18 09:27:56 -07:00
hanzo-dev c6c28fd9c3 feat(code): normalize continue across harnesses 2026-07-18 09:26:22 -07:00
hanzo-dev be4ae6a921 integrations: Cloudflare OAuth connect path alongside apikey (same KMS coordinate)
The cloudflare provider now offers a browser OAuth path in addition to the
shipped apikey path. /connect dispatches by request: a "token" key in the body
seals via apikey (verify-before-store, unchanged); its absence starts the
Authorization Code flow (confidential client, client_secret, no PKCE — the
framework's OAuth pattern) and the exchanged access token is sealed to the SAME
KMS coordinate (/orgs/{org}/integrations/cloudflare/api_token), so the DNS
provider layer is auth-method-agnostic.

Framework: connect dispatch is now capability-based (Verify and/or Authorize)
rather than Kind-only; Mount validates RedirectPath for any OAuth-capable
provider; bodyHasCredential picks the path. The OAuth leg gates on its own app
creds (Creds().ClientID) so a missing Cloudflare OAuth app degrades to an honest
503 without breaking the always-available apikey path.

Requires a registered Cloudflare OAuth app: CLOUDFLARE_OAUTH_CLIENT_ID/SECRET in
env, redirect https://api.hanzo.ai/v1/integrations/cloudflare/callback.
2026-07-18 09:14:07 -07:00
zeekayandhanzo-dev 998685c4fb rebuild(cloud): embed console-embed@sha-bd8a816 (casibase auth /v1/* fix + console per-project resources) + activate MCP builtin tool-plane (#292)
No Go change — this rebuild re-resolves the freshly-republished console-embed:latest
(console main bd8a81651) into the go:embed console served at console.hanzo.ai, and
ships builtin.go's auto '/v1 route → MCP tool' plane at /v1/tools/mcp (already in main,
newer than the deployed v1.801.69).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-18 08:57:15 -07:00
hanzo-dev ceba27fbbf feat(connectors): OAuth/apikey connector plane (Anthropic, OpenAI, Copilot, device flow, refresh)
Preserve divergent connector work: verify-before-store connector types
(Anthropic, OpenAI, Copilot), device-authorization flow, token refresh,
connector registry/store, and integration/store/link plumbing.
2026-07-18 01:26:55 -07:00
hanzo-dev c3e02036ba feat(connectors): OAuth/apikey connector plane (Anthropic, OpenAI, Copilot, device flow, refresh)
Preserve divergent connector work: verify-before-store connector types
(Anthropic, OpenAI, Copilot), device-authorization flow, token refresh,
connector registry/store, and integration/store/link plumbing.
2026-07-18 01:26:55 -07:00
hanzo-dev 4cdd2f2558 feat(channels): unified inbound channel ingest plane (envelope, pairing, policy, per-platform adapters)
Preserve divergent channel-ingest work: clients/channels package (envelope
normalization, pairing, delivery policy, Slack/Discord/Teams/Telegram
adapters, store), cmd/channels entrypoint, integration event-emit hooks,
and apps wiring.
2026-07-18 01:26:09 -07:00
hanzo-dev 2b6dbddf0c feat(channels): unified inbound channel ingest plane (envelope, pairing, policy, per-platform adapters)
Preserve divergent channel-ingest work: clients/channels package (envelope
normalization, pairing, delivery policy, Slack/Discord/Teams/Telegram
adapters, store), cmd/channels entrypoint, integration event-emit hooks,
and apps wiring.
2026-07-18 01:26:09 -07:00
hanzo-dev 6d42382aed fix(kms): unshadow the bare secrets-list route (/secrets/+ not /secrets/*)
The value routes registered the optional-greedy wildcard `/secrets/*`, which
fiber also matches with an empty tail — so the bare `GET .../secrets` list path
was answered by getSecret (400 "secret name is required") and listSecrets was
unreachable. Switch the getSecret/deleteSecret value routes to the required-
greedy `+` (one-or-more), so `/secrets` falls through to the exact list route
while `/secrets/<path>/<name>` still reads/deletes. reqWildcard reads the `+`
param.

Regression test list_route_test.go asserts the bare list path returns 200 with
a secrets array (was 400) and that value reads still work.
2026-07-18 01:19:23 -07:00
hanzo-dev 57b6ac109b integrations: Cloudflare apikey connector (verify-before-seal, org-admin gated)
Register Cloudflare as an apikey-kind connector on the /v1/integrations plane.
A customer-supplied scoped API token is verified live against Cloudflare's
GET /user/tokens/verify (must be status:active) before it is sealed into the
org's KMS namespace (/orgs/{org}/integrations/cloudflare/api_token); the
connection row holds only non-secret account metadata. connect/verify/disconnect
are org-admin gated from the validated principal (principal.IsOrgAdmin).

Extends the connector framework with the apikey credential seam shared by future
customer-credential providers: Provider.Kind/AdminOnly/Verify, VerifyInput,
connectByCredential (verify-before-seal, fail-closed), and the verify route.

Serves POST /v1/integrations/cloudflare/{connect,verify,disconnect} and
GET /v1/integrations.
2026-07-18 00:07:53 -07:00
z 305e7dd8d3 Merge feat/route-work-to-target: route coding run to chosen target machine 2026-07-17 23:32:54 -07:00
hanzo-dev 4fe61262a9 coding: verify + PR + close the session when a routed run completes
A routed run's machine pushes with its own credential and streams into the
session, but cloud still owns the completion — the integrity gate, the PR row, and
the session's terminal state (the machine never closes the session, so it was
staying "running" forever). Give a routed run the SAME cloud-side completion the
local keystone path runs after a sandbox push.

- completeChanged: the shared terminal for a run that reported changes — VerifyRef
  the pushed branch LANDED (fail-closed to a session error + no PR if absent), file
  the native PR, mirror done, close the session done. The local path (Run) now calls
  it too, so the two paths cannot drift.
- finalizeRouted: maps a machine's terminal report onto that completion — reported
  failure closes the session error (no PR), no-changes closes done (no PR), a changed
  push runs completeChanged. No secret crosses; cloud only reads the ref it can see.
- DeliverRoutedRunActivity runs the completion once, after a real report, on a
  cancel-immune bounded context, so a completed run is never re-executed by a retry.
  The completion seam is injected at the composition root (NewDispatcher), the same
  injected-seam shape index_on_push uses, so the free-function activity reaches the
  dispatcher's git/tracker/session seams without a global Dispatcher.
- RoutedRun carries Actor + AgentRef (cloud-side only, never sent to the machine) so
  the completion attributes the session close and files the PR with the right
  assignee.

Also document the mailbox's single-replica dependency at its definition (accepted,
inherited from cloud's KMS-lock replicas:1) with a future replica-aware note.

Tests: routed changed+verify -> PR filed + session done; verify fails -> no PR +
session error; no changes -> done no PR; reported error -> error no PR (verify never
runs); NewDispatcher wires the seam; the durable type bridge preserves attribution.
2026-07-17 23:13:07 -07:00
hanzo-dev a1074d0691 coding: verify + PR + close the session when a routed run completes
A routed run's machine pushes with its own credential and streams into the
session, but cloud still owns the completion — the integrity gate, the PR row, and
the session's terminal state (the machine never closes the session, so it was
staying "running" forever). Give a routed run the SAME cloud-side completion the
local keystone path runs after a sandbox push.

- completeChanged: the shared terminal for a run that reported changes — VerifyRef
  the pushed branch LANDED (fail-closed to a session error + no PR if absent), file
  the native PR, mirror done, close the session done. The local path (Run) now calls
  it too, so the two paths cannot drift.
- finalizeRouted: maps a machine's terminal report onto that completion — reported
  failure closes the session error (no PR), no-changes closes done (no PR), a changed
  push runs completeChanged. No secret crosses; cloud only reads the ref it can see.
- DeliverRoutedRunActivity runs the completion once, after a real report, on a
  cancel-immune bounded context, so a completed run is never re-executed by a retry.
  The completion seam is injected at the composition root (NewDispatcher), the same
  injected-seam shape index_on_push uses, so the free-function activity reaches the
  dispatcher's git/tracker/session seams without a global Dispatcher.
- RoutedRun carries Actor + AgentRef (cloud-side only, never sent to the machine) so
  the completion attributes the session close and files the PR with the right
  assignee.

Also document the mailbox's single-replica dependency at its definition (accepted,
inherited from cloud's KMS-lock replicas:1) with a future replica-aware note.

Tests: routed changed+verify -> PR filed + session done; verify fails -> no PR +
session error; no changes -> done no PR; reported error -> error no PR (verify never
runs); NewDispatcher wires the seam; the durable type bridge preserves attribution.
2026-07-17 23:13:07 -07:00
z 9922104692 Merge feat/kms-reseal-migration: CR-driven KMS re-seal migration tool (#79) 2026-07-17 23:10:37 -07:00
a44aaffbed auth: accept admin-console audience in the cloud JWT allowlist (#332)
The cloud already trusts hanzo-admin-guard (the admin surface) but not admin-console
(the admin console's own OIDC client), so a SuperAdmin token minted via admin-console
was rejected on /v1/admin with 'invalid audience' — forcing an awkward hanzo-admin-guard
detour. Add admin-console so the admin console's tokens work directly, matching
GATEWAY_ALLOWED_AUDIENCES which already lists it.

Co-authored-by: zeekay <z@hanzo.ai>
2026-07-17 23:03:19 -07:00
hanzo-dev 2936317959 docs: open cloud planes blueprint (HIP-0129)
Plane map with honest tiers in LLM.md: /v1/connectors custody (in flight), /v1/channels transport (planned, branch reserved), shipped planes named by package. Spec home HIP-0129; roadmap P1-P15 lives there.
2026-07-17 22:49:44 -07:00
zeekayandz d1f7667486 gateway: nest clients/gatewaypolicy → clients/gateway/edge (kill the compound)
"gatewaypolicy" is a compound (gateway+policy) and read as a second gateway
package. It is the ONE Gateway concern with clients/gateway — the per-org edge
policy STORE (OrgRPM ceiling + CORS + cache) that the /v1/gateway/config plane and
the package-cloud edge middleware both read.

They are two packages only to break a Go import cycle: clients/gateway imports root
cloud (cloud.Deps), and middleware_edge.go IS package cloud — so the store must be a
LEAF both can import. A flat merge cycles. Fix per the no-compound law: nest the leaf
UNDER gateway as clients/gateway/edge (edge.Policy/Store/New). One gateway namespace;
/v1/gateway/config surface unchanged; edge-middleware logic unchanged.

NOT redundant with the external hanzoai/gateway (KrakenD): that does coarse per-route
edge rate-limit + auth at ingress; this is per-AUTHENTICATED-org RPM (needs the decoded
token org), an app-level ceiling the edge proxy cannot compute. Different layer.

Pure rename (49/49, no logic change); full cmd/cloud binary links; gateway + gateway/edge
tests pass; gofmt/vet clean.
2026-07-17 22:47:51 -07:00
ee2cf3a11b flags: runtime flags resolve from /v1/flags only — drop redundant env gates (#331)
waitlist_* / public_signup / gateway_* are runtime flags; strip their Env: fallbacks so
they resolve from the /v1/flags DB engine → Default (single source of truth, flipped live,
no redeploy). Boot-time ReadOnly rows (subsystem_*, network_id_*) keep Env — that IS their
boot mechanism. Nothing read these env vars outside the flags engine (verified).

Co-authored-by: zeekay <z@hanzo.ai>
2026-07-17 22:46:51 -07:00
hanzo-dev 55fcaea0b7 agents/integrations: reach routed dispatch from the Slack trigger
Wire the load-bearing trigger so a coding run can be dispatched to a chosen
linked machine end-to-end. Extend the Slack coding grammar with an optional
routing prefix — code: <repo> on <machine> <task> — resolve <machine> org-scoped
(id or friendly label) and set Req.TargetID. An unknown or foreign machine is an
honest error, never a silent local fallback; an untargeted request is byte-
unchanged (repo <task>).

- agents.ResolveTarget: the ONE org-scoped id-or-label resolver, fail-closed, so a
  trigger surface turns 'on evo' into a target id without leaking another tenant's
  inventory.
- slack_coding: parseCoding yields (repo, target, task); a routed run skips the KMS
  agent-credential fetch (the machine authenticates with its own credential); the
  ack + result card report a routed run as queued-on-<machine>, followed live in
  mission-control, not a premature branch-pushed verdict.

Tests: ResolveTarget id/label precedence + cross-org not-found + unmounted fail-
closed; parseCoding on-prefix grammar (routing only when 'on' is the token after
the repo; 'on-call'/'only' untouched); routed result card is queued not done.
2026-07-17 22:42:59 -07:00
z 2aac986043 Merge fix/consensus-bump: luxfi/consensus v1.36.9 (unbreak force-moved checksum) 2026-07-17 22:33:25 -07:00
zeekayandhanzo-dev 141e417742 build(deps): bump ai → v1.822.3 — drop the last WqyJh audio-fork edges
cloud transitively pulled github.com/WqyJh/{go-cosyvoice,go-openai-realtime}
through ai v1.822.2 (the go-openai-fork release, which predated ai's TTS
switch to the hanzo-owned forks). ai v1.822.3 wires ai/tts onto
github.com/hanzoai/go-cosyvoice + go-openai-realtime, so tidy drops both WqyJh
edges from cloud's graph. cloud now pulls ZERO third-party OpenAI-lineage:
WqyJh 0, sashabaranov 0, ClickHouse 0. Single datastore sql registrant
(hanzo-ds/go). Builds clean.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-17 22:29:42 -07:00
hanzo-blue 173ec3a0a8 docs(kmsreseal): runbook SCOPE = exact per-host dry-run counts (#79) 2026-07-17 22:26:48 -07:00
hanzo-blue 4d63dde788 docs(kmsreseal): operator LLM.md — subcommands, dry-run findings, gated cutover (#79)
4-host map, G1 delta (78-union), G5 boot-cycle verdict (safe: direct master key),
seeding wedge-risk prerequisite. Dry-run only; cutover CTO-gated.
2026-07-17 22:25:33 -07:00
z 1d2e93a45e feat(admin): fleet-aggregate billing endpoints (metrics + invoices + subscriptions)
admin.hanzo.ai's SaaS-metrics/Invoices/Subscriptions pages were placeholders
awaiting cross-org /v1/admin/* endpoints. Add them as super-admin (core.Guard)
fleet-aggregate readers over the existing commerce billing engine:

  GET /v1/admin/metrics        — SaaS god-view: MRR/ARR/net-new/churn/active-subs/
                                 paying-customers/plan-mix/top-customers/recent.
                                 Single S2S proxy — commerce /v1/metrics/saas is
                                 already a cross-org aggregate (same gate finance
                                 Costs uses: RequirePlatformAdmin→IsServiceToken).
  GET /v1/admin/invoices       — cross-org invoice list; fan core.ListOrgs out
  GET /v1/admin/subscriptions  — cross-org subscription list; per-tenant reads
                                 merged (identical to revenue.go's fan-out).

Honest degradation: a failed per-org read contributes no rows, never fabricated.
go build ./clients/admin/... green, gofmt clean. Pairs with admin operator UI
(feat/admin-billing-fleet-ui).
2026-07-17 22:23:47 -07:00
zeekayandhanzo-dev 2c6f900aa2 refactor(go-openai): import the hanzoai/go-openai fork directly, drop the replace
cloud/clients{,/agent} used sashabaranov/go-openai only via a 'replace =>
hanzoai/go-openai' — which does not propagate to cloud's own consumers, so
gateway/iam/etc. each had to copy it. Now the fork declares its own module
path (github.com/hanzoai/go-openai v1.41.0): require it directly. Bumps the
lockstep fork adopters — ai v1.822.2, agent v0.1.3 — so the hz.Mount Completer
boundary shares ONE openai type set (was a hanzoai-vs-sashabaranov type
mismatch). No replace anywhere; upstream sashabaranov remains only as an
indirect dep of the go-cosyvoice TTS chain (owned next). cmd/cloud keeps its
single datastore sql registrant (hanzo-ds/go). Builds clean.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-17 22:11:47 -07:00
zandGitHub e655b848de Merge: cloud S3 client minio-go → hanzoai/s3-go (the one house S3 client)
7 files repoint minio-go v7 → hanzoai/s3-go (a byte-identical fork, package
minio, zero call-site churn); minio-go demoted to indirect. Red-verified:
presign + conditional-CAS byte-identical to the prior v7.0.100 dep, drop-in.
Deploy-gate: live SeaweedFS CAS smoke (If-Match/If-None-Match/412).
2026-07-17 22:05:03 -07:00
hanzo-blue 7fc7dcf125 feat(kmsreseal): CR-driven KMS re-seal migration tool (#79)
Embeds the fleet KMS into cloud by re-sealing the ~125 KMSSecret-referenced
secrets from the legacy standalone (unsealed at rest) into cloud's embedded
/v1/kms (AES-256-GCM sealed per secret). Driven by the KMSSecret CRs — the
authoritative (org,path,env,key) manifest — not a raw store copy.

- inventory: pure CR parse/validate/dedup; handles explicit keys, folder-sync
  (empty keys[]), the env-default divergence (cloud refuses empty env), and
  malformed CRs (fail-loud, never silently dropped). Reuses cloud/clients/kms
  ValidSegment/ValidSubpath so a coordinate the store would reject is never built.
- reseal: per-CR org-bound auth (owner==projectSlug), GET standalone -> POST cloud
  (cloud seals). Idempotent upserts, re-runnable. Plaintext transits memory only,
  wiped after write; results carry coordinates + status, never values.
- verify: read-only SHA-256 hash-compare standalone-vs-cloud + org-isolation matrix
  (cross-org 403, no-principal 403) against the real cloud guard.
- preflight: cloud /v1/kms reachability + JWT-validation probes + offline G1 delta.
- runbook: the ordered, rollback-safe cutover (standalone stays read-only).

Tests: round-trip against cloud's REAL embedded /v1/kms in-process (real seal,
real guard) via a zip app.Fiber().Test transport adapter; seal-proof (no plaintext
on disk); wrong-org refusal; folder-sync via LIST; hash-mismatch detection;
isolation matrix. go test ./cmd/kmsreseal green.
2026-07-17 22:03:16 -07:00
zeekay 0a47ea327d policy: kill /v1/featuregate/mode alias + rename featuregate → admission
featuregate READ like a synonym for flags — the source of the "isn't this the
same thing?" confusion. It is not: flags is the Policy decide-ENGINE (/v1/flags);
this package is the request-ADMISSION gate that composes it one-way (host→service
registry + waitlist.<svc> mode read + Enforce middleware + IAM approval check).
Renamed to `admission` — the precise systems term for policy-gating requests
(k8s-style admission control) — which also dodges the gate/gateway/gatewaypolicy
name cluster. flags stays THE engine; admission is a thin one-way consumer.

Also kills the /v1/featuregate/mode compat alias entirely (route + Enforce exempt
entry + test): one route, /v1/flags/waitlist. No shim, no adaptor, no backwards
compat — per the one-and-only-one-way law.

flags engine surface unchanged. Build/vet green; admission tests + apps frozen-Wire
order test pass (admission holds featuregate's slot). Deeper Policy collapse
(authz/entitlements/gatewaypolicy → one engine) is a separate staged HIP-0127 pass.
2026-07-17 21:26:28 -07:00
hanzo-dev bc086c53cf agents/coding: route a coding run to a chosen target machine
When a coding run carries a targetId (a registered /v1/agents/targets
machine), enqueue it as a durable task addressed to that target on the ONE
embedded tasks engine instead of running it in the cloud sandbox. No target
keeps the local sandbox path byte-unchanged.

- mailbox: an in-process rendezvous between the durable RoutedRunWorkflow and
  the external machine that claims a run over HTTP; tenant + machine isolation
  is a property of the (org,target) key, not a check a caller can skip.
- routing: per-target claim key (a capability, stored only as a SHA-256 hash,
  constant-time verified) is the machine identity; a fail-closed liveness gate
  (online + a live runner) is the dispatch admission.
- claim/report HTTP surface (org bearer + X-Target-Key) lets a machine claim
  and complete only runs addressed to it.
- coding.Dispatcher gains a routed branch: open the session on the target,
  enqueue the durable RoutedRunWorkflow (no secret in the payload — the machine
  authenticates with its own credential), return queued; fail closed on an
  unavailable target or a failed enqueue, never fall back to local.

Tests: dispatch-to-target, no-target-local-unchanged, dead-target-fail-closed,
cross-machine/cross-org claim denied, mailbox isolation + claim race.
2026-07-17 21:26:04 -07:00
hanzo-dev 83ab5130db Route the S3 object plane through the hanzoai/s3-go client
Swap the S3 client in the 7 direct importers from github.com/minio/minio-go/v7
to github.com/hanzoai/s3-go (package minio; a minio-go v7.0.98 fork). Drop-in:
same package name and same New/Client/Options, {Get,Put,List,MakeBucket,
RemoveObject,RemoveObjects}Options, ObjectInfo/Object/ErrorResponse surface and
credentials.NewStaticV4; conditional-CAS (SetMatchETag/SetMatchETagExcept) and
presign paths unchanged.

minio-go leaves the direct requires and stays indirect (luxfi/zapdb via
clients/kms). Run go mod tidy after the s3-go v1.0.0 tag is published to
populate go.sum.
2026-07-17 21:24:02 -07:00
antje 1a5cc134a0 gpu worker: a claimed job survives the engine recycle window
The supervisor recycles at queue-idle, but a freshly CLAIMED job is invisible
to the engine queue until its graph is submitted — so recycles fired over the
claim-to-submit window and staging failed on a dead engine, consuming the job
(observed twice in prod, seconds apart). Two invariants close it: a staging
latch the supervisor honors before recycling, and waitEngine() so a job
claimed while a recycle is already mid-flight waits out the restart instead
of dying on connection-refused.
2026-07-17 20:28:12 -07:00
z 7136fff969 feat(link): server-side failover router over linked accounts + per-account usage
Add the execution half of route.go's redundancy seam: a Router that routes a
signed-in caller's inference through one of their OWN linked provider accounts,
resolves that account's KMS-sealed credential, and cycles to the next in-org
account on a live 429 — never falling back to a platform key or crossing the
tenant boundary. Meters each served call per (org, provider, profile) and
exposes the per-account breakdown at /v1/billing/usage/accounts (+ the canonical
/v1/links/usage/accounts).

- resolver.go: the credential-fetch contract consumed (Resolver seam) + a
  KMS-backed impl keyed at orgs/<org>/providers/<provider>/<profile>; Credential
  redacts under every fmt verb, resolves only within one org's namespace, and
  never falls back to a platform key.
- select.go: the non-secret account selector (openclaw Model@provider:profile,
  X-Provider-Account header, session pin) — carries no org/subject.
- router.go: in-org candidate selection (Links.ListLinked only), cycle-on-429,
  fail-secure, per-account cooldown; PolicyPlan|MostRemaining|RoundRobin.
- routed.go + meter.go: a summing per-account usage counter beside the Links +
  the meter that bills api-key accounts via commerce and leaves subscriptions
  plan-paid (BillingMode), so a call is metered once per meaning.
- carrier.go + wire.go: a process-local credential carrier (never serialized) +
  the Deps composition root and the AIClient upstream adapter. Inert until called.

Tests prove routing through a linked account, in-org cycling on 429, the
cross-org isolation boundary (against the real store), fail-secure with no
platform-key fallback, and that a credential never reaches a log or error.
2026-07-17 19:19:33 -07:00
z a7e4d68acf feat(link): server-side failover router over linked accounts + per-account usage
Add the execution half of route.go's redundancy seam: a Router that routes a
signed-in caller's inference through one of their OWN linked provider accounts,
resolves that account's KMS-sealed credential, and cycles to the next in-org
account on a live 429 — never falling back to a platform key or crossing the
tenant boundary. Meters each served call per (org, provider, profile) and
exposes the per-account breakdown at /v1/billing/usage/accounts (+ the canonical
/v1/links/usage/accounts).

- resolver.go: the credential-fetch contract consumed (Resolver seam) + a
  KMS-backed impl keyed at orgs/<org>/providers/<provider>/<profile>; Credential
  redacts under every fmt verb, resolves only within one org's namespace, and
  never falls back to a platform key.
- select.go: the non-secret account selector (openclaw Model@provider:profile,
  X-Provider-Account header, session pin) — carries no org/subject.
- router.go: in-org candidate selection (Links.ListLinked only), cycle-on-429,
  fail-secure, per-account cooldown; PolicyPlan|MostRemaining|RoundRobin.
- routed.go + meter.go: a summing per-account usage counter beside the Links +
  the meter that bills api-key accounts via commerce and leaves subscriptions
  plan-paid (BillingMode), so a call is metered once per meaning.
- carrier.go + wire.go: a process-local credential carrier (never serialized) +
  the Deps composition root and the AIClient upstream adapter. Inert until called.

Tests prove routing through a linked account, in-org cycling on 429, the
cross-org isolation boundary (against the real store), fail-secure with no
platform-key fallback, and that a credential never reaches a log or error.
2026-07-17 19:19:33 -07:00
zeekay 559e54bd45 build(iam2): bump v0.15.4 → v0.16.0 (argon2id SOTA password hashing) 2026-07-17 17:35:27 -07:00
z 9c4f6df7a7 refactor(usage): unify account-usage onto the ONE /v1/usage surface
The account-usage plane (2024df4) wrongly opened a SECOND usage surface inside
clients/link (/v1/links/usage). Move it into clients/usage so usage owns ALL
usage and link owns links and nothing usage — one surface, orthogonal, one window.

Moves (package link -> usage): sample.go (the Sample value + Sanitize), datastore.go
(the hanzo.account_usage warehouse series + reads, now behind a `warehouse` type that
holds only the DDL latch over aiobject's shared datastore — no handle, so usage keeps
NO Shutdown), and the record/samples handlers (account.go). Reconciled with the usage
subsystem: cloudUsageTable -> the existing llmTable, dsTime -> the existing tsLiteral,
duplicate aString -> dsString.

Route table (was /v1/links/usage*):
  POST /v1/usage           record account-usage samples (the collector)
  GET  /v1/usage/samples   one provider account's own lane dash (time series)
  GET  /v1/usage/summary   THE one summary — merged (see below)
  GET  /v1/usage/analytics{,/access}  unchanged

Summary collision resolved by MERGE, not two endpoints: the account-usage global view
folds into the existing /v1/usage/summary as a labelled `accounts` block beside spend +
LLM, over ONE window (aiobject.ResolveCloudUsageWindow drives both). Nothing dropped —
the caller's own linked-account rows AND the org Hanzo-routed rows both ride the one
summary, each side reporting its own availability, never summed.

Decomplected the Link-refresh: reportUsage braided a warehouse write with a Link
upsert, and since POST /v1/links already sets an account's usage snapshot, the sample
-> snapshot path was a SECOND way to do that. record now records usage only; the link
registry stays link's own concern. Drops the 3 Link-registry tests (they exercised
/v1/links, unreachable in a usage-only mount) and the Link half of 2 more; the warehouse
+ value coverage moves intact. No back-compat alias (the route was hours old).

Wire guard unchanged: link keeps its Shutdown (SQLite store), usage keeps none.
2026-07-17 16:53:17 -07:00
hanzo-dev 69f3d23b53 fix(iam-edge): forward the public sign-in surface before the tenant gate
console.hanzo.ai is served one-binary off cloud, so its /v1/iam/* calls hit the
iam_edge — which required a validated org for EVERY route. That 401'd
'sign in to continue' on the sign-in routes themselves (get-app-login, login,
oauth token exchange), a chicken-and-egg that bricked console login (the
'unknown iam route' / 'sign in to continue' users saw). Forward the
unauthenticated-by-design sign-in surface (login-page config, credential submit,
signin/signup, captcha/verification aids, the OAuth token endpoint, OIDC
discovery) straight to IAM BEFORE the org gate. Tenant CRUD + org metadata stay
fully gated — no tenant-data route is opened. Test: TestIamEdgePublic.
2026-07-17 16:50:29 -07:00
hanzo-dev 62068b2d1e feat(integrations): Cloudflare apikey connector (verify-before-store, KMS custody, org-admin gate) 2026-07-17 16:14:21 -07:00
hanzo-dev 04a7644f42 feat(integrations): Cloudflare apikey connector (verify-before-store, KMS custody, org-admin gate) 2026-07-17 16:14:21 -07:00
hanzo-dev 754becb1e0 wip(fleet-samples plane (clients/samples + /v1/fleet board)): rescued from agent that hit the session limit
Committed as-is to preserve the work (the building agent died mid-verify).
Not yet built/tested green; NOT merged to main. Resume from here.
2026-07-17 16:04:56 -07:00
zeekay 607deab802 chore: trigger release build for iam2 v0.15.4 (federation fix)
e5b28e4 (iam2 v0.15.1→v0.15.4 bump) did not trigger a release run; nudge
the push-triggered release so the federation-security-fixed image ships.
2026-07-17 15:21:10 -07:00
zeekay e5b28e4f6b build(iam2): bump v0.15.1 → v0.15.4 (federation SuperAdmin-mint CRITICAL fix)
v0.15.4 closes the red-team CRITICAL in the federation broker: authorize
Application.Organization on write + reserved-org guard in federation
link/provision (was: social login could mint a SuperAdmin / take over a
cross-tenant account) + SSRF IP filter. Required before the hanzo.id social
cutover. Build pipeline healthy (consensus v1.36.3).
2026-07-17 15:03:48 -07:00
hanzo-dev 67e9d54902 Merge: surface the iam2 canary in /v1/flags 2026-07-17 12:39:04 -07:00
z 563ad78dee flags: surface the iam2 canary in /v1/flags
A read-only subsystem_iam2_active switch on the platform panel, mirroring
subsystem_iam_active, makes the clean-room iam2 selection visible in the /v1/flags
cockpit. The selector stays ONE thing — CLOUD_IAM_IMPL=iam2 at boot, applied on
the next reconcile — this switch reflects it, it does not add a second control.
Its description names the gate: the IAM cutover parity suite
(universe e2e/50-iam-cutover-parity) must be green against the iam2 shadow before
the canary is flipped.
2026-07-17 12:38:47 -07:00
hanzo-dev 2024df413b feat(account-usage): clients/link usage plane — samples, datastore series, /v1/links/usage
The account-usage plane over clients/link: a Sample value (one metering lane of
one provider account at one instant), a ReplacingMergeTree warehouse projection
(hanzo.account_usage + a dedup-preserving daily rollup MV) read back with explicit
read-time argMax dedup, and the /v1/links/usage surface — report samples, a
per-provider dash, and a global summary that sets a user's own linked-account plan
usage beside the org's Hanzo-routed cost of record, every row labelled by
source/scope/confidence and never summed together.

A windowless sample (a valid window class with no meter-reported duration or
reset) keys its class's nominal bucket, never the zero instant: every ranged read
filters window_start into [from,to) and the TTL drops epoch rows on arrival, so a
zero-keyed row would be written-but-never-read and would silently drop out of the
summary. Re-polls of a windowless counter collapse onto that one nominal instance
(ReplacingMergeTree by ts), so it is one row per lane, never summed across polls —
reconciling the two window-instance tests the rescued WIP left in contradiction.
2026-07-17 12:31:12 -07:00
hanzo-dev b413370b19 fix(identity): mint X-Billing-Account-Id from the claim, never from the client
The header was captured from client input and re-injected verbatim for any
validated principal. That was defensible while it was a mere attribution hint
no debit ever read — the comment said as much. It is not one anymore:
ai/object.Payer now resolves the PAYING account from it, so forwarding the
client's copy would let a caller name its own payer, which is the whole thing
the claim exists to prevent. A signup-org member could have sent
`X-Billing-Account-Id: org:hanzo` and pointed their spend at the shared pool.

It is now minted from the validated `billing_account` claim
(idClaims.mintedBillingAccount), mirroring iamauth.Claims.MintedBillingAccount
byte-for-byte, so the in-binary path binds what the gateway would and both
resolve one payer. The raw client copy is deleted on ingress and not restored.

The console read and the top-up now hand Payer that same claim, so the balance a
member SEES, the account a top-up FUNDS, and the account the ai gate DEBITS are
one wallet. Feeding Payer a different credential per call site is the modern
shape of the old org-vs-"org/user" split: a funded balance the gate refuses.

Tests drive real signed tokens through the boundary: the claim reaches the
header for person/org/project, a forged copy never survives (even on a token
that carries no claim, where a restored copy would be the only value present),
and an anonymous caller carries no payer at all.
2026-07-17 12:27:34 -07:00
z beea858117 refactor(flags,featuregate): decomplect the waitlist host-gate out of the flag engine
flags is now the PURE (Principal,context)->verdict engine: Register/Bool/Int/
String/Board/SetPlatformSwitch/Defs + /v1/flags/* + native evaluator + the
platform-switch seed. ZERO host->service / ModeForHost / waitlist.<svc> /
mode-route knowledge.

The complete launch waitlist-gate feature moves to clients/featuregate, which
COMPOSES flags one-way (flags.Bool/Register/SetPlatformSwitch/Def/Defs; flags
never imports featuregate):
- flags/waitlist_store.go -> featuregate/registry.go (host->service map)
- flags/waitlist.go -> featuregate/waitlist.go (mode decide + admin funcs +
  seed + waitlist.<svc> Def registration + Mount/Shutdown + the mode route)
- flags/waitlist_store_test.go -> featuregate/registry_test.go
- the registry OrgStore handle (was flags.Client.registry) is now featuregate
  package state, opened in featuregate.Mount, closed in Shutdown
- Enforce default gate is now the LOCAL WaitlistModeForHost
- /v1/flags/waitlist AND /v1/featuregate/mode compat alias served by
  featuregate (route name unchanged)
- apps.Wire re-adds featuregate after admin (after flags); wire_test frozen row
- admin/services.go swaps the flags import to featuregate for the board funcs
2026-07-17 12:06:06 -07:00
zeekay 763daed531 fix(flags): keep /v1/featuregate/mode as a TEMPORARY compat alias for /v1/flags/waitlist
The namespace collapse (132d01a) renamed the live waitlist-mode read to
/v1/flags/waitlist and made /v1/featuregate/mode a 404 — correct per the one-namespace
Policy primitive, but a BREAKING change to a public route whose external callers cannot
be fully enumerated from the monorepo (a deployed frontend could still call the old
path). Per the hard "never goes down for any customer" constraint, ship the collapse
WITHOUT the break: /v1/flags/waitlist is canonical; /v1/featuregate/mode is a temporary
alias to the same handler; both exempt from the Enforce gate.

Delete the alias (this route + its exempt entry in featuregate/middleware.go) once every
caller is confirmed on /v1/flags/waitlist — a one-line follow-up, gated on the owner.

Verified: gofmt clean, go build/vet green, exempt-path test asserts BOTH routes ungated.
2026-07-17 11:24:57 -07:00
hanzo-dev 6a269a581c Merge rip/services-kind: read one workload kind (App), drop the Service shim
services.hanzo.ai is dead (0 Service CRs cluster-wide; the fleet is 100% App).
clients/paas, clients/deploy, and clients/platform drop the two-kind read shim
and read apps.hanzo.ai only. The paas deploy endpoint (and release seam) now
always refuse a git-declared App with 409, naming the universe git path to
commit the tag to.
2026-07-17 11:01:28 -07:00
hanzo-dev 638cbe34ef paas/deploy/platform: read one workload kind (App), drop the Service shim
The services.hanzo.ai kind is dead: zero Service CRs exist cluster-wide and
the whole fleet is apps.hanzo.ai (kind App). These three cluster-facing planes
carried a two-kind read shim (App first, Service fallback) that is no longer
reachable, so strip it and read one kind — App.

- clients/paas: drop servicesGVR + crGVRs(); listApps/getApp/observeFleet read
  appsGVR directly (no cross-kind dedup). The deploy endpoint now always refuses
  (409): every App CR in the platform namespaces is git-declared and reconciled
  by Hanzo CD with selfHeal, so a patch here is reverted — the response names the
  git path to commit the tag to. releaseService refuses on the same grounds.
- clients/deploy: drop servicesCRGVR + appCRGVRs() and the "hanzo.ai/Service"
  registry entry; health/getAppCR/listAppCRs read appsCRGVR directly. coreSvcGVR
  (the core/v1 Service child object) is unchanged.
- clients/platform: drop servicesGVR + crGVRs(); resolveCR/getCR/deleteService
  read and delete appsGVR only. Tenant apps are still written and patched as App
  CRs in tenant-<org>.

Tests updated to the one-kind reality. Builds/vets/gofmt clean; go.mod untouched.
2026-07-17 11:00:25 -07:00
zeekay 132d01a0ff flags: move the waitlist mode read /v1/featuregate/mode -> /v1/flags/waitlist (one namespace)
The guard's public waitlist-mode read now lives under /v1/flags (the flags engine
owns it) — there is NO /v1/featuregate HTTP endpoint. The route, the Enforce
exempt prefix, and the doc/prose comments move; the featuregate Go PACKAGE (native
Enforce middleware) is NOT renamed, and the /v1/admin/services board is unchanged.

- clients/flags/routes.go            GET /v1/featuregate/mode -> GET /v1/flags/waitlist
- clients/flags/waitlist.go          doc comments repointed
- clients/featuregate/middleware.go  defaultExemptPrefixes /v1/featuregate/ -> /v1/flags/waitlist
- clients/featuregate/middleware_test.go  exempt-path assertion updated
- apps/apps.go                       stale prose comment repointed

Verified green: go build ./clients/flags/... ./clients/featuregate/... ./apps/...,
go vet, and CGO_ENABLED=0 go test ./clients/featuregate/...
2026-07-17 10:57:25 -07:00
hanzo-dev 141a8505b2 Merge: real semver only — ai v1.821.1 / iam v1.31.28 / luxfi from sumdb (kill pseudo-versions + force-moved-tag poison) 2026-07-17 10:48:06 -07:00
zandhanzo-dev b0fc5838eb Real semver across the board: ai v1.821.1, iam v1.31.28, luxfi from sumdb
Three coordinate-hygiene fixes so the pipeline resolves deterministically:
  - ai v1.820.0 -> v1.821.1. v1.820.0 pinned iam at an orphaned pseudo-version
    (a commit rebased out of existence); v1.821.1 pins the real iam tag v1.31.28.
  - iam -> v1.31.28, the real published tag; the pseudo-version and its replace
    are gone.
  - luxfi go.sum re-recorded from the immutable sum.golang.org via go mod tidy,
    so consensus/vm can no longer carry the hashes a force-moved git tag served.

Nothing but coordinates changed; ai v1.821.1 is v1.820.0's tree with one dep line
repinned, so the compiled result is identical to the shipped v1.801.49. Real
public semver only: no pseudo-versions, no replaces, no force-moved tags.
2026-07-17 10:47:19 -07:00
zeekay e2c900e4e7 fix(release): fail-closed container-tag floor so an orphaned tag is never reused
The v1.801.50 tag collision: a run pushed :v1.801.50 then was cancelled after
imagetools-create but before its git tag (orphaned container tag). The Tag steps
container-tag floor (cont_max) was fail-OPEN — `gh api ... 2>/dev/null || true`
yields "" on any API error — so a later run did NOT see :v1.801.50, recomputed
the same number, and REASSIGNED :v1.801.50 to a different image: an ambiguous
mutable prod tag (silent flip on any fresh-node reschedule).

Fail-CLOSED: if the container-tag lookup ERRORS (vs legitimately empty), retry the
whole attempt instead of proceeding on a git-only floor that cannot see the orphan.
A version that already has a pushed image is now never reused.

NOT reordering git-tag before imagetools-create (the other candidate fix): that
reintroduces the phantom "tag exists, image does not" this workflow was built to
prevent. Pairs with the crane-mirror timeout (5d8ec40) that stops the hang→cancel
which orphans tags in the first place. Compute-step cont_max left as-is (hint only).

[skip ci]
2026-07-17 10:29:03 -07:00
antjeandGitHub 3c560765d0 supervisor: recycle only at queue-idle; busy is not dead (#329)
Recycling on each completed render killed long renders mid-sample when short
jobs shared the engine (observed: every direct render died within ~6 minutes
while probe jobs cycled). The recycle now defers until the queue is empty.
Health: an engine that answers /queue with work in it is alive however slowly
it answers /system_stats; restarts require three consecutive silent probes
with an idle or unreadable queue.
2026-07-17 02:48:12 -07:00
hanzo-dev e87835bab0 fix(release): mirror LOGIN is best-effort too — a registry blip must not fail the release
The 'Mirror credential' step fail-safed only on a missing KMS token, not on the
docker-login to registry.hanzo.ai itself. A transient 502 from the mirror registry
(ingress blip; the registry was healthy 6m before and after) killed the whole
serialized release — no image, no tag — even though ghcr (the PRIMARY) was fine.
Both login paths now skip the mirror (MIRROR_OK unset) on failure and continue.
Complements 5d8ec40 (the crane-copy timeout): the mirror is now best-effort end to end.
2026-07-17 02:26:28 -07:00
z 5d8ec408d3 fix(release): bound the registry.hanzo.ai crane mirror with a timeout
An unbounded `crane copy` to registry.hanzo.ai can HANG (not just fail) — the
best-effort mirror once livelocked the Tag step and held the entire serialized
release lane (concurrency: release-cloud, cancel-in-progress:false), so no queued
release could run. A best-effort mirror must never be able to block the git-tag
receipt that follows it. `timeout 120` makes it truly best-effort.

[skip ci]
2026-07-17 02:06:48 -07:00
zeekay 8ea9134423 build(iam2): bump v0.14.0 → v0.15.1 (federation + signing-key generation)
v0.15.0 adds the OIDC/OAuth2 social-federation broker (Google/GitHub);
v0.15.1 mints signing keys for keyless reserved-org certs so the embedded
iam2 publishes a JWKS and can sign tokens (shadow-canary finding). Carries
the full parity + RFC surface into the cloud image for the hanzo.id cutover.
2026-07-17 01:35:15 -07:00
hanzo-dev ffdd5e2242 build(deps): adopt luxfi/consensus v1.36.9 (unbreak force-moved v1.36.2 checksum)
luxfi/consensus v1.36.2 was force-repushed with different go.mod content, so
cloud's committed go.sum no longer matches and 'go mod download' aborts with a
SECURITY ERROR — breaking EVERY release. Same recurring luxfi force-move pattern
as ad7e414 (keys). Bump to latest stable v1.36.9; clients/controlplane (only
importer) compiles clean, go mod verify passes.
2026-07-17 01:27:58 -07:00
antjeandGitHub 8c34978d89 render: poll window matches the dispatch cap; engine recycles after each render (#326)
* render: poll window matches the dispatch cap; engine recycles after each render

The 10m local history poll undercut the 4h startToCloseTimeout the dispatcher
grants — live renders (observed 8-70m) were marked failed while still sampling;
only the mirror later delivered them. renderWindow now matches the cap.

The engine leaks ~58GB per render. The handler signals a recycle after each
COMPLETED render (never on timeout — the engine may still be sampling and the
mirror rescues late finishes); the supervisor restarts on the signal.

* mirror: skip hidden files — AppleDouble forks pass the extension check

._foo.png is a mac resource fork, not a render; 700+ of them poisoned a
library within an hour of the mirror going live.

* deps: luxfi/consensus v1.36.2 -> v1.36.3 — the v1.36.2 tag was re-pushed

Cold builds fail sumdb verification against the moved tag (downloaded
eKzasq4O... vs sealed IbeWQF1w...). v1.36.3 is the immutable successor;
never re-tag a published version.
2026-07-17 01:26:40 -07:00
hanzo-dev ad26470ae2 feat(ai): bump ai v1.818.0 → v1.820.0 — router live-by-default + record-all + self-export/delete
Ships to prod: router.enabled=true (model=auto routes for every org by default),
per-request RoutingEvent recording for auto AND explicit models (up/down feedback
works on all models), per-org + global fit-gate-deploy-publish training, and the
self-scoped routing-data export/delete (data ownership). Pairs with the universe
CR ROUTER_ENDPOINT removal (heuristic 300ns is the live path).
2026-07-17 01:25:40 -07:00
hanzo-dev 37aeab8060 dedup: extract clients/payout from the 3 byte-mirror commerce.go copies
referrals/affiliates/authors each carried a byte-identical commerce.go (their own
doc-comments said so): the same commerce interface, httpCommerce, newCommerceClient,
deposit(), spendCents(), errUnconfigured — the S2S COMMERCE_SERVICE_TOKEN money-in
path (POST /v1/billing/deposit) + usage-rollup, triplicated.

Extract ONE clients/payout (attributed-credit -> commerce via commerceinproc): the
exported Commerce/Client/NewClient/ErrUnconfigured. Each program keeps a THIN
adapter — its own lowercase commerce interface + a commerceSeam that delegates to
payout.Client — so the program store/handler code AND their fakeCommerce test doubles
are untouched, and each program still names its own grant tag (grant:referral /
grant:affiliate / grant:author). ~330 duplicated lines collapse to one binding.

Zero behaviour change: identical HTTP contract, headers (X-Org-Id, Bearer), body,
fail-soft (ErrUnconfigured on deposit / 0 on spend when unwired), and errors.Is
sentinel. Adds payout unit tests (httptest) that give the extracted HTTP path REAL
coverage the fakes never did — ok clients/payout 0.010s.
2026-07-17 01:19:36 -07:00
hanzo-dev a41c2ae9e6 dedup: fold clients/gojabase into clients/goja (the Base binding is an option)
gojabase was the read-WRITE-Base sibling of goja: it wrapped a goja.Host and
added per-tenant Base/SQLite persistence, but duplicated the Host/Config/Request/
Response/New surface. Fold it into the ONE goja package as the Base-binding
CONSTRUCTOR — the persistence layer is now opted into via NewBase (vs New for a
read-only catalog bundle):

  goja.New   / goja.Host   / goja.Config   / goja.Request    read-only engine (plans/pricing)
  goja.NewBase / goja.BaseHost / goja.BaseConfig / goja.BaseRequest   + per-tenant Base

Moves gojabase.go -> clients/goja/base.go (renamed types, no goja. self-import),
store.go -> basestore.go, and both test files, all into package goja (zero
identifier collisions, coverage preserved). Repoints every importer —
dataroom/captable/sign (RW) to goja.Base*; plan/pricing already used goja and are
unchanged; base uses goja.TenantSegment. clients/gojabase deleted.

Behaviour is byte-identical: the engine, the per-request transaction commit-on-
<400, the injective TenantSegment, and the __db/__blob/__newId/__now host globals
are unchanged; only the package + exported names moved. No routes (both are
libraries). The gojabase[...] error prefix is kept as the RW-layer diagnostic label.
2026-07-17 01:19:36 -07:00
hanzo-dev d22d107e85 dedup: fold connectorruntime into the one automations subsystem
clients/connectorruntime mounts exactly ONE route —
POST /v1/automations/connectors/:id/run — the in-process goja runner paired with
automations own GET /v1/automations/connectors catalogue. It was a separate Wire
entry solely for that route. Fold connectorruntime.Mount in as a terminal
sub-mount of automations.Mount and drop its Wire entry + import -> ONE
automations subsystem.

The route is DISTINCT from every automations route and automations mounts no
/v1/automations/* wildcard, so there is no shadow; the runner still resolves the
shared engine lazily. clients/connectorruntime stays a focused package
(composition); its internal bundlecmd tool is untouched. Frozen wire row
removed.
2026-07-17 01:19:36 -07:00
hanzo-dev 5b020fc441 dedup: fold platform cron into the one tasks subsystem
clients/cron mounts NO routes — its Mount only launches a background starter
that registers durable schedules on the SAME shared engine (cloud.EmbeddedTasks)
that clients/tasks fronts. It was a separate Wire entry purely to get its
goroutine launched. Fold it in as a terminal sub-mount of tasks.Mount and drop
the cron Wire entry + import -> ONE tasks subsystem.

clients/cron stays a focused package (composition, not code-dumping): tasks
imports and invokes it. No routes change (cron never had any); the scheduler
still waits for the post-MountAll engine, so timing is unchanged. Frozen wire
row removed.
2026-07-17 01:19:36 -07:00
hanzo-dev d44fae4cf5 dedup: normalize the plan subsystem enable id "plans" -> "plan"
clients/plan.Mount was wired under the name "plans" while its package, and now
its generated standalone cmd, are "plan" — one subsystem, two names. Normalize
the Wire enable id (and cmd/plans -> cmd/plan, ServeSingle arg) to "plan".

Product routes are unchanged: the subsystem still serves /v1/plans/* (plural),
including its OwnsHealth /v1/plans/health probe — only the enable id / binary
name changes. No route drop; mount-all default still enables it (empty Enable =
all on). Updated the frozen wire row and the two cmd/cloud enable-id references;
TestMountAllAndServeHealth now maps plan to its real /v1/plans/health path
(enable id no longer equals route prefix for plan, as is already true for
account/runtime/agent).
2026-07-17 01:19:36 -07:00
c842f3b1df fix(release): resolve prebuilt artifact digests + unbreak cloud-flags publish (#327)
cloud#321 landed the Go-only Dockerfile (FROM cloud-flags:latest) but the
release.yml integration wasn't in it, and the reusable could not publish
cloud-flags at all — so every release since has FAILED at
'FROM cloud-flags:latest: not found'. Two fixes:

1. native/flags/Dockerfile base ghcr.io/hanzoai/mirror/rust -> public.ecr.aws
   (digest-identical). The hanzoai/ci reusable builds cloud-flags with the repo
   GITHUB_TOKEN, which 403s pulling the cross-repo-linked private mirror package;
   a public base is GITHUB_TOKEN-pullable, so cloud-flags finally publishes.

2. release.yml resolves console-embed/agent-skills/cloud-flags :latest to
   IMMUTABLE digests at release time (crane) and passes them as CONSOLE_IMAGE/
   SKILLS_IMAGE/FLAGS_IMAGE build-args to BOTH the smoke build and the push build,
   replacing CONSOLE_CACHEBUST. Reproducible (pinned, not floating :latest) AND
   fresh (a console/skills/flags change is a new digest). A MISSING artifact FAILS
   the release BEFORE build/smoke/tag — the receipt invariant holds, never a
   phantom tag on an image that could not embed the real console.

Preserves #322's functional + migration smoke gates (different sections).


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-17 01:18:50 -07:00
55810833c5 bench(finance): BenchmarkListUsage — the co-resident usage-read path (BUG 2) (#325)
The finance domain had no benchmark; this measures the read that replaced the
commerceinproc self-dispatch — finance.ListUsage over a per-org SQLite ledger,
at 100/1000/5000 seeded debits. Backs the reproducibility claim in the
hanzo-unified-tenant-cloud paper (1.25/10.5/61 ms). Also surfaces a real N+1:
store.Entries fetches postings per row the usage view never uses — a
postings-free read would cut this ~10x (follow-up).


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-17 01:06:20 -07:00
zandhanzo-dev d7f24d8e61 Money names its unit: Int -> Atto
There were two words for one idea. money.Amount.Minor() returns an integer at the
CURRENCY's decimals — money.USD declares 2, so cents. cloudmoney.Amount.Int()
returns an integer at 18 decimals — atto. Both are "the backing integer", neither
name says which, and they differ by 10^16.

That is not a style complaint. It is how

    Amount: cloudmoney.FromInt(u.Charge.Minor()), // exact 18-dp USD, no floor

got written, reviewed, and shipped. It type-checked (both sides *big.Int), it read
as "make an Amount from the integer", and it billed a $17.376 zen call as
$0.0000000000000017 until v1.801.44. The package documented the trap in prose —
"Take the decimal, never Amount.Minor()" — because a comment was the only place
the unit existed. Prose is not a type.

So name the unit, not the Go type: FromInt -> FromAtto, Int() -> Atto(),
IntString() -> AttoString(). FromCents/Cents already did this and were never
confused with anything. Now the units are visible at the call site, and the
mistake reads as one: FromAtto(x.Cents()) is obviously wrong where
FromInt(x.Minor()) was obviously fine. It no longer type-checks either —
FromAtto takes *big.Int, Cents() returns int64 — so the pairing that cost us the
money is now two independent kinds of error instead of zero.

Values are untouched: Atto/AttoString return exactly what Int/IntString did, so
the treasury ledger hash and every stored 18-decimal string are byte-identical.
No migration, no data change — only the names, and the compiler found every one
of them (45 sites; a regex could not have, because .Int() also belongs to big.Int
and decimal).

Zero regressions: the failing set is identical to origin/main.
2026-07-17 00:52:06 -07:00
zeekayandhanzo-dev ad7e414672 build(deps): adopt stable luxfi/keys v1.4.1 (unbreak force-moved v1.4.0 checksum)
luxfi/keys v1.4.0 was force-moved on the remote (tree 5153d639→80a3745a),
producing a go.sum checksum mismatch that broke `go mod tidy`/`go build` and
the hanzoai/cloud image build in CI. Every keys tag v1.1.0..v1.4.0 was
force-moved with content changes; only v1.4.1 is byte-stable (local==remote
tree 17551acd). Adopt v1.4.1 as the single stable version. Its go.mod floors
the unified luxfi stack, so the transitive set moves forward (geth 1.17.12→
1.20.1, consensus 1.35.32→1.36.2, crypto/database/warp/zap/…), all within v1.

go.mod/go.sum only; diff vs origin/main is luxfi/* exclusively. Embedded
iam2 v0.14.0, apps.go, and concurrent commerce/agent work untouched.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-17 00:50:13 -07:00
92123af64e build: decomplect the cloud image — cloud compiles ONLY Go (#321)
The console SPA, agent-skills catalog, and native flags staticlib are each now
built by their OWN CI as a versioned immutable image and PULLED into the cloud
build, instead of rebuilding node+python+rust from scratch every release. The
console stage (cold npm install + full Next static export, cache-busted every
build) was the ~20-min long pole; it is now a registry pull.

- Dockerfile: console/skills/flagslib stages -> FROM ${CONSOLE_IMAGE}/
  ${SKILLS_IMAGE}/${FLAGS_IMAGE} prebuilt pulls; COPY sources updated
  (/dist, /catalog, /libhanzo_flags.a). Mirror golang+alpine bases, GOPRIVATE,
  and every RED gate (SQLCipher proof, modernc guard, cek frozen-format) are
  unchanged. Pins are ghcr.io so both buildx lanes pull directly; mirrored to
  registry.hanzo.ai (S3) for GET-flow consumers. release.yml still owns the
  cloud image + v* tags (it resolves CONSOLE_IMAGE to a fresh console-embed
  digest, as CONSOLE_CACHEBUST did).
- native/flags/Dockerfile: the cloud-flags artifact (rust -> scratch /libhanzo_flags.a).
- hanzo.yml: images: cloud-flags (distinct artifact, never a v* tag) + zccache
  RUSTC_WRAPPER on the native-flags gate (no-op unless the runner carries it).

Companion artifact publishers: hanzoai/console#(console-embed),
hanzoai/openapi#(agent-skills).


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-17 00:49:57 -07:00
hanzo-dev 47d099075f refactor(flags): extract the Policy primitive — fold featuregate's waitlist into the flags engine
featureflags -> flags (Hanzo's Unleash-analog / hanzoai/flags): ONE runtime
decision engine, (Principal, context) -> verdict, evaluated in-process, hot.

Fold the launch-control waitlist off its DUPLICATE SQLite mode store
(clients/featuregate) onto the one engine:

  - a service's waitlist mode IS the platform switch waitlist.<svc>,
    evaluated through the same native evaluator as every other flag
  - the host->service registry folds into clients/flags (waitlist_store.go,
    mode column dropped — the mode is the switch)
  - featuregate.Enforce stays but as a CONSUMER of flags.WaitlistModeForHost
    (the decide), via an injectable Gate seam
  - /v1/featuregate/mode -> served by flags (the decide), same wire shape
  - /v1/admin/services -> an admin lens like /v1/admin/flags
  - per-user approval still reuses IAM (featuregate/approval.go, unchanged)
  - featuregate dropped from apps Wire() — it exposes only Enforce now

The flags package doc names the aspirational end-state: authz (access policy)
and entitlements (product-access policy) are the SAME (Principal,context)->
verdict shape and could COMPOSE this one engine. NOT touched here — flagged only.

Tests (green): featuregate RULE acceptance matrix + approvals; folded-registry
store (Seed/ServiceForHost/List/Upsert); flags env/board/parsers;
apps TestWireOrderMatchesFrozen. Native-engine + cek store tests need
CGO+libsqlcipher+libhanzo_flags (CI), unchanged.
2026-07-17 00:48:30 -07:00
hanzo-dev 10f185f563 billing bridge: ask commerce what it gates, stop keeping a copy
The bridge's mint guard kept its own list of commerce's mint routes under the
comment 'kept in lockstep with api/billing/handlers.go'. A comment cannot hold
two lists together, and it hadn't: 10 paths here against the 16 commerce gates.
Six money-mint routes were outside this guard entirely.

Commerce now DECLARES its gated surface — middleware.Mint records what it gates,
MintRoutes() exports it (commerce v1.49.0) — so the guard reads that declaration
instead of copying it. A mint route added in commerce is covered here with nobody
remembering anything, which is the only version of this that survives contact
with a busy repo.

Registration is what populates the registry, so the guard registers commerce's
billing routes before reading it. The request-level assertion is unchanged and is
the part that matters: an ordinary org user's call must never REACH commerce,
because arriving at all means arriving with the admin service token that
satisfies MayMintMoney.

Proof it bites: allowlisting 'deposit' fails with the escalation itself —
'POST /v1/billing/deposit reached commerce carrying Bearer svc-tok'.

16/16 gated routes refused. All six bridge tests pass.
2026-07-17 00:41:23 -07:00
antjeandGitHub b8c17f06aa gpu: mirror local studio renders to the library (independent of claims) (#324)
Every 30s (its own ticker in the connect select loop), scan --studio-dir/output
recursively for image files new or changed since the last scan and POST each to
<studio-url>/v1/library/upload with the worker's bearer, tagged ?node=<identity>
and ?subpath=<subfolder>. So EVERY render lands in studio.hanzo.ai — including
ones produced OUTSIDE the job path: a graph hand-run on the node, or a render
that finished after its activity was reaped (the stranded-late-render class). The
mirror's independence from claims/activities is the point.

State is a tiny in-memory map[relpath]size that skips unchanged files; the studio
endpoint dedupes byte-identical uploads, so a re-scan after restart is cheap and
harmless. One log line per newly stored file; upload failures are summarized once
per scan and retried next tick (no 5xx spam). New --studio-url flag (default
https://studio.hanzo.ai, HANZO_STUDIO_UPLOAD_URL honored) so the mirror works
WITHOUT jobs. No new deps. Test: cli/gpu_mirror_test.go (new/changed uploaded,
unchanged skipped, node+subpath+bearer carried).
2026-07-17 00:20:24 -07:00
antje 21673ccde9 agent: v0.1.2 — pass a completion billing refusal (402) through, not a 502
Bump github.com/hanzoai/agent v0.1.1 -> v0.1.2 and have the in-process completer
return the agent typed hz.UpstreamError{Status,Body} on a non-2xx completion. The
round then passes a caller-facing 4xx (402 insufficient_balance, 429, 403) through
verbatim so a no-credit user sees "add credits", not an opaque "agent: completion"
502. Proven live: POST /v1/agent with a real hk- key ran the round end-to-end and
the completion returned 402. Dropped the now-unused clip() helper. Simplified
commerce_errorscope_test to the real invariant (typed 403 never flattened to 5xx)
now that commerce v1.48.10 honors status itself; the scope stays as the boundary.
2026-07-17 00:01:18 -07:00
antje f3f0181052 deps: commerce v1.48.5 -> v1.48.10 — free $0 tiers stay self-serve (paid-tier gate keys on price, not includedCreditUsd) 2026-07-16 22:38:59 -07:00
zandhanzo-dev cd2f5dece8 MountAll's doc outlived Typed
It still said the MountFunc takes the app as `any` and that in-repo subsystems
recover it via Typed. Neither is true: MountFunc names *zip.App and Typed is
deleted. The app is handed to each mount as itself.
2026-07-16 22:24:17 -07:00
antje 22d899d684 feat(sync,git): Gitea-native sync provider + webhook reject parity; fix LoadConfig flag panic
CHANGE 1 — the /v1/sync git provider drives GITEA (the one git store), not the
retired cloud embedded store:
- clients/sync/gitea.go: a Gitea REST + go-git client (GIT_ADMIN_TOKEN, GITEA_URL).
  Inbound = fast-forward-only go-git fetch(source) -> push(Gitea), so a diverged ref
  is a conflict, never overwritten (split-brain guard, now on Gitea). Outbound = a
  Gitea push-mirror (sync_on_commit) so Gitea itself propagates every commit. Fails
  closed when GIT_ADMIN_TOKEN is unset.
- git_provider.go Reconcile and sync_api.go patch/delete now compose those Gitea
  primitives; the cloud embedded seams (InboundGitSync/ImportGitRepo/EnsureGitMirror)
  are retired from the sync path. resolve() decision, loop guard, cursor idempotency,
  and hop limit are unchanged.

CHANGE 3 — webhook reject parity + a pre-existing root test panic:
- Wrap /v1/git/webhook and the slack(events,commands)/discord/teams/telegram inbound
  webhooks in cloud.Terminal so a bad-signature 401 / malformed 400 survives the
  commerce /v1 500-flatten (uniform reject codes, matching /v1/sync and
  /v1/connector/github/webhook).
- config.go LoadConfig: guard the process-global flag registration with a sync.Once,
  so a re-entrant LoadConfig (many test callers in one binary) no longer panics
  "flag redefined: enable".

CHANGE 2 (retire the embedded git server) is NOT done: it is still a live dependency.
cloneURL resolves to api.hanzo.ai/v1/git (cloud's OWN embedded server) and the
coding-agent orchestrator clones from uploadPack, pushes to receivePack, and reads the
store via VerifyRef. Deferred — migrate coding to Gitea first.

Tests (CGO_ENABLED=0): clients/sync + clients/integrations green; new
clients/sync/gitea_test.go proves reconcile acts on Gitea and the fast-forward guard.
2026-07-16 22:20:41 -07:00
hanzo-dev 820c79a0e1 Merge: Mount takes *zip.App — delete the Typed shim and its 85 wrappers 2026-07-16 21:57:01 -07:00
antje 5aa2a8fa5f edge: scope commerce error envelope to its own routes (unblock release smoke — 14 endpoints 500→4xx)
commercemid.ErrorHandlerJSON() was installed as a /v1 GROUP middleware, but fiber
matches group middleware by PREFIX, not by the handle a route registered on. So on
the shared /v1 it wrapped EVERY subsystem mounted after commerce (projects, agents,
wallets, functions, integrations, marketplace, team, s3, analytics, knowledge,
automations, deploy, billing) and flattened their typed zip.HTTPError (403 "X-Org-Id
required", 400, …) into a blanket 500 — the store envelope always renders 500. The
authenticated release smoke (#322) correctly fails on 5xx, so this blocked every
release since it landed.

Fix stays cloud-side (no commerce dep bump, no money-path churn): commerceErrorScope
guards the envelope by commercePrefixes, so it stays on commerce and every other
subsystem renders its own status via zip default. Pre-commerce subsystems (kms,
o11y) already did; this makes the post-commerce ones match. Verified with the REAL
commerce middleware: /v1/projects 403 (was 500), /v1/store/current keeps the 500
envelope. Regression test added.
2026-07-16 21:50:00 -07:00
zandhanzo-dev fa60ebed6a Mount takes *zip.App: delete Typed and the 85 wrappers
MountFunc took `app any` and cloud.Typed asserted it back to *zip.App on every
mount — a runtime check doing the type system's job, with a failure branch that
could not fire because the only value ever passed is a *zip.App. Every subsystem
paid for it: 85 call sites read cloud.Typed(x.Mount) instead of x.Mount.

The reason given was circular. cloud said `any` was load-bearing because an
external module (licensing) exposed func(any, Deps) error; licensing said it used
`any` to avoid an import cycle in pkg/cloud. Each pointed at the other, and the
cycle cannot exist: this package already imports zip (build.go), and zip does not
import cloud. The `any` was justified by nothing.

So name the type. MountFunc is func(*zip.App, Deps) error — what every subsystem
already exported and what licensing's own doc claimed all along. Typed is deleted,
the 85 wrappers are gone, and the three in-repo mounts that hand-rolled the same
assertion (mountZen, mountMetrics, MountO11y) just take the app.

The compiler immediately found what the `any` had been hiding: four mounts still
shaped func(any, ...), one of them across a module boundary. That is the point —
a signature drift is now a build failure instead of a runtime error nobody would
see until a subsystem mounted.

Also here, because the same rip surfaced them:

  - iam: v1.31.27-0.20260716191958-4400762928a2 -> v1.31.28, and the replace
    pinning a second pseudo-version is dropped. The required pseudo-version named
    a commit that no longer exists (it was rebased away), so `go mod tidy` could
    not resolve it; v1.31.28 is a real tag and a strict superset of what the
    replace pointed at. A version, not a coordinate, and no replace.
  - licensing -> v0.1.5, which is where the typed Mount ships.
  - cloud.OrgConfig is aliased next to LicenseEntitlement. Both are named by
    CommerceClient's methods, but only one was exported, so the exported
    interface could not be implemented from outside without reaching into
    cloud/types — an omission, not a boundary.
  - build_registration_test.go tested Typed and nothing else. "Recovers the
    *zip.App" proved an adapter passed through its argument; "fails closed on a
    wrong type" cannot be compiled now. What is left is the assertion that a
    subsystem signature IS a MountFunc — which the build checks.

No regressions: the failing set is byte-identical to origin/main (11 pre-existing
TestAudit_*).
2026-07-16 21:47:19 -07:00
zeekay 48cd0e41e1 build(iam2): bump embedded iam2 v0.1.1 → v0.14.0 (parity-complete surface)
The CLOUD_IAM_IMPL=iam2 identity fold (clients/iam2, identitySpec) was pinned
to iam2 v0.1.1 — an early cut missing the whole console-parity surface. Bump
to v0.14.0 so the embed actually serves what hanzo.id needs:

  - RFC/IETF surface (HIP-0111): OAuth2 code+PKCE/refresh/client_credentials/
    password, RFC 8693 token-exchange, 7662 introspection, 7009 revocation,
    8414 AS-metadata, OIDC UserInfo, SCIM 2.0 Users
  - Casdoor verb-alias compat (transitional cutover bridge) for every verb the
    live console/gateway hard-code: users/orgs/apps/providers/roles/projects
  - operator bootstrap upsert (IAM CR reconciliation), TOTP MFA enrollment,
    organization-scoped projects (ScopeSwitcher)

No wiring change — main's identitySpec + co-mingle iam2server.Mount(app, db)
compile unchanged against the v0.14.0 API. zip already at v1.8.3.
2026-07-16 21:43:03 -07:00
antje ec50b73366 feat(connector,sync): /v1/connector/github/webhook namespace + reject paths survive the commerce /v1 500-flatten
CHANGE A — first-party vs external route naming. Rename /v1/github-webhook ->
/v1/connector/github/webhook, opening the external-platform namespace
/v1/connector/<provider>/webhook (github now; gitlab/others are sibling literal
routes later, each with its own signature scheme). /v1/git/webhook (first-party
Hanzo Git) and /v1/sync (the bridge) are unchanged. No live consumer breaks:
the GitHub App isn't created yet.

CHANGE B — 500 -> real 4xx on the reject paths. mountCommerce registers
commerce's ErrorHandlerJSON on app.Group("/v1"); that filter rewrites ANY error
a downstream /v1 handler PROPAGATES into a hardcoded HTTP 500. Every /v1
subsystem mounted after commerce is wrapped the same way -- git, sync AND
integrations alike. git does NOT escape it: its isolated tests read 401 only
because they don't co-mount commerce (its bad-sig path is never exercised in
prod, so the flatten went unseen). Add cloud.Terminal, which writes a returned
*zip.HTTPError in-band and returns nil, so the filter's c.Next() sees nil and
has nothing to flatten. Wrap /v1/sync (all verbs) and the connector webhook.
sync no-principal is now 401 (was 403) -- an authentication failure, matching
the webhook's bad-sig 401. The commerce filter is left untouched.

Tests reproduce the /v1 flatten filter and assert: sync unauth -> 401, connector
bad-sig -> 401, malformed body -> 400; connector resolves at the new path and
the old /v1/github-webhook 404s.
2026-07-16 21:32:46 -07:00
antje 146d65d380 edge: /v1/agent self-meters (fix 503 balance_unavailable on the agent orchestrator)
The global BillingGate priced /v1/agent/* at a flat 1c via a dead legacy
bot-reverse-proxy rule in DefaultPrice, so every agent request was gated: the
unauthenticated preset/conversation reads hit the balance check, failed closed,
and 503d before the orchestrator handler ever ran. The route was mounted and
winning precedence the whole time — the gate short-circuited ahead of it.

/v1/agent is self-metered: the reads are free and POST /v1/agent bills through
the /v1/chat/completions it runs in-process (gated + metered downstream), so the
edge must price it 0 or double-bill. Drop the legacy branch (and its now-unused
cloudEdgePriceCents const); price /v1/agent and /v1/agent/* at 0. Tests updated.
2026-07-16 21:06:09 -07:00
hanzo-dev b44bb4da4b Merge feat/openapi-spec: /v1/openapi.json generated from the live router
The spec IS the router, not a description of it: apps.Wire() -> MountAll ->
app.Fiber().GetRoutes(). 983 operations / 692 paths / 109 products, served beside
/zap so ZAP and OpenAPI are two projections of one route table rather than two
sources that can disagree.

A drift test proves the bijection and was proved to fire; it already caught
/v1/pricing-policy and /v1/pricing/policy collapsing onto one operationId.

# Conflicts:
#	serve.go
2026-07-16 20:42:42 -07:00
5c148bd48e fix(billing,smoke): usage reads the co-resident ledger (not a self-dispatch) + a real authenticated release smoke (#322)
A live authenticated smoke surfaced two production bugs; this fixes both and
adds the durable smoke that now guards every release.

BUG — /v1/billing/usage 500 for a valid caller. usage() proxied
"/v1/billing/usage" through commerceinproc, which re-dispatches BY PATH.
Commerce's own billing routes are behind //go:build cloud and never compiled
here, so the ONLY registration of that path is usage() itself — the S2S hop
re-entered the handler, which self-answered "sign in to view billing". This is
the SAME defect balance() was already fixed for. usage() now reads the usage
ledger DIRECTLY from cloud's own finance ledger (finance.ListUsage → the
wallet→revenue debits RecordUsage wrote), off the self-dispatching hop;
split-deploy falls back to the commerce S2S read, unchanged.

BUG — the balance gate 402'd read-only GETs (fixed in hanzoai/ai, pinned via
the go.mod bump). A $0-balance org could not VIEW its own resources. Fixed in
the ai module's BalanceGateFilter — reads never spend, so GET/HEAD/OPTIONS are
exempt; only writes/metered POSTs gate on balance.

SMOKE — cmd/smoke: a durable, authenticated per-subsystem prober. One
side-effect-free read per core subsystem; a read that 402s (balance gate) or
5xx-es (crash) fails the release. Baked into the image (Dockerfile) and wired
into release.yml as the functional gate after the boot check, so a release can
never ship with chat/billing/projects/kms/... down. The smoke token is
KMS/secret-sourced (never hardcoded); absent → the anonymous matrix still gates
public/authed and catches every 402-on-read / 5xx.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-16 20:35:56 -07:00
hanzo-dev 833b2764f1 cli: multi-identity credential store with active-pointer switch
Store every logged-in identity in ~/.hanzo/identities.json keyed by a stable
<owner>/<name> key (admin/z, hanzo/z), with an Active pointer; mirror the active
identity into credentials.json so every legacy single-file reader is unchanged.
A second login as a different owner for the same email (privilege separation:
hanzo-admin-guard vs hanzo-console) is stored beside the first, not over it.

New: auth list (whoami --all), auth switch <owner|owner/name> (alias use),
logout [<owner>]. Token refresh writes through the store (SaveActive) so the
active identity stays fresh after rotation. All files stay mode 0600.

One credential store, active-pointer switch, backward compatible.
2026-07-16 20:20:53 -07:00
hanzo-dev 6644af0533 dedup: fold hanzoai/o11y module wildcard into the one o11y subsystem
apps.Wire() had TWO {Name:"o11y"} entries: the in-repo read plane
(o11y.MountO11y) and the external hanzoai/o11y module wildcard
(cloud.Typed(o11ymod.Mount)). Fold the module wildcard in as the TERMINAL
sub-mount inside o11y.MountO11y (registered after every specific /v1/o11y/*
route, so Fiber in-order match still gives them precedence), delete the 2nd
Wire entry and the now-unused o11ymod import -> ONE o11y spec.

/v1/o11y/health is preserved exactly: the merged entry keeps OwnsHealth=false,
so the generic always-ok route is registered before MountAll (ahead of the
wildcard) — byte-identical to when the module co-entry, also OwnsHealth=false,
triggered it. Frozen wire order updated (two co-owned rows -> one); the
no-duplicate test no longer needs an o11y exemption.
2026-07-16 20:17:18 -07:00
hanzo-dev ced7fdf490 dedup: delete dead clients/session
Zero importers, no cmd/session, absent from apps.Wire(); its /v1/code/sessions
surface was never mounted (dark). Removes session.go, store.go, session_test.go.
2026-07-16 20:10:06 -07:00
hanzo-dev 24e51b17d9 fix(cloud): serve /v1/iam/* — the org-scoped edge to Hanzo IAM
The one-binary console (console.hanzo.ai — cloud serves console's static export)
reads org members + projects at /v1/iam/*, but cloud 404'd those (IAM isn't
folded in-process yet) and the old /org/iam BFF proxy is pruned from the static
export. So the browser got the SPA shell (HTTP 200 HTML), which the client
surfaced as "Request failed (HTTP 200)" — the Platform page died on "Could not
load".

iam_edge.go adds an org-scoped reverse edge at /v1/iam/* → the standalone IAM,
mounted when IAM is NOT folded in-process (else that subsystem owns the path — no
double-mount). The org is PINNED to the caller's validated, server-minted
X-Org-Id (never a raw client header) — load-bearing, since IAM's own authz is
permissive on the org-keyed routes, so without the pin one tenant could read
another's projects. A super admin may cross; a tenant may not; writes require an
org admin and must carry the caller's own org. Shares the ONE IAM identity
(iamHost/iamCred) with the API-key resolver (DRY).

Verified: 7 gate tests (pin, cross-tenant refuse, super-cross, allow-list, 401,
write-gate, own predicate) + go build clean.
2026-07-16 20:07:12 -07:00
zandGitHub 4dda15d2d0 chore(deps): bump hanzoai/tasks → v1.51.1 (fix api.hanzo.ai concurrent-map-writes crash)
Fixes the live recurring tasksd concurrent-map-writes fatal (hanzoai/tasks#18, v1.51.1). Build green; the failing 'Test' check is the pre-existing repo-wide LoadConfig flag-redefine panic (fails on main + all branches), unrelated to this go.mod-only bump.
2026-07-16 20:02:07 -07:00
hanzo-dev dcc440e589 Merge: tenant apps are App CRs — the last writer of the Service kind
clients/platform was the only thing still minting Service CRs; every other
declarer is already App. A role-less App dispatches to the operator's service
profile, the same reconcile the Service kind ran, so a tenant workload carries
over verbatim.

Existing tenant CRs still resolve (App first, Service second). A redeploy patches
the kind it IS rather than minting a twin — two CRs on one name is the
commerce-admin ownerRef flap, not a migration. Teardown deletes BOTH kinds:
either alone rebuilds the app the tenant just deleted, still billing.

113 tests pass, 7 new.
2026-07-16 19:40:04 -07:00
hanzo-dev 071c698792 platform: tenant apps are App CRs — the last writer of the Service kind
clients/platform was the only thing still minting Service CRs. Every other
declarer is already App: universe git has zero hanzo.ai/v1 kind:Service, the
operator calls App "the sole workload reconciler for the collapsed fleet", and 69
of the 80 live CRs are Apps. A tenant app was the exception for no reason — a
role-less App dispatches to the operator's service profile (controllers/app.rs
`classify("") => Dispatch::Service`), which is the same reconcile the Service kind
ran, so an App carries a tenant workload verbatim.

Nothing mints a new Service CR after this. What already exists still resolves:
reads, patches, scales and deletes walk crGVRs() — App first, Service second — so
the 3 live tenant CRs written before this keep working untouched.

A redeploy of a pre-collapse app patches the kind it IS rather than declaring an
App twin. Two CRs claiming one name is not a migration, it is the commerce-admin
flap: both kinds materialize the same children through the same materializer under
one field manager, so the Deployment's ownerRef just flips between them.

Teardown deletes BOTH kinds. Either kind alone re-materializes the Deployment, so
removing only the one that resolves first would rebuild the app the tenant just
deleted — running, and still billing, minutes after a successful delete.

Sequencing (deploy order matters): operator v0.7.7 carries the Claim guard that
makes App the deterministic owner when both kinds claim a name. This is safe
before it — the no-twin rule means a colliding pair is never created here — but
the guard is what makes an existing collision safe to clean up.

REMOVABLE once no Service CR remains in any tenant namespace: drop servicesGVR
from crGVRs() and the delete-both, and this file is App-only.

Tests: 113 pass; 7 new (kind is App, no twin on legacy redeploy, delete removes
both kinds, idempotent delete, resolution order, absence honest). The 10 tests
that asserted a Service CR is written now assert the kind we write.
TestMigrateOverLegacyPlatformApps fails identically on pristine main (sqlcipher
codec, environmental).
2026-07-16 19:39:56 -07:00
antje 7075d8ba01 apps: frozen wire list — add sync entry (main added /v1/sync to Wire; frozen drifted, breaking TestWireOrderMatchesFrozen); fix agent route comment 2026-07-16 19:38:00 -07:00
hanzo-dev 5d185a4c8b Merge: paas reads the kind the fleet runs on, and does not fight the git declarer
The PaaS control plane serves platform.hanzo.ai and read services only: 69 App CRs
run in the scanned namespaces against 7 Service CRs, so the SUPERADMIN drift board
rendered 7 rows for a 69-app fleet — 1 in production — and /v1/paas/health probed
the Service CRD, found it served, and reported ok over a blind board.

Reads now walk App first, Service second, deduped by name. Writes refuse a
git-declared App (Hanzo CD syncs it with selfHeal, so a patch is reverted on the
next sync) and name the file to commit to; a Service CR still patches.

35 tests pass, 0 fail.
2026-07-16 19:30:16 -07:00
hanzo-devandz 43958eb36e feat(shard): in-binary org→owner shard router for horizontal writer scale
Lifts the unified binary off replicas:1 on DigitalOcean (RWO-only block
storage, no RWX) WITHOUT any shared volume. Each org is pinned by rendezvous
hashing (ha.Owner over the static CLOUD_PEERS ring) to exactly one owner pod;
a request whose org this pod does not own is transparently forwarded to the
owner, so every per-org SQLite store (KMS, finance, and every org-keyed store),
audit append, per-org rate ceiling, and prepaid billing debit runs on ONE pod.

THE INVARIANT — no two pods ever write one tenant's SQLite file — is upheld by
two independent guarantees that compose: per-pod RWO PVC (different physical
files per pod) + org→owner routing (all of an org's writes on one pod).

- shardrouter.go: the middleware. Runs immediately after SanitizeIdentity so it
  keys on the VALIDATED, server-minted X-Org-Id (never a client header), hashes
  the SAME injective SanitizeOrg slug the on-disk path uses (routing key ≡ file
  key), and forwards via a fasthttp streaming proxy (SSE/chat pass through — a
  streaming chat still carries a per-org billing debit, so it routes too) with
  dial-only retry across an owner's roll gap and a 421 loop-guard on divergence.
  Static identical membership rules out dual-owner and routing loops.
- config.go: CLOUD_PEERS (id@addr ring) + POD_NAME (self). Sharding auto-enables
  only when >1 peer; Validate fails closed if self is not in the ring or if
  embedded iam (non-shardable process-local sessions) is co-enabled.
- serve.go: wires the middleware after IdentityMiddleware; shard-aware boot log.
- audit_serve.go: per-shard audit is automatic on the per-pod PVC; stamp the
  shard id on the AU-9 checkpoint stream so the tail-truncation monitor tracks
  N heads. writerpin.SingleWriter is correct PER SHARD (each pod sole writer of
  its orgs); the writer lease is pod-local under per-pod PVC and stays off.

No-op (byte-identical to today) when CLOUD_PEERS names ≤1 pod. N pinned at 3;
rebalance-on-N-change (a tenant-file move) is the documented follow-up.

Tests prove: exactly one owner per org (deterministic, total, evenly
distributed); all pods agree (no dual-writer); owned served locally, unowned
forwarded to the owner (local chain never runs, hop header set, response
streamed); org-less served locally; loop-guard 421; boot-gate fail-closed.
2026-07-16 19:28:27 -07:00
hanzo-dev bfcc9893a0 Merge: bill zen in the credit unit — FromInt(Minor()) understated every debit by 10^16 2026-07-16 19:26:12 -07:00
hanzo-dev ba34cae921 Bill zen in the credit unit, not the currency's minor unit
zen prices every SKU as an exact 18-dp value tagged money.USD. money.USD
declares 2 decimals, so Amount.Minor() rescales that value to CENTS, while
cloudmoney.FromInt reads its argument as 18-dp. Composing the two divided
every zen debit by 10^16: a $17.376 charge debited $0.000000000000001738,
and any charge under half a cent folded to a zero that metering.Record drops
before it reaches the ledger — no debit row, call served free.

The spend gate read the same composition. FromInt(est.Minor()).Cents() is
always 0, and AuthorizeVerdict only compares balance against size when
AmountCents > 0, so the size check was dead code and any org with a positive
balance could draw a request of any size. The dust debits never accumulated,
so the cap could not trip either.

Route the seam through one conversion. credit() carries the exact decimal
across and changes only the minor-unit convention — no rescale, no rounding,
no factor — because the decimal is the value and a currency's Decimals is a
rendering convention. cloudmoney.FromDecimal is the typed counterpart of
ParseUSD that makes this expressible without a string round-trip. nano()
takes the typed Amount rather than a bare *big.Int, so a cents integer can no
longer reach the warehouse fold.

The tests asserted Amount.Int() against Charge.Minor() — the same integer on
both sides — so they proved a round-trip and were blind to the unit. They now
assert against known dollar values built through a different constructor: a
$17.376 charge debits $17.376. Reintroducing Minor()-into-FromInt fails all
of them, including a gate test that admits an over-cap request.
2026-07-16 19:24:29 -07:00
hanzo-dev 164a2f9b56 Forward only the billing paths the console reads
The /v1/billing/* bridge attaches the commerce service token, and that token
satisfies commerce's MayMintMoney. The subpath was checked for traversal but
not against a set, so POST /v1/billing/deposit forwarded a mint for any
signed-in user — pinned to their own subject, which aims the mint rather than
stopping it. Commerce closed this on its direct path the day after the bridge
reopened it, and wider: the bridge needs only a validated principal, never an
admin bit.

The forwardable set is now a per-method table, consulted before the token is
attached, 404 on anything else. Per-method because payouts is a GET read and a
POST mint on one path — a method-blind set hands the mint to every reader.

Nothing reachable today: commerce's mint is not compiled into this binary and
commerce.hanzo.svc resolves here. But devnet runs a real mint-capable commerce
and points cloud-api at it under a second name for the same thing; unifying
those names would arm it. The gate should exist before that cleanup does.
2026-07-16 18:54:20 -07:00
blue 0c92f38c9d Gate the billing bridge on an allowlist: the service token is authorization
/v1/billing/* forwarded any subpath to commerce carrying the admin
COMMERCE_SERVICE_TOKEN, validated only against path traversal. Forwarding IS
authorization there: commerce gates money-mint on
MayMintMoney = IsServiceToken || IsSuperAdmin, so every forwarded path ran with
platform authority. Commerce 403s an org admin who POSTs /v1/billing/deposit
directly; through this bridge the same person was handed the platform's own
credential, and the subject-pinning aimed the credit at their own account.
Pinning is an IDOR control, not an authority control.

Add billingForwardable: a per-method allowlist of the endpoints the console
actually calls, enforced before the token is attached. An unlisted path 404s.
Allowlist, not denylist — a mint route commerce adds tomorrow is unreachable
with no change here. GET and POST are separate sets because `payouts` is a read
AND a mint-gated write; one method-blind set would hand the mint to every reader.
The POST set holds nothing that creates balance from a client-named amount.

Not exploitable in the current topology: commerce is co-resident in cloud, whose
build never compiles mount.go (//go:build cloud), so no api.Route billing bundle
is reachable and the forward loops back to this bridge. It is one COMMERCE_URL
away from live — devnet already runs a standalone commerce.

Tests: an ordinary org user's deposit/credit/refund/credit-grants/husd/allotment
now never reach commerce; the console's 12 real calls still forward; the store
bridge still cannot tunnel into billing.
2026-07-16 18:36:07 -07:00
hanzo-dev 08fc4ff7fb Embed zen v1.4.0 so the prompt-cache fix reaches production
zen v1.4.0 bills the prompt cache: cache_read was priced in the catalog,
published in /v1/models, and never charged. cloud pinned v1.3.11, so the
fix could not reach api.hanzo.ai, where the traffic lands.

The module graph cannot move: zen's own go.mod is byte-identical between
v1.3.11 and v1.4.0 (same sha256), and cloud holds exactly one zen edge.

NOTE: this is necessary but NOT sufficient. hmoney.Minor() returns cents
while cloudmoney.FromInt expects atto, so apps/zen.go understates every
debit by 1e16 and the spend gate reads AmountCents 0. Correct pricing is
still zeroed downstream. Tracked separately; that fix is what makes cache
billing real.
2026-07-16 18:21:12 -07:00
hanzo-dev 4efbb5faea openapi: serve GET /v1/openapi.json generated from the live router
The route table gets a third projection. /zap replays the /v1 handlers, the
console renders them, and GET /v1/openapi.json now describes them — all read
from the ONE router after MountAll, so none can drift and none holds a second
copy. There is no checked-in spec file and no second registry.

openapi.Live(app) reads app.Fiber().GetRoutes(true) (fiber's own filter drops
Use() middleware); every other function is pure over that []Route. Each
operation is tagged with its product — the first path segment after /v1/ —
so a CLI can build `hanzo <product> <resource> <verb>` with no judgment.

Reading the LIVE router is the only total source: POST /v1/kms/auth/login is
registered as Group("/v1/kms/auth").Post("/login") and no grep can find it,
and the route set is a function of deployment config, so the document varies
per deployment — correctly. Unauthenticated: it grants no capability, every
route it names stays auth-gated, and `hanzo --help` must build its tree
before login.

The drift guard (cmd/cloud/openapi_test.go) is a bijection over the fully
mounted apps.Wire(): 983 operations / 692 paths / 109 products, every live
route present, no operation invented. Shown to fail on a broken translation
(353 routes reported missing) before being restored.

Honest boundaries, asserted rather than papered over:
  - No schemas. The router holds func(*zip.Ctx) error; the request type is a
    local inside the handler (var req secretPutRequest; json.Unmarshal(...)),
    unreachable by reflection. cloud.Handle[S] is generic over the SERVICE,
    not the payload. The path to schemas is zip's typed ops, which today
    number zero — which is why zip's own generator emits nothing here.
  - No responses block. OpenAPI 3.1 makes it optional; fabricating 200/ok on
    ~900 routes would assert what nothing knows.
  - HEAD/CONNECT excluded — forced, not taste. CONNECT has no OpenAPI field;
    fiber auto-generates HEAD in startupProcess(), so including it would make
    the document depend on lifecycle stage.
  - Catch-alls are opaque: POST /v1/billing/deposit is not a route here.

Corrects LLM.md, which the code contradicted: Wire() does exist (apps.go:188),
MountAll does not sort, and byte-identical patterns MERGE rather than panic.
A high handler count is not a collision — app.Post(path, mw1, mw2, mw3, h) is
one registration with four handlers (apps/commerce.go:151), and 34 live routes
are that shape, so the generator never reads the count.
2026-07-16 18:14:23 -07:00
antje f3c2d2b98c cloud: mount hanzoai/agent /v1/agent (no-shim); delete dead clients/chat
clients/agent adapter injects the ai completion (in-process, billed) + tools.Default()
into github.com/hanzoai/agent; POST /v1/agent live path. Dispatch resolves the caller
via the ONE canonical tools.PrincipalFrom — no reconstructed principal. Builds green
on the v1.801.35 base. DEPLOY-BLOCKED: go.mod uses a local replace for hanzoai/agent
(CI needs a fetchable release + the local module-cache shallow-clone bug resolved).
2026-07-16 18:00:50 -07:00
hanzo-dev 491141790f Bill the prompt cache in production: zen v1.3.11 -> v1.4.0
cache_read was priced in the catalog and published in /v1/models, but never
charged: on an openai upstream every cached input token billed at the full
`in` rate, and on an anthropic one it left the bill entirely. zen v1.4.0
normalizes both dialects into one tally over three disjoint classes
(fresh + cached + cacheWrite == the whole prompt) and derives once per
response. cloud embeds zen via apps/zen.go, so the fix is inert at
api.hanzo.ai until this pin moves.

zen v1.4.0's go.mod is byte-identical to v1.3.11's, so the require edit and
the two go.sum lines are the whole change: no transitive pin moves, and
nothing else in the graph names zen.
2026-07-16 17:49:16 -07:00
hanzo-dev 8e6e81d6da Merge: fix the zen margin test's decimal + money types (unblocks CI on main) 2026-07-16 17:48:36 -07:00
z 5a8218bb53 apps: price the zen test with the decimal money actually uses
The zen margin test never compiled, so CI has been red on main since it landed:
it parsed prices with shopspring/decimal and handed the result to hmoney.New,
which takes hanzoai/decimal. Two identically-named types, one of them wrong.

    vet: cannot use d (struct type "github.com/shopspring/decimal".Decimal)
         as "github.com/hanzoai/decimal".Decimal value in argument to hmoney.New

Money has exactly one decimal. A second one that merely LOOKS like it is how a
price silently becomes a different number, which is why the compiler is right to
refuse. Parse with hanzoai/decimal (decimal.Parse — the same call zen itself
prices with); this was the only file in cloud importing shopspring.

Read the debit through Amount.Int(), not Amount.Minor(). The test held two money
types at once: zen's hanzoai/money.Amount (Charge/Cost) has Minor(), but
meterUsage returns cloud's clients/money.Amount, whose accessor is Int() — it
wraps Minor() and returns the identical *big.Int, so the assertions are unchanged.

The tests themselves are worth keeping: they prove the debit is the retail Charge
and never the upstream COGS, that a 3x-margin tier collects 3x, and that a
sub-cent call does not floor to zero. They just could not run.

Not run locally: ./apps links a prebuilt Rust staticlib (libhanzo_flags.a) that
is built in CI, not here. vet — the step CI actually failed — passes, and the
package builds.
2026-07-16 17:48:26 -07:00
hanzo-dev 1ebcd3259d GOPRIVATE names the namespace that is actually private
GOPRIVATE named zap-proto/*, which is public -- all 55 repos, all 6 modules
cloud needs served anonymously from the public proxy. The namespace that is
private went unnamed: github.com/hanzoai/* (467 private repos incl. ai,
account, commerce, orm, xorm, beego, csqlite). It resolved only by falling
through GOPROXY's direct fallback, and passed checksums only because go.sum
already pins everything.

containment.yml then set GOSUMDB=off to compensate. GOSUMDB does not scope to
a namespace: that disabled checksum verification for every module in the
build, public ones included, in the image that handles payments -- breaking
the invariant the Dockerfile three files away states and honors.

Public modules keep proxy and sumdb immutability; private ones go direct and
authenticated. Verified under the CI and Dockerfile env with -mod=readonly and
sumdb on: build 0, go mod verify all verified.
2026-07-16 17:33:16 -07:00
hanzo-devandantje b9856a4408 feat(sync): universal /v1/sync engine — GitHub/GitLab ⇆ native Hanzo Git
One reconcile loop, one place a sync happens. clients/sync owns the Sync
record (source/target/direction/trigger/cursor), a per-org sqlite `sync`
table, /v1/sync CRUD + /v1/sync/:id/run, and a kind→Provider registry with a
single Reconcile(sync,event) contract. Git is the first provider, composing the
existing git object-plane seams (InboundGitSync inbound, ImportGitRepo reconcile,
EnsureGitMirror outbound) — no second copy of any git op.

Triggers resolve to Syncs and enqueue the engine (cloud.Sync), never sync
directly: the GitHub App webhook now serves the path it actually fires at
(/v1/github-webhook, was a prod 404 at /v1/integrations/github/webhook) and hands
the verified push to the engine; the Gitea push webhook gains a loop guard
(skip pusher == GIT_SYNC_ACTOR). Loops break on the engine cursor (identical
fingerprints are a no-op) + actor guard; chained propagation (a sync's target is
another's source) is bounded by a hop limit.

Seams (root cloud): SyncFunc + RegisterSync/Sync, GitMirrorController +
EnsureGitMirror. Fail-closed on unmounted engine / missing secret. Tests
(CGO_ENABLED=0): engine loop-guard/idempotency/chain/hop-limit, git resolve,
CRUD+patch+run, webhook signature/isolation/enqueue, gitea loop guard.
2026-07-16 17:26:06 -07:00
hanzo-dev 97072d0443 build: GOPRIVATE names the namespace that is actually private
GOPRIVATE listed github.com/zap-proto/*. Every zap-proto repo is public — all 55
of them — and every zap-proto module this build needs is served by the public
proxy anonymously. It was never the reason anything resolved direct.

The namespace that IS private went unnamed: github.com/hanzoai/* — ai, account,
commerce, orm, xorm, beego, csqlite and ~30 more. Those only ever built by
falling through GOPROXY's `direct` fallback after the proxy 404'd them, and only
kept passing the checksum step because go.sum already pins them, so no sumdb
lookup happens. It worked by accident, one added dependency away from failing.

containment.yml compensated for that unnamed namespace with GOSUMDB=off, which
does not scope to hanzoai — it disables checksum verification for EVERY module in
the build, the public majority included. The Dockerfile's comment reasoned the
same way inverted ("hanzoai/* and luxfi/* are PUBLIC ... only zap-proto/* is
exempt"); hanzoai/* is largely private and luxfi/* (all 37 deps here) is public.

Naming github.com/hanzoai/* is what the off switch was standing in for. GOPRIVATE
implies GONOPROXY+GONOSUMDB for exactly that namespace, so the private modules go
direct+authenticated and skip the sumdb that cannot see them, while zap-proto and
luxfi keep the public proxy + checksum db — the immutable hashes that make a
force-moved tag unable to break or poison the build. GONOSUMDB and GOSUMDB=off
are dropped: scoped by GOPRIVATE, they are redundant, and blanket-off is a
supply-chain regression in a money image.

hanzoiam/* is not listed: 12b6e1c reverted the scim/saml/ldap embed, so nothing
in go.mod requires it. It goes back when the modules resolve, named as private.

Verified under the exact CI/Dockerfile env (GOPRIVATE=github.com/hanzoai/* only,
GOSUMDB=sum.golang.org, GOFLAGS=-mod=readonly): build exit 0, vet exit 0,
`go mod verify` = all modules verified, and `go mod download` resolves both a
public module (zap-proto/zip) and a private one (hanzoai/ai).
2026-07-16 17:21:31 -07:00
antje 563856fc88 deps: ai v1.816.0 -> v1.817.0 — global auto-routing switch is the '*' settings row, not ROUTER_ENABLED env (config-as-Base)
Runtime routing policy now sources from the GlobalDefaultOwner OrgSettings
row (admin.hanzo.ai-editable SQLite), env demoted to deprecated fallback.
Per-org override unchanged.
2026-07-16 17:16:33 -07:00
hanzo-dev d5266685d0 paas: read the kind the fleet runs on, and don't fight the git declarer
The PaaS control plane serves platform.hanzo.ai and read `services` only. The
fleet collapsed onto `kind: App` and this reader never followed: 69 App CRs run
across the scanned namespaces against 7 Service CRs, so the SUPERADMIN drift
board rendered 7 rows for a 69-app fleet — 1 row in `hanzo`, production, and that
row is a duplicate CR. It was not an error anyone could see. The board looked
plausible and was blind to 68 of 69 services; every read of an App-declared
service 404'd; and /v1/paas/health probed the Service CRD, found it served, and
reported ok — status theater over a blind board. clients/deploy already solved
this with an App-first/Service-second read order; this gives the same order to
the reader that needed it.

Both kinds are listed and deduped by name: one workload is one row even when an
App CR and a Service CR both claim the name, because a Deployment has one
controller ownerRef and the operator's Claim guard gives it to the App.

The write path is the harder half. Hanzo CD syncs 68 of the 69 App CRs from
universe `infra/k8s/operator/crs/` with selfHeal on, so patching an App CR here
is reverted on the next sync — the deploy would report success and silently roll
back. That is worse than refusing, so deploy and release now resolve which kind
holds the workload and refuse a git-declared one, naming the file to commit to.
A Service CR has no git declarer (cloud and kubectl write them directly), so it
still patches exactly as before — the tenant plane and the untransitioned CRs are
untouched. release.go's "no ArgoCD" claim was true when written and is not now.

This does not restore an admin deploy button for the git-declared fleet. Under
GitOps that button belongs in git, and ImageUpdate already names that seam
(registry→git→cluster). Whether /v1/paas/deploy should commit to universe or be
retired is a CTO call, not one to make silently inside a read fix.

Tests: 35 pass, 0 fail, including the pre-existing release suite. The fleet-sees-
App-CRs case is seeded to the shape of the real `hanzo` namespace.
2026-07-16 17:12:40 -07:00
antje 12b6e1c118 revert(iam2): back out scim/saml/ldap embed — hanzoiam repos do not resolve
Reverts 3aaf9d23. go.mod pinned github.com/hanzoiam/{ldap,saml,scim}, but none
of the three repos exist: https, ssh, and API all return not found, and ldap
was never fetched by any proxy or cache. Cold machines — including release
runners — fail go mod download, so main could neither build nor release.
Local builds passed only on module caches warmed 21:36-21:48 UTC today.

iam2 restored to v0.1.1, the state prod v1.801.38 ships. Re-land the embed
unchanged once the hanzoiam repos exist and resolve from a cold GOMODCACHE.
2026-07-16 17:01:45 -07:00
hanzo-dev 25cecf25a2 Merge: one name for the account, one for the org — and the last copy of the payer rule deleted 2026-07-16 16:38:08 -07:00
z ecc9b2ab54 billing: one name for the account, one name for the org
Two functions were called Payer and returned different things: account.Payer
returns the ACCOUNT that pays, principal.Payer returned the ORG whose ledger
holds it. Those are different values on the same request — a person in the shared
signup org pays from account "hanzo/alice" held in ledger "hanzo" — and one name
for both is how the gate came to key the pool while the debit spent the person.
Rename it to what it returns: principal.HomeOrg. An org names a ledger; an
account names a wallet within it.

Fix the last copy of the rule with it. clients/metering is cloud's vendored
metering client, and its IdentityFromGatewayHeaders still hardcoded `user := org`
under the same false premise, while its doc claimed cloud "mirrors" the module
"exactly so every product keys the SAME ledger entry" — a promise two independent
copies cannot keep. Both now call hanzoai/account.Payer, so they agree by
construction and the comment is true for a reason.

Its test asserted the divergence in words — "want hanzo (per-org billing key, not
org/sub)" — which is how a premise outlives the code that disproved it. It now
asserts equality with the rule.
2026-07-16 16:37:43 -07:00
hanzo-devandz 70e4ab8887 build(cloud): mirror base images to ghcr.io/hanzoai/mirror/* (kill public.ecr.aws 429s)
public.ecr.aws (ECR Public) rate-limits anonymous pulls with HTTP 429 on shared
CI runners; a 429 on any base pull aborts the release (a release died pulling
python:3.12-alpine). Repoint all five FROMs to 1:1 linux/amd64 mirrors in our
own GHCR namespace, pinned by digest for immutability:

  node    ghcr.io/hanzoai/mirror/node:24-alpine@sha256:0cb0e7c3195bce740b6c8d8b27432c92360e3b7f1528087f2c50640b177950c6
  python  ghcr.io/hanzoai/mirror/python:3.12-alpine@sha256:aa679aa4eed6eb56c1dc6ad3f1b98b7d2d788fd961596779d188fdedad97fb38
  rust    ghcr.io/hanzoai/mirror/rust:1-alpine3.22@sha256:b348cb409ac0a73de15065997a360063cf87465574a15e3e4469862cb8996f02
  golang  ghcr.io/hanzoai/mirror/golang:1.26-alpine3.22@sha256:47d47cb5cc3c7dac409dcb6c3a98a6263571218046cd02d709527feef804a77c
  alpine  ghcr.io/hanzoai/mirror/alpine:3.22@sha256:7c8cb692ae09657cbc4a3f3cbd0e8d5a2690ba38386aaaf252dbb060bf5eb2e6

The four the task named plus alpine:3.22 (the final-stage base — same registry,
same 429 exposure) so no FROM still hits public.ecr.aws. release.yml already
logs the build into ghcr.io (GH_PAT, docker/login-action) before building, so
buildx resolves these private mirrors today with no new plumbing. Only the five
FROM lines + one rationale comment change; nothing else in the Dockerfile.
2026-07-16 15:48:48 -07:00
hanzo-dev f9b2060faa feat(ai): bump ai v1.814.0 → v1.816.0 — router self-probe, flywheel trainer, casibase metering
Ships the full router flywheel into the cloud binary: the self-probe
(continuous tagged auto traffic → reward ledger), the fit→gate→auto-deploy
→publish trainer, the routing-latency guard (~0.68us heuristic), and
casibase-chat usage/o11y metering. Also carries the zen warehouse+span
wiring already on main.

Migrates the billing-subject callers (balance.go, account/billing.go +
tests) from the ai/object.Payer that v1.816.0 REMOVED to the extracted
github.com/hanzoai/account.Payer — the same rule in its new home (the
concurrent decomplect). Same replace directive for the force-pushed iam
pseudo-version account v0.2.0 pins.
2026-07-16 15:32:35 -07:00
z cfce8b10d0 probe: pin what zip typed ops can bind, before migrating 792 routes onto them
zip.Get[In, Out] is advertised as ONE op projected into three surfaces (REST ·
OpenAPI · MCP), and cloud's raw routes were slated to migrate onto it starting
with clients/agents. Measured first, against the route shapes cloud actually
has.

The registry works: registering one typed op populates /.well-known/openapi.json
and the /mcp tool surface, both of which are absent today only because cloud
registers zero typed ops (installOpenAPIRoutes/installMCP early-return on
len(a.ops)==0).

The binding does not. registerTyped's fiber handler passes c.Body() — nil for a
GET — into op.invoke and nothing else, and fiber's DefaultCtx.Context() returns
context.Background(). So a typed handler sees no path param, no query param, and
no header. 16 of clients/agents' 25 routes carry a path param and would receive a
zero In; the ?live/?host/?status/?agent filters would vanish; and the org, which
every agents handler reads via principal.Org(c) -> tenant(c) -> 403, is
unreachable. Migrating as-is would answer the wrong session for every :id route
and drop the authz gate on all of them.

MCP is the sharp edge: tool arguments arrive as the body, so MCP is the ONE
projection that DOES fill In, and mcpCall runs op.invoke with no identity at all.
A migrated org-scoped op would answer an anonymous caller, and the only way to
give it an org would be an org field in the typed In — caller-supplied, i.e. a
cross-tenant read. Both horns are unacceptable, so clients/agents does not
migrate at this zip version.

The org half needs no framework change: fiber's SetContext (already used by
TracingMiddleware) is honored by the ctx registerTyped hands to op.invoke, so
middleware can carry the validated org to a typed handler off the wire, over REST
and MCP alike, without an In field. TestPrincipalBridgeCarriesOrg proves it on
stock zip. Only URL binding is missing.

These are characterization tests: they pin the gap as the current contract and
fail with "UNBLOCKED: invert this test" the moment zip binds a URL, so the
migration restarts on a failing build rather than on someone remembering.
2026-07-16 15:14:25 -07:00
z 88453727c1 probe: pin what zip typed ops can bind, before migrating 792 routes onto them
zip.Get[In, Out] is advertised as ONE op projected into three surfaces (REST ·
OpenAPI · MCP), and cloud's raw routes were slated to migrate onto it starting
with clients/agents. Measured first, against the route shapes cloud actually
has.

The registry works: registering one typed op populates /.well-known/openapi.json
and the /mcp tool surface, both of which are absent today only because cloud
registers zero typed ops (installOpenAPIRoutes/installMCP early-return on
len(a.ops)==0).

The binding does not. registerTyped's fiber handler passes c.Body() — nil for a
GET — into op.invoke and nothing else, and fiber's DefaultCtx.Context() returns
context.Background(). So a typed handler sees no path param, no query param, and
no header. 16 of clients/agents' 25 routes carry a path param and would receive a
zero In; the ?live/?host/?status/?agent filters would vanish; and the org, which
every agents handler reads via principal.Org(c) -> tenant(c) -> 403, is
unreachable. Migrating as-is would answer the wrong session for every :id route
and drop the authz gate on all of them.

MCP is the sharp edge: tool arguments arrive as the body, so MCP is the ONE
projection that DOES fill In, and mcpCall runs op.invoke with no identity at all.
A migrated org-scoped op would answer an anonymous caller, and the only way to
give it an org would be an org field in the typed In — caller-supplied, i.e. a
cross-tenant read. Both horns are unacceptable, so clients/agents does not
migrate at this zip version.

The org half needs no framework change: fiber's SetContext (already used by
TracingMiddleware) is honored by the ctx registerTyped hands to op.invoke, so
middleware can carry the validated org to a typed handler off the wire, over REST
and MCP alike, without an In field. TestPrincipalBridgeCarriesOrg proves it on
stock zip. Only URL binding is missing.

These are characterization tests: they pin the gap as the current contract and
fail with "UNBLOCKED: invert this test" the moment zip binds a URL, so the
migration restarts on a failing build rather than on someone remembering.
2026-07-16 15:14:25 -07:00
hanzo-dev 94aa7b4791 ci(cloud): GOPRIVATE += github.com/hanzoiam/* (enterprise IAM modules)
The iam2 embed pulls github.com/hanzoiam/{scim,saml} (+ ldap under -tags iam2_ldap).
hanzoiam is a distinct org from hanzoai, so hanzoai/* did NOT cover it — the test
phase would route these private modules through the public proxy/sumdb and 404.
Git auth is already handled by the reusable CI's GH_PAT insteadOf (covers any
github.com private repo the PAT reads).
2026-07-16 15:04:30 -07:00
zeekay 3aaf9d2374 feat(iam2): wire SCIM + SAML into the cloud embed; LDAP GPL-isolated behind -tags iam2_ldap
Last code step of the iam2 migration. clients/iam2 blank-imports the Apache-2.0
enterprise features so their init() self-registers into iam2's feature registry;
iam2server.Mount -> feature.MountAll auto-mounts them under CLOUD_IAM_IMPL=iam2.
ONE mechanism (database/sql driver pattern) — no feature.Register call in cloud,
which (feature.Register appends with no dedup) would double-mount and collide routes.

  _ github.com/hanzoiam/scim   SCIM 2.0 provisioning  (/scim/*)
  _ github.com/hanzoiam/saml   SAML IdP + SP SSO      (/v1/iam/saml/*, /v1/iam/acs, /v1/iam/get-saml-login)

LDAP is GPL-isolated: hanzoiam/ldap links goldap (GPL-2.0), so it is opt-in only via
clients/iam2/ldap_enabled.go (//go:build iam2_ldap). The DEFAULT cloud binary stays
copyleft-free — proven: `go list -deps ./clients/iam2/` has no goldap; only under
-tags iam2_ldap does it pull github.com/lor00x/goldap + hanzoai/ldapserver.

deps: hanzoai/iam2 v0.1.1->v0.1.4; + hanzoiam/{scim,saml,ldap}; go mod tidy.
2026-07-16 14:59:47 -07:00
hanzo-devandz 2aeea28f7a feat(kms): one-time legacy ZapDB -> per-org SQLite migration (deploy-safe cutover)
Without this, deploying the per-org store over live data boots an EMPTY store and
orphans every secret — cloud KMS is the authoritative source the kms-operator syncs
out to every service, so that is a cluster-wide outage. New() now runs a one-time,
writer-only, keyed, FATAL-on-error migration BEFORE serving: it opens the legacy
{DataDir}/kms ZapDB (encrypted at rest with the master key), streams every SEALED
secret (kms/secrets/ prefix; the JSON value carries the full coordinate + AES-GCM
ciphertext + ML-KEM wrapped DEK) verbatim into its per-org SQLite file via store.put
— NEVER unsealing, so no plaintext is exposed and the AAD path-binding survives —
then archives the legacy dir to {DataDir}/kms.migrated so the OS lock is released for
good. Idempotent (prior .migrated marker or absent store = no-op; put upserts).

Tests: migrate roundtrip (opens to same plaintext), no-legacy-store no-op, and the
cross-org relocation defense still holds after migration. Full kms suite green (2
pre-existing admin-edge fails only).
2026-07-16 14:55:16 -07:00
hanzo-devandz 28ff703da0 feat(kms): per-org SQLite store — replaces OS-locked embedded ZapDB, lifts replicas=1
The embedded luxfi/kms ZapDB (a Badger fork) held an exclusive OS lock on ONE dir
for ALL orgs — the hard reason cloud ran replicas=1 behind a cross-process writer
lease. This replaces it with per-org encrypted SQLite ({DataDir}/orgs/{org}/kms.db
via cloud.OrgDB->cek, per-db DEK), which has no exclusive-opener lock, so distinct
tenants never contend and different pods can serve different tenants.

Crypto stays in the client (Seal/Open AES-256-GCM envelope, AAD-bound to the FULL
/orgs/{org} path): plaintext never reaches the file, and a record physically moved
into another org's file still fails to Open (cross-org swap defense preserved).
Adds cloud.PlatformDB for the reserved non-tenant (_platform) partition.

Verified: 54 PASS / 2 FAIL in clients/kms; the 2 fails (admin-edge dualmount/authz)
are PRE-EXISTING — proven failing identically on origin/main. concurrent_open_probe
proves per-org SQLite has no exclusive lock. Build + finance (pure-Go) green.

Branch only — NOT for main/deploy until red review passes.
2026-07-16 14:55:16 -07:00
zandhanzo-dev 6e417599c3 billing: the edge gate keys the account the debit spends
The gate keyed `user := home` — the org pool, always — on the premise that
prepaid billing is per-org. That premise is false. A person in the shared signup
org holds their OWN account: its members are strangers, not a team, and a shared
org is not a shared wallet. That is what IAM's signed billing_account claim states
and what ai's meter debits.

So the gate authorized against a balance nobody drained. Fund the pool and a
signup-org person still 402s, because their usage comes out of their own account;
fund the person and an empty pool blocks them anyway. Two layers, two answers, one
request.

Resolve through hanzoai/account.Payer — the same function ai debits with, on the
same credential — so the gate and the debit cannot name different accounts. The
premise is removed rather than restated.

The masquerade split is preserved by construction, not by care: the account is
resolved WITHIN the home org, so Account.Org IS the home org and a SuperAdmin
acting in another org still bills their own ledger. A claim naming a foreign
ledger is refused, so it cannot redirect a debit into the org being acted on.

Tests assert against the rule rather than a constant, so they cannot drift the way
the premise did: the signup-org person keys their own account, a real org pools,
the claim wins for a person and a project, and every case is checked equal to what
the debit computes.
2026-07-16 14:42:34 -07:00
hanzo-dev f00f549739 merge: iam2 embed subsystem — CLOUD_IAM_IMPL selects beego (default) or iam2
clients/iam2 mounts the clean-room zip+orm IAM at /v1/iam when CLOUD_IAM_IMPL=iam2,
else beego, byte-for-byte unchanged. Flag OFF by default → inert. iam2 v0.1.1 seam.
2026-07-16 14:38:03 -07:00
hanzo-dev 18537ce994 feat(iam2): select /v1/iam impl via CLOUD_IAM_IMPL (default beego, unchanged)
apps.Wire's identity slot now calls identitySpec(): CLOUD_IAM_IMPL=iam2 mounts the clean-room iam2 (zip+orm), anything else — including unset, the production default — keeps the legacy beego Casdoor embed byte-for-byte. The two impls own the SAME absolute prefixes (/v1/iam/*, /login/oauth/*) and cannot co-mount, so exactly one occupies the slot per boot and mount order is preserved. Off by default => completely inert until a canary flips the flag; selection (this) stays orthogonal to activation (cfg.Enabled).
2026-07-16 14:32:15 -07:00
hanzo-dev 509d0f8ff3 feat(iam2): clean-room IAM v2 embed subsystem (zip+orm, beego-free)
clients/iam2 folds the beego-free Hanzo IAM v2 into the unified cloud binary as an in-process identity plane — the either/or twin of clients/iam. Matches the cloud.Typed contract func(*zip.App, cloud.Deps) error: cloud hands subsystems a cloud.Deps (not an orm.DB), so Mount opens its OWN embedded SQLite ({DataDir}/iam2/iam.db, mirroring the beego embed's {DataDir}/iam layout), seeds config new-only+idempotent from the SAME init_data.json the beego iam uses (non-fatal, honest degrade), then iam2server.Mount registers the whole surface at the canonical absolute paths.

Fail-closed like clients/iam: a store-open or mount failure serves 503 on the identity prefixes while every co-resident subsystem stays up; iam2server.Mount's only panic path (a registered enterprise feature) is recovered in safeMount so it never crashes the shared binary. A TODO marks where the parallel-lane hanzoiam/{scim,saml,ldap} feature.Register lines land.

Pins github.com/hanzoai/iam2 v0.1.1 (seam held stable across the parallel internals refactor); transitive MVS bumps are all patch-level within v1.x (zip 1.8.3, orm promoted to direct, luxfi/crypto 1.20.1, argon2id 1.0.0, pgx 5.9.2). Inert until wired — see the apps.Wire gating follow-up.
2026-07-16 14:32:06 -07:00
hanzo-dev dc9cb10b43 fix(ci): bound the SECOND version scan too (compute step) — same unbounded --paginate
The compute-next-version step had the same full-registry --paginate as the tag
step (fixed in b72394f). It ran first, so it could hang before the build. Bound
it to one page too. Both version scans are now O(1 page), not O(registry).
2026-07-16 14:28:00 -07:00
hanzo-dev 10362142a1 feat(zen): warehouse + gen_ai span emission with exact margin — ai v1.814.0
zen's commerce Meter now also calls ai's TraceServedUsage (recordTrace
WITHOUT recordUsage — the commerce debit stays the ONE billing source,
never doubled), carrying zen's exact per-tier retail (Charge) and upstream
COGS (Cost) folded atto→nano, so zen* traffic in the unified binary lands
in hanzo.cloud_usage + the o11y span plane with TRUE margin instead of
being warehouse-blind. Rides the ai v1.813.1→v1.814.0 bump; balance.go
(+ its drift-guard test) migrated to the renamed Payer/PayerOf API —
same subjects, one rule.
2026-07-16 14:26:17 -07:00
zandGitHub 6961df4901 Merge pull request #315 from hanzoai/cloud-zen-family-events
feat(zen): embedded-zen meter writes the family RoutingEvent (last link) + ai v1.813.5/zen v1.3.11
2026-07-16 14:08:50 -07:00
hanzo-dev b64c2daa65 feat(zen): embedded-zen meter writes the family RoutingEvent (the last link)
The embedded zen mount serves the zen catalog in-process and never reaches ai's
pipeToFamily, so zen* calls produced ZERO routing events — starving stats, world,
spark retrain, and /v1/feedback joins. Wire cloud's zen Meter to ALSO write the
RoutingEvent through the ONE shared writer object.RecordFamilyRouting (source="family",
served arm = zen.Usage.Upstream, join key = zen.Usage.ResponseID — the client-visible
response id, new in zen v1.3.11 — tokens + retail cost), fire-and-forget beside the
existing debit. Bumps zen v1.3.7 → v1.3.11 (Usage.ResponseID); ai already v1.813.6
carries object.RecordFamilyRouting.
2026-07-16 14:08:28 -07:00
antje 2763842079 git: Gitea push-webhook ingest (POST /v1/git/webhook)
The external Hanzo Git server (Gitea fork, git.hanzo.ai) POSTs push events
here so a push landing on it drives the SAME push-to-deploy core the embedded
smart-HTTP receive-pack path drives: fireBranchBuild -> cloud.OnGitPush deploy
trigger + EmitLifecycle. One code path, no duplication.

HMAC auth: X-Gitea-Signature is hex HMAC-SHA256 of the raw body, verified
constant-time against GIT_WEBHOOK_SECRET (KMS-synced hanzo/prod:/git/webhook-secret).
Fail-closed 401 on unset secret or mismatch. Only X-Gitea-Event: push acts;
others 204. Zero-SHA / non-branch refs are no-ops.
2026-07-16 12:05:08 -07:00
zandGitHub 670153907d Merge decomplect/account-payer: migrate to ai.Payer — unbreaks main, ends the self-serve 402
main pinned ai v1.813.6 (which deletes billing_subject.go for the Payer/Account
refactor) without migrating the call sites, so main did not compile:
  clients/billing/balance.go:59: undefined: aiobject.BillingSubject

This migrates the call sites to the ONE rule: Payer(Credential)->Account
(owner = Person|Org|Project). Ends the live 402 where a self-serve signup's top-up
minted to the shared org pool 'hanzo' while the gate debited 'hanzo/alice' ($0) --
customer paid, locked out, money in a pool their org-mates could spend.

Green: go build -tags 'libsqlite3 sqlite_fts5' ./clients/... rc=0;
go test ./clients/billing ./clients/account rc=0. go.mod/go.sum identical to main.
2026-07-16 11:51:27 -07:00
hanzo-dev e081547ad3 fix(billing): migrate off deleted BillingSubject → ai.Payer/PayerOf — unbreaks main
main pins ai v1.813.6, which contains the Payer/Account refactor (billing_subject.go
and its PERSONAL_BILLING_ORGS/ORG_BILLING_ORGS lying default are deleted). But the
call sites were never migrated, so main does not compile:
  clients/billing/balance.go:59: undefined: aiobject.BillingSubject
  clients/billing/balance.go:61: undefined: aiobject.BillingSubjectFromUserKey

Migrates balance.go + finance.go + billing.go to the ONE rule —
Payer(Credential{Owner,Name}).Subject() / PayerOf(org,key).Subject() — and DRYs the
duplicate subject resolvers. This is the cloud half of the fix that ends the live
self-serve 402 (top-up minted to the shared org pool 'hanzo' while the gate debited
'hanzo/alice' = $0).

go.mod/go.sum untouched vs main. Green: go build -tags 'libsqlite3 sqlite_fts5'
./clients/... rc=0; go test ./clients/billing ./clients/account rc=0.
2026-07-16 11:50:51 -07:00
hanzo-dev adfca0b3fe refactor(billing): route top-up + console subject through ai.Payer, one rule
The console top-up (clients/account/billing.go) and the finance read
(clients/billing/finance.go) each re-implemented the billing subject as "always
the org" — the reverted lineage's rule. Against ai's gate, which bills a signup
person per-person, that is the split: money minted to subject "hanzo" (the pool)
while the gate debited "hanzo/alice" ($0) → the paid-up member 402'd.

Delete both twins; resolve the subject through the ONE rule, ai/object.Payer,
keyed on the IAM username (X-User-Name) the gate also keys on. Top-up credits and
console reads now land on the SAME account the gate debits — they cannot drift
because there is one function. The killed PERSONAL_BILLING_ORGS / ORG_BILLING_ORGS
env is inert here too (test proves hostile values change nothing).

Requires ai v1.809.5, which must be tagged FROM ai main after decomplect/account-payer
merges — NOT off a branch. The prior one-rule fix was tagged off an unmerged branch
(v1.806.8/.9), main never got it, and every later tag resurrected the allowlists;
that is why this bug is live. go.sum refreshes via `go mod tidy` once the tag exists.
Verified locally via a replace to the ai branch: clients build + twin tests green.
2026-07-16 11:47:10 -07:00
antje f4f506f9be ci(release): retire notify-universe — Hanzo CD owns git→cluster sync
The repository_dispatch deploy hub is gone (universe image-update.yml
removed in universe d07cf945; the dispatches were silently suppressed by
the flagged sender account regardless). Deploys are declared-tag bumps in
universe crs/, synced by Hanzo CD (ArgoCD, ns hanzo-cd) and reconciled by
the operator. [skip ci]
2026-07-16 11:36:10 -07:00
antjeandGitHub 7e9bc187f1 deps: hanzoai/ai v1.813.1 -> v1.813.6 (balance-exempt routing config + enso family helper) (#314)
ai#102: /v1/get-routing-defaults + org-settings CRUD + routing-ledger
export are configuration metadata, never wallet-gated — unblocks reading
org routing defaults for $0-balance orgs and the operator platform flip.
Edge auto-routing billing tests green.
2026-07-16 11:28:33 -07:00
antje 69e21ecff1 platform: allow the self-hosted fleet registry in the native build lane
registry.hanzo.ai/{hanzoai,luxfi,zooai}/ join the /v1/runner push allowlist —
Wave 0 of the native CI/CD migration. Until now only release.yml's crane
mirror could reach the fleet registry; the native BuildKit lane was
ghcr-only by policy.
2026-07-16 11:13:07 -07:00
hanzo-dev b72394f461 fix(ci): version-assignment scanned the WHOLE registry — livelocked releases
The "atomic free-version assignment" step paginated every container version
(`gh api --paginate .../versions`) to find the max release number. As the
registry accumulated tags this grew unbounded and hung the step for 30+ min,
livelocking every cloud release. Container versions are created newest-first and
version tags are monotonic, so the max is always on the newest page — query one
bounded page (?per_page=100) instead of the full history. Fast + correct.
2026-07-16 11:11:39 -07:00
hanzo-dev e07d454096 refactor(ledger): drop redundant "core" — the ledger adapter is apps.ledger
Values, not places: the finance-backed credit-ledger adapter is qualified by its
namespace (apps.ledger), not a braided ledgercoreCredit compound. Rename the type
ledgercoreCredit → ledger, the file commerce_ledger.go → ledger.go, and scrub
"ledgercore" from prose (the ledger IS the core — "core" adds nothing). No behavior
change; admin/core tests green, changed packages build.
2026-07-16 10:18:47 -07:00
antje a2812e1505 feat(chat): clients/chat — one /v1/chat tool-calling orchestrator
POST /v1/chat runs one LLM tool-calling round that lets a model manage a system
via tools. Composes existing cloud pieces, reinventing nothing:
- LLM routing + per-org reserve/settle billing: the ai subsystem's
  /v1/chat/completions, invoked in-process (Fiber Test) — the only path that
  both returns tool_calls AND carries the billing gate.
- tool plane (clients/tools): the org's registered MCP/registry tools are
  offered to the model and dispatched server-side (activation + price gated).
- capabilities: graph (advisory node-ops -> ops the client applies) and create
  (server-executed, tools = the org's registered MCP render services).
Returns {reply, actions, ops}. chat mounts before ai so /v1/chat resolves here
(the ai /v1/chat alias is shadowed); ai keeps /v1/chat/completions.
2026-07-16 10:02:29 -07:00
hanzo-devandantje 630722d168 feat(ai): bump ai subsystem v1.813.0 → v1.813.1
Carries the PAID-Enso revert + the family learning loop: per-family-call RoutingEvents
+ shadow A/B (records what the learned engine would have picked), /v1/feedback signal
contract (up/down/regenerate/switch/abandon/accept/revert/rating/dismiss) with the
online reward forward to the engine's /route/observe, ROUTER_ADMIN_TOKEN service-auth
on the training-data exports, and Zen/Enso provider branding. Lights up /v1/router/stats
+ world.hanzo.ai (shadow-vs-served agreement).
2026-07-16 09:53:56 -07:00
hanzo-dev 404486a479 feat(ai): bump ai subsystem v1.812.0 → v1.813.0 — router policy, rewards, margin, DO backfill
Ships the per-org router policy (get/update-router-policy + the org > '*' >
conf fold on every auto route), the routing-reward ledger, the nano margin
ledger (costNano/billedNano/marginNano + unpriced flagging), and the
gaps-only DigitalOcean usage backfill (POST /v1/admin/usage/backfill-do,
dry-run default). The release image also re-embeds console@main
(CONSOLE_REF=main), picking up the console Router page (v8.4.137+).
2026-07-16 09:28:20 -07:00
hanzo-dev 8fff1a33cc feat(credit): ONE ledger seam — commerce credit + admin grant mint into finance.Current()
The last mile of "one way to grant credit": cloud implements commerce's
creditledger.CreditLedger (v1.48.5) over the native finance ledger and injects it
at mountCommerce (EmbedConfig.Ledger). Now commerce's POST /v1/billing/credit AND
the admin.hanzo.ai grant (/v1/admin/customers/:org/credit) both mint into the SAME
per-org finance wallet the ai prepaid gate reads — a granted credit is immediately
spendable, no split ledger. The admin path drops its parallel finance.Deposit for
the one creditledger.Credit call (idempotency key rides through; commerce HTTP
deposit remains the split-deploy fallback).

- apps/commerce_ledger.go: ledgercoreCredit adapter (compile-time asserted against
  commerce's exported interface); org-pool wallet (Subject==Org); fails closed with
  no co-resident finance.
- apps/commerce.go: inject Ledger: ledgercoreCredit{} at mountCommerce.
- clients/admin/core/grant.go: grantDeposit prefers creditledger.Get() (the one
  ledger) over its own finance.Deposit.

Verified: changed pkgs build green on commerce v1.48.5 + ai v1.812.0; admin/core
grant tests pass. Ships with the ai free-tier removal (v1.811.0+) already on main.
2026-07-16 09:14:59 -07:00
antje 2db0bcf87f ci(release): pass GIT_AUTH_TOKEN build secret — Dockerfile renamed the id from gh_token (45387fb7) but the workflow still passed gh_token, so the private console clone ran credentialless and the release failed 2026-07-16 01:50:00 -07:00
antje 45387fb70f build: consume the standard GIT_AUTH_TOKEN build secret (was gh_token)
One secret id everywhere: BuildKit's gitsource convention (GIT_AUTH_TOKEN,
which the fabric's buildFrontendCmd already attaches) is also what the
Dockerfile's console-embed and go-mod fetch stages mount. gh_token was a
second name for the same credential.
2026-07-16 01:19:03 -07:00
antje efc1c5152d deps: hanzoai/ai v1.812.0 + commerce v1.48.5 — reward ledger, router policy, connections import reach prod
ai v1.809.4 → v1.812.0: RoutingEvent reward ledger (/v1/add-routing-reward,
/v1/export-routing-rewards), per-org router policy + observability, Enso
public arms + retrain-status, connected-provider usage import (OpenAI/
Anthropic), own-brand god-view gate (TokenIsOwnBrand), Bearer-aware
get-cloud-usages, billing nano margin ledger, unpriced-model flagging.

commerce v1.48.2 → v1.48.5: mint-gated org-keyed credit primitive +
injected-ledger seam; fixes the phantom luxfi/cevm test import that broke
go mod tidy for every consumer (this repo included).
2026-07-16 01:16:12 -07:00
antje 2cbbbb31ec feat(fabric): GIT_AUTH_TOKEN build secret — private-repo fetches for the build Jobs
The buildkit Job now surfaces the console-git-token Secret (optional) as the
GIT_AUTH_TOKEN env, and buildctl attaches it as the same-named build secret;
BuildKit's gitsource presents it as the HTTPS credential for the git context.
Fixes the fabric's inability to build PRIVATE repos (github.com/hanzoai/cloud
itself: 'could not read Username' — the rel_HvInGKU failure). Public repos and
Secret-less clusters fetch anonymously exactly as before.
2026-07-16 01:15:07 -07:00
antje f459e889d5 feat(git): public repos — anonymous read for the build fabric
A repo gains a visibility bit: PATCH /v1/git/repos/:name {"public":true}
(or create-time "public"). Public grants ANONYMOUS upload-pack/info-refs
only — receive-pack and the whole control plane stay org-authed, and a
private or missing repo answers the same uniform 404 so anonymous probing
cannot enumerate.

This is what lets the credential-less buildkit build Job (launchDirectBuild
git context) fetch from the embedded git server — the same reason public
GitHub repos build with no env. Private-repo builds remain a later
GIT_AUTH_TOKEN feature.

resolvePackRepo grows the allowPublic branch (fetch-side only): anonymous
callers address org-level repos by the orgRE-validated :org path segment;
the path-vs-identity guard for authenticated callers is unchanged.
2026-07-16 01:05:30 -07:00
antjeandGitHub d6b4361822 test(platform): end-to-end git push -> buildFromPush enqueue proof (#311)
Wire a REAL smart-HTTP git push through the actual production seam
(cloud.RegisterPushBuilder <-> cloud.OnGitPush) into the real platform
push builder, and assert the matching app's build is enqueued (a
building deployment for the pushed commit). The two halves were covered
in isolation (clients/git TestPushFiresBuildTrigger; platform
TestBuildFromPush_LaunchesMatchingApp) but nothing connected a live push
to the real builder end to end. Pure-Go dev build (CGO_ENABLED=0).
2026-07-16 00:45:07 -07:00
hanzo-dev b02257731f Merge land/slack-on-bridge: Slack chat path onto the generalized bridge — one chat code path 2026-07-16 00:15:57 -07:00
hanzo-dev 42e6bf84a1 refactor(integrations): Slack chat path onto the generalized bridge — one chat code path 2026-07-16 00:13:59 -07:00
zandGitHub f469043773 Merge feat/bots-control-plane: decomplect the bot plane, proxy instead of duplicate
Deletes clients/bot — a place-name that had collected three concerns (a mount, a
transport, and two unrelated domain wire protocols). One value per module:
  run       -> clients/bots      /v1/bots (no store)
  dispatch  -> clients/coding    /v1/coding-tasks
  transport -> clients/runtime   address, identity, framing, cleartext policy
  machine   -> clients/visor     /v1/compute/bots (moved off /v1/bots)
  session   -> clients/agents    the one registry (reverted to main)

Fixes a live route collision: visor's listBots silently won GET /v1/bots (the
router merges byte-identical patterns, first wins), so cloud served machine rows
as runs and orgs' real runs were invisible.

POST /v1/bots/run now returns 501 instead of charging $1.00 for a bot that never
booted -- the runtime has no launch endpoint. Stop fails closed: a bare 404 is
502, only a structured error body means already-stopped.

Net -613 lines. runtime never imports bots/coding; the reverse is a compile-error
cycle, so the one-way dependency is structurally enforced.
2026-07-15 23:27:48 -07:00
zandGitHub afbf789dc6 Merge fix/billing-balance-collision: read prepaid balance from the finance ledger, not a self-call
Cloud's /v1/billing/balance proxy was the only handler registered (commerce's
GetBalance is behind //go:build cloud, never compiled in) so it called itself,
depth 2. Reads the finance ledger via the existing finance.Current() seam and
single-sources the subject from aiobject.BillingSubject so console and gate
cannot drift.
2026-07-15 23:27:02 -07:00
zandGitHub 5d56b479ea Merge pull request #310 from hanzoai/bump/ai-v1.809.4
chore(ai): bump embedded ai v1.809.3 -> v1.809.4 (enso comp cold-cache fix)
2026-07-15 23:07:00 -07:00
hanzo-dev a0d75f185f chore(ai): bump embedded hanzoai/ai v1.809.3 -> v1.809.4 (enso comp cold-cache fix) 2026-07-15 23:06:41 -07:00
hanzo-dev c65427feaa bots: proxy the runtime instead of copying its state; name the transport
Red found the substrate was wrong: cloud kept its own registry of runs. It minted
an id the runtime had never heard of, so /v1/bots listed runs that did not exist,
stop closed records for runs that were never started, and run charged $1.00 for a
bot that never booted — the runtime has no launch operation at all, and nothing
in run ever contacted it. The registry that exists is the runtime's own tenant
store; it is the only thing that knows whether a sandbox is alive.

So cloud owns policy and the runtime owns the run:
  - GET /v1/bots and POST /v1/bots/:runId/stop proxy the runtime, gated by the
    validated principal and org. The org is cloud's, never the client's, and the
    runtime keys every run under tenants/{org}/ — so a foreign run resolves under
    the caller's org, where it does not exist, and 404s.
  - POST /v1/bots/run returns 501. There is no launch operation to call, so the
    honest answer is that it is not implemented. It no longer charges.
  - The agents session plane is reverted verbatim: no surface column, no Agent
    filter, no SessionOpen, no in-process stop. It never should have carried a
    second copy of the runtime's state.

Absence is only meaningful from a callee that could have said otherwise, so the
transport separates ErrNotFound (the operation ANSWERED absent) from ErrNotServed
(no such operation). A runtime without the stop route reports absent for every
run; treating that as "already stopped" made a stop that cannot fail. It is 502.

Decomplect the transport: clients/bot was named for the host it dials and braided
three concerns — an ops face plus two unrelated domain protocols. It is now
clients/runtime, a domain-free transport (address, identity, framing, cleartext
policy, /v1/bot/* ops face) that must not import bots/coding. Each domain owns its
own wire stub (bots/wire.go, coding/task.go), so the ZAP swap per HIP-0106/0120 is
a seam swap. Wire spec bot -> runtime; cmd/bot -> cmd/runtime.

The duplicate-route guard was a no-op: the router MERGES byte-identical patterns
into one route with chained handlers, so counting entries never saw the collision
it guarded. It now asserts one handler per route, and a test proves it fires on
the original bug. Fiber's Test() defaults to a 1s wall-clock deadline, which made
the isolation guards flake under load; they now pass an explicit timeout.

Route table unchanged: /v1/bots (runs), /v1/compute/bots (machines), /v1/bot/*
(runtime ops).
2026-07-15 22:52:24 -07:00
hanzo-dev aad2bd189c feat(git): JSON read/browse surface for the console repo-browser
The console repo-browser (hanzoai/console products/git) needs machine-readable
refs/tree/blob/commits/readme, but git only served those as HTML (ui.go) + the
repo-CRUD control plane. Add the JSON twin, reusing ONE set of go-git read
helpers so the HTML + JSON surfaces can never drift:

  GET /v1/git/repos/:name/refs                   → { branches, tags, default }
  GET /v1/git/repos/:name/tree?ref&path          → { entries:[{name,path,type,size,mode}] }
  GET /v1/git/repos/:name/blob?ref&path          → { path,size,encoding,content,binary,truncated }
  GET /v1/git/repos/:name/commits?ref&path&limit → { commits:[{sha,shortSha,message,author*,date}] }
  GET /v1/git/repos/:name/readme?ref             → { path, content, encoding }

- browse.go: the handlers + DTOs (mirror the console GitApi normalizers verbatim).
  ref+path ride as ?ref=&path= query params (the UI's own convention) so a slashed
  branch is unambiguous. Reuses org()/findRepo()/openGit()/resolveRef()/
  cleanTreePath(). Org-scoped (X-Org-Id); a repo outside the caller's org 404s.
  Distinct trailing segments — never shadow the :org/:repo smart-HTTP routes.
- ui.go: findReadme → readmeAt (returns filename+content); the ONE readme scan now
  backs both the HTML repo home and the JSON /readme (DRY, no duplication).
- browse_test.go: seeds a nested tree + README + binary file, asserts every endpoint
  + org isolation + honest 404/403 + empty-repo refs.

CGO_ENABLED=0 go test ./clients/git/ green (full package); go vet + build clean.
Stays v1.x.x (Go module). Ships with console@main on the next cloud release.
2026-07-15 22:23:09 -07:00
hanzo-dev 4b5c482051 fix(billing): read the prepaid balance from the finance ledger, not a self-call
GET /v1/billing/balance and /v1/finance/balance proxied to commerce at the SAME
path they are registered on. commerceinproc publishes the shared zip app and
re-dispatches by path, and commerce's own /v1/billing/* routes are never
registered in this binary (api.Route runs only from commerce's mount.go, which
is //go:build cloud; cloud ships -tags "libsqlite3 sqlite_fts5"). So the proxy
re-entered itself, hit its own sign-in gate with no principal, and reported
"billing upstream status 500".

Co-resident, the prepaid wallet is cloud's own finance ledger: wireFinance points
the ai gate's balance read at it, the edge meter debits it (metering.fetchAvailable
already resolves finance.Current() first), and an admin grant credits it
(core.grantDeposit already prefers it). Read it directly through that same seam.
The commerce S2S read stays as the split-deploy fallback.

The subject comes from ai/object.BillingSubject — the function the ai prepaid gate
itself resolves — instead of a re-implemented copy. cloud and ai each keeping their
own copy of that rule is what let them drift: the console scoped to the org while
the gate scoped to "org/user", so the view showed a funded org while the gate
refused the member.

Fail posture unchanged: a balance that cannot be read is UNKNOWN and surfaces as
502 — never rendered as a zero balance. The sign-in gate and org scoping are
untouched; the org still comes from the validated principal only.

Tests: self-dispatch pinned at depth 2 (the seam's real mechanics, which the
existing SetHandler-stub test cannot see); router semantics probed (byte-identical
patterns MERGE and silently shadow; equal-specificity param-name conflicts PANIC at
registration; most-specific wins over registration order); balance regression,
gate-subject parity, unreadable-is-not-zero, sign-in gate, and cross-org isolation.
2026-07-15 22:16:46 -07:00
antje 02ffc01b58 feat(insights): /v1/insights — the unified native surface on the ONE analytics engine
A wire adapter, not a second pipeline: POST /v1/insights/e accepts PostHog-
shaped payloads (single or {batch}) from @hanzo/insights and any compatible
SDK, maps $-properties onto the native CaptureEvent, and rides the SAME
capture path (tenant gate -> normalize -> scrub -> hanzo.events). GET
/v1/insights/events is the console's tenant-scoped recent-events read; GET
/v1/insights/health reports the surface. Flags deliberately stay at /v1/flags.

Scale path stays stateless: accept on any replica, pooled batch INSERT sink;
the queue-buffered (mq/pubsub -> Datastore consumer) upgrade swaps the exec
behind buildEventsInsert with no handler changes.
2026-07-15 21:35:30 -07:00
zandGitHub 20e7b148fc Merge pull request #309 from hanzoai/bump/ai-v1.809.3
chore(ai): bump embedded ai v1.809.2 -> v1.809.3 (comped preview balance fix)
2026-07-15 21:30:55 -07:00
hanzo-dev fb9fe44107 chore(ai): bump embedded hanzoai/ai v1.809.2 -> v1.809.3 (comped preview bypasses enforceBalanceGate) 2026-07-15 21:30:09 -07:00
hanzo-dev ec2105f75e Merge land/hybrid-connectors: injective tenant S3 key (close cross-tenant collision) + orgPathSafe guard 2026-07-15 21:21:56 -07:00
hanzo-dev 04bee3d212 fix(projects): injective tenant S3 key via verbatim principal.Org (close cross-tenant collision) + orgPathSafe traversal guard 2026-07-15 21:21:25 -07:00
hanzo-dev d251389f19 bots: state the list cap explicitly
The published contract has no pagination, so the list is capped either way; set
it at the store maximum here rather than inherit the store's 100-row default.
2026-07-15 20:52:36 -07:00
hanzo-dev ecae57354c bots: give the three "bot" values one namespace each; make the run control plane native
GET /v1/bots was registered twice: clients/visor (bot machines) and clients/bots
(bot runs). The router resolves byte-identical patterns by first-registration
without panicking, and visor mounts first, so visor's machine list answered the
console's run list and clients/bots.list was unreachable. The console normalized
machine rows into run rows, yielding one blank-runId row per kind=bot machine
with a dead sessionUrl, and hid the org's real runs.

Name the values apart, one home and one route namespace each:
  - bot run     -> clients/bots  /v1/bots            (unchanged; the console + CLI contract)
  - bot machine -> clients/visor /v1/compute/bots    (moved; a machine is compute)
  - bot runtime -> clients/bot   /v1/bot/*           (passthrough for runtime-owned ops)

Make the run control plane native. clients/bots holds no store: a run is recorded
on the agents session plane under agent label "bot", so the run id is the session
id and one registry serves every kind of agent work. list reads it org-scoped;
stop resolves (org, runId) against that record and drives the runtime only after
ownership is proven, so a run of another tenant is a 404 the runtime never hears
about. An unreachable runtime is a 502 with the record left live.

Two seams (Runs, Runtime) are injected in adapters.go, the only file in
clients/bots importing agents/bot, so handlers unit-test against fakes.

agents: sessions carry a surface (the modality a session runs on, matching the
runtime's own origin.surface) via the established additive-column migration;
SessionFilter gains Agent so a product face reads only its own sessions;
OpenSession takes SessionOpen; StopSession is the single-session twin of
StopSessions and shares its one write path (stopOne).

Contract change: a run id is now the session id, not bot_<hex>. No bot_ id was
durable anywhere -- the old run handler minted an id and stored it nowhere -- so
there is nothing to migrate. Ids stay opaque to clients.
2026-07-15 20:38:54 -07:00
antje 6079ee5f32 cli/gpu: heartbeat the claimed activity + fleet presence DURING a render (goroutine in claimAndRun) — a >120s render no longer drops the machine offline mid-render; fixes BYO-GPU fix/compose/render hitting the deadline + going offline 2026-07-15 20:21:31 -07:00
zandGitHub 74cea96986 Merge pull request #308 from hanzoai/bump/ai-v1.809.2
chore(ai): bump embedded hanzoai/ai v1.809.1 -> v1.809.2 (Enso limited-preview gating)
2026-07-15 19:19:07 -07:00
hanzo-dev 7204e96c27 chore(ai): bump embedded hanzoai/ai v1.809.1 -> v1.809.2 (Enso limited-preview gating)
Lands the Enso limited-preview gating into the live api.hanzo.ai binary
(cloud embeds ai in-process via ai.Mount). v1.809.2 adds ai's Enso family
routing (ENSO_URL -> enso pod) + waitlist gating: ModelAccess visibility in
/v1/models, 403 request-access for ungated SKUs, comped bypass of the balance
gate for granted callers, POST /v1/models/:model/access, org `hanzo` seed.

Independent of the concurrent zen v1.3.0 -> v1.3.7 bump (#307), which this
branch is based on top of.
2026-07-15 19:16:00 -07:00
hanzo-dev 840dacc659 Merge land/chat-connectors: generalized ChatBridge + Discord/Teams/Telegram adapters for hanzo.chat 2026-07-15 19:11:41 -07:00
hanzo-dev 81f495db18 land(chat): generalized ChatBridge + Discord/Teams/Telegram adapters for hanzo.chat 2026-07-15 19:09:52 -07:00
zandGitHub 42bbf2db9d Merge pull request #307 from hanzoai/fix/zen-bump-clean
fix(zen): bump embedded hanzoai/zen v1.3.0 → v1.3.7 (1M ladder fix)
2026-07-15 19:04:13 -07:00
hanzo-dev a37339c7e5 fix(zen): bump embedded hanzoai/zen v1.3.0 -> v1.3.7 (1M ladder need() fix)
The cloud binary serves the zen* family IN-PROCESS via zen.Mount, so the
embedded module version — not the zen pod image — is what serves zen5. v1.3.0
sized the ladder rung by the byte estimate alone: a ~230K-token prompt with a
32K max_tokens budget estimated ~258K (under glm-5.2's 262144 cap), stayed on
glm-5.2, which then saw prompt+output = 262145 and 400'd. v1.3.7 sizes the rung
by need() = messages + tool schema + max_tokens, overflowing a >262144 total to
the 1M deepseek-v4-pro rung. Brings the Enso family + gating in-binary too.
2026-07-15 19:03:37 -07:00
antje 478aeb2e33 fix(image): runtime needs libgcc — the flags staticlib references _Unwind_*
The hanzo-flags Rust staticlib compiles with unwinding (panic-guarded FFI);
its _Unwind_* references resolve from libgcc_s, which the alpine runtime did
not carry — /cloud failed relocation at exec ('Error relocating /cloud:
_Unwind_GetIP: symbol not found') and smoke red-gated the release. Add libgcc
to the runtime apk set.
2026-07-15 18:47:52 -07:00
hanzo-dev 090c7091bb Merge land/agent-deploy-sites: agent AI-brief site build+deploy to <slug>.hanzo.app, metered per deploy 2026-07-15 18:41:36 -07:00
hanzo-dev f3cb5f1d7b land(sites): agent AI-brief site build+deploy to <slug>.hanzo.app, metered per deploy 2026-07-15 18:40:05 -07:00
antje 8ad160e912 feat(flags): native flag engine — stateless Rust FFI + SQLite per project, /v1/flags
native/flags: hanzo-flags, a stateless Rust staticlib with PostHog-compatible
evaluation — the exact Insights rollout hash (sha1 first-15-hex / LONG_SCALE,
pinned by test vectors), the full vendored property-operator set (exact/regex/
semver/date/relative-date...), condition groups (variant-override groups first),
multivariate cumulative selection, payloads. Pure (defs JSON, ctx JSON) ->
response JSON behind a panic-guarded C ABI: hanzo_flags_evaluate/_free.
65 tests green.

clients/featureflags becomes the NATIVE engine (no external evaluator, no KV,
no network): definitions in per-(org, project) SQLite via cloud.OrgDB
(encrypted at rest via cek), hot in-memory platform snapshot (TTL 15s),
evaluation over cgo. The INSIGHTS_FLAGS_URL HTTP proxy is gone. Bool/Int/
String and the admin Board keep their shapes; sources are flags -> env ->
default (first cockpit write creates the definition — env fallback intact
until then, zero regression). engine_stub (!cgo) degrades loudly, fail-safe.

/v1/flags (org-scoped via principal, project via X-Project-Id): POST /v1/flags
(+/decide alias) evaluate; GET/PUT/DELETE /v1/flags/defs[/:key]; GET
/v1/flags/activity; GET /v1/flags/health. PUT /v1/admin/flags/:key (SuperAdmin)
is the cockpit write path through SetPlatformSwitch — flips apply immediately
in-pod, peers converge within one TTL.

Build: Dockerfile flagslib stage (rust:1-alpine musl staticlib, --locked) copied
to the exact ${SRCDIR}-relative cgo link path; make native; hanzo.yml
native-flags step + clients/featureflags in the hermetic unit gate.
go test ./clients/featureflags ./apps green (FFI live).
2026-07-15 18:38:01 -07:00
zeekayandhanzo-dev ccf9b61d09 fix(console): white-label the embedded console <title> by request Host
The console SPA shipped in the unified cloud binary is a STATIC export of
hanzoai/console whose <title> is baked to the default (Hanzo) brand at build
time. The embed cannot read the request Host, so every host — including
console.lux.cloud and console.zoo.cloud — served "Hanzo Cloud Console" in the
browser tab, a white-label violation. (The standalone Next.js app is host-aware
via generateMetadata, but it is retired from this serving path: cloud serves
the console in-process from the go:embed static export.)

Rewrite the SPA shell's <title> at the serving layer (serveIndex/indexFor) to
the request host's brand, reusing the existing brands registry (BrandForHost):
console.lux.cloud -> "Lux Cloud Console", console.hanzo.ai -> "Hanzo Cloud
Console" (unchanged), console.zoo.cloud -> "Zoo Cloud Console". Matches
hanzoai/console's own `${brandName} Console` output — one source of truth. The
Hanzo/default host returns the embedded bytes unchanged (no regression); a shell
with no <title> is never altered.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-15 18:25:22 -07:00
hanzo-dev ee0587f20c Merge land/featuregate: per-service waitlist-mode control plane 2026-07-15 18:10:27 -07:00
antje 294bfb6b24 deps: bump ai v1.808.1 -> v1.809.1
Brings live: get-cloud-usages Bearer + balance-exempt (usage panel works at
$0), the /v1/ai/connections usage-import endpoint. Console changes ride the
same release via CONSOLE_REF=main embed. Build green.
2026-07-15 17:38:07 -07:00
hanzo-dev a4f68617a0 wip(account-usage plane (clients/link usage + datastore)): rescued from agent that hit the session limit
Committed as-is to preserve the work (the building agent died mid-verify).
Not yet built/tested green; NOT merged to main. Resume from here.
2026-07-15 14:46:43 -07:00
hanzo-dev c5e59e6e4f wip(account-usage plane (clients/link usage + datastore)): rescued from agent that hit the session limit
Committed as-is to preserve the work (the building agent died mid-verify).
Not yet built/tested green; NOT merged to main. Resume from here.
2026-07-15 14:46:43 -07:00
hanzo-dev 7ceecbbc7a land(featuregate): per-service waitlist-mode control plane
Global SQLite registry {service, hosts, waitlistMode} + admin board/toggle
(/v1/admin/services*, /v1/featuregate/mode) + native Enforce middleware and
IAM-backed per-user approval resolver. Ported from recover/featuregate; dropped
the init()/cloud.RegisterWithShutdown self-registration in favor of the explicit
apps.Wire() MountSpec (order after admin, before tasks) + frozen wire_test row.

Distinct from clients/featureflags (a global read-only Insights waitlist_open
switch); this is the per-host launch lever with in-binary enforcement.
2026-07-15 13:20:25 -07:00
z fe5ee47725 platform: guard IAM object-store against a nil engine + commerce v1.48.2
Two co-residence fixes surfaced by the live founder-journey e2e:

1. /v1/platform/projects (authed) returned 500 'runtime error: invalid memory
   address' — iamobj.GetProjects/AddProject deref the package-global ormer.Engine,
   nil until the co-resident IAM subsystem initializes it. A nil store must fail
   cleanly: the iamStore() guard converts a nil-deref into a typed 503. (The store
   is initialized once iam co-mounts — v1.801.21 embedded-SQLite isolation.)
2. Bump commerce v1.48.1 -> v1.48.2: ErrorHandlerJSON preserves a downstream
   typed HTTP status, so a co-resident subsystem's 401/403/404 (e.g. platform's
   'X-Org-Id required') is no longer flattened to 500 by commerce's /v1 mount.

Regression tests: nil-store -> 503, success + real-error passthrough.
2026-07-15 13:11:54 -07:00
hanzo-dev 3fdbe9e4b2 controlplane seam (c): R1 — first-writer-wins registry + self-authorized Rekey (RED R1)
The ValidatorRegistry is the trusted key source for the whole cert, so key binding must be rogue-key-resistant BEFORE the driver sources idVerifier into the cert verifier. Register is now first-writer-wins: a node already bound to one ML-DSA key may not be silently re-bound to a DIFFERENT key (ErrIdentityKeyConflict); idempotent re-register of the identical key stays allowed. The ONLY sanctioned rotation is Rekey(node, newPub, authSig) — self-authorized by a signature under the node's CURRENT key over rekeyTBS under a DISTINCT rekey context (not popContext/certContext); an attacker without the current secret key cannot rotate. This is the registry action a consensus-ordered OpRekeyValidator applies (op plumbing lands with the RSM rewire).

5 R1 tests green; existing OnePodOneShare + full suite unaffected.
2026-07-15 12:41:21 -07:00
hanzo-dev fe26860183 controlplane seam (c): two-threshold self-guard + raw-craft verify tests (RED findings 1,3)
Finding 1: guardBFTFloor(len(keys), quorumWeight) fails closed unless the floor is the byzantine-safe BFT quorum (2n/3+1) for the validator-set size — so a mis-wired wallet-custody t (=3 for n=5) can neither compose nor verify a cert even if a future driver passes it. Called at the top of ComposeControlPlaneCert and VerifyControlPlaneCert. This is the exact two-threshold trap inc-1 hit (cluster.go wiring Pulsar threshold = quorum), now impossible in the cert core.

Finding 3: three raw-craft verify-side tests drive hand-built malicious certs (bypassing the honest composer) straight at VerifyControlPlaneCert — sub-quorum threshold-lie (ErrQCThresholdBelowFloor), attacker-key stuffing (ErrQCMerkleInclusion), weight inflation (ErrQCAggregateWeight) — plus a self-guard test (t=3 refused). 14 cert tests green.

Flag still false; ceremony untouched; existing byzantine suite green.
2026-07-15 12:41:21 -07:00
hanzo-dev 14467edfc5 controlplane inc-2 seam (c): real independent-sig ConsensusCert crypto core
The shipped Gen-3 weighted-quorum cert the design chose over the blocked threshold-Pulsar path. Each pod signs the canonical quorum message INDEPENDENTLY with its seam-a ML-DSA-65 key under a DISTINCT cert context (RED R4: certContext != popContext); the cert is a quasar.ConsensusCert carrying one EvidenceWeightedSigSet leg (a WeightedQuorumCert of N independent FIPS-204 sigs + weighted-Merkle quorum). Verification is quasar.VerifyConsensusCert under a control-plane ConsensusCertPolicy that requires the weighted-sig-set PQ leg — the shipped, audited verifier; NEVER the structural QuasarCert.Verify. No DKG, no threshold aggregate, no unshipped luxfi/pulsar core: soundness rests only on stock FIPS-204 verify + the weighted-validator-set Merkle commitment. Composes against luxfi/consensus v1.35.32 (BuildWeightedValidatorSet / BuildWeightedQuorumCert / VerifyConsensusCert).

VerifyControlPlaneCert binds the cert to the caller's expected position before the cryptographic verify (VerifyConsensusCert pins validator-set+policy but not the caller's height/round/block).

10 standalone crypto tests green: real cert verifies under policy; below-quorum / missing-leg / forged-sig / rogue-signer / wrong-position / wrong-validator-set all REJECTED with exact typed errors; R4 proven (a popContext sig is rejected as a cert sig); deterministic composition. Self-contained (does not touch the ceremony); existing suite stays green. Flag NOT yet flipped — ceremony rewiring + the ProductionBCCSigningReady() flip + stub deletion follow.
2026-07-15 12:41:21 -07:00
hanzo-dev ebaf7199ef deploy: rename the CD surface gitops -> /v1/deploy
The ArgoCD fork now lives at hanzoai/deploy (hanzoai/gitops redirects to it), so
the cloud CD control plane takes the matching single-word name: clients/gitops ->
clients/deploy, /v1/gitops/* -> /v1/deploy/*, and the subsystem name (Wire entry,
enablement key, health surface) is 'deploy'.

Same plane, same ArgoCD-grade fleet view over our own App CRs — applications,
resource tree, live manifest + diff, logs, sync, rollback.
2026-07-15 12:30:57 -07:00
antjeandGitHub 9819d1926b cloud(cmd): generate standalone connectorruntime binary
Completes the per-app-binary invariant: connectorruntime (HIP-0126) is in apps.Wire() but its generated cmd stub was dropped by a stale reconciliation merge. go generate ./apps restores it. cicd green.
2026-07-15 11:18:51 -07:00
hanzo-dev 13abe4432b Merge fix/iam-coresidence-db-isolation: isolate embedded IAM SQLite (unblock iam+ai) + CLOUD_ENABLE_STAGED
# Conflicts:
#	clients/iam/iam.go
2026-07-15 10:40:00 -07:00
hanzo-dev 28420819e8 Merge chore/regen-cmd-link: generate cmd/link standalone binary 2026-07-15 10:33:42 -07:00
antje b1576f51d3 cloud(ci): fix test gate path ./subsystems/ -> ./apps/ (main was red)
The subsystems/ -> apps/ rename merge (b9fc3ac5) missed hanzo.yml's test
gate, which still ran `go test ./subsystems/` — a directory that no longer
exists — so every main CI run failed "./subsystems [setup failed]". Point
it at ./apps/ (the renamed composition root). Unbreaks the release gate.
2026-07-15 10:21:30 -07:00
hanzo-dev 351bbff761 Merge origin/main (agents oversize-target fix) into branch integration 2026-07-15 10:19:28 -07:00
hanzo-dev 154209c7f1 Merge fix/402-credential-ordering: prefer fresh login JWT over stored hk- key in hanzo code 2026-07-15 10:18:18 -07:00
hanzo-dev a0e457893b Merge feat/connector-runtime-land: native in-process connector runtime (goja) 2026-07-15 10:17:08 -07:00
hanzo-dev 8e12bff7f5 fix(agents): reject an oversize target GPU array at the handler
Red LOW-2: body.Spec.GPUs was only truncated post-decode by Sanitize
(cap 32). register + PATCH now return 400 for a GPU array larger than the
cap instead of silently truncating, so an absurd payload is refused up
front and the client learns the bound. The 4MiB body limit already caps
the transient decode allocation; a normal-sized list still registers.
2026-07-15 10:14:47 -07:00
antje 0083a6894a cloud(cmd): generate missing standalone link binary (go generate ./apps)
The link app (unified AI login-manager registry, /v1/links) landed in
apps.Wire() without its generated cmd stub. Regenerating reconciles the
per-app-binary invariant: every Wire() app builds standalone via
apps.ServeSingle AND mounts into the unified binary. Idempotent.
2026-07-15 10:02:13 -07:00
hanzo-dev a5c711abba Merge origin/main (agents run-target) into apps/ rename integration 2026-07-15 09:49:42 -07:00
hanzo-dev 1d221f0e76 feat(agents): carry machine capability + live metrics on a run-target
A linked computer now reports what it IS (os/arch/cpus/memory/gpus, "spec")
and what it is DOING now (loadavg/memory/gpu-util, "metrics") so mission-
control can show which machine an agent runs on and whether it can take more
work — without copying the fact onto every session.

- targetspec.go: Spec/Metrics/GPU value types, JSON column codecs, and a
  total Sanitize that bounds strings/counts/sizes and coerces floats finite,
  so a hostile client can neither bloat the row nor smuggle a NaN/Inf.
- agent_targets gains spec/metrics/metrics_at (crashloop-safe addColumns,
  PRAGMA-guarded, no new index); register + PATCH accept them; a metrics
  PATCH is a heartbeat and the server owns the staleness clock (a client
  can't forge At).
- register upserts by (org, host) so re-linking the same machine refreshes
  one target instead of piling up duplicates.

Tests: sanitize bounds, store round-trip, GetTargetByHost org-scope, HTTP
capability+heartbeat, upsert-by-host. Existing target tests stay green.
2026-07-15 09:45:44 -07:00
hanzo-dev 9a689b049b fix(apps): add link mount to frozen wire-order fixture (main was red)
The feat/link-registry merge (7aa1e1a) added the `link` mount to Wire() —
{Name:"link", after agents} — but never updated the frozen mount-order fixture,
so TestWireOrderMatchesFrozen failed on main (84 specs vs 83 frozen). This merge
inherited that break; adding the missing frozen entry (link, no health, has
shutdown) at its Wire() position makes apps green again.
2026-07-15 09:43:55 -07:00
hanzo-dev b9fc3ac5bc Merge feat/per-app-cmd-binaries: subsystems/ -> apps/ rename + per-app cmd binaries
# Conflicts:
#	apps/wire_seams.go
2026-07-15 09:39:39 -07:00
hanzo-dev fbc2337819 fix(link): scope login-out session stop to the revoking user
A link revoke tore down live sessions by matching only {org, host,
provider, account}. Those fields come from a link row the caller sets at
upsert, so any org member could stop another member's sessions by
registering a wildcard link (e.g. provider-only) and revoking it.

Make Actor mandatory in agents.SessionMatch and AND it into the query.
The link adapter derives it from the revoking user's subject
(agents.BillingActor), the single place that binds a stop to the caller's
own sessions, so a stop/count can only ever reach that user's sessions.
A match with no actor stops nothing (fail-closed).

Tests: agents.TestStopSessions_ActorScoped (wildcard, host-forge,
no-actor fail-closed, count scope, own-account teardown) and
link.TestRevokeCannotStopCoTenantSessions (full stack through the real
adapter, co-tenant survives). TestRevokeStopsSessions stays green.
2026-07-15 09:31:20 -07:00
hanzo-dev f5be74651f Merge branch 'feat/link-registry' into _mergetmp 2026-07-15 03:32:21 -07:00
hanzo-dev 7aa1e1abbb feat(link): unified AI login-manager registry (/v1/links)
The org+user-scoped registry of which provider accounts (Claude Max, ChatGPT
Plus, a Hanzo/api key) a developer has signed into, on which machines, with each
account's latest usage snapshot — the cross-machine view console renders and the
source the redundancy route policy reads.

- clients/link: the Link atom (no secret — metadata + usage snapshot only),
  per-org SQLite store (org+subject leading-bound, upsert-on-identity, revoke),
  the /v1/links surface, and a pure RoutePolicy (Plan) that orders a user's linked
  accounts for redundancy (two Claude Max, then the metered API backstop) carrying
  the billing mode per candidate. The store holds no metering client — a
  subscription's usage is metered for visibility only and never charges commerce.
- clients/agents: a session now carries the linked account it ran under
  (Provider/Account tag), and StopSessions/CountActiveSessions expose the
  in-process action a link revoke takes to stop the sessions under a revoked
  account/device. Backward-compatible session migration (addColumns).
- subsystems: mount link after agents so a revoke can stop its sessions.

Org+user fail-closed isolation, subscription-vs-api-key billing distinction,
and revoke-stops-sessions are all tested (build/vet/gofmt clean; race+CGO green).
2026-07-15 03:31:37 -07:00
hanzo-dev 241c6d814a chore(cloud): bump hanzoai/ai v1.808.0 → v1.808.1 (relay reasoning normalization)
Pulls the DeepSeek <think></think> strip into api.hanzo.ai: reasoning-inlining
upstreams (zen5-pro/zen5-flash → deepseek-*) no longer leak the </think> template
token into the visible answer via the Anthropic-translation path that `hanzo code`
uses. Also carries the hk-key 402 tenant-gate fix. Builds clean (server + hanzo CLI).
2026-07-15 03:09:00 -07:00
zeekay 64699ec3b7 release: mirror via crane — IAM token realm rejects buildx's multi-scope request (crane single-scope proven E2E) 2026-07-15 02:59:17 -07:00
hanzo-dev e40705ff3d feat(cloud): per-app standalone cmd binaries via apps.ServeSingle + generator
Each app now builds as its own standalone binary AND still mounts into the
unified cloud binary — one source of truth (apps.Wire()). Two pieces:

1. apps.ServeSingle(name) — the ONE way to run a single app standalone: validate
   the name against Wire(), then cloud.Serve(Wire(), []string{name}). It is the
   path cmd/hanzo's 'hanzo <name>' already uses; promoting it to apps (which
   already imports cloud, so no cycle) lets every cmd/<app> stub reuse it instead
   of re-implementing the dispatch. Adding an app in Wire() is still the one edit.

2. cmd/gen-app-cmds — a tiny generator (go:generate directive in apps/apps.go)
   that parses Wire()'s {Name: "..."} literals and writes one cmd/<app>/main.go
   stub per app (apps.ServeSingle("<app>")). Generated, not hand-maintained; a
   re-run is idempotent (writes only on content change). 80 unique apps today.

Verify: go build ./apps/... green; apps.ServeSingle added; go generate ./apps is
idempotent; go test ./apps/... green (TestWireOrderMatchesFrozen at 84 specs); a
sample of generated cmd stubs (kms/account/agents/platform/storage/audit/
analytics/ads/ingress/billing/o11y) build standalone green. The full ./cmd/...
build is heavy (80 binaries x the cloud tree) and needs CI parallelism; the
stubs themselves are sound.
2026-07-15 02:46:42 -07:00
zeekay c6400c620d Merge remote-tracking branch 'origin/main' into ci-finish 2026-07-15 02:43:03 -07:00
zeekay bd8c1d7454 ci: mirror uses direct REGISTRY_USER/PASSWORD secret (Free plan hides org KMS secrets from private repos); test gate skips the cek env-gated test 2026-07-15 02:42:50 -07:00
hanzo-dev 0645a1cc05 refactor(cloud): rename subsystems/ -> apps/ (the composition root)
The cloud binary mounts ~84 'subsystems' into one unified binary; they read as
apps, so the package follows. Purely internal: dir subsystems/ -> apps/, package
'subsystems' -> 'apps', the one file subsystems.go -> apps.go. Three import sites
updated (cmd/cloud, cmd/hanzo). Broken path/filename references in comments
(subsystems.Wire, subsystems/commerce.go, subsystems.go) -> apps.*; the package
doc + LLM.md/docs paths follow.

No external breakage: the package is same-module (never in go.mod), no external
module imports it. The frozen-order test TestWireOrderMatchesFrozen compares
MountSpec.Name strings, not the package name — passes unchanged at 84 specs.

Standalone via 'hanzo <name>' and per-app cmd binaries are unchanged by this
rename (D2 adds the cmd binaries on top).

Verify: go build ./... green; go vet ./apps/... ./cmd/... clean; go test ./apps/...
green; TestWireOrderMatchesFrozen green. cmd/cloud TestMountAllAndServeHealth
fails identically on clean main (needs CLOUD_KMS_MASTER_KEY_REF), not this change.
2026-07-15 02:31:45 -07:00
hanzo-dev c22391f6fb fix(agents): build session target/host indexes after addColumns
migrateSessions created ix_sessions_org_target and ix_sessions_org_host in
the table DDL, which runs before addColumns adds target/host to a
pre-existing table. On a fresh DB the CREATE TABLE carries the columns so
the indexes build; booting over the prior release's agent_sessions table
the CREATE TABLE IF NOT EXISTS no-ops and the index references a
not-yet-added column ("no such column: target"), failing the release
migration smoke. Build the two indexes after addColumns so the columns
exist first on both the fresh and upgrade paths.

TestMigrateOverLegacySessionsTable locks it over the pre-target schema.
2026-07-15 02:24:57 -07:00
hanzo-dev 1bb0773823 fix(cloud/code): prefer the live login JWT over the hk- key to unblock 402
hanzo code 402s ('a billable tenant is required') on a deployment without
IAM_MINT_CLIENT_*: codeToken() preferred the hk- API key, but an hk- key only
mints a billing principal when the server can resolve it (iamKeys.resolve is a
no-op without the mint credential) — so the request arrives anonymous and zen's
/v1/messages billing gate refuses it. The /v1/models catalog call is loosely
gated, so the launcher banner still printed, masking the cause.

Reorder codeToken to prefer a FRESH hanzo login JWT, which carries the caller's
owner/project/sub claims verbatim and mints a billing principal on EVERY
deployment, then fall back to the hk- key chain (still works on mint-credentialed
servers). HANZO_API_KEY stays the deliberate operator override at the top.

freshAccessToken() is a new expiry-guarded accessor (decomplected from
accessToken, which whoami keeps expiry-agnostic): an EXPIRED jwt must not win —
it would 401 a session a valid hk- key would serve — so it falls through to the
key. No expiry record (a raw HANZO_TOKEN) is trusted as-is.

Pinned by TestCodeTokenPrecedence (fresh jwt beats hk-, expired jwt falls
through, HANZO_API_KEY overrides all). Structural hanzoai/jwt extraction is a
follow-up PR.
2026-07-15 02:19:47 -07:00
hanzo-dev 211a72c274 git: make the mounted service pointer atomic (fix -race)
The lifecycle reactors (notify / mirror-out / index-on-push) read the package
'mounted' var in DETACHED EmitLifecycle goroutines while Mount/Shutdown write it;
a reactor goroutine outliving a Shutdown (or a test Mount<->Shutdown cycle) tore
against the write. Convert 'mounted' to atomic.Pointer[cloud.Service[state]] —
Store on Mount/Shutdown, Load in every reader (reactors, export CloneURL/VerifyRef,
index-on-push, the GitHub importer) + the tests. Behavior unchanged (production
sets mounted once); the data race the -race detector flagged is gone.
2026-07-15 02:19:15 -07:00
hanzo-dev 8486d0f1d0 feat(cloud/connectorruntime): native in-process connector exec (HIP-0126)
Lands clients/connectorruntime — the native replacement for the standalone
ActivePieces Node engine: an ActivePieces JS connector action runs in goja
in-process (no auto pod, no in-cluster HTTP hop, no shared X-Piece-Run-Secret).
Same {action,auth,props} -> {ok,output,error} contract; org-gated via
principal.Org; the caller's credential travels in the request auth.

POST /v1/automations/connectors/:id/run  run one connector action in-process

Wired into the composition root (subsystems.Wire) the one way main does it —
an explicit MountSpec after automations (it pairs with the catalogue) — not the
old self-registering cloud.Register init (removed; main is explicit-Wire).
frozen order sequence updated to 84 specs.

Re-derived from the cloud-cr worktree's uncommitted work against current main:
the package's principal.Tenant -> principal.Org, the cloud.Register init -> the
explicit Wire() entry, and the go.mod esbuild dep flipped to direct. The
kb/sync_piece.go refactor that originally accompanied this is NOT carried —
clients/kb was removed on main since the branch forked, so there is nothing to
refactor; connectorruntime stands on its own (imports only cloud + principal).

Build green; go test ./clients/connectorruntime/... green (registry + runtime);
TestWireOrderMatchesFrozen green at 84 specs.
2026-07-15 02:13:38 -07:00
hanzo-dev 6049750fb1 Merge feat/ai-obs-attribution: o11y trace attribution + annotation queues
Trace attribution, per-project metrics, sessions, and annotation-queue routes
on the o11y plane (clients/o11y annotation store + queues, clients/eval
attribution/telemetry, principal scope). Genuine new feature (not on main),
rebased clean onto current main; builds and race tests green.
2026-07-15 02:07:08 -07:00
hanzo-dev bf7b3a9660 Merge feat/github-app-sync: GitHub App bidirectional repo mirror + sync
Install the Hanzo GitHub App -> list org repos -> import into git.hanzo.ai ->
bidirectional sync (outbound mirror already exists; inbound = HMAC-verified
webhook, fast-forward-ONLY, never force-overwrites native). Inert until the App
creds land in KMS.
2026-07-15 02:07:05 -07:00
hanzo-dev 2a44492f49 Merge fix/affiliates-topup: accrual tops up the open period (month-to-date converges); link clicks off the money-DB write path 2026-07-15 02:05:26 -07:00
hanzo-dev b15b320716 fix(affiliates): accrual tops up the open period so month-to-date spend converges
The monthly accrual latched once on the FIRST sweep's PARTIAL month-to-date spend
and no-oped every later sweep, so an affiliate whose dashboard swept early in the
month froze its share near zero (underpaid, though platform-safe). Store.Accrue now
inserts the period row on first sweep and, for the still-open period, TOPS IT UP
toward the current (higher) month-to-date reading in one transaction — adding only
the positive delta, so accrued_cents converges to the month-end value, never
decreases, and never overshoots. The per-event money invariant is untouched: each
row keeps margin + share from ONE reading (share <= margin) and the level-schedule
cap keeps sum(share) <= margin at every step.

Public link clicks move off the money-DB write path: clickLink folds pings into an
in-memory coalescing buffer (bounded), flushed batched on the next links read and on
shutdown, so a click flood can never contend with the accrual/payout writes.

Tests: TestAccrualConverges + TestAccrualConvergesAtMaxRate prove the share tracks a
growing month-to-date and sum(share) <= margin at every intermediate sweep; existing
invariant/idempotency/isolation/links tests stay green (26 pass, -race).
2026-07-15 02:05:08 -07:00
hanzo-dev 9a6505f39b code: index on push via durable hanzoai/tasks workflow
A push to a repo's default branch enqueues a durable IndexRepoWorkflow on
the embedded tasks engine (workflow id keyed by commit = idempotent per
push); a worker reads the tip tree from the object plane and folds its
text files into the org's code index, retried on failure. Fail-soft:
before the engine is wired the reactor indexes inline on its detached
lifecycle goroutine (the ai-ingest contract), so push-index is always live.

git and code never import each other — the reactor is git's third
lifecycle subscriber, the index reached through the SetIndexer func seam
wired once at the composition root.
2026-07-15 02:04:02 -07:00
hanzo-dev b2da6ae6a6 feat(o11y): trace attribution, per-project metrics, sessions + annotation-queue routes
eval telemetry (Trace): add ProjectID, SessionID, APIKeyHash (SHA-256 ref, never
plaintext), and StartTime/EndTime latency; DDL columns + additive migrations +
write/read paths. A run stamps its project (principal.ProjectScope), groups its
item-traces under the run as one session, times the model call, and records a
non-reversible ref of the caller credential. Trace list narrows by the caller's
project (default == whole org).

evals/metrics: thread the server-minted project scope into MetricsFilter,
usageWhere (AND project = ?) and the latency span filter. The default-project
(whole-org) board queries the ledger today; a named-project board is honest-empty
until cloud_usage carries a project column (ai write path) — activation is one
guard flip, the query plumbing is project-aware and tested.

principal.ProjectScope: the ONE helper for "default project == "" == whole org";
eval + o11y both read it.

o11y: explicit org-gated GET /v1/o11y/sessions pinning the runtime /api/sessions
route; native annotation-queues surface (SQLite metastore, org+project scoped) at
/v1/o11y/annotation-queues* — list/create/detail/update/delete, items add/list/
complete — returning the console {data,meta} envelope.

Tests: trace attribution (latency, session grouping, hashed-not-plaintext key),
per-project + cross-org trace isolation, usageWhere project predicate, annotation
queue lifecycle + org/project isolation + validation + principal gate.
2026-07-15 02:03:55 -07:00
hanzo-dev e27262c7ce GitHub App: bidirectional repo mirror + sync (git.hanzo.ai)
integrations (github.go/github_app.go/github_webhook.go): App install ->
installation-token mint (ghinstallation), list granted repos, background import,
inbound push webhook (HMAC-verified, fast-forward-only). Inert until
GITHUB_APP_{ID,PRIVATE_KEY,WEBHOOK_SECRET,SLUG} land.

git (github_import.go + cloud.GitImporter seam in git_import.go): import =
fast-forward mirror-in per branch; inbound = fast-forward-ONLY advance where
native is canonical, so a divergence is a recorded conflict and never a
force-overwrite; loop-prevention via the Origin stamp; per-repo status. Outbound
mirror uses the org installation token for github.com targets, shared-token
fallback otherwise (no regression).
2026-07-15 02:02:49 -07:00
hanzo-dev 821e7e3911 code: add tree + file endpoints (zread contract over the index)
GET /v1/code/tree?repo=  → get_repo_structure: the repo's files + per-file
  symbol counts, ordered, per-org isolated.
GET /v1/code/file?repo=&path= → read_file: the INDEXED content (the symbol
  chunks the search tiers hold), for fast context. NOT byte-verbatim — inter-
  symbol lines a parser did not chunk are absent; the S3-backed git object
  plane (clients/git) is the source of record for exact bytes, history, and
  blame. Comments say so; a follow-up routes byte-fidelity read_file/blame at
  the git plane.
Store.tree + Store.fileContent read the existing files/symbols/chunks tiers —
no schema change. Test covers tree structure, indexed-file read, 404 on an
unindexed path, and per-org isolation (org B sees an empty tree).
2026-07-15 02:01:18 -07:00
hanzo-dev d86ee8ed46 Merge feat/hanzo-agent-keystone: @hanzo Slack coding agent (ships INERT)
The coding-agent code lands fail-closed and INERT: no sandbox runs until an
operator provisions docker and sets HANZO_CODING_* env. The coding sandbox
stays DISABLED until the default-deny-egress + container resource-limit
hardening lands and red re-clears. Do NOT provision docker / set HANZO_CODING_*
until then.

Native coding tasks from Slack over the shared object plane: clients/coding
dispatcher, in-process agents session bus, tracker agent PR, bot NDJSON client,
git export seam, slack coding trigger.
2026-07-15 01:57:17 -07:00
hanzo-dev 1d290d0067 Merge feat/mission-control-sessions: agent session execution-context + targets
Mission-control backend over /v1/agents/sessions: sessions carry host/cwd/
repo/target and a compact last-event; new /v1/agents/targets registry
(register/list/detail/patch/delete) with live session-load, org fail-closed,
in the same agents.db. Composes with the compute fleet at the view layer.
2026-07-15 01:49:51 -07:00
hanzo-dev 17f274ca8c agents: session execution-context + targets (mission-control)
Sessions gain host/cwd/repo/target so mission-control can show where a
session runs and map sessions to a machine; the list projection carries a
compact last-event preview. Add /v1/agents/targets — register/list/detail/
patch/delete a dispatch destination (laptop|cloud|gpu|cluster|machine) with
a live session-load rollup — in the same agents.db, one tenancy column, org
fail-closed. A session's target resolves same-org at register/patch (#48).

Composes with the compute fleet (/v1/fleet/workers, /v1/clusters) at the
view layer rather than duplicating it.
2026-07-15 01:48:51 -07:00
zeekay 0155df3879 release: fix step splice — buildx keeps its with: block 2026-07-14 23:38:51 -07:00
zeekay a5bd1c9d82 release: dual-host — mirror release tags to registry.hanzo.ai
Server-side copies alongside the ghcr retag; the cluster deploys from OUR
registry, ghcr stays the public identity. Best-effort by contract — a mirror
hiccup never blocks the tag-receipt.
2026-07-14 23:38:27 -07:00
zeekay 731eb217e6 Merge remote-tracking branch 'origin/main' into unfork-commerce 2026-07-14 23:19:49 -07:00
zeekay e7e98e2b52 ci: canonical hanzo.yml test gate + hanzoai/ci caller on the arc pool
release.yml keeps sole ownership of the image + v* tags (tag-is-a-receipt);
this adds the vet/unit gate on every push/PR, on our own runners.
2026-07-14 23:19:38 -07:00
hanzo-dev bced234eba fix(cli/code): give agent its true 1M context on zen models
agent sizes a model's context window only from ids it recognizes.
An unknown gateway id like `zen5-pro` gets a hardcoded 128K budget and is
rejected client-side above it ("maximum context length is 131072") — even
though api.hanzo.ai and the DeepSeek-V4 upstream serve the full 1M window
fine (verified live: 140K-token prompts return 200).

Bridge each CC tier through a recognized carrier id (claude-opus-4-8[1m]
etc.) that unlocks the 1M budget, then rewrite it back to the served zen
alias via settings.json modelOverrides before the request leaves the
client — the carrier never reaches the server, so real claude-* routes are
untouched and responses stay branded/served as zen. One source of truth
(zenTiers) drives the wire env, picker branding, and overrides. The fast
tier stays direct (never needs >128K; SMALL_FAST_MODEL isn't override-
rewritten, so a carrier there would leak raw).

Verified end-to-end with agent v2.1.210: the wire model reaching
api.hanzo.ai is zen5 / zen5-pro carrying context-1m-2025-08-07, with zero
raw claude-* leaks.
2026-07-14 23:15:24 -07:00
zeekay 561a1b1e8c merge main: /v1/store owner carried onto the NATIVE mount
The store surface registers natively (api/store.Route on a group-scoped chain
mirroring the standalone /v1 bundle) instead of re-adding the gin AdaptNetHTTP
prefix; commercePrefixes keeps /v1/store pinned so store reads can never fall
through to the AI /v1/* balance gate again.
2026-07-14 23:03:43 -07:00
zeekay 4a972ea6a1 commerce v1.48.1 NATIVE co-residence — routes on the shared zip app, AdaptNetHTTP gone from the commerce path
- subsystems/commerce.go: commerce.Embed(App: app) — the SharedApp contract;
  commerce registers /v1/commerce/* + /_/commerce/* directly on cloud's router
  (one specificity space, no second engine, no net/http adaptation)
- live wire paths registered natively with commerce's own gates:
  /v1/billing/webhooks/:provider (HMAC-is-auth) and
  /v1/billing/auto-recharge/run-all (service-token + PlatformOnly) —
  commercePrefixes contract test pinned and passing
- commerceinproc.SetApp: S2S byte-stream enters the shared app's pipeline;
  RoundTrip normalizes client-style RequestURI at the ONE seam;
  entitlements + BalanceCents stay pure direct-Go (commerceclient)
- zip v1.8.2 (union tag: v1.8 features + the v1.7.4 chain-order fix + v1.7.5
  empty-leaf normalization — v1.8.1 was missing both)
- metering dispatch e2e ported to the zip harness and green
2026-07-14 23:01:39 -07:00
hanzo-dev 008b0c2b92 fix(commerce): own /v1/store/* so store reads reach commerce, not the AI /v1/* balance gate
The bare storefront surface api.Route(Group("/v1")) registers on the commerce
gin engine — GET /v1/store/current (the org-scoped default store the admin
dashboard AND the content storefront edge resolve), the /v1/store/:id/listing
upsert the publish edge writes, and the public /v1/store/:store/listing reads
karma.style serves at runtime — was dropped from commercePrefixes by the unfork.
Unowned, /v1/store/* fell through to the bare /v1/* AI catch-all, whose prepaid
LLM balance gate 402'd every store read for an org funded in commerce but $0 in
the ai ledger (org karma: $999.99 credit, GET /v1/store/current -> 402
insufficient_balance, blocking store provisioning + the storefront image fan-out).

Restore the /v1/store prefix so /v1/store/* routes to the commerce handler, which
resolves the org from the gateway X-Org-Id (TokenRequired -> ensureIAMOrg) and
lazily provisions the org's store — standalone-commerced parity. A store-metadata
read never requires an LLM balance. Per-route permission masks are unchanged
(money paths stay publishedRequired); gin still 404s an unknown /v1/store/* path.

Guarded by TestStoreSurfaceRoutedToCommerceNotAIGate (store read resolves on
commerce, never 402 at the catch-all) + the extended commercePrefixes pin test.
2026-07-14 22:55:03 -07:00
hanzo-dev 3da5b13053 feat(social): optional media[] URLs on posts
Add an optional media field (JSON array of URLs) to the social Post store,
additive and idempotent exactly like the marketing scheduled_at fix:

- store: media TEXT NOT NULL DEFAULT '[]' column + addColumn upgrade for
  pre-existing prod DBs (migrate-on-open, the only upgrade path for the
  encrypted single-file store); JSON encode/decode helpers; Post.Media []string
  always serialized as [] never null.
- api: normMedia bounds each URL to maxField and the list to maxMedia (10),
  the one sanitization seam on create + update (mirrors content/channel).
- tests: media round-trip in TestPostCRUD (create sets, update replaces) and
  TestMigrateAddsMediaColumnToOldPosts — old-schema DB gains the column and a
  media write no longer 500s; legacy rows default to [].
2026-07-14 22:51:48 -07:00
hanzo-dev d23e0d0f42 Merge feat/affiliate-profit-share-dashboard: OSS-dev affiliate profit-share + dashboard
Margin-based accrual (share = rate x Hanzo's margin, share <= margin invariant),
derived share ledger that never mutates the cost-of-record, shareable links with
click/signup/conversion, per-period earnings, opt-in privacy leaderboard, and the
SuperAdmin set-rate. Tenant-isolated, fail-closed.
2026-07-14 22:31:52 -07:00
hanzo-dev 8043a84f61 affiliates: profit-share (margin-based accrual) + dashboard (links, earnings, leaderboard)
Switch the affiliate accrual from revenue-share (spend x rate) to PROFIT-share
(margin x rate), where margin = a referred org's gross spend x the platform
gross-margin fraction (AFFILIATE_MARGIN_BPS, default 40%). The share is a rate
OF Hanzo's margin, never the customer's bill, so a payout can never exceed the
margin earned. The L1 rate is capped (maxL1RateBps=9300) so the whole L1+L2+L3
schedule stays <= 100% of the margin: total share <= margin, always.

The derived share ledger (affiliate_accruals) now records the margin base
alongside the gross spend and the share; it never mutates the cost-of-record
(commerce/cloud_usage) - it is a pure projection keyed by referrer.

Dashboard surface (all org-scoped, fail-closed):
- GET  /v1/affiliates/me/earnings   per-period + per-direct-referral aggregate
- GET/POST /v1/affiliates/me/links  shareable links + click/signup/conversion
- POST /v1/affiliates/me/handle     opt-in leaderboard display name
- POST /v1/affiliates/click         public click ping (vanity counter)
- GET  /v1/affiliates/leaderboard   opt-in handles + aggregate + your own rank
- POST /v1/admin/affiliates/:id/rate SuperAdmin set L1 rate (capped)

Tests: margin invariant (share <= margin per level + summed; charge unchanged),
set-rate cap, links lifecycle, cross-affiliate isolation (earnings + links),
leaderboard privacy (no org identity leaks, own rank always visible).
2026-07-14 22:31:21 -07:00
hanzo-dev e3e87f8b69 leaderboard: gamified usage analytics — leaderboards + activity graph (#43)
New /v1/usage/leaderboard + /v1/usage/activity + opt-in surface over a derived
datastore rollup (SummingMergeTree MV of hanzo.cloud_usage). Ranks top AI users
(personal/org) and orgs (global); per-day contribution heatmap + timeline.

- rollup.go: usage_rollup_daily target + incremental MV (type-exact projection,
  cannot fail a valid ledger insert) + deploy-gated run-once backfill.
- sql.go: injection-safe builders — org bound positionally, metric from a closed
  allowlist, limit a clamped int; org is the leading predicate.
- view.go: opt-in privacy — self/opted-in/admin named, else Anonymous; cross-org
  detail structurally impossible (org-bound reads).
- store.go: opt-in preference store (private by default), Base/SQLite via cek.
- board.go/activity.go/optin.go/backfill.go: handlers, fail-closed on principal.
- 39 tests (incl -race): builder injection-safety, tenant isolation, cross-tenant
  bleed, naming policy, opt-in default-private, authz resolvers, rollup lifecycle.
2026-07-14 22:28:37 -07:00
hanzo-dev ca1a266141 leaderboard: gamified usage analytics — leaderboards + activity graph (#43)
New /v1/usage/leaderboard + /v1/usage/activity + opt-in surface over a derived
datastore rollup (SummingMergeTree MV of hanzo.cloud_usage). Ranks top AI users
(personal/org) and orgs (global); per-day contribution heatmap + timeline.

- rollup.go: usage_rollup_daily target + incremental MV (type-exact projection,
  cannot fail a valid ledger insert) + deploy-gated run-once backfill.
- sql.go: injection-safe builders — org bound positionally, metric from a closed
  allowlist, limit a clamped int; org is the leading predicate.
- view.go: opt-in privacy — self/opted-in/admin named, else Anonymous; cross-org
  detail structurally impossible (org-bound reads).
- store.go: opt-in preference store (private by default), Base/SQLite via cek.
- board.go/activity.go/optin.go/backfill.go: handlers, fail-closed on principal.
- 39 tests (incl -race): builder injection-safety, tenant isolation, cross-tenant
  bleed, naming policy, opt-in default-private, authz resolvers, rollup lifecycle.
2026-07-14 22:28:37 -07:00
hanzo-dev df7ad43030 Merge fix/prod-headers-followups: brand-only Server fallback (zip v1.8.1) + trailing-dot brand resolution 2026-07-14 21:56:26 -07:00
hanzo-dev 4a8c25e27f fix(headers): brand-only Server fallback + trailing-dot brand resolution
Address Red's LOW-1/LOW-3 on the production-header posture:
- serve.go sets zip.Config.ServerHeader=cfg.Brand so the responses the
  ProductionHeaders middleware can't reach — the fasthttp transport's OWN
  pre-routing errors (431/400) — read Server: <brand> instead of the framework
  default. Requires zip>=v1.8.1 (transport now propagates ServerHeader). Repin
  v1.8.0->v1.8.1; go.mod diff is only the zip line.
- BrandForHostOK strips a trailing FQDN dot so api.lux.network. resolves to lux
  (fails safe to neutral before, never a wrong brand — brand-fidelity fix).
2026-07-14 21:56:15 -07:00
hanzo-dev d690c3b129 deps(commerce): v1.47.2 -> v1.47.3 — tier gate counts granted credits as spendable (fixes 402 for grant-funded orgs) 2026-07-14 21:47:20 -07:00
hanzo-dev d10a7528a8 Merge feat/prod-response-headers: inherit zip ProductionHeaders posture (brand-by-host Server, X-Api-Version, HSTS/nosniff) 2026-07-14 21:07:29 -07:00
hanzo-dev a0a932d1f2 feat: inherit zip ProductionHeaders — brand-by-host Server, X-Api-Version, HSTS/nosniff
Wire the shared production response-header posture (zip v1.8.0) into the edge,
right after RequestID so it covers every response — success, error, 404, and
the public-site static bytes:

- Server is the white-label brand of the request Host via BrandForHostOK (cloud's
  own registry), so a lux/zoo caller is never served "hanzo" and no response
  leaks the framework name. An unmatched Host falls back to this deployment's own
  brand (cfg.Brand), never a framework or hardcoded single brand.
- X-Api-Version carries the build version (Config.Version <- CLOUD_VERSION env,
  else the link-time cloud.Version default) under a brand-neutral key.
- HSTS + nosniff are the always-safe security floor; the console SPA keeps its
  own framing rules (no X-Frame-Options/CSP forced here).

Bumps zip v1.6.0 -> v1.8.0 for middleware.ProductionHeaders. X-Request-Id stays
owned by middleware.RequestID.
2026-07-14 21:07:18 -07:00
hanzo-dev 9363528d53 cli: auto-wire Hanzo MCP into hanzo code claude (port Rust resolve_mcp)
The Go hanzo code wired zero MCP — subagents got a bare model, no Hanzo tools.
Port the Rust CLI's resolve_mcp: resolve hanzo-mcp (installed → PATH, else uvx
hanzo-mcp), write an --mcp-config stdio document scoped to the cwd into the
isolated config dir, and pass --strict-mcp-config so the Hanzo server is the sole
MCP source (a repo .mcp.json is ignored — it can ship a bearer-exfiltrating stdio
server). A missing hanzo-mcp warns and continues (MCP is an enhancement, never a
blocker). So hanzo code claude now starts with the Hanzo tool lattice — code
search over the cloud index, web search, vision, fs/exec/git. Test covers the
opt-in, the config shape, and the not-found warn path.
2026-07-14 20:12:25 -07:00
hanzo-dev 9487becfec code+knowledge: default embed model to zen-embedding SKU (fix semantic tier)
Both indexers defaulted CLOUD_EMBED_MODEL to bge-m3 (the raw upstream), which
the gateway rejects (400) — only the zen-embedding SKU is served. Result:
index-time embeds failed silently, semantic code + KB search returned nothing
(vectors:0, degraded:true). Default to the served SKU. Pairs with the zen
bge-m3->zen-embedding alias so both the name and the SKU resolve.
2026-07-14 20:04:19 -07:00
hanzo-dev 8d4469fde6 integrations: GitLab OAuth provider (login + repo sync, KMS-env creds)
Adds the gitlab provider on the same OAuth registry as slack/google/github.
Reads GITLAB_CLIENT_ID/GITLAB_CLIENT_SECRET from ENV (KMS-synced, never in code);
Configured() is false until both are present so it ships INERT and fails closed
(honest 503 / failure redirect, never a fake OK). Callback = the app's
/v1/integrations/gitlab/callback; the generic dispatcher seals the access+refresh
tokens into the org KMS namespace.

Least-privilege by construction: requests only openid/profile/email/read_api/
read_repository/write_repository — the token receives the intersection of
requested + app-allowed, so we never request api/sudo/admin_mode/k8s_proxy/
*_runner/*_registry even if the app was provisioned with them (tested).
GITLAB_URL supports self-hosted GitLab. 5 tests: registration, least-privilege
authorize URL, exchange seals both tokens + resolves account, missing-secret
fails honestly, error body surfaced.
2026-07-14 18:19:52 -07:00
hanzo-dev bb7a652315 deps(commerce): v1.47.1 -> v1.47.2 — org-scoped /v1/store/current + lazy store provisioning
commerce v1.47.2 fixes GET /v1/store/current returning the phantom shared
"default" store: it now resolves the caller org's namespace and lazily,
idempotently provisions the org-scoped store (store.EnsureDefault) on first
authenticated hit — the store id the content storefront edge needs to publish
Listing.headerImage. Round-trip + cross-tenant tests ship in the module.
2026-07-14 17:59:51 -07:00
hanzo-dev 18a7b05c58 fix(marketing): additive ALTER so old prod campaigns tables gain scheduled_at
POST/GET /v1/marketing/campaigns 500'd on prod with "table
marketing_campaigns has no column named scheduled_at": migrateCampaigns()
uses CREATE TABLE IF NOT EXISTS, which NEVER alters an existing table, so a
prod DB created before scheduled_at was added to the DDL was frozen at its
original schema and every campaign write (INSERT/UPDATE name it) 500'd. The
store is an encrypted single-file SQLite only the binary can open, so a
hand-patch is impossible — the upgrade MUST happen in migrate-on-open.

- migrateCampaigns: after the CREATE, run an idempotent additive column
  upgrade — ALTER TABLE marketing_campaigns ADD COLUMN scheduled_at
  INTEGER NOT NULL DEFAULT 0 — swallowing SQLite's "duplicate column name"
  (the only error) so it's a no-op on a fresh DB.
- addColumn helper mirrors clients/social/store.go exactly (the ONE way we
  do additive migrations), keyed by an extensible {table,col,def} list.

Tests (store_migrate_test.go, CGO_ENABLED=0 pure-Go cek): a from-OLD-schema DB
(marketing_campaigns without scheduled_at + a legacy row) opens, migrate ADDs
the column, a scheduled_at write succeeds, and the legacy row survives with
scheduled_at defaulted to 0; plus a fresh-DB idempotency re-open. Without the
ALTER the old-schema test reproduces the exact prod 500.
2026-07-14 17:56:36 -07:00
hanzo-dev 959cf4d15c fix(commerce): restore /v1/billing/auto-recharge prefix dropped by the unfork — pin test moves beside the canonical list
The unfork (741644d) rebuilt commercePrefixes in subsystems/commerce.go from a
pre-fix snapshot, dropping /v1/billing/auto-recharge (landed as #274/3c48fb8)
and deleting its pin test with the old clients/commerce tree. Without the
prefix the durable cron's quarter-hour billing-autorecharge poke lands on the
account-bridge /v1/billing/* session gate and 403s — verified live before
3c48fb8 shipped in v1.801.1 (poke 200, 311 orgs swept, 23:45:05Z).

Same one-line-family fix, new canonical location; TestCommercePrefixesPinned
now lives beside the list it pins so a future rewrite can't silently regress
the wire path again.
2026-07-14 17:51:13 -07:00
hanzo-dev 0df009b329 fix(zen): resolve upstream provider keys with env fallback (fix DO 401)
zenKeyResolver read the co-resident KMS store ONLY. The upstream provider
keys (DO_AI_API_KEY, ANTHROPIC_API_KEY) are provisioned as env, injected
from the KMS-synced cloud-api-llm-keys secret — they are NOT sealed in the
embedded KMS store. So GetSecret missed, the resolver returned an empty key,
and zen's call to DO GenAI answered 401 'Unable to authenticate you'. Every
zen chat failed at the upstream while ai (which reads the key from env) worked.

Try the sealed KMS value first (so completing sealed-store provisioning later
needs no code change), then fall back to env. Absent from both still returns
'' so the call fails fast — never silent free usage. Tests cover env-fallback,
sealed-precedence, and absent-everywhere.
2026-07-14 17:29:07 -07:00
hanzo-dev ec86c4b67b fix(zen): resolve upstream provider keys with env fallback (fix DO 401)
zenKeyResolver read the co-resident KMS store ONLY. The upstream provider
keys (DO_AI_API_KEY, ANTHROPIC_API_KEY) are provisioned as env, injected
from the KMS-synced cloud-api-llm-keys secret — they are NOT sealed in the
embedded KMS store. So GetSecret missed, the resolver returned an empty key,
and zen's call to DO GenAI answered 401 'Unable to authenticate you'. Every
zen chat failed at the upstream while ai (which reads the key from env) worked.

Try the sealed KMS value first (so completing sealed-store provisioning later
needs no code change), then fall back to env. Absent from both still returns
'' so the call fails fast — never silent free usage. Tests cover env-fallback,
sealed-precedence, and absent-everywhere.
2026-07-14 17:29:07 -07:00
hanzo-dev 19233b485d fix(zen): resolve upstream provider keys env-first, then KMS
zenKeyResolver read the embedded KMS store ONLY. The operator injects the
provider keys (DO_AI_API_KEY, ANTHROPIC_API_KEY) as env from the KMS-synced
K8s secret cloud-api-llm-keys, but that value is not seeded into the embedded
ZapDB KMS store — so GetSecret missed, the resolver returned an empty key, and
zen's upstream call to DO GenAI answered 401 'Unable to authenticate you'.
Every zen chat failed while ai (which reads the key from env) worked.

Read env first, then KMS — the same order ai uses (object/kms.go). One key
source of truth shared by both zen and ai. Empty on both still returns '' so
the call fails fast, never silent free usage.
2026-07-14 17:24:48 -07:00
zeekay 741644dd25 unfork(commerce): import hanzoai/commerce v1.47.1 — delete the 1,397-file inlined fork
One canonical commerce repo, one-way dependency (cloud → commerce), no more
monorepo-split force-pushes to keep two trees in sync.

- clients/commerce/ (fork, no go.mod) DELETED; cloud imports the module
- subsystems/commerce.go: the ONE adapter — narrows cloud.Deps, boots
  commerce.Embed, mounts the gin handler at the commerce prefixes, wires the
  two in-process seams; luxfi/log imported plainly as log
- consumer bridges move cloud-side (they read cloud seams, not commerce):
  clients/metering    ← fork metering (finance-coupled billing-gate client)
  clients/commerceclient ← in-process entitlement client + BalanceCents
  (separate from commerceinproc: the entitlement client imports clients/plan,
  which imports cloud — commerceinproc must stay stdlib-only for build.go)
- commerce API path: api/api flattened to api (module v1.47.1)
- middleware precedence regression tests moved INTO the module beside the
  accesstoken fix they pin; the in-proc dispatch e2e stays in clients/metering
- subsystems/wire_test: freeze gitops (main had 83 specs vs 82 frozen)

Test surface green: subsystems, commerceclient (real embedded-ledger money
tests), commerceinproc, metering, catalogsync, bots, admin/finance, ml; root
package fails ONLY the 11 pre-existing env-gated tests (cek master key),
identical to origin/main.
2026-07-14 17:08:40 -07:00
hanzo-dev 5407638be7 cli: map agent Fable tier to zen5-max (top SKU)
Fable is agent's top model tier; it was pinned to zen5-pro, the
same as Opus. Point it at zen5-max (the largest zen5 SKU: Qwen3.5-397B ->
1M overflow) so the CC tier ladder is monotonic: Haiku->zen5-flash,
Sonnet->zen5, Opus->zen5-pro, Fable->zen5-max.
2026-07-14 16:20:45 -07:00
hanzo-dev 82d203a8a7 deps: o11y v1.5.28 + otel-collector v1.2.0 — evict koanf v1 monolith
Bumps the embedded o11y (v1.5.26->v1.5.28) and otel-collector
(v0.144.13->v1.2.0) to their koanf-v2 releases and drops the obsolete
otel-collector v0.144.10=>v0.144.13 replace. Removes the ambiguous
github.com/knadh/koanf/maps import (bundled v1.5.0 monolith vs split
module) that broke 'go build ./cmd/cloud'. Cloud builds green.
2026-07-14 16:20:45 -07:00
hanzo-dev 35730bd31a P2: /v1/gitops deploy dashboard API (clients/gitops)
The ArgoCD-grade GitOps control plane over the operator App CRs, native to the
cloud binary and parallel to /v1/git. SuperAdmin-only, fail-closed, Secrets never
surfaced. The console dashboard consumes these shapes:

  GET  /v1/gitops/applications        list: name, role, version(declared),
                                      runningVersion, health, sync, phase, endpoints
  GET  /v1/gitops/{name}/tree         flat node list (ArgoCD ApplicationTree) with
                                      ownerRef parentRefs + per-node health
  GET  /v1/gitops/{name}/resource/{ref} live manifest + desired-vs-live diff
                                      (ref = group:kind:namespace:name from a node)
  GET  /v1/gitops/{name}/logs         newest app pod logs (tail/container bounded)
  POST /v1/gitops/{name}/rollback     pin CR image tag to a prior semver — REUSES
                                      the P1 release seam (cloud.OnServiceRelease)
  POST /v1/gitops/{name}/sync         request an operator reconcile now

- health.go: per-resource health in the ArgoCD vocabulary (Healthy/Progressing/
  Degraded/Suspended/Missing), pure — P2b swaps to gitops-engine pkg/health.
- CR kind-collapse compat shim: reads BOTH apps.hanzo.ai (kind App, forward) and
  services.hanzo.ai (kind Service, live) — App wins the dedupe; removable
  post-cutover. spec.role surfaced.
- Wired into subsystems.Wire after paas (so the release seam is registered before a
  rollback delegates to it).

TODO seam (follow-on, noted): true GitOps on git.hanzo.ai — RegisterPushBuilder
commits the CR change to the manifest repo and the engine syncs repo→cluster;
desiredSource flips last-applied → git with no shape change.

Tests: health matrix, sync, ref-parse, membership, diff, observe, App-first/Service
-fallback resolution, list dedupe, tree ownerRef+selector. go build + test green.
2026-07-14 16:20:45 -07:00
hanzo-dev 18d086d072 ci(release): make the migration-smoke shared volume writable (fix baseline INFRA fail)
The two-boot migration smoke shares a docker named volume across boots, but a fresh
named volume is root:root 0755 while the cloud image runs non-root — so the baseline
(v1.799.19) could not create cek's <db>.cek.lock under /data and aborted before
"listening" ("cek: open lock ... permission denied"), failing the gate on infra, not a
regression. The single-boot smoke only passed because --tmpfs is world-writable.

chmod the shared volume 0777 via a root helper before the baseline boot, and again
between boots so the candidate can read the baseline's files even if runtime UIDs differ.
2026-07-14 16:20:09 -07:00
hanzo-dev 3c48fb8c0c fix(commerce): mount /v1/billing/auto-recharge as a commerce prefix — unbreak the durable-cron sweep poke
The durable platform cron (clients/cron) fires cron-billing-autorecharge every
15m as a poke: POST cloud.hanzo.svc:8000/v1/billing/auto-recharge/run-all with
the COMMERCE_SERVICE_TOKEN bearer. That path was not a commerce prefix, so it
fell through to the account-bridge /v1/billing/* catch-all, whose session gate
403s a service token ("sign in to view billing"). Live fires have been failing
on exactly that — verified in-pod: the poke returns 403 on v1.799.13.

Add /v1/billing/auto-recharge to commercePrefixes alongside /v1/billing/webhooks
— same class of route (token/signature IS the auth, no session possible).
commerce mounts at Wire order 100, ahead of the bridge, so the poke reaches gin
where commerce's own TokenRequired service-token branch + PlatformOnly gate
authenticate it. No other /v1/billing/* route changes owner.

TestAutoRechargePrefixMounted pins both prefixes so a future edit can't silently
re-break the sweep. Landed directly on main: the identical change merged four
times (#274/#275/#277/#280) and was each time force-pushed off main or closed +
branch-deleted; a PR is not a durable landing surface here.
2026-07-14 16:17:24 -07:00
hanzo-dev a5e34d8f6c Merge fix/release-migration-smoke: index-after-ADD-COLUMN store migrations + regression harness 2026-07-14 16:04:56 -07:00
hanzo-dev 36be0c4d40 fix(stores): create indexes over ALTER-added columns after the ADD COLUMN pass
tracker.migrate() indexed issues(org, repo) and issues(org, kind) in the base
DDL, but repo/kind are ALTER-added. On a legacy tracker.db (CREATE TABLE IF NOT
EXISTS no-ops) those indexes fail "no such column", migrate() fails, mount
fails, and the pod crashloops on deploy — the same class already fixed in
wallets and affiliates. Move both indexes after the ALTER pass.

Prevent recurrence with a shared regression harness (internal/migratetest):
each store contributes a legacy-DDL case that seeds its pre-migration schema and
asserts migrate() succeeds and is idempotent, proving migration-correctness as a
pure test over schema epochs instead of at prod boot. Cases added for tracker
(with a scoped-insert probe) plus the other ALTER+index stores — agents, social,
platform, provisioning, projects — which audit clean and are now locked.
2026-07-14 16:04:49 -07:00
hanzo-dev 0ccbd2eb16 ci(release): migration smoke — boot candidate over the prior release's on-disk schema
The plain smoke boots on a fresh /data, so every migrate() takes its CREATE-TABLE
path and no forward-migration runs — an index over a not-yet-ADDed column is valid
on a fresh store yet crashes on a pre-existing one (affiliates referrer_org in
v1.800.1, wallets project/agent before it), which is how a boot-crash reached prod
and took api.hanzo.ai down while every smoke stayed green.

Reproduce the real upgrade path: boot the prior released image (default
ghcr.io/hanzoai/cloud:v1.799.19, override via SMOKE_MIGRATION_BASELINE) to lay its
cek-encrypted on-disk schema into a persistent volume under one shared throwaway
master key, then boot the candidate over the same volume and require "listening". A
migrate() that assumes a fresh store fails the gate before any image is pushed.
2026-07-14 15:46:42 -07:00
hanzo-dev 986588eecd P1: native release seam — RegisterServiceReleaser patches services.hanzo.ai CR
Close push→build→image→CR: a proven, clean-semver image rolls live by patching
the matching operator hanzo.ai/v1 Service CR's spec.image directly, so the
operator reconciles the Deployment. This is the in-cluster, direct-CR replacement
for universe's image-update.yml GitOps hop (repository_dispatch → PR → ArgoCD),
with the same determinism (clean-semver only; resolve CR by metadata.name) and no
git round-trip.

- build.go: RegisterServiceReleaser / OnServiceRelease / ServiceReleaserRegistered
  inversion seam (mirrors RegisterPushBuilder), so a build-completion path rolls a
  proven image with no cloud⇄paas import cycle.
- clients/paas/release.go: releaseService — resolve CR by name (main-first),
  clean-semver gate (IsSemverTag), idempotent merge-patch of spec.image; registered
  at Mount as the releaser impl (paas owns the first-party Service CR plane).
- clients/platform/release.go: rolloutRelease — native CR patch primary, universe
  image-update dispatch kept as an additive GitOps mirror during cutover.

Tests: split/gate, patch, idempotent, reject-floating, unknown-service, main-first,
fail-closed, seam dispatch/no-op. go build + go test green (paas, platform, root).
2026-07-14 15:45:02 -07:00
hanzo-dev 45dd347761 Merge: fix affiliates migrate index order (cloud boot crash) 2026-07-14 15:22:16 -07:00
zeekay 894e49dcfd fix(affiliates): create referrer_org index AFTER ADD COLUMN (boot crash)
On a store whose affiliate_referrals table predates referrer_org, CREATE TABLE
IF NOT EXISTS is a no-op, so 'CREATE INDEX ... ON affiliate_referrals(referrer_org)'
in the same DDL batch failed with 'no such column: referrer_org' BEFORE the
ALTER ... ADD COLUMN pass ran — crashing cloud on boot (v1.800.1 CrashLoopBackOff,
api.hanzo.ai down). Move the index creation after ADD COLUMN so it is valid on
both a fresh store and a migrated one. Regression test seeds the old schema and
asserts migrate succeeds + backfills (it fails with the exact prod error when the
index is moved back into the DDL batch).
2026-07-14 15:22:16 -07:00
hanzo-dev 4b94fb9699 Merge: hanzo code claude self-identifies as a Hanzo Zen model (--append-system-prompt) 2026-07-14 15:21:18 -07:00
hanzo-dev 21a4807126 feat(code): claude agent appends the Hanzo Zen identity to its system prompt
`hanzo code claude` already pins the model to a zen5 alias (default zen5, the
GLM-5.2-class tier) and forces it on argv so a persisted /model selection
("best") cannot override it — but agent's base system prompt still tells
the model it is Claude, so a Hanzo-served model self-identifies as Claude when
asked. Append the Hanzo Zen identity via --append-system-prompt (an APPEND, not
--system-prompt: CC keeps its harness prompt for tool-use/safety/coding) so the
served model says it is a Hanzo Zen model. The identity is not a permission
bypass, so it is applied in --safe too (unlike --dangerously-skip-permissions).

codex/dev (OpenAI wire) are unchanged — the append is Anthropic-only.
2026-07-14 15:19:53 -07:00
hanzo-dev b2a4048815 Merge: bare hanzo + configurable default coding tool 2026-07-14 15:04:48 -07:00
zeekay 0495666e24 feat(cli): bare hanzo + configurable default coding tool
- `hanzo` (no args): log in if needed, then drop into the configured agent on
  a Hanzo cloud model — one word, billed to your account, all zen-native.
- `hanzo code` (no agent): runs the default agent instead of showing help.
- Default agent resolves HANZO_CODE_TOOL, then config `code_tool`, else dev.
- `hanzo config set code_tool claude|codex|dev` + `code_model` — git-style k/v,
  persisted to ~/.hanzo/config.
A named agent (`hanzo code claude`) still dispatches to its subcommand.
2026-07-14 15:04:48 -07:00
hanzo-dev 7a9b532f49 wallets: migrate columns before indexing them (fix deploy crashloop)
migrate() created ix_wallets_scope/ix_wallets_finance over project/agent/
finance_account inside the base DDL, before the idempotent ALTER TABLE that
forward-adds those columns. On a wallets table created before scoping (the prod
shape), CREATE TABLE IF NOT EXISTS no-ops, so the index build hit 'no such
column: project' -> migrate fails -> mount fails -> pod crashloop on any newer
image. Order by dependency: base tables, then ALTER-add every post-original
column (project/agent/chain/finance_account), then the indexes over them.

Regression test opens a legacy-schema wallets.db and asserts clean, idempotent
migrate + a fully-scoped insert.
2026-07-14 14:54:01 -07:00
hanzo-dev ae24015e2c Merge: resolve API keys to a principal at the identity edge (fixes zen 402) 2026-07-14 14:25:50 -07:00
zeekay c9b122830a feat(auth): resolve API keys to a principal at the identity edge
The identity boundary validated JWTs only; an opaque API key (hk-/sk-/pk-)
yielded no principal, so a subsystem that gates on the minted identity (zen's
billing gate) refused a key request as anonymous — the zen 402 'a billable
tenant is required' for a funded hk- key.

keyResolver turns a key into the SAME idClaims a JWT yields (via IAM's
authenticated get-user?accessKey, the confidential hanzo-console client), so the
ONE minting path serves both credentials and key auth and session auth can never
disagree on who a request is. An unresolved key stays anonymous — a bad key
never grants trust; an unconfigured resolver keeps keys anonymous rather than
mis-resolved. Brief generic TTL cache keeps the hot path off the network.
2026-07-14 14:25:50 -07:00
hanzo-dev a0fb6f048d fix(coding): terminal ops outlive run ctx; cloud→bot POST refuses cleartext
Red LOW-6: a timed-out coding run must still close the session and mirror
its terminal result. Run terminal-side ops (fail/CloseSession/mirror/CreatePR)
under context.WithoutCancel so an expired run ctx cannot strand a session in
'running'.

Red MEDIUM-3: the cloud→bot coding POST carries the org hk- git credential and
the shared gateway bearer. Fail closed on a cleartext http:// target; a plaintext
in-cluster hop is allowed ONLY when the operator asserts mesh mTLS via
BOT_GATEWAY_ALLOW_PLAINTEXT=1. Error never echoes the credential.
2026-07-14 13:59:57 -07:00
zeekay 0b8aa9b748 deps: bump hanzoai/zen v1.2.0→v1.3.0 (cost-basis + margin pricing) 2026-07-14 13:32:08 -07:00
blue 28ceaec797 style: gofmt coding orchestrator + tracker agentpr + slack_coding test 2026-07-14 13:22:13 -07:00
blue dd4c95b44d feat(coding): @hanzo agent keystone — native coding tasks from Slack
Turn @hanzo from a chatbot into an engineer. A Slack message
`@hanzo code: <repo> <task>` branches off the chat-only reply into a
durable coding run: register a live agent session, dispatch to the
bot-gateway sandbox, mirror progress into the session live, verify the
pushed branch landed in native /v1/git, open a native PR work item, and
report the branch + PR back in-thread. Non-code mentions keep the
existing chat path unchanged.

- clients/coding: transport-agnostic orchestrator (Dispatcher over
  interface seams; unit-tested with fakes, org-isolated, no credential
  leak into session events/PR body). Cannot import git (cycle via
  integrations) so CloneURL/VerifyRef are injected at the composition root.
- clients/agents/inproc: in-process session API (Open/Log/Close) — the
  twin of the /v1/agents/sessions control plane, same store + live bus.
- clients/tracker/agentpr: in-process Kind:pr Source:agent work item,
  org-scoped, get-or-create repo board (KEY-N).
- clients/bot/coding: in-process NDJSON client for POST /v1/coding-tasks;
  credential travels in the body only, never argv/URL/logs.
- clients/git/export: CloneURL + VerifyRef seams (org-scoped ref check).
- clients/integrations/slack_coding: the code: trigger, credential fetch
  from KMS (fail-closed), ack, detached bounded run, result Block Kit card.
- subsystems/wire_seams: compose the Dispatcher (git seams + adapters)
  and inject into the Slack surface.

Tenant isolation fail-closed: org is the only tenant key on every seam;
sandbox pointed only at the caller org's clone URL with an IAM-scoped
credential; cross-org repo targeting is refused by git's path-vs-identity
guard. Tests: CGO_ENABLED=0 go test green across all touched packages.
2026-07-14 12:45:37 -07:00
zeekay c6e69b502d deps: bump hanzoai/zen v1.1.0→v1.2.0 (coherent lineup + thinking effort)
Pulls in the reworked zen family (one SKU per capability, every upstream
verified on DO) and the hanzoai/thinking depth fold (effort → each upstream's
native reasoning shape). Adds hanzoai/thinking v0.1.0 as a direct require.
2026-07-14 12:40:54 -07:00
hanzo-dev 7e4ddcbc44 Merge: hanzo code claude defaults to zen5 (GLM-5.2) 2026-07-14 12:39:26 -07:00
zeekay 8f466533db feat(cli): hanzo code claude defaults to zen5 (GLM-5.2)
zen5 is the flagship GLM-5.2-class alias (1M ctx, tool-capable — the glm-5.2
upstream returns tool_use/stop_reason:tool_use, verified live). Matches the
intent to launch agent on GLM-5.2 through api.hanzo.ai. Was zen5-pro
(DeepSeek). The four CC tier slots stay pinned to served zen5 aliases so the
classifier, subagents, and /compact never hit an unserved claude-* id.
2026-07-14 12:38:19 -07:00
hanzo-dev 8244dd0388 git lifecycle: escape branch name in Slack blocks (Red residual LOW)
ev.Branch was an un-escaped mrkdwn sink: git refnames allow < > & !, and the
receive-pack fire path (branchTips → for-each-ref) applies no branchRE, so a
hostile branch (e.g. x<!channel>y) flowed verbatim into the summary and the
*Branch* field of every subscribed channel. slackEscape it in both places, and
defensively escape the org/repo display text too (a deploy event's repo derives
from an unconstrained RepoURL, though it is subscription-gated to a valid name).

TestNotifyEscapesMrkdwn now also pushes a hostile branch through smart-HTTP with
the real git CLI (go-git rejects such a refspec) and asserts it is neutralized in
both the summary and the Branch field.
2026-07-14 12:06:41 -07:00
hanzo-dev 3741c9adcf git lifecycle: address Red review (fix-then-ship)
HIGH-1 mirror-out no longer starves the shared pack plane: dedicated mirrorSem
(separate from packSem), per-push context.WithTimeout, git http.lowSpeed abort,
and cmd.WaitDelay so a stalled downstream's network-helper child can't wedge the
slot past the deadline.

MED-2/MED-3 full (org,project,repo) identity: subscriptions + mirror targets key
on project too (DDL + every list/delete + the notify + mirror fire paths); deploy
emitters thread project (platform a.ProjectID normalized, projects org-level);
repo delete cascade-deletes its subscriptions + mirrors in one tx (no
exfil-on-recreate via an orphaned target).

MED-1 decouple allowlists: outbound mirror TARGET set = {github.com, gitlab.com}
only; the local git host is rejected as a target (no internal SSRF /
privileged-cred presentation). Inbound-fetch credential gate unchanged.

MED-4 Slack mrkdwn escaping of user-derived text (commit subject, pusher, deploy
detail) so a crafted commit subject can't inject <!channel>/disguised links.

LOW-1 keep the shared bot token least-privilege (no chat:write.public — it would
also arm the @hanzo assistant to post uninvited); notifications require the bot
be invited. INFO: reject non-deliverable build.started at subscribe time; DRY
repoFromURL into cloud.RepoFromCloneURL.

Re-verified: new tests for project-scoped routing, delete-cascade, mrkdwn escape,
and stalled-downstream isolation; suites green under -race; gofmt+vet clean;
go.mod untouched.
2026-07-14 11:49:03 -07:00
ec4112183a test(subsystems): base gained Shutdown via per-org embed (#298) — sync frozen wire order (#300)
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-14 11:23:45 -07:00
c307842105 analytics: capture (write) plane — POST /v1/analytics + /v1/tracker → hanzo.events (#296)
* analytics: add capture (write) plane — POST /v1/analytics + /v1/tracker → hanzo.events

The analytics subsystem served only read lenses over hanzo.events; nothing
wrote the table, so the web/commerce lenses were permanently honest-empty. This
adds the symmetric ingest: products POST batches to cloud (the ONE native front
door) and cloud writes org-scoped rows into the datastore warehouse the read
side already queries.

- POST /v1/analytics, /v1/analytics/batch, /v1/tracker (beacon alias) — all
  tenant-gated in-handler; tenant_id is always principal.Org, never client input.
- Writes ride ai/object.DatastoreExec (the SAME pooled client the reads use).
- The writer owns the hanzo.events DDL (EnsureEventsTable, idempotent/latched).
- Privacy scrub: credential/PII-shaped property keys dropped, email values
  redacted, before any row is built.
- Pure core (normalizeEvent/scrubProps/buildEventsInsert) unit-tested; HTTP
  contract tests cover no-principal 403, forged-org 403, oversized 400,
  datastore-down 503; a build-tagged live test proves the full round trip
  against a real datastore.

* analytics: accept anonymous capture, attributed to the brand-public org

Marketing sites emit anonymous pageviews (no session). captureTenant now falls
back — when there is no validated principal — to the PUBLIC brand org derived
SERVER-SIDE from the request Host via the white-label registry (BrandForHostOK),
never a client-claimed org. A forged X-Org-Id is still ignored, and an
unrecognized Host is refused (anonymous events are never dumped into a default
org). Gated by CLOUD_ANALYTICS_PUBLIC_CAPTURE (default on, matching the existing
public insights-capture posture). Verified live: an anonymous pageview to
Host hanzo.ai lands under tenant_id=hanzo.

---------

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-14 11:21:54 -07:00
7583c9394a marketing: grow /v1/marketing into the full GTM engine (#299)
Fold the remaining GTM subsystems into clients/marketing (native Go, per-org
SQLite, twin of clients/crm), so nothing Python is load-bearing in the mount
path:

- Email drip sequences on the embedded hanzoai/tasks engine (drip.go): a
  per-minute durable schedule sweeps due enrollments; each (enrollment, step)
  is claimed once so a step delivers at most once across restarts/redelivery,
  then the walk advances or completes. Mirrors clients/cron (engine owns time,
  SQLite owns the schedule) — no Redis, no bespoke ticker.
- The ONE send seam (suppress.go): every marketing delivery funnels through
  state.deliver, which enforces the per-org suppression/opt-out list and then
  hands off to the platform notify rail. Expose notify.Send so the existing
  sender is reachable in-process — one sender, not a second. Plus a signed
  public one-click unsubscribe.
- Audiences (audiences.go): cohort filters evaluated live against the org's
  hanzo.events analytics via the ai/object datastore, tenant_id-scoped and
  honest-empty when the warehouse is not wired.
- Promo codes (promos.go): the First-1,000 90%-off launch promo (discounts.md)
  realized as a non-cash wallet credit through the finance ledger, with the
  hard 1,000 cap, one-per-org, one-per-instrument and team-seat-cap guards.
- Content calendar (calendar.go): scheduled posts as documents published by a
  task-executed hook; social publish returns an honest 501 until a connector
  is wired (clients/social's push is fail-closed, the automations connector
  registry is package-private).
- Campaign scheduling (scheduled_at + the scheduled state).

Real tests: drip scheduling + per-step idempotence + tenant isolation,
suppression enforcement at the gate, promo eligibility math + abuse guards.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-14 11:21:24 -07:00
hanzo-dev 19e2513cff git lifecycle event stream + Slack-notify and outbound-mirror subscribers
Generalize the single-registrant git push→deploy hook into a many-subscriber
lifecycle stream without regressing push→deploy. RegisterPushBuilder/OnGitPush
stay exactly as-is (deploy is the subscriber-of-record); alongside them a new
RegisterLifecycleSubscriber/EmitLifecycle fans ONE LifecycleEvent out to N
reactors, best-effort, detached, on a cancel-immune context, panic-contained.

Emit points: PushLanded at the one branch-build funnel (covers HTTP/SSH/push,
now carrying before/after tips + pusher); BuildStarted/DeployLive/DeployFailed
at the platform (startGitBuild / applyLive / failDeploymentCtx) and projects
(deployGit / deployArtifact / completeDeployment) transitions.

Subscriber 1 — Slack notify: per-org repo→channel subscriptions
(/v1/git/repos/:name/subscriptions) stored in the existing per-org git.db,
delivered as Block Kit via the ONE integrations chat.postMessage path
(integrations.NotifySlack/PostSlackBlocks; the automations connector now shares
it). Adds chat:write.public so a freshly-subscribed public channel works
without a manual invite.

Subscriber 2 — outbound mirror: per-repo downstream targets
(/v1/git/repos/:name/mirrors) force-push ONLY the advanced branch to
allowlisted hosts (github.com/gitlab.com/git.hanzo.ai), token via env-only
http.extraHeader under the pack-slot semaphore. LifecycleEvent.Origin is the
loop-prevention seam for a future inbound sync.

All routes org-scoped + fail-closed; tenant isolation, injection, allowlist,
and loop-prevention covered by tests.
2026-07-14 11:17:52 -07:00
zandGitHub 9301e57ed4 Merge pull request #297 from hanzoai/feat/company-formation
Hanzo Company — incorporation + fundraising state machine (/v1/company)
2026-07-14 11:17:14 -07:00
4a0b40aa71 feat(base): per-org multi-tenant Base hosting at /v1/base/* (kill superbase wrapper) (#298)
Upgrade the in-process Base embed from the single-instance waitlist-only fold
(#193/#211/#248) into two orthogonal lanes on the ONE engine:

  - LANE 1 (unchanged surface): the platform waitlist app → public /v1/waitlist/*.
  - LANE 2 (new): managed Base hosting → ONE Base app PER ORG, opened lazily and
    LRU-pooled, each on its own SQLite under {DataDir}/base/{TenantSegment}/ (the
    HIP-0302 'SQLite per tenant' model the gojabase leaves use). Served
    authenticated under /v1/base/*, the org resolved from the VALIDATED cloud
    principal (principal.Org) — physical per-org isolation, the console Bases
    manager's backend and the in-binary replacement for the superbase pod.

Base serves under BASE_API_PREFIX=/v1/base so its collections API mounts natively
(self-URLs included) without colliding with cloud's other /v1 routes; the waitlist
plugin binds a FIXED /v1/waitlist regardless. Per-org apps validate bearers against
Hanzo IAM's JWKS as their exclusive auth source (apis.StoreKey{JWKSURL,
ExternalAuthOnly}) — ONE IAM, no second auth path. The cloud binary owns the ZAP
transport, so embedded apps set ZAP_DISABLED. Shutdown releases the platform app +
every pooled per-org app.

Reuses gojabase.TenantSegment (the ONE injective, traversal-safe org->path encoder).
Test: subsystem boots in the harness, a collection record round-trips per-org
isolated over the real HTTP path (acme's record invisible to globex; distinct
on-disk dirs).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-14 11:16:33 -07:00
hanzo-dev 1e363e9475 captable: harden the http_test fiber timeout (pre-existing flake); wire company after guide/x402 in the frozen order 2026-07-14 11:12:53 -07:00
hanzo-dev ff57ba838f company: make genesis idempotent (no double share issuance) + propagate save error 2026-07-14 11:10:51 -07:00
hanzo-dev e01eb60d90 feat(company): Hanzo Company — incorporation + fundraising state machine
Adds clients/company (/v1/company): one formation state machine per org
(structure → founders+KYC → $999 → documents → esign → on-chain equity
genesis → company), with a SKIP path for already-incorporated orgs that
imports corporate docs → data room and a cap-table sheet → captable.

- machine.go: pure transition table + per-edge guards; the payment gate,
  KYC gate, and skip path are unit-tested with no I/O.
- providers.go: narrow seams for billing/kyc/docs/esign/captable/anchor/
  filing/upgrade. Billing wires the shared ResourceMeter ($999). KYC and
  state filing are honest stubs (no fabricated verification/filing).
- genesis.go: KMS-signed Hanzo-L1 equity-genesis anchor mirroring
  clients/treasury; computes the root always, commits on-chain when wired,
  honest pending otherwise.
- adapters.go + captable.facade + dataroom.Ingest: new in-proc facades so
  company writes the cap table and data room without an HTTP hop.
- google provider completed in clients/integrations (OAuth + KMS token
  custody); the automations google_sheets/google_drive connectors are
  consolidated to one `google` connector sharing that token; company import
  reads Drive/Sheets through it.
- wired into subsystems.Wire() after referrals; docs/company-dogfood.md
  walks Hanzo/Lux/Zoo through the import path.
2026-07-14 11:10:51 -07:00
82b83297fe feat(guide): Business AI Guide — /v1/guide launch checklist + agent (#282)
clients/guide + /v1/guide/*: a per-org checklist engine over a
machine-readable curriculum (Step: id/title/why/how/done/dependencies/
signal/tool). Per-org progress on cloud.OrgStore; next-step + dependency
gating are pure functions; auto-detect reconciles a step to done when its
signal maps to real org state (acted = agent action ledger; analytics =
shared warehouse events). Business AI 'do it for me' drafts with deps.AI
then executes the step's bound MCP tool via automations.InvokeTool AS THE
CALLER — attributable, metered, audited, never exceeding the caller's
authorization. Built-in default.yaml (7 steps: positioning→landing→
analytics→waitlist→email→referral→launch) seeds it before marketing's
checklist.yaml; an org-custom PUT replaces it cleanly.

clients/automations: decomplect tool dispatch (dispatchTool) from its two
doors — the HTTP MCP handler and the new in-process InvokeTool/ToolExists
seam — so both share one dispatch, org-scoped credential, concurrency
bound, meter and audit.

Tests (pure-Go, CGO_ENABLED=0): validation/cycle-detection, next-step +
dependency gating, auto-detect reconcile (present/absent/error/terminal),
the acted detector end-to-end, the agent (tool success/failure/assisted/
unknown-tool), and the HTTP surface (403 gate, transitions, 409 gating,
curriculum replace/revert, per-org isolation, do-delegation).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-14 11:05:32 -07:00
b89f0f9602 feat(growth): multi-level referral upline + OSS-author royalties in one accrual walk (#295)
Extend the existing affiliate accrual into a depth-capped multi-level upline and
fold OSS-author royalties into the SAME per-org spend walk, over one attribution
spine.

Affiliates — multi-level upline (clients/affiliates):
- The referredBy graph is the existing affiliate_referrals edge made walkable
  (referred_org -> referrer_org, denormalized). referredBy is set-once/immutable
  (UNIQUE referred_org) with cycle detection at set time (wouldCycleOrg climbs the
  proposed referrer's upline and refuses a loop).
- Per-level schedule L1 20% / L2 5% / L3 2%, depth cap 3. L1 uses the affiliate's
  own negotiable rate (default 20%, preserving prior single-level behavior); L2/L3
  are platform constants. accrual rows carry the level for analytics.
- The admin sweep is source-centric: fold over every referred org, read spend ONCE,
  walk the upline and accrue to each ancestor's approved affiliate, latched
  at-most-once per (affiliate, source, period). The per-affiliate dashboard read
  walks the downline (same latch key, never double-accrues).
- User-level referredBy graph (user_referrals): set-once + cycle-checked, mirroring
  the org edge; recorded from the referee's user to the affiliate's owner user.
- Payout no-overdraw guarantee unchanged (pending-guard + treasury reserve backing).

Authors — OSS royalties (clients/authors):
- Default author share 25% (was 5%).
- GitLab provider alongside GitHub: host-aware repo canonicalization (github.com +
  gitlab.com), provider-dispatched forge seam (linkedAccount/repoAdmin/fetchFile).
- Append-only, on-chain-ready royalty ledger (author_ledger) with a nullable
  compute_proof column, written per accrual in the same transaction as the balance
  move. compute_proof stays NULL — the hanzod attestation is a follow-up, not faked.
- AccrueForOrg seam: the affiliate sweep drives author royalty for each source org
  with the spend it already read — one accrual walk. Nil-safe when authors is
  unmounted (mirrors treasury.Reserve).

Surfaces:
- GET /v1/affiliates/me — my code, link, downline by level (L1/L2/L3 with rates +
  counts), accrued/pending/paid, payouts.
- GET /v1/admin/referrals — unified SuperAdmin cross-tenant analytics (top referrers,
  conversion, accrual liability by level). The one-time-bonus board moves to
  GET /v1/admin/referrals/bonuses (referrals) so the two compose without colliding.

Tests: 3-level walk math + depth cap, cycle rejection (org + user), set-once
immutability (org + user), no-overdraw payout, GitLab verify + ledger row with NULL
compute-proof, both surfaces. Deterministic fiber test timeout (30s ceiling; the 1s
default flaked under machine load).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-14 10:53:48 -07:00
f7b5178f0d kb: wikilink graph — extraction, /v1/kb/graph, and vault import (#294)
Complete the native knowledge graph on the kb subsystem.

Wikilink edges: the kb-page after_save hook now extracts [[Page Title]]
references from the page body (the same flattened Lexical text the vector
indexer embeds) and reconciles them into kb-link edge documents — a Link
(source) + Data (target_title) reference, no parallel store. Index and link
maintenance run as one combined page hook so a vector outage never skips link
extraction. Targets resolve to a page by value at read time, so a rename or
trash of a target needs no edge rewrite; a page's own trash removes its
outgoing edges.

GET /v1/kb/graph: the org's knowledge as nodes (kb-page/kb-memory/kb-source,
plus connector and dangling-link endpoints) and edges (parent tree, wikilinks,
connector provenance), org/project scoped, shaped for a force-directed
renderer.

POST /v1/kb/import: an Obsidian-importer-equivalent that ingests an Obsidian
vault zip, Notion export zip (markdown/HTML), Evernote .enex, or Roam JSON as
a kb-page tree with links preserved. Each format is a pure normalizer package
(obsidian/notion/roam/evernote over vault + lexical), mapping to pages filed
through the same framework.Ingest path a connector sync uses — the after_save
hook then extracts their wikilinks, one link path for authored and imported
pages alike.

framework: add the in-process Delete (twin of the HTTP delete, runs on_trash)
used by edge reconciliation.

Tests: table-driven wikilink extraction; per-format normalizer tests with real
fixture files; end-to-end graph, import, and full edge lifecycle (extract,
reconcile-on-edit, source-trash cleanup, target-trash dangling) over the real
framework store.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-14 10:53:02 -07:00
hanzo-dev 02fab4bc40 analytics: accept anonymous capture, attributed to the brand-public org
Marketing sites emit anonymous pageviews (no session). captureTenant now falls
back — when there is no validated principal — to the PUBLIC brand org derived
SERVER-SIDE from the request Host via the white-label registry (BrandForHostOK),
never a client-claimed org. A forged X-Org-Id is still ignored, and an
unrecognized Host is refused (anonymous events are never dumped into a default
org). Gated by CLOUD_ANALYTICS_PUBLIC_CAPTURE (default on, matching the existing
public insights-capture posture). Verified live: an anonymous pageview to
Host hanzo.ai lands under tenant_id=hanzo.
2026-07-14 10:52:17 -07:00
hanzo-dev 4dcd1d6e8f analytics: accept anonymous capture, attributed to the brand-public org
Marketing sites emit anonymous pageviews (no session). captureTenant now falls
back — when there is no validated principal — to the PUBLIC brand org derived
SERVER-SIDE from the request Host via the white-label registry (BrandForHostOK),
never a client-claimed org. A forged X-Org-Id is still ignored, and an
unrecognized Host is refused (anonymous events are never dumped into a default
org). Gated by CLOUD_ANALYTICS_PUBLIC_CAPTURE (default on, matching the existing
public insights-capture posture). Verified live: an anonymous pageview to
Host hanzo.ai lands under tenant_id=hanzo.
2026-07-14 10:52:17 -07:00
d251740a37 feat(wallets,x402): scoped custody {org,project,agent,account} + native x402 pay-per-use (#293)
Wallets custody scoping (clients/wallets)
- One Scope type {org, project, agent, account} is the ONE key both the KMS
  secret ref (keyRef) and the store lookup derive from. Org stays the hard
  isolation boundary; project/agent/account are optional narrowings within it.
- keyRef derives from the full scope, injection-safe (narrowings validated to a
  slash-free segment). An org-only wallet keeps its exact legacy ref, so scoping
  is additive, not a migration.
- listWalletsByScope is the one scope-filtered read path (org bound, narrowings
  filter within); create + list handlers thread the scope. Store gains project +
  agent columns (forward ALTER, dup-column tolerant).

Native x402 pay-per-use subsystem (clients/x402, /v1/x402)
- challenge (402 + PaymentRequirements) -> client signs ERC-3009 -> proof ->
  Verify (EIP-712 secp256k1 recovery via luxfi/crypto, the same primitive wallets
  signs with) -> settle -> serve. Enforce is a zip middleware a priced route
  group applies.
- Idempotent + replay-safe: settlement id is deterministic in (from, nonce); a
  spent nonce reused for different terms is a replay (402), a re-submitted
  authorization is an idempotent retry (settled + metered once). Two independent
  guards: the PK-atomic store claim and the ledger's own RequestID/Ref idempotency.
- Settlement wires into the metering spine: the payer's org is debited through
  metering so paid usage appears in billing/usage like any metered spend, and the
  recipient wallet's ledger is credited. Ledger settlement is LIVE; on-chain
  broadcast of the authorization is a seam (not wired).
- Marketplace seam: a Registry (Publish) maps resource -> Terms (price + recipient
  wallet ref); x402 resolves the recipient via wallets.ResolvePaymentTarget and
  enforces. The registry itself is another subsystem's work.

Removes the dead, unreferenced clients/commerce/payment/x402 (gin + btcec) so the
binary has exactly one x402.

Tests: scope derivation + injection + scoped seal/sign + scope lookup isolation;
EIP-712 verify round-trip/tamper/term-binding; challenge->verify->serve, nonce
replay rejection, settle-once on retry, free passthrough, payer-required, and
end-to-end ledger settlement (payer debited once, recipient credited once).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-14 10:51:18 -07:00
ba24d1ce97 feat(tools): unified tool plane + marketplace (registry, activation, external-MCP, full-cloud-control, x402 seam) (#292)
* tools: unified tool-plane registry, activation, external-MCP + builtin sources

ONE registry where a tool is {name, source, schema, per-(org,project) activation,
optional price}. Sources register a Provider (List+Dispatch) into it; the registry
enforces the one policy: precedence dedup, activation gate (403), x402 Charger seam
(fail-closed for priced tools), then dispatch. Ships two package-owned sources:
external MCP servers (KMS-sealed auth, SSRF-guarded) and full-cloud-control builtin
(every /v1 route, dispatched in-process replaying the caller credential). Real
tests: precedence, activation 403 + cross-org isolation, priced fail-closed,
external-MCP dispatch, builtin route dispatch, SSRF guard.

* tools+marketplace: register sources, x402 price seam, wire subsystems

Sources register a Provider into the tool plane from their OWN Mount (no source
duplicates its listing logic): connectors (automations), functions, agents, and
skills (discovery-only). Adds the registry Pricer seam + public activation
pass-throughs (Activate/Deactivate/Activated/Exists) so marketplace install IS the
tool activation write and a published listing's price reaches per-call dispatch
enforcement.

clients/marketplace: /v1/marketplace listing/discovery/install over the plane;
monetized listings declare price + recipient wallet and settle through the tools
Charger (x402) seam — proven end-to-end (publish→install→dispatch charges the
seller). Wires tools + marketplace into subsystems.Wire() (frozen-order updated).

Tests: marketplace install==activation + cross-org isolation, phantom 422, publish
validation, monetized dispatch settlement, discovery overlay.

---------

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-14 10:48:38 -07:00
hanzo-dev 684fa2b79b feat(git): repo maintenance — bitmap + commit-graph for fast clones
A repo populated by fetch/receive-pack accumulates objects via index-pack,
which writes a pack but no reachability bitmap and no commit-graph — so every
upload-pack walks the whole object graph to build the clone pack (O(objects)
per clone). Add maintenance on the git-exec seam: repack -adb --write-bitmap-index
+ commit-graph write --reachable --changed-paths, under one pack slot with the
same memory bounds as every pack op (a multi-GB gc can't OOM the pod).

- POST /v1/git/repos/:name/gc — on-demand repack, org-scoped, 404 fail-closed.
- Post-push autoMaintain: fire-and-forget git gc --auto after every receive-pack
  (HTTP + SSH), slot-yielding (skips when clones are busy) so repos self-maintain
  as pushes accumulate packs, no scheduler.

Tests: runMaintenance writes bitmap + commit-graph and the repo stays clonable;
the /gc endpoint repacks (200 maintained) + 404s unknown/cross-tenant repos.
2026-07-14 08:50:26 -07:00
antje 8505bee2a2 deps: bump ai v1.807.0 -> v1.808.0
Bearer-auth get-cloud-usages + brand-scoped god-view (one UsagePanel across
app/chat/billing). Blue+RED reviewed; build+tests green.
2026-07-14 08:32:16 -07:00
hanzo-dev e0d8e5241c fix(git): read the framework-decoded pack request body, don't re-inflate
Fiber's Body() transparently inflates a Content-Encoding: gzip request per
its header (BodyGunzipWithLimit + SetBodyRaw), so c.Body() is already the
decoded pkt-line stream. packRequestBody inflated it a second time and
returned 400 "invalid gzip request body" for every compressed clone/fetch.
git gzips the ref-negotiation once a repo carries enough refs, so this broke
clone for all but trivially small repos — the small-repo round-trip tests
sent the request plain and never exercised the compressed path.

Read c.Body() verbatim (the framework owns Content-Encoding). Add a gzipped
upload-pack regression test through the live Fiber server that fails on the
double-decode and passes on the fix.
2026-07-14 08:16:46 -07:00
hanzo-dev 8f2746057b deps(ai): bump hanzoai/ai v1.806.15 -> v1.807.0
Ships the zen consolidation: ai discovers the Zen family from the zen service
(GET $ZEN_URL/v1/models) and fronts it, holding no zen routing/pricing/identity
of its own. Pulls the corrected per-upstream reasoning vocabulary and the
money/decimal billing types transitively.
2026-07-14 07:43:32 -07:00
hanzo-dev f820ac0a5c feat(git): streaming git-CLI object plane — bounded memory, protocol v2 (red-reviewed GO) 2026-07-14 07:08:45 -07:00
hanzo-dev e4e09768d2 chore(deps): fold released hanzoai/ai v1.806.15 (#290,#291) into main 2026-07-14 07:08:45 -07:00
hanzo-dev 3d5b891e37 git: harden the streaming object plane (red review deltas)
Address the red-team findings on feat/git-streaming-object-plane:

HIGH-1 mirror credential exfiltration: only attach GIT_MIRROR_TOKEN to hosts
on an allowlist (GIT_MIRROR_ALLOW_HOSTS, default {github.com, git.hanzo.ai});
any other https source fetches anonymously so a tenant-supplied URL can't
capture the shared token. Set http.followRedirects=false on the mirror fetch
so the token can't ride a cross-host redirect.

MED-2 pack RAM: add -c pack.threads=1 -c pack.windowMemory=64m
-c pack.deltaCacheSize=64m -c core.bigFileThreshold=16m to every pack/fetch
subprocess, and a concurrency semaphore (GIT_PACK_MAX_CONCURRENCY, default 2)
around all pack ops so concurrent large clones/mirrors can't multiply RAM in
the shared cgroup.

MED-3 org traversal: gate the org identifier through orgRE (alnum-led, no
'/'/'\\', no leading '.') in org(), so a SuperAdmin X-Org-Id switch to
"../../etc" is rejected at the git boundary before it reaches absRepoPath.

MED-4 mirror SSRF: resolve the source host and refuse loopback / private /
link-local / metadata (IMDS) / unspecified / multicast targets;
GIT_MIRROR_ALLOW_PRIVATE_HOSTS allowlists internal hosts for tests /
deliberate internal mirrors. Generic rejection message (no probe oracle).

MED-5 pack-to-disk DoS: -c receive.maxInputSize (GIT_RECEIVE_MAX_INPUT_SIZE,
default 2g) on receive-pack so a gzip-amplified or runaway push can't fill
the pod disk.

MED-6 disconnect reaping: gitPackStream.Close now closes the read pipe and
Kills the process before Wait, and runPackSSH Kills on a channel-copy error,
so an abandoned clone can't leave git blocked on a full pipe (leaked
proc/goroutine/FDs).

LOW-7 strip URL userinfo in mirrorSource (credentials via env only, never a
ps-visible argv). LOW-8 close std pipes on cmd.Start failure.

Tests: org-traversal rejection, disconnect-reaping (Close returns promptly +
process reaped + slot released), mirror credential host allowlist, and mirror
SSRF + userinfo strip; existing suites stay green.
2026-07-14 06:54:11 -07:00
hanzo-dev d2a38d9aed git: back the object plane with the streaming git CLI for bounded memory
The heavy git paths buffered whole packs in RAM via go-git's pure-Go
server transport and FetchContext: a clone serialized the entire outgoing
packfile into a bytes.Buffer, a mirror indexed the whole incoming pack in
memory, and receive-pack read the full push body. Mirroring a multi-GB
repo OOM-killed the 1 Gi cloud pod.

Route the object plane through the streaming git CLI instead — the way
gitea/GitLab serve smart-HTTP — so packs stream to and from disk with
memory bounded by an OS pipe:

- clone/fetch serve: `git upload-pack --stateless-rpc`, git stdout streamed
  straight to the HTTP response (SendStream); no pack buffer.
- push receive: `git receive-pack --stateless-rpc`, request body -> git
  stdin -> index-pack to disk; only the small report-status is buffered.
- info/refs: `git <svc> --stateless-rpc --advertise-refs` + pkt-line header.
- mirror-in: `git fetch --prune --tags +refs/*:refs/*` against the on-disk
  bare repo; ls-remote --symref resolves the source default branch for HEAD.
- SSH: plain `git upload-pack`/`git receive-pack` over the channel.

One git-exec seam (gitexec.go) builds every subprocess with a hardened,
minimal env (GIT_CONFIG_NOSYSTEM, GIT_CONFIG_GLOBAL=/dev/null, no inherited
secrets, GIT_TERMINAL_PROMPT=0, GIT_NO_REPLACE_OBJECTS) and arg slices only.
Protocol v2 is forwarded from the client Git-Protocol header / SSH env
(validated). Mirror source credentials are injected only via env git-config
http.extraHeader (GIT_MIRROR_TOKEN), never argv or logs, under
GIT_ALLOW_PROTOCOL=http:https. Tenant isolation stays on the validated
absolute bare-repo path (storage.absRepoPath); the handler org/path guards
are unchanged.

Push-to-deploy is preserved via a branch-tip diff (before/after
for-each-ref) that fires cloud.OnGitPush for every advanced branch, and
metering (recordUsage) runs after every push. The go-git server transport
is removed; go-git remains only for bounded init/ref-read/object-building.

Runtime: add git to the alpine stage (upload-pack/receive-pack/http-backend/
git-remote-https).

Tests: real git-CLI clone+push round-trip, a 48 MiB clone proving the pack
streams with ~120 KB server heap growth, mirror from an external
git-http-backend source, and the existing tenant-isolation + push-to-deploy
suites, all green.
2026-07-14 06:12:40 -07:00
d575792666 fix(deps): correct hanzoai/ai v1.806.15 go.sum hash (#291)
The v1.806.15 tag was re-pointed to the current fix commit; a stale local
module cache wrote the previous tag content's hash into go.sum in #290, so
cloud CI's fresh download failed verification (checksum mismatch / SECURITY
ERROR). Re-fetched the module clean so go.sum records the actual v1.806.15
content hash. No code change; go.mod pin unchanged.

Verified: CGO_ENABLED=1 go build -tags "libsqlite3 sqlite_fts5" ./cmd/cloud → 0.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-13 23:37:21 -07:00
1c7128419a fix(deps): bump hanzoai/ai v1.806.13 -> v1.806.15 (#290)
Carries the hanzo-code-claude corrections (ai 68be82cf, first in v1.806.14):
per-upstream reasoning-effort vocabulary (GLM/DeepSeek max|high, not OpenAI
low|medium|high) and round-tripping assistant thinking as reasoning_content
so DeepSeek/Kimi tool-call loops no longer 400/stall. v1.806.15 also carries
ai's luxfi/geth proxy-resolution CI fix and the native Responses work.

Prod (cloud v1.799.16) embedded ai v1.806.13, which predates these fixes.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-13 23:28:17 -07:00
zandGitHub 0b8fd867c3 Merge pull request #285 from hanzoai/fix/cloud-gomod-cache-v4
fix(ci): retry Zen resolution after publishing tag
2026-07-13 17:07:34 -07:00
hanzo-dev 88b97a7fb7 fix(ci): retry Zen resolution after publishing tag 2026-07-13 17:07:28 -07:00
zandGitHub cea10dfefb Merge pull request #284 from hanzoai/fix/cloud-gomod-cache-v3
fix(ci): cold-bust poisoned Zen module cache
2026-07-13 17:04:49 -07:00
hanzo-dev 67b808bee5 fix(ci): cold-bust poisoned Zen module cache 2026-07-13 17:04:25 -07:00
zandGitHub e6268be2f7 Merge pull request #283 from hanzoai/fix/code-full-auto-tool-default
fix(code): full-auto defaults on tool-capable model
2026-07-13 17:02:32 -07:00
hanzo-dev e862cfaace fix(code): default agents to full-auto tool-capable model 2026-07-13 17:02:10 -07:00
hanzo-dev 4aff04accd analytics: add capture (write) plane — POST /v1/analytics + /v1/tracker → hanzo.events
The analytics subsystem served only read lenses over hanzo.events; nothing
wrote the table, so the web/commerce lenses were permanently honest-empty. This
adds the symmetric ingest: products POST batches to cloud (the ONE native front
door) and cloud writes org-scoped rows into the datastore warehouse the read
side already queries.

- POST /v1/analytics, /v1/analytics/batch, /v1/tracker (beacon alias) — all
  tenant-gated in-handler; tenant_id is always principal.Org, never client input.
- Writes ride ai/object.DatastoreExec (the SAME pooled client the reads use).
- The writer owns the hanzo.events DDL (EnsureEventsTable, idempotent/latched).
- Privacy scrub: credential/PII-shaped property keys dropped, email values
  redacted, before any row is built.
- Pure core (normalizeEvent/scrubProps/buildEventsInsert) unit-tested; HTTP
  contract tests cover no-principal 403, forged-org 403, oversized 400,
  datastore-down 503; a build-tagged live test proves the full round trip
  against a real datastore.
2026-07-13 16:09:17 -07:00
hanzo-dev 92530c589a analytics: add capture (write) plane — POST /v1/analytics + /v1/tracker → hanzo.events
The analytics subsystem served only read lenses over hanzo.events; nothing
wrote the table, so the web/commerce lenses were permanently honest-empty. This
adds the symmetric ingest: products POST batches to cloud (the ONE native front
door) and cloud writes org-scoped rows into the datastore warehouse the read
side already queries.

- POST /v1/analytics, /v1/analytics/batch, /v1/tracker (beacon alias) — all
  tenant-gated in-handler; tenant_id is always principal.Org, never client input.
- Writes ride ai/object.DatastoreExec (the SAME pooled client the reads use).
- The writer owns the hanzo.events DDL (EnsureEventsTable, idempotent/latched).
- Privacy scrub: credential/PII-shaped property keys dropped, email values
  redacted, before any row is built.
- Pure core (normalizeEvent/scrubProps/buildEventsInsert) unit-tested; HTTP
  contract tests cover no-principal 403, forged-org 403, oversized 400,
  datastore-down 503; a build-tagged live test proves the full round trip
  against a real datastore.
2026-07-13 16:09:17 -07:00
hanzo-dev 47a98eb808 git: serve smart-HTTP at the git-host root so git clone https://git.hanzo.ai/<org>/<repo>.git works
Add host-guarded root-level /:org/:repo/{info/refs,git-upload-pack,git-receive-pack}
reusing the existing handlers. onGitHost gates on the request Host == git host
(defaultSSHHost(Domain)); on api/console hosts the routes fall through (c.Next())
so a bare /:org/:repo never shadows another surface. Advertised clone URL stays
/v1/git until cutover. Test: TestRootSmartHTTP_HostGuard (git host → handler runs;
other host → 404).
2026-07-13 15:41:24 -07:00
hanzo-dev 7b5a8a5642 cloud: mount zen co-resident + 18-dp-native metering
zen mounts as a /v1-scoped Claim middleware BEFORE ai's /v1/* catch-all
(Wire position 100, ahead of ai at 150). Claim routes every request whose
model is a zen SKU to zen's serving layer in-process and c.Next()s the rest
to ai, so zen owns the zen family (identity, tools, 1M ladder, codec) and ai
owns every other model + /v1/models. The frozen wire sequence is updated.

Billing for zen* moves from ai's in-handler metering (skipped — zen claims
before ai's beego catch-all) to zen's own Gate+Meter wired to the same
commerce metering client the edge gate uses — ONE billing source for zen*,
never double-billed, never free. Granularity is org/project/user mirroring
the edge gate: home org (principal.BillingOrg) pays, project
(principal.ValidatedProject) scopes spend caps, user is attributed. An admin
acting in another org bills their own home org.

metering.Usage now carries a typed money.Amount (native 18-dp USD — the
finance ledger's own precision) as the canonical debit; Record passes it
straight to finance with NO flooring to cents or micros, so an exact
per-token cost from zen debits exactly. Legacy AmountCents/AmountMicros
reconstruct the same Amount for older callers. See hip-00NN.

Bumps hanzoai/zen v1.0.0 -> v1.1.0 (additive API: TenantResolver, ctx-aware
Meter/Gate, Tenant.BillingOrg/Project, Usage.RequestID).
2026-07-13 15:39:38 -07:00
hanzo-dev 4bfede81a0 fix(code): disable zen5-ultra (403s), Fable tier falls through to zen5-pro; bump ai v1.806.12->v1.806.13
zen5-ultra routes to anthropic-claude-opus-4.8, which 403s on this account.
Drop the ultra backend chain from the Fable tier so it resolves to zen5-pro
(the working heavy-reasoning tier) instead. Picks up ai v1.806.13:
upstream-429 surfacing (typed apiError, status reaches client) + the
Anthropic thinking-budget -> upstream reasoning-vocabulary adapter
(glm max|high, openai low|med|high) so deep CC thinking reaches the deep
tier and invalid vocab no longer gets GLM-5.2 429'd. See hip-00NN.
2026-07-13 14:47:11 -07:00
hanzo-dev e474e65c4a chore(deps): bump hanzoai/money v0.2.0 -> v0.2.1 (honest 18-dp decimal floor) 2026-07-13 12:46:06 -07:00
antje cffc595a28 Merge remote-tracking branch 'origin/main'
# Conflicts:
#	go.mod
2026-07-13 12:34:25 -07:00
hanzo-dev 55e2b84d96 merge: git-native feature + de-gitea → main
# Conflicts:
#	go.mod
#	go.sum
2026-07-13 12:22:40 -07:00
hanzo-dev b1c93222ae chore: de-gitea — scrub branding/docs, keep legal+provider 2026-07-13 11:54:03 -07:00
antje 0445122adc Merge commit 'e896a505' 2026-07-13 11:19:05 -07:00
antje 7e78a12235 merge: restore force-push-clobbered lineage — CLI device login + gpu --studio-dir, atto-USD money, datastore debrand, deps
origin/main was rewritten (tracker/git work force-pushed over ~13 shipped
commits). This merge reunites both histories; go.mod resolved as the
highest-version union + go mod tidy.
2026-07-13 11:19:05 -07:00
antje 6cccfb79e7 refactor(money): delegate exact-money machinery to shared hanzoai/money + hanzoai/decimal
The big.Int 18-decimal fixed-point implementation lives ONCE in the shared
packages; cloud/clients/money becomes the thin policy layer that pins the
credit unit to 18-decimal USD. Finance/treasury/admin call sites unchanged
in behavior — money, ledger, sqlstore, finance, admin-core tests green.
2026-07-13 11:13:59 -07:00
hanzo-dev 47b9c38878 tracker: narrow Kind to {issue,pr,epic}; contract draws the three-plane boundary
Correct the taxonomy after confirming the real architecture: helpdesk tickets,
CMS content, ERP docs and knowledge entries are framework.DocType records (Hanzo
Base), and CRM Company/Contact/Opportunity is a bespoke relational store — a
DIFFERENT plane from the tracker. The tracker Issue is the engineering/project
work-item primitive, not the universal record.

- kinds narrowed to issue|pr|epic (drop deal/ticket/doc = domain records; drop
  task = the async plane, hanzoai/tasks). sources kept (legit work-item origins).
- contract.go rewritten to THREE orthogonal planes — tracker Issue (work items),
  framework.DocType (domain records), hanzoai/tasks (async execution) — and the
  thin one-directional seams (ExtRef links, Issue<->task), never embedding.
- Issue/IssueFilter/field docs + test seed aligned to the narrowed set.
2026-07-13 10:44:51 -07:00
hanzo-dev 370f161b2f tracker(contract): the Hanzo work-item contract — ONE Issue, every surface a filter
Pin the alignment law as in-repo design source of truth (doc-file, like the
git package doc and tasks/CONTRACT.md): there is ONE work-item primitive — the
tracker Issue — and no second issue table. Every product surface (hanzo.team
board, git Issues/PRs tabs, CRM pipeline, helpdesk queue, CMS task list, an
agent's work) is a FILTER over the one table, never a parallel store.

Documents: the four immutable discriminators (Kind/Source/Repo/ExtRef) and
their orthogonality; the surface->filter mapping; the two tenancy roots (IAM
project = physical file; tracker Project = KEY-N team within it; git repo bound
by the Repo discriminator, a third thing); and the tasks seam — tracking
(intent+state) is not execution (durable async = hanzoai/tasks), composing in
exactly two directions (Issue->enqueue task, task->patch Issue board state),
never inverted.
2026-07-13 10:38:58 -07:00
hanzo-dev 70e0138b20 tracker: ONE polymorphic issue primitive — git/CRM/helpdesk/CMS/team all compose
Extend the tracker Issue with four immutable discriminators (Kind, Source,
Repo, ExtRef) so a project task, a git issue, a pull request, an epic, a CRM
deal and a helpdesk ticket are ALL the same row. Every product surface becomes
a FILTER over this one table — never a second tracker:

  - hanzo.team board  = ListIssues(Filter{})            (or {Status})
  - git repo Issues    = Filter{Repo, Kind:"issue"}
  - git repo PRs       = Filter{Repo, Kind:"pr"}
  - CRM pipeline       = Filter{Kind:"deal"}
  - helpdesk queue     = Filter{Kind:"ticket"}

store.go: additive kind/source/repo/ext_ref columns (DEFAULTed so existing
rows read as native team issues; idempotent ALTER for pre-spine DBs) +
org_repo / org_kind indexes; ListIssues takes IssueFilter (status+kind+repo+
source, each an optional bound predicate).

tracker.go: closed kinds/sources sets, normKind/normSource, issueFilter() from
?status=&kind=&repo=&source=; create accepts + validates the discriminators;
billing category stays the constant "issue" row, decoupled from work-item Kind.

Discriminators are identity — set once at Create, immutable on Update — so a
row never migrates between surfaces. Tests: TestIssuePolymorphicSpine proves
the filter slices + defaults + cross-repo source view.
2026-07-13 10:32:00 -07:00
hanzo-dev 9085031c83 git(ui): native Hanzo Git web UI — repo list, browse, blob, commits
The embedded git host gains its browser surface (ui.go + ui_templates.go),
server-rendered in the ONE cloud binary — the native replacement for the
standalone Gitea web app, so git.hanzo.ai can retire it. Routes at /git/*:
org repo list, repo home (branches/HEAD/tree/clone/README), tree browse, file
view (binary-aware), commit log. Reads the SAME org-scoped store + go-git object
storage the API uses; every page scoped to the IAM-validated X-Org-Id with the
path-vs-identity guard (no cross-tenant read). html/template auto-escaping is the
XSS boundary. Build+vet clean; all clients/git tests green.
2026-07-13 10:01:43 -07:00
hanzo-dev f1ab5ae856 git(env): drop CLOUD_ prefix on embedded git SSH env — GIT_SSH_{HOST,ADDR,HOST_KEY}
The embedded Hanzo Git subsystem's env is now bare GIT_* (was CLOUD_GIT_*). Not
set in any prod deployment (code defaults), so the rename is behavior-neutral;
takes effect on next build. Cleaner + gitea-free naming.
2026-07-13 09:06:07 -07:00
hanzo-dev 301517fd16 fix(ci): bustable BuildKit cache ids — a poisoned module cache wedged the release
The release has been failing on:
  go: github.com/hanzoai/otel-collector@v0.144.10: unknown revision v0.144.10

That tag EXISTS on GitHub and resolves fine from a clean cache (verified:
go get github.com/hanzoai/otel-collector@v0.144.10 -> exit 0). The failure is
the BuildKit cache mount, not the pin: with no explicit id, BuildKit keys the
cache by target path alone, so a negative lookup recorded while the tag did not
yet exist is remembered FOREVER and there is no way to evict it.

Give every cache mount an explicit, bumpable id (cloud-gomod-v2 /
cloud-gobuild-v2). Bumping the suffix forces a cold cache. This unwedges the
release without touching the (correct) otel-collector pin, and gives us the
lever we were missing the next time a phantom pin poisons the cache.
2026-07-13 09:00:34 -07:00
hanzo-dev ecc8345e90 git: rebrand OUR embedded git = Hanzo Git; default provider = git (git.hanzo.ai)
Per directive: only OUR internal embedded git is rebranded (Hanzo Git), and the
default source provider is our own git.hanzo.ai ("git"). External providers
github/gitlab/bitbucket/gitea.com stay available as sources — gitea.com is fine
as an external source, we just never brand OUR host as Gitea.
2026-07-13 08:43:15 -07:00
hanzo-dev 3a8f4069b8 fix(edge): raise BodyLimit to 16 MiB — the 4 MiB default capped the context window
The context window is a BODY-SIZE fact, not only a model fact: a chat request
carries its whole prompt in the request body. zip/fiber defaults BodyLimit to
4 MiB (zip.go: 'if cfg.BodyLimit == 0 { cfg.BodyLimit = 4 << 20 }') and cloud
never set it — so a 1M-token prompt (~4.3 MB of JSON) was refused by fasthttp
BEFORE any handler ran, with the opaque 400 'Error when parsing request' that
reads like a malformed payload rather than a size cap.

Effect: the 1M-context routes were unreachable in practice. Measured on prod
(v1.799.10, direct to pod:8000, bypassing ingress+gateway):
  1.5 MB body -> 350,152 prompt tokens -> 200 (glm-5.2 correctly auto-rolled)
  2.0 MB body -> 466,148 prompt tokens -> 200
  4.0 MB+     -> 400 'Error when parsing request'
So the cascade logic was right all along; the transport silently capped it at
~900k tokens. 16 MiB gives a 1M-token prompt ~3.7x headroom. The edge is
authenticated + rate-limited and fasthttp streams, so this is not a memory
vector. Env GATEWAY_BODY_LIMIT, mirroring the GATEWAY_READ_BUFFER_SIZE
precedent (same class of bug one layer up: the 4 KiB header default).

Tests pin the invariant so it cannot silently regress to the framework default.
2026-07-13 08:26:57 -07:00
hanzo-dev 170b47eecb chore(deps): re-pin ai v1.806.12 + iam v1.31.25 (de-mattn) — mattn/go-sqlite3 now 0 in go.sum AND build graph 2026-07-13 00:28:26 -07:00
hanzo-dev 4fca0e1fc5 chore(deps): de-mattn — beego/v2 v2.4.2 + xorm v1.4.1 (sqlite v0.3.0 already) + drop the mattn/go-sqlite3 replace (go list -deps mattn=0) 2026-07-12 23:45:47 -07:00
antje e896a50578 merge: karma storefront loop — published Asset → listing image; product.created → ecom render; Campaign→product integrity
# Conflicts:
#	build.go
#	go.mod
#	go.sum
2026-07-12 23:33:27 -07:00
hanzo-dev f541e26b9b content: reverse storefront loop (product.created → ecom render) + Campaign→product integrity
Closes the Studio→CMS→Commerce loop the other direction. The forward edge
(storefront.go) publishes a rendered Asset onto the product image; this adds the
REVERSE — a new catalog product triggers its ecom render — plus the join-key
integrity gate. Decomplected over the COMMERCE event stream so commerce and content
never import each other.

Reverse loop (B):
- events: SubjectProductCreated/Updated ("commerce.product.*", on the COMMERCE
  stream) + PublishProductCreated/Updated on the existing events.Publisher (nil-safe
  no-op when NATS is absent, mirroring the order publishers).
- commerce product REST: publishProductEvents middleware fires the catalog event after
  a successful create/replace, reading the process Publisher off the gin context
  exactly like the order publishers — a total no-op when unwired.
- clients/catalogsync: in-process consumer subsystem on the COMMERCE stream mapping
  product.created → content.EnsureCatalogAsset (design == slug). Inert until
  CLOUD_COMMERCE_NATS_URL names the NATS carrying the events.
- content.EnsureCatalogAsset: product slug → ONE ecom Asset draft, idempotent (skip if
  a non-archived ecom asset already exists) and quiet (errNotConfigured / lane not
  installed = skip, never an error loop).

Integrity (C):
- Storefront gains ProductExists (reusing the same commerce S2S seam, X-Org-Id-pinned);
  enforceCatalogRefs before_save on Campaign.product fails closed on a dangling handle,
  validates only on set/change, and skips when commerce is unconfigured or erroring.
  Campaign-only on purpose: Asset.design is authored render-first (before the product
  exists), so gating it would reject the loop's first step.

Wired after content in subsystems.Wire() (frozen order test updated). Tests cover the
EnsureCatalogAsset idempotency/skip paths, the Campaign dangling-handle rejection, the
ProductExists S2S wire, the catalog event envelope + no-op, the REST middleware
classification, and the consumer dispatch ACK/NAK semantics.
2026-07-12 23:13:07 -07:00
antjeandGitHub 70bf435c19 refactor(datastore): cloud on hanzo-ds/go — kill datastore-go/v2 (#279)
swap the 3 remaining driver call sites (audit_mirror, commerce/db/
datastore, o11y/event_ingest) hanzoai/datastore-go/v2 -> hanzo-ds/go,
and take o11y v1.5.26 (fully driver-clean). cloud graph now has zero
hanzoai/datastore-* — the driver's one home is hanzo-ds/go.
2026-07-12 23:02:12 -07:00
hanzo-dev f213106edc deps: bump ai v1.806.9->v1.806.10 + fix phantom o11y v1.5.23->v1.5.25
- ai v1.806.10: gofumpt-clean CI (unblocks ai release pipeline) +
  deepseek-r1-distill context parity + swarm datastore standardization
  (hanzo-ds/go v1.0.1, datastore-go v2.47.2).
- o11y v1.5.23 was a PHANTOM pin (tag never published; graph jumps
  1.5.21 -> 1.5.25). Cloud only built via warm CI module cache; a fresh
  resolve failed 'unknown revision v1.5.23'. Repin to the real latest
  v1.5.25. Verified: go build ./cmd/cloud = exit 0.
2026-07-12 22:58:00 -07:00
antje 5103980096 feat(cli): hanzo login defaults to the RFC 8628 device flow — QR + link, approve from any device
The ONE interactive login: the CLI mints a device+user code from IAM
(hanzo.id /v1/iam/oauth/device, client hanzo-app — the first-party client
seeded with device_code), renders the verification link as text and a
terminal QR, and polls /v1/iam/oauth/token until approved. No password
ever touches the terminal; works headless (GPU boxes, ssh). Same endpoints
and client as hanzo-dev's live device flow, so one server-side config
serves both. --username/--password-stdin keep the password grant for
automation; --token unchanged.
2026-07-12 22:56:18 -07:00
antje 6c0c756906 fix(cli): gpu connect detects Apple Silicon — report chip + unified memory
detectGPUs only probed nvidia-smi, so a Metal/MPS box registered as
CPU-only in the fleet. On darwin/arm64 report the Apple chip as one GPU
with hw.memsize unified memory, MiB-formatted like nvidia-smi.
2026-07-12 22:51:45 -07:00
antje 0c9ebb650d feat(cli): hanzo gpu connect --studio-dir supervises the local render backend
The gpu-jobs claim loop renders on the LOCAL studio server (127.0.0.1:8188);
naming a checkout makes connect own that server's lifecycle too — launch from
the venv, health-probe /system_stats, one grace re-check, free the port and
relaunch on death or hang. Replaces the GB10 watchdog script and hand-rolled
systemd units: the hanzo CLI is the one way a BYO box joins the fleet, render
backend included. --daemon bakes the flag into hanzo-gpu.service.
2026-07-12 22:23:42 -07:00
antjeandGitHub 120ce15db4 chore(o11y): consume Datastore() accessor — o11y v1.5.23 (#276)
renames the store call site DatastoreDB() -> Datastore() (redundant DB
dropped) and bumps hanzoai/o11y v1.5.19 -> v1.5.23, which also completes
the clickhouse->datastore debrand indirects and pins a valid module zip
(v1.5.21/.22 zips were case-collision-invalid; .23 verified downloadable).
CGO_ENABLED=0 go build ./... green.
2026-07-12 22:04:40 -07:00
antje ab51d0cc83 feat(finance): exact 18-decimal (atto-USD) money — kill cents flooring
The double-entry ledger stored money as int64 cents, so every sub-cent AI
charge was floored to 0 (free AI) and every call skimmed the fractional
cent on rounding. Money is now EXACT to 18 decimals end to end.

- clients/money: new immutable big.Int Amount in atto-USD (1e-18) — the
  EVM/ERC-20 uint256 unit, so the off-chain ledger and an on-chain credit
  token are the SAME value with no boundary rounding. No float, no deps.
- ledger core + SQLite adapter: Posting/Entry/Balance int64 → money.Amount;
  amounts stored as TEXT (atto overflows SQLite INTEGER past ~$9.20) with an
  O(1) running-balance column and a one-time cents→atto ×1e16 rebuild
  migration (existing prefunds carry over exactly on first open). Anchor
  ComputeRoot hashes exact atto.
- finance wallet / types seam / Formance adapter / admin grant+deposit /
  metering / treasury threaded through. Treasury reserve-fund facade stays
  int64-cents (revenue-share is cents-granular).
- ai debit path (v1.806.6) emits exact decimal-USD; wireFinance parses it to
  atto. Balance READ stays coarse cents (gates a >0 threshold only). Grant
  response returns balanceAtto so a sub-cent debit is visible.

Pin ai v1.806.3 → v1.806.6 (exact nano-USD billing; datastore-go/v2 registers
driver 'datastore', no clash with cloud's hanzo-ds/go 'clickhouse').
2026-07-12 21:59:09 -07:00
hanzo-dev 65b33e2830 content: Storefront edge — published catalog Asset → storefront product image
A published content.Asset (kind in ecom/product/lifestyle, design==product slug)
materializes its S3 URL into the org's Hanzo Commerce store Listing headerImage —
the runtime display layer karma.style already reads (GET /v1/store/:store/listing).
Replaces the build-time studio->S3->library.json->sync-*->site pipeline with ONE
publish-edge side effect, decomplected exactly like the social Distributor.

- storefront.go: Storefront seam + commerce S2S impl (Bearer COMMERCE_SERVICE_TOKEN
  + X-Org-Id over clients/commerceinproc; store via GET /v1/store/current; upsert
  PUT /v1/store/:id/listing/:design). Fail-closed (not_configured) with no token.
- content.go: wire sf edge at Mount; fire StorefrontPublish on the published edge;
  TransitionResult.Storefront.
- Tenant-scoped by IAM org; assets referenced by S3 URL; no sync scripts.

Tests (CGO_ENABLED=0): gate, url resolution, transition side-effect, fail-closed,
and the real commerce S2S wire incl. X-Org-Id tenant pinning.
2026-07-12 21:53:29 -07:00
hanzo-dev 6fc65809a2 deps: iam v1.31.24 (tenancy: Org.Parent + recursive authz; signup/signin IP capture); datastore-go v2.47.0
datastore-go was pinned at v2.47.1 — a version the module proxy cannot resolve (the /v2 module path does not exist at that tag), so cloud did not build from a clean cache. v2.47.0 resolves.
2026-07-12 21:44:10 -07:00
z ccca97bc7d chore(deps): bump hanzoai/ai v1.806.8 -> v1.806.9 (>256k glm-5.2 auto-rolls to DS4-Pro) 2026-07-12 20:21:22 -07:00
hanzo-dev 69b260efd9 cloud: clickhouse->datastore debrand (zero clickhouse .go, datastore:// DSN, drop legacy CLICKHOUSE env) + pin o11y v1.5.21 (restore /v1/sentry, one datastore driver) 2026-07-12 20:08:39 -07:00
z eafb63cde5 chore(deps): bump hanzoai/ai v1.806.7 -> v1.806.8 (ONE-rule billing subject) 2026-07-12 20:03:12 -07:00
zandhanzo-dev f824fb01f9 refactor(billing): mirror ai ONE rule — billing subject is always the org
billingSubject(org,name) = org (lowercased), always. Deletes the personalBillingOrgs
and orgBillingOrgs allowlist parsers (PERSONAL_BILLING_ORGS / ORG_BILLING_ORGS). Keeps
the console billing view in lockstep with ai/object.BillingSubject so the view and the
gateway gate scope to the SAME subject. Tests prove the killed envs are ignored.

Needs go.mod bump github.com/hanzoai/ai -> v1.806.8 (run in a module-resolving env).
2026-07-12 20:01:25 -07:00
zeekay 087d91427c revert: otel-collector v0.144.10 — v0.144.13 forced mock v0.14.3 whose renamed API breaks o11y v1.5.17 (cascade pending) 2026-07-12 19:46:48 -07:00
zeekay bfaa70a57d deps: otel-collector v0.144.13 + hanzo-ds/mock v0.14.3 — chproto-free datastore stack 2026-07-12 19:44:41 -07:00
zeekay b49cf651a5 debrand: datastore-go v2.47.2 (0 clickhouse/CH identifiers) + chproto->dsproto 2026-07-12 19:43:38 -07:00
zeekay f3bb3107e0 deps: datastore-go v2.47.1 — cloud go.mod/go.sum now 0 clickhouse + 0 signoz
datastore-go's transport moved to hanzo-ds/native, so the ClickHouse/ch-go
indirect is pruned from cloud entirely. No cloud package imports a ClickHouse
or signoz module.
2026-07-12 19:31:48 -07:00
hanzo-dev 313c9055f7 deps: adopt otel-collector v0.144.10 (hanzo-ds datastore drivers) — drop signoz + -hanzo.0 replaces
- otel-collector v0.144.8 (+ replace => v0.144.8-hanzo.0) -> clean require v0.144.10
- dropped: replace SigNoz/signoz-otel-collector => hanzoai/signoz-otel-collector
  and the // indirect SigNoz/signoz-otel-collector require
- cloud now pulls hanzo-ds/{go,native,mock} transitively; 0 signoz in go.mod/go.sum;
  no cloud package imports a ClickHouse driver directly (build green)
Residual: one ClickHouse/ch-go // indirect graph line from a transitive dep's
go.mod (not compiled in) — clears when that dep drops it.
2026-07-12 19:19:54 -07:00
hanzo-dev fc7a04a1ed deps: bump hanzoai/ai v1.806.6 -> v1.806.7
Measured DO context windows + config-only resolver (the old table refused
deepseek-v4-pro at 131072) + oversized-prompt reroute to the 1M model.
This is what lifts hanzo code claude to the full 1M context.
2026-07-12 18:44:29 -07:00
hanzo-dev 3e52fdcdac debrand(o11y): drop signoz — query plane reads o11y_traces.distributed_o11y_index_v3
The o11y read plane was renamed signoz_* -> o11y_* (databases, tables, query
identifiers) at the source; the ClickHouse cutover migration
(o11y/deploy/clickhouse/migrations/0001_rename_signoz_to_o11y.sql) renames the
physical objects data-preservingly. Cloud's eval telemetry queried the OLD
physical name directly, so it would break post-cutover AND leaked the brand:

  eval/metrics.go: spanTable signoz_traces.distributed_signoz_index_v3
                        ->   o11y_traces.distributed_o11y_index_v3

Plus every prose/table-name reference across the in-repo o11y read plane and
commerce OTel bootstrap: signoz_traces/signoz_logs -> o11y_traces/o11y_logs,
SigNoz-fork provenance comments -> o11y/upstream. No source brand leaks remain.

DEPLOY: apply 0001_rename_signoz_to_o11y.sql during the collector cutover window
so prod ClickHouse objects match the new identifiers before this image serves.

Residual: go.mod keeps a REDIRECTED github.com/SigNoz/signoz-otel-collector key
(replace -> our fork; no SigNoz code fetched) — pulled transitively by
hanzoai/otel-collector's own go.mod; purge belongs to that fork's rename.
2026-07-12 17:43:34 -07:00
hanzo-dev a033c23591 feat(billing): adopt ai v1.806.6 — thread exact USD debit through the usage recorder
ai v1.806.6's UsageEvent replaced Cents(int64) with USD (exact decimal string,
atto-precise) so a sub-cent AI call bills precisely upstream. Adapt cloud's recorder:
- types.UsageInput gains USD (supersedes Cents when set).
- build.go SetUsageRecorder forwards u.USD.
- finance.RecordUsage rounds USD -> cents at the ledger boundary (usdToCents,
  round-half-up via math/big, no float). The local finance ledger is cents-denominated,
  so sub-cent floors to 0 exactly as the prior int64-Cents contract did — the atto path
  is commerce, not this store. Money-safety locked by TestUsdToCents.

Also carries ai v1.806.6's glm-5.2 1M context window (already in v1.806.5). Build ./...
green, finance tests pass.
2026-07-12 17:17:02 -07:00
hanzo-dev 1b75a97a77 fix(code): default hanzo code to glm-5.2, not the reserved word best
agent treats `best` (like opus/sonnet/haiku) as a reserved model
alias and rewrites it to a claude-* id. api.hanzo.ai does not serve those
claude-* ids (a request 403s — see anthropicWire), so `hanzo code claude`
on the default `best` died at session start. Default to glm-5.2: the
stable GLM-5.2-class 1M-ctx frontier, a concrete catalog id CC passes
through unchanged; the backend still cascades on rate-limit / down.
2026-07-12 17:05:02 -07:00
hanzo-dev 605ead5a4c fix(billing): bill the SUBJECT's wallet — a personal account gets a personal balance
The ledger already scoped correctly (org = which books, subject = which wallet in them), but both hooks passed the org for BOTH, collapsing every member onto the tenant's pool wallet. Since every signup lives in 'hanzo', a brand-new $0 account read HANZO's balance and sailed through the gate: we were enforcing our own wallet, not theirs. Now the gate reads, and usage debits, the subject ai already resolves — a person => their own wallet (personal plan), an org-owned application/service key => the org's account. That is the product: sign up as yourself with personal billing, then stand up an org whose users are your customers (Organization.Parent + AdministersOrg in hanzoai/iam). The invariant that must never break: the gate READ and the usage DEBIT key on the SAME wallet, or spend outruns the balance that admitted it — both use subject, keep them together. Also unpins o11y v1.5.16, whose tag was re-pointed upstream so its hash no longer matches go.sum (build fails verification); v1.5.17 is the unpoisoned tag.
2026-07-12 16:52:20 -07:00
hanzo-dev 72313f5f14 refactor(team): rip the last Huly refs — zero Huly anywhere
- tracker.go: prose 'Huly/Svelte' -> 'prior Svelte'
- model.json: Slack-mapping wire attrs hulyChannel/hulyChannelClass ->
  teamChannel/teamChannelClass (seed-model data, NOT Go-referenced; valid JSON).
The clients/team/*.go debrand already landed on main. This closes it out.

NOTE: teamChannel wire IDs are new-workspace seed data; the deployed front
(front:v0.7.391) still speaks hulyChannel for Slack channel mapping, so a lockstep
front rebuild is needed for Slack mapping on NEW workspaces — flagged, not silent.
2026-07-12 16:50:11 -07:00
hanzo-dev 649fcf87bd fix(billing): bill the SUBJECT's wallet — a personal account gets a personal balance
The ledger already scoped correctly (org = which books, subject = which wallet in them), but both hooks passed the org for BOTH, collapsing every member onto the tenant's pool wallet. Since every signup lives in 'hanzo', a brand-new $0 account read HANZO's balance and sailed through the gate: we were enforcing our own wallet, not theirs. Now the gate reads, and usage debits, the subject ai already resolves — a person => their own wallet (personal plan), an org-owned application/service key => the org's account. That is the product: sign up as yourself with personal billing, then stand up an org whose users are your customers (Organization.Parent + AdministersOrg in hanzoai/iam). The invariant that must never break: the gate READ and the usage DEBIT key on the SAME wallet, or spend outruns the balance that admitted it — both use subject, keep them together. Also unpins o11y v1.5.16, whose tag was re-pointed upstream so its hash no longer matches go.sum (build fails verification); v1.5.17 is the unpoisoned tag.
2026-07-12 16:48:12 -07:00
hanzo-dev c290cdaf45 chore(deps): bump ai v1.806.4 -> v1.806.5 (GLM-5.x 1M context window)
Deployed glm-5.2 was capped at 16384 tokens (the context_length_util.go
fallback) — long agent sessions 402'd. ai v1.806.5 sets glm-5.x to a
1M window with 131072 modern fallback (commit d248ff98).
2026-07-12 16:30:39 -07:00
hanzo-dev deeb3a7067 cli: pin CC tier slots to fixed zen5 aliases
The four agent tier slots (Haiku/Sonnet/Opus/Fable) are a fixed
zen5-* capability contract, decoupled from the resolved main model id.
Previously OPUS tracked ANTHROPIC_MODEL, coupling the tier to the main
choice; now OPUS=zen5-pro and FABLE=zen5-ultra are stable.

Decomplects which-tier from which-model: the tier->alias map is fixed
in the client; the alias->upstream map lives in models.yaml. Swap an
upstream (GLM -> Qwen 3.6 -> a future frontier) and every SDK/CLI/CC
integration keeps working unchanged. zen5-ultra carries its own backend
fallback chain (zen5-ultra -> zen5-pro -> zen5 -> zen5-flash) so the
Fable tier degrades gracefully if the premium upstream is unavailable.
2026-07-12 16:18:08 -07:00
hanzo-dev d32e9ffca0 fix(code): pin all agent model slots to zen5 aliases
agent's subagents, permission classifier, and /compact default to
built-in claude-* model ids (claude-haiku-*, claude-opus-*, claude-sonnet-*).
api.hanzo.ai does not serve those ids — a request 403s, which:
  - kills the classifier ('auto mode cannot determine safety of Bash')
  - kills every subagent (the 403 that dead-ended session cff690fc)
  - leaves /compact to run on a non-1M model -> 262145 > 262144 -> unresumable

anthropicWire now pins every CC tier slot to a served zen5 alias (the
Hanzo-standard mapping of CC tiers onto top OSS models resold via DO GenAI):
  ANTHROPIC_MODEL             = <model> (default best -> zen5/glm-5.2)
  ANTHROPIC_SMALL_FAST_MODEL  = zen5-flash  (DeepSeek-4 Flash, classifier)
  ANTHROPIC_DEFAULT_HAIKU     = zen5-flash
  ANTHROPIC_DEFAULT_SONNET    = zen5       (GLM-5.2)
  ANTHROPIC_DEFAULT_OPUS      = <model>
  ANTHROPIC_DEFAULT_FABLE     = zen5-pro  (DeepSeek-V4 Pro)

Live-proven: zen5/zen5-flash/zen5-pro/zen5-ultra all return 200 on the account;
only the raw claude-* ids 403. tests: TestAnthropicWirePinsZen5Tiers +
TestAnthropicWireExplicitModel lock the mapping and forbid raw claude-*.
2026-07-12 16:18:08 -07:00
hanzo-dev 7a547f0a4c refactor(team): drop Huly branding — no brand prefix, no vendor name
Debrand the team subsystem (our fork is Hanzo Team, not Huly):
- prose/comments: Huly -> Team / the platform
- hulyName() -> personName() (internal Person.name formatter)
- HULY_MODEL_VERSION -> MODEL_VERSION (no brand prefix at all, per the naming rule)
- TestHulyName -> TestPersonName

model.json WIRE class IDs (slack:class:SlackChannelMapping_hulyChannel, attrs
hulyChannel/hulyChannelClass) are deliberately UNTOUCHED: the deployed front
(ghcr.io/hanzoai/front:v0.7.391) speaks them, so renaming needs a lockstep front
rebuild — tracked separately, not a silent prod break.

Build + team tests green.
2026-07-12 16:18:08 -07:00
hanzo-dev 002e949447 feat(billing): honor ORG_BILLING_ORGS in console billing subject + bump ai
Bump hanzoai/ai to the ORG_BILLING_ORGS build (v1.806.3 + the allowlist), so the
gateway's BillingSubject promotes an allowlisted org (e.g. hanzo) to ONE shared
pool. Mirror the same scoped override in the console billing BFF (billingSubject)
so the console view scopes to the SAME subject the gate reads. Default empty =
zero behavior change.

Also refresh the stale hanzoai/o11y v1.5.16 go.mod checksum in go.sum (metadata
only; the zip/code hash was already correct — the readonly build passes), which
a moved tag had left inconsistent and which blocked every cloud build.
2026-07-12 16:18:08 -07:00
hanzo-dev b51616ce65 feat(cli): hanzo code defaults to the virtual best model
Change defaultCodeModel glm-5.2 -> best so `hanzo code claude` (no model arg)
auto-routes to the best-available coding model by quality and cascades on
rate-limit / out-of-credit / down (server-side, controllers/failover.go).
Explicit overrides still work (`hanzo code claude glm5.2`); the fuzzy resolver
validates `best` against /v1/models, where it is a real listed catalog entry.
2026-07-12 16:17:39 -07:00
zeekayandhanzo-dev 4fa6862602 datastore: cloud source → hanzoai/datastore-go/v2 (registers "datastore", not "clickhouse")
Completes the datastore standardization for cloud's own source: audit_mirror,
commerce/db, o11y/event_ingest now import github.com/hanzoai/datastore-go/v2
(the rebranded fork registering the UNIQUE sql driver name "datastore") instead
of the interim github.com/hanzo-ds/go (which registers "clickhouse" and collides
with upstream). Bumps ai v1.806.3 → v1.806.4 (its object/datastore.go likewise on
datastore-go/v2). cmd/cloud now links ZERO upstream ClickHouse/clickhouse-go/v2.
Remaining hanzo-ds/go is transitive via o11y/pkg/datastoremetrics — needs an o11y
release built off datastore-go/v2 (v1.5.16 tag currently resolves to the interim
hanzo-ds/go variant).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-12 11:21:51 -07:00
antjeandGitHub 5cc3b68444 fix(commerce): service-token S2S auth beats the IAM scope gate (completions 500 → clean 402/allow) (#273)
An in-process metering/billing dispatch (cloud → commerce via buildMeteringClient)
carries the verified COMMERCE_SERVICE_TOKEN *and* an X-Org-Id header (to select the
tenant namespace). The X-Org-Id makes IAMTokenRequired stamp iam_authenticated=true
with ZERO permissions (an S2S call carries no X-User-Permissions), so on an
Admin-masked billing route (/v1/billing/{balance,tier,usage}) TokenRequired's IAM
branch ran first, hasScope(0, Admin)=false → 403 "IAM principal lacks required
permission scope". The service token was never consulted; the balance read / usage
debit failed and the completions edge rendered that internal failure as a client 500.

Fix: in TokenRequired, check the service-token branch BEFORE the IAM branch. A request
bearing the KMS-sourced service token is the trusted platform S2S caller and must
authenticate as the service, never be subjected to the per-user IAM scope gate — even
when it also carries X-Org-Id. The two branches' logic is byte-identical; only the
order changed.

- Does NOT loosen the gate for external callers: a real IAM user never holds the
  service token, so they still hit the IAM branch and are masked-gated exactly as
  before (TestTokenRequired_UserIAMScopeGateUnchanged, IAMBranchEnforcesMasks).
- Prepaid gate stays fail-closed: funded → allowed, unfunded → clean 402
  insufficient_balance (TestInProcMeteringDispatch_ServiceTokenAuthPath drives the
  real chain over commerceinproc with finance NOT co-resident).
- The service token is KMS/env-sourced and never logged.

The 500-render itself lives in the hanzoai/ai dep (it serves /v1/chat/completions and
turns the internal balance/debit failure into a 500); once the cloud-side dispatch no
longer 403s, that failure no longer occurs.
2026-07-12 11:19:48 -07:00
antje 039d2a593d chore(deps): ai v1.806.3 (fast-500 nil-CruSession guard) + o11y v1.5.16 (datastore debrand)
ai v1.806.3: nil-CruSession guard on both session funnels + auth-before-autoroute
(routers/base.go; fixes fast 500 on unauthenticated session read).
o11y v1.5.16: datastore debrand (DatastoreDB()). luxfi pins unchanged.
2026-07-12 10:40:56 -07:00
antje f477163b73 Merge remote-tracking branch 'origin/chore/audit-datastore-env' into chore/datastore-roll 2026-07-12 10:37:08 -07:00
antje 65fc5439cd fix(finance): bill the org billing ACCOUNT (pool) for both read + debit, not the per-user subject
The ai gate read the account pool but debited the per-user wallet (read-subject != debit-subject),
so a funded account gated but never depleted. Key BOTH the balance read and the usage debit on the org
(its default-account pool wallet) in the wireFinance hook, so a funded account gates AND meters
consistently. Fund accounts, not users.
2026-07-12 10:18:38 -07:00
9d05b9d331 feat(cron): ONE durable platform cron on the tasks engine — retire every ticker and k8s CronJob (#268)
* feat(cron): ONE durable platform cron on the tasks engine — retire every ticker and k8s CronJob

clients/cron replaces the k8s CronJob fleet AND the commerce sweep ticker
with durable schedules on the shared embedded hanzoai/tasks engine
(v1.50.0 — the release whose sweeper actually fires on the sharded store).
The engine owns time; runs are durable workflows visible in the Tasks
console (/_/tasks, tasks.hanzo.ai) under the CRON_ORG shard (default
hanzo), namespace default, queue cloud-cron.

Entries are DATA in universe git: a ConfigMap labeled
cron.hanzo.ai/enabled="true" carries schedule + EITHER job.yaml (a
batchv1 Job manifest run to completion with Forbid concurrency, entry
label, TTL self-reap) OR poke.json ({url,method,bearerEnv,timeout} —
bearer resolved from THIS process's KMS-synced env, never stored).
A reconcile workflow — itself a durable schedule (*/5) plus one boot
pass — diffs ConfigMaps against cron-prefixed schedules: upserts are
drift-gated (rewriting resets the fire anchor), deletes never touch
foreign ids. Fire-time activities re-read their ConfigMap, so payload
edits apply next tick.

The commerce auto-recharge sweeper (clients/commerce/sweep.go) is
deleted; its 15m POST /v1/billing/auto-recharge/run-all becomes the
cron-billing-autorecharge poke entry — same request, same token, one
cron system.

Tests (-race): TestPokeEndToEnd + TestJobEndToEnd drive the FULL durable
path against a real embedded engine (org-shard schedule → trigger →
loopback worker → activity → completed run in the org shard — the exact
console-visibility + dispatch-routing the design stands on);
TestReconcileConverges pins upsert/delete/foreign-id/anchor semantics;
TestParseEntry pins the ConfigMap contract. commerce + subsystems suites
green.


* feat(tasks): UI at /tasks (no /_/) + org/project/user-native gate (tasks v1.51.0)

The Tasks console moves to console.hanzo.ai/tasks + tasks.hanzo.ai/tasks —
the /_/ prefix is gone. Subsystem routes mount before the console SPA
catch-all, so /tasks wins the path and every other console route keeps the
SPA fallback. The gate now threads the FULL validated identity — org,
project (X-Project-Id, minted by identity middleware from the validated
project claim), user, email — into the engine via tasks v1.51.0's
WithIdentity; convention: project ↔ tasks namespace inside the org shard.
ZAP plane unchanged (loopback ungated + IAM-gated cluster listener).

The embedded SPA bundle is rebuilt with base /tasks/ in a follow-up sync
commit (hanzoai/admin builds it; clients/tasks/ui/dist is the embed).


* ci: retrigger (Actions dropped the push event)

* chore(tasks/ui): sync SPA rebuilt with base /tasks/ (hanzoai/admin@448b189)


---------

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-12 09:53:06 -07:00
antje 433ad0c4b6 chore(audit): dual-read CLOUD_AUDIT_DATASTORE_* with legacy CLOUD_AUDIT_CLICKHOUSE_* fallback
Prefer the datastore-named env keys, fall back to the clickhouse-named legacy
keys so a rollout mid-flight (new binary / old manifest, or vice versa) never
drops the audit OLAP mirror connection. Code-only; no roll.
2026-07-12 09:48:18 -07:00
antje c8a2cb5d3d fix(deps): ai v1.805.11 (datastore fork) — kill clickhouse double-registration; native commerce read + per-subject finance deposit
ai v1.805.10 pulled upstream ClickHouse/clickhouse-go via ai/object, which double-registered
the clickhouse driver against cloud's hanzo-ds/go fork → boot panic (CI smoke caught it). ai v1.805.11
merges the clickhouse→hanzo-ds purge, so the upstream driver is no longer imported (verified: 0 in the
build, binary boots clean). Also: commerce.BalanceCents (native in-proc balance read — backfill + cockpit
read real balances, no commerce.inproc) and POST /v1/admin/finance/deposit (fund a SPECIFIC wallet:
hanzo/z, not just the pool).
2026-07-12 09:47:50 -07:00
hanzo-dev b55fe5fee4 chore(deps): bump hanzoai/ai v1.805.10 → v1.805.13 (Anthropic /v1/messages tools translation) 2026-07-12 09:28:30 -07:00
hanzo-dev e3efd3c17d merge: hanzo code CLI launcher (claude/codex/dev on any Hanzo cloud model) 2026-07-12 09:25:20 -07:00
hanzo-dev d8ae08da5c merge: LOW-1 — ml/provisioning balance Gate keys on home org (principal.Payer), matching the debit (Red fast-follow) 2026-07-12 09:08:47 -07:00
hanzo-dev f6905b18c8 fix(billing): ml + provisioning pre-create GATE keys on home org (principal.Payer), matching the paired debit
LOW-1 fast-follow (Red). In these two create-handlers the DEBIT correctly keyed on the
HOME org (principal.Payer(c)) but the pre-create balance GATE still keyed on the
EFFECTIVE org — the two swapped only the meter, not the gate (their Gate signature had
gained a projectValidated arg, so the class swap missed them). Effect: a SuperAdmin
masquerading into a victim org was balance-gated on the VICTIM's funds while the debit
landed on admin's ledger (money already correct; an availability/consistency nit).

Fix: swap the Gate org arg -> principal.Payer(c) in both handlers, restoring the
invariant the other ~8 resource meters hold — gate + debit BOTH key on home; data scope
(namespace/project) stays effective.

Test: TestResourceMeter_GateKeysOnPayerForMasqueradingAdmin resolves principal.Payer(c)
from a masquerade ctx (X-User-Owner=admin, X-Org-Id=victim -> admin) and asserts the
recCommerce balance check keyed on admin (home), never victim. go build ./...=0 (links),
gofmt clean, root ResourceMeter suite + clients/ml green (provisioning store tests are the
pre-existing CLOUD_KMS_MASTER_KEY_REF env gate, orthogonal).
2026-07-12 09:07:58 -07:00
zeekayandhanzo-dev fc9fdd8203 git: re-integrate SSH + ZAP + client-less REST push onto the generics framework
Ports the prior Git-over-SSH + ZAP transport + client-less REST push work
(commit 36c52d1, written against the old *svc-receiver service) onto the
current cloud.Service[state] generics framework on main. A port, not a
rewrite: the DRY architecture is preserved — ONE control-plane core, ONE
git pack path, three thin transports (REST, ZAP, SSH) over them.

Handler pattern adapted: every `*svc` method became a free function taking
`s *cloud.Service[state]` (matching git.go's existing create/list/get/del),
`cloud.TenantStore` → `cloud.NewOrgStore`, methods → `cloud.Handle(s, fn)`
route registration, `s.log`→`s.Log`, `s.stores/storage/keys/ssh`→`s.State.*`,
`tenant(c)`→`org(c)` (principal.Org). Package-scoped helpers (storeFor,
provision, recordUsage, refState, cloneURL, sshURL, firePushBuilds, session)
are the current framework's free-function forms.

New files:
- core.go   coreCreate/List/Get/Delete/Usage — the ONE transport-agnostic
            control-plane impl; REST + ZAP both call it.
- pack.go   serve{Upload,Receive}Pack + ssh{Upload,Receive}Pack — the ONE git
            pack code path both smart-HTTP and SSH drive (io.Reader/Writer,
            transport-agnostic); readerOnly guards the SSH channel write side.
- ssh.go    golang.org/x/crypto/ssh server. Host key from CLOUD_GIT_SSH_HOST_KEY
            (KMS env) or on-disk ed25519 generated+persisted 0600.
            PublicKeyCallback resolves key→(org,user) by SHA256 fingerprint,
            fail CLOSED. Session accepts only git-upload-pack/git-receive-pack,
            enforces path-org == key-org. Listen CLOUD_GIT_SSH_ADDR (:2222).
- keystore.go  fingerprint-indexed global SSH public-key registry (public keys
               only — nothing to hash).
- keys.go   POST/GET/DELETE /v1/git/keys.
- push.go   POST /v1/git/repos/:name/push — build tree+commit from posted
            utf-8/base64 files, advance ref, create-on-first-push (composes the
            ONE provision), fire the SAME firePushBuilds hook receive-pack fires,
            return commit + cloneUrl + sshUrl.
- zap.go    git/zap/{createRepo,listRepos,getRepo,deleteRepo,usage} envelope
            adapters over the SAME core funcs, on the shared /zap plane. No
            second ZAP server, no gRPC.

Modified:
- git.go       repoView/toView gain sshUrl; state holds sshHost/ssh/keys; Mount
               opens the keystore + starts the SSH listener; routes() wires
               push+keys+ZAP; Shutdown stops SSH + closes the keystore. REST
               control-plane handlers are now thin adapters over core.go.
- smart_http.go  uploadPack/receivePack drive the shared pack.go path via
                 resolvePackRepo; firePushBuilds + session helpers retained.

Tests (go test ./clients/git/... — 16 pass): SSH key register accept/reject +
cross-org reject + end-to-end SSH clone+push over an in-process listener; a
zapface WS round-trip proving the ZAP path hits the SAME core as REST; REST
push create/update/build-hook, first-push provisioning, validation, and
cross-org isolation. tenant_isolation_test + git_test bind the SSH listener to
an ephemeral loopback port so tests never collide on :2222.

CGO_ENABLED=0 GOWORK=off go test ./clients/git/... passes; go build clean;
gofmt clean.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-12 09:05:56 -07:00
antje 63efcc84ca feat(finance): route edge meter + admin grant onto the finance ledger, + commerce→finance backfill
metering.Client fetchAvailable/Record now resolve finance.Current() first (micros→cents
ceil so metered_ai's AmountMicros debit is never dropped), commerce HTTP only when finance is
not co-resident. admin ApplyGrant credits the finance wallet (fixes the broken commerce.inproc
grant path). New DepositInput.Ref makes deposits idempotent; finance.MigrateOrg + SuperAdmin
POST /v1/admin/finance/backfill?org= carry each org's commerce balance into its finance wallet
exactly once at cutover. All money — balance, usage, deposit — now flows through the ONE per-org
double-entry ledger.

Verified: build + go vet clean; finance/metering/admin/core tests pass (micros fold, ref
idempotency, backfill exactly-once, zero-HTTP-when-co-resident all locked by tests).
2026-07-12 09:02:36 -07:00
antje afe5ce9fc9 feat(finance): per-org double-entry prepaid wallet ledger (finance.Client) + ai native hooks
New clients/finance: ONE lightweight SQLite file per customer (orgs/<org>/finance.db),
double-entry wallet on the treasury/ledger core — deposit=funding→wallet, usage=wallet→revenue,
balanced within the org file, idempotent. Orthogonal money interface types.FinanceClient
(finance.Client, mirrors commerce.Client); NOT bolted onto the Deps bag and NO pick/RPC/disabled
cruft — a narrow published seam (finance.Publish/Current) every money consumer resolves.

BuildDeps.wireFinance constructs+publishes it and installs the ai router's typed
BalanceReader/UsageRecorder hooks (ai v1.805.10) so the prepaid gate reads balance + debits usage
by a DIRECT in-proc call, no HTTP. Rob-Pike small interfaces between systems, not a god-struct.

Remaining before cutover: edge meter + admin grant onto finance.Current(), backfill commerce
balances into <org>:wallet, then build/deploy/prefund/drop BALANCE_EXEMPT_USERS.
2026-07-12 09:01:38 -07:00
antjeandGitHub 3080a2fb07 chore(deps): pin all hanzoai/luxfi deps to clean v1 OSS tags (#269)
Drop pseudo-versions and +incompatible for our own modules; everything now
resolves to a published v1 semver tag (v1 only, per policy):

  goa            v0.0.0-pseudo        -> v1.0.0   (first tag)
  sign           v0.0.0-pseudo        -> v1.0.0   (v1 path; 36 bogus v2 tags dropped)
  captable       v0.0.0-pseudo        -> v1.0.0   (first tag)
  pubsub-go      v1.0.1-0.pseudo      -> v1.53.0  (existing clean tag; HEAD==v1.53.0)
  gochimp3       v0.0.0-pseudo        -> v1.0.0   (module path fixed: Elandiro -> hanzoai)
  goauthorizenet v0.0.0-pseudo        -> v1.0.0   (go.mod added)
  sendgrid-go    v3.4.2-...+incompat  -> v1.3.0   (go.mod added; v1 path)
  luxfi/keys     v1.2.2 (force-moved) -> v1.4.0   (clean current tag)

CGO_ENABLED=0 go build ./... green.
2026-07-12 00:58:01 -07:00
775cf27ce9 fix(release): race-safe version assignment — unjam the cloud release lane (#271)
Root cause of the jammed lane (failed releases + phantom tags like v1.786.221):
the push step published the racy v<X.Y.Z> tag BEFORE it was confirmed free, and
the tag step hard-failed when a concurrent release had claimed that number
(computed 220, already tagged → 'rejected, already exists'). Result: a pushed
image with no matching git tag, mutable :vX clobbering, and dead releases.

Fix — decouple the immutable artifact from the versioned release:
- Push step publishes ONLY the unique sha-<sha7> (+ floating latest); never the
  racy version tag.
- Tag step assigns the next FREE v<X.Y.Z> ATOMICALLY: recompute fresh, retag the
  proven sha-image → :vX/:X.Y.Z/:X.Y via imagetools (metadata-only, byte-identical),
  git-tag it, and RETRY on collision (the git-tag push is the serialization point —
  a loser recomputes). Concurrent releases each grab a distinct free number.
- Compute-step collision is now a non-fatal hint (Tag step owns assignment).
- notify-universe rolls the Tag step's FINAL version, not the compute guess.

Preserves the invariant: git tag vX ⇔ image :vX pushed + smoke-passed. Logic
unit-tested locally (free-find + collision-skip). YAML valid.

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-11 23:58:40 -07:00
hanzo-dev afbf01d13f merge: admin-org predicate (owner==admin) + home-org billing (debit/gate on X-User-Owner, data scope on X-Org-Id) — admin masquerade bills the admin ledger 2026-07-11 23:41:30 -07:00
hanzo-dev f42309dcd1 billing: debit+gate on HOME org (X-User-Owner); embedded-commerce sudo = owner==admin
THE billing fix (B): split 'who pays' (HOME org, X-User-Owner) from 'whose data'
(EFFECTIVE org, X-Org-Id), which the old code conflated onto one org. A platform
SuperAdmin masquerading into another org now spends from the admin ledger; data
scope stays on the acted-on org.

- clients/principal: Owner(c) (X-User-Owner, bounded+cloned), BillingOrg(c) (home w/
  effective fallback, Validated-gated), Payer(c) (bare-string for the in-handler meters).
- middleware_identity: mint X-User-Owner=home for every validated principal (before the
  admin org-switch) + strip it on ingress (authorityHeaders) so a client can't forge who pays.
- middleware_billing: identityFromCtx keys User+Org (balance check AND debit) on BillingOrg.
- 11 resource-meter live sites (visor/s3/security/platform-run/functions/ml/provisioning/
  bots/tracker/automations): Gate+Meter+MeterUsage bill principal.Payer(c) (home); Org stays
  effective for the namespace. Background/reconcile paths (a.Org/r.Org/b.Org, meterRun) and
  the studio_render 'org' param bill the RESOURCE's own org — FLAGGED (see Red handoff).
- metered_ai + types: ChatRequest/EmbedRequest gain BillingOrg; meteredAI gate+record key on
  billedOrg(BillingOrg,Org); threaded principal.Payer(c) through the code engine (Synthesize/
  Embed) so internal RAG bills home too. Inner AI call keeps req.Org (data scope).

Part A (embedded commerce, clients/commerce mirror of the commerce repo): drop the spoofable
IsSuperAdmin boolean; auth.IAMClaims.IsSuperAdmin() gates homeOrg()=="admin" (HomeOrg from
X-User-Owner ?: Owner); iammiddleware reads X-User-Owner->HomeOrg (Owner stays effective);
EdgeAuth strips+mints X-User-Owner before the ?org override, stops minting X-User-IsSuperAdmin;
all .SuperAdmin() call sites -> .IsSuperAdmin(). Org-scoped IsAdmin untouched.

Tests (green): TestIdentityFromCtx_AdminMasqueradeBillsHomeOrg (owner=admin+effective=victim ->
debit on admin, data on victim), TestBilledOrg, TestMeteredAI_AdminMasqueradeBillsHomeOrg
(end-to-end debit user=admin), vendored TestIAMClaims_IsSuperAdmin masquerade+anti-escalation,
EdgeAuth strips forged X-User-Owner. go build ./...=0, go vet=0.

Rollout: pre-gateway (no X-User-Owner) billing falls back to effective (home==effective for a
normal caller; masquerade fails CLOSED). Deploy gateway (mints X-User-Owner) first.
2026-07-11 23:36:25 -07:00
hanzo-dev dfda1eced2 cli(code): fail loudly on unresolved models; declare the Hanzo provider for codex
Three defects found by testing a real completion instead of --version:
  - an unreachable catalog silently passed the typed id straight through, so an
    agent booted on a model that does not exist and failed opaquely mid-session;
    resolution is now strict.
  - a stale ANTHROPIC_API_KEY in the shell outranks ANTHROPIC_AUTH_TOKEN and
    silently wins; it is cleared for the claude wire.
  - codex ignores OPENAI_BASE_URL and talks to chatgpt.com unless a provider is
    declared; declare Hanzo and select it (wire_api=responses, per upstream).
2026-07-11 23:34:33 -07:00
hanzo-dev 9276d9ba2b docs(identity): last 'global admin' prose -> SuperAdmin (zero repo-wide) 2026-07-11 23:22:37 -07:00
hanzo-dev 4b45286ff9 refactor(identity): ONE fact per predicate — SuperAdmin = admin-org membership, OrgAdmin = own-org admin
Cloud was the odd one out: it minted SuperAdmin from TWO signals
(claims.IsAdmin && owner == adminOrg) while IAM's canonical User.IsSuperAdmin() is
just user.Owner == conf.AdminOrg, and IsOrgAdmin folded SuperAdmin into itself. Two
predicates that each meant one-and-a-half things.

Decomplected — two orthogonal facts, one predicate each:
  IsSuperAdmin = owner == adminOrg          (platform sudo; the SAME equality IAM uses)
  IsOrgAdmin   = the IAM isAdmin bit         (admin of one's OWN org; implies nothing about super)
A gate admitting either now writes IsSuperAdmin(c) || IsOrgAdmin(c) explicitly, so the
superset is visible AT the gate, not hidden inside a predicate. GuardScoped already did.

The admin org holds ONLY SuperAdmins (provisioned in, never promoted), so membership IS
the fact — the isAdmin bit is the orthogonal org scope, never a super gate. The KMS
machine-principal exclusion STAYS (a real guard: an admin-org machine token must never
be super). Test renamed + strengthened to lock the one predicate. Build ./... clean,
identity/admin/principal suites all green, KMS-machine exclusion tests pass.
2026-07-11 23:21:17 -07:00
d613345067 security: golang-jwt/jwt v3 → v4 (no-patch high; v3 line abandoned) (#270)
The 4 commerce auth/token files imported the vulnerable golang-jwt/jwt v3.2.2
(dependabot high, no v3 patch — fix is off-v3). v4.5.2 (already present) keeps
StandardClaims/Valid()/int64 dates → API-compatible, zero code ripple vs the
v5 RegisteredClaims rewrite. Dropped the orphan v3 require. Full build + commerce
auth tests green. (dgrijalva/jwt-go remains transitive-only = unreachable.)

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-11 23:19:39 -07:00
antjeandGitHub 4121895cc4 chore(deps): bump embedded iam to v1.31.22 (#266)
Advances the in-process IAM embed (clients/iam, HIP-0106) from the
v1.31.20 pre-release pin (31a798fb, #117) to the published v1.31.22 tag
(387c4a60). Brings the project claim mint + default-project seed (#120)
into cloud's embedded IAM so console admin surfaces match standalone
hanzo.id, already on v1.31.22.

API-compatible: iamserver.InitEmbed and object.Project CRUD unchanged
(iam go.mod hash identical across versions), no clients/iam adaptation.
CGO_ENABLED=0 go build ./... green; clients/iam + clients/platform tests pass.
2026-07-11 22:51:42 -07:00
4957073775 cloud: bump zip v1.6.0 + licensing v0.1.4 — off the deleted App.Mount, one-way surface (#267)
zip v1.6.0 deleted the deprecated/redundant surface (App.Mount/Route/ModuleFn/
UseFiber). cloud is off App.Mount (#257); licensing v0.1.4 migrated its one
App.Mount call. Build 0, framework tests green.

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-11 22:51:03 -07:00
hanzo-dev 56db31b79f cli: hanzo code — launch claude/codex/dev on any Hanzo cloud model
One way to start any coding agent on any Hanzo model: endpoint, credential and
model are injected, so nothing is exported by hand. Two wires cover all three
agents (claude speaks Anthropic; codex and @hanzo/dev, a codex fork, speak
OpenAI), model ids resolve fuzzily (glm5.2 -> glm-5.2), and agents run full-auto
unless --safe. Config gains apiKey; the key the rest of the toolchain already
keeps in ~/.hanzo/config.json is read (never written) so one key serves every tool.
2026-07-11 22:36:34 -07:00
zandantje b1f3da2ac9 chore(datastore): purge clickhouse-go/datastore-go → hanzo-ds + debrand (Group A, v3)
Rebased on current origin/main (37a438a3, +4: #262/#258/#261/prepaid-wiring).
- ai v1.805.9 (= v1.805.7 hardened prepaid gate + purge) — NOT the money-regressing v1.805.8
- o11y v1.5.15, otel-collector v0.144.8-hanzo.0
- datastore clients → hanzo-ds/go; ClickhouseDB()→DatastoreDB(); clickhouse→datastore rip
- go.mod clickhouse-free; go list -deps ./cmd/cloud clickhouse=0; 0 BALANCE_EXEMPT in linked ai
- boots clean fail-closed (single sql.Register); luxfi pins as live; wire/DSN/env contracts preserved
2026-07-11 22:25:32 -07:00
d5d8e74099 docs(projects,tracker): mark the two Project structs as deliberately distinct (#264)
WS4 audit flagged projects.Project and tracker.Project as duplicate types.
They are not: projects.Project is the Slug-keyed deployable site (one global
projects.db, org column); tracker.Project is a KEY-prefixed Linear-style issue
team living in a per-(org,project) tracker.db WITHIN one of those sites. Only
the universal storage-row skeleton (id/org/name/description/timestamps) is
shared; the natural keys carry different semantics (DNS label vs issue-ID
prefix) and the field sets are not subsets. Record the distinction on each type
so the finding is not re-litigated. Comment-only; no API or behavior change.

Co-authored-by: Hanzo <dev@hanzo.ai>
2026-07-11 22:13:06 -07:00
37a438a33c feat(billing): thread validated-project bool through ResourceMeter.Gate (#262)
principal.ValidatedProject(c) now flows through ResourceMeter.Gate into
metering.AuthInput.ProjectValidated, so resource-creation caps harden
consistently with the edge BillingGate. Removes the hardcoded
ProjectValidated:false at resource_billing.go.

- Gate signature gains projectValidated bool (adjacent to project, mirroring
  principal.ValidatedProject's return + AuthInput's field order).
- Every *zip.Ctx caller passes principal.ValidatedProject(c); the three
  no-principal/client-body paths (metered_ai LLM decorator, background agent
  run, content studio in.Project) pass false — unvalidated stays soft, never
  fabricated true.
- Meter/Record path unchanged: Usage carries no ProjectValidated; cap
  enforcement is Gate-only, so threading it there would be dead code.

Stays SOFT in prod today: ValidatedProject is true only for a validated NAMED
project, and IAM seeds none yet, so no org has a project claim. No behavior
change now; named-project caps auto-harden per-org as IAM seeds them.

Co-authored-by: Hanzo <dev@hanzo.ai>
2026-07-11 21:44:51 -07:00
220cd8bcf8 feat(platform): collapse buildType surface to pack|dockerfile (one way) (#261)
The build path (buildFrontendCmd) already chose its BuildKit frontend on
dockerfile presence alone: an explicit Dockerfile → dockerfile.v0, otherwise
hanzoai/pack via gateway.v0. buildType never gated the build — it was stored
metadata whose closed set still advertised the retired nixpacks/buildpacks/
static strategies and defaulted git apps to nixpacks.

Collapse it to the one true surface:
- closed set is {pack, dockerfile}; git source defaults to pack, not nixpacks
- image source always yields buildType image (no build), forced by source, so a
  client can no longer stamp a git strategy on a prebuilt-image app
- Goa contract (design.go + generated openapi3.json/yaml) enum, description and
  examples track the same {pack, dockerfile, image} set

Adds TestBuildTypeSurface pinning the default (pack), the dockerfile escape
hatch, rejection of every retired strategy, and image-source forcing.

Co-authored-by: Hanzo <dev@hanzo.ai>
2026-07-11 21:44:48 -07:00
dcdd317cc8 refactor(identity): collapse hand-rolled org/project reads onto principal (#258)
Six identity reads re-derived org/project inline instead of going through
principal — the ONE accessor. Route them all through it so the trust
decision lives in exactly one place and cannot drift.

- analytics tenant(): delegate to principal.Org — fixes a retained-buffer
  bug (org keyed the cloud_usage ledger past request end as a zero-copy
  fasthttp view; principal.Org clones). Gate unchanged.
- git/security projectScope(): read via principal.Project; absent header
  and literal "default" both map to the un-suffixed default scope via
  principal.IsDefaultProject, keeping today's (org,project) key shape.
- admin core.ResolveScope / core.GuardScoped / me / core.EmitAudit: read
  org via principal.Org (validated principal + org), composed with the new
  principal.IsOrgAdmin predicate. Admin-org bucket kept local.

Co-authored-by: Hanzo <dev@hanzo.ai>
2026-07-11 21:44:08 -07:00
hanzo-dev f6afc51056 feat(billing): prepaid AI gate — wire ai v1.805.7 to in-process commerce, no exempt
Pin hanzoai/ai v1.805.7 (exempt concept removed: no BALANCE_EXEMPT_USERS/KEYS, no
fail-open; balance-unverifiable now DENIES). BuildDeps sets aiobject.CommerceTransport
= commerceinproc.Transport() and self-configures the ai module's commerceEndpoint to the
in-process placeholder when commerce is co-resident, so the embedded ai router's balance
read + usage debit hit the raw commerce gin (service-token, no socket) — the SAME one
ledger the metering client bills — not the customer proxy that 401s a service token.
Every external inference is now prepaid-gated with zero exempt principals.

hanzo/z was a normal customer, never admin; exempting it leaked money. Closed.
2026-07-11 21:37:00 -07:00
be0f0f002b feat(platform): delegate project lifecycle to IAM (one source of truth) (#260)
Projects are now owned by Hanzo IAM (hanzo.id) as the org-scoped
(owner,name) resource. Platform REFERENCES that store instead of owning
one: apps still live under a project, but create/list/get/delete/exists
of the bare project delegate to IAM in-process.

- New ProjectStore port (projects.go) with an in-process iamProjects
  adapter over github.com/hanzoai/iam/object — no HTTP hop, and IAM's
  canonical *object.Project is used verbatim (no platform-local clone).
- Delete platform's project ownership: the Project struct, the
  platform_projects table, and Create/Get/GetByID/List/Update/Delete
  project store methods. DeleteProject is replaced by DeleteProjectApps
  (cascade of the app tree only; the project row is IAM's).
- Apps key on the IAM project NAME (platform_apps.project_id holds it);
  the internal proj_ id indirection is gone, so reconcile/preview/domains
  use app.ProjectID directly and drop their project lookups.
- run.go get-or-creates the default project via the ProjectStore, using
  principal.DefaultProject as the single source of truth for "default".
- tenant() gates on principal.Validated(c) (canonical), not a raw
  c.User()=="" whitespace-blind check.
- Tests: fake ProjectStore + TestProjectLifecycleDelegatesToIAM proving
  create/delete route through IAM and delete cascades platform's apps.

Co-authored-by: Hanzo <dev@hanzo.ai>
2026-07-11 21:01:24 -07:00
36405143c1 feat(identity): bind X-Project-Id from the validated project claim; harden per-project caps (#259)
SanitizeIdentity now mints X-Project-Id from the validated JWT `project` claim
(idClaims.mintedProject), exactly like X-Org-Id from `owner` — the raw client
X-Project-Id is never a source. Still checked non-foreign to the effective org, so
a global admin viewing another org drops their own-org project. Mirrors the edge
(iamauth.Claims.MintedProject), so the in-binary path binds the same header.

principal.ValidatedProject flips CONDITIONALLY: (project, true) only for a
validated, NON-default project claim — the signal a project-scoped spend cap uses
to HARD-enforce. The default project stays soft (IAM does not seed default
projects yet; a blanket true would wrongly hard-402 every org). Named project caps
auto-harden one org at a time as projects are seeded.

Tests: claim-sourced project binding + forged/override client X-Project-Id ignored
+ cross-org claim refused on admin org-switch + X-Org-Id forgery still blocked;
ValidatedProject claim-backed→(project,true), absent/default→soft, unvalidated→soft.

Follow-up (WS3, out of scope): resource_billing.go Gate hardcodes
ProjectValidated:false — resource-creation caps stay soft until it threads
principal.ValidatedProject.

Co-authored-by: Hanzo <dev@hanzo.ai>
2026-07-11 21:00:51 -07:00
hanzo-dev 45a68002d5 refactor(auth): rename GlobalAdmin -> SuperAdmin (SOC2/FedRAMP term)
Eliminate "global admin" terminology across the cloud repo in favor of the
standard SuperAdmin term. Pure naming refactor; behavior preserved.

Identifiers (commerce auth + middleware + root):
- (*auth.IAMClaims).GlobalAdmin() -> SuperAdmin() + all call sites
- IAMClaims.IsGlobalAdmin field -> IsSuperAdmin (JSON tag isGlobalAdmin
  -> isSuperAdmin; SuperAdmin() still honors owner=="admin", the canonical
  predicate, so behavior is preserved)
- edgeauth isGlobalAdmin() helper -> isSuperAdmin(); requireGlobalAdmin ->
  requireSuperAdmin; test-local globalAdmin vars -> superAdmin
- const HeaderUserIsGlobalAdmin -> HeaderUserIsSuperAdmin, value
  "X-User-IsGlobalAdmin" -> "X-User-IsSuperAdmin" (commerce-internal header:
  minted+stripped in edgeauth, read in iammiddleware; no external sender)
- auth/globaladmin_test.go -> auth/superadmin_test.go

clients/admin: the deprecated isGlobalAdmin JSON alias (sibling of the
canonical isSuperAdmin, same value) is REMOVED rather than renamed — a rename
would collide with the existing IsSuperAdmin field, and the alias was a
transitional back-compat scaffold for this very migration. Tautological
alias-equality assertions dropped; isSuperAdmin checks retained.

Prose: "global admin" / "global-admin" / "GLOBAL-ADMIN" / "GlobalAdmin" in
comments and docs (LLM.md, k8s manifests) rewritten to SuperAdmin, case-aware.

Left untouched: cloud gateway header X-User-IsAdmin; principal.IsSuperAdmin /
IsOrgAdmin; svcorg's DefaultNamespace (global) kind.

build ./... clean; vet clean; SuperAdmin behavior-locking tests pass
(commerce auth/edgeauth/iammiddleware/catalog/costs/checkout, admin
scope/money, principal, root identity). gofmt clean. Zero GlobalAdmin /
global-admin remaining in .go/.md/.yaml.
2026-07-11 18:20:58 -07:00
hanzo-dev 675fdbb637 feat(admin): per-provider credit ledger + usage funding endpoints
Two SuperAdmin endpoints for multi-provider credit-management (the console renders
them; contract is authoritative):
  GET /v1/admin/providers/credit  -> [{provider, grant_cents, burn_cents,
      remaining_cents, runway_days, has_credit, is_paid_only}]
  GET /v1/admin/usage/funding?from&to -> [{provider, model, funding, tokens,
      cost_cents, requests}], funding in {credit,paid,paid_only,byo}

DRY: reuses the admin auth guard (core.Guard), finance.go's DO billing read (real
k grant, live remaining/burn/runway), and the ONE cloud_usage warehouse (no new
store). DO is the seeded real row; others land as keys arrive. Funding is
provider-level-derived in v1; the per-call split lands when the ai metering write
stamps a funding column (next commit). Envelope = core.OK {status,msg,data:[...]}.
Two-ledger model kept distinct: UPSTREAM provider credits here vs DOWNSTREAM Hanzo
customer billing (commerce). Pure logic TDD'd.
2026-07-11 17:24:20 -07:00
hanzo-dev d7716ed604 refactor(principal): IsSuperAdmin + IsOrgAdmin — two named predicates, not bare c.IsAdmin()
Per the SuperAdmin convention (SOC2/FedRAMP; standard term SuperAdmin, NEVER
'global admin'): name the two admin scopes explicitly.
- principal.IsSuperAdmin(c) = c.IsAdmin() — platform sudo; SuperAdmin ⟺ owner==admin org
  (X-User-IsAdmin minted only for that identity).
- principal.IsOrgAdmin(c) = IsSuperAdmin(c) || X-User-IsOrgAdmin — admin of one's own org.
GuardScoped now reads IsSuperAdmin (fast-path) + IsOrgAdmin (scoped) instead of bare
c.IsAdmin()/OrgAdmin. Pure rename — no behavior change, identity suite green.
2026-07-11 17:22:13 -07:00
hanzo-dev 0ae41f9f90 fix(deps): bump hanzoai/ai v1.805.5 -> v1.805.6 (failover credit-cascade)
v1.805.6 makes credit/quota exhaustion + agreement gates (402/403/insufficient_quota)
cascade to the next provider in the fallback chain — Phase 2 of multi-provider
credit-management + fixes the do-ai->anthropic opus fallback.
2026-07-11 17:15:52 -07:00
hanzo-dev 3c2179d928 admin: close same-tenant over-visibility gap in GuardScoped
GuardScoped admitted ANY validated org member to their own org's admin
panels (/v1/admin/{overview,orgs,users,usage,analytics,me,bases}), not
just org-admins: SanitizeIdentity discarded the JWT org-level isAdmin bit
for non-admin-org principals (it only minted the GLOBAL X-User-IsAdmin
when owner==adminOrg), so GuardScoped's fallback (validated User+Org) let
a plain member through.

Mint a new, unforgeable X-User-IsOrgAdmin signal and require it:

- middleware_identity.go: add X-User-IsOrgAdmin to authorityHeaders
  (stripped on every ingress request, so a client can never forge it),
  and mint it for any validated principal with claims.IsAdmin &&
  !isKMSMachinePrincipal — covering both a global admin and an org-admin,
  owner-scoped and machine-excluded.
- clients/principal: add OrgAdmin(c) = IsAdmin() OR X-User-IsOrgAdmin.
- clients/admin/core.GuardScoped: require principal.OrgAdmin(c) in
  addition to validated User+Org; a validated non-admin member now gets
  the same 403 as an unvalidated caller. Guard (global-only) untouched.

Tests: orgAdminHdr now carries the bit; add TestScope_MemberWithoutOrg
AdminDenied (member without the bit is 403 on every scoped panel) and
TestSanitizeIdentity_OrgAdminHeader (org-admin gets the bit not global,
member/machine get neither, forged bit stripped on bearer + anon paths).
2026-07-11 17:11:53 -07:00
hanzo-dev 8f23770eef Merge branch 'moneysafe' 2026-07-11 16:58:15 -07:00
hanzo-dev 496f4e76f4 admin: fail-closed grant durability + deposit idempotency (no unaudited/double money)
Finding 3 (grant durability — ARCHITECT: FAIL-CLOSED). EmitAudit is a silent
no-op when State.AuditStore == nil, so on a no-audit-store deployment a
SuperAdmin grant moved REAL money with NO durable cloud-side before/after record.

Decision: FAIL-CLOSED. Grants are a real-money op and the audit store is always
present in any real deployment — audit_serve.go REQUIRES a persistent data dir
for the trail (hard boot error otherwise), and a nil store arises ONLY from the
explicit CLOUD_AUDIT_DISABLED dev opt-out. So ApplyGrant now refuses the grant
with 503 BEFORE any money moves when AuditStore == nil: no unaudited money move.
The check sits after org validation and before the deposit, so the existing
validation errors (amount/cap/unknown-org) are unaffected.

Residual (documented, not regressed): a post-deposit Append failure (store
present but the write errors) keeps the money-moved success response — the money
DID land, and failing it would both misreport the grant and, absent idempotency,
invite a double-credit retry. It stays a loud log, backstopped by the
request-level AuditTrail middleware which independently records the request and
fails the request CLOSED on its own write error (serve.go / audit_middleware.go).

Finding 4 (deposit double-credit on retry — WIRED). commerce.Deposit posted
/v1/billing/deposit with no idempotency key, so a commit-then-timeout (cloud's
15s client) could drive an operator double-credit on retry. The commerce backend
DOES enforce idempotency: POST /v1/billing/deposit reads X-Idempotency-Key,
scoped billing-deposit:<subject> — a completed key REPLAYS the receipt, an
in-flight key 409s, an absent key is legitimately additive (never dedup by
amount). Verified in ~/work/hanzo/commerce/api/billing/deposit.go.

Fix: thread a DETERMINISTIC key from the grant. Deposit/post gained an
idempotencyKey arg sent as X-Idempotency-Key. ApplyGrant derives it as
sha256(org|amountCents|currency|source|nonce) where nonce is the operator-supplied
Idempotency-Key — so a retried grant carrying the SAME nonce dedupes at commerce
while two DISTINCT grants (even same org+amount) never collide, and a nonce reused
for a DIFFERENT amount still lands (dedup can never silently DROP a real grant).
No nonce => no key => additive default preserved; we do NOT fabricate a
content-only key (it would wrongly dedupe two legitimate identical comps).
Residual (cross-service, frontend): effective end-to-end once the operator
console sends an Idempotency-Key per grant attempt, reused verbatim on retry.

Finding 2 (GuardScoped over-visibility — NEEDS CROSS-SERVICE, not fixed here).
GuardScoped admits any validated principal with non-empty c.User()+c.Org().
SanitizeIdentity (middleware_identity.go) mints X-User-Id for EVERY validated org
member and mints the admin signal X-User-IsAdmin ONLY for a GLOBAL admin
(owner==adminOrg) — it DISCARDS the JWT org-level isAdmin bit for non-admin-org
principals. So a regular member's sanitized identity is byte-identical to an
org-admin's, and GuardScoped admits regular members to their own org's admin
panels (same-tenant over-visibility of financials + user directory; NOT
cross-tenant — ResolveScope pins to c.Org()). The finding is REAL, but the
prescribed fix (require org-admin) CANNOT be done in clients/admin/*: the
org-admin signal is not minted, and middleware_identity.go is read-only in this
scope. Requiring the only admin signal we have (c.IsAdmin(), GLOBAL) would break
the deliberate, test-locked org-scoped tier (scope_test.go admits org-admins).
Correct cross-service fix: in SanitizeIdentity's `case owner != ""` branch, mint
X-User-IsOrgAdmin=true when claims.IsAdmin (owner safe, non-KMS), add it to
authorityHeaders (stripped on ingress so unforgeable), then GuardScoped requires
c.IsAdmin() || X-User-IsOrgAdmin=="true". Left for the middleware owner.

Tests: TestGrantCredit_NilAuditStoreFailsClosed (503, no deposit, balance
unchanged); TestGrantCredit_IdempotencyKeyForwarded (same nonce=>same key,
distinct nonce=>distinct key, no nonce=>no key). Existing money must-pass tests
(TestGrantCredit_DepositLandsAndAudited, TestSuspendReactivate_*, TestAdminAudit_*)
still pass unchanged.
2026-07-11 16:54:33 -07:00
hanzo-dev e8b7e34913 admin: decomplect partial-commerce reporting on /overview (one partial pattern)
Finding 1 (undercount masked as healthy). core.OrgMoney swallowed the
Commerce.Spend/Credits errors to a silent zero, and /overview derived the
"commerce" source freshness from a SINGLE probe org — so if commerce was down
for 40 of 41 orgs the fleet spend/credits totals were an undercount while the
source still read "healthy". revenue.go / finance.go already report this
correctly via a `partial` flag + the core.ErrPartialRevenue sentinel.

Fix (decomplect to ONE partial pattern, not a second one):
- OrgMoney now returns (spend, credits int64, ok bool); ok is false when the
  spend OR credits read failed — the SAME (row, ok) contract revenue.revenueOf
  uses. An unwired commerce is NOT a failure (Spend/Credits return (0,nil) when
  unconfigured), so ok stays true and Ready() still distinguishes not-configured.
- overview folds ANY per-org OrgMoney failure into the commerce source as
  degraded, reusing core.ErrPartialRevenue, and derives freshness from the SAME
  per-org reads the totals fold instead of a single probe org.

orgs is a per-ROW panel (OrgRow[] via OKList; NO sources[] channel): a failed
read degrades THAT org's row to an honest zero — there is no aggregate total to
mislabel, so no change beyond the signature. The customer per-row/detail call
sites are best-effort and unchanged in behavior.

Test: TestOverview_CommercePartialOnPerOrgError — commerce 500s for one org of
two; the commerce source reports not-ok with an error while the healthy org's
spend still contributes (honest partial total, not a hard panel fail).
2026-07-11 16:54:05 -07:00
hanzo-dev 4415067a2f test(graph): align unreachable-upstream tests + doc to the code's honest-empty-200 (not 502) 2026-07-11 16:36:30 -07:00
hanzo-dev 519e5f5012 chore(kms): delete dead clients/kms/replication — a lone test with no production source
The subpackage contained only replication_test.go referencing an undefined Producer
(no NewProducer/BackupOnce/etc. anywhere), so it never compiled and failed go vet.
Zero importers. Pike: the best code is no code.
2026-07-11 16:33:39 -07:00
hanzo-dev 69baee05a5 style(admin): gofmt clients/admin/revenue/revenue.go (new split file) 2026-07-11 16:19:17 -07:00
hanzo-dev ae95251738 Merge branch 'splitA' into splitmerge 2026-07-11 15:37:47 -07:00
hanzo-dev da7fc15bb5 admin: rewire Mount onto core + domains, retire in-package handlers
Rewrite the top-level admin package as the Mount + aggregator only. state ->
core.State (exported fields), and every top-level handler + helper is retyped to
*cloud.Service[core.State] and calls core.* for the shared kernel:
  - routes() registers the org-scoped panels (me/overview/orgs/users/usage/
    analytics/bases) behind core.GuardScoped and the platform reads
    (roles/applications/products/compute/o11y/sync + flags/waitlist) behind
    core.Guard, then delegates to audit/customer/revenue/finance Routes().
  - analytics.go keeps only the analytics-specific derivation (growth/retention/
    churn/active/LTV) folding over the core activity model + spend series.
  - o11y/compute/bases/waitlist/flags/types retyped to core.State + core.*.

Delete the now-moved audit.go/customers.go/grants.go/revenue.go/finance.go/
scope.go (their handlers live in the domain packages; their shared helpers in
core). doTokenFromEnv moves to admin config; iamAuditQuery moves to the audit
package.

Move tests with their code: audit store tests -> audit package; grantTag test ->
core; finance pure-math tests -> finance package. The full-mount integration
tests (admin/scope/cockpit/finance-aggregation) stay in the admin package and
now drive the real routes() so the harness mirrors Mount exactly. Behavior,
routes, tenant scoping and every assertion unchanged.
2026-07-11 15:35:31 -07:00
hanzo-dev 385c45dc5e admin: extract shared core kernel + carve audit/customer/revenue/finance domains
Introduce clients/admin/core as the subsystem's shared kernel — the State
struct (upstream clients + adminOrg + audit store) and the one-copy business
primitives every admin surface composes: the two-tier gate (Guard/GuardScoped),
the /v1 envelope writers (OK/OKList/OKRaw/Fail), CallerCreds, the tenant-scope
predicate (TenantScope/ResolveScope/ScopedOrgs/Descendants), the IAM fan-in
(ListOrgs/OrgMoney/FindOrg/Display/SrcOf/SourceStatus), the ONE credit-write path
(ApplyGrant + EmitAudit + grantTag/grantNote/CreditRequest) and the fleet
activity/time-series model shared by analytics and revenue
(CustActivity/TxnPoint/SeriesPoint/FleetActivity/SpendSeries + bucket helpers).

Carve one package per handler domain over that kernel:
  - audit    -> /v1/admin/audit{,/verify} (store-backed + IAM fallback)
  - customer -> /v1/admin/customers* + /v1/admin/grants (list/detail/credit/
                suspend/reactivate/grants ledger)
  - revenue  -> /v1/admin/revenue
  - finance  -> /v1/admin/finance (+ the pure ComputeFinance derivation)

Each domain imports core for the kernel and shared logic; no business logic is
duplicated. Routes are registered per-domain via <domain>.Routes(app, s).
2026-07-11 15:35:03 -07:00
hanzo-dev 95dfcff940 feat(bots): GET /v1/bots + POST /v1/bots/:runId/stop — org-scoped list+stop over the bot-gateway
Completes the bots surface (launch -> launch+list+stop). Both are thin,
org-scoped proxies onto the in-cluster bot-gateway (BOT_GATEWAY_URL, the same
server-side knob clients/bot uses), carrying the caller's validated tenant
context (X-Org-Id pinned to principal.Org, never a request param).

- GET /v1/bots normalizes the gateway's session rows into
  {runId,task,surface,status,sessionUrl,startedAt}, deriving sessionUrl here
  (the one place a session URL is built). Honest-empty {"bots":[]} on an
  unconfigured/unreachable gateway, a non-2xx, or an undecodable body -- never 5xx.
- POST /v1/bots/:runId/stop returns {runId,status:stopped}; a run the caller's
  org does not own is 404; an unreachable gateway is a clean 502.

Both require a validated principal so org-scoping can't ride a forged X-Org-Id.
Hermetic list+stop tests against a fake gateway assert normalization, sessionUrl
derivation, caller-org scoping, honest-empty, and 404/502 paths.
2026-07-11 15:15:56 -07:00
hanzo-dev c739ec57d7 refactor(admin): extract iam upstream client into clients/admin/iam (finishes the upstream-client layer) 2026-07-11 14:53:10 -07:00
hanzo-dev 298522ab87 fix(deps): bump hanzoai/ai v1.805.4 -> v1.805.5 (enforceBalanceGate fail-open)
v1.805.5 adds the controller-side balance-gate fail-open (BALANCE_GATE_FAIL_OPEN_ON_ERROR)
on top of v1.805.4's nil-guard, so a broken/misconfigured Commerce billing backend
degrades to allowed-but-ungated instead of 500-ing every authenticated chat for
non-exempt users. Together they let the CR drop the commerce.hanzo.svc:8001 bridge
and keep commerceEndpoint unset per the in-process design.
2026-07-11 13:21:45 -07:00
hanzo-dev 2e03d9237b build(cloud): bump hanzoai/o11y v1.5.12 -> v1.5.13 — Hanzo Sentry /v1/sentry backend routes live 2026-07-11 12:39:20 -07:00
hanzo-dev 3d797a8c3a merge: Hanzo Sentry /v1/sentry cloud edge — mount + DSN-ingest gate exemption (Red-GO) 2026-07-11 12:37:25 -07:00
hanzo-dev 0be9250634 fix(deps): bump hanzoai/ai v1.805.3 -> v1.805.4 (nil balanceGate 500)
ai v1.805.4 guards the nil balanceGate deref in resolveBillingKey that returned a
bare HTTP 500 on EVERY authenticated /v1 request (models, chat/completions,
messages) whenever commerceEndpoint is unconfigured (balance enforcement disabled).
RateLimitFilter calls resolveBillingKey before BalanceGateFilter's own nil guard,
so the embedded AI subsystem crashed every authed request while anonymous requests
(no token) 401'd correctly. Fail-open: a disabled billing subsystem resolves no
billing subject and never crashes a request.
2026-07-11 12:27:38 -07:00
hanzo-dev ad9f83f6d0 feat(o11y): expose /v1/sentry edge mount + DSN-ingest gate exemption
Cloud edge for Hanzo Sentry (the /v1/sentry product face served by the embedded
o11y runtime):

- mountSentry registers the /v1/sentry/* wildcard, forwarding to the SAME gated
  runtime handler the /v1/o11y wildcard uses (one runtime, two path families; no
  path rewrite — the Sentry routes are literal /v1/sentry/... in the runtime).
- gate() now exempts the DSN-authenticated Sentry ingest writes (isSentryIngestPath:
  POST /v1/sentry/{project}/envelope|store/, tight method+prefix+suffix match) from
  the principal gate — the runtime authenticates the DSN key, not a Hanzo session —
  while EVERY Sentry read/write API stays principal-gated (no cross-tenant leak).

Uses only the existing o11y runtime-handler API, so it builds against the current
pinned o11y (v1.5.12) and is inert (404) until the o11y dep is bumped to a build
that carries the /v1/sentry routes.

FOLLOW-ONS (coordinated separately):
- Bump the hanzoai/o11y dep to the tag containing the /v1/sentry routes to activate.
- Gateway needs a byte-identical isErrorIngestPath sibling for POST
  /v1/sentry/{project}/envelope|store/ so the tokenless DSN ingest is not 403'd at
  the edge (do NOT touch ~/work/hanzo/gateway here).

Test: TestGateExemptsSentryIngestButGatesReads (ingest exempt, reads/writes gated).
2026-07-11 11:58:40 -07:00
hanzo-dev 5cb73ab8b1 fix(social): honest at-rest comment — token NOT yet encrypted (RED HIGH)
The Account.Token comment claimed the store is SQLCipher-encrypted and 'keeps
it encrypted at rest'. False: openStore uses cek.Open, which today runs the
no-key plaintext fallback (real WithRawKey-from-KMS lands with the connect
flow). Token column is empty today, so no plaintext secret ships.
2026-07-11 11:36:16 -07:00
hanzo-dev 7edc983318 merge(social): publish edge + scheduler parity (fail-closed provider seam) 2026-07-11 11:35:11 -07:00
hanzo-dev 88b0b29ba7 refactor(admin): complete first-principles upstream clients — money.Cents + digitalocean
Finishes the batch-F rework (the extraction landed in 7e93408; this is the second,
first-principles half that the earlier cherry-pick stopped short of):

- clients/admin/money — type Cents int64; the unit lives in the type, so
  ConsumedCents/MRRCents/CreditsCents collapse to Consumed/MRR/Credits. int64 casts
  only at the operator-contract boundary (wire unchanged, JSON tags kept).
- clients/admin/digitalocean — do.go extracted from the inline client; Client.{Ready,
  Balance,History}. Named digitalocean (not do) so it never collides with the local do var.
- clients/admin/commerce — reworked to the money.Cents unit + collapsed the always-equal
  (org,user) into one subject; dropped the MRRCents + duplicate-rollup shims.

Applied cleanly (0 conflicts) atop the extraction + tenant->org main. Build ./... green,
vet clean, admin+commerce+digitalocean tests pass (pure-Go cek gate).
2026-07-11 11:17:10 -07:00
hanzo-dev 7e93408074 refactor(admin): first-principles upstream clients — money.Cents + commerce/health/digitalocean/health packages
Extracts admin's inline upstream reader clients into self-contained, package-namespaced
units and gives billing a single value type. Decomplect + package-as-namespace, per the
Hickey/DRY bar:

- clients/admin/money — type Cents int64; the unit lives in the type (ConsumedCents/
  MRRCents/… collapse to Consumed/MRR). int64 casts only at the operator-contract edge.
- clients/admin/commerce — Client.{Ready,Spend,Credits,Plan,Ledger,Costs,Deposit}
  (Deposit = the one write). Collapses the always-equal (org,user) into one subject;
  the bare-slug X-Org-Id+user invariant is baked into the client. Drops MRRCents +
  duplicate rollup-balance shims.
- clients/admin/digitalocean — Client.{Ready,Balance,History}
- clients/admin/health — Client.{Ready,Up}

Wire unchanged (JSON tags kept, DTOs stay int64 cents). Integrated onto current main:
preserves the commerceinproc.BaseURL(...) in-process routing and the tenant->org
vocabulary. Build ./... green, vet clean, admin+commerce+integrations tests pass
(pure-Go, cek gate). Completes the batch-F rework the user authorized.
2026-07-11 11:12:08 -07:00
hanzo-dev af48c53c62 feat(social): publish edge + in-process scheduler (fail-closed provider seam)
Fold the live social stack's publish + schedule onto the native /v1/social domain:

- Publish edge (publish.go): the ONE publishPost path — claim (at-most-once across
  the HTTP handler and a scheduler tick), fan out to the channel's connected accounts
  through the injectable Publisher seam, record the honest outcome (published + external
  id, or failed + reason) on the post. POST /v1/social/posts/:id/publish + on-create
  fanout (scheduled-for-now-or-earlier) both call it.
- Scheduler (scheduler.go): an in-process periodic sweep (the native equivalent of the
  live stack's Temporal timer + hourly missing-post poller) that advances every org's
  scheduled -> published when the time arrives; idempotent, per-org, clean shutdown.
  Mirrors clients/commerce/sweep.go.
- Provider seam: fail-closed default (notConfiguredPublisher) that reports EXACTLY which
  OAuth-app credentials are missing (the live orchestrator's env var names) and NEVER
  fakes success. No Hanzo deployment carries provider creds today, so this is prod's
  honest state. GET /v1/social/providers reports per-network publish-readiness.
- Store: token on accounts (SQLCipher-encrypted at rest; live stack stores it plaintext),
  external_id/account_id/error on posts, ClaimForPublish/MarkPublished/MarkFailed,
  DueScheduled (the ONE deliberate cross-org system read), RecoverStuckPublishing.

Tenant isolation preserved: every publish is org-scoped; the scheduler's cross-org sweep
only reads (org,id) to dispatch into the org-scoped path. 12 tests (9 new) prove publish
success/not-configured/no-account, idempotency, per-org isolation, the scheduler sweep,
crash recovery, and live capabilities. go build -tags 'cloud cloud_mount' ./... green;
frozen wire-order guard green.
2026-07-11 11:08:49 -07:00
hanzo-dev d788864450 cek: complete plaintext-store coverage on live-prod + flatten to top-level package
Rebased blue's RED-reviewed encrypt-at-rest onto the CURRENT live-prod commit
(v1.786.185). The release train added stores blue's base never saw, all opening
PLAINTEXT — closed every one through the same cek.Open seam:
  - tenantdb.go   (the SOLE per-tenant opener → code/git/functions/tracker)
  - gojabase, gatewaypolicy (gateway.db), dataroom (link_index.db)
Result: ZERO plaintext sql.Open("sqlite") left in the cloud data plane
(commerce self-encrypts under its own KMS key; cek internals excepted).

Flattened internal/cek → top-level one-word package cek. Trimmed the AI-slop
import comments to one line.

Verified CGO=1+SQLCipher: go build ./... green, cek tests green, and blue's
cek.Open shipping path pre-proven on ALL 47 real prod DBs (ciphertext + plaintext
shredded + exact row-count parity, 0 fail).
2026-07-11 10:56:53 -07:00
blueandhanzo-dev 15533dc02d ci(cek): run frozen-format guard inside the Docker image build
The frozen-fixture test ran in Go CI but not inside the image under the pinned
Alpine libsqlcipher, so a sqlcipher-dev pin/base bump that changes the on-disk
format could go green in CI while bricking prod. Add the frozen-format gate to
the Docker RED gate (beside TestEncryptionProof), and make requireCipher honor
SQLITE_REQUIRE_CODEC=1 (a would-be skip becomes a FAILURE) so the in-image gate
is airtight: any format change now fails the IMAGE build, not just Go CI.

Verified: SQLITE_REQUIRE_CODEC=1 go test -run TestFrozenFixtureOpens ./internal/cek = ok.
2026-07-11 10:52:44 -07:00
blueandhanzo-dev 3b139ab55d fix(cek): frozen-format CI fixture + exact sqlcipher pin + confidentiality-only scope doc
RED re-review closers:
1. Cross-version brick guard (vector b): the version-freeze was soft (unpinned
   apk sqlcipher-dev resolves from the live Alpine repo). Now:
   - Dockerfile pins sqlcipher-dev=4.6.1-r0 → a repo bump fails the build LOUDLY
     (never a silent prod brick).
   - Commit a FROZEN encrypted fixture (internal/cek/testdata/frozen/store.db
     + .dek, written under cipher_compatibility 4) + TestFrozenFixtureOpens that
     copies it to temp, opens via cek, and reads a known canary row. A future
     libsqlcipher format change makes the FROZEN store fail to open → red CI,
     which build-time SQLITE_REQUIRE_CODEC (fresh-db only) cannot catch.
     TestGenerateFrozenFixture (CEK_GEN_FIXTURE=1) regenerates it on an
     intentional format rev.
2. Scope doc (vector c): cek package doc now states it provides CONFIDENTIALITY
   at rest, NOT integrity/authenticity/anti-rollback vs a PV-write
   (node-compromise) adversary — out of the stated read-only model. The
   logical-id+epoch binding is deliberately NOT added (out-of-model complexity).

9/9 cek tests green on real SQLCipher (cgo) + nocgo; gofmt/vet clean. Migration
still deferred; live cutover gated on the user's supervised go.
2026-07-11 10:52:44 -07:00
blueandhanzo-dev b6a85738ed fix(cek): address RED review — shred plaintext, fileID KEK, fatal-missing-key, content-parity
RED review fixes on the encrypt-at-rest codec:
1. [HIGH] Shred the pre-migration <db>.plain.bak after the verified keyed reopen
   (overwrite+remove, single call site in openEncrypted). Nothing reaped it
   before, so every migrated store left a COMPLETE plaintext replica on the
   volume forever — the exact threat the codec exists to kill. Test asserts the
   backup is gone post-migration.
2. [MED] KEK no longer binds to the CLOUD_DATA_DIR-relative path (brittle: a
   dir change bricked the plane). A random per-file id is stored in the sidecar
   head (fileID(16) || wrapped-DEK) and the KEK derives from it — intrinsic to
   the file, config-independent. Test moves a store to a new dir + wrong
   CLOUD_DATA_DIR and it still opens.
3. [MED] Missing key on an encryption-capable build is now FATAL (fail-closed,
   same posture as KMS), not a silent plaintext downgrade. Encrypting() is wired
   into the boot log (serve.go). No new gate/env — keyed off the existing
   CLOUD_KMS_MASTER_KEY_REF + build capability.
4. [MED] Parity gate gains a rowid-independent per-table content hash (commutative
   multiset sum of per-row hashes over user columns), catching value mutations
   that count+schema+integrity_check miss while ignoring benign rowid renumbering.
   Dropped the inaccurate 'byte-faithful' wording. Cross-version cipher stability
   is enforced by FREEZING libsqlcipher in the build (an at-open cipher_compat pin
   is infeasible with mattn's URI-key requirement — the URI param is ignored and a
   post-open pragma runs after mattn reads the header; proven); any mismatch fails
   closed (test: corrupt/missing sidecar refuses).
5. [LOW] recoverInterrupted scrubs stale -wal/-shm; migration removes them before
   the swap so a plaintext WAL never sits beside an encrypted db. No statement
   logging on keyed conns; the key never rides a logged DSN/error.

8/8 tests green on real SQLCipher 3.53.1 (cgo) + nocgo fatal-key path.
2026-07-11 10:52:44 -07:00
blueandhanzo-dev 999b86a2a4 feat(cek): encrypt-at-rest for all cloud SQLite stores (plaintext→SQLCipher, zero-loss migrate-on-open)
The CGO+libsqlcipher cloud binary shipped encryption DORMANT: all 29 stores
opened via bare sql.Open("sqlite", path), so /var/lib/cloud/*.db (crm PII,
treasury ledger, wallets, audit, team/entitlements) were plaintext despite
CLOUD_KMS_MASTER_KEY_REF being present. The SQLCipher primitives were linked
(hanzoai/sqlite cek.go) but never USED.

internal/cek.Open is the ONE encryption-at-rest seam every store now routes
through. When the master key is configured it transparently, under a per-file
flock: mints a per-DB random DEK (SQLCipher page key), wraps it AES-256-GCM
under KEK=HKDF-SHA256(master, principal) in a <db>.dek sidecar, and — for an
existing PLAINTEXT file — migrates it to SQLCipher via sqlcipher_export, then
verifies schema + per-table row-count parity + integrity_check by re-opening
the encrypted copy EXACTLY as the app will, BEFORE an atomic swap. Fail-secure:
key set on a non-encrypting build is a hard error; unverifiable migration leaves
plaintext intact; wrong master fails closed; crash mid-swap is recovered.

Proven with real SQLCipher (TDD): plaintext→ciphertext header, row parity,
.dek unwrap, .plain.bak preserved, idempotent reopen, wrong-key rejected,
dev-mode gated, nocgo fail-closed.

Migration is deferred to a maintenance-window cutover (this image built via
arcd) — NEVER encrypt against the current binary, which cannot open SQLCipher.

Rewires 29 open-sites; base/o11y external-module DBs are a follow-on.
2026-07-11 10:51:13 -07:00
8dfbbc314d cloud: migrate off deprecated app.Mount → All+AdaptNetHTTP (one mount path); tidy go.sum (#257)
zip #5 deprecated (*App).Mount(prefix, h): it is exactly
  app.All(prefix+"/*", zip.AdaptNetHTTP(h))
kept only as a behaviour-identical alias. Move every foreign-http.Handler
mount onto the explicit primitive so there is ONE way to put a route on the
app, and so the cloud money binary keeps building once zip deletes App.Mount.

Migrated all THREE route-mount call sites (the task scoped two; commerce is a
third — its exclusion note referred to the commerce.Mount *function*, not the
app.Mount(p, handler) inside it):
  - clients/plugin/plugin.go:125   app.Mount(p.Prefix, h) → app.All(p.Prefix+"/*", zip.AdaptNetHTTP(h))
  - clients/iam/iam.go:159         app.Mount(p, handler)  → app.All(p+"/*", zip.AdaptNetHTTP(handler))
  - clients/commerce/mount.go:147  app.Mount(p, handler)  → app.All(p+"/*", zip.AdaptNetHTTP(handler))
Behaviour-identical (Mount IS this composition). Also updated the four doc
comments that named the deprecated zip.App.Mount so no dangling reference to a
soon-deleted method remains. grep-confirmed ZERO app.Mount( route-mounts left.

go.sum: removed the two inert zap-proto/zip v1.3.0 hash lines (nothing in the
module graph requires v1.3.0 — orphan cruft a working `go mod tidy` would drop).
Full `go mod tidy` is blocked by a PRE-EXISTING, unrelated force-moved tag on
luxfi/keys@v1.2.2 (server serves h1:nuD+y5…; committed go.sum records
h1:XH5mRm…), so the two dead lines were removed surgically instead — luxfi/keys
and every other entry left byte-identical to origin/main. No GONOSUMCHECK hack.

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-11 10:50:09 -07:00
hanzo-dev dbcb55eb0e fix(commerce): derive org from IAM on the token path — no commerce-owned org
Per the one-authority directive: commerce must not require its OWN Organization
record; IAM is the sole org/user/auth authority. The service-token path already
auto-projects the org from X-Org-Id via the cached GetOrCreate resolver, but the
IAM-principal path c.Next()'d without resolving the org — it depended on
iammiddleware running upstream, and when it hadn't, GetOrganization MustGet-
panicked (500) / the org was absent, so an IAM org with no pre-existing commerce
row could not view or create billing. ensureIAMOrg now resolves the validated
X-Org-Id through the SAME cached GetOrCreate resolver on the IAM path too
(idempotent), so any IAM org "just works" and a thin billing record is
auto-projected on first use — commerce derives the org from IAM, never its own
table. This is also the root cause behind the "hanzo has no commerce org record"
wall on the live 2-org proof.
2026-07-11 10:45:33 -07:00
hanzo-dev 758af74d8d fix(billing): account handlers never panic on an unresolvable org
The billing-account CRUD (List/Create/Get/Update/Delete + project bindings +
members + loadOwnedAccount) called middleware.GetOrganization, which MustGet-
panics (→ recovered 500) when the request's org has no commerce Organization
record — the same class as the already-deployed spend-alert fix. Switch all to
GetOrganizationOK with a safe default (reads → empty, mutations → 400/404), so
an IAM principal whose org lacks a commerce record gets a clean response, not a
500. Completes the metering-path panic hardening.
2026-07-11 10:33:24 -07:00
472e7f2067 cloud: zip v1.5.0 + OnShutdown teardown — fix teardown-before-drain race (#256)
Bump github.com/zap-proto/zip v1.3.0 → v1.5.0 (verified drop-in) and move
subsystem teardown onto zip's OnShutdown hook, deleting the hand-rolled
ShutdownAll reverse-loop.

Before: serve.go called ShutdownAll(specs) — a reverse-mount teardown loop —
BEFORE app.ShutdownWithContext. Subsystems were torn down while the listener
was still accepting and in-flight requests were still draining: a latent race
(a request could use a store that teardown had just closed).

After: MountAll registers each enabled spec's ShutdownFunc via app.OnShutdown
right after the subsystem mounts. zip drains those hooks LIFO — AFTER the
listeners stop accepting and in-flight requests drain — so registration-at-mount
reproduces the exact reverse-mount teardown order ShutdownAll gave, minus the
race. app.ShutdownWithContext now owns the whole teardown.

Scope is deliberately narrow: MountSpec, MountAll, Wire(), Typed, and the
OwnsHealth health loop all STAY (the imperative composition-root flatten was
proven unsound — a generic /v1/:name/health route is shadowed by 3-segment
subsystem routes). Only ShutdownAll and its serve.go call site are deleted;
audit/gateway-policy/telemetry teardown keep their positions.

Test: build_onshutdown_test.go drives a real in-flight request over a loopback
listener, shuts down mid-request, and proves (1) Shutdown blocks until the
request drains and (2) the MountAll-registered hooks then run LIFO = reverse
mount order. A second test locks the enablement axis + nil-Shutdown guard.

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-11 05:35:45 -07:00
hanzo-dev 9eeaa47588 fix(o11y): exempt DSN-authed error-ingest from the principal gate
The Sentry error-ingest wire endpoints (POST /v1/o11y/api/<project>/envelope|store/)
authenticate with a DSN public key downstream in the o11y handler, not a Hanzo
principal. cloud's o11y gate() 403'd them for lacking X-User-Id, so the tokenless
ingest that the gateway allowlists (isErrorIngestPath) was blocked one layer deeper
— error-tracking could never ingest end-to-end.

Add the cloud-side counterpart: gate() now also exempts isErrorIngestPath
(byte-for-byte the gateway's matcher — method + /v1/o11y/api/ prefix + envelope|store
suffix, never a bare prefix). Reads under /v1/o11y/api/vN/... and the Issues
list/detail/update stay principal-gated; the exempted ingest still fails closed on a
bad/absent DSN key (401/503). Tested: TestGateExemptsErrorIngestButGatesReads.
2026-07-10 23:58:25 -07:00
ba9f632108 fix(commerce): sessionless webhook org resolve must not MustGet-panic (#254)
The live signed-webhook e2e (#118) got past HMAC verification and then
500'd: resolveWebhookOrg called middleware.GetOrganization — gin MustGet
— but webhook ingress runs OUTSIDE the auth-token group, so no
middleware ever set "organization" and every signature-VALID provider
delivery panicked ("key organization does not exist"). Switch to the
GetOrganizationOK variant that exists precisely for signature-verified
sessionless ingress; the header/env/default fallback chain below it now
actually runs.

Regression test drives resolveWebhookOrg on a bare sessionless gin
context and asserts no panic.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 23:39:24 -07:00
eed7221488 fix(commerce): seed the Square registry provider from env — unbreak inbound webhook validation (#253)
Every inbound Square webhook 401'd 'square processor not configured':
payment/providers/square registers an EMPTY Provider whose Configure()
only runs from BD's per-tenant charge resolver — but tryValidateWebhook
(the sessionless inbound path) reaches the registry slot directly. The
env-registering thirdparty/square init could never win the slot either
(import-order clobber / defer-to-existing), so webhook validation had no
configured processor in ANY deployment — 100% of live Square deliveries
were rejected.

init() now seeds Configure() from the deployment env (same vars +
SQUARE_ENVIRONMENT sandbox switch thirdparty reads; charge creds required,
webhook fields optional with ValidateWebhook's own fail-closed contract).
BD's per-tenant Configure still overrides per request for outbound money.

Regression guard drives env -> configFromEnv -> Configure ->
ValidateWebhook over a real Square-spec HMAC (url+body), plus tamper and
sandbox-switch cases.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 22:56:08 -07:00
5e8fd1e932 feat(commerce): in-process auto-recharge sweep — retire the external CronJob (#252)
Standalone commerce was swept by a k8s CronJob (curl image + cluster hop +
token secret) POSTing /v1/billing/auto-recharge/run-all every 15m at
commerce.hanzo.svc:8001. With commerce folded into the cloud binary that
loop is redundant moving parts: the binary now dispatches the SAME request
loopback through its own embedded gin handler — full middleware chain
(TokenRequired service-token branch -> PlatformOnly mint gate -> datastore/
KMS context), byte-identical to the wire path (guarded by
TestSweepOnceWireContract).

- interval: COMMERCE_AUTORECHARGE_INTERVAL (default 15m; 0/off disables;
  garbage disables fail-safe — a card-charging loop must never guess)
- token: COMMERCE_SERVICE_TOKEN (absent -> disabled with a loud warn,
  not a 403-every-tick loop)
- first fire after one FULL interval (never front-run the outgoing CronJob
  during rollout)
- runs only where commerce mounts (single-writer pod; reader role never
  mounts commerce) -> exactly one sweeper, same guarantee the single
  CronJob schedule gave
- stops with Embedded.Stop

Unblocks deleting the commerce-auto-recharge CronJob + the standalone
commerce/commerce-sandbox pods (#118).


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 22:33:02 -07:00
hanzo-dev 5cda438a8c merge(social): fold native /v1/social domain (ship-dormant; no ingress cutover)
# Conflicts:
#	subsystems/subsystems.go
#	subsystems/wire_test.go
2026-07-10 21:46:12 -07:00
hanzo-dev 2928ccfdab merge(ads): fold native /v1/ads domain
# Conflicts:
#	subsystems/subsystems.go
#	subsystems/wire_test.go
2026-07-10 21:45:26 -07:00
hanzo-dev 0dec0cff70 merge(marketing): fold native /v1/marketing domain 2026-07-10 21:44:39 -07:00
hanzo-dev 2d924110b4 feat(content): real Generator + Distributor over the framework CMS, hardened
Turns the agentic-marketing content loop from scaffolding into a working
edge pair, then closes every finding from the adversarial review.

Edges (wired at Mount, fail-closed until configured):
- Generator: zen5 copy via deps.AI.ChatCompletion (metered Bill.Gate/MeterUsage)
  + studio assets via the ComfyUI Qwen-Image-Edit-2511 graph.
- Distributor: hanzoai/social Public API fan-out, per-brand key custodied in KMS.
  errNotConfigured until a brand connects a key — no key, no fan-out.

Hardening:
- Publish TOCTOU interlock: a per-item, store-backed lease (framework fw_locks,
  the ONE cross-process coordination primitive) serializes concurrent publishes
  of the same item across drivers and pods; non-idempotent fan-out runs once.
- source_media SSRF gate: the URL validator fails closed on unparseable/backslash
  input.
- external_ids is server-managed: the before_save guard rejects a client write;
  Publish records it under the lease as a trusted server write.

Adversarial regression suite pins each guarantee from the outside (raw framework
PUT, direct ops, concurrency): red_adversarial / red_rereview / red_final /
red_lease_final, plus blue_hardening and per-file unit tests.

One open item recorded as a HARD GATE in clients/content/LLM.md: lease
TTL-preemption re-opens the double-post window for a brand with ~15+ slow
channels (fan-out (N+1)x20s > 5m TTL, external_ids recorded only at fan-out end).
ZERO exposure until a brand connects a social key; the fix (lease heartbeat/renew
preferred) MUST land in the same change that connects the first real key.
2026-07-10 20:54:56 -07:00
hanzo-dev 57f0cde63d fix(social): add social to frozen wire-order guard (RED ship-blocker) 2026-07-10 19:13:09 -07:00
0f86372c3b hardening(consolidation): split-proof the framework-content guard + DataDir doc (RED INFO) (#251)
Two INFO items from RED's #250 re-review:
- TestFrameworkContentModulesLinked asserted only the module registry. erp's
  ledger-posting HOOKS register in a SEPARATE init() step, so a future split of
  registerHooks() out of erp's module init() could drop the hooks while the guard
  stayed green. Add framework.RegisteredHookCount() and assert it > 0 so the guard
  fails if erp's hooks (computeJournalTotals, journalEntry/paymentEntry
  submit+cancel, …) are ever unlinked from the binary.
- gojabase Config.DataDir comment said "{tenantSlug}.db" (the pre-C1 name); the
  on-disk segment is TenantSegment (injective, traversal-safe base32 of raw org
  bytes). Comment now matches the code.

go build + go test ./subsystems/... ./clients/framework/... green.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 19:11:18 -07:00
hanzo-dev 24961ee941 feat(ads): native /v1/ads domain in the cloud binary
Net-new per-org ad-campaign domain on the ONE cloud framework (zip/Fiber +
cloud.Deps + per-org SQLite), the same shape clients/crm uses — twin of crm.
Registered in subsystems.Wire() right after crm; frozen wire sequence updated.

/v1/ads surface (all org-scoped, tenant-isolated on the bearer owner claim):
  GET    /v1/ads/health             (auto, serve.go liveness)
  GET    /v1/ads/summary            per-org roll-up (total/active/budget/spend)
  GET    /v1/ads/campaigns          list (?status=)
  POST   /v1/ads/campaigns          create
  GET    /v1/ads/campaigns/:id      detail
  PUT    /v1/ads/campaigns/:id      update
  DELETE /v1/ads/campaigns/:id      delete

Campaign = root of the ad hierarchy (campaign -> ad sets -> ads; ad-set/ad legs
hang off this seam): Platform (meta/google/tiktok/x), Status (draft/active/
paused/completed), Objective, Budget/Spend (cents).

Tests: per-org isolation + full CRUD + summary round-trip, both green.
2026-07-10 18:50:50 -07:00
hanzo-dev 439de5c01f feat(social): fold the live social stack as native /v1/social domain
In-process fold of github.com/hanzoai/social (social-backend/frontend/
orchestrator, a Postiz-style scheduler) onto the ONE cloud framework
(zip/Fiber + per-org SQLite), twin of clients/crm + sibling of the
marketing fold. NOT a proxy to the standalone social pods.

Two entities faithful to the live Public API (clients/content publish.go
already talks to it): Account = a connected channel (the stack's
integration), Post = content published/scheduled to a channel. Scheduling
is a Post with status=scheduled + a future scheduleAt, not a third entity.

Every query filters WHERE org=? on principal.Org (validated bearer owner,
HIP-0026); one tenant can never read/mutate another's rows. Registered in
subsystems Wire() after crm with a ctxShutdown that closes the DB.

Surface (org-scoped, /v1 only): summary + accounts CRUD + posts CRUD.
Generic liveness serves GET /v1/social/health.

Build: go build -tags 'cloud cloud_mount' ./... GREEN; 3 store tests pass
(per-org isolation across accounts+posts, post CRUD+summary, account CRUD).
2026-07-10 18:44:03 -07:00
hanzo-dev df72126108 feat(marketing): fold hanzoai/marketing as native /v1/marketing domain
Mount github.com/hanzoai/marketing in-process on the ONE cloud framework
(zip/Fiber + cloud.Deps + per-org SQLite), the same shape clients/crm uses —
NOT a proxy to a standalone marketing pod. Registered in subsystems.Wire()
right after its twin crm; frozen wire sequence updated to match.

/v1/marketing surface (all org-scoped, tenant-isolated on the bearer owner claim):
  GET    /v1/marketing/health             (auto, serve.go liveness)
  GET    /v1/marketing/summary            per-org roll-up (total/active/budget/spend)
  GET    /v1/marketing/campaigns          list (?status=)
  POST   /v1/marketing/campaigns          create
  GET    /v1/marketing/campaigns/:id      detail
  PUT    /v1/marketing/campaigns/:id      update
  DELETE /v1/marketing/campaigns/:id      delete

Campaign faithful to the repo domain: Channel (email/sms/social/meta/google/
tiktok), Status (draft/active/paused/completed), Objective, Budget/Spend (cents).
Genetic-optimizer / ML-forecasting / ad-platform integrations not folded yet.

Tests: per-org isolation + full CRUD + summary round-trip, both green.
2026-07-10 18:01:08 -07:00
hanzo-dev 89aae92c6a Merge fix/genai-tracer-clobber-repin: pin ai to v1.805.3
Bumps github.com/hanzoai/ai v1.805.2 -> v1.805.3 (GenAI tracer pinned at adopt
time; gen_ai spans survive the embedded o11y/SigNoz global tracer-provider
reassignment and reach o11y_traces). go.mod/go.sum only, no cloud source change.
2026-07-10 17:58:22 -07:00
hanzo-dev ca9c4134fe deps: pin ai to v1.805.3 (GenAI tracer immune to o11y global-provider clobber)
Bumps github.com/hanzoai/ai v1.805.2 -> v1.805.3. v1.805.3 pins the GenAI
tracer at adopt time so the embedded o11y/SigNoz runtime reassigning the
process-global OTel tracer provider can no longer redirect gen_ai spans off
the o11y sink — they reach o11y_traces. No cloud source change; go.mod/go.sum only.
2026-07-10 17:57:43 -07:00
hanzo-dev 3bdf72a80f feat(content): native agentic-marketing content loop on the framework CMS
Add clients/content — the ONE Go-native replacement for the bespoke karma
Python pipeline. A framework app-lane (module "marketing": Campaign, SocialPost,
Asset) + a thin /v1/content/* control-plane. Store-less: content IS framework
documents; the subsystem is a stateless orchestrator over the framework store +
the zen5/studio/social edges.

- lifecycle.go: ONE state machine (draft→in_review→approved→queued→published,
  +archived), a pure value read by the before_save hook (enforces edge legality
  at the storage boundary) and the transition endpoint (same check + fan-out).
- hooks.go: before_save gate on every publishable DocType.
- content.go: board (cross-DocType aggregate), lifecycle, transition (+best-effort
  distribution), generate/publish/channels. Org-scoped via principal.Org; never a
  5xx from a foreseeable condition (fail-closed 503 / honest 4xx).
- generate.go/publish.go: Generator + Distributor seams with fail-closed defaults;
  real zen5 (deps.AI) + studio (ComfyUI) + hanzoai/social wiring documented for the
  follow-up. Exported Generate/Publish/Transition are the ONE impl the HTTP surface
  AND the automations connector call.
- framework: add Get (read-one), exported ErrNotFound/ErrConflict/ErrBadRef +
  IsValidationError so in-process callers classify errors without string-matching.
- automations: connector_content.go exposes content_generate/transition/publish as
  flow steps + MCP tools (in-process, org-scoped) so the loop runs autonomously
  (cron flow: generate → wait_for_approval → transition → publish).
- subsystems: Wire content after knowledge; frozen wire-order test updated.

Tests: lifecycle table, before_save hook, end-to-end loop over the real framework
store (install→create→board→transition→publish), forge-403, cross-org isolation.
go build ./... + go test (content/framework/automations/subsystems) all pass.
2026-07-10 16:48:54 -07:00
3f185baa6f fix(subsystems): restore cms/erp/help framework content modules (#248 regression) (#250)
#248 (Wire() composition root) rewrote subsystems.go and dropped the blank
imports for cms/erp/help. Those three are NOT mount subsystems — no HTTP surface,
never in Wire(). Each registers DocType fixtures and, for erp, ledger-posting
lifecycle hooks (computeJournalTotals, journalEntry submit/cancel, paymentEntry
submit/cancel, …) into the always-on clients/framework engine from a package
init() (framework.RegisterModule). Dropping the imports left them out of the
binary: /v1/framework/* carried no erp/cms/help and the erp ledger hooks were
silently gone. No mount test caught it (frozen[]/Wire() cover only mount specs).

Fix (RED-verified): re-add the three blank imports under an explicit "framework
content modules" comment; add framework.RegisteredModules() + a guard test
(TestFrameworkContentModulesLinked) asserting the engine carries each lane so the
drop cannot silently recur. Also relax TestDepGatedSubsystemsFailClosed to accept
any non-2xx — a dep-disabled subsystem denying 403 is fail-closed; >=500 was
wrongly strict (o11y denies 403, ai returns 5xx).

go list -deps ./cmd/cloud shows cms/erp/help; go build ./... green;
go test ./subsystems/... ./cmd/cloud/... -race green.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 16:24:14 -07:00
hanzo-dev decc5cf85c fix(billing): metering gate no longer panics on an unresolvable org
The S2S metering path (ScopeRules → ListSpendAlerts; AuthorizeVerdict →
AuthorizeSpendCap) can reach these handlers with no "organization" context key
when X-Org-Id does not resolve to a commerce org (e.g. the agents scheduler
probing an unprovisioned org). middleware.GetOrganization MustGet-panicked
there → a recovered 500 storm on the money path (observed live: "key
organization does not exist", ~every 5-15s). Switch the metering-verdict
handlers + billingSubject to GetOrganizationOK with a safe default — no org
means no org-scoped caps, so ListSpendAlerts returns empty and
AuthorizeSpendCap allows — instead of panicking. Regression test proves neither
handler panics with no org.
2026-07-10 16:16:19 -07:00
hanzo-dev 3e54403f68 feat(principal): carry Project + BillingAccount from the verified identity
types.Claims gains Project + BillingAccount; principal.BillingAccount(c) reads
the gateway-minted X-Billing-Account-Id, added to subScopeHeaders so the raw
client copy is stripped on ingress and re-injected only for a validated
principal (mirrors X-Project-Id). It is an ATTRIBUTION hint only — the debited
account is always resolved server-side by commerce from the org's
ProjectBinding, never trusted from the header — so a mislabelled account can
only ever misattribute the caller's OWN spend within its own org, never
redirect spend to another tenant. Inert until IAM emits the billing_account
claim + the gateway mints the header (slices 63/64).

Also fixes a stale test left by a88f6612 (/metrics is now a console page, not an
API 404 — dropped from the must-404 list).
2026-07-10 16:00:57 -07:00
6072681f22 cloud: explicit composition root — Wire() replaces the init-registry; order-ints deleted (#248)
Replace the init()-registry (blank imports + cloud.Register(name, order, …) +
magic order-ints) with ONE explicit subsystems.Wire() []cloud.MountSpec. Slice
position IS the mount order — MountAll iterates it as-given, ShutdownAll in reverse.

Core (build.go/serve.go/cmd): delete MountSpec.Order, Register,
RegisterWithShutdown, the HealthOwner option func, and var Registry. MountAll and
ShutdownAll take the []MountSpec slice; Serve threads it in (cloud never imports
subsystems → no cycle). cmd/cloud + cmd/hanzo call subsystems.Wire(). Typed and the
factory hooks (RegisterKMSClientFactory, RegisterCommerceClientFactory,
RegisterOrgScopeResolver, RegisterPushBuilder, RegisterTelemetryInstaller) are
untouched — a different mechanism.

In-repo (~64 clients/*): delete each init() registration; export the mount fn where
unexported (commerce.MountFromDeps, o11y.MountO11y) and the shutdowns. ctxShutdown
bridges the func() error shutdowns in one place.

Externals wired explicitly; the wave-2 tags no longer self-register:
  ai v1.805.2 (@150 catch-all), o11y v1.5.12 (@70 wildcard) — origin/main already
  pins these (genai-span) but did NOT wire them, so main currently DROPS ai + the
  o11y wildcard from its live registry (main's own TestRegistryAssemblesSubsystems
  fails on that). This composition root RESTORES both. authz v1.10.7 (@70),
  licensing v0.1.3 (@110, Mount is already a MountFunc — wired direct), metrics
  v1.110.2 (@40, takes its own metrics.Deps via mountMetrics). vfs unchanged: it
  never registered a subsystem, so it is not wired.

Order is proven by subsystems.TestWireOrderMatchesFrozen against the sequence
captured empirically from origin/main @9e8ac44 (68 self-registering specs) with ai
+ the o11y wildcard restored at their order-int slots. The Service[state] refactor
reshuffled same-order tie positions on main; the frozen sequence adopts main's new
order (functionally inert — tied subsystems own disjoint route prefixes).

Also fixes a pre-existing clean-main build break (#247): clients/sign/sign.go
referenced an undefined `log` — main does not compile without it, so the whole tree
(subsystems → cmd/cloud) failed to build. One-line fix (log → deps.Logger); flag
for the owner to factor out if preferred.

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 15:42:09 -07:00
bc4f082f00 fix(sign): use deps.Logger for the VFS-nil guard (semantic merge fix, #117) (#249)
#247's VFS-nil health-only guard was authored against the pre-refactor sign.go
(which had a local `log` var); it auto-merged cleanly onto the concurrently-landed
cloud.Service refactor (principal.Org / s.Log) but referenced an undefined `log`,
red-ing the build (clients/sign/sign.go: undefined: log). Point it at deps.Logger
(guaranteed non-nil by the guard above it). No behavior change.

go build ./... + go build -tags "libsqlite3 sqlite_fts5" ./cmd/cloud +
go test -race ./clients/{gojabase,captable,sign,dataroom}/... all GREEN.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 15:24:58 -07:00
9e8ac4454c fix(consolidation): gojabase blob seam + sign SEQUENTIAL/off-DB PDFs + cap-table dilution (#117 M5/M7/M8) (#247)
Second wave of RED's consolidation review — the mediums that needed new
hanzoai/captable + hanzoai/sign bundle versions (now merged) plus their cloud-side
wiring and tests.

M5 (captable bundle @119bf9c0) — capTable summed ALL option rows regardless of
  status, so EXERCISED/EXPIRED/CANCELLED grants inflated fullyDilutedShares and
  every ownership %. Fixed to count OUTSTANDING options only. New
  TestDilutiveOptionsExcludeTerminal proves terminal-state grants don't dilute.

M8 — ONE blob seam in gojabase (Config.Blob + a tenant-scoped globalThis.__blob =
  { put(key,b64), get(key) }), keys namespaced {Name}/{TenantSegment} so a bundle
  can only reach its own tenant's objects. sign (bundle @315a3fe6) now routes PDF
  bytes THROUGH it instead of inlining 32 MiB base64 in document_data.data: create
  stores the original blob key, seal writes a sealed key, view/download read via
  __blob.get. The sign leaf passes deps.VFS as the seam (health-only without it).
  document_data holds only the key now (type BLOB_KEY). Same VFS/S3 plane dataroom
  uses — one blob strategy, not two.

M7 (sign bundle @315a3fe6) — SEQUENTIAL signing order was declared but never
  enforced. assertTurn gates signField+signComplete so a later signer can't act
  until every earlier signer has SIGNED. New TestSequentialSigningOrder proves an
  out-of-turn signer is refused (403) and the turn opens once the earlier one signs.

sign tests now inject an in-memory VFS and assert PDF bytes land on the seam
(tenant-scoped key), NOT in the tenant SQLite. All four gate packages pass under
go test -race against the published module versions.

Gate: go build ./... + go build -tags "libsqlite3 sqlite_fts5" ./cmd/cloud +
go test -race ./clients/{gojabase,captable,sign,dataroom}/... all GREEN.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 15:19:42 -07:00
hanzo-dev efa47ed112 Merge feat/genai-span-host-provider: hoist gen_ai host tracer install into cloud.Serve; pin ai v1.805.2 + o11y v1.5.12 2026-07-10 15:10:08 -07:00
hanzo-dev eb74b1e216 telemetry: bootstrap the host tracer provider inside cloud.Serve — one site, every entrypoint
cloud installs the ONE global tracer provider wired to the o11y in-process trace
sink, but in in-process-sink mode it sets no OTLP/ZAP exporter endpoint. The embedded
hanzoai/ai module then found no endpoint and DISABLED its gen_ai span emit, so the
gen_ai plane in o11y_traces was dark (cloud's own request spans flowed; every LLM
call's gen_ai span was gated off). The provider install + adopt also lived ONLY in
cmd/cloud/main.go, leaving every 'hanzo <svc>' entrypoint (which shares cloud.Serve)
telemetry-dark.

Hoist the bootstrap into cloud.Serve — the ONE serve body cmd/cloud AND every
'hanzo <svc>' share — so both install the provider and adopt it into ai identically,
BEFORE MountAll mounts ai (ai's InitTelemetry reads the adopted-ready flag at mount).
cloud.Serve cannot import clients/o11y (clients/o11y imports cloud -> cycle), so the
concrete bootstrap (SetTracerProvider + aiobject.AdoptHostTracerProvider) lives in
clients/o11y and registers via cloud.RegisterTelemetryInstaller — the same cycle-free
inversion as RegisterKMSClientFactory / RegisterPushBuilder. Serve flushes the
provider on shutdown BEFORE ShutdownAll tears the sink down, so buffered spans drain.

Removes cmd/cloud/telemetry.go (moved to clients/o11y/traceprovider.go). Repins ai to
the branch carrying AdoptHostTracerProvider.

Red review (SHIP): MED-1 — this hoist (both entrypoints adopt identically). Tests:
cloud.installTelemetry dispatch/no-op (telemetry_test.go); clients/o11y
installTraceProvider adopt-latching + OTLP-env-retire + disabled-noop
(traceprovider_test.go).

ai: -> v1.805.1-0.20260710213829-b5eb789d3878 (feat/genai-span-host-provider @ b5eb789d)
2026-07-10 15:09:19 -07:00
hanzo-dev 18743b1155 Merge remote-tracking branch 'origin/main' into wt3/integrate
# Conflicts:
#	clients/affiliates/affiliates.go
#	clients/auditlog/auditlog.go
#	clients/authors/authors.go
#	clients/automations/automations.go
#	clients/automations/mcp.go
#	clients/billing/billing.go
#	clients/billing/finance.go
#	clients/bots/bots.go
#	clients/captable/captable.go
#	clients/dataroom/dataroom.go
#	clients/framework/permission.go
#	clients/functions/billing_test.go
#	clients/functions/functions.go
#	clients/functions/invoke.go
#	clients/functions/metrics.go
#	clients/gateway/gateway.go
#	clients/git/git.go
#	clients/git/mirror.go
#	clients/git/smart_http.go
#	clients/graph/graph.go
#	clients/integrations/integrations.go
#	clients/knowledge/connectors.go
#	clients/knowledge/subsystem.go
#	clients/projects/deploy.go
#	clients/projects/domains.go
#	clients/projects/fork.go
#	clients/projects/projects.go
#	clients/referrals/referrals.go
#	clients/security/security.go
#	clients/tracker/tracker.go
#	clients/treasury/treasury.go
#	clients/visor/fleet.go
#	clients/wallets/wallets.go
2026-07-10 15:03:50 -07:00
hanzo-dev 9cc58786f3 Merge remote-tracking branch 'origin/main' into wt3/integrate
# Conflicts:
#	clients/agents/agents_test.go
#	clients/agents/model_validation_test.go
#	clients/agents/scheduler_test.go
#	clients/captable/captable.go
#	clients/crm/applications.go
#	clients/crm/applications_test.go
#	clients/dataroom/dataroom.go
2026-07-10 14:48:11 -07:00
1791491878 cloud: IAM-native vocabulary — org/user/project everywhere, tenant concept removed (TenantDB→OrgDB) (#246)
* cloud: IAM-native vocabulary — org/user/project everywhere, tenant concept removed (TenantDB→OrgDB)

"tenant" was a non-IAM synonym for org. Replace it with the IAM-native nouns
org / user / project / billing account so there is ONE name for the concept.

Root framework primitives (package cloud) — now 100% tenant-free:
  TenantDB                     -> OrgDB            (tenantdb.go -> orgdb.go)
  TenantStore[T]               -> OrgStore[T]
  NewTenantStore               -> NewOrgStore
  tenantDBPath / openTenantDB  -> orgDBPath / openOrgDB
  TenantScopeResolver          -> OrgScopeResolver (tenant_scope.go -> org_scope.go)
  RegisterTenantScopeResolver  -> RegisterOrgScopeResolver
Dropped the legacy X-Tenant-Id / X-Tenant-ID entries from the identity
strip-list (nothing reads them; X-Org-Id is the live header).

Cross-subsystem seams:
  principal.Tenant(c) -> principal.Org(c)  (clients/principal; no collision —
    Project/User kept, already IAM-native). ~50 call sites + ~70 stale doc refs.
  types.TenantConfig -> types.OrgConfig; CommerceClient.GetTenantConfig ->
    GetOrgConfig  (+ commerce client impl, disabled/rpc/entitlements consumers,
    the cloud.OrgConfig alias in deps.go).

Adopters deep-cleaned (identifiers, comments, test names): clients/code, git,
functions, tracker, provisioning, projects. Docs: subsystems.go composition
comments, README.md, and a new LLM.md "Framework doctrine" section.

GATED EXCEPTION (flagged, intentionally unchanged): clients/platform derives
LIVE k8s namespaces, registry image refs, and quota/limit objects from a
"tenant-<org>" string prefix. Renaming it orphans deployed namespaces + built
images, so the literal string is retained behind a // NAMING(gated) note in
clients/platform/k8s.go. The rbac test file + its funcs were renamed
(tenant_rbac_test.go -> org_rbac_test.go).

Out of scope (separate waves; independent domains, touched only for the
principal.Org call-site + stale-ref fix): commerce internal tenant tables
(active commerce-dissolve branch), runner tenantsource, treasury ledger, and
other subsystems' own tenant vocab.

Build: GOWORK=off CGO_ENABLED=0 go build ./... -> exit 0.
Tests: every touched package green. The 38 pre-existing failures (fakeAI
missing Embed; undefined Producer; commerce integration tests; o11y/graph/zt/
kms route-precedence & service tests) fail identically on the base commit.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>

* rebase: apply org vocabulary to post-branch main commits (metering/HA/tracing comments); drop unused TenantConfig alias

---------

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 14:45:00 -07:00
hanzo-dev e5cc2bcc54 merge: integrate cloud.Service[state] refactor onto origin/main (zip v1.3.0, o11y, renames) 2026-07-10 14:39:39 -07:00
zeekayandhanzo-dev d739b9d063 style(config): gofmt align HA reader/writer fields in LoadConfig
Follow-up to the reader-tier commit — align the CLOUD_WRITER_URL/
CLOUD_READER_RETRY_BUDGET/CLOUD_WRITER_LEASE struct-literal keys with gofmt so
the CI fmt gate is clean. No behavior change.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 14:37:32 -07:00
1351eb5354 fix(consolidation): tenant-injective per-tenant SQLite + un-stage folds + harden gojabase (#117) (#245)
RED adversarial review of the one-binary consolidation (gojabase + captable/sign/
dataroom folds). Fixes the two ship-blockers plus the in-repo mediums/slop.

C1 (HIGH) — cross-tenant collapse via non-injective filename encoding.
  gojabase.slugify ToLower+folded [^a-z0-9._-]→'_', collapsing DISTINCT owners
  (Acme/acme, "a b"/"a_b") onto ONE per-tenant SQLite file — a cross-tenant break,
  the exact fold principal.Tenant keeps the org verbatim to avoid. Replaced with
  gojabase.TenantSegment: lowercased unpadded base32 of the RAW org bytes —
  INJECTIVE (distinct orgs never share a file) and traversal-safe ([a-z2-7], never
  "."/".."/"/"). Table test proves injectivity (Acme≠acme≠a b≠a_b) + traversal
  containment. IAM owner charset permits case/separator variants, so this was a
  real (not theoretical) collapse. Fresh-tenant-only: the folds pre-date any
  release, so no on-disk data uses the old names — nothing to migrate.

H2 (HIGH) — code==docs on staging. captable/sign/dataroom were un-staged in
  config.go (stagedSubsystems={iam,ingress}) yet every leaf/README claimed
  "STAGED / standalone keeps authority". Evidence (do-sfo3-hanzo-k8s): NO
  standalone captable or dataroom deployment exists; the esign pod runs but its
  SQLite holds ZERO tenant data (User=0, DocumentData=0, Recipient=0, Field=0 —
  only BackgroundJob/RateLimit churn). Decision (a): keep un-staged, delete the
  false comments (captable.go, sign.go, dataroom.go, subsystems.go, README ×2).
  No migration needed (dead/empty apps).

M3  drop dead commercesvc api.Route(/v1/{billing,checkout,store,subscription,
    account}) — unreachable in embed mode (only /v1/commerce/* mounted, no
    rewrite); money path is clients/billing, unaffected. Removed unused import.
M4  gojabase per-tenant *sql.DB cache is now an LRU (cap CLOUD_GOJABASE_MAX_DBS=256)
    with idle eviction (CLOUD_GOJABASE_IDLE_TTL_SEC=300) and dispatch PINNING —
    only idle handles evict, never a DB mid-transaction. Bounds fd/memory for N
    tenants.
M6  dataroom deny_list is now ENFORCED in view.authenticate and WINS over allow
    (checked first); surfaced on linkOut. One shared email matcher (DRY).
M9  sign refuses to boot with a self-signed cert in a production env — requires the
    KMS-custodied CLOUD_SIGN_CERT_PEM/KEY_PEM; dev still self-signs.
M10 added a -race concurrent multi-tenant dispatch test (heavy eviction churn +
    case-variant tenants) proving pool + cache + eviction are race-free and
    isolation holds under concurrency.

Slop/decomplect:
  - goja: a response with no explicit valid status now FAILS CLOSED (default 500,
    rolls the transaction back) instead of committing at a defaulted 200.
  - newID (gojabase) + randKey (dataroom): a crypto/rand failure now returns an
    error / fails the dispatch instead of a predictable time/zero fallback.
  - subsystems.go: removed 6 duplicate blank imports (pubsub, eval, exec, plan,
    plugin, pricing).
  - dataroom: unified on ONE tenant encoding — the object-store key prefix now uses
    gojabase.TenantSegment, matching the SQLite filename (was raw org).

Gate: go build ./... + go build -tags "libsqlite3 sqlite_fts5" ./cmd/cloud +
go test -race ./clients/{gojabase,captable,sign,dataroom}/... all GREEN.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 14:35:24 -07:00
hanzo-dev 47e141c3f3 feat(ai): meter EVERY inference at the one chokepoint — no exempt path
The metering decorator (metered_ai.go) wraps deps.AI so every chat + embed
call is authorized against the caller's org balance/budget/freeze BEFORE the
call and debited its billing account AFTER — the single DRY chokepoint through
which no inference runs unattributed. The billing scope (org, project) rides
the request value (ChatRequest/EmbedRequest), so it is compile-time impossible
to call AI without declaring who pays; no bypass, no side-channel key.

Closes the exempt holes — clients/code (index+search+ask), clients/knowledge
(search + index embeds), clients/crm (application screen) — all previously ran
the balance-exempt M2M path unmetered; each now threads org/project to the
chokepoint.

- types.AIClient.Embed takes *EmbedRequest (scope); ChatRequest carries
  Org/Project; ChatResponse surfaces token counts for exact metering.
- httpAI stays pure transport but surfaces resp.Usage tokens and stamps
  hanzo.org/hanzo.project on its gen_ai spans (feeds per-tenant o11y isolation).
- token-based micro-USD debit (CLOUD_AI_PRICE_UUSD_PER_1K, default $2/1M tok);
  metering.Usage gains AmountMicros so a sub-cent call meters exactly instead
  of rounding to zero and slipping through unbilled.
- reuses ResourceMeter over Deps.Metering — same per-org invariants; a
  transparent pass-through when commerce is unconfigured (dev never blocked).
- fixes a latent slice-1 test break (crm/agents AIClient fakes lacked Embed).
- tests: pricing, scope forwarding, system-call, ModelLister preservation.
2026-07-10 14:32:27 -07:00
zeekayandhanzo-dev a88f661206 fix(webui): /metrics is a console page, not an API 404 — drop it from apiPrefixes
The console catch-all 404'd /metrics because it was in apiPrefixes (the Prometheus
scrape-path convention). But the real Prometheus surface is on the SEPARATE ops
listener (:9090, healthMux); on the product API (:8000) /metrics is the console
MetricsModule page. Drop /metrics from apiPrefixes so it falls through to the SPA
shell. One surface per port: :8000 product+SPA, :9090 ops. (ServiceMonitor repointed
to the ops port in the same change.)

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 14:30:53 -07:00
zeekayandhanzo-dev 74bb668a1f feat(ha): reader-proxy edge + writer fcntl lease for zero-downtime cloud rolls
The cloud writer embeds an exclusive-lock ZapDB KMS store. A new probe
(clients/kms.TestConcurrentOpen_LiveWriterStoreIsNotROShareable) proves that
opening that store READ-ONLY while the writer is live FAILS ("Log truncate
required to run DB") — Badger's RO open replays the live memtable WAL and
refuses to truncate it. So the prior groundwork's assumption that a reader can
open the KMS store RO off the writer's PVC is false for a LIVE writer (only the
sequential close-then-reopen case worked). The audit SQLite store IS
concurrently shareable (audit/shareability_probe_test.go); the KMS store is the
one that is not, and every mutation is audited on the writer anyway.

Reader tier is therefore a transparent, always-ready reverse proxy (opens no
stores) to the single writer:
 - reader_proxy.go: CLOUD_ROLE=reader boots serveReaderProxy BEFORE BuildDeps —
   forwards every request to CLOUD_WRITER_URL, streams SSE, preserves inbound
   Host. Dial-only retry (retryTransport) absorbs the writer's roll gap: it
   retries ONLY when the connection was never established (no ready endpoint /
   refused), so a non-idempotent POST is never double-executed; bounded by
   CLOUD_READER_RETRY_BUDGET (default 25s) then 502.
 - The reader Deployment rolls RollingUpdate(maxUnavailable:0), so the edge
   Service always has a ready endpoint — this removes the ~30s console blip that
   the writer's Recreate/replicas:1 causes today.

Writer zero-gap roll (opt-in, default OFF = byte-identical Recreate):
 - writer_lease.go (+_unix/_other): CLOUD_WRITER_LEASE takes an exclusive fcntl
   flock on {DataDir}/.writer.lock BEFORE opening the RWO stores and releases it
   LAST at shutdown (after every store closes). A surge writer blocks until the
   old one releases, so the exclusive ZapDB/audit stores are handed off, never
   double-opened. Fail-closed on timeout.

Removes the dead ReaderGuard (the reader no longer runs the full pipeline; it is
the proxy). Unset CLOUD_ROLE + unset CLOUD_WRITER_LEASE ⇒ writer, byte-identical
to today.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 14:12:46 -07:00
hanzo-dev 57959fc491 Merge branch 'wt3/F' into wt3/integrate 2026-07-10 14:11:39 -07:00
hanzo-dev 37c44f6bcc feat(billing): GCP-style billing accounts — funding separate from org
BillingAccount is a funding entity separate from the org: it holds a spend
limit + level + freeze and funds 1..N of the org's projects (dedicated or
shared). A project resolves to its account via ProjectBinding; usage debits
carry Transaction.AccountId so an account's balance and calendar-month spend
sum from the SAME append-only ledger the balance gate + per-scope caps read —
no parallel ledger, no stored total, never drifts.

The authorize verdict layers an account cap + freeze on top of the per-scope
caps (most-restrictive-wins, fail-closed). Resolution is ALWAYS server-side
from the org namespace — the account is never a forgeable client header.
Backward-compatible: (org,"default") folds to the org-wide pool, so an org
with no account behaves byte-for-byte as before.

- models/billingaccount, models/projectbinding (+ hashid kinds 284/285,
  query reserved-kind list)
- Transaction.AccountId indexed ledger axis
- resolveAccountId + accountSpentCents + account cap/freeze in AuthorizeSpendCap
- real account + project-binding CRUD replacing the org-wrapper stubs
- tests: account cap, freeze kill-switch, dedicated-binding isolation,
  backward-compat pool fold
2026-07-10 14:07:31 -07:00
69c3ffed0d cloud: zip v1.3.0 — zap-proto/fiber engine, route precedence is now a framework guarantee (#244)
zip v1.2.1 -> v1.3.0 rides zap-proto/fiber v3.2.1 (gofiber v3.2.0 + native
ServeMux-1.22 specificity precedence: most-specific wins regardless of
registration order, ambiguous overlaps panic at registration, use/mount stay
declaration-ordered barriers). TestIAMKeysBeatsWildcard now passes as a
FRAMEWORK property — subsystem order-ints are no longer load-bearing for
route precedence (their init-ordering role remains; removal rides the
composition-root refactor). Also swaps the 6 direct gofiber test-file imports
to the fork (production code was already 100% behind zip).

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 13:44:13 -07:00
hanzo-dev 0e34afc12f feat(ai): embeddings via the one AIClient — org/project-aligned, no static key
Both semantic subsystems (clients/code, clients/knowledge) embedded via their own
static CLOUD_AI_API_KEY HTTP client — a side-channel credential separate from the
chat path, and the root of vectors:0 (the key was dead + entitlement-gated). Add
Embed() to types.AIClient and route both embedders through deps.AI — the SAME
client chat runs through, which authenticates with the IAM client-credentials
(M2M) token when no static key is set: ONE org/project-aligned identity, one way,
no static key to rotate. Emits gen_ai OTel spans so embeddings are observable end
to end like chat.

This is the ai-module leg of the canonical path
ingress -> gateway -> commerce -> ai module; metering + no-exempt + billing-on land
next on this seam. Supersedes the CLOUD_EMBED_* dedicated-provider detour.

- types.AIClient: + Embed(ctx, model, inputs)
- clients/aihttp: httpAI.Embed (raw POST reusing the static/M2M authed transport)
- clients/disabled, clients/rpc: stub Embed (fail-closed / not-wired)
- clients/code, clients/knowledge: embed through deps.AI; drop the static key path
- knowledge test: exercise the deps.AI seam (inject a deterministic fake embedder)
2026-07-10 13:19:47 -07:00
zeekayandhanzo-dev e34a2802a2 docs(o11y): reword LLM.md so the one-concept grep stays exact
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 13:03:21 -07:00
zeekayandhanzo-dev 322655eb73 refactor(o11y): collapse 5 subsystems -> ONE o11y; flat version-less paths (#71)
Decomplect the observability estate to exactly ONE top-level concept.

## 5 -> 1 registration collapse
The plane was FIVE separately-registered subsystems (o11yscope 69,
o11y-runtime 71, o11y-event-ingest 68, o11y-otlp-ingest 72,
o11y-trace-inproc 73) whose names leaked five public concepts (five config
toggles + five /v1/<name>/health routes). The k8s-style ordering was an
internal impl detail. They collapse to ONE
`cloud.RegisterWithShutdown("o11y", 69, mountO11y, shutdownO11y, HealthOwner)`:
mountO11y performs the ordered sub-mounts in-process (mountEventIngest ->
mountScope -> mountRuntime -> mountIngest -> mountTraceSink), so the PUBLIC
concept is a single `o11y`. Behavior is preserved EXACTLY — every specific
/v1/o11y/* route still registers inside the one order-69 mount, hence BEFORE
the upstream hanzoai/o11y wildcard (order 70), so Fiber's in-order match still
gives the specific routes precedence. The upstream module co-owns the `o11y`
name at order 70 (the wildcard); HealthOwner on this entry keeps /v1/o11y/health
registered exactly once (never a duplicate).

## Flat, version-less public surface (one /v1/, no nested /api/vN)
The upstream SigNoz engine version is an internal impl detail resolved inside
the handlers, never leaked into a route:
- /v1/o11y/vm/{query,query_range} (was /v1/o11y/vm/api/v1/*) — VM proxy; the
  upstream VM api/v1/* path stays INSIDE the handler (queryRaw). SuperAdmin
  gate + {up,sum(up),count(up)} allowlist unchanged.
- /v1/o11y/{query,query_range} (new, query.go) — the flat builder query;
  resolves to the v3 engine route SERVER-SIDE (the version-less alias would
  float to v5, which 400s the console's v3 composite payload), delegating to
  the same gated runtime handler the wildcard uses.

o11y stays EMBEDDED in-process (the everything-binary); OTLP ingest + trace
sink stay opt-in. Registers exactly one `o11y` in clients (grep-verified),
alongside analytics/evals/usage/audit.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 13:02:28 -07:00
fd308f4b8c cloud: dissolve commercesvc → commerce.Client (fold-finisher + decomplect) (#242)
Absorb the thin clients/commercesvc mount wrapper AND the fail-closed
clients/commerce_local.go stub into the in-tree commerce library, and expose
a real in-process commerce.Client — one package now owns the commerce library,
its /v1/commerce subsystem, and its cloud.CommerceClient, self-registered via
cloud's inversion hooks (the clients/kms pattern; cf. #241 gatewaysvc→gateway).
cloud never imports commerce, so there is no cloud⇄commerce import cycle.

commerce.Client (client.go): a real in-process types.CommerceClient. The former
stub failed CLOSED on every CheckEntitlement (the Phase-2 gap). It now resolves
org→active-subscription→plan-tier→license-features from commerce's OWN models +
the @hanzo/plans vocabulary:

  - org → namespace via commerce's canonical org.Resolve (the same binding the
    money path uses), so a read can never cross tenants;
  - active, unexpired subscriptions via a plain Status filter (matches commerce's
    own read idiom, so grant- and payment-created subs are both found);
  - plan tier → flat license features via a new plan.LicenseEntitlement seam that
    runs @hanzo/plans' toLicenseFeatures (the vocabulary stays in one place, not
    re-implemented in Go);
  - Active iff the plan's features carry "licensing.product:<id>".

MONEY-SAFETY: Active:true only for a real active sub whose real plan really
licenses the product. Every unresolvable piece (commerce not co-resident, org
unresolvable, subscription query error, plans vocabulary unavailable) returns an
ERROR → the entitlements gate treats it as "cannot verify ⇒ 503"; a clean
"no plan licenses this product" is Active:false (→ 402), never an error and never
a fabricated grant.

Decomplect: commerce.Mount takes a MountConfig VALUE (Brand/Env/DataDir/Domain)
+ a logger — the values it uses, not the whole cloud.Deps bag; mountFromDeps is
the one place Deps is narrowed and carries the PCI Payments/Vault warnings.

Network path preserved: pickCommerceClient still selects the ZAP-RPC/disabled
client when commerce is NOT enabled in-process (CLOUD_COMMERCE_ZAP_ADDR). This
fold does NOT force the live in-process cutover — that stays operator-gated.

Deleted: clients/commercesvc, clients/commerce_local.go(+test), the legacy
//go:build cloud clients/commerce/mount.go(+ its two tests), and the redundant
cmd/commerce --cloud boot path (cloud_boot.go/cloud_stub.go) — one way to serve
commerce inside cloud: the folded subsystem.

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 13:02:00 -07:00
zeekayandhanzo-dev 3cdca83cc8 test(audit): prove store shareable by RO reader under live writer (HA carve)
Concurrent RO opener reads all records while the serialized writer appends;
writer chain stays intact and a fresh RW re-open recovers the head with no
fork/gap. Audit-store analog of clients/kms TestReaderReadOnlyRoundTrip.
Locks in the reader/writer shareability invariant the cloud HA carve depends
on. Test-only; zero runtime change (writer path byte-identical).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 12:54:13 -07:00
hanzo-dev 051b63d0db fix(embed): dedicated CLOUD_EMBED_* provider — unbraid embeddings from chat
The code + knowledge embedders read CLOUD_AI_BASE_URL/CLOUD_AI_API_KEY,
which are ALSO the chat/synth AIClient's config (config.go:308). That
braided embeddings onto the IAM-gated api.hanzo.ai path, whose /embeddings
403s customer keys and 401s gateway keys — only a superadmin JWT passes —
so the vector tier indexed vectors:0 platform-wide.

Split embeddings onto dedicated CLOUD_EMBED_BASE_URL / CLOUD_EMBED_API_KEY /
CLOUD_EMBED_MODEL, each falling back to the shared CLOUD_AI_* for back-compat.
Both semantic subsystems (clients/code + clients/knowledge) read the same
three vars — one embed config, orthogonal to chat. Deployment points them at
DigitalOcean GenAI (bge-m3 @1024-dim, first-party, not IAM-gated) via the
DO_AI_API_KEY already in cloud-api-llm-keys; chat stays on CLOUD_AI_*.
2026-07-10 12:09:15 -07:00
hanzo-dev 4b62d0981c fix(platform): preserve a kept secret on empty setEnv (never wipe KMS)
Secrets are write-only: toAppView masks a secret value to "" on read, so a client
editing env re-submits kept secrets with an empty value. sealSecretEnv now treats an
empty secret value as "keep the already-sealed value" — it is preserved, not resealed
to empty (which wiped the KMS secret). Only a non-empty value seals (and requires KMS).
Makes the masked-read -> setEnv round-trip a no-op for untouched secrets instead of a
data-loss footgun; unblocks the console env editor's Keep path. Adds a focused test.
2026-07-10 11:55:55 -07:00
f24801c715 cloud: drop svc suffix — clients/gatewaysvc → clients/gateway (#241)
The gateway subsystem package carried an "svc" suffix as a naming habit
only: there is no clients/gateway to collide with, and the package imports
no "gateway" library, so it renames cleanly to the plain domain name.

- git mv clients/gatewaysvc → clients/gateway (history preserved)
- gatewaysvc.go / gatewaysvc_test.go → gateway.go / gateway_test.go
- package gatewaysvc → package gateway; Mount() error strings + doc comment
  updated to gateway.*
- update the blank import in subsystems/subsystems.go and the doc comments
  in deps.go, middleware_edge.go, clients/gatewaypolicy/policy.go

commercesvc is intentionally NOT renamed in this PR. clients/commerce
already exists in-tree — the embedded upstream Hanzo Commerce library
absorbed by #114 (package commerce, ~1583 files) — and commercesvc.go
imports it as github.com/hanzoai/cloud/clients/commerce. Moving
clients/commercesvc → clients/commerce is therefore a directory-level
collision, not an import-alias shadow; re-aliasing does not free the path.
Renaming commerce cleanly needs the operator to decide the target name (or
relocate the embedded library), so it is flagged for follow-up, not forced.

Build: GOWORK=off CGO_ENABLED=0 go build ./...  → exit 0
Test:  GOWORK=off CGO_ENABLED=0 go test ./clients/gateway/... \
         ./clients/gatewaypolicy/... .           → ok

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 11:52:55 -07:00
hanzo-dev 99bec227da fix(platform): wire kmsIdentity (secret env) + mount platform,git in deploy enable list 2026-07-10 11:48:42 -07:00
zeekayandhanzo-dev 9b12e55d62 feat(o11y): SuperAdmin VM read proxy for the console infra-health board (#71)
The console's /metrics and /status SuperAdmin infra-health board read
VictoriaMetrics `up{}` via the console's Next.js `/telemetry/[...path]` route.
That server route is STRIPPED by the static-export embed (cloud go:embeds
console as output:'export'), so the browser call 404s — the last console error.

Add a same-origin, SuperAdmin-gated VM read proxy on the cloud `/v1` API that
serves the exact three queries the board issues, registered at o11yscope order
69 (before the o11y wildcard at 70):

  GET /v1/o11y/vm/api/v1/query?query=up
  GET /v1/o11y/vm/api/v1/query_range?query=sum(up)|count(up)&start&end&step

Security follows scope.go's contract ("the client never supplies a raw query"):
  - admin(c) gate (X-User-IsAdmin, reserved admin org) — 403 for everyone else.
  - The ?query param is ALLOWLISTED to exactly {up, sum(up), count(up)}; anything
    else is 400. Range args (start/end/step) validated as positive integers. No
    generic PromQL passthrough — this can never become an exfiltration/DoS surface.
  - VM's native Prometheus envelope is returned VERBATIM (c.Bytes) so the console's
    parseInstant/parseRange work unchanged. Reuses the existing newVMClient()
    (O11Y_VM_URL); adds vmClient.queryRaw for the verbatim forward.

Pairs with console 1beb6fca9 (telemetry.ts repoint). Verified: go build + go vet
clean on clients/o11y.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 11:47:37 -07:00
zandGitHub 1fa10831cc Merge pull request #240 from hanzoai/feat/114-absorb-commerce-source
feat(#114): absorb commerce source in-tree (clients/commerce) — one module, one binary
2026-07-10 11:32:50 -07:00
hanzo-dev 1c22329dfb feat(#111): billing money-path in-process — retire the standalone commerce hop
Every co-resident subsystem that spoke the commerce billing S2S surface over HTTP
to the standalone pod (commerce.hanzo.svc:8001) now dispatches straight into the
in-tree commerce handler (#114) — no socket, no serialization change. This is what
lets the standalone be retired.

- New seam clients/commerceinproc: commercesvc.Mount publishes the embedded
  commerce http.Handler (now carrying /v1/billing via api.Route) once at boot; a
  self-routing http.RoundTripper dispatches each S2S request to it in-process when
  co-resident, else falls back to plain HTTP (the pre-#111 split-deploy behavior).
  Behaviour-preserving: same path, same Bearer+X-Org-Id headers, same body/status
  bytes either way — proven by commerceinproc_test (in-process dispatch, HTTP
  fallback, base resolution).
- Converted: clients/{billing,account,admin,referrals,authors,affiliates,usage} —
  each keeps its OWN request-building + tenant subject-pinning (the security
  boundary is untouched); only the transport swaps. account keeps its shared
  httpClient for HUSD EVM JSON-RPC and routes ONLY commerce calls in-process.
- The request-edge METERING gate (build.go buildMeteringClient) debits the
  in-process handler; base pinned non-empty when co-resident so the gate never
  silently no-ops (free-money hole) even after CLOUD_COMMERCE_HTTP_URL is dropped.

GATES: go build ./... + cmd/cloud (sqlite tags) green; commerceinproc + all 8
converted packages' tests pass. LIVE money-parity gate (in-process vs standalone
balance/deposit/usage on shared Postgres + metering debits) precedes retiring the
standalone — that live cutover is gated, not in this commit.
2026-07-10 11:29:07 -07:00
hanzo-dev 848c28def5 feat(#114): absorb commerce source in-tree (clients/commerce) — one module, one binary
Move hanzoai/commerce out of the external-dep seam and INTO the cloud module,
so /v1/commerce (and, next, /v1/billing) is served by one repo, one binary, one
way. No external github.com/hanzoai/commerce* require remains.

Absorbed the EXACT versions cloud already compiled (behavior-preserving — not
main), copied from the module cache so the in-tree tree == what the live binary
built:
  - github.com/hanzoai/commerce            v1.46.40  -> clients/commerce
  - github.com/hanzoai/commerce/metering   v0.1.4    -> clients/commerce/metering
  - github.com/hanzoai/commerce/thirdparty/ethereum v1.40.0 -> clients/commerce/thirdparty/ethereum

- ONE Go module: removed the three nested go.mod/go.sum; commerce's deps merged
  into cloud/go.mod via `go mod tidy` (MVS keeps the highest existing patch — the
  single major conflict, luxfi/zap, already resolved to cloud's v1.2.1 in the old
  build, so nothing recompiles differently). New direct deps discovered by tidy:
  huandu/facebook, square-go-sdk/v3.
- Imports rewritten repo-wide: github.com/hanzoai/commerce* ->
  github.com/hanzoai/cloud/clients/commerce* (1103 files: the absorbed tree +
  cloud's own build.go/deps.go/middleware_billing.go/clients/{bots,functions}/…
  metering importers). commercesvc leaf now imports the in-tree package.
- Excluded the app/ frontend pnpm workspace (59M, NOT referenced by any .go —
  the built ui/dist + billing/ui/dist + checkout/ui/dist that Go //go:embed's are
  kept). //go:build cloud files kept (excluded from the plain build as before).

GATES (all green):
  go build ./...                                   -> ok
  go build -tags "libsqlite3 sqlite_fts5" ./cmd/cloud -> ok (633MB binary boots)
  go test ./clients/commercesvc/... ./clients/commerce/{,datastore,api/billing}  -> ok
  go test -run 'Billing|Metering|SpendCap|Commerce' .  -> ok
2026-07-10 11:13:14 -07:00
hanzo-dev 04e245bdbd Merge branches 'wt3/B' and 'wt3/C' into wt3/integrate 2026-07-10 11:11:22 -07:00
hanzo-dev 225432b1a8 refactor(admin): adopt cloud.Service[state], drop per-package svc 2026-07-10 11:09:43 -07:00
hanzo-dev ea02c3baf0 refactor(ml): adopt cloud.Service[state], drop per-package svc 2026-07-10 11:08:54 -07:00
hanzo-dev 9bda7878af refactor(affiliates): adopt cloud.Service[state], drop per-package svc 2026-07-10 11:06:59 -07:00
hanzo-dev a7e67f1608 refactor(do): adopt cloud.Service[state], drop per-package svc 2026-07-10 11:03:59 -07:00
hanzo-dev 48cdbcc8e3 Merge branch 'main' into wt3/integrate 2026-07-10 11:03:17 -07:00
hanzo-dev 77317b55fe refactor(tracker): adopt cloud.Service[state], drop per-package svc 2026-07-10 11:00:06 -07:00
hanzo-dev 0b7c6cfbdc Merge branch 'wt3/G' 2026-07-10 10:59:09 -07:00
hanzo-dev 8184b97c35 Merge branch 'wt3/D' 2026-07-10 10:59:01 -07:00
hanzo-dev 76c73ffa68 refactor(treasury): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:58:36 -07:00
hanzo-dev d96c00fe2c Merge branch 'wt3/E' 2026-07-10 10:58:23 -07:00
hanzo-dev 6e1e87b371 refactor(framework): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:58:04 -07:00
hanzo-dev a1e556227e refactor(paas): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:57:45 -07:00
hanzo-dev 85795b85c6 refactor(automations): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:57:01 -07:00
hanzo-dev ecc622d888 Merge branch 'wt3/H' 2026-07-10 10:56:11 -07:00
hanzo-dev 2f6ce9a24f refactor(wallets): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:56:03 -07:00
hanzo-dev 54a7f43263 refactor(platform): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:55:07 -07:00
hanzo-dev 04371fc01f refactor(ingress): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:54:49 -07:00
hanzo-dev 218d3c3cee Merge branch 'wt3/A' 2026-07-10 10:54:04 -07:00
hanzo-dev 4c55806a97 refactor(crm): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:53:35 -07:00
hanzo-dev 64c8b3e44f refactor(storage): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:52:53 -07:00
hanzo-dev 8a4aca362b refactor(agents): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:52:40 -07:00
hanzo-dev 62c9ab5f47 refactor(sbom): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:51:55 -07:00
hanzo-dev 3380848531 refactor(dataroom): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:49:35 -07:00
hanzo-dev 48d9b15713 refactor(git): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:49:27 -07:00
hanzo-dev 6bf3cdb262 refactor(sign): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:48:48 -07:00
hanzo-dev 3aa53f2a29 refactor(projects): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:48:43 -07:00
hanzo-dev 9888189c5f refactor(integrations): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:47:19 -07:00
hanzo-dev aff2a55469 refactor(authors): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:46:54 -07:00
hanzo-dev 3861d68904 refactor(graph): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:46:37 -07:00
hanzo-dev 7abfdb8718 refactor(kms): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:46:22 -07:00
hanzo-dev 787ad3a6fb refactor(functions): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:46:02 -07:00
hanzo-dev 38c6795a73 refactor(provisioning): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:45:26 -07:00
hanzo-dev ec180f5366 refactor(account): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:43:16 -07:00
hanzo-dev dad7ec6aa1 refactor(analytics): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:42:02 -07:00
hanzo-dev 6205664012 refactor(zt): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:40:59 -07:00
hanzo-dev 496f176d77 refactor(visor): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:39:47 -07:00
hanzo-dev 46091d2098 refactor(referrals): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:39:41 -07:00
hanzo-dev 6809e4d574 refactor(captable): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:39:40 -07:00
831742eb3c fix(deps): pin hanzoai/beego/v2 to tag v2.4.1 (kill controlplane-containment shallow-clone flake) (#239)
cloud pinned beego at a PSEUDO-VERSION
(v2.4.1-0.20260710093857-0ad99bdf8b90). A pseudo-version forces `go` to
resolve the dep via a live VCS fetch-by-commit-hash into the shared ARC
GOMODCACHE cache/vcs. Parallel containment jobs racing that shallow bare
repo hit `fatal: shallow file has changed since we read it` → intermittent
FAIL, which has forced admin/green-locally squash-merges (#235, #236) and
defeated the containment gate.

beego HEAD (0ad99bdf) is exactly one benign commit past tag v2.4.0 (silences
a missing-default-app.conf stderr warning, beego#29). Cut that commit as a
proper patch tag v2.4.1 and pin to it — byte-identical code, but `go` now
fetches the immutable refs/tags/v2.4.1 ref instead of shallow-fetching a
bare commit, so no cache/vcs race. beego is private (no module-proxy), but
the tag ref path is deterministic and does not trip the shallow-file check.

go.sum regenerated via `go mod tidy` (authentic v2.4.1 hash). `go build ./...`
green. iam stays on tag v2.3.10 (already a tag, not a pseudo-version).


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 10:39:12 -07:00
hanzo-dev 249dd7a5da telemetry: declare the host tracer provider to embedded ai (gen_ai spans -> o11y)
cloud installs the ONE global tracer provider wired to the o11y in-process trace
sink, but in in-process-sink mode it sets no OTLP/ZAP exporter endpoint. The
embedded hanzoai/ai module then found no endpoint and DISABLED its gen_ai span
emit, so the gen_ai plane in o11y_traces was dark (cloud's own request spans
flowed; every LLM call's gen_ai span was gated off).

After otel.SetTracerProvider, call aiobject.AdoptHostTracerProvider() so ai emits
every gen_ai span through THIS provider -> the o11y in-process sink -> o11y_traces.
Repins ai to the branch carrying AdoptHostTracerProvider. The OTLP-env unset stays
as orthogonal defense against any other lib forking a competing provider.

ai: github.com/hanzoai/ai v1.804.1 -> v1.804.2-0.20260710172916-723e03f81bca
2026-07-10 10:38:51 -07:00
hanzo-dev fe9f67582c refactor(billing): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:37:09 -07:00
hanzo-dev 3b07f84380 refactor(bots): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:36:03 -07:00
hanzo-dev f6f8592a33 refactor(security): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:35:49 -07:00
hanzo-dev 26b6cab29c refactor(team): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:35:49 -07:00
hanzo-dev 8686a11769 refactor(knowledge): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:35:00 -07:00
hanzo-dev d9bfa35b00 refactor(prompts): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:33:58 -07:00
hanzo-dev 9a65c02be4 refactor(templates): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:33:18 -07:00
hanzo-dev 9a2f0e5cfd refactor(gatewaysvc): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:32:10 -07:00
hanzo-dev 5f299546f7 refactor(auditlog): adopt cloud.Service[state], drop per-package svc 2026-07-10 10:31:20 -07:00
hanzo-dev e7fd53820f feat(cloud): export cloud.NewBase for complex-Mount subsystems
Simple subsystems use one-line cloud.Mount[S]. Subsystems whose Mount is more
than build+routes (background reconciler, package-global for cross-package hooks,
shutdown cancel — e.g. platform) construct the Service value directly with
cloud.NewBase + &cloud.Service[state]{...} and wire routes via Handle — still the
ONE generic type, one Base derivation.
2026-07-10 10:25:53 -07:00
hanzo-dev db5b8e9b56 feat(cloud): one generic subsystem type — cloud.Service[S], kill per-package svc
The ONE server abstraction: a subsystem is cloud.Base (shared deps derived once:
Log/KMS/Bill/Brand/Env/Domain/DataDir) + its own typed State. Handlers are FREE
FUNCTIONS func(*Service[S], *zip.Ctx) error bound with cloud.Handle — so a package
declares NO service/receiver type, only its State (plain data) and its handlers.
cloud.Mount[S] is the one generic entrypoint (build state, wire routes).

Pike minimalism: generics carry the state type, functions carry behaviour. Kills
the 'type svc struct{ …re-plumbed deps… }' shape copied ~40×.

Proven on clients/usage: svc type gone, log via embedded Base (s.Log), commerce
reader in State (s.State.commerce), all handlers/helpers free functions. Build +
vet + test green, routes + tenant isolation unchanged.
2026-07-10 10:25:53 -07:00
ccb925d474 feat(#105): un-stage commerce (in-binary /v1/commerce) — DRAFT, gated on cutover prep (#237)
* feat(#105): un-stage commerce — serve /v1/commerce in-process from the one binary

Phase 2 final step (tasks #96#105). commerce was the last staged
in-process subsystem; drop it from stagedSubsystems so the mount-all default
serves /v1/commerce (+ /_/commerce) from the cloud binary instead of proxying
to the standalone commerce pod. iam + ingress STAY staged (the IAM embed
corrupts its own Beego bootstrap under mount-all).

commerce owns the money path, so the cutover is data-neutral BY CONSTRUCTION:
the authoritative stores stay put and shared — balances/deposits/credits live
in Hanzo SQL (SQL_URL), analytics in DATASTORE_URL, blobs in S3 — and the
in-process commerce.Embed reads the SAME stores via the SAME backend env the
standalone CR carried. Only the small per-org merchant SQLite + tenant `base`
tree migrate into the cloud data dir. No money is copied or split.

Tests: go test -run TestEnabled ./  → ok (staged contract: commerce now mounts
under the empty-Enable default; iam/ingress still gated to explicit CLOUD_ENABLE).

Also gofmt: fixes a pre-existing struct-literal misalignment in LoadConfig.


* fix(commercesvc): isolate in-process commerce data under {DataDir}/commerce

In-process commerce writes orgs/ and base/ under its DataDir. cloud sets
deps.DataDir=/var/lib/cloud, where cloud ALSO owns /var/lib/cloud/orgs (its
own per-org subsystem SQLite, HIP-0302) and /var/lib/cloud/base (its Base/IAM
store) — verified live. Sharing the root would open two apps on the same
SQLite files (base/data.db) and corrupt them. Always nest commerce under
{deps.DataDir}/commerce (was only the empty-DataDir fallback), keeping the
commerce ledgers physically separate on the same cloud-api-data PVC. This is
the target path the #105 data migration copies the per-org merchant SQLite +
tenant base into.


---------

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 10:05:20 -07:00
hanzo-dev c1cf59020c Merge branch 'wt2/preview' 2026-07-10 10:01:28 -07:00
hanzo-dev 194f5c1218 Merge branches 'wt2/bots' and 'wt2/gateway' 2026-07-10 10:01:11 -07:00
hanzo-dev b40226e347 cloud(platform): preview deployments + promote + rollback 2026-07-10 10:00:38 -07:00
hanzo-dev 8d6e68b9ff cloud(gateway): per-org edge config — cache TTL + CORS + rate, org-scoped round-trip 2026-07-10 09:59:20 -07:00
hanzo-dev 00f0916b31 cloud(bots): POST /v1/bots/run — launch a computer-using agent, return VNC session 2026-07-10 09:56:41 -07:00
hanzo-dev 609f578b27 feat(git): mirror external repos into the embedded git server + self-host cloud's own repo 2026-07-10 09:50:22 -07:00
hanzo-dev 633163aa1a feat(platform): native release semantics on /v1/runner (retires release.yml build/tag/notify) 2026-07-10 09:48:41 -07:00
hanzo-devandGitHub 4660a1c8d2 fix(code): normalize embedder base to /v1 — deployed CLOUD_AI_BASE_URL lacks /v1, so /v1/code semantic tier posted to /embeddings (405) → vectors:0. Append /v1 when absent; ask/RAG now works. (#238) 2026-07-10 09:46:47 -07:00
hanzo-devandGitHub 1b9939a54f fix(code): normalize embedder base to /v1 — deployed CLOUD_AI_BASE_URL lacks /v1, so /v1/code semantic tier posted to /embeddings (405) → vectors:0. Append /v1 when absent; ask/RAG now works. (#238) 2026-07-10 09:46:21 -07:00
297ea6bf6a feat(sbom): registry pull-on-miss — materialize attached SBOMs into ClickHouse (#77) (#236)
The consumer half of the SBOM datastore lane. The registry is the source of
truth: CI produces the CycloneDX SBOM and `cosign attach`es it to the image
digest. Cloud now PULLS that attached artifact and materializes its components
into the global hanzo.sbom_component table — no CI push-ingest.

clients/sbom/pull.go (new): registry pull with go-containerregistry (pure-Go,
no binary deps). Resolves ref→digest, locates the CycloneDX SBOM by the cosign
SBOM tag (sha256-<hex>.sbom) first, OCI 1.1 referrers second, and parses it
through the SAME parseComponents the POST /v1/sbom path uses (one flattener).

Triggers:
- Pull-on-miss: GET /v1/sbom/{ref} with 0 rows and an image ref pulls the
  attached SBOM, upserts it, and rereads FINAL — the console goes live with no
  console change (the panel already GETs by repository:tag).
- Deploy-time: platform applyLive fires sbom.Prefetch(ref) async when a
  deployment goes live (best-effort, idempotent; platform→sbom, one direction).

go.mod: + github.com/google/go-containerregistry v0.21.7 (direct), authentic
go.sum via go mod tidy.

Tests: hermetic end-to-end over a real in-memory OCI registry (cosign tag +
OCI referrers + no-attachment + bare-digest), all green with -race. Plus an
env-gated live end-to-end (pull_live_test.go) proven against registry:2 + real
ClickHouse serving a real cyclonedx-gomod SBOM: GET → pull-on-miss → production
pullSBOM → parse → INSERT → reread → 200 with the real components + cache hit.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 09:34:28 -07:00
hanzo-dev 2208539090 feat(platform,git): git-push-to-deploy — a push triggers a native build
A push landed on the embedded git server (clients/git) now fires a build
for every app that tracks that repo+branch — no GitHub, no Actions.

- build.go: GitPushEvent + RegisterPushBuilder/OnGitPush — the same
  package-level inversion as kmsClientFactory, so git never imports
  platform and there is no git<->platform cycle.
- clients/git/smart_http.go: after a push lands (post-metering),
  firePushBuilds fires OnGitPush for each branch ref that advanced
  (tags + deletes skipped). Best-effort: a trigger failure is logged,
  never fails the push the client already committed.
- clients/platform/push.go: buildFromPush resolves every git-source app
  whose RepoURL+branch matches the pushed ref and launches a build via
  the ONE shared build-launch core.
- clients/platform/deploy.go: extract startGitBuild (ctx-only) as that
  single core; deployGit maps it onto the HTTP deploy, buildFromPush onto
  the push trigger — one build-launch path, no duplication.
- clients/platform/validate.go: the cloud's own embedded-git apex
  (deps.Domain) is always a trusted build source, so a self-hosted-git
  app builds with no env — host all repos on our own git, not GitHub.

Tests: TestPushFiresBuildTrigger (real go-git push -> OnGitPush fires with
org/repo/branch/commit/cloneURL), TestBuildFromPush_{LaunchesMatchingApp,
NoMatchIsNoop,IgnoresImageApp}. Full platform + git suites green.
2026-07-10 09:26:32 -07:00
zandGitHub bde5c47ddc fix(team): remap migrated-workspace socialId to deterministic person on connect (#235)
Jul-5 team-go→cloud migrated workspaces threw "Confirmed social identity is attached to the wrong person" on transactor connect: the confirmed hanzo:<account> SocialIdentity was still attached to a team-go-era Person id, not the deterministic person-<account>. reconcile() now runs remapMigratedSocialIds(uid) to re-point it — migrated-only, idempotent, non-destructive. CI red check (controlplane-containment) is an unrelated shared-runner Go module-cache flake on hanzoai/beego; fix is clients/team-only and green locally (go build ./..., make build, go test/-race/vet/gofmt).
2026-07-10 09:25:49 -07:00
zeekayandhanzo-dev ae49159652 fix(o11y): edge-trusted local authz in the embed (o11y v1.5.10)
Set O11Y_AUTHZ_PROVIDER=local so the in-process o11y authorizes org-scoped,
gateway-authenticated users locally instead of round-tripping to an external IAM
Casbin enforcer the one-binary has no credentials for. That round-trip was 401ing
every /v1/o11y read (provision Grant -> add-policy authz_unavailable), breaking the
console overview-metrics widgets on ~9 secondary product pages. Same enforced
policy; tuples in-process. Bumps o11y 1.5.9 -> 1.5.10.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 08:49:11 -07:00
zeekay acd6614c72 deps: bump hanzoai/o11y v1.5.8 -> v1.5.9 (surface swallowed IdentN errors — diagnose o11y read 401) 2026-07-10 08:25:36 -07:00
zeekayandhanzo-dev 8d0f6f3686 deps: bump hanzoai/o11y v1.5.7 -> v1.5.8 (role-selector digits — fix o11y-admin grant panic)
v1.5.8 allows digits in TypeRole selectors, unbreaking the built-in o11y-admin role
grant that panicked (→500) on EVERY authenticated embedded o11y data read. Completes
the o11y-telemetry fix (v1.5.6 aliases + v1.5.7 /api passthrough + v1.5.8 grant):
/v1/o11y/{query_range,services,rules,dashboards} now resolve for the console
overview-metrics widgets.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 06:24:50 -07:00
zeekayandhanzo-dev 18a4814c84 deps: bump hanzoai/o11y v1.5.6 -> v1.5.7 (embedded /api/* passthrough — fix o11y data 404s)
v1.5.7 routes /api/*-prefixed paths straight to the router in the ExternalPath wrapper,
fixing the double-strip that 404'd EVERY embedded o11y data call. With v1.5.6's
version-less aliases, /v1/o11y/{query_range,services,rules,dashboards}+/metrics now
resolve — fixes the console overview-metrics widgets on studio/gateway/cli/registry/
desktop/console/dashboards/alerts/metrics.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 05:26:55 -07:00
zeekayandhanzo-dev 0eee4c5f6a deps: bump hanzoai/o11y v1.5.5 -> v1.5.6 (version-less /v1/o11y/<resource> aliases)
v1.5.6 registers the version-less /api/<resource> aliases on the app.Server router
the embedded runtime uses (o11y e01015954), so the console's /v1/o11y/{query_range,
services,rules,dashboards} + /metrics resolve instead of 404 — fixes the o11y-telemetry
overview widgets on studio/gateway/cli/registry/desktop/console/dashboards/alerts/metrics.
Also folds in the v1.5.5 C1 cross-tenant llmobs read fix.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 04:37:05 -07:00
28f4698d24 config: keep commerce staged (money-path data migration required first) (#234)
captable/sign/dataroom stay un-staged — their standalone stacks held no live
data. commerce owns real transactional data + the money path (commerce-api /
pay / webhooks + credit-deposit flow); un-staging split-brains /v1/commerce onto
a fresh in-process dataset. Proxy the authoritative standalone until commerce's
in-binary cutover (data migration + money-path repoint + cron) is done properly.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 04:21:35 -07:00
zeekayandhanzo-dev 4326d2a447 fix(zt): networks/edge READS return empty-200 when ZT unconfigured (not gate 503)
The prior fix degraded the orgRouters-error path, but zt's gate() fail-closes with
503 BEFORE that when ZT_CLIENT_ID/SECRET are unset — so /networks + /edge still 503'd
a console error on every load. Short-circuit the two READ handlers to an honest-EMPTY
list (200) when unconfigured, ahead of the gate. WRITES keep the fail-closed 503.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 03:57:05 -07:00
066aa15853 fix(hanzo): control-plane verbs emit no server-init noise (#231)
Every `hanzo` control command (version, whoami, config, apps, login, …) printed
three lines of server-mode init chatter first:

  init global config instance failed ... open conf/app.conf: no such file...
  failed to load persistent registry signing key: KMS_SERVICE_TOKEN ... required
  generating ephemeral registry signing key for non-production runtime

These come from dependency package init() functions that run before main(), so
main.go's client-vs-server branch cannot gate them, and Go's alphabetical
package-init order (beego < cloud < iam) means no cloud-side os.Stderr swap can
pre-empt them — verified with an init-order probe. Masking the output would also
leave a wasteful KMS fetch + RSA keygen firing on every CLI call. Fixed at the
source instead:

  - hanzoai/iam#117: registry signing key resolved lazily (sync.Once) at its use
    sites, not in package init(); server token paths keep identical semantics,
    the CLI never touches KMS or generates a key.
  - hanzoai/beego#29: the benign conf/app.conf probe is silent when the default
    file is absent (the CLI case), loud only on a present-but-broken file.

Bump both deps to the fixed versions and document the quiet-output invariant at
the CLIENT MODE gate in cmd/hanzo/main.go.

Result: `hanzo version|whoami|config path|apps --help` emit ZERO stderr; server
subcommands (`hanzo ai --help`, serves) initialize unchanged.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 03:42:39 -07:00
9cf947e722 config: phase-2 flip — un-stage commerce/captable/sign/dataroom (serve from the one binary) (#232)
All four are folded in-process and validated on cloud-unified-canary
(v1.786.167): commerce embedded, captable/sign/dataroom "mounted in-process
(goja + per-tenant Base)", /v1/{commerce,captable,sign,dataroom}/health all 200.
Dropping them from stagedSubsystems makes the mount-all default serve them, so
the main cloud (empty CLOUD_ENABLE) runs them natively and their standalone
Postgres/Next pods retire. iam + ingress stay staged (IAM embed corrupts its own
bootstrap under mount-all; iam served by the standalone pod).


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 03:38:19 -07:00
zeekayandhanzo-dev 0bd907594b fix(graph,zt): degrade indexer/oracle/network/edge reads to honest-empty (no 502/503 page errors)
console.hanzo.ai /indexer, /oracles, /networks, /edge fired 502/503 console errors
because these list handlers returned the upstream error when the chain indexer /
price-feed oracle / ZT controller is unreachable or unconfigured (ZT_CLIENT_ID
optional). Same graceful fold already shipped for visor clusters/machines/gpus:
log + return an honest-EMPTY list (200). An empty list is honest (you have none),
never fabricated; ZT WRITES stay fail-closed. Clears the console errors on 4
secondary product pages for every org without those backends deployed.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 03:25:15 -07:00
75e8ef9c03 cloud: remove /v1/console/* namespace — routes to their real domains (keys/onboard→/v1/iam) (#227)
"console" is just our cloud FE name; there must be NO /v1/console/* API domain.
Rename clients/console → clients/account and re-home every route onto its REAL
domain, forwards-only (no /v1/console aliases, no compat shim):

  keys   GET/POST/DELETE  /v1/console/keys        → /v1/iam/keys
  onboard POST            /v1/console/onboard      → /v1/iam/onboard
  csrf   GET              /v1/console/csrf         → /v1/csrf
  embed-status GET        /v1/console/embed-status → /v1/embed-status
  topup  POST             /v1/console/topup/wallet → /v1/commerce/topup/wallet
  health                  /v1/console/health       → dropped (generic /v1/<name>/health)
  waitlist                /v1/console/waitlist     → dropped (SPA uses clients/base /v1/waitlist)
  billing/commerce bridges /v1/billing/*,/v1/commerce/*  paths unchanged, relocated

The same handlers, CSRF protection (requireCSRF), per-IP rate limiting, and the
VALIDATED-principal tenancy are preserved — only the PATHS change.

IAM wildcard ordering: keys/onboard live on /v1/iam/*, which clients/iam (order 50)
mounts as a WILDCARD. The package registers TWO subsystems so the specific routes win
Fiber's first-match scan: `account` (order 48) mounts /v1/iam/{keys,onboard} + /v1/csrf
+ /v1/embed-status + /v1/commerce/topup/wallet BEFORE the wildcard (and before the
commerce embed at 100); `account-bridge` (order 122) mounts the /v1/billing/* +
/v1/commerce/* catch-all bridges AFTER clients/billing (121) + the commerce embed (100).
Both share one svc + a process-wide CSRF key so a /v1/csrf token verifies on the bridge
writes. Proven by TestIAMKeysBeatsWildcard (native /v1/iam/keys beats the wildcard;
/v1/iam/oauth/token still falls through) and TestRegisteredOrders (account<50, bridge=122).

waitlist.go/waitlist_test.go deleted; httpClient relocated to topup.go. All 54 tests
pass; go build ./... green.

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 03:24:55 -07:00
557de3b002 feat(dataroom): fold dataroom into cloud in-process on gojabase (#101) (#230)
Fold hanzoai/dataroom (Papermark fork: Next.js + Prisma + Postgres) FULLY into
the unified cloud binary via the gojahost pattern (HIP-0106, task #101 / epic
Postgres, no Next.js.

REUSE the shared binding, don't build a second. clients/dataroom runs the
self-contained dataroom goja bundle (byte-identical to hanzoai/dataroom/goja/
bundle.js, go:embed) on the REUSABLE clients/gojabase host — the SAME RW-Base
binding captable (#97) pilots and esign (#100) reuses — which injects
__db/__newId/__now, opens one SQLite file per tenant, and runs each dispatch in
one transaction (commits iff status<400). The leaf adds only: the per-tenant
Schema (schema.go), the object-storage seam for document bytes, a bcrypt HostFn
for link passwords, and the public link->org index. Zero domain logic in Go.

gojabase gains ONE generic, domain-free seam — Config.HostFns — the extension
point esign/dataroom both reach for (dataroom injects __bcrypt; the reserved
__db/__newId/__now always win). captable is unaffected (nil HostFns).

Storage: document BYTES go through the cloud object-storage seam (deps.VFS — the
S3/SeaweedFS data plane, a Go storage host-fn in the leaf, NOT local FS); the
bundle persists only the opaque key. View-analytics events (page-by-page
tracking) are Base rows in the tenant DB.

Auth: admin routes require a validated cloud principal (principal.Tenant -> org);
public viewer routes resolve their org from the link index. Passwords hashed with
bcrypt in Go — never plaintext. Registered order 134, HealthOwner, STAGED behind
CLOUD_ENABLE. go.mod unchanged.

Proof (clients/dataroom/flow_test.go, in-process over real per-tenant Base + the
VFS seam): create dataroom -> upload document (bytes to storage) -> attach ->
create email+password-gated share link -> open as public viewer -> authenticate
(wrong password 401, disallowed email 403) -> record per-page views -> analytics:
  {"pages":[{"pageNumber":1,"views":2,"totalDuration":6000,"avgDuration":3000},
            {"pageNumber":2,"views":1,"totalDuration":900,"avgDuration":900}],
   "totalPageViews":3,"totalViews":1}
plus viewer byte download round-trip and cross-org isolation. Live binary boots
with CLOUD_ENABLE=dataroom, mounts in-process, health 200, admin 403 fail-closed.

Companion bundle PR: hanzoai/dataroom#6.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 03:09:22 -07:00
fd9980325a feat(sign): fold hanzoai/sign into cloud via the reusable gojabase host (task #100) (#229)
Fold the esign product (Documenso fork — open-source DocuSign) FULLY into the
unified cloud binary, per HIP-0106 (epic #96, task #100). Cloud serves /v1/sign/*
itself — the TS domain on dop251/goja backed by per-tenant Hanzo Base/SQLite. No
Next.js, no Prisma, no Postgres. Reuses the SAME clients/gojabase RW-Base binding
the captable pilot (#96) established — ONE binding, not a second.

clients/gojabase: add an additive, backward-compatible Config.HostFns passthrough
— extra native host globals injected per dispatch alongside __db/__newId/__now.
esign uses it for __pdf; captable is unchanged (tests green).

clients/sign (leaf on gojabase):
  - schema.go — the per-tenant sign.db DDL (gojabase Config.Schema).
  - signer.go — THE HARD PART as Go host-functions injected via HostFns as
    __pdf = { stamp (pdfcpu renders field values onto the PDF), sign (real
    x509/PKCS#7 seal via digitorus/pdfsign) }; signer sourced from KMS PEM,
    persisted PEM, or a self-signed dev cert. Signing orchestration stays TS.
  - sign.go — Mount + route table; owner routes gated by principal.Tenant,
    recipient token routes org-in-path. Registered + STAGED behind CLOUD_ENABLE
    (config.stagedSubsystems); retires the standalone esign pod on cutover.
  - sign_test.go — end-to-end wire proof: create→recipient→fields→send→sign→
    complete seals a REAL signed PDF (/ByteRange,/Type /Sig,PKCS7) + full audit,
    per-tenant Base-backed, with cross-tenant isolation.

Bundle: github.com/hanzoai/sign (goja/bundle.js) — the ESM-free domain port on
the gojabase contract (__db/__newId/__now/__pdf, handle{route,params,query,orgId,
body}, one txn per dispatch). go.mod additive; pdfcpu pinned v0.11.0,
hhrutter/pkcs7 v0.2.0 (no bumps).


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 03:06:27 -07:00
hanzo-dev afd7b4f5ff release: cloud v1.786.166 — org-scoped o11y (C1) + BYO billed_cost (C3)
Ship the crown-jewel security fix: o11y v1.5.5 (org-scoped llmobs span views — closes
the cross-tenant read exposed by the Observe->/v1/o11y repoint) + ai v1.804.1 (BYO
billed_cost). The leaky observation read is already deleted upstream (#217); this repin
ships the org-scope via the cloud-embedded o11y runtime.

(Coordinator asked for v1.786.162 but that tag + through v1.786.165 were already cut on
origin; this is the next free tag.)
2026-07-10 02:52:34 -07:00
hanzo-dev 5f0320bf56 fix(security): repin o11y v1.5.5 (org-scoped llmobs) + ai v1.804.1
Crown-jewel security repin off the pseudo-versions to the merged release tags:
- o11y v1.5.4 -> v1.5.5: the llmobs span-view SQL is now org-scoped
  (gen_ai.hanzo.org_id = <validated tenant>, fail-closed) — closes the cross-tenant
  read the Observe->/v1/o11y repoint exposed. Cloud EMBEDS the o11y runtime
  (clients/o11y), so this is the ship vehicle for the fix.
- ai <pseudo> -> v1.804.1: gen_ai span emits _o11y.gen_ai.billed_cost alongside
  total_cost (BYO invoice reconciliation) + the enriched gen_ai span.

The leaky cloud_usage-as-observations read stays DELETED (upstream via #217); the
metering warehouse (cloud_usage via /v1/usage + the #218 /v1/evals/metrics dashboard)
is untouched. No go mod tidy (luxfi/keys go.sum fragility) — only the two repins.
Build: go build ./... green. Test: clients/eval + clients/o11y green.
2026-07-10 02:52:22 -07:00
8e0a3892ce feat(captable): fold Captable,Inc into cloud via goja + reusable Base binding (#96 pilot) (#228)
Cloud now serves /v1/captable/* ITSELF, per tenant, on Base/SQLite — the PILOT of
epic #96 (fold the Captable,Inc app into the unified binary; drop Next.js/Prisma/
Postgres). Where clients/plan + clients/pricing host a read-ONLY @hanzo catalog in
goja, captable hosts the tRPC business LOGIC (ported to a self-contained goja
bundle in github.com/hanzoai/captable) and gives it PERSISTENCE over per-tenant
Base/SQLite. The bundle carries logic; the Go host carries storage.

REUSABLE Base-goja binding (clients/gojabase) — the deliverable esign (#100) +
dataroom (#101) rebase onto. It is the storage-bearing sibling of clients/goja
(the pure JS engine): given a Bundle + a per-tenant Schema (DDL) + DataDir, it
- opens ONE SQLite file per tenant (lazy, migrated once, cached; slug-contained),
- injects per dispatch a tenant-bound __db bridge (query/exec) + __newId + __now,
- runs globalThis.handle inside ONE transaction that commits iff status<400 and
  handle didn't throw (atomic multi-statement mutations for free), and
- carries ZERO domain logic. clients/goja gains DispatchWith (per-call native
  globals) as the read-WRITE extension of the read-only plan/pricing path.

clients/captable leaf: go:embed'd bundle (hanzoai/captable.Bundle) + the per-
tenant schema (Prisma model → SQLite DDL) + a company seed (OnOpen) + the
/v1/captable/* zip routes. Org resolves from the VALIDATED principal
(principal.Tenant), never a client header; that org selects the DB file AND
scopes every row. Registered order 133; STAGED behind CLOUD_ENABLE (joins iam/
ingress/commerce in config.stagedSubsystems) so main stays shippable and the
standalone captable service keeps authority until the phase-2 cutover.

Full fold over Base: stakeholders, share classes, equity plans, securities
issuance (shares + options), share transfers (full + partial, atomic), SAFEs +
convertible notes, rounds + investments (a priced round issues shares and
dilutes), and a computed cap table (fully-diluted ownership, per-class
authorized-vs-issued, convertibles + rounds summary).

Proven:
- clients/gojabase: RW round-trip, per-request rollback (caught-500 + raw-throw),
  per-tenant isolation, OnOpen seed, slug traversal-containment (real SQLite).
- clients/captable: the REAL embedded bundle vs REAL SQLite through the whole
  lifecycle (create stakeholder → issue share class → issue shares → priced round
  + investment dilution → transfer → cap table), and an HTTP wire test via the
  zip/Fiber test client (trusted headers) proving create→read-back, issuance→cap
  table, the bundle's OWN 404 (not a proxy 502), the 403 principal gate, and
  cross-tenant isolation.
- Live binary boots under CLOUD_ENABLE=captable and serves /v1/captable/health
  200 in-process (a proxy would 502).

go build ./... + go build -tags "libsqlite3 sqlite_fts5" ./cmd/cloud green.
go.mod pins github.com/hanzoai/captable at its merged main commit; go.sum authentic.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 02:48:03 -07:00
hanzo-dev 3185eb182c feat(platform): native POST /v1/runner build endpoint (retires GitHub Actions builds)
The server side of the ex-/v1/arcd surface: one native build API on the
runner fabric that hanzo build, git-push-to-deploy, and cloud's own
self-release all call — no GitHub builders, no Actions.

- runner.go: POST /v1/runner. Privileged (caller supplies the output
  image), so gated by a constant-time build-callback token AND an
  image-ref allowlist (ghcr.io/{hanzoai,luxfi,zooai}/*). Fails closed
  when no token is configured.
- k8s.go: launchDirectBuild — validated at the same choke point as the
  tenant build; plus buildFrontendCmd + buildJobSpec extracted so tenant
  and direct builds share ONE Job spec and ONE frontend selector.
- hanzoai/pack is the default BuildKit frontend (zero-config, gateway.v0);
  a Dockerfile is the explicit escape hatch (dockerfile.v0).

Tests: token 503/403, missing-field 400, image-allowlist 403, happy-path
202. Full platform suite green; whole cloud module builds.
2026-07-10 02:28:35 -07:00
8df5362b39 cloud: per-project SQLite via one cloud.TenantDB resolver (DRY tenant DB) (#226)
Before: subsystems opened SQLite inconsistently. clients/code used the good
per-ORG-file pattern ({DataDir}/orgs/{slug}/code.db, resolved per-request), while
git/functions/tracker each opened ONE shared DB at Mount ({DataDir}/git.db,
functions.db, tracker.db) scoped only by an org column per-row — a decomplected
tenancy gap where the physical boundary was a single file for every tenant.

After: ONE resolver — cloud.TenantDB(dataDir, org, project, subsystem) — is the
single way any subsystem opens a tenant SQLite DB. Path convention:
  project-scoped: {DataDir}/orgs/{orgSlug}/projects/{projectSlug}/{subsystem}.db
  org-scoped:     {DataDir}/orgs/{orgSlug}/{subsystem}.db
It MkdirAll 0700s, opens via the sole "sqlite" driver (github.com/hanzoai/sqlite),
applies the shared single-writer + WAL pragmas, and folds org/project through the
injective SanitizeOrg slugger so distinct tenants can never share a file and no
segment can traverse. A generic cloud.TenantStore[T] caches per-tenant stores
(opened once each) so the hand-rolled per-subsystem map is DRY'd into one value.

SanitizeOrg (the one injective org-slug normalizer) moves to the root cloud
package beside OrgHasUnsafeRune and TenantDB; provisioning.SanitizeOrg delegates
to it, byte-identical, so S3/KMS/knowledge slugs are unchanged.

Subsystems migrated onto the resolver:
- code:      adopts the helper; stays ORG-scoped (no project axis) — same path.
- git:       single-shared git.db -> per-ORG file. Kept org-scoped (NOT project)
             because /v1/git/usage is a deliberate org-wide rollup across every
             project; the project stays a row column.
- functions: single-shared functions.db -> per-ORG file (no project axis).
- tracker:   single-shared tracker.db -> per-(org, IAM-project) file — tracker is
             project-scoped (principal.Project); its KEY-based projects are rows
             WITHIN each per-project file.
- tasks:     left as-is — it opens NO SQLite (delegates to the shared durable
             engine owned by durable.go), so there is nothing to migrate.

Data safety: the single-shared -> per-tenant switch is fail-closed (an invalid
org/project errors rather than falling through to another tenant's file). These
are new subsystems with little/no production data; existing rows in a prior
shared *.db would live under {DataDir}/{subsystem}.db and are NOT auto-migrated —
a deployment carrying such data must relocate rows into the per-tenant files
before cutover. No silent data drop.

Tests prove isolation: two orgs -> two files, no cross-read; two projects under
one org -> two nested files; project-scoped path nests; the cache opens each
tenant once. Root tenantdb_test.go plus per-subsystem end-to-end file-isolation
tests (git/functions org, tracker project).

Co-authored-by: zeekay <ai@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 02:19:36 -07:00
hanzo-dev c907e6d8eb Merge branch 'wt/v1run'
# Conflicts:
#	clients/platform/platform.go
2026-07-10 01:55:41 -07:00
hanzo-dev 8a76311c0b cloud(run): POST /v1/run container-serverless handler (reuses Service-CR deploy path) 2026-07-10 00:52:55 -07:00
zeekayandhanzo-dev c12595c799 fix(embed): cachebust on console main HEAD (re-embed on console-only change)
CONSOLE_CACHEBUST was the cloud sha, so a console-only push could not trigger a
fresh embed without a cloud commit — the whole point (freshness) leaked between
cloud pushes. Resolve hanzoai/console main HEAD (git ls-remote, extraheader
cleared since actions/checkout's GITHUB_TOKEN 404s cross-repo; gh absent on the
runner) and use it as the cachebust; fall back to the cloud sha if empty. A
console change now moves the cache key on its own.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-10 00:47:37 -07:00
hanzo-devandGitHub c2d0c0d861 controlplane inc-2 seam (a): per-pod ML-DSA-65 identity keys (contained, flag stays false) (#221)
* controlplane inc-2 seam (a): per-pod ML-DSA-65 identity keys

Replace the symmetric-HMAC proof-of-possession with per-pod asymmetric ML-DSA-65 (github.com/luxfi/crypto/mldsa, MLDSA65, FIPS 204 Level 3) identity keys — the same key material that becomes the cert-signing key in seam (c). idKey is now a fresh random keypair (crypto/rand), NEVER seed-derived; the registry stores the public key; signPoP uses the FIPS 204 5.2 deterministic variant with a domain-separation context; VerifyPoP is asymmetric VerifySignatureCtx.

Acceptance: the two byzantine-safety tests TestRed_B_DerivablePoPForgesHonestLeg and TestRed_B_LoneNodeForgesFullQuorum are un-skipped and now GREEN (an attacker rebuilding a pod's custody from public inputs gets a different key whose PoP fails); TestSafety_RogueAndForgedLegs_Rejected is re-armed to forge a CORRECTLY-DERIVED leg under a real attacker key (not a bit-flip) and still rejects it.

Scope: seam (a) only. ProductionBCCSigningReady() stays false and all other stubs remain — the flag flips later in the same change that lands the real cert (seam c) and deletes the stub types. Full suite green (zero skips), -race clean, gofmt clean, containment intact (no-tag build still matches no packages).

* controlplane seam (a): R3 — doc.go PoP narrative now reflects discharged crypto

RED review R3: refresh the stale CLASS-B caveat + ShareCustody stub-catalog entry so prose matches the code. Per-pod identity keys are real random ML-DSA-65 (not seed-derived); the two TestRed_B_* are un-skipped + green; only the z-share / cert crypto remains stub (ProductionBCCSigningReady stays false until seam c). Doc-only; no code change.
2026-07-10 00:33:24 -07:00
hanzo-devandGitHub afc5805412 feat(o11y): native-Go LLM-obs event ingest (folds retired console-worker) (#222)
Adds the write path for LLM-observability events (traces/observations/scores)
that the retired console-worker (Node BullMQ->Valkey->Datastore) used to
provide, folded into the cloud binary as a normal o11y subsystem.

- POST /v1/o11y/ingestion (validated tenant via principal.Tenant) -> parse batch
  -> route by type -> per-table batch insert into the Hanzo Datastore via the
  branded github.com/hanzoai/datastore-go/v2 client (promoted to a direct dep;
  first direct user in cloud). ZERO clickhouse imports/identifiers -- datastore-go
  brings the same upstream ch-go line (MVS-unified v0.71.0) the SigNoz o11y query
  runtime uses, so it coexists in one binary (verified).
- Grounding: the embedded o11y runtime (github.com/hanzoai/o11y) is a SigNoz fork
  serving INFRA o11y and mounts app.All("/v1/o11y/*") at order 70; it has no
  LLM-obs ingestion. This is a cloud-native SPECIFIC route registered at order 68
  so Fiber's in-order match binds it AHEAD of the wildcard (same rule scope.go
  uses for /v1/o11y/{logs,metrics,status} at 69). A route at the OTLP-ingest
  order (72) would be swallowed by the proxy.
- Oversized event bodies overflow to object storage (blob ref stored inline),
  threshold via CLOUD_O11Y_INGEST_BLOB_BYTES (default 1 MiB).
- Always mounted (no feature flag); the Datastore (O11Y_DATASTORE_DSN) is a
  required dependency. Fail-soft: no DSN / unreachable datastore -> unmounted,
  never blocks boot. Inert in prod until the console producer repoints here.

Tests: go test -tags 'cloud cloud_mount' ./clients/o11y/... green (8 cases:
routing, batching, blob overflow/disabled, sink+blob error propagation, threshold).
Full build green: go build -tags 'cloud cloud_mount' ./...

FLAGGED for cutover review:
- ASSUMED table/column schema (worker source ships dist-only; reconcile
  traces/observations/scores columns with 002_llm_observability.sql before the
  console producer is repointed).
- Durable EmbeddedTasks hand-off: flush runs INLINE today (removes BullMQ+Valkey);
  the durable enqueue->activity path is the next reviewed step (a Datastore insert
  must be a durable Activity, not run in a workflow fn).
2026-07-10 00:33:13 -07:00
hanzo-devandGitHub c581b97994 fix(idem): test uses hanzoai/sqlite not modernc directly — ONE driver, airtight (#225)
Pre-existing stray direct modernc.org/sqlite import in a test file (from #201).
Swap to the canonical _ "github.com/hanzoai/sqlite" (its !cgo backend IS modernc,
identical behavior) so NO file anywhere imports modernc directly. Test-only —
not in the shipped binary — but keeps 'hanzoai/sqlite only' truly airtight.
2026-07-10 00:30:11 -07:00
hanzo-devandGitHub 64868ba755 feat(md): agent-ready markdown content-negotiation via zap-proto/md (#224)
ONE data model, MANY adapters: handlers keep returning structured JSON; this
middleware re-serializes successful application/json responses through
zap-proto/md when the caller asks (Accept: text/markdown or ?format=md), so
token-efficient markdown is a request-time choice, not a second code path.
/v1/code/ + /v1/agents/ may default to markdown; caller override always wins.
Fail-safe: md render error leaves JSON untouched (never a 500); streams/HTML/
bytes pass through.
2026-07-10 00:24:43 -07:00
hanzo-devandGitHub d4e452cf1d feat(code): /v1/code — SOTA hybrid code-intelligence (FTS5+trigram+tree-sitter+sqlite-vec), per-org (#223)
Native, per-org code-intelligence subsystem for AI coding agents and hanzo.app.
Retrieval is HYBRID — three orthogonal tiers fused with reciprocal-rank fusion
(the SOTA lesson that embeddings alone under-serve code search):

  - lexical  — FTS5 trigram over code-tokenized text (camelCase/snake_case split,
    operators kept); substring + regex via trigram pre-filter + regexp verify (Zoekt).
  - symbolic — go/parser for Go (real def→ref call edges) + compact lexical
    extractors for TS/JS/Python/Rust/Solidity; go-to-symbol + edge table.
  - semantic — AST-boundary chunks embedded via the SAME gateway /embeddings
    clients/knowledge uses; cosine KNN over a float32 vector table.

Storage is ONE SQLite file per org at {DataDir}/orgs/{slug}/code.db (HIP-0302):
the tenant boundary is PHYSICAL. Every request is principal-gated (principal.Tenant)
— no validated principal ⇒ 403, a client X-Org-Id is never trusted.

Routes (order 134, before the AI /v1/* catch-all):
  GET  /v1/code/search   ?q=&type=text|regex|symbol|semantic|hybrid&repo=&limit=
  POST /v1/code/context  {query,budgetTokens,repo}  → budget-packed context bundle
  GET  /v1/code/ask      ?q=&repo=  (or POST)        → cited RAG answer (deps.AI)
  POST /v1/code/index    {repo,files,prune}          → (re)index, incremental by hash

Parsing is pure-Go and vectors are brute-force cosine because the repo's canonical
build is CGO_ENABLED=0 (Makefile); CGO tree-sitter and the sqlite-vec loadable
extension would break `go build ./...`. The vectors table is the schema-compatible
sqlite-vec `vec0` drop-in seam. Builds + tests green under both CGO=0 (modernc) and
CGO=1 (sqlite_purego).
2026-07-10 00:24:21 -07:00
zeekayandhanzo-dev 97cefcca43 fix(visor): degrade /v1/clusters gracefully when Visor is unreachable
listMachines/listGpus already log+fall-through to a BYO-only list when Visor is
down; listClusters alone returned the error, which surfaced as a 502 + a console
error on the Clusters/GPUs page for every org where Visor isn't deployed
(visor.hanzo.svc unresolvable). Mirror the graceful fold: log, drop managed
pools, still return the org's BYO clusters. 200, honest empty, no page error.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-09 23:43:53 -07:00
3f812b00d6 fix(iam-embed): bump hanzoai/iam v1.31.18 -> v1.31.19 (NewEnforcer panic fix) (#219)
Unblocks the embedded IAM subsystem. v1.31.18 getPermissionEnforcer called
authz.NewEnforcer(&DefaultLogger{}, false) — the (logger,bool) form Casbin
type-switches params[1]->persist.Adapter, panicking "bool is not persist.Adapter:
missing method AddPolicy" the moment InitEmbed runs against a FRESH store (exactly
the unified cloud iam subsystem). v1.31.19 (fe50caf7) uses NewEnforcer()+SetLogger.

Proven on a fresh store: CLOUD_ENABLE=iam,ai boots green ("iam embedded in-process")
and serves /v1/iam/.well-known/openid-configuration 200 (was fail-closed 503/404);
iam + ai co-reside, process listens :8080/:9653/:9090.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-09 23:26:10 -07:00
hanzo-devandGitHub e55c7838cc feat(websearch): native in-process keyless meta-search — retire the SearXNG proxy (#220)
Finishes the native web-search half of /v1/websearch (HIP-0106: no external
search SaaS, one fewer non-Go dependency). The route no longer reverse-proxies
to the retired SearXNG pod; it runs metaSearch in-process:

- search.go: keyless meta-search over public engines (Bing default, DDG opt-in),
  parses HTML with x/net/html, merges+dedupes by normalized URL, returns the
  exact SearXNG {query,number_of_results,results[]} envelope the LibreChat
  searxng client decodes verbatim. A failing/bot-challenged engine contributes
  zero and never fails the request (degrades to fewer results, never a 5xx).
- websearch.go: /v1/websearch/search now serves searchNative; removed
  newSearchProxy + searchUpstream + WEBSEARCH_UPSTREAM. Auth unchanged (F2):
  validated principal OR shared X-API-Key; neither ⇒ 401/503, never open.
- tests: converted the proxy route/guard tests to native (mocked engine via
  WEBSEARCH_BING_URL fixture); added metaSearch parse + graceful-degrade tests.

go test ./clients/websearch/... green; full binary builds.
2026-07-09 23:16:01 -07:00
40eb682076 feat(evals): native AI-observability dashboard at GET /v1/evals/metrics (#218)
Wire the metrics board handler and route that the completed data layer was
missing. metrics.go already had the ClickHouse ledger + GenAI-span aggregation
(assembleTotals/Series/ByModel, usageWhere, latency percentiles) and the
in-memory honest-empty path; this adds:

- metricsBoard handler: principal gate (403 without a validated tenant),
  SuperAdmin all-orgs via c.IsAdmin(), range preset -> window/bucket
  (24h|7d|30d, ?interval override), ?project threaded. nil telemetry or a
  non-default project -> honest-empty board (never 503, never fabricated).
- Telemetry.Metrics added to the interface (dsTelemetry + memTelemetry already
  implement it).
- Route registered in Mount() and the test mountApp().

All clients/eval tests pass, including the three handler tests that previously
404d (TestMetricsHandlerHonestEmpty / RequiresPrincipal / NonDefaultProjectEmpty).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-09 22:56:43 -07:00
hanzo-devandGitHub 2bff14a285 feat(eval+cli): collapse llmobs to o11y span plane + compute-ladder CLI verbs (#217)
* feat(eval): drop cloud_usage-as-observations read; collapse to o11y span plane

Step 1 of the unified AI-observability plan: the observation of record is the
o11y gen_ai span plane (/v1/o11y/observations), not a second projection of the
metering warehouse. evalsvc no longer reads hanzo.cloud_usage as 'observations'.

- Remove Telemetry.ListObservations + Observation/ObservationFilter types + the
  dsTelemetry (cloud_usage) and memTelemetry impls + the now-dead asInt64 coercer.
- Remove GET /v1/evals/observations route, its handler, and observationView/
  toObservationView. The console Observe > Observations view now reads o11y.
- cloud_usage stays the metering warehouse (read by /v1/usage + billing) — only
  the duplicate obs projection is gone. eval keeps its unique datasets/evaluators/
  runs and its own eval_traces/eval_scores tables.
- Bump ai dep to the enriched-gen_ai-span build (forward-only from v1.804.0).

Build+vet+test ./clients/eval green.

* cloud(cli): compute ladder — hanzo run/agent/bot verbs (thin /v1 clients)

Preserve in-flight CLI work: hanzo run (artifact/function), hanzo agent
(headless managed agent -> /v1/agents/:ref/run), hanzo bot (computer-using
agent -> operative/visor). Thin clients over IAM token + cloud /v1.
2026-07-09 22:55:21 -07:00
70084b91ee fix(config): accept aud=hanzo-world in defaultJWTAudiences (#216)
world.hanzo.ai is the OIDC client `hanzo-world`; IAM stamps its access
tokens with aud=hanzo-world (each app's aud is its client_id). That
audience was missing from cloud's baked identity-sanitizer allowlist, so
signed-in world tokens resolved anonymous and api.hanzo.ai returned 401.
Append the client_id (forwards-only) and add a membership + resolved-env
acceptance test.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-09 22:14:23 -07:00
1d4e5960a4 feat(commerce): embed commerce in-process as a cloud subsystem (task #89 phase 1) (#215)
Cloud now serves /v1/commerce/* + /_/commerce/* ITSELF via a new
clients/commercesvc leaf that wraps commerce.Embed's gin handler — the same
wrap-don't-rewrite fold clients/iam and clients/kms use — instead of proxying to
a remote commerce pod. deps.Commerce resolves to the in-process CommerceClient
(CommerceInProcess) when commerce is co-resident.

Why a leaf (not the upstream commerce.Mount): commerce's own Mount/init sit
behind //go:build cloud, and cloud builds without that tag, so cloud's plain
build never compiled the registration — commerce was blank-imported yet absent
from cloud.Registry (proxied at runtime). The leaf lives in-repo and imports only
commerce's cloud-free surface (Embed, api.Route, Version), so the upstream
commerce.Mount that imports cloud stays tagged out — no cloud<->commerce import
cycle. Zero go.mod/go.sum churn (commerce v1.46.40 already required).

STAGED (prod-safe): commerce joins iam/ingress in stagedSubsystems, so the
mount-all default is unchanged — the in-process cutover happens only on explicit
CLOUD_ENABLE=...,commerce. Phase 2 flips the default once validated in prod. The
remote proxy seam (CLOUD_COMMERCE_HTTP_URL / CLOUD_COMMERCE_ZAP_ADDR) is
untouched; the disabled/RPC fallbacks still compile.

GetTenantConfig is answered in-process (org + brand); CheckEntitlement fails
closed until commerce exports its subscription->plan->features resolver (Phase 2)
— the specified "cannot verify => never open" default clients/entitlements relies
on.

Proven: CLOUD_ENABLE=commerce -> GET /v1/commerce/tenant, /v1/commerce/catalog,
/_/commerce/healthz all 200 from the embedded gin engine; an unknown
/v1/commerce path returns gin's own 404 (a proxy would 502).


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-09 22:08:05 -07:00
hanzo-dev eccb271e0d Merge branch 'wt/meters' into feat/collapse-llmobs-to-o11y 2026-07-09 21:48:20 -07:00
hanzo-dev 31fdf8672e cloud(billing): usage-native meters — function GB-seconds + build minutes 2026-07-09 21:44:15 -07:00
hanzo-dev b4aeb20be4 cloud(sites): opt-in COOP/COEP cross-origin isolation for WebGL 2026-07-09 21:39:21 -07:00
hanzo-dev 79416150b2 cloud(sites): correct MIME for WebGL/WASM game assets (Unity/Emscripten/Godot)
Served sites typed .wasm/.data/.mem/.unityweb/.pck via mime.TypeByExtension only,
which returns "" for the engine payloads (empty Content-Type) and can mis-type
.wasm — breaking WebAssembly.instantiateStreaming (requires application/wasm) and
the loader's streaming fetch of Unity/Emscripten .data/.mem and Godot .pck. Pin
those in one gameAssetType map; everything else defers to the stdlib table.
Unblocks hosting WebGL game builds. Tested (TestGameAssetContentType).
2026-07-09 21:12:27 -07:00
hanzo-dev 177bc6fb9f cloud(cli): rename the native build endpoint /v1/arcd/enqueue -> /v1/runner
'arcd' is the client-side GitHub-Actions BYO product (github.com/arc-runner);
the platform's own native CI/compute pool is 'runner'. Rename the enqueue path
and de-brand the build command help/comments accordingly. Pairs with the
platform route move pages/api/v1/arcd/enqueue.ts -> pages/api/v1/runner.ts.
2026-07-09 20:19:10 -07:00
hanzo-dev e47204a5bf cloud(ci): BuildKit cache mounts on go build/test/mod — warm Go cache across builds
The 4 go invocations (mod download, sqlite double-register gate, encryption-proof
test, final build) had no cache mount, so every release recompiled the full CGO
graph (k8s + otel-collector + sqlcipher, CGO=1) from scratch on the ephemeral ARC
runner — the Build step was ~1316s/22min, dwarfing every other step. Mount
/go/pkg/mod + /root/.cache/go-build (sharing=locked) so the persistent ARC dind
BuildKit cache keeps the Go build+module cache warm. First build cold; subsequent
builds reuse compiled artifacts — target single-digit-minute rebuilds.
2026-07-09 20:03:43 -07:00
hanzo-devandGitHub d17666c384 Merge pull request #214 from hanzoai/feat/runner-ship
cloud(runner): embed the arc JIT runner as `hanzo runner`
2026-07-09 19:26:41 -07:00
hanzo-dev e95b1b4993 cloud(runner): embed the arc JIT runner as hanzo runner
Migrate arc-runner/arc (arc-archive) cmd/arcd host-role JIT GitHub-Actions
runner into hanzoai/cloud as package runner/, exposed as `hanzo runner` —
third verb of the one-binary trifecta: engine serves models, gpu connect shares
compute, runner claims CI. One org login, outbound-only, GPU/vulkan-aware.

- runner/: host-role JIT daemon (config, GitHub App auth, poll, JIT launcher,
  /v1 control surface, WSL labels). In-cluster controller role deferred.
- cli: wire `hanzo runner`; also register `engine` in controlCommands (was
  constructed but missing from the router — silently undispatchable).
- Security (red+cto reviewed): /v1 defaults 127.0.0.1 + localGuard (Host
  allowlist vs DNS-rebind, Origin vs CSRF); child env stripped of secrets;
  fork repos skipped by default (private/internal-org scoping is the real
  containment; documented honestly).
- deps: +go-github/v52, +ghinstallation/v2; luxfi/keys v1.2.0->v1.2.2 (v1.2.0
  tag mutated at origin — bump past it, verified under -mod=readonly, no bypass).
2026-07-09 19:25:05 -07:00
hanzo-dev 37ba7c6bd6 o11y: release chtraces exporter on failed Start (no leak on fail-soft path)
CreateTraces opens the ClickHouse conn + spawns the writer's ticker goroutine, so
a failed Start must Shutdown to release them rather than leak on the fail-soft
mount path.
2026-07-09 17:39:47 -07:00
6bafc5e9f4 test(kms): make dual-mount two-scope assertion load-robust (#212)
The validated-org-principal check asserted ==200, coupling this cross-package
gate test to the orgs/users/me handler's downstream success (IAM/datastore),
which flaked under full-suite parallel load. Assert the gate/shadow decision
only — admitted (not 403) and mounted (not 404) — since tenant-scoped data
correctness is proven in clients/admin/scope_test.go. Anonymous->403 and the
SuperAdmin-only platform routes are unchanged.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-09 17:32:13 -07:00
zeekayandhanzo-dev 0189d578e1 fix(embed): bust console clone layer with cloud sha (gh/ls-remote-free)
The prior two attempts (a94b3ca sha-pin via git ls-remote, c86acc3 via gh api)
both FAILED at version-compute: the ARC runner has no gh CLI, and git ls-remote
404s because actions/checkout installs a global http.extraheader carrying THIS
repo's GITHUB_TOKEN (scoped to hanzoai/cloud), overriding URL creds on the
cross-repo hanzoai/console lookup.

Bulletproof instead: no console-HEAD resolution at all. release.yml passes the
cloud commit sha as --build-arg CONSOLE_CACHEBUST (unique per push); the
Dockerfile references it in the proven `git clone --depth 1 --branch main`
RUN, so the layer cache key changes every build and re-clones console main HEAD
fresh. No gh, no ls-remote, no extraheader. Correctness over cache reuse — the
console stage rebuilds each release, but the embed is never the frozen snapshot.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-09 17:30:51 -07:00
31d0a858be fix(base+admin): base serves /v1/base/health (HealthOwner) even embed-off; dual-mount red test tracks the admin two-scope model (#211)
- clients/base: register /v1/base/health in Mount BEFORE the CLOUD_BASE_EMBED
  gate and mark cloud.HealthOwner — same always-on liveness pattern as
  clients/plan + clients/pricing. Fixes cmd/cloud TestMountAllAndServeHealth
  (base was the only listed subsystem not self-serving health).
- clients/kmssvc red_dualmount_test: #192 made the admin cockpit two-scope —
  orgs/users/me are org-scoped (guardScoped: a validated org principal is
  admitted + hard-scoped to its own org; anonymous still 403), while
  audit/roles/finance/flags/revenue stay SuperAdmin-only (guard: 403 for a
  non-admin principal). Assert both, plus kms's public /v1/kms/config never
  shadows either. No production code semantics changed — the stale test tracked
  the pre-two-scope admin-only contract.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-09 17:26:29 -07:00
zeekayandhanzo-dev c86acc3390 fix(embed): resolve console HEAD via gh api, not git ls-remote
git ls-remote failed 'Repository not found' on the cross-repo hanzoai/console
lookup: actions/checkout installs a global git http.extraheader carrying THIS
repo's GITHUB_TOKEN (scoped to hanzoai/cloud only), which overrides the URL
creds and 404s. gh api honors GH_PAT (org read) and is unaffected. Unblocks the
console-embed-freshness fix (a94b3ca) — the release aborted at version-compute
before building.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-09 17:24:26 -07:00
hanzo-dev 80281a865e o11y: dogfood ZAP Router — cloud's own spans -> in-process ClickHouse sink, no socket
cloud already embeds the o11y trace write side (chtraces), so shipping its OWN
spans over the ZAP wire to a collector that then writes the same store is pure
waste. Route them through the ZAP locality-adaptive Router (luxfi/zap v1.2.1):
when sender and sink share this binary, the Cost-0 InProcessInterface wins and
the LIVE proto batch is handed to the sink by value — zero ZAP-wire serialize,
zero socket, no second collector hop.

- clients/o11y/tracesink.go: Router + Cost-0 InProcessInterface on Destination
  "hanzo.o11y.traces"; a chtraces exporter (the REAL o11y_index_v3 writer, reused
  as a consumer.Traces — its pdata->SpanV3 conversion is unexported) fed in
  process. Handler bridges SDK-exporter proto spans -> pdata via one in-memory
  OTLP round-trip. OPT-IN (O11Y_TRACES_ZAP_INPROCESS) + fail-soft: any error
  leaves cloud's spans on the wire; can never take cloud down. NewTraceExporter +
  routerTraceClient own the transport; cmd/cloud stays the composition root.
- cmd/cloud/telemetry.go: install the ONE tracer provider over the Router
  (in-process primary, ZAP wire fallback when the sink isn't registered). Enable
  when the in-process sink is on OR a wire endpoint is set. Composition-root
  single-provider invariant (ai's GenAI tracer inherits it) preserved.
- go.mod: github.com/luxfi/zap v1.2.0 -> v1.2.1 (adds Router/InProcessInterface).

TDD: span from cloud's provider reaches the in-process handler with no wire
client and no socket; proto->pdata round-trip preserves the span; router prefers
in-process, falls back to wire on ErrNoRoute, surfaces ErrNoRoute when neither.
2026-07-09 17:21:29 -07:00
694a5b2cf5 refactor(kms): merge clients/kmssvc into clients/kms — one KMS package (#210)
The kmssvc dir was an artificial split: clients/kms is the KMS library
(embeds luxfi/kms + SecretStore + in-process client), clients/kmssvc was
the Fiber subsystem mounting /v1/kms/* — and it was ALSO package kms, dir-
named kmssvc only to dodge a dir-name collision. That svc suffix is a
workaround, not a concept.

Move every kmssvc file into clients/kms (kmssvc.go → mount.go; login.go,
env_required/kms/login_ratelimit/paas_sync/red_*/v6 tests). subsystems.go
imports clients/kms (order 10, /v1/kms/*); exactly one cloud.Register("kms").
Zero kmssvc refs remain. Full cloud build + kms package tests (incl red_*
adversarial) green. (Pre-existing clients/kms/replication test build failure
is unrelated — broken on main before this change.)

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-09 17:13:54 -07:00
hanzo-devandGitHub b16ff7d5bd org+idem: fenced single-writer + exactly-once request layer (lift replicas:1 -> N safe) (#201)
Composes the ha lease-round (v0.1.1) and vfs/replica.FencedStore (v0.6.3) into
the cloud per-org substrate, and adds the request-layer exactly-once dedup, so
per-org SQLite is safe under a multi-replica Deployment (not just replicas:1).
Four orthogonal concerns, one home each:

  internal/org/fence.go   CASFencer: the INTERIM monotone round source. A per-org
                          writer lease {round,owner} over the object store's CAS
                          (a single linearizable register); takeover strictly
                          bumps the round, renewal keeps it. Implements ha.Fencer,
                          so the Lux BFT round drops in behind the same seam later.
                          HRW is only an optimization (cuts contention); safety
                          does not depend on a fresh/agreed membership view — a
                          split view costs liveness, never safety, because the
                          fence backstops it.
  internal/org/condstore.go  MinioConditionalStore: the concrete atomic-CAS store
                          (minio If-Match against the SeaweedFS gateway), promoted
                          from the orphaned internal/writefence. satisfies
                          replica.ConditionalStore.
  internal/idem/          exactly-once request execution: request-id PK written in
                          the SAME txn as the effect (atomic dedup+effect), shipped
                          in the per-org snapshot so a retry re-routed after a
                          rolling upgrade is deduped on the successor. 'fail if
                          already done' via ErrAlreadyApplied.
  internal/org/shared.go  re-export FencedStore/ConditionalStore/Lease/Fencer/
                          Round/ErrStaleRound so the org API stays one surface.

Deletes internal/writefence (was orphaned, zero importers): its fence primitive
is promoted to vfs/replica.FencedStore (the storage substrate's rightful home),
its minio store to condstore.go — one and one way, forward-only.

Safety (no data loss + no double-exec) rests on the composition, proven by
handoff_test.go against the four hazards: (a) partition minority cannot advance
the round -> cannot write; (b) rolling-upgrade handoff -> successor CarryForwards
the predecessor's last landed write + dedups; (c) duplicate request -> idem runs
once; (d) deposed writer -> refused by election, and if it still ships, fenced by
FencedStore. Ship-before-ack: a request is 'done' only once its fenced ship lands.

Interim round source = single linearizable register (object CAS) = crash-fault
tolerant. Roadmap: replace readLease/claim with the quasar PQ-BFT agreed round
(Byzantine-tolerant, deterministic finality, 3/5 quorum) — same ha.Fencer seam,
same FencedStore admission.

Cross-repo: pins ha@fence-lease + vfs@fenced-store (pseudo-versions); retag to
ha v0.1.1 + vfs v0.6.3 once those merge. go.sum touches only ha+vfs; the
pre-existing luxfi/keys@v1.2.0 re-tag mismatch (blocks `go mod tidy` on main
today) is unrelated and untouched.
2026-07-09 16:47:19 -07:00
zeekayandhanzo-dev a94b3caedc fix(embed): re-clone console per build — stop shipping a frozen embed
The console-clone+build layer was keyed only on static text ('git clone
--branch main'), so on the persistent ARC dind BuildKit cache EVERY cloud
build re-embedded the SAME stale console snapshot. New console work — the
native Tracker module, and everything since the cache was first warmed —
silently never shipped: console.hanzo.ai/tracker rendered an old surface
with zero /v1/tracker calls even on a freshly-deployed image.

Fix (values, not places): release.yml resolves hanzoai/console main HEAD
(git ls-remote) at build time and threads it through --build-arg CONSOLE_REF;
the Dockerfile fetches that exact ref (init+fetch+checkout, sha- or branch-
capable). A changed sha moves the layer cache key, so each build embeds the
live console commit — deterministically pinned, never frozen.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-09 16:45:56 -07:00
b7c1544293 feat(admin): operator cockpit — two-scope model, flags/launch, waitlist/access, bases (#192)
admin.hanzo.ai as ONE cockpit for BOTH tiers off ONE identity predicate. scope.go decomplects the rule into a single place (resolveScope/scopedOrgs/descendants): owner==admin (c.IsAdmin, SuperAdmin) => cross-tenant, all orgs; any other validated admin => their OWN org, hard-scoped server-side. guardScoped admits a SuperAdmin OR a validated org-pinned caller and the handler scopes the data; the platform control plane (roles/audit/finance/revenue + launch/release/flags/access) stays s.guard (SuperAdmin only). me/overview/orgs/users/usage/analytics/bases are org-scoped; a non-super caller can never read another tenant for any input (their org is the sanitized, un-forgeable c.Org()).

Feature flags / launch / access via Hanzo Insights (one engine, not two): clients/featureflags is a hot-apply evaluation seam over Insights /flags (env = fallback default, 15s TTL, fail-safe degrade); /v1/admin/flags surfaces the launch switches (public_signup, waitlist_open, waitlist_access_capacity, ...) with deep-links to the Insights flag manager + activity log. /v1/admin/waitlist + /boost proxy the Base waitlist engine (server-authed, KMS secret, audited grant). /v1/admin/bases is the scoped tenant-Base panel seam (honest-empty until the Base engine is embedded).

Fix: waitlist.go shadowed the ok() envelope writer with a local bool (compile error) — renamed to configured. Tests: scope_test.go proves the two-scope invariant (super sees all; org-admin hard-pinned to own org; platform routes 403 an org-admin; users read pinned to own org); featureflags_test.go proves hot-apply + env fallback. go build ./... green; go test ./clients/admin/ + ./clients/featureflags/ green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-09 16:31:39 -07:00
hanzo-devandGitHub 94d67de1b0 feat(edge): embed the gateway CORS + per-IP rate-limit role in cloud (#181)
* feat(edge): embed the gateway CORS + per-IP rate-limit role in cloud

Fold the hanzoai/gateway edge role into cloud so it can serve api.hanzo.ai
directly, dropping the redundant KrakenD hop. cloud already validates the IAM
JWT + strips/re-mints identity headers (SanitizeIdentity), runs the per-tenant
ScopeRateLimit, and owns balance/spend-cap quota (BillingGate) — the gateway
duplicated the JWT/identity role and added only CORS + a per-IP flood cap.

Adds middleware_edge.go (package cloud), two orthogonal middlewares wired into
the serve.go chain AFTER Logger/sites and BEFORE identity:

- EdgeCORS: credentialed reflect-Origin CORS matching the gateway's policy
  (methods/headers/max-age). DEFAULT OFF (empty CLOUD_CORS_ORIGINS) so the
  shared Traefik ingress `cors-allow-all` stays the sole CORS authority on the
  recommended rollout — enabling both would double the ACAO header. Set
  CLOUD_CORS_ORIGINS only on a direct DO-LB->cloud edge. Handles + short-circuits
  the OPTIONS preflight (204) before any auth work.

- EdgeRateLimit: per-client-IP fixed-window flood cap (default 100/1s, gateway
  service-tier parity, strategy:ip) that runs BEFORE identity — the one gap
  ScopeRateLimit (keyed on the validated tenant) structurally can't see: an
  anonymous flood with no valid JWT. Keyed on the leftmost X-Forwarded-For;
  in-cluster direct callers (no XFF) are exempt, matching the standalone
  gateway's public-only scope. Opportunistic eviction keeps the bucket map
  bounded at edge IP cardinality (default ON, CLOUD_EDGE_RATELIMIT=false to
  disable). This preserves the gateway's protection rather than dropping it.

Config: CORSOrigins, EdgeRateEnabled/PerIP/WindowSec (config.go).
Tests: TestOriginMatcher, TestEdgeCORS_*, TestEdgeRateLimit_* (all green).
CGO_ENABLED=0 go build ./... + go test . green.

* feat(gateway): /v1/gateway runtime-mutable edge-policy plane

Make the embedded gateway role RUNTIME-CONFIGURABLE instead of baked into static
config: an operator/SuperAdmin retunes CORS, the per-IP flood cap, or a tenant's
rate ceiling via PUT /v1/gateway/config with NO redeploy — replacing the gateway's
image-baked KrakenD config.

clients/gatewaypolicy (leaf pkg, stdlib + hanzoai/sqlite only, no cloud import so
both the middleware and the HTTP subsystem share it cycle-free):
- Policy{CORSOrigins, PerIPRPM, WindowSec (platform), OrgRPM (per-org)} — every
  field is enforced by a consumer; no stored-but-ignored knob.
- Store: one encrypted per-tenant SQLite (gateway.db), org-keyed rows. The admin
  org row is the PLATFORM policy, layered over the static env/flag boot defaults.
  Cached resolvers Platform()/OrgRPM()/Effective() (5s TTL, fail-open to static),
  merge(base,over) makes a partial PUT additive. Fail-soft: a store-open error
  degrades to static-only (reads work, writes error) — the edge never goes down.

clients/gatewaysvc: the /v1/gateway subsystem (order 139) — GET/PUT config over
the SAME store, IAM-gated like clients/pricing/enablement.go:
- platform fields (CORS/per-IP) writable ONLY by SuperAdmin (c.IsAdmin()); routed
  to the platform row explicitly (PutPlatform) so an org-switched SuperAdmin still
  lands on it.
- per-org OrgRPM writable by the org admin (own org via principal.Tenant, never a
  raw header) or a SuperAdmin targeting ?org=<slug>.

Wiring: deps.GatewayPolicy (BuildDeps constructs it, layered over staticEdgePolicy;
serve.go closes it at shutdown). EdgeCORS/EdgeRateLimit now read the PLATFORM policy
LIVE (recompiling the CORS matcher only when the allowlist changes; the per-IP
limit/window per request). ScopeRateLimit gains the runtime per-org OrgRPM
override (most-restrictive-wins with the commerce-configured ceiling).

Tests: gatewaypolicy (static-only, platform layering, additive merge, per-org +
platform-default OrgRPM, persist-across-reopen); gatewaysvc (principal required,
org-self OrgRPM, org-admin platform 403, SuperAdmin platform, org-switched-still-
platform, empty-body 400). CGO_ENABLED=0 go build ./... + go test . green.
2026-07-09 16:25:43 -07:00
788a4aac1c feat(base): embed base app + waitlist in-process (/v1/waitlist), retire standalone superbase (#193)
Mounts /v1/waitlist/* served in-process off the embedded hanzoai/base app over
the durable cloud PVC — the in-binary replacement for the standalone superbase
pod. STAGED + fail-closed: no-op unless CLOUD_BASE_EMBED=1. Registered as the
"base" subsystem (order 60) in clients/base.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-09 16:21:02 -07:00
4b7f24356d fix(sbom): lazily create hanzo.sbom_component (datastore connects after Mount) (#207)
The datastore connects ASYNCHRONOUSLY: ai/object.InitDatastore flips
DatastoreEnabled true only AFTER Mount returns. Mount ran the CREATE TABLE
DDL only when DatastoreEnabled() was already true, so in prod it was skipped
and never retried -> GET /v1/sbom/{ref} 502 'Unknown table expression
identifier hanzo.sbom_component' while /v1/sbom/health reported datastore:true.

Add a lazy, idempotent ensureTable(ctx) guarded by a mutex+bool that latches
ONLY success (a transient failure retries; sync.Once would cache the failure).
It CREATE DATABASE IF NOT EXISTS hanzo then CREATE TABLE IF NOT EXISTS, and is
called from ingest and resolve right after requireDatastore() passes; on error
they return a retryable 503. Mount now routes its best-effort boot DDL through
the same ensureTable and is non-fatal (a Mount-time miss no longer aborts the
subsystem).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-09 15:47:23 -07:00
9ca9163309 fix(brand): map <brand>.cloud/.network hosts to the brand (white-label) (#209)
The live cloud console runs on <brand>.cloud hosts that route straight to the
cloud Service (console.lux.cloud, console.zoo.cloud, …). BrandForHostOK only
matched a brand's primary marketing Domain (lux.network, zoo.ngo), so a request
Host on lux.cloud/zoo.cloud fell through to the deployment brand — emitting Hanzo
branding on a Lux/Zoo surface (agent-skills catalogue + any Host-branded reply).
Add AltDomains per brand (lux→lux.cloud; zoo→zoo.network,zoo.cloud;
hanzo→hanzo.cloud,hanzo.app; pars→pars.ai) and match them in BrandForHostOK.
Base-URL/issuer scoping still uses the primary Domain.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-09 15:39:16 -07:00
5103936e9a feat(agentskills): serve /.well-known/agent-skills discovery, white-labeled by Host (#208)
New subsystem clients/agentskills serves the Agent Skills Discovery surface from
a catalog GENERATED by hanzoai/openapi's skills.py and embedded via go:embed:

  GET /.well-known/agent-skills/index.json        the brand's MASTER catalogue
  GET /.well-known/agent-skills/<skill>/SKILL.md   one skill document

WHITE-LABEL: the brand is decided per request from the Host (new BrandForHostOK,
mirroring platform.ts getWhiteLabelBrand) — api.hanzo.ai serves Hanzo,
api.lux.network serves Lux (lux.id), api.zoo.ngo serves Zoo — never Hanzo
branding on a Lux/Zoo surface. An unmatched Host degrades to the deployment brand
(CLOUD_BRAND), not blindly Hanzo. Order 8 registers these exact routes BEFORE
IAM's /.well-known/* wildcard (50) and the console catch-all, so they win Fiber's
first-match. Public, GET-only, no secrets.

The binary does not re-derive skills — it serves the embedded bytes, so the
sha256 digests in index.json match the served SKILL.md exactly. Only a tiny,
self-consistent `ai` fallback is committed (catalog/.gitignore); `make
agentskills` / the Dockerfile `skills` stage regenerate the FULL catalog (all 68
services × hanzo/lux/zoo) from the openapi SOT before `go build`, mirroring
webui/dist. End-to-end serve test drives the real router (index + SKILL.md +
white-label + digest + 404); cmd/cloud links clean.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-09 15:16:36 -07:00
269c1736e9 feat(sbom): global SBOM datastore + /v1/sbom ingest & resolve (#206)
Add clients/sbom: a self-contained subsystem riding the ONE shared
ClickHouse client (ai/object.Datastore*) — no second connection — that
ingests CycloneDX SBOMs from CI and serves them by image digest/ref.

The store (hanzo.sbom_component, ReplacingMergeTree) is GLOBAL/cross-tenant
by design: an SBOM belongs to a content-addressed image digest, not a
tenant, so any tenant deploying that image resolves the same component set.
Ingest is gated to the canonical cloud super-admin check (c.IsAdmin(),
owner==AdminOrg) which the build fleet carries; resolve exposes only an
image's immutable bill-of-materials (no tenant data).

  POST /v1/sbom        ingest (super-admin/CI): flatten document.components[]
  GET  /v1/sbom/{ref}  resolve by digest OR ref (FINAL dedupe, type,name order)
  GET  /v1/sbom/health liveness + datastore bool (not JWT-gated)

Registered id "sbom" order 137 with cloud.HealthOwner (binds before the ai
/v1/* catch-all at 150). Mirrors clients/analytics for structure, coercers,
and the honest-503 datastore gate.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-09 14:28:03 -07:00
bfad6f2bb6 chore(deps): pin hanzoai/ai → v1.804.0 (cookie self-heal + one super-admin rule) (#205)
Clean-semver pin bringing three ai releases into the cloud binary:
- v1.803.1 fix(account): cookie-session self-heal (#71) — a signed-in admin
  whose beego session already holds a guest u-<hash> is rebound to the
  canonical identity from hanzo_iam_token, so /v1/admin/* stops 403ing.
- v1.804.0 refactor(authz): ONE super-admin rule — membership in the `admin`
  org (owner == AdminOrg); drops the configurable globalAdminOrgs + built-in.
  Matches cloud's clients/admin (isSuperAdmin canonical, isGlobalAdmin alias)
  and the console isSuperAdminAccount gate.

Supersedes the pseudo-version pin (#197) and the intermediate v1.803.1 pin
(#198, closed). Verified: go build -tags "libsqlite3 sqlite_fts5" ./cmd/cloud.


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-09 13:46:20 -07:00
9faa0a9ef7 feat(platform,sites): pure-Go zip/tar.gz static-site upload + custom-domain serving (#204)
* feat(platform,sites): pure-Go zip/tar.gz static-site upload + custom-domain serving

Adds a self-service static-site deploy to the unified cloud binary's PaaS
surface and lets the site edge serve a customer's own domain from S3.

- projects: walkArtifact accepts a ZIP (archive/zip) as well as tar(.gz),
  sniffed by magic bytes; one deploy contract (index.html at root, same size
  and traversal guards), three container formats. A single wrapping top-level
  directory (a zip made from a project folder) is stripped so index.html lands
  at the root.
- projects: the deploy handler reads the artifact from a multipart file upload
  (a browser <input type=file>) OR the raw request body (a curl one-liner).
- sites: the host edge now serves a bound CUSTOM domain (a customer apex/host
  pointed at this edge) from that project's S3 prefix, resolved by the full
  host. Only external hosts (never one of our self domains) with a LIVE binding
  are served; every other host — our api/console hosts, or an unbound host
  routed here — Continues to the normal pipeline, so the API path pays no
  per-request lookup and a customer binding can never shadow a real Hanzo host.
- projects: POST/GET .../domains binds and lists a site's custom domains
  (admin-gated until DNS-ownership verification is wired here).
- surface: the static engine is exposed under /v1/platform/sites/* (the PaaS
  namespace) in addition to /v1/projects/*, so the one user flow is create a
  site -> upload a zip -> bind a domain -> live.

Pure Go, CGO-off. New unit tests cover the zip walker, format dispatch,
single-root strip, custom-domain routing (served/passthrough/self-host/not
-live), hostname validation, and host binding.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>

* feat(projects): authorize custom-domain binding by the platform-operator org

A custom-domain bind is authorized for a global admin OR the platform-operator
org (the deployment's brand org, env CLOUD_PLATFORM_OPERATOR_ORGS, default the
brand). The operator manages customer DNS until per-tenant DNS-ownership
verification is wired here. Safe because a bound domain is inert until its owner
points DNS at this edge — the real gate is DNS control.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>

---------

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-09 13:12:56 -07:00
de85b94790 debrand: signoz → o11y (branding) + repoint collector module to hanzoai/otel-collector (#202)
* debrand: signoz -> o11y (branding) + repoint collector module

Drop "SigNoz"/"signoz" where it is BRANDING (comments, docs, prose) to o11y,
and repoint cloud's direct collector import to the renamed module.

- go.mod/go.sum: github.com/hanzoai/signoz-otel-collector v0.144.6 (direct)
  -> github.com/hanzoai/otel-collector v0.144.7 (direct). The old module stays
  as an // indirect dep because hanzoai/o11y v1.5.2 (separate repo, out of
  scope) still imports it; the fork also pulls upstream
  github.com/SigNoz/signoz-otel-collector v0.144.5 // indirect.
- clients/o11y/ingest.go: chlogs/chtraces imports -> otel-collector.
- Branding prose swapped to o11y in telemetry.go, subsystems.go, embed.go,
  logs.go, metricsread.go, scope.go, ingest_test.go, agents.go, LLM.md,
  docs/consolidation.md.

KEPT (not branding):
- ClickHouse schema read by cloud (written by the deployed collector):
  signoz_traces / signoz_logs / distributed_signoz_index_v3, severity_text
  columns. Renaming reads without migrating the live schema breaks them; the
  v0.144.6->.7 patch bump does not migrate table names.
- Upstream API in hanzoai/o11y/pkg/signoz: import path, alias, type SigNoz,
  and signoz.New / SigNoz.Start references (that repo is debranded separately).
- Upstream package names: signozclickhousemetrics.
- Honest attribution: "SigNoz's dd-sketch fork of ch-go".

Build: go build ./... = 0, go vet = 0, go test ./clients/o11y ./clients/admin
= ok. go mod tidy is blocked by a pre-existing luxfi/keys@v1.2.0 go.sum
checksum mismatch (identical on origin/main) -> CI-authoritative.

* o11y: read o11y_* ClickHouse tables + bump collector to v0.144.8

Direct ClickHouse table reads renamed signoz_* -> o11y_* to match the
o11y read plane and the data-preserving RENAME migration (hanzoai/o11y#28):

  o11y_traces.distributed_o11y_index_v3  (was signoz_traces.distributed_signoz_index_v3)
  o11y_logs.distributed_logs_v2          (was signoz_logs.distributed_logs_v2)

Files: clients/o11y/logs.go, clients/o11y/metricsread.go,
clients/o11y/ingest.go, clients/admin/o11y.go (+ o11y_test.go).

Bump github.com/hanzoai/otel-collector v0.144.7 -> v0.144.8 (writer side
now CREATEs/WRITEs the same o11y_* physical schema). Collector go.mod is
unchanged between the two tags (identical go.mod hash) — pure source
rename, so the module graph is unchanged; go mod tidy left to CI
(pre-existing luxfi/keys tidy block is CI-authoritative).

go build ./... = 0. clients/admin + clients/o11y tests green (SQL
assertions now match o11y_* target names). Lockstep deploy: collector
v0.144.8 -> o11y#28 RENAME migration -> o11y+cloud readers.

* cloud: embed o11y v1.5.4 — version-less /v1/o11y + o11y_ schema reads + debrand

Bumps hanzoai/o11y v1.5.2→v1.5.4 (version-less surface + o11y_ ClickHouse table
reads + the signoz→o11y debrand) and repoints embed.go to the renamed runtime
package pkg/signoz→pkg/o11y (type SigNoz→O11y). Pairs with otel-collector v0.144.8
(writes o11y_) + the lockstep cutover migration. go build ./... = 0.

---------

Co-authored-by: hanzo <z@hanzo.ai>
2026-07-09 13:12:34 -07:00
904d4e5dcd deps: bump hanzoai/o11y v1.5.1 -> v1.5.2 (o11y /v1/o11y path normalizer) (#199)
Embeds hanzoai/o11y#26: the mount normalizes the /v1/o11y/<resource> public
contract onto the internal SigNoz /api/vN routes (kills the /api/ leak, fixes
the llmobs /v1/o11y/* 404). Pairs with the cloud CR O11Y_GLOBAL_EXTERNAL__URL=""
change (universe#461) — deploy together.

Co-authored-by: hanzo <z@hanzo.ai>
2026-07-09 08:57:35 -07:00
zeekayandhanzo-dev 08970ab1d3 fix(identity): read hanzo_iam_token cookie — unbreak embedded-console org-scoped /v1
The ai (casibase) layer SETS the httpOnly hanzo_iam_token cookie (the IAM JWT) after
login (ai/controllers/account.go iamTokenCookieName), but cloud's own identity
middleware only read [iam_access_token, access_token, hanzo_token] — NOT
hanzo_iam_token. So the embedded console (browser holds ONLY that cookie, no
Authorization header) resolved to no validated principal → every org-scoped /v1
endpoint (agents, gpus, machines, platform, orgs, entitlements, …) 403'd
'X-Org-Id required', and modules rendered empty. Add hanzo_iam_token (first) to
cookieTokenNames so cloud reads the SAME cookie the ai layer sets → validates the
JWT → X-Org-Id from owner → org-scoped surfaces authorize. Verified: the JWT is
present in the browser (1533-char httpOnly hanzo_iam_token); only the name was wrong.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-09 02:28:51 -07:00
df11066933 chore(deps): bump hanzoai/ai → e1c4cde0 (cookie-session self-heal, #71) (#197)
Pulls the get-account cookie-path fix (hanzoai/ai#81): a signed-in admin
whose beego session already holds an anonymous guest u-<hash> is now
self-healed from the hanzo_iam_token credential to its canonical identity,
so /v1/admin/* stops 403ing under the console cookie session. Verified:
cloud binary builds with -tags "libsqlite3 sqlite_fts5".


Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-09 00:00:57 -07:00
cd790a228b org: elect via hanzoai/ha; vfs/replica stays replication-only (#196)
internal/org/shared.go split its aliases along the real seam: election
(Member/Owner/IsOwner/Replicas) now re-exports github.com/hanzoai/ha; the
Replicator/Store/DB/DBPath stay github.com/hanzoai/vfs/replica. membership.go
and cipher.go are unchanged (the alias types line up: org.Member = ha.Member).
writefence doc updated to name ha as the election primitive.

One concern, one home: who-writes (ha) vs how-state-ships (vfs). No behavior
change; internal/org + writefence pass with -race.

NOTE: `go mod tidy` is blocked in this repo by a PRE-EXISTING, unrelated
luxfi/keys@v1.2.0 go.sum checksum mismatch; ha was added via `go get` +
marked direct by hand. Re-run tidy once that pin is fixed.

Co-authored-by: hanzo <z@hanzo.ai>
2026-07-08 23:11:34 -07:00
zandGitHub d7e578f475 Merge pull request #195 from hanzoai/feat/ai-login-manager
feat(cloud): analytics.datastore entitlement gate for paid per-org usage analytics
2026-07-08 17:20:58 -07:00
hanzo-dev c7b547ce22 feat(cloud): analytics.datastore entitlement gate for paid per-org usage analytics 2026-07-08 17:01:45 -07:00
hanzo-dev 67e48170bc test(cli): uploadOutputs takes active org 2026-07-08 16:38:10 -07:00
hanzo-dev 95e60faae1 Merge commit '02043ca' into HEAD 2026-07-08 16:37:47 -07:00
hanzo-dev bade567e65 harden(build): -mod=readonly + GOSUMDB-on + digest-pinned bases (RED money-image gate)
Per RED's conditional GO on the encryption image:
- GOFLAGS -mod=mod -> -mod=readonly: the committed go.sum is the SOLE source
  of truth; any needed-hash drift FAILS the build instead of silently
  re-recording an unverified hash. Verified go.sum is complete + sumdb-
  consistent (go build/download clean with GOSUMDB ON).
- Drop GOSUMDB=off: a money image must not blanket-disable the checksum
  database. GONOSUMDB scoped to zap-proto/* only (first-party-direct).
- Digest-pin the three base images (node:24-alpine, golang:1.26-alpine3.22,
  alpine:3.22) @sha256 for a reproducible money image.

The ldd/readelf link proof stays belt-and-suspenders behind the ciphertext
proof (verified at build time on the musl image).
2026-07-08 16:16:07 -07:00
hanzo-dev 4c3c1053fb fix(console-sec): rate-limit keys on validated principal, not spoofable XFF (RED)
The limiter guards the OFF-GATEWAY path where nothing trusted stamps
X-Forwarded-For, so keying on the client-settable XFF let an attacker send a fresh
value per request and reset the 30/min bucket at will. These are all post-auth
money-write routes, so key on the un-spoofable VALIDATED principal (X-Org-Id/
X-User-Id, minted by SanitizeIdentity from a verified JWT); fall back to the socket
peer (L4 RemoteAddr) for an unauthenticated request (which the handler 403s anyway).
Test now rotates XFF during the flood (must NOT reset) and asserts a second principal
keeps its own bucket.
2026-07-08 16:14:38 -07:00
hanzo-dev 1e2f549034 feat(console-sec): CSRF token on money writes + per-IP rate limit (RED hardening)
Cloud-direct/off-gateway money path loses the edge WAF/limiter and the embed
session-bridge's Sec-Fetch-Site gate passes VACUOUSLY when Origin/Referer/SFS are
all absent (RED). Adds two positive controls to the console write surface:

CSRF (csrf.go) — GET /v1/console/csrf issues a token bound to the validated
principal (X-User-Id+X-Org-Id), MAC'd with keyed BLAKE3 (luxfi/crypto,
blake3.KeyedHash) under a server-only KMS key (CONSOLE_CSRF_KEY; ephemeral
per-process fallback). requireCSRF enforces X-CSRF-Token on the AMBIENT-cookie path
ONLY (no Authorization + a Cookie present) — Bearer/Basic/gateway/API callers are
immune to CSRF and skip it, so nothing non-browser breaks. A cross-site page cannot
read the same-origin token (SOP) nor set the custom header (no CORS preflight
granted), and the token is identity-bound so it can't be replayed as another user.

Rate limit (ratelimit.go) — per-IP token bucket (30/min) on mint/rotate/revoke key
+ wallet top-up; distinct from commerce's spend-cap, restores frequency protection
lost off-gateway. Keyed on XFF first-hop.

Wraps POST/DELETE keys, POST onboard, POST topup, POST billing, POST/PUT/PATCH/DELETE
commerce. Reads stay open. Tests: ambient-no-token 403, valid-token 200,
cross-identity replay 403, Bearer-skips-CSRF 200, rate-limit 429; existing suites
unchanged (no cookie ⇒ CSRF skipped). luxfi/crypto for the MAC (NOT stdlib/JWT).

Needs arcd build + RED review; CONSOLE_CSRF_KEY to be provisioned via KMS for
restart/multi-replica-stable tokens (coordinate ac742480).
2026-07-08 16:14:38 -07:00
hanzo-dev 5b8e4b7daa fix(cloud-direct): stamp X-User-Name; mint hk- keys by owner/username not owner/uuid
The in-binary direct-Bearer path (console SPA -> cloud, gateway bypassed) stamps
X-User-Id = the JWT subject (a UUID) via idClaims.userID(). The console key ops
built the IAM id as <owner>/<X-User-Id> = hanzo/<uuid>, but IAM's mint-user-keys /
get-user resolve only <owner>/<name> (hanzo/z) -> 'password or code is incorrect'
-> hk- mint 502 on the cloud-direct path. (The gateway path worked because the
gateway minted X-User-Id == username.)

Fix (narrow blast radius, per RED-preferred approach — does NOT reorder userID()):
- idClaims.username(): the IAM username (name claim, then preferred_username),
  NEVER the subject.
- SanitizeIdentity stamps X-User-Name from the validated username, DISTINCT from
  X-User-Id. X-User-Name is already an authorityHeader (stripped on ingress,
  re-injected only from validated claims -> forgery-proof).
- resolveCaller carries a distinct caller.username (X-User-Name, fallback to
  X-User-Id for the gateway path); new caller.keyID() = <owner>/<username> is used
  ONLY by the user-key ops. caller.id / caller.name are UNCHANGED, so the
  billing/topup/commerce subjects are byte-identical -> zero money-path impact.

Tests: TestKeys_DirectBearerPath_MintsByUsernameNotUUID (mint targets hanzo/z, not
the UUID), TestSanitizeIdentity_StampsUserName, TestSanitizeIdentity_UserNameForgeryStripped;
existing key + identity suites unchanged (gateway path falls back to owner/name).

Needs arcd/CI image build (no local docker) + RED review before deploy.
2026-07-08 16:14:38 -07:00
hanzo-dev 68f0becf1c build: CGO=1 + libsqlcipher encrypted image (was CGO=0 PLAINTEXT) + KAT gates
The unified binary embeds IAM's per-org SQLCipher store and commerce's
per-tenant money DBs; the prior CGO_ENABLED=0 build shipped pure-Go
modernc — PLAINTEXT at rest. Rebuild CGO=1 against system libsqlcipher
(hanzoai/iam's proven recipe: libsqlite3 tag + libsqlcipher symlink +
-DSQLITE_HAS_CODEC), runtime base scratch -> alpine:3.22 + sqlcipher-libs
(CGO needs libc + the codec .so).

Baked-in RED gates (a failing gate = NO image):
- modernc double-registration guard: 0 modernc in the CGO=1 ./cmd/cloud
  graph (the one 'sqlite' driver is mattn/SQLCipher).
- TestEncryptionProof: real ciphertext-at-rest under SQLITE_REQUIRE_CODEC=1.
- cek.go golden-vector KAT (TestUnwrapGoldenFixture + round-trip): a frozen
  pre-luxfi-swap 61-byte DEK sidecar still decrypts under the shipped
  luxfi/crypto-AEAD code — existing encrypted stores stay readable.
- readelf/ldd link proof: the binary binds sqlite3_* to libsqlcipher, never
  a plaintext libsqlite3.

Console embed stage unchanged (same-origin console). RED must review before
the image ships.
2026-07-08 16:03:57 -07:00
hanzo-dev f4f271b39e deps: converge SQLite onto one driver — bump sqlite/orm/base/commerce/o11y/replicate
Bump the six hanzo modules to their driver-converged releases so the CGO=1
unified binary has EXACTLY ONE database/sql 'sqlite' registration
(mattn/SQLCipher), ending the 'sql: Register called twice for driver
sqlite' panic:
  sqlite     v0.1.5   -> v0.2.3   (SetPersistWAL + OpenPragma primitives)
  orm        v0.5.2   -> v0.6.1
  base       v1.4.6   -> v1.5.7   (+ replicate v0.9.5, the last modernc leak)
  commerce   v1.42.29 -> v1.46.40 (+ go:embed plans fix)
  o11y       (pseudo) -> v1.5.1
  replicate  v0.8.0   -> v0.9.5

Retarget the mattn v2.0.3+incompatible replace v1.14.16 -> v1.14.47 (the
SetFileControlInt/SQLITE_FCNTL_PERSIST_WAL-capable version hanzoai/sqlite
v0.2.3 needs for SetPersistWAL).

Verified CGO=1: 0 modernc in the ./cmd/cloud dep graph; the 517MB binary
builds and boots (--help) with NO double-register panic.
2026-07-08 16:00:21 -07:00
hanzo-dev 02043cacea gpu connect: materialize uploaded inputs + route output to active org
studio.render now (1) writes any inputs shipped with the job into the
local studio input dir via its own /upload/image, so an uploaded photo
(which lives in orgs/{org}/input on the dispatching pod, unreadable here)
resolves for LoadImage before the render; and (2) forwards the job's
active org as the studio_active_org cookie on /upload/output, so the
finished render lands in that org's gallery even when the worker token's
home org differs (a@hanzo.ai home=hanzo, rendering for karma).
2026-07-08 15:50:46 -07:00
hanzo-devandGitHub dcff8ab973 feat(cloud): SuperAdmin canonicalization + per-org entitlements API (#194)
SuperAdmin: /v1/admin/me and /v1/admin/users now emit the canonical
`isSuperAdmin` key alongside the deprecated back-compat alias `isGlobalAdmin`
(both populated with the SAME fact — owner == AdminOrg). The console may read
either during the rename migration and sees the same truth. No DB change: the
signal was always a derived boolean, never a stored column.

Entitlements: new clients/entitlements subsystem (order 139) — the per-org
product-enablement plane the console's paid-product sidebar reads.

  GET  /v1/orgs/:org/entitlements  -> { "enabled": [...] }
  POST /v1/orgs/:org/entitlements  { add?, remove? } -> { "enabled": [...] }

Two authorities, never braided: ENABLEMENT (this store: durable per-tenant
SQLite, (org,product) key, settings-store discipline) vs ENTITLEMENT (commerce:
deps.Commerce.CheckEntitlement at write time). A non-super-admin may only enable
a product the org's plan already grants (402 otherwise); disabling is never
gated; a super admin bypasses the commerce gate and may target any :org. Org
scoping mirrors clients/kms: :org must equal the validated owner claim unless the
caller is a super admin; a bearer-less forge fails the principal gate (403).

Tests (TDD, all green): store tenant-isolation + all-or-nothing Apply;
forged-request 403; cross-org 403; malformed org/product 400 (commerce not
consulted); entitled enable 200; unentitled enable 402 (nothing persisted);
super-admin bypass 200 (commerce not consulted); nil-commerce member-add 503
(fail-closed); remove never gated; empty mutation 400. Plus admin_test asserts
isSuperAdmin present and equal to isGlobalAdmin on both /me and /users.
2026-07-08 15:12:41 -07:00
zandGitHub 15c51ebea6 Merge pull request #191 from hanzoai/feat/world-pricing
feat(world): plan enforcement contract + GET /v1/world/limits
2026-07-08 13:55:41 -07:00
hanzo-dev 6ed1484419 chore: pin hanzoai/plans to released v1.4.0 2026-07-08 13:55:37 -07:00
hanzo-dev 35209eec56 feat(world): plan enforcement contract + GET /v1/world/limits
Bumps @hanzo/plans to the World-pricing catalog (world-enterprise tier +
world.model_api gate) and adds the single-sourced enforcement contract for
the /v1/world data plane.

- clients/plan: export Entitlements(ctx, id) — the one Go seam to read a
  plan's canonical entitlement block from the @hanzo/plans catalog (runs the
  bundle 'entitlements' route; no data duplication, no fromLegacy re-impl).
- clients/world/entitlement.go: WorldLimits + WorldLimitsFromEntitlements
  (pure) + ResolveWorldLimits(ctx, planID) — values sourced from world.*
  entitlements, never hardcoded. FreeWorldLimits is the fail-closed floor
  (catalog outage degrades to Free, never grants model/stream).
- GET /v1/world/limits?plan=<id>: machine-readable contract echo so agents/
  dashboard self-config against the live catalog instead of hardcoding tiers.
- Tests: contract mapping (all tiers), fail-closed on unmounted catalog, and
  end-to-end Entitlements against the real embedded bundle (world.model_api
  present on pro/enterprise, absent on free).

Per-request enforcement (org->plan resolution + rate limiter wiring) is the
documented follow-up owned with feat/world-model-engine; both gates resolve
through ResolveWorldLimits so policy stays single-sourced.
2026-07-08 13:49:29 -07:00
faf7adec55 fix(wallets): retry Safe deploy on 'wallet not found' (same ring commit race, DRY) (#190)
Proxy capture of cloud->ring proved the Safe flow hits the ring's commit-after-
response read-after-write race TWICE, not once: createVault->createWallet ('vault
not found', already retried) AND createWallet->deploy ('wallet not found', which
502'd custody=safe). With ALL requests pinned to one node (via a debug proxy) the
deploy STILL 404'd, so it is a Postgres commit-visibility lag, not node affinity.

Extract doRetryNotFound(...notFound) (bounded 6x/250ms linear, ctx-aware, fail-fast
on any other error; do() only unmarshals on 2xx so out is safe across retries) and
use it for BOTH createWallet ('vault not found') and deploySafe ('wallet not
found'). go test ./clients/wallets/... green; cmd/cloud builds.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-08 13:42:44 -07:00
7776439ed5 fix(kmssvc): require explicit env on secret writes (no silent default) (#189)
The embedded KMS write path POST /v1/kms/orgs/{org}/secrets defaulted a
missing env to "default" (envOr), committing the write to a bucket that
project/env/path readers (the kms-operator, cluster syncs) never resolve.
That split is what let an IAM z-password land in env=default while prod kept
serving the stale value. env is a first-class component of the storage key
(kms/secrets/{path}/{env}/{name}) and cannot be aliased, so a write with no
env now fails loud (400). GET/DELETE/LIST keep the envOr compat default (a
read/delete can't plant a value another reader trusts; legacy readers that
omit env must keep working). No PATCH route exists on this surface.

Regression tests: write without env -> 400 (and lands nowhere); write
env=prod is readable via the operator's project/env/path resolution (sha256
round-trip, values never printed) and is not visible in env=default. The
fail-closed-without-master-key test now sends a valid env so it still
exercises the 503 master-key gate rather than 400-ing on input.


Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-08 12:04:34 -07:00
hanzo-dev 48e77f643a feat(config): additive CLOUD_ENABLE_STAGED lever — activate staged subsystems without an allowlist
Enabled() staged path is now orthogonal to the Enable allowlist: a staged
subsystem (iam/ingress) mounts when named in EITHER Enable (strict allowlist) OR
the new EnableStaged (additive). CLOUD_ENABLE_STAGED=iam + empty CLOUD_ENABLE =
all-non-staged prod default PLUS iam — the faithful iam-fold canary/cutover shape
with NO hand-enumerated allowlist that silently drops a newly-added subsystem.

Proven: TestEnabled_StagedActivatesAdditively (iam on, non-staged default intact,
unnamed staged sibling stays off).
2026-07-08 12:01:06 -07:00
7d33041580 deps: bump hanzoai/ai -> #71 session-resolution fix (ai#79); iam stays v1.31.18 (#188)
Completes the #71 auth repair as a clean dep bump (the fix lives in ai + iam, not
the cloud tree):
- github.com/hanzoai/ai v1.802.0 -> v1.802.1-0.20260708185316-0321c35877f0
  (ai#79 0321c358: self-heal get-account identity — stop degrading real logins to
  u-<hash> guests; fail-closed 401). Pseudo-version pins the commit while the
  semantic-release patch tag mints (1 commit ahead of v1.802.0).
- github.com/hanzoai/iam v1.31.18 already pinned in main (iam#109 fail-closed
  guest-mint gate) — MVS keeps it over ai's older iam pin.
go mod tidy added the authentic gopsutil/v4 transitive hashes (iam util); go mod
verify OK; -mod=readonly CGO_ENABLED=0 go build ./cmd/cloud green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-08 11:58:25 -07:00
hanzo-dev 2e2f8f2a1e fix(iam): isolate embedded IAM's SQLite via IAM_DATABASE_URL — unblock iam+ai co-residence
Both the iam and ai casdoor-derived forks resolve their SQLite handle from the
SAME env key (dataSourceName) + the one beego web.AppConfig global. A deployment
sets dataSourceName for ai; with iam enabled, IAM's bootstrap resolved that same
value and xorm-opened ai's DB (auto-migrating casdoor tables into it) -> the
documented boot crash that pinned every post-embed release and kept iam staged.

IAM's conf already honors an IAM-scoped IAM_DATABASE_URL above the shared
dataSourceName; pin it to IAM's own iam.db under DataDir so the two forks get
independent stores, order-independent, NO fork edit. Operator override respected.

Unit-proven: TestIsolateDatabase (iam-owned DSN, != ai dataSourceName, respects override).
2026-07-08 11:55:47 -07:00
hanzo-dev ed706d5356 Merge fix/embed-session-principal-bridge: embed session→principal bridge + RED H2/H3 + iam v1.31.18 (H1)
Makes console.hanzo.ai (go:embed console) authenticate its money surfaces: the
first-party IAM session cookie → validated principal (sessionAccessToken → v.validate),
RED-hardened (H2 Secure cookie, H3 same-origin bridge gate), on iam v1.31.18 (H1
session-regeneration + iam-main security fixes). Pairs with console v8.4.122 which
addresses billing/commerce/keys at the canonical bare /v1 in embed mode.
2026-07-08 10:13:34 -07:00
hanzo-dev ae01163d6d deps(embed): pin hanzoai/iam v1.31.18 — H1 session-regen + iam-main security fixes ∪ InitEmbed
v1.31.18 is iam main (guard-leak stamp #108, guest-signin fail-closed #109, capauth
PermAttenuate #106) UNION the InitEmbed line UNION RED H1 (SessionRegenerateID on the
sign-in transition). The prior embed pin v1.31.17 had diverged off an old base and was
MISSING those iam-main security fixes, so pinning v1.31.17+H1 would have shipped the
money embed without them. v1.31.18 ships H1 + guard-leak/guest/capauth + InitEmbed
atomically into this binary's in-process IAM. Transitive indirect bumps (purego/
plan9stats/locafero/gopsutil-v4/viper/tidwall-match) are MVS-driven by iam v1.31.18.
2026-07-08 10:13:16 -07:00
hanzo-dev 3f7f238db7 auth(embed): RED fixes H2+H3 on the session→principal bridge
H2 (HIGH) — pin the IAM session cookie Secure. clients/iamsvc/iamsvc.go derived
Secure from web.BConfig.Listen.EnableHTTPS, which is FALSE (the binary listens plain
:8000 behind the TLS-terminating ingress) → the session cookie shipped non-Secure.
The embed bridge turns that opaque sid into a money bearer (hk- mint, balance/top-up),
so a non-Secure cookie is capturable off any plaintext leg and replayable. Pinned
Secure: true (the deployed edge is always HTTPS).

H3 (MED-HIGH) — gate the ambient-cookie bridge to same-origin. billing.go/commerce.go
forward GET verbatim to commerce; a SameSite=Lax cookie still rides a top-level GET, so
a cross-site link could drive the victim's own money action if any commerce GET mutates.
validatedPrincipal now fires the session bridge ONLY for a same-origin request
(sessionBridgeSameOrigin: Sec-Fetch-Site same-origin|none, else Origin/Referer
host==Host) — refusing cross-site AND sibling-subdomain (same-site). Bearer/JWT-cookie
paths (non-ambient) are unaffected. +TestSessionBridgeSameOrigin (7 cases) green.

REMAINING for money: H1 (session-fixation — SessionRegenerateID on the IAM sign-in
transition) lands in hanzoai/iam (compiled into this binary); coordinating.
2026-07-08 09:55:08 -07:00
hanzo-dev 56a0405d92 auth(embed): bridge first-party IAM session cookie → validated principal
The go:embed console (console.hanzo.ai → cloud:8000) authenticates against the
in-process IAM, which sets an OPAQUE, httpOnly session cookie (cloud_session_id)
and stores the user's IAM-minted access-token JWT SERVER-SIDE against that session.
The console's Next BFF token-minting routes are stripped by the static export, so a
browser request to a cloud-native route (/v1/console/keys, /v1/billing/*) carries
only the session cookie — no bearer — and validatedPrincipal refused it, 401ing
every authenticated surface (API keys, billing, every product page = shell).

validatedPrincipal now resolves that session cookie to the server-stored access
token (sessionAccessToken via web.GlobalSessions) as a LAST RESORT (after Bearer/
Basic/JWT-cookie), then validates it through the SAME v.validate (sig/iss/aud/exp).
Identity is bound to the VALIDATED session: the client holds only an unguessable,
httpOnly sid; the session never asserts identity itself. No-op on gateway-fronted
binaries (a bearer is present) and on binaries with no IAM session manager
(web.GlobalSessions == nil) — tested. CSRF: cloud_session_id is SameSite=Lax, so a
cross-site request never carries it; and cookieTokenNames already establishes cloud's
JWT-cookie auth posture. This is the v8.4.5-flagged 'set the cookie the sanitizer
looks for' path, done cloud-side from the session store (no cross-repo IAM release).

RED review requested before it fronts money (session-fixation / CSRF surface).
2026-07-08 09:55:08 -07:00
ea232dbcd7 feat(automations): waitlist points connectors — x/discord verify + award_points seam (#187)
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-08 09:21:54 -07:00
57200605cd fix(console): default HANZO_CHAIN_ID to genesis-canonical 36963 (#186)
topupConfig defaulted to the placeholder 36900; align to the
genesis-canonical Hanzo mainnet chain id 36963 (lux/genesis, and the
rest of cloud clients/treasury+wallets already use 36963). Still
env-overridable via HANZO_CHAIN_ID.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-08 09:01:20 -07:00
hanzo-devandGitHub a84180c542 refactor(cloud): one /v1/o11y owner (embedded runtime); split settings out of observe (#185)
The observability surface was mounted THREE ways over the same /v1/o11y/* paths:
clients/observe (order 44), clients/o11y's o11yscope (order 69), and the
hanzoai/o11y wildcard runtime (70/71). observe also served /v1/settings/:product,
which is console product config, not observability. Collapse to one and one way.

- ONE owner of /v1/o11y/{logs,metrics,status}: clients/o11y's o11yscope (order 69).
  observe's richer logic is folded IN so nothing is lost — the REAL per-org RED
  metrics + LLM usage (metricsread.go, was a stub in o11y) and the two-view logs
  (admin infra stdout / per-org request-from-traces). Tenant isolation preserved:
  org is principal.Tenant bound as a positional ClickHouse param, the product is
  shape-validated → alias-mapped (console slug → workload) → allowlisted
  (knownServices, SSRF/injection boundary). observe's productAlias merged into
  resolveService so no product loses data. Admin god-view gates on c.IsAdmin()
  (== owner=="admin" SuperAdmin after SanitizeIdentity), never a per-org isAdmin.

- /v1/settings/:product moved OUT of observe into clients/settings (it is NOT
  observability). Behavior/contract preserved verbatim from observe: {config,
  secretKeys} shape, KMS ref orgs/{org}/settings/{product}/{key}, (org,product)
  store isolation, secrets-to-KMS-or-fail-closed. Replaces the orphaned, divergent
  clients/settings stub with the live behavior and wires it in (order 138).

- /v1/query does not exist (no registrant, no consumer) — nothing to fold.
  /v1/observe/health was the auto-derived GET /v1/<id>/health for id "observe";
  it vanishes with the subsystem (o11yscope gets /v1/o11yscope/health; the runtime
  serves its own gate-exempt /v1/o11y/api/v*/health*). Both documented.

- DELETE clients/observe; drop its import; add clients/settings; fix the stale
  subsystems.go o11y comment ("reverse proxy to the dedicated o11y Deployment" →
  the embedded reality: scoped reads 69 + in-process runtime 71 + OTLP ingest 72).

Net -1037 LoC. cmd/cloud + cmd/hanzo build; clients/o11y + clients/settings tests
pass (20/20), covering tenant isolation, secrets-never-plaintext, product
validation, alias resolution, and route precedence over the wildcard proxy.
2026-07-08 08:58:18 -07:00
674b599833 refactor(cloud): canonical subsystem names — drop svc suffixes, one noun per capability (#184)
One canonical short-noun name per subsystem (registered name + Go package dir +
route). No public route breaks: renames that change a live /v1 prefix keep the
old route as a back-compat alias (mount both, same handlers).

Renamed (internal-only, route unchanged):
  usagesvc  -> usage        (register string)
  zt        -> zero-trust   (register string; pkg dir kept `zt`, routes /v1/networks|mesh|edge unaffected)
  auditlog  -> audit        (register string; route already /v1/audit)
  s3        -> storage       (dir+pkg+register; route /v1/s3 kept — route-safe)
  tasksvc   -> tasks         (dir+pkg; register already `tasks`)
  iamsvc    -> iam           (dir+pkg; register already `iam`)
  mpcseal   -> mpc           (internal lib dir+pkg; importers repointed, local alias kept)
  gojahost  -> goja          (internal lib dir+pkg; importers repointed)

Renamed with public route + back-compat alias:
  kb        -> knowledge     (dir+pkg+register; canonical /v1/knowledge added, /v1/kb alias kept; framework module id `kb` retained = data-model id)

Log "subsystem" labels aligned to canonical names. Subsystem test enable-lists
and stale clients/<old> path comments updated. stagedSubsystems already
canonical ({iam,ingress}).

Held (route collisions — CTO decision):
  ml -> models       COLLIDES /v1/models (OpenAI-compat catalog owns it) — kept `ml`
  websearch -> search COLLIDES /v1/search (provisioned search resource) — kept `websearch`
  kmssvc dir         register+route already canonical (`kms`, /v1/kms); dir kept
                     to avoid colliding with the `clients/kms` SecretStore core lib.

Not present in repo: gatewaysvc / gatewaypolicy (gateway is a separate deployment).

Build: CGO_ENABLED=0 go build ./... green; go vet green; tests green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-08 08:55:27 -07:00
hanzo-devandGitHub 873118d77c refactor(cloud): decomplect subsystem registration — Typed adapter, OwnsHealth flag, generic pick[T], clean ids (#183)
One coherent change to the subsystem-registration layer. Concrete types over
`any` at the call sites, indirection deleted, generics only where they remove
real duplication.

CHANGE 1 — kill the per-subsystem `any`-unwrap boilerplate
  Every in-repo subsystem's init() hand-wrote the identical
    func(app any, deps cloud.Deps) error { a, ok := app.(*zip.App); if !ok {…}; return Mount(a, deps) }
  Add ONE adapter, cloud.Typed(func(*zip.App, Deps) error) MountFunc, that does
  the *zip.App recovery in a single place (fail-closed, never panics). All ~50
  subsystems collapse to `cloud.Register("x", n, cloud.Typed(Mount))` /
  `cloud.RegisterWithShutdown(..., cloud.Typed(Mount), Shutdown)`. Redundant
  shutdown wrappers dropped where Shutdown already matches ShutdownFunc; kept
  only where a no-arg Shutdown() needs signature adaptation.
  MountFunc's param STAYS `any` on purpose: the pinned external subsystem modules
  (hanzoai/ai, authz, base, commerce, metrics, o11y, licensing) register with
  `func(any,…)`, and a `func(any,…)` literal is not assignable to a
  `func(*zip.App,…)` parameter — retyping MountFunc would break those modules at
  compile time. The assertion is now central, not per-subsystem. MountAll takes
  the concrete *zip.App (threaded from Serve).

CHANGE 2 — OwnsHealth flag replaces the "<name>svc" health kludge
  Some subsystems serve their OWN fail-closed /v1/<name>/health; the generic
  always-ok liveness route in Serve would shadow it. The old fix encoded routing
  policy in the id ("kmssvc" parked the generic route at an unrouted path). Now
  Register/RegisterWithShutdown take `opts ...Option`; cloud.HealthOwner sets
  MountSpec.OwnsHealth, and Serve's generic-health loop skips a HealthOwner. The
  id is once again the clean route name. Invariant now uniform and checkable:
  a subsystem serves /v1/<name>/health  IFF  it registers cloud.HealthOwner.
  Migrated every health-owner to it: kms, paas, s3 (named in scope) plus
  analytics, console, platform, ml (same kludge) and notify, plans, pricing,
  security (had coincidental id==route; security's real probe reports a rule
  count the generic route was silently dropping). pickKMSClient gate + all
  tests + stale comments updated from the "kmssvc"/"s3svc"/… ids to kms/s3/….

CHANGE 3 — clean package renames (no collision)
  clients/paassvc → clients/paas, clients/projectsvc → clients/projects
  (package decls, filenames, the sole importer, error strings, userAgent, and
  doc refs repo-wide). clients/kmssvc + clients/tasksvc KEEP their package names
  — the `svc` disambiguates the subsystem from the same-named library it imports
  (clients/kms, hanzoai/tasks); their ids are already clean (kms via CHANGE 2,
  tasks).

CHANGE 4 — generic pick[T]
  The five identical co-resident-or-RPC-or-disabled resolvers (IAM, Base,
  Commerce, O11y, MQ) collapse into one
    pick[T](cfg, log, name, label, zapAddr, rpc func(string)T, disabled func()T) T.
  KMS/AI/VFS/Payments/Vault keep bespoke pickers — their construction genuinely
  differs (embedded store / gateway preference / S3-admin backend / never
  co-resident), so they are left alone.

Verified: CGO_ENABLED=0 go build ./cmd/hanzo/ and ./cmd/cloud/ both exit 0;
go vet clean on every changed package; `hanzo --help` lists kms/paas/projects/
s3/tasks svc-free; cloud root + renamed + health-owner package tests pass; new
build_registration_test.go covers Typed + HealthOwner. Net −199 lines.
2026-07-08 08:13:52 -07:00
hanzo-dev 94c5c40d13 Merge branch 'feat/control-plane-ceremony'
# Conflicts:
#	config.go
2026-07-08 07:59:17 -07:00
hanzo-dev 806beeee34 chore(cloud): repin luxfi/consensus v1.35.30 -> v1.35.32 (DoS bound + 1-based fix)
Picks up the increment-2 crypto hygiene: the PartyID<=ValidatorSetSize DoS
bound on the quasar/pulsar Finalize path (Item7a) + the structural-Verify lock
(Item7b). v1.35.32 corrects a 1-based off-by-one in v1.35.31 that rejected the
Nth validator; verified the controlplane N=7 ceremony finalizes under -race.
LOW severity (ingestLeg bounds PartyID upstream) but the fix is now live-pinned.
2026-07-08 07:56:58 -07:00
hanzo-dev 0acb4440b2 merge: two-plane epoch write-fence primitive (strict-> + atomic CAS) 2026-07-08 07:51:42 -07:00
hanzo-dev eded7e4ef3 merge: controlplane containment cage (CI guard + fail-closed assert + external-cert seam) 2026-07-08 07:51:41 -07:00
fd05bf9860 feat(ingress): embedded runtime-configurable edge subsystem (/v1/ingress) (#182)
Add clients/ingress — an embedded edge plane in the cloud binary so the ONE
binary can BE the fleet edge: terminate TLS, run ACME, and reverse-proxy by Host
to upstreams, configured LIVE over /v1/ingress with no static routes.yaml and no
restart to change a route (hot-apply via an atomic engine snapshot swap).

Control plane (zip): /v1/ingress/{routes,services,middlewares,tls,status},
SuperAdmin-gated, per-tenant SQLite persistence, route Host globally unique;
every mutation reloads the engine.

Data plane (net/http): :80 (ACME HTTP-01 + router) and :443 (SNI TLS termination
via x/crypto/acme/autocert + router). Started only in edge role
(CLOUD_INGRESS_EDGE_ENABLED); app role keeps the listeners off — role = runtime
config, one binary.

Proxy: github.com/vulcand/oxy/v2 (Traefik lineage) weighted round-robin, the
Traefik router->service->middleware model. Middlewares: redirectScheme,
stripPrefix, addPrefix, headers.

STAGED subsystem (config.stagedSubsystems): linked but mounts ONLY when named in
CLOUD_ENABLE, so prod is untouched. Orthogonal to /v1/gateway (auth/rate-limit).

Build: CGO_ENABLED=0 go build ./... green; go test ./clients/ingress green (11 tests).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-08 07:45:37 -07:00
hanzo-dev 0ba6d72c44 fix(writefence): injective mirror key + single-read Get (red pass)
Red-team findings on the write-fence primitive:
 - HIGH: mirrorKey(plugin,shard) = "writefence/"+plugin+"/"+shard was not
   injective — mirrorKey("kms/tenant-a","secrets") collided with
   mirrorKey("kms","tenant-a/secrets"), so a push framed as one shard could
   overwrite another's epoch/writer/payload. Real shard scopes carry '/'
   (vfs replica.DBPath yields "projects/site"), so it is reachable. Fixed with
   a %d:-length-prefixed key; TestMirrorKeyInjective_NoCrossShardAliasing locks
   it (was red's failing PoC, now green).
 - LOW: MinioConditionalStore.Get did StatObject+GetObject (two round trips);
   tightened to one GET whose obj.Stat() ETag is consistent with the read bytes
   — closes the window rather than relying on the CAS to absorb a stale version.
Core CAS/epoch soundness unchanged (red GO: N=64 same-epoch race → one winner,
retry-bounded, strict-> rejects epoch==recorded). Still shadow-only, unwired.
go test -race -count=200 green.
2026-07-08 07:26:02 -07:00
hanzo-dev 0f9e44b0ee harden(containment): grep also catches -tags negation form (red pass)
Red-pass finding: the check-#1 grep char class [A-Za-z0-9_, ] missed the
build-constraint negation form `-tags '!x,controlplane'` (the `!` broke the
match before reaching controlplane). Add `!` to the class so it is caught.

Verified the deeper guarantees hold (evasion-agnostic), so this is belt-only:
 - ZERO non-test importers of clients/controlplane (grep-confirmed).
 - The package has ZERO untagged files, so importing it into serve code fails
   the untagged `go build ./...` — check #3 catches ANY -tags syntax, incl.
   GOFLAGS=-tags=controlplane (verified: the pkg becomes buildable => check #3's
   'matched no packages' assertion fails => CI red).
Runtime asserts + external-cert selfComposedCert seam confirmed wired through
guarded constructors. No stub-crypto path reaches a serve binary.
2026-07-08 07:18:31 -07:00
hanzo-dev 7a07a38f1f harden(controlplane): CI guard also catches the testing.Testing() spoof vector
Self-review finding: containment.go's runtime guard trusts testing.Testing(),
which is backed by a linker-set string var (testing.testBinary, set by `go
test` itself per cmd/go/internal/load/test.go). Confirmed locally that
`go build/run -ldflags="-X testing.testBinary=1"` spoofs it to true in a REAL
(non go-test) binary — verified with a throwaway program before writing this.

containment.yml's grep step now also fails the build on any reference to
`testing.testBinary` outside the Go toolchain itself, so a build path that
tried to ship that spoof gets caught the same way a `-tags controlplane`
build path does. Documented as a known residual in the workflow's header:
this is a mitigation (CI catches it), not a cryptographic close — that needs
increment-2's real signing, tracked in doc.go.

Also fixed the exclusion patterns to be grep-implementation-agnostic (some
recursive greps don't prefix paths with "./"), verified against a planted
violation for both checks.
2026-07-08 07:09:47 -07:00
hanzo-dev 63389d6e45 fix(writefence): strict-epoch, atomic-CAS write-fence for the (plugin,shard) mirror
Closes the same-epoch double-write on the HIP-0107 data-plane push path
(github.com/hanzoai/vfs/replica, wired in internal/org): today the only
admission checks are replica.IsOwner (a pure local computation over a
possibly-stale membership view) and the StatefulSet Recreate deployment
shape (role.Role) — both comment-only, non-atomic, and the underlying
Store/Backend.Put is an unconditional overwrite ("Overwriting is allowed").
A deposed/partitioned writer and a freshly-elected one can both push.

internal/writefence/fence.go adds Fence.Push: a single atomic
read-check-CAS that (1) rejects any candidateEpoch <= the epoch currently
recorded for the shard (strict >, closing the same-epoch case) and (2)
performs the epoch-advance and payload append in ONE conditional write
against the store's live version token, so two racing writers cannot both
land — the store is the sole arbiter, never an in-memory cache. Retries
once on a lost CAS race, re-checking strict monotonicity against the new
state, so a same-epoch racer's retry fails ErrStaleEpoch rather than
silently duplicating the admit.

EpochSource is the pluggable seam clients/controlplane's lease epoch drops
into once it graduates from shadow (Stage 1 today) — this package imports
nothing from controlplane. ConditionalStore models the S3 If-Match / GCS
generation-match primitive; store.go backs it for real with minio-go's
native SetMatchETag/SetMatchETagExcept (already vendored at v7.0.100, no
go.mod bump). fake_test.go models the same semantics in-process with a
barrier hook that deterministically reproduces the concurrent-CAS race.

Tests prove: strict-epoch rejection of a same-epoch retry (same and
different writer), the raw CAS rejecting a race loser, the full
concurrent-Push race resolving to exactly one winner, a legitimately
higher epoch being admitted, a stale lower epoch being rejected, and
per-shard scoping. Not yet wired into the live push path (that remains
gated by controlplane's shadow flag per HIP-0116); this is the fence
primitive plus a precise wiring recommendation for hanzoai/vfs's block
layer, which currently exposes no conditional-write capability to adopt.
2026-07-08 07:07:17 -07:00
hanzo-dev c0f1f07c4b harden(controlplane): CI+runtime containment cage + external-cert type seam
Stage-1 ceremony's crypto is stub/forgeable by design (doc.go); this closes
the drift risks doc.go's increment-2 worklist flagged:

- .github/workflows/containment.yml (PR-gated): greps every build/release
  surface in the repo for `-tags controlplane` and fails the build if found,
  plus a positive proof that `go build ./...` links clients/controlplane into
  no cmd/ main and that the package still matches zero packages with no tag.

- containment.go: mustHarnessOnly fail-closed panics the moment this
  package's stub crypto is touched (package-import-time for the
  PartialZVerifier registration, construction-time for NewSigner/
  NewStubComposer) unless ProductionBCCSigningReady() (hardcoded false) or
  testing.Testing() (the Go toolchain's own go-test signal, unspoofable by a
  real build) holds. Proven end-to-end via a real subprocess
  (TestContainment_NonHarnessProcessRefuses), not just in-process logic.

- selfComposedCert typed seam (driver.go/signer.go): CertComposer.Compose now
  returns an unexported wrapper only it can produce; verifyOwnCertStructure
  accepts only that type, never a bare *quasar.QuasarCert. An externally-
  received cert has no way to become one, so it cannot reach the structural
  check even by mistake. VerifyExternalCert is the sole seam for such a cert
  and fails closed (increment-2 crypto not implemented). Locked from a
  black-box vantage in external_cert_test.go.

Containment verified unchanged: `go build ./clients/controlplane/...` (no
tag) still matches zero packages; `go build ./...` still links no cmd/ main
to the package; full `-tags controlplane -race` suite green, no test weakened.
2026-07-08 07:05:39 -07:00
hanzo-dev 984186bb71 analytics: bake console website-id into the console-embed build
The cloud-embedded console (console.hanzo.ai + team) is built from hanzoai/console
build:embed. hanzoai/console now ships <HanzoAnalytics/> (env-gated on
NEXT_PUBLIC_ANALYTICS_WEBSITE_ID). Default it to the console.hanzo.ai property
(7dce54ee-41f6-4751-96bf-fe005067c7c7, public per-site) in the console build stage
so the one native analytics tag renders on the next cloud build. GA4/Pixel off.
2026-07-08 06:41:24 -07:00
940da9a06a fix(anchor): EIP-2 low-S normalize the MPC signature (fixes 'invalid sender') (#180)
The luxfi/mpc threshold signer returns a NON-canonical r|s: s is frequently in the
upper half (s > N/2). luxfi/geth's tx validation (ValidateSignatureValues,
homestead=true) REJECTS high-S signatures, so the anchor's MPC-signed self-tx
failed on submit with 'invalid sender' (live: POST /v1/admin/treasury/anchor ->
status error, note 'submit: send tx: invalid sender'). recoverableSig now
canonicalizes s to N-s when it exceeds N/2 before searching the recovery id, so
the 65-byte r|s|v it hands tx.WithSignature is EIP-2-valid and recovers to the
treasury MPC wallet. Tests: TestRecoverableSig_LowSNormalization (forced high-S ->
low-S, still recovers). go test ./clients/wallets/... green; cmd/cloud builds.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-08 06:25:30 -07:00
hanzo-dev 3ce162b573 test(controlplane): re-red — lock double-write closure as full-ceremony invariant
Adversarially verify blue's class-A fixes hold under op COMPOSITION inside a
single block (which the original red suite exercised only as separate blocks or
single ops). Six hostile compositions — bare reassign, release+reassign,
release+assign, membership-remove+reassign, remove+release+assign, assign-steal
— are each refused end-to-end through the N=7 ceremony, and the live writer is
unchanged across every voter with its lease mirror consistent. Plus: the
authorized proven-dead handoff stays single-valued under redundant reassigns,
and assign+release of a fresh resource leaves no orphan writer (mirror desync
would be a second authority). GO: the double-write class is fully closed.
2026-07-08 06:09:40 -07:00
432a365834 fix(wallets): retry Safe createWallet on 'vault not found' (ring commit-after-response race) (#179)
The ring's :8081 commits a newly-created vault to its DB AFTER writing the
createVault 201 response, so cloud's back-to-back createVault->createWallet (fired
microseconds apart on one keep-alive connection) races the commit and read-misses
the just-created vault -> 404 'vault not found' -> custody=safe 502. A slower
client (curl, separate processes) never observes the gap, which is why manual
repro succeeded. Bounded retry (6x, linear 250ms backoff, ctx-aware) on exactly
that 404; every other error still fails fast. Idempotent per attempt (fresh body).
go build ./cmd/cloud green; go test ./clients/wallets/... green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-08 05:02:46 -07:00
f8b557281e treasury: bind the reserve MPC wallet as the on-chain anchor signer (#63) (#178)
Wires the #162 BindAnchorSigner seam to a real quorum signer. New:
- wallets.TreasuryAnchorSigner(org,chain): resolves-or-provisions the org's stable
  KindTreasury wallet on the ring (reserved account 'treasury' / wallet
  'reserve-anchor', idempotent) and returns its EVM address + a sign closure.
- The closure produces an EVM-recoverable r‖s‖v signature: the ring returns a bare
  r‖s (64B, no recovery id) but tx.WithSignature needs 65B, so recoverableSig finds
  the v whose recovery yields the wallet address (fails closed otherwise).
- POST /v1/admin/treasury/bind-anchor (global-admin): calls TreasuryAnchorSigner +
  BindAnchorSigner, so subsequent /v1/admin/treasury/anchor commits the ledger root
  signed by the treasury MPC wallet, not the lone KMS key. Returns the bound address
  (fund it for gas on the Hanzo L1).

Tests: TestRecoverableSig (both parities recover to the signer) + _NoMatch (fail
closed). go build ./cmd/cloud green; go test ./clients/wallets/... green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-08 04:37:06 -07:00
zeekayandhanzo-dev ba16080fd0 fix(o11y): restore the missing metrics-query companion — unbreak the cloud build
The o11y-scope landing added clients/o11y/{scope,status}.go referencing a metrics-
query layer (vmClient, newVMClient, promLabel, metricsQuery, queryMetrics,
metricsResult, metricPoint, usageRollup, boundRangeMinutes) whose source file was
never committed → `go build ./cmd/cloud` failed (undefined symbols), taking the
whole deploy plane down (no new cloud image buildable from main). Restore the file
to the surface's own honest-empty contract: newVMClient reads O11Y_VM_URL and an
unset/unreachable VM degrades every query to an honest-empty series (never a
fabricated point); status.go's VM up-inventory works when VM is wired. queryMetrics
returns the honest-empty RED series until the VM query_range wiring lands. Full
`go build ./cmd/cloud` now links; go test ./clients/o11y passes.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-08 04:31:12 -07:00
hanzo-devandGitHub 3a8de3cc60 chore(deps): hanzoai/ai v1.800.10-pre -> v1.802.0 (auto-routing defaults + decision collection) (#177)
Brings the embedded ai subsystem up to v1.802.0:
- #76 opt-in auto-routing (virtual auto/zen-router model, X-Routed-Model)
- #77 per-org enable/disable (OrgSettings precedence)
- #78 admin-settable defaults (reserved "*" row, /v1/get-routing-defaults)
  + RoutingEvent collection (no prompt text) + /v1/export-routing-ledger

Edge contract unchanged; auto_routing_billing_test green against the new
module (ok github.com/hanzoai/cloud). Pre-existing clients/o11y compile
break on main is untouched (fix lands separately).
2026-07-07 23:31:08 -07:00
hanzo-devandGitHub 0f7f6fe9ea refactor(auto): decomplect — remove the /v1/auto reverse-proxy + clients/auto (#176)
Kill the second automation surface. /v1/auto was a per-org reverse proxy
(clients/auto + clients/auto/proxy) to the standalone hanzoai/auto engine
(auto.hanzo.svc) — a duplicate of the native, in-process /v1/automations
Connectors+Automations engine (clients/automations, cloud.EmbeddedTasks,
706-piece catalogue). One engine, one surface: /v1/automations is the ONE
native automation engine. The external engine + its console link-out are
retired (console + universe in paired PRs).

- Remove the order-140 blank import of clients/auto from subsystems.
- Delete clients/auto/ (auto.go + proxy/).

No functional loss: /v1/automations already serves flows/versions/runs/
pieces/MCP natively. clients/kb keeps its own AUTO_UPSTREAM piece-runner
coupling (a separate, pre-existing bridge to a never-implemented engine
endpoint) — reported for a follow-up, not touched here.

go build ./... green, go vet green, go test ./clients/automations + root ok.
2026-07-07 23:11:43 -07:00
7c0a653e76 refactor(automations): rename connector catalogue pieces -> connectors (HIP-0126) (#174)
* refactor(automations): rename connector catalogue pieces -> connectors (HIP-0125)

The automations connector CATALOG surface drops the ActivePieces term "pieces" for the ONE Hanzo term "connectors":

- GET /v1/automations/pieces -> /v1/automations/connectors; /pieces kept as a
  byte-identical back-compat alias (same handler) so live clients never break.
- Catalog{PieceCount,Pieces} -> {ConnectorCount,Connectors}; PieceMetadata/
  PieceAuth/PieceAction/PieceTrigger -> Connector*; JSON tags pieceCount/pieces
  -> connectorCount/connectors; embedded catalog.json + OpenAPI updated to match.
- Test proves the /pieces alias mirrors /connectors byte-for-byte.

Deliberately UNCHANGED (persisted @xyflow builder wire contract; renaming would
break live clients + stored flows): the flow-step protocol PieceName/pieceName,
PIECE/PIECE_TRIGGER, corePiece. Aligning those is a staged migration (HIP-0125).

* chore(automations,git,framework): scrub AI-slop placeholder comments (Rob Pike pass)

Comment-only, zero behavior change. Removes agent-note narration and future-work hedges, keeps the real WHY:
- automations.go: drop "a separate agent later OVERWRITES this file" narration; keep the Catalog-is-the-wire-contract invariant.
- framework/naming.go: "value for now" -> "value derived from now" (it reads the now arg, not a hedge).
- git/git.go: drop TODO(billing) + "in the MVP" hedge; state the git.usage meter fact.
- git/storage.go: drop TODO(vfs)/MVP/follow-up narration; keep the WHY osfs (not vfs) is used (vfs.FS does not implement go-billy).
Kept as real WHY/invariants (not slop): connector_core.go loopback-test SSRF guard, connector_slack.go httptest override, affiliates/store.go  sentinel + PendingCents; types.go was already cleaned in the rename commit.

* docs(automations): point connector-rename references at HIP-0126 (0125 was taken)

---------

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 23:01:24 -07:00
b503e3d512 feat(cloud): kill the /v1/auto ActivePieces reverse-proxy — one native engine (#175)
CTO decision: ONE automation engine = the native Go /v1/automations
(clients/automations on cloud.EmbeddedTasks). This removes the redundant
/v1/auto reverse-proxy subsystem (clients/auto), a per-org proxy to the
standalone ActivePieces Deployment (auto.hanzo.svc).

- delete clients/auto/ (auto.go + proxy/)
- drop the order-140 blank import from subsystems.go

Safe: no live caller of cloud/v1/auto — console link-outs to auto.hanzo.ai,
and clients/kb calls the engine directly via its own AUTO_UPSTREAM client
(untouched here). The native /v1/automations surface is unaffected.

NOTE (does NOT retire the ActivePieces Deployment): clients/kb/sync_piece.go
still executes connector pieces via the engine at /v1/auto/pieces/{piece}/run;
the native engine exposes the piece CATALOGUE but not piece EXECUTION yet, so
auto.hanzo.svc must stay until native reaches piece-run parity.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 22:51:02 -07:00
dbb03eb911 chore: cut AI-slop comment narration (Rob-Pike pass) (#173)
Comment-only tightenings, zero behavior change:
- pubsub/o11y: drop the misleading "GC won't collect" narration on the
  package-level server/collector refs; state the real reason (shutdown
  reachability) or the actual invariant (metrics ref is a write-only keepalive).
- iamsvc: condense the 11-line InitEmbed block that verbatim-restated the
  package doc down to the fail-closed WHY that matters at the call site.

No code changed (git diff: comments only).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 22:50:59 -07:00
0466ae3953 fix(wallets): Safe custody must create the MPC wallet via the ring PRODUCT API (#172)
Safe deploy (POST /v1/wallets/{id}/smart-wallet) resolves the owner wallet by its
db.Wallet PRIMARY KEY (orm.Get). The :9800 internal /keygen mints a threshold key
but persists NO db.Wallet row, so deploy 404'd 'wallet not found' (live: every
custody=safe create -> 502). The ring's Safe surface is VAULT-scoped: the only
create path that persists a db.Wallet AND returns its id is
POST /v1/vaults/{id}/wallets.

safeCustody.Provision now: createVault -> createWallet (vault-scoped, returns db
id + internal WalletID + EOA) -> deploySafe(dbId). KeyRef stays
<internalWalletId>|<smartWalletId> (owner-sign via :9800 uses the internal id;
propose via :8081 uses the smart-wallet id); the db id is only needed for the
one-time deploy. safeclient gains createVault + createWallet; the stub test now
emulates the vault/wallet-create routes. go test ./clients/wallets/... green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 22:11:00 -07:00
c12fec04ac feat(usage,audit): org-scoped usage summary + audit trail endpoints (#171)
Add two org-scoped cloud-api surfaces for the enterprise console:

- /v1/usage/summary (clients/usage): the org's unified footprint roll-up —
  spend by category over time + wallet (from the commerce ledger) plus LLM
  usage totals (from the warehouse). Composes existing sources server-side;
  each degrades independently to honest zeros with a source marker. Org from
  the validated bearer only (principal.Tenant); a forged X-Org-Id with no
  principal 401s and never reaches commerce.

- /v1/audit (clients/auditlog): the per-org twin of the admin god-view — an
  org admin reads ONLY their own org's events off the SAME tamper-evident,
  hash-chained store. Org PINNED server-side (a client ?org is ignored);
  filters time/actor/action/resource/resourceId/result + pagination.

- audit: extract the shared audit.Wire projection (used by both the admin and
  org routes, one JSON contract) and add a ResourceID filter to audit.Query.

Tests: usage (pure roll-up/categorization + HTTP scoping/honest-zeros),
auditlog (real in-memory recorder: scope isolation, filters, pagination,
401/501), audit (ToWire + ResourceID). CGO_ENABLED=0 go build + go test green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 22:06:00 -07:00
hanzo-dev 9479ce783a fix(controlplane): close the release+assign double-write sibling
Reviewing my own displacement fix adversarially: red tested release+reassign,
but a standalone release of a LIVE holder was still admitted, and after it the
shard is unowned — so release(victim) at H then assign(attacker) at H+1 puts a
second live writer on the shard (same class, pure policy, survives real crypto).

Fix: a LIVE holder's lease is immutable — releasable only when the holder is
proven dead (ErrUnauthorizedRelease), symmetric with reassign. Closes the whole
double-write class, not just red's two tested paths. + TestPolicy_ReleaseLiveHolderRefused;
TestRSM_DeterministicConvergence now marks the holder proven-dead (out-of-band)
before releasing. Suite green.
2026-07-07 22:02:16 -07:00
fbd1765d9d clients/world: GDELT + allowlisted-RSS news data plane (/v1/world) (#169)
* treasury: anchor signs through a quorum-gateable seam, not a lone key

anchor_evm.go held the signer's private key in-process and did types.SignTx.
Decouple WHERE the key lives from the tx builder via a txSigner seam:

- keySigner  — the existing local KMS-provisioned key (default; unchanged result,
  proven byte-identical to types.SignTx).
- mpcSigner  — delegates the 32-byte EVM signing hash to a quorum-gated custody
  backend (the reserve's 3-of-5 treasury MPC wallet), bound via BindAnchorSigner
  (the finance seam). The bound signer wins over any local key.

submit() now hashes the tx, delegates the hash to the resolved signer, and
applies the recoverable signature — agnostic to single-sig vs threshold. Fails
closed when neither signer is available (never fabricates a signature).

Test proves both paths recover to the correct sender and the quorum signer is
invoked exactly once; the live ring is a config swap.

* feat(gpu): BYO-GPU worker uploads render outputs to the org gallery

After studio.render completes on the local GPU, the worker fetches each finished
output from the local studio (/view) and POSTs it to the org studio's /upload/output
with the user's IAM bearer — landing it in orgs/{org}/output (S3-mirrored to the
gallery). No S3/rclone credentials ever touch the box; the session token is the only
credential. Upload target resolves from input.uploadUrl, then HANZO_STUDIO_UPLOAD_URL,
then studio.hanzo.ai. Proven end-to-end against studio 0.14.9 (aud hanzo-console).

* feat(gpu): per-machine share policy — advertised on the fleet record, enforced at claim

A linked GPU can be shared to specific orgs/projects/job-types/models with limits via
ONE policy object on the machine record (SharePolicy). It rides in the fleet
registration (input.policy) and is enforced ONCE, at claim: a job outside the policy
is failed back so an eligible worker takes it. nil/zero policy = fully permissive
(unchanged behaviour). Loaded from HANZO_GPU_POLICY (inline JSON) or
HANZO_GPU_POLICY_FILE. Unit-tested (reject matrix + loader).

Server-side multi-org queue fanout + metering-to-org+project remain follow-ups; the
worker enforces its own policy today (workers still claim their own org's queue).

* feat(world): GDELT + allowlisted-RSS news data plane (clients/world)

First vertical slice of the World news backend in the unified cloud binary:

  GET  /v1/world/news       merged, filtered, freshest-first feed  -> {items:[…]}
  GET  /v1/world/pipeline   per-(org,project) pipeline config
  PUT  /v1/world/pipeline   upsert feeds + keyword/region/source filters
  GET  /v1/world/stream     SSE live refresh (ZAP-native, org+project scoped)

- Ports world/api/{gdelt-doc,rss-proxy}.js: GDELT 2.0 Doc artlist + host-
  allowlisted RSS/Atom (~180-domain SSRF allowlist, enforced at PUT boundary,
  at fetch time, and on redirect targets).
- Org/project isolation on every path (principal.Tenant/Project); SQLite
  pipelines table PK(org,project); in-memory TTL feed cache (10m).
- RegisterWithShutdown order 142; one blank-import line in subsystems.go.
- Tests: httptest-stubbed upstreams (deterministic/offline) + live-verified.

---------

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 22:00:29 -07:00
hanzo-dev 4f0db23d6c harden(controlplane): quorum-safety assertion, honest cert-verify, domain-sep commit
Follow-up rails from the red pass + the cryptographer audit (all on top of the
class-A fixes):
- checkQuorumSafety(N,quorum,f) asserts N>=3f+1, quorum>=2f+1, 2q>N+f at cluster
  construction (fail closed). The 2q>N+f margin at N=7 is exactly 1 and is the
  whole basis of the no-fork property, so a future sizing change can never
  silently break safety. + TestSafety_QuorumParametersAreByzantineSafe.
- verifyOwnCertStructure: renamed the driver's structural self-composed cert
  check away from 'independent triple-gate verification' and documented that an
  external cert must go through the cryptographic VerifyUnderPolicy (increment-2),
  never this structural path (red #8).
- commitZ: domain-separate the z-share commitment by session + party
  (H(cp-commit||sid||party||z)) so a commitment cannot be replayed across
  sessions/parties (red #4 hardening).
- doc.go: record the red->blue outcome (no-fork core held; 4 class-A closed), the
  CLASS-B caveat (stub secrets are public-seed-derivable -> safety suite meaningful
  only under real crypto), and the increment-2 security worklist (distributed DKG,
  authenticated handoff + KMS fence, RSM-level authz re-verify, external-cert
  crypto verify, CI guard against -tags controlplane releases).

Suite green under -tags controlplane -race; default build unaffected.
2026-07-07 21:58:34 -07:00
hanzo-dev cdbf3442b4 fix(controlplane): close red's class-A byzantine findings (blue->red->blue)
Red found a CRITICAL double-write + 3 more class-A breaks (pure orchestration/
policy, survive real crypto) with failing exploit tests. All closed; red's 4
class-A tests now pass without weakening them; class-B (stub-crypto-forgeable)
deferred to the real-crypto increment with explicit t.Skip TODOs.

#1/#2 CRITICAL double-write (policy.go, placement.go): displacement of a LIVE
  shard writer now requires proven-death by out-of-band evidence. A proposer-
  written same-block release authorizes nothing (it is not holder consent), and
  membership removal no longer manufactures proven-dead. Fail-closed increment-1
  posture; authenticated graceful handoff + KMS fence are increment-2.
#3 HIGH barrier forgeable (driver.go, custody.go, signer.go, transport.go):
  Round1 commitments are now proof-of-possession authenticated exactly as Round2
  legs, so one node cannot forge a quorum of spoofed commitments to defeat
  commit-before-reveal.
#4 MEDIUM apply fork gate (rsm.go): RSM.Apply re-checks ParentRoot == the
  applied-state commitment, so a block that does not extend local state can never
  mutate it (defense-in-depth for a future recovery/gossip path).

Corrected TestPolicy_ShardReassign_WithRelease (it asserted the vulnerable
same-block-release-authorizes-displacement behavior) to assert the fix. Updated
rsm_test blocks to extend state properly (the new parent-root gate). Suite green
under -tags controlplane; default build unaffected (package is tag-gated).
2026-07-07 21:54:35 -07:00
7ad0745ce3 wallets: add Safe (Gnosis-Safe) smart-wallet custody over the luxfi/mpc ring (#62) (#168)
New KindSafe custody composes the ring's TWO planes without importing luxfi/mpc:
- :9800 internal threshold API (mpcclient) — keygen the owner MPC EOA + owner-sign
- :8081 product API (new safeclient) — CREATE2 Safe deploy + EIP-712 Safe-tx propose

safeclient.go mints a SHORT-LIVED HS256 ring JWT (iss=mpc.lux.network, aud=mpc-api,
role=admin, org-scoped) hand-rolled (crypto/hmac, no jwt dep) from the ring's
MPC_JWT_SECRET — resolved from cloud's in-process KMS via
CLOUD_WALLETS_MPC_JWT_SECRET_REF, NEVER a plaintext env value. The deploy route is
role-gated (owner|admin), so role=admin clears it.

safeCustody.Provision: keygen (owner EOA) -> deploy Safe(owners=[EOA], threshold=1)
on the wallet's EVM chain (per-wallet, default Hanzo L1 36963); KeyRef encodes both
ring handles (<mpcWalletId>|<smartWalletId>); Address = predicted Safe contract.
Sign: owner-approval signature via :9800 (uniform /v1/wallets/:id/sign). New route
POST /v1/wallets/:id/safe-tx composes the ring propose (EIP-712 MPC-sign) via a
safeProposer capability type-assert (no Kind switch). Fails closed
(ErrMPCNotConfigured) until CLOUD_WALLETS_MPC_API_ADDR + the JWT secret are wired.

Tests: TestSafeCustody drives a stub emulating both ring planes (asserts the minted
JWT is HS256-valid with correct iss/aud/role/org) + TestSafeCustody_FailClosed.
go test ./clients/wallets/... green; CGO_ENABLED=0 go build ./cmd/cloud green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 21:45:19 -07:00
0b07cce775 cloud: register /v1/wallets subsystem + unblock main (Config.Role collision) (#167)
Two coupled changes so main builds green AND the MPC custody surface is live:

1. Fix the broken build on main. #160 (CLOUD_ROLE writer/reader HA split) and
   #163 (Stage-0 control-plane inert config) each added a `Role` field to the
   SAME Config struct on separate branches; the merge left Config.Role
   redeclared (role.Role vs string) + a duplicate struct-literal key, so
   `go build ./cmd/cloud` failed (release lane stuck at v1.786.124). Rename the
   inert #163 field to ControlPlaneRole (env ROLE, consumed by nothing yet). The
   HA Role (role.Role, CLOUD_ROLE, used by serve.go/build.go) is unchanged.

2. Register the wallets subsystem. clients/wallets (#151/#161) was never blank-
   imported into subsystems.go, so its init() never ran and /v1/wallets was
   unrouted (404) despite the code shipping. Add the order-127 blank import so
   the accounts/wallets/custody/keys/sign surface mounts — KMS custody always
   on; mpc/treasury fail closed until CLOUD_WALLETS_MPC_ADDR +
   CLOUD_WALLETS_MPC_API_KEY_REF are wired. This is the seam the treasury anchor
   (#162 BindAnchorSigner) binds through.

Verified: CGO_ENABLED=0 go build ./cmd/cloud green; wallets + config tests ok;
local boot logs 'wallets mounted' (defaultCustody=kms) then 'listening', no panic.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 21:03:30 -07:00
hanzo-dev 3e6a35609c feat(controlplane): Stage-1 increment-1 byzantine ceremony driver + harness
FULL-BFT byzantine ceremony driver for the cloud control plane, built
against published luxfi interfaces (consensus v1.35.30 protocol/quasar +
protocol/quasar/pulsar PulsarRoundSigner, pulsar v1.9.0 pkg/pulsar). Behind
the controlplane build tag, NOT wired into serve.

- Real PulsarRoundSigner drives Round1/Round2/Finalize (canonical
  non-grindable nonce, canonical signer set, z aggregation, ConsensusCert).
- Ceremony driver over an abstract Transport: proposer -> per-voter Round1
  commitment -> ALL-Round1 barrier -> Round2 share -> >=2/3 legs -> compose
  triple-PQ QuasarCert -> independent QuasarCert.Verify -> fail-secure apply.
- One-pod-one-share custody; policy gate refuses invariant-violating blocks
  (shard-writer reassign without lease-release/proven-dead predecessor).
- In-process N=7 harness: happy path, liveness (drop2 finalize / drop3 SAFE
  HALT), safety (equivocation, one-pod-two-shares, rushing, dup/rogue legs).

Stubbed for later increments (drop-in seams): ZAP transport, KMS share
custody, NonceMPC pool, DKG keygen, and pulsar's unshipped SignatureCore +
PartialZVerifier crypto cores + ComposePolaris cert composition.

Supersedes PR #163 classical pins: drops the bft promotion.
go build/vet/test green (CGO_ENABLED=0); race-clean.
2026-07-07 18:15:28 -07:00
hanzo-dev c97a4e42c1 fix(config): resolve Role field collision from concurrent merge
Two PRs landed on main that both added a Config.Role field — the #160
HA writer/reader role (role.Role, load-bearing in Serve) and the Stage-0
control-plane role (string). The text-merge compiled to a duplicate field
and broke the default build. Rename the inert control-plane field to
ControlPlaneRole (env ROLE unchanged); the HA Role keeps its name and all
cfg.Role.IsReader()/String() consumers are untouched.
2026-07-07 18:12:41 -07:00
hanzo-devandGitHub ce7b6fe935 feat(cloud): CLOUD_ROLE writer/reader HA split + de-alias ZapDB + writer-pin seam (#160)
* refactor(kms): de-alias badger→zapdb (the embedded store IS ZapDB)

clients/kms/kms.go imported the store as `badger "github.com/luxfi/zapdb"`.
The store is luxfi/zapdb — the canonical Lux embedded KV, a hardened Badger
fork whose Go package is still literally `package badger`. The alias made call
sites read like raw dgraph-io/badger. Rename the alias to `zapdb` so every call
site is self-documenting; behaviour is byte-identical (same package, same API).

Confirms the invariant: `grep -rn dgraph-io/badger` across cloud = 0. There is
no raw Badger anywhere; the one embedded store is ZapDB.

* feat(cloud): CLOUD_ROLE writer/reader split + read-only KMS reader + writer-pin

Introduces an explicit HA role so read replicas can be added WITHOUT ever
risking a second writer opening the RWO stores. Default is byte-identical to
today: unset CLOUD_ROLE ⇒ Writer ⇒ the single pod that owns the RWO PVC.

- role: CLOUD_ROLE ∈ {writer(default), reader}. Serve fails CLOSED on an
  explicitly-invalid value (a wrong guess demotes the real writer or risks a
  second one). Pure, tested, imports nothing from cloud.
- kms: Config.ReadOnly opens the ZapDB store READ-ONLY with the lock guard
  BYPASSED — a reader serves secrets off a restored replica and NEVER takes the
  exclusive write lock (the mechanism proven safe by luxfi/zapdb's
  WithReadOnly + BypassLockGuard; zapdb-replicate uses the same to coexist with
  a live writer). Reader with no restored store / no key fails closed. Tested
  round-trip: writer writes → reader reopens read-only → reads back; reader
  writes rejected.
- writerpin: the single-writer election seam. SingleWriter (production-correct
  for StatefulSet replicas:1) is the default; ConsensusPin (Quasar leaderless
  election) is an HONEST stub that fails closed with ErrNotImplemented rather
  than fabricating a pin. Tested.
- wiring: Serve resolves+logs the role and the backing pin; pickKMSClient opens
  KMS read-only for readers. Writer path unchanged.

NOT YET wired (reported for Red/CTO): consensus election (writerpin gates no
store-open yet — k8s guarantees the single writer); reader gating of the
audit chain / durable tasks / per-tenant SQLite (still open writable) — the KMS
reader path is the completed slice. Data replication runs as sidecars at the
manifest layer (hanzoai/replicate for SQLite, luxfi/zapdb-replicate for ZapDB),
not via in-process import.

* feat(ha): fail-closed reader write-guard + prove in-process KMS backup

ReaderGuard: one boundary middleware rejects mutating verbs on a Reader
(405), gating EVERY store (KMS+audit+tasks+SQLite), not just KMS's
read-only open — a mis-routed write can no longer silently persist to a
reader's ephemeral dir and vanish on restart (H4). No-op on a Writer.

replication_test: real *zapdb.DB writer streams incremental age-encrypted
db.Backup blocks WHILE live; reader Restores into its OWN separate dir —
refutes the C1 'second-process open fails' path and proves the producer.
Fail-closed test: no recipient => no block (never plaintext to S3).

* test(ha): reader-guard verb matrix + replication edge cases

ReaderGuard: GET/HEAD/OPTIONS reach the store, POST/PUT/PATCH/DELETE all
405 without reaching it; Writer path (guard unmounted) serves every verb.
replication: wrong-identity restore fails closed; restore requires manifest
+ identity (unhydrated store never serves empty); repeated/no-op/overwrite
backups restore to the exact latest value (chain-correctness invariant).

* test(config): align IAM single-replica test with staged-subsystem contract

The 'empty list -> iam-enabled' subtest predates IAM becoming a STAGED
subsystem (stagedSubsystems["iam"]=true): the empty-Enable mount-all
default deliberately does NOT mount IAM (it corrupts the shared Beego
global and crashes `ai` with SQLITE_CANTOPEN). So empty list is
iam-DISABLED and >1 replica is allowed; the guard fires only when iam is
EXPLICITLY enabled. Code was correct; the test asserted the pre-staging
behavior. Pre-existing red on main, unrelated to the HA change.
2026-07-07 17:55:06 -07:00
hanzo-devandGitHub f3a959b4bd feat(cloud): Stage 0 — control-plane deps + inert config (#163)
Promote the luxfi consensus stack (consensus v1.25.15, bft v0.1.5,
p2p v1.21.1, validators v1.2.0) from indirect to direct requires, and add
four INERT control-plane config fields. Zero behavior change, reversible.
Deps + inert config only — no engine imported/started, no routes, no
serve.go/build.go behavior change.

v1.25.15 is the minimal clean tag: it already carries NewBFT (consensus.go:168)
+ engine/bft, its graph pulls validators v1.2.0 (Manager), it requires exactly
pulsar v1.1.1 (which stays v1.1.1 — zero drift), and it is the MVS-selected
version, so promotion is a no-op to the compiled graph. A lower tag would
downgrade the whole build's consensus (behavior change); a higher tag drifts
pulsar + consensus code.

The four are held direct by controlplane_deps.go: blank imports behind the
never-set //go:build controlplane_deps tag, so nothing links into the binary.
go mod tidy keeps them direct (it reads all build tags); deleting the file
reverts them to indirect. NodeID/Peers/Role/ControlPlaneQuorum parse in
LoadConfig (NODE_ID/PEERS/ROLE/CONTROL_PLANE_QUORUM) but no subsystem reads them.

Architecture direction (proposed, not shipped): the control plane is designed
to run Quasar (post-quantum BFT, protocol/quasar Submit->Finalized) under a
strict-PQ cert profile with a Pulsar RoundSigner threshold signer.

tidy also corrected pre-existing drift on main (nats-io/nats.go indirect->direct
via clients/kafka/interop_test.go; pruned 7 superseded go.sum lines) — verified
identical on pristine origin/main.
2026-07-07 17:55:03 -07:00
hanzo-devandGitHub c35926554e fix(cloud): correct luxfi/precompile go.sum hash + ZAP-only telemetry (drop plaintext OTLP fallback) (#165)
Red review findings:
- go.sum: luxfi/precompile v0.5.37 zip hash disagreed with sum.golang.org
  (h1:Yh3dJ+... vs authoritative h1:2v0z...) → cold-cache CI SECURITY ERROR.
  Corrected to the sumdb-vouched hash.
- telemetry.go: remove the plaintext OTLP-HTTP fallback (newTraceExporter) that a
  stray/standard OTEL_EXPORTER_OTLP_ENDPOINT could use to silently downgrade
  tenant-carrying trace spans to cleartext. ONE wire now: ZAP. Dropped the
  otlptracehttp import (also severs its transitive grpc pull) and the dead
  otlpEndpoint parameter. OTLP stays only the collector's interop receiver.
2026-07-07 17:54:54 -07:00
hanzo-devandGitHub 02c47ded29 test(billing): prove auto-routing bills as the resolved model at the edge (#166)
The ai subsystem serves a virtual `auto`/`zen-router` model that resolves to a
concrete model id before pricing/billing, meters its own token cost keyed on the
SERVED model, and reports it via the X-Routed-Model header. The cloud edge prices
/v1/ai/* by PATH (0, self-metered), never by the request model, so `auto` bills
as whatever it resolved to — and the edge passes X-Routed-Model through untouched.

- auto_routing_billing_test.go: TestAutoRoutingBillsAsResolvedModel (edge does not
  double-bill /v1/ai/* + header pass-through) and TestDefaultPriceAiPathModelAgnostic.
- AUTH_BILLING_CONTRACT.md §4a: document the binding.

No code change needed — cloud already meters ai from the subsystem's own usage
record (which keys off the resolved request.Model), so the edge binds correctly.
2026-07-07 17:52:02 -07:00
c6adea2036 zaptrace: encode OTLP via zap2pb; drop direct google.golang.org/protobuf (#164)
Route ExportTraceServiceRequest wire encoding through github.com/zap-proto/zap2pb
(the sanctioned ZAP<->protobuf boundary) instead of importing
google.golang.org/protobuf {proto,encoding/protowire} directly. Wire bytes are
byte-identical (repeated ResourceSpans under field 1); TestUploadTracesOverZAP
still decodes the spans over the real ZAP transport.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 17:28:07 -07:00
7bf3fc4601 treasury: anchor signs through a quorum-gateable seam, not a lone key (#162)
anchor_evm.go held the signer's private key in-process and did types.SignTx.
Decouple WHERE the key lives from the tx builder via a txSigner seam:

- keySigner  — the existing local KMS-provisioned key (default; unchanged result,
  proven byte-identical to types.SignTx).
- mpcSigner  — delegates the 32-byte EVM signing hash to a quorum-gated custody
  backend (the reserve's 3-of-5 treasury MPC wallet), bound via BindAnchorSigner
  (the finance seam). The bound signer wins over any local key.

submit() now hashes the tx, delegates the hash to the resolved signer, and
applies the recoverable signature — agnostic to single-sig vs threshold. Fails
closed when neither signer is available (never fabricates a signature).

Test proves both paths recover to the correct sender and the quorum signer is
invoked exactly once; the live ring is a config swap.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 16:30:19 -07:00
5efc28cfaa wallets: reconcile mpc custody to the ring's internal threshold API (#161)
The prior mpcclient targeted a DECIDED-but-nonexistent dashboard route tree
(/v1/wallets/{id}/sign, /v1/treasury/*) authed with a hand-minted HS256 JWT.
The deployed luxfi/mpc ring's real, working server-to-server custody surface is
the internal threshold API (cmd/mpcd/main.go, :9800): POST /keygen + POST /sign,
gated on the static MPC_INTERNAL_API_KEY bearer token — the exact contract the
ring's own /sign handler documents for a custody adapter.

Reconcile cloud to that contract:
- mpcclient.go: keygen + sign over the internal API; static bearer key (KMS),
  no JWT/dependency; deterministic idempotency key per (org,wallet,digest).
- custody.go: mpc + treasury provision via keygen, sign via /sign with the
  wallet's EVM chain id; Rotate preserves the address (ring-managed shares).
  Treasury quorum governance moves to the finance policy layer over this same
  primitive (no separate ring route).
- wallets.go: CLOUD_WALLETS_MPC_API_KEY_REF (KMS ref of the bearer key).
- test: stub emulates the internal /keygen+/sign contract.

Feature-flagged: unset CLOUD_WALLETS_MPC_ADDR ⇒ mpc/treasury fail closed.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 16:20:12 -07:00
871a6eea1e feat(billing): populate per-product usage axis + server-side ?product=/?groupBy=product (#159)
The console per-product Metrics dashboard groups usage on metadata.product /
metadata.agent, but commerce RecordUsage persists only provider/model (no product
field), so the breakdowns rendered honest-empty even though every non-LLM product
already meters+gates per-org via ResourceMeter (provider=<product>, default fee
$1.00, fail-closed 402 on zero balance).

clients/billing/usage.go is the ONE read-side adapter: usage() injects a canonical
metadata.product onto each ledger row (agent->agents, provisioning->kind,
token-metered->inference, else provider) from the SAME charged ledger, and honors
the previously-ignored ?product=<id> (server-side filter) and ?groupBy=product
(per-product spend rollup {product,requests,amountCents}). A row already carrying
metadata.product/agent wins, so it degrades to a no-op once the meter/commerce
persist them natively (forward-compatible).

No change to what is charged or gated; the balance floor stays enforced by default.
scopedBillingQuery is extracted so proxy() and usage() build the subject boundary
one way. AUTH_BILLING_CONTRACT.md documents coverage + the native-field checklist.

Tests: productOf table + enrich/filter/group units + handler-level ?product= /
?groupBy=product through the real route (33 billing tests green).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 16:11:54 -07:00
53d3fcb00b feat(cloud): embed the REAL hanzoai/console bundle; fail-hard Dockerfile (#158)
The console image stage now FAILS the build when build:embed does not emit a
real static bundle (non-empty out/index.html + out/_next), instead of silently
degrading to the committed fallback shell. A broken console export can no longer
ship the placeholder to prod. Escape hatch: --build-arg ALLOW_PLACEHOLDER=1 for
a pure-Go dev image with no Node console.

hanzoai/console build:embed produces a real 7.7M static export (361KB index.html
+ 4.3M _next chunks); //go:embed bakes it into the ONE cloud binary. Also drops
the last console2 references (repo is hanzoai/console).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 15:52:57 -07:00
hanzo-dev bd72bfbb06 Merge branch 'feat/addon-fast-follows' 2026-07-07 15:11:12 -07:00
hanzo-dev d5bf2a0dc7 cloud: rename console2→console (one canonical name)
The frontend repo is hanzoai/console (console2 was renamed away). Kill the
dead name across the build path + source so there is one name, one way:
- Dockerfile: clone hanzoai/console.git; ARG CONSOLE_REPO / CONSOLE_REF
- Makefile: CONSOLE_DIR; webui + build-standalone targets
- config.go: drop dead console2.hanzo.ai from the ZAP-WS origin allowlist
- comments across clients/* reference the console repo + its TS modules by
  their real name

No behavior change beyond dropping one unused CORS origin. Root pkg builds.
2026-07-07 15:11:09 -07:00
hanzo-devandGitHub e73b16403f provisioning: Red low fast-follows (rollback orphan-key, kv-auth + envtest proofs, datastore tag) (#156)
Red review = SHIP; these close the 4 cloud-side low findings so the PR lands
with no known edges.

low-1 (rollback atomicity): createDedicated's inject-failure branch now calls
  removeAddonURL BEFORE tearing the backend down. injectAddonURL is not atomic —
  a strategic-merge PATCH can LAND server-side yet still return err (dropped
  response / post-commit timeout); scrubbing the maybe-written <KIND>_URL first
  means a committed-but-errored inject can't leave the instance pointing at a
  deleted backend (a dangling DSN is worse than Base). Proven by
  TestDedicated_InjectPartialWriteRollsBackOrphanKey (fake now models the
  write-then-error partial failure; asserts inject THEN remove ran, key gone).

low-2 (kv fail-open corner): TestDedicatedKV_RequirepassEnforced boots the REAL
  ghcr.io/hanzoai/kv image with the exact engine.args + mounted requirepass
  config and asserts an UNAUTHENTICATED PING is REJECTED, then that default:<pw>
  authenticates — locking down the one corner where, if the image ignored the
  positional config, the instance would boot unauthenticated. Raw RESP over TCP
  (zero new client deps); gated on CLOUD_KV_SMOKE_IMAGE + docker so the default
  suite stays green, real in CI.

low-3 (strategic-merge sibling preservation): TestPatchAddonSecret_RealAPIServer
  runs the ACTUAL k8sOrchestrator addon methods against a REAL kube-apiserver
  (controller-runtime envtest) — inject KV_URL then SQL_URL => BOTH survive in
  .data; RemoveAddonSecretKey drops one, keeps the other; idempotent on absent
  key/Secret. Replaces the fake orchestrator's assumption with a server-proven
  fact. Gated on KUBEBUILDER_ASSETS (skip without envtest binaries). Adds
  controller-runtime v0.23.3 as a TEST-ONLY dep — pinned to the release that
  keeps k8s.io at v0.35.3 (NO production client-go bump).

low-4 (datastore tag symmetry): dedicated datastore image tag floating ':26' ->
  env("CLOUD_DEDICATED_DATASTORE_TAG", "26.2.3.2"), symmetric with sql/kv/docdb.
  A floating ':26' resolves to whichever datastore lineage (bridge vs fork, distinct
  data dirs) last pushed under it — a per-org instance must boot a deterministic
  image.

go build ./... green; go test ./clients/provisioning/... green (envtest PASS
against a live apiserver, kv-smoke skips without docker).

(cherry picked from commit 04d841c4906b58fa06b1bc407b55c97e6661f169)
2026-07-07 14:50:18 -07:00
c84df87754 feat(o11y): wire native datastore metrics ingest into the embedded runtime (#157)
* feat(o11y): wire native datastore metrics ingest into the embedded runtime

Bumps hanzoai/o11y to the native datastore metrics driver and starts an
in-process ZAP metric receiver (clients/o11y/metrics.go) that writes metrics to
the datastore over upstream ch-go via o11y/pkg/datastoremetrics — no histogram
fork. Reuses the embedded runtime.TelemetryStore.ClickhouseDB() connection, so
the query plane (read) and metrics (write) share one datastore conn.

Opt-in + fail-soft: gated on O11Y_METRICS_ZAP_LISTEN, a no-op until set, errors
logged and swallowed so metrics ingest can never take the query plane down. This
unblocks retiring the standalone signoz-otel-collector metrics path once verified
(verify-then-cutover). CGO_ENABLED=0 build + vet + existing o11y/observe tests green.

* chore: re-pin o11y@main (native datastore metrics driver merged)

---------

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 14:27:47 -07:00
hanzo-dev 0aa1201cc5 provisioning: Red low fast-follows (rollback orphan-key, kv-auth + envtest proofs, datastore tag)
Red review = SHIP; these close the 4 cloud-side low findings so the PR lands
with no known edges.

low-1 (rollback atomicity): createDedicated's inject-failure branch now calls
  removeAddonURL BEFORE tearing the backend down. injectAddonURL is not atomic —
  a strategic-merge PATCH can LAND server-side yet still return err (dropped
  response / post-commit timeout); scrubbing the maybe-written <KIND>_URL first
  means a committed-but-errored inject can't leave the instance pointing at a
  deleted backend (a dangling DSN is worse than Base). Proven by
  TestDedicated_InjectPartialWriteRollsBackOrphanKey (fake now models the
  write-then-error partial failure; asserts inject THEN remove ran, key gone).

low-2 (kv fail-open corner): TestDedicatedKV_RequirepassEnforced boots the REAL
  ghcr.io/hanzoai/kv image with the exact engine.args + mounted requirepass
  config and asserts an UNAUTHENTICATED PING is REJECTED, then that default:<pw>
  authenticates — locking down the one corner where, if the image ignored the
  positional config, the instance would boot unauthenticated. Raw RESP over TCP
  (zero new client deps); gated on CLOUD_KV_SMOKE_IMAGE + docker so the default
  suite stays green, real in CI.

low-3 (strategic-merge sibling preservation): TestPatchAddonSecret_RealAPIServer
  runs the ACTUAL k8sOrchestrator addon methods against a REAL kube-apiserver
  (controller-runtime envtest) — inject KV_URL then SQL_URL => BOTH survive in
  .data; RemoveAddonSecretKey drops one, keeps the other; idempotent on absent
  key/Secret. Replaces the fake orchestrator's assumption with a server-proven
  fact. Gated on KUBEBUILDER_ASSETS (skip without envtest binaries). Adds
  controller-runtime v0.23.3 as a TEST-ONLY dep — pinned to the release that
  keeps k8s.io at v0.35.3 (NO production client-go bump).

low-4 (datastore tag symmetry): dedicated datastore image tag floating ':26' ->
  env("CLOUD_DEDICATED_DATASTORE_TAG", "26.2.3.2"), symmetric with sql/kv/docdb.
  A floating ':26' resolves to whichever datastore lineage (bridge vs fork, distinct
  data dirs) last pushed under it — a per-org instance must boot a deterministic
  image.

go build ./... green; go test ./clients/provisioning/... green (envtest PASS
against a live apiserver, kv-smoke skips without docker).

(cherry picked from commit 04d841c4906b58fa06b1bc407b55c97e6661f169)
2026-07-07 14:22:13 -07:00
52843e1ac2 feat(cloud): embed native PubSub (clients/pubsub :4222) + Kafka adaptor (clients/kafka :9092) — Lux-consensus/no-ZK, disabled-by-default, interop-verified (#155)
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 14:15:02 -07:00
hanzo-devandGitHub 7a29b86d53 provisioning: uniform on-demand add-on instance binding + <KIND>_URL injection (#154)
Extend the dedicated-instance strategy so all four on-demand data add-ons —
Hanzo KV / SQL / DocDB / Datastore — route through ONE mechanism, and bind an
enabled add-on to an app instance by injecting its DSN as <KIND>_URL into the
instance's addons Secret (disabling reverts to Base).

- store: additive instance column (idempotent ALTER, threaded through Resource/
  cols/scan/Insert) + ListByInstance(org,instance).
- dedicated: add sql (Datastore type=postgresql, POSTGRES_* env, PGDATA subdir)
  and kv (type=valkey, per-instance requirepass via a MOUNTED config Secret since
  the kv-server binary reads no password from env; DSN user=default). Engine
  gains adminUser/env/args/secretMount so the CR builder stays one code path.
- addon_inject: injectAddonURL/removeAddonURL + orchestrator PatchAddonSecret
  (strategic-merge, create-if-absent, key-preserving) / RemoveAddonSecretKey
  (JSON-merge delete, idempotent). Reloader annotation + rev bump on the Secret.
- create: instance bind field (validated); inject AFTER the row Insert as part of
  the atomic provision (rollback on failure). drop: revert to Base BEFORE tearing
  the backend down.
- sql/kv move off the shared-logical registry (each org OWNS its instance); the
  orphaned shared postgres/redis provisioners + pgx/go-redis direct deps removed.

Tests: instance column round-trip + ListByInstance isolation; sql/kv DSN + CR
shape; inject merges (second add-on never clobbers the first); un-bound create
skips injection; drop removes URL before teardown; inject failure rolls back the
whole provision. go build/vet/test green.
2026-07-07 14:08:35 -07:00
hanzo-dev 2c7dc05ec1 deps: bump hanzoai/ai -> isglobaladmin in /get-account (8e65b8c3)
Pulls ai's additive isGlobalAdmin field on /get-account so console
recognizes global admins. Pure dependency bump: re-pins re-tagged
luxfi/* modules from source (GOPRIVATE, sumdb-bypassed) after the
documented content-hash drift, prunes cloud.google.com/go/compute and
stale hanzoai/iam v1.31.16 (ai dropped the GCP SDK and requires iam
v1.31.17). go build ./... green (CGO_ENABLED=0).
2026-07-07 12:57:45 -07:00
44dd4d869b o11y: embed in-process OTLP ingest (traces+logs) into cloud (#153)
Fold the standalone otel-collector Deployment into the unified cloud binary:
an in-process OpenTelemetry Collector accepts OTLP (grpc :4317, http :4318) and
writes spans+logs into the same ClickHouse datastore cloud already reads for the
o11y query plane (signoz_traces / signoz_logs, cluster insights). Consumers point
at cloud.hanzo.svc instead of otel-collector.hanzo.svc.

Trimmed, driver-compatible pipeline (reuses the signoz clickhouse exporters that
compile against cloud upstream clickhouse-go v2.44.0):
  otlp -> memory_limiter, resource(namespace=hanzo, env), batch
       -> clickhousetraces (traces), clickhouselogsexporter (logs)

- OFF by default (CLOUD_OTLP_INGEST_ENABLED); fail-soft; ShutdownFunc flushes.
- DSN via env (envprovider), never on disk; metrics self-telemetry off so only
  :4317/:4318 bind (no :9090 class clash).
- telemetry.go: add OTLP-HTTP exporter path so cloud can loop back to the
  in-process ingest at localhost:4318 (ZAP stays default/canonical).

DEFERRED: metrics pipeline (signozclickhousemetrics) needs SigNoz dd-sketch
ch-go fork (chproto.DD/Store/IndexMapping) that will not compile against cloud
upstream ch-go; metrics ingest stays on the standalone collector. See
clients/o11y/LLM.md.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-07 12:12:26 -07:00
hanzo-dev c607d24b85 Merge branch 'feat/gpu-engine-serve'
# Conflicts:
#	Makefile
#	clients/treasury/anchor_evm.go
#	clients/treasury/ledger/sqlstore/sqlstore.go
#	clients/treasury/treasury.go
#	clients/treasury/treasury_test.go
#	config_iam_replicas_test.go
#	go.mod
#	go.sum
#	subsystems/subsystems.go
2026-07-07 10:43:36 -07:00
48ed003d6d feat(wallets): configurable KMS/MPC/treasury custody subsystem (/v1/wallets/*) (#151)
One custody seam over three orthogonal signing backends selected per-wallet by
Kind:

- KindKMS  single-sig custody IN-PROCESS via the embedded luxfi/kms client
  (deps.KMS). The fully-exercised spine: a real secp256k1 key is generated, its
  private bytes sealed under the KMS envelope, and every Sign recovers to the
  wallet address. No network hop.
- KindMPC / KindTreasury custody DELEGATE over HTTP to the deployed luxfi/mpc
  cluster via a thin typed REST client (the clients/mpcseal precedent). cloud
  never imports github.com/luxfi/mpc. Unconfigured -> fail closed
  (ErrMPCNotConfigured); a signature is never fabricated.

Config seam: KMS always available; mpc/treasury built only when
CLOUD_WALLETS_MPC_ADDR is set and the HS256 JWT secret resolves from a KMS ref
(never a plaintext env). Per-tenant SQLite (org column on every row, every query
filtered by org). Finance seam (WalletForLedgerAccount) is a pure lookup only.

Tests (incl -race): KMS single-sig end-to-end (sig recovers to address, sealed
at rest, rotate changes address), per-tenant isolation, custody seam selects
backend (fail-closed mpc/400 unknown), mpc path wired against a faithful stub.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-05 20:48:59 -07:00
115c8bbdd6 test(identity): make global-admin gate test config-agnostic on adminOrg (#150)
Reconciles TestGlobalAdminGate_RequiresAdminOrgAndIsAdmin with the task #51 decision
to pin IAM_ADMIN_ORG to the operator org (hanzo). The assertions are unchanged — the
gate is owner==adminOrg AND isAdmin — but the comment/labels no longer editorialize
that owner==admin is the only valid adminOrg. adminOrg is deployment config; the test
pins it to "admin" hermetically and proves the two invariants that hold for ANY
adminOrg: isAdmin is required (a non-admin in the admin org gets nothing) and owner is
required (an admin of any OTHER org gets nothing). Renamed the different-org case off
"hanzo" (which prod now pins AS the admin org) to a neutral "globex" so it reads
unambiguously.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-05 20:34:43 -07:00
900175390f feat(treasury): per-tenant Base storage — one SQLite file per tenant, IAM-selected (#149)
The finance ledger of record ran on a single process-wide {DataDir}/treasury.db.
Select the store per request from the validated IAM owner instead, so every
tenant's books live on their OWN Hanzo Base file and one tenant's writes can
never appear in another's read.

- sqlstore.Manager: opens+caches one *Store per tenant (mutex-guarded map). The
  house/reserve ledger is one fixed file ({DataDir}/treasury.db, preserved — no
  migration of live reserve capital); customer ledgers are {DataDir}/finance/{slug}.db.
- tenantSlug: injective (never folds acme/ACME), path-traversal-guarded, reserves
  the house slug. Verbatim stem for a DNS-ish org, else a sha256 slug. Consumes the
  treasury's canonical hanzoai/sqlite opener (Open) — ledgercore's per-tenant opener
  is a test-only helper that would double-register the sqlite driver.
- treasury.Mount binds the ledger of record to the HOUSE store; myAccounts reads the
  caller's OWN per-tenant file (house scope still honours the Formance/Postgres opt-in).
- StorageDriver(): one place decides the driver — sqlite (default, what prod runs) or
  postgres (opt-in via FORMANCE_LEDGER_URL). Postgres option preserved, never the default.

Tests: per-tenant isolation (A's write never in B's read; distinct files; cache
identity; traversal stays in-dir; no case-fold) + default-driver=sqlite + opt-in
preserved. go build ./cmd/cloud green; ./clients/treasury/... green (incl -race).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-05 20:05:46 -07:00
66b149f97f fix(config): stage IAM off mount-all (unblock release smoke) + lock global-admin gate (task #51) (#148)
* fix(config): stage IAM off the mount-all default — unblock the release boot smoke

Every cloud release since the IAM embed (#142) has failed its boot smoke and
the fleet stayed pinned to a pre-embed image (v1.786.110), so the treasury +
finance merges (#143/#144/#145/#147) never shipped.

Root cause (from the failed release smoke logs): with CLOUD_ENABLE unset the
binary mounts every registered subsystem, so iamsvc.Mount now runs
iamserver.InitEmbed(). In the smoke/Docker env InitEmbed panics opening its own
SQLite (IAM_DATA_DIR=/data/iam absent on the tmpfs) and is recovered to a
fail-closed 503 — but IAM and the ai subsystem are sibling casibase/casdoor
forks linked against the SAME beego module, so InitEmbed's half-initialised
shared process-global (web.BConfig / xorm adapter) then makes ai's own
bootstrap fail identically:

  iam  ERROR iamserver.InitEmbed: bootstrap panicked: unable to open database file (14)
  ai   INFO  ai: initializing runtime
  cloud: mount: mount ai: ai: bootstrap: unable to open database file (14)  -> SMOKE FAIL

This would crash api.hanzo.ai in prod too (CLOUD_ENABLE is unset there), not
just the smoke.

Fix: make IAM a STAGED subsystem — excluded from the empty-Enable mount-all
default, mounted ONLY when named in CLOUD_ENABLE. This is exactly the HIP-0106
staged-rollout contract iamsvc already documents ('operator adds iam to
--enable only after the fold is verified'), now enforced in code. It restores
the pre-#142 mount-all set (iamsvc is the only subsystem #142 added to it), so
the boot smoke goes green again; hanzo.id keeps being served by the standalone
iam pod until an explicit, verified cutover. Local mount-all boot now reaches
'listening' with iam 'subsystem disabled' and ai mounted clean.

pickIAMClient already falls back to the remote/disabled IAM client when
Enabled("iam") is false (build.go), which is current prod behaviour, so no
deps.IAM regression. One activation mechanism (the enable-list), one place.

* test(identity): lock global-admin = owner==adminOrg AND isAdmin

The cloud admin surfaces (incl. /v1/admin/treasury/*) grant global admin only
to a validated principal whose org IS the admin org AND whose token carries
isAdmin. This locks that invariant end-to-end through the real JWKS-validated
SanitizeIdentity boundary, with the two cases that matter for the treasury flip:

  - a hanzo-org ADMIN (owner=hanzo, isAdmin=true)  -> NOT global admin
  - a NON-admin in the admin org (owner=admin, isAdmin=false) -> NOT global admin

The sole global admin z@hanzo.ai is global admin because IAM promotes @hanzo.ai
into the admin org (owner==adminOrg), NOT because it lives in 'hanzo'. This test
is the guard proving the boundary must NOT be widened to owner==hanzo (e.g.
IAM_ADMIN_ORG=hanzo), which would elevate every hanzo-org admin to see all
tenants' finances. The gate stays owner==adminOrg AND isAdmin; the fix for z is
that its token carries owner=admin, never a wider gate.

---------

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-05 19:58:06 -07:00
hanzo-dev dde2df77db build(hanzo): pure-Go recipe for cmd/hanzo so the one sqlite driver registers once
The hanzo CLI (cmd/hanzo) had no build target, so a naive
`go build ./cmd/hanzo` used the machine default CGO_ENABLED=1 and panicked
at init: "sql: Register called twice for driver sqlite".

Root cause: with CGO on, github.com/hanzoai/sqlite (the canonical Hanzo
driver, imported by ~15 clients/*/store.go) compiles its mattn/SQLCipher
backend and registers "sqlite"; the embedded upstream deps that import
modernc.org/sqlite directly (base/core, o11y, commerce/db, orm/db) register
"sqlite" a second time -> panic.

Fix: build cmd/hanzo the same pure-Go way cmd/cloud and the Dockerfile
already ship (CGO_ENABLED=0). hanzoai/sqlite's !cgo backend IS modernc, so
the fork and every modernc importer resolve to a single registration. This
extends the existing CGO_ENABLED?=0 policy (see Makefile header) to the new
binary instead of adding a second way to build; no dependency is dropped and
hanzoai/sqlite stays canonical.

  make hanzo   # -> ./bin/hanzo, pure Go, one 'sqlite' registration
2026-07-05 19:52:05 -07:00
hanzo-dev 3185258a3e chore(deps): mark luxfi/crypto & luxfi/geth direct
clients/treasury/anchor{,_evm}.go (Phase 2 ledger-root anchor) import
luxfi/geth and luxfi/crypto directly, so they are no longer indirect.
go mod tidy result; no version change.
2026-07-05 19:52:04 -07:00
d1974cae85 feat(treasury): delegate the native ledger to ledgercore — ONE double-entry engine (#145)
Collapse the treasury's separately-written double-entry SQL onto ledgercore
(github.com/hanzo-fi/ledger) — the SAME engine the ledger's own store uses — so
there is exactly one double-entry implementation across the stack (church of
Rich Hickey: one double-entry value, not three places).

- Reimplement clients/treasury/ledger/sqlstore to back the ledger.Store/ledger.Tx
  port with ledgercore instead of hand-rolled treasury_postings SQL. The
  accounting truth — every balance and the reserve overdraw guard — is now
  ledgercore's (postings -> moves -> balances + hash-chained log, idempotency-key
  dedup, WithTx atomic read-then-write). The adapter only maps the treasury's
  vocabulary (int64 cents, Kind/Program/Ref key, signed-Posting Entry) onto it.

- KEEP the port/adapter seam: Open()'s signature is unchanged, so treasury.go and
  the Formance-HTTP opt-in are untouched — native (ledgercore) stays the default
  backend. The engine (ledger.go) and the on-chain Root are UNCHANGED: each Entry
  is round-tripped verbatim (as ledgercore transaction metadata), so the Root is
  byte-identical to the previous store's, independent of ledgercore's own postings.

- Policy (revenue-share bps) stays in a small side table — it is Hanzo config, not
  double-entry accounting, so it does not belong in the shared engine.

- Pin bun to v1.2.9 (replace): ledger-fi floors v1.2.18, which removed
  schema.Formatter/NewFormatter/Append that hanzoai/o11y still uses; ledgercore's
  compiled closure uses no v1.2.18-only API, so v1.2.9 satisfies both. hanzo-fi/ledger
  is pinned to the PR-3 branch commit until it merges.

Tests (all green, incl. -race): overdraw guard, at-most-once payout, snapshot
reconcile, scope isolation, and Tx rollback all pass unchanged against the
ledgercore-backed store. The whole cloud module builds under -mod=readonly, and
the treasury test binary links NO modernc driver (so it does not reintroduce the
"sqlite registered twice" panic).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-05 19:14:26 -07:00
99dada4dde feat(treasury): activate Hanzo L1 (36963) anchor — EIP-1559 gas + deploy tool (#147)
The 36963 coreth fee market pins a 25 gwei min base fee, so a legacy tx priced
at base+1 strands the moment the base fee ticks up. anchor_evm.go now submits a
DynamicFeeTx (1 gwei tip floor, 2x-base-fee cap) — proven accepted on-chain as a
type-2 tx.

Adds clients/treasury/cmd/anchorctl: a one-shot in-cluster tool that provisions
the KMS-held signer (key -> KMS, only the address printed), funds it from a
genesis account, deploys contracts/TreasuryAnchor.sol, and can send anchor(bytes32).
Includes the compiled TreasuryAnchor.bin (solc 0.8.26, optimizer 200, cancun).

Deployed live: contract 0x53141dF42DF13Aad0512f2F08c3E3216EEFac5F2, owner = signer
0x703D4227d58d0b6A20BD721c940CED170470f634 (KMS ref hanzo/treasury-anchor/TREASURY_ANCHOR_SIGNER_KEY).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-05 19:11:05 -07:00
48bbcc04fb fix(test): canonical make test-race — one sqlite registration under -race (#48) (#146)
The cloud registers the "sqlite" driver exactly once in every build mode EXCEPT
a naive `go test -race`: -race forces CGO=1, which links the fork's mattn
"sqlite" (github.com/hanzoai/sqlite) ALONGSIDE the embedded deps that import
modernc directly (ai/base/commerce/o11y/orm), so both register "sqlite" and the
binary panics at init ("sql: Register called twice for driver sqlite") — the
pre-existing failure in clients/{graph,kmssvc,o11y}.

`make test` (CGO=0) and `make test-cgo` (-tags sqlite_purego) already avoid this
by resolving the whole binary to modernc's single registration. This adds the
missing peer for the race detector: `make test-race` runs
`CGO_ENABLED=1 go test -race -tags sqlite_purego ./...` — CGO on for the race
instrumentation, but the fork forced to its pure-Go backend so mattn never
registers and "sqlite" is registered exactly once. The ONE way to race-test the
cloud.

Proof: `go test -race ./clients/o11y/` panics; `go test -race -tags sqlite_purego
./clients/{graph,kmssvc,o11y}/` all pass.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-05 18:42:18 -07:00
0a68db08e4 feat(finance): per-org /v1/finance/* projecting the commerce + treasury planes (#144)
The finance.hanzo.ai + console Finance surfaces render real per-org data
instead of preview stubs. This adds no billing system — it PROJECTS the two
that already exist (the commerce customer wallet + the treasury reserve fund)
into the @hanzo/finance-ui contract (USD cents, optional-safe), scoped to the
validated IAM owner.

clients/billing/finance.go — six commerce-projected reads, reusing this
package's commerceProxy + per-org subject-pinning (one commerce read path):
  GET /v1/finance/balance          commerce balance (holds -> pendingCents)
  GET /v1/finance/credits          commerce deposit rows (grants, positive)
  GET /v1/finance/usage?range=     commerce withdraw rows -> series+lines+total
  GET /v1/finance/invoices         honest empty (no invoice ledger exists yet)
  GET /v1/finance/payment-methods  commerce portal, masked to brand+last4
  GET /v1/finance/ledger?range=    commerce ledger -> signed per-org postings

clients/treasury/treasury.go — GET /v1/finance/treasury reshaped from the
reserve Report into the TreasurySummary shape (reserve/committed/available +
honest Hanzo L1 anchor); the transparency policy rides along additively.

Tenant isolation: org A never sees org B (per-org subject pinned server-side,
client cannot widen scope); payment methods re-masked defensively so a PAN can
never leak. Honest empty/typed shapes where a data source does not exist yet.

Tests: go test -race ./clients/billing/... ./clients/treasury/... green;
go build ./cmd/cloud green.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-05 14:19:36 -07:00
b07c945f80 feat(treasury): native reserve fund + backed payouts + Formance ledger-of-record + Hanzo L1 anchor (#143)
* feat(treasury): native double-entry reserve fund + backed-payout seam (#treasury)

The platform's OWN fund/reserve accounting, one layer ABOVE the per-org commerce
credit ledger. A store-agnostic, cloud-decoupled double-entry engine
(clients/treasury/ledger) — the SEED of the native hanzoai/finance central ledger
(the Go replacement for the Formance stack) — plus a Base/SQLite adapter
(ledger/sqlstore) and the cloud client (clients/treasury).

Core (clients/treasury/ledger): accounts + balanced journal entries (Σ postings==0,
refused otherwise), ONE shared fund:reserve pool with per-program payout sinks,
revenue-share policy (bps, one place), and the reserve GUARD — a fund debit that
would overdraw is refused, atomically, so growth-loop payouts are backed capital not
unbounded minting. Zero cloud/zip/SQLite imports; persistence is the Store/Tx port,
so it lifts to hanzoai/finance as a directory move.

Surface: GET /v1/treasury (org transparency), GET /v1/admin/treasury (report +
journal + anchor), POST /v1/admin/treasury/{policy,sweep,seed,anchor} (global-admin).
treasury.Reserve(program,ref,memo,cents) is the ONE seam the 3 loops call: backed →
proceed to credit; not backed → honestly pending; unmounted → passthrough
(backward-safe). Idempotent by ref (at-most-once fund debit). ledger.Root commits the
whole journal for the Hanzo L1 anchor (Phase 2 wires the KMS-signed submit).

Tests (-race, green): double-entry balances, revenue-share accrual + per-period
idempotency, reserve guard (backed→blocked), at-most-once, concurrent no-overdraw,
admin gate, Reserve passthrough+enforced, sqlstore round-trip + tx rollback.

* feat(finance): Formance ledger-of-record backend + backed payouts + scope-aware /v1/finance/*

Adopt Formance as the ledger of record behind a ledger.Backend PORT, without
reimplementing double-entry: two adapters satisfy the port — the native Base/SQLite
engine (offline/default, ships the reserve fund today) and clients/treasury/formance
(a real HTTP client to the Postgres-backed Formance Ledger v2 API: world→fund accrual,
fund→payout debit, 400 INSUFFICIENT_FUND→not-backed=the overdraw guard Formance
enforces, reference→idempotency). Select by FORMANCE_LEDGER_URL — a config flip. Root
computed via a SHARED hash so the L1 anchor is backend-agnostic.

Back the growth-loop payouts: referrals/affiliates/authors now DEBIT the reserve fund
via the ONE treasury.Reserve seam before crediting the recipient wallet — fund down,
wallet up, reconciled. Not backed → honestly pending (referrals) or 402 + VoidPayout
restores pending (affiliates/authors). Idempotent by ref (at-most-once). Unmounted →
passthrough (backward-safe; existing loop suites stay green).

Scope-aware /v1/finance/* — ONE engine, three tenancy surfaces (admin/console/finance
product): tenant derived from IAM, house/reserve locked to global-admin under
/v1/admin/finance/*, per-org callers see ONLY their own org:<tenant>:* accounts.
GET /v1/finance/accounts (per-org; admin ?scope=house|?org=<t>). Storage tiers doc'd:
authoritative OLTP ledger (native/Formance) + ClickHouse OLAP projection over the same
o11y event stream (audit mirror — no second metering pipeline).

Tests (-race, green): Formance adapter (accrual+idempotency, debit guard+replay,
snapshot) via a fake Formance server; scope isolation (per-org never sees house);
backed-payout enforced+blocked+at-most-once; VoidPayout restores pending.

* feat(treasury): Phase 2 — Hanzo L1 (36963) ledger-root anchor (contract + luxfi/geth submit + KMS signer)

Make the off-chain books tamper-evident on the LIVE Hanzo L1 (verified running:
network/chainId 36963, hanzod-0 producing blocks, EVM at network-36963).

- contracts/TreasuryAnchor.sol: minimal immutable witness — owner-gated anchor(bytes32)
  appends a timestamped root + emits Anchored; latest()/count for cheap verification.
  No upgradeability, no token — one job.
- anchor_evm.go: real luxfi/geth submitter — dial → chainID/nonce/gasPrice → sign a
  LegacyTx (anchor(bytes32) call when TREASURY_ANCHOR_CONTRACT set, else a 0-value
  self-tx carrying the root) with types.SignTx → send → await receipt → persist. The
  signer key is provisioned from KMS (KMSSecret → env TREASURY_ANCHOR_SIGNER_KEY,
  ref TREASURY_ANCHOR_SIGNER_KMS_REF) — NEVER plaintext in code/manifest.
- ledger.Root/ComputeRoot: deterministic SHA-256 hash-chain over the whole journal +
  reserve, shared by both backends so the anchor is backend-agnostic. A change to any
  historical posting changes the root.
- POST /v1/admin/treasury/anchor submits when wired; else returns the root that WOULD
  be committed + the EXACT remaining step. GET /v1/admin/treasury shows last anchored
  root/tx/block + synced flag. Persisted across restart (treasury_anchor.json).

Honest status: the on-chain submit is COMPLETE + compiling + config-gated but NOT
driven live this pass — the node's external JSON-RPC is unreachable from the build
env and needs an operator to: deploy TreasuryAnchor on 36963, provision the KMS
signer (fund it), set TREASURY_ANCHOR_{RPC_URL,CONTRACT,SIGNER_KEY}. In-cluster the
cloud binary reaches hanzod-rpc-internal:9630, so it's one deploy-config away.

Builds green (cmd/cloud links luxfi/geth); tests -race green; gofmt clean.

---------

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-05 13:38:29 -07:00
hanzo-dev 21660cfa0f feat(cli): hanzo engine install|serve|status
Manage a local hanzo-engine (the `hanzoai` OpenAI + Anthropic model server)
from the canonical `hanzo` CLI:
- install: runs the canonical install.sh / install.ps1 (single source of truth
  for platform detection + signature verification — no re-implementation).
- serve:  launches the installed binary (`hanzoai --port P run -m MODEL`);
  syscall.Exec on Unix so signals + exit code flow through.
- status: probes the local engine, reusing the /v1/models probe that
  `hanzo gpu connect --serve-engine` advertises with.
Tests cover wiring, ready/unreachable status, and binary discovery.
2026-07-05 13:27:21 -07:00
hanzo-dev ab5c81b699 feat(treasury): Phase 2 — Hanzo L1 (36963) ledger-root anchor (contract + luxfi/geth submit + KMS signer)
Make the off-chain books tamper-evident on the LIVE Hanzo L1 (verified running:
network/chainId 36963, hanzod-0 producing blocks, EVM at network-36963).

- contracts/TreasuryAnchor.sol: minimal immutable witness — owner-gated anchor(bytes32)
  appends a timestamped root + emits Anchored; latest()/count for cheap verification.
  No upgradeability, no token — one job.
- anchor_evm.go: real luxfi/geth submitter — dial → chainID/nonce/gasPrice → sign a
  LegacyTx (anchor(bytes32) call when TREASURY_ANCHOR_CONTRACT set, else a 0-value
  self-tx carrying the root) with types.SignTx → send → await receipt → persist. The
  signer key is provisioned from KMS (KMSSecret → env TREASURY_ANCHOR_SIGNER_KEY,
  ref TREASURY_ANCHOR_SIGNER_KMS_REF) — NEVER plaintext in code/manifest.
- ledger.Root/ComputeRoot: deterministic SHA-256 hash-chain over the whole journal +
  reserve, shared by both backends so the anchor is backend-agnostic. A change to any
  historical posting changes the root.
- POST /v1/admin/treasury/anchor submits when wired; else returns the root that WOULD
  be committed + the EXACT remaining step. GET /v1/admin/treasury shows last anchored
  root/tx/block + synced flag. Persisted across restart (treasury_anchor.json).

Honest status: the on-chain submit is COMPLETE + compiling + config-gated but NOT
driven live this pass — the node's external JSON-RPC is unreachable from the build
env and needs an operator to: deploy TreasuryAnchor on 36963, provision the KMS
signer (fund it), set TREASURY_ANCHOR_{RPC_URL,CONTRACT,SIGNER_KEY}. In-cluster the
cloud binary reaches hanzod-rpc-internal:9630, so it's one deploy-config away.

Builds green (cmd/cloud links luxfi/geth); tests -race green; gofmt clean.
2026-07-05 13:21:41 -07:00
hanzo-dev 4330dd5444 feat(gpu): engine.serve — a connected GPU serves hanzo-engine models
`hanzo gpu connect --serve-engine` advertises a local hanzo-engine (the OpenAI +
Anthropic model server on :1234) on the org fleet, alongside the existing
studio.render worker. The worker probes GET {engine-url}/v1/models, publishes the
endpoint + model list in its presence record, and prints (or with --register-provider
POSTs) the /v1/add-provider call that routes api.hanzo.ai model traffic to this GPU as
an OpenAI-compatible (Type=Local) provider.

- cli/gpu.go: --serve-engine/--engine-url/--engine-endpoint/--register-provider;
  probeEngine, refreshEngine, engineAdvertisement, capabilities, provider hint;
  `hanzo gpu status` shows the engine endpoint.
- clients/visor/fleet.go: byoWorker + fleetRegistration carry capabilities + engine;
  GET /v1/fleet/workers advertises the endpoint (additive, omitempty).
- docs/bring-your-gpu.md: Connect (BYO) vs Deploy (cloud) -> engine.serve + studio.render.
- tests: probe/advertise/registration + full stub-cloud round-trip (no model needed).

One fleet, two job types: engine.serve (model serving) + studio.render (diffusion).
2026-07-05 13:12:30 -07:00
hanzo-dev 4cc966fb45 feat(finance): Formance ledger-of-record backend + backed payouts + scope-aware /v1/finance/*
Adopt Formance as the ledger of record behind a ledger.Backend PORT, without
reimplementing double-entry: two adapters satisfy the port — the native Base/SQLite
engine (offline/default, ships the reserve fund today) and clients/treasury/formance
(a real HTTP client to the Postgres-backed Formance Ledger v2 API: world→fund accrual,
fund→payout debit, 400 INSUFFICIENT_FUND→not-backed=the overdraw guard Formance
enforces, reference→idempotency). Select by FORMANCE_LEDGER_URL — a config flip. Root
computed via a SHARED hash so the L1 anchor is backend-agnostic.

Back the growth-loop payouts: referrals/affiliates/authors now DEBIT the reserve fund
via the ONE treasury.Reserve seam before crediting the recipient wallet — fund down,
wallet up, reconciled. Not backed → honestly pending (referrals) or 402 + VoidPayout
restores pending (affiliates/authors). Idempotent by ref (at-most-once). Unmounted →
passthrough (backward-safe; existing loop suites stay green).

Scope-aware /v1/finance/* — ONE engine, three tenancy surfaces (admin/console/finance
product): tenant derived from IAM, house/reserve locked to global-admin under
/v1/admin/finance/*, per-org callers see ONLY their own org:<tenant>:* accounts.
GET /v1/finance/accounts (per-org; admin ?scope=house|?org=<t>). Storage tiers doc'd:
authoritative OLTP ledger (native/Formance) + ClickHouse OLAP projection over the same
o11y event stream (audit mirror — no second metering pipeline).

Tests (-race, green): Formance adapter (accrual+idempotency, debit guard+replay,
snapshot) via a fake Formance server; scope isolation (per-org never sees house);
backed-payout enforced+blocked+at-most-once; VoidPayout restores pending.
2026-07-05 13:08:24 -07:00
hanzo-dev bd51d6eef6 feat(treasury): native double-entry reserve fund + backed-payout seam (#treasury)
The platform's OWN fund/reserve accounting, one layer ABOVE the per-org commerce
credit ledger. A store-agnostic, cloud-decoupled double-entry engine
(clients/treasury/ledger) — the SEED of the native hanzoai/finance central ledger
(the Go replacement for the Formance stack) — plus a Base/SQLite adapter
(ledger/sqlstore) and the cloud client (clients/treasury).

Core (clients/treasury/ledger): accounts + balanced journal entries (Σ postings==0,
refused otherwise), ONE shared fund:reserve pool with per-program payout sinks,
revenue-share policy (bps, one place), and the reserve GUARD — a fund debit that
would overdraw is refused, atomically, so growth-loop payouts are backed capital not
unbounded minting. Zero cloud/zip/SQLite imports; persistence is the Store/Tx port,
so it lifts to hanzoai/finance as a directory move.

Surface: GET /v1/treasury (org transparency), GET /v1/admin/treasury (report +
journal + anchor), POST /v1/admin/treasury/{policy,sweep,seed,anchor} (global-admin).
treasury.Reserve(program,ref,memo,cents) is the ONE seam the 3 loops call: backed →
proceed to credit; not backed → honestly pending; unmounted → passthrough
(backward-safe). Idempotent by ref (at-most-once fund debit). ledger.Root commits the
whole journal for the Hanzo L1 anchor (Phase 2 wires the KMS-signed submit).

Tests (-race, green): double-entry balances, revenue-share accrual + per-period
idempotency, reserve guard (backed→blocked), at-most-once, concurrent no-overdraw,
admin gate, Reserve passthrough+enforced, sqlstore round-trip + tx rollback.
2026-07-05 12:48:24 -07:00
f6702f76bc feat(iam): embed IAM in the unified cloud binary (last binary-consolidation piece) (#142)
* feat(iam): embed IAM in the unified cloud binary as an in-process subsystem

Folds Hanzo IAM -- the identity provider serving hanzo.id (login/authorize/
token/jwks/userinfo, /v1/iam/* admin, OAuth2/OIDC, LDAP/RADIUS) -- into the
unified hanzoai/cloud binary as the LAST binary-consolidation piece
(HIP-0106: "one Go binary embeds IAM + KMS + o11y").

clients/iamsvc wraps IAM's own Beego runtime: iamserver.Init() runs the full
bootstrap without binding a listener, and web.BeeApp.Handlers is mounted
verbatim on cloud's zip.App at every prefix IAM owns (/v1/iam/*,
/.well-known/*, /login/oauth/*, /_/iam/*, /cas/*, /scim/*). No auth logic is
reimplemented -- the same controllers answer, so OAuth/OIDC semantics
(authorize clientId org-resolution, JWT audiences, SuperAdmin owner=="admin",
argon2id password hashing) are preserved byte-for-byte. Registered at order 50
(identity authority, mounts before dependents).

- go.mod: pin hanzoai/iam v1.28.12 -> v1.31.16 (latest; carries the
  authorize-login org-resolution fixes #95/#96 the operator SSO chain needs).
- subsystems.go: blank-import clients/iamsvc; IAM no longer "NOT fused in".

Auth-critical middleware interactions verified: /v1/iam/* prices to 0 in
DefaultPrice (ungated -- the M2M /v1/iam/oauth/token mint is never charged);
SanitizeIdentity strips only forgeable X-User-*/X-Org-* headers, never the
Authorization bearer or iam_session_id cookie IAM's session/oauth logic reads.

Activation is STAGED via the enable-list gate: "iam" is NOT added to the live
--enable until IAM config is present in the cloud runtime and the fold is
verified (login/authorize/token/jwks + operator SSO chain). The standalone iam
pod keeps serving hanzo.id via ingress until then.

Build gate: CGO_ENABLED=0 go build ./... && go test . green. clients/iamsvc
tests prove registration (order 50) + full-path preservation through the mount.

* fix(iam): red-review — embed-mode bootstrap, fail-closed, single-replica guard

Addresses the red review of cloud#142 (mount mechanism approved; activation
blocked on standalone-only side effects in the wrapped entrypoint).

1. [HIGH] Embed-mode bootstrap. iamsvc now calls iamserver.InitEmbed (new in
   iam v1.31.17) instead of the standalone Init: skips StopOldInstance
   (lsof/SIGKILL — panics on distroless, kills a co-resident on shared netns),
   skips LDAP/RADIUS listeners (RADIUS binds unmanaged UDP with an empty shared
   secret), skips export/os.Exit, binds no listener. Standalone hanzo iam / iamd
   is byte-for-byte unchanged (Init delegates to the same shared bootstrap with
   every flag on). Also covers [MED] #3 — directory listeners never start
   in-process.

4. [MED] Fail-closed, not fail-loud. InitEmbed returns an error (recovers
   bootstrap panics); a broken/misconfigured IAM degrades THIS subsystem to a
   503 fail-closed on every IAM prefix (mountFailClosed) — every co-resident
   subsystem (KMS, o11y) stays up. Mirrors the KMS "no master key -> health-only"
   blast-radius isolation.

2. [HIGH] Single-replica enforcement. Embedded IAM uses Beego's process-local
   "memory" session store. Config.Validate now REFUSES to boot iam-enabled above
   CLOUD_REPLICAS=1 (a real runtime guard, not convention); the helm chart pins
   replicas=1 + injects CLOUD_REPLICAS whenever "iam" is in --enable.

5. [MED] Bump verified iam v1.31.16 -> v1.31.17. The slim-JWT change keeps every
   claim cloud reads (owner, isAdmin, email, name kept; aud is a registered
   claim, untouched) — IdentityMiddleware unaffected. authz v1.10.4 policy-API
   swap is IAM-internal (cloud builds green, no direct use). redirect_uri
   exact-match + AutoSignin CC-JWT normalization are version-skew CUTOVER gates:
   version-match the standalone pod + verify registered redirect_uris are exact
   before adding "iam" to the live --enable (runtime-data checklist, not code).

Tests (CGO_ENABLED=0):
- TestIAMEmbedBehindMiddlewareChain — unauth POST /v1/iam/oauth/token, /login,
  jwks + /login/oauth/authorize return 2xx through the REAL SanitizeIdentity +
  BillingGate chain (never 402/503); forged X-User-IsAdmin is stripped; a priced
  control path is denied at zero balance (proves the gate is engaged).
- TestDefaultPriceExemptsIAM — every IAM prefix prices to 0.
- TestValidateIAMSingleReplica — iam + replicas>1 refused; 1/unset/off ok.
- TestMountFailClosed503 — the fail-soft path serves 503 on every IAM prefix.

Build+test green; standalone hanzo iam still links; helm renders replicas=1 for
iam-enabled, replicaCount otherwise. Depends on iam v1.31.17
(hanzoai/iam#feat/iam-embed-entrypoint). STILL STAGED — the standalone iam pod
serves hanzo.id until red GREEN + runtime e2e.

---------

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-05 12:21:42 -07:00
hanzo-dev 3d11e0be1a fix(iam): red-review — embed-mode bootstrap, fail-closed, single-replica guard
Addresses the red review of cloud#142 (mount mechanism approved; activation
blocked on standalone-only side effects in the wrapped entrypoint).

1. [HIGH] Embed-mode bootstrap. iamsvc now calls iamserver.InitEmbed (new in
   iam v1.31.17) instead of the standalone Init: skips StopOldInstance
   (lsof/SIGKILL — panics on distroless, kills a co-resident on shared netns),
   skips LDAP/RADIUS listeners (RADIUS binds unmanaged UDP with an empty shared
   secret), skips export/os.Exit, binds no listener. Standalone hanzo iam / iamd
   is byte-for-byte unchanged (Init delegates to the same shared bootstrap with
   every flag on). Also covers [MED] #3 — directory listeners never start
   in-process.

4. [MED] Fail-closed, not fail-loud. InitEmbed returns an error (recovers
   bootstrap panics); a broken/misconfigured IAM degrades THIS subsystem to a
   503 fail-closed on every IAM prefix (mountFailClosed) — every co-resident
   subsystem (KMS, o11y) stays up. Mirrors the KMS "no master key -> health-only"
   blast-radius isolation.

2. [HIGH] Single-replica enforcement. Embedded IAM uses Beego's process-local
   "memory" session store. Config.Validate now REFUSES to boot iam-enabled above
   CLOUD_REPLICAS=1 (a real runtime guard, not convention); the helm chart pins
   replicas=1 + injects CLOUD_REPLICAS whenever "iam" is in --enable.

5. [MED] Bump verified iam v1.31.16 -> v1.31.17. The slim-JWT change keeps every
   claim cloud reads (owner, isAdmin, email, name kept; aud is a registered
   claim, untouched) — IdentityMiddleware unaffected. authz v1.10.4 policy-API
   swap is IAM-internal (cloud builds green, no direct use). redirect_uri
   exact-match + AutoSignin CC-JWT normalization are version-skew CUTOVER gates:
   version-match the standalone pod + verify registered redirect_uris are exact
   before adding "iam" to the live --enable (runtime-data checklist, not code).

Tests (CGO_ENABLED=0):
- TestIAMEmbedBehindMiddlewareChain — unauth POST /v1/iam/oauth/token, /login,
  jwks + /login/oauth/authorize return 2xx through the REAL SanitizeIdentity +
  BillingGate chain (never 402/503); forged X-User-IsAdmin is stripped; a priced
  control path is denied at zero balance (proves the gate is engaged).
- TestDefaultPriceExemptsIAM — every IAM prefix prices to 0.
- TestValidateIAMSingleReplica — iam + replicas>1 refused; 1/unset/off ok.
- TestMountFailClosed503 — the fail-soft path serves 503 on every IAM prefix.

Build+test green; standalone hanzo iam still links; helm renders replicas=1 for
iam-enabled, replicaCount otherwise. Depends on iam v1.31.17
(hanzoai/iam#feat/iam-embed-entrypoint). STILL STAGED — the standalone iam pod
serves hanzo.id until red GREEN + runtime e2e.
2026-07-05 11:36:15 -07:00
hanzo-dev 68a7e52c4a refactor(o11y): zaptrace ships OTLP over ZAP with no gRPC dep
The ZAP-native trace exporter marshaled its payload via the generated
otlp/collector/trace/v1.ExportTraceServiceRequest, whose sibling
trace_service_grpc.pb.go (no build tag) drags google.golang.org/grpc into the
graph — contradicting the exporter's own contract (ZAP wire, never gRPC).

Encode the ExportTraceServiceRequest envelope directly from the grpc-free trace
messages with protowire: it is a single 'repeated ResourceSpans resource_spans
= 1', so appending each ResourceSpans under field 1 is byte-identical to the
generated marshaler (proven by the existing round-trip test, which still decodes
with the canonical collector type). go list -deps ./zaptrace now shows no grpc.
Hanzo services speak ZAP/HTTP/WS, never gRPC.

Caveat: the cloud module still pulls google.golang.org/grpc transitively via
hanzoai/ai (sibling-owned), hanzoai/o11y (embedded SigNoz — intrinsically an
OTLP/gRPC collector) and hanzoai/base (GCS gRPC transport). Not removable by a
cloud-local change; tracked separately. go.sum: incidental tidy prune of stale
vfs/age checksums.
2026-07-05 11:19:32 -07:00
hanzo-dev 7a2857b1c9 feat(iam): embed IAM in the unified cloud binary as an in-process subsystem
Folds Hanzo IAM -- the identity provider serving hanzo.id (login/authorize/
token/jwks/userinfo, /v1/iam/* admin, OAuth2/OIDC, LDAP/RADIUS) -- into the
unified hanzoai/cloud binary as the LAST binary-consolidation piece
(HIP-0106: "one Go binary embeds IAM + KMS + o11y").

clients/iamsvc wraps IAM's own Beego runtime: iamserver.Init() runs the full
bootstrap without binding a listener, and web.BeeApp.Handlers is mounted
verbatim on cloud's zip.App at every prefix IAM owns (/v1/iam/*,
/.well-known/*, /login/oauth/*, /_/iam/*, /cas/*, /scim/*). No auth logic is
reimplemented -- the same controllers answer, so OAuth/OIDC semantics
(authorize clientId org-resolution, JWT audiences, SuperAdmin owner=="admin",
argon2id password hashing) are preserved byte-for-byte. Registered at order 50
(identity authority, mounts before dependents).

- go.mod: pin hanzoai/iam v1.28.12 -> v1.31.16 (latest; carries the
  authorize-login org-resolution fixes #95/#96 the operator SSO chain needs).
- subsystems.go: blank-import clients/iamsvc; IAM no longer "NOT fused in".

Auth-critical middleware interactions verified: /v1/iam/* prices to 0 in
DefaultPrice (ungated -- the M2M /v1/iam/oauth/token mint is never charged);
SanitizeIdentity strips only forgeable X-User-*/X-Org-* headers, never the
Authorization bearer or iam_session_id cookie IAM's session/oauth logic reads.

Activation is STAGED via the enable-list gate: "iam" is NOT added to the live
--enable until IAM config is present in the cloud runtime and the fold is
verified (login/authorize/token/jwks + operator SSO chain). The standalone iam
pod keeps serving hanzo.id via ingress until then.

Build gate: CGO_ENABLED=0 go build ./... && go test . green. clients/iamsvc
tests prove registration (order 50) + full-path preservation through the mount.
2026-07-05 11:02:49 -07:00
b641365ed7 feat(authors): native /v1/authors OSS-author deploy-royalty loop over the commerce ledger (#141)
The THIRD growth loop next to referrals (one-time credit) and affiliates
(partner commission): pays open-source AUTHORS a royalty on the metered platform
spend of orgs who DEPLOY their projects on Hanzo. Mirrors clients/affiliates
exactly — one SQLite store, server-side tenant isolation, one Mount (HIP-0106),
the SAME commerce ledger path (a credits payout is a grant, tag grant:author),
and an at-most-once accrual latch.

Flow: connect GitHub (IAM-linked account or supplied login) → verify repo
ownership (OAuth admin-check OR a hanzo.json verify-code file) → a deploy of a
verified author repo by ANY org is recorded (provenance) → sweep accrues 5% of
that org's month-to-date spend, at-most-once per (author, deploying-org, period),
self-deploys excluded → staff pay out as credits (real grant) or cash (record-only),
never exceeding pending.

Surface: GET /v1/authors, POST /v1/authors/{connect,repos/verify,deploys/record};
GET /v1/admin/authors, POST /v1/admin/authors/{sweep,:id/approve,:id/suspend,:id/payout}.

10 tests, all -race green: repo canonicalization, both verify methods, deploy
attribution + idempotency, spend×share accrual + at-most-once, lazy dashboard
sweep, credits-one-grant/cash-record-only/pending-guard payout, admin gate.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-05 06:11:55 -07:00
bfaa3c4360 feat(affiliates): native /v1/affiliates partner-commission loop over the commerce ledger (#140)
Mirrors clients/referrals: one SQLite store, server-side tenant isolation, one
HIP-0106 Mount, admin surface global-admin-gated + enveloped for the console
proxy. Affiliates earn an ONGOING commission (default 20%) on the metered spend
of the customers they refer — the recurring, partner-revenue growth loop beside
referrals' one-time both-sides credit.

- apply (org) -> status applied; staff approve mints the code (vanity opt-in,
  uniqueness-enforced, else a derived slug) + sets the rate.
- attribute (?aff capture) records referred_org->affiliate (first-touch, one per
  referred org, self blocked; approved affiliates only).
- accrual sweep: commission = referred org spend this period x rate, latched
  at-most-once per (affiliate, referred_org, period) in one txn; also lazy on the
  affiliate's own dashboard read.
- payout: a credits method issues a commerce grant (tag grant:affiliate); cash
  methods are record-only; can never exceed pending (accrued - paid), reserved
  atomically before any grant.

Tests (go test -race, 9 green): apply->approve, vanity uniqueness (409),
accrual = spend x rate, idempotent-per-period sweep, payout-as-credits issues one
grant + cash record-only + pending guard, admin gate 403, attribution
self/unknown/first-touch, Mount.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-05 05:34:03 -07:00
e78cacb49d feat(referrals): native /v1/referrals viral loop over the commerce ledger (#139)
* feat(referrals): native /v1/referrals viral loop over the commerce ledger

Per-org referral program mirroring clients/crm's structure (one SQLite store,
server-side tenant isolation, HIP-0106 Mount). Grants promo credit through the
SAME commerce deposit path as clients/admin.grantCredit (trial/Credit bucket,
tag grant:referral).

- Stable deterministic referral code per org (base32 of a hash of the org id) +
  a white-labeled ?ref link; persisted directory for O(1) reverse lookup.
- POST /v1/referrals/claim: record referrer<->referee (referee = validated
  caller), status signed_up. Self-referral blocked, one-per-referee idempotent
  (first-touch wins).
- Qualify signal = referee metered spend (honest 'actually used the product').
  On qualify, grant BOTH sides: referrer +$10, referee +$5. At-most-once via a
  credited_at latch — no sweep and no concurrent read can double-pay.
- Trigger: lazy on the referrer's GET /v1/referrals + POST /v1/admin/referrals/
  sweep (cron path). GET /v1/admin/referrals directory, both global-admin gated.
- Constants (bonus amounts + ledger tag) in one place. Commerce behind an
  interface for testable double-grant/idempotency proofs.

Tests: code derivation, self-ref block, idempotent claim, qualify->double-grant
with balances moving through the (fake) ledger, at-most-once idempotency, lazy
qualify on read, admin gate + directory, real Mount. All green (go test -race).

* feat(referrals): envelope the /v1/admin/referrals surface for the console admin proxy

---------

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-05 05:01:58 -07:00
hanzo-dev e2161d7e5b refactor(org): use the shared hanzoai/vfs/replica — kill the duplicate Replicator+election
cloud's internal/org was the ORIGIN of the HA-SQLite machinery; it's now promoted to the
shared hanzoai/vfs/replica lib that every service adopts. Delete the duplicated impls
(replica.go Replicator+Store+DB+DBPath, owner.go Member+Owner+IsOwner+Replicas+HRW) and
re-export them as aliases (shared.go). cloud-specific pieces stay: membership.go (live IAM
source), cipher.go (KMS envelope — already satisfies replica.Cipher), vfsstore.go (Store over
deps.VFS, now using the exported replica.Version). One and one way: ONE Replicator + election,
in vfs/replica, used by cloud AND visor. Builds + org tests green (vfs v0.6.2).
2026-07-05 00:43:11 -07:00
45d9e5a885 fix(deps): bump hanzoai/ai v1.800.7 → v1.800.9 (zen context length → unblocks console chat) (#138)
Zen models were capped at the 4096 fallback in getContextLength, so every
console chat (grounded assistant ~4190-token system prompt) 402'd
'exceeds maximum token count: 4096'. v1.800.9 special-cases the zen* prefix
to 131072. Fixes the P0 console-chat gate.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-05 00:23:17 -07:00
hanzo-devandGitHub 726256b434 Merge pull request #137 from hanzoai/fix/o11y-embed-metrics-port-9090
fix(o11y embed): disable runtime :9090 self-metrics reader — crash-loop guard
2026-07-04 23:15:42 -07:00
hanzo-dev 4e89e21a7c o11y embed: disable the runtime's :9090 self-metrics reader (crash-loop guard)
The embedded o11y runtime's OTel instrumentation defaults to a Prometheus pull
reader bound to 0.0.0.0:9090 (pkg/instrumentation) — the SAME port as cloud's
health listener (CLOUD_HEALTH_LISTEN=:9090). Activating the embed therefore made
the whole cloud process crash-loop with 'listen tcp :9090: bind: address already
in use' (verified on the canary), taking down all of api.hanzo.ai — a listener the
standalone o11y pod never contended for.

buildEmbeddedHandler now defaults O11Y_INSTRUMENTATION_METRICS_ENABLED=false (via
setenvDefault, operator-overridable to a free port) before construction. Cloud owns
process-level observability (exports its own OTel telemetry), so the embed serves
/v1/o11y in-process without a second metrics listener. Extracted the env defaults
into applyEmbedEnvDefaults + TDD (guard + operator-override).

Verified on the cloud-unified-canary: with this default the .104 embed goes Ready
and serves /v1/o11y in-process (health 200); without it the pod crash-loops on :9090.
CGO=0 go build/test green.
2026-07-04 23:15:22 -07:00
hanzo-devandGitHub 3f0c91eb34 Merge pull request #136 from hanzoai/feat/o11y-embed-mainbased
o11y embed: shared community.NewServer (DRY) + health-exempt gate; o11y v1.5.0
2026-07-04 22:46:06 -07:00
hanzo-dev 81dd41e2ab o11y embed: use shared community.NewServer (DRY); exempt health from the gate
Refactor clients/o11y/embed.go onto o11y v1.5.0's shared builder community.NewServer
+ community.NewConfig — the EXACT construction the standalone o11y pod runs — so
the in-process runtime cannot drift from the pod's auth (pkg/identn/iamidentn,
Hanzo IAM gateway-header identity). Collapses the duplicated ~70-line signoz.New
factory list (drift risk) to one call. Enable signal now reads the flat operator
knob O11Y_DATASTORE_DSN (what the pod sets), falling back to the structured
O11Y_TELEMETRYSTORE_DATASTORE_DSN.

Exempt liveness/readiness paths from gate(): the runtime serves them without
identity (k8s probes pass that way), so gating them only breaks unauthenticated
health probes (admin System Health CLOUD_O11Y_HEALTH_URL, the external o11y.*
hosts) without protecting anything. Data routes stay gated (RED forge test still
403s /v1/o11y/api/v1/query_range).

go.mod: o11y v1.4.1 -> v1.5.0 (identical go.mod hash — no new transitive deps).
Build-gate: CGO_ENABLED=0 go build ./... = 0, go test . = ok, go test ./clients/o11y = ok.
2026-07-04 22:44:04 -07:00
zandGitHub 9b9aaf08d9 Merge pull request #135 from hanzoai/feat/o11y-embed-main-iamidentn
feat(o11y): embed the MAIN-based runtime (iamidentn) — auth matches the pod
2026-07-04 22:30:16 -07:00
hanzo-dev c6eba9ed42 feat(o11y): embed the MAIN-based runtime (iamidentn) — auth now matches the pod
The #133 embed pinned o11y v1.3.13, whose runtime authenticates via o11y-native
JWT (tokenizer.GetIdentity on the Authorization bearer). The live gateway-header
traffic the standalone o11y:0.2.0 pod serves — identity injected as X-Org-Id/
X-User-Id/X-User-Email by the gateway — would 401 against that. So activating the
v1.3.13 embed could not replace the pod.

This repoints the embed to the MAIN o11y line (v1.4.1), which resolves identity
through the IdentN resolver's iamidentn provider (default-enabled) from those
gateway session headers, with iamauthz (Hanzo IAM Casbin) for authorization —
the SAME auth model as the running pod. Gateway-header traffic authenticates
(200), not 401.

- clients/o11y/embed.go: build the runtime via pkg/signoz.New with the SAME
  provider factories the standalone cmd/community server uses (noop zeus,
  licensing, gateway, auditor, meterreporter; iamauthz; ClickHouse
  telemetrystore; sqlite sqlstore; IdentN to iamidentn), then app.NewServer to
  server.PublicHandler (new accessor, o11y v1.4.1). runtime.Start runs the
  registry background services (incl. the ruler/alert-rule-manager)
  non-blocking; we never call server.Start (cloud owns its HTTP listeners; OpAMP
  stays out-of-process). Gate/proxy-fallback structure (clients/o11y/o11y.go) is
  unchanged: still O11Y_TELEMETRYSTORE_DATASTORE_DSN-gated, still fail-soft to
  the reverse proxy.
- go.mod: hanzoai/o11y v1.3.13 to v1.4.1 (main line, iamidentn). Drop the stale
  replace prometheus/alertmanager to hanzoai/alertmanager v0.28.2 — it forced
  o11y's code onto the old fork whose api/v2 returns hanzoai/common types that
  clash with o11y v1.4.1's upstream prometheus/common structs. o11y v1.4.1 (and
  the pod) build against upstream prometheus/alertmanager v0.31.1; cloud has no
  direct alertmanager import, so it now matches.

Telemetry backend (ClickHouse datastore StatefulSet, cluster insights) is
untouched — the embedded runtime queries it over ClickHouse-native :9000.

Build-gate: CGO_ENABLED=0 go build ./... OK; go test ./clients/o11y/... OK; vet OK.
2026-07-04 22:22:49 -07:00
hanzo-devandGitHub 341e7c0aaf Merge pull request #134 from hanzoai/bump/tasks-v1.49.0
chore(deps): bump hanzoai/tasks v1.48.0 -> v1.49.0 (durable social primitives + gated auth)
2026-07-04 21:43:17 -07:00
hanzo-dev aff9fcee87 chore(deps): bump hanzoai/tasks v1.48.0 -> v1.49.0
v1.49.0 adds the durable workflow primitives social-orchestrator needs to run
on cloud's embedded gated engine (ServeGated :9999): signal-to-running-workflow
re-dispatch, continueAsNew, startChild, typed search attributes, workflowId
conflict policy, and the signalWithStart wire fix. No cloud code change — the
embedded engine + gated listener pick up the fixes on rebuild.
2026-07-04 21:42:56 -07:00
hanzo-devandGitHub 84ae716f74 feat(o11y): embed the o11y runtime in-process; retire the proxy path (#133)
Constructs the ONE hanzoai/o11y runtime IN-PROCESS (clients/o11y/embed.go) —
the SAME bootstrap the standalone cmd/server runs (o11y.New with its provider
factories -> app.NewServer -> server.PublicHandler) — and installs it via
o11y.SetHandler, so /v1/o11y/* is served by THIS binary against the ClickHouse
`datastore` (StatefulSet, cluster insights) instead of reverse-proxying a
standalone o11y Deployment. The standalone o11y pod can now retire; the
ClickHouse datastore stays as the telemetry backend.

- clients/o11y/embed.go: buildEmbeddedHandler wires telemetrystore (ClickHouse/
  datastore), sqlstore (sqlite under cloud's data root), querier, dashboards,
  alerts; starts the registry services + the alert rule manager (StartBackground).
  Enabled by O11Y_TELEMETRYSTORE_DATASTORE_DSN (the DSN is the one knob).
- o11y.go Register callback: prefer the in-process runtime; fall back to the
  reverse proxy when the embed is disabled (no DSN) or fails to init — fail-soft,
  zero downtime. Proxy handler + gate + tests retained for the fallback path.
- Bump hanzoai/o11y v1.3.12 -> v1.3.13 (adds Server.PublicHandler + StartBackground).
- Drop the stale `replace gorilla/mux => containous/mux`: it was a copied Traefik
  replace block; Traefik is not in the graph and nothing calls the containous API,
  but the fork lacks mux.MiddlewareFunc that o11y's otelmux needs. Standard
  gorilla/mux v1.8.1 satisfies every consumer.

Deferred (reported, not faked): OpAMP collector management (a second websocket
listener) is not started in-process — telemetry ingest continues on the existing
collector->datastore path. Build/test gate is CGO_ENABLED=0 (as prod ships): o11y
+ hanzoai/sqlite resolve to a single modernc sqlite driver registration.
2026-07-04 21:27:54 -07:00
zandGitHub 0a294af85e billing(#70): enforce per-scope spend caps + rate limits at the edge (metering v0.1.4)
Red SHIP; conflicts (only #45 clients/team) resolved to main; 22 money-path+regression tests green; go build clean. Edge calls standalone commerce /authorize; inert by default.
2026-07-04 20:53:45 -07:00
hanzo-dev 8b7a1cc57c Merge remote-tracking branch 'origin/main' into feat/scope-spend-limits
# Conflicts:
#	clients/team/account.go
#	clients/team/account_store.go
#	clients/team/account_store_test.go
#	clients/team/account_test.go
#	clients/team/bots.go
#	clients/team/roster_test.go
#	clients/team/store.go
#	clients/team/team.go
#	clients/team/token/token.go
#	clients/team/token/token_test.go
#	clients/team/transactor.go
2026-07-04 20:47:26 -07:00
bb2e6f5e8e chore: pin tasks v1.48.0 (was mutable pseudo-version) (#132)
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-04 20:43:20 -07:00
8d2a33990f feat(durable): expose embedded tasks engine on a gated cluster ZAP listener (#131)
Consolidation: kill the standalone tasksd pod by running its consumers on cloud's
in-process embedded engine. After Embed wires the loopback (ungated, in-process
ai-ingest) listener, call emb.ServeGated(ctx, 9999, validator) to expose the SAME
engine cluster-wide under mandatory identity gating.

RequireIdentity: every request on :9999 must carry an IAM auth_token, validated
against {IAMIssuer}/v1/iam/.well-known/jwks (HIP-0111) and org-scoped to its owner --
the same trust anchor as the HTTP SanitizeIdentity boundary. The loopback dialer for
ai-ingest is untouched (127.0.0.1:19999, ungated, cloud's own trust boundary).

Fail-soft: a missing IAMIssuer or a bind failure logs and leaves the gated surface
down without disturbing ai-ingest. 9999 mirrors the port the retired tasksd exposed,
so a consumer repoint changes only the host (tasks.hanzo.svc -> cloud.hanzo.svc).

Depends on hanzoai/tasks#8 (ServeGated + identity over ZAP). Pinned here to that
branch's commit; repin to the tagged release once #8 merges. universe adds the :9999
Service port.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-04 20:42:28 -07:00
hanzo-dev ddcc961a52 chore: pin tasks v1.48.0 (ServeGated) for the gated cluster ZAP listener 2026-07-04 20:42:24 -07:00
hanzo-dev 06c00b333f feat(durable): expose embedded tasks engine on a gated cluster ZAP listener
Consolidation: kill the standalone tasksd pod by running its consumers on cloud's
in-process embedded engine. After Embed wires the loopback (ungated, in-process
ai-ingest) listener, call emb.ServeGated(ctx, 9999, validator) to expose the SAME
engine cluster-wide under mandatory identity gating.

RequireIdentity: every request on :9999 must carry an IAM auth_token, validated
against {IAMIssuer}/v1/iam/.well-known/jwks (HIP-0111) and org-scoped to its owner --
the same trust anchor as the HTTP SanitizeIdentity boundary. The loopback dialer for
ai-ingest is untouched (127.0.0.1:19999, ungated, cloud's own trust boundary).

Fail-soft: a missing IAMIssuer or a bind failure logs and leaves the gated surface
down without disturbing ai-ingest. 9999 mirrors the port the retired tasksd exposed,
so a consumer repoint changes only the host (tasks.hanzo.svc -> cloud.hanzo.svc).

Depends on hanzoai/tasks#8 (ServeGated + identity over ZAP). Pinned here to that
branch's commit; repin to the tagged release once #8 merges. universe adds the :9999
Service port.
2026-07-04 20:35:02 -07:00
hanzo-dev c6510261e6 billing(#70): fix Red HIGH-2 project-spoof, MED-4 DenyResource, INFO-7
HIGH-2: principal.ValidatedProject(c) (project, validated) — returns false
today (X-Project-Id is a caller-chosen label, not claim-bound), so the edge
gate + resource meter pass ProjectValidated=false and commerce degrades
project-scoped hard caps to soft. ONE lever to harden when IAM mints a
project claim. MED-4: DenyResource renders ErrSpendCapExceeded -> 402
spend_cap_exceeded (was 503). INFO-7: canonicalService unifies the edge
service label with the resource provider (ml/visor->compute, agents->agent,
security->security.scan) so a cap binds on both surfaces. Bump metering
v0.1.3 -> v0.1.4. Tests green (incl DenyResource spend_cap).
2026-07-04 20:25:04 -07:00
hanzo-dev 509f0ce08e fix(team/account): public-URL OAuth callback origin behind the gateway (#45)
Adds TEAM_PUBLIC_URL / PUBLIC_ORIGIN config: callbackOrigin() returns the
configured public origin (e.g. https://hanzo.team) for the OAuth redirect_uri
instead of the request Host, so cloud emits the registered public callback even
behind the gateway (where the request Host is the internal cluster service).
Unset = unchanged (falls back to originOf). Lets hanzo.team route through the
gateway UNIFORMLY like api.hanzo.ai — removes the need for the temporary
direct-to-cloud edge route.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-04 20:15:12 -07:00
hanzo-dev 68530100d2 billing: enforce per-scope spend caps + rate limits at the edge (#70)
BillingGate uses metering.AuthorizeVerdict (funds+cap, one round trip):
renders a distinct 402 spend_cap_exceeded (scope/cap/spent) and sets
X-Spend-Warn at the soft threshold; gates on the request price. New ONE
ScopeRateLimit middleware composes zip/middleware.RateLimit per-scope
(org/project/service), dynamic rpm from commerce (short-TTL cache,
fail-open), 429 + X-RateLimit-* — wired after identity, before billing.
ResourceMeter threads project + service(=provider) so resource creation
is scope-gated too. Scope always from the validated principal. Bump
metering v0.1.2 -> v0.1.3. Money-path tests green (402/warn/429/isolation).
2026-07-04 19:47:10 -07:00
hanzo-dev 98fc642ac7 feat(vfs): real in-process deps.VFS on SeaweedFS S3 — team files/avatars (#45)
pickVFSClient returns an S3-backed types.VFSClient (clients/s3vfs.go) when
S3_ADMIN_ACCESS_KEY/SECRET_KEY are set, else DisabledVFS (R-7 fail-closed
preserved). Reuses the SAME s3admin.Admin construction as clients/s3 (DRY, one
credential path). Put/Get/Delete over the shared 'team-blobs' bucket, per-tenant
key prefix (files.go builds team/blobs/<verified-org>/<ws>/<blobId>). S3 NoSuchKey
maps to types.ErrBlobNotFound (honest 404/idempotent-204); any other S3 error →
502 fail-closed (never a dishonest 404). Bucket create-if-absent self-heals a boot
blip. Red-reviewed SHIP. This is the repoint gate: hanzo.team avatars/attachments
now work off cloud.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-04 16:58:04 -07:00
hanzo-dev 017ebc0761 feat(team): mount clients/team as native cloud subsystem — port from team-go (#45)
Ports hanzoai/team-go into the unified cloud binary as a zip-native subsystem
(order 138, /v1/team/*): account (IAM OAuth bridge + workspaces/members on SQLite),
transactor (Huly wire over wsx, serverVersion 0.6.0 preserved), bots-as-members
(in-process agents.ListForOrg → Employees, removal-reconcile), files (FrontStorage
contract, org+workspace-membership scoped, byte-derived content-type allow-list).

Security (Red-reviewed, all closed): fail-closed SERVER_SECRET degrade-health-only
(never crashes the binary/CI smoke-boot), token exp/nbf, seg() traversal guard,
setCookie verify, cross-tenant blob isolation, VFSClient.Delete fail-closed (deps.VFS
never nil, R-7). Supersedes the stale clients/team a parallel branch swept onto main.

Real deps.VFS wiring (avatars) follows in the next patch (.97) before the hanzo.team
front repoint, so nothing regresses. Migration + repoint + rip of the standalone
team-go Deployment are the remaining cutover steps.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-04 16:32:19 -07:00
80288d87df admin: trial/prepaid grant source + /v1/admin/grants (list+issue) (#129)
- POST /v1/admin/customers/:org/credit gains `source` (trial|prepaid). A staff
  comp defaults to TRIAL (non-cash Credit bucket, grant:admin tag → billing/bucket
  DepositKind Credit); only explicit "prepaid" mints real money (admin-grant tag →
  Prepaid). Fail-closed: unknown→trial, so a comp never silently becomes payout-able
  cash. source recorded in the audit before/after + response.
- grantCredit refactored to a shared applyGrant core (ONE credit-write path).
- NEW GET /v1/admin/grants — the credit-grant ledger across all orgs, projected
  from the tamper-evident audit trail (action admin.customer.credit): org, amount,
  source, reason, staff actor, date, txid, result. Honest-empty without a local
  audit store.
- NEW POST /v1/admin/grants — issue a grant to any org from the operator Grants
  view (org in body), funneled through the SAME applyGrant core.
- Both global-admin gated (s.guard). grantTag unit-tested.

go build/vet/test ./clients/admin green.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-04 14:54:39 -07:00
hanzo-dev f871bded96 admin: trial/prepaid grant source + /v1/admin/grants (list+issue)
- POST /v1/admin/customers/:org/credit gains `source` (trial|prepaid). A staff
  comp defaults to TRIAL (non-cash Credit bucket, grant:admin tag → billing/bucket
  DepositKind Credit); only explicit "prepaid" mints real money (admin-grant tag →
  Prepaid). Fail-closed: unknown→trial, so a comp never silently becomes payout-able
  cash. source recorded in the audit before/after + response.
- grantCredit refactored to a shared applyGrant core (ONE credit-write path).
- NEW GET /v1/admin/grants — the credit-grant ledger across all orgs, projected
  from the tamper-evident audit trail (action admin.customer.credit): org, amount,
  source, reason, staff actor, date, txid, result. Honest-empty without a local
  audit store.
- NEW POST /v1/admin/grants — issue a grant to any org from the operator Grants
  view (org in body), funneled through the SAME applyGrant core.
- Both global-admin gated (s.guard). grantTag unit-tested.

go build/vet/test ./clients/admin green.
2026-07-04 14:32:44 -07:00
hanzo-devandGitHub 3f63ea482e chore(kms): drop hanzoai/kms/sdk/go straggler, bump luxfi/kms v1.11.6→v1.11.8 (#128)
Align cloud to the target: KMS is the embedded luxfi/kms (clients/kms) alongside
embedded IAM; the last external hanzoai/kms dependency is removed.

- go.mod: bump github.com/luxfi/kms v1.11.6 → v1.11.8 (match the deployed image);
  remove github.com/hanzoai/kms/sdk/go v1.1.1; go mod tidy.
- clients/mpcseal (NEW): the minimal client-side-CEK sealing client for the
  SEPARATE luxfi/mpc node ring — a faithful, behavior-identical inline of the
  subset of the former hanzoai/kms/sdk/go that clients/fleet + clients/provisioning
  use (NewClient/Unlock/Set/Get/Delete + Argon2id→HKDF→AES-256-GCM). luxfi/kms has
  no drop-in equivalent (its pkg/ is server/ZAP/store, not a Vault client), so
  inlining the used subset is the minimal correct change that removes the external
  dep without altering the wire protocol or trust model. Drops the HPKE Wrap/Unwrap
  the callers never used.
- clients/{fleet,provisioning}: swap import path only; call sites untouched.
- clients/kmssvc/login.go + docs/consolidation.md: comment/table refs → luxfi.

Verified: go build ./... = 0, go vet = 0, gofmt clean, clients/provisioning tests
pass. go.mod + go.sum carry zero hanzoai/kms references.

Follow-up (separate, tested change): fold fleet/provisioning sealing into cloud's
embedded deps.KMS once types.KMSClient gains Delete + verified against the live MPC
ring — one KMS surface. Once the universe KMS-collapse PR merges, the deprecated
hanzoai/kms repo/package can be archived.
2026-07-04 14:25:58 -07:00
hanzo-dev 809890b706 feat(principal,fleet,ml): make the data plane tenant+PROJECT aware
The gateway now mints X-Project-Id (an org SUB-SCOPE) alongside X-Org-Id.
Thread it through the keyed surfaces, backward-compatibly — the default
project ("default", or an absent header) resolves to today's exact keys,
so existing single-project tenants are byte-identical.

- principal.Project(c): the ONE read accessor, mirroring c.Org() (zero-copy
  header read, cloned on retain). Defaults to DefaultProject when the header
  is empty. principal.DefaultProject / IsDefaultProject own the default-scope
  semantics in one place (shared contract value with iamauth.DefaultProject).
- fleet: registry refs shard by project via the ONE scopeRef seam —
  "<org>/fleet/clusters" for the default project, "<org>/<project>/fleet/
  clusters" for a non-default one (index, sealed kubeconfig, cache key).
- ml: tenant namespace is "ml-<org>" for the default project and
  "ml-<org>-<project>" for a non-default one; both org and project are
  validated against strict DNS-label regexes (no lossy fold) and the composed
  label is length-checked against the 63-char ceiling, keeping the
  (org, project) -> namespace map injective. A hanzo.ai/project attribution
  label is stamped for non-default projects.
- visor BYO fleet + ml federation resolve project via principal.Project.

Billing stays keyed on the paying org (a project has no separate prepaid
balance); project is isolation + attribution, not a billing key.
2026-07-04 14:19:15 -07:00
hanzo-devandGitHub 7518dd395f fix(deps): repin luxfi/age v1.5.0 -> v1.5.1 (cold-cache checksum SECURITY ERROR) (#127)
luxfi/age v1.5.0 was upstream-retagged (transient files GC'd from the tag
tree), so the tag's zip content on the origin no longer matches the h1 hash
recorded in go.sum. Cold-cache builds (fresh CI, empty GOMODCACHE) fail with:

    verifying github.com/luxfi/age@v1.5.0: checksum mismatch
    SECURITY ERROR

v1.5.1 dereferences the same commit, is immutable, and is sum.golang.org
verified (h1:Gj8iHMMi0lGkKT/mlXV2HVBr2m3vt2v0eKVsTMTtAQM=). Surgical: age
require + go.sum only. go mod verify clean.
2026-07-04 14:15:49 -07:00
08d3c4ac06 feat(crm): Startup Program applications — public intake, AI screen, pipeline (#125)
* feat(crm): startup-program applications resource (intake + AI screen + pipeline)

Public unauthenticated intake POST /v1/crm/applications (rate-limited + honeypot)
writes a dedicated crm_applications record (all fields in metadata JSON), a
best-effort CRM Company+Contact projection, and kicks off an AI screen via the
gateway (score / tier1 / suggested credits / summary / draft reply) that
auto-advances applied->screened. Staff GET/PATCH drive a stage machine
(applied->screened->qualified->credits-offered->onboarded, +rejected w/ reason).
Non-fatal if the LLM is unavailable.

* test(crm): startup applications — intake, honeypot, idempotency, AI screen, stage machine

10 tests: public intake creates application+CRM projection with all fields in
metadata; honeypot drop; validation; idempotent resubmit; end-to-end AI screen
with a fake gateway (score/tier1/credits/reply + auto-advance applied->screened);
non-fatal screen on gateway error; staff PATCH stage machine (advance/skip-block/
reject-requires-reason); pure canTransition + parseScreen + detectTier1.

---------

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-04 13:32:59 -07:00
hanzo-devandGitHub 64e5a9a593 chore(tasksvc): swap embedded Tasks UI to a fresh from-source build (#126)
PR #124 embedded the last-known-good admin-tasks build as a de-risking
fallback. This replaces it with a FRESH build of current admin-tasks HEAD,
built from source in a combined gui+admin workspace (hanzogui@7.3.x +
@hanzogui/admin@7.3.0 workspace-linked; @hanzogui@7.3.x is unpublished, so it
must build inside that workspace — see clients/tasksvc/ui/README.md).

Same contract (base=/_/tasks/, api=/v1/tasks); full component parity
(namespaces/workflows/schedules/batches/deployments/activities/nexus/history).
Embed tests still assert the real bundle (not the placeholder).
2026-07-04 13:29:39 -07:00
hanzo-dev 473b892a2d feat(fleet): unify BYO clusters into the ONE /v1/clusters surface (visor)
One and one way: BYO k8s / BYO-GPU / bare-metal attach now lives on the SAME fleet
surface as managed clusters (visor /v1/clusters), not a parallel /v1/ml/clusters.
- clients/fleet: the shared per-org BYO-cluster Registry — kubeconfig sealed in the
  org's KMS, validated by reaching the cluster (node + nvidia/amd GPU inventory),
  tenant-scoped by the ZAP-propagated X-Org-Id. ONE source of truth.
- visor: POST /v1/clusters (attach) + DELETE /v1/clusters/:id (detach), and BYO
  clusters MERGE into GET /v1/clusters beside managed ones. Nominal management fee
  (rides the compute-fee config — no bespoke env var; customer brings the compute).
- ml: deleted the parallel /v1/ml/clusters; dynForOrg federates ML serving onto the
  org's registered cluster via the shared registry (home client when none).
Builds + vet + tests green.
2026-07-04 13:16:34 -07:00
hanzo-devandGitHub 707a1ab223 feat(tasksvc): embed the real Tasks UI, retire tasks-ui pod (#124)
clients/tasksvc served /_/tasks from github.com/hanzoai/tasks/ui, whose
ui/dist is an empty 'No UI build present' placeholder — so tasks.hanzo.ai
still routed to the standalone tasks-ui pod (a Temporal-Web-UI fork).

cloud is the ONE process that serves tasks.hanzo.ai (durable.go's embedded
engine + /v1/tasks surface), so cloud now owns the UI embed too: a local
clients/tasksvc/ui package bakes the real admin-tasks SPA build (base=/_/tasks/,
api=/v1/tasks) into the binary via //go:embed. One binary, one origin, the
real UI — which lets the tasks-ui Deployment/Service/CR be retired.

Tests prove the embedded bundle is the real SPA (not the placeholder), the
SPA deep-link fallback, immutable asset caching, and GET-only.
2026-07-04 13:14:17 -07:00
hanzo-dev b7188fb04f test(observe): red-team route-precedence probe — scoped GET wins over the o11y proxy wildcard (#59)
Companion security test to the observe subsystem: proves a /v1/o11y/logs
request lands on the org-scoped handler (order 44), never falling through to
the unscoped hanzoai/o11y reverse-proxy wildcard (order 70) that would bypass
tenant scoping (attack #4).
2026-07-04 13:03:37 -07:00
hanzo-dev dcc7bd7ed9 fix(observe): coerce response_status_code (LowCardinality(String)) before numeric compare
Validated the handler SQL against the live signoz_traces schema: response_status_code
is LowCardinality(String), so a raw >= 500 raises NO_COMMON_TYPE and asInt64 on it
yields 0. Wrap with toInt32OrZero() in the RED errs count and the request-log status,
matching the verified live query (real per-org buckets returned).
2026-07-04 13:02:27 -07:00
hanzo-dev 0a92b92b25 feat(observe): live per-org Settings/Status/Logs/Metrics for console products (#59)
New /v1/o11y/{logs,metrics,status} + /v1/settings/:product cloud subsystem
(order 44, wins over the hanzoai/o11y proxy wildcard) backing the console
product-detail tabs with REAL, org-scoped data — no stubs.

- Logs   : ClickHouse signoz_logs (admin: raw app stream) / signoz_traces
           org-tagged request stream (every other tenant), live-tail cursor.
- Metrics: per-org RED (rate/errors/p50/p95) from org-tagged spans
           (attributes_string['hanzo.org']) + per-org LLM usage (cloud_usage).
- Status : live in-cluster health probe (latency) + VictoriaMetrics up{service}.
- Settings: per-(org,product) SQLite CRUD; secret fields -> KMS, never SQLite.

Tenant isolation server-side: org = principal.Tenant (validated owner claim),
bound as a positional ClickHouse param / mandatory WHERE org=? — never a client
header/param/raw-query. Only IAM_ADMIN_ORG sees unattributed infra logs.
Reuses the shared ai/object datastore client (one conn, KMS creds).

Tests: 8/8 pass — store isolation, principal gate on every endpoint,
cross-tenant read denial, secrets-never-in-SQLite (fail closed w/o KMS),
product traversal/injection rejection, honest status down.
2026-07-04 13:02:27 -07:00
hanzo-dev c7c6b6bb26 fix(billing): restore per-item ledger attribution (Meter records kind as Usage.Model)
The 1deec70 agents-metering refactor split MeterUsage out of Meter and
dropped the Model:kind write, so EVERY per-product debit (functions/invoke,
s3/op, provisioning, ml, tracker, automations, security) recorded an empty
model — losing per-item revenue attribution in the commerce ledger. Restore
the one-place mapping in Meter (all 8 resource callers flow through it).

Also fix two stale test doubles that read the retired X-IAM-Org-Id header;
commerce reads X-Org-Id only (same $0-revenue class as the admin.go fix), so
they saw an empty org. Full suite: 58 ok, 0 fail.
2026-07-04 12:58:32 -07:00
035fb11a0b fix(serve): truthful comment — ZAP :9653 is plaintext TCP, needs mesh mTLS (red finding #3) (#123)
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-04 12:46:04 -07:00
hanzo-dev a379519f76 feat(visor,cli): bring-your-own GPU fleet — console surfaces + hanzo gpu CLI
Register a machine's GPU into the cloud fleet with one command and see it on
the console's existing Machines + GPUs pages, tagged provider=byo.

- clients/visor: a BYO worker is a heartbeating presence activity in the org's
  `fleet` tasks namespace (cloud.EmbeddedTasks). fleet.go reads it and folds it
  into the SAME machineView/gpuView the console renders (provider=byo,
  location=on-prem, gpu model + VRAM, online/offline by heartbeat), plus a raw
  GET /v1/fleet/workers. /v1/machines and /v1/gpus union Visor's inventory with
  the BYO workers and degrade gracefully (BYO stays visible if Visor is down).
- cli: `hanzo gpu connect|status|disconnect` — reuses the `hanzo login`
  IAM token (org from its claims; token auto-refresh), detects GPUs via
  nvidia-smi, registers + heartbeats the fleet presence record, and runs an
  outbound worker loop claiming from `gpu-jobs` (pluggable handlers: echo,
  studio.render→local ComfyUI). --daemon installs a systemd --user unit.
- go.mod: hanzoai/tasks v1.46.0 → v1.47.0 (the claim + lease-reaper surface).

E2E (z@hanzo.ai): connect → GB10 'spark' shows provider=byo on
/v1/fleet/workers + /v1/machines + /v1/gpus → echo job claimed + completed.
2026-07-04 12:28:10 -07:00
hanzo-devandGitHub fdf62e7f3c fix(bots): auto-create the bound agent on launch so a bot is messageable (#122)
launchBot bound an agent *name* but never created that agent, so
messageBot's in-process run (/v1/agents/:agent/run -> Resolve) 404'd
"agent not found" — a launched bot could not be messaged.

launchBot now create-if-absent's the bound agent via the SAME
POST /v1/agents the console uses (one create path, forwarding the
caller's validated identity -> org-scoped, IDOR-safe), BEFORE launching
the metered machine so a bad request (e.g. a non-catalog model) 400s
before anything is provisioned. Idempotent: an existing agent (409) is
reused. An omitted model takes the deployment default
(deps.AIDefaultModel, a valid catalog model) threaded from config — no
hardcoded model id.

Also add create/update-time model validation: a client-supplied model
outside the gateway's served catalog is a clean 400 (via the optional
types.ModelLister the real gateway client implements) instead of a
confusing run-time 502; fail-open when the catalog can't be enumerated.

Tests: agents model-validation + default (real store); httpAI.Models
against a fake gateway; visor launch->auto-create->message->resolve E2E
incl. the before/after 404->200 gap proof, idempotency, and bad-model
fail-fast (no machine provisioned).
2026-07-04 12:17:29 -07:00
hanzo-devandGitHub 75f0987994 deps(rag): bump hanzoai/ai v1.800.6 -> v1.800.7 — force gateway embedder at resolution time (fixes RAG ingest hang, #72) (#121) 2026-07-04 11:46:26 -07:00
hanzo-devandGitHub 5a0e8ea486 feat(notify): KMS-only provider creds, remove env fallback (#120)
notify's send surface now reads provider credentials EXCLUSIVELY from cloud's
embedded KMS (cloud.Deps.KMS) at the org-scoped, rotatable ref
orgs/<org>/notify/<svc>/<key> — the same /orgs/<org> namespace
clients/integrations uses, so a cred is seedable + rotatable via
POST /v1/kms/orgs/:org/secrets with no operator-injected env Secret and no
restart. The org is the VALIDATED principal's tenant, never a client header.

Removes the env-first fallback (envCreds/envFirst + the os import): no secret
is ever read from the environment, hard-coded, or logged. A missing key leaves
the value empty and constructProvider fails closed.

Tests rewritten to inject a fake KMS (no env), plus a per-org isolation test
and a regression that creds() ignores the legacy TWILIO_* env entirely.
2026-07-04 11:35:28 -07:00
hanzo-devandGitHub 27e52fc379 deps(rag): bump hanzoai/ai v1.800.4 -> v1.800.6 — default embedder targets the Hanzo gateway, not api.openai.com (#119)
Pulls hanzoai/ai#71: the RAG default embedder (object/init.go seed) now points at
the Hanzo gateway (CLOUD_AI_BASE_URL / CLOUD_AI_API_KEY, model text-embedding-qwen3)
instead of an empty ProviderUrl that hit api.openai.com directly. A server-side
embed to api.openai.com from in-cluster crawled ~180-210s and then failed, so RAG
ingest (/v1/rag/embed) hung AND the Qdrant vector collection was never created
(writeDocsToVector sample embed timed out before ensureVectorCollection ran).
Gateway embeddings are <1s (proven live). The seed self-heals an existing
api.openai.com-direct default to the gateway on boot, so this deploy converges the
live default-embed provider with no manual console repoint.
2026-07-04 11:11:47 -07:00
z c19c07dad5 fix(websearch): admit the validated console principal on /v1/websearch/search
The console (console2 WebSearch module) reaches search through the /cloud proxy
with a signed-in USER BEARER, not the shared X-API-Key. searchGuard required the
key on every call (F2 hardening), so the console got 503/401 — "backend not
initialized" — even though searxng+crawl are deployed and the upstream defaults
are correct.

Reconcile to the ONE-WAY gate the rest of the /v1 data plane uses: at the zip
layer, a request with a validated principal (principal.Validated — X-User-Id
minted by the identity middleware from a verified JWT) proxies straight to
SearXNG; a request with NO principal falls to the unchanged key-based searchGuard
(the hanzo.chat server path). A caller with neither is still refused, so F2 (no
open metasearch proxy) holds — proven by TestSearchNoPrincipalNoKeyRefused.

searchGuard (net/http) is untouched; its 503/401 tests stay green. New coverage:
TestSearchValidatedPrincipalBypassesKey (console bearer, key unset -> 200),
TestSearchNoPrincipalNoKeyRefused (anonymous, key unset -> 503).
2026-07-04 08:50:05 -07:00
zandGitHub 98ed1332e1 Merge pull request #117 from hanzoai/fix/agent-run-internal-egress
agent-runner mints M2M inference token from in-cluster IAM (fixes /v1/agents/:ref/run 502). Forward-integrated with main (#118 admin-guard audience). Reconciles live sha-7912539 onto a semver release.
2026-07-04 08:37:34 -07:00
cd1b77685c fix(identity): accept hanzo-admin-guard as a JWT audience (#118)
cloud-api SanitizeIdentity validates the forwarded IAM bearer against
defaultJWTAudiences before granting global-admin (owner==adminOrg). The
admin.hanzo.ai guard is client hanzo-admin-guard, so its tokens carry
aud=hanzo-admin-guard, which was missing from the allowlist -> the bearer
failed validation, resolved anonymous, and the SuperAdmin gate read false
-> 403, even though the token owner IS admin.

Append the guard client_id (forwards-only, mirrors gateway iamauth). Admin
authority still requires owner==adminOrg, so no widening. Pairs with
hanzoai/gateway audience fix.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-04 06:35:12 -07:00
hanzo-dev 7912539748 fix(cloud): agent-runner mints M2M inference token from in-cluster IAM (fixes /v1/agents/:ref/run 502)
pickAIClient built the M2M token URL from cfg.IAMIssuer (=https://hanzo.id), a
Cloudflare-fronted host. In-cluster the runner's server-side POST to
https://hanzo.id/v1/iam/oauth/token 403s with CF edge error 1006, so the
oauth2 client-credentials fetch fails and EVERY POST /v1/agents/:ref/run 502s:
  'cloud: chat completion: oauth2: cannot fetch token: 403 Forbidden / 1006'.

New aiM2MTokenURL resolves the token endpoint split-horizon, mirroring the KMS
login-broker (clients/kmssvc) exactly — one policy, no drift:
  1. CLOUD_AI_IAM_TOKEN_URL override
  2. in-cluster IAM_URL (already wired to http://iam.hanzo.svc for JWKS)
  3. public IAMIssuer fallback (single-process deploys)
IAMIssuer stays https://hanzo.id for JWT iss-validation (untouched). The chat
base URL is pointed in-cluster via CR env CLOUD_AI_BASE_URL (universe).

Proven in-cluster: mint token from iam.hanzo.svc + chat to gateway.hanzo.svc
both 200 (real completion). Unit test pins the 3-branch resolution order.
2026-07-04 05:12:39 -07:00
hanzo-dev a160fd5249 deps(billing): bump hanzoai/ai v1.800.3 -> v1.800.4 — widget (hz_) keys bill owner org
Pulls in the ai fix that closes the hz_ widget-key free-inference hole: widget
keys now bill the OWNER ORG (object.WidgetKeyOwner), so reserveBudget +
recordUsage + the balance gate all engage instead of running free/unmetered.
Bounded to the restricted widget model set + token cap; fail-secure when a widget
key is unattributable.
2026-07-04 04:41:47 -07:00
hanzo-devandGitHub da8fe6e5d5 fix(kms): broker uses in-cluster IAM_URL for token exchange, not public issuer (CF 403s in-cluster loopback) (#116)
The per-tenant KMS secret-sync login broker derived its IAM token-exchange URL from the public issuer (hanzo.id), which Cloudflare 403s for in-cluster server-side POSTs → the sync could never authenticate. Prefer in-cluster IAM_URL (+ CLOUD_KMS_IAM_TOKEN_URL override), fall back to issuer. Unblocks PaaS per-tenant secret env (proven: git-built ai-demo app deployed on maxpower).
2026-07-04 04:05:44 -07:00
c3a811d23a feat(notify): fold notifyd OTP send surface into the unified cloud binary (#115)
Mounts /v1/notify/{send,send/sms,send/email,health} natively in-process as the
cloud subsystem "notify" (order 139) — the native, in-process replacement for
the standalone notifyd (github.com/hanzoai/notify) Deployment.

notifyd's ONLY production consumer is Hanzo IAM's OTP send
(POST /v1/notify/send?sync=true, event=iam.otp_sent), and the live tenant's
template/provider/event tables are empty, so this folds exactly that contract
and nothing more. It reuses notifyd's OWN public provider packages
(service/{twilio,twilioemail,plivo,mail}) and wire types (pkg/types) — no
duplication of provider plumbing; only the internal-only cred->constructor glue
is mirrored.

Security: unlike the ClusterIP-internal notifyd (which trusted a raw X-Org-Id),
/v1/notify/send is reachable via the public gateway here, so it gates on a
VALIDATED principal and derives the org from principal.Tenant — the same
trust-boundary move clients/auto makes. Credentials come from env (the
KMS-synced notify-twilio Secret) and KMS via cloud.Deps.KMS; none is hard-coded
or logged. Ships a built-in iam.otp_sent template so the fold is strictly more
available than notifyd is today (whose empty store would 400 an OTP send).

Sync-only: the Temporal notify-send async plane is intentionally NOT folded;
async (no ?sync=true) returns 503, exactly as notifyd does without a worker.

Build-gated: go build ./... green; go test ./clients/notify/... green; gofmt/vet
clean. go.mod adds only hanzoai/notify + its provider transitive deps.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-04 03:48:05 -07:00
hanzo-dev c4dda6a800 fix(zt): message-hygiene — 503 body names no internal env
The fail-closed 503 body no longer names ZT_CLIENT_ID/ZT_CLIENT_SECRET;
it now reads 'networking is not configured on this deployment' — a
customer-facing string the console renders as a clean 'not available yet'
state. Ops still see the env names in the Warn log at Mount. No behavior
change; the gate() fail-closed contract is identical.
2026-07-04 03:24:53 -07:00
zandGitHub 7e27214b11 Merge pull request #114 from hanzoai/feat/slack-integrations-bridge
feat(integrations): @hanzo Slack agent bridge — events/commands/link on /v1/integrations/slack/*
2026-07-04 03:18:33 -07:00
hanzo-dev e5d7b1d38a fix(integrations): Red delta — recover async turns + shed before dedupe (M-2, M-1)
RED M-2 [MED] — contain panics in the async agent-turn goroutine. The dispatched
turn (handleSlack* → slackAgentReply → agents.RunOnBehalf, a large surface over
UNTRUSTED Slack input) ran UNRECOVERED — middleware.Recover() only wraps the sync
request goroutine — so a panic would crash the ENTIRE shared multi-tenant cloud
binary (every tenant, every subsystem). Introduced slackSpawn: runs an
already-slotted turn in a recovered goroutine (recover defer registered LAST so it
runs FIRST; the slot release still runs after it, so a panicking turn frees its
slot). Test TestSlackTurnPanicRecoveredAndSlotReleased.

RED M-1 [MED] — shed BEFORE burning the dedupe key. The old order was
mark-then-dispatch: MarkSlackEvent recorded the event_id, then a pool-full drop
silently 2xx-acked → Slack never retried and a later retry was deduped away, so
the @mention vanished. New order (events + slash): verify HMAC → resolve org →
TRY-ACQUIRE a pool slot; on shed record NOTHING and return a retriable 429 (Slack
re-delivers when a slot frees — the turn never ran, so no double-run and no burned
key); on acquire → MarkSlackEvent (release the slot on duplicate/error) → spawn.
The fast empty-2xx ack is kept for the normal path. Test
TestSlackShedReturnsNon2xxAndDoesNotRecord.

Also corrected the slack_dedupe.go replica note (we no longer "always 2xx-ack" —
a capacity shed returns a retriable non-2xx and records nothing, so it is not a
double-process).

RED M-3 [MED] deploy single-replica (Recreate + persistent CLOUD_DATA_DIR) — the
universe cloud manifest, owned by the deploy lane; the in-code single-writer
invariant comment is kept accurate here.

go build ./... = 0, go vet ./... = 0, go test ./clients/integrations/
./clients/agents/ -race green (25 tests).
2026-07-04 03:03:36 -07:00
hanzo-dev 4249b0f989 fix(integrations): address Red review — wire routes, per-org pool, replica scope
RED H1 [HIGH] — wire the bridge into integrations.Mount (was only in a test
helper → the front-door didn't exist in prod). The 5 literal routes are
registered BEFORE the /:provider wildcards (registration-order precedence, same
discipline as clients/agents' static-before-:ref) and are PUBLIC at the JWT
layer: IdentityMiddleware only POPULATES a principal (never rejects) and
DefaultPrice returns 0 for /v1/integrations/* so BillingGate passes through —
reached exactly like /:provider/callback. Auth is HMAC (events/commands) /
signed __Host- cookie (link legs) INSIDE the handler. New test
TestSlackRoutePrecedence proves GET /slack/link hits slackLink (not :provider),
/v1/integrations/slack still resolves the provider view, and the webhook is
reachable with no principal.

RED M1 [MED] — per-org concurrency sub-limit. The agent-turn pool was one
process-global semaphore; one org bursting @hanzo could starve every tenant.
Replaced with orgLimiter (global cap + per-org cap, SLACK_AGENT_ORG_CONCURRENCY
default 8). The org is now resolved SYNC in the webhook path so the pool keys on
the RESOLVED tenant before a slot is taken. New test TestOrgLimiter.

RED M2 [MED] — corrected the false "holds across replicas" dedupe claim: the
table is per-process embedded SQLite (single-writer per HIP-0302), so the billed
webhook path MUST run single-replica (stated as the shipping invariant); a
shared SETNX store is the multi-replica follow-up.

RED L1 [LOW] — moved the dedupe-table DDL into the store's migrate() (store.go)
— fail-loud at Mount, one place — and removed the lazy first-use ensure whose
LoadOrStore-before-run could permanently disable the path on a transient DDL
error. slackBridgeReady now only inits the process pool + link seen-set.

Deferred (flagged for clients/integrations owner): L2 UNIQUE(provider,
external_id)+first-org-wins refusal on duplicate team connect; L3 purge
user:<slackUser>:refresh secrets on disconnect (currently inert after
disconnect, no leak).

go build ./... = 0, go vet ./... = 0, go test ./clients/integrations/
./clients/agents/ -race green (18 + 5 tests).
2026-07-04 02:41:14 -07:00
hanzo-dev 060659ae8b feat(integrations): Slack agent bridge on the one-binary integrations plane (#45)
Port the hardened @hanzo Slack agent front-door from team-go/pkg/slack into
the unified Hanzo Cloud integrations plane, so Slack is ONE connector aligned
with the one-binary north star. It CONSUMES the existing Slack OAuth provider
(the per-org bot token it seals) and the framework seams
(OrgForExternalID / TokenFor / ConnectionFor); it adds no new custody path and
edits no existing file.

clients/agents:
- onbehalf.go: exported in-process RunOnBehalf(ctx, org, userSub, ref, input) —
  the clean in-process twin of the HTTP run handler (no gateway hop, no
  Cloudflare/IPv6 exposure). Resolves the agent org-scoped, runs it through the
  SAME runAgent -> executeRun -> meter path, bills billingActor(org, userSub)
  against org's ledger. Takes org+userSub DIRECTLY (caller pre-authenticated).

clients/integrations:
- slack_events.go: Slack Events webhook + slash command. HMAC-verified over the
  EXACT raw body with a 5-min replay window; url_verification challenge; routes
  @mention + DM to an on-behalf-of run; durable dedupe on event_id; fast empty
  ack + bounded async worker pool. Posts the reply into the thread with the
  org's bot token, or the link prompt EPHEMERALLY.
- slack_link.go: transplant-safe 3-leg per-user link (__Host- init/link cookies,
  leg1<->leg2 nonce continuity checked BEFORE any exchange, single-use). Binds
  Slack<->Hanzo via hanzo.id OIDC (hanzo-slack client) and seals the refresh
  token per (org, "slack", "user:<slackUser>:refresh").
- slack_verify.go: Slack signature verify + single-use link-state crypto
  (constant-time HMAC over s.stateKey; orthogonal to the OAuth-connect state).
- slack_dedupe.go: durable event-dedupe table as Store methods (no store.go edit).

PER-ORG ISOLATION (ship bar): an event's org comes ONLY from
OrgForExternalID(team_id) — never the payload; the reply uses THAT org's bot
token (TokenFor); the run is THAT org's agent (RunOnBehalf org-scoped). Tests
prove team A's event never resolves/tokens/runs as org B.

Mount wiring (5 routes) is handed to the clients/integrations owner — this
change adds NO Mount edit (clean separation); handlers are (s *svc) methods.

Tests (go test -race, green): HMAC reject (bad/missing/stale), dedupe
idempotency, per-org isolation (end-to-end bot-token capture proves the reply
used the connecting org's token), link transplant-rejected (no/mismatched init
cookie refused before exchange), RunOnBehalf bills the right actor.
2026-07-04 02:10:35 -07:00
hanzo-dev cfcd40ac3a fix(deps): align luxfi/age v1.5.0 go.sum hash with sum.golang.org
The recorded zip h1 for github.com/luxfi/age v1.5.0 (zC/Fw…) did not match
the immutable Go checksum transparency log (sum.golang.org), which records
G69Hb… — the same bits the module proxy and local cache serve. The stale
hash made the ENTIRE module unbuildable: every `go build` failed with a
checksum mismatch / SECURITY ERROR. The /go.mod hash already matched sumdb;
only the zip h1 was wrong. Aligning it to the transparency-log-verified
value unblocks the repo (`go mod verify` -> all modules verified). age is an
indirect dependency; no version bump.
2026-07-04 02:10:18 -07:00
024dbc186a feat(admin): GET /v1/admin/o11y — global fleet observability over the one datastore (#111)
Cross-org fleet o11y for admin.hanzo.ai (global-admin only, s.guard fail-closed):
fleet totals (requests/tokens/cost/errors/orgs/models from hanzo.cloud_usage;
latency p50/p95/p99 + error-rate + services from signoz_traces; log volume from
signoz_logs), usage + log-volume timeseries, and top-N orgs/models/services
leaderboards, plus the fleet Langfuse generation rollup. Un-org-scoped by design
— the one place a fleet operator crosses tenants; a non-admin bearer is refused
403 before a row is read. Reuses the shared aiobject.DatastoreQuery transport
(no second connection) and the compute/analytics honest-empty pattern; admin
reads only, owns no table. Time bounds are positional params, bucket interval a
server-side constant — injection-safe. Pure builders + parsers unit-tested.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-04 01:54:32 -07:00
zandGitHub e477579d45 Merge pull request #113 from hanzoai/feat/automations-phase1
feat(automations): Connectors+Automations engine — Phase 1 (HIP-0106 #51)
2026-07-04 01:53:55 -07:00
zandGitHub 7f30bae821 Merge pull request #112 from hanzoai/fix/gosum-luxfi-age
fix(go.sum): realign luxfi/age@v1.5.0 to the proxy content hash (unblocks cloud releases)
2026-07-04 01:53:14 -07:00
hanzo-dev 4bc4502279 fix(go.sum): realign luxfi/age@v1.5.0 to the proxy content hash
luxfi/age@v1.5.0 was force-retagged upstream: proxy.golang.org now serves
content hashing to G69HbSV… while go.sum pinned the stale zC/Fw… → every
cloud release fails at `go mod download` with a SECURITY ERROR (checksum
mismatch), blocking the whole pipeline. Dockerfile already sets GOSUMDB=off,
so the mismatch is against the committed go.sum, not the sumdb. Realign to the
exact hash CI computes from the proxy (same fix IAM shipped as fee44857).
2026-07-04 01:53:00 -07:00
hanzo-dev c347ba1484 fix(automations): RED fix set — exactly-once run bookkeeping + SSRF/caps/audit hardening
Addresses CTO's post-RED fix set (isolation boundary already approved airtight).

MED-1 (exactly-once metering/audit/persistence across ALL entrypoints): the
durable path is now the SINGLE owner of run bookkeeping. FlowRunWorkflow runs a
RecordRunStartActivity keyed on the workflow id (workflow.GetInfo — a scheduled
cron mints a fresh id per tick, so each tick is its own metered run despite the
schedule embedding one fixed FlowRunInput.RunID). Store CreateRunIfAbsent (row
idempotency) + ClaimMeter (atomic metered-flag 0->1) meter+audit only the winner,
so manual /run, MCP, and cron never double-bill. Manual /run no longer meters/
audits — it only CreateRunIfAbsent for immediate visibility. RecordRunEndActivity
records terminal status. Proof: TestScheduledRunMeteredExactlyOnce (a tick meters
once + shows in listRuns; two ticks = two distinct runs) + TestRunStartBookkeeping-
Idempotent (recordRunStart twice = one meter, one row).

MED-2 (honest SSRF blocklist): isPublicIP now rejects the IANA special-use ranges
Go's net helpers miss — 100.64/10 CGNAT (Alibaba metadata 100.100.100.200),
0/8, 192.0.0/24, 192.0.2/24, 192.88.99/24, 198.18/15, 198.51.100/24, 203.0.113/24,
240/4, 64:ff9b::/96 NAT64 — plus v4-mapped-v6 normalization. Comment no longer
overclaims a complete cloud-metadata blocklist. TestIsPublicIP covers each range +
public IPs still allowed.

MED-3 + LOW-4: step-count (<=256) + serialized-tree (<=512KB) caps at create /
version / operation time -> honest 422; resume payload bounded (<=64KB) -> 413.

LOW-2: per-org concurrency limiter (429) on run-starts + synchronous MCP tool calls
(bounds the core.delay goroutine lever). TestConcurrencyLimiter + TestFlowStepCap +
TestResumePayloadBounded.

LOW-1: MCP meters/audits AFTER Run, outcome derived from the real result — a failed
/ SSRF-blocked / not-connected call audits as error and is NOT billed. TestMCPAuditOutcome.

LOW-3: updateFlow validates publishedVersionId names an existing version OF THIS
FLOW in-org (else 422). TestUpdateFlowPublishedVersionValidated.

INF-1: register() panics at init on a <connector>_<action> tool-name collision so a
future connector can't silently make MCP dispatch ambiguous. TestToolNameCollisionPanics.

Tests: 25/25 green (CGO=0 build/vet/test; -race clean under cgo). Full module builds;
cmd/cloud links. catalog.json untouched.
2026-07-04 01:38:15 -07:00
hanzo-dev 4c7b07b340 feat(integrations): Slack connect lights up on the public client_id alone
Authorize needs only SLACK_CLIENT_ID (a public value in every consent URL);
the SECRET is required only at the callback token exchange. Gate available/
connect on client_id so an org reaches Slack's Allow screen as soon as the
public id is set, while a deployment still missing SLACK_CLIENT_SECRET fails
the exchange with an honest ?error=slack (never a dead-end).
2026-07-04 01:31:54 -07:00
zeekayandhanzo-dev a7277ba3e0 fix(deps): bump hanzoai/ai v1.800.2 -> v1.800.3 for brand .cloud CORS fix
Pulls the cors_filter static-allowlist fix so console.lux.cloud (and
zoo/pars brand consoles) stop getting 403 "origin is not allowed" on
/v1/signin. Cleared stale sum.golang.org-poisoned go.sum entries for
re-tagged luxfi/{age,precompile,keys} (GOPRIVATE direct re-records the
current content hashes; matches the repo's GOSUMDB-off CI recipe).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-04 01:20:55 -07:00
hanzo-dev 2ff71fff3c feat(automations): merge full 701-piece catalog + /v1/automations OpenAPI
- catalog/catalog.json: 706 pieces (5 Tier-A executable connectors + 701
  ActivePieces catalogue entries); action/trigger props normalized string[]
  -> []PropSpec so PieceMetadata unmarshals at Mount (boot-parses green).
- docs/automations-openapi.yaml: OpenAPI 3.0.3, 13 paths all under /v1/automations.
- http_test: assert catalogue invariant (PieceCount==len, Tier-A present) not
  the seed-pinned count.
2026-07-04 01:09:57 -07:00
hanzo-dev 4592fc1ebd feat(automations): Phase-1 Connectors+Automations engine (HIP-0106 #51)
Native-Go /v1/automations/* subsystem in the unified cloud binary. Composes
three existing seams, never reinvents them:

- clients/integrations  — per-org connector creds via integrations.TokenFor
  (KMS-sealed, fail-closed); connectors never touch KMS directly.
- cloud.EmbeddedTasks    — the ONE shared in-process durable engine; a flow
  runs as a durable workflow in the OWNER's namespace (per-org lazy worker,
  mirroring ai/object/ingest_tasks.go).
- clients/principal      — the ONE tenant gate on every data handler.

Isolation is physical: ONE SQLite file, org column + org-led index on every
table; the durable activity's SOLE credential scope is FlowRunInput.Owner —
the VALIDATED org set at flow-start, never a client-supplied field.

Surface: pieces catalogue (go:embed), flows CRUD + versions + FlowOperation
apply, durable runs (start/list/get/resume via SignalWorkflow), enable/disable
(POLLING -> CreateSchedule), and a HIP-0300 MCP JSON-RPC tool surface
(/v1/automations/mcp) exposing every connector action as <connector>_<action>.

Connectors (Tier-A, self-registering): core (http_request SSRF-guarded via a
dialer Control hook, delay, code data-mapper, wait_for_approval signal
waitpoint), slack (send_message), github + google_sheets/drive (fail closed
until integrations custodies their tokens).

Metering + audit on flow-run start and MCP tool call. Order 148 (after
integrations 137, before ai's /v1/* catch-all 150).

Tests (16, all green with -race): store org-isolation, HTTP org-gating (403),
durable flow run reaching SUCCEEDED with threaded step outputs on an embedded
tasks engine, connector token isolation (in.Owner is the sole cred scope),
MCP tools/list + gated tools/call dispatch, pieces catalogue.
2026-07-04 01:03:06 -07:00
336f60f8d4 feat(o11y): HTTP request + LLM/agent traces over ZAP (single provider) (#110)
* feat(o11y): emit an OTel SERVER span per /v1/* request over the ZAP wire

Cloud installed a ZAP tracer provider (cmd/cloud initTelemetry) but nothing in
the handler chain opened a span, so no request ever flowed through it — the o11y
Monitoring tab saw zero hanzo-cloud request traces (receiver="zap" span count
was flat-zero while logs streamed over ZAP).

TracingMiddleware (middleware_tracing.go) opens one SERVER span per /v1/*
request off the GLOBAL tracer (= the ZAP provider), records the OTel HTTP
semantic-convention attributes (method, route, status) + request_id/org, maps
error/5xx to an error span status, and writes the span context back onto the
request via SetContext so every downstream span (agent.run -> agent.step -> the
chat client span in clients/aihttp) parents under it: one trace tree per
request. Health/readiness/metrics + non-/v1 paths are skipped so probes never
flood the trace store. Wired right after RequestID in the canonical pipeline
(serve.go) so the whole authenticated chain nests under it.

c.Path()/c.Method()/headers are zero-copy views over the fasthttp request
buffer, which is recycled for the next request BEFORE the batch span processor
serializes the span asynchronously — so retained views corrupt (live: a
GET /v1/models span exported with http.route="/v1/chat/c..."). strings.Clone
pins our own copy for every retained attribute. Tests cover emission, attribute
mapping, error status, parent/child propagation, the skip set, and an env-gated
on-wire live test (CLOUD_ZAP_LIVE_ENDPOINT) that ships real spans to a ZAP
receiver — the async-export + ctx-reuse path the in-memory recorder can't model
(and the one that surfaced the corruption).

* feat(o11y): make cloud the single tracer-provider owner — one wire (ZAP)

The fused cloud binary set the ZAP provider first, then ai.Bootstrap (during
MountAll) called hanzoai/ai object.InitTelemetry which, seeing the CR's
OTEL_EXPORTER_OTLP_ENDPOINT, installed a SECOND, competing OTLP provider. OTel
global delegation is first-writer-wins for handles created before the first
SetTracerProvider (cloud's package-level tracers keep ZAP), but the ai GenAI
tracer is resolved lazily AFTER the second Set, so its spans stranded on
OTLP(:4318) while ZAP owned the rest — the split that left receiver="zap" span
count at zero for hanzo-cloud (verified live: spans arrived only via
receiver="otlp").

Composition-root fix: once cloud installs the ZAP provider, clear the
OTLP-exporter env (OTEL_EXPORTER_OTLP_ENDPOINT / _TRACES_ENDPOINT) so no embedded
subsystem installs a competing OTLP provider. Exactly one provider (ZAP), one
wire, deterministic regardless of CR env drift. In the fused binary OTLP is only
ever the collector's interop RECEIVER, never cloud's exporter; standalone
cmd/aid (no ZAP endpoint) is unaffected and keeps its OTLP path.

---------

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-04 00:56:42 -07:00
hanzo-dev 435ceeddfb fix(kb): RED H1/M1 — inject X-Piece-Run-Secret on the auto piece-run call
The auto engine now gates /v1/auto/pieces/{piece}/run on a shared secret (it
trusts X-Org-Id absolutely, so the write+SSRF surface needs an in-band caller
proof). cloud is the ONLY legitimate caller — it resolves each org's real token
and pins the provider URL — so it presents the secret (from KMS, the same
PIECES_RUNNER_SECRET) as X-Piece-Run-Secret. pieceSync fails closed if the
secret is unset (a doomed call the engine would 403 anyway).
2026-07-04 00:05:48 -07:00
hanzo-dev 9d299ab169 feat(auto+kb): hybrid connector layer — /v1/auto proxy + activepieces long-tail
Mounts workflow automation + the ~280-app activepieces long tail in-platform,
per-org, through the ONE knowledge store. Go core + JS on-demand.

clients/auto: /v1/auto/* per-org REVERSE PROXY to the standalone Hanzo Auto
engine (one engine, one store — not re-embedded). The auto engine trusts
X-Org-Id absolutely, so this proxy IS the trust boundary: it GATES on a
validated principal (refuses the anon-forge X-Org-Id-with-no-credential path)
and re-stamps outbound identity from validated values only (strips every
smuggled authority alias). Pure gate+proxy in clients/auto/proxy (5 isolation
tests: anon-forge 403, per-org forward, smuggled-header strip, path preserved).

clients/kb: the first LONG-TAIL connector (notion). Identical OAuth lifecycle
(HMAC-org-bound state, KMS token path) but its PULL runs the activepieces JS
piece through the auto engine's on-demand runner (sync_piece.go) instead of
native Go — then files each record via the SAME framework.Ingest path. One
ingestion path; a JS-sourced doc lands in the same per-org store+index as a
Go-sourced one. clients/kb/notion is the pure record-shaper (6 tests).

ONE catalog: /v1/kb/connectors/catalog lists native Go + long-tail piece
connectors in one list, each badged kind native|piece (3 tests).

RED LOW-1: collection() + kmsRef() now route org through provisioning.SanitizeOrg
(the codebase's ONE normalizer) so the physical Qdrant namespace + KMS path are
injective in the owner ("a b" != "a_b") — defense in depth under the payload.org
filter. Injectivity tests + KB integration tests updated to derive the collection
through the helper (robust to the normalizer).

All tests green under CGO=0 (production config). Full binary boots; /v1/auto
mounted, anon-forge 403, catalog gated, spine (kb/framework health) 200.
2026-07-04 00:05:48 -07:00
zeekayandhanzo-dev 03f024ac9d fix(billing): payment-methods → commerce portal read; pin the full subject-key set
Address review: proxy the customer card list to commerce's admin-group PORTAL
endpoint (GET /v1/billing/portal/payment-methods), not the user-group
/payment-methods. PortalPaymentMethods 400s without a ?customerId=, so pinning
only ?user= would break it — generalize the proxy's subject pinning to the FULL
commerce edge-auth key set {user,userId,customerId} (now a shared
billingSubjectKeys var, identical to clients/console + commerce), pinned to the
caller's OWN org on every request. This leaves NO billing endpoint unfiltered
regardless of which param it reads (usage/balance/gpu-eligibility read user;
portal/payment-methods requires customerId) and is strictly more tenant-safe.
pinSubjectBody reuses the same var. The console keeps requesting the same-origin
/v1/billing/payment-methods (mounted here); the portal hop is server-side only.

Tests updated: widen-scope now asserts every subject key is pinned (org dropped);
payment-methods asserts the portal path + customerId pin. 14/14 pass.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-03 23:37:00 -07:00
hanzo-dev 2a4659f834 chore(deps): bump hanzoai/ai → v1.800.2 (durable worker retention + fail-fast OpenAI embedding default) 2026-07-03 22:47:33 -07:00
zeekayandhanzo-dev ba0fdff868 feat(billing): proxy GPU launch-gate + payment-methods on customer /v1/billing
The customer billing proxy (clients/billing) exposed only usage+balance, so the
console GPU launch gate — card-on-file check + prepaid eligibility read + the
prepay-only charge — had no org-scoped cloud route and fell through to the
console pkg's /v1/billing/* wildcard (admin-shaped 403). Extend the SAME
commerce-proxy helper (no new HTTP/auth machinery) with the three enforcement
routes commerce 1.46.28 serves (api/billing/gpu_charge.go + portal):

  GET  /v1/billing/gpu-eligibility -> commerce GET  (read-only launch gate:
       {eligible,reason,prepaidAvailable,cardOnFile,...}; amountCents +
       minPrepaidCents + currency pass through)
  POST /v1/billing/gpu-charge      -> commerce POST (prepay-only, card-required
       debit; commerce enforces both gates + gpu-tagging server-side; status
       forwarded verbatim: 201 ok / 402 card_required|insufficient_prepaid)
  GET  /v1/billing/payment-methods -> commerce GET  (masked brand+last4 cards
       for the card-on-file check; type passes through)

All org-scoped to the caller's OWN org from the VALIDATED IAM owner claim
(principal.Tenant), identical to usage/balance — a client can never widen scope:
the GET subject is pinned to ?user=<org> (commerce's privileged payment-methods
branch filters CustomerId on it), and the POST body subject is pinned to the
{user,userId,customerId} set (mirrors clients/console + commerce edge-auth), so
a forged body can never charge another tenant. New commerceProxy.post + the
pinSubjectBody helper; no principal -> 401, unconfigured -> 501.

Tests: gpu-eligibility scope+passthrough+forged-subject overwrite;
payment-methods scope+type; gpu-charge body-subject pin + 402 verbatim + 401
no-principal + 501 unconfigured; pinSubjectBody unit. 14/14 pass.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-03 22:40:11 -07:00
zeekayandhanzo-dev d1779528a2 feat(billing): customer-facing org-scoped /v1/billing/{usage,balance}
On console.hanzo.ai the ingress routes /v1/* straight to cloud-api:8000 (the
console Next BFF is only at "/"), so the console's /v1/billing/usage +
/v1/billing/balance calls land on cloud-api — NOT the console's per-tenant
commerce proxy. cloud-api wired commerce billing ONLY under the admin-gated
aggregate (/v1/admin/*), so a normal org owner (davelorenzini/maxpower) hitting
/v1/billing/usage had no customer route and was denied -> 403 -> the "Access
required" wall on EVERY product overview + o11y usage panel.

Add a customer-facing, org-scoped billing READ surface (clients/billing):
GET /v1/billing/usage and /v1/billing/balance. Org = the VALIDATED IAM owner
claim (principal.Tenant — the trusted X-Org-Id the identity middleware minted
from the caller's verified session; never a client header), so a customer reads
ONLY their OWN org. Proxies commerce with COMMERCE_SERVICE_TOKEN + X-Org-Id=<org>
and the per-org billing subject pinned to user=<org> (admin.orgSubject /
metering identityFromCtx — verified live: user=<org> returns the real wallet);
returns commerce's raw body + status verbatim (the console parses the raw ledger).
Tenant isolation: no client-supplied subject/org query is ever forwarded, so
scope can never be widened. The all-orgs god view stays admin-only (clients/admin).

Tests: subject pinned to caller org, forged user/userId/customerId/org dropped,
start/end/currency pass through, no-principal -> 401 (commerce untouched),
unconfigured -> 501, commerce status forwarded verbatim.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-03 22:28:52 -07:00
zeekayandhanzo-dev bd8bb52514 fix(deps): bump hanzoai/ai → v1.800.0 (async Sora-style /v1/videos #68)
Pulls ai#68: async OpenAI Sora-style video API onto the cloud router.
POST /v1/videos/generations returns a video_<uuid> job immediately (was
sync ~104s → console /ai proxy 502); GET /v1/videos/{id} polls; GET
/v1/videos/{id}/content streams the MP4. Metering exactly-once (hold on
create, settle on completion, reaper releases abandoned), ownership-secured.

ai v1.800.0 go.mod is identical to v1.799.3 — no dependency-graph change;
only the hanzoai/ai hash lines move. Verified: cloud binary builds clean
(CGO_ENABLED=0) and the router now serves /v1/videos/generations,
/v1/videos/:id, /v1/videos/:id/content (spark-video backend).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-03 22:27:59 -07:00
d356b24e40 feat(tracker): native /v1/tracker issue tracker on SQLite (#109)
Durable replacement for the Huly/Svelte hanzo.team tracker whose upstream
each-block reactive-batching render race left issue lists rendering zero
rows. Native Go over one org-scoped SQLite store: rows return as plain JSON
and render deterministically (no Svelte reactivity in the path).

clients/tracker/store.go  — projects + issues on {DataDir}/tracker.db, the
  same modernc/SQLCipher driver + MaxOpenConns(1)+WAL pattern as projectsvc/
  crm. Per-project monotonic issue numbering allocated inside one tx (never
  races under the single-writer conn). Cascade delete in a tx. org column is
  the tenancy key; every query filters WHERE org=?.
clients/tracker/tracker.go — /v1/tracker/projects[/:key][/issues[/:num]] CRUD.
  org = principal.Tenant (validated IAM owner claim, HIP-0026), 403 otherwise.
  Status/priority closed sets; board/list via ?status=. Create wired to the
  shared per-org billing seam (free by default; ops prices via
  CLOUD_TRACKER_FEE_CENTS). Registered order 129, before the AI /v1/* catch-all.
subsystems/subsystems.go — one blank import links it into the binary.

Store CRUD/numbering/status-filter/cascade/tenant-isolation proven green on
real SQLite (clients/tracker/tracker_test.go).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-03 21:39:07 -07:00
hanzo-dev 3f2f92dbd3 chore(deps): bump hanzoai/ai → v1.799.3 (real default embedding provider — ingest produces real vectors); ships with the durable-enqueue default-namespace fix 2026-07-03 21:08:46 -07:00
hanzo-dev 232dfcb1e0 fix(durable): enqueue ingest into the 'default' namespace, not a per-org one
ROOT CAUSE of durable ingest silently running inline (round-trip: github ingest blocked
25s+, no workflow in Tasks): the embedded engine only registers 'default' at boot and does
NOT lazily create namespaces on ExecuteWorkflow, so dialing Namespace:<org> made the worker
poll a non-existent namespace and BLOCK → EnqueueIngest hung → handler fell back to inline.
Fix: dial 'default' (always registered). Data isolation is unchanged — it's in the workflow
INPUT (IngestSource is owner-scoped), never the namespace. Now github/crawl enqueue a real
durable workflow that appears in Tasks (under default).
2026-07-03 20:36:16 -07:00
hanzo-dev e883c83acd fix(deps): correct hanzoai/ai to v1.799.2 (v1.799.1 concurrent work + OpenAI-compat /v1/audio/speech) — supersedes the mis-numbered v1.796.6 downgrade 2026-07-03 18:13:45 -07:00
hanzo-dev 73b97198d0 chore(deps): bump hanzoai/ai → v1.796.6 (OpenAI-compat /v1/audio/speech; audio+image+video all native) 2026-07-03 18:12:22 -07:00
hanzo-devandGitHub 866a783b71 fix(kms): close V6+V1 pre-activation blockers on the PaaS KMS→Secret sync (#108)
V6: accept the owner-bound per-tenant machine audience (<owner>-platform-kms) so a real client_credentials sync token clears SanitizeIdentity + the /v1/kms guard; decoupled from global-admin (isKMSMachinePrincipal). V1: per-IP rate limit + MaxConnsPerHost(h2-off) on the public login broker. Blue→Red→Blue→Red: Red SHIP (0 crit/high/medium). Activation runbook in the PR body + EnsureOrgIdentity doc.
2026-07-03 17:55:23 -07:00
hanzo-devandGitHub 9f4856695a feat(kb): Knowledge Base + unified AI memory + connectors on the DocType engine (#107)
Fourth app lane (after cms/erp/help): a Notion-like KB + agent memory + app
connectors as a 'kb' module on clients/framework — no new Base, no new database.

Fixtures (module kb): kb-page (wiki tree via a self-Link parent + Lexical
RichText body), kb-memory (agent memory: note/fact/observation), kb-source
(connector-ingested docs), kb-connector (connection metadata; OAuth token in KMS,
never in the doc/logs). All CRUD/permissions/tenant-isolation/install are the
framework's generic /v1/framework/* surface + the generic @hanzo/ui renderer.

Indexing (index.go, the ONE vector-write path): an after_save hook embeds every
knowledge write (page/memory/source) via the gateway and upserts it into the org's
OWN Qdrant collection (kb_<org>) with an org-pinned payload; on_trash removes it.
Human wiki + AI memory are ONE per-org knowledge store, indexed once. Fail-open at
index time (a vector outage never blocks a knowledge write); fail-honest at query.

Retrieval (subsystem.go): POST /v1/kb/search is the org-scoped RAG entry point —
collection AND payload filter both pinned to principal.Tenant, so a caller can only
retrieve its OWN knowledge. Degrades to an honest empty result when the index is
down.

Connectors (connectors.go, sync.go): per-org OAuth to GitHub/Slack/Google that
ingest external docs INTO the same store + index (via framework.Ingest → same
after_save hook — one ingestion path, never forked). OAuth state is HMAC-bound to
the validated org (defeats login-CSRF/mix-up); tokens live in KMS at a per-org path.
GitHub is end-to-end (repo READMEs + issues); Slack/Google share the OAuth
lifecycle + normalizer with an honest 'listing not yet implemented' depth marker.

framework.Ingest/UpdateData/FindByField/Search: the in-process create-with-hooks
API first-party producers (the connector sync) use so off-request writes run the
exact validate + lifecycle pipeline and stay physically org-scoped.

Tests (16, all green): engine-Validate on every fixture; page self-Link + Lexical
body; connector-has-no-token; per-org pointID/collection/kmsRef isolation; payload
org-pin; OAuth-state org-binding + tamper/cross-provider/wrong-key rejection;
Ingest validates+fires-hooks+org-scoped. Integration (real SQLite + mock
Qdrant/embeddings over the real HTTP surface): install → create kb-page in org A →
after_save indexes into kb_A → search as A retrieves it → search as B sees NOTHING
(no cross-org leak); forged-org search → 403; kb-memory lands in the same kb_A
namespace.
2026-07-03 17:15:08 -07:00
hanzo-dev 1a115aab29 harden(integrations): red-team defense-in-depth on the OAuth connector plane
Adversarial review of the connector framework (state HMAC, nonce custody,
per-org KMS token custody, console redirect). Core design held: state forgery,
nonce replay/race, cross-tenant KMS pathing, principal-forge, and the seam
fail-closed contract were already sound. Fixes are defense-in-depth + red-style
proving tests; the contract is unchanged (no /api/, no /v1/slack route).

Fixes
- Ingest sanitization (vector 7): provider-supplied NON-secret metadata
  (account label / external id / bot user id / scopes) is now stripped of C0
  control chars + DEL and length-bounded at the ONE framework ingest point in
  callback, before it is logged, stored, or reflected. Kills log-line/separator
  injection via a crafted Slack workspace name and bounds per-org row growth.
  Secret token VALUES bypass this and go straight to the KMS seal.
- Open-redirect hardening (vector 4): success/failRedirect fold into one
  query-escaping consoleRedirectURL builder (DRY + unit-testable). Confirms the
  Location host is always the env-fixed console origin; hostile provider detail
  can't break out of the query into host/scheme/path or inject CRLF.
- Request bounds (vector 10): callback rejects an oversized OAuth `code`
  (maxCodeLen, also covers the in-process ZAP plane); verify() rejects an
  oversized state token before any base64 work (maxStateLen).
- kmsDelete uses errors.Is(kms.ErrSecretNotFound) for wrap-safe idempotency.

Tests (all real, -race green; 39 pass)
- state: dot-injection/degenerate split, MAC-checked-before-parse,
  validly-signed-but-hostile-org rejected, overlong rejected.
- store: concurrent single-winner nonce consume (race proof).
- http: no-open-redirect property, end-to-end metadata sanitization,
  disconnect anonymous-forge -> 403 (secret+row survive), github scaffold
  callback fails closed at the Configured gate before any exchange,
  oversized-code rejected.
2026-07-03 17:12:25 -07:00
hanzo-dev ec6869279c test(integrations): real TDD suite for the connector framework
29 tests, all green under -race:
- state: sign/verify, tamper, expired, wrong-provider, wrong-key, malformed, key resolve
- store: org isolation, connected_at preservation, external-id resolve, nonce single-use, GC, idempotent delete
- slack: authorize URL params, exchange parse (httptest ok+error), revoke, registration shape
- integrations: list shape, connect unconfigured->503 / no-principal->403 / KMS-down->503 / invalid-org->400,
  callback happy path (real KMS-sealed token + connection row + 302 console), replay rejected, tampered state rejected,
  disconnect (KMS+row deleted, idempotent), org isolation, seam fail-closed when unmounted.
KMS custody uses a REAL clients/kms.Client with a 32-byte master key; no mocks in prod paths.
2026-07-03 17:12:25 -07:00
hanzo-dev e0287a0e6f feat(integrations): generic OAuth connector framework (Slack ref, GitHub scaffold)
Provider-agnostic /v1/integrations plane: one registry, N providers.
Slack = full reference impl (OAuth v2 bot-token); GitHub = scaffold + #51 seam.
Per-org token custody in KMS (sealed); state-authed HMAC callback with
single-use nonce; org derived ONLY from signed state on the public callback.
Generic /v1/integrations/{provider}/callback — no /v1/slack/* (team-go owns it).
Wired at order 137 (after security 136, before AI 150).
2026-07-03 17:12:25 -07:00
hanzo-devandGitHub 4d2d6b0ed6 fix(ci+sqlite): tag-after-push release + single "sqlite" driver in both cgo/nocgo (#105)
release.yml — invert the tag/build order so a git tag can NEVER exist without a
pushed, boot-verified image (the phantom v1.786.42/43 → ImagePullBackOff cause):

  main push → compute next version → build → SMOKE (boot to "listening") →
  push image → git tag (receipt) → notify universe

- Tag is minted only AFTER the push step succeeds; any build/smoke/push failure
  fails the run before the tag step → fail-run-no-tag.
- concurrency group `release-cloud` (cancel-in-progress:false) serializes runs so
  two main pushes can't collide on a number; the queued run re-reads tags and
  lands on the next patch → monotonic.
- next version = max(highest git tag, highest pushed ghcr container tag) + 1,
  folding in container tags so a pushed-but-untagged number is never reused.
- removed the `tags: v*` trigger (this workflow now OWNS tags — a hand-cut tag has
  no image behind it and won't build); notify-universe fires only on a successful
  build+tag, so universe is never told about a phantom.

sqlite — fix `panic: sql: Register called twice for driver sqlite` under
CGO_ENABLED=1 (blocks `go test ./...` + clean cgo rebuilds). #96 moved cloud's
stores to github.com/hanzoai/sqlite (mattn under cgo) while several embedded deps
still import modernc.org/sqlite directly (ai, tasks, base, commerce, o11y, orm) →
two packages register "sqlite" under cgo. Prod is CGO_ENABLED=0 (all one modernc
package, deduped) so prod never paniced; the panic is cgo-only.

- bump github.com/hanzoai/sqlite v0.1.4 → v0.1.5: adds the `sqlite_purego` opt-out
  build tag that forces the fork's pure-Go (modernc) backend under cgo. Default
  cgo path is unchanged (mattn/SQLCipher) so IAM/commerce encryption is untouched.
- bump github.com/hanzoai/ai → the commit that routes object/adapter.go + cmd
  tools through hanzoai/sqlite instead of modernc (never modernc directly).
- Makefile: CGO_ENABLED?=0 default (matches the shipped Dockerfile) so `make
  build`/`make test` register "sqlite" once and exactly mirror prod; new `test-cgo`
  target proves the cgo path via `-tags sqlite_purego`.

Verified: CGO_ENABLED=0 `go build/test ./...` and CGO_ENABLED=1 `-tags
sqlite_purego go build/test ./...` both pass with NO panic (the eval package that
panicked now passes in both modes). Pre-existing clients/s3 + clients/functions
billing-attribution test failures are unrelated (present on clean main, both
modes) and out of scope.
2026-07-03 17:05:47 -07:00
hanzo-devandGitHub 7d072d9366 fix(console): close encoded-traversal gap on the /v1/billing money surface (#106)
billing.go's isSafeSegment left percent-escape (`%2f`/`%2e`) and matrix-param
(`;`) segments undecoded, so `/v1/billing/x/..%2fadmin` forwarded
`x/..%2fadmin` verbatim; the Go http client + commerce's own router
decode+normalize it downstream into a path that tunnels PAST /v1/billing into
another surface. commerce.go had already patched its call site with an inline
`%;` check — braiding the policy across call sites.

Harden the ONE shared segment guard instead: isSafeSegment now rejects empty,
`.`/`..`, slash, backslash, percent-escape, matrix-param, and any control char.
Both bridges (billing + commerce) get the complete guard from one place, and
commerce.go's call site drops the now-redundant inline check.

Regression test: `..`, `%2f`, `%2e%2e`, and `;` all 400 and never reach
upstream (proven to fail against the pre-fix guard).
2026-07-03 17:05:08 -07:00
hanzo-devandGitHub 9e6490f415 feat(console): per-tenant /v1/commerce/* store bridge for the static console (task #41) (#104)
The store twin of the just-merged /v1/billing/* bridge (#102). #81 namespaced
the console's commerce store calls to the canonical same-origin /v1/commerce/*
(SPA->server->/commerce proxy); the statically-exported console now terminates
every dynamic call at the unified cloud binary's /v1, so the binary must
reverse-proxy /v1/commerce/* to the commerce service. Without it the commerce
embed was incomplete.

clients/console/commerce.go serves GET|POST|PUT|PATCH|DELETE /v1/commerce/<path>
-> commerce's BARE store surface /v1/<path> (the console-side 'commerce'
namespace is stripped: the deployed commerce cmd/commerced mounts
api.Route(Group('/v1')), so products/orders/customers/... live at /v1/<kind>
while money lives at /v1/billing/*). Exactly the mapping console2's next.config
rewrite proved live (/v1/commerce/:path* -> /commerce/v1/:path* ->
commerce.svc/v1/:path*).

IDOR-safe: the org is the VALIDATED caller's own (resolveCaller ->
principal.Validated / c.Org()), never a client value; a bearer-less forged
X-Org-Id has no validated principal and is refused 403 before any commerce call.
Reuses the commerceDo(base,token) S2S transport billing.go/topup.go share
(admin COMMERCE_SERVICE_TOKEN + X-Org-Id, which commerce's EdgeAuth trusts only
behind the service token). Least privilege: a store-head allow-list (identical
to console2 proxy-allow.ts COMMERCE_HEADS) so the bridge can never tunnel to
/v1/billing (its own subject-scoped bridge), /v1/checkout, or tenant admin.

Hardened over a naive port: rejects percent-encoded path segments (%2f/%2e),
which the router leaves undecoded in the wildcard param but the Go http client +
commerce's router normalize downstream -- 'product/..%2fbilling' would otherwise
tunnel to /v1/billing past the allow-list (RED). Mirrors console2 pathIsClean.

/v1 only. CGO_ENABLED=0 go build ./... ok; go test ./clients/console/ ok.
2026-07-03 16:54:19 -07:00
hanzo-devandGitHub 3f5000e8ff feat(platform): activate PaaS KMS→Secret sync — org-scoped coords + login broker + per-tenant identity (#42) (#103)
Companion to #89 (KMS-sealed PaaS secret env) + universe #321. The sync was
INERT for two structural reasons this closes, and the per-tenant scoping the
task requires is now enforced at cloud's ONE auth boundary — proven by test.

WHY IT WAS INERT (coordinate drift + wrong CR shape):
  - Seal path ≠ read path. cloud sealed at /platform/tenant-<org>/<app> but the
    kms-operator reads through cloud's org-scoped surface /v1/kms/orgs/<org>/
    secrets/... which folds to /orgs/<org>/... — a DIFFERENT record, never found.
  - The CR set projectSlug="platform" (a literal) and omitted secretsScope.keys,
    which the CRD REQUIRES (MinItems=1; luxfi/kms has no list endpoint). Either
    alone starves the sync.
  - hostAPI carried a /v1/kms suffix; the operator appends /v1/kms/... itself, so
    login + read URLs doubled the prefix.

THE FIX (secrets.go):
  - Seal at the org-scoped coordinate  orgs/<org>/platform/<app>/<KEY>  — the EXACT
    store path cloud's org-scoped read surface addresses. Seal and read are now one
    coordinate (proven: TestPaaSSecretSealReadAlignment).
  - CR carries projectSlug=<org>, secretsPath=platform/<app>, envSlug=default, and
    the explicit sorted key roster. hostAPI = KMS root.

PER-TENANT SCOPING (the NON-NEGOTIABLE) — enforced, not hoped:
  - The operator authenticates as a per-tenant IAM machine identity (owner=<org>)
    via the NEW /v1/kms/auth/login broker (kmssvc/login.go): it exchanges the
    caller's clientId/clientSecret at IAM's client_credentials endpoint and returns
    IAM's owner-scoped token verbatim. cloud is a relay, not an issuer.
  - cloud's org-scope guard admits /orgs/<org>/... ONLY when the VALIDATED owner ==
    that org (SanitizeIdentity derives owner from the token, ignoring client
    X-Org-Id). So tenant-A's credential can NEVER read tenant-B's path — 403 before
    the store is touched (proven: TestPaaSSecretCrossTenantDenied).
  - credsSecret is a PER-TENANT name in the tenant namespace — never a shared
    platform-wide reader (that would be a cross-tenant hole = NO-SHIP).

PROVISIONING (ensureTenantKMSAuth) — fail-closed, one privileged seam:
  - On app-create/deploy cloud ensures the tenant's owner=<org> credential is
    projected into tenant-<org> as the creds Secret the CR references, via an
    injected tenantKMSIdentity provider. nil provider (default) ⇒ honest "pending"
    (operator can't log in ⇒ reads nothing) — NEVER a shared or wrong-org identity.
  - FLAGGED: the concrete provider needs a scoped IAM admin credential cloud does
    not yet hold (clients/admin/iam.go replays the caller's cred, no service
    identity). Until wired/verified, sync stays safely pending. Flip ON = provision
    the per-tenant identity; the security invariant holds regardless of how it is
    minted (the guard is the enforcement point).

Tests (CGO_ENABLED=0 — prod/CI build mode; CGO test builds double-register sqlite,
a pre-existing module issue): alignment round-trip, cross-tenant 403 (+ A→A 200,
B→B 404, unauth 403), login broker happy/bad-cred/malformed, per-tenant provisioning
(fail-closed when unprovisioned, org-bound projection, no cross-tenant ask). Existing
kmssvc red-team guard vectors unchanged and passing.
2026-07-03 16:44:28 -07:00
1343b10fc5 feat(console): per-tenant /v1/billing/* bridge for the static console (task #41) (#102)
The BFF catch-all sweep's one real "server work -> Go handler" case. console2's
app/billing/v1/[...path]/route.ts injected the commerce SERVICE token and pinned the
caller's billing subject server-side (work a static export cannot do). Ported to
clients/console/billing.go: GET|POST /v1/billing/* forwards to commerce with the admin
COMMERCE_SERVICE_TOKEN, scoping every request to the VALIDATED caller's own subject
(billingSubject + scopedBillingSearch + scopedBillingBody, the Go port of console2's
billing-scope.ts), so a tenant can only read/act on its OWN ledger.

IDOR-safe: the subject is the validated principal (resolveCaller: principal.Validated /
c.Org() / c.User()), never a client userId/org. A forged X-Org-Id with no validated
X-User-Id is refused (403). Unset COMMERCE_SERVICE_TOKEN -> honest 501.

DRY: commerceDo (topup.go) refactored to take (base, token) so the wallet top-up AND
this bridge share one S2S transport. No behavior change to topup (its tests pass).

Tests (billing_test.go): billingSubject personal/dedicated, subject pin + org drop +
passthrough (query & write body), forged-value overwrite, 403 no-principal, 501 no-token,
and the end-to-end scoped forward to a fake commerce. go build ./clients/console/ = 0;
go test (CGO_ENABLED=0) ok. The default-CGO modernc-vs-CGO sqlite double-register that
panics the package is a pre-existing repo-wide issue (separate sqlite-one-driver lane).

Co-authored-by: Hanzo AI <ai@hanzo.ai>
2026-07-03 16:28:32 -07:00
79667baa43 fix(identity): spoof-proof X-Project-Id/X-App-Id at the SanitizeIdentity boundary (#101)
SanitizeIdentity minted an un-forgeable X-Org-Id but passed the org sub-scopes
X-Project-Id / X-App-Id through verbatim, so a caller could assert ANOTHER org's
project as a compute_usage attribution key or per-project sub-scope. Sanitize the
sub-scopes in the ONE trust boundary:

- Delete every X-Project-Id/X-App-Id on ingress (no raw client copy survives),
  then re-inject only for a validated principal, against the acted-as org.
- Refuse a cross-org X-Project-Id: a project REGISTERED to a DIFFERENT org than
  the validated org is dropped; the caller's own registered project and
  unregistered free-form within-org labels survive (projectIsForeign; fail-closed
  on a registry error).
- Drop both sub-scopes entirely on the anonymous path.
- X-App-Id is a caller label, not an isolation boundary (no cloud subsystem
  scopes access by it; the un-forgeable org bounds any mislabel) - forwarded on
  the validated path, dropped when anonymous.

Dependency-inverted like sites.SetResolver: projectsvc registers a
TenantScopeResolver at Mount; cloud never imports the project registries. The
visor proxy forwards the now-validated sub-scopes so compute attribution lands on
the caller's own project. X-Org-Id anti-forgery is unchanged.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-03 16:24:37 -07:00
hanzo-dev cfe80a3c05 chore(deps): bump hanzoai/ai → v1.796.5 (zen-video spark-video provider); integrated blue/cloud-ml-sa-auth (cloud-ml SA k8s auth) 2026-07-03 16:24:20 -07:00
hanzo-dev cc629adef4 feat(framework): RichText fieldtype for the CMS WYSIWYG body
Add a RichText fieldtype to the DocType engine so a content field can hold a
Lexical EditorState JSON string (the console renders it with a native WYSIWYG).
The value is opaque text — validate.go coerces it verbatim (clipped to the scalar
bound), stored in the schemaless doc blob, round-tripping through create→get with
no shape enforcement (that's a UI concern). Minimal + fail-closed: the fieldtype
is added to the validated allow-set, so an unknown type is still rejected.

CMS: the seeded content body (Article/Page/Post) becomes RichText, and a
Data field is added for optional per-project scoping (the console's org→project
switcher filters content by it; empty = org-level). One CMS engine; project is a
filter.

Tests: TestDocTypeValidate accepts a RichText field; TestFieldTypeValidation
proves a Lexical JSON string round-trips verbatim through validation.
2026-07-03 16:18:14 -07:00
hanzo-blueandhanzo-dev 288a22ac62 ml(clients): authenticate as dedicated cloud-ml SA via HANZO_ML_TOKEN_FILE
The ML control plane (clients/ml, /v1/ml) ran under the pod SA (cloud-api),
braiding KServe/Kubeflow cluster reach onto the product-API identity. newDynamic
now re-scopes the in-cluster client to a mounted cloud-ml ServiceAccount token
when HANZO_ML_TOKEN_FILE is set (keeps in-cluster host+CA, swaps only identity),
fail-closed if the configured token is unreadable. Unset -> unchanged behaviour
(pod SA) for local/dev and pre-cutover. Pairs with universe ml-rbac.yaml
(cloud-mlsvc ClusterRoleBinding -> cloud-ml). go build + go vet clean.
2026-07-03 16:14:29 -07:00
hanzo-dev 9c2d91f4de docs(cloud): wave map — remaining Go services and their merge disposition
The execution queue for "all Hanzo Go services merge into the one cloud binary"
(HIP-0106): wave 0 already-merged (8 embedded modules + 37 native clients),
wave 1 tasks+visor (this build), wave 2+ mount queue (notify2, extract-svc,
playground, ...), and keep-standalone with reasons (iam/gateway/kms-MPC/registry
/s3/docdb/chain daemons). Waves are sequential through go.mod to avoid the
in-flight collision this wave hit.
2026-07-03 16:11:00 -07:00
hanzo-dev 7e9e4ca6ab feat(tasks): mount Tasks HTTP+UI surface on the ONE in-process engine
Consolidates the Tasks product surface into cloud — the follow-up durable.go
named ("consolidating that surface into cloud is the follow-up"). durable.go
already embeds the ONE tasks engine in-process (loopback ZAP :19999) for ai's
durable ingest; this mounts THAT SAME engine's HTTP handlers, so the Tasks
UI/API reads the same durable state as ingest. One engine, one binary, one way —
no second Embed.

- clients/tasksvc (order 147, before ai's /v1/* catch-all): adapts the shared
  engine's HTTP surface onto the zip mux at /v1/tasks/* + the embedded React UI
  at /_/tasks/*. The engine is created after MountAll, so the surface resolves
  cloud.EmbeddedTasks() lazily per request (503 fail-soft until wired).
- gate: settings/cluster/health stay open (no per-org data); the data surface
  (namespaces/workflows/mcp/events) refuses an unvalidated principal (403, never
  the unscoped store) and threads the gateway-validated org into the engine via
  tasks/pkg/auth.WithIdentity — per-(org,ns) shard isolation, matching the rest
  of the cloud data plane (clients/principal).
- durable.go: export EmbeddedTasks() — the single shared-engine accessor.
- bump hanzoai/tasks v1.43.0 → v1.46.0 (in-proc identity seam + one sqlite
  driver). No local replace directives.

Proof: /v1/tasks/cluster returns nodeId "cloud-tasks" (durable.go's engine),
settings 200, data routes 403 without a principal, /_/tasks UI 200.
2026-07-03 16:11:00 -07:00
hanzo-devandGitHub 602f5203ae debrand: langfuse -> o11y/observability in our prose & comments (#100)
Drop the Langfuse brand from our own strings (code comments, docs,
config labels), mirroring the signoz->o11y product rename. Meaning preserved;
comments/docs/labels only, no functional change.

Intentionally KEPT (references to the external Langfuse product / upstream
dependency / integration contract, not our brand):
- LiteLLM success_callback/failure_callback ["langfuse"] + LANGFUSE_* env
  var names (the litellm langfuse-callback contract; renaming breaks emission)
- infra/k8s/langfuse/* (deploys upstream langfuse/langfuse:3 OSS image)
- o11y/langfuse-otlp-fanout.yaml + console-langfuse-keys (trace-fanout lane)
- console NOTICE (MIT attribution to Langfuse GmbH for clean-room UX)

Trace pipeline (ai emit -> collector -> backend -> console Observe) unchanged.
2026-07-03 16:06:15 -07:00
hanzo-dev cc0710ad10 Merge native ERP + Helpdesk DocType lanes (feat/erp-help-framework-modules, RED-PASSED 9/9 race)
ERP (clients/erp): ERPNext-core DocType fixtures + native-Go GL/stock hooks —
idempotent deterministic-leg postings (exactly-once under concurrent submit),
on_cancel reversal, finite-guarded totals, double-entry submit gates.
Help (clients/help): Frappe Helpdesk-core fixtures, pure DocTypes, no hooks.
Both register on the framework engine at init; installed per-org via
/v1/framework/modules/{erp,help}/install. No new HTTP surface — ERP/Help ARE
documents on /v1/framework/*, drawn by the same generic DocType renderer as CMS.

Verified CGO=0 (production Dockerfile config): binary boots clean (no double
sqlite driver register), /v1/framework/health 200, /v1/models unshadowed (503
real AI handler), erp+help tests green under -race.
2026-07-03 15:51:13 -07:00
97b90ae449 feat(visor): /v1/bots surface + machine agent-binding proxies (CLOUD, mirrors machines) (#98)
Mounts /v1/bots in cloud as the sibling of /v1/machines — a Bot is an
Agent(cloud /v1/agents) + a kind=bot Machine(vm) + their AgentBinding,
composed as a thin proxy over the SAME Visor client the machines routes use:

  GET    /v1/bots                    list (vm /v1/machines?kind=bot + bindings join)
  POST   /v1/bots/launch             machine launch{kind:bot} THEN bind-agent
  GET    /v1/bots/:id                machine + its binding (404 if not a bot)
  DELETE /v1/bots/:id                unbind THEN terminate the machine
  POST   /v1/bots/:id/:action        message=agent run | stop|pause=unbind

Plus the machine agent-binding proxies cloud lacked (vm already serves them):

  POST   /v1/machines/:id/bind-agent
  GET    /v1/machines/:id/agent-binding
  DELETE /v1/machines/:id/agent-binding
  GET    /v1/agent-bindings

Every route org-gated by the validated principal (principal.Tenant), forwarded
to vm as ?owner=<org> — 403 without a valid IAM owner, exactly like machines.
No vm change: kind=bot launch + bind-agent are already live at visor:19000.
message runs the bot's bound agent via the ONE agent runner (/v1/agents/:agent/run).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-03 15:47:45 -07:00
hanzo-dev 6832090ee3 fix(durable): pin tasks engine DataDir to cloud data root (distroless has no /tmp)
Embed's default os.MkdirTemp("") resolves to /tmp, absent in the distroless cloud image
→ 'tasks.Embed: tempdir: stat /tmp: no such file or directory' → fail-soft to inline.
Pin DataDir to {deps.DataDir}/tasks (MkdirAll first) so the in-process engine actually
boots. Verified: without this the warn+inline-fallback fired cleanly in prod (v1.786.48).
2026-07-03 15:38:59 -07:00
hanzo-dev bcc097ef05 feat(durable): embed the ONE tasks engine in-process — unified durable queue
cloud embeds hanzoai/tasks IN-PROCESS (loopback ZAP, durable.go) and injects a per-org
dialer into ai's ingest — there is no external tasks service to auth to, no per-org token
minting, no HTTP inner-cloud hop. Long ingests (github/crawl/s3) run as durable workflows
in the owner's namespace (CONTRACT §6); upload stays inline. Fail-soft: embed error →
ai dialer unset → inline fallback. Bumps ai → v1.796.4 (per-org ingest dialer). One engine,
one binary, one way. NOTE: embedded store is memdb today (survives worker crash via retry,
not process restart); console /tasksd still points at the cluster tasks Service for the UI
(consolidating that surface into cloud is the follow-up).
2026-07-03 15:21:08 -07:00
hanzo-dev 59ea8ffba9 fix(erp): address Red review — idempotent postings + cancel reversal + finite guard
Red verdict FIX-THEN-SHIP (0 critical; all findings within-tenant integrity —
isolation, forge-proofing, gates, ledger perms, console deletions all refuted/solid).

- HIGH (TOCTOU double-post): on_submit postings ran before the atomic docstatus flip
  with hash-named legs, so N concurrent submits over-posted the ledger 4-6x. Every
  GL/stock leg now has a DETERMINISTIC name (voucher-<kind>-<index>, via prompt
  autoname) and postLeg is idempotent (pre-read + re-check on create-conflict), so
  posting is exactly-once under any concurrency AND replayable after a partial
  failure — no engine change, uses the existing store API. Balances stay SUM(ledger).
- MED (cancel did not reverse): on_cancel hooks append reversing ledger rows (swap
  debit/credit; negate qty), sharing the SAME leg computation as submit.
- LOW (non-finite total -> 500): finite guards in the totals hooks -> clean 422.
- LOW (comment): tightened the ledger-immutability doc — manager bypass is
  within-tenant authority only (Red confirmed no cross-org escalation).

Regressions: 8 concurrent submits -> exactly 1x200 + GL posted once (2 legs,
race-clean); cancel -> net GL zero-sum + net stock zero; overflow qty*rate -> 422.
go test -race 9/9 (CGO=1) + CGO=0 + vet clean + full cmd/cloud binary.
2026-07-03 15:11:51 -07:00
1ad09837ea eval: mount GET /v1/evals/observations reading hanzo.cloud_usage (#99)
Every AI call already writes the proven hanzo.cloud_usage ledger (model,
provider, tokens, cost_cents, org, user, status) via ai/object's zapWriteUsage
— the same recordTrace funnel that emits the OTel GenAI span. This mounts the
missing native route the console Observe > Observations surface calls, reading
that ledger as Langfuse-v3 GENERATION observations (org-scoped by the validated
principal, bound positional params, bounded LIMIT). No new emission path: one
recordTrace, fanned to o11y (span) + Langfuse (span) + this ledger (read).

- telemetry.go: Observation model + ObservationFilter; ListObservations on the
  Telemetry interface, dsTelemetry (cloud_usage query), memTelemetry (honest
  empty); asInt64 coercer (cloud_usage UInt32 tokens / UInt64 cost — asFloat
  only handles float types).
- eval.go: listObservations handler + observationView/toObservationView mapping
  to the console Observation shape; GET /v1/evals/observations route.
- observations_test.go: view mapping (success/error), asInt64 coercion, mem
  telemetry empty + org-required.

Requires a cloud rebuild + deploy-by-sha to go live (route is code, not env).

Co-authored-by: hanzo <a@hanzo.ai>
2026-07-03 15:06:29 -07:00
hanzo-dev d6ddfc74af chore(deps): bump hanzoai/ai → v1.796.3 (routed /v1/docs/ingest + code-aware splitter + brand-neutral per-org store + transport-agnostic durable-ingest seam) 2026-07-03 15:01:54 -07:00
03be81ef15 feat(o11y): ZAP-native span export + GenAI spans on LLM/agent paths (#95)
- zaptrace: otlptrace.Client over the ZAP wire (github.com/zap-proto/http) —
  spans marshaled as OTLP protobuf, shipped over ZAP frames, NEVER OTLP-HTTP
  (:4318)/gRPC(:4317). Target = collector zapreceiver (:4319).
- cmd/cloud/telemetry.go: initTelemetry uses the ZAP exporter; enable via
  OTEL_EXPORTER_ZAP_ENDPOINT (default otel-collector.hanzo.svc:4319); keeps the
  no-op-when-unset posture.
- clients/aihttp.go ChatCompletion: OTel GenAI client span (gen_ai.system/
  operation.name/request.model + response.model + usage.{input,output}_tokens),
  RecordError on failure. Captures the previously-discarded resp.Usage.
- clients/agents/agents.go: runAgent opens a per-run root span (agent.run),
  executeRun a child agent.step span; the LLM client span nests under them —
  one trace per run: run -> step -> chat.

Test: zaptrace TestUploadTracesOverZAP green (span over the real ZAP wire).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-03 14:56:48 -07:00
1c30259d24 feat(provisioning): docdb dedicated engine = FerretDB on Hanzo SQL (SQLite) (#97)
Supersede the Base swap (84d4a77). Hanzo Base has a REST/realtime document
API but NOT the MongoDB wire protocol, so it is not a drop-in docdb — a
customer's mongodb:// driver cannot speak to it. The managed "document
database" must accept existing MongoDB drivers unchanged.

The per-org dedicated docdb instance is now a FerretDB v1.24 instance
speaking the MongoDB wire protocol on :27017, backed by Hanzo SQL (SQLite,
pure-Go) — Mongo databases map to SQLite files under /state, collections to
tables, documents to JSON1 rows. ZERO raw mongod (no WiredTiger), ZERO
Postgres, ZERO go.mongodb.org driver in cloud (FerretDB is a deployed pod,
not a Go import). Per-instance SCRAM auth via FerretDB's SQLite-backend
new-auth (FERRETDB_TEST_ENABLE_NEW_AUTH + FERRETDB_SETUP_*); the returned
password is the instance admin credential, sealed in KMS.

engine fields:
- image ghcr.io/hanzoai/docdb-sqlite:1.24.0 — pinned FerretDB v1 with the
  SQLite backend handler, mirrored from upstream by hanzoai/docdb CI (v2 and
  the hanzoai/docdb Postgres/DocumentDB fork both dropped SQLite; v1 is the
  last line that carries it). Distinct package from the Postgres-backed
  ghcr.io/hanzoai/docdb that backs shared chat-docdb.
- fsGroup 1000: FerretDB is distroless and runs as UID:GID 1000 (no
  entrypoint can chown), so a fresh block PVC must be group-writable via the
  pod securityContext.fsGroup the operator stamps from spec.fsGroup — else
  the instance CrashLoops on "permission denied" writing /state.
- FERRETDB_STATE_DIR + FERRETDB_SQLITE_URL pin both process state and the
  SQLite files onto the mounted /state PVC (persist across restarts).

Verified end-to-end against the FerretDB v1.24 SQLite image with this exact
env: mongosh Insert/Find/Update/Delete over the wire protocol, and /state
held per-db admin.sqlite + events.sqlite with "SQLite format 3" magic — no
WiredTiger datadir, no Postgres PG_VERSION (both locally and in hanzoai/docdb
CI on the mirrored image).

TestDedicated_DocdbIsFerretOnSQL asserts the FerretDB image, SQLite backend
env, mongodb:// connString, per-instance SCRAM credential, fsGroup 1000, and
the absence of any Postgres/IAM/Base env. unavailableKinds stays empty.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-03 14:40:02 -07:00
hanzo-dev 55e338ea84 feat(erp+help): native ERP + Helpdesk DocType lanes on the framework
Second and third app lanes on clients/framework, reusing the generic DocType
engine + install path + generic renderer (like clients/cms) — zero forked
engine, zero HTTP surface of their own, per-org on Base/SQLite.

- clients/erp: 20 ERPNext-core DocTypes (module "erp") — masters (item/
  warehouse/customer/supplier/account/department/employee), submittable
  transactions with child Tables (sales-order/-invoice/purchase-order/
  stock-entry/journal-entry/payment-entry), and two read-only hook-posted
  ledgers (gl-entry, stock-ledger-entry). Business logic as native-Go hooks:
  line/document totals (before_save), balanced GL on invoice/journal/payment
  submit, append-only stock ledger on stock-entry submit, and double-entry /
  non-empty submit gates. Posting is org-scoped via ev.Org + ev.Store.
- clients/help: 5 Helpdesk DocTypes (module "help") — hd-ticket (status
  workflow) + hd-agent/hd-team/hd-sla/hd-canned-response. Pure fixtures, no
  hooks — the purest DRY proof; self-contained (no cross-lane Link).
- SLUG names (erp-*/hd-*), series naming for transactions, field naming for
  masters (console slugifies on write), hash for ledgers — all reachable via
  the generic renderer; no collision with CMS (Author/Media/Page/...) or CRM.
- subsystems: blank-import erp + help so their init() registers the lanes.

Tests: fixtures-valid/model-spec/install-transact-roundtrip/submit-gates/
tenant-isolation/ledger-read-only — go test -race green (CGO=1 and CGO=0),
go vet clean, full cmd/cloud binary builds.
2026-07-03 14:39:26 -07:00
zandGitHub cf027cfeb1 Merge pull request #91 from hanzoai/platform/whitelabel-signin-cloud
feat(auth): cloud accepts every white-label brand's issuer + audience (lux/zoo/pars login)
2026-07-03 14:28:01 -07:00
hanzo-dev 80c15d47bc feat(security): hanzo security scan CLI + decomplect engine into detect pkg
The pure detection engine moves to clients/security/detect (a stdlib-only
LEAF): one engine, two surfaces — the /v1/security HTTP subsystem and the
new local CLI both consume it, neither drags the other in. The subsystem
now calls detect.ScanContent/Rules/SeverityRank; behavior is unchanged.

hanzo security scan [path...] walks a tree, runs the engine, and exits
non-zero when a finding at/above --fail-on (default low; 'none' = report
only) is present — a pre-commit/CI/agent guardrail with no server, auth,
or network. Skips vendored/binary files; never prints a raw secret (masked
preview only). hanzo security rules lists the catalog. -o json supported.
Tests: 9 CLI (find+fail, clean-pass, fail-on threshold/none, json, vendor+
binary skip, bad flag, rules, control-verb), engine+subsystem unchanged.
2026-07-03 14:26:29 -07:00
zeekayandhanzo-dev efba12fee9 feat(auth): accept every white-label brand's cloud audience — one binary, all brands
Extends the white-label issuer-set validation (this branch) to the AUDIENCE half:
a lux/zoo/pars session token carries aud=<brand>-cloud (HIP-0111: client_id == app
== aud), so the audience allowlist must include each or the cloud-native identity
sanitizer 401s a valid lux token even after the issuer gate passes.

- brand.go BrandAudiences() derives <brand>-cloud for every registry brand
  (hanzo-cloud, lux-cloud, zoo-cloud, pars-cloud, bootnode-cloud) — one source of
  truth, mirroring BrandIssuers(); no hand-listed audience.
- config.go jwtAudiencesFromEnv() now ALWAYS unions BrandAudiences() into the
  resolved allowlist (baked like the brand issuers). A legacy hanzo-only
  GATEWAY_ALLOWED_AUDIENCES env override still accepts lux-cloud — the brand auds
  don't depend on getting the deploy env perfectly right. Fail-secure: only ADDS
  the known-good <brand>-cloud client_ids, never an arbitrary aud. unionStrings
  dedupes so an env-supplied entry is never duplicated.

Tests: BrandAudiences (registry-derived, covers every brand), jwtAudiencesFromEnv
brand-union (baked default AND a hanzo-only env override both accept lux-cloud, no
duplicate). Paired with hanzoai/ai#64 (the per-brand signin code exchange).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-03 14:26:24 -07:00
zeekayandhanzo-dev 8016ea759e wip(auth): white-label issuer-set validation — one binary validates all brand issuers
Widen identityValidator to a trusted issuer SET (primary UNION BrandIssuers) so one
cloud binary validates hanzo AND lux/zoo/pars tokens off the one shared IAM JWKS.
Fail-secure: only known-good brand issuers added. NOT built (phantom go-sqlite3
v2.0.3 dep blocks) / NOT gated / NOT deployed — needs: per-brand EXCHANGE client
verification, build-dep fix, hanzo-login no-regression gate, red review.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-03 14:26:24 -07:00
d1f82f214d fix(sqlite): every store imports the hanzoai/sqlite fork, never modernc directly (#96)
github.com/hanzoai/sqlite is the ONE Hanzo SQLite driver: it registers the
"sqlite" database/sql name under BOTH build tags (cgo → mattn+SQLCipher,
encrypted at rest; !cgo → pure-Go modernc, wrapped internally). Fourteen
cloud stores plus the pg→sqlite migration blank-imported modernc.org/sqlite
DIRECTLY, so a CGO_ENABLED=1 build registered "sqlite" twice — the fork's
mattn registration AND the direct modernc one — and panicked at init
("sql: Register called twice for driver sqlite"), taking down the whole
fused hanzo/cloud binary.

Swept every direct `_ "modernc.org/sqlite"` → `_ "github.com/hanzoai/sqlite"`
(sql.Open("sqlite", …) calls unchanged — same driver name). go.mod promotes
the fork to a direct require and demotes modernc to indirect (it survives
only as the fork's !cgo backend). Stale comments calling modernc the
"primary" driver corrected. Production already builds CGO_ENABLED=0 (one
modernc package, no collision); this makes the driver choice consistent and
unblocks a CGO_ENABLED=1 + libsqlcipher encrypted build.

NOTE: five UPSTREAM modules (base/core, ai/object, o11y sqlstore,
commerce/db, orm/db) still import modernc directly; a CGO_ENABLED=1 fused
build stays collision-prone until they adopt the fork too. Out of scope for
this repo; tracked as the cross-repo follow-up.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-03 14:22:50 -07:00
hanzo-dev dcd97b2552 chore(deps): bump hanzoai/ai → v1.796.2 (durable tasks ingest + routed /v1/docs/ingest + code-aware splitter + brand-neutral per-org store); tasks → v1.43.0 2026-07-03 14:12:56 -07:00
hanzo-dev 1c2af2d697 Merge CMS-on-Framework keystone (feat/cms-framework-module, RED-PASSED 1eea6105)
First business app lane native on the Hanzo Framework DocType engine:
generic module-install path (/v1/framework/modules[/:module[/install]]) +
the CMS content model as fixtures (Page/Post/Article/Media/Navigation/Author,
module cms). Additive — no change to the proven /v1/framework/* isolation.
RED verdict: SHIP (0 crit/high/med).
2026-07-03 13:16:00 -07:00
hanzo-dev 350bbdacec chore(deps): bump hanzoai/ai → v1.796.1 (unified ingest routed + code-aware splitter + brand-neutral per-org docs store) 2026-07-03 12:58:38 -07:00
8e4488153c feat(security): native /v1/security/* — dependency-free secrets scanner (#94)
The first Semgrep-class capability shipped natively in the cloud binary,
per hanzoai/security POSTURE.md's plan of record. One subsystem, the
established clients/ pattern (self-registering Mount, org-scoped store
under DataDir, audit + metering wired), zero external tools.

- engine.go — the reusable detection core: pure (path,content)→findings,
  no I/O. Pattern rules (AWS/GCP/GitHub/Stripe/Slack/npm keys, private-key
  blocks, JWTs) + a Shannon-entropy-gated generic-assignment rule so
  `secret = "changeme"` is not flagged but a real high-entropy token is.
  THE INVARIANT: a finding never carries the raw secret — only a masked
  preview (4+4 ends, middle starred; short secrets fully starred) and the
  SHA-256 fingerprint (dedupe + rotation tracking). Persisting plaintext
  would make the findings DB the very thing we scan to prevent.
- store.go — per-tenant SQLite ({DataDir}/security.db), scans + findings
  tables, org column is the isolation boundary on every query; SaveScan is
  one transaction so a scan is never half-written. Mirrors clients/git.
- security.go — mounts /v1/security/{health,rules,scans,scans/:id,
  findings,findings/:id}. submitScan runs the engine, persists redacted
  findings, meters one unit, emits a tamper-evident audit record (the
  tally, never the secrets). Registered cloud.RegisterWithShutdown(
  "security", 136, …) + one blank-import line in subsystems.

Tests (14, no skips, no fakes): engine_test proves each rule fires, the
entropy gate, line mapping, dedupe, severity ordering, and that no field
ever echoes the raw secret; security_test proves the HTTP surface,
cross-tenant isolation (evil sees 0 of acme's scans/findings, 404 on id),
the no-principal 403, the severity filter, and that a clean scan persists
a real zero-findings record. go build ./... + go vet + -race all clean.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-03 12:50:51 -07:00
hanzo-dev 1eea6105e9 feat(framework+cms): generic app-lane module fixtures + CMS content model
Framework: a generic, DRY module-install path so an app lane (CMS/ERP/Helpdesk)
declares its DocTypes as fixtures (framework.RegisterModule, sibling to
RegisterHook) and installs them per-org via the engine's own gate:
  GET  /v1/framework/modules                list registered lanes
  GET  /v1/framework/modules/:module        lane fixtures + which are installed in-org
  POST /v1/framework/modules/:module/install  ensure fixtures exist (managerOnly, idempotent)
'modules' reserved so the static routes are never shadowed by a document route.

CMS (clients/cms): the first lane — the content model as fixtures only, NO HTTP
surface of its own. Page/Post/Article (slug-named, status Draft/Published,
author Link), Media (Attach-backed DAM), Navigation (JSON menu), Author. A CMS
collection IS a framework DocType (module 'cms'); content IS documents;
publishing IS a status field. Registered at init; installed per-org.

Secure by default: install is managerOnly (owner seeded trust-on-first-use),
create-if-absent (never clobbers a customised DocType), stamps the module tag,
and every op stays per-org via principal.Tenant.

Tests: install/idempotency/unknown-404/tenant-isolation/forged-principal-403/
non-owner-403/module-tag; CMS fixture validity + content-model spec + a full
HTTP install->create Author->create Page(link)->publish->filter round-trip.
2026-07-03 12:44:06 -07:00
hanzo-dev 4f9fb6f97b fix(framework): RED MEDIUM — decode URL-encoded path params (space-named DocTypes/docs/roles)
The router (zip over fasthttp) runs with Fiber's default UnescapePath:false, so
c.Param() returns path segments verbatim. A DocType/document/role name that is
legal per docTypeNameRe but contains a space ('Sales Invoice', 'System Manager')
arrives percent-encoded ('%20') and never matched its stored value: GET/PUT/
DELETE /v1/framework/:doctype/:name, submit/cancel, and revokeRole (:user/:role)
all 404'd, while create+list (no name in the path) worked — records could be made
yet be unreachable, and a granted System Manager could not be revoked.

Fix scoped to the ONE seam: pathParam() percent-decodes every framework path
param (getDocType/replaceDocType/deleteDocType, access(:doctype), docName,
revokeRole). NOT a global fiber.Config UnescapePath flip — that would change
segment splitting on the KMS secret-path, model-catalog, git and s3 wildcards
(c.Params("*")) that legitimately carry encoded slashes; the framework-local
decode is orthogonal and zero-blast-radius. Malformed escapes fall through to an
honest 404, never a panic.

Unblocks space-named DocTypes for the CMS/ERP/Help app lanes. Tests: red→green
round-trip (create -> GET/PUT/submit/cancel/DELETE by name) + space-named role
revoke; full framework suite green under -race.
2026-07-03 12:27:31 -07:00
84d4a77872 feat(provisioning): docdb dedicated engine = Hanzo Base, not MongoDB/FerretDB (#93)
Reconciles task #52 (eliminate Mongo) onto main's dedicated-per-org instance
model. The managed "document database" (docdb) is now a dedicated per-org Hanzo
Base instance — JSON document collections on per-tenant SQLite with native
realtime (SSE /v1/realtime), IAM-native — NOT a per-org FerretDB/Mongo instance.

- dedicated.go: docdb engine swapped ghcr.io/hanzoai/docdb:0.1.0 (mongodb://,:27017,
  POSTGRES_*) -> ghcr.io/hanzoai/base (http://.../v1, :8090, /data), dsType "base".
  New engine fields: dataMount (emits spec.volumeMounts so the data PVC actually
  mounts — the operator does NOT auto-mount) and iamAuth (IAM-native: no per-
  resource password generated/sealed/returned; admin Secret carries IAM_URL/
  KMS_URL/IAM_CLIENT_* from the cloud binary's own IAM identity). baseInstanceEnv
  helper. createDedicated honors iamAuth (no pw path). datastoreCR emits
  volumeMounts. Runs on the operator's GENERIC Datastore controller — spec.type is
  free-form, image/ports/volumeMounts drive the StatefulSet verbatim, NO operator
  Rust change (verified). datastore engine stays ClickHouse (not Mongo).
- provisioning.go: header + sanitizeIdent comments de-Mongo'd.
- go.mod: mongo-driver demoted direct -> // indirect (zero Go imports of
  go.mongodb.org remain; it survives only as a transitive requirement).
- test: TestDedicated_DocdbIsBase asserts the docdb CR is the base image on :8090
  with /data mounted, connString http://.../v1 (no mongodb://), no credential
  (IAM-native), IAM env in the admin Secret. PASS.

Audit (unchanged): zero customer docdb data, so the swap is clean.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-03 11:56:46 -07:00
hanzo-dev 564338b8a7 Merge remote-tracking branch 'origin/main' into feat/framework-doctype-engine 2026-07-03 11:20:55 -07:00
hanzo-devandGitHub c1f8e08457 fix(enablement): RED MEDIUM — scope self-opt-in/view to the VALIDATED tenant (#92)
Red found the self-service enablement write path (POST /v1/enablement/optin|optout)
+ the view keyed on raw c.Org() instead of principal.Tenant(c). On the bearer-less
direct-to-pod path SanitizeIdentity restores a client X-Org-Id with no validated
principal, so an off-gateway caller could opt an org it does not own into/out of a
beta (cross-tenant enablement write; bounded — betaOrgs membership of already-beta
items only, no money/data/global-state, not reachable through the gateway).

Fix (DRY — pricing already imports principal, the READ catalog gate already uses it):
- enablementOpt: resolve subject via principal.Tenant(c), 401 on !ok (was raw c.Org()).
- enablementView: resolve via the existing trustedOrg(c) (validated-principal gate).
- Corrected the docstrings (subject is the VALIDATED tenant, not 'never client-supplied').

Red's two PoC attack tests (enablement_attack_test.go) now GREEN; full enablement
suite unchanged + green + -race. Closes the one MEDIUM from Red's cockpit review.
2026-07-03 10:10:42 -07:00
hanzo-dev 85a9e3091d fix(admin): decode commerce transactions {count,transactions} wrapper — analytics ledger was silently empty
Live-verified: /v1/billing/transactions returns {count, transactions:[...]}, not a bare array.
My client decoded a bare [] (test fake also returned bare — mock hid the bug), so the analytics
retention/churn/active/usage ledger read got ZERO rows despite real usage (maxpower: 268 txns).
Now decodes the wrapped shape (bare-array fallback for robustness); test fake mirrors the live shape.
2026-07-03 09:55:46 -07:00
hanzo-devandGitHub 5cae3b9a7b feat(platform): KMS-sealed secret env vars (close the 501) (#89)
User app secret env is no longer refused — it is sealed into cloud's embedded
KMS and wired to the pod via an operator-materialized k8s Secret, never
plaintext, never logged.

The path a secret takes (secrets.go):
  1. SEAL   — createApp / PUT .../env seal every secret:true value into deps.KMS
              at a per-tenant/app coordinate (platform/<tenant-ns>/<app>/<KEY>);
              the persisted env_json value is blanked. Fails CLOSED if KMS is
              unavailable — a plaintext secret never lands in the DB as a fallback.
  2. DECLARE — on deploy (applyLive, the ONE shared choke point) cloud writes a
              canonical KMSSecret CR (secrets.lux.network/v1alpha1) into
              tenant-<org> declaring a managed Secret <app>-env sourced from that
              KMS scope. Best-effort: a missing CRD/RBAC degrades to an honest
              'pending' status, never a failed deploy.
  3. MOUNT  — the Service CR renders each secret env as valueFrom.secretKeyRef →
              that Secret (optional:true so the pod boots pre-sync); the hanzo
              operator (which already supports secretKeyRef) mounts it into the
              Deployment env. cloud is never in the plaintext path at runtime.

Also: PUT /v1/platform/projects/:p/apps/:a/env to set/rotate env post-create;
honest secretSync status (pending|syncing|ready|failed) from the KMSSecret CR
conditions on the app view; KMSSecret teardown on app/project delete.

Tests: seal blanks+seals+fails-closed; injective KMS refs (no cross-tenant
collision); canonical KMSSecret CR shape; apply/patch/delete; sync-status
mapping; and an end-to-end deploy asserting the Service CR carries secretKeyRef
(never plaintext) and the KMSSecret CR is authored. go build ./...=0, vet clean.
2026-07-03 09:45:30 -07:00
hanzo-dev 9fa89757b4 feat(admin): operator cockpit — customers/revenue/analytics + enablement registry
/v1/admin/* (global-admin gated, reuse s.guard→c.IsAdmin owner==AdminOrg):
- GET  /v1/admin/customers          fleet customer list (balance/spend/plan/status), concurrent enrichment
- GET  /v1/admin/customers/:org     detail (balance/usage/keys-presence/txns/users) — no card data, no key values
- POST /v1/admin/customers/:org/credit      real commerce deposit, audited before/after
- POST /v1/admin/customers/:org/{suspend,reactivate}  IAM isForbidden flip (login+token enforced), audited
- GET  /v1/admin/revenue            fleet revenue aggregate + per-customer table + ARPU + real spend trend
- GET  /v1/admin/analytics          native SaaS analytics: cohort retention, growth, churn, DAU/WAU/MAU,
                                     revenue/ARPU, usage — real from IAM createdTime + commerce ledger,
                                     honest-empty + computed[] transparency map (no fabricated curves)

Enablement registry #30/#31 (extends the ONE pricing catalog overlay, DRY):
- Overlay gains explicit Beta flag → tri-state off|beta|ga; off is an ABSOLUTE kill switch
  (visibleTo ignores betaOrgs when !beta). Additive migration + backfill (no regression).
- GET/PUT /v1/admin/enablement       global-admin: list + set state (+grant orgs)
- GET  /v1/enablement                 caller effective view + available betas
- POST /v1/enablement/{optin,optout}  self-service, subject = SANITIZED caller org, refuses non-beta
  (cannot bypass off, cannot target another org, cannot change global state)

Clients: commerce deposit/transactions/subscriptionSummary; iam getUserRaw/updateUserRaw (caller-cred replay).
Tests: analytics math (retention/churn/growth/spend), cockpit handlers (credit deposit+audit, suspend+audit,
no-secret-leak, revenue), enablement (tri-state, opt-in-refuses-non-beta, admin-only, full flow, caller-scope).
All green + -race clean; cmd/cloud builds.
2026-07-03 09:40:29 -07:00
4db0666537 feat(provisioning): TRUE multitenancy — dedicated per-tenant DB instances (#88)
datastore (ClickHouse) + docdb (FerretDB) were honest-gated (unavailableKinds)
because a SHARED backend can't scope a per-tenant role. Replace the gate with a
DEDICATED-instance strategy: each create launches the org's OWN instance via an
operator Datastore CR + admin Secret in tenant-<org> (derived from the VALIDATED
org, never a request field). Isolation is BY INSTANCE — a cross-tenant grant is
impossible, there being one tenant on the instance — which un-gates both kinds.

- dedicated.go: engine table (image/ports/admin-env/DSN per kind), instanceName
  (<prefix>-<orgHash10>-<name>, DNS-1123), the k8s orchestrator (ensure tenant
  ns + RBAC wait, apply/observe/delete the Datastore CR + admin Secret + reap the
  retained PVC) behind an interface a fake stands in for, createDedicated,
  reconcileDedicated (provisioning->ready off the operator's status.phase), and
  dropDedicated.
- Billing (first-class): a provision debit carrying the size dimension
  (Model=<kind>:<size>) lands on the CALLER's org via the ONE commerce meter, and
  a recurring GB-day footprint sweep charges every running instance's own org —
  the reserved hook, now unblocked by the instance's declared size. Drop removes
  the row, stopping the meter, and reaps the PVC so no storage leaks.
- unavailableKinds now empty (mechanism kept); shared datastore/docdb
  provisioners deleted (one way only); the 5 shared kinds untouched.
- Datastore CR (not DocDB) is used because only its controller writes
  status.phase, the readiness signal; type forced per engine.

Tests: hermetic dedicated suite (fake orch + mock commerce) proves CR/Secret
shape, two-org isolation, ready reconcile, drop+PVC reap, and per-org billing
attribution; a build-tagged livecluster test proves the whole path against the
real operator. go build ./cmd/cloud + go test ./clients/provisioning/... green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-03 09:37:36 -07:00
hanzo-devandGitHub 92659f9616 feat(console): port waitlist/embed-status/topup routes to /v1/console/* (task #41) (#87)
Completes the console subsystem's standalone-route port for the True 1-binary
FE. The three remaining console2 Next server routes that do REAL server work
(not vanishing BFF proxies) now terminate natively in the unified binary, so
console2 can drop them from its static export:

  POST /v1/console/waitlist       waitlist.go — session-gated join to the Base
                                  waitlist plugin; the recorded email is BOUND to
                                  the gateway-verified X-User-Email (a signed-in
                                  user can't enroll a third party), honest 501
                                  when WAITLIST_URL is unset.
  GET  /v1/console/embed-status   embed.go — server-authoritative entitlement
                                  (owning brand org / global admin only) + a
                                  time-boxed reachability probe. SSRF-free: the
                                  target is <app>.<brand-domain> for the FIXED
                                  deployment brand (deps.Brand) — no client host
                                  in the target at all.
  POST /v1/console/topup/wallet   topup.go — verify an HUSD transfer on-chain
                                  (plain eth JSON-RPC, no EVM dep) and credit the
                                  VALIDATED caller's own org for the ON-CHAIN
                                  amount via the S2S commerce billing API. IDOR-
                                  safe (ignores any client userId); honest 501
                                  greenfield gate until HUSD is deployed.

All three resolve the caller from the VALIDATED principal only (same trust
boundary as keys/onboard); a forged X-User-Id/X-Org-Id is refused. docs is a
pure host->URL redirect with no server work, so it stays client-side in console2
(no handler here). Registered in the ONE routes() place; full unit coverage
(fake IAM/waitlist/RPC/commerce), go build ./... clean, binary boot-proven to
serve the console SPA at / with every /v1/console/* route resolving (403/501/503,
never 404).
2026-07-03 09:30:26 -07:00
zandGitHub f0152aecbb Merge pull request #86 from hanzoai/chore/zip-v1.2.1-431
chore(deps): zip v1.2.1 — activate the api.hanzo.ai 431 fix
2026-07-03 03:10:50 -07:00
hanzo-dev 2d9ffd3b7c chore(deps): zip v1.2.0->v1.2.1 — HTTP transport honors ReadBufferSize (activates the 431 fix)
v1.786.33 set zip.Config.ReadBufferSize=32768 (GATEWAY_READ_BUFFER_SIZE) but
zip v1.2.0's HTTP transport built a bare fasthttp.Server and dropped it — the
edge still 431'd at 4 KiB. zip v1.2.1 propagates the App Config onto the
transport's fasthttp.Server, so the 32 KiB header ceiling now takes effect on
cloud:8000 (the api.hanzo.ai/v1/* backend).
2026-07-03 03:10:42 -07:00
3eff9b9521 test(crm): lock /v1/crm/summary live-count immediacy (no lagging rollup) (#85)
The console E2E saw /v1/crm/summary miss a just-created record. Root cause was a
stale/eventually-consistent read; the current handler already counts LIVE
(s.Counts -> SELECT COUNT(*) per table on the same store the writes hit), so a
create/delete is reflected with ZERO lag — verified live (create company ->
summary companies +1 immediately).

Add TestSummaryReflectsCreateImmediately: create -> Counts shows +1, delete ->
Counts shows -1, all in one synchronous flow. This guards against any regression
to a materialized/async rollup. No production code change needed — the fix is the
regression lock.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-03 03:10:29 -07:00
46c3f7fc13 fix(analytics): honest 503 on ClickHouse timeout; fix(provisioning): honest-gate datastore+docdb (no cross-tenant role) (#84)
Gap 2 (analytics 502 on ClickHouse i/o timeout): /v1/analytics/{overview,timeseries,top}
returned a raw 502 when a DatastoreQuery hit a connectivity failure even though
requireDatastore()'s not-connected path already 503s. warehouseErr() now maps a
transport/connectivity error -> 503 'warehouse unavailable' (retryable); a REACHABLE
warehouse that rejected the query (bad SQL/protocol) stays 502. Table-driven test.

Gap 3 (docdb/datastore provisioning) — SAFETY REWORK (replaces the earlier
readWriteAnyDatabase change, which was cluster-wide = a cross-tenant hole):
- datastore + docdb are honest-GATED (unavailableKinds -> 503 'not yet available',
  refused BEFORE billing or any backend write) because their backends cannot mint a
  per-tenant-SAFE credential:
    * datastore (ClickHouse): no grant-capable per-tenant admin (GRANT ALL -> Code 497);
      unblocking is backend-side (StatefulSet grant-capable admin).
    * docdb (FerretDB/DocumentDB): engine implements ONLY cluster-wide roles
      (clusterAdmin -> Postgres SUPERUSER, readWriteAnyDatabase) — no per-db role.
- docdbProvisioner.Create keeps requesting the CORRECT per-db 'readWrite' role (the
  tenant-safe target); when FerretDB supports it, drop the gate and it works as-is.
- Tests: gated kinds -> 503 with the provisioner never run; the 5 guaranteed kinds
  (sql/vector/kv/search/s3) are asserted NOT gated.

Bar met: 5/7 data kinds fully work; datastore+docdb show an honest 'coming soon',
never a security hole.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-03 03:10:25 -07:00
zandGitHub a033ab2d14 Merge pull request #83 from hanzoai/fix/edge-read-buffer-431
fix(edge): raise fasthttp header buffer to 32 KiB (fix api.hanzo.ai 431 on multi-domain SSO)
2026-07-03 02:43:20 -07:00
hanzo-dev 6d4f08b29e fix(edge): raise fasthttp header buffer to 32 KiB so multi-domain SSO sessions don't 431
The public HTTP edge (zip/fiber) uses fasthttp's default 4 KiB per-conn
read buffer, which caps total request-header size and returns 431 (Request
Header Fields Too Large) above it. Once an admin-guard Domain=.hanzo.ai SSO
cookie is set on every subdomain, a browser's request headers cross ~4 KiB
and every request to api.hanzo.ai/v1/* (gateway -> cloud passthrough) 431s.

Raise the edge ceiling to a sane 32 KiB (nginx large_client_header_buffers
parity) via zip.Config.ReadBufferSize, env GATEWAY_READ_BUFFER_SIZE (shared
with the gateway edge so both trust boundaries agree on ONE value; tunable
down if the per-conn memory budget demands). Internal zip services keep the
4 KiB framework default — only the browser-facing edge opts up.

Repro (pre-fix): POST cloud:8000/v1/agents with a 9 KiB Cookie -> 431
Server: fasthttp. Post-fix: same request -> 403 (auth), no 431.
2026-07-03 02:42:54 -07:00
zandGitHub 6520338886 Merge pull request #82 from hanzoai/fix/agents-id-name-resolve
fix(agents): make a created agent gettable AND runnable by the returned id
2026-07-03 01:53:45 -07:00
hanzo-dev 66707fbed5 fix(agents): resolve /v1/agents/:ref by id OR name so created agents are runnable
create and list return an agent's public id (agent_<hex>), but get/run/update/
delete/runs resolved the URL path segment ONLY against the name column — so a
client that used the id create returned got 404 "agent not found". A created
agent was listed but neither gettable nor runnable by the identifier the API
handed back.

One-way fix: Store.Resolve(org, ref) matches either the public id or the
org-unique name (id wins on the astronomically-unlikely in-org collision),
org-scoped fail-closed so a cross-tenant ref is 404, never a leak. Every
path-addressed handler (get/update/delete/run/runs) resolves through it and keys
every downstream store op on the resolved a.Name. Route param :name -> :ref to
say what it accepts. The run path's validated-principal gate and single
product=agent debit are unchanged.

Tests (go test -race): Resolve by id and by name return the SAME agent; full
create -> get-by-returned-id -> run-by-returned-id all 200 the same agent with
real output; cross-org ref denied 404; a run addressed by the returned id meters
exactly once (product=agent).
2026-07-03 01:52:54 -07:00
zandGitHub 6562cfb49e Merge pull request #81 from hanzoai/feat/standalone-embed-real-console
feat(webui): standalone binary embeds the REAL console (build:embed pipeline)
2026-07-03 01:20:34 -07:00
hanzo-dev 5a4f199ff8 feat(webui): make the standalone binary embed the REAL console (build:embed)
The 1-binary console (HIP-0106) shipped as a 3-file STUB: nothing ran
console2's static export before `go build`/the image build, so `//go:embed
all:webui/dist` baked only the fallback shell.

- `make webui` (new): runs hanzoai/console2 `npm run build:embed` and overlays
  the static export into webui/dist so a plain `go build` embeds the full
  @hanzo/gui console. `make build-standalone` = webui → build. CONSOLE2_DIR
  points at a console2 checkout (default ../console2).
- Dockerfile console stage + webui.go + the stub shell: docs corrected — the
  pipeline is `build:embed` (a static export), not `next build`; the export now
  prerenders clean (console2 build-embed.mjs neutralizes the root layout's
  request-time headers() read), so the image embeds the real console instead of
  silently degrading to the shell. Bumped the export heap to 8192 for headroom.

Verified: build:embed → webui/dist (index.html 368 KB, /_next/static assets) →
CGO_ENABLED=0 go build ./cmd/cloud → the running binary serves the real console
at / (200, references /_next/, not the stub), the SPA shell for deep links
(/orgs), fingerprinted assets immutable-cached, and the /v1 API on the SAME
origin ({"service":"base","status":"ok"}); unmatched /v1/* is a real 404, not
HTML. webui_test.go (7 tests) green against the real bundle.
2026-07-03 01:16:17 -07:00
hanzo-devandGitHub d7c55382ea chore(deps): bump ai v1.790.4→v1.790.5 — provider-admin API + path-normalization hardening (#80) 2026-07-03 01:08:29 -07:00
hanzo-devandGitHub 3792398b0e chore(deps): bump hanzoai/ai v1.790.3→v1.790.4 — text-to-video (/v1/videos/generations) + canonical /v1/crawl (both red-cleared) (#79) 2026-07-03 00:44:54 -07:00
hanzo-dev 1937995f6b feat(visor): /v1/compute/{regions,sizes} — compute catalog on the cloud path (org-gated DRY passthrough; closes console compute-catalog 404, drops /vm proxy need) 2026-07-03 00:31:43 -07:00
hanzo-dev 5307fb4a7d Merge remote-tracking branch 'origin/main' into feat/framework-doctype-engine 2026-07-03 00:21:50 -07:00
hanzo-dev a9669e8fd0 fix(framework): atomic owner-seed — 'exactly one' System Manager under concurrency (Red LOW)
Red measured 3-6 System Managers seeded when concurrent role-less members first
administered a fresh org: managerOnly did a check-then-insert (OrgHasRoles then
AssignRole) with a TOCTOU window. Fix: store.SeedOwnerIfUnowned is a SINGLE
conditional INSERT ... SELECT ... WHERE NOT EXISTS(SELECT 1 FROM fw_roles WHERE
org=?), so the unowned-check and the insert are one atomic statement — exactly
one concurrent first-caller's row lands. RowsAffected==1 => this caller is the
seeded owner; ==0 => re-resolve (a concurrent grant may have made them a
manager) else 403. No UNIQUE index (multiple SMs are legit later via AssignRole;
only the AUTO first-seed must be singular). Removed the now-dead OrgHasRoles.

Test: TestAtomicOwnerSeed — 8 concurrent role-less first-callers → exactly 1
seeded winner + exactly 1 System Manager row. 22 tests total, race-clean.
2026-07-03 00:21:42 -07:00
hanzo-devandGitHub 988a76cdbc chore(deps): bump hanzoai/ai v1.790.2→v1.790.3 — Great-Audit security fixes (F1 unauth-admin RAG/scrape, F-sk zero-billing, F4 401s) (#78) 2026-07-03 00:06:18 -07:00
hanzo-devandGitHub 711bf591f0 fix(security): gate /v1/websearch/search fail-closed (Great-Audit F2) (#77)
searchGuard treated the searxng X-API-Key as OPTIONAL — a MISSING key passed —
so GET /v1/websearch/search was an open proxy to the Hanzo-operated metasearch
instance (unauthenticated request-forgery + cost surface). Its scrape sibling
(scrapeHandler) already fails closed; this brings search to parity:
  - key unset         → 503 (surface not configured, never open-to-all)
  - X-API-Key missing → 401 (constant-time compare of "" vs want fails)
  - X-API-Key mismatch→ 401

Safe for the real caller: the LibreChat searxng client sends the configured
searxngApiKey (universe chat configmap wires searxngApiKey=${WEBSEARCH_API_KEY})
as X-API-Key, so only anonymous callers are turned away.

Tests: TestSearchMissingKeyRejected (was ...Allowed) → 401; new
TestSearchUnsetKeyFailsClosed → 503; TestSearchProxyRewritesToSearchPath and
TestMountRoutesThroughRouter now present the key. go build/vet/test green.
2026-07-03 00:04:28 -07:00
hanzo-dev 32f31416be Merge remote-tracking branch 'origin/main' into feat/framework-doctype-engine 2026-07-03 00:03:02 -07:00
hanzo-dev 2207cfeb70 fix(framework): secure-by-default perms + Single submit-immutability (Red LOW-1/LOW-2)
LOW-1 — Single submit-immutability: updateDocument/createDocument for a Single
now route through writeSingle, which enforces the SAME draft-only guard as the
non-Single path (a submitted/cancelled Single → 409, not a silent mutation) and
preserves a redacted Password across an unchanged update.

LOW-2 — secure-by-default permissions (no open-to-all footgun):
- permission.can() is now DEFAULT-CLOSED: removed the 'empty perms => open to
  every org member' branch. A permless doctype is manager-only; a role-less
  member is denied.
- DocType.normalize() seeds a System Manager perm at define time, so a stored
  doctype is never silently permless (explicit in UI + audit).
- Owner seeding moved from resolveAccess (any member is SM until a role exists)
  to managerOnly as trust-on-first-use: the FIRST validated principal to
  administer an org with no roles becomes its persisted System Manager (the
  owner) — exactly one member, deterministically, never cross-tenant.

Tests: +2 (TestSingleSubmitImmutability, TestPermlessDefaultClosed); 21 total
race-clean. go build ./... CGO=1 & =0 green, vet + gofmt clean. Fixed binary
boot-verified (framework health 200, gate 403, forge 403).
2026-07-02 23:49:37 -07:00
zeekayandhanzo-dev 0dc29a7894 fix(deps): luxfi/age@v1.5.0 go.sum = canonical sumdb hash (fixes container SECURITY ERROR)
The prior fix re-fetched luxfi/age with checksum-checking off, which recorded the
DIRECT-vcs hash. luxfi/age@v1.5.0 was force-re-tagged, so the direct tree hash
differs from the immutable proxy zip hash — the container build (GOPROXY=proxy +
GOSUMDB=sum.golang.org, GOPRIVATE dropped for luxfi/* on purpose) verifies against
the sumdb and hit "checksum mismatch / SECURITY ERROR" on `go mod download`.
Replaced the h1: zip hash with the canonical value from
sum.golang.org/lookup/github.com/luxfi/age@v1.5.0 (the /go.mod hash already
matched). Now go.sum == what the proxy+sumdb serve → container verification passes.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-02 23:38:46 -07:00
zeekayandhanzo-dev 3a51c88fe6 fix(docker): console2 build:embed failure degrades to fallback shell (non-fatal)
The console-embed stage's contract is "a missing static target is a degrade, not
an error" — but the guard only handled the target being ABSENT. When console2
exposes build:embed AND it CRASHES (currently: /signin Server-Components
prerender error kills `next build`), the `&&` chain failed the whole cloud image,
so a frontend prerender bug took down the entire Go backend build (release runs
for projectsvc S3 fix + kms refactors all failed here, not on Go).

Complete the stated contract: wrap build:embed so a build FAILURE also degrades to
the committed fallback shell (/out stays empty → Go embeds webui/dist/index.html).
The standalone console2 Deployment is the primary console; this embed is a
same-origin convenience and must never gate the backend image.

(console2 /signin static-export prerender crash tracked separately for the
console track — this makes cloud CI robust to it either way.)

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-02 23:24:43 -07:00
zeekayandhanzo-dev cca56eb945 fix(deps): unblock cloud build — versioned replace for phantom go-sqlite3 + luxfi/age checksum
Two dep-rot issues blocking the cloud (Go backend) build:
1. A transitive dep requires the non-existent mattn/go-sqlite3 v2.0.3+incompatible.
   The unversioned replace didn't stop Go reading v2.0.3's go.mod during graph
   load. Fixed with a VERSIONED replace (v2.0.3+incompatible => v1.14.16, the last
   real go-sqlite3, drop-in package sqlite3). cloud's primary sqlite is
   modernc.org/sqlite (pure-Go); hanzoai/sqlite (encrypted, package `sqlite`) is a
   separate driver, adopting it is a real migration not this phantom fix.
2. luxfi/age@v1.5.0 go.sum checksum mismatch → removed stale lines + re-fetched.

go build ./internal/org/ (the sqlite consumer) now clean; module graph resolves.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-02 23:16:17 -07:00
hanzo-dev d14dfc3a3b feat(framework): native-Go DocType engine — the foundation that retires Frappe
The Hanzo Framework: Frappe's DocType/metadata core rebuilt native in Go on
Base/SQLite, mounted at /v1/framework/* (subsystem order 129). ONE engine +
ONE generic UI renders every business app — CMS content-types, ERPNext
DocTypes, Helpdesk all become just DocTypes on this engine. No Frappe/Python
runtime dependency; the engine is pure Go.

- DocType registry: define/list/get/replace/delete metadata per-org
- Generic metadata-driven document CRUD with ?filters=/fields=/order_by=/limit=
- Fieldtypes: Data/Int/Float/Currency/Check/Date/Datetime/Text/SmallText/
  LongText/Select/Link/Table/Attach/JSON/Password (all validated)
- Naming: hash / field: / prompt / series patterns (INV-.YYYY.-.#####)
- Link relations (in-org ref check + fetch_from), Table child rows
- docstatus 0/1/2 with submit/cancel for submittable doctypes
- Per-org permissions (DocType perms by role) + per-org role store
- Go lifecycle hook interface (before_insert/before_save/after_save/
  on_submit/on_cancel/on_trash) — gpython/goja runner is a later add to the
  SAME interface
- Password fields: argon2id hash on write, redact on read (fail-secure)

Security: org derived ONCE via clients/principal.Tenant (validated principal
only; forged X-Org-Id refused 403). Every table + query is org-scoped. 19
tests (race-clean) prove cross-org isolation, forged-principal refusal,
permission enforcement, field-type validation, and the docstatus lifecycle.
Boot-verified locally (health 200, doctypes 403, forge refused).
2026-07-02 23:16:10 -07:00
hanzo-dev efdbd118e3 refactor(kms): filenames match packages — kms/kms.go + kmssvc/kmssvc.go 2026-07-02 23:03:41 -07:00
hanzo-dev 16df83233b refactor(kms): drop the kmsembed compound — core is clients/kms, HTTP-mount subsystem is clients/kmssvc
The embedded luxfi/kms core (cloud/types-only leaf, built by build.go before the
app exists to break the import cycle) is now just 'kms'; the Fiber /v1/kms/* mount
subsystem (imports cloud) is 'kmssvc' (its existing internal name). One clean name
each, no unnecessary compound. build+vet+tests green.
2026-07-02 22:54:02 -07:00
zeekayandhanzo-dev da078d967c fix(projectsvc): SeaweedFS-compatible public-read bucket policy
publicReadPolicy used Principal {"AWS":["*"]} + array Resource, which
SeaweedFS's S3 policy engine rejects with 'Policy has invalid resource' —
aborting ensureBucket (SetBucketPolicy) BEFORE any files upload, so every
projectsvc deploy failed ('object storage'/'invalid resource') and no site
was ever served. Use scalar Principal "*" + scalar Resource, which SeaweedFS
accepts and is equally valid on AWS S3 / MinIO. Verified: mc anonymous
set-json with this exact shape succeeds against the s3.hanzo.ai SeaweedFS
gateway; a site uploaded to the now-public hanzo-sites bucket serves 200 at
https://s3.hanzo.ai/hanzo-sites/<org>/<slug>/index.html.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-02 22:52:59 -07:00
hanzo-devandGitHub bca1e24a11 fix(websearch): decode Crawl4AI object-shaped markdown (scrape returned empty) (#72)
Found by testing the ACTUALLY-deployed hanzoai/crawl:0.0.1 (= Crawl4AI
0.8.6) against clients/websearch's crawl adapter: 0.8.x/0.9.x return the
/crawl result's `markdown` as an OBJECT
{raw_markdown, fit_markdown, markdown_with_citations, ...}, and signal the
batch with a boolean `success` (no `status`). crawlResult.Markdown was
typed `string`, so json.Decode errored on the object form → crawl()
returned an error → EVERY scrape returned {success:false} with empty
content. hanzo.chat Web Search's scrape half was therefore dead even once
crawl is running.

Fix: markdownField.UnmarshalJSON accepts either a bare string OR the
object (preferring the cleaned fit_markdown, raw_markdown fallback); the
crawlResponse envelope now also accepts boolean `success` alongside the
legacy `status`. Neither envelope field is required — Results[0].Success
is authoritative.

Tests (real 0.8.6 response shape):
  TestScrapeHandlesCrawl4AIObjectMarkdown — object markdown + bool success
    → success:true, returns fit_markdown (was: {success:false}).
  TestMarkdownFieldAcceptsBareString — bare-string form still works.
  All 12 clients/websearch tests pass; go build + go vet clean.

Contract verified live: crawl4ai 0.8.6 POST /crawl {urls:[...]} returns
synchronously (no task_id polling) with url/markdown/success/metadata —
matches the adapter otherwise.
2026-07-02 22:30:15 -07:00
zandGitHub bf77781a6d Merge pull request #76 from hanzoai/feat/paas-domains
feat(platform): customer domains — self-serve subtree hosts + verified BYO custom domains
2026-07-02 22:01:40 -07:00
hanzo-devandGitHub 8422ee1a88 feat(platform): live deployment logs — stream real build + app pod logs (#75)
deploymentLogs returned only the recorded timeline + a Job reference, ending
with a '(live BuildKit Job logs stream in phase 2)' placeholder. This closes
that phase-2 gap: it streams the ACTUAL pod logs from the cluster — the BuildKit
Job's pod while a git build runs, and the running app's pod once deployed — so
the console's per-deployment Logs pane shows real output, operator-consistent.

- logs.go: buildLogs (job-name=<jobName> pod in the build ns), appLogs
  (app.kubernetes.io/instance=<slug> pod in tenant-<org>), one podLogsBySelector
  path (newest pod, tail-bounded 400 lines, byte-capped 256 KiB keeping the tail,
  8s time-boxed). Every read is org-scoped and time-boxed.
- k8s.go: add a typed kubernetes.Interface clientset (from the SAME rest.Config)
  held ONLY for the Pods().GetLogs subresource the dynamic client cannot express;
  nil-safe — a construct failure leaves logs degrading to the timeline and never
  disables the CR control plane (all on dyn).
- deploy.go: deploymentLogs now appends real build + app logs and stamps a
   (build|app|none) so the console can label the pane. HONEST DEGRADE:
  an unreachable cluster / absent pod yields the recorded timeline + a stated
  'not available' note — source stays reflecting real streamed content, never a
  fabrication.
- 8 tests over a fake typed clientset: newest-pod selection, tenant-namespace
  scoping (acme never reads victim's pod), no-pod/no-clientset honest degrade,
  and the handler surfacing live build logs (source=build) vs degrading
  (source=none) — asserting the phase-2 placeholder is gone.
2026-07-02 21:52:39 -07:00
hanzo-devandGitHub 43f51cdb04 feat(console): native /v1/console/{keys,onboard} — port console2's standalone IAM server routes (#74)
The console can be a static export only once its two NON-proxy Next server
routes (app/keys, app/onboard) — which mint/revoke the user's hk- Cloud API
key and create the user's org as the confidential hanzo-console IAM client —
have a native home. Port them to a clients/console subsystem mounted at
/v1/console/* in the one binary (task #41, True 1-binary FE): the embedded SPA
calls /v1/console/* on its own origin, and the last stateful Node handlers go.

- clients/console/iam.go: confidential-client (client_secret_basic) IAM caller
  for mint/revoke/get the hk- key + create/read/update an org. Honest 501 when
  IAM_MINT_CLIENT_ID/SECRET are unset (mirrors identity.ts mintConfigured()).
- clients/console/console.go: /v1/console/{keys(GET/POST/DELETE),onboard(POST),
  health}. Every route requires a VALIDATED principal; the IAM id is DERIVED as
  <owner>/<name> from the gateway-minted X-User-Id/X-Org-Id, never a request
  value — a caller can only ever act on their OWN key/org (red-bar structural).
- clients/console/onboarding.go: faithful Go port of console2 onboarding.ts —
  pure slug + reserved-name policy (admin/built-in/app + hanzo/lux/zoo/pars).
- Registered as consolesvc (order 122) so /v1/consolesvc/health does not shadow
  the real fail-closed /v1/console/health probe.
- 16 tests: unauth 403 (forged X-Org-Id refused, IAM never touched), mint/get/
  revoke scoped to the derived id + show-once + no secret leak on GET, 501/502
  honesty, onboard first-run(create+move)/additional(create-only)/reserved 400/
  taken 409/personal auto-suffix, + pure-policy unit tests.
2026-07-02 21:52:11 -07:00
hanzo-dev bee976a64d feat(platform): customer domains — self-serve subtree hosts + verified BYO custom domains
Close the two PaaS domain gaps so a customer can put their app on their own
domain, operator-native, from the console.

- Seed a canonical default host <slug>.<org>.<sitesHost> on app create, so
  every app has a working HTTPS URL the moment it deploys (operator issues the
  cert). Never removable.
- New /v1/platform/.../domains surface (list/add/verify/remove). An org-subtree
  host is active immediately; a BYO custom host (yourco.com) is claimed PENDING
  and returns the exact DNS records to publish (TXT ownership token at
  _hanzo-challenge.<host> + CNAME to the app host).
- Verify resolves DNS: a matching TXT token proves control (DNS-01 model), then
  the host is rendered into the app's operator Service CR ingress via applyIngress
  (cert-manager TLS comes for free). Honest still-pending on not-yet, never fake.
- platform_domains table: host PRIMARY KEY = global uniqueness (one org per host,
  like site_hosts); pending→verified lifecycle. Cascade-deleted with app/project.
- validateOrgDomains extended: a non-subtree host renders ONLY when this org owns
  a VERIFIED claim; unverified/foreign/apex hosts still refused (RED hardening kept).
- ingressSpec extracted (one TLS shape shared by serviceCR + applyIngress);
  observeDomains surfaces operator status.endpoints/phase for honest live state.
- Tests: verified-custom accept + pending/foreign refuse; full add→verify→remove
  HTTP flow with fake DNS; global uniqueness (two orgs/two apps); apex refusal;
  default-host seeding; CR ingress render. go build + go test green.
2026-07-02 21:48:04 -07:00
hanzo-devandGitHub 8f858e0d0b chore(deps): bump hanzoai/ai v1.790.1→v1.790.2 — full DO model lineup + file-scoped RAG + SD3.5 image (atop diffusion) (#73) 2026-07-02 21:26:05 -07:00
hanzo-dev d0a1a507da fix(websearch): decode Crawl4AI object-shaped markdown (scrape returned empty)
Found by testing the ACTUALLY-deployed hanzoai/crawl:0.0.1 (= Crawl4AI
0.8.6) against clients/websearch's crawl adapter: 0.8.x/0.9.x return the
/crawl result's `markdown` as an OBJECT
{raw_markdown, fit_markdown, markdown_with_citations, ...}, and signal the
batch with a boolean `success` (no `status`). crawlResult.Markdown was
typed `string`, so json.Decode errored on the object form → crawl()
returned an error → EVERY scrape returned {success:false} with empty
content. hanzo.chat Web Search's scrape half was therefore dead even once
crawl is running.

Fix: markdownField.UnmarshalJSON accepts either a bare string OR the
object (preferring the cleaned fit_markdown, raw_markdown fallback); the
crawlResponse envelope now also accepts boolean `success` alongside the
legacy `status`. Neither envelope field is required — Results[0].Success
is authoritative.

Tests (real 0.8.6 response shape):
  TestScrapeHandlesCrawl4AIObjectMarkdown — object markdown + bool success
    → success:true, returns fit_markdown (was: {success:false}).
  TestMarkdownFieldAcceptsBareString — bare-string form still works.
  All 12 clients/websearch tests pass; go build + go vet clean.

Contract verified live: crawl4ai 0.8.6 POST /crawl {urls:[...]} returns
synchronously (no task_id polling) with url/markdown/success/metadata —
matches the adapter otherwise.
2026-07-02 21:21:25 -07:00
hanzo-devandGitHub 05d0fb5d11 fix(templates): resolve 38 broken screenshot preview URLs in catalog (#71)
Map catalog preview URLs to screenshots that actually exist in gallery.
All 69 templates now have resolvable preview images at gallery.hanzo.ai.
2026-07-02 21:17:21 -07:00
hanzo-dev 9708db5c45 feat(graph): chain-data cloud client — /v1/indexers + /v1/oracles
Front the Lux chain-data plane over HTTP so the console's Indexer and
Oracles pages read REAL chain state from api.hanzo.ai/v1/* instead of
rendering "not connected":

- GET /v1/indexers  -> luxfi/indexer explorer REST (/health + latest
  block): per-network chain/network/height/health. lag honestly omitted
  (the indexer REST exposes indexed height, not the chain HEAD).
- GET /v1/oracles   -> luxfi/graph GraphQL priceFeeds (O-Chain PriceFeed
  registry): real on-chain price feeds; honest-empty when none.

Principal-gated (403 without a validated IAM principal); brand-scoped
(each brand's cloud is wired to its own indexer/graph, a ledger is public
within a brand). Honest 502 on unreachable upstream, never a fabricated
row. Mirrors clients/visor + clients/zt structure; interface-seam tests
against a fake upstream. Registered order 135.
2026-07-02 20:27:25 -07:00
hanzo-devandGitHub ec8691d6ac feat(o11y): emit OTel traces via go.opentelemetry.io/otel (service.name=hanzo-cloud) (#67)
Env-gated on OTEL_EXPORTER_OTLP_ENDPOINT; non-fatal; clean no-op when unset (safe to ship before the collector is live). Installs the global tracer provider with a service.name resource so the console Monitoring tab filters this product. Mirrors ai/object/telemetry.go. Traces-only; metrics/logs are a tracked follow-up.
2026-07-02 20:18:47 -07:00
hanzo-devandGitHub 2625817e68 chore(deps): bump hanzoai/ai v1.789.1→v1.790.1 — wire /v1/images/generations diffusion (zen3-image → do-ai fal) (#70) 2026-07-02 19:46:27 -07:00
zandGitHub 32159c4d2a Merge pull request #64 from hanzoai/feat/sites-subdomain-router
feat(sites): wildcard-subdomain site router over S3 + Cloudflare purge-on-redeploy
2026-07-02 18:14:28 -07:00
hanzo-dev 0a80a6cf71 fix(sites): RED — unified reserved-list enforced at create+bind+serve, stream not buffer, 405
RED review fixes on the sites router:

1) [HIGH] ONE reserved-subdomain source (clients/sites/reserved.go:
   baseReserved baked-in + operator SetReservedExtra, never subtractable),
   consulted at THREE points that can no longer drift: serve (siteSlug),
   project-create (createProject -> 400), and host-bind (Store.BindHost ->
   errReservedHost). site_hosts can now NEVER physically hold a reserved host,
   so a reserved subdomain never resolves even if the ingress regex drifts —
   the serve gate is a backstop, not the sole guard. Widened the set to app/
   auth/payment/brand labels (console, sites, internal, gateway, login, secure,
   account, signin, auth, pay, wallet, admin, brand terms, ...).

2) [MED DoS] Serve now STREAMS objects (Fiber SendStream, Content-Length from
   info.Size, fasthttp closes the reader) instead of io.ReadAll-buffering up to
   64 MiB per request on the unauthenticated edge — removes the OOM vector.
   Same for the 404.html path.

3) [LOW] Non-GET/HEAD on a site host → 405 + Allow: GET, HEAD.

Tests: IsReserved, reserved-host-never-serves backstop, 405, BindHost-rejects-
reserved (even with a forced project row), create-rejects-reserved-slug via the
real handler. All green; no regressions.
2026-07-02 18:09:27 -07:00
hanzo-dev 3bc0ca2ebc fix(platform): harden cold-start deploy flywheel (RED L2+L1+I2)
Close the two LOW follow-ups on the cold-start tenant-RBAC fix, plus confirm
the fresh-org fail-closed status. All on top of v1.786.23 (already SHIP).

L2 — git reconciler no longer treats a transient tenant-RBAC delay as TERMINAL
and no longer head-of-line-blocks other orgs. reconcileBuild now does ONE
non-blocking readiness probe (ensureTenantReady: create-namespace-if-absent +
single SelfSubjectAccessReview) instead of the synchronous image path's ~45s
in-line waitForTenantRBAC. If the operator's RoleBinding has not landed, the
deployment stays 'building' and re-drives on the next 10s tick — never a
permanent fail (there is no client to retry a git build) and never a 45s stall
of the shared sequential reconciler. Only the elapsed build deadline fails it
honestly. errTenantProvisioning from applyLive is also caught as transient
(defense in depth). Namespace-create is decomplected into one shared
ensureNamespaceExists; ensureNamespace (sync, blocking) and ensureTenantReady
(async, probing) compose it.

L1 — image deploy path gains a per-org in-flight-deploy cap (inflightGate,
maxConcurrentDeploys, default 8 via CLOUD_PLATFORM_MAX_CONCURRENT_DEPLOYS),
mirroring the git build cap. deployImage acquires a slot before applyLive's
~45s RBAC wait and releases on any return; over-cap is a retryable 429 refused
BEFORE recording an attempt. Bounds request-goroutine pile-up on a wedged
operator; per-org (one org's saturation never throttles another); fail-closed.

I2 — confirmed the truly-fresh-org path fails closed on 503 (RBAC pending),
NOT a raw 502: the namespace IS created (the trigger for the operator's
RoleBinding), then the bounded RBAC wait yields errTenantProvisioning -> 503.

Tests (all -race green): reconciler stays 'building' then goes live on a later
tick once RBAC lands (not 'failed'); over-cap image deploy -> 429 with per-org
isolation + slot-release re-admit; fresh-org deploy -> 503 with namespace
created + no Service CR + honest 'error' deployment recorded.
2026-07-02 18:06:45 -07:00
244b43b3d7 fix(admin): reconcile fleet revenue with commerce — X-Org-Id + bare-slug subject (#65)
The /v1/admin money panels (finance/orgs/overview) read $0 for every org despite
real balances (lux $10,000, maxpower $20,498) because the commerce client used
the wrong org selector on BOTH axes:

- commerce.go get(): sent X-IAM-Org-Id, which commerce does NOT read. Commerce
  EdgeAuth resolves the per-org billing namespace from the TRUSTED X-Org-Id header
  (trusted only with the COMMERCE_SERVICE_TOKEN bearer). X-IAM-Org-Id silently
  fell back to the default (COMMERCE_SERVICE_ORG) namespace.
- admin.go orgSubject(): keyed the wallet subject as "org/org"; commerce keys the
  per-org wallet under the BARE org slug (user=<org>) within the X-Org-Id namespace
  (the 2026-07 commerce durability rework, commerce >=1.46.8).

Either alone zeroed the reconciliation; both were present. The prior comments
encoded the wrong model ("commerce resolves from COMMERCE_SERVICE_ORG, header
advisory") — corrected to the verified contract.

Verified LIVE against commerce /v1/billing/{balance,usage-rollup}:
  user=lux      + X-Org-Id: lux      -> $10,000.00 (1,000,000c)
  user=maxpower + X-Org-Id: maxpower -> $20,498.13 (2,049,813c)
  user=lux/lux  OR X-IAM-Org-Id      -> $0 (the bug)

The fleet-wide /v1/costs COGS god-view is org-independent and correctly sends no
org (unchanged).

Regression guard: TestCommerce_ReconcilesWithXOrgIdBareSlug — a contract-accurate
fake commerce that returns money ONLY for X-Org-Id + bare-slug user; proven
red->green (fails on org/org, passes on the fix). Full admin suite green.

Co-authored-by: blue <blue@hanzo.ai>
2026-07-02 18:05:16 -07:00
zandGitHub e0e67e7fa3 Merge pull request #66 from hanzoai/feat/agent-sessions
feat(agents): live agent-session control plane (/v1/agents/sessions)
2026-07-02 17:41:13 -07:00
hanzo-dev 9d12d11a64 agents/sessions: clone SSE root filter (fix Ctx-recycle data race) + prove event-seq under concurrency
Red review of the live agent-session control plane.

FIX (MEDIUM, systems lens): sessionsStream retained root := c.Query("root")
verbatim. c.Query is a zero-copy view into the fasthttp request buffer, and the
SendStreamWriter loop OUTLIVES the handler (runs after the Ctx is recycled), so
the long-lived root filter raced a reused buffer — within-org stream-filter
corruption / UB. tenant() already clones org for this exact reason; clone root
the same way. Not cross-tenant (org is cloned + bus-filtered); fixes the race
and honors the file's own 'never touch Ctx after return' invariant.

TEST (vector #4): add TestSessionEventSeqConcurrent — 64 parallel AppendEvents
to one session must yield seqs exactly {1..N}, no gaps (no lost write) no dupes
(no raced MAX+1). Proves the single-conn + UNIQUE(session_id,seq) guarantee
under -race instead of only asserting it.
2026-07-02 17:38:16 -07:00
hanzo-dev 8ca58e924c feat(agents): live agent-session control plane (/v1/agents/sessions)
The canonical cloud registry every surface hangs off: live agent SESSIONS +
the subagent tree, streamed over ZAP, remote-controllable. This is the
view/control/stream layer; durable execution rides hanzoai/tasks, not a
bespoke scheduler.

Model + store (agents.db, same tenancy pattern as agents/runs):
- Session{id,agent,org,actor,status,parentSessionId,rootSessionId,title,
  startedAt,endedAt,taskWorkflowId,taskRunId,events[]}. Subagent tree =
  sessions linked by parentSessionId; the outer agent is the root, each
  spawned subagent a child, all sharing rootSessionId. Parent must exist
  in the SAME org (TOCTOU-checked in the write path) so a tree can never
  cross tenants. Per-session monotonic event Seq.

REST (org-scoped via principal.Tenant, fail-closed):
- POST /v1/agents/sessions            register (opt parentSessionId)
- GET  /v1/agents/sessions            list (filter root/parent/status)
- GET  /v1/agents/sessions/:id        detail + children + recent events
- GET  /v1/agents/sessions/:id/tree   full subagent-flow graph (1 query)
- PATCH /v1/agents/sessions/:id       status/title (terminal is monotonic)
- POST /v1/agents/sessions/:id/events append message/tool-call/spawn/log
- POST /v1/agents/sessions/:id/{pause,resume,stop,message}  control
  Routes register BEFORE /v1/agents/:name (Fiber matches in registration
  order); /stream precedes /:id for the same reason.

ZAP live stream:
- GET /v1/agents/sessions/stream (SSE) rides the ZAP machine transport
  natively (zip SendStreamWriter streams through ListenZAP — proven by
  zip stream_test). In-process bus is the single fan-out seam a direct
  ZAP push subscription attaches to. Org-filtered, non-blocking, laggard-
  drop; GET endpoints are the source of truth.

Durable execution = hanzoai/tasks (architecture alignment):
- Root session -> a tasks workflow; subagent -> child workflow (same
  rootSessionId). TaskController seam mirrors the tasks SDK Client
  (Signal/Cancel); control forwards to it when a session is task-backed,
  else records the command as a durable control event for stream-
  consuming surfaces. Default is the disabled (record-only) controller;
  the live client.Dial(TASKS_URL) plug-in point is marked in Mount.

Run integration (#5): the ONE runAgent path (HTTP + scheduler) opens a
root session per run (best-effort, never fails the run), so every run is
visible in the same registry.

Tests (real): store tree-linking + cross-tenant/dangling parent deny +
event seq/counts; HTTP tree assembly; cross-tenant read/tree/control/
append/parent deny; control authz (no validated principal -> 403) +
tasks forward (signal/cancel, forward-failure 502, record-only fallback);
event append/seq + status monotonicity; run-opens-session; bus fan-out/
org-filter/overrun/close. go build+vet+test clean; -race clean.
2026-07-02 17:38:16 -07:00
hanzo-dev fed3e40b12 feat(sites): wildcard-subdomain site router over S3 + Cloudflare purge-on-redeploy
Add clients/sites: a HOST-routed public site server that turns
<slug>.hanzo.app into the static site a project deployed to OUR S3
(<org>/<slug>/ in CLOUD_PROJECTS_BUCKET). Installed as the FIRST middleware
in the compose root, ahead of identity/billing, so a published site is a
public artifact served straight from S3 — never a tenant API call.

Tenant isolation (RED-focus): the org + S3 prefix come ONLY from the store
lookup keyed by the validated subdomain slug, never from the request path or
a client header. Object keys are rooted-clean (path.Clean under '/') so no
../ or encoded traversal can escape the <org>/<slug>/ prefix into another
project or org. A globally-unique site_hosts binding table makes a bare
subdomain resolve deterministically to exactly one tenant (project slugs are
only org-unique); binding is first-come and cannot be hijacked.

Cache: one canonical policy (sites.CacheControlFor) applied both when writing
objects at deploy and when serving them — HTML public,max-age=60,s-maxage=86400;
content-hashed assets immutable 1y; middle TTL otherwise; per-project
cacheControl override on the document TTL. Cloudflare purge-by-cache-tag
(site-<org>-<slug>) on redeploy AND delete; creds from KMS/env
(CF_API_TOKEN/CF_ZONE_ID), honest no-op when unset. Cache state (TTL +
lastPurgeAt) exposed on the project API.

Tests: traversal/cross-tenant isolation proof, host-routing + reserved-host
exclusions, first-come/no-hijack subdomain binding, CF purge client.
2026-07-02 17:29:17 -07:00
hanzo-dev 89a51599fd feat(zt): tenant-scoped networking surface fronting Hanzo Zero Trust
Add clients/zt — a thin, org-scoped facade over the Hanzo Zero Trust
controller's OpenZiti Edge Management API (/edge/management/v1), backing
the console's Networks, Service Mesh and Edge pages (which render
"not connected" today).

Surface (all org-scoped by the validated principal):
  GET /v1/networks[/:id]  the org's ZT overlay, projected from its edge-routers
  GET /v1/mesh/services   ZT edge services
  GET /v1/edge/nodes      ZT edge-routers + real online/disabled/offline status

- client.go: one HTTP path — Ziti password-auth (KMS-injected
  ZT_CLIENT_ID/ZT_CLIENT_SECRET, zt-session header), cached session with
  re-auth-and-retry-once on 401, generic {data,meta} pager, honest error
  mapping, TLS trust via ZT_CA_PEM. Fails closed 503 when unconfigured.
- types.go: ZT wire structs + console view structs + PURE mapping.
  Tenant isolation is the "org-<org>" role attribute (the ONE tenancy
  convention ZT expresses natively); list/get filter to the caller's org.
- zt.go: routes/handlers, registered as subsystem "zt" (order 134).
- http_test.go: fake controller (interface seam) — asserts 200, tenant
  isolation, shape, health mapping, 401 re-auth retry, fail-closed 503.

Honest-empty over fabrication throughout: no org tag -> invisible; no
routers -> no network; no metrics -> omitted (UI renders em dash).
2026-07-02 17:28:44 -07:00
zandGitHub a18ec0eac8 Merge pull request #63 from hanzoai/feat/visor-subsystem
feat(cloud): mount visor /v1/visor/* as a subsystem in the unified binary
2026-07-02 17:25:16 -07:00
hanzo-dev 8956fbde1d chore(cloud): pin visor v1.108.5 for the mounted subsystem 2026-07-02 17:24:25 -07:00
zandGitHub f38f57ab99 Merge pull request #62 from hanzoai/feat/admin-compute-endpoint
feat(admin): /v1/admin/compute — cross-tenant compute analytics from the datastore
2026-07-02 17:22:58 -07:00
hanzo-dev 18c1852215 feat(admin): /v1/admin/compute — cross-tenant compute analytics from the datastore
New global-admin read GET /v1/admin/compute powering the console Bots + Machines
operator boards. Aggregates hanzo.compute_usage(org, app, project, kind, event,
machine_id, size, price_cents, ts) grouped by (org, app, project, kind) over the
shared datastore client (aiobject.DatastoreQuery — the clients/analytics transport,
no second conn). `kind` is an OPEN LowCardinality spectrum (bot|machine|cluster|
nodepool|container|function|…) matched as a PLAIN STRING — ?kind= narrows to any
kind (Bots=bot, Machines=machine; future Clusters/Functions reuse this endpoint),
?org filters, ?range=24h|7d|30d bounds. Two-level roll-up: inner argMax(event,ts)
per machine -> outer counts machines, active (latest non-terminal), sum(price_cents),
max(ts). Honest-empty when the warehouse/table isn't wired yet (visor/commerce
emitter pending) — never a fabricated fleet. Global-admin only (s.guard); stays v1.x.x.
2026-07-02 17:15:02 -07:00
hanzo-dev c2f7003016 feat(visor): /v1/machines,/v1/gpus,/v1/clusters — compute unified into cloud via Visor
New clients/visor subsystem fronts Visor (the cloud OS at visor.hanzo.svc) and
serves the console's Machines/GPUs/Clusters pages as clean, tenant-scoped REST off
the unified cloud binary — replacing the god-mode /paas admin proxy that 501s.

Routes (every route org-scoped by the validated principal → Visor ?owner):
  GET/POST /v1/machines, GET/DELETE /v1/machines/:id   -> get-machines / machines/launch / delete-machine
  GET /v1/gpus (+ /v1/gpus/alerts)                     -> per-accelerator inventory derived from GPU machines
  GET /v1/clusters, node-pool create/scale/delete      -> get-node-pools / *-node-pool

View JSON mirrors the console normalizers exactly (visor.ts/compute.ts/platform.ts)
so the FE renders with no change. No fabrication: GPU rows are real accelerators of
real GPU machines, clusters are real node pools, and telemetry Visor lacks is
omitted (renders — not 0). Auth: KMS service credential (Basic) or forwarded bearer.

Tests: tenant scoping/isolation, machine/gpu/cluster shape, GPU slug derivation,
launch quote+real+delete. go build ./... + go vet + go test all green.
2026-07-02 17:08:17 -07:00
hanzo-dev 97cad6068c feat(platform): native console aggregates — /v1/{environments,pipelines,builds,releases}
The console Environments/Pipelines/Builds/Releases pages rendered "not
connected" because they call top-level REST that no cloud subsystem served.
Serve them natively from the platform control plane, DERIVED from the SAME
per-org project/app/deployment/build records (no new data model, no fabrication):

  - GET /v1/environments — distinct Application.Environment targets across the
    org's apps, each aggregating its apps (services), with a derived
    type/status. List-only: an environment is a scope on apps, not a record.
  - GET /v1/pipelines — one per app: its build/deploy config (repo|image) plus
    the status/timing of its latest deployment. List-only: a pipeline is an app.
  - GET /v1/builds — the REAL arcd BuildKit build records (platform_builds),
    joined to app repo + deployment commit. List-only: builds are triggered by
    the app deploy path (git source) — one trigger, not a duplicate here.
  - GET /v1/releases — deployments actually applied to the cluster
    (status deploying|live): a released image tag on an app/environment.

Every route is org-scoped through the same validated-principal gate (s.tenant →
requires c.User()); the response is the exact `{ "<plural>": [...] }` wrapper the
console FE normalizers read. Three org-wide store aggregates back them
(ListAllApplications / ListDeploymentsByOrg / ListBuildsByOrg), org the only
tenancy predicate. Real records or an honest empty — never fabricated history.

Tests: shape (200 + wrapper + derived fields), org isolation (second org sees
empty, no cross-tenant leak), forgeable-org refusal (no X-User-Id → 403).
2026-07-02 17:04:37 -07:00
hanzo-dev d1aa749779 feat(do): native DigitalOcean VPC + Load Balancer surface (/v1/vpcs, /v1/load-balancers)
Add clients/do: an org-scoped facade over digitalocean/godo's native VPCs
and LoadBalancers services, backing the console's VPC + Load Balancers pages
(which render 'not connected' today because nothing serves them).

- Routes: GET/POST /v1/vpcs, GET/DELETE /v1/vpcs/:id and the same for
  /v1/load-balancers. Real godo calls, honest empty/error states, never
  fabricated.
- Tenant isolation: DO is a single account, so a resource's physical DO name
  is 'o'<orgHash>-<friendly> via provisioning.BucketName (the SAME org-hash
  convention clients/s3 uses). List filters the account inventory to the
  caller's prefix; get/delete confirm prefix ownership before acting; a
  cross-tenant id reads 404 (existence-oracle guard).
- Fail-closed: absent DO_API_TOKEN every op is an honest 503.
- FE shape matches console2 VpcModule/LoadBalancerModule verbatim (vpcs[],
  loadBalancers[] with the exact field names).
- Registered as subsystem 'do' (order 123). godo v1.197.0 added to go.mod.
- Tests: per-org VPC + LB isolation, forge-path 403, fail-closed 503.
2026-07-02 17:02:11 -07:00
zandGitHub 322cf7ace4 Merge pull request #61 from hanzoai/fix/agents-ai-inference
fix(agents+platform): real /v1/agents/:name/run inference + tolerate async tenant-RBAC on first deploy
2026-07-02 16:43:35 -07:00
hanzo-dev 34966f18ee chore(deps): realign luxfi/age + luxfi/pq go.sum zip hashes (force-re-tag drift)
Same class as f238188: upstream force-re-tagged luxfi/age v1.5.0 and
luxfi/pq v1.0.3 (content moved, /go.mod hashes unchanged), so the committed
zip h1: sums no longer match the served bits and `go build ./...` fails
verification. These deps entered the graph via hanzoai/commerce/metering
v0.1.2 (the agents scheduler/billing). Re-record the current zip hashes.
2026-07-02 15:48:48 -07:00
hanzo-dev 7609c9981e agents: bind /v1/agents/metrics + /v1/agents/activity (unshadow from :name)
The console2 Agents dashboard calls two org-wide routes that were never
reachable: the bare /v1/agents/:name wildcard captured "metrics"/"activity"
as an agent name (Fiber matches in registration order), so both 404'd and the
dashboard rendered a permanent "not connected" state.

- Register the two static routes BEFORE :name so they win the match.
- GET /v1/agents/metrics?range=24H|7D|30D -> a per-agent invocations-over-time
  histogram bucketed from REAL agent_runs rows; the Resource Usage rollup is
  all-null because this store meters no CPU/mem/storage/cost (honest em-dash,
  never a fabricated trend). Shape mirrors console2 normalizeMetrics exactly
  ({range,series:[{key,points:[{t,v}]}],resource:{...}}).
- GET /v1/agents/activity -> org-wide recent-activity feed: each recorded run
  is an invoked/failed event, each agent's own create/update timestamps are
  created/updated events; merged newest-first, capped 50. Shape mirrors
  normalizeActivity ({activity:[{id,kind,agent,message,at}]}).
- Store: add RunsSince(org,since,limit) — org-wide runs across all agents,
  tenancy on the org column; powers both surfaces.
- Tests prove the surfaces are not shadowed (200, not 404), reflect only real
  runs, and stay org-isolated.
2026-07-02 15:45:44 -07:00
hanzo-dev d5fbf844f3 fix(agents): wire real inference so /v1/agents/:name/run executes
deps.AI was permanently nil under the default all-enabled config
(pickAIClient returned nil for cfg.Enabled("ai") and no in-process ai
subsystem ever filled it), so every agent run 503'd "inference is not
configured on this deployment" before the already-live per-org metering.

New AI client (clients/aihttp.go), two credential modes:
- AIHTTPAt: static-key OpenAI-compatible client (CLOUD_AI_API_KEY) — an
  operator/pre-provisioned-key override.
- AIHTTPM2M: durable default — mints+auto-refreshes an IAM client-
  credentials token from the binary's OWN identity (IAM_CLIENT_ID/SECRET)
  via x/oauth2/clientcredentials. No static key to rotate, no expiry cliff,
  no new secret to store. On Hanzo the identity resolves to
  admin/hanzo-cloud, which the gateway treats as balance-exempt, so cloud's
  per-org ResourceMeter stays the single revenue debit (no double-bill).

- build.go pickAIClient: static key -> M2M -> ZAP RPC -> fail-closed stub.
  Never returns nil (the live bug). Secret never logged.
- config.go: CLOUD_AI_BASE_URL (default https://api.hanzo.ai/v1),
  CLOUD_AI_API_KEY (optional), CLOUD_AI_DEFAULT_MODEL (default
  deepseek-v4-flash), AIAuthClientID/Secret from IAM_CLIENT_ID/SECRET.
- clients/aihttp_test.go: httptest OpenAI emulation — default-model
  substitution, content parse, 4xx/5xx mapping, empty-choices error, and
  M2M token mint+use+cache.

Composes with the live metering (Gate/MeterUsage) untouched. Model routing
is the gateway's job; empty model -> cheap default is the only cloud-side
fallback (no in-code model aliasing).
2026-07-02 15:34:54 -07:00
hanzo-dev 1be9f0b40d fix(admin/finance): RED — honest revenue.Configured + fleet-wide /v1/costs (no false margin)
Addresses Red MED-1/MED-2/INFO-3:
- MED-1: revenue.Configured now means the source was actually READ (listOrgs
  succeeded), not merely wired. A transient IAM failure → configured:false, never a
  fabricated zero that flips margin negative into a false 'burning' alarm. Per-org
  read failures mark the commerce source not-ok (partial), never presented as whole.
- MED-2: /v1/costs is a fleet-wide god-view — commerce resolves the namespace from
  COMMERCE_SERVICE_ORG, NOT from a request header (it never reads X-IAM-Org-Id).
  Dropped the no-op org arg from costs(); corrected the false 'resolves namespace
  from X-IAM-Org-Id' claims in commerceClient + orgSubject docs.
- Test: TestFinance_RevenueSourceDown_NoFabrication proves no fake revenue/margin
  when the IAM org list is unreadable while COGS still flows.
(INFO-3 false-green fix lands console-side in financeHealth.) 10 admin tests green.
2026-07-02 14:28:30 -07:00
hanzo-dev 062a0f46c0 feat(admin/finance): margin COGS from commerce /v1/costs — one vendor-COGS source
The finance board's cost side now CONSUMES commerce /v1/costs (the single
vendor-COGS source of truth: DigitalOcean compute + the LLM providers we resell)
instead of re-reading DigitalOcean's billing API to derive a DO-only cost. This
removes cloud's duplicate DO COGS read and gives the board the multi-vendor
per-vendor breakdown for free.

- commerceClient.costs() reads GET /v1/costs over the admin S2S service token
  (COMMERCE_SERVICE_TOKEN, no IAM user → commerce requireCostsAdmin M2M path).
- financeCost carries {configured,totalCents,vendors[],period}; margin cost is
  now the multi-vendor TotalCents, not DO month-to-date spend.
- DigitalOcean stays ONLY as the orthogonal promo-credit/runway treasury view
  (commerce does not track our prepaid credit); its MTD spend feeds runway alone.
- /v1/admin/finance shape is additive (cost.digitalocean preserved) and the
  global-admin guard is unchanged.
- tests: fake commerce now serves /v1/costs; margin = revenue - COGS; DO-off path
  proves COGS still flows from commerce (decoupled).
2026-07-02 13:51:46 -07:00
hanzo-dev 7c8be23f14 fix(platform): tolerate async tenant-RBAC on first deploy (cold-start race)
A brand-new tenant's namespace is created by ensureNamespace, but the operator's
tenant-RBAC controller projects cloud-api's `cloud-api-platform` RoleBinding
(get/create resourcequotas/limitranges/services.hanzo.ai in tenant-<org>)
ASYNCHRONOUSLY. The first-ever deploy raced ahead of that RoleBinding and failed
with `resourcequotas ... is forbidden`, self-healing only on a manual retry.

Gate ensureNamespace on a SelfSubjectAccessReview readiness poll
(waitForTenantRBAC): before touching the quota objects, ask the apiserver — as
cloud-api's OWN identity — "can I get resourcequotas in tenant-<org>?" and wait,
with bounded exponential back-off (~45s ceiling), for the operator's RoleBinding
to land. An already-onboarded tenant is confirmed by a single fast probe (no
sleep), so existing deploys are not slowed. On timeout, fail CLOSED with a
retryable errTenantProvisioning (deployErrStatus -> HTTP 503, honest
"provisioning, retry") — never a fabricated success. Creating a SSAR needs no
tenant RBAC (system:basic-user), so the probe is itself immune to the window it
closes. No RBAC or namespace derivation is loosened.

Tests (clients/platform/tenant_rbac_test.go): retry-until-RoleBinding-lands
succeeds end-to-end (quota+Service CR written); bounded timeout fails closed
(503, no quota/CR written); ctx-cancel aborts promptly; ready tenant resolves in
one probe (auto-glue fast path unchanged).
2026-07-02 13:39:42 -07:00
zeekay 930ded6efb Merge remote-tracking branch 'origin/rip/api-to-v1' 2026-07-02 13:07:18 -07:00
hanzo-dev 1deec7088c feat(agents): /v1/agents per-org fail-closed metering + long-running scheduler
Lands the agent-backend metering feature onto the zip->zap-proto-migrated main
(cloud is already fully migrated on main; 8 subsystem pins + MountAll clean).

- /v1/agents/* self-meters a per-run fee to commerce: pre-authorize the org's
  prepaid credit balance fail-closed (402 insufficient_balance), debit on success,
  attributed to product "agent". Added to selfMeteredPrefixes so the edge gate
  never double-bills. Run path (money-moving) requires a VALIDATED principal
  (c.User() non-empty), refusing the no-bearer forge path; scheduled runs carry an
  unforgeable 'scheduler'-prefixed actor.
- ResourceMeter.Gate now forwards costCents as AuthInput.AmountCents so the gate
  enforces available >= fee (not merely > 0) — a 1-cent balance can no longer
  authorize a run that takes the ledger negative. MeterUsage generalizes the
  per-org debit (Actor/Model/token attribution) while Meter keeps its signature.
- Long-running agents: cron scheduler scans once a minute over a partial index
  (ix_agents_scheduled), bounded per-org cap (CLOUD_AGENT_MAX_LONG_RUNNING);
  scheduler.stop drains in-flight runs inside the SIGTERM budget via ShutdownAll.

Deps: commerce/metering v0.1.0 -> v0.1.2 (Actor field + AuthInput.AmountCents).
All other pins inherited from migrated main (ai v1.789.1, authz v1.10.3,
base v1.4.6, commerce v1.42.29, licensing v0.1.1, metrics v0.4.1, o11y v1.3.12,
vfs v0.4.4). hanzoai/zip stays out of the graph; zip == zap-proto/zip v1.2.0.

go mod verify clean; go build ./... EXIT 0; MountAll boot-smoke clean (no
want *zip.App); agents -race + root/ml/provisioning billing tests green.
2026-07-02 12:44:31 -07:00
zeekayandhanzo-dev f238188227 chore(deps): realign luxfi/age + luxfi/pq go.sum zip hashes (force-re-tag drift)
Verified the /api/ -> /v1/ rip is COMPLETE on origin/main: zero owned /api/
route registrations, zero owned /api/ client strings. The rip landed in
32138db (productsvc: drop residual /api/ prefix) plus the o11y and eval
cleanups. Every remaining /api/ reference is a non-owned external contract:

  - clients/o11y/o11y.go, o11y_test.go: doc comments ("no /api/, no rewrite")
    documenting the now-removed upstream rewrite.
  - clients/pricing/pricing.go: https://openrouter.ai/api/v1/models — a
    third-party vendor URL.
  - clients/platform/*_test.go: "apps/api/deploy" where `api` is a user's
    APP NAME inside /v1/platform/... paths (not an API prefix).
  - zapface/dispatch.go: comment already says "the /v1 convention".
  - clients/prompts/catalog.json: a prompt-catalog data blob describing a
    different project (prompts.chat), not this repo's routes.

The IAM /api/add-usage-record callout referenced in older cloud docs is a
Casdoor/casibase-lineage endpoint the LEGACY Node cloud-api used (documented
in hanzoai/commerce auth/iam_admin.go). The current Go cloud-api does NOT
call it: billing meters to commerce via hanzoai/commerce/metering. No
cross-service IAM usage-record dependency exists in this repo.

The only change here is a deps-hygiene fix so the build verifies clean:
luxfi force-re-tagged age@v1.5.0 and pq@v1.0.3, drifting their module-zip
h1: hashes vs the recorded go.sum (go.mod hashes unchanged). Realigned to
the upstream hashes at the SAME versions — no major/minor bump.

  CGO_ENABLED=0 GOWORK=off go build -mod=readonly ./...   -> exit 0
  go test ./clients/{o11y,eval,pricing,ml,admin} ./zapface . ./clients/platform -> all ok

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-02 12:33:09 -07:00
hanzo-dev 2ee521c6da fix(platform): serialize apply-CR→finalize-live per app (RED LOW-1)
applyService (operator Service CR write) was not jointly ordered with
FinalizeLive (live-pointer DB write): under concurrent same-app deploys an
OLDER deploy's CR write could land AFTER a NEWER one already went live, leaving
the live Service CR image lagging the recorded live version. deployImage had no
supersede gate at all.

Introduce applyLive — the ONE deploy mechanic shared by the image-source path
and the git build reconciler — running supersede-check → applyService →
FinalizeLive as one per-app-serialized critical section (appMutex, fixed-shard,
O(1) memory). An older deploy that loses the race is superseded and never writes
its CR. FinalizeLive's monotonic CAS still backstops DB monotonicity.

go vet + full clients/platform test suite green.
2026-07-02 12:00:17 -07:00
hanzo-dev 20ea2881ce fix(platform): HIGH-2 secrets-free cloud-api + MED-1 version-monotonic finalize (RED)
HIGH-2 — cloud-api touches NO K8s Secret:
- delete ensurePullSecret + secretsGVR + CLOUD_PLATFORM_PULL_DOCKERCONFIG;
  the per-tenant ghcr-pull Secret is provisioned by the OPERATOR's tenant-RBAC
  controller (from a KMS-synced source). serviceCR only REFERENCES it by name.
  cloud-api's ServiceAccount holds no secrets grant and issues no Secrets call.

MED-1 — monotonic live-version finalize (no build-time inversion):
- Store.FinalizeLive: ONE atomic conditional UPDATE that advances an app to live
  ONLY when its version >= the currently-live version (no read-then-write TOCTOU).
- reconcileBuild gates on buildSuperseded before applying the (older) CR, and
  records a late/older build 'superseded' (image still succeeded) instead of
  regressing the running workload. deployImage shares the same FinalizeLive.
- tests: TestFinalizeLiveIsMonotonic (store CAS) + TestBuildReconcilerVersionMonotonic
  (e2e inversion: newer-first-live, older-late-superseded, CR never downgraded).
2026-07-02 10:55:55 -07:00
hanzo-dev 4d59d648fc feat(platform): git build→deploy watcher (close phase-2) + tenant GHCR pull secret
The git deploy path launched a BuildKit Job fire-and-forget and left the
deployment stuck 'building' — deploy.go documented the build watcher as
'phase 2'. Implement it as the ONE owner of the handoff:

- reconcile.go: a restart-safe reconciler (state in the store, not a
  goroutine) that scans 'building' deployments, checks each build Job, and on
  success applies the operator Service CR with the built image (the SAME
  applyService the image path uses) → deployment 'deploying', app 'live'. On
  failure/deadline it records the honest error. Started from Mount, stopped on
  Shutdown. Org-scoped: every write targets tenant-<row.Org>.
- store.ListBuildingDeployments: cross-org 'building' query (reconciler input).
- k8s.jobOutcome/jobResult: ONE Job terminal-state classifier shared by the
  concurrent-build cap and the reconciler.
- k8s.serviceCR imagePullSecrets + ensurePullSecret: the built image is PRIVATE
  (ghcr.io/hanzoai/tenant-<org>/*); provision the tenant GHCR pull secret from
  CLOUD_PLATFORM_PULL_DOCKERCONFIG (KMS-synced; no-op when unset) and reference
  it so the operator's pod can pull.

Tests: jobOutcome classifier + ListBuildingDeployments (oldest-first, cross-org,
building-only). Full platform suite green (RED authz/cmd-injection incl.).
2026-07-02 10:47:22 -07:00
hanzo-dev 929c55c182 refactor(eval): route telemetry over the ONE shared datastore client
clients/eval/telemetry.go opened a SECOND direct clickhouse.Open with a
parallel CLOUD_EVALS_CLICKHOUSE_* cred namespace, bypassing the shared ZAP
datastore mesh that clients/analytics + ai/object already use. Consolidate:

- eval telemetry now routes every write/read over ai/object's shared client
  (aiobject.DatastoreExec / DatastoreQuery / DatastoreEnabled), the same peer
  the o11y ledger + /v1/analytics use. One connection, one pool, one
  retry/backoff, one KMS-injected cred namespace (DATASTORE_*).
- DELETE the CLOUD_EVALS_CLICKHOUSE_* namespace and the private clickhouse.Open.
- Ownership stays clean: eval owns only its two tables (hanzo.eval_traces,
  hanzo.eval_scores); ai/object owns hanzo.cloud_usage / hanzo.observations.
- No batch primitive needed — eval Records are single-row, mapping to the
  shared DatastoreExec INSERT ... VALUES (?) the o11y write path already uses.
- Async-connect aware: readiness gates per-op on DatastoreEnabled() (honest
  'unavailable' in the boot window), tables ensured idempotently, latched once.
- Reads bind org + narrowers positionally (?) — no interpolation; LIMIT always
  applied. Tenant isolation + score finiteness invariants unchanged.

provisioning's direct CH client is a distinct control-plane concern — untouched.

go build + go test ./clients/eval/... green.
2026-07-02 08:40:16 -07:00
hanzo-devandGitHub 3c8b509380 ci(release): smoke-test the image before it can publish (#59)
A green go build/vet/test does not catch a binary that PANICS at startup.
v1.786.14/.15/.16 compiled clean but crashed at boot with

    cloud: mount metrics: metrics.Mount: app is *zip.App, want *zip.App

(a runtime type-assert from an incomplete hanzoai/zip -> zap-proto/zip
migration), published green, and CrashLooped in prod. The only gate that
catches this class is running the binary.

Restructure the single build-push into build(load) -> smoke -> push:

1. Build once to a local cloud:smoke tag (push:false, load:true), warming
   the BuildKit builder cache.
2. Boot that exact image with a minimal, prod-representative env (writable
   ephemeral /data + a throwaway 32-byte KMS master key so the KMS plane
   mounts on its normal ready path) and assert it reaches "listening" with
   NO startup-crash signature (metrics.Mount / mount metrics / panic /
   want *zip.App), else exit 1 BEFORE any push. Container always rm -f.
3. Re-run build with push:true and the real tags: identical context /
   platform / secrets, so every layer is a cache hit from step 1 and it
   only publishes the already-tested image.

Stays on the self-hosted arcd amd64 scale set + GH_PAT; notify-universe
unchanged.

Proven to DISCRIMINATE against the published images:
  ghcr.io/hanzoai/cloud:v1.786.18 (known-good) -> SMOKE PASS (exit 0)
  ghcr.io/hanzoai/cloud:v1.786.15 (known-bad)  -> SMOKE FAIL (exit 1)
2026-07-02 05:48:29 -07:00
hanzo-dev 7deb08323a merge(cloud): forward-integrate v1.786.18 F1 data-plane gate into the platform mount
SECURITY: cloud origin/main (dcd1a485) had DIVERGED from the LIVE image
v1.786.18 (9e597d07) at merge-base 2e4d402c — v1.786.18 was tagged+deployed
from fix/cloud-1.786.18-zip-and-gates but NEVER merged back to main. It carries
the F1 forged-X-Org-Id cross-tenant gate that main LACKED:
  - clients/principal (validated-principal helper) — new package
  - bot + o11y reverse-proxy forge gates (bot.go, o11y.go + red_forge_test.go)
  - whole-data-plane principal gating across agents/crm/eval/functions/git/kms/
    ml/plan/pricing/projectsvc/prompts/provisioning/s3

Building .20 from main+platform ALONE would have REGRESSED this live fix
(reopened the forged-X-Org-Id hole — the ".17 insecure, do NOT deploy" hole).
This merge makes the artifact a true SUPERSET of the security floor.

Only conflict: clients/provisioning/provisioning.go import block — resolved to
keep BOTH the F1 principal.Validated(c) gate AND the platform sanitizeOrg
injectivity crit-fix (OrgHasUnsafeRune + raw-byte hash). go.mod/go.sum merged
clean (main and .18 converged on identical dep floors: ai 1.789.1, authz
1.10.3, base 1.4.6, commerce 1.42.29, o11y 1.3.12, vfs 0.4.4, zap-proto/zip).

Result = main (analytics/crm/templates/git + zip migration) + /v1/platform mount
+ v1.786.18 F1 gate. All 24 subsystems registered. go build ./... + go vet green.
Tests green together: platform (TestRED_CommandInjectionBlocked, cross-tenant,
injective), F1 (TestRed_BotProxyForwardsForgedOrgNoPrincipal,
TestRed_O11yProxyGatesForgedOrgNoPrincipal), provisioning, principal, root cloud.
2026-07-02 05:34:55 -07:00
hanzo-dev a826a5bdde merge(cloud): mount /v1/platform PaaS subsystem into main (blue/paas-v1platform)
Merge RED-PASSED blue/paas-v1platform@99985110 onto main@dcd1a485, mounting the
per-org container-app PaaS control plane at /v1/platform (HIP-0106) — the deploy
engine behind one-click app deploys (ERP/Helpdesk ride on it).

Conflicts resolved (5 files, all combine-both-sides — no logic dropped):
  - subsystems/subsystems.go: KEEP every registration — platform (order 124) +
    analytics/crm/git/templates/prompts/agents/functions + all pre-existing.
  - clients/provisioning/provisioning.go: main's zip canonical import +
    branch's sanitizeOrg injectivity crit-fix (OrgHasUnsafeRune reject,
    raw-byte SHA-256, no TrimSpace) — both preserved.
  - middleware_identity.go: main's zap-proto/zip + doc; branch's OrgHasUnsafeRune
    (root cloud pkg) + SanitizeIdentity handler hardening — both preserved.
  - provisioning_test.go / middleware_identity_test.go: both test sets kept.

Integration: main migrated the repo hanzoai/zip -> zap-proto/zip; the branch
predated it, so the new clients/platform/{deploy,platform,http_test}.go were
rewritten to the canonical github.com/zap-proto/zip (incompatible zip.Ctx types
otherwise). go.mod unchanged from main (zap-proto/zip v1.2.0 direct); no new dep
(k8s.io/{apimachinery,client-go} v0.35.0 already present via ml/paassvc).

Crit-fixes preserved EXACTLY (RED-PASSED, proven green):
  argv build (TestRED_CommandInjectionBlocked), org-slug injective
  (TestSanitizeOrg{Injective,WhitespaceInjective}, TestBuildImageRefIsInjective),
  ResourceQuota (TestEnsureNamespaceAppliesQuota), no cross-tenant
  (TestHTTPCrossTenantIsolation, TestNamespaceIsDerivedFromOrgNotInput,
  TestServiceCRAlwaysPinnedToTenantNamespace).

go build ./... + go vet + go test (platform/provisioning/root cloud) all green.
2026-07-02 05:22:07 -07:00
hanzo-dev 9e597d07b9 fix(cloud): purge hanzoai/zip (forward zip-canonical-home) + complete F1 data-plane gates
Startup crash on v1.786.15/.16: `cloud: mount metrics: metrics.Mount: app is
*zip.App, want *zip.App`. Cloud's core migrated to github.com/zap-proto/zip
(v1.786.13→.15) so app is *zap-proto/zip.App, but eight hanzo modules cloud
imports still pinned the OLD github.com/hanzoai/zip and registered subsystems
that type-assert app.(*hanzoai/zip.App). Distinct import paths = distinct Go
types, so MountAll's first such subsystem ("metrics", hanzoai/metrics@v0.4.0)
failed the assert at runtime (build/vet/test stayed green — the mismatch is
runtime-only). authz/base/o11y were the same latent break behind it.

Forward fix (the canonical-home migration was already released upstream; cloud
merely lagged): bump every lagging module to its migrated tag —
  ai v1.789.1-… → v1.789.1        authz v1.10.1 → v1.10.3
  base v1.4.1 → v1.4.6            commerce v1.42.27 → v1.42.29
  licensing v0.1.0 → v0.1.1       metrics v0.4.0 → v0.4.1
  o11y v1.3.7 → v1.3.12           vfs v0.4.1 → v0.4.4
authz pinned to v1.10.3 specifically: v1.10.2/v1.10.4 carry an unrelated
GetPolicy 2-value change that breaks the pinned hanzoai/iam; v1.10.3 has the zip
migration AND the iam-compatible 1-value signature. commerce pinned to v1.42.29
(v1.43.0 regressed back to hanzoai/zip). clients/analytics (from the merged .16
work) is cloud's own code and is migrated in-place. Result: hanzoai/zip is gone
from go.mod/go.sum and the whole module graph; the compiled binary mounts
metrics+o11y+all subsystems and reaches "listening" with no panic.

Also folds in the COMPLETE F1 close (RED found the gate was partial — two
reverse-proxy paths still forwarded a forged X-Org-Id):
- clients/bot: gate proxy() on principal.Validated before forwarding X-Org-Id to
  bot-gateway (RED PoC red_forge_test.go now passes: no-principal forge → 403).
- clients/o11y: wrap the installed reverse-proxy handler in gate() — refuse any
  request with no X-User-Id before it reaches the o11y runtime (forge twin test).
- clients/crm: extend the forge guard to WRITE+DELETE verbs (belt-and-suspenders).
- middleware_identity: refresh the stale FAIL-MODE comment — post-F1 the DATA
  plane also fails secure on a cold-cache JWKS failure (bounded by stale-on-error).

Base = F1 (fix/cloud-data-plane-principal-gate, 473e0986) + origin/main (analytics
.16, 2e4d402c). One healthy image: F1 + analytics + zip-fix + bot/o11y gates.
2026-07-02 04:37:38 -07:00
hanzo-dev dcd1a4852b fix(analytics): import canonical github.com/zap-proto/zip (not hanzoai/zip)
Aligns clients/analytics with serve.go + every other in-tree clients/* subsystem,
which import github.com/zap-proto/zip. The original commit imported the OLD
github.com/hanzoai/zip, making analytics.Mount assert *hanzoai/zip.App while serve
passes *zap-proto/zip.App — a boot-time mount type mismatch. Removes the last in-tree
hanzoai/zip importer so once the migration lane re-releases the external subsystem
modules on zap-proto/zip, main boots with ONE zip.App type. (The DEPLOYED v1.786.17
is built off v1.786.13, which is all-hanzoai/zip, and is unaffected.)
2026-07-02 04:33:45 -07:00
hanzo-dev e1e5508e57 Merge remote-tracking branch 'origin/main' into fix/cloud-1.786.17-zip-and-gates 2026-07-02 04:18:41 -07:00
hanzo-dev 473e098687 fix(cloud): gate the whole data plane on a validated principal (RED HIGH — live cross-tenant)
RED found the cloud data plane trusted a bare X-Org-Id with no validated
principal. Off-gateway (direct api.cloud.hanzo.ai / in-cluster), a request with
`X-Org-Id: victim` and NO credential read/wrote/deleted another tenant's data:
CRM PII, KMS secrets, prompts, agents, evals, ML namespaces, projects, git repos.
SanitizeIdentity RESTORES a client X-Org-Id on the bearer-less "Phase-1 data"
path but leaves X-User-Id empty, so c.Org() alone is forgeable; c.User() (set
ONLY from a verified bearer/cookie) is the authentic signal.

ONE canonical gate — new clients/principal:
  - Validated(c): request carries a validated principal (X-User-Id set).
  - Tenant(c): verbatim org, gated + bounded + cloned; ("",false) => caller 403s.
The S3/eval fix (already shipped) is now the single source of truth, used
everywhere instead of six drifting hand-rolled copies.

Data-plane resolvers gated (were bare c.Org()):
  crm, prompts, agents, functions, git, eval -> principal.Tenant (verbatim)
  kms.guard (closes forged X-Org-Id==:org bypass), ml (per-org k8s ns),
  projectsvc, provisioning, s3 -> principal.Validated + own normalize/admin-bucket
  plan, pricing (public catalog) -> validated principal selects its overlay, else
  the public "hanzo" default (never a forged org's overlay)
  middleware_billing (no victim-ledger drain/probe), audit_middleware (aligned)

Breaks no real client: the console BFF always mints a user-bound bearer
(X-User-Id set); opaque-API-key callers hit /v1/ai/*, not these subsystems.

Tests (F4): clients/principal unit suite (forged-org-no-principal refused,
verbatim no-fold, empty/overlong refused, Validated only from X-User-Id); crm
forged-org 403 on every collection (PII); kms Vector8 forged X-Org-Id==:org with
no principal -> 403 (secrets). Existing per-subsystem tests send X-User-Id on the
legit path as the BFF does.

go build ./... green; go vet green; go test ./clients/... . green.
2026-07-02 03:34:45 -07:00
hanzo-dev 2e4d402c09 feat(analytics): native-Go /v1/analytics on datastore/ClickHouse, per-org
Adds clients/analytics (order 132, registered as analyticssvc so the real
/v1/analytics/health owns the probe, not serve.go's generic liveness route) —
the backend for the console Native Analytics module (unified-analytics.md §5).

Two read lenses over the ONE hanzo warehouse, reusing the SAME clickhouse-go/v2
client the ai o11y ledger opens (ai/object DatastoreQuery/DatastoreEnabled/
EnsureCloudUsageTable/ResolveCloudUsageWindow) — no second CH client, DRY:
  - LLM lens (REAL): hanzo.cloud_usage — requests/tokens/spend/models/errorRate
  - web+commerce lens: hanzo.events — honest-empty until the collector emits

Surface (read-only, org-scoped, /v1):
  GET /v1/analytics/overview     per-org KPIs (llm real; web/commerce honest-empty)
  GET /v1/analytics/timeseries   requests/tokens/spend over hour|day buckets
  GET /v1/analytics/top          top models (real) + top products (honest-empty)
  GET /v1/analytics/health       datastore connectivity + lens-table availability

Tenant isolation is the security bar: tenant() requires a VALIDATED principal
(c.User(), set by SanitizeIdentity only for a verified bearer) AND a valid org
(c.Org(), the minted owner claim) — closing the Phase-1 no-bearer forged-X-Org-Id
data path exactly as clients/s3 does. Every query binds the org POSITIONALLY
(query.go llmWhere/eventsWhere), so a maxpower token can never read another org.
ClickHouse creds are KMS-injected env (DATASTORE_*), never hardcoded.

Tests: query-boundary isolation (org bound, never interpolated, incl SQLi slug),
honest-empty, real-number KPIs, errorRate, gap-filled series, top-models pct;
HTTP: no-principal->403, forged-org-no-bearer->403, datastore-down->honest 503,
bad-range->400, health owned-by-analytics honest 503 when down.
2026-07-02 03:32:27 -07:00
hanzo-dev 4b7ddf0717 Merge feat/per-product-metering: per-product credit-drawdown for functions + s3
Integrates the fail-closed per-org credit-drawdown gate into cloud's last two
free data-plane subsystems (functions invoke, s3 op) via the ONE shared
cloud.ResourceMeter, plus the per-org billing-key fix (gate keys on the org slug).

HELD — NOT DEPLOYED. cloud origin/main is non-bootable until the zip migration
(#25 / o11y) lands; this ships with the cloud unfreeze. No dependency changes
(go.mod identical to main), so it adds no new build risk beyond the pre-existing
#25 o11y blocker. Verified: clients/functions 5/5, clients/s3 6/6, root billing
tests green; go build + vet clean on all changed packages.
2026-07-02 02:19:32 -07:00
hanzo-dev 6a56c2ff79 fix(billing): key the credit-drawdown gate on the org slug, not {org}/{sub}
The prepaid credit balance is PER-ORG (one credit pool covers the whole org),
so the metering gate must query the ledger by the org slug. identityFromCtx keyed
User on "{org}/{sub}", which queries an empty per-user ledger and 402s a fully
funded org — a revenue-blocking false-decline for every real request. Key User
on the org slug (bare sub only when org is absent), mirroring
metering.IdentityFromGatewayHeaders so cloud and every product key the SAME
ledger entry. The {org}/{sub} actor identity belongs on the usage audit trail,
not the gate; metering v0.1.0 carries no Actor field, so it is omitted until the
module ships the User/Actor split.

Covered by resource_billing_test.go (per-org debit asserts user==org). Root +
functions + s3 packages: go build + vet clean, tests green.
2026-07-02 02:19:16 -07:00
hanzo-dev 5c37a8350e feat(billing): per-product credit-drawdown metering for functions + s3 (DRY ResourceMeter)
Wire fail-closed credit-drawdown metering into the two remaining cloud data-plane
subsystems that were free, reusing the ONE shared cloud.ResourceMeter (the same
Gate+Meter provisioning and ml already use). No free tier: pre-authorize the
org balance (insufficient -> 402, unreachable -> 503, nothing runs), then debit
on success. No second metering path.

- functions: POST /v1/functions/:name/invoke gates before sandbox compute and
  debits the caller org on a real execution (product "functions", unit
  "invoke", fee CLOUD_FUNCTION_FEE_CENTS, $1.00 default). A sandbox transport
  failure ran no billable compute -> not charged.
- s3: the data-plane guard is the ONE place it meters -- every guarded op gates
  before S3 is touched and debits on handler success (product "s3", unit
  "op", fee CLOUD_S3_FEE_CENTS). A handler error is not billed.
- resource_billing: Meter now records the billed unit as Usage.Model (kind), so
  provisioning kinds (sql/vector/kv/...), functions "invoke", s3 "op" and ml
  kinds all get per-item attribution in the ledger under their product label.
- middleware_billing: add /v1/functions/ and /v1/s3/ to selfMeteredPrefixes so
  the edge gate never double-bills the subsystem's own charge.

Tests (real commerce + sandbox doubles): functions 5/5, s3 guard 6/6 -- 402 on
empty balance with no compute run, debit-on-success to the CALLER org (never the
client default), no-bill-on-failure, free-fee ungated, unconfigured no-op,
tenant isolation. go build + vet clean; affected packages green.
2026-07-02 01:35:53 -07:00
zandGitHub 9a66c437e8 Merge pull request #58 from hanzoai/feat/saas-finance
feat(admin): SaaS finance dashboard — DO burn-down + revenue + margin/runway (cloud)
2026-07-02 01:00:34 -07:00
hanzo-dev 21c0984f27 feat(admin): GET /v1/admin/finance — SaaS profitability dashboard aggregate
Add a global-admin-only finance panel to the /v1/admin/* surface for
admin.hanzo.ai: DigitalOcean credit burn-down (our primary ~$40k credit
venue), month-to-date spend, revenue, MRR, gross margin, and runway.

- digitalocean.go: DO billing client (GET /v2/customers/my/balance +
  /billing_history). Authoritative DO sign convention (from DO's public
  OpenAPI spec): the three money fields are decimal-DOLLAR strings; a
  NEGATIVE account_balance = credit we hold, so creditRemaining =
  -account_balance (clamped at 0). dollars parsed to int64 cents at the
  edge. Token DO_API_TOKEN from env (KMSSecret); unset => {configured:false},
  never a fabricated balance.
- commerce.go: mrrCents reader — sums active/trialing subscriptions'
  monthly-normalized plan price (yearly => /12) for fleet MRR.
- finance.go: financeData shape + computeFinance, a PURE derivation
  (grossMargin = revenue - cost, marginPct, runwayDays = credit/burn or
  null, profitable). Handler fans out to DO + commerce, honest-empty on
  every unconfigured/unreachable path. Mounted under s.guard (global-admin
  only; no-principal/tenant-admin/forged => 403).
- Tests: computeFinance math (profitable + burning-faster + null-runway +
  unconfigured), DO sign/parse, MRR normalization, full-pipe aggregation
  with fake DO+commerce, honest-unconfigured-DO path; /v1/admin/finance
  added to the gate test so 403-for-non-admin is proven.

CGO_ENABLED=0 go build ./cmd/cloud: exit 0. go test ./clients/admin/: ok.
2026-07-02 00:49:14 -07:00
hanzo-dev a82299bfb3 chore: zip v1.2.0 + zap-proto/http v0.2.0 — SSE streams over ZAP
Bump to the streaming transport so cloud's SSE endpoints (o11y traces, MCP
notifications, chunked bodies) push over ZAP as live streams, not buffered blobs.
Additive — existing buffered responses unchanged. Build + tests green.
2026-07-02 00:46:38 -07:00
zandGitHub f68fbbc84d Merge pull request #57 from hanzoai/feat/v1-git
feat(git): native /v1/git layer (clone/push over go-git+VFS) + fix main build (fork.go zip import)
2026-07-01 23:59:47 -07:00
hanzo-dev e57c366f07 fix(projectsvc): fork.go on zap-proto/zip — finish the zip migration
The template-fork merge (#56) left clients/projectsvc/fork.go +
fork_test.go importing github.com/hanzoai/zip while their package sibling
projectsvc.go already moved to github.com/zap-proto/zip (5d2c993), so
`go build ./cmd/cloud` failed with a Ctx/Handler type mismatch. One import
line each — one and one way, the whole package on zap-proto/zip.
2026-07-01 23:56:53 -07:00
hanzo-dev a6e98343c1 feat(git): native S3-ready Git layer — /v1/git/* smart-HTTP clone/push
Adds clients/git: org+project-scoped Git hosting inside the unified cloud
binary — the "internal Gitea, native" foundation agents push code into.

Control plane (X-Org-Id tenancy, HIP-0026; optional X-Project-Id sub-scope):
  POST   /v1/git/repos        create a bare repo -> repoView (201)
  GET    /v1/git/repos        list the tenant's repos
  GET    /v1/git/repos/:name  repo detail (branches, HEAD, sizeBytes)
  DELETE /v1/git/repos/:name  delete + purge storage (204)
  GET    /v1/git/usage        per-repo + total bytes for the tenant

Smart-HTTP git protocol (real `git clone`/`git push` work natively):
  GET  /v1/git/:org/:repo/info/refs?service=git-upload-pack|git-receive-pack
  POST /v1/git/:org/:repo/git-upload-pack    (clone/fetch)
  POST /v1/git/:org/:repo/git-receive-pack   (push)

Storage: bare go-git repos on a go-billy filesystem; go-git's server
transport (plumbing/transport/server) reads/writes it for clone AND push.
MVP backs billy with osfs under {DataDir}/git/<org>/<project>/<repo>.git;
a documented TODO(vfs) seam swaps in hanzoai/vfs (S3/SeaweedFS) — vfs.FS
does not yet implement the go-billy surface go-git's dotgit requires.

Billing: every repo tracks sizeBytes, re-measured on create and after each
push; each measurement emits a meterable `git.usage org=.. repo=.. bytes=..`
log line. TODO(billing) seam for a commerce metering event.

Tenant isolation on every query (org empty -> 403; path :org must match the
authenticated tenant). Registered as subsystem "git" order 132.

go-git/v5 v5.19.1 + go-billy/v5 promoted indirect -> direct (no version bump).

Verified: CGO_ENABLED=0 go build ./cmd/cloud (exit 0); go test ./clients/git
(CRUD+isolation, info/refs advertisement, in-process go-git clone/commit/
push/re-clone round-trip, cross-tenant 403) all pass; real `git` CLI
clone/push/re-clone round-trip confirmed against a live server.
2026-07-01 23:56:53 -07:00
zandGitHub 78fed9688f Merge pull request #56 from hanzoai/feat/template-fork
feat(fork): template → project fork (cloud)
2026-07-01 23:48:00 -07:00
hanzo-dev 42c4da4eb4 chore: zip canonical home — hanzoai/zip -> zap-proto/zip@v1.1.0 + one-verb Listen
Move to the ZAP-family canonical framework (zap-proto/zip) and its final API:
app.Listen(cfg.ZAPListenAddr, "http://"+cfg.ListenAddr) — one verb, transport is
the address scheme (ZAP primary + HTTP extra from one call). Free MCP tool surface
rides along at /mcp over both transports. go.sum re-recorded vs the immutable proxy.
Whole tree builds; cloud/zapface/storagelock tests pass.
2026-07-01 23:45:54 -07:00
hanzo-dev c489487998 feat(projectsvc): fork a gallery template into a real project
Add POST /v1/projects/fork — the ONE way to start a project from the Hanzo
starter-kit gallery in-console. The handler reads the ONE embedded templates
catalog (templates.Get; no catalog copy), maps the template's freeform
framework label to the projectsvc build-hint enum, and funnels through the SAME
createProject path POST /v1/projects uses, so slug validation, org scoping, ID
minting, and conflict handling are not duplicated.

- clients/templates: export List()/Get(slug) so projectsvc reads the catalog
  through one door; the HTTP GET handler now uses Get too (DRY).
- clients/projectsvc: extract create -> createProject (the shared internal
  create path); fork.go seeds a CreateProject from the template (name=title or
  override, slug=target or template slug, framework mapped, repo=gallery source)
  and calls createProject; org-scoped (X-Org-Id) exactly like the other routes.
- Framework mapping (mapFramework): Vite wins -> vite; Next.js -> next;
  React -> react; enum names pass through; bare HTML/* -> static.

Tests: end-to-end wire tests over the real route (template->project mapping,
org scoping/isolation, dup 409, missing-slug 400, unknown-template 404) plus a
mapFramework table pinned to the real gallery labels.
2026-07-01 23:41:50 -07:00
hanzo-dev 5d2c99303f feat: serve /v1 over ZAP — zip v0.5.0, real dual transport
zip v0.5.0 un-stubs the ZAP transport, so cloud now serves BOTH transports from
the ONE app: app.Serve(cfg.ZAPListenAddr, cfg.ListenAddr) binds ZAP (primary,
:9653 via CLOUD_ZAP_LISTEN — already in config) alongside HTTP (:8000). The log
already advertised the zap addr; now it is actually bound. Every /v1 route answers
identically over either transport (no RPC registry, routes ARE the ZAP surface).

- go.mod: hanzoai/zip v0.2.1 -> v0.5.0 (+ zap-proto/http v0.1.0 transitively).
- serve.go: app.Listen(http) -> app.Serve(zap, http).

Verified: go build ./... green (whole 54-file zip surface); go vet clean;
cloud + zapface + storagelock + admin tests pass. Prod ZAP port :9653 will be
open (was closed — the stub never bound).
2026-07-01 22:56:37 -07:00
hanzo-dev 82cdb7308b chore: strip casibase/casdoor — cloud is Hanzo-referential, one and one way
cloud is greenfield original (not a casibase fork); the residual casibase/casdoor
names in comments + one type + one error string were off-brand AND contradicted
that provenance (e.g. storagelock's 'casibase-derived cloud-api' lineage story).
De-branded to Hanzo-referential throughout — the {status,msg,data,data2} WIRE shape
is unchanged (console2 depends on it); it is simply OUR /v1 envelope now.

- zapface: casibaseEnvelope type -> envelope; all 'casibase /v1' -> '/v1'.
- storagelock: dropped the casibase-lineage narrative; 'casibase's XORM knob' ->
  'the storage driver knob'; classify string -> 'driverName=postgres'. SQLite is
  the only backend, full stop (no transitional-config language).
- subsystems: iam '(Casdoor)' -> '(Hanzo IAM)'.
- clients/admin: 'casibase envelope' -> '/v1 envelope' throughout.
- tests: de-branded; the integration test's simulated 'casdoor_session_id' cookie
  -> the REAL 'iam_access_token' contract (cookieTokenNames), so it's more faithful.

Verified: go build + go vet clean; storagelock/zapface/clients-admin tests pass.
2026-07-01 22:56:37 -07:00
hanzo-dev 999851104f fix(platform,provisioning): close RED CRIT-2 residual — whitespace-collapse org injectivity
The org identifier was TrimSpace'd at both trust-boundary sites
(middleware_identity.go on claims.Owner + client X-Org-Id, and
provisioning.sanitizeOrg before hashing), so two DISTINCT IAM orgs differing
only by edge/internal/unicode whitespace ('acme' vs 'acme ' vs 'ac me' vs an
NBSP/ZWSP variant) collapsed onto ONE tenant-<slug> namespace / image ref /
bucket / DB — a cross-tenant fold (IAM org name is an unvalidated varchar, so a
fold-sibling is registrable and mints a valid token).

FIX — normalize+VALIDATE at the trust boundary, reject rather than fold:
- cloud.OrgHasUnsafeRune: refuse any org bearing a whitespace / control /
  zero-width-format (Cf) rune. fasthttp OWS-trims header values, so folding
  such an org could never round-trip through transport — rejection (fail
  secure) is the only injective option. Visible case/'.'/'-' still fold
  injectively via the org-slug hash.
- middleware_identity.go: owner is taken verbatim from the validated principal
  (no TrimSpace) and refused if unsafe -> request resolves org-less, every
  tenant() gate fails closed 403. Client X-Org-Id refused likewise.
- provisioning.sanitizeOrg: reject unsafe-rune inputs (defense-in-depth for
  non-header callers e.g. clients/s3) and hash the RAW bytes, never a trimmed
  copy. c.Org() is now the sole tenancy source and injective end-to-end.

Regression tests: {acme, 'acme ', 'ac me', NBSP/ZWSP/BOM/tab variants} ->
distinct-or-rejected, never colliding (provisioning + platform + middleware
JWT-owner path). go build ./... + go vet + affected suites green.
2026-07-01 21:40:57 -07:00
hanzo-devandGitHub ed5749287f feat(crm): native-Go /v1/crm on Base — companies/contacts/opportunities, per-org (#55)
First slice of the unified-backend-go program: a native-Go port of the Twenty
CRM core model (company/person/opportunity standard objects, composites
flattened to scalar columns) mounted at order 131 in the one cloud binary.

- Base/SQLite store ({DataDir}/crm.db), tenant isolation = org column on every
  query (c.Org() from the validated IAM owner claim, HIP-0026). Mirrors
  clients/prompts + clients/eval exactly (the ONE storage pattern).
- Full CRUD for all three entities + per-org summary counts; in-org referential
  integrity (a relation can never point across tenants; errBadRef -> 422).
- /v1/crm/{summary,companies,contacts,opportunities} — /v1 only, no /api.
- Tests: per-org isolation, CRUD round-trip, referential integrity,
  delete-clears-refs, list filters/counts, HTTP round-trip + validation. 7/7 pass.

No proxy to a NestJS backend; this is the thesis (business apps as native-Go
/v1 subsystems on Base) embodied as one working brick.
2026-07-01 20:14:44 -07:00
hanzo-dev 8fdff378a3 fix(platform): close CRIT-1 cmd-injection, CRIT-2 org collision, MED-3 quotas (RED)
/v1/platform (PaaS) — RED do-not-ship findings. Tenancy core untouched.

CRIT-1 — OS command injection in the privileged BuildKit Job:
  launchBuildJob now emits buildctl as EXEC-FORM argv ([]string, no `sh -c`),
  so no shell parses any input. repo.url / dockerfile / git-ref are validated
  (validate.go): https-only URL to an allowlisted git host, no shell/flag
  metachars; safe relative dockerfile (no `..`); safe branch/tag/commit ref
  (no `#`, no metachars). Output image ref is forced server-side — a client
  cannot override --output/--opt. Validation runs at the build choke point AND
  early at createApp (400). red_cmdinj_poc_test.go flipped to a passing guard.

CRIT-2 — sanitizeOrg collision (non-injective) → cross-tenant takeover:
  deleted the lossy platform.sanitizeOrg; tenant()/tenantNamespace()/
  buildImageRef() now use the ONE injective provisioning.SanitizeOrg (DRY,
  reused — commit 01bf3a12). Image ref made injective too: org+app are now
  separate '/'-joined path components (ghcr.io/hanzoai/tenant-<org>/<app>),
  neither slug can contain '/', so (a-b,c) vs (a,b-c) no longer collide.

MED-3 — quotas / replica bounds / shared-build DoS:
  clampReplicas caps replicas to [1,20] (env CLOUD_PLATFORM_MAX_REPLICAS) at
  createApp, applyService, and scaleService (fail-secure default). ensureNamespace
  applies a ResourceQuota + LimitRange per tenant namespace (idempotent).
  launchBuildJob caps concurrent builds per org (default 3, errTooManyBuilds→429).

Tests: cmd-injection blocked (5 vectors), org-slug + image-ref injectivity,
replica clamp (unit+HTTP), namespace quota/limitrange, concurrent-build cap.
go build ./... green; clients/platform + provisioning + s3 tests green.
2026-07-01 19:17:28 -07:00
hanzo-dev 834ebf58f3 templates: read-only starter-kit gallery at /v1/templates (69 templates from hanzoai/gallery; browse + fork/deploy handoff) 2026-07-01 19:13:52 -07:00
hanzo-dev 116df7c7ab prompts: read-only starter catalog at /v1/prompts/catalog (107 prompts, browse+import; org store stays honestly empty) 2026-07-01 19:08:54 -07:00
hanzo-dev b72053d8a8 fix(evals): bound /v1/evals/runs — per-org concurrency + wall-clock deadline (RED MED)
A synchronous run drives up to maxRunItems paired LLM calls against the SHARED
in-process gateway. Two bounds stop one org from degrading every tenant:

- Per-org concurrency cap (maxConcurrentRunsPerOrg=4): a run acquires a per-org
  slot after passing validation; excess concurrent runs fail fast with 429 (never
  queued — queuing just relocates the exhaustion). Slot released on every return.
- Total wall-clock deadline (maxRunDuration=10m): the item loop runs under a
  context.WithTimeout; a run that exceeds it is cancelled, remaining items are
  recorded as honest errors, and the partial summary returns (Scored counts only
  real successes → 502 when nothing scored). A runaway can never pin a request +
  a gateway slot indefinitely.

Also (RED LOW, verified NOT present at 249a136e despite belief): getDataset now
sizes its collection via store.CountItems (SELECT COUNT(*)) instead of loading up
to maxListLimit full item bodies just to len() them (~96MB amplification on a
large dataset).

Tests: TestRunConcurrencyCap (semaphore fill/refuse/release), TestRunDeadline
Bounded (blocking runner + tiny deadline → cancels, 502, honest item errors, no
hang). go build ./clients/eval/ . green, go vet clean, all eval tests pass.
2026-07-01 18:58:41 -07:00
hanzo-dev 468e48f934 fix(evals): HIGH — gate tenant() on validated principal + clone org key (RED)
Two cross-tenant fixes on the /v1/evals API layer (RED review):

1. Principal gate (HIGH): tenant() now requires a non-empty c.User() (X-User-Id,
   set ONLY by SanitizeIdentity from a verified token/session and stripped from
   client input). Its Phase-1 residual RESTORES a client X-Org-Id on the
   NO-principal path (bearer-less / opaque hk-/sk- key / invalid bearer), so
   without this gate a direct-to-pod request 'X-Org-Id: victim' with no auth read
   /wrote/deleted the victim org's datasets (PII golden outputs), scores, traces
   and runs. This is the same trust signal the audit layer uses (actorFromCtx).

2. Buffer-aliasing (correctness on the isolation key): c.Org() is a zero-copy
   view into the fasthttp request buffer, reused after the request ends. The org
   is our tenant KEY and is retained (telemetry events, run records); tenant()
   now strings.Clone()s it so a stored org can never silently mutate into another
   value once the buffer is recycled (was manifesting as run scores landing under
   a corrupted org id).

red_cap_test.go TestRed_ForgedHeaderCannotCrossTenant now sends a no-principal
X-Org-Id:victim request asserting 403; eval_test.go asserts tenant()='' without a
validated principal. go build ./clients/eval/ . green, go vet clean, all eval
tests pass.
2026-07-01 18:58:41 -07:00
hanzo-dev 1579716b34 feat(evals): native /v1/evals over Base+datastore, retire console proxy
Replace the Langfuse-fork proxy (crash-looping console P1012) with a native,
org-scoped evals system. Storage split per CTO directive:
- metastore (store.go): Base/SQLite, per-org config — datasets, dataset-items,
  evaluators, score-configs, dataset-run defs. Composite (org,id) keys so an id
  is never a cross-org global key (no existence oracle; two orgs may reuse ids).
- telemetry (telemetry.go): datastore/ClickHouse MergeTree, append-only traces +
  scores-as-events; every read binds org as a named param + LIMIT; behind an
  interface with an in-memory impl for tests. Reuses the Langfuse v3 CH shapes.
- runner (runner.go): pluggable EvalRunner (Complete + Judge); gateway runner is
  the spine (any model/judge, no token cap), DO can drop in as an adapter later.

Tenant isolation is c.Org() (validated bearer owner) ONLY — never client
X-Project-Id/X-Org-Id (the cross-tenant break the old proxy shipped). Score
integrity: NaN/Inf rejected, values validated against the org's score-config,
categorical labels checked against the allowed set. Content caps + name regex
guard injection/traversal/amplification.

24 tests pass: metastore + HTTP cross-tenant isolation, forged-header guard,
score-integrity, content caps, run orchestration over a stub runner.

/v1 only. TDD (go test green). gofmt+vet clean. go build ./... green.
2026-07-01 18:58:41 -07:00
hanzo-dev e836e24f0f cloud(s3): drop CLOUD_ prefix on the shared S3 env family (S3_*)
One clean env family for the ONE shared S3 access path (clients/s3admin)
and the provisioning control plane. Rename across s3admin, clients/s3,
projectsvc, and provisioning — code, comments, log/error strings, tests:

  CLOUD_S3_ADMIN_ENDPOINT   -> S3_ADMIN_ENDPOINT
  CLOUD_S3_ADMIN_ACCESS_KEY -> S3_ADMIN_ACCESS_KEY
  CLOUD_S3_ADMIN_SECRET_KEY -> S3_ADMIN_SECRET_KEY
  CLOUD_S3_SECURE           -> S3_SECURE
  CLOUD_S3_REGION           -> S3_REGION
  CLOUD_S3_PUBLIC_ENDPOINT  -> S3_PUBLIC_ENDPOINT
  CLOUD_S3_PUBLIC_SECURE    -> S3_PUBLIC_SECURE

The cloud CR (universe 9e57d71d) already stamps BOTH the old and new
ACCESS_KEY/SECRET_KEY spellings from the s3-credentials secret, so this
image roll is drop-in: old names stay populated until the new image
lands, then the S3_* names take over. Everything else resolves from code
defaults (s3.hanzo.svc:9000 / us-east-1 / s3.hanzo.ai). go build + vet +
tests green across all four packages.
2026-07-01 18:40:51 -07:00
hanzo-dev 8be42c8b6c feat(org): per-org envelope encryption — SQLite ciphertext at rest, wired live
No TODO, no placeholder: the "full customer encryption" brick is real and wired
into the Replicator. Each org's SQLite snapshot is sealed with a distinct
AES-256-GCM key DERIVED from the KMS master via HKDF(master, label, orgID) — the
master never leaves the process, per-org keys are in-memory only. The SeaweedFS
object is ciphertext; orgs are cryptographically isolated (orgID bound as GCM
AAD, so a blob can't be replayed under another org); rotating the master re-keys
everything. Nonce is derived from (key, plaintext) so identical content seals
identically — the Replicator's version-skip keeps working under encryption.

Wired: NewReplicator(..., WithEncryption(cipher, orgID)) → Push seals, Pull opens.
Omit the option and the DB is stored in the clear (local dev only).

Pure Go stdlib (crypto/aes, crypto/cipher, crypto/hmac, crypto/sha256, crypto/subtle) —
package org stays dependency-free + testable. Tests: round-trip, wrong-master reject,
cross-org isolation, tamper detection, deterministic-per-content, master-rotation
rekey, and an end-to-end encrypted Replicator (stored bytes are ciphertext, reader
with the key restores plaintext, no plaintext leak without the key).
2026-07-01 17:57:50 -07:00
hanzo-dev c7e22c5675 harden(platform): bind custom ingress domains to the caller's org (RED — domain hijack)
A custom domains[] entry was rendered straight into the operator Service CR
ingress.hosts, so a tenant could claim another org's host or a Hanzo apex
(api.hanzo.ai) and the operator would serve an Ingress for it. Require every
custom host to be under the caller's OWN '<org>.<sitesHost>' subtree (e.g.
maxpower may only claim *.maxpower.hanzo.app); anything else is refused 501
(verified arbitrary custom domains are phase-2 domain CRUD). Closes the
cross-tenant/apex domain-hijack vector reachable in the first slice. +2 tests
(unit + HTTP); 23 tests green.
2026-07-01 16:46:39 -07:00
hanzo-dev af218ac1ad fix(platform): require a validated principal in tenant() (RED HIGH)
/v1/platform mutates cluster state (operator Service CRs + BuildKit Jobs in
tenant-<org>) — more consequential than a data read — so trusting X-Org-Id alone
would let a direct-to-pod caller forge X-Org-Id:victim with NO bearer and
deploy/read into another tenant (SanitizeIdentity's documented Phase-1 residual).
tenant() now gates on c.User() (X-User-Id, set ONLY for a validated principal),
mirroring the Red-hardened clients/s3.tenant. Proven live: forged X-Org-Id with
no token -> 403 (was 200); real JWT -> 200; forged header + real token ->
validated owner wins. Every legitimate caller (gateway/console BFF) carries a
user-bound bearer, so no real client breaks.
2026-07-01 16:36:01 -07:00
hanzo-dev d604351c90 fix(eval): tenant() must use the sanitized org, never client X-Project-Id (cross-tenant)
RED MED-1 (cross-tenant eval-score read). eval.tenant() scopes the console API
key pair (console-pk-{org}/console-sk-{org} in KMS) and was PREFERRING the raw
`X-Project-Id` request header over the bearer-pinned org, then feeding it to
resolveKeys(). X-Project-Id is a project sub-scope WITHIN an org and is
DELIBERATELY excluded from SanitizeIdentity.authorityHeaders (client-controllable,
per middleware_identity.go). So a caller who set `X-Project-Id: victim-org` made
resolveKeys fetch ANOTHER org's console-pk/console-sk from KMS and read that org's
eval scores/datasets — a cross-tenant break. Reading a raw X-Org-Id header was
equally unsafe (SanitizeIdentity strips a client copy and re-mints c.Org() from
the token).

Fix: tenant() returns ONLY c.Org() — the org SanitizeIdentity pinned from the
validated bearer owner (HIP-0026) — the same authoritative selector
agents/prompts/provisioning use. X-Project-Id no longer influences KMS key
selection anywhere in the evals facade (it was used nowhere else). Per-PROJECT
key scoping, if ever needed, must derive the project from a membership check UNDER
c.Org(), never a raw sub-scope header (Phase-2; keys stay org-scoped today).

The console2 BFF (app/cloud) was the sole compensating control (it drops
X-Project-Id without forwardScope); this closes the defect AT THE SOURCE so
isolation no longer hangs on one omitted proxy header. New test
TestTenantIgnoresClientProjectID pins that a forged X-Project-Id never becomes the
tenant. go build ./clients/... + go test ./clients/eval/... green.
2026-07-01 16:24:12 -07:00
hanzo-dev 9470345d0c feat(platform): native per-org /v1/platform PaaS subsystem (Dokploy port, Goa-designed)
Port the standalone Dokploy (platform.hanzo.ai) tRPC backend into the unified
cloud binary as clients/platform, mounted at /v1/platform (HIP-0106). Per-org,
IAM-validated, Base/SQLite store; the deploy path writes an operator hanzo.ai/v1
Service CR into the caller's OWN tenant-<org> namespace (derived from the
validated X-Org-Id, never a request input) and the operator reconciles it. Git
apps build via an in-cluster BuildKit Job (arcd model); image apps deploy
directly. Complements clients/paassvc (admin fleet board) and clients/projectsvc
(static sites) with the container-app PaaS.

Goa is the design-first contract (clients/platform/design, goa gen -> OpenAPI 3);
the runtime is native zip handlers (one router, behind SanitizeIdentity) — the
generated net/http server is deliberately NOT mounted so the identity trust
boundary is not routed through the fiber<->net/http adaptor.

- store.go: projects/applications/deployments/builds, org column tenancy
- k8s.go: tenant-<org> namespace derivation + Service CR apply/scale/delete + BuildKit Job
- platform.go: Mount + project/app CRUD + tenant() gate + health
- deploy.go: deploy/start/stop + deployment history/logs, fail-closed (no fabricated success)
- 20 tests: store CRUD, cross-tenant isolation (RED bar), fail-closed deploy,
  fake-cluster deploy-into-tenant-ns success, secret-env rejection

go build ./... green; go test ./clients/platform/ green.
2026-07-01 16:20:48 -07:00
hanzo-dev 01bf3a12fe fix(s3,provisioning): close Red re-review findings — control-plane forge + injective org slug
Red re-review (0 critical, 1 high, 1 med, 1 low): the s3 data-plane HIGH was
confirmed CLOSED, but Red found the same forge open on the provisioning control
plane (worse: destroy DB + credential exfil), proved my org-fold dispute WRONG
with a reachable cross-tenant collision, and asked to lock the fix's dependency.

- [HIGH] provisioning.tenant() now requires ctx.User() (provisioning.go:454) —
  same gate as the s3 fix. Without it, an in-cluster caller could forge
  'X-Org-Id: victim' with NO bearer and POST /v1/sql (allocate a DB in the
  victim's namespace + receive its connection string + password), DELETE
  /v1/sql/:name (destroy the victim's DB), or enumerate resources. SanitizeIdentity
  restores a forged X-Org-Id on the no-principal Phase-1 path but strips X-User-Id;
  gating on it refuses only the anonymous forge. Test:
  TestForgedOrgWithoutPrincipalRefused (forged org + no principal -> 403 across
  POST/DELETE/GET, provisioner never runs).
- [MED, dispute WITHDRAWN — Red was right] provisioning.sanitizeOrg is now
  INJECTIVE (provisioning.go:468): identity on a clean [a-z0-9-] slug, else the
  fold + '-'+16hex SHA-256(raw owner) — mirroring iam/object/orgdb.go:orgSlug.
  The old lossy fold collapsed 'Acme'/'acme' and 'team.a'/'team-a' onto one slug,
  and since the whole tenant->bucket/DB namespace hashes THAT slug, two distinct
  orgs shared one physical namespace (reachable: the IAM org name is a varchar
  with no shape validator, so a fold-sibling is registerable + mints a valid
  token). Tests: TestSanitizeOrgInjective (the exact collisions no longer collide,
  incl. derived orgHash) + updated TestSanitizeOrg.
- [LOW] locked the s3/provisioning fixes' cross-file dependency:
  TestSanitizeIdentity_AnonPathHasNoUserId asserts a client-forged X-User-Id does
  NOT survive the anon path (ctx.User()=="") while X-Org-Id does — so a future
  refactor that restored X-User-Id fails this test first.

All fold consumers (orgHash -> SQL/KV/CH/S3 physical namespaces) inherit the
injective slug through the ONE sanitizeOrg. go test green (provisioning + s3 +
s3admin + root identity); cmd/cloud builds; gofmt/vet clean; zero go.sum drift.
2026-07-01 15:40:11 -07:00
hanzo-dev 2473043d3c fix(s3): address Red review — require validated principal, harden keys, shorten presign TTL
Red adversarial review (0 critical, 1 high, 3 med, 4 low). Fixes:

- [HIGH] tenant() now REQUIRES a validated principal (ctx.User()/X-User-Id).
  SanitizeIdentity restores the client's raw X-Org-Id on the no-principal
  'Phase-1 data path' but leaves X-User-Id empty; a pure data plane trusting
  X-Org-Id alone let an in-cluster caller (co-namespace pod) forge
  'X-Org-Id: victim' with NO bearer and get cross-tenant object CRUD. Gating on
  X-User-Id refuses ONLY that anonymous forge path — every legitimate caller
  reaches s3 through the console BFF /cloud proxy which mints a user bearer, so
  no real client breaks. Object storage never serves an unauthenticated
  principal. Test: TestForgedOrgWithoutPrincipalRefused (forged org + no
  principal -> 403 across the full route surface).
- [MED] presign TTL 15m -> 5m: bounds a minted capability's post-revocation
  lifetime (presigned URLs have no server-side revocation; the TTL IS the
  window). Documented the unwired-rate-limiter platform gap.
- [LOW] cleanKey rejects control bytes (\x00-\x1f) + backslash: a null byte
  serializes as %00 (C-string truncation risk for a downstream consumer) and
  '\' is a non-Go path separator. Tests extended.
- [LOW] friendlyBucket re-validates the recovered name against bucketNameRE:
  a prefixed-but-non-conforming bucket (only reachable out-of-band, never via
  createBucket) is treated as not-owned, so listBuckets never echoes an
  unaddressable name. Test added.
- Documented the ESCALATED residuals (not subsystem-fixable): single omnipotent
  SeaweedFS identity (isolation is app-layer only until STS/scoped creds), and
  the intentional S3-vs-KMS org-normalization divergence (S3 must fold to match
  provisioning's bucket naming; KMS keys secrets by exact owner).

Verdict was fix-then-ship; no critical, HIGH is bounded (not internet-reachable,
gateway strips X-Org-Id at the edge). go test ./clients/s3/... green (20 tests);
cmd/cloud builds; gofmt clean; zero go.sum drift.
2026-07-01 15:40:11 -07:00
hanzo-dev fbf3dbaca3 feat(s3): native org-scoped /v1/s3 object-storage file manager
Adds the DATA plane over the shared SeaweedFS S3 gateway as /v1/s3/* on the
unified cloud binary (HIP-0106), the companion to clients/provisioning's s3
CONTROL plane. One console, one backend — no external s3.hanzo.ai UI.

- clients/s3admin: the ONE shared S3 access path. Both projectsvc/blob (deploy
  blob store) and the new s3 subsystem build their minio client here from the
  SAME CLOUD_S3_ADMIN_* creds (DRY — no second S3 client anywhere). Leaf pkg
  (minio-go only), so no import cycle. Separate public-host client mints
  presigned URLs a browser can follow.
- clients/projectsvc/blob.go: refactored to build its minio client via s3admin
  (was inline minio.New). Existing projectsvc tests unchanged + green.
- clients/s3: /v1/s3/{health,buckets,buckets/:bucket,buckets/:bucket/objects,
  buckets/:bucket/objects/*}. Org-scoped bucket-per-org via the EXACT
  provisioning.BucketName scheme (exported) = bucketName(physicalName(org,name)):
  org-hash prefixed AND '_'->'-' folded to a DNS-safe S3 name — so a bucket
  provisioned via POST /v1/s3 {name} is browsable here AND a bucket created here
  is a valid S3 name (the raw physicalName has underscores S3 rejects). Upload/
  download = presigned PUT/GET URLs (browser goes direct to S3; admin cred never
  leaves the server; object key path-clean-guarded; time-boxed 15m). Fail-closed
  503 without creds. Registered 's3svc' order 118 (< provisioning 120) so the
  static /v1/s3/buckets + /v1/s3/health win Fiber's first-match scan ahead of
  /v1/s3/:name and the generic-health route does not shadow the real probe.
- subsystems.go: one-line blank import.

Tests: go test ./clients/s3/... ./clients/s3admin/... green — fail-closed 503,
org 403, route-ordering (s3 owns /v1/s3/buckets + /health, not provisioning
:name), bucket-name + object-key traversal 400, cross-tenant physical-name
isolation, AND bucket-name consistency with provisioning + DNS-safety (no '_').
Zero go.sum drift (minio-go already a dep). cmd/cloud builds.
2026-07-01 15:40:11 -07:00
hanzo-dev 6b4f399143 chore(deps): bump ai -> 296a9e9b (ai ledger owns cloud_usage only; drop colliding observations write) 2026-07-01 15:17:04 -07:00
hanzo-dev 08e2b1b0a9 chore(deps): bump ai -> a5a199e9 (o11y ledger reaches ClickHouse directly)
Pulls hanzoai/ai#a5a199e9: the cloud_usage/observations ledger now uses a direct
clickhouse-go/v2 client (object.InitDatastore in the shared Bootstrap) instead of
the dead ZAP 'datastore peer'. Fixes GET /v1/get-cloud-usages 'datastore peer not
connected'. Requires DATASTORE_ADDR/USER/PASSWORD env (wired on the cloud CR).
2026-07-01 14:58:40 -07:00
19a19253bd fix(provisioning): return PUBLIC endpoints, never the internal .svc host (#53)
Tenants were shown the internal admin address (e.g. vector.hanzo.svc:6333) in
create/get/list responses + the connectionString — unusable from an app and a
leak. Add publicEndpoint(kind): HTTP kinds (vector/search/docdb/s3) → the unified
api.hanzo.ai gateway (/v1/<kind>/*); native-wire DBs → <kind>.hanzo.ai on the
native port (sql.hanzo.ai:5432, kv.hanzo.ai:6379, datastore.hanzo.ai:8123). So a
customer gets a real, routable endpoint for their app + hanzo.app. Per-kind
override via PUBLIC_<KIND>_HOST/_PORT. The DSN host:port is remapped too.

Co-authored-by: zeekay <z@zeekay.io>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-01 13:46:19 -07:00
hanzo-dev f8bfeae427 fix(prompts): self-heal from the legacy clients/prompt schema on shared prompts.db
Live 1.786.5 returned 500 "no such column: id" on GET /v1/prompts. The removed
clients/prompt facade (shipped in 1.786.4) had created a `prompts` table in the
SAME {DataDir}/prompts.db with a versioned-rows layout that has NO `id` column.
clients/prompts' `CREATE TABLE IF NOT EXISTS prompts` no-ops on that existing
table, so every `SELECT id,... FROM prompts` failed against the persisted PVC.

migrate() now inspects the `prompts` table via PRAGMA table_info; when it exists
but lacks `id` (the legacy signature) it drops the legacy pair and rebuilds the
forward schema. The legacy rows carry no durable product data (the facade shipped
one release; the console Compute pages never wrote through it), so this is
forward-only and idempotent — once `id` exists it never fires again.

TDD: TestMigrateSelfHealsFromLegacyPromptSchema seeds the exact legacy schema +
a row, opens the store over the same file, and asserts List/Upsert succeed. Green.
2026-07-01 13:42:56 -07:00
hanzo-dev 0a37ebe31d Merge blue/cloud-tenant-endpoints: per-org /v1/{prompts,agents,functions}
Mounts the red-approved (3 rounds) per-org product control planes natively
in the unified cloud binary (the "all products in the cloud binary" thesis):

  - clients/prompts   order 126  /v1/prompts/*    versioned prompt library
  - clients/agents    order 127  /v1/agents/*     autonomous agents + runs
  - clients/functions order 128  /v1/functions/*  serverless fns + invoke

All three are org-scoped by the gateway-minted X-Org-Id (HIP-0026), fail
closed when it is absent, and persist to a per-tenant modernc.org/sqlite
store under CLOUD_DATA_DIR (no CGO). Each ships adversarial red_*_test.go
coverage (all green).

DRY reconciliation with main: main had independently added clients/prompt
(singular) also registering "prompts" for /v1/prompts. Two subsystems both
named "prompts" would double-append to cloud.Registry and double-mount the
same routes. blue's clients/prompts is the red-approved superset (adds
/v1/prompts/metrics — which console2's PromptsModule consumes — plus DELETE,
strict fail-closed X-Org-Id, and a reserved-name guard), so it becomes the
ONE owner of /v1/prompts/*; the earlier clients/prompt facade is removed.

Build green (CGO_ENABLED=0 go build ./...); prompts/agents/functions +
root + cmd/cloud tests all pass.
2026-07-01 13:16:58 -07:00
hanzo-dev 7b5e9bf642 feat(prompts): serve /v1/prompts from a native SQLite store (kill the console loop)
The console (Langfuse image 3.159.55) 307-redirects /api/public/v2/prompts →
/v1/prompts, delegating prompts TO cloud. promptsvc used to proxy the other
way → redirect loop → 500. Port prompts NATIVE: an org-scoped, versioned
prompt registry in a single SQLite file (base pattern, modernc driver, WAL),
list/get/create. No proxy, no loop, all-SQLite by construction. Verified:
GET /v1/prompts → {data:[]}, POST creates v1, GET lists it, GET /:name
returns versions. Honest-empty when none.
2026-07-01 13:06:23 -07:00
hanzo-dev 0ad5852438 feat(kms): embed luxfi/kms in cloud (/v1/kms/*), fail-secure sealed store
HIP-0106 'all Go embeds in cloud': the KMS secrets plane runs in-process
in the cloud binary instead of the standalone Infisical fork.

- clients/kmsembed: cloud-free Client (implements types.KMSClient) over a
  luxfi/zapdb SecretStore. AES-256-GCM envelope (per-secret DEK sealed
  under a 32-byte master KEK); plaintext never touches disk. New() uses a
  fail-SECURE 3-way store-open (keyed→encrypted on-disk; no-key+no-store→
  ephemeral in-memory, no plaintext registry; no-key+existing-store→fail
  loud, never silently shadow encrypted data). Sign fails closed when no
  MPC backend is co-hosted. One place for key-shape validation.
- clients/kms: Fiber subsystem mounting /v1/kms/* (order 10), org-scoped
  CRUD via cloud's auth; /v1/kms/health + /v1/kms/config.
- build.go pickKMSClient: Enabled('kmssvc') → in-process kmsembed.New,
  fail-closed to DisabledKMS on error (never nil).
- config: CLOUD_KMS_MASTER_KEY_REF / CLOUD_KMS_MPC_ADDR / _VAULT_ID.

Reviewed blue+red: 40 tests (7 build + ~33 adversarial), gofmt/vet clean,
go.sum zero-diff. Red verdict: ship (fail-closed + confidentiality
invariants hold across the full key/store matrix).
2026-07-01 12:44:33 -07:00
hanzo-dev f1330eef74 chore(deps): bump ai → v1.789.0 — unify iam hotfix into main
Brings the released ai v1.789.0 onto main: it has BOTH the V1IamRewriteFilter
(the /v1/iam/* account-surface fix shipped off-main as the 1.785.35 hotfix) AND
the canonical X-Project-Id/X-Environment/X-User-Id header sweep. Resolves the
divergence — main is now the one lineage (prompts + bot + headers + org refactor
+ SeaweedFS replication + iam fix). Build + embed/identity tests green.
2026-07-01 12:24:46 -07:00
hanzo-dev e46d654521 refactor(org): tenant IS the organization — per-org writer pinning + SeaweedFS replication
Drop the confusing "tenant" concept: in Hanzo the ORGANIZATION is the tenant
boundary (identity, billing, per-org SQLite are all org-scoped). Renamed
internal/tenant → internal/org and made ownership explicitly PER-ORG: one replica
writes ALL of an org's databases (root + per-project + per-user), for locality +
intra-org consistency; the org moves as a unit on failover.

- owner.go: Owner/IsOwner/Replicas over Rendezvous (HRW) hashing of orgID — every
  replica computes the same writer-owner from the same membership, NO coordinator.
- membership.go: live replica set via a pluggable Source (StaticSource/CLOUD_REPLICAS
  now; K8s Endpoints / zapd gossip later); lock-free AmOwner hot-path.
- replica.go: Replicator Push (owner → SeaweedFS) / Pull (reader ← SeaweedFS,
  version-skip) + DBPath (orgs/<org>[/<scope>]/<service>.db, HIP-0302).
- vfsstore.go: object store bound to hanzoai/vfs (SeaweedFS) — NO minio, NO external
  S3 SDK. hanzoai/sqlite + hanzoai/base back the local DB handle (the DB interface).

Package is PURE stdlib (local vfsClient interface, zero cloud deps) so it builds +
tests without the dep tree. 16 tests pass: determinism, single-owner, even
distribution (±35%/50k), exact minimal-reshuffle, ordered failover, push/pull
round-trip, skip-unchanged, ownership handover, DBPath layout, vfs adapter. gofmt clean.
2026-07-01 11:07:23 -07:00
hanzo-dev 63142078c7 feat(bot): mount /v1/bot/* → bot-gateway; name datastore/docdb by the primitive
- botsvc (order 143): reverse-proxies /v1/bot/* to the in-cluster bot-gateway,
  stripping the /v1/bot prefix (the gateway serves bare paths: /v1/bot/health →
  bot-gateway /health) and forwarding the gateway-minted identity headers. The
  console2 Bot module's /v1/bot/health probe now resolves instead of 404.
  Verified e2e against the real bot-gateway: /v1/bot/health → 200
  {"service":"bot","status":"ok"}.
- provisioner: rename the datastore/docdb provisioners by the HANZO PRIMITIVE
  (datastoreProvisioner/newDatastore, docdbProvisioner/newDocdb) not the backing
  tech — the ClickHouse/MongoDB driver imports + wire-protocol schemes stay
  (functionally required), but the type names read as the primitive.

Note: /v1/s3, /v1/datastore, /v1/docdb are ALREADY mounted (provisioning loops
its 7 kinds); /v1/memory is served by the ai monolith. functions has no backend
— left honest (not fabricated).
2026-07-01 10:49:09 -07:00
hanzo-dev 2bdc066c6f feat(prompts): mount /v1/prompts in the unified binary (console Langfuse facade)
The console2 Prompts module hit GET /v1/prompts → 404 "not routed on this
host" because no subsystem owned the route. Add promptsvc: a thin facade
(order 144, mirrors evalsvc) proxying list / get-by-name / create to the
console public prompts API (/api/public/v2/prompts) under the project-scoped
console key pair (HTTP Basic) — the same console + auth the eval facade
already composes. No prompt logic reimplemented; the console owns storage,
versioning, and labels.

Verified locally: the binary boots with "prompts surface mounted", GET
/v1/prompts routes to promptsvc (503 honest "no console API key" in the
keyless test env, not a 404), /v1/prompts/health → 200 (no route shadow).
In prod the console-keys secret is already wired (evalsvc), so prompts
resolve real data. eval tests + embed tests stay green.
2026-07-01 10:41:25 -07:00
hanzo-dev 29af637039 feat(tenant): rendezvous-hash owner — coordination-free per-tenant writer pinning
The load-bearing primitive of the horizontally-scalable OSS cloud (Hanzo V8).
Every replica of the unified binary computes the SAME writer-owner for a tenant's
per-tenant SQLite from the SAME membership set — via Rendezvous (HRW) hashing —
so there is NO election, NO lock service, NO discovery. This removes the whole
"who finds/owns/reaches service X" plumbing class for tenant state:

- Owner(tenant, members): deterministic, order-independent, exactly one owner.
- IsOwner(tenant, self, members): the per-write hot-path check.
- Replicas(tenant, members, n): owner + ordered failover successors (pre-warm S3).

HRW gives minimal reshuffle on membership change (only a departed replica's ~1/N
tenants migrate; the rest stay put) — cheap rolling deploys + scale-out. Pure Go
(crypto/sha256, no deps). 6 tests: determinism, single-owner, even distribution
(±35%/50k), minimal-reshuffle (exact), ordered failover. All pass; gofmt clean.

Wires into: owner holds the SQLite WAL (1 writer + N readers), streams WAL to
SeaweedFS/S3 (HIP-0107); non-owners read the S3 copy or forward strong writes.
Per-tenant envelope encryption (DEK wrapped by KMS master) makes the S3 file
ciphertext — tenants crypto-isolated. Membership feeds from K8s Endpoints / zapd.
2026-07-01 10:25:02 -07:00
hanzo-dev 104e034f66 chore(deps): bump ai → X-User-Id canonical identity header (drop X-IAM-*)
Pulls hanzoai/ai 56a55d1c so the unified binary's monolith reads the
canonical X-User-Id (cloud middleware_identity already injects it) instead
of the never-sent X-IAM-User-Id. Completes the X-IAM-* → canonical sweep
(org/project/env/user) in the compiled-in monolith. Builds + tests pass.
2026-07-01 10:16:18 -07:00
hanzo-dev ea8e4b61d7 chore(deps): bump ai → project/env canonical-header fix (X-Project-Id/X-Environment)
Pulls hanzoai/ai 7aab19aa into the unified binary so the monolith's
tenant-context filter reads the canonical X-Project-Id / X-Environment
(was X-IAM-*, never sent → project+env scoping was empty across ~200 /v1
routes) + the CORS allow-list accepts them. Pseudo-version pending the ai
v1.788.1 release tag; re-pin to the clean tag on CI cut. Full binary
builds, embed + identity tests pass.
2026-07-01 10:11:56 -07:00
hanzo-dev 5ebb43dabe fix(tenancy): read canonical X-Project-Id for project scope (was X-IAM-Project-Id)
evalsvc.tenant() read `X-IAM-Project-Id`, which nothing sends — console2
stamps the canonical `X-Project-Id` — so a selected project always fell
through to org-level and project selection scoped ZERO backend calls.
Switch the reader to `X-Project-Id` (one canonical header, per the
one-way rule); resolveKeys already accepts the project slug. Update the
identity-sanitizer comment to name the canonical sub-scope.

Build green, eval + identity tests pass.
2026-07-01 09:51:39 -07:00
hanzo-dev 2772cc4623 refactor(console): one console path, no build tags — delete dead clients/console
The repo carried TWO console embeds from parallel work: webui.go (wired in
Serve via mountConsole, no build tag — the working one) and clients/console
(a second take gated by //go:build cloud, imported by nobody after the earlier
build fix). The tag hid it from a normal build and the stale bundle doc comment
made it look like the whole binary needed -tags cloud. It never did.

- Delete clients/console/ — dead duplicate (unimported, tag-excluded, would
  double-mount "/"). Zero //go:build cloud tags remain in the repo.
- subsystems.go: correct the doc comment — subsystems register unconditionally;
  plain `go build ./cmd/cloud` (no tags) links and mounts the full set. Drop the
  now-stale clients/console note.

One console path (webui.go), builds normally. Verified: go build ./cmd/cloud
(no tags) → 427MB binary, go vet clean.
2026-07-01 09:50:02 -07:00
hanzo-dev bbf5094621 fix(deps): restore 5 drifted luxfi go.sum h1 hashes (canonical sum.golang.org)
The audit-trail commit f9b400cb rewrote 5 luxfi h1: module-zip hashes
(age, keys, pq, precompile, zap) to non-canonical values recorded under a
local GOPRIVATE/GONOSUMDB env, breaking 'go mod download' in the release
Docker build (checksum-DB SECURITY ERROR). go.mod is byte-identical to the
last-green build 55eef30e, so restore its go.sum. All 5 now match
sum.golang.org. No code change.
2026-07-01 09:23:05 -07:00
hanzo-devandGitHub f9b400cb8c feat(cloud): compliance-grade audit trail — tamper-evident, append-only, live (v1.785.34) (#51)
* fix(deps): reconcile 5 drifted luxfi go.sum hashes (age keys precompile pq zap)

These 5 luxfi modules were re-published under the same version tags (monorepo
re-tag); every local + CI module cache and the proxy agree on the new zip
hashes while the committed go.sum still pinned the old ones, so ALL builds fail
with a checksum-mismatch SECURITY ERROR. Reconcile go.sum to the hashes every
source agrees on (what go mod tidy would write). GONOSUMDB already trusts luxfi
(fetch direct). Pre-existing drift, orthogonal to the audit feature; needed to
build.

* feat(cloud): compliance-grade audit trail — tamper-evident, append-only, live

FedRAMP AU-* / SOC 2 CC-* audit control for the unified cloud binary. Every
security-relevant request against this binary is captured as a structured,
hash-chained record in an append-only store the app can only INSERT into, and
queryable through the global-admin-gated /v1/admin/audit surface.

WHAT

- audit/ package (record + chain + store + redact + query/verify), no route
  knowledge, pure security logic:
  * record.go — the AU-3 event model (actor/action/resource/auth/outcome/
    source-ip/ua/request-id/before-after) + the hash-chain math:
    hash = SHA256(canonical(record, hash+prevhash zeroed) || prevHash),
    genesis-anchored, DRY (add a field → covered by the hash automatically).
  * store.go — a single serialized Recorder (mutex + SQLite MaxOpenConns(1))
    owning the chain head; INSERT-only SQLite primary ({DataDir}/audit.db,
    zero-loss, synchronous) + optional best-effort ClickHouse mirror. Restart
    recovers the head so the chain continues (never forks).
  * query.go — filtered Query (parameterized; org/actor/action/resource/result/
    time) + Verify (walks the chain, recomputes every hash, reports the exact
    seq where a tamper/delete/reorder first breaks it).
  * redact.go — secret-key denylist (deny-by-key-name, recursive, fail-closed)
    for the before/after an explicit emit point supplies.

- audit_middleware.go (cloud pkg) — the ONE place every security-relevant
  request is recorded (decomplected: one predicate, every route). Sits AFTER
  SanitizeIdentity (validated, unforgeable actor/isAdmin) and BEFORE BillingGate
  (so billing 402/503 + admin 403 denials are audited too). Captures METADATA
  ONLY — never request/response bodies — so a secret in a body can't leak.
  Records mutations + all /v1/admin/* + all 401/403. Fails the request CLOSED
  (503) if the trail write fails (AU-5). Resolves the effective status from a
  returned *zip.HTTPError so error-returning denials are audited.

- audit_mirror.go — ClickHouse MergeTree OLAP mirror (insert-only by engine),
  best-effort projection for fleet retention/query. Driver already in go.mod.

- clients/admin/audit.go — rewires GET /v1/admin/audit to cloud's REAL store
  (was an IAM get-records proxy; kept as a federated fallback) + adds
  GET /v1/admin/audit/verify. Both behind the existing global-admin s.guard.

TESTS (all real, on-disk SQLite, no mocks)
- audit/: chain seals+links, verify passes clean, DETECTS field-tamper /
  deletion / reorder (out-of-band UPDATE/DELETE on a 2nd connection), restart
  continues the chain, concurrent appends stay gapless+verified (-race), redact
  strips secrets + fails closed, SQL-injection filter is inert.
- cloud/: middleware records a mutation with validated identity, audits a 403
  denial, SKIPS safe reads, audits admin reads, NEVER captures a secret-bearing
  body, no-op when unconfigured, fails closed on write error.
- clients/admin/: /v1/admin/audit returns real records + integrity summary,
  filters, verify endpoint, 403 without global-admin (no data leak), nil-store
  fallback.

THREAT MODEL
- Forge actor/admin: impossible at request level (SanitizeIdentity strips
  X-User-IsAdmin, actor from validated JWT).
- Forge the chain: an out-of-band edit re-hashes differently; keeping the chain
  valid requires recomputing the whole suffix — bounded by an externally-pinned
  head (Head()) for AU-9 (tail-truncation detection). Documented.
- Skip the middleware: mounted at the compose root before MountAll; the /zap
  plane replays through the same Fiber app (all middleware), so no bypass.
- Fail-closed is POST-RESPONSE: prevention is the AC layer (runs before the
  action); the trail is detection/accountability. Documented precisely.

Store is a compliance control: empty DataDir is a hard boot error unless
CLOUD_AUDIT_DISABLED=true (explicit opt-out). Secrets from env/KMS only; no
plaintext credential ever enters a record.

* harden(audit): scrub credential-shaped path segments + expand redaction denylist

Defense-in-depth from self-review before adversarial handoff:
- scrubCredentialSegments/scrubToken: a token that ever rides in a URL PATH
  (an hk-/sk-/pk-/fw_/hz_ key, reusing isAPIKey) is replaced with a marker in
  both Record.Path and resource.ID, so a secret in the path is never recorded
  verbatim. Normal identifiers (:name/:slug/:id/uuid/numeric) pass through.
  Proven by TestAudit_ScrubsCredentialInPath.
- Redaction denylist gains passphrase, privkey, social_security, phrase (covers
  seedPhrase/recoveryPhrase) — closing the key-name gaps found by enumerating
  real credential field names. TestRedact_StripsSecrets now asserts them.

* harden(audit): close raw-secret leak in path/resource-id/user-agent

Self-review PoC found a real residual leak beyond prefixed keys: a raw
high-entropy secret (64-hex, or a JWT) in the URL path — and a bearer/key in
the client User-Agent — were recorded verbatim (isAPIKey only matched
hk-/sk-/pk-/fw_/hz_ prefixes). Closed:
- scrubToken now also catches JWTs (eyJ + two dots) and long unbroken
  high-entropy alphanumeric runs (>=32, mixed, no separators) — a raw API
  key/hex secret. UUIDs (hyphens), slugs, names, emails, numeric ids pass
  through (TestScrubToken_NoFalsePositives).
- scrubFreeText scrubs credential-shaped words from the User-Agent (splits on
  space/=/;/,) and caps length at 512. Normal UA prefix preserved.
Proven: TestAudit_ScrubsCredentialInPath (prefixed+raw-hex+JWT),
TestAudit_ScrubsSecretInUserAgent. Query-string secrets already safe (c.Path()
excludes the query string).

* fix(audit): close audit-evasion via /health suffix on mutations (SECURITY)

Self-review PoC found a real evasion: isSecurityRelevant skipped ANY path
ending in /health, so a mutating POST /v1/admin/orgs/x/health (wildcard route
or an attacker-named segment) slipped past the audit trail entirely — the
worst class of bug for a compliance control (silent bypass).

Fix: check the unconditional security signals FIRST and without exception — a
401/403 denial, any /v1/admin/* call, and any POST/PUT/PATCH/DELETE are ALWAYS
audited whatever the path. Only after that is a safe read dropped (all safe
reads, incl. liveness probes, are request-log noise → not recorded). The
suffix-based /health exemption is gone; it can no longer suppress a mutation.
Proven by TestAudit_HealthSuffixCannotEvadeAudit (POST .../health IS audited;
GET /v1/kms/health is not) and the unchanged TestAudit_SkipsSafeReads.

* harden(audit): no false-attribution — anonymous request records no org/sub

Self-review: SanitizeIdentity's Phase-1 residual restores a client-supplied
X-Org-Id for the data path, so an UNAUTHENTICATED attacker sending
X-Org-Id: victim-org could stamp an audit event with a victim's org (false
attribution), even though X-User-Id/IsAdmin are correctly stripped.

Fix: actorFromCtx gates the recorded actor on a VALIDATED principal — a
non-empty c.User() (X-User-Id, which SanitizeIdentity sets only from a verified
JWT). With no validated sub (anonymous, or an invalid/garbage bearer that failed
validation), the actor is recorded EMPTY: the event stands as an honest
anonymous mutation identified by SourceIP, never mis-attributed to a claimed
org. With a validated sub, org/sub/email are authoritative.
Proven by TestAudit_AnonRequestNotAttributedToForgedOrg (runs the real
SanitizeIdentity ahead of AuditTrail).

* fix(audit): close 4 scrub-bypass classes from Red review (MEDIUM)

Red's adversarial review found the path/UA credential scrub (5dfdf0e4) had
4 bypass classes that let a secret reach the immutable trail:
  1. base64url with -/_  (looksLikeHighEntropyToken rejected any non-alnum)
  2. all-alpha opaque >=len  (required digits>0)
  3. percent-encoded prefix  (hk%2D… defeated the isAPIKey match)
  4. UA glued by :/()[]  (scrubFreeText split on too few delimiters)

Fixes:
- looksLikeHighEntropyToken now accepts the FULL base64url alphabet
  [A-Za-z0-9_-] (RFC 4648 §5), drops the digit requirement, exempts dotted
  values + canonical UUIDs, threshold lowered to 24 (128-bit base64 / 24-hex).
- scrubToken percent-decodes before every credential test (url.PathUnescape),
  so %2D/%5F can't hide structure.
- scrubFreeText tokenizes on a broad delimiter superset (= ; , : / \ ( ) [ ]
  { } " ' < > | & ?) and rebuilds in one pass preserving delimiters —
  replacing the fragile strings.ReplaceAll.

Proven by TestScrubToken_RedReviewBypassClasses (all 4 classes + UA glue) and
the expanded TestScrubToken_NoFalsePositives (uuids, model names like
claude-opus-4-20250514/text-embedding-3-large, slugs, normal UAs unchanged).

* feat(audit): operationalize AU-9 tail-truncation anchor (Red LOW #2)

Red: the Head() pin is inert unless operationalized — a hash chain can't detect
that the last K records were deleted (the surviving prefix self-verifies); only
an independent, durable head-digest series catches the count regression.

Adds a checkpoint emitter to the Recorder:
- StartCheckpoints(interval, logFn): a periodic goroutine emits the head digest
  {count, head, ts} to the append-only observability log (o11y) every interval
  (CLOUD_AUDIT_CHECKPOINT_INTERVAL, default 5m), plus a FINAL checkpoint on
  Close so the shutdown head is anchored.
- CheckpointSink: when the ClickHouse mirror implements it, the digest is ALSO
  persisted to an INDEPENDENT audit_log_checkpoints table (MergeTree) — so
  truncating the local SQLite chain cannot rewrite the anchor history.
- Detection (compare consecutive checkpoints, alert on count regression) lives
  in o11y where alert rules belong; the binary emits the tamper-evident anchor
  to an independent sink. /v1/admin/audit/verify already returns (count,head) as
  the pollable anchor too.

Proven: TestCheckpoint_EmitsHeadDigest (log + independent sink get count=7 head
on Close), TestCheckpoint_CountMonotonicDetectsTruncation (delete tail → prefix
still self-verifies, but count regresses 10→6 = the o11y alert signal).
Race-clean.

* harden(audit): close dotted-exemption + standard-b64 + nested-encoding scrub gaps

Self re-review (pre-empting Red's scoped re-review) found the round-1 scrub fix
still had gaps: the dotted-exemption let a raw secret bypass by appending '.x',
standard-base64 tokens (with +/) slipped, and nested percent-encoding (%252D)
survived a single decode.

Rewrote looksLikeHighEntropyToken to SCAN for the longest UNBROKEN base64-ish
run (>=24) anywhere in the value, over BOTH url-safe (-/_) and standard (+//)
alphabets — so '<rawsecret>.x' still trips (pre-dot run >= 24) and a standard-
base64 secret is caught. UUIDs stay exempt; real slugs/model-names/filenames
(report.pdf, text-embedding-3-large) have no 24-char run so they pass.
percentDecode now iterates (bounded x3) to normalize nested encodings.

All bypass classes closed (Red's 4 + dotted/standard/double-encoded), zero
false positives — TestScrubToken_RedReviewBypassClasses + NoFalsePositives
extended. The 20-char short-secret is a deliberate non-match (lowering below 24
would over-scrub legit hex-ish ids).

* fix(audit): address Red re-review — model-id over-scrub, UA glue, checkpoint (2 MED + 1 LOW)

Red re-review of the scrub/checkpoint code found 2 MEDIUM + 1 LOW:

MEDIUM 1 — model-id over-scrub (AU-3 regression): the round-3 run-scanner
counted '-' as a token char, so hyphenated model ids (claude-3-5-sonnet-
20241022, 26-char run) were redacted on audited routes (PATCH/DELETE
/v1/ml/models/:name, /v1/admin/catalog/models/*) — an auditor lost WHICH model
changed. Fix: isHighEntropyRunChar EXCLUDES '-' (kept +/_ for base64). Model
ids break into short runs (max ~9, far under 24); real secrets stay unbroken
>=24 runs — even a url-safe token using '-' as a separator has a >=24 run on one
side (AbCdEf-GhIjKl_MnOpQrStUvWxYz012345 -> 27). 6 model ids added to
TestScrubToken_NoFalsePositives.

MEDIUM 2 — UA free-text bypass: isFreeTextDelimiter omitted . @ # ~, so a
prefixed key glued by them (client@sk-live-KEY) stayed one token whose prefix
was no longer sk-/hk-. Fix: add . @ # ~ to the delimiter set; also flag a lone
eyJ-prefixed JWT header segment regardless of length (a JWT header is never a
legit id). Real UA dots are version separators (<24, safe). Fixed the stale
isFreeTextDelimiter docstring that referenced a nonexistent exemption. Proven by
4 glue-char probes in TestScrubToken_RedReviewBypassClasses.

LOW — checkpoint robustness: (a) StartCheckpoints now guards double-start with a
 flag (the field/WaitGroup write was -race-flagged on a 2nd call) and
the docstring is corrected; (b) the on-Close final checkpoint to the independent
sink is now SYNCHRONOUS with a bounded 5s ctx (was fire-and-forget — the AU-9
independent anchor could be stale exactly at shutdown when an attacker truncates).
Proven by TestCheckpoint_DoubleStartIsSafe (-race) + TestCheckpoint_CloseSyncsToSink.

34 tests green, race-clean, -tags cloud.

* harden(audit): structured-id exemption beats hyphen-exclusion (11%->0.09% token bypass)

The prior fix (exclude '-' from the entropy run to protect model ids) opened an
~11% bypass for 32-byte url-safe-base64 secrets whose '-' happened to break
every 24-run (measured over 10k random tokens). Excluding '-' was too blunt.

Better construction: INCLUDE '-' in the run alphabet again (so a base64url token
embedding '-' is caught by its run), but exempt STRUCTURED IDs up-front via
isStructuredID — a value with >=3 hyphen groups where every part is <=12 chars
(dictionary words / short numbers: claude-3-5-sonnet-20241022). A raw secret
does not decompose that way. Measured: 0 model over-scrub, 0.09% residual on
32-byte base64 tokens that randomly resemble an id AND are prefixless AND sit in
a URL path (real keys carry hk-/sk- prefixes caught by isAPIKey; JWTs by
looksLikeJWT). An interior-hyphen raw secret with LONG parts still redacts.

Proven: TestScrubToken_NoFalsePositives (12 model ids/slugs pass) +
TestScrubToken_RedReviewBypassClasses (interior-hyphen long-part secret redacts).
34 tests green, -tags cloud.

* fix(audit): lexical structured-id test closes Red MEDIUM + guard started race (LOW)

Red final re-review: isStructuredID was SHAPE-only (>=3 hyphen groups, parts
<=12) — attacker-satisfiable. A secret chunked to that shape
(AbCdEfGhIjKl-MnOpQrStUvWx-YzAbCdEfGhIj, or deadbeef-cafebabe-01234567-89abcdef)
was exempted; ~2.15% of random 128-bit tokens leaked by chance. Entropy-count
alone can't separate them (deepseek-r1-distill-qwen-32b has 24 non-hyphen chars,
same as a 128-bit secret).

Fix: isStructuredID now requires every group to be WORD-LIKE (isWordLikeGroup) —
lexical content, not shape. A group is rejected if it is dense MIXED-CASE (base64
chunk) or a long ALL-HEX-WITH-LETTERS run >=8 (hex chunk like deadbeef); an
all-digit version date (20241022) stays word-like. Measured: 0 model over-scrub,
0 crafted-attacker bypass, natural random-token leak 0.0004% (128-bit) / 0%
(192-bit+) — down from 2.15%. Real keys (hk-/sk- prefix) and JWTs are caught
regardless.

LOW: StartCheckpoints' started check-and-set now under r.mu (was -race-dirty on
a concurrent 2nd call; prod-unreachable but now clean).

Proven: TestScrubToken_RedReviewBypassClasses (4 chunked-secret classes redact) +
TestScrubToken_NoFalsePositives (17 model ids/slugs pass). 34 tests green, -race,
-tags cloud.

* harden(audit): two-stage detection + document single-case-chunk residual bound

Restructured looksLikeHighEntropyToken into two stages after tracing the
fundamental limit Red is probing:
- Stage 1 (UNCONDITIONAL): a >=24 UNBROKEN run over [A-Za-z0-9_+/] (hasHighEntropyRun,
  '-' and '.' are separators). Catches every raw secret WITHOUT internal separators
  (hex, base64) at 100% — the realistic 'client bug put a raw key in the URL' case.
  Never over-scrubs a hyphenated id (runs are short).
- Stage 2 (separated values): exempt a lexical structured-id (isStructuredID: >=3
  word-like hyphen groups); otherwise redact. Catches mixed-case/hex chunks.

ACCEPTED RESIDUAL BOUND (documented in code): a secret deliberately chunked into
>=3 single-case-ALPHABETIC groups <=12 chars is lexically indistinguishable from
a hyphenated model id (deepseek-r1-distill-qwen-32b carries the SAME 24-char
entropy budget) — NOT closable by any length/case/count rule without a word
dictionary (over-engineering for a defense-in-depth URL/UA backstop). It does not
widen exposure for any REAL credential: Hanzo keys are prefixed (isAPIKey, any
length), JWTs are eyJ-prefixed (looksLikeJWT), bodies are never read. The residual
is an adversary chunk-encoding their OWN secret into a URL to seed an admin-only
audit row — contrived, low-value. The realistic accidental leak (unbroken raw key)
is caught by stage 1.

Removed dead isHighEntropyRunChar. 34 tests green, -race, -tags cloud.

* feat(paassvc): native in-process PaaS deploy control plane (/v1/paas/*)

Port the standalone Dokploy platform's observe + deploy halves into the unified
cloud binary as clients/paassvc — the 'one and only one way to deploy' made
native. Follows the clients/ml pattern exactly: a self-contained dynamic k8s
client, cloud.Register'd from init() (order 128), global-admin-gated, fail-closed
when no cluster.

Surface (global-admin only; user-facing view lives in console2):
  GET  /v1/paas/apps             fleet drift board (declared/running/latest/drift+health)
  GET  /v1/paas/apps/:app        one service row by CR name (main->test->dev)
  POST /v1/paas/apps/:app/deploy deploy a tag by merge-patching Service CR .spec.image
  GET  /v1/paas/health           real k8s reachability + Service CRD probe

- drift.go: 1:1 port of apps-drift.ts (computeDrift/isSemverTag + 6 DriftKinds,
  identical severities). Pure, zero IO.
- paas.go: observeFleet lists hanzo.ai/v1 services across hanzo/-testnet/-devnet
  (inventory.ts DEFAULT_TARGETS), joins the live Deployment for the running tag
  (the operator Service CR status does NOT surface the running image — confirmed
  against the live CRD), health/phase/endpoints from the reconciled CR status.
  deploy merge-patches .spec.image (deploy-executor.ts parity) -> operator rolls it.
- Stateless: reads the cluster live (CRs are the source of truth); no apps-table
  copy, no cron readers (dropped vs the Node platform).

Tested green:
- 30+ unit cases incl. the 9 drift-contract cases ported verbatim; go test green,
  gofmt clean, go vet clean, full cloud binary builds (-tags cloud).
- Live-cluster probe (paasintegration tag, PAAS_IT-gated): observeFleet returned
  82 real rows matching kubectl; an idempotent same-image patch on pricing
  round-tripped through the operator with generation unchanged (6->6) = write
  path proven WITHOUT triggering a rollout, zero disturbance to live state.

Design + full port map: universe/docs/architecture/paas-in-cloud.md.
RBAC (cloud-paassvc -> cloud-api SA): universe infra/k8s/cloud/paassvc-rbac.yaml.

Additive: platform.hanzo.ai stays as the internal-admin console; this is the
native backend + (next) the console2 user UI. No forced retirement.

* polish(audit): close hex-chunk residual (hex rule 8->4) + correct residual doc

Red final review (SHIP verdict) flagged 2 non-blocking polish items:
1. Lower isWordLikeGroup hex rule from len>=8 to len>=4 — Red verified across 19
   real model ids that NONE has an all-hex-with-letters group of len>=4, so this
   closes the small-hex-chunk leak (md5/sha in 'xxxx-xxxx' display grouping:
   abcd-ef01-2345-6789-…) at 100% with ZERO model-id over-scrub. hexChunkMinLen=4.
2. Correct the residual doc: the accepted bound is now ONLY single-case-ALPHABETIC
   base32 chunks (lowercase/uppercase-only, no hex-letter runs >=4) — mixed-case
   base64 AND all hex-chunk sizes are now caught. The remaining case is genuinely
   unclosable without a word dictionary and exposes no real credential.

Proven: TestScrubToken_RedReviewBypassClasses now includes 4-char hex groups
(abcd-ef01-…) which redact; TestScrubToken_NoFalsePositives (19 model ids) still
pass. 36 tests green, -race, -tags cloud. Red re-review: none needed.
2026-07-01 07:22:39 -07:00
hanzo-devandGitHub 55eef30e3b feat(paassvc): native in-process PaaS deploy control plane (/v1/paas/*) (#52)
Port the standalone Dokploy platform's observe + deploy halves into the unified
cloud binary as clients/paassvc — the 'one and only one way to deploy' made
native. Follows the clients/ml pattern exactly: a self-contained dynamic k8s
client, cloud.Register'd from init() (order 128), global-admin-gated, fail-closed
when no cluster.

Surface (global-admin only; user-facing view lives in console2):
  GET  /v1/paas/apps             fleet drift board (declared/running/latest/drift+health)
  GET  /v1/paas/apps/:app        one service row by CR name (main->test->dev)
  POST /v1/paas/apps/:app/deploy deploy a tag by merge-patching Service CR .spec.image
  GET  /v1/paas/health           real k8s reachability + Service CRD probe

- drift.go: 1:1 port of apps-drift.ts (computeDrift/isSemverTag + 6 DriftKinds,
  identical severities). Pure, zero IO.
- paas.go: observeFleet lists hanzo.ai/v1 services across hanzo/-testnet/-devnet
  (inventory.ts DEFAULT_TARGETS), joins the live Deployment for the running tag
  (the operator Service CR status does NOT surface the running image — confirmed
  against the live CRD), health/phase/endpoints from the reconciled CR status.
  deploy merge-patches .spec.image (deploy-executor.ts parity) -> operator rolls it.
- Stateless: reads the cluster live (CRs are the source of truth); no apps-table
  copy, no cron readers (dropped vs the Node platform).

Tested green:
- 30+ unit cases incl. the 9 drift-contract cases ported verbatim; go test green,
  gofmt clean, go vet clean, full cloud binary builds (-tags cloud).
- Live-cluster probe (paasintegration tag, PAAS_IT-gated): observeFleet returned
  82 real rows matching kubectl; an idempotent same-image patch on pricing
  round-tripped through the operator with generation unchanged (6->6) = write
  path proven WITHOUT triggering a rollout, zero disturbance to live state.

Design + full port map: universe/docs/architecture/paas-in-cloud.md.
RBAC (cloud-paassvc -> cloud-api SA): universe infra/k8s/cloud/paassvc-rbac.yaml.

Additive: platform.hanzo.ai stays as the internal-admin console; this is the
native backend + (next) the console2 user UI. No forced retirement.
2026-07-01 07:19:32 -07:00
hanzo-dev 6712290e1f fix(cloud): address Red review — exact tenant keying + content caps
Red HIGH-1 (cross-tenant CRUD, PROVEN): the isolation key used a lossy
sanitizeOrg (lowercase/punct->'-'/32-char truncate) so distinct IAM owners
(acme/ACME/acme!/32-char-prefix) collapsed into one storage bucket. Fixed:
tenant() now keys on the EXACT validated org from SanitizeIdentity — never
normalized. Removed the magic 'admin' bucket (empty org -> 403, even for
admins). sanitizeOrg deleted from the key path (kept only as a cosmetic,
clearly-labeled namespace normalizer in functions).

Red MED-1 (46MB response amplification): prompt content now capped at 64KiB;
version history is bounded, metadata-only (no per-version content echo);
metrics uses a true COUNT.

Red INFO: >900s timeout now clamps to 900 (was reset to 30) in create+invoke.

Red's three adversarial tests INVERTED into regression guards that assert
isolation HOLDS: TestRed_OrgKeyExactIsolation (8 collision classes, all
isolated), TestRed_NoAdminBucketConfusion, TestRed_PromptContentCapped.
All suites green under -race; fused subsystems graph links.
2026-07-01 07:12:33 -07:00
hanzo-dev 52c44a0c8c fix(build): main was broken — clients/console import excluded by //go:build cloud
Commit 625c234 added clients/console (a second, //go:build cloud-gated take on
the go:embed console) and imported it unconditionally in the subsystems bundle.
The default build (go build ./cmd/cloud, no -tags) excludes those files, so the
whole binary failed: 'build constraints exclude all Go files in clients/console'
— which also fails the CI image build, blocking every deploy.

The working, wired console embed is webui.go's mountConsole (called from Serve
after all /v1 routes). clients/console is redundant with it and would double-mount
'/'. Drop the broken import so main builds; consolidating onto ONE console path
is a clean follow-up.

Verified: go build ./cmd/cloud → 426MB binary; booted it and the ONE process
serves console '/' (200 HTML), SPA fallback /gpus (200 HTML), /v1/metrics/health
(200), /v1/nope (503 non-HTML — decline-list holds), /healthz (200).
2026-07-01 06:40:13 -07:00
hanzo-dev 8b6451ed10 feat(cloud): mount per-org /v1/{prompts,agents,functions} product subsystems
Three native HIP-0106 subsystems, each org-scoped by the gateway-minted
X-Org-Id (SanitizeIdentity trust boundary), Base/SQLite in DataDir, secrets
by KMS reference only. Follows the projectsvc template; wired via one
additive block in subsystems/subsystems.go (orders 126/127/128).

- prompts   /v1/prompts/*    versioned prompt library (create=new version)
- agents    /v1/agents/*     agent defs + real run via deps.AI, recorded runs
- functions /v1/functions/*  serverless registry + invocations/metrics;
                             invoke delegates to the code-exec sandbox and
                             fails closed (503) when unconfigured — never
                             runs tenant code in-process, never fabricates.

Tenant isolation proven at the store AND HTTP layers (Fiber app.Test):
no-org -> 403, cross-org list empty, cross-org get/delete/run -> 404.
All suites green (CGO=1).
2026-07-01 06:39:31 -07:00
hanzo-dev 625c234e14 feat(console): go:embed the console2 SPA into cloud — the one-binary foundation
Hanzo V8: Open Edition. cloud/clients/console go:embeds dist/ (the console2 static
export) and mounts the SPA at "/" with SPA-fallback, order 990 — the last-resort
catch-all AFTER every /v1/* route (isAPIPath refuses to HTML-fallback /v1,/zap,/_,
/healthz so JSON clients get honest 404s). Registered in subsystems.go.

This is the seam that makes ONE Go binary the whole cloud — edge + gateway + every
subsystem + the frontend. Placeholder dist/index.html is overwritten by the
console2 static-export bundle at image-build time. Build verified: go build -tags
cloud ./clients/console/ clean.
2026-07-01 05:59:48 -07:00
hanzo-devandGitHub 2d89b8d4f7 feat(cloud): embed + serve the console UI from the ONE binary (go:embed) (#50)
One artifact, one origin: the same hanzoai/cloud binary now serves the
console (@hanzo/gui, from hanzoai/console2) at the web root AND the /v1 API
from one process — no separate console Service, no second origin. Flagship
OSS-cloud consolidation (HIP-0106).

Serve (webui.go)
- The console is compiled in via `//go:embed all:webui/dist` and mounted as
  the app's TERMINAL catch-all in Serve — LAST, after every /v1 subsystem
  route, the /zap plane, and the health contract. Fiber v3 matches in
  registration order, so real API routes always win; only paths that match
  nothing else reach the SPA.
- SPA fallback: `/` and any client-side route (`/orgs`, `/models`, …) serve
  index.html (Cache-Control: no-cache) so deep links / reloads work.
  Fingerprinted assets (assets/, _next/) are served immutable for a year,
  with brotli/gzip precompressed-sibling negotiation when the build emits
  .br/.gz. Served through a stdlib http.Handler (correct Content-Type,
  conditional GET) adapted onto zip via zip.AdaptNetHTTP.
- API precedence + namespace safety: an UNMATCHED path under an API/ops
  prefix (/v1/, /zap, /healthz, /readyz, /metrics) returns a real 404 — never
  the SPA shell — so clients calling a mistyped /v1/… never get HTML 200.
- Same-origin: the embedded console calls /v1 on its own host; the session
  cookie is first-party — no CORS, no second-origin token dance.
- Reuses the hanzoai/static plugin's SPAMode semantics; implemented in-binary
  because static.Handler is disk/S3-only today (its New() takes a Root/S3
  bucket, not an fs.FS) so it can't serve an embed.FS — teaching it fs.FS is
  the clean follow-up to collapse onto the shared plugin.

Build pipeline (Dockerfile)
- New `console` stage builds the console2 static bundle → /out; the Go build
  overlays it into webui/dist BEFORE `go build` so go:embed bakes it in.
- webui/dist/index.html is a committed fallback shell (a real same-origin /v1
  bootstrap) so `go build` always compiles and the binary always serves a UI
  even without the Node toolchain; the image build overwrites it with the real
  console. Built assets are .gitignore'd — generated at build time, never
  committed as source.

Tests (webui_test.go) — boot the app + assert, end-to-end via app.Fiber().Test:
GET / → shell; deep links → shell 200 (not 404); /v1/models → API (not SPA);
unmatched /v1/… → 404 (not HTML); assets served directly; HEAD; and path
traversal (../, %2e%2e) cannot escape the embed FS. 7/7 green.

Honest current state: console2 ships 15 Next server route handlers
(app/**/route.ts, KMS-token proxies) so it emits a Node server bundle, not a
static export — the image embeds the fallback shell until console2 exposes a
build:embed static target or those routes land here as native /v1 endpoints.
The Go embed/serve plumbing is complete and needs no change to light up the
full console the moment the static bundle exists.

Drive-by: brand_test.go asserted the pre-pin hanzo issuer (iam.hanzo.ai);
brand.go was pinned to hanzo.id in 21ac43f1, so the test was stale — aligned
to the shipped behavior (brand.go unchanged). Root package: 34/34 green.
2026-07-01 05:53:35 -07:00
hanzo-dev 3834cc1bbd fix(deps): re-record 3 drifted luxfi go.sum hashes (age@v1.5.0 +2) — canonical proxy; unblock release build 2026-06-30 22:23:20 -07:00
hanzo-dev db3b156bd0 refactor(o11y): forward path verbatim — no /api/ rewrite
The o11y fork now registers its routes at their exact public path (/v1/o11y/*),
so the reverse proxy forwards unchanged — removed rewritePath (/v1/o11y→/api) and
the TestRewritePath test. One and one way: the route IS the path on both sides.
Cloud o11y tests pass.
2026-06-30 21:35:19 -07:00
zeekay b09052305b Merge branch 'feat/projects-store-and-deploy' 2026-06-30 20:18:54 -07:00
hanzo-devandGitHub 88cde76b90 feat(cloud): /v1/exec (Code Interpreter → sandbox) + /v1/websearch (Hanzo search+crawl) (#49)
* feat(cloud): /v1/exec (Code Interpreter → sandbox) + /v1/websearch (SearXNG+Firecrawl-compat over Hanzo search+crawl)

hanzo.chat's Run Code and Web Search agent tools speak fixed LibreChat
provider contracts. cloud-api is the single /v1 edge, so it owns those
surfaces and routes them to Hanzo's own infra — never an external SaaS.

- clients/exec (order 140): mounts /v1/exec, /v1/exec/*, /v1/upload,
  /v1/download/*, /v1/files/* — the @librechat/agents CodeExecutor contract
  (POST /exec {lang,code} X-API-Key -> {stdout,stderr,files}). Transparent
  reverse proxy to a SANDBOXED executor (CODE_EXEC_UPSTREAM). NO os/exec here;
  the executor is the isolation boundary. X-API-Key (CODE_EXEC_API_KEY, KMS)
  enforced constant-time, fail-closed.
- clients/websearch (order 141): mounts /v1/websearch/search (SearXNG JSON,
  proxied to a Hanzo-operated metasearch WEBSEARCH_UPSTREAM) and
  /v1/websearch/v1/scrape (Firecrawl shape, backed by Hanzo Crawl/Crawl4AI —
  {url}->{success,data:{markdown,metadata}}). WEBSEARCH_API_KEY (KMS).
- Both register before ai (150) so their specific paths win over ai's /v1/*
  catch-all. Mirrors the clients/o11y reverse-proxy pattern.

Tests: proxy verbatim-forward, path rewrite, auth fail-closed/reject,
crawl->firecrawl shape adaptation. All green.

* test(cloud): mount-through-Fiber integration tests for exec + websearch

Prove Mount() registers the overlapping static+wildcard routes (/v1/exec &
/v1/exec/*, /v1/websearch/*) on a real zip/Fiber router without panicking,
and that requests route end-to-end through the router to the guarded
handlers (proxy forward, firecrawl-shaped scrape, auth reject). Closes the
gap where direct-handler tests bypassed route registration.
2026-06-30 18:39:23 -07:00
hanzo-dev 18b7577017 refactor(cloud): adminsvc → admin (drop svc, one word) — consistency with the svc-drop 2026-06-30 17:55:00 -07:00
hanzo-dev 2f6c824f3a refactor(cloud): drop the svc suffix — one word per subsystem client
o11ysvc→o11y, evalsvc→eval, mlsvc→ml, plansvc→plan, pluginsvc→plugin,
pricingsvc→pricing, productsvc→product, provisioningsvc→provisioning. The suffix
was stutter (svc = service). Package name == dir == the bare noun now.

o11y is `package o11y` importing `github.com/hanzoai/o11y` with a PLAIN import (no
alias): the import name is file-scoped and you never qualify your own package, so
`o11y.SetHandler` resolves to the upstream — the local `upstream()` URL func is
untouched. subsystems.go import paths + gojahost comment updated. Renamed packages
+ subsystems build clean.
2026-06-30 17:52:59 -07:00
hanzo-devandGitHub 9df0efbc2c feat(adminsvc): god-mode /v1/admin/* surface for admin.hanzo.ai console (#48)
Aggregator facade mounting the /v1/admin/* surface the Hanzo Admin Console
(admin.hanzo.ai, apps/operator) calls, matching its api.ts contract
field-for-field. Fans out over HTTP to the real upstreams — IAM (orgs, users,
roles, applications, audit, me), commerce (spend, credits), o11y (health) —
exactly like the o11ysvc/productsvc read facades; holds no store of its own.

Every route is GLOBAL-ADMIN ONLY, fail-closed: the guard reuses c.IsAdmin(),
which after SanitizeIdentity is true only for a JWT-validated principal whose
org is the admin org (IAM's IsGlobalAdmin), matching the gateway's admin-guard.
Anonymous and tenant-admin callers are denied 403 on every route (regression
locked in TestGate_DeniesEveryRoute). The IAM fan-out replays the caller's own
cookie/bearer — no adminsvc service credential — so it never reads more than the
caller could, and IAM re-checks IsGlobalAdmin. Commerce uses the existing
KMS-synced COMMERCE_SERVICE_TOKEN; no secret is hard-coded or logged.

Panels with no in-binary feed yet return the honest empty state, never a
fabricated number: the usage timeseries + per-product breakdown (insights/
datastore) and the product/workload registry + infra tiles (platform apps
table). The operator renders these as empty/em-dash by design.

Endpoints: overview, orgs, users, roles, applications, audit, usage, products,
me, sync. Registered order 146; blank-imported in subsystems.go.

Tests: gate denial across all routes x anonymous/tenant-admin/tenant-user, gate
allow for global admin, real aggregation (orgs/users/overview/usage) against
mock IAM+commerce, credential-replay assertion, IAM-error-surfaced (not
fabricated), honest-empty series/products. go build ./... + go test green.
2026-06-30 17:48:43 -07:00
hanzo-dev 675257b127 fix(storagelock): a db NAME is not a backend — reject only real Postgres
`dbName=hanzo_cloud` alongside driverName=sqlite crash-looped cloud-api (fail-closed
on a benign leftover). Decomplect: the guard's one job is "reject Postgres" =
driverName=postgres OR a postgres:// DSN. A database name selects nothing, so drop
`dbName` from forbiddenEnvs entirely (the Go binary never reads it). Also correct the
lineage label: the legacy cloud-api is casibase (Go) — it lives on as hanzoai/ai,
which mounts INTO this hanzoai/cloud orchestrator — NOT "Python/TS". Tests updated:
dbName is never a violation; driverName=postgres + postgres DSN still are. Forwards
perfection, no backwards-compat leftover.
2026-06-30 17:32:17 -07:00
hanzo-devandGitHub 32138dbf0b chore(productsvc): top-level /v1 — drop residual /api/ prefix (#47)
Rename cloud-api's own public product routes from /api/<route> to
top-level /v1/<route>, per the openapi v1.0.0 lock-in (no /api/ prefix;
the subdomain is api.* so /api/ double-prefixes):

  /api/search-docs/indexes -> /v1/search-docs/indexes
  /api/search-docs/stats   -> /v1/search-docs/stats
  /api/vector/collections  -> /v1/vector/collections
  /api/vector/stats        -> /v1/vector/stats

These are the only /api/ paths cloud-api REGISTERS (serves). The remaining
/api/ literals are upstream calls cloud-api MAKES to other services that
genuinely serve /api/ — left untouched:
  - evalsvc: /api/public/* (Langfuse console API proxy targets)
  - o11ysvc: /v1/o11y/* -> /api/* runtime rewrite (destination)
  - pricingsvc: openrouter.ai/api/v1/models (external)

Hard cutover (no dual-serving, per no-backwards-compat). Coordinated with:
universe cloud-api-v1 AUTH_PUBLIC_PATHS, python-sdk + hanzo-docs RAG
clients, and the openapi cloud spec — all moving to /v1 together.
2026-06-30 17:06:27 -07:00
zeekayandhanzo-dev 71371fdff2 fix(deps): realign go.sum to current origin (luxfi force-re-tags) + integrate main (goa/pluginsvc); clear corrupted VCS cache
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-30 16:38:16 -07:00
zeekay 9a0c9768a7 Merge remote-tracking branch 'origin/main' into feat/projects-store-and-deploy 2026-06-30 16:22:28 -07:00
hanzo-dev 675336cec7 Merge commit 'c1647e32' into deploy/cloud-convergence 2026-06-30 15:48:06 -07:00
hanzo-dev 21ac43f13c fix(auth): pin hanzo IAM issuer to hanzo.id + reject missing exp (red FIX1/INFO)
FIX1 (red, HIGH — the deploy landmine): brand.go defaulted the `hanzo` brand
IAMIssuer to https://iam.hanzo.ai, but the live .well-known/openid-configuration
on BOTH hanzo.id and iam.hanzo.ai reports issuer=https://hanzo.id +
jwks_uri=https://hanzo.id/v1/iam/.well-known/jwks (iam.hanzo.ai is a routing
alias, not the token issuer). With the baked default, SanitizeIdentity's issuer
check would fail on every real token -> every principal anonymized -> ALL global
admin gets 403 (fail-secure, no forgery opened, but admin broken platform-wide).
The cloud CLI already defaults to hanzo.id; lux/zoo/pars already point at their
own .id issuers. Pin hanzo -> https://hanzo.id so the correct config is
default-by-default. (JWKS derivation then yields the correct hanzo.id JWKS.)

INFO (red): go-jose ValidateWithLeeway only enforces exp when present
(`if c.Expiry != nil`), so a token with NO exp would never expire. Reject a
missing exp explicitly, exactly like a missing iss. +1 test (22 green).

No subsystem reads cfg.IAMIssuer except SanitizeIdentity + a log line, so the
brand default change is contained.
2026-06-30 15:39:08 -07:00
hanzo-dev c1647e324b feat(billing): per-org fail-closed gate+meter for non-LLM provisioning + compute
Non-LLM resources were free: anyone could provision sql/vector/kv/s3/datastore/
docdb/search (provisioningsvc) and ml models/train jobs/experiments (mlsvc, GPU)
for $0 — only LLM calls were metered. This adds the same per-org commerce gate
the LLM edge gate uses, in-handler, so every create is paid for.

ONE shared primitive (no copy-paste per kind), reusing Deps.Metering (the single
commerce client) — the in-handler analogue of BillingGate:

  cloud.ResourceMeter (resource_billing.go)
    Gate(ctx, org, kind, costCents)   pre-create balance gate, fail-CLOSED
    Meter(org, kind, amountCents, …)  post-success debit, per-org, async
    DenyResource(c, err)              402 insufficient_balance / 503 unavailable
    ResourceFeeCents(prefix, kind)    configurable flat fee, $1.00 default

Wired into BOTH create paths (provisioningsvc + mlsvc) via the shared type:
gate runs after request validation and BEFORE any backend/k8s object is created
(no free provisioning, even on a commerce outage); meter runs only after the
resource is persisted/created.

Multitenancy (the whole point): org is the caller's resolved slug from tenant(c)
— the SAME value that namespaces the resource, now JWT-derived by the #66
identity sanitizer (not a spoofable header). It is sent to commerce as BOTH the
user identity AND X-IAM-Org-Id, OVERRIDING the client default org, so the balance
checked and the ledger debited are always the caller's own — never a default,
never another tenant. Proven by tests asserting commerce sees X-IAM-Org-Id:<caller>
(not the client default "hanzo") on both the balance check and the debit.

Env-aware (3-env split): the gate fires in EVERY env; test/dev are sandbox-but-
billed against their own per-env commerce/Square (structural, not a code branch).
Env is threaded config→deps as an attribution label and is NEVER a billing
bypass — proven by a test that testnet/devnet still refuse at zero balance.

Cost model: real, configurable per-kind flat fee (CLOUD_PROVISION_FEE_CENTS[_KIND],
CLOUD_COMPUTE_FEE_CENTS[_KIND]); 0 makes a kind free and un-gated; invalid/negative
is ignored so a typo can't silently free a paid resource. Ongoing storage GB-month
and GPU-hour reuse the SAME Meter primitive with a usage-derived amount from a
future runtime watcher — no live-size source here, so no size is fabricated.

Tests: resource_billing_test.go (gate allow/refuse/free/fail-closed/fail-open,
caller-org-not-default for both balance and debit, tenant isolation, env-never-
bypasses, unconfigured/nil no-op, fee resolution, deny shapes) + per-subsystem
integration tests proving the gate is wired into the real create path (402 before
backend on zero balance, 201 + caller-org debit when funded, free-kind un-gated).
go build ./... clean, go test ./... green, gofmt + vet clean.
2026-06-30 15:20:05 -07:00
hanzo-dev eb77547072 fix(auth): close forgeable-admin trust boundary on cloud-api
zip.Ctx.Org()/IsAdmin() read X-Org-Id / X-User-IsAdmin verbatim, trusting the
gateway to be their sole minter. But cloud-api is reachable WITHOUT the gateway
in front (in-cluster cloud-api.hanzo.svc:8000, and historically the public
cloud-api.hanzo.ai), so a direct caller could forge `X-User-IsAdmin: true` and
pass every admin gate: the new /v1/admin/catalog writes, /v1/pricing/sync, and
the provisioningsvc/mlsvc literal "admin" tenant bucket.

Add SanitizeIdentity, an early middleware (before BillingGate + every subsystem)
that strips every client-supplied authority header and re-derives identity ONLY
from a validated IAM JWT. Validation is a tiny go-jose JWKS validator
(auth_identity.go) that mirrors gateway/v2/iamauth — deliberately NOT imported
to avoid a module cycle (gateway/v2 already imports hanzoai/cloud) and pulling
the gateway's KrakenD/gin/traefik tree for ~150 lines. Admin authority is
granted ONLY to a verified GLOBAL admin (owner == AdminOrg), so an org-admin
(IAM also sets isAdmin=true for org owners) can't escalate. Non-admins are
pinned to their own org; a verified global admin's org-switch is honored. One
middleware makes every existing c.IsAdmin()/c.Org() reader trustworthy with no
handler changes.

Phase-1 residual (documented in middleware_identity.go): with no validatable
bearer the client X-Org-Id is passed through for DATA scoping (the console
browser data path depends on it) — closing that is Phase-2; the ADMIN boundary
is closed on every path because X-User-IsAdmin is never restored from a header.
Fail-secure: a validator misconfig (issuer/JWKS) makes admin 403, never opens.

Tests (middleware_identity_test.go): 18 subtests — forged header grants nothing,
org-admin can't escalate or cross-tenant, global-admin org-switch honored,
expired/wrong-key/wrong-audience/api-key/missing-issuer all anonymous, cookie +
HTTP-Basic paths. go-jose promoted to a direct require (already in the graph).
2026-06-30 15:02:06 -07:00
hanzo-dev 574ce320da fix(deps): canonical luxfi/zap@v0.8.11 go.sum hash (re-tagged module drifted local cache -> CI checksum mismatch) 2026-06-30 14:50:54 -07:00
hanzo-dev 51dc208f8b Merge branch 'feat/catalog-enablement' 2026-06-30 14:45:50 -07:00
hanzo-dev 928cc6a0ea fix(pricingsvc): gate root /v1/pricing + fail-closed overlay + override caps
Red review of feat/catalog-enablement found two in-branch leaks; fixed:

FIX #2 (HIGH): the root GET /v1/pricing returned the WHOLE bundle blob
(hanzoModels, thirdPartyModels, providers, freeModels, families) un-gated
via the `fixed` passthrough — an un-gated second source for everything the
leaf routes hide. New GateRootData() (catalog.go) gates the root in place:
hanzoModels+thirdPartyModels via VisibleCatalog (hanzoModels tagged "Hanzo"
so a disabled Hanzo provider cascades), providers via VisibleProviders, and
the id-reference lists freeModels + families[].models kept only if the
referenced model survived (admins keep all). Route moved out of `fixed` to
app.Get("/v1/pricing", gatedRoot). Audited the rest of `fixed`
(subscriptions/blockchain/iam/base/paas/tools/gpu/policy/cloud/compute):
all draw from the plans catalog with ZERO model/provider identity keys —
no gating needed; summary stays gated (providers sub-dict) with counts as
aggregate stats.

FIX #3 (fail-closed): empty DataDir was a Warn + :memory: fallback — a
security control that silently fails OPEN (admin-hidden models re-expose on
pod restart). Now a hard boot error (prod sets CLOUD_DATA_DIR;
provisioningsvc already requires it, so the unified binary always has one).

FIX #5 (DoS guard): overrides now bounded at 64 KiB + depth 32
(checkOverride) — bounds the recursive merge under a forged-admin write.

Tests: TestGateRootData (root gated identically to leaves: disabled/beta/
admin across hanzoModels/thirdPartyModels/freeModels/families/providers,
summary counts untouched), TestCheckOverride (object|null, size+depth caps),
TestMount_EmptyDataDir_FailsClosed, + e2e GET /v1/pricing gating and an
over-deep override PATCH->400. go build ./... + go test ./... green, gofmt.

NOT fixed here (infra, tracked separately): forgeable X-User-IsAdmin via
direct-to-pod cloud-api route — pre-existing, shared by every cloud IsAdmin
route; needs gateway routing + NetworkPolicy restriction in universe/operator.
2026-06-30 14:34:25 -07:00
hanzo-dev 2d0d63daf6 deps(ai): v1.785.14 -> v1.786.1 — cloud-repo image reaches feature-parity with prod
Decision (b): the cloud repo is the ONE authoritative builder of ghcr.io/hanzoai/cloud
(it assembles every subsystem incl. o11ysvc). But cloud pinned ai v1.785.14 while the
prod AI-built image (1.785.26) embeds ai code through the blue-money P0 security wave.
Bump ai to v1.786.1 — which contains ALL of it (balance ledger + overdraft gate, JWT
iss/aud validation, secret redaction, single-pod ledger invariant, aud env-keys,
redact allowlist, global-admin {admin,built-in}) — so the cloud-repo image is an
UPGRADE, never a regression below 1.785.26. luxfi/zap v0.8.8 -> v0.8.11 (tidy).
Build verified: go build ./cmd/cloud clean (453MB binary). Unblocks shipping o11ysvc.
2026-06-30 14:30:33 -07:00
hanzo-dev 976f6ec1ff fix(ci): ECR Public mirror for golang base — unblock release build (Docker Hub 429)
The last 5 release builds failed at `FROM golang:1.26-alpine` with
"toomanyrequests: unauthenticated pull rate limit" (429) from Docker Hub on the
shared runner, so no new cloud image has shipped — the deployed image predates the
o11ysvc mount (o11y /v1/o11y/* still 503) and the commerce mount. Switch the build
base to public.ecr.aws/docker/library/golang:1.26-alpine (immutable ECR Public
mirror, no rate limit) — the same fix already shipped in hanzoai/console2. Build
logic unchanged. Unblocks shipping o11ysvc → o11y live → retire old Langfuse console.
2026-06-30 14:12:28 -07:00
hanzo-dev 62e1f658d6 feat(pricingsvc): catalog enablement overlay + admin API
Add the backend admin layer that makes "admin enables -> customer sees"
real for the model/provider catalog, without forking the static
@hanzo/pricing bundle (still the sole source of truth for catalog
content/shape).

One overlay store, one gate:
- catalog.go: SQLite/Base overlay (table catalog_overlay, PK (kind,id);
  default = enabled, so an empty store is a no-op). Pure gate
  VisibleCatalog/VisibleProviders applies {enabled,betaOrgs,overrides}
  onto the bundle output: visible iff own AND provider overlay admit the
  org (enabled || org in betaOrgs); overrides merge via RFC 7386. Admins
  see every entry, annotated under _overlay.
- admin.go: global-admin (c.IsAdmin) write surface — GET /v1/admin/catalog
  (full catalog + state), PATCH /v1/admin/catalog/models/* (slashed ids via
  greedy wildcard) and /providers/:name. Partial-update PATCH; override
  validated as JSON object|null.
- pricingsvc.go: gate wired into the catalog read path (models, free,
  featured, providers, summary, model/:name); non-catalog routes unchanged.
  Overlay opened at {DataDir}/catalog.db (in-memory fallback), closed in
  Shutdown.

Default behavior unchanged for live customers (all enabled). Tests:
pure-gate units (default-all-visible, disabled-hidden-except-beta,
override-merged deep, admin-sees-all, provider cascade), store round-trip,
and an end-to-end HTTP test (wildcard routing, IsAdmin 403, enable->see flow).

console2 admin UI is a separate agent's job; it consumes these endpoints.
2026-06-30 14:02:59 -07:00
hanzo-dev 49318d610f fix(deps): base v1.3.2 -> v1.4.1 — unblock release Docker build
The committed go.sum pinned hanzoai/base@v1.3.2, whose tag was force-re-tagged
upstream (live content hash drifted from the recorded hash). The release
Dockerfile verifies modules against the committed go.sum with GOSUMDB=off, so
`go mod download` hit "checksum mismatch / SECURITY ERROR" and every release
build failed.

v1.4.1 is the latest base tag that (a) has a stable, immutable hash and (b)
still registers as a cloud subsystem (v1.4.2+ dropped cloud.Register and would
break subsystem assembly — 13 vs 14 subsystems, /v1/base/health 404). Pin v1.4.1:
fresh-cache go mod download is clean, registry assembles 14 subsystems, all
health endpoints 200, full suite green.
2026-06-28 23:15:53 -07:00
zeekay e0c68fd4a5 feat(projects): /v1/projects org-scoped store + deploy pipeline
New projectsvc subsystem (HIP-0106) — the ONE org-scoped store of
buildable/deployable sites, shared by hanzo.app (builder) and
console.hanzo.ai (Projects module). Both read/write the same records
through the gateway (X-Org-Id from the IAM JWT); no second copy of state.

- CRUD: POST/GET/PATCH/DELETE /v1/projects (+ /:slug)
- Deploy: POST /v1/projects/:slug/deploy
  - artifact mode: tar(.gz) of built site -> OUR S3 (s3.hanzo.ai,
    CLOUD_PROJECTS_BUCKET) under <org>/<slug>/, public-read, live URL
  - git mode: queue + CI completion hook (/deployments/:id/complete)
- Deploy history: GET /v1/projects/:slug/deployments(/:id)
- SQLite store (modernc), versioned deployments, tenant isolation by org
- Reuses CLOUD_S3_ADMIN_* creds (one S3 path, like provisioningsvc)
- Path-traversal + size/file guards on artifacts; index.html required
- Published contract in CONTRACT.md for console2 to consume

Tests: store CRUD/isolation/ordering, deployment versioning, slugify,
provider detection, safeRel traversal guard, tar/tar.gz walker. All pass.
2026-06-28 23:07:51 -07:00
5665cc491d build: drop GOPRIVATE for luxfi/hanzoai — use immutable public proxy (fix go.sum checksum mismatch) (#46)
The build routed luxfi/* + hanzoai/* DIRECT via git insteadOf, which re-fetches a
re-pointed tag's tree (luxfi/age@v1.5.0) whose hash differs from go.sum's proxy
hash → 'verifying github.com/luxfi/age@v1.5.0: checksum mismatch / SECURITY
ERROR'. luxfi/hanzoai are PUBLIC: resolve them via the IMMUTABLE public proxy +
the committed go.sum (which already pins the proxy hashes). Only zap-proto/*
stays first-party-direct. Matches the drop-GOPRIVATE fix in hanzoai/iam +
luxfi/kms.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-28 20:21:35 -07:00
hanzo-dev 577366a7b8 feat(cloud): pluginsvc — runtime plugin loader (goa wasm + ZAP-pluggable proxy)
cloud becomes a thin runtime host: alongside the compiled-in application
subsystems it mounts services from a runtime manifest (CLOUD_PLUGINS) with no
rebuild. Two plugin kinds, both reduced to app.Mount(prefix, http.Handler):

  wasm  — a polyglot module (Rust/WASM, Python, TypeScript) loaded in-process
          via github.com/hanzoai/goa (wazero/gpython/goja; pure Go, stays
          CGO_ENABLED=0). Drop a .wasm + manifest entry → mounted.
  proxy — a standalone server (e.g. the beego apps ai, vm) reached over a
          pluggable transport. The "zap" transport registers via
          pluginsvc.RegisterTransport; proxying defaults to HTTP until then.

Adding/updating a service = edit the manifest + drop a .wasm or redeploy the
standalone — the cloud binary is unchanged unless its own core changes.
Registered at order 900. Static binary preserved; package + full suite green.
2026-06-28 20:09:36 -07:00
z 0fee2d3e55 docs(brand): add hero banner 2026-06-28 20:05:32 -07:00
z 7a611434c7 chore(brand): dynamic hero banner 2026-06-28 20:05:31 -07:00
hanzo-dev b0368c9076 test(cloud): scope registry assertion to the application-layer matrix (edge/infra subsystems run as own deployments) 2026-06-28 17:11:07 -07:00
hanzo-dev 81bbb0f859 refactor(arch): cloud = application layer only; drop edge/infra subsystem imports (gateway, iam, kms, mcp run as own deployments behind gateway/ingress). Scope registry test to app matrix. Blast-radius isolation, smaller binary. 2026-06-28 17:09:22 -07:00
hanzo-dev 955342795e refactor(tenant): remove last X-Hanzo-Org; cloud -> ai v1.785.14. Single canonical X-Org-Id tenant header across the whole stack. 2026-06-28 16:59:27 -07:00
hanzo-dev 88b9fff607 build(deps): cloud -> ai v1.785.13 (tenant identity decomplected to X-Org-Id; X-Hanzo-Org kept as orthogonal service-token selector) 2026-06-28 16:47:23 -07:00
hanzo-dev d87ee75829 fix(cloud): full subsystem registry — force guarded beego/v2 v2.3.10 + explicitly import kms/iam/gateway/mcp; resolves ai(beego-v1)+iam(beego-v2) grace flag collision. TestRegistryAssemblesSubsystems green; cloud test suite fully passing. 2026-06-28 16:32:21 -07:00
hanzo-dev 152de74bab feat(cloud): storage-lockdown invariant (reject legacy PG env) + PG->SQLite migration tooling from zap-listener; superseded ZAP-listener/mount-adapter code dropped (zapface + registry are the one canonical way) 2026-06-28 16:17:10 -07:00
hanzo-dev 3c76a9c689 fix(deps): revert idv to v1.0.0 (v1.0.2 release is broken — missing provider/onyxplus.go) 2026-06-28 15:58:38 -07:00
hanzo-dev e7ba359e49 build(deps): cloud -> ai v1.785.11 (fail-closed mount + P0 security: balance/overdraft, JWT iss/aud, redaction, authz, tenant) + idv v1.0.2 2026-06-28 15:57:18 -07:00
hanzo-dev 981c8fc4e7 Merge remote-tracking branch 'origin/main' 2026-06-28 15:23:09 -07:00
hanzo-dev 0ddb5d40f4 fix(build): route luxfi via proxy + clean go.sum; ai v1.785.10 (fail-closed mount); drop removed amqp from tests
- Root cause of broken build: GOPROXY=direct fetched re-tagged git content + dropped //go:embed files (accel/crypto/geth). luxfi is public -> use proxy.
- ai@v1.785.10: Mount fails closed (503) when DB unconfigured, honoring BuildDeps three-mode contract.
- amqp removed from registry/health tests (subsystem retired).
- TestRegistryAssemblesSubsystems still red: ai(beego v1) vs iam(beego v2) grace flag collision blocks in-process coexistence - architectural follow-up.
2026-06-28 15:21:14 -07:00
zeekay 5bf4429661 fix(deps): bump hanzoai/iamsdk/v2 v2.1.0 -> v2.1.2 (JWKS token verify)
v2.1.2 verifies JWTs via the published JWKS instead of parsing the configured
cert PEM. cloud-api's /v1/signin (the ai dep's code->session exchange) configured
an unparseable Certificate (IAM returns the cert NAME 'cert-built-in' to
global-admin callers, not a PEM) -> 'iamsdk: not valid PEM' -> every
hanzo-cloud/hanzo-console SPA login failed (console + admin). go.sum realigned
(first-party re-tag dep-rot). cmd/cloud builds clean.
2026-06-28 14:28:00 -07:00
hanzo-dev 25be112454 chore(deps): bump luxfi/hanzoai deps to latest; fix re-tagged keys v1.2.2 + gateway v2.14.10; regenerate clean go.sum 2026-06-28 14:04:15 -07:00
hanzo-dev 00fb7351e5 Merge PR #37: Hanzo branding + LICENSE attribution 2026-06-28 13:12:17 -07:00
hanzo-dev 7aa7cf5660 Merge PR #45: fix(deps) bump hanzoai/ai -> 2e8fc6f15947 (401 on missing/invalid Bearer) 2026-06-28 13:12:17 -07:00
Blue a9db9e05a7 build(cloud): bump ai -> 2e8fc6f1 (invalid hk- key -> 401, nil-user fix); cloud:1.785.24 2026-06-28 04:29:32 -07:00
Blue acec71759a build(cloud): bump ai -> 069b83ce (authenticate-before-parse; 401 not 200 on invalid key + bad body)
Pulls hanzoai/ai @069b83ce into cloud:1.785.23: residual 200-leak fix for
/v1/chat/completions, /v1/embeddings, /v1/rerank, /v1/messages — an invalid
credential with a malformed/incomplete body now returns 401 (was 200/400),
authenticating before the body is parsed. ai go.mod unchanged, so only the ai
require + its go.sum zip hash move.
2026-06-28 03:28:29 -07:00
hanzo-dev 021dc990f6 fix(deps): revert erroneous base/pq go.sum realign — keep ONLY ai bump
The base/pq 'realign' in the prior commit adopted anomalous bits from a local
direct-fetch; the build container (and the working cloud:1.785.20 build) resolve
the ORIGINAL stable hashes via the module proxy/cache. Net go.sum change is now
exactly the hanzoai/ai bump (c08be563). luxfi re-tag churn
(fix/threshold-*-checksum-convergence) does NOT touch these pinned hashes.
2026-06-28 01:41:07 -07:00
hanzo-dev 4dd69bce68 fix(deps): bump hanzoai/ai -> c08be563 (401/402/400 auth status, not 200)
Pull in the ai auth-status fix: invalid/unknown hk- key -> 401, insufficient
balance -> 402, bad model -> 400 (was HTTP 200 with an error body on all of
them). Cloud-api is the single backend validating hk- keys for both
api.cloud.hanzo.ai and the gateway (api.hanzo.ai), which proxies the status.

Realign go.sum zip hashes for hanzoai/base v1.3.2 and luxfi/pq v1.0.3 to the
current origin (upstream re-tags; /go.mod hashes unchanged) so the build
resolves in a fresh container. Verified: CGO_ENABLED=0 go build ./cmd/cloud
links clean.
2026-06-28 01:28:44 -07:00
zeekay 6890c89e98 fix(deps): correct go.sum for re-tagged luxfi/pq + hanzoai/base
CI fetches via proxy.golang.org,direct; both tags were force-re-pushed so
the committed zip hashes no longer matched what the proxy serves:
  luxfi/pq    v1.0.3  pFlQm1... -> ksw1dm... (proxy commit 90d2223)
  hanzoai/base v1.3.2 BdTNDNe... -> 7GcHpg... (proxy commit 33d12949)
Minimal go mod tidy fix (2 lines); go.mod unchanged; cmd/cloud builds clean.
Unblocks the /v1/memory embed (hanzoai/ai fe516793).
2026-06-28 01:01:05 -07:00
zeekay a1503c51bf deps: bump hanzoai/ai -> fe516793 (embeds /v1/memory) + realign luxfi go.sum (pq/base/tls re-tag dep-rot) 2026-06-28 00:16:20 -07:00
zeekay fd17255784 fix(o11y): proxy rewrites /v1/o11y/* -> /api/* for the runtime's controllers
The o11y runtime (SigNoz query server) serves its API under /api; the registered
handler owns the documented /v1/o11y/* -> /api/* rewrite. The proxy now strips the
public prefix and prepends /api so /v1/o11y/v3/query_range reaches /api/v3/query_range
(verbatim forwarding hit the SPA fallback instead of the API).
2026-06-27 23:27:34 -07:00
zeekay 7b8c025ae3 feat(o11y): install runtime handler via reverse proxy to the o11y deployment
The o11y subsystem (hanzoai/o11y, order 70) mounts /v1/o11y/* but delegates to a
handler installed via o11y.SetHandler — never called in the unified cloud binary,
so the surface 503'd 'o11y runtime not initialized'. The heavy o11y runtime runs
as a dedicated Deployment; cloud now installs a reverse proxy to it (O11Y_UPSTREAM,
default o11y.hanzo.svc:80) so /v1/o11y/* serves real telemetry. Path preserved
verbatim; gateway-terminated identity forwarded.
2026-06-27 23:24:25 -07:00
hanzo-dev 6027df1e4d fix(deps): bump hanzoai/ai -> 254ea3b6 (401 on missing/invalid Bearer)
Pulls in ai fix: /v1/chat/completions, /v1/embeddings, /v1/rerank now
return HTTP 401 (not 200) on a missing/invalid Bearer token, matching
/v1/models. Valid-key completions + per-org billing unchanged.
2026-06-27 23:05:48 -07:00
zeekay 9ffdcbd253 chore(deps): bump hanzoai/kms/sdk/go v1.0.0 -> v1.1.1 (luxfi/constants dep-rot fix → unblocks mlsvc image build) 2026-06-27 22:04:23 -07:00
hanzo-dev 6783e9ae20 build: realign first-party go.sum hashes to current origin (upstream re-tags)
luxfi/* and hanzoai/* tags were re-pointed upstream (base v1.3.2, pq v1.0.3,
zap v0.8.8, et al.); the committed go.sum went stale and a clean image build
failed 'go mod download' verification. Re-record the current direct-fetch
hashes (proven non-first-party set untouched). No go.mod change.
2026-06-27 18:19:52 -07:00
hanzo-dev e702b54fbd build: realign hanzoai/base v1.3.2 go.sum hash (upstream re-tag)
base@v1.3.2 was re-pointed after cloud's go.sum was recorded; the committed
zip hash (BdTNDNe3…) is the stale public-proxy first-seen content, while the
live tag (direct git, the path CI's GOPRIVATE takes) hashes to 7GcHpg…. CI
fetches base DIRECT and fast-fails the build at the checksum mismatch — the
last stale hash blocking a green main (pq was realigned in dd557d7a; this is
the same upstream-re-tag fix, mirroring 5b4e3cef for luxfi age/keys/zap).

go.mod /go.mod hash unchanged (graph-load verified it); only the module zip
hash needed realigning. Verified: clean-cache readonly linux/amd64 -mod=mod
direct build is green.
2026-06-27 18:14:42 -07:00
hanzo-dev dd557d7a4c build: realign luxfi/pq v1.0.3 go.sum hash (upstream re-tag)
The committed zip hash went stale after luxfi/pq@v1.0.3 was re-tagged
upstream; cloud's clean image build failed go mod download verification.
Update to the current origin hash.
2026-06-27 18:10:26 -07:00
hanzo-dev 5bf55f2fde chore(deps): bump hanzoai/ai → f2cd2681 (per-user billing subject)
Pulls the per-user billing-subject fix into cloud-api: the gateway now keys the
balance gate + usage debit on object.BillingSubject(owner,name), so individuals
in the shared 'hanzo' org are billed independently (own balance, own $5) instead
of sharing+draining the single (hanzo,hanzo) balance.
2026-06-27 18:01:45 -07:00
zeekay ae0dc50a3b feat(mlsvc): tenant-scoped /v1/ml + /v1/train k8s bridge (kserve/trainer/katib)
New cloud subsystem (order 130) fronting the kubeflow forks via the k8s
dynamic client, scoped per-org by namespace (ml-<org>):

- /v1/ml/models           CRUD + PATCH + /predict (kserve InferenceService;
                          predict proxies to the model's v2 data plane /infer)
- /v1/train/jobs          CRUD (trainer TrainJob)
- /v1/train/experiments   CRUD + /trials (katib Experiment/Trial)
- /v1/ml/health, /v1/train/health  real probes: k8s reachability + CRD presence
  (200 ok / 503 degraded with the real reason; never status-theater)

Tenant boundary is the per-org Kubernetes namespace; the org->namespace map is
injective (strict slug regex, no lossy fold) so two tenants can never share a
namespace. User-supplied labels can't override the tenant org marker. The k8s
client is built in-process from the in-cluster service account with a KUBECONFIG
fallback (self-contained like provisioningsvc's backends, not on shared
cloud.Deps); it fails closed (503 / degraded health) when unconfigured.

Promotes k8s.io/apimachinery + client-go to direct requires. Registered via
blank import in subsystems.go. Unit tests cover the security-critical pure
helpers (tenant injectivity, name validation, label-override guard, GVRs).
2026-06-27 14:56:14 -07:00
hanzo-dev 54ddb3783b chore(deps): bump hanzoai/ai -> 703fe6b5 (OpenAI stream role + usage-chunk gating)
Picks up the streaming fix: first delta carries role:assistant and the
empty-choices usage chunk is gated behind stream_options.include_usage —
resolves hanzo.chat 'reading role' no-reply (separate from the dbx fix).
2026-06-26 17:43:37 -07:00
hanzo-dev 9ae1cc50ef chore(deps): bump hanzoai/ai -> aa326e8a (Message []struct JSON columns)
Picks up JSONList[T] (sql.Scanner + driver.Valuer over JSON) for
Message.VectorScores/Suggestions/ToolCalls/SearchResults, fixing the dbx
'unsupported type []model.SearchResult, a slice of struct' 500 that killed
console2 sign-in (welcome-message insert) and every hanzo.chat AI message
save. No new deps; transitive hunyuan -> v1.3.48 (already required by ai main).
2026-06-26 17:25:00 -07:00
hanzo-dev 5b4e3cef35 fix(build): go.sum direct-live hashes for re-tagged luxfi age/keys/zap
luxfi re-tagged age v1.5.0, keys v1.1.0 and zap v0.8.8 in place. The public
module proxy serves each tag's first-seen (now stale) content, while the
Dockerfile fetches first-party DIRECT via GOPRIVATE — so `go mod download`
hit SECURITY ERROR (checksum mismatch) against the stale proxy zip hashes
committed in go.sum. Record the live DIRECT zip hashes for all three
(the /go.mod hashes are unchanged). Verified clean in the exact build env
(golang:1.26-alpine, GOPRIVATE=hanzoai/luxfi/zap-proto, GOPROXY=proxy,direct):
go mod download + CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build ./cmd/cloud ./cmd/hanzo.
2026-06-26 15:44:14 -07:00
hanzo-dev 3c091a411c fix(build): GOPROXY=proxy,direct so public nested-tag modules resolve
The cloud merge pulled in tencentcloud-sdk-go monorepo modules whose tags are
nested paths (tencentcloud/hunyuan/v1.0.1074). GOPROXY=direct forced ALL modules
through direct VCS, which CANNOT resolve those nested tags ('unknown revision') —
go mod download failed in-build (worked locally only via cache). Route public
deps through the module proxy; first-party (hanzoai/luxfi/zap-proto) still go
direct+token via GOPRIVATE, so no private leak and re-pointed tags still resolve.
2026-06-26 15:16:51 -07:00
hanzo-dev ee3ba882f5 Merge feat/product-search-vector-endpoints into main (union)
Union merge — keep BOTH main's and the feature branch's work:

- subsystems/subsystems.go: register main's provisioningsvc (order 120)
  AND the feature's productsvc (order 145) alongside evalsvc/plansvc/pricingsvc.
- serve.go: collapse the two parallel :9090 listeners into ONE. Keep the
  feature's robust healthSrv/healthMux lifecycle (ReadHeaderTimeout, graceful
  Shutdown, fatal-on-bind) and FOLD main's HIP-0113 /metrics into healthMux,
  so the single ops port serves /healthz /readyz /health /metrics. Keep the
  feature's /zap zapface WebSocket plane. Drop the duplicate inline ops
  goroutine (and its now-unused io import) to avoid a double-bind CrashLoop.
- config.go: union HIP-0111 IAM-issuer-from-brand AND the ZAP web-origin allowlist.
- go.mod/go.sum: union both dep sets; take the feature's newer hanzoai/ai
  (2962d31c, /v1/embeddings + /v1/rerank) and main's newer luxfi
  (database v1.19.3, threshold v1.9.9, age v1.5.0). RESTORE the feature's
  'replace sashabaranov/go-openai => hanzoai/go-openai v1.40.0' that the merge
  dropped — ai's reasoning path needs Delta.ReasoningContent (fork-only).
  go.sum regenerated via go mod tidy honoring 34aa84e1 (first-party sumdb skip).

Build: go build ./... green (CGO=1; only luxfi/accel ld warnings). go vet clean.
2026-06-26 14:32:30 -07:00
hanzo-dev e7b34cadf8 feat(evals): mount /v1/evals/* — LLM-observability facade (HIP-0106)
evalsvc is a thin facade: proxies /v1/evals/{datasets,dataset-items,evaluators,
scores} → console's public REST API (Langfuse v3 fork, owns the eval/observability
data model) and orchestrates POST /v1/evals/runs against the in-process model
gateway (run model over dataset → score → trace). No eval logic reimplemented —
evals ARE LLM observability; cloud unifies the surface, console owns the logic.
Registered at order 145, before the AI /v1/* catch-all.
2026-06-26 14:08:08 -07:00
hanzo-dev 008433e546 feat(cli): hanzo cloud-control CLI (login/apps/deploy/clusters/build/k8s)
Extend cmd/hanzo with a gcloud/doctl-class control plane (client mode),
selected by the first token alongside the existing server-mode subsystem
dispatch. Thin client over IAM (hanzo.id), the platform REST control plane
(platform.hanzo.ai/v1), and the cloud /v1 API — no parallel API.

- login/logout/whoami/auth: IAM password grant, token in ~/.hanzo (0600)
- apps list|get|sync: platform apps board (declared/running/latest/drift)
- deploy: rolling zero-downtime redeploy via the container redeploy surface
- clusters list|get|create|select|install-baseline|target: dedicated DOKS
- build: platform-native (arcd) build enqueue
- k8s target; config get|set|list|path
- global --org/--output/--platform-url/--iam-issuer/--platform-token
- secrets only via env/~/.hanzo, never hardcoded; platform kubeconfig never fetched
- stdout kept machine-readable (server-graph init chatter redirected to stderr)

44 unit tests (client + command wiring via httptest). Verified live:
login -> whoami -> apps list (79 apps) -> deploy pricing (gen 4->5, zero-downtime).
2026-06-26 13:47:01 -07:00
hanzo-dev b47a1761fc chore(deps): bump hanzoai/ai → 2962d31c (/v1/embeddings + /v1/rerank)
Pulls hanzoai/ai feat/embeddings-rerank, which adds the OpenAI-compatible
POST /v1/embeddings and Cohere/Jina-compatible POST /v1/rerank endpoints on the
same auth + provider-routing path as /v1/chat/completions. Embeddings reuse the
already-configured OpenAI Direct key (kms://OPENAI_API_KEY); rerank needs no new
key (bi-encoder cosine over the resolved embedding model, native Jina/Cohere
proxy when keyed).
2026-06-26 13:30:26 -07:00
4d6925242c feat(provisioning): /v1 control plane that creates logical resources in live shared backends (#44)
* feat(cloud): derive IAM issuer from brand, plumb into Deps (HIP-0111)

The unified cloud binary is one artifact serving every brand's API host
(api.hanzo.ai, api.lux.cloud, api.zoo.cloud, api.cloud.pars.network). Brand
is a per-deployment value, but the IAM issuer was hardcoded to iam.hanzo.ai
for every brand and Config.IAMIssuer was never plumbed into Deps — so a lux
or zoo deployment would validate JWTs against the wrong issuer (or not at all,
subsystems having no issuer to use).

White-label the token-validation issuer by brand:
- brand.go: PUBLIC brand→IAM registry (issuer + domain). hanzo→iam.hanzo.ai,
  lux→lux.id, zoo→zoo.id, pars→pars.id, bootnode→id.bootno.de. One source of
  truth; public values live in code, not KMS.
- config: when CLOUD_IAM_ISSUER/--iam-issuer is unset, derive it from the
  brand via the registry (no longer silently iam.hanzo.ai for all brands).
- deps + build: add Deps.IAMIssuer, set from cfg in BuildDeps, so subsystems
  validate against {issuer}/v1/iam/.well-known/jwks per HIP-0111.

Root package: go build + go vet + go test green (registry + issuer-derivation
tests). The fused -tags cloud binary build is blocked by a pre-existing
workspace go.sum mismatch (luxfi/age via go.work), orthogonal to this change.

* deps: converge luxfi/threshold -> v1.9.9, fix luxfi/age v1.5.0 re-tag hash

Both threshold and age were re-tagged upstream, leaving cloud/go.sum with
hashes that no longer match what the proxy serves. threshold@v1.9.4's
recorded sum broke `go work sync` and the fused -tags cloud build:
  verifying github.com/luxfi/threshold@v1.9.4/go.mod: checksum mismatch

Converge on the single workspace-wide versions:
- threshold v1.9.4 -> v1.9.9 (latest 1.9.x; matches base + mpc)
  go.mod require bumped; go.sum gains v1.9.9 zip+go.mod sums; drops the
  unused v1.9.4 zip sum; keeps v1.9.4/go.mod (consensus@v1.25.0 still
  requires it in MVS) with the correct post-retag hash.
- age v1.5.0: corrected the stale zip hash
  (zC/Fw/ptZwAXr9nqrxmrcf8752EIl1Lq9RECp9OmCO0= ->
   G69HbSV4R3vKEH9B0CulnRaMdSdf4RalMgP8xKmxHeI=) to match the re-tagged
  module; go.mod sum was already correct. Version unchanged (v1.5.0,
  consistent with base/kms/mpc).

All hashes authoritative (match proxy + mpc/base go.sum). No suppression
flags, no downgrade.

* feat(provisioning): control plane that creates logical resources in live shared backends

Adds clients/provisioningsvc, registered at order 120 and linked by one blank
import in subsystems/subsystems.go. It turns "create a database" into a real
logical resource inside the already-live shared product backends, scoped to the
gateway-minted org (X-Org-Id / c.Org()).

Surface (kind in databases|vector|datastore|kv|search|storage|docdb):
  POST   /v1/<kind>        {"name":"<slug>"} -> 201 {id,kind,name,status,host,
                            port,username,database,connectionString,password?}
  GET    /v1/<kind>        -> 200 [{id,name,kind,status,host,port,createdAt}]
  GET    /v1/<kind>/<name> -> 200 {id,name,kind,status,host,port,username,database}
  DELETE /v1/<kind>/<name> -> 204
  (GET /v1/provisioning/health is auto-registered by Serve.)

Backends (admin creds + in-cluster .svc defaults via env):
  databases -> Postgres   (pgx)            CREATE ROLE + CREATE DATABASE
  vector    -> Qdrant      (net/http)      PUT /collections/{name}
  datastore -> ClickHouse (clickhouse-go)  CREATE DATABASE + USER + GRANT
  kv        -> Redis       (go-redis)      ACL SETUSER (keyspace-scoped)
  search    -> Meilisearch (net/http)      POST /indexes
  storage   -> S3/MinIO    (minio-go)      MakeBucket
  docdb     -> MongoDB     (mongo-driver)  createCollection + createUser

Physical resources are namespaced org_<org>_<name> so tenants never collide;
the name is validated to a slug at the boundary and all SQL identifiers are
quoted — injection-safe.

Secrets: per-resource passwords (databases/kv/datastore/docdb) are sealed in
Hanzo KMS (github.com/hanzoai/kms/sdk/go, client-side encrypted); only a
secret_ref is persisted in SQLite. When KMS is unconfigured the service
degrades safely — the password is returned once in the create response and
nothing is written in plaintext. vector/search/storage have no per-resource
password (shared key auth out of band).

Metadata lives in ONE pure-Go SQLite DB ({DataDir}/provisioning.db, modernc),
UNIQUE(org,kind,name); multi-step writes run in a transaction.

Drivers were already indirect deps; importing them promotes them with no
version bumps. Tests cover the store (insert/get/list/delete, org isolation,
duplicate->conflict), name validation, org sanitization, identifier safety,
and token generation.

* fix(provisioning): close cross-tenant physical-name collision + native kind naming

BLOCKING SECURITY FIX. physicalName folded the org→name boundary by joining
hyphen-underscored org and name, so two distinct tenants could map to ONE
physical backend resource: physicalName("acme","my-db") ==
physicalName("acme-my","db") == "org_acme_my_db" (bucketName collided too).
On KV that is a cross-tenant credential takeover (idempotent ACL SETUSER
overwrites tenant A's user/keyspace); on SQL/datastore/S3 a cross-tenant DoS
and existence oracle. UNIQUE(org,kind,name) did not protect the physical layer.

- physicalName(org,name) = "o" + hex(sha256(org))[:16] + "_" + sanitizeIdent(name):
  a FIXED-WIDTH org hash makes the boundary unambiguous, so cross-org folds are
  cryptographically negligible. bucketName derives from the (now injective)
  physical via the '_'→'-' bijection, so one guard covers every backend. Both
  stay backend-valid (Postgres 63-char identifier limit, S3 3-63 char bucket).
- store: add global UNIQUE(physical_name) index + PhysicalExists pre-check; the
  create handler now FAILS CLOSED with 409 BEFORE touching a backend on any
  residual name-fold, never silently sharing a physical resource. Row maps
  physical_name -> (org,kind,name) so names stay traceable.
- tests: injectivity (physicalName + bucketName), handler org-gate (empty
  X-Org-Id -> 403 for non-admin), KMS safe-degrade (password returned once,
  nothing persisted in plaintext, secret_ref empty).

NATIVE NAMING (Hanzo brand: product name, never upstream OSS name):
kind "databases"->"sql", "storage"->"s3"; final set = sql, vector, datastore,
kv, search, s3, docdb. env CLOUD_STORAGE_*->CLOUD_S3_*. Wire connection schemes
(postgres://, redis://, mongodb://) unchanged — protocol, not branding.

Minor: package-level Shutdown closes the store (mirrors plansvc); comment that
trusting X-User-IsAdmin is acceptable (blast radius = literal "admin" bucket).

---------

Co-authored-by: zeekay <z@zeekay.io>
2026-06-26 13:15:36 -07:00
hanzo-dev 3b56654865 chore(deps): bump hanzoai/ai → 41869ad8 (self-scoped /v1/update-preferences)
Picks up the account-backed user-preferences endpoint so console2 (and any
product) can persist cross-product, cross-device customizations onto the IAM
user account.
2026-06-26 11:15:07 -07:00
hanzo-dev 6971464edc feat(productsvc): expose console Search/Vector panels on cloud-api
The console Search/Indexes and Vector panels are hardcoded to call
api.cloud.hanzo.ai/api/search-docs/* and /api/vector/* with a bearer
service key. cloud-api owns those paths now: productsvc proxies them to
the in-cluster Meilisearch (search.hanzo.svc) and Qdrant (vector.hanzo.svc)
and translates each upstream response into the exact JSON the console's
tRPC routers decode (SearchIndex/SearchStats, VectorCollection/VectorStats).

Read-only, shape-translating glue — no search/vector logic reimplemented.
Bearer key enforced with a constant-time compare (gateway bypasses these
paths via AUTH_PUBLIC_PATHS since the key is opaque, not a JWT). Endpoints
degrade to an honest empty body when the upstream is unreachable, matching
the console panels' graceful-empty contract.

Verified locally against the live search/vector services: 6 real indexes
(43,162 docs), 2 real Qdrant collections; wrong/absent key -> 401.
2026-06-25 15:18:51 -07:00
0b4372f1a0 chore: bump luxfi/database v1.19.3 (#41)
Co-authored-by: zeekay <z@zeekay.io>
2026-06-25 15:15:47 -07:00
151e151f44 deps: converge luxfi/threshold → v1.9.9, fix luxfi/age v1.5.0 re-tag hash (#43)
* feat(cloud): derive IAM issuer from brand, plumb into Deps (HIP-0111)

The unified cloud binary is one artifact serving every brand's API host
(api.hanzo.ai, api.lux.cloud, api.zoo.cloud, api.cloud.pars.network). Brand
is a per-deployment value, but the IAM issuer was hardcoded to iam.hanzo.ai
for every brand and Config.IAMIssuer was never plumbed into Deps — so a lux
or zoo deployment would validate JWTs against the wrong issuer (or not at all,
subsystems having no issuer to use).

White-label the token-validation issuer by brand:
- brand.go: PUBLIC brand→IAM registry (issuer + domain). hanzo→iam.hanzo.ai,
  lux→lux.id, zoo→zoo.id, pars→pars.id, bootnode→id.bootno.de. One source of
  truth; public values live in code, not KMS.
- config: when CLOUD_IAM_ISSUER/--iam-issuer is unset, derive it from the
  brand via the registry (no longer silently iam.hanzo.ai for all brands).
- deps + build: add Deps.IAMIssuer, set from cfg in BuildDeps, so subsystems
  validate against {issuer}/v1/iam/.well-known/jwks per HIP-0111.

Root package: go build + go vet + go test green (registry + issuer-derivation
tests). The fused -tags cloud binary build is blocked by a pre-existing
workspace go.sum mismatch (luxfi/age via go.work), orthogonal to this change.

* deps: converge luxfi/threshold -> v1.9.9, fix luxfi/age v1.5.0 re-tag hash

Both threshold and age were re-tagged upstream, leaving cloud/go.sum with
hashes that no longer match what the proxy serves. threshold@v1.9.4's
recorded sum broke `go work sync` and the fused -tags cloud build:
  verifying github.com/luxfi/threshold@v1.9.4/go.mod: checksum mismatch

Converge on the single workspace-wide versions:
- threshold v1.9.4 -> v1.9.9 (latest 1.9.x; matches base + mpc)
  go.mod require bumped; go.sum gains v1.9.9 zip+go.mod sums; drops the
  unused v1.9.4 zip sum; keeps v1.9.4/go.mod (consensus@v1.25.0 still
  requires it in MVS) with the correct post-retag hash.
- age v1.5.0: corrected the stale zip hash
  (zC/Fw/ptZwAXr9nqrxmrcf8752EIl1Lq9RECp9OmCO0= ->
   G69HbSV4R3vKEH9B0CulnRaMdSdf4RalMgP8xKmxHeI=) to match the re-tagged
  module; go.mod sum was already correct. Version unchanged (v1.5.0,
  consistent with base/kms/mpc).

All hashes authoritative (match proxy + mpc/base go.sum). No suppression
flags, no downgrade.

---------

Co-authored-by: zeekay <z@zeekay.io>
2026-06-25 14:49:15 -07:00
hanzo-devandGitHub e1b4813776 feat(cloud): derive IAM issuer from brand, plumb into Deps (HIP-0111) (#42)
The unified cloud binary is one artifact serving every brand's API host
(api.hanzo.ai, api.lux.cloud, api.zoo.cloud, api.cloud.pars.network). Brand
is a per-deployment value, but the IAM issuer was hardcoded to iam.hanzo.ai
for every brand and Config.IAMIssuer was never plumbed into Deps — so a lux
or zoo deployment would validate JWTs against the wrong issuer (or not at all,
subsystems having no issuer to use).

White-label the token-validation issuer by brand:
- brand.go: PUBLIC brand→IAM registry (issuer + domain). hanzo→iam.hanzo.ai,
  lux→lux.id, zoo→zoo.id, pars→pars.id, bootnode→id.bootno.de. One source of
  truth; public values live in code, not KMS.
- config: when CLOUD_IAM_ISSUER/--iam-issuer is unset, derive it from the
  brand via the registry (no longer silently iam.hanzo.ai for all brands).
- deps + build: add Deps.IAMIssuer, set from cfg in BuildDeps, so subsystems
  validate against {issuer}/v1/iam/.well-known/jwks per HIP-0111.

Root package: go build + go vet + go test green (registry + issuer-derivation
tests). The fused -tags cloud binary build is blocked by a pre-existing
workspace go.sum mismatch (luxfi/age via go.work), orthogonal to this change.
2026-06-25 14:42:24 -07:00
hanzo-dev 974d22bc9c fix(zapface): serve /zap via native Fiber WebSocket (zip/wsx), not net/http adaptor
The net/http adaptor path 404'd: fasthttp's synthetic ResponseWriter can't be
hijacked, so coder/websocket.Accept failed and Fiber returned 404 for /zap
(confirmed live: 'GET /zap status 404'). Switch to zip/wsx (fasthttp/websocket)
which upgrades natively. Handler now returns a zip.Handler that mints the auth
slot BEFORE upgrade (401 fail-closed), captures the cookie/bearer in the
per-connection closure, and runs the binary-ZAP read loop with ws.ReadMessage/
WriteMessage. serve.go mounts app.Get("/zap", ...). Adds fasthttp/websocket
to go.sum (zip/wsx dep). End-to-end WS integration test rewritten against a
real zip app + native upgrade — green.
2026-06-25 14:08:16 -07:00
hanzo-dev 9bfef1a633 fix(serve): bind the :9090 health listener (/healthz, /readyz)
HealthListenAddr was declared but never bound — the operator's liveness
probe targets :9090/healthz and readiness :9090/readyz, so the pod failed
liveness and got SIGTERM'd in a ~90s CrashLoop (clean exit-0 'shutdown
requested'). Bind a stdlib health server on the health port serving
/healthz + /readyz (+ /health), separate from the :8000 API so the health
surface never shares failure modes with the API stack. Graceful-shutdown it
alongside the app.
2026-06-25 13:57:10 -07:00
hanzo-dev 5946d93ea1 test(zapface): end-to-end WebSocket integration + unauth-reject
Drive a real ZAP binary frame through the full server path (WS upgrade ->
mintCap -> rpc.ParseRequest -> dispatch -> Fiber /v1/* mount -> casibase
envelope -> rpc.BuildResponse -> WS reply), asserting: real provider data
round-trips, the session cookie is replayed to the /v1 handler, query/body
mapping works, unknown method surfaces !ok, and an unauthenticated upgrade
fails closed with HTTP 401.
2026-06-25 13:46:53 -07:00
hanzo-dev 8d1b23b358 feat(zapface): browser ZAP-over-WebSocket plane at /zap
Bind the scaffolded ZAP face: a WebSocket endpoint at /zap that speaks the
@zap-proto/web wire (github.com/zap-proto/go/rpc envelope + console2's inner
ZapRequest/ZapReply structs) and dispatches each call into the EXISTING /v1
casibase handlers in-process (adaptor.FiberApp) — one dispatch path, two
transports, zero duplicated business logic.

- zapface/wire.go: inner ZapRequest{method@0,payload@8}/ZapReply{ok@0,status@4,
  result@8,errorJson@16} codec + SuperJSON envelope ({"json":V}).
- zapface/dispatch.go: (method,input) -> /v1 HTTP replay; casibase
  {status,msg,data} -> ZapReply; get-* => GET+query, mutations => POST+body.
- zapface/server.go: coder/websocket upgrade, cookie/bearer auth slot
  (mintCap, fail-closed), per-frame rpc.ParseRequest -> dispatch ->
  rpc.BuildResponse.
- serve.go: mount app.All("/zap", ...) after MountAll.
- config.go: CLOUD_ZAP_WEB_ORIGINS allowlist.
- deps: promote coder/websocket + zap-proto/go@v1.3.0 (rpc pkg) direct.

Wire proven byte-exact vs the REAL @zap-proto runtime console2 ships, both
directions (TS buildRequest -> Go parse; Go reply -> TS parseResponse +
SuperJSON.parse). go test ./zapface green.
2026-06-25 13:29:13 -07:00
zeekay ece1634bfd feat(serve): HIP-0113 ops listener (:9090 /healthz /readyz /metrics)
Decomplect health/ops from the product API: the ops endpoints move off the
app listener (:8000, /v1/*) onto cfg.HealthListenAddr (:9090), unauthenticated
and unversioned. Liveness (/healthz) and readiness (/readyz) are now distinct.
stdlib-only, zero new deps. Makes cloud the reference impl for HIP-0113 and
unbreaks the cloud-api probe (was /v1/health → 404 on the unified binary).
2026-06-25 12:42:17 -07:00
zeekay 34aa84e12f fix(build): unpoison luxfi/age go.sum + first-party-scoped sumdb skip
luxfi/age v1.5.0 was re-pointed to a newer commit; sum.golang.org pins
the first-seen hash immutably, so a fresh build fetching our own module
hit `verifying github.com/luxfi/age@v1.5.0: checksum mismatch · SECURITY
ERROR`.

- go.sum: re-record age v1.5.0 zip h1: to live content (G69Hb… → zC/Fw…);
  /go.mod hash was unchanged.
- Dockerfile: add explicit GONOSUMDB scope (first-party only) and drop the
  fragile `rm -f go.sum && go mod download` self-heal — it masked the stale
  go.sum and re-recorded unverified hashes on any transient error. Correct
  committed go.sum + GOPROXY=direct is the one durable way.

Never global GONOSUMDB=* / GOINSECURE. Root cause is the upstream
force-re-tag practice, which must stop.
2026-06-25 00:58:58 -07:00
zeekay 469f072611 ci(deploy): notify universe with image-update on release
cloud had no universe dispatch → never auto-deployed. Add the same
image-update notify-universe job gateway/iam use. One contract.
2026-06-25 00:54:13 -07:00
35535cb1d1 build(hip-0106): unpoison module graph + tidy unified cloud binary (#40)
* build: HIP-0106 unified binary builds pure-Go static

- commerce v1.42.5 -> v1.42.27 (pure-Go modernc SQLite + tracked embed catalogs)
- pin luxfi/kms v1.11.6 (past force-re-tagged v1.11.0)
- exclude legacy ugorji/go (gin msgpack ambiguity)
- regenerate go.sum clean (sumdb off; force-re-tag poisoning)
- Dockerfile: GOPRIVATE + GOSUMDB=off + GOPROXY=direct + gh_token secret

Produces CGO_ENABLED=0 static /cloud (206M); all subsystems link.
Follow-up: repin hanzoai/kms off the dead v1.0.x pseudo-version to v0.159.x.

* build: fresh-origin go.sum + Dockerfile self-heal (v0.2.1)

v0.2.0's go.sum was regenerated from a stale module cache, so a clean container
build mismatched the force-re-tagged hanzoai/kms/sdk/go. Regenerate from fresh
origin (matches Docker's fetch), add self-heal (rm go.sum + retry on poisoning)
+ GOFLAGS=-mod=mod. New tag (not a force-re-tag of v0.2.0 — that's the anti-pattern).

* feat(subsystems): unified cloud = app layer only; infra/edge run separately

Per CTO: the fused binary is the APPLICATION layer. Removed from subsystems.go:
- amqp (unused)
- iam → iam.hanzo.ai (Casdoor), kms → kms.hanzo.ai (luxfi/kms): isolated control plane
- mcp: own deployment
- gateway, ingress: the edge (route *to* this binary)

Keeps: ai, authz, base, commerce, licensing, metrics, o11y, vfs, plansvc, pricingsvc.
Binary 154M→79M; v0.3.0 ships uncompressed (no UPX). Validated: boots ready on
SQLite with the default set, infra/edge excluded.

* build(deps): bump zap-proto/go to v1.3.0 (indirect)

* build(hip-0106): unpoison module graph + tidy unified cloud binary

- re-record poisoned luxfi/* + hanzoai/* go.sum hashes (threshold/keys/kms
  and kms SDK force-re-tagged at same version; origin authoritative,
  GOSUMDB off + GONOSUMDB covers both orgs).
- replace mattn/go-sqlite3 v2.0.3+incompatible (deleted upstream tag) -> v1.14.16.
- go mod tidy drops the unimported direct require hanzoai/gateway
  v2.9.7+incompatible (gateway subsystem is a separate deploy, not yet
  wired into the unified binary's subsystems.go).
- cmd/cloud and cmd/hanzo build green for darwin + linux/amd64.

---------

Co-authored-by: zeekay <z@zeekay.io>
2026-06-24 19:21:21 -07:00
hanzo-dev c05e1c372e deps(kms): refresh hanzoai/kms@v0.159.1 hash after brand-scrub history rewrite
kms history was rewritten to remove white-label brand leaks
(Liquidity/redacted); the v0.159.1 tag now points at a rewritten commit
with a new tree hash. go.mod content unchanged (only h1: tree hash
changes). cloud builds green (go build ./... exit 0). go mod verify: all
modules verified.
2026-06-24 18:05:23 -07:00
hanzo-dev a7cc3dbf69 deps: bump luxfi/kms -> v1.11.7 (clean tag after OSS brand-hygiene history rewrite) 2026-06-24 17:07:03 -07:00
hanzo-dev 47cb805284 chore(deps): bump ai to d9c02eca (ratelimit tier ?user= fix)
Pulls hanzoai/ai#fix(ratelimit): tier lookup queries commerce by org slug
(?user=) instead of ?apiKey=, fixing the 400 that starved paid orgs of
their rate limits. No other dep changes (age/threshold lines reordered,
same immutable bits).
2026-06-24 04:39:22 -07:00
hanzo-dev c08d3a8a2d build: pin luxfi/age+threshold go.sum to PROXY bits (Dockerfile is proxy-first)
The prior commit refreshed these via local direct fetch (GOPRIVATE), recording
the GitHub-rewritten bits. The cloud Dockerfile fetches via proxy.golang.org
first (GONOPROXY=hanzoai only), so the build downloaded the proxy bits and
failed go.sum verification on threshold@v1.9.4/go.mod. Restored to the exact
proxy hashes from v1.785.13 (which built clean). Only the ai bump remains the
real go.mod/go.sum delta.
2026-06-23 22:07:54 -07:00
hanzo-dev 887a51599d deps(ai): bump to 83876bf0 — hk- API-key resolution uses /v1/iam/get-user
Pulls the ai fix where the controller hk- key lookup hit the legacy
/api/get-user (served as @hanzo/id SPA HTML, breaking API-key auth on
/v1/chat/completions). Refreshes go.sum for force-retagged luxfi/age@v1.5.0
and luxfi/threshold@v1.9.4 (upstream re-tag drift, GOPRIVATE — not caused by
this change). cloud (CGO_ENABLED=0) builds clean.
2026-06-23 22:00:55 -07:00
hanzo-dev 1386c70292 deps: bump ai -> 91659573 (complete per-org balance sweep)
Folds in the zap-native + scraper per-org balance fixes so EVERY balance
check (gate, controller backstop, ZAP premium gate, zap balance query,
scraper preflight) reads the one per-org balance. Final image for the
per-org billing unification.
2026-06-23 21:33:13 -07:00
hanzo-dev 577cb2dcfd deps: bump ai -> e6402611 (per-org balance backstop)
Completes the per-org billing unification: both the BalanceGateFilter AND the
resolveProviderForUser backstop now key by org slug + stamp X-Hanzo-Org, so the
single per-org credit is the balance every LLM call checks.
2026-06-23 21:27:09 -07:00
Hanzo 1c2da48aee deps: bump ai -> ee423689 (zen→DO-AI routing + provider secret self-heal)
Makes the LLM layer real:
- zen3/zen4/aliases re-pointed from dead Fireworks serverless to DO-AI
- do-ai provider key unified to kms://DO_AI_API_KEY (env-first resolution)
- provider re-seed self-heals ClientSecret/ProviderUrl/State on boot
2026-06-23 21:17:53 -07:00
hanzo-dev 98a044c669 build: resync luxfi go.sum to proxy/checksum-DB bits (fix re-tag drift)
Several luxfi modules were force-rewritten upstream so go.sum captured the
rewritten direct-fetch bits, which conflict with the proxy's checksum-DB
artifacts: luxfi/age v1.5.0, luxfi/threshold v1.9.4, luxfi/zap v0.8.8.
Combined with the GOPROXY split (luxfi via proxy, hanzoai/zap-proto direct),
go.sum now pins the proxy/sumdb-authoritative hashes. luxfi stays in GONOSUMDB
so the few proxy-absent versions (e.g. luxfi/constants@v1.5.8 → 404) fall to
direct without a sumdb-lookup error while still pinned by go.sum.

Validated locally with the exact Dockerfile env: full 'go mod download' +
'CGO_ENABLED=0 go build ./cmd/cloud' succeed, 'go mod verify' = all modules
verified, binary embeds ai v1.785.9-...-44cd5f9a (per-org balance gate).
2026-06-23 21:12:47 -07:00
hanzo-dev 4344031d91 build: split GONOPROXY/GONOSUMDB so luxfi/* resolves via proxy (fix age re-tag)
GOPRIVATE forces BOTH direct-fetch and sumdb-bypass for every match, so
luxfi/age went direct to GitHub and hit the force-rewritten v1.5.0 tag
(h1:KEjq... != go.sum/sum.golang.org h1:G69H...), failing go mod download.
All luxfi/* modules we use are on the public proxy, so drop luxfi/* from
GONOPROXY (keep only hanzoai/* + zap-proto/*, whose just-pushed pseudo-
versions the proxy 404s). luxfi/* now resolves via proxy.golang.org =
immutable checksum-DB bits matching go.sum. Validated locally with the exact
Dockerfile env: luxfi/age proxy-clean, hanzoai/ai direct-clean.
2026-06-23 21:04:20 -07:00
hanzo-dev 09f19a7828 build: proxy-first GOPROXY so force-rewritten upstream tags can't poison builds
luxfi/age v1.5.0 was re-pushed on GitHub with content differing from the bits
sum.golang.org recorded (h1:G69H... original vs h1:KEjq... rewritten), so the
GOPRIVATE-forced direct fetch failed go.sum verification. Public luxfi/* are all
on the proxy; resolve through proxy.golang.org first (immutable, checksum-DB
artifacts) and fall back to direct only for repos the proxy 404s (private). Pins
public deps to verified bits; private resolution unchanged.
2026-06-23 20:58:16 -07:00
hanzo-dev 3c46dd741e deps: bump ai -> 44cd5f9a (per-org LLM balance gate)
Unifies the LLM balance gate with commerce's per-org credit: the gate now
keys billing by org slug and stamps X-Hanzo-Org so a single per-org credit
(X-Org-Id=<org>) is the balance the gate checks and usage debits. Fixes
insufficient_balance on funded orgs (gate previously queried per-user in the
default 'hanzo' namespace).
2026-06-23 20:53:12 -07:00
hanzo-dev dd57a8db70 deps: bump ai -> f3a36aa2 (iam SDK /v1/iam GetUrl + signout nil-guard)
Fixes console2/cloud login end-to-end: the IAM SDK now loads the app cert from
/v1/iam/* so Signin's ParseJwtToken succeeds (was 'iamsdk: not valid PEM'),
establishing a real session that admin endpoints accept.
2026-06-22 01:01:45 -07:00
hanzo-dev 688f3425cd build: drop re-added stale luxfi/threshold go.sum entry (re-tagged upstream) 2026-06-21 20:55:49 -07:00
hanzo-dev 47de155564 deps: bump hanzoai/dbx -> 6b6ceb7 (composite fields as JSON)
Fixes get-account 'unsupported type []model.SearchResult, a slice of struct'
and the matching scan errors — Message.SearchResults/VectorScores/Suggestions/
ToolCalls and all slice/map model fields now round-trip via JSON in the data
layer. Completes SQLite-native cloud-api login.
2026-06-21 20:54:03 -07:00
hanzo-dev 46e74b1af3 build: tolerate re-pushed private module tags (GOFLAGS=-mod=mod)
luxfi/threshold@v1.9.4 is being re-tagged upstream, so its checksum drifts
from go.sum and 'go mod download' fails in CI. Record private-module hashes at
build time (-mod=mod; GOPRIVATE keeps them off the public sumdb) and drop the
stale threshold entry so it re-records cleanly.
2026-06-21 19:42:28 -07:00
hanzo-dev 4b175102f4 deps: bump hanzoai/ai -> 152107f4 (dbx.Sync creates casibase schema on SQLite)
Unblocks the Base/SQLite cloud-api: the ai subsystem now creates its tables
from the Go structs on a fresh embedded SQLite store (no external migrations).
2026-06-21 19:37:14 -07:00
hanzo-dev a98366d098 deps: bump hanzoai/ai -> v1.785.9-0...a98523d4 (StringList []string scan fix)
Pins the merged ai main commit that adds StringList (sql.Scanner/Valuer over
JSON) for list columns — fixes the casibase data-layer panic
'unsupported Scan ... string into *[]string' that broke OAuth sign-in.
2026-06-21 19:15:26 -07:00
hanzo-dev eb60e364d2 deps: bump hanzoai/ai v1.785.7 -> v1.785.8 (CopyRequestBody for POST body parsing)
v1.785.8 sets beego CopyRequestBody in Bootstrap so the unified binary's AI
controllers can read POST bodies (json.Unmarshal of c.Ctx.Input.RequestBody);
without it /v1/chat/completions returned 'unexpected end of JSON input'. Completes
the unified-AI serve path: routing (bare /v1/*) + no-panic (session mgr) +
scratch-safe (memory sessions) + body parsing (CopyRequestBody).
2026-06-21 10:56:59 -07:00
hanzo-dev 27c7b46b2b deps: bump hanzoai/ai v1.785.6 -> v1.785.7 (memory session provider for scratch image)
v1.785.7 adds the scratch-safe memory session provider on top of the bare /v1/*
mount + session-manager build. Without it the unified binary 503'd on every
request (file session provider can't write in the read-only scratch root). With
all three fixes, /v1/chat/completions and the other OpenAI routes serve through
the unified binary.
2026-06-21 10:40:18 -07:00
hanzo-dev 632376b753 fix(pricing): drop bare /v1/models alias so AI owns the OpenAI model list
In the unified binary the pricing subsystem (order 112) mounted a bare /v1/models
alias that shadowed the AI subsystem's (order 150) OpenAI-compatible /v1/models —
the {data:[{id,…}]} model list the api.hanzo.ai gateway forwards to cloud-api and
clients (cowork model picker) consume. Pricing's annotated catalog already lives
at /v1/pricing/models, so the bare alias only introduced a shape regression.
Remove it; pricing stays strictly under /v1/pricing/*. Now /v1/models, like the
other OpenAI routes, resolves to AI's beego handler via its /v1/* catch-all.
2026-06-21 10:32:25 -07:00
hanzo-dev 3176460ddc deps: bump hanzoai/ai v1.785.5 -> v1.785.6 (bare /v1/* mount + session manager)
v1.785.6 carries BOTH unified-binary fixes:
1. AI mounts casibase routes at bare /v1/* (not /v1/ai/*) so the api.hanzo.ai
   gateway, which forwards /v1/chat/completions etc. unchanged, resolves.
2. beego session manager built in Bootstrap so forwarded requests don't panic.

Together these make /v1/chat/completions, /v1/chat, /v1/models, /v1/messages
serve through the unified binary exactly as the gateway sends them.
2026-06-21 10:29:05 -07:00
hanzo-dev 13238567d6 deps: bump hanzoai/ai v1.785.4 -> v1.785.5 (build beego session manager in Bootstrap)
ai v1.785.5 fixes the embedded /v1/ai/* HTTP 500: the unified binary never
calls beego.Run(), so beego.GlobalSessions was nil and every forwarded request
panicked in SessionStart. v1.785.5 builds the session manager in the shared
Bootstrap, so /v1/ai/chat/completions, /v1/ai/models and all nested routes
serve. Cloud binary builds green against it.
2026-06-21 10:14:16 -07:00
hanzo-dev f6b79a0a74 ci(build): self-contained arcd build, GHCR login via GH_PAT
The ghcr.io/hanzoai/cloud package is linked to hanzoai/ai (cloud->ai rename),
so this repo GITHUB_TOKEN is denied write (permission_denied: write_package) via
the shared workflow. Build self-contained on the hanzo-build-linux-amd64 scale
set and log into GHCR with GH_PAT (admin:org+write:packages). gh_token still
feeds the Dockerfile private-module fetch. Dropped GHA cache (same denial +
artifact quota).
2026-06-21 09:31:43 -07:00
hanzo-dev 53b1fb49b9 deps: consume gateway/v2 v2.14.8 (proper /v2 module path)
gateways v2 tags were invalid Go modules (go.mod lacked the /v2 path), so
gateway v2.9.7+incompatible could not resolve on a clean fetch and cloud failed
to build. gateway v2.14.8 fixes the module path; import the /v2 path in the
subsystems bundle and pin v2.14.8. Build + go mod verify clean; binary boots
with no init panic.
2026-06-21 09:23:34 -07:00
hanzo-dev dca49c1d52 deps: pin gateway v2.9.6+incompatible (v2.9.7 added a go.mod without /v2 path)
gateway v2.9.7 introduced a go.mod still declaring module path
github.com/hanzoai/gateway at a v2 tag, which makes v2.9.7+incompatible an
invalid version (a module with a go.mod at major>=2 must use a /vN path).
v2.9.6 is the last go.mod-free v2 tag, so +incompatible is valid there; API is
identical for the subsystem blank-import. One patch down, no code change.
2026-06-21 09:10:37 -07:00
hanzo-dev cbc792f075 deps: refresh go.sum for force-pushed private tags (hanzoai/base v1.3.2, luxfi/threshold v1.9.4, ...)
Several private module tags were re-tagged after the committed go.sum was
generated, so a clean CI fetch failed go.sum verification (SECURITY ERROR:
checksum mismatch). Regenerated the affected private entries from current
remote content via go build -mod=mod (versions unchanged). Build is green and
boots without panic; go mod verify passes.
2026-06-21 09:03:59 -07:00
hanzo-dev 50e033e570 ci(build): route amd64 to ARC scale set hanzo-build-linux-amd64 by name
ARC ephemeral runners only match jobs targeting the scale-set name as a label.
The shared workflows default [self-hosted,linux,amd64] matches only classic
static runners (evo pool, offline) so the job sat queued with the listener
reporting assigned-job=0. gateways successful builds use runs-on:
hanzo-build-linux-amd64 (runner hanzo-build-linux-amd64-cvs28-runner-*); pass
that as runner-amd64.
2026-06-21 08:22:13 -07:00
hanzo-dev 1ed94b79a6 ci(build): delegate to shared arcd docker-build.yml (self-hosted, billing-immune)
GitHub-hosted runners for this org are billing-frozen (jobs fail in ~5s:
"recent account payments have failed"), so the bespoke ubuntu-latest release
workflow can never start. Use the canonical hanzoai/.github docker-build.yml
reusable workflow, which runs on the self-hosted arcd pools and injects GH_PAT
as the gh_token BuildKit secret the Dockerfile needs for private cross-org Go
modules. amd64-only (cluster arch) to complete without the arm64 pool.
2026-06-21 08:09:19 -07:00
hanzo-dev 9bf6f04235 ci(build): authenticate private cross-org Go modules in image build
The unified binary pulls private hanzoai/* AND luxfi/* modules; the public
proxy 404s on them and the default GITHUB_TOKEN cannot read cross-org repos, so
the Docker build failed at go mod download.

- Dockerfile: split deps layer; GOPRIVATE + BuildKit gh_token secret +
  git insteadOf to fetch private modules over authenticated git (mirrors the
  proven hanzoai/ai Dockerfile). COPY --chmod=0755 the binary so the scratch
  image can never ship a non-executable /cloud (the 0644 CrashLoop class).
- release.yml: source the gh_token from the org GH_PAT (cross-org RO PAT that
  actually exists), not the never-configured HANZO_GH_RO_TOKEN.
2026-06-21 08:07:41 -07:00
hanzo-dev 8a6be4c683 deps: cut cloud-api onto unified binary — ai v1.785.4 (+#29 runtime init, /v1/billing path), beego v2.3.10 grace guard, go-openai fork
Fixes user-token AI on api.hanzo.ai:
- ai v1.785.4: AI runtime initializes in Mount() (#29) so /v1/ai/* serves real
  completions (no more 503 "ai runtime not initialized"); ai self-meters
  Commerce on the correct /v1/billing/* path (1.784.2 casibase used the dead
  /api/v1/billing/* -> 404 for real user JWTs, blocking the prepaid balance
  gate + usage auto-debit).
- beego v2.3.10: grace flag-registration guard so the unified binary (ai beego
  v1 + iam beego v2) does not panic ("flag redefined: graceful") at init.
- ai v1.785.4 also swaps deprecated denisenkom/go-mssqldb -> maintained
  microsoft/go-mssqldb (one mssql driver registration; no "sql.Register called
  twice" panic).
- replace sashabaranov/go-openai => hanzoai/go-openai v1.40.0 (ReasoningContent
  field) — mirrors ai own replace, which does not transit to this main module.

Verified: CGO_ENABLED=0 go build ./cmd/cloud produces a 316MB executable that
boots clean (base + full subsystem set) with no init panic; full boot stops
only on expected in-cluster config (IAM_KEYS_URL/IAM_AUDIENCE), supplied by the
cloud-api CR env.
2026-06-21 08:05:32 -07:00
Antje Worring c47f89f85e cloud: pin commerce/metering v0.1.0 (drop local replace) 2026-06-20 17:31:18 -07:00
Antje Worring 8d7d324a11 cloud: zip-native fail-closed billing gate (wraps commerce/metering) 2026-06-20 17:13:49 -07:00
z 1579c773ac deps: pin luxfi/kms v1.11.6 (published) — pkg/iam v1.18.5 referenced phantom v1.11.3 2026-06-19 01:15:08 -07:00
z 6f7bd64ca4 deps: pin goldap-free iam (pkg/iam v1.18.5, iam v1.19.6) — resell-clean (zero GPL-2.0) 2026-06-19 01:08:48 -07:00
antje 7a2d11c8e5 chore: add LICENSE (Apache-2.0) — Hanzo-native 2026-06-19 00:39:25 -07:00
hanzo-dev 4f77c15de0 chore: add Apache-2.0 LICENSE (Copyright 2026 Hanzo AI Inc) 2026-06-18 23:40:49 -07:00
bf48e97120 refactor(cmd): single subsystems bundle — define the mounted set once (DRY) (#19)
cmd/cloud and cmd/hanzo each blank-imported the same 16-subsystem list, so
adding/removing a subsystem meant editing two files (repeat-yourself). Move
the list into one package, github.com/hanzoai/cloud/subsystems; both
entrypoints blank-import only that. One source of truth for what's linked into
a Hanzo binary — dispatcher and full-surface binary mount an identical set by
construction.

(Bundle is a sibling subpackage, not the root cloud package: subsystems import
cloud for Deps+Register, so a root bundle would cycle.)

Verified: go build -tags 'cloud cloud_mount' . ./subsystems ./cmd/cloud
./cmd/hanzo green (-mod=readonly); hanzo --help still lists 18 subcommands
(16 subsystems + cloud + datastore); go test ./... green.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-17 09:29:28 -07:00
b85f0eaa66 fix(deps): commerce v1.39.1->v1.42.5 — mount commerce into the unified binary (#18)
cloud pinned commerce v1.39.1, which predates commerce's cloud-mount
integration: cloud.Register("commerce", 100, ...) in init() behind
//go:build cloud, added at v1.42.5. So commerce silently was NOT in the fused
surface or hanzo's subcommands. v1.42.5 (latest tag) registers correctly —
`hanzo --help` now lists commerce; the unified binary composes 16 subsystems.
iam stays v1.19.4 (no cascade). go build -tags 'cloud cloud_mount' + full
go test ./... green.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-17 09:21:19 -07:00
050bc2b5c8 feat(cmd): land unified hanzo binary + extract shared cloud.Serve (DRY) (#17)
Adds cmd/hanzo — one binary dispatched by subcommand: `hanzo <svc>` serves one
subsystem, `hanzo cloud` the full fused surface, `hanzo iam` the standalone
Beego IdP (iamserver.Run), `hanzo datastore` documents the ClickHouse boundary.

DRY: the cloud-server body (compose root + HIP-0106 /v1/<name>/health contract
+ graceful shutdown) is extracted from cmd/cloud's main() into cloud.Serve(enable)
— the ONE shared place. cmd/cloud now calls cloud.Serve(nil), gaining graceful
shutdown + health endpoints (strict superset of its prior body, no regression).
cmd/hanzo dispatches through the same cloud.Serve.

Beego non-collision holds: iam registers routes inside iamserver.Init(), not
package init(); one Beego v2 path; visor (Beego v1) intentionally unlinked.
iam v1.19.4 moves indirect->direct (cmd/hanzo imports iam/iamserver).

Verified: go build -tags 'cloud cloud_mount' . ./cmd/cloud ./cmd/hanzo green
(-mod=readonly), go vet clean, hanzo --help lists 17 subcommands.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-17 05:47:13 -07:00
8e5b85f7f3 fix(deps): iam v1.19.1→v1.19.4 + pkg/iam v1.18.1→v1.18.4 — fix cloud-api boot panic (#16)
iam v1.19.0–v1.19.3 panic at boot: routers/router.go registered
GET /v1/iam/run-authz-command → ApiController.RunAuthzCommand, a method never
added, so Beego panics at route registration when iamserver.Init() runs (this
hit cloud-api too). v1.19.4 (cut from iam main: dangling route removed +
initAdminUser seeds via conf.AdminOrg) fixes it. Transitive zap-proto/go
v0.3.0→v1.1.0 required by pkg/iam v1.18.4. go build -tags 'cloud cloud_mount'
./cmd/cloud green; -mod=readonly verified.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-17 05:36:42 -07:00
414229184b fix(go.sum): re-record re-tagged hanzoai/lux module checksums (unblocks build) (#15)
A chain of re-tagged modules produced checksum SECURITY ERRORs that
block 'go build ./...':
  - github.com/hanzoai/base@v1.3.2          (h1)
  - github.com/luxfi/threshold@v1.9.4       (h1)
  - github.com/hanzoai/kms/sdk/go@v1.0.0    (h1 + go.mod)
Re-record the current proxy hashes. go build ./... -> exit 0 (cmd/cloud links).

Co-authored-by: zooqueen <dev@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-17 00:53:19 -07:00
Abhishek KrishnaandGitHub fd386b3b1a fix(go.sum): refresh stale hash for retagged luxfi/threshold (#7)
github.com/luxfi/threshold v1.9.4 was re-pushed at the same version
tag without updating go.sum in this repo. The recorded h1: digest no
longer matches the upstream zip on proxy.golang.org, so cold clones
fail at `go mod verify` / `go build`:

  verifying github.com/luxfi/threshold@v1.9.4: checksum mismatch
      downloaded: h1:H69e9QkvygDtWkni1FkD5ztLdiTasmpJXcuVzgebdxo=
      go.sum:     h1:/TsgIzo/e/DIx++J0+9eNuS7HkpSaXbVk+HvhlUOsmE=
  SECURITY ERROR

Regenerating go.sum (single line update) restores parity with
upstream and the build succeeds.

Reproduction (pre-fix):
  git clone https://github.com/hanzoai/cloud
  cd cloud && GOPRIVATE='github.com/hanzoai/*,github.com/luxfi/*' \
    go build ./cmd/cloud
  # verifying github.com/luxfi/threshold@v1.9.4: checksum mismatch

No code changes outside go.sum.
2026-06-16 20:28:51 -07:00
Abhishek KrishnaandGitHub bf6dbad118 ci: add release workflow to publish ghcr.io/hanzoai/cloud (#8)
The README advertises `docker run -p 8080:8080 ghcr.io/hanzoai/cloud:latest`
but there's no release pipeline in the tree to produce that image.
This workflow publishes the Docker image on git-tag push (matching
v*), on default-branch push (as :latest and :sha-<short>), and on
manual dispatch.

Includes a Buildx secret pattern for fetching private upstream
modules (hanzoai/iam, hanzoai/commerce, hanzoai/gateway, ...) using
either the default GITHUB_TOKEN or an org-level HANZO_GH_RO_TOKEN
when cross-org private reads are required.
2026-06-16 20:28:47 -07:00
Abhishek KrishnaandGitHub 7b0200ab6a chore: add helm/cloud minimal chart (#9)
Reference Helm chart that renders a single Deployment + Service +
optional PVC for the unified cloud binary. Mirrors the binary's
CLI flags via .Values.hanzo (brand, domain, dataDir, iamIssuer,
enable). Pod runs as nonroot UID 65532 matching the Dockerfile.

Not a substitute for luxfi/operator + Service CRD — this chart is
for users who want raw k8s manifests without installing the
operator first.
2026-06-16 20:28:44 -07:00
Abhishek KrishnaandGitHub 1afeeb041f chore: add deploy/compose.yml for single-node VPS deployment (#6)
Reference Docker Compose manifest for the unified cloud binary, with
matching .env.example and a deploy/README.md that documents required
vs optional environment variables (the binary refuses to mount IAM
without HANZO_IAM_ISSUER, so make that explicit).

No code or config changes outside deploy/.
2026-06-16 20:28:41 -07:00
Abhishek KrishnaandGitHub 7ab0d887bb chore(scripts): add scripts/smoke-runtime.sh — boot + probe the real binary (#11)
`cmd/cloud-smoke` (existing) exercises the in-process mount path on a
mock zip.App with two health endpoints. It does not boot the actual
`cmd/cloud` binary or hit the HTTP surface customers will use.

This script closes that gap: it builds `./cmd/cloud`, boots it under
the same default-safe `--enable` list used in deployments (omits the
`iam` subsystem until the v1.19.2 boot panic is patched), waits for
the listener to bind, then probes the five endpoints whose expected
status is fixed by the HIP-0106 contract:

  /healthz                200   process health probe
  /v1/models              200   model catalog (no auth)
  /v1/plans               200   plansvc (goja-hosted)
  /v1/pricing             200   pricingsvc (goja-hosted)
  /v1/base/collections    401   base alive, auth-gated

If any probe regresses, the script dumps the tail of the boot log and
exits non-zero — making it usable as a CI gate and as a local "does my
clone actually serve?" check.

Env knobs (`PORT`, `LISTEN`, `BIN`, `DATA_DIR`, `ENABLE`,
`KEEP_RUNNING`, `BOOT_TIMEOUT`) let it drop into different
environments without a Makefile change. The `IAM_*` env vars default
to the production hanzo.id JWKS — required by `kms` for inbound JWT
validation even with `iam` disabled — and can be overridden per
deployment.

Pairs with the Makefile in #5: `make smoke` already exists for the
mount-time path; this is the runtime counterpart and can be wired as a
sibling target (`make smoke-runtime`) in a follow-up once #5 lands.
2026-06-16 20:28:37 -07:00
Abhishek KrishnaandGitHub f65f264567 chore: add Makefile with build/test/docker targets (#5)
Minimal developer ergonomics for the unified cloud binary. Targets
wrap go build / go test / docker build for the existing Dockerfile,
plus a `make run` shortcut that matches the README quickstart
(--enable=iam,base,kms,gateway,o11y).

No code changes outside the new Makefile.
2026-06-16 20:28:34 -07:00
Abhishek KrishnaandGitHub 8c72dbf11f chore: add .gitignore and .dockerignore (#10)
Repo currently has neither file. Two practical consequences:

1. `docker build .` copies the entire context including `.git/`, IDE
   metadata, OS detritus (`.DS_Store`), and any local `.env` — bloats
   the build context and risks baking secrets into image layers.
2. Without a `.gitignore`, the build output binary (`/cloud` per the
   `Dockerfile` final stage), local `.env` files, and editor leftovers
   are easy to commit by accident.

Both files cover the standard Go-project surface (binary at `/cloud`,
test outputs, coverage, env files), plus IDE/OS noise. The
`.dockerignore` additionally drops docs and tests so they don't enter
the runtime image — the binary is what ships, the README lives on
GitHub.

No behavior change; the binary the Dockerfile builds is bit-identical.
What changes is build-context size and the safety margin around
accidental commits.
2026-06-16 20:28:30 -07:00
70363681b0 fix(auth): default CLOUD_IAM_ISSUER to https://iam.hanzo.ai (was .id typo) (#14)
IAM issues JWTs with iss=https://iam.hanzo.ai, but cloud-api defaulted the
expected issuer to https://iam.hanzo.id — so EVERY hanzo.id-login JWT was
rejected with 'invalid issuer claim (iss)' and the AI gateway's JWT auth path
was dead for all users (only hk-*/sk-* keys worked). One-char .id->.ai fix.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-16 17:55:57 -07:00
hanzo-devandGitHub 45dd3c4a83 test(cloud): drop cmd/cloud-smoke, add real orchestrator integration test (#13)
The cloud-smoke command was a throwaway harness that hand-mounted fake health
routes and avoided the real subsystem matrix (citing build issues in cmd/cloud
that are now fixed — the full binary builds). Replaced with a proper go test in
package main that exercises the actual path:

- TestRegistryAssemblesSubsystems: every subsystem main.go imports self-registers
  via init() into cloud.Registry (proves the unified binary wires the matrix).
- TestMountAllAndServeHealth: BuildDeps -> MountAll -> serve; the self-contained
  subsystems (base, authz, amqp, metrics, plans, pricing) mount in-process and
  serve /v1/<name>/health = 200 via the real zip/fiber + jsonenc stack
  (app.Fiber().Test, no listener / external services).
- TestDepGatedSubsystemsFailClosed: ai, o11y mount and return >=500 from the
  disabled-dep stub — proving the BuildDeps three-mode contract end to end.

Discovered (not fixed here — separate subsystem bug): enabling iam panics with
"'RunAuthzCommand' method doesn't exist in the controller ApiController".

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-16 09:56:21 -07:00
hanzo-devandGitHub ea46670309 fix(go.sum): refresh moved private-tag hashes so it builds clean (#12)
hanzoai/base@v1.3.2 and hanzoai/kms/sdk/go@v1.0.0 were re-tagged upstream, so
the recorded go.sum hashes no longer matched — `go build`/`go test` failed with
checksum mismatch (SECURITY ERROR) for anyone fetching fresh. Refreshed the
private-module hashes against the current tags.

Verified: go build ./... and go test ./... pass in default -mod=readonly mode.
No local replace directives (the 9 replaces are all published version pins for
the krakend/traefik gateway stack). cmd/cloud links to a single 272M binary.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-16 09:47:05 -07:00
Antje Worringandhanzo-dev 20f9df47c5 feat(cloud): pin metrics v0.4.0 — ZAP MsgMetricBatch receiver + per-tenant + durable
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-15 14:15:56 -07:00
Antje Worringandhanzo-dev 7c0d39e316 feat(cloud): pin metrics v0.3.0 — per-tenant observability isolation (X-Org-Id)
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-10 13:44:48 -07:00
Antje Worringandhanzo-dev 13911e452c feat(cloud): pin metrics v0.2.0 — durable native metrics+logs+traces
The unified binary now serves the full native observability stack at
/v1/{metrics,logs,traces}/* — WAL-durable (survives restart, verified), zero
prometheus, zero Grafana. This is the working replacement for the Loki/Tempo/
SigNoz vendoring.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-10 08:20:43 -07:00
Antje Worringandhanzo-dev 275650bb89 feat(cloud): mount native hanzoai/metrics v0.1.0 (ZAP-native metrics store)
The prometheus-free replacement for the Grafana/Prometheus observability
backends. Registers at order 40, serves /v1/metrics/{health,batch,write,query};
ingests luxfi/metric.MetricBatch (the ZAP MsgMetricBatch wire shape). Verified
live: write+query and batch+query roundtrips return correct series. Binary stays
at zero prometheus. (Also refreshed the stale kms/sdk/go go.sum hash from the
wave's re-tag.)

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-09 22:24:44 -07:00
hanzo-dev 9530b85e8b build(deps): re-pin plans/pricing/licensing to release tags; drop local replace
Replace the dev pseudo-versions + the local `replace github.com/hanzoai/licensing
=> ../licensing` directive with the published release tags now that the three
subsystems are merged + tagged:

  - github.com/hanzoai/plans     v1.2.0  (was pseudo @147eced7)
  - github.com/hanzoai/pricing   v1.3.0  (was pseudo @0c4b4c12)
  - github.com/hanzoai/licensing v0.1.0  (was v0.0.0 + replace => ../licensing)

licensing@v0.1.0 requires github.com/hanzoai/cloud@v0.0.0-00010101...; that
self-reference resolves to this main module, so no replace is needed. go.mod/
go.sum are tidy and `go build -mod=readonly ./cmd/cloud` produces the 303M
unified binary (boots with --enable=plans,pricing,licensing; all health 200).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-08 19:58:14 -07:00
hanzo-dev 0c44327f7b Merge feat/plans-pricing-goja-clean into feat/mount-licensing
Combine the licensing Mount (PR #3) with the plans+pricing goja mounts
(PR #4) onto one branch for the unified-binary re-pin.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-08 19:54:22 -07:00
hanzo-dev cfbf5e014b feat(plans+pricing): mount @hanzo/plans + @hanzo/pricing via base+goja
Mount the Node @hanzo/plans (data) and @hanzo/pricing (Express) services
INTO the unified cloud binary under /v1/plans/* and /v1/pricing/*, running
their JS inside the dop251/goja engine (the same engine base/plugins/gojavm
uses) — per HIP-0106. No service rewrite; Ship-of-Theseus to pure Go later.

New packages (the glue — clean module boundaries, no service source copied):
  - clients/gojahost: a reusable goja VM-pool host. Compiles a service repo's
    goja/bundle.js once, pre-warms a runtime pool (mirrors gojavm's pool +
    compile-once + per-runtime ensureLoaded discipline), injects the catalog
    JSON as globals, and dispatches handle({route,params,tenant}) -> {status,
    body} with ctx-cancel interrupt. Eager-loads one runtime so a bad bundle
    fails at Mount, not first request.
  - clients/plansvc: Mount(app, deps) for /v1/plans/*. Loads the @hanzo/plans
    bundle + embedded catalog, registers zip routes (subscriptions, cloud,
    blockchain, dns, gpu, regions, storage, tools, policy, schema, vocab,
    resolve/:id, entitlements/:id), threads X-Org-Id as the tenant for
    per-reseller (tenant_id,id) catalog scoping. The entitlements.mjs
    transforms (fromLegacy/toLicenseFeatures/resolvePlan) run in goja.
  - clients/pricingsvc: Mount(app, deps) for /v1/pricing/* + /v1/models.
    Express does NOT run in goja, so the Express transport is dropped; the
    server.mjs read handlers run in goja via the bundle. The sync.mjs markup
    (toMTok/processOpenRouterModel/...) also runs in goja via applyMarkup();
    the admin-gated POST /v1/pricing/sync does the live OpenRouter fetch in Go
    (net/http) and feeds raw JSON into the goja markup. _internal (provider
    costs/routing) is stripped from public responses.

Wiring: cmd/cloud/main.go blank-imports both wrappers; each init() calls
cloud.Register (plans order 111, pricing 112, after iam/commerce/licensing).
go.mod references hanzoai/plans + hanzoai/pricing as their own private Go
modules (the JS + data live there, embedded; nothing copied into cloud).

Tests: clients/{gojahost,plansvc,pricingsvc}/*_test.go exercise the real
embedded bundles (vocab, resolve+license_features, 404s, _internal strip,
exact markup math). Verified end-to-end: binary boots with --enable=plans,
pricing; all routes serve real data through goja over HTTP; X-Org-Id tenant
scoping confirmed (reseller override-wins, isolation holds).

Also corrects a stale go.sum entry for github.com/hanzoai/kms/sdk/go@v1.0.0
(the module was retagged; the recorded hash no longer matched the origin,
blocking any build that pulls base/iam/commerce -> kms). Updated to the
current origin hash.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-08 18:42:54 -07:00
hanzo-dev b714cf28ac feat(cloud): mount licensing subsystem + commerce entitlement-copy
Wire the private hanzoai/licensing Go subsystem into the unified cloud
binary per HIP-0106, following the iam/commerce/ai Mount(app, deps)
pattern. Clean module boundary: licensing is imported as its own private
module (its mount.go self-registers via init(); cmd/cloud blank-imports
it). No subsystem source is copied into cloud — the signer/fingerprint
secret logic stays in the licensing module.

- cmd/cloud: blank-import github.com/hanzoai/licensing (order 110, after
  iam=50 and commerce=100, since the /v1/licensing/issue flow depends on
  both identity and entitlements).
- go.mod: require licensing + local replace (co-developed private module;
  production resolves via tag/pseudo-version, drop the replace).
- types.CommerceClient: add CheckEntitlement(ctx, orgID, productID) plus
  the LicenseEntitlement transport type. This is the entitlement flow that
  gates issuance: commerce answers "does this tenant own the licensed
  product?" and returns the plan's FLAT license-features per the
  @hanzo/plans toLicenseFeatures vocab contract; the licensing mount copies
  them verbatim into the signed token's `features` so the engine enforces
  exactly the plan that was bought.
- clients: implement CheckEntitlement on the disabled (fail-closed) and
  ZAP-RPC commerce stubs; in-process pass-through already satisfies it.

Tenant-scoped via orgID (X-Org-Id). Real KMS stays a licensing follow-up
(scaffold TODO); the Mount + entitlement-copy are the deliverable here.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-08 18:23:19 -07:00
Antje Worringandhanzo-dev 539a6b1d68 fix(cloud): pin gateway v2.9.7 (clean, no local replaces)
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-08 17:30:42 -07:00
Antje Worringandhanzo-dev ac6d1c0abd fix(cloud): correct gateway pin to v2.9.6+incompatible (prev go.mod was broken)
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-08 16:46:14 -07:00
Antje Worringandhanzo-dev dcaa461d0e fix(cloud): pin gateway v2.9.6 — ZERO prometheus in the unified binary
gateway dropped the legacy opencensus SaaS exporters (stackdriver was the last
prometheus source). Combined with o11y v1.3.7 + alertmanager/krakend-otel forks +
base v1.3.2 + kms Corona, the binary now links zero real prometheus packages.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-08 16:45:19 -07:00
Antje Worringandhanzo-dev 7e28f27c38 fix(cloud): pin base v1.3.2 — network metrics on luxfi/metric (prometheus 6->1)
Real prometheus in the unified binary is now a single leaf package
(prometheus/prometheus/model/value via a gateway dep). Down from 16+.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-08 16:44:05 -07:00
Antje Worringandhanzo-dev 1080b7fc6e fix(cloud): prometheus 16->6 pkgs — krakend-otel fork + gateway v2.9.5
replace krakend-otel => hanzoai/krakend-otel v0.13.1 (prom-free fork); pin
gateway v2.9.5 (opencensus prometheus exporter removed, counters -> luxfi/metric).
With the alertmanager fork + o11y v1.3.7 + iam v1.18.1, the only prometheus left
is the hanzoai/common+alertmanager fork shim core (6 pkgs) — needs those forks to
shed prometheus/common internally.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-08 16:41:05 -07:00
Antje Worringandhanzo-dev 1f3fed619e fix(cloud): pin iam/pkg/iam v1.18.1 — kill duplicate-beego graceful flag panic
pkg/iam v1.18.0 imported upstream github.com/beego/beego/v2 while the rest of the
binary uses the hanzoai/beego prom-free fork; both register a global 'graceful'
flag in init() -> panic at startup. v1.18.1 (already fixed on iam main, just
untagged) uses the fork only. The unified binary now boots and shows the
white-label -brand/-domain/-enable tenancy flags.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-08 16:11:47 -07:00
Antje Worringandhanzo-dev d278bb53f5 fix(cloud): green the unified binary — o11y v1.3.7 + alertmanager fork replace
o11y's prometheus->hanzoai/alertmanager replace must be carried by the main
module (cloud), since a dependency's replace is ignored by consumers. With
o11y v1.3.7 (no-prometheus) + kms v0.159.1 (Corona), the full HIP-0106 binary
(11 subsystems on zip+ZAP, /v1 routing, luxfi/log+metric) compiles end-to-end.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-08 16:06:39 -07:00
Antje Worringandhanzo-dev 4e02c1814e fix(deps): pin hanzoai/kms v0.159.1 (Corona signing fix)
Clears the kms SignWithRingtail blocker. cloud's remaining build failure is
hanzoai/o11y v1.3.6 (incomplete prometheus/common -> hanzoai/common fork).

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-08 12:49:18 -07:00
Antje Worringandhanzo-dev 747d624b2a fix(deps): drop 13 cross-repo local replaces; pin siblings to real published versions
Build still blocked separately on kms SignWithRingtail (luxfi/kms API gap) and
o11y type-mixing — tracked upstream; this lands the no-local-replace requirement.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-06-08 12:11:00 -07:00
947 changed files with 135706 additions and 7944 deletions
-12
View File
@@ -1,12 +0,0 @@
# ~7-line canonical caller — all real config lives in /hanzo.yml.
# Test gate on OUR arc pool; release.yml owns the image + v* tags.
name: CI/CD
on:
workflow_dispatch:
push:
branches: [main]
pull_request:
jobs:
cicd:
uses: hanzoai/ci/.github/workflows/build.yml@v1
secrets: inherit
-155
View File
@@ -1,155 +0,0 @@
name: containment
# Guards the Stage-1 byzantine ceremony containment (clients/controlplane,
# build tag `controlplane`). Its increment-1 crypto is stub/forgeable BY
# DESIGN (SHA256-of-public-inputs commitments, symmetric-HMAC
# proof-of-possession, seed-derived threshold shares — see
# clients/controlplane/doc.go) and MUST NEVER reach a release/serve binary.
# Three independent checks; any one failing blocks the PR:
#
# 1. grep (tag) — no build/release invocation anywhere in the repo
# (Dockerfile, Makefile, shell scripts, any workflow) may pass a `-tags`
# value containing `controlplane` to go build/vet/run/install. The
# package's own `//go:build controlplane` tag declarations
# (clients/controlplane/*) are the thing being guarded, not a violation,
# and are excluded by path.
# 2. grep (spoof) — no build/release invocation may pass
# `-X testing.testBinary=1` (or any -ldflags containing it) to a REAL
# `go build`. That linker flag is what `go test` itself uses to make
# testing.Testing() report true (cmd/go/internal/load/test.go) — the
# runtime guard in containment.go trusts that signal, so this is the one
# concrete way to spoof it in a non-test binary. This grep is what turns
# "someone could type this" into "CI fails the PR that types it".
# 3. build — `go build ./...` (no tag — exactly what the Dockerfile
# and Makefile run) must not link clients/controlplane into any cmd/
# main, and `go build ./clients/controlplane/...` with no tag must match
# zero buildable packages (proves the tag still gates every file in it).
#
# Runtime belt-and-suspenders (defense in depth, not a substitute for the
# above): clients/controlplane/containment.go fail-closed panics if its stub
# crypto is ever constructed outside a go-test binary (testing.Testing()==
# false) — see TestContainment_NonHarnessProcessRefuses in
# containment_test.go. KNOWN RESIDUAL: testing.Testing() is a linker-set
# string var (testing.testBinary), not cryptographically bound to "actually
# is a test" — `-ldflags="-X testing.testBinary=1"` spoofs it in a real
# binary. Check #2 above is the mitigation: it fails the PR that would ship
# that flag. Closing the residual for real needs a signal `go build` cannot
# produce at all (increment-2, tracked in doc.go) rather than one merely
# absent by convention.
on:
pull_request:
push:
branches: [main]
permissions:
contents: read
jobs:
controlplane-containment:
runs-on: [hanzo-build-linux-amd64]
steps:
- uses: actions/checkout@v4
- name: grep — no build/release path may set -tags controlplane, or spoof testing.Testing()
run: |
set -euo pipefail
hits=0
# NOTE: exclusions are plain substring matches on the path (not
# anchored to a leading "./") so this is robust across grep
# implementations that format recursive-search paths differently.
# char class includes `!` so a build-constraint negation form
# (`-tags '!x,controlplane'`) cannot slip the grep. (Belt only: the
# positive-proof step below is the syntax-agnostic guarantee — the
# package has zero untagged files, so importing it into serve code
# fails the untagged `go build ./...` regardless of any -tags syntax.)
if grep -RnE -- '-tags[= ]*["'"'"']?[!A-Za-z0-9_, ]*\bcontrolplane\b' \
--exclude-dir=.git --exclude-dir=node_modules --exclude-dir=.claude --exclude-dir=vendor \
. 2>/dev/null \
| grep -v '\.git/' \
| grep -v 'clients/controlplane/' \
| grep -v '.github/workflows/containment.yml:'; then
echo "::error::found a build/release invocation passing -tags controlplane — clients/controlplane's stub crypto must never enter a release/serve binary (see clients/controlplane/doc.go)"
hits=1
fi
if grep -RnE -- 'testing\.testBinary' \
--exclude-dir=.git --exclude-dir=node_modules --exclude-dir=.claude --exclude-dir=vendor \
. 2>/dev/null \
| grep -v '.github/workflows/containment.yml:'; then
echo "::error::found a reference to testing.testBinary outside the Go toolchain itself — this is the linker var that spoofs testing.Testing() in a real (non go-test) binary; the containment.go runtime guard trusts that signal, so setting it anywhere in a real build path defeats it (see doc.go)"
hits=1
fi
if [ "$hits" -ne 0 ]; then exit 1; fi
echo "OK: no build/release path sets -tags controlplane or spoofs testing.Testing()"
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
- name: go env for private modules
env:
GH_PAT: ${{ secrets.GH_PAT }}
# GOPRIVATE names exactly the namespace that is private. github.com/hanzoai/*
# is: ai, account, commerce, orm, xorm, beego, csqlite and ~30 more are
# private repos, so they must resolve direct+authenticated and skip a sumdb
# that cannot see them. Everything else stays on the public proxy + checksum
# db, which is what makes a module hash immutable: zap-proto (all 55 repos)
# and luxfi (all 37 deps here) are public and proxy-served.
#
# This previously named zap-proto — public, and never the reason anything
# here was direct — and then set GOSUMDB=off to compensate for hanzoai/*
# being absent, which disabled checksum verification for EVERY module in the
# build, public ones included. Naming the private namespace is what the off
# switch was standing in for.
run: |
git config --global url."https://x-access-token:${GH_PAT}@github.com/".insteadOf "https://github.com/"
{
echo "GOPRIVATE=github.com/hanzoai/*"
echo "GOPROXY=https://proxy.golang.org,direct"
} >> "$GITHUB_ENV"
- name: zen streaming-fix floor — go.mod must pin github.com/hanzoai/zen >= v1.4.1
# The SSE body-close fix (zen commit 50328b8, first released in zen v1.4.1)
# is what makes streaming completions return a body instead of an empty
# stream. A stale-branch merge that reverts go.mod's zen pin below the floor
# silently re-breaks streaming, and `next build`'s ignoreBuildErrors hides
# the runtime break — so no image may be cut on a regressed pin. This is the
# durable root-cause guard: it reads the EFFECTIVE module version (post-MVS,
# exactly what the build links) and fails the PR/push below the floor.
run: |
set -euo pipefail
FLOOR="v1.4.1"
V="$(go list -m -f '{{.Version}}' github.com/hanzoai/zen)"
echo "effective github.com/hanzoai/zen = ${V} (floor ${FLOOR})"
# semver-correct compare: the lowest of {V, FLOOR} under `sort -V` must be
# the FLOOR, i.e. V >= FLOOR. (sort -V orders v1.4.2 above v1.4.10 too.)
low="$(printf '%s\n%s\n' "$V" "$FLOOR" | sort -V | head -1)"
if [ "$low" != "$FLOOR" ]; then
echo "::error::github.com/hanzoai/zen is pinned at ${V}, below the streaming-fix floor ${FLOOR} — this re-breaks SSE streaming (empty completions). Re-pin zen to >= ${FLOOR} in go.mod before merging."
exit 1
fi
echo "OK: zen ${V} is at or above the streaming-fix floor ${FLOOR}"
- name: positive proof — clients/controlplane is unreachable from the default build
run: |
set -euo pipefail
go build ./...
for m in $(go list ./cmd/...); do
if go list -deps "$m" | grep -qx 'github.com/hanzoai/cloud/clients/controlplane'; then
echo "::error::$m links clients/controlplane into a real binary — containment breach"
exit 1
fi
done
out="$(go build ./clients/controlplane/... 2>&1 || true)"
if ! printf '%s' "$out" | grep -q 'matched no packages'; then
echo "::error::clients/controlplane built successfully WITHOUT -tags controlplane (containment breach): $out"
exit 1
fi
echo "OK: containment holds — clients/controlplane has zero buildable files by default and is linked into no cmd/ binary"
-684
View File
@@ -1,684 +0,0 @@
name: release
# Cuts a release of ghcr.io/hanzoai/cloud. The invariant this workflow exists to
# enforce:
#
# a git tag v<X.Y.Z> exists ⇔ an image ghcr.io/hanzoai/cloud:v<X.Y.Z>
# was pushed AND booted to "listening" in the smoke test.
#
# The tag is a RECEIPT for a proven image, minted only AFTER a successful push —
# never a trigger for a build that might fail. The prior design triggered builds
# FROM pushed tags, so a tag could exist with no image behind it (a failed or
# never-run build) — universe would then try to roll that tag and the pods went
# ImagePullBackOff (phantom v1.786.42/43). Here the order is inverted:
#
# main push → compute next version → build → SMOKE → push image → tag → notify
#
# so a push/smoke/build failure fails the run BEFORE the tag step and leaves no
# tag; universe is only ever notified of a version whose image is proven present.
#
# DO NOT push v* tags by hand anymore. This workflow OWNS them. A hand-cut tag has
# no image behind it (exactly the phantom this prevents) and won't build (there is
# no `tags:` trigger). Every merge to main IS the release; skip one with the usual
# `[skip ci]` in the commit/merge message (a docs-only change need not ship).
#
# concurrency: a single serialized lane (cancel-in-progress:false — a
# mid-flight push must finish, never be killed between "image pushed" and "tag
# created"). Two main pushes can therefore never compute the same next number:
# the queued run starts only after the running one tags, re-reads the tags, and
# lands on the next patch. Monotonic by construction.
#
# The next version is max(highest git tag, highest pushed container tag) + 1 (patch
# bump only — never a major/minor jump). Folding in the container tags means we
# never reuse a number that already has a pushed image, even if some earlier run
# pushed an image but died before tagging.
#
# ── Infra notes (unchanged, still true) ─────────────────────────────────────────
# Self-hosted arcd amd64 scale set — NEVER GitHub-hosted runners (this org's
# GitHub-hosted Actions are billing-frozen). GHCR login uses GH_PAT, not the repo
# GITHUB_TOKEN: the ghcr.io/hanzoai/cloud package is linked to a DIFFERENT repo
# (hanzoai/ai, from the cloud→ai module rename), so this repo's GITHUB_TOKEN is
# denied write to it (permission_denied: write_package). GH_PAT (admin:org +
# write:packages) writes any hanzoai package regardless of package-repo linkage,
# and is the BuildKit gh_token the Dockerfile uses to fetch private cross-org Go
# modules. amd64-only: the cluster is amd64; one platform completes on the live
# scale set without waiting on the arm64 pool.
on:
push:
branches: [main]
workflow_dispatch:
# Cut tags on the cloud repo (git tag push) → contents: write. packages: write
# to push the image; id-token for provenance.
permissions:
contents: write
packages: write
id-token: write
# One serialized release lane. Never cancel in-flight: a run killed between
# "image pushed" and "tag created" is exactly the drift we are preventing.
concurrency:
group: release-cloud
cancel-in-progress: false
jobs:
build-amd64:
# ARC ephemeral runners match jobs targeting the scale-set NAME as a label.
runs-on: [hanzo-build-linux-amd64]
outputs:
# The FINAL assigned version comes from the Tag step (atomic free-version
# assignment), NOT the compute step — under concurrency the compute value may
# have been superseded. notify-universe must roll the version that was actually
# tagged + whose image was actually retagged.
version: ${{ steps.tag.outputs.version }}
version_v: ${{ steps.tag.outputs.version_v }}
steps:
- name: Checkout (full history + all tags — the version floor is read from tags)
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
- name: Compute next version (monotonic patch bump over git + container tags)
id: ver
env:
GH_PAT: ${{ secrets.GH_PAT }}
run: |
set -euo pipefail
git fetch --tags --force --quiet
# Highest semver git tag (vX.Y.Z), normalised without the leading v.
git_max="$(git tag -l 'v[0-9]*.[0-9]*.[0-9]*' \
| sed 's/^v//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true)"
# Best-effort: highest ALREADY-PUSHED container tag, so a number that has
# an image (even from a run that died before tagging) is never reused.
cont_max=""
if command -v gh >/dev/null 2>&1; then
cont_max="$(GH_TOKEN="$GH_PAT" gh api \
'/orgs/hanzoai/packages/container/cloud/versions?per_page=100' \
--jq '.[].metadata.container.tags[]?' 2>/dev/null \
| sed 's/^v//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true)"
fi
# Floor = highest of the two; fall back to 1.786.0 only if the repo has
# no tags at all (first release ever).
max="$(printf '%s\n%s\n%s\n' "1.786.0" "$git_max" "$cont_max" \
| grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1)"
major="${max%%.*}"; rest="${max#*.}"; minor="${rest%%.*}"; patch="${rest##*.}"
version="${major}.${minor}.$((patch + 1))"
# This version is a STARTING HINT only. It labels the smoke-built image and
# seeds the OCI metadata; the FINAL version is assigned atomically in the Tag
# step (which recomputes + retries on collision), so a taken number here is
# NOT fatal — the Tag step finds the next free one. Just note it and proceed.
if git rev-parse -q --verify "refs/tags/v${version}" >/dev/null; then
echo "note: hint v${version} already tagged — the Tag step will assign the next free version"
fi
echo "version=${version}" >> "$GITHUB_OUTPUT"
echo "version_v=v${version}" >> "$GITHUB_OUTPUT"
echo "major_minor=${major}.${minor}" >> "$GITHUB_OUTPUT"
echo "sha_short=$(git rev-parse --short "$GITHUB_SHA")" >> "$GITHUB_OUTPUT"
echo "Next release: v${version} (git_max='${git_max:-none}' container_max='${cont_max:-none}')"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
driver: docker-container
driver-opts: network=host
- name: Mirror credential (registry.hanzo.ai)
# Dual-host: pull the KMS deploy kubeconfig (same Universal Auth flow
# the hanzoai/ci reusable uses), read the cluster-synced
# registry-credentials dockerconfig, and log in. Best-effort: absent
# creds → GHCR-only release, never a blocked tag.
env:
KMS_CLIENT_ID: ${{ secrets.KMS_CLIENT_ID }}
KMS_CLIENT_SECRET: ${{ secrets.KMS_CLIENT_SECRET }}
KMS_ENDPOINT: ${{ vars.KMS_ENDPOINT || 'https://kms.hanzo.ai' }}
REGISTRY_USER: ${{ secrets.REGISTRY_USER }}
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
run: |
set -uo pipefail
# Direct credential first (repo/org secret — works on private repos,
# where the Free plan hides org KMS secrets); KMS kubeconfig fallback.
if [ -n "${REGISTRY_USER:-}" ] && [ -n "${REGISTRY_PASSWORD:-}" ]; then
if echo "$REGISTRY_PASSWORD" | docker login registry.hanzo.ai -u "$REGISTRY_USER" --password-stdin; then
echo "MIRROR_OK=1" >> "$GITHUB_ENV"
else
echo "mirror login failed (registry.hanzo.ai unreachable) — mirror skipped, release continues"
fi
exit 0
fi
[ -z "${KMS_CLIENT_ID:-}" ] && { echo "no KMS creds — mirror skipped"; exit 0; }
TOKEN=$(curl -sf "$KMS_ENDPOINT/v1/kms/auth/login" -H 'Content-Type: application/json' -d "{\"clientId\":\"$KMS_CLIENT_ID\",\"clientSecret\":\"$KMS_CLIENT_SECRET\"}" | jq -r '.accessToken // empty')
[ -z "$TOKEN" ] && { echo "KMS login failed — mirror skipped"; exit 0; }
KC=$(curl -sf "$KMS_ENDPOINT/v1/kms/orgs/hanzo/secrets/deploy/KUBECONFIG?env=prod" -H "Authorization: Bearer $TOKEN" | jq -r '.secret.value // empty')
[ -z "$KC" ] && { echo "no KUBECONFIG in KMS — mirror skipped"; exit 0; }
echo "$KC" | base64 -d > "$RUNNER_TEMP/kubeconfig"
command -v kubectl >/dev/null 2>&1 || {
KVER=$(curl -fsSL https://dl.k8s.io/release/stable.txt)
mkdir -p "$HOME/.local/bin"
curl -fsSL "https://dl.k8s.io/release/${KVER}/bin/linux/amd64/kubectl" -o "$HOME/.local/bin/kubectl" && chmod +x "$HOME/.local/bin/kubectl"
export PATH="$HOME/.local/bin:$PATH"
}
CFG=$(KUBECONFIG="$RUNNER_TEMP/kubeconfig" kubectl -n hanzo get secret registry-credentials -o jsonpath='{.data.\.dockerconfigjson}' 2>/dev/null | base64 -d || true)
[ -z "$CFG" ] && { echo "registry-credentials unreadable — mirror skipped"; exit 0; }
UP=$(echo "$CFG" | jq -r '.auths["registry.hanzo.ai"].auth // empty' | base64 -d)
[ -z "$UP" ] && { echo "no registry auth — mirror skipped"; exit 0; }
echo "::add-mask::${UP#*:}"
if echo "${UP#*:}" | docker login registry.hanzo.ai -u "${UP%%:*}" --password-stdin; then
echo "MIRROR_OK=1" >> "$GITHUB_ENV"
else
echo "mirror login failed (registry.hanzo.ai unreachable) — mirror skipped, release continues"
fi
- name: Log in to ghcr.io (GH_PAT — writes the cloud package despite its ai-repo linkage)
uses: docker/login-action@v3
with:
registry: ghcr.io
username: hanzo-dev
password: ${{ secrets.GH_PAT }}
- name: Resolve decomplection artifact digests (the Go-only build's prebuilt inputs)
id: artifacts
run: |
set -euo pipefail
# cloud compiles ONLY Go; it pulls three prebuilt artifacts (console SPA,
# agent-skills catalog, native flags staticlib). Resolve each published
# :latest to an IMMUTABLE digest so THIS release is reproducible (pinned,
# not floating :latest) AND a console/skills/flags change is picked up —
# its CI republished :latest, so this resolves to the NEW digest. A MISSING
# artifact FAILS the release HERE, before build/smoke/push/tag: the receipt
# invariant means we never tag an image that couldn't embed the real console.
command -v crane >/dev/null 2>&1 || {
mkdir -p "$HOME/.local/bin"
curl -fsSL "https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz" \
| tar -xz -C "$HOME/.local/bin" crane
}
export PATH="$HOME/.local/bin:$PATH"
resolve() {
local repo="$1" d
d="$(crane digest "ghcr.io/hanzoai/${repo}:latest" 2>/dev/null || true)"
[ -n "$d" ] || { echo "::error::decomplection artifact ghcr.io/hanzoai/${repo}:latest is not published — refusing to cut a release that would embed a stale/placeholder ${repo}"; return 1; }
printf 'ghcr.io/hanzoai/%s@%s' "$repo" "$d"
}
CONSOLE_IMAGE="$(resolve console-embed)" || exit 1
SKILLS_IMAGE="$(resolve agent-skills)" || exit 1
FLAGS_IMAGE="$(resolve cloud-flags)" || exit 1
{
echo "console_image=${CONSOLE_IMAGE}"
echo "skills_image=${SKILLS_IMAGE}"
echo "flags_image=${FLAGS_IMAGE}"
} >> "$GITHUB_OUTPUT"
echo "resolved: console=${CONSOLE_IMAGE} skills=${SKILLS_IMAGE} flags=${FLAGS_IMAGE}"
- name: OCI labels
id: meta
uses: docker/metadata-action@v5
with:
images: ghcr.io/hanzoai/cloud
tags: type=raw,value=${{ steps.ver.outputs.version_v }}
# ── Build → SMOKE → push → tag ───────────────────────────────────────────
# 1. Build once to a LOCAL tag (load into the daemon, do NOT push). Warms
# the BuildKit cache — the expensive console/npm + Go layers land here.
# 2. Boot that exact image and assert it reaches "listening" with no
# startup-crash signature (the gate).
# 3. Re-run build with push:true and the real tags: identical context /
# platform / secrets, so every layer is a cache hit from step 1 and the
# step only exports + pushes the already-tested image. Nothing that failed
# the smoke test can reach the registry.
# 4. Only after the push succeeds, mint + push the git tag (the receipt).
- name: Build (load locally for the smoke test)
uses: docker/build-push-action@v6
with:
context: .
platforms: linux/amd64
push: false
load: true
tags: cloud:smoke
labels: ${{ steps.meta.outputs.labels }}
# cloud compiles ONLY Go: pull the three prebuilt artifacts pinned to the
# digests resolved above (reproducible, and fresh — a console/skills/flags
# change is a new digest). No node/python/rust toolchain in this build.
build-args: |
CONSOLE_IMAGE=${{ steps.artifacts.outputs.console_image }}
SKILLS_IMAGE=${{ steps.artifacts.outputs.skills_image }}
FLAGS_IMAGE=${{ steps.artifacts.outputs.flags_image }}
# GIT_AUTH_TOKEN: BuildKit secret the Dockerfile consumes to fetch private
# cross-org Go modules (hanzoai/*, luxfi/*) over authenticated git.
secrets: |
GIT_AUTH_TOKEN=${{ secrets.GH_PAT }}
- name: Smoke test — the binary MUST boot to "listening" with no crash signature
run: |
set -euo pipefail
IMAGE=cloud:smoke
CID=""
cleanup() { [ -n "$CID" ] && docker rm -f "$CID" >/dev/null 2>&1 || true; }
trap cleanup EXIT
# Minimal, production-representative boot env:
# • a writable ephemeral /data root — the audit store, the embedded
# KMS secrets plane and every per-tenant SQLite open files under
# CLOUD_DATA_DIR; an unwritable dir would fail EVERY image before
# MountAll and the gate would stop discriminating good from bad; and
# • a throwaway 32-byte KMS master key so the KMS plane mounts on its
# normal ready path exactly as prod does (no real secret is used).
# The subsystem that crashed the incident (metrics, mount order 40)
# mounts AFTER kms (order 10), so the boot must get past kms for the
# gate to observe the panic — this env does exactly that.
KEY="$(head -c 32 /dev/urandom | base64 | tr -d '\n')"
CID="$(docker run -d \
--tmpfs /data:rw,size=64m \
-e CLOUD_DATA_DIR=/data \
-e CLOUD_ENV=smoke \
-e CLOUD_KMS_MASTER_KEY_REF="$KEY" \
"$IMAGE")"
# Poll up to 60s for boot to either finish ("listening" is logged once
# every subsystem has mounted and both transports are about to bind) or
# die (a Mount panic exits the process). A healthy boot is ~1-2s; the
# ceiling only guards a cold daemon.
listening=0
for _ in $(seq 1 60); do
logs="$(docker logs "$CID" 2>&1 || true)"
if printf '%s' "$logs" | grep -q '"message":"listening"'; then listening=1; break; fi
if [ "$(docker inspect -f '{{.State.Running}}' "$CID" 2>/dev/null || echo false)" != "true" ]; then break; fi
sleep 1
done
logs="$(docker logs "$CID" 2>&1 || true)"
echo "::group::cloud:smoke boot logs"
printf '%s\n' "$logs"
echo "::endgroup::"
# (1) No startup-crash signature. Catches the incident's Mount
# type-assert panic AND any generic Go panic — case-insensitive so a
# re-worded variant can't slip through — BEFORE a byte is pushed.
if printf '%s' "$logs" | grep -Eiq 'metrics\.Mount|mount metrics|panic|want \*zip\.App'; then
echo "SMOKE FAIL: startup-crash signature in boot logs (see above)"
exit 1
fi
# (2) Reached "listening" — proof that MountAll returned for every
# enabled subsystem (a failed Mount returns before this line).
if [ "$listening" -ne 1 ]; then
echo "SMOKE FAIL: binary never reached \"listening\" (a subsystem did not mount)"
exit 1
fi
# (3) Still alive — a server that logged "listening" then exited (e.g. a
# listener bind failure) is not a healthy image.
if [ "$(docker inspect -f '{{.State.Running}}' "$CID" 2>/dev/null || echo false)" != "true" ]; then
echo "SMOKE FAIL: process exited after \"listening\""
exit 1
fi
echo "SMOKE PASS: cloud:smoke booted to \"listening\" with no crash signature"
# ── Functional smoke — authenticated per-subsystem probe (the REAL gate) ─────
# The boot check above proves the process REACHES "listening"; this proves the
# mounted HTTP surface actually WORKS. /smoke (cmd/smoke, baked into the image)
# hits ONE side-effect-free read per core subsystem and FAILS the release on any
# broken code — above all a 402 on a READ (the balance-gate-over-blocks-reads
# regression) or a 5xx (a crash, e.g. the /v1/billing/usage self-dispatch 500).
# So a release can never ship with chat/billing/projects/kms/... down.
- name: Functional smoke — per-subsystem probe (fails the release if a core endpoint is broken)
env:
KMS_CLIENT_ID: ${{ secrets.KMS_CLIENT_ID }}
KMS_CLIENT_SECRET: ${{ secrets.KMS_CLIENT_SECRET }}
KMS_ENDPOINT: ${{ vars.KMS_ENDPOINT || 'https://kms.hanzo.ai' }}
# A KMS-provisioned short-lived smoke bearer, injected as a secret (NEVER
# hardcoded). Absent → the anonymous matrix still gates public/authed and
# catches every 402-on-read / 5xx.
SMOKE_TOKEN: ${{ secrets.SMOKE_TOKEN }}
run: |
set -euo pipefail
IMAGE=cloud:smoke
CID=""
cleanup() { [ -n "$CID" ] && docker rm -f "$CID" >/dev/null 2>&1 || true; }
trap cleanup EXIT
KEY="$(head -c 32 /dev/urandom | base64 | tr -d '\n')"
CID="$(docker run -d \
--tmpfs /data:rw,size=64m \
-e CLOUD_DATA_DIR=/data -e CLOUD_ENV=smoke -e CLOUD_KMS_MASTER_KEY_REF="$KEY" \
"$IMAGE")"
# Wait for the HTTP listener to bind (or the process to die).
up=0
for _ in $(seq 1 60); do
lg="$(docker logs "$CID" 2>&1 || true)"
printf '%s' "$lg" | grep -q '"message":"listening"' && { up=1; break; }
[ "$(docker inspect -f '{{.State.Running}}' "$CID" 2>/dev/null || echo false)" != "true" ] && break
sleep 1
done
if [ "$up" != 1 ]; then
echo "::group::boot logs"; docker logs "$CID" 2>&1 || true; echo "::endgroup::"
echo "FUNCTIONAL SMOKE INFRA FAIL: image never reached \"listening\""
exit 1
fi
# Token: prefer the injected secret; else mint from KMS (a provisioned smoke
# identity); else run the anonymous matrix. Never hardcoded.
if [ -z "${SMOKE_TOKEN:-}" ] && [ -n "${KMS_CLIENT_ID:-}" ]; then
KT=$(curl -sf "$KMS_ENDPOINT/v1/kms/auth/login" -H 'Content-Type: application/json' \
-d "{\"clientId\":\"$KMS_CLIENT_ID\",\"clientSecret\":\"$KMS_CLIENT_SECRET\"}" | jq -r '.accessToken // empty' || true)
[ -n "$KT" ] && SMOKE_TOKEN=$(curl -sf "$KMS_ENDPOINT/v1/kms/orgs/hanzo/secrets/smoke/TOKEN?env=prod" \
-H "Authorization: Bearer $KT" | jq -r '.secret.value // empty' || true)
fi
if [ -n "${SMOKE_TOKEN:-}" ]; then echo "::add-mask::$SMOKE_TOKEN"; echo "smoke: AUTHENTICATED matrix"; else echo "smoke: ANONYMOUS matrix (no SMOKE_TOKEN wired)"; fi
# /smoke is baked into the image (Dockerfile) — exec it INSIDE the container,
# so it probes the real mounted surface at localhost:8080 with no port/network
# plumbing. A non-zero exit here fails the release BEFORE any image is pushed.
docker exec \
-e SMOKE_BASE_URL=http://127.0.0.1:8080 \
-e SMOKE_TOKEN="${SMOKE_TOKEN:-}" \
"$CID" /smoke
# ── Migration smoke — the gate the v1.800.1 crashloop would have tripped ─────
# The plain smoke above boots on a FRESH /data, so every subsystem's migrate()
# takes its CREATE-TABLE path and no forward-migration is exercised — which is
# exactly why a DDL valid on a fresh store but broken on a pre-existing one (an
# index over a not-yet-ADDed column: affiliates referrer_org in v1.800.1, wallets
# project/agent before it) sailed through CI and took api.hanzo.ai down. This
# step reproduces the REAL prod upgrade path: boot the PRIOR released image to lay
# its on-disk (cek-encrypted) SQLite schema into a persistent volume, then boot
# the candidate over that SAME volume and require it to still reach "listening".
# A migrate() that assumes a fresh store dies here, before any image is pushed.
- name: Migration smoke — candidate MUST boot over the PRIOR release's on-disk schema
env:
# The image whose on-disk schema a prod upgrade migrates FROM — the tag the
# fleet runs today. Bump to the last-DEPLOYED tag as releases roll (override
# without a code change via the SMOKE_MIGRATION_BASELINE repo/org variable).
BASELINE: ${{ vars.SMOKE_MIGRATION_BASELINE }}
run: |
set -euo pipefail
BASELINE="${BASELINE:-ghcr.io/hanzoai/cloud:v1.799.19}"
CANDIDATE=cloud:smoke
VOL="cloudmig-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
B1=""; B2=""
cleanup() { docker rm -f "$B1" "$B2" >/dev/null 2>&1 || true; docker volume rm "$VOL" >/dev/null 2>&1 || true; }
trap cleanup EXIT
docker volume create "$VOL" >/dev/null
# ONE throwaway 32-byte master key for BOTH boots: cek seals each per-db DEK
# under it on the baseline boot and unwraps it on the candidate boot. A
# mismatched key fails closed (never opens), so sharing it is what puts the
# MIGRATE path — not a decrypt error — under test.
KEY="$(head -c 32 /dev/urandom | base64 | tr -d '\n')"
boot() { # $1=image $2=name -> prints container id
docker run -d --name "$2" \
-v "$VOL":/data \
-e CLOUD_DATA_DIR=/data \
-e CLOUD_ENV=smoke \
-e CLOUD_KMS_MASTER_KEY_REF="$KEY" \
"$1"
}
wait_listen() { # $1=container -> 0 if "listening", 1 if it died / timed out
for _ in $(seq 1 90); do
lg="$(docker logs "$1" 2>&1 || true)"
printf '%s' "$lg" | grep -q '"message":"listening"' && return 0
[ "$(docker inspect -f '{{.State.Running}}' "$1" 2>/dev/null || echo false)" != "true" ] && return 1
sleep 1
done
return 1
}
# A fresh docker named volume is root:root 0755, but the cloud image runs
# NON-ROOT, so it cannot create cek's <db>.cek.lock under /data (the single-boot
# smoke only worked because --tmpfs is world-writable). Make the shared volume
# writable for BOTH boots, re-opening it between them so the candidate can read
# the baseline's files even if their runtime UIDs differ.
chmod_vol() { docker run --rm --user 0 -v "$VOL":/data --entrypoint sh "$CANDIDATE" -c 'chmod -R 0777 /data'; }
# Boot 1 — the prior release writes its real schema into the volume. It must
# reach "listening" (proof every subsystem migrated + its DB is on disk); if
# the pinned baseline can't boot in this env the gate is blind, so fail loud.
echo "migration baseline: $BASELINE"
pulled=0; for _ in 1 2 3; do if docker pull "$BASELINE"; then pulled=1; break; fi; sleep 5; done
[ "$pulled" = 1 ] || { echo "MIGRATION SMOKE INFRA FAIL: cannot pull baseline $BASELINE"; exit 1; }
chmod_vol
B1="$(boot "$BASELINE" cloudmig_base)"
if ! wait_listen "$B1"; then
echo "::group::baseline boot logs"; docker logs "$B1" 2>&1 || true; echo "::endgroup::"
echo "MIGRATION SMOKE INFRA FAIL: baseline $BASELINE did not reach \"listening\" — cannot stage the prior schema (inspect/bump SMOKE_MIGRATION_BASELINE)"
exit 1
fi
docker stop "$B1" >/dev/null
chmod_vol
# Boot 2 — the candidate migrates that on-disk schema IN PLACE. This is the gate.
B2="$(boot "$CANDIDATE" cloudmig_cand)"
listening=0; wait_listen "$B2" && listening=1
logs="$(docker logs "$B2" 2>&1 || true)"
echo "::group::candidate migration boot logs"; printf '%s\n' "$logs"; echo "::endgroup::"
# 'no such column'/'no such table' is the exact index-before-ADD-COLUMN crash;
# 'panic' catches any generic Mount failure. The load-bearing check is the
# "listening" assertion below — a migrate() crash exits before it.
if printf '%s' "$logs" | grep -Eiq 'panic|no such column|no such table'; then
echo "MIGRATION SMOKE FAIL: candidate logged a DDL/migration error over the prior schema (the v1.800.1-class regression)"
exit 1
fi
if [ "$listening" -ne 1 ]; then
echo "MIGRATION SMOKE FAIL: candidate did NOT reach \"listening\" over the $BASELINE schema — a subsystem's migrate() crashes on a pre-existing store"
exit 1
fi
echo "MIGRATION SMOKE PASS: candidate booted to \"listening\" over the $BASELINE on-disk schema"
- name: Push (cache hit from the smoke build — publishes the tested image)
id: push
uses: docker/build-push-action@v6
with:
context: .
platforms: linux/amd64
push: true
# Push ONLY the immutable, per-commit sha- tag (always unique — never races)
# and the floating latest. The v<X.Y.Z> version is NOT pushed here: the
# compute-step version may be claimed by a concurrent release between compute
# and now, and pushing it would clobber that release's :vX image (mutable tag
# corruption). The version is assigned + the proven sha-image retagged to it
# ATOMICALLY in the Tag step below, so :vX exists iff its git tag exists.
tags: |
ghcr.io/hanzoai/cloud:sha-${{ steps.ver.outputs.sha_short }}
ghcr.io/hanzoai/cloud:latest
labels: ${{ steps.meta.outputs.labels }}
# SAME artifact digests as the smoke build → every layer is a cache hit from
# step 1 and the pushed image is byte-identical to the one smoke proved.
build-args: |
CONSOLE_IMAGE=${{ steps.artifacts.outputs.console_image }}
SKILLS_IMAGE=${{ steps.artifacts.outputs.skills_image }}
FLAGS_IMAGE=${{ steps.artifacts.outputs.flags_image }}
secrets: |
GIT_AUTH_TOKEN=${{ secrets.GH_PAT }}
# THE RECEIPT + ATOMIC VERSION ASSIGNMENT (race-safe). Reached only because
# build + smoke + push all succeeded, so a proven image exists under the unique
# sha- tag. Here we assign the next FREE v<X.Y.Z> and retag that proven image to
# it — atomically, with the git-tag push as the serialization point:
# * Recompute the next version FRESH (compute-step's value may have been claimed
# by a concurrent release in the build window).
# * If that version's git tag already exists, bump and retry.
# * Retag the proven sha-image → :vX (+ :X.Y.Z + :X.Y) via imagetools (metadata
# only, NO rebuild — byte-identical to the smoke-passed image).
# * Push the git tag; the FIRST pusher of vX wins, a loser deletes its local tag
# and recomputes. So concurrent releases each grab a distinct free number and
# the invariant "git tag vX ⇔ image :vX pushed+smoke-passed" holds under race.
- name: Tag the proven image (atomic free-version assignment — race-safe)
id: tag
env:
GH_PAT: ${{ secrets.GH_PAT }}
run: |
set -euo pipefail
git config user.name "hanzo-dev"
git config user.email "dev@hanzo.ai"
SHA_IMG="ghcr.io/hanzoai/cloud:sha-${{ steps.ver.outputs.sha_short }}"
PUSH_URL="https://x-access-token:${GH_PAT}@github.com/${GITHUB_REPOSITORY}.git"
for attempt in $(seq 1 8); do
git fetch --tags --force --quiet
git_max="$(git tag -l 'v[0-9]*.[0-9]*.[0-9]*' \
| sed 's/^v//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true)"
# Newest page only — NOT --paginate. Container versions are created
# newest-first and version tags are monotonic, so the highest version
# is always among the most-recent versions; paginating the WHOLE
# registry history is what livelocked this step as tags accumulated.
# Fail-CLOSED. An ORPHANED container tag — image pushed by a run that
# died or was cancelled after imagetools-create but before its git tag —
# MUST raise the floor, or a later run reassigns that same number to a
# different image (an ambiguous mutable prod tag; the v1.801.50 flip). A
# git-only floor can't see the orphan, so if the container-tag lookup
# ERRORS (vs legitimately returning no tags) we retry the whole attempt
# rather than silently proceeding — a version with a pushed image is never
# reused. (Reordering git-tag before imagetools-create is the WRONG fix: it
# reintroduces the phantom "tag ⇔ no image" this workflow exists to prevent.)
cont_max=""
if command -v gh >/dev/null 2>&1; then
if cont_raw="$(GH_TOKEN="$GH_PAT" gh api \
'/orgs/hanzoai/packages/container/cloud/versions?per_page=100' \
--jq '.[].metadata.container.tags[]?' 2>/dev/null)"; then
cont_max="$(printf '%s\n' "$cont_raw" \
| sed 's/^v//' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1 || true)"
else
echo " container-tag lookup failed — retry so an orphaned tag can't be reused (attempt $attempt)"; sleep 3; continue
fi
fi
max="$(printf '%s\n%s\n%s\n' "1.786.0" "$git_max" "$cont_max" \
| grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1)"
major="${max%%.*}"; rest="${max#*.}"; minor="${rest%%.*}"; patch="${rest##*.}"
VER="${major}.${minor}.$((patch + 1))"; V="v${VER}"
if git rev-parse -q --verify "refs/tags/$V" >/dev/null; then
echo " $V already tagged — recomputing (attempt $attempt)"; sleep 3; continue
fi
docker buildx imagetools create \
-t "ghcr.io/hanzoai/cloud:${V}" \
-t "ghcr.io/hanzoai/cloud:${VER}" \
-t "ghcr.io/hanzoai/cloud:${major}.${minor}" \
"$SHA_IMG"
# Dual-host: mirror the release tags to OUR fleet registry (server-
# side copy) so the cluster never depends on GHCR to deploy. crane,
# not buildx imagetools: the IAM token realm doesn't answer buildx's
# multi-scope token request (spec gap, tracked), crane's single-scope
# flow works. Best-effort — a mirror hiccup never blocks the receipt.
if [ "${MIRROR_OK:-}" = "1" ]; then
command -v crane >/dev/null 2>&1 || {
curl -fsSL https://github.com/google/go-containerregistry/releases/download/v0.20.2/go-containerregistry_Linux_x86_64.tar.gz \
| tar -xz -C "$HOME/.local/bin" crane 2>/dev/null || {
mkdir -p "$HOME/.local/bin"
curl -fsSL https://github.com/google/go-containerregistry/releases/download/v0.20.2/go-containerregistry_Linux_x86_64.tar.gz \
| tar -xz -C "$HOME/.local/bin" crane
}
export PATH="$HOME/.local/bin:$PATH"
}
for MT in "${V}" "${VER}" "${major}.${minor}"; do
# Bounded: registry.hanzo.ai can *hang* (not just fail), and this
# is best-effort — an unbounded crane copy once livelocked the whole
# tag step and held the serialized release lane. timeout makes the
# mirror truly best-effort so the git-tag receipt below always runs.
timeout 120 crane copy "$SHA_IMG" "registry.hanzo.ai/hanzoai/cloud:${MT}" \
|| echo "::warning::mirror registry.hanzo.ai/hanzoai/cloud:${MT} failed or timed out"
done
fi
git tag -a "$V" -m "release $V — image ghcr.io/hanzoai/cloud:$V (retagged from sha-${{ steps.ver.outputs.sha_short }}, smoke-passed ${GITHUB_SHA})"
if git push "$PUSH_URL" "$V" 2>/dev/null; then
echo "Tagged $V → ghcr.io/hanzoai/cloud:$V"
echo "version=${VER}" >> "$GITHUB_OUTPUT"
echo "version_v=${V}" >> "$GITHUB_OUTPUT"
exit 0
fi
echo " push of $V lost the race — recomputing (attempt $attempt)"
git tag -d "$V" >/dev/null 2>&1 || true
sleep 3
done
echo "::error::could not acquire a free version tag after 8 attempts"
exit 1
# ── Promote: the declared-tag bump that makes the release DEPLOY ─────────────
# The tag minted above is the receipt for a pushed, smoke-passed image; THIS job
# records it as the desired state Hanzo CD reconciles. The universe-crs ArgoCD
# Application (ns hanzo-cd, `automated` sync + selfHeal) syncs
# infra/k8s/operator/crs/*.yaml → cluster and the operator rolls the Deployment,
# so a tag bump committed here reaches api.hanzo.ai with NO hand-dispatch and NO
# hand-edit of the CR.
#
# This is the SAME yq-bump → `deploy(<svc>): <tag>` universe commit the hanzoai/ci
# reusable (build.yml deploy step) does for every other service. cloud owns it
# HERE because its image is built by this workflow, not the ci reusable — its
# hanzo.yml carries no main `images:` entry and `# NO deploy`, so the shared
# deploy step never bumps cloud's CR. A direct in-cluster CR patch is NOT enough:
# ArgoCD selfHeal reverts any live edit not also recorded in git within ~45s.
# The retired notify-universe repository_dispatch had no receiver after the
# image-update.yml deploy hub was deleted in the Hanzo CD cutover; the git commit
# IS the sanctioned path now.
promote:
needs: build-amd64
# Only a real release promotes: build+smoke+push+tag all succeeded, so a
# proven v* image exists. A failure earlier leaves version_v empty → skipped.
if: ${{ needs.build-amd64.outputs.version_v != '' }}
runs-on: [hanzo-build-linux-amd64]
steps:
- name: Record the proven tag in universe crs/cloud.yaml (Hanzo CD rolls it)
env:
# GH_PAT already pushes this repo's git tags above (contents:write on the
# hanzoai org), so it writes hanzoai/universe too — the SAME token the ci
# reusable falls back to for the universe deploy commit.
GH_PAT: ${{ secrets.GH_PAT }}
VERSION_V: ${{ needs.build-amd64.outputs.version_v }}
run: |
set -euo pipefail
[ -n "${GH_PAT:-}" ] || { echo "::error::no GH_PAT — cannot record the declared-tag bump in universe"; exit 1; }
# Bare arc runners ship no yq — provision the static binary (sudo-free,
# same pattern the ci reusable and this workflow's kubectl/crane fetches use).
if ! command -v yq >/dev/null 2>&1; then
mkdir -p "$HOME/.local/bin"; export PATH="$HOME/.local/bin:$PATH"
curl -fsSL https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 \
-o "$HOME/.local/bin/yq" && chmod +x "$HOME/.local/bin/yq"
fi
git clone -q --depth 1 \
"https://x-access-token:${GH_PAT}@github.com/hanzoai/universe.git" \
"$RUNNER_TEMP/universe"
CR="$RUNNER_TEMP/universe/infra/k8s/operator/crs/cloud.yaml"
[ -f "$CR" ] || { echo "::error::crs/cloud.yaml not found in universe"; exit 1; }
CUR="$(yq -r '.spec.image.tag // ""' "$CR")"
echo "cloud CR: ${CUR:-<empty>} → ${VERSION_V}"
# Monotonic guard: never roll the CR BACKWARD. Release runs finish under a
# serialized lane but a slow older run must never overwrite a newer promote.
# Skip iff the CR already holds a semver >= the version we just cut.
CURN="${CUR#v}"; NEWN="${VERSION_V#v}"
if printf '%s' "$CURN" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+$'; then
top="$(printf '%s\n%s\n' "$CURN" "$NEWN" | sort -V | tail -1)"
if [ "$top" = "$CURN" ] && [ "$CURN" != "$NEWN" ]; then
echo "::notice::cloud CR already at v${CURN} (≥ ${VERSION_V}) — not rolling back"; exit 0
fi
fi
yq -i ".spec.image.tag = \"${VERSION_V}\"" "$CR"
if git -C "$RUNNER_TEMP/universe" diff --quiet; then
echo "::notice::crs/cloud.yaml already at ${VERSION_V} — nothing to record"; exit 0
fi
git -C "$RUNNER_TEMP/universe" -c user.name=hanzo-ci -c user.email=dev@hanzo.ai \
commit -qam "deploy(cloud): ${VERSION_V} (${GITHUB_REPOSITORY}@$(echo "${GITHUB_SHA}" | cut -c1-7))"
# Rebase-safe push: universe main advances on every service's deploy, so a
# concurrent commit must not make cloud's promote lose the whole roll. Retry
# a few times, rebasing between attempts.
for attempt in $(seq 1 5); do
if git -C "$RUNNER_TEMP/universe" push -q origin HEAD:main; then
echo "recorded deploy(cloud): ${VERSION_V} — Hanzo CD (universe-crs) will roll it to api.hanzo.ai"
exit 0
fi
echo " universe push lost the race — rebasing (attempt ${attempt})"
git -C "$RUNNER_TEMP/universe" pull -q --rebase origin main || true
sleep 3
done
echo "::error::could not record the cloud tag bump in universe after 5 attempts"; exit 1
+12
View File
@@ -13,6 +13,14 @@
coverage.txt
coverage.html
# cek encryption sidecars are key material — never commit one from a source tree.
# Tests seal their stores under t.TempDir(); a sidecar in a package dir (e.g. from a
# test that opened ":memory:") is a mistake. The cek testdata fixtures are the one
# intentional exception.
*.dek
*.cek.lock
!cek/testdata/**
# Environment files
.env
.env.*
@@ -36,3 +44,7 @@ Thumbs.db
.claude/
.worktrees/
native/flags/target
# Artifacts from treating an in-memory DSN as a filename (see cek.inMemory).
:memory:
:memory:.*
+206
View File
@@ -0,0 +1,206 @@
name: CI/CD
# The ONE pipeline for cloud, on our own runners against git.hanzo.ai.
#
# THE LAW: `.github/workflows` holds exactly one file — a sync nudge that runs
# zero CI. Everything that gates, builds or deploys lives here.
#
# It absorbs the three pipelines that used to run beside each other:
#
# .github/workflows/cicd.yml → `gate` (unchanged: the ~7-line caller; every
# real detail still lives in the repo-root hanzo.yml, which
# platform.hanzo.ai reads too, so no build logic moved)
# .github/workflows/containment.yml → `containment` (verbatim; only its own
# self-exclusion path follows the file)
# .hanzo/workflows/deploy.yml → deleted, not moved (below)
#
# CI gates. It does not deploy, and deliberately builds no cloud image: that
# image and its v* tags have ONE owner, clients/platform/release.go (POST
# /v1/runner {release:true}) — a second builder on the same commit is the
# double-build hanzo.yml's images: block already refuses for this reason.
# deploy.yml claimed to do both and could do neither: it shelled
# `buildctl-daemonless.sh`, absent from the image this fleet actually serves for
# `hanzo-build-linux-amd64` (catthehacker/ubuntu:act-24.04 —
# universe:infra/k8s/git-runner/statefulset.yaml), read a GIT_CLONE_TOKEN secret
# that exists on neither the repo nor the org, and ended in a `kubectl patch app`
# that cd.hanzo.ai's selfHeal undoes on its next poll. Rollout is, and stays, a
# reviewed tag pin in hanzoai/universe.
on:
push:
branches: [main]
pull_request:
# The sync (sync-from-github.yml) dispatches this workflow by name after a
# fast-forward: a push made with the workflow token does not trigger workflows,
# so without this trigger every synced commit would gate nothing. deploy.yml
# declared no dispatch trigger, which is why that curl could only 404.
workflow_dispatch:
concurrency:
group: cicd-${{ github.ref }}
cancel-in-progress: true
jobs:
# Test gate + the decoupled native flags staticlib image, driven by hanzo.yml.
gate:
uses: hanzoai/ci/.hanzo/workflows/build.yml@v2
secrets: inherit
# Guards the Stage-1 byzantine ceremony containment (clients/controlplane,
# build tag `controlplane`). Its increment-1 crypto is stub/forgeable BY
# DESIGN (SHA256-of-public-inputs commitments, symmetric-HMAC
# proof-of-possession, seed-derived threshold shares — see
# clients/controlplane/doc.go) and MUST NEVER reach a release/serve binary.
# Three independent checks; any one failing blocks the merge:
#
# 1. grep (tag) — no build/release invocation anywhere in the repo
# (Dockerfile, Makefile, shell scripts, any workflow) may pass a `-tags`
# value containing `controlplane` to go build/vet/run/install. The
# package's own `//go:build controlplane` tag declarations
# (clients/controlplane/*) are the thing being guarded, not a violation,
# and are excluded by path.
# 2. grep (spoof) — no build/release invocation may pass
# `-X testing.testBinary=1` (or any -ldflags containing it) to a REAL
# `go build`. That linker flag is what `go test` itself uses to make
# testing.Testing() report true (cmd/go/internal/load/test.go) — the
# runtime guard in containment.go trusts that signal, so this is the one
# concrete way to spoof it in a non-test binary. This grep is what turns
# "someone could type this" into "CI fails the PR that types it".
# 3. graph — no cmd/ main may reach clients/controlplane through its
# untagged import graph (`go list -deps`), and
# `go build ./clients/controlplane/...` with no tag must match zero
# buildable packages (proves the tag still gates every file in it).
#
# Runtime belt-and-suspenders (defense in depth, not a substitute for the
# above): clients/controlplane/containment.go fail-closed panics if its stub
# crypto is ever constructed outside a go-test binary (testing.Testing()==
# false) — see TestContainment_NonHarnessProcessRefuses in
# containment_test.go. KNOWN RESIDUAL: testing.Testing() is a linker-set
# string var (testing.testBinary), not cryptographically bound to "actually
# is a test" — `-ldflags="-X testing.testBinary=1"` spoofs it in a real
# binary. Check #2 above is the mitigation: it fails the PR that would ship
# that flag. Closing the residual for real needs a signal `go build` cannot
# produce at all (increment-2, tracked in doc.go) rather than one merely
# absent by convention.
containment:
runs-on: [hanzo-build-linux-amd64]
# Bounded so a wedged run frees its runner. Absent this the job inherits
# GitHub's six-hour default, and the failure that actually happens is a
# runner reclaimed mid-build (exit 130, "runner has received a shutdown
# signal") whose job then sits on the slot for the rest of the morning.
# Generous: these checks read the import graph, they do not link, and the
# whole pipeline runs 13-26 min cold.
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
- name: grep — no build/release path may set -tags controlplane, or spoof testing.Testing()
run: |
set -euo pipefail
hits=0
# NOTE: exclusions are plain substring matches on the path (not
# anchored to a leading "./") so this is robust across grep
# implementations that format recursive-search paths differently.
# char class includes `!` so a build-constraint negation form
# (`-tags '!x,controlplane'`) cannot slip the grep. (Belt only: the
# positive-proof step below is the syntax-agnostic guarantee — the
# package has zero untagged files, so importing it into serve code
# fails the untagged `go build ./...` regardless of any -tags syntax.)
if grep -RnE -- '-tags[= ]*["'"'"']?[!A-Za-z0-9_, ]*\bcontrolplane\b' \
--exclude-dir=.git --exclude-dir=node_modules --exclude-dir=.claude --exclude-dir=vendor \
. 2>/dev/null \
| grep -v '\.git/' \
| grep -v 'clients/controlplane/' \
| grep -v '.hanzo/workflows/cicd.yml:'; then
echo "::error::found a build/release invocation passing -tags controlplane — clients/controlplane's stub crypto must never enter a release/serve binary (see clients/controlplane/doc.go)"
hits=1
fi
if grep -RnE -- 'testing\.testBinary' \
--exclude-dir=.git --exclude-dir=node_modules --exclude-dir=.claude --exclude-dir=vendor \
. 2>/dev/null \
| grep -v '.hanzo/workflows/cicd.yml:'; then
echo "::error::found a reference to testing.testBinary outside the Go toolchain itself — this is the linker var that spoofs testing.Testing() in a real (non go-test) binary; the containment.go runtime guard trusts that signal, so setting it anywhere in a real build path defeats it (see doc.go)"
hits=1
fi
if [ "$hits" -ne 0 ]; then exit 1; fi
echo "OK: no build/release path sets -tags controlplane or spoofs testing.Testing()"
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
- name: go env for private modules
env:
GH_PAT: ${{ secrets.GH_PAT }}
# GOPRIVATE names exactly the namespace that is private. github.com/hanzoai/*
# is: ai, account, commerce, orm, xorm, beego, csqlite and ~30 more are
# private repos, so they must resolve direct+authenticated and skip a sumdb
# that cannot see them. Everything else stays on the public proxy + checksum
# db, which is what makes a module hash immutable: zap-proto (all 55 repos)
# and luxfi (all 37 deps here) are public and proxy-served.
#
# This previously named zap-proto — public, and never the reason anything
# here was direct — and then set GOSUMDB=off to compensate for hanzoai/*
# being absent, which disabled checksum verification for EVERY module in the
# build, public ones included. Naming the private namespace is what the off
# switch was standing in for.
run: |
git config --global url."https://x-access-token:${GH_PAT}@github.com/".insteadOf "https://github.com/"
{
echo "GOPRIVATE=github.com/hanzoai/*"
echo "GOPROXY=https://proxy.golang.org,direct"
} >> "$GITHUB_ENV"
- name: zen streaming-fix floor — go.mod must pin github.com/hanzoai/zen >= v1.4.1
# The SSE body-close fix (zen commit 50328b8, first released in zen v1.4.1)
# is what makes streaming completions return a body instead of an empty
# stream. A stale-branch merge that reverts go.mod's zen pin below the floor
# silently re-breaks streaming, and `next build`'s ignoreBuildErrors hides
# the runtime break — so no image may be cut on a regressed pin. This is the
# durable root-cause guard: it reads the EFFECTIVE module version (post-MVS,
# exactly what the build links) and fails the PR/push below the floor.
run: |
set -euo pipefail
FLOOR="v1.4.1"
V="$(go list -m -f '{{.Version}}' github.com/hanzoai/zen)"
echo "effective github.com/hanzoai/zen = ${V} (floor ${FLOOR})"
# semver-correct compare: the lowest of {V, FLOOR} under `sort -V` must be
# the FLOOR, i.e. V >= FLOOR. (sort -V orders v1.4.2 above v1.4.10 too.)
low="$(printf '%s\n%s\n' "$V" "$FLOOR" | sort -V | head -1)"
if [ "$low" != "$FLOOR" ]; then
echo "::error::github.com/hanzoai/zen is pinned at ${V}, below the streaming-fix floor ${FLOOR} — this re-breaks SSE streaming (empty completions). Re-pin zen to >= ${FLOOR} in go.mod before merging."
exit 1
fi
echo "OK: zen ${V} is at or above the streaming-fix floor ${FLOOR}"
- name: positive proof — clients/controlplane is unreachable from the default build
run: |
set -euo pipefail
# Containment is an IMPORT-GRAPH property, so prove it with the import
# graph. This step used to lead with `go build ./...`, which compiles
# AND LINKS every cmd/ main — and linking needs
# native/flags/target/release/libhanzo_flags.a, the Rust staticlib
# clients/flags pulls in under cgo (clients/flags/engine.go). Nothing
# in THIS job builds it: `make native` runs in the gate job, in that
# job's own workspace. It passed on GitHub only because the arc
# runners reuse a workspace and target/ is gitignored, so an earlier
# gate job's leftover .a was still sitting there; on a container
# runner with a fresh volume every cmd/ main died at
# `ld: cannot find .../libhanzo_flags.a`. The compile was never the
# proof anyway — `go list -deps` reads the same untagged file set
# without linking, and the gate job is the ONE place that builds.
for m in $(go list ./cmd/...); do
if go list -deps "$m" | grep -qx 'github.com/hanzoai/cloud/clients/controlplane'; then
echo "::error::$m links clients/controlplane into a real binary — containment breach"
exit 1
fi
done
out="$(go build ./clients/controlplane/... 2>&1 || true)"
if ! printf '%s' "$out" | grep -q 'matched no packages'; then
echo "::error::clients/controlplane built successfully WITHOUT -tags controlplane (containment breach): $out"
exit 1
fi
echo "OK: containment holds — clients/controlplane has zero buildable files by default and is linked into no cmd/ binary"
+37 -8
View File
@@ -19,11 +19,26 @@
# Pinned to ghcr.io so BOTH buildx lanes (release.yml + platform arcbuild) pull
# it directly; the SAME tags are mirrored to registry.hanzo.ai (S3-backed) for
# GET-flow consumers (docker/kaniko/crane). Override any pin with
# --build-arg <NAME>_IMAGE=… — release.yml resolves CONSOLE_IMAGE to a fresh
# console-embed digest, exactly as CONSOLE_CACHEBUST re-fetched console before.
ARG CONSOLE_IMAGE=ghcr.io/hanzoai/console-embed:latest
ARG SKILLS_IMAGE=ghcr.io/hanzoai/agent-skills:latest
ARG FLAGS_IMAGE=ghcr.io/hanzoai/cloud-flags:latest
# --build-arg <NAME>_IMAGE=… .
#
# IMMUTABLE per-commit tags, never `:latest`. These defaults are LOAD-BEARING:
# the builder that actually runs our releases is the native one (POST /v1/runner
# → launchDirectBuild → BuildKit), and it passes no --build-arg, so whatever is
# written here is what gets baked. `release.yml`, which the previous comment said
# would resolve a fresh digest, is a stub and resolves nothing.
#
# With `:latest` the embedded console was therefore decided by WHEN the build ran,
# not by what we shipped — and it bit: cloud v1.801.215 was built ~12 minutes
# before console CI finished publishing the console-embed carrying v8.5.26, so a
# release whose whole purpose was that console change silently baked the previous
# one and shipped green. Same image, two contents, no diff to show for it.
#
# BUMP: when a console/skills/flags change must reach production, move its pin
# here in the same commit that claims it. That is what makes a cloud release
# reproducible and makes "what console is in v1.801.N" answerable from git.
ARG CONSOLE_IMAGE=ghcr.io/hanzoai/console-embed:sha-147ecd3-amd64
ARG SKILLS_IMAGE=ghcr.io/hanzoai/agent-skills:sha-b931a11-amd64
ARG FLAGS_IMAGE=ghcr.io/hanzoai/cloud-flags:sha-e1ca02a-amd64
# ── toolchain base images: the golang + alpine FROMs below pull from our own
# GHCR mirror (ghcr.io/hanzoai/mirror/*), pinned by digest. WHY: public.ecr.aws
@@ -163,10 +178,21 @@ LABEL org.opencontainers.image.revision="${REVISION}" \
# libgcc: the hanzo-flags Rust staticlib (clients/featureflags FFI) references the
# _Unwind_* unwinder symbols; musl needs libgcc_s at load time or the binary fails
# relocation ("Error relocating /cloud: _Unwind_GetIP: symbol not found").
RUN apk add --no-cache ca-certificates tzdata sqlcipher-libs git libgcc \
# tini: /cloud runs as PID 1, and PID 1 inherits every orphaned descendant in the
# container. git is not a single process — fetch/clone fan out to git-upload-pack,
# git-index-pack, git-rev-list and git-pack-objects. When cloud Kill()s a wedged
# direct child (gitPackStream.Close does exactly that, correctly), those
# grandchildren orphan and reparent to PID 1. A Go program never reaps adopted
# orphans, so each one becomes a permanent zombie holding a PID slot.
# Measured 2026-07-26 on worker-xl-37bw71: 18,553 zombie `git` out of 18,741
# processes, all parented to /cloud, which drove the node to PID pressure and
# got cloud ITSELF evicted. A zombie costs no CPU and no memory, so nothing but
# an eviction ever surfaces it. tini reaps them.
RUN apk add --no-cache ca-certificates tzdata sqlcipher-libs git libgcc tini \
&& SC="$(find /usr/lib /lib -name 'libsqlcipher.so*' 2>/dev/null | sort | head -1)" \
&& test -n "$SC" \
&& ln -sf "$SC" /usr/lib/libsqlite3.so.0
&& ln -sf "$SC" /usr/lib/libsqlite3.so.0 \
&& test -x /sbin/tini
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=build /usr/share/zoneinfo /usr/share/zoneinfo
COPY --from=build /etc/passwd /etc/passwd
@@ -175,4 +201,7 @@ COPY --from=build /cloud /cloud
COPY --from=build /smoke /smoke
EXPOSE 8080 9090 9653
USER 65532:65532
ENTRYPOINT ["/cloud"]
# tini as PID 1 forwards signals to /cloud unchanged (so SIGTERM still drains
# normally) and reaps the orphans described above. `--` keeps cloud's own args
# untouched; the CR passes none today, but that stays true if it ever does.
ENTRYPOINT ["/sbin/tini", "--", "/cloud"]
+192
View File
@@ -0,0 +1,192 @@
# IAM cutover — Casdoor pod → embedded IAM in cloud (supervised)
Flip `hanzo.id`'s identity plane from the standalone Casdoor pod to the clean-room
IAM rewrite embedded in this binary (`clients/iam`, `github.com/hanzoai/iam`
**v1.33.6**). This is the last step of HIP-0106 (one binary embeds IAM + KMS + o11y).
**This is a single supervised session. Do not run it piecemeal or in the
background.** Every step below is: **action → verify → rollback**. Money/auth is on
the line — a bad flip 401s every login and every metered request.
The blocking code work is **done and shipped**:
- IAM v1.33.6 serves the legacy path aliases the deployed fleet hard-codes
(`/v1/iam/oauth/access_token`, `/v1/iam/oauth/refresh_token`, `/v1/iam/userinfo`)
next to the canonical paths — so no hard-coded caller 404s at the flip.
- `cloud` pins `github.com/hanzoai/iam v1.33.8` (go.mod) and compiles it in.
> **`iam` is NOT staged.** `stagedSubsystems` in `config.go` holds only `ingress`, so
> the empty-`CLOUD_ENABLE` "mount everything" default — **the live posture on
> `universe/infra/k8s/operator/crs/cloud.yaml`** — mounts the embedded IAM. Step 1 is
> therefore a **precondition of the next deploy**, not of a later flip: a cloud that
> boots before the store is migrated opens an EMPTY `iam2.db` and seeds only from
> `init_data.json`. `iam_edge.go` forwards `/v1/iam/*` to the standalone pod ONLY
> under a non-empty `--enable` that omits `iam`, which is the one way to hold cloud
> on the old plane while Step 1 runs.
## Preconditions (verify before touching anything)
1. Live image is at or above the tag that carries embedded IAM v1.33.6
(`universe/infra/k8s/operator/crs/cloud.yaml` `spec.image.tag`). The subsystem is
compiled in but inert until enabled — safe to deploy ahead of the flip.
2. `spec.replicas: 1` and `strategy: Recreate` (already required — the embedded
store is single-writer/single-open). **`config.go` refuses to boot iam-enabled
above 1 replica.** Never scale up with iam on.
3. `CLOUD_DATA_DIR=/var/lib/cloud` on the RWO `cloud-api-data` PVC. The embedded IAM
store is **`/var/lib/cloud/iam/iam2.db`** (`clients/iam/iam.go` `paths()`) — the v2
store. `iam.db` is a different database; opening it serves the wrong identities
without failing.
4. You have the live Casdoor store to migrate FROM and its KMS master key:
- encrypted sharded root: `<dir>/iam.db` + `<dir>/orgs/*/iam.db` (+ `.dek` sidecars)
- `IAM_KMS_MASTER_KEY` = the 64-hex master key (from KMS — never an arg, never logged)
5. `migrate-v1` is built from the **iam** repo (`github.com/hanzoai/iam`,
`cmd/migrate-v1`, same v1.33.6 tag) with a C `sqlcipher` binary on PATH (for
`--wal-inclusive`).
## Step 1 — Migrate the live store into the embedded datadir (BEFORE any seed)
The embedded IAM seeds new-only from `init_data.json`. Migrating real rows must
happen **before** the subsystem ever seeds, or the seed masks/collides with them.
The source is opened **read-only** — the live Casdoor pod is untouched.
**1a. Dry-run = the drift/parity gate.** `--dry-run` runs the full extraction and
prints the per-entity report WITHOUT writing. Require every entity's count to match
the live source and **drift = 0** before proceeding.
```
migrate-v1 \
--src-datadir /path/to/live/casdoor/store \
--src-master-key-env IAM_KMS_MASTER_KEY \
--wal-inclusive \
--dest /var/lib/cloud/iam/iam2.db \
--dry-run
```
- `--wal-inclusive` checkpoints each shard's uncheckpointed `-wal` via the C
sqlcipher binary → COMPLETE extraction. Without it, uncheckpointed WAL rows are a
hard error (or, with `--ignore-wal`, silently dropped — do NOT use for a real cutover).
- `--dest` accepts either form and both land on the same file here (`storePath` in
`iam/cmd/migrate-v1/main.go`): a `.db` path is taken verbatim, anything else is
treated as a data-dir and gets `/iam2.db` appended. So `…/iam/iam2.db` and `…/iam`
are equivalent. What matters is that the written file is exactly
`/var/lib/cloud/iam/iam2.db` — the path `clients/iam` opens.
**Verify:** dry-run report shows expected counts for users, orgs, applications,
providers, certs; zero drift; zero errors.
**Rollback:** none needed — nothing written.
**1b. Real migration.** Same command **without** `--dry-run`, writing into the
(empty) embedded datadir on the cloud PVC. Do this while iam is still staged OFF.
**Verify:** re-run `--dry-run` against `--src /var/lib/cloud/iam/iam2.db` (or open it
read-only) and confirm counts equal the source.
**Rollback:** `rm -f /var/lib/cloud/iam/iam2.db*` (only the freshly-written store) and
re-run. Nothing else consumes it until cloud next boots iam-enabled.
## Step 2 — Boot cloud with the embedded IAM subsystem
`iam` is **not** staged, so the live CR's empty `CLOUD_ENABLE` already enables it —
there is no env var to add. Deploying the image IS this step. Nothing here is
additive or reversible by a flag: plan Step 1 to complete before the next roll.
Apply the CR; the operator rolls the Recreate Deployment (single pod, brief blip —
expected). On boot, `clients/iam` opens `/var/lib/cloud/iam/iam2.db` (the migrated
store), seeds new-only from `init_data.json` (idempotent — real rows already present,
so seed only adds anything genuinely missing), and mounts the full `/v1/iam/*` surface
IN-PROCESS. `iam_edge.go` stops mounting (`serve.go`: `if !cfg.Enabled("iam") …`), so
there is **no double-mount** and no forward to the Casdoor pod.
**Verify (still on the internal Service, before repointing the edge):**
```
kubectl -n hanzo exec deploy/cloud -- \
curl -s localhost:8000/v1/iam/.well-known/openid-configuration | jq .issuer
# → "https://hanzo.id"
kubectl -n hanzo logs deploy/cloud | grep 'iam embedded in-process'
```
A boot failure serves fail-closed 503 on `/v1/iam/*` (cloud + every other subsystem
stay up) — it does NOT crash the binary.
**Rollback:** set `CLOUD_ENABLE` to an explicit list that OMITS `iam` and re-apply the
CR. Deleting an env var does not roll this back — an empty `CLOUD_ENABLE` is
iam-ENABLED. An explicit list is an allowlist, so it must name every other subsystem
this deployment serves; take it from the CR's own subsystem set, not from memory.
With `iam` out of that list the edge re-mounts and forwards to Casdoor again.
hanzo.id is unaffected (still pointed at Casdoor until Step 3).
## Step 3 — Repoint the hanzo.id identity backend at the edge
`universe/infra/k8s/ingress/routes.yaml`: router `hanzo-id-iam-api` (priority 100)
matches `Host(hanzo.id) && (PathPrefix(/v1/iam) || PathPrefix(/oauth) ||
PathPrefix(/.well-known))``service: iam-hanzo-ai`. Repoint that **service** from
the Casdoor pod to embedded cloud:
```yaml
iam-hanzo-ai:
loadBalancer:
passHostHeader: true
servers:
- url: http://cloud.hanzo.svc.cluster.local:8000 # was: http://iam.hanzo.svc.cluster.local:80
```
Leave the `hanzo-id` service (`id.hanzo.svc:80`, the @hanzo/id login SPA) as-is — only
the API backend moves. The ingress file-provider applies the ConfigMap edit **HOT**
(fsnotify) — **do NOT `rollout restart deploy/ingress`** (that triggers the per-node
ACME storm / TLS outage documented in `universe/CLAUDE.md`).
**Verify:** run the Step-4 parity checks against the public host `https://hanzo.id`.
**Rollback:** revert the one `url:` back to `http://iam.hanzo.svc.cluster.local:80`;
hot-reapplies in seconds. Instant, complete rollback to Casdoor.
## Step 4 — Playwright / curl parity (drive it, don't just curl a status)
Against `https://hanzo.id` (through the repointed edge). Use Playwright for the browser
login (real interaction, not an HTTP status peek):
1. **Discovery + JWKS**: `/.well-known/openid-configuration`,
`/v1/iam/.well-known/openid-configuration`, `/v1/iam/.well-known/jwks` — issuer
`https://hanzo.id`, keys present.
2. **Browser login → code → token** (Playwright): `/login/oauth/authorize` → sign in
→ callback with `code` → token exchange → a verifiable JWT; the app authenticates.
3. **client_credentials** (KMS bridge / gateway guards shape) at BOTH
`/v1/iam/oauth/token` and the alias `/v1/iam/oauth/access_token` → 200 + token.
4. **refresh** at the alias `/v1/iam/oauth/refresh_token` (the `hanzo` CLI shape) → 200
+ rotated token.
5. **userinfo** at BOTH `/v1/iam/oauth/userinfo` and the alias `/v1/iam/userinfo`
(commerce shape) → same principal.
6. **Real callers**: force one live KMS-bridge / gateway-guard token fetch and one
`hanzo login` + `hanzo` CLI refresh against hanzo.id → all 200.
**Accepted delta (not a regression):** BARE `/oauth/token|access_token|userinfo`
(no `/v1/iam` prefix) on hanzo.id 404 post-cutover — the rewrite serves the
`/v1/iam/…`-prefixed canonical + alias paths only, discovery advertises those, and no
live caller uses the bare form (grep-verified: every fleet caller uses `/v1/iam/oauth/*`
or an app-local `/oauth/*` proxy that rewrites to `/v1/iam/*`). Bare `/oauth/authorize`
still 302s to the login SPA via the priority-150 router (unchanged). Optionally tighten
the `hanzo-id-iam-api` rule to drop the bare `/oauth` prefix in the same edit.
**If any parity check fails: roll back Step 3 immediately** (one `url:` revert) and
diagnose with iam still embedded-but-unrouted.
## Step 5 — Retire the standalone Casdoor `iam` (final, only after parity holds)
With hanzo.id served by embedded IAM and parity green, remove the standalone Casdoor
workload. It is operator-managed via its App/CR
(`universe/infra/k8s/operator/crs/iam.yaml`, legacy
`hanzo-operator/crs/iam.yaml` + `iam-v1.yaml`). **First scale to 0** (reversible),
soak, then delete the CR + drop its basename from the Hanzo CD
`universe-crs` `include` glob (so ArgoCD stops governing it).
**Verify:** hanzo.id fully green with the Casdoor pod at 0 replicas for a full soak
(logins, token refresh, metered `/v1/*` traffic). Then delete.
**Rollback (pre-delete):** scale the Casdoor Deployment back to 1 and revert Step 3's
edge `url:` — hanzo.id is back on Casdoor in seconds. **After** the CR is deleted this
is no longer a one-step rollback (re-apply the CR from git), so hold the scale-to-0
soak until you are certain.
## Guardrails (do NOT, until this supervised session)
- Do not roll a cloud image onto the live CR before Step 1 completes — the empty
`CLOUD_ENABLE` mounts embedded IAM on boot, so the deploy itself performs Step 2.
- Do not repoint `iam-hanzo-ai` before Step-2 in-process verification passes.
- Do not delete or scale down the Casdoor `iam` workload before Step-4 parity holds.
- Do not run `migrate-v1` against prod without the read-only source + a green `--dry-run`.
- Do not `rollout restart deploy/ingress` (ACME storm). The routes edit is hot-reloaded.
- Keep `replicas: 1` / `strategy: Recreate` — embedded IAM is single-writer.
+293 -5
View File
@@ -1,10 +1,40 @@
# LLM.md — hanzoai/cloud
Guidance for AI agents working in this repo. `hanzoai/cloud` (HIP-0106) is ONE Go
binary + CLI that mounts every Hanzo subsystem into a single process; the same
artifact serves `api.hanzo.ai`, `api.lux.cloud`, `api.zoo.cloud`, `api.osage.cloud`
and every white-label reseller. Brand, enabled subsystems, and org scope are
deployment configuration.
**Canonical repo.** `hanzoai/cloud` (HIP-0106) is the Open AI Cloud as ONE Go
binary + `hanzo` CLI: every Hanzo subsystem (iam, base, kms, ai, gateway,
commerce, o11y, tasks, …) mounted into a single multi-org process. The same
artifact serves `api.hanzo.ai`, `api.lux.cloud`, `api.zoo.cloud`,
`api.osage.cloud`, and every white-label reseller — brand, enabled subsystems,
and org scope are deployment configuration. This is the impl home for the cloud
control plane; the OpenAPI it emits at `GET /v1/openapi.json` is the single
source for the generated per-language SDKs.
## Role in the SDK model
- Full Cloud SDK is GENERATED from THIS binary's OpenAPI; SDK impl lives in
`hanzo-<lang>/sdk`, docs/wrappers in `hanzoai/<lang>-sdk`, meta in `hanzoai/sdk`.
- AI/agents flagship lib is separate: Python `hanzo` (`hanzoai/python-sdk`),
Node `@hanzo/ai` (`hanzo-js/ai`). Completeness: Python > Rust > C++ > Go.
- DRY: one impl, one place; discovery repos link OUT, never duplicate impl.
- Full spec: `~/work/hanzo/SDK-ARCHITECTURE.md`.
## Brand rules (hard)
- Hanzo is a full AI cloud, NOT a proxy — never "LLM gateway", never position vs
LiteLLM. Zen models are our OWN family; never name upstream models.
- `/v1/` only, never `/api/`. Voice: "Hanzo — the Open AI Cloud."
## Install / run
- `docker run -p 8080:8080 ghcr.io/hanzoai/cloud:vX.Y.Z` (pin a released tag) ·
`go install github.com/hanzoai/cloud/cmd/hanzo@latest` · `brew install hanzoai/tap/hanzo`
- Build in MODULE mode only: `make build` / `GOWORK=off go build ./...` — never
workspace mode (see "Build & module graph" below).
## Key entry points
- `cmd/cloud` — server binary · `cmd/hanzo` (`cli/`) — control CLI · `webui.go` — embedded console
- `apps/apps.go:Wire()` — composition root (the one ordered subsystem slice)
- `deps.go` / `cloud.Deps` — process-wide handles · `clients/<name>/` — every subsystem
- `openapi/` — live spec derived from the router (no checked-in spec file)
---
## Open Cloud planes
@@ -22,6 +52,8 @@ reverse).
| `/v1/compute/bots` | Hosting: `@hanzo/bot` Node containers | `clients/bots` | Shipped |
| `/v1/tasks` | Durable engine | `clients/tasks` | Shipped |
| `/v1/gpus` + fleet | BYO GPU presence | `clients/fleet` + `clients/visor` | Shipped |
| `/v1/cloud` | Cloud accounts: link DO/AWS/GCP/Azure, discover native k8s clusters, fold into the fleet | `clients/venue` (new) | In flight (branch `feat/cloud-account-connectors`; blue-held for red) |
| `/v1/blueprint` | Cost: OSS-template SBOM (compose→images) + compute-cost estimate | `clients/blueprint` (new) | Shipped |
| IAM | Identity: users, orgs, roles | IAM | Shipped |
| KMS | Secret custody: sealed secrets | `clients/kms` | Shipped |
@@ -86,6 +118,16 @@ Dockerfile's dedicated `-tags libsqlite3` CGO stage, and fail under `make test`
by design (clients/git, kms, flags, x402, cmd/kmsreseal, finance). Bundle-embed
tests (clients/tasks/ui) need `make deploy-ui` first (real bundle is gitignored).
Store-heavy subsystem tests are fsync-bound, not CPU-bound. A mount opens its own
SQLite stores, so a test that mounts several subsystems commits many times, and
`t.TempDir()` under `/tmp` puts every commit behind the ext4 journal — on a box with
a concurrent build the same mount that costs milliseconds idle costs ~90s, at ~0%
CPU, blocked in `jbd2_log_wait_commit`. Point `TMPDIR` at tmpfs to measure the real
cost: `TMPDIR=/dev/shm/t GOWORK=off go test -p 1 ./clients/guide` runs the eight-seam
cross-subsystem harness (`clients/guide/drivehome_e2e_test.go`) in under a second.
Prefer one package per `go test` invocation regardless: `./...` links every main
package at once (`cmd/cloud` alone links >6GB).
## Framework doctrine
One way to do everything. Composable, orthogonal, DRY. A new subsystem is a
@@ -131,6 +173,56 @@ package under `clients/<name>` that obeys these seams — nothing more.
the SOLE driver (blank-imported once, in orgdb.go); subsystems never import a
SQLite driver themselves. The caller owns its schema/migration and Close.
## Zero-downtime HA for per-org stores (rolling-upgrade safe)
The per-org store path (`cloud.OrgStore` + `internal/org`) is HA over embedded
SQLite: `ha` decides WHO writes (HRW election + a monotone fencing round), `vfs`
FencedStore decides HOW state ships (hydrate-on-open + fenced ship to S3), and this
layer decides WHEN ownership transitions. Three orthogonal lanes; SQLite stays
embedded underneath.
- **Durability is THE path, capability-detected — no flag.** `buildDurability`
probes at boot: no object store reachable (dev / native-Go) → local-only, same
code path; a reachable store → `org.ProbeCAS` PROVES its conditional-PUT
atomicity (two racing If-None-Match creates + If-Match updates, exactly one winner
each) before fencing any tenant data. A store that can't be proven atomic fails
SAFE to local-only + a loud alert (never fence where two writers could win one
round). Replaced the old `CLOUD_RESEARCH_DURABLE` opt-in — the atomicity gate (H2)
is now a self-check the binary runs.
- **Live membership (no static peer list).** `membership_k8s.go` lists Ready,
non-terminating pods by label (`CLOUD_PEER_SELECTOR`) via the K8s API each 2s
refresh, so a rolling upgrade's changing pod set is tracked and a draining/dead pod
(`DeletionTimestamp` set, or NotReady) leaves the writer election at once. Out of
cluster / no selector → static self set (`podWriterEligible` is the ONE ready gate;
visor has the twin, the shared `hanzoai/ha/k8s` source folds them).
- **M3 live re-acquire, no restart.** A store that opened degraded (read-only) is
promoted IN PLACE when this replica becomes the org's elected owner:
`Durable.PendingPromotion` gates it, `TryClaim` probes the lease (CAS only, no file
I/O), then `OrgStore.promote` quiesces the read-only handle and reopens as owner
(Hydrate renews + CarryForward-restores under the FRESH handle — the file swap is
why the reopen is required). The reopen claims a strictly higher round, fencing the
prior owner — never two live writers.
- **Graceful drain.** SIGTERM → `SetDraining()``/readyz` 503 (drain-aware, ops
port) → K8s marks NotReady → peers re-elect this pod's orgs to live successors
(which hydrate via M3) → the pod stays serving a short grace, then in-flight drains
and final state ships (`OrgStore.CloseAll`, ship-before-close). The shard router
routes on the live set when the durable plane is on, so a draining pod's orgs go to
the ready successor — not to the gone pod. Manifest: readiness → `/readyz` on the
metrics port, `terminationGracePeriodSeconds` ≥ ~40s, RBAC pods:list,watch, the
downward-API `POD_NAME`/`POD_NAMESPACE`, `CLOUD_PEER_SELECTOR` (all in `helm/cloud`).
- **Proof.** `internal/org/rollingupgrade_test.go` rolls 3 pods over 8 orgs with
continuous writes and asserts zero lost acked writes, zero split-brain (no
(org,round) acked by two pods), and continuous availability — across both a pod
restart (fresh rehydrate) and an in-place ownership flap (M3, no restart).
- **Two extensibility seams (for the tiered-storage perf pass).** The fence's
`ConditionalStore` is constructed in `buildDurability`, so a KV read/write-through
cache (L1 over the S3 L2) wraps it as a one-line decorator. The ship mechanism is a
swappable `snapshotCodec` (default `wholeFile`), so WAL-frame delta shipping
replaces it without touching the fence/round. `WithCheckpoint` injects the ship
checkpoint (`durableCheckpoint`) — the crypto envelope's re-encrypt integration
point: on a defer-encryption-to-checkpoint backend it MUST route through the
driver's re-encrypting Checkpoint so ship-before-ack reads FRESH ciphertext (P5).
## The route table has three projections, and the router is the source
`serve.go` composes ONE route table and projects it three ways, all after
@@ -212,6 +304,24 @@ package under `clients/<name>` that obeys these seams — nothing more.
"stopped" runs that were never started. Isolation holds because the org is the
validated one cloud sends, never a client's, and the runtime keys every run
under `tenants/{org}/`.
- **A customer IS an IAM user; marketing keeps no contact list.** Who to email is
read IN-PROCESS from the embedded IAM (`clients/marketing/roster.go`
`iam/pkg/store.GetMailableUsers` over `clients/iam.DB()`), the same seam
`clients/platform` uses for the IAM-owned Project — no HTTP hop to `/v1/iam`
from inside the binary, IAM's `model.User` verbatim, read-only, masked. The org
is `principal.Org`, which IS IAM's `Owner`, so an audience can only ever resolve
its own tenant; `GetMailableUsers` REFUSES an empty org rather than falling back
to the all-orgs view that `GetProjects` deliberately allows. IAM not co-mounted
is a 503, never an empty audience — a send reported successful to nobody is the
worse failure. An audience with no event filter is every mailable customer;
with one, `matchCohort` joins the warehouse `distinct_id`s to that roster and
COUNTS what matched nobody instead of inventing an address.
- **A product announcement is not its own engine.** It is a one-step sequence with
an audience enrolled into it — `POST /v1/marketing/sequences/:id/enroll` takes
`audienceId` where it takes `address` — so it inherits the drip engine's
claimed-once delivery, the ONE send gate (`state.deliver` → suppression →
`notify.Send`), and the signed unsubscribe footer. Never add a blast path beside
`deliver`: the gate is only absolute because it is the only door.
- **Absence is only meaningful from a callee that could have said otherwise.**
`runtime.ErrNotFound` (the operation ANSWERED "no such target") is separate from
`runtime.ErrNotServed` (the operation does not exist). Conflating them makes a
@@ -226,6 +336,39 @@ package under `clients/<name>` that obeys these seams — nothing more.
curriculum is a machine-readable contract (embedded `default.yaml`; org-custom via
PUT replaces it) so `hanzoai/marketing` can author the full `checklist.yaml`
against the same `Step`/`Curriculum` shape.
- **The EXPERIMENT is a composition, not a fourth engine (`clients/experiments`,
`/v1/experiments`).** A/B testing is ONE value whatever the variant KIND (feature
flag, ad creative, email subject, model id): the primitive owns only the
experiment registry (definition + decision); it COMPOSES three planes it never
duplicates. ASSIGNMENT = `flags.Assign(org,project,key,subject,props)`
subject→variant is a deterministic `engineEvaluate` (sha1 rollout hash), no second
bucketing, no assignment store; create writes a multivariate flag def
(`flags.PutDef`) and decide rewrites its weights to 100% for the winner
(`flags.GetDef`+`PutDef`). MEASUREMENT = `analytics.Outcomes(...)` — one
org-scoped `hanzo.events` query (the `eventsWhere` isolation invariant), no second
event store; the analyze fold joins each subject's analytics outcome to its flags
variant by `distinct_id`. EVIDENCE = `research.Record`/`research.List` — per-variant
samples land as immutable `kind:"ab"` rows; significance (two-proportion z-test,
`math.Erfc`, no dep) is a PURE function over them. `clients/campaign` runs a
creative A/B by composing `experiments.Assign`/`experiments.Analyze` — it never
reinvents assignment or evidence. Add a new variant KIND by putting a payload on
the variant; the primitive does not care what it is.
- **The OSS-template compute cost is DERIVED from the compose, not a fourth ledger
(`clients/blueprint`, `/v1/blueprint`).** A blueprint's `docker-compose.yml` is
parsed to its SBOM (the bill of container images) and its services' CPU/memory
footprint priced through ONE documented rate card (microdollars per vCPU-/GB-hour,
DigitalOcean-droplet-derived + platform margin; tunable via
`CLOUD_BLUEPRINT_UCPU_HR`/`_UGB_HR`). Sizing is the declared
`deploy.resources.reservations`/`limits` (or legacy `cpus`/`mem_*`) else a default
footprint per inferred class (db/cache/web/worker/other). `blueprint.EstimateTemplate(id)`
returns `{sbom, vcpuHr, gbHr, microUsdPerHour, estCentsPerMonth}`: `estCentsPerMonth`
is the "~$X/mo to run" the console shows; `microUsdPerHour` is the exact rate the
deploy path meters the deploying org on via the SAME commerce spine `resource_billing`
uses. The author royalty (`clients/authors`, `defaultShareBps=2000`) already accrues
20% of a deploying org's metered spend — this plane only DEFINES the compute component
of that spend from a real rate card; it never touches the ledger or the accrual sweep.
Distinct from `clients/sbom` (CycloneDX packages INSIDE one image, keyed by digest);
this is the bill of IMAGES a stack runs, keyed by template.
## Identity vocabulary is IAM-native
@@ -277,6 +420,112 @@ image to a prior semver and `/{name}/sync` requests a reconcile. SUPERADMIN-only
`gitops-engine` (`hanzoai/deploy/gitops-engine` v0.7.2, no replace) in-process for the
reconcile half behind `DEPLOY_ENGINE_ENABLED` (default off), with a prune-safety fuse.
## The index (`clients/index`, `/v1/index`)
The in-binary index, speaking the Meilisearch REST dialect so a Meilisearch client
repoints by changing one host. It replaced the standalone Meilisearch containers
(`chat-meilisearch`, `search-fts5`).
**Four different things, four names — do not merge them.** `hanzoai/search` is the
SEARCH PRODUCT (our own Meilisearch build, serving `search.hanzo.ai` and the docs
corpus). `clients/websearch` queries the OUTSIDE world. `hanzoai/crawl` fetches it.
`clients/index` is the storage primitive an application writes documents into and
queries back. It is NOT at `/v1/search`: that path belongs to the `hanzoai/ai` RAG
plane, whose `/v1/search/{name}` pattern silently swallowed this subsystem's
single-segment routes (`/health`, `/version` answered 404 in production while every
deeper route worked). `GET /v1/openapi.json` is what shows two owners of one path.
Tenancy is the point: a standalone Meilisearch has ONE
global keyspace behind a master key, so every consumer sharing an instance shares its
indexes — here the tenant is `principal.Org` and every query filters `WHERE org=?`,
so two orgs may both hold an index named `messages`. The credential is the org's
ordinary API key, because the JS client already sends `Authorization: Bearer`.
**The index is a term table, NOT FTS5, and must stay that way.** FTS5 is a
compile-time module and this binary links the SYSTEM SQLite so the SQLCipher codec is
real; that library ships `ENABLE_FTS3` + `HAS_CODEC` with no fts5, and the
`sqlite_fts5` build tag only affects the VENDORED amalgamation, so it is inert here.
An index built on FTS5 opens on a pure-Go build, passes its tests, and then cannot
create a single table in the shipped image. `terms` is keyed
`(org, uid, term, pk)` so a prefix query is an index range scan; it behaves the same
in every build lane. Verify any SQLite module against the production lane
(`-tags "libsqlite3 sqlite_fts5"` + `-lsqlcipher`) before designing on it.
The store is `{DataDir}/index.db`, and a rename must carry the WHOLE family: cek
keeps the wrapped data key beside it as `<path>.dek`, so moving the `.db` alone
strands the key and every document becomes undecryptable — data loss that presents
as an empty index.
## Releases are cut by a merge to main
`.github/workflows` is intentionally empty of CI. The image and its `v*` tags have ONE
owner, `clients/platform/release.go`: compute the next version → build → SMOKE the
pushed image → tag → roll out. The tag is a RECEIPT for a proven image, so a
change that breaks boot never reaches production and leaves no phantom tag.
The final step has ONE writer: patch the operator `hanzo.ai/v1` Service CR's
`spec.image` and let the operator reconcile. It used to write twice — that patch plus
a `repository_dispatch` mirror at `hanzoai/universe` — composed best-effort so the
step passed if EITHER landed. Two writers for one fact, and the composition HID their
disagreement: patch fails, mirror succeeds, cluster and git now describe different
production states with nothing reporting a problem. The mirror was also never running
— it read `UNIVERSE_DISPATCH_TOKEN`, never set on the deployment, so it failed closed
on every release and the CR patch was already doing all the work. A rollout with
nowhere to write is now an ERROR: the image is built, smoke-passed and tagged but NOT
live, and a release that claims otherwise is worse than one that fails.
### Site releases already have a lifecycle — do not build a second one
`clients/projects` owns the full versioned-release model for static sites, and it is
the ONE way:
- `<org>/.releases/<slug>/rel_<128-bit manifest digest>/` — immutable, content-
addressed, and a SIBLING of the mutable `<org>/<slug>/` prefix, so neither a
full-artifact deploy nor a project delete (both of which purge that subtree) can
shred a release the pointer still names.
- `Store.ActivateRelease` — the flip is one atomic `UPDATE … WHERE EXISTS (release
row)`, so it cannot point a site at a release that was never created, and two
concurrent activations cannot leave the pointer disagreeing with whichever won.
`MarkLive` deliberately does NOT touch `current_release`.
- `servePrefix` (`clients/projects/sites.go`) — the ONE read rule, re-validating the
id against `releaseIDRE` before it can widen a prefix. An unrecognized id falls back
to the legacy prefix, so there is no flag day and no migration.
- Rollback is activating an older id. Routes are already mounted on both site
surfaces via `siteReleases`.
A parallel `clients/cd` + `clients/site` lifecycle (kind-agnostic `Target`, a
`CURRENT` pointer object next to the bundles) was built and then DELETED unmerged: it
re-implemented all of the above with a weaker pointer — a `v<N>` counter instead of a
content digest, and a plain PUT that could name a release whose row does not exist.
Its one genuinely new finding is recorded as a gap below, not as a second system.
**Known gap: releases are never garbage-collected.** `promote` writes a new immutable
prefix per distinct content and nothing prunes them; `DeleteReleases` only runs on
project delete. Retention belongs in `clients/projects` next to `promote`. When it is
added, `activate` must also verify the bytes still exist before flipping — today
`ActivateRelease` proves only that the ROW exists, which is sufficient only while
nothing can prune the bytes out from under it.
It is driven by the GitHub App. A push arrives HMAC-verified at
`/v1/connector/github/webhook`, which fires `cloud.OnGitPush` — the SAME
single-registrant seam the embedded git server uses, not a second CI — and
`clients/platform` decides what the push means: an app tracking the repo rebuilds,
and cloud's own upstream cuts a release. Cloud is the machine, so it calls the
release in-process and the build token is never handed to a caller. The trigger runs
BEFORE the token mint and the mirror, because a build reads from GitHub and must not
be lost to a sync outage.
`isReleasePush` is narrow on purpose: the release repo BY URL (an org does not
identify a repo), `main` only, a pinned commit only. Single-flight, because the
version is computed from existing tags and two overlapping runs would compute the
same one. Bot-authored pushes are excluded, so the release's own tag and mirror
pushes cannot retrigger it.
The org an inbound webhook belongs to comes from the App INSTALLATION id via the
`connections` row written by the install callback. Without that row every delivery is
acked `200 {"ignored":"unknown installation"}` and silently does nothing — a 200 on
that path is not evidence it worked; check for sync/build activity.
## The `hanzo` CLI targets THIS binary — one contract, one IAM login
The `hanzo` CLI (`cli/`) is the same unified binary; its control-plane verbs speak the
@@ -300,3 +549,42 @@ app; `?org=` cannot widen it). The rolling restart needs `patch` on `apps/deploy
here (TS-Dokploy contract, 404), and `/v1/platform/*` needs a co-resident IAM store this
deployment does not fold in (IAM runs as a separate svc) so it 500s; the live apps backend
is `/v1/paas`, whose board reads k8s directly with no IAM-store dependency.
## GTM: `/v1/campaign` orchestration → channels → connectors → analytics
The go-to-market stack decomplects a campaign from its execution. A **Campaign is a
VALUE** (`clients/campaign`: `{name, audience, content[], schedule, budget, channels[],
status}`) that SPANS channels; a **Channel is an EXECUTOR** (`channel.go`, the
`Channel` interface) it fans out to. The three channels are orthogonal and each
CONSUMES the connector plane via `integrations.TokenFor` — the campaign object never
touches a credential:
- **paid → `/v1/ads`** — `ads.LaunchPaid/PaidSpend/PausePaid` (`clients/ads/provider.go`)
resolve the org's ad token (`meta_ads`/`google_ads`/… via `TokenFor(org, <id>,
"access_token")`) and run the campaign on the provider. Meta is executed for real;
fail-closed when the org has not connected (424). This is the ONLY place `/v1/ads`
touches the connector plane.
- **organic → `/v1/publish`** (rename of `clients/social`) and **email → `/v1/marketing`**
are DESIGNED follow-ons: register their executors the same way in `apps/wire_seams.go`
(`campaign.RegisterChannel(campaign.NewChannel(kind, launch, spend, pause))`). Until
wired, a fan-out records that channel "unavailable" (honest), never fabricated.
Channels are injected at the composition root (`apps/wire_seams.go`), the SAME
injected-function decoupling the coding dispatcher uses — `campaign` never imports
`ads`, `ads` never imports `campaign`. Fan-out (`launch.go` `fanOut`) is best-effort
per channel; the org (the ONLY tenant key) is passed verbatim to every executor, so a
campaign can only ever resolve its OWN org's token.
**Metrics = the ONE analytics plane, not a second store.** `GET /v1/campaign/:id/metrics`
reads the funnel from `analytics.CampaignMetrics(org, campaignID, variant, start, end)`
(`clients/analytics/campaign.go`) — an org+`utm_campaign`(+`utm_content`)-scoped query
over `hanzo.events`, org and campaign bound POSITIONALLY (same tenancy invariant as
every analytics query) — joined with each channel connector's reported spend
(`Channel.Spend`). Derived KPIs: CTR/CVR/CAC/ROAS. Honest-empty when the warehouse is
absent.
**Creative A/B composes the experiment primitive** (`experiment.go`), it does not
reinvent it: a creative A/B is an experiment whose variant = a creative (tagged
`utm_content`) and whose metric = the analytics read. The `AssignFunc`/`EvidenceFunc`
seams are wired at the root to the flags-assignment + evidence primitive; nil-safe
until it lands (single-creative honest default).
+56 -10
View File
@@ -24,9 +24,12 @@ CONSOLE_DIR ?= ../console
# Path to a hanzoai/openapi checkout — the SOT the agent-skills catalog is generated from.
OPENAPI_DIR ?= ../openapi
# The shipped binary is pure Go (Dockerfile: CGO_ENABLED=0 → scratch). Default all
# build/test targets to that mode so `make build`/`make test` exercise exactly
# what prod runs — and, critically, register the ONE "sqlite" driver exactly once:
# The shipped binary is NOT pure Go. Dockerfile builds /cloud with
# CGO_ENABLED=1 go build -tags "libsqlite3 sqlite_fts5"
# (CGO_ENABLED=0 there builds only the /smoke helper), so production links the live
# libsqlcipher codec: databases are encrypted in place, per-commit, and shareable by
# a second opener. The default below is pure Go for a DIFFERENT reason — it
# registers the ONE "sqlite" driver exactly once:
# cloud's stores use github.com/hanzoai/sqlite (its !cgo backend IS modernc), and
# the embedded deps (ai/base/commerce/o11y/orm/tasks) that import modernc directly
# then resolve to the SAME package → a single registration. A plain CGO_ENABLED=1
@@ -34,9 +37,17 @@ OPENAPI_DIR ?= ../openapi
# and panics at init ("sql: Register called twice for driver sqlite"); `make
# test-cgo` proves the cgo path via the fork's `sqlite_purego` opt-out tag, which
# forces the fork to modernc too so the whole binary registers "sqlite" once.
#
# CONSEQUENCE, worth knowing before trusting a green run: neither target links
# libsqlcipher, so NEITHER exercises the engine the image ships. Without the codec,
# cek falls back to the pure-Go envelope, whose properties differ — a store is
# single-writer and durable at close rather than in-place and per-commit. The tests
# that pin the shipped storage posture (clients/kms concurrent-open, audit
# shareability) therefore skip in both targets. `make test-codec` below is the one
# that runs them, and needs a real libsqlcipher to do it.
CGO_ENABLED ?= 0
.PHONY: help native webui deploy-ui agentskills build build-standalone hanzo run smoke test test-cgo vet tidy docker docker-push clean
.PHONY: help native webui deploy-ui agentskills build build-standalone run smoke test test-cgo test-codec vet tidy docker docker-push clean
help: ## Show this help.
@awk 'BEGIN{FS=":.*##";printf "\nUsage: make <target>\n\nTargets:\n"} /^[a-zA-Z_-]+:.*##/{printf " \033[36m%-16s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST)
@@ -76,9 +87,13 @@ build: ## Build the unified cloud binary into ./bin/cloud (embeds whatever webui
build-standalone: webui build ## Build the REAL 1-binary console: console build:embed → webui/dist → go build.
hanzo: ## Build the hanzo control-plane CLI into ./bin/hanzo (pure Go, same mode as cmd/cloud — registers the ONE "sqlite" driver exactly once; a plain CGO_ENABLED=1 `go build ./cmd/hanzo` links the fork's mattn backend alongside the embedded modernc importers and panics, see header).
@mkdir -p bin
CGO_ENABLED=$(CGO_ENABLED) $(GO) build -ldflags="$(LDFLAGS)" -o bin/hanzo ./cmd/hanzo
# NOTE: cloud builds ONLY the `cloud` binary — the stateless unified API. The Go
# `hanzo` CLI (cmd/hanzo + cli/) is RETIRED: the shipped `hanzo` is the Rust CLI
# (~/work/hanzo/cli, `curl hanzo.sh`), which talks to this API over HTTP via its
# OpenAPI-generated command surface. The `code` wrapper (incl. the zen-tier 1M
# mechanism) now lives in the Rust CLI. cmd/hanzo + cli/ remain only as the
# reference for the still-to-port client-side tools (GPU fleet worker `link`,
# `runner`, `engine`, `security`) and are no longer built here.
run: build ## Run with iam,base,kms,gateway,o11y enabled (matches README quickstart).
./bin/$(BIN) --enable=iam,base,kms,gateway,o11y --brand=hanzo --domain=api.hanzo.ai
@@ -86,11 +101,42 @@ run: build ## Run with iam,base,kms,gateway,o11y enabled (matches README quickst
smoke: ## Build and run cmd/cloud-smoke (mount-time integration check).
$(GO) run ./cmd/cloud-smoke
test: ## Run unit + integration tests (pure-Go, exactly as prod ships).
CGO_ENABLED=$(CGO_ENABLED) $(GO) test ./...
# The data plane has no plaintext-at-rest mode: cek refuses to open a store without
# a master key, on every build. The server makes that a boot decision (serve.go); a
# test run has no boot, so the suite declares its own dev posture HERE — once, for
# every package — instead of each package carrying a copy. A key already in the
# environment always wins, so CI's real key is never overridden.
DEV_KMS_KEY := AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
TEST_ENV = CLOUD_KMS_MASTER_KEY_REF="$${CLOUD_KMS_MASTER_KEY_REF:-$(DEV_KMS_KEY)}"
# The release image builds with -tags "libsqlite3 sqlite_fts5" (see Dockerfile).
# libsqlite3 needs cgo and the C library, but sqlite_fts5 does not — and without it
# any store whose migration declares an FTS5 table fails to open, so a subsystem
# built on full-text search (clients/code) cannot be tested at all. Carry the tag
# the shipped build carries, so the suite exercises the same schema surface.
TEST_TAGS := sqlite_fts5
test: ## Run unit + integration tests (pure-Go, with the FTS5 tag the image ships).
$(TEST_ENV) CGO_ENABLED=$(CGO_ENABLED) $(GO) test -tags "$(TEST_TAGS)" ./...
test-cgo: ## Prove the cgo build works too — forces the fork's pure-Go backend via -tags sqlite_purego so the embedded modernc importers don't double-register "sqlite".
CGO_ENABLED=1 $(GO) test -tags sqlite_purego ./...
$(TEST_ENV) CGO_ENABLED=1 $(GO) test -tags "sqlite_purego $(TEST_TAGS)" ./...
# The only target that builds what the image builds (Dockerfile: CGO_ENABLED=1,
# -tags "libsqlite3 sqlite_fts5"). The other two link no codec, so cek falls back to
# the pure-Go envelope and the tests pinning the shipped storage posture — a store
# shareable by a second opener, durable per-commit rather than at close — skip
# instead of running.
#
# The tag alone is not enough: it selects the C engine, but the codec is a RUNTIME
# probe of the libsqlcipher that engine links. A csqlite built against plain SQLite
# compiles and passes the one-engine guard while CodecLinked() stays false, so the
# storage tests would still quietly skip. SQLITE_REQUIRE_CODEC=1 — the same
# assertion the Dockerfile makes before it builds /cloud — turns that into a
# failure, so this target either exercises the shipped engine or says it cannot.
# It therefore FAILS on a machine without SQLCipher, which is the honest result.
test-codec: ## Run the suite against the engine the image ships (cgo + a real libsqlcipher).
SQLITE_REQUIRE_CODEC=1 $(TEST_ENV) CGO_ENABLED=1 $(GO) test -tags "libsqlite3 $(TEST_TAGS)" ./...
vet: ## go vet across the module.
CGO_ENABLED=$(CGO_ENABLED) $(GO) vet ./...
+32
View File
@@ -0,0 +1,32 @@
Hanzo Cloud
Copyright (c) 2026 Hanzo Industries, Inc.
This product includes software from the following upstream projects. Their names
appear here and NOWHERE ELSE in this codebase's user-visible surfaces — Hanzo
products are named for Hanzo (Hanzo Analytics, Hanzo Insights, Hanzo IAM, …).
Umami (https://github.com/umami-software/umami), licensed under MIT:
Copyright (c) 2022 Umami Software, Inc. <hello@umami.is>
Hanzo Analytics (analytics.hanzo.ai) is a fork. The cloud destination adapter
speaks its public collect contract.
PostHog (https://github.com/PostHog/posthog), licensed under MIT:
Copyright (c) 2020-present PostHog Inc.
Hanzo Insights (insights.hanzo.ai) is a fork. The cloud destination adapter
speaks its capture contract.
Casibase (https://github.com/casibase/casibase), licensed under Apache-2.0:
Copyright (c) The Casibase Authors
The Hanzo AI module (the /v1 AI, RAG, and search surfaces) derives from it.
Casdoor (https://github.com/casdoor/casdoor), licensed under Apache-2.0:
Copyright (c) The Casdoor Authors
Hanzo IAM (hanzo.id) derives from it.
+74 -63
View File
@@ -1,21 +1,30 @@
<p align="center"><img src=".github/hero.svg" alt="cloud" width="880"></p>
<p align="center"><img src=".github/hero.svg" alt="Hanzo Cloud" width="880"></p>
# cloud
# Hanzo Cloud
Unified Go control plane and binary for the Hanzo platform (HIP-0106).
**The Open AI Cloud as one Go binary.** Identity, secrets, data, AI, gateway, observability, and the console — every Hanzo-native subsystem mounted into a single multi-org process. Per [HIP-0106](https://github.com/hanzoai/HIPs/blob/main/HIPs/hip-0106-unified-hanzo-cloud-binary.md).
[![Status](https://img.shields.io/badge/status-beta-blue)]()
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)]()
The same artifact serves `api.hanzo.ai`, `api.lux.cloud`, `api.zoo.cloud`, `api.osage.cloud`, and every white-label reseller. Brand, enabled subsystems, and org scope are deployment configuration — one binary, one origin, no sidecars.
## Quick start
```bash
docker run -p 8080:8080 ghcr.io/hanzoai/cloud:latest
# Run the unified binary (pin a released version)
docker run -p 8080:8080 ghcr.io/hanzoai/cloud:v1.801.206
# Or install the CLI + server
go install github.com/hanzoai/cloud/cmd/hanzo@latest
brew install hanzoai/tap/hanzo
```
Open <http://localhost:8080> for the embedded console; the API is served under `/v1` on the same origin.
## What this is
`hanzoai/cloud` is one Go binary that mounts every Hanzo subsystem (iam, kms, base, gateway, ai, commerce, vfs, mq, dns, amqp, mcp, o11y, ...) into a single multi-org process. Same artifact serves `api.hanzo.ai`, `api.osage.cloud`, `api.lux.cloud`, `api.zoo.cloud`, and every white-label reseller. Brand, enabled subsystems, and org scope are deployment configuration.
`hanzoai/cloud` is one Go binary that mounts every Hanzo subsystem (iam, kms, base, gateway, ai, commerce, vfs, mq, dns, amqp, mcp, o11y, tasks, …) into a single multi-org process. The same artifact serves `api.hanzo.ai`, `api.osage.cloud`, `api.lux.cloud`, `api.zoo.cloud`, and every white-label reseller. Brand, enabled subsystems, and org scope are deployment configuration.
## `hanzo` — cloud control CLI
@@ -47,16 +56,31 @@ authed (it cannot validate user tokens), so `apps`/`deploy`/`clusters` use
Install: `go install github.com/hanzoai/cloud/cmd/hanzo@latest`, or `brew install hanzoai/tap/hanzo`.
## Specs
## Subsystems mounted
Implements:
- HIP-0014 Application Deployment
- HIP-0026 IAM
- HIP-0027 KMS
- HIP-0037 AI Cloud Platform
- HIP-0105 In-Process Extension Runtime
- HIP-0106 Unified Cloud Binary
- HIP-0302 Encrypted SQLite + ZapDB Durability
Each subsystem exposes `func Mount(app *zip.App, deps cloud.Deps) error` and wires its own `/v1/<name>/*` routes onto the shared app.
- `iam` — identity & access (users, orgs, roles, OIDC/JWKS per HIP-0026)
- `base` — per-org SQLite + in-process extension runtimes (HIP-0105)
- `kms` — secret custody (sealed secrets, HIP-0027)
- `commerce` — checkout, billing, pricing, invoicing (light router; NOT in PCI-DSS scope)
- `ai` — AI control plane: inference, RAG, model hub, agents, MCP management
- `gateway` — HTTP routing + policy
- `o11y` — metrics / traces / logs
- `vfs` — virtual filesystem / object-store abstraction
- `mq` — message queue
- `dns`, `amqp`, `mcp`, `auto`, `tasks`, … — full list per HIP-0106
## Deployment modes
Same binary; different startup configuration:
```bash
cloud --enable=iam,base,kms,commerce,ai,gateway,o11y --brand=hanzo --domain=hanzo.ai
cloud --enable=iam,base,kms,commerce,ai,gateway,o11y --brand=osage --domain=osage.cloud
cloud --enable=iam,base,kms,commerce,ai,gateway,o11y --brand=lux --domain=lux.cloud
cloud --enable=iam,base,kms,commerce,ai,gateway,o11y --brand=zoo --domain=zoo.cloud
```
## Architecture
@@ -70,49 +94,15 @@ Implements:
| Mount() | Mount() | Mount() | Mount() | Mount() |
+----------+----------+----------+----------+----------+
per-org SQLite (HIP-0302) | Hanzo IAM JWKS (HIP-0026)
replicate -> S3 (HIP-0107) | ZAP inter-subsystem RPC
replicate -> S3 (HIP-0107) | ZAP inter-subsystem RPC
```
Every subsystem exposes `func Mount(app *zip.App, deps cloud.Deps) error`. White-label fork pattern: customers fork this repo to launch their own ecosystem.
---
# Hanzo Cloud
The unified Go binary that imports every Hanzo-native subsystem and dispatches
requests per deployment configuration. One artifact, many subsystems.
Per [HIP-0106](https://github.com/hanzoai/HIPs/blob/main/HIPs/hip-0106-unified-hanzo-cloud-binary.md).
## Subsystems mounted
- `iam` — identity & access
- `base` — per-org SQLite + extension runtimes (per HIP-0105)
- `kms` — secrets
- `commerce` — checkout, billing, pricing, invoicing (light router; NOT in PCI-DSS scope)
- `ai` — LLM control plane / RAG / model hub / MCP management (was hanzoai/cloud pre-rename)
- `gateway` — HTTP routing + policy
- `o11y` — metrics / traces / logs
- `vfs` — virtual filesystem / object-store abstraction
- `mq` — message queue
- `dns`, `amqp`, `mcp`, `auto`, `tasks`, ... (full list per HIP-0106)
## Deployment modes
Same binary; different startup configuration:
```bash
cloud --enable=iam,base,kms,commerce,ai,gateway,o11y --brand=hanzo --domain=hanzo.ai
cloud --enable=iam,base,kms,commerce,ai,gateway,o11y --brand=osage --domain=osage.cloud
cloud --enable=iam,base,kms,commerce,ai,gateway,o11y --brand=lux --domain=lux.cloud
cloud --enable=iam,base,kms,commerce,ai,gateway,o11y --brand=zoo --domain=zoo.cloud
```
Every subsystem mounts through the same `Mount(app, deps)` seam. Cross-subsystem calls ride a narrow in-process interface; no subsystem reaches into another's store.
## White-label fork pattern
Customers fork `hanzoai/cloud` to launch their own ecosystem in one binary. Brand
detection, enabled subsystems, ZAP endpoints (payments / vault backends) are all
detection, enabled subsystems, and ZAP endpoints (payments / vault backends) are all
deployment configuration.
## Web framework
@@ -153,18 +143,39 @@ even without the Node toolchain. The image build overwrites `webui/dist` with th
real console bundle. See `webui_test.go` for the boot-and-assert tests
(`/` → shell, deep link → shell 200, `/v1/*` → API, unmatched `/v1` → 404).
Current state: `hanzoai/console` exposes `build:embed` (`scripts/build-embed.mjs`),
which stashes its Next server route handlers (BFF proxies that collapse to the
cloud `/v1/*` the SPA calls same-origin), wraps the client catch-all pages for
`output: 'export'`, neutralizes the root layout's request-time `headers()` read,
and emits a real static export at `out/` (a ~360 KB `index.html` + `_next/`
chunks). The image build (and `make webui`) run it and overlay `webui/dist`, so
`//go:embed` bakes the FULL `@hanzo/gui` console into the ONE binary. The
Dockerfile console stage FAILS HARD if that bundle is missing or degenerate —
the placeholder shell can never silently ship to prod (escape hatch:
`--build-arg ALLOW_PLACEHOLDER=1` for a pure-Go dev image).
The `hanzoai/console` `build:embed` script (`scripts/build-embed.mjs`) stashes its
Next server route handlers (BFF proxies that collapse to the cloud `/v1/*` the SPA
calls same-origin), wraps the client catch-all pages for `output: 'export'`,
neutralizes the root layout's request-time `headers()` read, and emits a real
static export at `out/` (a ~360 KB `index.html` + `_next/` chunks). The image
build (and `make webui`) run it and overlay `webui/dist`, so `//go:embed` bakes
the FULL `@hanzo/gui` console into the ONE binary. The Dockerfile console stage
FAILS HARD if that bundle is missing or degenerate — the placeholder shell can
never silently ship to prod (escape hatch: `--build-arg ALLOW_PLACEHOLDER=1` for a
pure-Go dev image).
## Specs
Implements:
- HIP-0014 Application Deployment
- HIP-0026 IAM
- HIP-0027 KMS
- HIP-0037 AI Cloud Platform
- HIP-0105 In-Process Extension Runtime
- HIP-0106 Unified Cloud Binary
- HIP-0129 Open Cloud Planes
- HIP-0302 Encrypted SQLite + ZapDB Durability
## Status
Scaffold. The Mount(app, deps) integration for each subsystem lands per
HIP-0106's migration phases.
In production. The unified binary serves `api.hanzo.ai` and the white-label cloud
surfaces today, with per-org SQLite (HIP-0302) and the embedded console. Subsystems
continue to land per HIP-0106's migration phases; `apps/apps.go:Wire()` is the one
ordered list of everything mounted. For repo-level engineering doctrine (module
graph, route-table projections, cross-subsystem seams), see [`LLM.md`](./LLM.md).
## Hanzo — the Open AI Cloud
Open source · every language · on-chain settlement. [hanzo.ai](https://hanzo.ai) · [docs.hanzo.ai](https://docs.hanzo.ai)
**SDKs in every language** — [Python](https://github.com/hanzoai/python-sdk) (flagship) · [TypeScript](https://github.com/hanzo-js/sdk) · [Go](https://github.com/hanzo-go/sdk) · [Rust](https://github.com/hanzo-rs/sdk) · [C++](https://github.com/hanzo-cpp/sdk) · [Swift](https://github.com/hanzo-swift/sdk) · [Kotlin](https://github.com/hanzo-kt/sdk) · [umbrella](https://github.com/hanzoai/sdk)
+72
View File
@@ -0,0 +1,72 @@
// Copyright 2026 Hanzo AI Inc. All Rights Reserved.
package cloud
import (
"net/http"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
)
// AccountFromPrincipal decomplects the console's IDENTITY onto the ONE truth every
// /v1/admin/* call already authorizes on — the validated cloud principal — instead of
// the embedded casibase account model.
//
// THE COMPLECTION IT REMOVES. The operator SPA authenticates via /v1/signin (a cloud
// PKCE session → the X-User-* principal the middleware mints) but read its IDENTITY
// from /v1/get-account, which the embedded IAM (casibase) answers from ITS OWN session
// cookie. A PKCE session is not a casibase session, so get-account returned
// owner:"hanzo" (anonymous) or "Unauthorized operation" — and the SPA's SuperAdmin
// gate (owner == "admin" && isAdmin), reading that, bounced the operator UI to login
// even though the SAME session got 200 from every /v1/admin/* route. Two session
// models for one surface; the identity source and the authorization source disagreed.
//
// THE DECOMPLECTION. Identity is now the principal: when a VALIDATED principal is
// present (X-User-Id is set ONLY by IdentityMiddleware from a real credential, never a
// raw client header — see middleware_identity.go), /v1/get-account reflects it. owner
// is the HOME org (principal.Owner) so a SuperAdmin org-switched into a tenant stays a
// SuperAdmin; isAdmin is the validated bit. With NO principal it falls through
// (c.Next()) to the casibase account surface unchanged — the anonymous sign-in page
// and any legacy casibase-session caller are untouched. One truth, additive, fail-open
// to the old path. MUST be registered AFTER IdentityMiddleware (needs the minted
// headers) and BEFORE MountAll (so it precedes the casibase /v1/get-account handler).
// accountPath is the account read this middleware fronts. It is a named constant
// because the interception is a PATH MATCH: when the /v1 surface was namespaced
// (/v1/get-account → /v1/ai/account) a literal left un-updated here would not
// error — the middleware would simply stop firing, fall through to the casibase
// account surface, and hand the SPA the anonymous owner again. That is precisely
// the bug this file exists to fix, silently restored.
const accountPath = "/v1/ai/account"
func AccountFromPrincipal() zip.Handler {
return func(c *zip.Ctx) error {
if c.Method() != http.MethodGet || c.Path() != accountPath {
return c.Next()
}
user := c.User() // X-User-Id — minted only from a validated credential
if user == "" {
return c.Next() // no principal → the casibase account surface (unchanged)
}
owner := principal.Owner(c) // HOME org (a SuperAdmin stays one when org-switched)
if owner == "" {
owner = c.Org()
}
name := c.Header("X-User-Name")
if name == "" {
name = user
}
return c.JSON(http.StatusOK, map[string]any{
"status": "ok",
"msg": "",
"data": map[string]any{
"owner": owner,
"name": name,
"displayName": name,
"email": c.Header("X-User-Email"),
"isAdmin": c.IsAdmin(),
"type": "normal-user",
},
})
}
}
+43
View File
@@ -0,0 +1,43 @@
package cloud
import (
"context"
tasksclient "github.com/hanzoai/tasks/pkg/sdk/client"
)
// Callbacks the ai module makes back into the host. Declared here, installed by
// apps/ — importing ai/object here would put ~1480 packages under every
// subsystem, and nothing here reads back from it.
// UsageEvent mirrors the ai module's payload. Separate on purpose: sharing the
// type would reintroduce the import.
type UsageEvent struct {
Subject string
Namespace string
USD string // exact decimal USD ("0.00132"), never a rounded cent
Currency string
Model string
Provider string
RequestID string
}
type (
TierReaderFunc func(ctx context.Context, subject, namespace string) (string, error)
BalanceReaderFunc func(ctx context.Context, subject, namespace, currency string) (int64, error)
UsageRecorderFunc func(ctx context.Context, u UsageEvent) error
IngestDialerFunc func(org string) (tasksclient.Client, error)
)
var (
tierReader TierReaderFunc
balanceReader BalanceReaderFunc
usageRecorder UsageRecorderFunc
ingestDialer IngestDialerFunc
)
// nil means that subsystem isn't co-resident; apps/ leaves it uninstalled.
func TierReader() TierReaderFunc { return tierReader }
func BalanceReader() BalanceReaderFunc { return balanceReader }
func UsageRecorder() UsageRecorderFunc { return usageRecorder }
func IngestDialer() IngestDialerFunc { return ingestDialer }
+221 -71
View File
@@ -1,7 +1,7 @@
// Package apps is the composition root: the single, explicit list of which
// Hanzo cloud subsystems are linked into the binary AND the order they mount in.
//
// Wire() returns []cloud.MountSpec in mount order (slice position == order). There
// Wire() returns []cloud.AppSpec in mount order (slice position == order). There
// is no init()-registry and no order-int: adding, removing, or reordering a
// subsystem is a one-line edit to Wire(), read top-to-bottom. cmd/cloud and
// cmd/hanzo both call Wire() and thread the slice into cloud.Serve — the set is
@@ -14,9 +14,13 @@
// HIP-0106: the unified cloud binary is the APPLICATION layer plus the embedded KMS
// secrets plane and the embedded IAM identity plane ("one Go binary embeds IAM +
// KMS + o11y"). The edge/infra tier (mcp, gateway, ingress-edge) runs as its own
// deployments for blast-radius isolation; several application folds (iam, base,
// commerce, captable, dataroom, sign, ingress) are STAGED — linked here but mounted
// only when the operator names them in CLOUD_ENABLE.
// deployments for blast-radius isolation.
//
// Every subsystem below mounts under the default (empty CLOUD_ENABLE) EXCEPT the
// staged ones — see config.go's stagedSubsystems, which is the single source of
// that set. A subsystem is staged when its Mount can abort startup, so it must be
// named in CLOUD_ENABLE deliberately; one whose Mount fails closed instead (as
// clients/iam does) is not staged.
//
// Ordering provenance: order-int ascending; ties in the exact order the
// pre-refactor init()-registry mounted them, captured empirically from origin/main
@@ -35,6 +39,8 @@ import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
@@ -42,7 +48,6 @@ import (
// External subsystem modules. As of the atomic wave-2 bump they NO LONGER
// self-register (no cloud.Register in their init) — the composition root wires
// each one explicitly below, so removing an entry here is the ONLY way to drop it.
"github.com/hanzoai/ai"
"github.com/hanzoai/authz"
"github.com/hanzoai/licensing"
"github.com/hanzoai/metrics"
@@ -58,28 +63,37 @@ import (
"github.com/hanzoai/cloud/clients/agents"
"github.com/hanzoai/cloud/clients/agentskills"
"github.com/hanzoai/cloud/clients/analytics"
"github.com/hanzoai/cloud/clients/ask"
"github.com/hanzoai/cloud/clients/auditlog"
"github.com/hanzoai/cloud/clients/authors"
"github.com/hanzoai/cloud/clients/automations"
"github.com/hanzoai/cloud/clients/base"
"github.com/hanzoai/cloud/clients/benchmark"
"github.com/hanzoai/cloud/clients/billing"
"github.com/hanzoai/cloud/clients/blueprint"
"github.com/hanzoai/cloud/clients/books"
"github.com/hanzoai/cloud/clients/bots"
"github.com/hanzoai/cloud/clients/campaign"
"github.com/hanzoai/cloud/clients/captable"
"github.com/hanzoai/cloud/clients/catalogsync"
"github.com/hanzoai/cloud/clients/channels"
"github.com/hanzoai/cloud/clients/cloudflare"
"github.com/hanzoai/cloud/clients/code"
"github.com/hanzoai/cloud/clients/company"
"github.com/hanzoai/cloud/clients/compliance"
"github.com/hanzoai/cloud/clients/content"
"github.com/hanzoai/cloud/clients/crm"
"github.com/hanzoai/cloud/clients/dataroom"
"github.com/hanzoai/cloud/clients/deploy"
"github.com/hanzoai/cloud/clients/destinations"
"github.com/hanzoai/cloud/clients/dns"
"github.com/hanzoai/cloud/clients/do"
"github.com/hanzoai/cloud/clients/domain"
"github.com/hanzoai/cloud/clients/entitlements"
"github.com/hanzoai/cloud/clients/esign"
"github.com/hanzoai/cloud/clients/eval"
"github.com/hanzoai/cloud/clients/exec"
"github.com/hanzoai/cloud/clients/experiments"
"github.com/hanzoai/cloud/clients/flags"
"github.com/hanzoai/cloud/clients/framework"
"github.com/hanzoai/cloud/clients/functions"
@@ -87,23 +101,32 @@ import (
"github.com/hanzoai/cloud/clients/git"
"github.com/hanzoai/cloud/clients/graph"
"github.com/hanzoai/cloud/clients/guide"
"github.com/hanzoai/cloud/clients/help"
"github.com/hanzoai/cloud/clients/iam"
"github.com/hanzoai/cloud/clients/iam2"
"github.com/hanzoai/cloud/clients/index"
"github.com/hanzoai/cloud/clients/ingress"
"github.com/hanzoai/cloud/clients/integrations"
"github.com/hanzoai/cloud/clients/kafka"
"github.com/hanzoai/cloud/clients/kms"
"github.com/hanzoai/cloud/clients/knowledge"
"github.com/hanzoai/cloud/clients/leaderboard"
"github.com/hanzoai/cloud/clients/legal"
"github.com/hanzoai/cloud/clients/link"
"github.com/hanzoai/cloud/clients/marketing"
"github.com/hanzoai/cloud/clients/marketplace"
"github.com/hanzoai/cloud/clients/ml"
"github.com/hanzoai/cloud/clients/notify"
"github.com/hanzoai/cloud/clients/o11y"
// NOTE: clients/o11y is deliberately NOT imported. It is loaded at run time
// as a plugin (see the o11y entry in Wire), and this line is the whole reason
// that works: an import here would keep its 2.7k-package graph — the
// otel-collector, prometheus, gonum — linked into cloud whether or not any
// Wire entry referenced it. Unlinking a subsystem means deleting its import,
// not just its mount.
"github.com/hanzoai/cloud/clients/paas"
"github.com/hanzoai/cloud/clients/plan"
"github.com/hanzoai/cloud/clients/platform"
"github.com/hanzoai/cloud/clients/plugin"
"github.com/hanzoai/cloud/clients/prefs"
"github.com/hanzoai/cloud/clients/pricing"
"github.com/hanzoai/cloud/clients/product"
"github.com/hanzoai/cloud/clients/projects"
@@ -111,12 +134,13 @@ import (
"github.com/hanzoai/cloud/clients/provisioning"
"github.com/hanzoai/cloud/clients/pubsub"
"github.com/hanzoai/cloud/clients/referrals"
"github.com/hanzoai/cloud/clients/research"
"github.com/hanzoai/cloud/clients/rollingcap"
"github.com/hanzoai/cloud/clients/runtime"
"github.com/hanzoai/cloud/clients/sbom"
"github.com/hanzoai/cloud/clients/security"
"github.com/hanzoai/cloud/clients/settings"
"github.com/hanzoai/cloud/clients/sign"
"github.com/hanzoai/cloud/clients/share"
"github.com/hanzoai/cloud/clients/social"
"github.com/hanzoai/cloud/clients/storage"
"github.com/hanzoai/cloud/clients/sync"
@@ -125,29 +149,33 @@ import (
"github.com/hanzoai/cloud/clients/templates"
"github.com/hanzoai/cloud/clients/tools"
"github.com/hanzoai/cloud/clients/tracker"
"github.com/hanzoai/cloud/clients/translate"
"github.com/hanzoai/cloud/clients/treasury"
"github.com/hanzoai/cloud/clients/usage"
"github.com/hanzoai/cloud/clients/validators"
"github.com/hanzoai/cloud/clients/venue"
"github.com/hanzoai/cloud/clients/visor"
"github.com/hanzoai/cloud/clients/wallets"
"github.com/hanzoai/cloud/clients/webhooks"
"github.com/hanzoai/cloud/clients/websearch"
"github.com/hanzoai/cloud/clients/world"
"github.com/hanzoai/cloud/clients/x402"
"github.com/hanzoai/cloud/clients/zt"
// Framework CONTENT modules — NOT mount subsystems (they carry no HTTP surface
// and are absent from Wire()). Each registers its DocType fixtures and, for erp,
// its ledger-posting lifecycle hooks into the clients/framework DocType engine
// from a package init() (framework.RegisterModule) — the idiomatic
// register-into-a-registry pattern (cf. database/sql drivers). The framework
// engine is mounted (always-on, /v1/framework/*) but its module registry is
// populated ONLY by these blank imports. Dropping one silently strips that
// lane's DocTypes and hooks — for erp, the immutable ledger postings — from the
// binary with NO mount change and NO failing mount test. #248 dropped them;
// TestFrameworkContentModulesLinked now guards against a recurrence. Keep.
// Framework CONTENT modules — pure fixture lanes that carry no HTTP surface and
// are absent from Wire(). Each registers its DocType fixtures and, for erp, its
// ledger-posting lifecycle hooks into the clients/framework DocType engine from a
// package init() (framework.RegisterModule) — the idiomatic register-into-a-
// registry pattern (cf. database/sql drivers). The framework engine is mounted
// (always-on, /v1/framework/*) but its module registry is populated ONLY by these
// blank imports. Dropping one silently strips that lane's DocTypes and hooks — for
// erp, the immutable ledger postings — from the binary with NO mount change and NO
// failing mount test. #248 dropped them; TestFrameworkContentModulesLinked now
// guards against a recurrence. Keep. (help is a framework lane too but ALSO mounts
// a thin /v1/help public plane, so it is a real import + a Wire() spec below,
// alongside knowledge — the other lane with a companion subsystem.)
_ "github.com/hanzoai/cloud/clients/cms"
_ "github.com/hanzoai/cloud/clients/erp"
_ "github.com/hanzoai/cloud/clients/help"
)
// init wires the cross-subsystem func seams — the composition root is the one place
@@ -168,29 +196,13 @@ func init() {
})
}
// identitySpec selects the ONE identity backend that owns /v1/iam/* (+ /login/oauth/*)
// for this boot. CLOUD_IAM_IMPL=iam2 picks the clean-room iam2 (zip+orm, beego-free);
// anything else — including unset, the production default — keeps the legacy beego
// Casdoor embed, byte-for-byte today's behavior. The two impls register the SAME
// absolute prefixes and therefore cannot co-mount, so selection (this func) stays
// separate from activation (cfg.Enabled): exactly one spec occupies the identity slot
// in Wire, preserving mount order either way. os.Getenv (not the unexported
// cloud.getenv, which is unreachable from package apps) is the read — CLOUD_IAM_IMPL is
// the deliberate, off-by-default opt-in that keeps iam2 inert until a canary flips it.
func identitySpec() cloud.MountSpec {
if os.Getenv("CLOUD_IAM_IMPL") == "iam2" {
return cloud.MountSpec{Name: "iam2", Mount: iam2.Mount}
}
return cloud.MountSpec{Name: "iam", Mount: iam.Mount}
}
// Wire returns every linked subsystem as a cloud.MountSpec, in mount order. The
// Wire returns every linked subsystem as a cloud.AppSpec, in mount order. The
// slice position IS the order: cloud.MountAll iterates it as-given, registering each
// subsystem's teardown as a zip shutdown hook so teardown runs in reverse (LIFO).
// Enablement is a separate axis: cloud.Serve mounts only the specs cfg.Enabled(name)
// admits, so a STAGED subsystem is linked but inert until named.
func Wire() []cloud.MountSpec {
return []cloud.MountSpec{
func Wire() []cloud.AppSpec {
return []cloud.AppSpec{
// embedded NATS :4222 + JetStream.
{Name: "pubsub", Mount: pubsub.Mount, Shutdown: pubsub.Shutdown},
// embedded Kafka adaptor :9092.
@@ -207,7 +219,7 @@ func Wire() []cloud.MountSpec {
// hanzoai/metrics — native o11y. It declares its OWN narrow metrics.Deps (no
// hanzoai/cloud import), so Typed cannot adapt it; mountMetrics builds that Deps
// from cloud.Deps and calls metrics.Mount explicitly.
{Name: "metrics", Mount: mountMetrics},
{Name: "metrics", Mount: cloud.Global(mountMetrics), Global: true},
// Embedded runtime edge (/v1/ingress/*). STAGED — edge listeners stay off unless
// the operator names "ingress" in CLOUD_ENABLE.
{Name: "ingress", Mount: ingress.Mount, Shutdown: ingress.Shutdown},
@@ -215,32 +227,44 @@ func Wire() []cloud.MountSpec {
// /v1/commerce/topup/wallet). MUST mount before the IAM /v1/iam/* wildcard (50) so
// they win Fiber's first-match scan (framework-guaranteed since zip v1.3.0).
{Name: "account", Mount: account.MountAccount},
// Embedded IAM identity plane (/v1/iam/*, /.well-known/*, /login/oauth/*, /_/iam/*,
// /cas/*, /scim/*) — the identity authority, mounts before its dependents. STAGED:
// the operator adds "iam" to --enable only after IAM config + the fold are verified.
// Which IMPLEMENTATION owns these prefixes is selected by CLOUD_IAM_IMPL
// (identitySpec): the clean-room iam2 (zip+orm, beego-free) when =="iam2", else the
// legacy beego Casdoor embed — the default (unset = today's behavior, byte-for-byte).
// Both register the SAME absolute paths and cannot co-mount, so this is an either/or
// switch at this ONE slot, never a shadow prefix.
identitySpec(),
// Embedded IAM identity plane (/v1/iam/*, /login/oauth/*) — the identity authority,
// mounts before its dependents. The ONE implementation: the clean-room iam-v2
// (zip-native + hanzoai/orm, beego-free); the retired Casdoor iam-v1 embed is GONE.
// STAGED: the operator adds "iam" to --enable only after IAM config + the fold are
// verified (login/authorize/token/jwks + the operator SSO chain).
{Name: "iam", Mount: iam.Mount, Prefixes: iam.Prefixes},
// Embedded Base app engine + viral waitlist (/v1/waitlist/*). STAGED behind
// CLOUD_BASE_EMBED. OwnsHealth: native /v1/base/health.
{Name: "base", Mount: base.Mount, Shutdown: base.Shutdown, OwnsHealth: true},
// The ONE observability subsystem: the in-repo o11y READ plane + runtime-handler
// install (o11y.SetHandler), with the hanzoai/o11y module wildcard /v1/o11y/*
// folded in as the TERMINAL sub-mount INSIDE o11y.MountO11y. Every specific
// /v1/o11y/* route registers before that wildcard, so Fiber's in-order match gives
// them precedence. NOT OwnsHealth: /v1/o11y/health stays the generic always-ok
// route (registered before MountAll), exactly as when the former module co-entry —
// which also set OwnsHealth=false — triggered it.
{Name: "o11y", Mount: o11y.MountO11y, Shutdown: o11y.ShutdownO11y},
{Name: "authz", Mount: authz.Mount},
// The ONE observability subsystem — and the first one that is NOT linked in.
// It runs as its own binary (cmd/o11y) and mounts at /v1/o11y over a private
// unix socket; the whole read plane, the runtime handler, the OTLP collector
// and the trace sink moved into that process untouched (it calls the same
// o11y.MountO11y). Nothing about the ROUTES changed: the plugin's own in-order
// registration still puts every specific /v1/o11y/* route ahead of the
// hanzoai/o11y module wildcard, and NOT OwnsHealth still leaves /v1/o11y/health
// the generic always-ok route Serve registers before MountAll — which therefore
// still wins over this mount's /v1/o11y/* and answers without waking the child.
//
// No Shutdown: teardown moved with the resources. zip.Load registers its own
// OnShutdown that stops the child, and the child flushes its collector/sink in
// its own app.OnShutdown. The host has nothing left of o11y's to close.
//
// Why this one first: o11y is the heaviest app in the graph — the
// otel-collector, prometheus and gonum are here and nowhere else — and it is
// imported by NOTHING but this line, so unlinking it is a pure subtraction.
//
// o11y owns TWO public prefixes — /v1/o11y and /v1/sentry/* (mountSentry,
// the Sentry-protocol ingest). Both are named here: a prefix left out
// would 404 silently rather than fail, which for Sentry ingest means
// quietly dropping every error event in the fleet.
cloud.PluginSpec("o11y", o11yPlugin(), "/v1/o11y", "/v1/sentry"),
{Name: "authz", Mount: cloud.Global(authz.Mount), Global: true},
// Embedded commerce plane /v1/commerce/*, /_/commerce/* — the hanzoai/commerce
// MODULE via the adapter in commerce.go (un-forked; the in-process
// CommerceClient is wired directly in pickCommerceClient).
{Name: "commerce", Mount: mountCommerce},
{Name: "licensing", Mount: licensing.Mount},
{Name: "commerce", Mount: cloud.Global(mountCommerce), Global: true},
{Name: "licensing", Mount: cloud.Global(licensing.Mount), Global: true},
// clients/plan.Mount. Enable id normalized "plans" -> "plan" to match the
// package + generated cmd/plan (one subsystem, one name). Its product routes
// stay /v1/plans/* (incl. the OwnsHealth /v1/plans/health probe) — unchanged.
@@ -261,8 +285,8 @@ func Wire() []cloud.MountSpec {
// (121) + the commerce embed (100). Same clients/account package as "account" (48).
{Name: "account-bridge", Mount: account.MountBridge},
{Name: "do", Mount: do.Mount},
{Name: "platform", Mount: platform.Mount, OwnsHealth: true},
{Name: "projects", Mount: projects.Mount},
{Name: "platform", Mount: platform.Mount, Shutdown: ctxShutdown(platform.Shutdown), OwnsHealth: true},
{Name: "projects", Mount: projects.Mount, Shutdown: ctxShutdown(projects.Shutdown)},
// The /v1/dns forward head: relays the console DNS dashboard to the DNS
// control plane under the caller's own validated bearer (clients/dns).
{Name: "dns", Mount: dns.Mount},
@@ -287,8 +311,24 @@ func Wire() []cloud.MountSpec {
{Name: "functions", Mount: functions.Mount},
{Name: "tracker", Mount: tracker.Mount},
{Name: "templates", Mount: templates.Mount},
// OSS-template compute-cost basis /v1/blueprint/* — parses a blueprint's
// docker-compose into its SBOM (bill of container images) and prices the
// stack's CPU/memory footprint through a documented rate card. The per-hour
// rate the deploy path meters an org on, and the "~$X/mo to run" the console
// shows; it is the compute cost the 20% author royalty (clients/authors) is
// taken from. OwnsHealth: serves its own /v1/blueprint/health. Reference
// content (embedded blueprints), no store → no Shutdown. After templates, its
// sibling catalog concern; before the AI /v1/* catch-all.
{Name: "blueprint", Mount: blueprint.Mount, OwnsHealth: true},
{Name: "framework", Mount: framework.Mount, Shutdown: ctxShutdown(framework.Shutdown)},
{Name: "knowledge", Mount: knowledge.Mount},
// Hanzo Support PUBLIC plane /v1/help/* (help center KB read + customer ticket
// intake) — the anonymous face the secure-by-default framework surface can't
// serve. A framework lane like knowledge: its DocType fixtures register via
// init() (help.Module), and this mounts the thin public subsystem. Owns no store
// (delegates to the framework in-process API), so no Shutdown. After framework
// (whose in-process API it calls at request time); before the AI /v1/* catch-all.
{Name: "help", Mount: help.Mount},
// Marketing content loop /v1/content/* (generate → CMS → transition → publish).
// After framework (its DocType store the ops read/write) + knowledge (the sibling
// framework lane); before the AI /v1/* catch-all so /v1/content/* resolves here.
@@ -300,8 +340,20 @@ func Wire() []cloud.MountSpec {
// After content (whose EnsureCatalogAsset it drives). Inert until CLOUD_COMMERCE_NATS_URL
// names the NATS carrying commerce catalog events — the reverse of the forward edge.
{Name: "catalogsync", Mount: catalogsync.Mount, Shutdown: catalogsync.Shutdown},
// The platform-global webhook layer: /v1/webhooks registry (per-org SQLite) +
// ONE durable JetStream consumer that delivers ANY bus event to org-registered
// HTTP subscribers. A bus consumer like catalogsync — placed adjacent to it and
// after the commerce embed whose COMMERCE stream it reads. Fail-soft: the
// registry serves even with the bus down; the consumer retries in the background.
// Owns per-org store handles + a worker pool → Shutdown drains both.
{Name: "webhooks", Mount: webhooks.Mount, Shutdown: webhooks.Shutdown},
{Name: "ml", Mount: ml.Mount, OwnsHealth: true},
{Name: "usage", Mount: usage.Mount},
// Gamified usage analytics: /v1/usage/leaderboard + /v1/usage/activity + the
// per-day contribution graph, over the datastore rollup (#43). Co-owns /v1/usage/*
// with usage (a distinct concern — who leads + your activity graph) at its own
// exact paths; owns the opt-in SQLite store. Before the ai /v1/* catch-all.
{Name: "leaderboard", Mount: leaderboard.Mount, Shutdown: leaderboard.Shutdown},
{Name: "crm", Mount: crm.Mount},
// Native /v1/marketing/* — the in-process fold of github.com/hanzoai/marketing
// (per-org campaign store on Base/SQLite), twin of crm. Owns a DB handle, so
@@ -311,6 +363,13 @@ func Wire() []cloud.MountSpec {
// twin of crm/marketing. Owns a DB handle, so its Shutdown closes it cleanly
// on SIGTERM (ctxShutdown adapts func() error).
{Name: "ads", Mount: ads.Mount, Shutdown: ctxShutdown(ads.Shutdown)},
// Top-level GTM orchestration /v1/campaign/* — the capability layer that fans a
// campaign VALUE out to its channels (paid→ads, organic→publish, email→marketing),
// each CONSUMING the connector plane via integrations.TokenFor. Metrics read from
// the ONE analytics plane (never a second store); creative A/B composes the
// experiment seam. The paid channel executor is wired in wire_seams.go. Owns a DB
// handle, so its Shutdown closes it cleanly on SIGTERM.
{Name: "campaign", Mount: campaign.Mount, Shutdown: ctxShutdown(campaign.Shutdown)},
// GDA/SDM validator onboarding /v1/validators/* — wallet-sig + ETH-mainnet
// GenesisNFT ownerOf → seal luxd staking identity into KMS → write a NEW-node
// LuxNetwork CR (node.lux.cloud, never the live luxd) → enqueue an owner-gated
@@ -324,33 +383,58 @@ func Wire() []cloud.MountSpec {
{Name: "analytics", Mount: analytics.Mount, OwnsHealth: true},
{Name: "git", Mount: git.Mount},
// Universal sync (/v1/sync/links + engine). Registers the cloud.SyncEngine the
// GitHub/Gitea webhooks enqueue to; git is its first provider. Owns per-org
// GitHub/Hanzo Git webhooks enqueue to; git is its first provider. Owns per-org
// DB handles, so its Shutdown closes them on SIGTERM.
{Name: "sync", Mount: sync.Mount, Shutdown: ctxShutdown(sync.Shutdown)},
{Name: "visor", Mount: visor.Mount},
// Connect-a-cloud-account plane /v1/cloud/*: an org links its DigitalOcean /
// AWS / GCP / Azure accounts (labeled, KMS-sealed, keyless where possible),
// Hanzo DISCOVERS each account's native Kubernetes clusters and FOLDS them into
// the ONE fleet (clients/fleet.Register) — so they surface in visor's
// /v1/clusters and run work like any BYO/managed cluster. No second registry.
{Name: "venue", Mount: venue.Mount},
// Cap table on Base via goja. STAGED behind CLOUD_ENABLE.
{Name: "captable", Mount: captable.Mount, Shutdown: captable.Shutdown},
{Name: "code", Mount: code.Mount, Shutdown: code.Shutdown},
{Name: "zero-trust", Mount: zt.Mount},
// ngrok-native public sharing: /v1/share/* provisions a per-org zrok
// account so `hanzo share <port>` publishes a local service to a public
// https://<token>.share.hanzo.ai URL. Fail-closed until ZROK_ADMIN_TOKEN.
{Name: "share", Mount: share.Mount},
// Data rooms via goja + per-tenant Base. STAGED behind CLOUD_ENABLE. OwnsHealth.
{Name: "dataroom", Mount: dataroom.Mount, Shutdown: dataroom.Shutdown, OwnsHealth: true},
{Name: "graph", Mount: graph.Mount},
{Name: "security", Mount: security.Mount, Shutdown: ctxShutdown(security.Shutdown), OwnsHealth: true},
{Name: "integrations", Mount: integrations.Mount, Shutdown: integrations.Shutdown},
// Per-org Cloudflare asset plane /v1/integrations/cloudflare/{pages,workers,r2,kv,d1}/*.
// Mounts AFTER integrations because it reads the org's Cloudflare token through
// the integrations custody seam (integrations.TokenFor) — one token, one
// custody boundary. Stateless: no store, no shutdown.
// Marketing destinations /v1/destinations/* — the native fan-out that
// TRANSLATES the canonical /v1/event stream to each connected ad/analytics
// platform (GA4 Measurement Protocol, Meta Conversions API, X/LinkedIn/TikTok/
// Reddit) and forwards it server-side. Mounts AFTER analytics (whose fan-out
// sink it installs) and integrations (a destination may reuse an OAuth
// connection's token via integrations.TokenFor). Owns a DB handle → Shutdown.
{Name: "destinations", Mount: destinations.Mount, Shutdown: ctxShutdown(destinations.Shutdown)},
// First-class per-org Cloudflare asset plane /v1/cloudflare/{zones,pages,workers,
// ai,r2,kv,d1}/* (sibling of /v1/dns, /v1/domain). Mounts AFTER integrations
// because it reads the org's Cloudflare token through the integrations custody
// seam (integrations.TokenFor) — one token, one custody boundary. Connecting the
// provider stays on the integrations plane; this plane only MANAGES resources.
{Name: "cloudflare", Mount: cloudflare.Mount},
{Name: "sbom", Mount: sbom.Mount, OwnsHealth: true},
{Name: "team", Mount: team.Mount, Shutdown: ctxShutdown(team.Shutdown)},
{Name: "settings", Mount: settings.Mount, Shutdown: settings.Shutdown},
{Name: "prefs", Mount: prefs.Mount, Shutdown: prefs.Shutdown},
{Name: "notify", Mount: notify.Mount, OwnsHealth: true},
{Name: "channels", Mount: channels.Mount, Shutdown: channels.Shutdown},
{Name: "gateway", Mount: gateway.Mount},
{Name: "entitlements", Mount: entitlements.Mount, Shutdown: entitlements.Shutdown},
{Name: "exec", Mount: exec.Mount},
{Name: "websearch", Mount: websearch.Mount},
// The in-binary full-text index (/v1/index): a per-org inverted index on
// Base/SQLite speaking the Meilisearch dialect, so a Meilisearch client
// repoints at it unchanged. websearch above queries the OUTSIDE world;
// this one indexes ours. NOT /v1/search — that path belongs to the
// hanzoai/ai RAG plane, and the collision silently ate two routes.
{Name: "index", Mount: index.Mount, Shutdown: ctxShutdown(index.Shutdown), OwnsHealth: true},
{Name: "world", Mount: world.Mount, Shutdown: ctxShutdown(world.Shutdown)},
// The bot runtime's ops face (/v1/bot/*). The transport itself is domain-free;
// the run control plane is "bots" below.
@@ -359,10 +443,25 @@ func Wire() []cloud.MountSpec {
{Name: "bots", Mount: bots.Mount},
{Name: "audit", Mount: auditlog.Mount},
{Name: "affiliates", Mount: affiliates.Mount},
// Hanzo Sign (e-signature) via goja + per-tenant Base. STAGED behind CLOUD_ENABLE. OwnsHealth.
{Name: "sign", Mount: sign.Mount, Shutdown: sign.Shutdown, OwnsHealth: true},
// Hanzo e-signature via goja + per-tenant Base. Mounts under the default. OwnsHealth.
{Name: "esign", Mount: esign.Mount, Shutdown: esign.Shutdown, OwnsHealth: true},
{Name: "product", Mount: product.Mount},
{Name: "evals", Mount: eval.Mount},
{Name: "benchmark", Mount: benchmark.Mount},
// The R&D EVIDENCE plane (HIP-0512) + its R&D Ops Board UI at /research —
// the arena's sibling: benchmark measures, research is the versioned diary
// every product's runs accrue into. Non-staged: mounts under the default.
{Name: "research", Mount: research.Mount, Shutdown: ctxShutdown(research.Shutdown)},
// The unified EXPERIMENT primitive (/v1/experiments): A/B testing as ONE value
// whatever the variant kind (feature | ad creative | email | model). It is a
// COMPOSITION — assignment via flags, measurement via analytics, evidence via
// research — so it mounts AFTER all three. It owns only the experiment registry
// store → Shutdown. clients/campaign composes it (experiments.Assign/Analyze).
{Name: "experiments", Mount: experiments.Mount, Shutdown: ctxShutdown(experiments.Shutdown)},
// The revenue BOOKS spine (/v1/books): a native double-entry general ledger that
// reads commerce transactions (the sole posting source) and books the accounting twin —
// plus bank import (PDF/OFX/CSV/Plaid/Teller), reconciliation, and the AI Ask brain.
{Name: "books", Mount: books.Mount, Shutdown: ctxShutdown(books.Shutdown)},
{Name: "treasury", Mount: treasury.Mount, Shutdown: ctxShutdown(treasury.Shutdown)},
{Name: "admin", Mount: admin.Mount},
// Launch-control gate (per-service waitlist): the COMPLETE feature — host→service
@@ -403,6 +502,14 @@ func Wire() []cloud.MountSpec {
// google token custody; captable/dataroom facades) and before the /v1/* AI
// catch-all so its routes resolve here.
{Name: "company", Mount: company.Mount, Shutdown: company.Shutdown},
// The corporate back-office surfaces — orthogonal to company/captable/billing:
// Hanzo Compliance (/v1/compliance) orchestrates KYC/KYB verification providers
// + tracks accreditation state + surfaces the SOC 2 audit posture; Hanzo Legal
// (/v1/legal) is the versioned template + generation engine + e-sign/filing seams.
// Both are TOOLING with providers/professionals in the loop, never advice or
// certification. They mount before the /v1/* AI catch-all so their routes resolve.
{Name: "compliance", Mount: compliance.Mount, Shutdown: ctxShutdown(compliance.Shutdown), OwnsHealth: true},
{Name: "legal", Mount: legal.Mount, Shutdown: ctxShutdown(legal.Shutdown), OwnsHealth: true},
// Chat orchestrator — POST /v1/chat: ONE LLM tool-calling round over the tool
// plane. It COMPOSES the ai completion path (in-process, so per-org billing
// runs) + the unified tool registry, and splits the model's tool calls into
@@ -410,7 +517,24 @@ func Wire() []cloud.MountSpec {
// catch-all so /v1/chat resolves here (Fiber first-match); the ai module's
// beego /v1/chat alias behind its /v1/* glob is thereby shadowed, while ai
// keeps /v1/chat/completions + /v1/completions.
{Name: "agent", Mount: agent.Mount},
{Name: "agent", Mount: cloud.Global(agent.Mount), Global: true},
// The UNIFIED GROUNDED ADVISOR — POST /v1/ask. DISTINCT from /v1/chat/completions
// (ai's RAW model) and /v1/agent (tool-calling): it routes a plain-language question
// to the domain(s) that can GROUND it, reads the REAL figures from each domain's own
// endpoint IN-PROCESS under the caller's own creds (agent.go's replay pattern, so
// per-tenant isolation is inherited), hands the model the EXACT figures, and returns
// the grounded answer + figures + the domain reads that backed them. The model NEVER
// invents a figure. Contributors plug in via a registry (books today; o11y/billing
// next) WITHOUT a router edit. Mounts BEFORE the ai /v1/* catch-all so /v1/ask wins
// Fiber's first-match; after books/agent so the domains it composes are wired.
{Name: "ask", Mount: ask.Mount},
// POST /v1/translate (HIP-0516) — the ONE translation surface: a quality tier
// on the model plane and a bulk tier on MADLAD-400, behind one endpoint, one
// auth path, one meter. It COMPOSES the model plane (deps.AI, which already
// gates + debits its own tokens) rather than standing up a second inference
// stack, and owns only its per-org translation memory → Shutdown. Mounts
// BEFORE the zen/ai catch-all so /v1/translate resolves here.
{Name: "translate", Mount: translate.Mount, Shutdown: ctxShutdown(translate.Shutdown)},
// The bare /v1/* AI catch-all — the LAST route position. Every owning subsystem above
// wins its own namespace (Fiber first-match); AI is the fallback for the rest of /v1/*.
// zen mounts as a /v1-scoped Claim middleware BEFORE ai: it routes zen* models
@@ -418,8 +542,8 @@ func Wire() []cloud.MountSpec {
// c.Next()s everything else to ai. zen owns the zen family; ai owns every
// other model and the /v1/models list. Order is load-bearing — Claim must
// run before ai's catch-all. (See hip-00NN.)
{Name: "zen", Mount: mountZen},
{Name: "ai", Mount: ai.Mount},
{Name: "zen", Mount: mountZen, Prefixes: []string{"/v1"}},
{Name: "ai", Mount: cloud.Global(mountAI), Global: true},
// Runtime wasm/proxy plugins — mounts dead last.
{Name: "plugins", Mount: plugin.Mount},
}
@@ -444,6 +568,32 @@ func ServeSingle(name string) error {
return fmt.Errorf("ServeSingle: unknown app %q — run `hanzo code ls`/`hanzo` for the list", name)
}
// o11yPlugin says where to find the o11y binary. Its two knobs map 1:1 onto
// zip.Plugin's own fields, so there is no third notion of "where a plugin is"
// and nothing to translate:
//
// CLOUD_O11Y_ADDR — already listening there; start nothing, just mount it.
// CLOUD_O11Y_BIN — the binary's path on disk.
//
// The default is a file named "o11y" beside the running cloud binary, which is
// the container layout: both binaries in the image, still one artifact to ship.
// Resolving it from os.Executable rather than $PATH means a host always loads
// the o11y it was built and shipped with, not whichever one a PATH happens to
// find.
func o11yPlugin() zip.Plugin {
if addr := strings.TrimSpace(os.Getenv("CLOUD_O11Y_ADDR")); addr != "" {
return zip.Plugin{Addr: addr}
}
path := strings.TrimSpace(os.Getenv("CLOUD_O11Y_BIN"))
if path == "" {
path = "o11y"
if self, err := os.Executable(); err == nil {
path = filepath.Join(filepath.Dir(self), "o11y")
}
}
return zip.Plugin{Path: path}
}
// mountMetrics adapts hanzoai/metrics into a cloud.MountFunc. Unlike the other
// externals, metrics declares its OWN narrow Deps (Logger, DataDir, Brand) and does
// not import hanzoai/cloud, so cloud.Typed cannot bridge it: the composition root
+161 -15
View File
@@ -25,12 +25,14 @@ import (
"github.com/hanzoai/cloud"
accountclient "github.com/hanzoai/cloud/clients/account"
"github.com/hanzoai/cloud/clients/commerceclient"
"github.com/hanzoai/cloud/clients/commerceinproc"
"github.com/hanzoai/cloud/clients/commerce"
"github.com/hanzoai/cloud/clients/commerce/transport"
financeclient "github.com/hanzoai/cloud/clients/finance"
"github.com/hanzoai/cloud/clients/principal"
"github.com/hanzoai/commerce"
commercemod "github.com/hanzoai/commerce"
commercebilling "github.com/hanzoai/commerce/api/billing"
catalogapi "github.com/hanzoai/commerce/api/catalog"
planapi "github.com/hanzoai/commerce/api/plan"
commercestore "github.com/hanzoai/commerce/api/store"
commercedatastore "github.com/hanzoai/commerce/datastore"
commercemid "github.com/hanzoai/commerce/middleware"
@@ -43,10 +45,10 @@ import (
func init() {
// In-process CommerceClient factory — pickCommerceClient calls it when the
// commerce subsystem is enabled. Registered HERE (not called directly from
// package cloud) because commerceclient's entitlement client imports
// package cloud) because the commerce client's entitlement client imports
// clients/plan, which imports cloud: the hook keeps the package graph acyclic.
cloud.RegisterCommerceClientFactory(func(cfg *cloud.Config, _ log.Logger) cloud.CommerceClient {
return commerceclient.InProcessClient(cfg.Brand)
return commerce.InProcessClient(cfg.Brand)
})
}
@@ -66,6 +68,21 @@ var commercePrefixes = []string{
// whose prepaid BALANCE gate 402'd every store read (a store-metadata read
// must never require an LLM balance).
"/v1/store",
// Platform-admin catalog CMS: GET/POST/PUT/DELETE /v1/catalog/entries +
// POST /v1/catalog/seed — the SuperAdmin CRUD admin.hanzo.ai's editor drives.
// commerce's setupRoutes wires only the PUBLIC read (/v1/commerce/catalog);
// the CRUD lives on the standalone /v1 bundle (api.Route → catalogApi.AdminRoute),
// which the co-resident embed skips, so mountCommerce mounts it below on the
// same /v1 gate chain. Own the prefix here so it reaches commerce (each handler
// is requireSuperAdmin-gated) instead of the AI /v1/* balance catch-all.
"/v1/catalog",
// Platform-admin subscription/DNS plan authority CMS: GET/POST/PUT/DELETE
// /v1/plans/entries + POST /v1/plans/seed (increment 3a) — the SuperAdmin CRUD
// the console plan editor drives. The PUBLIC read stays GET /v1/billing/plans;
// this CRUD rides the /v1 bundle (api.Route → planApi.AdminRoute), which the
// embed skips, so mountCommerce mounts it below. Own the prefix so it reaches
// commerce (each handler requireSuperAdmin-gated), not the AI /v1/* 402 gate.
"/v1/plans",
// Payment-provider webhook receiver (POST /v1/billing/webhooks/:provider —
// Square et al). The provider's HMAC over the registered notification URL +
// body IS the auth; a bearer gate is impossible for provider callbacks.
@@ -86,7 +103,7 @@ var commercePrefixes = []string{
// own gate chains (see commercePrefixes).
func mountCommerce(app *zip.App, deps cloud.Deps) error {
if app == nil {
return fmt.Errorf("commerce: nil zip.App")
return fmt.Errorf("commerce: nil app")
}
if deps.Logger == nil {
return fmt.Errorf("commerce: nil deps.Logger")
@@ -114,7 +131,7 @@ func mountCommerce(app *zip.App, deps cloud.Deps) error {
dataDir = filepath.Join(deps.DataDir, "commerce")
}
embedded, err := commerce.Embed(context.Background(), commerce.EmbedConfig{
embedded, err := commercemod.Embed(context.Background(), commercemod.EmbedConfig{
DataDir: dataDir,
// RequireIdentity stays gateway-owned: the gateway in front of the cloud
// binary is the trust boundary per HIP-0026.
@@ -124,11 +141,27 @@ func mountCommerce(app *zip.App, deps cloud.Deps) error {
App: app,
// ONE LEDGER: commerce's POST /v1/billing/credit mints into cloud's native
// finance ledger (the SAME per-org account the AI spend-gate reads), so a
// granted credit is immediately spendable. commerce.Embed calls
// granted credit is immediately spendable. commercemod.Embed calls
// creditledger.Set(this) before routes register; nil would leave commerce on
// its own datastore (standalone), but in this unified binary finance is
// co-resident, so we inject the finance-backed ledger adapter.
Ledger: ledger{},
// ONE KEY for one process. commerce encrypts its per-tenant money stores
// under a 32-byte KEK; cloud already resolves one (CLOUD_KMS_MASTER_KEY_REF,
// the same master durableCipher and cek derive from), so it hands that over
// rather than a SECOND key being provisioned for the same process through
// commerce's own env var.
//
// It is not a convenience. Without a key commerce refuses to boot on a
// libsqlcipher-linked build — correctly, it will not open money data
// unencrypted — and that refusal is silent from out here: Mount never
// reaches transport.SetApp, so every S2S billing read falls through to the
// network and DNS-resolves the in-process placeholder. On 2026-07-27 that
// read as "Insufficient balance" on funded accounts, fleet-wide.
//
// Empty ref ⇒ nil ⇒ commerce reads its own env, unchanged, which is the
// standalone and pure-Go dev path.
MasterKey: deps.MasterKey,
})
if err != nil {
lg.Error("commerce embed failed — serving fail-closed 503 (cloud stays up)", "err", err)
@@ -146,6 +179,26 @@ func mountCommerce(app *zip.App, deps cloud.Deps) error {
storeV1.Use(iammiddleware.IAMTokenRequired())
commercestore.Route(storeV1, commercemid.TokenRequired())
// Platform-admin catalog CMS on the SAME /v1 bundle: GET/POST/PUT/DELETE
// /v1/catalog/entries + POST /v1/catalog/seed. setupRoutes wires only the
// public read (/v1/commerce/catalog); the CRUD rides the standalone /v1 bundle
// (api.Route → catalogApi.AdminRoute), which the co-resident embed skips — so
// register it here, exactly as the standalone does. storeV1's IAMTokenRequired
// populates the claims each handler's requireSuperAdmin reads (anon → 403, a
// platform admin edits); it is cross-tenant data, never org-scoped.
catalogapi.AdminRoute(storeV1)
// Platform-admin subscription/DNS plan authority CRUD on the SAME /v1 bundle:
// GET/POST/PUT/DELETE /v1/plans/entries + POST /v1/plans/seed (increment 3a).
// Mirrors the catalog mount: the standalone wires it on the /v1 bundle
// (api.Route → planApi.AdminRoute), which the co-resident embed skips. The
// embed seed SOURCE is injected here (the composition root) — commercebilling.
// SeedRows, the SAME @hanzo/plans embed the boot seed + resolveSubscriptionPlan
// read — so api/plan never imports api/billing. Each handler is
// requireSuperAdmin-gated (anon → 403); prices are admin-editable but the mint
// gates score the IMMUTABLE embed, so an edit never moves a charge gate.
planapi.AdminRoute(storeV1, commercebilling.SeedRows)
// Provider webhook intake at the LIVE registered path. Chain mirrors the
// commerce-standalone posture: gated request context, then the sessionless
// HMAC-verified handler.
@@ -219,8 +272,8 @@ func mountCommerce(app *zip.App, deps cloud.Deps) error {
// the console billingRead block above: without a co-resident handler this authorize
// fell through to the account bridge's /v1/billing/* wildcard (order 122), which — being
// service-token-forwardable (billing.go billingForwardable) — re-forwarded it to
// COMMERCE_URL (the public api.hanzo.ai edge = THIS binary) over commerceinproc's
// self-routing transport, re-entering the same wildcard until the depth-8 guard refused
// COMMERCE_URL (the public api.hanzo.ai edge = THIS binary) over the commerce transport's
// self-routing dispatch, re-entering the same wildcard until the depth-8 guard refused
// → 502 → the gate fails OPEN (the cap is a policy overlay, so no traffic was blocked,
// but ~135 502s/30m spammed the money path and each burned 8 full-app dispatches). The
// plain GET /v1/billing/spend-alerts (registered above) already broke this loop for the
@@ -277,13 +330,106 @@ func mountCommerce(app *zip.App, deps cloud.Deps) error {
commercebilling.DeleteSpendAlert,
)
// POST /v1/billing/topup/token — the INLINE Square card top-up (the console's
// "Billing → Credits → add credits": the Square Web Payments SDK tokenizes the card
// IN THE BROWSER → a single-use nonce → this endpoint charges it and credits the
// caller's balance). commerce's api.Route() billing bundle is NOT compiled into the
// co-resident embed, so — exactly like plans/invoices/spend-alerts above — without
// this registration the POST fell through to the account bridge's /v1/billing/*
// wildcard (order 122). That wildcard is service-token-forwardable for topup/token
// (billing.go billingForwardable), so billingData re-forwarded it to COMMERCE_URL
// (default the public api.hanzo.ai edge = THIS binary) over the commerce transport's
// self-routing dispatch, re-entering the SAME wildcard until the depth-8 guard
// refused → the "commerce transport: in-process dispatch depth 8 exceeded" 502 that broke
// top-up outright. Registering commerce's real TopupWithToken co-resident here
// (order 100 < 122) shadows the wildcard and serves the charge in-process at depth 1
// — no HTTP hop, no self-dispatch. topup/token STAYS in billingForwardable as the
// split-deploy fallback (a standalone commerce still serves it); co-residence just
// wins first.
//
// Chain — the browser money-WRITE posture, byte-for-byte what the bridge applied:
// RequireCSRF — the ambient-cookie anti-CSRF gate the bridge's requireCSRF
// wrapped POST /v1/billing/* with (a Bearer/gateway caller is
// not CSRF-able; an ambient-cookie write needs the token).
// RequestContext — the gated request context commerce's handler + ledger read.
// IAMTokenRequired — resolves the org from the gateway-validated X-Org-Id into
// Locals("organization"), which TopupWithToken.GetOrganization
// + topupDestination read as the org billing key.
// PinBillingSubject — pins ?user= to the caller's OWN account.Payer subject (the
// SAME rule the ai spend-gate debits and billingData pins), so
// the credit lands on the caller's subject (person=org/name) and
// can never be widened; fail-closed for an unvalidated caller —
// the IDOR boundary stays exactly where billingData put it.
// The card PAN never touches this binary: TopupWithToken charges the Square nonce only,
// and the settled charge itself is the mint authority (mintauth.WithAuthorized).
app.Post("/v1/billing/topup/token",
accountclient.RequireCSRF(),
commercemid.RequestContext(),
iammiddleware.IAMTokenRequired(),
accountclient.PinBillingSubject(),
commercebilling.TopupWithToken,
)
// The remaining console billing WRITES that share topup/token's self-dispatch loop
// class — each is a POST the console makes (billingForwardable in billing.go), each had
// NO co-resident handler, so each fell through to the account bridge's /v1/billing/*
// wildcard (order 122) and re-entered it over the commerce transport until the depth-8 guard
// refused (the same "in-process dispatch depth 8 exceeded" 502 that broke top-up). Each
// commerce handler exists in the vendored module (v1.49.13); registering them co-resident
// (order 100 < 122) shadows the wildcard and serves the write in-process at depth 1. They
// STAY in billingForwardable as the split-deploy fallback (same precedent as topup/token
// + spend-alerts). Chain matches the bridge's write posture byte-for-byte:
//
// - RequireCSRF — the ambient-cookie anti-CSRF gate the bridge wrapped POST
// /v1/billing/* with (Bearer/gateway callers are not CSRF-able).
// - RequestContext — the gated request context commerce's handlers read.
// - IAMTokenRequired — resolves the org from the gateway-validated X-Org-Id into
// Locals("organization") — the namespace GetOrganization reads.
// - PinBillingSubject — pins the caller's OWN account.Payer subject into BOTH query and
// body AND fail-closes an unvalidated caller. It is the auth gate
// on every one, and the IDOR control on the subject-scoped one.
//
// payment-methods (save a card-on-file / vault a Square nonce) is SUBJECT-scoped: commerce's
// CreatePaymentMethod reads `customerId` from the BODY, so PinBillingSubject's body-pin is
// load-bearing here — a member can only vault a card for their OWN subject, exactly the
// boundary billingData's scopedBillingBody enforced. The Square nonce goes to Square; the
// PAN never touches this binary.
app.Post("/v1/billing/payment-methods",
accountclient.RequireCSRF(),
commercemid.RequestContext(),
iammiddleware.IAMTokenRequired(),
accountclient.PinBillingSubject(),
commercebilling.CreatePaymentMethod,
)
// subscriptions/:id/{cancel,reactivate} are org-NAMESPACE-scoped: commerce's handlers
// resolve the subscription by `:id` WITHIN the caller's org namespace (a foreign org's id
// is a 404 miss), so tenancy is the namespace IAMTokenRequired resolves and PinBillingSubject
// is the fail-closed-anon auth gate — its pinned subject params are ignored by these
// handlers (the SAME role it plays for the org-scoped payment-config read). The bridge's
// subject-pin was likewise a no-op for these, so nothing is dropped.
app.Post("/v1/billing/subscriptions/:id/cancel",
accountclient.RequireCSRF(),
commercemid.RequestContext(),
iammiddleware.IAMTokenRequired(),
accountclient.PinBillingSubject(),
commercebilling.CancelBillingSubscription,
)
app.Post("/v1/billing/subscriptions/:id/reactivate",
accountclient.RequireCSRF(),
commercemid.RequestContext(),
iammiddleware.IAMTokenRequired(),
accountclient.PinBillingSubject(),
commercebilling.ReactivateBillingSubscription,
)
// In-process seams:
// - commerceinproc routes the S2S billing byte-stream into the co-resident
// - the commerce transport routes the S2S billing byte-stream into the co-resident
// app (the metering debit path) instead of a socket to a standalone pod.
// - commerceclient reads the Embedded's datastore DIRECTLY (entitlements +
// - the commerce client reads the Embedded's datastore DIRECTLY (entitlements +
// BalanceCents) — no HTTP shape at all.
commerceinproc.SetApp(app)
commerceclient.PublishEmbedded(embedded)
transport.SetApp(app.Fiber())
commerce.PublishEmbedded(embedded)
// Usage-cap enforcement on the FINANCE path. The unified binary records usage in
// the finance ledger (fin.RecordUsage), NOT commerce's transaction store — which
@@ -340,7 +486,7 @@ func hasCommercePrefix(path string) bool {
}
// of falling through to another subsystem's catch-all.
func mountCommerceFailClosed(app *zip.App) {
func mountCommerceFailClosed(app cloud.Router) {
failed := func(c *zip.Ctx) error {
c.SetHeader("Content-Type", "application/json")
return c.Bytes(http.StatusServiceUnavailable, []byte(`{"error":"commerce unavailable","code":503}`))
+21
View File
@@ -32,6 +32,8 @@ func TestCommercePrefixesPinned(t *testing.T) {
"/v1/billing/auto-recharge": false,
"/v1/billing/webhooks": false,
"/v1/store": false,
"/v1/catalog": false,
"/v1/plans": false,
}
for _, p := range commercePrefixes {
if _, ok := want[p]; ok {
@@ -84,6 +86,25 @@ func TestStoreSurfaceRoutedToCommerceNotAIGate(t *testing.T) {
if code, _ := doReq(t, app, http.MethodPut, "/v1/store/karma-store/listing/valentina"); code == http.StatusPaymentRequired {
t.Fatalf("PUT /v1/store/:id/listing/:slug fell through to the AI balance gate (402) — the whole store surface must be commerce-owned")
}
// The platform-admin catalog CMS (admin.hanzo.ai's editor) must reach commerce —
// where each handler is requireSuperAdmin-gated (anon → 401/403) — never the AI
// /v1/* balance gate, which would 402 the editor's list/edit instead.
if code, _ := doReq(t, app, http.MethodGet, "/v1/catalog/entries"); code == http.StatusPaymentRequired {
t.Fatalf("GET /v1/catalog/entries fell through to the AI balance gate (402) — /v1/catalog must be a commercePrefix so the SuperAdmin CMS reaches commerce")
}
if code, _ := doReq(t, app, http.MethodPut, "/v1/catalog/entries/cloud-starter"); code == http.StatusPaymentRequired {
t.Fatalf("PUT /v1/catalog/entries/:slug fell through to the AI balance gate (402) — the whole catalog CMS must be commerce-owned")
}
// The platform-admin plan authority CMS (increment 3a) must also reach commerce —
// requireSuperAdmin-gated (anon → 401/403) — never the AI /v1/* 402 gate.
if code, _ := doReq(t, app, http.MethodGet, "/v1/plans/entries"); code == http.StatusPaymentRequired {
t.Fatalf("GET /v1/plans/entries fell through to the AI balance gate (402) — /v1/plans must be a commercePrefix so the plan authority CMS reaches commerce")
}
if code, _ := doReq(t, app, http.MethodPut, "/v1/plans/entries/pro"); code == http.StatusPaymentRequired {
t.Fatalf("PUT /v1/plans/entries/:slug fell through to the AI balance gate (402) — the whole plan authority CMS must be commerce-owned")
}
}
// doReq drives one request through the mounted app and returns (status, body).
+9 -7
View File
@@ -7,13 +7,15 @@ import (
)
// TestFrameworkContentModulesLinked guards the framework CONTENT modules
// (cms/erp/help) against silent removal. They are not mount subsystems, so they
// never appear in Wire(); they register their DocTypes and — for erp — the
// ledger-posting lifecycle hooks into the framework engine from a package
// init(), reached ONLY via the blank imports in apps.go. #248 dropped
// those imports, which stripped the erp ledger hooks from the binary with no
// mount change and no failing mount test. This asserts the engine's module
// registry carries each lane, so that money-adjacent regression cannot recur.
// (cms/erp/help) against silent removal. cms/erp are pure fixture lanes reached
// ONLY via blank imports in apps.go; help is a real import + a Wire() spec (it also
// mounts the /v1/help public plane) but still contributes its DocTypes the SAME way,
// from a package init() (framework.RegisterModule). #248 dropped the blank imports,
// which stripped the erp ledger hooks from the binary with no mount change and no
// failing mount test. This asserts the engine's module registry carries each lane,
// so that money-adjacent regression cannot recur — for help, dropping the import
// would also drop its Wire() spec and fail TestWireOrderMatchesFrozen, but this keeps
// the fixture-linkage guard uniform across all three lanes.
func TestFrameworkContentModulesLinked(t *testing.T) {
got := make(map[string]bool)
for _, m := range framework.RegisteredModules() {
+44
View File
@@ -0,0 +1,44 @@
package apps
import (
"context"
"github.com/hanzoai/ai"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/membership"
"github.com/zap-proto/zip"
)
// cloud builds these callbacks but does not install them: importing the ai
// module or the Kubernetes client from cloud drags ~1700 packages into every
// subsystem, since they all import cloud for Deps. Registration lives here,
// where both are already linked.
// Outside a cluster K8s errors and cloud falls back to its static peer set.
func init() { cloud.Peers = membership.K8s }
// mountAI installs the money and ingest callbacks, then mounts ai. A nil
// callback is left alone — cloud leaves one nil exactly when that subsystem
// isn't co-resident, and the module's own fallback applies.
func mountAI(app *zip.App, deps cloud.Deps) error {
if f := cloud.TierReader(); f != nil {
aiobject.SetTierReader(aiobject.TierReaderFunc(f))
}
if f := cloud.BalanceReader(); f != nil {
aiobject.SetBalanceReader(aiobject.BalanceReaderFunc(f))
}
if f := cloud.UsageRecorder(); f != nil {
aiobject.SetUsageRecorder(func(ctx context.Context, u aiobject.UsageEvent) error {
return f(ctx, cloud.UsageEvent{
Subject: u.Subject, Namespace: u.Namespace, USD: u.USD,
Currency: u.Currency, Model: u.Model, Provider: u.Provider,
RequestID: u.RequestID,
})
})
}
if d := cloud.IngestDialer(); d != nil {
aiobject.SetIngestDialer(d)
}
return ai.Mount(app, deps)
}
+20 -6
View File
@@ -27,10 +27,18 @@ type ledger struct{}
// compile-time proof the adapter satisfies commerce's exported seam.
var _ creditledger.CreditLedger = ledger{}
// Credit posts a balanced deposit (funding:platform → wallet) to the org's POOL
// account (Subject == Org, the wallet the gate reads) and returns the ledger entry
// id + the org's new available balance in cents. Idempotent on IdempotencyKey:
// finance dedups on Ref, so the same key credits AT MOST once.
// Credit posts a balanced deposit (funding:platform → wallet) to the ADDRESS the
// input names — (Org, Subject), where an empty Subject is the org POOL — and returns
// the ledger entry id + that account's new available balance in cents. Idempotent on
// IdempotencyKey: finance dedups on Ref, so the same key credits AT MOST once.
//
// Subject exists because the pool is not always the account the gate reads. In the
// shared signup org each member spends from their own wallet, so a pool-only credit
// funds a balance nobody can spend while the member it was meant for is refused at
// $0. The caller resolves the address with the SAME rule the gate resolves the payer
// with (account.Payer, via principal), so a credit and the spend it funds land on one
// wallet by construction. An org whose members share one balance names no subject and
// is byte-for-byte unchanged.
func (ledger) Credit(ctx context.Context, in creditledger.CreditInput) (string, int64, error) {
fin := finance.Current()
if fin == nil {
@@ -44,9 +52,13 @@ func (ledger) Credit(ctx context.Context, in creditledger.CreditInput) (string,
if tag == "" {
tag = "grant:admin" // non-cash grant bucket (finance is a single wallet; Tags is a memo)
}
subject := in.Subject
if subject == "" {
subject = in.Org // pooled org: the slug IS the pool account the gate reads
}
id, err := fin.Deposit(ctx, types.DepositInput{
Org: in.Org,
Subject: in.Org, // org-pool wallet == the account the AI gate reads
Subject: subject,
Amount: money.FromCents(in.AmountCents),
Currency: cur,
Notes: in.Reason,
@@ -56,7 +68,9 @@ func (ledger) Credit(ctx context.Context, in creditledger.CreditInput) (string,
if err != nil {
return "", 0, err
}
bal, berr := fin.Balance(ctx, in.Org, in.Org, cur, false)
// Read back the account that was CREDITED, never the pool — reporting a pool
// balance after crediting a member is how a caller concludes the grant vanished.
bal, berr := fin.Balance(ctx, in.Org, subject, cur, false)
if berr != nil {
return id, 0, berr
}
+24
View File
@@ -0,0 +1,24 @@
// Copyright © 2026 Hanzo AI. MIT License.
package apps
import (
"os"
"testing"
sqlitedrv "github.com/hanzoai/sqlite"
)
// TestMain makes this package's store-backed tests (the finance-ledger money proofs in
// starter_test.go) build-tag agnostic, exactly as the root package's TestMain does. On
// an encryption-capable (cgo) build cek REFUSES to open a store without a master key;
// on a pure-Go build a key is itself refused. So supply a throwaway dev key ONLY when
// the build can encrypt AND the environment did not already provide one (CI may inject
// the real key) — never overriding a provided key. Resolved once per process,
// order-independent.
func TestMain(m *testing.M) {
if sqlitedrv.EncryptionAvailable() && os.Getenv("CLOUD_KMS_MASTER_KEY_REF") == "" {
_ = os.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=") // 32 zero bytes, dev-only
}
os.Exit(m.Run())
}
+134
View File
@@ -0,0 +1,134 @@
// One SQLite engine, reachable under the names that engine owns.
//
// database/sql is a global registry: two packages registering the same name
// panic the process at init ("sql: Register called twice for driver sqlite"),
// before main() runs and before a single request is served. That panic has
// already taken this binary down once — fourteen stores blank-imported
// modernc.org/sqlite while hanzoai/sqlite registered "sqlite" too (dba5d73b).
// The names collide in pairs: hanzoai/sqlite and modernc.org/sqlite both take
// "sqlite"; hanzoai/csqlite and mattn/go-sqlite3 both take "sqlite3". One blank
// import re-arms it.
//
// The guard lives here because apps is the composition root: cmd/cloud's main
// imports nothing but this package and the root cloud package, so this test
// binary's first-party graph is the shipped binary's less that one main.
// Whatever registers a driver in production registers here, and the gate
// already runs ./apps/ — so a second engine fails CI instead of a pod.
//
// The observable is the registered name -> owning package map, read back through
// reflection on the driver database/sql actually holds. It is closed by
// construction: an engine is caught whether it collides on a name, takes a fresh
// name, or hides behind a fork of a name already allowed.
//
// HIP-0106: one binary, many subsystems.
package apps
import (
"database/sql"
"reflect"
"strings"
"testing"
)
// sqliteEngines are the packages allowed to own a registered SQLite driver — one
// per build tag, both reached through the github.com/hanzoai/sqlite facade that
// every cloud store imports. Nothing else may appear.
var sqliteEngines = map[string]string{
"github.com/hanzoai/csqlite": "cgo: the C engine behind hanzoai/sqlite, linked against libsqlcipher for at-rest encryption",
// cgo-free: hanzoai/sqlite has no engine of its own on this path — driver_nocgo.go
// blank-imports modernc, which registers "sqlite" in its own init, so modernc IS
// the fork's pure-Go backend and owns the name. It is reached ONLY through the
// fork (nothing here imports it directly), and at-rest encryption is not lost by
// its being unkeyed: cek wraps it in the pure-Go SQLCipher envelope, which reads
// and writes the same page format as the C codec.
"modernc.org/sqlite": "cgo-free: the engine behind hanzoai/sqlite's !cgo backend, encrypted at rest by cek's SQLCipher envelope",
}
// sqliteNames are the driver names that engine may answer to. "sqlite" is
// required; "sqlite3" is optional because only the cgo backend registers it.
var sqliteNames = map[string]string{
"sqlite": "registered by hanzoai/sqlite under both build tags; the name every cloud store opens",
"sqlite3": "registered by hanzoai/csqlite's own init, same engine as \"sqlite\"; also the name mattn/go-sqlite3 takes",
}
// sqliteRegistrations maps every SQLite-family driver name registered in this
// binary to the package that owns it. A name is in the family if the name looks
// like SQLite or its owning package does, so an engine cannot slip past by
// picking a plain name.
//
// Owner resolution goes through sql.Open, which is lazy — it hands back the
// registered driver without connecting. A driver that refuses an empty DSN and is
// not SQLite-shaped by name is logged and skipped: it cannot be identified, and
// no SQLite engine in this tree behaves that way.
func sqliteRegistrations(t *testing.T) map[string]string {
t.Helper()
family := func(s string) bool {
s = strings.ToLower(s)
return strings.Contains(s, "sqlite") || strings.Contains(s, "sqlcipher")
}
reg := map[string]string{}
for _, name := range sql.Drivers() {
db, err := sql.Open(name, "")
if err != nil {
if family(name) {
t.Errorf("driver %q is registered but its owner cannot be read: %v", name, err)
} else {
t.Logf("driver %q: owner not readable (%v); not SQLite-shaped by name", name, err)
}
continue
}
rt := reflect.TypeOf(db.Driver())
_ = db.Close()
for rt.Kind() == reflect.Pointer {
rt = rt.Elem()
}
if pkg := rt.PkgPath(); family(name) || family(pkg) {
reg[name] = pkg
}
}
return reg
}
// TestSQLiteOneEngine fails if the binary links a second SQLite engine.
func TestSQLiteOneEngine(t *testing.T) {
reg := sqliteRegistrations(t)
if len(reg) == 0 {
t.Fatal(`no SQLite driver registered: every cloud store opens "sqlite", so the composition root must carry github.com/hanzoai/sqlite`)
}
owners := map[string][]string{}
for name, pkg := range reg {
owners[pkg] = append(owners[pkg], name)
}
if len(owners) > 1 {
t.Errorf("cloud links %d SQLite engines, not one: %v.\n"+
"Two engines mean two on-disk behaviours, and the moment either takes a name the other holds, this binary panics at init.",
len(owners), owners)
}
for pkg, held := range owners {
if _, ok := sqliteEngines[pkg]; !ok {
t.Errorf("%s owns SQLite driver name(s) %v but is not an allowed engine.\n"+
"Open \"sqlite\" through github.com/hanzoai/sqlite instead, or add %s to sqliteEngines here with the reason it must be its own engine.",
pkg, held, pkg)
}
}
}
// TestSQLiteDriverNames fails if the engine answers to a name outside the allowed
// set, which is how a collision with an upstream driver gets built.
func TestSQLiteDriverNames(t *testing.T) {
reg := sqliteRegistrations(t)
if _, ok := reg["sqlite"]; !ok {
t.Errorf("driver \"sqlite\" is not registered; registered SQLite names: %v", reg)
}
for name := range reg {
if _, ok := sqliteNames[name]; !ok {
t.Errorf("SQLite driver name %q is registered but not allowed.\n"+
"Every extra name is a name an upstream driver can no longer take without panicking this binary.\n"+
"Drop the registration, or add %q to sqliteNames here with the reason it must exist.",
name, name)
}
}
}
+291
View File
@@ -0,0 +1,291 @@
// Copyright © 2026 Hanzo AI. MIT License.
package apps
// starter_test.go proves the one claim a starter grant lives or dies on: the money
// lands at the address the SPEND GATE READS. A grant that credits a wallet the gate
// does not read is worse than no grant — the account looks funded and still 402s, and
// that exact bug has shipped twice before (clients/principal/wallet.go names both).
//
// So these tests never assert against the balance the grant itself reports. They
// re-derive the gate's address the way the gate does — account.Payer(...).Subject(),
// the ONE rule — and read THAT. If the two ever diverge, the read returns 0 and every
// test here fails.
//
// It lives in package apps because this is where the real chain is assembled: the
// cloud ledger adapter (ledger{}) that translates (Org, Subject) into a finance
// posting. Testing cloud.EnsureStarterCredit against a stub ledger would prove only
// that the stub agrees with itself. The middleware's own behaviour — the hot-path
// cache, which principals are eligible — is proven in middleware_starter_test.go,
// where a request context is cheap to build.
import (
"context"
"sync"
"testing"
"github.com/hanzoai/account"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/finance"
"github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/cloud/clients/principal"
"github.com/hanzoai/cloud/types"
commercecredit "github.com/hanzoai/commerce/billing/credit"
"github.com/hanzoai/commerce/billing/creditledger"
)
// wireLedger assembles the production money chain over a temp data dir: a real
// finance ledger, published as the process-wide client, with the real cloud adapter
// injected as commerce's credit seam. This is the same pair build.go/apps wire at
// boot — no fakes anywhere in the path under test.
func wireLedger(t *testing.T) finance.Client {
t.Helper()
fin := finance.New(t.TempDir())
finance.Publish(fin)
creditledger.Set(ledger{})
t.Cleanup(func() {
creditledger.Set(nil)
finance.Publish(nil)
})
return fin
}
// starterWalletOf builds the address a credential resolves to, by the SAME rule the gate
// uses. Owner is the home org (the ledger), name the person. No `billing_account`
// claim is supplied because IAM v2 mints none, so Payer takes the fallback — which is
// precisely the production shape.
func starterWalletOf(owner, name string) principal.Wallet {
return principal.Wallet{
Ledger: owner,
Account: account.Payer(account.Credential{Owner: owner, Name: name}).Subject(),
}
}
// gateBalance reads a principal's balance EXACTLY as the spend gate does: resolve the
// payer with account.Payer (what principal.WalletOf calls), then read that subject in
// the home org's ledger (what build.go's BalanceReader and metering's fetchAvailable
// both call).
func gateBalance(t *testing.T, fin finance.Client, owner, name string) int64 {
t.Helper()
w := starterWalletOf(owner, name)
bal, err := fin.Balance(context.Background(), w.Ledger, w.Account, "usd", false)
if err != nil {
t.Fatalf("gate balance read (owner=%q name=%q subject=%q): %v", owner, name, w.Account, err)
}
return bal.Cents()
}
// TestStarterCredit_LandsAtTheAddressTheGateReads is THE test. A new account is
// granted, and the balance is then read through the gate's own address rule — not the
// grant's return value. They must agree, or the grant funds a wallet nobody spends
// from.
func TestStarterCredit_LandsAtTheAddressTheGateReads(t *testing.T) {
fin := wireLedger(t)
if got := gateBalance(t, fin, "acme", "alice"); got != 0 {
t.Fatalf("a brand-new org must start at zero, got %d cents", got)
}
reported, err := cloud.EnsureStarterCredit(context.Background(), starterWalletOf("acme", "alice"))
if err != nil {
t.Fatalf("EnsureStarterCredit: %v", err)
}
if got := gateBalance(t, fin, "acme", "alice"); got != commercecredit.StarterCreditCents {
t.Fatalf("the GATE reads %d cents at acme, want %d — the grant landed at an address the gate does not read",
got, commercecredit.StarterCreditCents)
}
// A second member of the same org reads the SAME pool: one grant funds the tenant,
// not each employee. This is what makes "once per account" mean once per org.
if got := gateBalance(t, fin, "acme", "bob"); got != commercecredit.StarterCreditCents {
t.Fatalf("a second member reads %d cents, want the same pool %d", got, commercecredit.StarterCreditCents)
}
if reported != commercecredit.StarterCreditCents {
t.Fatalf("reported balance %d, want %d", reported, commercecredit.StarterCreditCents)
}
}
// TestStarterCredit_AmountIsServerAuthoritative pins the granted amount to the shared
// constant. EnsureStarterCredit takes no amount and no request body, so no client
// field can reach it — this catches drift between the constant and what lands.
func TestStarterCredit_AmountIsServerAuthoritative(t *testing.T) {
fin := wireLedger(t)
if _, err := cloud.EnsureStarterCredit(context.Background(), starterWalletOf("acme", "alice")); err != nil {
t.Fatalf("EnsureStarterCredit: %v", err)
}
if got := gateBalance(t, fin, "acme", "alice"); got != 500 {
t.Fatalf("granted %d cents, want exactly 500 ($5.00, the one canonical amount)", got)
}
}
// TestStarterCredit_RetryGrantsOnce covers the sequential replays: a retried request,
// a re-login, a process restart that empties the hot-path cache. Every one derives the
// same address-keyed ref, so the ledger credits once.
func TestStarterCredit_RetryGrantsOnce(t *testing.T) {
fin := wireLedger(t)
for i := 0; i < 5; i++ {
if _, err := cloud.EnsureStarterCredit(context.Background(), starterWalletOf("acme", "alice")); err != nil {
t.Fatalf("EnsureStarterCredit call %d: %v", i, err)
}
}
if got := gateBalance(t, fin, "acme", "alice"); got != commercecredit.StarterCreditCents {
t.Fatalf("after 5 grants the gate reads %d cents, want %d — the grant stacked",
got, commercecredit.StarterCreditCents)
}
}
// TestStarterCredit_ConcurrentGrantsOnce is the one that matters for money. A
// sequential retry test passes against a read-then-write guard that is still racy;
// only concurrency distinguishes a real idempotency barrier from a checked one.
// Twenty goroutines start together and grant the same wallet.
//
// The barrier under test is finance.Deposit's dedup on Ref, which does its
// EntryByRef check INSIDE the same transaction as the insert, over a store pinned to
// SetMaxOpenConns(1). If that ever loosens — a second connection, a deferred
// transaction, a check moved out of the tx — this test turns the regression into a
// balance that is a multiple of $5.
func TestStarterCredit_ConcurrentGrantsOnce(t *testing.T) {
fin := wireLedger(t)
const racers = 20
var start sync.WaitGroup
var done sync.WaitGroup
start.Add(1)
errs := make([]error, racers)
for i := 0; i < racers; i++ {
done.Add(1)
go func(i int) {
defer done.Done()
start.Wait() // release all goroutines at once
_, errs[i] = cloud.EnsureStarterCredit(context.Background(), starterWalletOf("acme", "alice"))
}(i)
}
start.Done()
done.Wait()
for i, err := range errs {
if err != nil {
t.Fatalf("concurrent grant %d failed: %v", i, err)
}
}
if got := gateBalance(t, fin, "acme", "alice"); got != commercecredit.StarterCreditCents {
t.Fatalf("after %d CONCURRENT grants the gate reads %d cents, want %d — the idempotency key is not a barrier under race",
racers, got, commercecredit.StarterCreditCents)
}
}
// TestStarterCredit_PerAccountNotPerProcess proves the key is scoped to the address:
// three different orgs each get their own grant. An over-broad key would fund the
// first and silently skip every one after it.
func TestStarterCredit_PerAccountNotPerProcess(t *testing.T) {
fin := wireLedger(t)
orgs := []string{"acme", "globex", "initech"}
for _, org := range orgs {
if _, err := cloud.EnsureStarterCredit(context.Background(), starterWalletOf(org, "founder")); err != nil {
t.Fatalf("EnsureStarterCredit(%s): %v", org, err)
}
}
for _, org := range orgs {
if got := gateBalance(t, fin, org, "founder"); got != commercecredit.StarterCreditCents {
t.Fatalf("org %s reads %d cents, want %d", org, got, commercecredit.StarterCreditCents)
}
}
}
// TestStarterCredit_FundedAccountIsNotGranted is the guard against a retroactive
// payout to the whole customer base. A first-contact trigger sees every EXISTING
// account on the first request after a deploy; if "unseen" meant "new", every funded
// org in the fleet would take a free $5.
func TestStarterCredit_FundedAccountIsNotGranted(t *testing.T) {
fin := wireLedger(t)
// An existing customer with money on the books.
if _, err := fin.Deposit(context.Background(), types.DepositInput{
Org: "acme", Subject: "acme", Amount: money.FromCents(2500), Currency: "usd",
Notes: "prior top-up", Ref: "prior-topup-1",
}); err != nil {
t.Fatalf("seed deposit: %v", err)
}
if _, err := cloud.EnsureStarterCredit(context.Background(), starterWalletOf("acme", "alice")); err != nil {
t.Fatalf("EnsureStarterCredit: %v", err)
}
if got := gateBalance(t, fin, "acme", "alice"); got != 2500 {
t.Fatalf("funded account reads %d cents, want its original 2500 — a retroactive starter credit was paid", got)
}
}
// TestStarterCredit_SpentAccountIsNotGranted is the other half of that guard, and the
// reason a zero balance alone is not the eligibility test. An org that spent its
// balance down to exactly zero reads $0 but is not new; only its usage history
// distinguishes it from a fresh signup.
func TestStarterCredit_SpentAccountIsNotGranted(t *testing.T) {
fin := wireLedger(t)
ctx := context.Background()
if _, err := fin.Deposit(ctx, types.DepositInput{
Org: "acme", Subject: "acme", Amount: money.FromCents(1000), Currency: "usd",
Notes: "prior top-up", Ref: "prior-topup-1",
}); err != nil {
t.Fatalf("seed deposit: %v", err)
}
if err := fin.RecordUsage(ctx, types.UsageInput{
Org: "acme", Subject: "acme", Amount: money.FromCents(1000), Currency: "usd", RequestID: "prior-usage-1",
}); err != nil {
t.Fatalf("seed usage: %v", err)
}
if got := gateBalance(t, fin, "acme", "alice"); got != 0 {
t.Fatalf("precondition: spent-out org should read 0, got %d", got)
}
if _, err := cloud.EnsureStarterCredit(ctx, starterWalletOf("acme", "alice")); err != nil {
t.Fatalf("EnsureStarterCredit: %v", err)
}
if got := gateBalance(t, fin, "acme", "alice"); got != 0 {
t.Fatalf("a spent-out account was granted %d cents — zero balance was mistaken for a new account", got)
}
}
// TestStarterCredit_PoolIsNotThePersonWallet is the NEGATIVE CONTROL for every other
// test here, and the reason the shared signup org is excluded from the grant.
//
// In that org the members are strangers, so account.Payer resolves each to their OWN
// wallet, not the org pool. An org-keyed grant there lands somewhere no member's gate
// reads. This test asserts that gap exists — it is what makes the passing address
// tests above meaningful rather than vacuous, and it pins the trap so nobody
// "simplifies" the exclusion away.
func TestStarterCredit_PoolIsNotThePersonWallet(t *testing.T) {
fin := wireLedger(t)
org := account.SignupOrg
// Credit the POOL directly (what an org-keyed grant would do).
if _, err := cloud.EnsureStarterCredit(context.Background(), principal.Wallet{Ledger: org, Account: org}); err != nil {
t.Fatalf("EnsureStarterCredit(pool): %v", err)
}
// A member of that org reads their PERSON wallet — EMPTY. This is the
// "looks fixed and 402s anyway" failure.
if got := gateBalance(t, fin, org, "alice"); got != 0 {
t.Fatalf("a signup-org member reads %d cents from a pool grant; the address model is wrong", got)
}
// Same ledger, different account: the pool DID receive it, so the zero above is an
// ADDRESS mismatch, not a failed write.
pool, err := fin.Balance(context.Background(), org, org, "usd", false)
if err != nil {
t.Fatalf("pool balance: %v", err)
}
if pool.Cents() != commercecredit.StarterCreditCents {
t.Fatalf("pool holds %d cents, want %d — the write itself failed, so this proves nothing about addressing",
pool.Cents(), commercecredit.StarterCreditCents)
}
}
// TestStarterCredit_NoLedgerIsInert proves a split deploy is a no-op, not a crash and
// not a false success: with no co-resident ledger there is nothing to grant into, the
// account stays at $0, and the gate refuses it.
func TestStarterCredit_NoLedgerIsInert(t *testing.T) {
creditledger.Set(nil)
finance.Publish(nil)
got, err := cloud.EnsureStarterCredit(context.Background(), starterWalletOf("acme", "alice"))
if err != nil {
t.Fatalf("no-ledger must be inert, got error: %v", err)
}
if got != 0 {
t.Fatalf("no-ledger reported balance %d, want 0", got)
}
}
+100 -1
View File
@@ -1,12 +1,23 @@
package apps
import (
"context"
"encoding/json"
"time"
"github.com/hanzoai/cloud/clients/ads"
"github.com/hanzoai/cloud/clients/automations"
"github.com/hanzoai/cloud/clients/campaign"
"github.com/hanzoai/cloud/clients/coding"
"github.com/hanzoai/cloud/clients/experiments"
"github.com/hanzoai/cloud/clients/framework"
"github.com/hanzoai/cloud/clients/git"
"github.com/hanzoai/cloud/clients/guide"
"github.com/hanzoai/cloud/clients/integrations"
"github.com/hanzoai/cloud/clients/principal"
)
// wire_seams.go wires cross-subsystem in-process seams that cannot be a MountSpec
// wire_seams.go wires cross-subsystem in-process seams that cannot be a AppSpec
// because they compose functions ACROSS packages that must not import each other.
//
// The coding orchestrator (clients/coding) needs git's CloneURL + VerifyRef, but
@@ -20,4 +31,92 @@ import (
// failures are non-fatal and dropped).
func init() {
integrations.SetCodingDispatcher(coding.NewDispatcher(git.CloneURL, git.VerifyRef, nil))
// Guide growth-OBSERVE seam: the /v1/guide/profile observe layer reads the org's
// real platform truth through injected, org-scoped, honest-degrading probes.
// clients/guide imports NONE of these subsystems (decomplected), so the
// composition root — the ONE place that imports them all — binds the reads, the
// same injected-function pattern the coding dispatcher above uses. Each probe is
// PROVABLY org-scoped: framework.ModuleInstalled keys GetDocType on the org;
// integrations.Connected keys store.Get on the org and never surfaces the token.
// HasDeployment/RevenueCents/RecordCount are LEFT NIL: deploy is cluster/admin-
// scoped (not per-org) and commerce-revenue-of-record + the crm record count have
// no clean per-org in-process read yet — binding a read whose org-scoping we
// cannot guarantee would be the bug. Until a provably-org-scoped read lands their
// signals honest-degrade to not-present (the vocabulary is the contract; a nil
// seam can never be a spurious true).
guide.BindSignals(guide.Signals{
ModuleInstalled: framework.ModuleInstalled,
ConnectorPresent: integrations.Connected,
})
// Inbound-event seam: a verified provider webhook (or chat channel) in
// clients/integrations fires the automations engine's Deliver here — the ONE place
// that imports both, so integrations never has to import automations (which imports
// it, for credential custody). A primitive-typed adapter keeps the seam free of the
// automations types.
integrations.SetAutomationTrigger(func(ctx context.Context, org, source, name, dedupeKey string, depth int, payload map[string]any) (int, error) {
return automations.Deliver(ctx, org, automations.TriggerEvent{
Source: source, Name: name, DedupeKey: dedupeKey, Depth: depth, Payload: payload,
})
})
// GTM PAID channel: the /v1/campaign orchestrator fans out to executors that
// satisfy campaign.Channel; the composition root is the ONE place that imports
// both campaign and the ad plane, so it adapts ads' connector-consuming
// execution funcs (provider.go — each resolves the org's ad token through
// integrations.TokenFor and fails closed) onto the primitive-typed channel
// seam. campaign never imports ads and ads never imports campaign — the same
// injected-function decoupling the coding dispatcher above uses.
campaign.RegisterChannel(campaign.NewChannel(campaign.KindPaid,
func(ctx context.Context, org string, p campaign.Plan) (campaign.Ref, error) {
r, err := ads.LaunchPaid(ctx, org, ads.PaidPlan{
Platform: p.Platform, Account: p.Account, Name: p.Name,
Objective: p.Objective, BudgetCents: p.BudgetCents, ScheduleAt: p.ScheduleAt,
})
return campaign.Ref{Platform: r.Platform, Account: r.Account, ExternalID: r.ExternalID, Status: r.Status, Detail: r.Detail}, err
},
func(ctx context.Context, org string, ref campaign.Ref) (int64, error) {
return ads.PaidSpend(ctx, org, ads.PaidRef{Platform: ref.Platform, Account: ref.Account, ExternalID: ref.ExternalID})
},
func(ctx context.Context, org string, ref campaign.Ref) error {
return ads.PausePaid(ctx, org, ads.PaidRef{Platform: ref.Platform, Account: ref.Account, ExternalID: ref.ExternalID})
},
))
// GTM creative A/B composes the merged EXPERIMENT primitive (clients/experiments)
// — campaign never reinvents bucketing or measurement. Assign resolves the
// subject's variant (creative) from the experiment's flag; Analyze is pull-model
// (it reads the metric from analytics itself), returned as opaque JSON so
// campaign stays decoupled from the analysis type. Campaign-linked experiments
// live in the org's default project. Both are nil-safe upstream: an org that
// never created a "campaign:<id>" experiment runs a single creative (Assign
// errors → "" → Content[0]).
campaign.SetExperiment(
func(ctx context.Context, org, experimentID, subject string) (string, error) {
a, err := experiments.Assign(ctx, org, principal.DefaultProject, experimentID, subject, nil)
if err != nil {
return "", err
}
return a.Variant, nil
},
func(ctx context.Context, org, experimentID string, start, end time.Time) (json.RawMessage, error) {
an, err := experiments.Analyze(ctx, org, principal.DefaultProject, experimentID, start, end, 0.05)
if err != nil {
return nil, err
}
return json.Marshal(an)
},
)
// GTM ORGANIC + EMAIL channels wire HERE the same way once their executors land
// (designed follow-ons — the publish rename + marketing email-connector wiring):
//
// campaign.RegisterChannel(campaign.NewChannel(campaign.KindOrganic,
// publish.Syndicate, publish.NoSpend, publish.Unpublish)) // social connectors
// campaign.RegisterChannel(campaign.NewChannel(campaign.KindEmail,
// marketing.Broadcast, marketing.NoSpend, marketing.Halt)) // email connectors
//
// Until wired, those channels record "unavailable" on a fan-out (honest) and a
// campaign runs a single creative — never a fabricated launch or variant.
}
+123 -91
View File
@@ -22,98 +22,124 @@ var frozen = []struct {
name string
ownsHealth bool
hasShutdown bool
global bool // receives the bare *zip.App — see AppSpec.Global
}{
{"pubsub", false, true}, // was order 5
{"kafka", false, true}, // was order 6
{"agentskills", false, false}, // was order 8
{"flags", true, true}, // was order 9; native engine: /v1/flags health + store shutdown
{"kms", true, false}, // was order 10
{"metrics", false, false}, // was order 40
{"ingress", false, true}, // was order 42
{"account", false, false}, // was order 48
{"iam", false, false}, // was order 50
{"base", true, true}, // was order 60; per-org embed added Shutdown (#298)
{"o11y", false, true}, // ONE observability subsystem (was co-owned orders 69+70): read plane + the hanzoai/o11y module wildcard folded in as MountO11y's terminal sub-mount. OwnsHealth=false keeps /v1/o11y/health the generic always-ok route the module co-entry used to trigger.
{"authz", false, false}, // was order 70
{"commerce", false, false}, // was order 100
{"licensing", false, false}, // was order 110
{"plan", true, false}, // was order 111; enable id normalized plans->plan (routes stay /v1/plans/*)
{"pricing", true, false}, // was order 112
{"storage", true, false}, // was order 118
{"provisioning", false, false}, // was order 120
{"billing", false, false}, // was order 121
{"rollingcap", false, false}, // rolling spend-cap gate (after billing); golden drifted — refrozen
{"account-bridge", false, false}, // was order 122
{"do", false, false}, // was order 123
{"platform", true, false}, // was order 124
{"projects", false, false}, // was order 125
{"dns", false, false}, // new: /v1/dns zone plane (after projects)
{"domain", false, false}, // new: Hanzo Domains registrar (/v1/domain), after dns
{"prompts", false, false}, // was order 126
{"agents", false, true}, // was order 127
{"link", false, true}, // new: unified AI login manager (/v1/links), after agents
{"wallets", false, true}, // was order 127
{"x402", false, true}, // new: x402 pay-per-use settlement (after wallets)
{"paas", true, false}, // was order 128
{"deploy", true, false}, // after paas (release seam), before functions
{"functions", false, false}, // was order 128
{"tracker", false, false}, // was order 129
{"templates", false, false}, // was order 129
{"framework", false, true}, // was order 129
{"knowledge", false, false}, // was order 130
{"content", false, true}, // new: marketing content loop (after knowledge)
{"catalogsync", false, true}, // new: reverse loop (product.created → render) after content
{"ml", true, false}, // was order 130
{"usage", false, false}, // was order 131
{"crm", false, false}, // was order 131
{"marketing", false, true}, // new: marketing domain fold (after crm)
{"ads", false, true}, // new: ads domain fold (after crm)
{"validators", false, true}, // new: NFT-gated node provisioning (after ads); golden refrozen
{"social", false, true}, // new: /v1/social fold (after crm)
{"analytics", true, false}, // was order 132
{"git", false, false}, // was order 132
{"sync", false, true}, // /v1/sync engine (owns per-org DB handles → Shutdown)
{"visor", false, false}, // was order 133
{"captable", false, true}, // was order 133
{"code", false, true}, // was order 134
{"zero-trust", false, false}, // was order 134
{"dataroom", true, true}, // was order 134
{"graph", false, false}, // was order 135
{"security", true, true}, // was order 136
{"integrations", false, true}, // was order 137
{"cloudflare", false, false}, // new: /v1/cloudflare edge plane (after integrations)
{"sbom", true, false}, // was order 137
{"team", false, true}, // was order 138
{"settings", false, true}, // was order 138
{"notify", true, false}, // was order 139
{"channels", false, true}, // new: /v1/channels transport plane (after notify; must mount after integrations so RegisterIngress installs before webhooks emit)
{"gateway", false, false}, // was order 139
{"entitlements", false, true}, // was order 139
{"exec", false, false}, // was order 140
{"websearch", false, false}, // was order 141
{"world", false, true}, // was order 142
{"runtime", false, false}, // was order 143; was "bot" until the transport was named for what it is
{"authors", false, true}, // was order 143
{"bots", false, false}, // was order 143
{"audit", false, false}, // was order 144
{"affiliates", false, false}, // was order 144
{"sign", true, true}, // was order 145
{"product", false, false}, // was order 145
{"evals", false, false}, // was order 145
{"treasury", false, true}, // was order 146
{"admin", false, false}, // was order 146
{"admission", false, true}, // launch-control gate: composes flags (registry+seed+mode route+Enforce); Shutdown closes the registry store
{"tasks", false, false}, // was order 147; platform cron folded in as a sub-mount of tasks.Mount (was a separate entry)
{"automations", false, true}, // was order 148; connectorruntime (POST /v1/automations/connectors/:id/run) folded in as a sub-mount of automations.Mount
{"tools", false, true}, // new: unified tool plane (after automations)
{"marketplace", false, true}, // new: marketplace over the tool plane (after tools)
{"referrals", false, false}, // was order 149
{"guide", false, true}, // new: Business AI Guide (after referrals, before ai)
{"company", false, true}, // new: Hanzo Company formation state machine (after guide)
{"agent", false, false}, // new: /v1/agent tool-calling round (before zen/ai catch-all)
{"zen", false, false}, // zen* claim middleware before ai's catch-all (hip-00NN)
{"ai", false, false}, // was order 150
{"plugins", false, false}, // was order 900
{"pubsub", false, true, false}, // was order 5
{"kafka", false, true, false}, // was order 6
{"agentskills", false, false, false}, // was order 8
{"flags", true, true, false}, // was order 9; native engine: /v1/flags health + store shutdown
{"kms", true, false, false}, // was order 10
{"metrics", false, false, true}, // was order 40
{"ingress", false, true, false}, // was order 42
{"account", false, false, false}, // was order 48
{"iam", false, false, false}, // was order 50
{"base", true, true, false}, // was order 60; per-org embed added Shutdown (#298)
// hasShutdown flipped true->false when o11y became a PLUGIN (cloud.PluginSpec,
// its own cmd/o11y binary). Deliberate and load-bearing, not drift: the host no
// longer owns any o11y resource to close. The collector/sink/Datastore moved into
// the child, which flushes them in its OWN app.OnShutdown, and zip.Load registers
// the host-side hook that stops the child. A Shutdown on this spec would now be a
// host closing something it does not have. Name/OwnsHealth/Global are UNCHANGED —
// position, health routing and the app-wide grant are all still pinned here.
{"o11y", false, false, true}, // ONE observability subsystem (was co-owned orders 69+70), now out-of-process. OwnsHealth=false keeps /v1/o11y/health the generic always-ok route, which Serve registers before MountAll and therefore ahead of the plugin's /v1/o11y/* mount.
{"authz", false, false, true}, // was order 70
{"commerce", false, false, true}, // was order 100
{"licensing", false, false, true}, // was order 110
{"plan", true, false, false}, // was order 111; enable id normalized plans->plan (routes stay /v1/plans/*)
{"pricing", true, false, false}, // was order 112
{"storage", true, false, false}, // was order 118
{"provisioning", false, false, false}, // was order 120
{"billing", false, false, false}, // was order 121
{"rollingcap", false, false, false}, // rolling spend-cap gate (after billing); golden drifted — refrozen
{"account-bridge", false, false, false}, // was order 122
{"do", false, false, false}, // was order 123
{"platform", true, true, false}, // was order 124
{"projects", false, true, false}, // was order 125
{"dns", false, false, false}, // new: /v1/dns zone plane (after projects)
{"domain", false, false, false}, // new: Hanzo Domains registrar (/v1/domain), after dns
{"prompts", false, false, false}, // was order 126
{"agents", false, true, false}, // was order 127
{"link", false, true, false}, // new: unified AI login manager (/v1/links), after agents
{"wallets", false, true, false}, // was order 127
{"x402", false, true, false}, // new: x402 pay-per-use settlement (after wallets)
{"paas", true, false, false}, // was order 128
{"deploy", true, false, false}, // after paas (release seam), before functions
{"functions", false, false, false}, // was order 128
{"tracker", false, false, false}, // was order 129
{"templates", false, false, false}, // was order 129
{"blueprint", true, false, false}, // new: OSS-template compute-cost basis /v1/blueprint (after templates); owns health, embedded content → no Shutdown
{"framework", false, true, false}, // was order 129
{"knowledge", false, false, false}, // was order 130
{"help", false, false, false}, // new: Hanzo Support public plane /v1/help (after knowledge; framework lane with a companion subsystem, no store → no Shutdown)
{"content", false, true, false}, // new: marketing content loop (after knowledge)
{"catalogsync", false, true, false}, // new: reverse loop (product.created → render) after content
{"webhooks", false, true, false}, // new: platform-global /v1/webhooks registry + bus-driven dispatcher (after catalogsync); owns per-org stores + worker pool → Shutdown
{"ml", true, false, false}, // was order 130
{"usage", false, false, false}, // was order 131
{"leaderboard", false, true, false}, // new: gamified usage analytics (after usage), owns opt-in SQLite (Shutdown)
{"crm", false, false, false}, // was order 131
{"marketing", false, true, false}, // new: marketing domain fold (after crm)
{"ads", false, true, false}, // new: ads domain fold (after crm)
{"campaign", false, true, false}, // new: /v1/campaign GTM orchestration (after ads); fans out to channels
{"validators", false, true, false}, // new: NFT-gated node provisioning (after ads); golden refrozen
{"social", false, true, false}, // new: /v1/social fold (after crm)
{"analytics", true, false, false}, // was order 132
{"git", false, false, false}, // was order 132
{"sync", false, true, false}, // /v1/sync engine (owns per-org DB handles → Shutdown)
{"visor", false, false, false}, // was order 133
{"venue", false, false, false}, // new: /v1/cloud connect-a-cloud-account plane (after visor); folds discovered clusters into the fleet
{"captable", false, true, false}, // was order 133
{"code", false, true, false}, // was order 134
{"zero-trust", false, false, false}, // was order 134
{"share", false, false, false}, // ngrok-native public sharing (/v1/share/*)
{"dataroom", true, true, false}, // was order 134
{"graph", false, false, false}, // was order 135
{"security", true, true, false}, // was order 136
{"integrations", false, true, false}, // was order 137
{"destinations", false, true, false}, // new: /v1/destinations CDP fan-out (after integrations, before cloudflare)
{"cloudflare", false, false, false}, // new: /v1/cloudflare edge plane (after integrations)
{"sbom", true, false, false}, // was order 137
{"team", false, true, false}, // was order 138
{"settings", false, true, false}, // was order 138
{"prefs", false, true, false}, // new: per-user preference plane (after settings); Shutdown closes the store
{"notify", true, false, false}, // was order 139
{"channels", false, true, false}, // new: /v1/channels transport plane (after notify; must mount after integrations so RegisterIngress installs before webhooks emit)
{"gateway", false, false, false}, // was order 139
{"entitlements", false, true, false}, // was order 139
{"exec", false, false, false}, // was order 140
{"websearch", false, false, false}, // was order 141
{"index", true, true, false}, // new: in-binary full-text index (Meilisearch dialect), after websearch
{"world", false, true, false}, // was order 142
{"runtime", false, false, false}, // was order 143; was "bot" until the transport was named for what it is
{"authors", false, true, false}, // was order 143
{"bots", false, false, false}, // was order 143
{"audit", false, false, false}, // was order 144
{"affiliates", false, false, false}, // was order 144
{"esign", true, true, false}, // was order 145; renamed sign->esign
{"product", false, false, false}, // was order 145
{"evals", false, false, false}, // was order 145
{"benchmark", false, false, false}, // benchmark plane (after evals, before treasury)
{"research", false, true, false}, // R&D evidence plane + /research board (HIP-0512), arena sibling after benchmark; Shutdown closes per-org stores
{"experiments", false, true, false}, // unified A/B EXPERIMENT primitive: composes flags(assign)+analytics(measure)+research(evidence); Shutdown closes the registry stores
{"books", false, true, false}, // AI bookkeeper (/v1/books, per-org SQLite); Shutdown closes org stores
{"treasury", false, true, false}, // was order 146
{"admin", false, false, false}, // was order 146
{"admission", false, true, false}, // launch-control gate: composes flags (registry+seed+mode route+Enforce); Shutdown closes the registry store
{"tasks", false, false, false}, // was order 147; platform cron folded in as a sub-mount of tasks.Mount (was a separate entry)
{"automations", false, true, false}, // was order 148; connectorruntime (POST /v1/automations/connectors/:id/run) folded in as a sub-mount of automations.Mount
{"tools", false, true, false}, // new: unified tool plane (after automations)
{"marketplace", false, true, false}, // new: marketplace over the tool plane (after tools)
{"referrals", false, false, false}, // was order 149
{"guide", false, true, false}, // new: Business AI Guide (after referrals, before ai)
{"company", false, true, false}, // new: Hanzo Company formation state machine (after guide)
{"compliance", true, true, false}, // new: Hanzo Compliance — KYC/KYB + accreditation + audit posture (after company)
{"legal", true, true, false}, // new: Hanzo Legal — template + generation engine + e-sign/filing (after compliance)
{"agent", false, false, true}, // new: /v1/agent tool-calling round (before zen/ai catch-all)
{"ask", false, false, false}, // new: unified grounded advisor /v1/ask (before zen/ai catch-all)
{"translate", false, true, false}, // new: /v1/translate, two tiers over one endpoint (HIP-0516); Shutdown closes the per-org translation memories
{"zen", false, false, false}, // zen* claim middleware before ai's catch-all (hip-00NN)
{"ai", false, false, true}, // was order 150
{"plugins", false, false, false}, // was order 900
}
// TestWireOrderMatchesFrozen proves the composition root's mount order is
@@ -134,6 +160,12 @@ func TestWireOrderMatchesFrozen(t *testing.T) {
if (s.Shutdown != nil) != w.hasShutdown {
t.Errorf("position %d (%s): hasShutdown = %v, frozen = %v", i, s.Name, s.Shutdown != nil, w.hasShutdown)
}
// Global hands a subsystem the bare *zip.App, and with it the ability to
// gate every route in the binary. Freezing it here means a new grant cannot
// arrive as a quiet field on one line of a 128-entry literal.
if s.Global != w.global {
t.Errorf("position %d (%s): Global = %v, frozen = %v — an app-wide capability changed", i, s.Name, s.Global, w.global)
}
if s.Mount == nil {
t.Errorf("position %d (%s): Mount is nil", i, s.Name)
}
+52 -25
View File
@@ -44,10 +44,12 @@ import (
// exact served cost after. zen's Meter/Gate are wired here; ai's edge gate stays
// 0 for these paths.
//
// Billing granularity is org / project / user, mirroring the edge gate's
// Billing granularity is wallet / project / user, mirroring the edge gate's
// identityFromCtx exactly:
// - the HOME org (principal.BillingOrg) is the balance key — who PAYS. An admin
// acting in another org bills the admin's home org, never the org acted on.
// - the WALLET (principal.WalletOf) is the balance key — who PAYS. Its Ledger is
// the home org, so an admin acting in another org bills the admin's own books,
// never the org acted on; its Account is the wallet within them, resolved by
// the ONE rule (account.Payer). A request with no resolvable wallet is refused.
// - the project (principal.ValidatedProject) scopes spend caps; a project may
// carry its own billing account, resolved server-side by commerce from the
// org's project binding (the X-Billing-Account-Id header only attributes,
@@ -56,7 +58,7 @@ import (
//
// It is wired BEFORE ai in Wire() so Claim's c.Next() falls through to ai's
// catch-all. zen's catalog reads its upstream keys from KMS via the Key resolver.
func mountZen(a *zip.App, deps cloud.Deps) error {
func mountZen(a cloud.Router, deps cloud.Deps) error {
z, err := zen.New(zen.Config{
Logger: deps.Logger,
Key: zenKeyResolver(deps.KMS),
@@ -72,30 +74,48 @@ func mountZen(a *zip.App, deps cloud.Deps) error {
return nil
}
// cloudTenantResolver is the multi-tenant billing-identity resolver: it keys
// the balance on the validated HOME org (who pays), resolves the project scope
// from a validated claim, and attributes the user. It mirrors the edge gate's
// identityFromCtx so a zen* debit lands on the SAME ledger axes the edge gate
// would have used — home-org balance, project+service scope, user actor — and a
// masquerading admin bills their own home org, never the org acted on. A request
// with no validated principal resolves to an empty Tenant, which zen's Valid()
// gate refuses (no free, anonymous usage).
// cloudTenantResolver is the multi-tenant billing-identity resolver: it resolves
// the money's ADDRESS once (principal.WalletOf), scopes the project from a
// validated claim, and attributes the user. It mirrors the edge gate's
// identityFromCtx by calling the SAME resolver, not by keeping a second copy in
// step — so a zen* request gates and debits the very wallet the edge gate, the ai
// gate, the ai debit and GET /v1/billing/balance address.
//
// THE ADDRESS IS TWO HALVES AND BOTH TRAVEL. It used to send only the ledger
// (BillingOrg) and let the gate read the org POOL, on the premise that prepaid
// balance is per-org. That premise is false for a member of the shared signup org,
// where members are strangers to each other and each holds their own account:
// zen then gated a pool a member cannot spend from while the ai path debited the
// member's own wallet — the third recurrence of one bug, catalogued in
// clients/principal/wallet.go. A brand-new $0 signup read the signup org's funded
// pool and served for free; a member who had bought credit was refused because the
// purchase had landed in that same pool.
//
// A request with no resolvable wallet returns the ZERO Tenant, which zen's Valid()
// gate refuses with its 402 (no free, anonymous usage) — the ok-bit is the answer
// and it propagates; there is no substitute payer.
func cloudTenantResolver(c *zip.Ctx) zen.Tenant {
home, _ := principal.BillingOrg(c)
w, ok := principal.WalletOf(c)
if !ok {
return zen.Tenant{}
}
project, _ := principal.ValidatedProject(c)
return zen.Tenant{
Org: c.Org(),
User: c.User(),
BillingOrg: home,
BillingOrg: w.Ledger,
Wallet: w.Account,
Project: project,
}
}
// commerceGate is zen's pre-serve authorization backed by cloud commerce. It
// asks the metering client whether the home org can cover the request's
// estimated cost (priced at the tier that WILL serve, so an overflow is gated
// against its real cost). The balance key is the home org (User); the project
// scopes the spend cap. The estimate is exact 18-dp atto-USD from zen, folded to
// asks the metering client whether the request's WALLET can cover its estimated
// cost (priced at the tier that WILL serve, so an overflow is gated against its
// real cost). The balance key is the wallet (Tenant.Payer — the account within
// the ledger BillingOrg names), the same address the debit below spends from and
// the same one the ai path and the balance view read; the project scopes the
// spend cap. The estimate is exact 18-dp atto-USD from zen, folded to
// whole cents for the balance check (a sub-cent estimate gates as "any positive
// balance", the same contract as the edge gate's AmountCents). An unconfigured
// metering client (nil) admits everything — zen's own tenant gate still refuses
@@ -106,7 +126,7 @@ func commerceGate(m *metering.Client) zen.Gate {
return nil
}
return func(ctx context.Context, t zen.Tenant, model string, est hmoney.Amount) error {
if t.BillingOrg == "" {
if t.BillingOrg == "" || t.Payer() == "" {
return fmt.Errorf("a billable tenant is required (no anonymous usage)")
}
// zen's estimate is an exact 18-dp USD value. Fold it to whole cents for
@@ -121,8 +141,9 @@ func commerceGate(m *metering.Client) zen.Gate {
// request of ANY size against any positive balance.
cents := credit(est).Cents()
v, err := m.AuthorizeVerdict(ctx, metering.AuthInput{
User: t.BillingOrg,
User: t.Payer(), // the ACCOUNT within the ledger — never the bare org
Org: t.BillingOrg,
Actor: t.User,
AmountCents: cents,
Project: t.Project,
Service: zenService,
@@ -142,7 +163,7 @@ func commerceGate(m *metering.Client) zen.Gate {
}
// commerceMeter is zen's post-serve usage recorder backed by cloud commerce.
// It debits the home org for the EXACT served cost as a typed money.Amount
// It debits the gated WALLET for the EXACT served cost as a typed money.Amount
// (native 18-dp USD — the same precision the co-resident finance ledger holds),
// so an exact per-token cost is never floored to cents or micros. Attributed to
// the requested zen SKU with real token counts. The debit is detached (background
@@ -160,7 +181,7 @@ func commerceMeter(m *metering.Client) zen.Meter {
type commerceMeterImpl struct{ m *metering.Client }
func (g commerceMeterImpl) Record(ctx context.Context, u zen.Usage) {
if u.Tenant.BillingOrg == "" {
if u.Tenant.BillingOrg == "" || u.Tenant.Payer() == "" {
return // never debit an unattributable request
}
// Beside the commerce debit, land the SAME warehouse row + gen_ai span every
@@ -211,8 +232,14 @@ func (g commerceMeterImpl) Record(ctx context.Context, u zen.Usage) {
}
// meterUsage projects a served zen.Usage onto the commerce debit. It is the ONE
// place the debit's amount is chosen, and it is pure — no ledger, no warehouse —
// so the money property is a unit test rather than an integration.
// place the debit's amount AND its address are chosen, and it is pure — no ledger,
// no warehouse — so the money property is a unit test rather than an integration.
//
// The address is Tenant.Payer, the very expression commerceGate authorized against:
// the gate and the debit read one value, so they cannot come to mean two wallets.
// Actor stays Tenant.User — for a machine key the payer is the org while the actor
// is the key, so an address read off the actor bills the wrong account and an actor
// read off the address loses the audit trail. They are different axes; both travel.
//
// The amount is the RETAIL Charge: what the caller pays. Cost is the upstream
// COGS we pay to serve the call; it is never the debit. It rides only the
@@ -223,7 +250,7 @@ func (g commerceMeterImpl) Record(ctx context.Context, u zen.Usage) {
// customer price (usageBilledCents), never its CostIn/CostOut COGS.
func meterUsage(u zen.Usage) metering.Usage {
return metering.Usage{
User: u.Tenant.BillingOrg,
User: u.Tenant.Payer(), // the ACCOUNT the gate authorized
Org: u.Tenant.BillingOrg,
Actor: u.Tenant.User,
Model: u.Model,
+64
View File
@@ -0,0 +1,64 @@
// Copyright 2026 Hanzo AI Inc. All Rights Reserved.
package apps
import (
"context"
"errors"
"testing"
)
// stubKMS is a KMSClient whose GetSecret returns a sealed value when present or a
// not-found error otherwise — mirroring the co-resident store that, in production,
// holds no upstream provider keys (they are provisioned as KMS-injected env).
type stubKMS struct{ sealed map[string]string }
func (s stubKMS) GetSecret(_ context.Context, ref string) ([]byte, error) {
if v, ok := s.sealed[ref]; ok {
return []byte(v), nil
}
return nil, errors.New("secret not found")
}
func (stubKMS) PutSecret(context.Context, string, []byte) error { return nil }
func (stubKMS) Sign(context.Context, string, []byte) ([]byte, error) { return nil, nil }
// TestZenKeyResolver_EnvFallback pins the production wiring: the upstream provider
// key is provisioned as env (from the cloud-api-llm-keys secret), NOT sealed in the
// co-resident KMS store, so a KMS miss must resolve to the env value rather than
// returning "" (which would send an empty bearer upstream → provider 401).
func TestZenKeyResolver_EnvFallback(t *testing.T) {
const env = "DO_AI_API_KEY"
t.Setenv(env, "env-provisioned-key")
// KMS store has no upstream keys (the real deployment state) — resolve from env.
if got := zenKeyResolver(stubKMS{})(context.Background(), env); got != "env-provisioned-key" {
t.Fatalf("KMS-miss: got %q, want env value", got)
}
// A nil KMS client (KMS disabled) — still resolve from env.
if got := zenKeyResolver(nil)(context.Background(), env); got != "env-provisioned-key" {
t.Fatalf("nil-KMS: got %q, want env value", got)
}
}
// TestZenKeyResolver_EnvTakesPrecedence pins the resolution ORDER: env is read
// FIRST, then KMS — the same order ai uses on the prod hot path. The operator
// injects provider keys as env from the KMS-synced secret, so the env is the live
// value; the co-resident store is the fallback. A key present in BOTH surfaces
// resolves to the env value.
func TestZenKeyResolver_EnvTakesPrecedence(t *testing.T) {
const env = "ANTHROPIC_API_KEY"
t.Setenv(env, "env-key")
got := zenKeyResolver(stubKMS{sealed: map[string]string{env: "sealed-key"}})(context.Background(), env)
if got != "env-key" {
t.Fatalf("got %q, want env-key (env precedence)", got)
}
}
// TestZenKeyResolver_AbsentEverywhere keeps the fail-fast contract: absent from both
// surfaces resolves to "" so zen refuses rather than serving for free.
func TestZenKeyResolver_AbsentEverywhere(t *testing.T) {
if got := zenKeyResolver(stubKMS{})(context.Background(), "MISSING_KEY_XYZ"); got != "" {
t.Fatalf("got %q, want empty", got)
}
}
+164
View File
@@ -0,0 +1,164 @@
// Copyright 2026 Hanzo AI Inc. All Rights Reserved.
package apps
import (
"encoding/json"
"io"
"net/http/httptest"
"testing"
"github.com/hanzoai/account"
"github.com/hanzoai/cloud/clients/principal"
"github.com/hanzoai/zen"
"github.com/zap-proto/zip"
)
// The money's address is (ledger, account), and the zen path is the one that used
// to carry only the ledger. These tests pin the whole address across the two places
// zen touches money — the pre-serve gate and the post-serve debit — against the
// resolver every OTHER path already uses (principal.WalletOf, i.e. account.Payer).
//
// The failure they exist to prevent is not "zen bills the wrong number"; it is zen
// billing the wrong WALLET, which reads as two unrelated symptoms at once: a $0
// signup served for free off the signup org's funded pool, and a member who had
// bought credit refused because the purchase landed in that same pool.
// tenant drives cloudTenantResolver over the SAME zip.Ctx the identity boundary
// feeds in production, so a test sets the headers SanitizeIdentity mints.
func tenant(t *testing.T, headers map[string]string) zen.Tenant {
t.Helper()
app := zip.New(zip.Config{DisableStartupMessage: true})
app.Get("/t", func(c *zip.Ctx) error {
return c.JSON(200, cloudTenantResolver(c))
})
req := httptest.NewRequest("GET", "/t", nil)
for h, v := range headers {
req.Header.Set(h, v)
}
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("tenant probe: %v", err)
}
defer func() { _ = resp.Body.Close() }()
b, _ := io.ReadAll(resp.Body)
var out zen.Tenant
if err := json.Unmarshal(b, &out); err != nil {
t.Fatalf("tenant decode: %v (%s)", err, b)
}
return out
}
// walletOf is the address every non-zen money path resolves: the edge BillingGate,
// the ai prepaid gate, the ai usage debit and GET /v1/billing/balance.
func walletOf(t *testing.T, headers map[string]string) principal.Wallet {
t.Helper()
app := zip.New(zip.Config{DisableStartupMessage: true})
app.Get("/w", func(c *zip.Ctx) error {
w, ok := principal.WalletOf(c)
return c.JSON(200, map[string]any{"ledger": w.Ledger, "account": w.Account, "ok": ok})
})
req := httptest.NewRequest("GET", "/w", nil)
for h, v := range headers {
req.Header.Set(h, v)
}
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("wallet probe: %v", err)
}
defer func() { _ = resp.Body.Close() }()
b, _ := io.ReadAll(resp.Body)
var out struct {
Ledger string `json:"ledger"`
Account string `json:"account"`
}
if err := json.Unmarshal(b, &out); err != nil {
t.Fatalf("wallet decode: %v (%s)", err, b)
}
return principal.Wallet{Ledger: out.Ledger, Account: out.Account}
}
// signupMember is the live shape of a self-serve account: home org == the shared
// signup org, X-User-Id == the JWT `sub` (a UUID), X-User-Name == the IAM username.
// 100% of self-serve accounts look like this, and it is the ONLY shape where the
// pool and the person differ — which is why the bug was invisible to tenant orgs.
var signupMember = map[string]string{
"X-User-Id": "3f2a9c14-7b6e-4d05-9a11-8c73e2f0b4d6",
"X-User-Name": "z@hanzo.ai",
"X-Org-Id": account.SignupOrg,
"X-User-Owner": account.SignupOrg,
"Authorization": "Bearer test",
}
// orgMember is a member of a REAL tenant org, whose members share one balance.
var orgMember = map[string]string{
"X-User-Id": "0d1e2f30-4a5b-6c7d-8e9f-a0b1c2d3e4f5",
"X-User-Name": "bob",
"X-Org-Id": "acme",
"X-User-Owner": "acme",
"Authorization": "Bearer test",
}
// TestZenAddressIsTheOneAddress is the deliverable: for the same request, zen's
// gate, zen's debit and every other money path name ONE (ledger, account) pair.
//
// It fails on the old code for signupMember only — zen answered ("hanzo","hanzo")
// while everything else answered ("hanzo","hanzo/z@hanzo.ai") — and that is exactly
// the blast radius: every self-serve signup, no tenant org.
func TestZenAddressIsTheOneAddress(t *testing.T) {
for name, h := range map[string]map[string]string{
"signup org member (pool != person)": signupMember,
"tenant org member (pool == person)": orgMember,
} {
want := walletOf(t, h)
got := tenant(t, h)
if got.BillingOrg != want.Ledger {
t.Errorf("%s: zen ledger = %q, want %q", name, got.BillingOrg, want.Ledger)
}
// The gate reads Payer; so does the debit, through meterUsage.
if got.Payer() != want.Account {
t.Errorf("%s: zen gate address = %q, want %q — a funded member 402s, or an empty one serves free", name, got.Payer(), want.Account)
}
if debit := meterUsage(zen.Usage{Tenant: got}); debit.User != want.Account || debit.Org != want.Ledger {
t.Errorf("%s: zen debit address = (%q,%q), want (%q,%q) — the gate and the debit have separated", name, debit.Org, debit.User, want.Ledger, want.Account)
}
}
}
// TestZenDebitAddressesWhatTheGateAuthorized: the gate and the debit read the SAME
// expression, so no header shape can make them disagree. This is the inversion the
// codebase has shipped twice (clients/principal/wallet.go): gate on the person,
// spend from the pool.
func TestZenDebitAddressesWhatTheGateAuthorized(t *testing.T) {
for _, tn := range []zen.Tenant{
{Org: "hanzo", BillingOrg: "hanzo", Wallet: "hanzo/alice", User: "hanzo/alice"},
{Org: "acme", BillingOrg: "acme", User: "acme/robot"},
{Org: "victim", BillingOrg: "hanzo", Wallet: "hanzo/root", User: "hanzo/root"},
} {
u := meterUsage(zen.Usage{Tenant: tn})
if u.User != tn.Payer() {
t.Errorf("tenant %+v: debit account = %q, gate account = %q", tn, u.User, tn.Payer())
}
if u.Org != tn.BillingOrg {
t.Errorf("tenant %+v: debit ledger = %q, want %q", tn, u.Org, tn.BillingOrg)
}
// The actor is a different axis and must survive: for a machine key the
// payer is the org while the actor is the key.
if u.Actor != tn.User {
t.Errorf("tenant %+v: actor = %q, want %q — attribution lost to the address", tn, u.Actor, tn.User)
}
}
}
// TestZenRefusesWithoutAWallet: an unresolvable principal yields the ZERO tenant,
// which zen's Valid() refuses. An anonymous caller with a forged X-Org-Id must never
// name a ledger — it could otherwise probe, and then drain, a victim org's balance.
func TestZenRefusesWithoutAWallet(t *testing.T) {
tn := tenant(t, map[string]string{"X-Org-Id": "victim"})
if tn.Valid() {
t.Fatalf("forged X-Org-Id with no principal resolved a billable tenant: %+v", tn)
}
if tn.Payer() != "" || tn.BillingOrg != "" {
t.Fatalf("unattributable request resolved an address: %+v", tn)
}
}
-50
View File
@@ -1,50 +0,0 @@
// Copyright 2026 The Hanzo Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package cloud
import (
"os"
"testing"
)
// TestJWTAudiences_AcceptsAdminGuard is the operator-cockpit keystone. The
// admin.hanzo.ai forward-auth guard is the confidential client `hanzo-admin-guard`,
// so IAM mints its access tokens with aud=hanzo-admin-guard (each app's aud is its
// client_id). The guard forwards that bearer to cloud-api /v1/admin/*; the identity
// sanitizer only grants SuperAdmin (owner==adminOrg) to a VALIDATED principal, and
// validation enforces this audience allowlist. If hanzo-admin-guard is not accepted
// the token resolves anonymous and the SuperAdmin gate reads false -> 403, even
// though the token's owner IS admin. Pin the client_id into the baked default so the
// forwarded bearer validates.
func TestJWTAudiences_AcceptsAdminGuard(t *testing.T) {
os.Unsetenv("CLOUD_JWT_AUDIENCES")
os.Unsetenv("GATEWAY_ALLOWED_AUDIENCES")
has := func(list []string, v string) bool {
for _, s := range list {
if s == v {
return true
}
}
return false
}
if !has(defaultJWTAudiences, "hanzo-admin-guard") {
t.Fatalf("defaultJWTAudiences must include hanzo-admin-guard (the admin-cockpit guard client_id); got %v", defaultJWTAudiences)
}
if !has(jwtAudiencesFromEnv(), "hanzo-admin-guard") {
t.Fatalf("resolved JWT audiences must include hanzo-admin-guard; got %v", jwtAudiencesFromEnv())
}
}
-48
View File
@@ -1,48 +0,0 @@
// Copyright 2026 The Hanzo Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package cloud
import (
"os"
"testing"
)
// TestJWTAudiences_AcceptsHanzoTeam pins the hanzo.team OIDC client. IAM mints
// team's access tokens with aud=hanzo-team (each app's aud is its client_id);
// the team OAuth callback sets that token as the hanzo_iam_token cookie, and
// the usage/wallet page (/v1/team/billing/ui/) reads /v1/billing/balance +
// /v1/usage/summary same-origin on it. If hanzo-team is not accepted the
// cookie resolves anonymous and every wallet read 401s — and the callback's
// own validator (NewTokenValidator shares this allowlist) refuses the login.
func TestJWTAudiences_AcceptsHanzoTeam(t *testing.T) {
os.Unsetenv("CLOUD_JWT_AUDIENCES")
os.Unsetenv("GATEWAY_ALLOWED_AUDIENCES")
has := func(list []string, v string) bool {
for _, s := range list {
if s == v {
return true
}
}
return false
}
if !has(defaultJWTAudiences, "hanzo-team") {
t.Fatalf("defaultJWTAudiences must include hanzo-team (the hanzo.team client_id); got %v", defaultJWTAudiences)
}
if !has(jwtAudiencesFromEnv(), "hanzo-team") {
t.Fatalf("resolved JWT audiences must include hanzo-team; got %v", jwtAudiencesFromEnv())
}
}
-47
View File
@@ -1,47 +0,0 @@
// Copyright 2026 The Hanzo Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package cloud
import (
"os"
"testing"
)
// TestJWTAudiences_AcceptsHanzoWorld pins the world.hanzo.ai OIDC client. IAM
// mints world's access tokens with aud=hanzo-world (each app's aud is its
// client_id). Those bearers hit cloud-api; the identity sanitizer only trusts a
// principal whose aud is in this allowlist. If hanzo-world is not accepted the
// token resolves anonymous and the analyst's api.hanzo.ai calls 401. Pin the
// client_id into the baked default so the forwarded bearer validates.
func TestJWTAudiences_AcceptsHanzoWorld(t *testing.T) {
os.Unsetenv("CLOUD_JWT_AUDIENCES")
os.Unsetenv("GATEWAY_ALLOWED_AUDIENCES")
has := func(list []string, v string) bool {
for _, s := range list {
if s == v {
return true
}
}
return false
}
if !has(defaultJWTAudiences, "hanzo-world") {
t.Fatalf("defaultJWTAudiences must include hanzo-world (the world.hanzo.ai client_id); got %v", defaultJWTAudiences)
}
if !has(jwtAudiencesFromEnv(), "hanzo-world") {
t.Fatalf("resolved JWT audiences must include hanzo-world; got %v", jwtAudiencesFromEnv())
}
}
+10 -2
View File
@@ -8,13 +8,14 @@ package audit
import (
"context"
"database/sql"
"encoding/json"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"github.com/hanzoai/cloud/cek"
)
// openTemp opens a Recorder backed by a fresh on-disk SQLite file (not :memory:,
@@ -119,6 +120,7 @@ func TestVerify_PassesOnUntamperedChain(t *testing.T) {
// reports the exact seq where the chain breaks. THIS is the tamper-evidence
// property — an audit trail that can be silently forged is worse than none.
func TestVerify_DetectsFieldTamper(t *testing.T) {
requireSharedStore(t) // tampers through a second handle while the Recorder holds the store open
rec, path := openTemp(t)
ctx := context.Background()
for i := 0; i < 10; i++ {
@@ -154,6 +156,7 @@ func TestVerify_DetectsFieldTamper(t *testing.T) {
// the chain: the record after the hole has a PrevHash that no longer matches the
// now-preceding record, and the seq sequence gaps. Either way Verify flags it.
func TestVerify_DetectsDeletion(t *testing.T) {
requireSharedStore(t) // tampers through a second handle while the Recorder holds the store open
rec, path := openTemp(t)
ctx := context.Background()
for i := 0; i < 10; i++ {
@@ -182,6 +185,7 @@ func TestVerify_DetectsDeletion(t *testing.T) {
// TestVerify_DetectsReorder proves swapping two records' positions (an attacker
// trying to reorder events) breaks the prev-hash linkage.
func TestVerify_DetectsReorder(t *testing.T) {
requireSharedStore(t) // tampers through a second handle while the Recorder holds the store open
rec, path := openTemp(t)
ctx := context.Background()
for i := 0; i < 6; i++ {
@@ -612,7 +616,11 @@ func TestCheckpoint_CountMonotonicDetectsTruncation(t *testing.T) {
// unaffected; Verify then re-reads and must catch the damage.
func tamperOutOfBand(t *testing.T, path, stmt string) {
t.Helper()
db, err := sql.Open("sqlite", path)
// Opened through cek, like the Recorder itself: the store is encrypted at rest,
// so a bare sql.Open cannot read it. The modelled adversary is one with database
// access AND the key (an insider, or a compromised process) — file access alone
// no longer suffices, which is the point of encrypting it.
db, err := cek.Open(path)
if err != nil {
t.Fatalf("tamper open: %v", err)
}
+10 -2
View File
@@ -22,10 +22,13 @@ import (
"testing"
"time"
"github.com/hanzoai/cloud/cek"
_ "github.com/hanzoai/sqlite"
)
func TestShareability_ReaderSharesLiveWriterStore(t *testing.T) {
requireSharedStore(t)
dir := t.TempDir()
path := dir + "/audit.db"
@@ -52,9 +55,14 @@ func TestShareability_ReaderSharesLiveWriterStore(t *testing.T) {
// Reader: a SECOND, independent connection opened READ-ONLY against the SAME
// files while the writer appends. This is what a reader-role pod does.
ro, err := sql.Open("sqlite", "file:"+path+"?mode=ro")
// Opened through cek: the store is encrypted at rest, so a bare sql.Open has no
// key and cannot read it. cek has no read-only mode, so this is a second RW
// handle that only ever reads — it still proves the reader sees the live
// writer's committed records, which is the claim, but it does not by itself
// prove the reader takes no write lock.
ro, err := cek.Open(path)
if err != nil {
t.Fatalf("reader Open(ro): %v", err)
t.Fatalf("reader Open: %v", err)
}
defer ro.Close()
+27
View File
@@ -0,0 +1,27 @@
package audit
import (
"testing"
sqlitedrv "github.com/hanzoai/sqlite"
)
// requireSharedStore skips a test that needs a SECOND opener to observe the same
// store as the first.
//
// Without the live libsqlcipher codec, cek falls back to the pure-Go envelope:
// it decrypts the file into a handle-private RAM copy and seals it back on close.
// Two opens of one path therefore never see each other's writes and the last close
// wins — envelope.go states this outright ("SINGLE WRITER per file"). A test that
// tampers or reads through a second handle is asserting a property the envelope
// provably does not have, so it would fail for the wrong reason.
//
// The property IS real on the shipped build, which links the codec and keeps the
// database in place with per-commit durability and ordinary SQLite sharing. Note
// that `make test` runs CGO_ENABLED=0, so nothing in CI exercises that path today.
func requireSharedStore(t *testing.T) {
t.Helper()
if !sqlitedrv.CodecLinked() {
t.Skip("pure-Go envelope: a store is single-writer (handle-private RAM copy, sealed on close), so a second opener cannot observe it; this property belongs to the codec-linked build")
}
}
+11 -1
View File
@@ -175,12 +175,22 @@ func (k *iamKeys) lookup(ctx context.Context, key string) *idClaims {
if strings.TrimSpace(env.Data.Owner) == "" {
return nil
}
owner := strings.TrimSpace(env.Data.Owner)
return &idClaims{
Owner: strings.TrimSpace(env.Data.Owner),
Owner: owner,
Name: strings.TrimSpace(env.Data.Name),
PreferredUsername: strings.TrimSpace(env.Data.Name),
Email: strings.TrimSpace(env.Data.Email),
IsAdmin: env.Data.IsAdmin,
// The org came from the SUBJECT: IAM resolved this accessKey to a user row,
// and that row's owner is the tenant. No application mints it and no claim
// carries it, so it is NOT the app-selected value homeOrg exists to reject —
// it is the same "the organization comes from the token subject" rule IAM
// states for itself in internal/authz/authz.go. Recorded here so the identity
// boundary can tell a KEY principal (which legitimately has no `orgs`, because
// a machine is a member of nothing) from a HUMAN token that has merely lost
// its claim — the latter must still fail closed.
subjectOrg: owner,
}
}
+248 -70
View File
@@ -41,14 +41,35 @@ import (
type idClaims struct {
jwt.Claims
Owner string `json:"owner"` // org slug (the org)
Project string `json:"project"` // org SUB-SCOPE within owner (empty ⟹ default project)
BillingAccount string `json:"billing_account"` // WHO PAYS, stated by IAM (empty ⟹ pre-claim token)
Name string `json:"name"` // display name (id fallback)
PreferredUsername string `json:"preferred_username"` // id fallback
Email string `json:"email"`
IsAdmin bool `json:"isAdmin"`
Orgs []model.OrgRef `json:"orgs"` // membership SET (home first); empty on legacy tokens
Owner string `json:"owner"` // org slug (the org)
Project string `json:"project"` // org SUB-SCOPE within owner (empty ⟹ default project)
BillingAccount string `json:"billing_account"` // WHO PAYS, stated by IAM (empty ⟹ pre-claim token)
Name string `json:"name"` // display name (id fallback)
PreferredUsername string `json:"preferred_username"` // id fallback
Email string `json:"email"`
IsAdmin bool `json:"isAdmin"`
// Type is IAM's account kind: "application" for a client_credentials MACHINE
// identity (object/token_oauth.go stamps Type:"application"), else a human kind
// ("normal-user", …). It is the discriminator that keeps a machine token — of ANY
// app, not only the KMS-sync one — from ever being granted SuperAdmin. Empty on a
// token that predates the claim ⟹ treated as non-machine (fail toward the KMS-aud
// check below, never toward granting admin).
Type string `json:"type"`
Orgs []model.OrgRef `json:"orgs"` // membership SET (home first); empty on legacy tokens
// subjectOrg is the org resolved from the token SUBJECT rather than from any
// claim — set ONLY by the API-key resolver (iamKeys.lookup), from the IAM user
// row the accessKey belongs to. It is the machine-credential answer to "whose
// org is this?", and it exists because an hk-/sk- key is not a member of
// anything: IAM mints no `orgs` claim for one, so the membership set homeOrg
// reads for a human is legitimately empty here.
//
// UNEXPORTED AND UNTAGGED ON PURPOSE. encoding/json cannot populate it, so no
// token can carry it and no caller can forge it — it is only ever set by the
// code path that already authenticated the key against IAM. That is the same
// rule the identity headers follow: never decoded from the request, only minted
// from something verified.
subjectOrg string
}
// mintedProject returns the project id to stamp into X-Project-Id, or "" when the
@@ -100,11 +121,28 @@ func (c *idClaims) userID() string {
// the distinct-from-userID() value stamped as X-User-Name so the direct-Bearer
// path builds owner/name correctly — the gateway historically minted
// X-User-Id==name, which userID() (sub-first) breaks on the in-binary path.
// PREFERRED_USERNAME FIRST, and `name` only as a legacy fallback. The order used
// to be reversed on the belief that IAM's `name` claim carried IAM's canonical
// username. It does not: OIDC gives `name` DISPLAY semantics and IAM fills it from
// User.DisplayName, so a real token read `name = "Zach Kelling"` — a human label
// with a space in it, not the `<name>` half of `<owner>/<name>`.
//
// The cost landed on the money path, which addresses a wallet as `<org>/<username>`
// (clients/principal/wallet.go). Preferring `name` addressed `hanzo/Zach Kelling`,
// a wallet no funding path can name, while the balance sat in `hanzo/z`. Every
// signed-in completion 402'd against a funded account. That file already documents
// three prior recurrences of one bug — "two layers derived the same address two
// ways"; this is the same bug arriving through the claim rather than the header.
//
// Fallback is retained, not removed: a token minted before IAM emitted
// preferred_username has only `name`, and for those the old reading is still the
// best available answer. New tokens carry the username explicitly, so the fallback
// stops being reached as they roll over rather than needing a flag day.
func (c *idClaims) username() string {
if c.Name != "" {
return c.Name
if c.PreferredUsername != "" {
return c.PreferredUsername
}
return c.PreferredUsername
return c.Name
}
// jwtSigAlgs is the accepted signature-algorithm allowlist passed to
@@ -125,43 +163,46 @@ var jwtSigAlgs = []gojose.SignatureAlgorithm{
// (cert-hanzo/cert-lux/cert-zoo/...), keyed by the token kid, so a single
// jwksURL verifies all brands. Only the issuer-string comparison had to widen.
type identityValidator struct {
issuers []string
audiences []string
cache *jwksCache
keys keyResolver // resolves an opaque API key to a principal; nil ⟹ keys stay anonymous
issuers []string
cache *jwksCache
keys keyResolver // resolves an opaque API key to a principal; nil ⟹ keys stay anonymous
// claims memoizes token ⇒ verified claims so an already-authenticated caller
// (notably a ZAP socket, which replays its credential on every frame) is not
// signature-verified again per call. See identity_cache.go.
claims *identityCache
}
// newIdentityValidator builds a validator whose trusted-issuer set is the primary
// issuer UNIONED with every white-label brand issuer (BrandIssuers) plus any
// WHITELABEL_ISSUERS override. ttl<=0 uses the 15m JWKS default. The union is
// fail-secure: it only ADDS the known-good brand issuers, never an arbitrary one.
func newIdentityValidator(issuer, jwksURL string, audiences []string, ttl time.Duration) *identityValidator {
//
// Trust is IAM-native: signature (JWKS) + issuer (this set) + expiry. There is NO
// per-app audience allowlist — the `aud` (a minting app's client_id) is IAM's to
// assign, not cloud's to mirror, so a new first-party app needs zero cloud change.
func newIdentityValidator(issuer, jwksURL string, ttl time.Duration) *identityValidator {
return &identityValidator{
issuers: trustedIssuers(issuer),
audiences: audiences,
cache: newJWKSCache(jwksURL, ttl),
keys: sharedKeys(), // ONE resolver+cache, shared with OrgForKey (analytics capture)
issuers: trustedIssuers(issuer),
cache: newJWKSCache(jwksURL, ttl),
claims: newIdentityCache(),
keys: sharedKeys(), // ONE resolver+cache, shared with OrgForKey (analytics capture)
}
}
// kmsMachineAudSuffix is the fixed suffix of a per-org PaaS-KMS sync machine
// identity's audience. Each org's KMS sync authenticates as a dedicated,
// NON-shared IAM application named "<org>-platform-kms" (Organization=<org>,
// client_credentials grant), so IAM stamps the token's aud == the app's own
// clientId == "<org>-platform-kms" (a non-shared app's audience is its clientId,
// object/token_jwt.go tokenAudience) and owner == <org>
// (object/token_oauth.go GetClientCredentialsToken sets owner = app.Organization).
// identity's audience. Each org's KMS sync authenticates as a dedicated, NON-shared
// IAM application named "<org>-platform-kms" (Organization=<org>, client_credentials
// grant), so IAM stamps the token's aud == the app's own clientId == "<org>-platform-kms"
// (object/token_jwt.go tokenAudience) and owner == <org> (object/token_oauth.go
// GetClientCredentialsToken sets owner = app.Organization).
//
// That audience is, by construction, absent from CLOUD_JWT_AUDIENCES — it is
// per-org, not a fixed app — which is EXACTLY why the sync stayed pending: the
// machine token failed the audience check below, SanitizeIdentity treated it as
// anonymous, and the /v1/kms org-scope guard 403'd it before the store. The fix is
// to accept this one audience, but ONLY when it equals the token's OWN owner claim
// plus this suffix, so it certifies "the KMS sync identity for its own org" and
// grants nothing wider. Org-scoping is still enforced downstream by owner at the guard
// (owner == :org); this only lets a legitimately-minted, owner-scoped machine token
// clear validation. A per-org application means a per-org clientSecret — never
// a shared platform-wide reader, which would be a cross-org hole.
// Validation no longer consults the audience at all (trust is signature + issuer +
// expiry), so a machine token clears validate() like any other. This suffix survives
// for the OPPOSITE reason: to RECOGNISE a machine principal (isKMSMachinePrincipal) so
// SanitizeIdentity can DENY it SuperAdmin even when it carries owner==adminOrg — a
// client_credentials machine identity must never wield platform-admin. The match is
// bound to the token's OWN owner claim (<owner>-platform-kms), so it certifies "the
// KMS sync identity for its own org" and grants nothing wider.
const kmsMachineAudSuffix = "-platform-kms"
// kmsMachineAudience returns the audience an org org's PaaS-KMS sync identity
@@ -174,6 +215,74 @@ func kmsMachineAudience(owner string) string {
return owner + kmsMachineAudSuffix
}
// homeOrg returns the USER's own organization — the tenant whose ledger pays and
// whose membership decides platform authority. It reads the FIRST entry of the
// signed `orgs` claim, which IAM builds home-first by construction from the
// authoritative user row (store.MemberOrgRefs: `refs := []OrgRef{{Org: user.Owner,
// …}}`, then explicit membership rows, deduped home-wins).
//
// IT IS DELIBERATELY NOT claims.Owner. The `owner` claim has never carried the
// user's org: IAM stamps the APPLICATION's org into it (oidc/jwt.go Sign:
// `Owner: app.Organization`). Same user, same password, two apps ⇒ two different
// `owner` values. That read as correct for years only because, pre-onboarding, the
// app org and the user org were both "hanzo"; onboarding broke the coincidence, not
// the claim. IAM knew — internal/authz/authz.go refuses to trust these claims
// internally and says the organization "comes from the token SUBJECT … never from
// the token's `owner`/`organization` claims" — and the Sign/SignUserToken pair
// documents the divergence while naming cloud's SanitizeIdentity as the consumer.
//
// Consuming `owner` made the tenant CALLER-SELECTABLE: a user picked which org's
// ledger to spend by choosing which app to authenticate through (a hanzo user via
// lux-cloud billed lux), and — because the same value gated SuperAdmin — a token
// from any app owned by the reserved admin org conferred platform admin. One
// poisoned value, two defects; one accessor, both closed.
//
// TWO PRINCIPAL KINDS, TWO SOURCES — they are different questions, so they are two
// branches rather than one fallback chain:
//
// - A MACHINE credential (hk-/sk- API key) is a member of nothing, so IAM mints it
// no `orgs` claim at all. Its org comes from the token SUBJECT: iamKeys.lookup
// resolves the accessKey to its IAM user row and records that row's owner in
// subjectOrg. Reading it here is not a fallback to `owner` — it never passed
// through an application, so it carries none of the app-selection hazard.
//
// - A HUMAN token carries the membership set, and its first entry is the home org.
//
// EMPTY STILL MEANS EMPTY for a human. A token with neither (minted before IAM
// v1.33.0) resolves NO home org and the caller must fail closed — an org-less
// request, every org() gate 403. It must never fall back to `owner`, which is
// precisely the app-selected value this exists to stop trusting. Callers log that
// case so a real legacy principal is visible rather than silently denied.
//
// Order matters: subjectOrg is checked FIRST because a key principal's empty `orgs`
// is correct-by-design, not a degraded token. Reading the membership set first and
// failing closed on it is what 403'd every customer API key on org-scoped routes
// (/v1/agents, /v1/gpus, /v1/billing/*) in v1.801.244 while leaving unscoped
// /v1/models working — which is why the pre-pin probe, which only ever asserted
// /v1/models, could not see it.
// A MACHINE JWT (client_credentials, IAM `type` == "application" — e.g. the
// per-org "<org>-platform-kms" sync identity) is the third case, and it reads
// `owner` DELIBERATELY. That is not the hazard this function exists to remove: the
// hazard was a HUMAN whose org followed whichever app they logged in through, and a
// machine cannot choose — it IS the application, its `owner` is that application's
// own organization, and obtaining the token at all requires that app's client
// secret. There is no user to mis-attribute. Omitting this branch would fail closed
// on the KMS sync identity, whose org-scoped data access runs through this same
// boundary (see isKMSMachinePrincipal, which gates ONLY the admin grant precisely so
// that access keeps working).
func (c *idClaims) homeOrg() string {
if c.subjectOrg != "" {
return c.subjectOrg // API key: resolved from the subject
}
if isMachinePrincipal(c) {
return c.Owner // machine JWT: the app IS the principal
}
if len(c.Orgs) == 0 {
return "" // human token with no membership set: fail closed
}
return c.Orgs[0].Org
}
// isKMSMachinePrincipal reports whether a validated token is a per-org KMS-sync
// machine identity: its audience set contains the owner-bound machine audience
// (<owner>-platform-kms). Such a principal is a client_credentials machine identity
@@ -197,6 +306,44 @@ func isKMSMachinePrincipal(claims *idClaims) bool {
return false
}
// isMachinePrincipal reports whether a validated token is a MACHINE (non-human)
// identity — the predicate SanitizeIdentity uses to DENY SuperAdmin and the org-admin
// signal. A client_credentials token carries IAM's `type` == "application"
// (object/token_oauth.go), which catches EVERY machine app regardless of its audience;
// this is the fix for the audience-decomplection widening SuperAdmin's reach — a
// generic admin-org machine app must not become platform-admin just because it belongs
// to the admin org. It UNIONS the owner-bound KMS-sync check so a machine token is
// still excluded even on the (defensive) path where `type` is absent but the
// <owner>-platform-kms audience is present. Fail-closed: unknown/empty type is treated
// as NON-machine so a real human admin (who may carry no `type`) is never locked out —
// the KMS-aud fallback still catches the one machine family we can identify audience-only.
func isMachinePrincipal(claims *idClaims) bool {
return claims.Type == "application" || isKMSMachinePrincipal(claims)
}
// isMember reports whether org is in the token's signed membership set — the
// `orgs` claim IAM mints for a USER token, home org first. It is the ONE test that
// turns a client's org SELECTION into an effective org (SanitizeIdentity), and
// therefore into the ledger that pays (principal.BillingOrg).
//
// The comparison is VERBATIM, no folding, for the same reason the owner claim is
// taken verbatim: "acme" and "ACME" are DISTINCT orgs in IAM, and a fold would let
// a member of one select the other. An empty org is never a member, so an absent
// selection leaves the caller in their home org. An empty set (a legacy token, an
// opaque key, a machine principal — IAM never mints `orgs` for a client_credentials
// token) admits nothing, which is exactly the pre-claim behavior.
func isMember(orgs []model.OrgRef, org string) bool {
if org == "" {
return false
}
for _, o := range orgs {
if o.Org == org {
return true
}
}
return false
}
// validate parses raw, verifies its signature against the JWKS, and enforces
// issuer/audience/expiry. Returns the claims on success, an error otherwise.
func (v *identityValidator) validate(raw string) (*idClaims, error) {
@@ -214,14 +361,13 @@ func (v *identityValidator) validate(raw string) (*idClaims, error) {
return nil, err
}
// Fail SECURE on a misconfigured (empty) trust set: an empty issuer OR audience
// allowlist must REJECT every token, never silently disable that axis. In
// production both are always resolved non-empty (BrandIssuers + the baked
// audience defaults, unioned in config.go so they are "never empty"), so this
// fires ONLY on an operator misconfiguration (CLOUD_JWT_AUDIENCES="" emptying the
// resolved set, or an empty issuer set) — and then it denies, it never admits (I2).
if len(v.issuers) == 0 || len(v.audiences) == 0 {
return nil, fmt.Errorf("identity validator misconfigured: empty issuer or audience allowlist")
// Fail SECURE on a misconfigured (empty) trust set: with no trusted issuer every
// token must be REJECTED, never silently admitted. In production the set is always
// non-empty (the primary issuer + BrandIssuers, unioned in config.go so it is
// "never empty"), so this fires ONLY on an operator misconfiguration — and then it
// denies, it never admits (I2).
if len(v.issuers) == 0 {
return nil, fmt.Errorf("identity validator misconfigured: empty issuer set")
}
// Reject a missing issuer: an empty issuer must never pass the set check.
@@ -234,28 +380,22 @@ func (v *identityValidator) validate(raw string) (*idClaims, error) {
if claims.Expiry == nil {
return nil, fmt.Errorf("missing expiry")
}
// Issuer must be one of the trusted brand issuers. go-jose's jwt.Expected
// checks a SINGLE issuer, so the issuer is validated here against the set and
// left out of Expected (audience + expiry stay with Expected).
// Issuer must be one of the trusted brand issuers. go-jose's jwt.Expected checks a
// SINGLE issuer, so the issuer is validated here against the set and left out of
// Expected (only expiry/not-before stay with Expected).
if !issuerAllowed(claims.Issuer, v.issuers) {
return nil, fmt.Errorf("untrusted issuer %q", claims.Issuer)
}
// Audience: the static allowlist (CLOUD_JWT_AUDIENCES / brand app client_ids)
// PLUS the per-org PaaS-KMS sync machine audience bound to THIS token's own
// owner (<owner>-platform-kms). The machine audience is added only when the
// allowlist is active (non-empty — always so in production) and only for the
// token's own org, so accepting it never widens org-scoping: the /v1/kms guard still
// gates on owner == :org. Without this, a real client_credentials machine token
// (aud == its per-org clientId, never in the allowlist) fails here and the
// sync silently stays pending — the activation blocker.
// The audience allowlist is guaranteed non-empty (checked above), so the
// audience axis is ALWAYS enforced — never silently skipped.
auds := v.audiences
if mach := kmsMachineAudience(claims.Owner); mach != "" {
auds = append(append(make([]string, 0, len(v.audiences)+1), v.audiences...), mach)
}
expected := jwt.Expected{AnyAudience: jwt.Audience(auds)}
if err := claims.Claims.ValidateWithLeeway(expected, 2*time.Minute); err != nil {
// Audience is NOT an access gate. A valid signature from a trusted issuer (both
// checked above) proves IAM minted this token for one of ITS OWN registered apps;
// the `aud` merely names which app. Cloud does not keep a per-app allowlist to
// mirror IAM's registry — that mirror drifted and silently 401'd every new
// first-party app until hand-edited. Org scope is the `owner` claim, enforced by
// every downstream guard; SuperAdmin is owner==adminOrg AND !isKMSMachinePrincipal
// (SanitizeIdentity). Expiry + not-before are STILL enforced here: Expected{} with a
// zero Time validates against time.Now(); an empty AnyAudience skips ONLY the
// audience match (go-jose/v4 jwt/validation.go).
if err := claims.Claims.ValidateWithLeeway(jwt.Expected{}, 2*time.Minute); err != nil {
return nil, fmt.Errorf("claims: %w", err)
}
return &claims, nil
@@ -375,14 +515,52 @@ func (c *jwksCache) fetch() (*gojose.JSONWebKeySet, error) {
// Token extraction — mirrors iamauth's Bearer / Basic / API-key helpers.
// ----------------------------------------------------------------------------
// isAPIKey reports whether tok is an opaque, backend-validated key (hk-/sk-/…)
// rather than a JWT, so the sanitizer skips JWT parsing for it.
// APIKeyPrefixes are the Hanzo API key families. A published key (pk-) is
// write-only and scoped, so it is safe in a public bundle; a secret key (sk-)
// authenticates a server. Everything else is OAuth2, which is a JWT and not a key.
//
// hk- is sk- under an older name and is on its way out. It stays accepted here
// because IAM mints it — cloud only validates (see account.go mintKey, which
// delegates to iam.mintUserKey) — so dropping it here before IAM renames the
// family would reject every key IAM hands out. Retire it in that order: IAM mints
// sk-, holders re-key, then delete the entry below.
//
// fw_ and hz_ were listed here and never minted by anything: dead entries that
// widened what counts as a credential for no reason. Gone.
//
// This is the ONE authority. Admission mirrors it rather than importing it (it
// stays free of cloud-internal imports); if this list changes, that copy must too.
var APIKeyPrefixes = []string{"pk-", "sk-", "hk-"}
// PublishablePrefix is the ONE publishable spelling: pk- is the key you may ship
// in a browser bundle, sk- is the one you may not. Stripe's split, same reason.
const PublishablePrefix = "pk-"
// IsPublishableKey reports whether tok is a publishable key.
//
// A publishable key is NOT a credential: it identifies a tenant so a public
// surface can WRITE (ingest events), and it must never mint a principal that can
// READ. Cloud resolved any isAPIKey token — pk- included — into "the same
// principal a JWT yields", which made a key documented as "safe to show" into a
// full bearer for the org that owns it. IdentityFromRequest now refuses it, so
// publishable means publishable.
//
// It stays in APIKeyPrefixes on purpose: OrgForKey must still resolve a pk- to
// its owning org, because that is exactly how the ingest door learns which tenant
// a browser beacon belongs to. Resolvable, not authenticating.
func IsPublishableKey(tok string) bool {
return strings.HasPrefix(strings.TrimSpace(tok), PublishablePrefix)
}
// isAPIKey reports whether tok is an opaque, backend-validated key rather than a
// JWT, so the sanitizer skips JWT parsing for it.
func isAPIKey(tok string) bool {
return strings.HasPrefix(tok, "hk-") ||
strings.HasPrefix(tok, "sk-") ||
strings.HasPrefix(tok, "pk-") ||
strings.HasPrefix(tok, "fw_") ||
strings.HasPrefix(tok, "hz_")
for _, p := range APIKeyPrefixes {
if strings.HasPrefix(tok, p) {
return true
}
}
return false
}
// bearerFromAuth extracts the token from a "Bearer <token>" header value.
+52
View File
@@ -0,0 +1,52 @@
package cloud
import (
"crypto/rand"
"crypto/rsa"
"testing"
"time"
)
// TestValidate_AudienceIsNotAGate proves the IAM-native trust model: a token signed
// by a trusted issuer validates REGARDLESS of its `aud` (the minting app's client_id).
// Cloud keeps no per-app audience allowlist mirroring IAM's registry — so a brand-new
// first-party app works with zero cloud change, and the specific app tokens the old
// per-app tests pinned (admin-guard, world, team, commerce) are accepted by the SAME
// rule as everything else. Trust is signature + issuer + expiry; org scope is the
// owner claim, enforced downstream. Replaces the three audience_*_test.go files that
// asserted a static allowlist which no longer exists.
func TestValidate_AudienceIsNotAGate(t *testing.T) {
key, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
t.Fatalf("genkey: %v", err)
}
jwks := jwksServer(t, &key.PublicKey)
v := newIdentityValidator(testIssuer, jwks.URL, 0)
future := time.Now().Add(time.Hour)
// Every audience — the apps the deleted per-app tests pinned AND a never-registered
// one — validates identically, because aud is not an access gate.
for _, aud := range []string{
"hanzo-admin-guard", // admin.hanzo.ai forward-auth cockpit
"hanzo-world", // world.hanzo.ai analyst tokens
"hanzo-team", // hanzo.team wallet page
"hanzo-commerce", // commerce.hanzo.ai admin AI assistant
"a-brand-new-first-party-app-never-listed-anywhere",
} {
tok := signWith(t, key, tokenClaims(aud, "acme", "", false, future))
id, err := v.validate(tok)
if err != nil {
t.Fatalf("aud=%q from a trusted issuer must validate (no allowlist), got %v", aud, err)
}
if id.Owner != "acme" {
t.Errorf("aud=%q: owner must be carried through, got %q", aud, id.Owner)
}
}
// Expiry is STILL enforced (dropping the aud gate must not disable time checks):
// a token expired beyond the 2m leeway is rejected whatever its aud.
expired := signWith(t, key, tokenClaims("hanzo-commerce", "acme", "", false, time.Now().Add(-time.Hour)))
if _, err := v.validate(expired); err == nil {
t.Error("expired token must be REJECTED even though audience is no longer gated")
}
}
+49 -35
View File
@@ -1,15 +1,15 @@
package cloud
// V6 (the activation blocker) — the identity validator must accept a per-org
// PaaS-KMS sync machine token: a client_credentials JWT whose aud is the org's
// own IAM application clientId "<owner>-platform-kms" (a per-org value, NEVER in
// CLOUD_JWT_AUDIENCES) — but ONLY when that audience is bound to the token's OWN
// owner claim. Before the fix the machine token failed the audience check,
// SanitizeIdentity resolved anonymous, and the /v1/kms guard 403'd it, so the sync
// silently stayed pending. These are white-box unit tests of validate() itself;
// the end-to-end proof through SanitizeIdentity + the real guard lives in
// clients/kms (v6_aud_e2e_test.go). Reuses the jwksServer/signWith/tokenClaims
// helpers from middleware_identity_test.go (same package).
// The per-org PaaS-KMS sync identity authenticates as its own IAM application
// "<owner>-platform-kms" (client_credentials), so its token carries owner=<org> and
// aud=<owner>-platform-kms. Validation no longer gates on the audience at all (trust
// is signature + issuer + expiry), so a machine token clears validate() like any
// other. The owner-bound machine aud survives only to IDENTIFY such a principal
// (isKMSMachinePrincipal) so SanitizeIdentity can DENY it SuperAdmin even in the admin
// org — a client_credentials machine identity must never wield platform-admin. These
// are white-box unit tests of that identification; the end-to-end proof through
// SanitizeIdentity + the real guard lives in clients/kms (v6_aud_e2e_test.go). Reuses
// the jwksServer/signWith/tokenClaims helpers from middleware_identity_test.go.
import (
"crypto/rand"
@@ -18,19 +18,16 @@ import (
"time"
)
func TestIdentityValidator_KMSMachineAudience(t *testing.T) {
func TestIdentityValidator_KMSMachinePrincipal(t *testing.T) {
key, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
t.Fatalf("genkey: %v", err)
}
jwks := jwksServer(t, &key.PublicKey)
// The static allowlist deliberately contains NO *-platform-kms audience, so any
// acceptance below can come ONLY from the owner-bound machine-aud rule, not the
// allowlist — this is what makes it a fix and not a config workaround.
v := newIdentityValidator(testIssuer, jwks.URL, []string{"hanzo-console"}, 0)
v := newIdentityValidator(testIssuer, jwks.URL, 0)
future := time.Now().Add(time.Hour)
t.Run("machine token for its own org is accepted", func(t *testing.T) {
t.Run("own-org machine token validates and is recognised as a machine principal", func(t *testing.T) {
c, err := v.validate(signWith(t, key, tokenClaims("maxpower-platform-kms", "maxpower", "", false, future)))
if err != nil {
t.Fatalf("machine token rejected: %v", err)
@@ -38,34 +35,51 @@ func TestIdentityValidator_KMSMachineAudience(t *testing.T) {
if c.Owner != "maxpower" {
t.Fatalf("owner=%q, want maxpower", c.Owner)
}
})
t.Run("machine aud for a DIFFERENT org is rejected (owner-bound)", func(t *testing.T) {
// owner=maxpower but aud=acme-platform-kms: the accepted machine aud is bound
// to the token's OWN owner (maxpower-platform-kms), so this must fail — it is
// not a blanket "*-platform-kms" wildcard.
if _, err := v.validate(signWith(t, key, tokenClaims("acme-platform-kms", "maxpower", "", false, future))); err == nil {
t.Fatal("cross-org machine audience must be rejected (owner-bound)")
if !isKMSMachinePrincipal(c) {
t.Fatal("aud==<owner>-platform-kms must be recognised as a machine principal")
}
})
t.Run("arbitrary audience still rejected (fix is scoped, not a disable)", func(t *testing.T) {
if _, err := v.validate(signWith(t, key, tokenClaims("some-random-app", "maxpower", "", false, future))); err == nil {
t.Fatal("an arbitrary audience must still be rejected")
t.Run("admin-org machine token is recognised so SuperAdmin is denied", func(t *testing.T) {
c, err := v.validate(signWith(t, key, tokenClaims("admin-platform-kms", "admin", "", true, future)))
if err != nil {
t.Fatalf("admin machine token rejected: %v", err)
}
if !isKMSMachinePrincipal(c) {
t.Fatal("admin-org machine token must be recognised (SanitizeIdentity denies it SuperAdmin)")
}
})
t.Run("machine aud with empty owner is rejected (fail closed)", func(t *testing.T) {
// aud="-platform-kms" with owner="": kmsMachineAudience("")=="" so no machine
// audience is granted and the bare suffix is not in the allowlist.
if _, err := v.validate(signWith(t, key, tokenClaims("-platform-kms", "", "", false, future))); err == nil {
t.Fatal("machine aud with empty owner must be rejected")
t.Run("machine aud bound to a DIFFERENT org is not this owner's machine principal", func(t *testing.T) {
// owner=maxpower, aud=acme-platform-kms: the machine-principal match is bound to
// the token's OWN owner (maxpower-platform-kms), not a "*-platform-kms" wildcard.
// It validates (aud is not gated) and is owner-scoped to maxpower downstream.
c, err := v.validate(signWith(t, key, tokenClaims("acme-platform-kms", "maxpower", "", false, future)))
if err != nil {
t.Fatalf("token rejected: %v", err)
}
if isKMSMachinePrincipal(c) {
t.Fatal("a cross-org machine aud must not count as this owner's machine principal")
}
})
t.Run("normal static-allowlist token still accepted (regression)", func(t *testing.T) {
if _, err := v.validate(signWith(t, key, tokenClaims("hanzo-console", "maxpower", "", false, future))); err != nil {
t.Fatalf("static-allowlist token rejected: %v", err)
t.Run("ordinary app token is not a machine principal", func(t *testing.T) {
c, err := v.validate(signWith(t, key, tokenClaims("hanzo-console", "maxpower", "", false, future)))
if err != nil {
t.Fatalf("token rejected: %v", err)
}
if isKMSMachinePrincipal(c) {
t.Fatal("an ordinary app token is not a machine principal")
}
})
t.Run("empty-owner token is never a machine principal (fail closed)", func(t *testing.T) {
c, err := v.validate(signWith(t, key, tokenClaims("-platform-kms", "", "", false, future)))
if err != nil {
t.Fatalf("token rejected: %v", err)
}
if isKMSMachinePrincipal(c) {
t.Fatal(`empty-owner token must never be a machine principal (kmsMachineAudience("")=="")`)
}
})
+36
View File
@@ -0,0 +1,36 @@
// Copyright 2026 Hanzo AI, Inc. All rights reserved.
package cloud
import (
"strings"
"testing"
)
// TestUsernamePrefersPreferredUsername pins the claim precedence the money path
// depends on. OIDC `name` is a DISPLAY name — IAM fills it from User.DisplayName,
// and a real token carried "Zach Kelling". Reading it as the username addressed
// wallet `hanzo/Zach Kelling`, which no funding path can name, while the balance
// sat in `hanzo/z`; every signed-in completion 402'd against a funded account.
func TestUsernamePrefersPreferredUsername(t *testing.T) {
// Both present: the username wins, never the human label.
c := &idClaims{Name: "Zach Kelling", PreferredUsername: "z"}
if got := c.username(); got != "z" {
t.Fatalf("username() = %q; want %q (preferred_username must win over the display name)", got, "z")
}
// A display name must never be returned when the username is available, and a
// space is the tell that a display name leaked into an account key.
if strings.ContainsRune(c.username(), ' ') {
t.Fatalf("username() = %q; an account key can never contain a space", c.username())
}
// Legacy token minted before IAM emitted preferred_username: `name` is all
// there is, so it stays the answer rather than becoming empty.
legacy := &idClaims{Name: "z"}
if got := legacy.username(); got != "z" {
t.Fatalf("legacy username() = %q; want %q (fallback must be retained)", got, "z")
}
// Neither present: empty, never a guess.
if got := (&idClaims{}).username(); got != "" {
t.Fatalf("empty username() = %q; want \"\"", got)
}
}
+8 -74
View File
@@ -9,10 +9,10 @@ import (
)
// TestValidate_FailSecureOnEmptyTrustSet proves I2: a validator whose resolved
// issuer OR audience allowlist is empty REJECTS an otherwise-valid, correctly
// signed token — the axis is never silently disabled. Production always resolves
// non-empty sets; this guards the misconfiguration path (CLOUD_JWT_AUDIENCES=""
// or an empty issuer set), which must fail closed, not open.
// issuer set is empty REJECTS an otherwise-valid, correctly signed token — the axis
// is never silently disabled. Production always resolves a non-empty set; this
// guards the misconfiguration path (an empty issuer set), which must fail closed,
// not open.
func TestValidate_FailSecureOnEmptyTrustSet(t *testing.T) {
key, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
@@ -23,20 +23,15 @@ func TestValidate_FailSecureOnEmptyTrustSet(t *testing.T) {
tok := signWith(t, key, tokenClaims("hanzo-console", "acme", "", false, future))
// Sanity: a properly configured validator accepts the token.
if _, err := newIdentityValidator(testIssuer, jwks.URL, []string{"hanzo-console"}, 0).validate(tok); err != nil {
if _, err := newIdentityValidator(testIssuer, jwks.URL, 0).validate(tok); err != nil {
t.Fatalf("baseline valid token must be accepted, got %v", err)
}
// Empty audience set → deny.
if _, err := newIdentityValidator(testIssuer, jwks.URL, nil, 0).validate(tok); err == nil {
t.Error("empty audience allowlist must REJECT (fail-secure), not accept")
}
// Empty issuer set → deny (construct directly; trustedIssuers never yields empty
// with a primary, so bypass it to exercise the guard).
vEmptyIss := &identityValidator{issuers: nil, audiences: []string{"hanzo-console"}, cache: newJWKSCache(jwks.URL, 0), keys: newIAMKeys()}
vEmptyIss := &identityValidator{issuers: nil, cache: newJWKSCache(jwks.URL, 0), keys: newIAMKeys()}
if _, err := vEmptyIss.validate(tok); err == nil {
t.Error("empty issuer allowlist must REJECT (fail-secure), not accept")
t.Error("empty issuer set must REJECT (fail-secure), not accept")
}
}
@@ -112,7 +107,7 @@ func TestBrandIssuers(t *testing.T) {
// full brand set, so a lux token would pass the issuer gate on the hanzo binary.
func TestNewIdentityValidator_MultiIssuer(t *testing.T) {
os.Unsetenv("WHITELABEL_ISSUERS")
v := newIdentityValidator("https://hanzo.id", "http://iam.hanzo.svc/v1/iam/.well-known/jwks", []string{"hanzo-cloud", "lux-cloud"}, 0)
v := newIdentityValidator("https://hanzo.id", "http://iam.hanzo.svc/v1/iam/.well-known/jwks", 0)
if !issuerAllowed("https://lux.id", v.issuers) {
t.Fatalf("validator must trust the lux issuer, set=%v", v.issuers)
}
@@ -123,64 +118,3 @@ func TestNewIdentityValidator_MultiIssuer(t *testing.T) {
t.Fatalf("validator must reject an untrusted issuer, set=%v", v.issuers)
}
}
// TestBrandAudiences proves every brand's cloud audience (<brand>-cloud) is derived
// from the brands registry — one source of truth, mirroring BrandIssuers.
func TestBrandAudiences(t *testing.T) {
got := BrandAudiences()
for _, want := range []string{"hanzo-cloud", "lux-cloud", "zoo-cloud", "pars-cloud", "bootnode-cloud"} {
found := false
for _, g := range got {
if g == want {
found = true
break
}
}
if !found {
t.Errorf("BrandAudiences()=%v missing %q", got, want)
}
}
}
// TestJWTAudiencesFromEnv_BrandUnion proves the resolved audience allowlist ALWAYS
// includes every brand's <brand>-cloud aud (so a lux token validates), whether the
// list comes from the baked default or a hanzo-only env override — and that an
// env-supplied entry is not duplicated.
func TestJWTAudiencesFromEnv_BrandUnion(t *testing.T) {
has := func(list []string, v string) bool {
for _, s := range list {
if s == v {
return true
}
}
return false
}
// Baked default path (no env).
os.Unsetenv("CLOUD_JWT_AUDIENCES")
os.Unsetenv("GATEWAY_ALLOWED_AUDIENCES")
def := jwtAudiencesFromEnv()
for _, want := range []string{"hanzo-cloud", "lux-cloud", "zoo-cloud", "pars-cloud"} {
if !has(def, want) {
t.Errorf("baked audiences %v missing brand aud %q", def, want)
}
}
// A legacy hanzo-only env override must STILL accept lux-cloud (brand union),
// with no duplicate of the env-supplied hanzo-cloud.
os.Setenv("GATEWAY_ALLOWED_AUDIENCES", "hanzo-app,hanzo-console,hanzo-cloud")
defer os.Unsetenv("GATEWAY_ALLOWED_AUDIENCES")
got := jwtAudiencesFromEnv()
if !has(got, "lux-cloud") {
t.Fatalf("hanzo-only env override must still accept lux-cloud, got %v", got)
}
n := 0
for _, s := range got {
if s == "hanzo-cloud" {
n++
}
}
if n != 1 {
t.Fatalf("hanzo-cloud must appear exactly once (no duplicate), got %d in %v", n, got)
}
}
-16
View File
@@ -137,19 +137,3 @@ func BrandIssuers() []string {
}
return out
}
// BrandAudiences returns the OAuth `aud` (== IAM client_id == app name) of every
// white-label brand's cloud login app: `<brand>-cloud` (hanzo-cloud, lux-cloud,
// zoo-cloud, pars-cloud, bootnode-cloud). A brand's session token carries
// aud=<brand>-cloud (HIP-0111: client_id == app == aud), so the audience allowlist
// must include each to accept a lux/zoo/pars token on the ONE binary. Derived from
// the same `brands` registry as BrandIssuers — one source of truth, no hand-listing.
func BrandAudiences() []string {
out := make([]string, 0, len(brands))
for id := range brands {
if id != "" {
out = append(out, id+"-cloud")
}
}
return out
}
+311 -36
View File
@@ -2,13 +2,20 @@ package cloud
import (
"context"
"database/sql"
"encoding/base64"
"fmt"
"os"
"strings"
"time"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud/clients/commerceinproc"
"github.com/hanzoai/cloud/cek"
"github.com/hanzoai/cloud/clients/commerce/transport"
"github.com/hanzoai/cloud/clients/metering"
"github.com/hanzoai/cloud/internal/org"
"github.com/hanzoai/ha"
s3 "github.com/hanzoai/s3-go"
sqlitedrv "github.com/hanzoai/sqlite"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
@@ -63,14 +70,16 @@ func BuildDeps(cfg *Config) Deps {
)
deps := Deps{
Logger: logger,
Brand: cfg.Brand,
Version: cfg.Version,
Env: cfg.Env,
Domain: cfg.Domain,
IAMIssuer: cfg.IAMIssuer,
DataDir: cfg.DataDir,
AIDefaultModel: cfg.AIDefaultModel,
Logger: logger,
Brand: cfg.Brand,
Version: cfg.Version,
Env: cfg.Env,
Domain: cfg.Domain,
IAMIssuer: cfg.IAMIssuer,
DataDir: cfg.DataDir,
MasterKey: masterKeyBytes(cfg),
AIDefaultModel: cfg.AIDefaultModel,
AIFallbackModel: cfg.AIFallbackModel,
}
// For each subsystem: enabled → leave nil (Mount fills it); not
@@ -102,6 +111,7 @@ func BuildDeps(cfg *Config) Deps {
deps.O11y = pick(cfg, logger, "o11y", "O11y", cfg.O11yZAPAddr, clients.O11yRPCAt, clients.DisabledO11y)
deps.VFS = pickVFSClient(cfg, logger)
deps.MQ = pick(cfg, logger, "mq", "MQ", cfg.MQZAPAddr, clients.MQRPCAt, clients.DisabledMQ)
deps.Durable, deps.LiveMembers = buildDurability(cfg, logger)
// Payments and Vault never co-resident. Disabled stub when no
// endpoint, otherwise RPC.
@@ -141,8 +151,8 @@ func staticEdgePolicy(cfg *Config) edge.Policy {
// the request-edge debit on every paid AI call.
//
// CO-RESIDENT (task #111): when commerce is folded in-process (Enabled("commerce")),
// the gate DEBITS the in-process commerce handler over commerceinproc's self-routing
// transport — a direct Go call, no socket to commerce.hanzo.svc:8001. The base is
// the gate DEBITS the in-process commerce handler over the commerce transport's
// self-routing dispatch — a direct Go call, no socket to commerce.hanzo.svc:8001. The base is
// pinned NON-EMPTY (real CLOUD_COMMERCE_HTTP_URL, else the in-process placeholder) so
// the gate stays ENABLED even after the standalone + its env are retired — a metering
// gate that silently no-ops is a free-money hole, so it must never drop to
@@ -159,9 +169,9 @@ func buildMeteringClient(cfg *Config, log luxlog.Logger) *metering.Client {
inProcess := cfg.Enabled("commerce")
if inProcess {
if base == "" {
base = commerceinproc.PlaceholderBase
base = transport.PlaceholderBase
}
httpClient = commerceinproc.Client(0) // in-process dispatch; no network timeout
httpClient = transport.Client(0) // in-process dispatch; no network timeout
}
m, err := metering.New(metering.Config{
BaseURL: base,
@@ -204,7 +214,7 @@ func boolStr(b bool, t, f string) string {
// wireTierReader installs the embedded ai module's per-tier SKU gate reader so it
// resolves the caller's commerce subscription tier through the SAME co-resident
// commerce client the metering gate bills over — in-process (commerceinproc) when
// commerce client the metering gate bills over — in-process (the commerce transport) when
// commerce is folded in, S2S HTTP with the service token otherwise — NEVER an authed
// self-call to the cloud edge. That self-call is the toothless-gate bug: the edge
// 401/403s a service call to /v1/billing/*, so the ai module's own HTTP lookup always
@@ -218,9 +228,9 @@ func wireTierReader(m *metering.Client, log luxlog.Logger) {
if m == nil || !m.Enabled() {
return
}
aiobject.SetTierReader(func(ctx context.Context, subject, namespace string) (string, error) {
tierReader = func(ctx context.Context, subject, namespace string) (string, error) {
return m.Tier(ctx, subject, namespace)
})
}
log.Info("ai per-tier SKU gate wired to co-resident commerce (in-process tier read, fail-safe)")
}
@@ -256,16 +266,16 @@ func wireFinance(cfg *Config, log luxlog.Logger) {
// the SAME wallet, or spend can outrun the balance that admitted it. Both use
// subject; keep them together. The gate reads a coarse cents balance (a >0
// threshold only); the DEBIT is 18-decimal-exact.
aiobject.SetBalanceReader(func(ctx context.Context, subject, namespace, currency string) (int64, error) {
balanceReader = func(ctx context.Context, subject, namespace, currency string) (int64, error) {
bal, err := fin.Balance(ctx, namespace, subject, currency, false)
if err != nil {
return 0, err
}
return bal.Cents(), nil
})
}
// The DEBIT is exact: the ai module emits the cost as a decimal-USD string, parsed
// here to 18-decimal USD (1e-18) so a sub-cent call bills precisely and is never floored.
aiobject.SetUsageRecorder(func(ctx context.Context, u aiobject.UsageEvent) error {
usageRecorder = func(ctx context.Context, u UsageEvent) error {
amt, err := money.ParseUSD(u.USD)
if err != nil {
return err
@@ -274,7 +284,7 @@ func wireFinance(cfg *Config, log luxlog.Logger) {
Org: u.Namespace, Subject: u.Subject, Amount: amt,
Currency: u.Currency, Model: u.Model, Provider: u.Provider, RequestID: u.RequestID,
})
})
}
log.Info("finance ledger wired (per-subject wallet in the org ledger, 18-decimal-exact, fail-closed)", "dataDir", cfg.DataDir)
}
@@ -548,7 +558,7 @@ func EmitLifecycle(ctx context.Context, ev LifecycleEvent) {
// call that reads the embedded commerce datastore (hanzoai/commerce MODULE, since
// the un-fork) + the @hanzo/plans vocabulary, no network hop (the HIP-0106
// co-resident default). The factory inversion stays because the concrete client
// (clients/commerceinproc) imports clients/plan, which imports cloud — a direct
// (clients/commerce) imports clients/plan, which imports cloud — a direct
// call here would be a package cycle. Absent the registration it fails closed
// rather than pretending.
//
@@ -575,7 +585,7 @@ func pickCommerceClient(cfg *Config, log luxlog.Logger) CommerceClient {
// commerceClientFactory constructs the embedded in-process Commerce client.
// apps/commerce.go registers it in init(); pickCommerceClient calls it so
// package cloud depends on the CommerceClient interface + this hook, never the
// concrete commerceinproc package (whose entitlement client pulls clients/plan,
// concrete clients/commerce package (whose entitlement client pulls clients/plan,
// which imports cloud — the hook is what keeps the package graph acyclic).
var commerceClientFactory func(cfg *Config, log luxlog.Logger) CommerceClient
@@ -712,6 +722,226 @@ func pickVFSClient(cfg *Config, log luxlog.Logger) VFSClient {
return clients.DisabledVFS()
}
// durableBucket holds every org's HA-SQLite snapshot (and its writer lease). One
// bucket, keys laid out orgs/<slug>[/…]/<subsystem>.db per HIP-0302 — the durable
// twin of the on-disk DataDir layout.
//
// OPERATIONAL REQUIREMENTS the fence depends on (enforce in the SeaweedFS deployment,
// not in code):
//
// - Object versioning + a no-expiry / no-lifecycle-deletion policy on this prefix.
// The writer lease (orgs/<slug>/.owner) is the round's system of record; if the
// gateway silently drops or rolls back that object, a monotone round can reset and
// un-fence a zombie writer (Red M4). A local high-water-round floor per pod is the
// future in-process defense; the object lifecycle is the operational one.
// - RWO, per-writer PVCs for DataDir — NEVER an RWX shared volume (Red M5). Two pods
// on one DataDir corrupt SQLite regardless of this fence; single-writer here is the
// durable-copy fence, and per-pod RWO is the local-file guarantee the shard router
// already relies on (see shardrouter.go).
const durableBucket = "org-db"
// durableProbePrefix namespaces the boot CAS-atomicity probe's throwaway objects
// (org.ProbeCAS writes one per boot) away from the orgs/ tree. A bucket lifecycle rule
// may reap ".probe/*"; the objects are tiny and never read after the probe.
const durableProbePrefix = ".probe/cas-"
// buildDurability constructs the deployment's HA-durability factory, or nil when the
// deployment has no object store to be durable against (dev/single-node — every
// OrgStore then stays local-only). It composes the SeaweedFS S3 If-Match
// ConditionalStore (the SAME s3admin identity deps.VFS uses), the writer membership
// over CLOUD_PEERS (the SAME set the shard router elects on, so the store-layer owner
// and the routed owner agree), and the per-org envelope Cipher rooted at the KMS
// master. Any construction failure fails SAFE to nil (local-only) rather than crash
// the boot; an encryption-capable build with no usable cipher is REFUSED — a build
// that promises encryption never ships plaintext snapshots to the object store.
// It returns the durable factory AND the live-members reader for the shard router (the
// SAME election snapshot), non-nil together only when the plane is active; both nil when
// local-only (the router then stays on the static ordinal set).
func buildDurability(cfg *Config, log luxlog.Logger) (*Durability, func() []ha.Member) {
// A multi-replica deployment REQUIRES the durable plane: with >1 writer, a per-org
// store that is not hydrate-on-open + fenced is the outage this exists to fix.
// disabledDurability logs at the severity the replica count warrants, so a
// misconfigured prod deployment is never SILENTLY non-durable (Red L2).
multiReplica := len(parsePeers(cfg.ShardPeers)) > 1
// Durability is THE path — there is no operator toggle. It self-detects capability
// at boot: no object store reachable (dev / native-Go / no S3 creds) → local-only,
// same code path, graceful; a reachable store → PROVE its conditional-PUT atomicity
// (org.ProbeCAS) before fencing a single byte of tenant data. A store that cannot be
// proven atomic fails SAFE to local-only with a loud alert — never a silent-wrong
// fence on a store that could split-brain.
admin := s3admin.New()
if !admin.Configured() {
disabledDurability(log, multiReplica, "no S3 admin creds (S3_ADMIN_* unset)")
return nil, nil
}
client, err := admin.Client()
if err != nil {
disabledDurability(log, multiReplica, fmt.Sprintf("S3 client construction failed: %v", err))
return nil, nil
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
if err := ensureDurableBucket(ctx, admin, client); err != nil {
// Non-fatal: the bucket likely already exists; a later ship/hydrate retries.
log.Warn("durability bucket ensure failed (continuing)", "bucket", durableBucket, "err", err)
}
cancel()
// Prove the store enforces conditional-PUT atomically BEFORE fencing any tenant data
// (the auto-H2 self-check that replaces the old opt-in flag). A store that cannot be
// proven atomic fails SAFE to local-only + a loud alert — never a silent fence on a
// store that could admit two writers for one round (split-brain). The result is
// cached for the process life (deps.Durable is set once), so this probe runs once.
// cond is the linearizable register the fence stands on, constructed HERE (not
// hard-wired inside the fence) so a cache tier slots in as a decorator: a
// read-through/write-through KV-in-front-of-S3 store (github.com/hanzoai/kv-go) can
// wrap this one line to serve low-latency hydrate reads while the authoritative CAS
// still lands on S3 — the fence reads and CASes through whatever ConditionalStore it
// is handed. (Safety note for that tier: a stale cached lease read only costs a claim
// retry, never safety — the S3 CAS is authoritative — so a write-through cache is
// sound over the SAME cond that backs both the lease and the data ships.)
cond := org.NewS3ConditionalStore(client, durableBucket)
probeCtx, probeCancel := context.WithTimeout(context.Background(), 15*time.Second)
err = org.ProbeCAS(probeCtx, cond, durableProbePrefix)
probeCancel()
if err != nil {
disabledDurability(log, multiReplica, fmt.Sprintf("object-store conditional-PUT atomicity NOT confirmed — %v", err))
return nil, nil
}
// Membership: LIVE when in-cluster + CLOUD_PEER_SELECTOR is set (a rolling upgrade's
// changing pod set is tracked, a draining/dead pod is never elected an org's owner),
// else the STATIC CLOUD_PEERS/self set — capability-detected, no flag (see
// membership_k8s.go). A single-pod deployment with no peers is its own sole writer.
// The 2s refresh keeps a drained pod out of every peer's election within a bound the
// terminationGracePeriod covers, so a rolling handoff loses no request.
self := firstNonEmptyStr(strings.TrimSpace(cfg.ShardSelf), hostnameOr("cloud-0"))
peers := parsePeers(cfg.ShardPeers)
if len(peers) == 0 {
peers = []org.Member{{ID: self, Addr: self}}
}
src := membershipSource(peers, cfg.PeerSelector, httpPortOf(cfg.ListenAddr), log)
members := org.NewMembership(self, src, 2*time.Second)
_ = members.Start(context.Background()) // initial refresh populates Members() before first request
cipher := durableCipher(cfg, log)
if cipher == nil && cek.Encrypting() {
// The master that satisfied cek must decode here too, so this is a genuine
// misconfig, not a dev path: never ship plaintext snapshots AND never silently
// drop durability — fail closed and log LOUDLY for the replica count.
disabledDurability(log, multiReplica, "encryption-capable build but no durable cipher (would ship plaintext snapshots)")
return nil, nil
}
log.Info("durability enabled", "bucket", durableBucket, "self", self, "peers", len(peers), "atomic_cas", true, "encrypted", cipher != nil)
// members.Members is the live election snapshot; hand it to the shard router so it
// routes on the SAME set the fencer elects over — the store-layer owner and the routed
// owner never disagree. WithCheckpoint WIRES the ship checkpoint (durableCheckpoint) so
// ship-before-ack folds the WAL into the real path before reading it — the crypto
// envelope's re-encrypt integration point (P5).
return org.NewDurability(cond, members, cipher, org.WithCheckpoint(durableCheckpoint)), members.Members
}
// durableCheckpoint makes the real on-disk file reflect every committed write before a
// durable ship reads it — the checkpoint the codec runs before snapshotting (WithCheckpoint).
// Two steps, correct on every backend:
//
// 1. A TRUNCATE checkpoint with the busy fail-closed guard: busy!=0 means a reader held the
// WAL so the main file is missing committed frames, and shipping it would silently lose an
// acked write. This folds the WAL and — on the WRITE-time-encrypting backends (cgo
// libsqlcipher page-level, and plaintext) — leaves the real path already fresh.
// 2. sqlitedrv.Checkpoint re-encrypts the pure-Go ENVELOPE backend's real path. The envelope
// defers encryption to Checkpoint/Close, so after step 1 the real path is STALE ciphertext
// until re-encrypted; without this the ship reads stale bytes and loses acked writes on
// takeover (the envelope backend landed in hanzoai/sqlite v0.4.0 — every pure-Go and
// mislinked-cgo keyed open routes through it). It is a successful no-op on the write-time
// backends, so it runs unconditionally.
//
// Step 1's connection is released before step 2 so the envelope re-encrypt sees a clean handle.
func durableCheckpoint(ctx context.Context, db *sql.DB) error {
if err := walCheckpointTruncate(ctx, db); err != nil {
return err
}
if err := sqlitedrv.Checkpoint(db); err != nil {
return fmt.Errorf("durable checkpoint re-encrypt: %w", err)
}
return nil
}
// walCheckpointTruncate folds the WAL into the main file with the busy fail-closed guard,
// on its own connection (released on return, before the envelope re-encrypt).
func walCheckpointTruncate(ctx context.Context, db *sql.DB) error {
conn, err := db.Conn(ctx)
if err != nil {
return fmt.Errorf("durable checkpoint conn: %w", err)
}
defer conn.Close()
var busy, logFrames, checkpointed int
if err := conn.QueryRowContext(ctx, "PRAGMA wal_checkpoint(TRUNCATE)").Scan(&busy, &logFrames, &checkpointed); err != nil {
return fmt.Errorf("durable checkpoint: %w", err)
}
if busy != 0 {
return fmt.Errorf("durable checkpoint did not complete (busy=%d, log=%d, checkpointed=%d) — refusing to ship a snapshot missing committed WAL frames", busy, logFrames, checkpointed)
}
return nil
}
// disabledDurability records that the durable plane is OFF, at ERROR when the
// deployment is multi-replica (per-org stores then survive only via shard routing +
// per-pod RWO PVC; a lost/rescheduled PVC loses committed data — the operator MUST
// configure S3) and at INFO when single-replica/dev (local-only is the expected
// posture). One place, so no disabled path is silent on a deployment that needs HA.
func disabledDurability(log luxlog.Logger, multiReplica bool, why string) {
if multiReplica {
log.Error("DURABILITY DISABLED on a MULTI-REPLICA deployment — per-org stores survive only via shard routing + per-pod RWO PVC; a lost/rescheduled PVC loses committed data. Configure S3_ADMIN_* to enable the durable plane.", "reason", why)
return
}
log.Info("durability disabled — single-replica/dev, per-org stores stay local-only", "reason", why)
}
// ensureDurableBucket creates the durable bucket if absent (idempotent).
func ensureDurableBucket(ctx context.Context, admin s3admin.Admin, client *s3.Client) error {
ok, err := client.BucketExists(ctx, durableBucket)
if err != nil {
return err
}
if ok {
return nil
}
return client.MakeBucket(ctx, durableBucket, s3.MakeBucketOptions{Region: admin.Region()})
}
// durableCipher builds the per-org envelope Cipher from the base64 KMS master
// (CLOUD_KMS_MASTER_KEY_REF — the SAME key cek derives from). nil when no valid
// 32-byte master is configured (pure-Go dev → plaintext durable object, matching the
// plaintext local file).
func durableCipher(cfg *Config, log luxlog.Logger) *org.Cipher {
ref := strings.TrimSpace(cfg.KMSMasterKeyRef)
if ref == "" {
return nil
}
master, err := base64.StdEncoding.DecodeString(ref)
if err != nil {
log.Warn("durable cipher: KMS master key ref is not valid base64", "err", err)
return nil
}
c, err := org.NewCipher(master)
if err != nil {
log.Warn("durable cipher: invalid KMS master key", "err", err)
return nil
}
return c
}
// hostnameOr returns the OS hostname, or def when unavailable — a stable self id for
// a single-pod deployment that sets no CLOUD_POD_NAME.
func hostnameOr(def string) string {
if h, err := os.Hostname(); err == nil && h != "" {
return h
}
return def
}
func pickPaymentsClient(cfg *Config, log luxlog.Logger) PaymentsClient {
if cfg.PaymentsZAPAddr != "" {
log.Info("deps.Payments → ZAP RPC", "addr", cfg.PaymentsZAPAddr)
@@ -732,13 +962,12 @@ func pickVaultClient(cfg *Config, log luxlog.Logger) VaultClient {
// deps for everything shared. Every subsystem in the fleet exports exactly this
// signature, so Wire references each one directly and the compiler checks it.
//
// app was once `any`, on the stated grounds that an external module (licensing)
// exposed func(any, Deps) error and narrowing would break it — while licensing
// said it used `any` to avoid an import cycle in pkg/cloud. Each cited the other,
// and the cycle could not exist: this package already imports zip, and zip does
// not import cloud. The `any` bought nothing and cost every subsystem a Typed()
// wrapper plus a runtime type assertion whose failure branch was unreachable.
type MountFunc func(app *zip.App, deps Deps) error
// app is a Router, not the concrete *zip.App, and that is the whole safety
// property: middleware a subsystem installs lands on the subtrees its AppSpec
// declares, never over the binary. Routes register exactly as before — absolute
// paths, same precedence. See scope.go. A subsystem that genuinely gates
// everything says so with Global: true and gets the bare app.
type MountFunc func(app Router, deps Deps) error
// ShutdownFunc releases a subsystem's process-lifetime resources (background
// goroutines, open DB handles) on graceful shutdown. It must be idempotent and
@@ -746,11 +975,11 @@ type MountFunc func(app *zip.App, deps Deps) error
// deadline so a slow teardown is cut off rather than hanging SIGTERM.
type ShutdownFunc func(ctx context.Context) error
// MountSpec describes one subsystem to mount. There is NO Order field: the slice
// AppSpec describes one subsystem to mount. There is NO Order field: the slice
// position in apps.Wire() IS the mount order — the composition root lists
// subsystems in the exact sequence they mount (and, reversed, tear down), so order
// is data read top-to-bottom in one file, not ints scattered across the tree.
type MountSpec struct {
type AppSpec struct {
Name string
Mount MountFunc
Shutdown ShutdownFunc // optional; nil means the subsystem has nothing to tear down.
@@ -759,11 +988,30 @@ type MountSpec struct {
// (a real, fail-closed probe). Serve's generic liveness loop skips these so
// its always-ok route never shadows the subsystem's real probe.
OwnsHealth bool
// Prefixes are the route subtrees whose middleware this subsystem may install.
// Empty means the convention it already follows — /v1/<Name>, the same subtree
// Serve's generic liveness route assumes — so only a subsystem that gates
// something else has to name it. It bounds MIDDLEWARE, not route registration:
// routes still register at absolute paths anywhere, as they always have.
Prefixes []string
// Global says this subsystem gates the whole binary and receives the bare
// *zip.App. It is the one way to reach app-wide middleware, so every grant is a
// decision someone made in writing, and apps.TestWireFrozen fails on a new one.
// Today it is held only by linked modules whose own Mount still takes *zip.App
// (see cloud.Global) — none of which installs middleware.
Global bool
}
// MountAll mounts every ENABLED subsystem in specs, in slice order — the order is
// the composition root's (apps.Wire()); MountAll does NOT sort. app is the
// concrete *zip.App from Serve, handed to each MountFunc as itself.
// the composition root's (apps.Wire()); MountAll does NOT sort.
//
// app is the concrete *zip.App from Serve. A Global spec receives it. Everyone else
// receives a scope bound to their declared Prefixes, so a subsystem's middleware
// reaches its own subtrees and nothing else, whatever its slice position. A
// subsystem that installs middleware outside them fails the mount — the binary
// refuses to boot half-gated rather than serving with a stranger's gate on.
//
// Teardown is wired HERE, at mount time: right after a subsystem mounts, its
// ShutdownFunc (if any) is registered via app.OnShutdown. zip drains those hooks
@@ -772,16 +1020,27 @@ type MountSpec struct {
// its dependents is torn down after them) with no subsystem torn down while a
// request still uses it. Only ENABLED specs mount, so only they register a hook;
// teardown needs no separate enablement gate.
func MountAll(app *zip.App, specs []MountSpec, cfg *Config, deps Deps) error {
func MountAll(app *zip.App, specs []AppSpec, cfg *Config, deps Deps) error {
logger := deps.Logger
for _, spec := range specs {
if !cfg.Enabled(spec.Name) {
logger.Debug("subsystem disabled", "name", spec.Name)
continue
}
if err := spec.Mount(app, deps); err != nil {
var router Router = app
var sc *scope
if !spec.Global {
sc = newScope(app, spec.Name, spec.Prefixes)
router = sc
}
if err := spec.Mount(router, deps); err != nil {
return fmt.Errorf("mount %s: %w", spec.Name, err)
}
if sc != nil {
if err := sc.err(); err != nil {
return fmt.Errorf("mount %s: %w", spec.Name, err)
}
}
// Register teardown as a zip shutdown hook. zip runs hooks LIFO after the
// drain (zip.App.Shutdown), so this reproduces the reverse-mount order the
// hand-rolled reverse-loop gave — without the teardown-before-drain race.
@@ -792,3 +1051,19 @@ func MountAll(app *zip.App, specs []MountSpec, cfg *Config, deps Deps) error {
}
return nil
}
// masterKeyBytes decodes the base64 KMS master (CLOUD_KMS_MASTER_KEY_REF) into
// the 32 raw bytes a subsystem needs to encrypt its own store. nil on absent or
// malformed — never a partial or wrong-length key, because a wrong key encrypts
// against a store no other key can open.
func masterKeyBytes(cfg *Config) []byte {
ref := strings.TrimSpace(cfg.KMSMasterKeyRef)
if ref == "" {
return nil
}
master, err := base64.StdEncoding.DecodeString(ref)
if err != nil || len(master) != 32 {
return nil
}
return master
}
+3 -3
View File
@@ -16,7 +16,7 @@ import (
// noopMount mounts nothing: the fake specs below carry the behavior under test in
// their Shutdown, not their Mount.
func noopMount(*zip.App, cloud.Deps) error { return nil }
func noopMount(cloud.Router, cloud.Deps) error { return nil }
// freeAddr reserves an ephemeral loopback port and hands back its address; the
// listener is closed so the app under test can bind it.
@@ -70,7 +70,7 @@ func TestMountAll_ShutdownHooksLIFOAfterDrain(t *testing.T) {
}
// Mount order a, b, c ⇒ LIFO teardown must be c, b, a.
specs := []cloud.MountSpec{
specs := []cloud.AppSpec{
{Name: "a", Mount: noopMount, Shutdown: record("a")},
{Name: "b", Mount: noopMount, Shutdown: record("b")},
{Name: "c", Mount: noopMount, Shutdown: record("c")},
@@ -183,7 +183,7 @@ func TestMountAll_ShutdownRegistration_EnablementAndNil(t *testing.T) {
}
}
specs := []cloud.MountSpec{
specs := []cloud.AppSpec{
{Name: "enabled", Mount: noopMount, Shutdown: record("enabled")},
{Name: "disabled", Mount: noopMount, Shutdown: record("disabled")},
{Name: "nilsd", Mount: noopMount}, // enabled, but no Shutdown
+1 -2
View File
@@ -4,7 +4,6 @@ import (
"testing"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
)
// TestMountFunc_IsTheSubsystemSignature pins the registry's mount contract: the
@@ -24,5 +23,5 @@ import (
//
// What remains is the only claim worth making, and the build enforces it.
func TestMountFunc_IsTheSubsystemSignature(t *testing.T) {
var _ cloud.MountFunc = func(*zip.App, cloud.Deps) error { return nil }
var _ cloud.MountFunc = func(cloud.Router, cloud.Deps) error { return nil }
}
+128 -21
View File
@@ -109,25 +109,60 @@ func SetMasterKey(k []byte) {
}
}
// resolveMaster resolves the process master key exactly once:
// EnsureDevKey installs a deterministic development master key when NO key is
// configured AND the live codec is not linked — i.e. a pure-Go dev/CI build. It
// lets that build run through the SAME encrypted path as production (per-db DEK,
// SQLCipher-format envelope) with zero configuration, instead of a divergent
// plaintext path. It is a NO-OP when a key is already configured (production uses
// it) or the live codec is linked (a production binary, which MUST supply the real
// KMS key and fails closed without it). Call once at boot, before the first Open.
// Returns true when a dev key was installed. Not safe against a concurrent Open —
// resolveMaster caches on first use, so this must run first.
func EnsureDevKey() bool {
if len(masterOverride) == 32 || strings.TrimSpace(os.Getenv(masterKeyEnv)) != "" {
return false // a real key is configured — use it
}
if sqlitedrv.CodecLinked() {
return false // production build — require the real key (resolveMaster fails closed)
}
SetMasterKey(devKey())
return true
}
// devKey derives the deterministic development master key. It is intentionally
// well-known — dev/CI data is not secret — and exists only so a pure-Go build
// encrypts through the production code path rather than diverging to plaintext. It
// is never reachable on a codec-linked (production) build; EnsureDevKey gates it.
func devKey() []byte {
sum := sha256.Sum256([]byte("hanzo-cloud-dev-cek-master-v1"))
return sum[:]
}
// resolveMaster resolves the process master key exactly once. Every build encrypts
// a keyed store — the live libsqlcipher codec when it is linked, the pure-Go codec
// envelope otherwise (EncryptionAvailable is always true) — so a store is either
// keyed-and-encrypted or it does not open. There is no plaintext-at-rest mode:
//
// (key, nil) — 32-byte key AND an encryption-capable build → encrypt.
// (nil, nil) — no key AND a non-encrypting (pure-Go) build → dev/CI plaintext.
// (nil, error) — key malformed; OR key set on a non-encrypting build; OR NO key
// on an encryption-capable build. The last is the production
// fail-closed: a capable binary never silently ships plaintext.
// (key, nil) — 32-byte key configured → encrypt (live codec or envelope; both
// write ciphertext at rest).
// (nil, error) — no key, OR a malformed key. Opening the data plane with no key is
// the fatal case: a build that can encrypt must never ship plaintext.
// Dev/CI supplies a deterministic dev key at boot (SetMasterKey) so
// it runs encrypted with zero config; production supplies the KMS key.
func resolveMaster() ([]byte, error) {
masterOnce.Do(func() {
raw := masterOverride
if len(raw) == 0 {
b64 := strings.TrimSpace(os.Getenv(masterKeyEnv))
if b64 == "" {
// EncryptionAvailable is always true, so this is always fatal: a build
// that can encrypt never opens the data plane unencrypted. Callers that
// want a keyless dev run inject a dev key via SetMasterKey before Open.
if sqlitedrv.EncryptionAvailable() {
masterErr = fmt.Errorf("cek: %s is required on an encryption-capable build; "+
"refusing to open the data plane unencrypted (set the KMS master key, "+
"or run a pure-Go dev build)", masterKeyEnv)
masterErr = fmt.Errorf("cek: %s is required; refusing to open the data plane "+
"unencrypted (set the KMS master key, or inject a dev key at boot)", masterKeyEnv)
}
return // pure-Go dev/CI: plaintext is expected (no codec linked)
return
}
decoded, err := base64.StdEncoding.DecodeString(b64)
if err != nil {
@@ -140,37 +175,99 @@ func resolveMaster() ([]byte, error) {
masterErr = fmt.Errorf("cek: master key must decode to 32 bytes, got %d", len(raw))
return
}
if !sqlitedrv.EncryptionAvailable() {
masterErr = fmt.Errorf("cek: %s is set but this build cannot encrypt (pure-Go sqlite); "+
"rebuild CGO_ENABLED=1 linked against libsqlcipher, or unset it for a dev build", masterKeyEnv)
return
}
// A configured key always encrypts: the live codec when linked, the pure-Go
// codec envelope otherwise. There is no "key set but cannot encrypt" case.
masterKey = raw
})
return masterKey, masterErr
}
// Encrypting reports whether cek will encrypt at rest (a valid master key is
// configured on an encryption-capable build). cloud calls this once at boot for
// the posture log; a false result on a capable build means resolveMaster errored
// and the first store Open will fail closed.
// configured). cloud calls this once at boot for the posture log; a false result
// means resolveMaster errored (no or invalid key) and the first store Open will
// fail closed rather than write plaintext.
func Encrypting() bool {
k, err := resolveMaster()
return err == nil && len(k) == 32
}
// inMemory reports whether path names an in-memory database rather than a file.
// SQLite spells that ":memory:", or any file: URI carrying mode=memory (including
// the shared-cache form, where several handles address ONE in-memory database by
// name). None of them produce a file, so none of them are cek's business.
func inMemory(path string) bool {
p := strings.TrimSpace(path)
if p == ":memory:" {
return true
}
if !strings.HasPrefix(p, "file:") {
return false
}
if strings.HasPrefix(p, "file::memory:") {
return true
}
// mode=memory as a query parameter, e.g. file:name?mode=memory&cache=shared.
q := p
if i := strings.IndexByte(p, '?'); i >= 0 {
q = p[i+1:]
}
for _, kv := range strings.Split(q, "&") {
if strings.TrimSpace(kv) == "mode=memory" {
return true
}
}
return false
}
// Exists reports whether a store lives at path — the question "has this org/
// subsystem been created yet?", asked by every caller that discovers stores by
// walking the data directory.
//
// os.Stat on the database file alone does NOT answer it. The pure-Go codec holds
// the database in its envelope and materializes the file on close, so a store that
// is OPEN right now has only its sidecar on disk; the live codec writes the
// database in place and has both. A caller that stats only the database therefore
// sees a store on one build and not the other, and on the pure-Go build skips
// precisely the stores that are in use. cek mints the sidecar eagerly, before the
// first byte of database is written, on both codecs — so "database or sidecar" is
// the marker that holds on every build and at every point in a store's life.
//
// The layout stays cek's own: callers ask this rather than knowing the suffix.
func Exists(path string) bool {
if _, err := os.Stat(path); err == nil {
return true
}
_, err := os.Stat(path + dekSuffix)
return err == nil
}
// Open returns a *sql.DB for the SQLite database at path, encrypted at rest when
// a master key is configured. It is the single drop-in replacement for
// sql.Open("sqlite", path) across every cloud store.
func Open(path string) (*sql.DB, error) {
// An in-memory database never reaches disk, so there is nothing at rest to
// encrypt and no master key to require. Treating the spelling as a filename
// instead creates a FILE literally named ":memory:" — silently making an
// ephemeral store durable, and making every opener of ":memory:" in a given
// working directory share one store.
if inMemory(path) {
// Opened on the registered driver with the DSN exactly as given: an
// in-memory DSN is already a complete DSN, and the keyed builder would wrap
// it as a filename (file::memory: becomes a file named "file::memory:").
db, err := sql.Open("sqlite", path)
if err != nil {
return nil, fmt.Errorf("cek: open in-memory %q: %w", path, err)
}
return db, nil
}
master, err := resolveMaster()
if err != nil {
return nil, err
}
if master == nil {
// Only reachable on a non-encrypting dev/CI build (a capable build with no
// key already errored above). Preserve the prior bare-path behavior.
return sql.Open("sqlite", path)
// Unreachable: resolveMaster returns a 32-byte key or a non-nil error, never
// (nil, nil). Fail closed anyway — cek never opens a store plaintext.
return nil, fmt.Errorf("cek: no master key resolved for %q", path)
}
return openEncrypted(path, master)
}
@@ -339,6 +436,16 @@ func openExisting(path string, master []byte) (*sql.DB, error) {
// the encrypted copy reproduces the source schema + per-table content hash +
// integrity_check, re-opened via the exact keyed path the app uses).
func migrateThenOpen(path string, master []byte) (*sql.DB, error) {
// Converting an existing plaintext database to SQLCipher uses libsqlcipher's
// ATTACH ... KEY + sqlcipher_export (see exportPlaintext), which only the live C
// codec provides. The pure-Go envelope opens and creates encrypted stores but
// cannot run that in-engine conversion; refuse cleanly rather than fail deep in
// the export. Legacy-plaintext migration is a production operation, and
// production links the live codec; a pure-Go dev/CI run starts from fresh
// encrypted stores (createFresh) instead.
if !sqlitedrv.CodecLinked() {
return nil, fmt.Errorf("cek: converting plaintext database %q to encrypted requires the live libsqlcipher codec; run a libsqlcipher-linked build to migrate it, or remove it to start from a fresh encrypted store", path)
}
dekPath := path + dekSuffix
// Plaintext header ⇒ an earlier attempt did not commit: discard any stale
// sidecar/tmp and redo from the plaintext source of truth.
+34 -2
View File
@@ -46,8 +46,8 @@ func requireCipher(t *testing.T) {
}
t.Skip(msg)
}
if !sqlitedrv.EncryptionAvailable() {
skipOrFail("sqlite build cannot encrypt (pure-Go); run with CGO + libsqlcipher")
if !sqlitedrv.CodecLinked() {
skipOrFail("sqlite build lacks the live libsqlcipher codec (pure-Go envelope encrypts, but these tests migrate via sqlcipher_export); run with CGO + libsqlcipher")
return
}
probe := filepath.Join(t.TempDir(), "probe.db")
@@ -317,6 +317,38 @@ func TestMissingKeyFatalOnCapableBuild(t *testing.T) {
}
}
// TestEnsureDevKeyEncryptsOnPureGo proves the zero-config dev posture: a pure-Go
// build with no configured key installs a deterministic dev key and runs through
// the SAME encrypted path as production — the store is ciphertext at rest, never
// plaintext. On a codec-linked (production) build EnsureDevKey is a no-op.
func TestEnsureDevKeyEncryptsOnPureGo(t *testing.T) {
if sqlitedrv.CodecLinked() {
t.Skip("codec-linked build: EnsureDevKey is a no-op; production requires the real key")
}
resetMaster(nil)
os.Unsetenv(masterKeyEnv)
if !EnsureDevKey() {
t.Fatal("EnsureDevKey should install a dev key on a pure-Go build with no key")
}
if EnsureDevKey() {
t.Fatal("EnsureDevKey must be idempotent — a no-op once a key is installed")
}
path := filepath.Join(t.TempDir(), "settings.db")
db, err := Open(path)
if err != nil {
t.Fatalf("open with dev key: %v", err)
}
if _, err := db.Exec(`CREATE TABLE t(x)`); err != nil {
t.Fatalf("ddl: %v", err)
}
_ = db.Close()
if isPlaintextHeader(path) {
t.Fatal("SECURITY: dev-key store is plaintext at rest")
}
}
// TestContentHashCatchesMutation (RED MED #4): the content hash catches a value
// change that row-count + schema + integrity_check all miss, AND does NOT
// false-positive on the benign implicit-rowid renumbering sqlcipher_export does.
+64
View File
@@ -0,0 +1,64 @@
package cek
import (
"os"
"path/filepath"
"testing"
)
// TestInMemoryNeverTouchesDisk pins the distinction cek has to make: an in-memory
// database is not a path. Treating ":memory:" as a filename created a real, durable,
// encrypted file of that name in the working directory — every opener in that
// directory silently sharing one store, and an ephemeral store outliving its process.
func TestInMemoryNeverTouchesDisk(t *testing.T) {
EnsureDevKey()
dir := t.TempDir()
wd, err := os.Getwd()
if err != nil {
t.Fatalf("getwd: %v", err)
}
if err := os.Chdir(dir); err != nil {
t.Fatalf("chdir: %v", err)
}
t.Cleanup(func() { _ = os.Chdir(wd) })
for _, dsn := range []string{":memory:", "file::memory:", "file:x?mode=memory&cache=shared"} {
db, err := Open(dsn)
if err != nil {
t.Fatalf("Open(%q): %v", dsn, err)
}
if _, err := db.Exec(`CREATE TABLE t (v TEXT)`); err != nil {
t.Fatalf("Open(%q): unusable: %v", dsn, err)
}
_ = db.Close()
}
ents, err := os.ReadDir(dir)
if err != nil {
t.Fatalf("readdir: %v", err)
}
for _, e := range ents {
t.Errorf("in-memory open left %q on disk", filepath.Join(dir, e.Name()))
}
}
// TestInMemoryPredicate pins which DSNs are in-memory — a real path must never be
// mistaken for one, or it would open unencrypted.
func TestInMemoryPredicate(t *testing.T) {
for dsn, want := range map[string]bool{
":memory:": true,
" :memory: ": true,
"file::memory:": true,
"file:x?mode=memory": true,
"file:x?cache=shared&mode=memory": true,
"/var/lib/cloud/orgs/acme/kms.db": false,
"memory.db": false,
"file:/var/lib/x.db": false,
"file:x?mode=ro": false,
"": false,
} {
if got := inMemory(dsn); got != want {
t.Errorf("inMemory(%q) = %v, want %v", dsn, got, want)
}
}
}
+3 -3
View File
@@ -105,8 +105,8 @@ type Config struct {
PlatformURL string `json:"platform_url,omitempty"`
CloudURL string `json:"cloud_url,omitempty"`
ClientID string `json:"client_id,omitempty"`
APIKey string `json:"apiKey,omitempty"` // hk-… key; what `hanzo code` hands the agents
CodeTool string `json:"code_tool,omitempty"` // default agent for bare `hanzo` / `hanzo code`: dev|claude|codex
APIKey string `json:"apiKey,omitempty"` // hk-… key; what `hanzo code` hands the agents
CodeTool string `json:"code_tool,omitempty"` // default agent for bare `hanzo` / `hanzo code`: dev|claude|codex
CodeModel string `json:"code_model,omitempty"` // default model for `hanzo code` (else defaultCodeModel)
}
@@ -578,7 +578,7 @@ func newRootCmd() *cobra.Command {
pf := root.PersistentFlags()
pf.StringVar(&f.org, "org", "", "organization (overrides config / HANZO_ORG)")
pf.StringVarP(&f.output, "output", "o", "", "output format: table|json")
pf.StringVar(&f.platformURL, "platform-url", "", "platform base URL (default "+defaultPlatformURL+")")
pf.StringVar(&f.platformURL, "platform-url", "", "platform base URL (default "+defaultPlatformURL+"); `hanzo build` targets --cloud-url instead")
pf.StringVar(&f.iamIssuer, "iam-issuer", "", "IAM issuer (default "+defaultIAMIssuer+")")
pf.StringVar(&f.cloudURL, "cloud-url", "", "cloud API base URL (default "+defaultCloudURL+")")
pf.StringVar(&f.clientID, "client-id", "", "IAM OAuth client id (default "+defaultClientID+")")
+43 -26
View File
@@ -6,8 +6,8 @@ package cli
// remember.
//
// hanzo code claude # Claude Code on the default model
// hanzo code codex deepseek-v4-pro
// hanzo code dev glm5.2 # ids are resolved fuzzily: glm5.2 -> glm-5.2
// hanzo code codex enso-pro
// hanzo code dev zen5 # ids are resolved fuzzily: zen5pro -> zen5-pro
// hanzo code ls # what can I run?
//
// Two wire protocols cover all three agents: Claude Code speaks Anthropic,
@@ -71,7 +71,7 @@ func (t zenTier) slotID() string {
var zenTiers = []zenTier{
{zen: "zen5-flash", env: "ANTHROPIC_DEFAULT_HAIKU_MODEL", name: "Zen5 Flash", desc: "Hanzo Zen5 Flash — fast, cheap tier"},
{zen: "zen5", carrier: "claude-sonnet-4-6[1m]", env: "ANTHROPIC_DEFAULT_SONNET_MODEL", name: "Zen5", desc: "Hanzo Zen5 — frontier tier (1M context)"},
{zen: "zen5-pro", carrier: "claude-opus-4-8[1m]", env: "ANTHROPIC_DEFAULT_OPUS_MODEL", name: "Zen5 Pro", desc: "Hanzo Zen5 Pro — DeepSeek-V4 class (1M context)"},
{zen: "zen5-pro", carrier: "claude-opus-4-8[1m]", env: "ANTHROPIC_DEFAULT_OPUS_MODEL", name: "Zen5 Pro", desc: "Hanzo Zen5 Pro — deep reasoning (1M context)"},
{zen: "zen5-pro", carrier: "claude-fable-5[1m]", env: "ANTHROPIC_DEFAULT_FABLE_MODEL", name: "Zen5 Pro (max effort)", desc: "Hanzo Zen5 Pro, top tier (1M context)"},
{zen: "zen5-coder", carrier: "claude-sonnet-5", name: "Zen5 Coder", desc: "Hanzo Zen5 Coder — code-specialized (1M context)"},
}
@@ -144,19 +144,32 @@ func openaiWire(base, token, _ string) map[string]string {
}
type codeAgent struct {
bin string // executable to exec
wire wire // how it finds the cloud
fullAuto []string // flags that bypass approval prompts
continueArgs []string // harness-native form of Hanzo -c/--continue
modelArg []string // how the model is passed on argv (empty: via env)
carrier func(model string) string // maps the resolved model to a client-recognized id (claude: zen→carrier); nil = pass through
provider func(base string) []string // agents that need the endpoint declared, not just env'd
clear []string // env that would shadow the wire (a stale key in the shell)
configHome string // env var that relocates the agent's config dir to ~/.hanzo ("" = share the user's own install)
seed func(dir string) error // one-time defaults for the isolated config dir
appendSystem []string // --append-system-prompt + text; ALWAYS applied (identity, not a permission bypass — present in --safe too)
mcp bool // auto-wire the Hanzo MCP server (code/vector/web/vision tools) as an stdio server scoped to the cwd
install string // hint when the binary is missing
bin string // executable to exec
wire wire // how it finds the cloud
fullAuto []string // flags that bypass approval prompts
continueArgs []string // harness-native form of Hanzo -c/--continue
modelArg []string // how the model is passed on argv (empty: via env)
carrier func(model string) string // maps the resolved model to a client-recognized id (claude: zen→carrier); nil = pass through
provider func(base, model string) []string // agents that need the endpoint declared, not just env'd
clear []string // env that would shadow the wire (a stale key in the shell)
configHome string // env var that relocates the agent's config dir to ~/.hanzo ("" = share the user's own install)
seed func(dir string) error // one-time defaults for the isolated config dir
appendSystem []string // --append-system-prompt + text; ALWAYS applied (identity, not a permission bypass — present in --safe too)
mcp bool // auto-wire the Hanzo MCP server (code/vector/web/vision tools) as an stdio server scoped to the cwd
install string // hint when the binary is missing
}
// codeContextWindow is the input context (tokens) the served coding model
// budgets from. The enso and zen5 flagship tiers serve 1M; the flash tiers
// serve 131072. Codex is told this so it sizes context to the real window
// instead of a flat 256K cap — the cause of "maximum context exceeded" at
// 262144 even on a 1M-capable model. This wrapper only launches Hanzo coding
// models (default zen5), so non-flash defaults to the 1M flagship window.
func codeContextWindow(model string) int {
if strings.Contains(strings.ToLower(model), "flash") {
return 131072
}
return 1000000
}
// codex and @hanzo/dev share a lineage (dev is a Codex fork), hence a wire.
@@ -169,19 +182,23 @@ func codexLike(bin, install string) codeAgent {
fullAuto: []string{"--dangerously-bypass-approvals-and-sandbox"},
continueArgs: []string{"resume", "--last"},
modelArg: []string{"-m"},
provider: func(base string) []string {
provider: func(base, model string) []string {
// api.hanzo.ai exposes the standard OpenAI /v1/models shape, not
// Codex's private remote model-catalog schema, so skip that refresh
// and supply the model's window here — sized to the SERVED model
// (enso / zen5 flagship = 1M, flash tiers = 131072) so Codex budgets
// the real context instead of a flat 256K cap (the "maximum context
// exceeded at 262144" bug). Auto-compact at 90% leaves headroom.
win := codeContextWindow(model)
return []string{
"-c", "model_provider=hanzo",
"-c", `model_providers.hanzo.name="Hanzo"`,
"-c", fmt.Sprintf(`model_providers.hanzo.base_url="%s/v1"`, strings.TrimSuffix(base, "/")),
"-c", `model_providers.hanzo.env_key="OPENAI_API_KEY"`,
"-c", `model_providers.hanzo.wire_api="responses"`,
// api.hanzo.ai exposes the standard OpenAI /v1/models shape,
// not Codex's private remote model-catalog schema. Skip that
// optional refresh and supply the coding model's metadata here.
"-c", `features.remote_models=false`,
"-c", `model_context_window=262144`,
"-c", `model_auto_compact_token_limit=235929`,
"-c", fmt.Sprintf("model_context_window=%d", win),
"-c", fmt.Sprintf("model_auto_compact_token_limit=%d", win*9/10),
}
},
install: install,
@@ -255,14 +272,14 @@ func newCodeCmd(envOf func() *Env, _ *globalFlags) *cobra.Command {
Long: "Run @hanzo/dev, Claude Code, or Codex against api.hanzo.ai with the endpoint,\n" +
"credential and model injected — no env vars to remember. `hanzo code` alone runs\n" +
"dev (the Hanzo agent); name an agent to pick another. Model ids resolve fuzzily\n" +
"(glm5.2 -> glm-5.2), -c resumes either harness, and agents run full-auto unless\n" +
"(zen5pro -> zen5-pro), -c resumes either harness, and agents run full-auto unless\n" +
"you pass --safe. Unknown options pass through; -- forces verbatim passthrough.",
Example: " hanzo code # dev, the default agent\n" +
" hanzo code claude\n" +
" hanzo code claude -c\n" +
" hanzo code codex -c\n" +
" hanzo code codex deepseek-v4-pro\n" +
" hanzo code dev glm5.2 -- --resume\n" +
" hanzo code codex enso-pro\n" +
" hanzo code dev zen5 -- --resume\n" +
" hanzo code ls",
// Bare `hanzo code` (or `hanzo code <model>/-- args` with no agent name) runs
// the configured default agent (code_tool, else dev). A recognized agent name
@@ -447,7 +464,7 @@ func codeArgv(agent codeAgent, base, model string, safe bool, rest []string) []s
argv = append(argv, agent.fullAuto...)
}
if agent.provider != nil {
argv = append(argv, agent.provider(base)...)
argv = append(argv, agent.provider(base, model)...)
}
if len(agent.modelArg) > 0 { // claude takes the model via env, codex/dev on argv
argv = append(argv, agent.modelArg...)
+25 -2
View File
@@ -100,19 +100,42 @@ func TestCodeUnknownOptionsAndPostSeparatorArgsPassThrough(t *testing.T) {
}
func TestCodexProviderUsesNativeResponsesMetadata(t *testing.T) {
// defaultCodeModel is zen5 — a 1M flagship tier — so the window must be 1M,
// NOT the old flat 262144 cap that surfaced as "maximum context exceeded"
// even on a 1M-capable model. Auto-compact is 90% of the window.
argv := codeArgv(codeAgents["codex"], "https://api.hanzo.ai", defaultCodeModel, false, nil)
for _, want := range []string{
`model_provider=hanzo`,
`model_providers.hanzo.base_url="https://api.hanzo.ai/v1"`,
`model_providers.hanzo.wire_api="responses"`,
`features.remote_models=false`,
`model_context_window=262144`,
`model_auto_compact_token_limit=235929`,
`model_context_window=1000000`,
`model_auto_compact_token_limit=900000`,
} {
if !slices.Contains(argv, want) {
t.Errorf("Codex argv %q does not contain %q", argv, want)
}
}
// A flash tier budgets its smaller real window, not 1M.
flash := codeArgv(codeAgents["codex"], "https://api.hanzo.ai", "zen5-flash", false, nil)
if !slices.Contains(flash, `model_context_window=131072`) {
t.Errorf("zen5-flash argv %q must budget 131072, not the flagship 1M", flash)
}
}
// TestCodeContextWindowSizing pins the model→window map: every non-flash tier
// gets the 1M flagship window; only flash tiers drop to 131072.
func TestCodeContextWindowSizing(t *testing.T) {
for _, m := range []string{"zen5", "zen5-pro", "zen5-coder", "enso", "enso-ultra"} {
if w := codeContextWindow(m); w != 1000000 {
t.Errorf("codeContextWindow(%q) = %d, want 1000000", m, w)
}
}
for _, m := range []string{"zen5-flash", "enso-flash"} {
if w := codeContextWindow(m); w != 131072 {
t.Errorf("codeContextWindow(%q) = %d, want 131072", m, w)
}
}
}
// TestCodeTokenPrecedence locks in the 402 unblock: a fresh `hanzo login` JWT
+8 -1
View File
@@ -16,6 +16,13 @@ func (e *Env) platform(gf *globalFlags) *Platform {
return newPlatform(e.PlatformURL, e.platformToken(gf.platformToken))
}
// runner builds the client for POST /v1/runner. That route is served by the
// cloud binary, not by the platform app, so it is reached at CloudURL — the
// platform host answers 500 for it. One route, one implementation, one door.
func (e *Env) runner(gf *globalFlags) *Platform {
return newPlatform(e.CloudURL, e.platformToken(gf.platformToken))
}
// dashIfEmpty renders a string cell, "-" when empty.
func dashIfEmpty(s string) string {
if s == "" {
@@ -279,7 +286,7 @@ func newBuildCmd(envOf func() *Env, gf *globalFlags) *cobra.Command {
if br.OrganizationID == "" {
br.OrganizationID = e.Org // optional; server defaults to DEFAULT_BUILD_ORG_ID
}
job, err := e.platform(gf).EnqueueBuild(cmd.Context(), br, e.buildToken(buildToken))
job, err := e.runner(gf).EnqueueBuild(cmd.Context(), br, e.buildToken(buildToken))
if err != nil {
return err
}
+15 -1
View File
@@ -38,6 +38,20 @@ func withPlatform(t *testing.T, h http.HandlerFunc) string {
return srv.URL
}
// withCloud is withPlatform's sibling for routes the CLOUD binary serves.
// /v1/runner is one: the platform host does not implement it, so a build sent
// to PlatformURL answers 500 there and 401 here. Pointing this at CloudURL is
// what the test is asserting.
func withCloud(t *testing.T, h http.HandlerFunc) string {
t.Helper()
sandbox(t)
srv := httptest.NewServer(h)
t.Cleanup(srv.Close)
t.Setenv("HANZO_CLOUD_URL", srv.URL)
t.Setenv("HANZO_PLATFORM_TOKEN", "svc-tok")
return srv.URL
}
// apps list hits the LIVE board path /v1/paas/apps and renders the fleet table.
func TestAppsListCommandTable(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, r *http.Request) {
@@ -214,7 +228,7 @@ func TestBuildCommandValidation(t *testing.T) {
}
func TestBuildCommand(t *testing.T) {
withPlatform(t, func(w http.ResponseWriter, r *http.Request) {
withCloud(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/runner" {
t.Errorf("path = %s", r.URL.Path)
}
+375 -36
View File
@@ -18,7 +18,7 @@ package cli
// --serve-engine adds the `engine.serve` capability: the worker probes a local
// hanzo-engine (the OpenAI + Anthropic model server on :1234), advertises its model
// endpoint in the presence record, and prints (or with --register-provider, POSTs)
// the /v1/add-provider call that routes api.hanzo.ai model traffic to this GPU. One
// the POST /v1/ai/providers call that routes api.hanzo.ai model traffic to this GPU. One
// fleet, two job types: engine.serve (model serving) alongside studio.render.
import (
@@ -42,8 +42,10 @@ import (
"sort"
"strconv"
"strings"
"sync"
"syscall"
"time"
"unicode"
"github.com/spf13/cobra"
)
@@ -90,6 +92,7 @@ const (
const (
studioCap = "studio.render"
engineCap = "engine.serve"
fnCap = "fn.run" // ephemeral script execution (uv-run) on this node
)
// The node-level command surface — `hanzo link | unlink | status` — is wired in
@@ -152,6 +155,8 @@ type engineAdvertisement struct {
type gpuInfo struct {
Name string `json:"name"`
MemoryTotal string `json:"memoryTotal,omitempty"`
Arch string `json:"arch,omitempty"` // native target, e.g. "gfx1151" (AMD)
Unified bool `json:"unified,omitempty"` // memory is a unified CPU/GPU pool (APU / SoC)
}
// registration is the fleet presence activity's Input — the shape cloud's
@@ -172,6 +177,10 @@ type registration struct {
Memory int64 `json:"memory,omitempty"`
Version string `json:"version"`
JobQueue string `json:"jobQueue"`
Rocm string `json:"rocm,omitempty"` // host ROCm version (AMD only)
Hip string `json:"hip,omitempty"` // host HIP version (AMD only)
Cuda string `json:"cuda,omitempty"` // host CUDA toolkit version (NVIDIA only)
Driver string `json:"driver,omitempty"` // host NVIDIA driver version
GPUs []gpuInfo `json:"gpus"`
Capabilities []string `json:"capabilities,omitempty"`
Engine *engineAdvertisement `json:"engine,omitempty"`
@@ -276,6 +285,9 @@ func newWorker(env *Env, jobsNS string) (*worker, error) {
"echo": echoHandler,
"studio.render": w.studioRenderHandler,
}
if _, err := exec.LookPath("uv"); err == nil {
w.handlers[fnCap] = fnRunHandler // functions-runner: needs uv, nothing else
}
return w, nil
}
@@ -311,25 +323,74 @@ func detectGPUs() []gpuInfo {
return nil
}
// detectNvidiaGPUs reports NVIDIA accelerators via nvidia-smi (name + total VRAM).
// detectNvidiaGPUs reports NVIDIA accelerators via nvidia-smi: name, memory,
// and the sm arch (compute capability). A unified SoC (GB10 Grace-Blackwell)
// reports memory.total as "[N/A]" — there is no dedicated VRAM counter — so it
// reports the MACHINE's RAM snapped to hardware capacity, unified=true, the
// same convention the AMD APU and Apple paths use.
func detectNvidiaGPUs() []gpuInfo {
out, err := exec.Command("nvidia-smi", "--query-gpu=name,memory.total", "--format=csv,noheader").Output()
out, err := exec.Command("nvidia-smi", "--query-gpu=name,memory.total,compute_cap", "--format=csv,noheader").Output()
if err != nil {
return nil
}
return parseNvidiaSmiCSV(out, detectMemTotal()/(1<<20))
}
// parseNvidiaSmiCSV parses `nvidia-smi --query-gpu=name,memory.total,compute_cap`
// output ("NVIDIA GB10, [N/A], 12.1"). Pure — hostMiB is injected for the
// unified-SoC memory figure.
func parseNvidiaSmiCSV(out []byte, hostMiB int64) []gpuInfo {
var gpus []gpuInfo
sc := bufio.NewScanner(bytes.NewReader(out))
for sc.Scan() {
line := strings.TrimSpace(sc.Text())
if line == "" {
fields := strings.Split(sc.Text(), ",")
if len(fields) < 2 || strings.TrimSpace(fields[0]) == "" {
continue
}
name, mem, _ := strings.Cut(line, ",")
gpus = append(gpus, gpuInfo{Name: strings.TrimSpace(name), MemoryTotal: strings.TrimSpace(mem)})
g := gpuInfo{Name: strings.TrimSpace(fields[0]), MemoryTotal: strings.TrimSpace(fields[1])}
var memMiB int64
if _, err := fmt.Sscanf(g.MemoryTotal, "%d MiB", &memMiB); err != nil || memMiB <= 0 {
// No dedicated VRAM counter — a unified SoC. The machine's RAM is
// the GPU's RAM.
if hostMiB > 0 {
g.MemoryTotal = fmt.Sprintf("%d MiB", snapUnified(hostMiB))
g.Unified = true
} else {
g.MemoryTotal = ""
}
}
if len(fields) >= 3 {
if cap := strings.TrimSpace(fields[2]); cap != "" && cap != "[N/A]" {
g.Arch = "sm_" + strings.ReplaceAll(cap, ".", "")
}
}
gpus = append(gpus, g)
}
return gpus
}
// nvidiaSoft is the host CUDA/driver inventory, detected once — the NVIDIA
// mirror of amdSoftware. Empty on non-NVIDIA hosts.
type nvidiaSoft struct{ cuda, driver string }
var nvidiaSoftware = sync.OnceValue(func() nvidiaSoft {
var v nvidiaSoft
if out, err := exec.Command("nvidia-smi", "--query-gpu=driver_version", "--format=csv,noheader").Output(); err == nil {
v.driver = strings.TrimSpace(strings.SplitN(string(out), "\n", 2)[0])
}
if b, err := os.ReadFile("/usr/local/cuda/version.json"); err == nil {
var m struct {
Cuda struct {
Version string `json:"version"`
} `json:"cuda"`
}
if json.Unmarshal(b, &m) == nil {
v.cuda = m.Cuda.Version
}
}
return v
})
// detectAmdGPUs reports AMD accelerators — discrete Radeon cards and gfx APUs alike
// (e.g. evo's gfx1151 Radeon 8060S on the RYZEN AI MAX+ 395). Resolution order:
// rocm-smi (marketing name + gfx target), then the kfd topology under /sys (gfx
@@ -338,11 +399,11 @@ func detectNvidiaGPUs() []gpuInfo {
func detectAmdGPUs() []gpuInfo {
if out, err := exec.Command("rocm-smi", "--showproductname", "--csv").Output(); err == nil {
if gpus := parseRocmSmiCSV(out); len(gpus) > 0 {
return fillAmdVRAM(gpus, amdVRAMTotals(sysfsDRM))
return apuProcessorNames(fillAmdVRAM(gpus, amdVRAMTotals(sysfsDRM)), detectCPUModel())
}
}
if gpus := parseKfdTopology(sysfsKfdNodes); len(gpus) > 0 {
return fillAmdVRAM(gpus, amdVRAMTotals(sysfsDRM))
return apuProcessorNames(fillAmdVRAM(gpus, amdVRAMTotals(sysfsDRM)), detectCPUModel())
}
if out, err := exec.Command("vulkaninfo", "--summary").Output(); err == nil {
if gpus := parseVulkaninfoSummary(out); len(gpus) > 0 {
@@ -388,12 +449,17 @@ func parseRocmSmiCSV(out []byte) []gpuInfo {
if name == "" {
continue
}
arch := ""
if gfx >= 0 && gfx < len(fields) {
if v := strings.TrimSpace(fields[gfx]); v != "" {
arch = v
name += " (" + v + ")"
}
}
gpus = append(gpus, gpuInfo{Name: name})
if fam := amdFamily(arch); fam != "" {
name = fam + " \u00b7 " + name
}
gpus = append(gpus, gpuInfo{Name: name, Arch: arch})
}
return gpus
}
@@ -422,7 +488,12 @@ func parseKfdTopology(nodesDir string) []gpuInfo {
if simd <= 0 || gfxVer <= 0 {
continue // CPU node or non-GPU
}
gpus = append(gpus, gpuInfo{Name: "AMD GPU (" + gfxName(gfxVer) + ")"})
arch := gfxName(gfxVer)
name := "AMD GPU (" + arch + ")"
if fam := amdFamily(arch); fam != "" {
name = fam + " (" + arch + ")"
}
gpus = append(gpus, gpuInfo{Name: name, Arch: arch})
}
return gpus
}
@@ -452,9 +523,119 @@ func gfxName(v int) string {
return fmt.Sprintf("gfx%d%d%d", v/10000, (v/100)%100, v%100)
}
// amdVRAMTotals returns each amdgpu card's total VRAM in MiB, in DRM card order,
// read from /sys/class/drm/card*/device/mem_info_vram_total (vendor 0x1002).
func amdVRAMTotals(drmDir string) []int64 {
// amdFamily maps a gfx target to the APU/SoC family marketing name, so the board
// says what the silicon IS ("AMD Strix Halo"), not just the iGPU series. Discrete
// cards return "" and keep their own name.
func amdFamily(arch string) string {
switch arch {
case "gfx1151":
return "AMD Strix Halo"
case "gfx1150":
return "AMD Strix Point"
case "gfx1103":
return "AMD Phoenix"
}
return ""
}
// apuProcessorNames renders a unified-memory APU as the PROCESSOR it is — the
// cpuinfo marketing name ("AMD Ryzen AI Max+ 395 w/ Radeon 8060S") beats the
// iGPU series, because that is the product the owner bought. Discrete cards and
// non-Ryzen hosts keep their existing names.
func apuProcessorNames(gpus []gpuInfo, cpuModel string) []gpuInfo {
name := amdAPUName(cpuModel)
if name == "" {
return gpus
}
for i, g := range gpus {
if !g.Unified {
continue
}
if g.Arch != "" {
gpus[i].Name = name + " (" + g.Arch + ")"
} else {
gpus[i].Name = name
}
}
return gpus
}
// amdAPUName normalizes an AMD APU cpuinfo model into its marketing name:
// "AMD RYZEN AI MAX+ 395 w/ Radeon 8060S" -> "AMD Ryzen AI Max+ 395 w/ Radeon
// 8060S". BIOS strings shout; the board should not. Non-Ryzen models return "".
func amdAPUName(cpuModel string) string {
m := strings.Join(strings.Fields(cpuModel), " ")
if !strings.Contains(strings.ToLower(m), "ryzen") {
return ""
}
words := strings.Split(m, " ")
for i, w := range words {
if w == "AMD" || w == "AI" || w != strings.ToUpper(w) {
continue // brand/initialism stays; mixed-case is already right
}
if strings.ContainsFunc(w, unicode.IsDigit) {
continue // model numbers ("8060S", "395") keep their casing
}
if r := []rune(w); len(r) > 1 && strings.ContainsFunc(w, unicode.IsLetter) {
words[i] = string(r[0]) + strings.ToLower(string(r[1:]))
}
}
return strings.Join(words, " ")
}
// amdSoft is the host ROCm/HIP toolchain inventory, detected once: ROCm from
// /opt/rocm/.info/version, HIP from `hipconfig --version` (which exists even on
// TheRock-style installs that lack the .info file). Empty on non-AMD hosts.
type amdSoft struct{ rocm, hip string }
var amdSoftware = sync.OnceValue(func() amdSoft {
var v amdSoft
if b, err := os.ReadFile("/opt/rocm/.info/version"); err == nil {
v.rocm = strings.TrimSpace(string(b))
}
if out, err := exec.Command("hipconfig", "--version").Output(); err == nil {
v.hip = strings.TrimSpace(string(out))
}
return v
})
// pickAmdMem chooses the honest memory figure for a card: dedicated VRAM for a
// discrete GPU; for an APU whose "VRAM" is a token carve-out (Strix Halo: 1 GiB
// VRAM beside a ~118 GiB GTT pool) the figure is the MACHINE's unified RAM —
// the same convention Apple silicon reports (an M4 Max says 128 GiB, not the
// wired-down remainder) — snapped to hardware capacity. Unified when GTT wins.
func pickAmdMem(vramMiB, gttMiB, hostMiB int64) (miB int64, unified bool) {
if gttMiB > vramMiB*4 {
m := gttMiB
if hostMiB > m {
m = hostMiB
}
return snapUnified(m), true
}
return vramMiB, false
}
// snapUnified rounds a kernel-visible unified-memory figure up to the hardware
// DIMM capacity (the next 16 GiB multiple) when the gap is a plausible firmware
// reservation (≤ 6 GiB): a 128 GiB Strix Halo shows 124.4 GiB to Linux because
// BIOS + the VRAM carve-out are invisible to the OS. A larger gap (say a 16 GiB
// carve-out) is real capacity the pool lost — reported as-is, never invented.
func snapUnified(miB int64) int64 {
const step = 16 << 10 // 16 GiB in MiB
next := ((miB + step - 1) / step) * step
if next-miB <= 8<<10 {
return next
}
return miB
}
// amdMem is one card's memory inventory in MiB: dedicated VRAM plus the GTT
// (system-memory) pool an APU actually computes in.
type amdMem struct{ vramMiB, gttMiB int64 }
// amdVRAMTotals returns each amdgpu card's memory totals in MiB, in DRM card order,
// read from /sys/class/drm/card*/device/mem_info_{vram,gtt}_total (vendor 0x1002).
func amdVRAMTotals(drmDir string) []amdMem {
entries, err := os.ReadDir(drmDir)
if err != nil {
return nil
@@ -469,32 +650,44 @@ func amdVRAMTotals(drmDir string) []int64 {
sort.Slice(cards, func(i, j int) bool {
return atoiSafe(strings.TrimPrefix(cards[i], "card")) < atoiSafe(strings.TrimPrefix(cards[j], "card"))
})
var mems []int64
readMiB := func(path string) int64 {
b, err := os.ReadFile(path)
if err != nil {
return 0
}
var n int64
if _, err := fmt.Sscan(strings.TrimSpace(string(b)), &n); err != nil || n <= 0 {
return 0
}
return n / (1024 * 1024)
}
var mems []amdMem
for _, c := range cards {
dev := filepath.Join(drmDir, c, "device")
if vendor, _ := os.ReadFile(filepath.Join(dev, "vendor")); strings.TrimSpace(string(vendor)) != "0x1002" {
continue
}
b, err := os.ReadFile(filepath.Join(dev, "mem_info_vram_total"))
if err != nil {
vram := readMiB(filepath.Join(dev, "mem_info_vram_total"))
if vram == 0 {
continue
}
var bytesTotal int64
if _, err := fmt.Sscan(strings.TrimSpace(string(b)), &bytesTotal); err == nil && bytesTotal > 0 {
mems = append(mems, bytesTotal/(1024*1024))
}
mems = append(mems, amdMem{vramMiB: vram, gttMiB: readMiB(filepath.Join(dev, "mem_info_gtt_total"))})
}
return mems
}
// fillAmdVRAM attaches VRAM totals to the GPU list positionally when the counts
// fillAmdVRAM attaches memory totals to the GPU list positionally when the counts
// match (the common single-GPU case always does); otherwise the names stand alone.
func fillAmdVRAM(gpus []gpuInfo, memsMiB []int64) []gpuInfo {
if len(memsMiB) != len(gpus) {
// An APU reports its unified GTT pool, not the token VRAM carve-out.
func fillAmdVRAM(gpus []gpuInfo, mems []amdMem) []gpuInfo {
if len(mems) != len(gpus) {
return gpus
}
hostMiB := detectMemTotal() / (1 << 20)
for i := range gpus {
gpus[i].MemoryTotal = fmt.Sprintf("%d MiB", memsMiB[i])
miB, unified := pickAmdMem(mems[i].vramMiB, mems[i].gttMiB, hostMiB)
gpus[i].MemoryTotal = fmt.Sprintf("%d MiB", miB)
gpus[i].Unified = unified
}
return gpus
}
@@ -742,9 +935,10 @@ type connectOpts struct {
serveEngine bool
engineURL string // local URL to probe hanzo-engine
engineEndpoint string // public URL to advertise (defaults to engineURL)
registerProvider bool // auto POST /v1/add-provider for the engine
registerProvider bool // auto POST /v1/ai/providers for the engine
studioDir string // local Studio checkout to launch + supervise on :8188
studioURL string // studio base the render mirror uploads finished images to
mirror bool // sweep local renders into the org studio library (default on)
}
func runConnect(cmd *cobra.Command, env *Env, opts connectOpts) error {
@@ -819,7 +1013,7 @@ func runConnect(cmd *cobra.Command, env *Env, opts connectOpts) error {
mirrorDir := ""
seen := map[string]int64{}
var mirC <-chan time.Time
if opts.studioDir != "" {
if opts.studioDir != "" && opts.mirror {
mirrorDir = filepath.Join(opts.studioDir, "output")
mir := time.NewTicker(heartbeatEvery)
defer mir.Stop()
@@ -878,11 +1072,16 @@ func runConnect(cmd *cobra.Command, env *Env, opts connectOpts) error {
// presence record (activityId==runId==identity, no requestId so a reconnect
// overwrites any prior/terminal record with a fresh online row).
func (w *worker) register(ctx context.Context) error {
// Ensuring the namespaces is BEST-EFFORT: they already exist in any live org, so a
// transient API blip here must not kill the worker. It used to be fatal, and with
// systemd Restart=always/RestartSec=5 one 503 turned into an infinite crash-loop
// (observed at 141 restarts, machine offline the whole time). If a namespace truly
// is missing, the presence write below fails and surfaces the real error.
for _, ns := range []string{fleetNS, w.jobsNS} {
if _, err := w.call(ctx, http.MethodPost, "/v1/tasks/namespaces", map[string]any{
"namespaceInfo": map[string]any{"name": ns},
}, nil); err != nil {
return fmt.Errorf("ensure namespace %q: %w", ns, err)
fmt.Fprintf(os.Stderr, "ensure namespace %q (continuing): %v\n", ns, err)
}
}
_, err := w.call(ctx, http.MethodPost, "/v1/tasks/namespaces/"+fleetNS+"/activities", map[string]any{
@@ -908,6 +1107,10 @@ func (w *worker) buildRegistration() registration {
Memory: w.memory,
Version: Version,
JobQueue: w.jobsNS,
Rocm: amdSoftware().rocm,
Hip: amdSoftware().hip,
Cuda: nvidiaSoftware().cuda,
Driver: nvidiaSoftware().driver,
GPUs: w.gpus,
Capabilities: w.capabilities(),
Engine: w.engine,
@@ -926,9 +1129,23 @@ func (w *worker) capabilities() []string {
if w.serveEngine {
caps = append(caps, engineCap)
}
if _, ok := w.handlers[fnCap]; ok {
caps = append(caps, fnCap)
}
return caps
}
// hasNonRenderLane reports whether this worker serves any lane beyond the render
// path (echo is a smoke type, not a lane).
func (w *worker) hasNonRenderLane() bool {
for name := range w.handlers {
if name != "echo" && name != studioCap {
return true
}
}
return false
}
// studioReachable probes the local studio's /queue (bounded), authenticated. A node
// that can answer it is up enough to accept a render.
func (w *worker) studioReachable(ctx context.Context) bool {
@@ -1023,10 +1240,12 @@ func (w *worker) claimFrom(ctx context.Context, taskQueue string) (claimedActivi
// queue name, so one worker never steals another GPU's targeted job (an empty
// taskQueue on claim would match any lane and do exactly that).
func (w *worker) claimAndRun(ctx context.Context, out io.Writer) error {
// Poison-loop guard: a node that can't serve renders must not claim render lanes —
// it would only fail every claimed job on the gated execute seam. It still
// heartbeats presence (shows in the fleet), just idle, until it becomes ready.
if !w.studioReady {
// Poison-loop guard, per-LANE: a node that can't serve renders must not sit on
// render jobs — but a node with other lanes (fn.run) still claims. When renders
// are this worker's only real lane and the studio isn't ready, stay idle; a
// claimed render on a non-ready node is declined below so an eligible worker
// takes it.
if !w.studioReady && !w.hasNonRenderLane() {
return nil
}
act, claimed, err := w.claimFrom(ctx, w.gpuQueue())
@@ -1052,6 +1271,12 @@ func (w *worker) claimAndRun(ctx context.Context, out io.Writer) error {
return nil
}
if act.Type.Name == studioCap && !w.studioReady {
cause := "studio not ready on this node — declined so a render-capable worker takes it"
_, _ = w.call(ctx, http.MethodPost, w.actPath(wf, run, "fail"), map[string]any{"cause": cause, "identity": w.identity}, nil)
fmt.Fprintf(out, " → declined (%s)\n", cause)
return nil
}
h, ok := w.handlers[act.Type.Name]
if !ok {
cause := fmt.Sprintf("no handler for job type %q", act.Type.Name)
@@ -1370,6 +1595,117 @@ func echoHandler(_ context.Context, input json.RawMessage) (any, error) {
return map[string]any{"echo": v}, nil
}
// fnRunInput is the fn.run job payload: an inline script executed on this node in
// an ephemeral `uv run` environment. The queue is org-scoped, so the submitter is
// running code on their OWN fleet — same trust domain as a CI runner.
type fnRunInput struct {
Name string `json:"name,omitempty"` // label for logs
Script string `json:"script"` // Python source (required)
Requirements []string `json:"requirements,omitempty"` // uv --with deps, e.g. ["numpy", "torch==2.5.*"]
Env map[string]string `json:"env,omitempty"` // extra environment
TimeoutSeconds int `json:"timeoutSeconds,omitempty"` // default 3600, cap 21600
}
const (
fnDefaultTimeout = time.Hour
fnMaxTimeout = 6 * time.Hour
fnOutputTail = 128 << 10 // keep the LAST 128 KiB of combined output
)
// fnValidate parses and bounds an fn.run payload. Requirements must look like
// package specs — a leading dash would inject uv flags.
func fnValidate(input json.RawMessage) (fnRunInput, error) {
var in fnRunInput
if err := json.Unmarshal(input, &in); err != nil {
return in, fmt.Errorf("fn.run: bad input: %w", err)
}
if strings.TrimSpace(in.Script) == "" {
return in, fmt.Errorf("fn.run: input needs a `script`")
}
for _, r := range in.Requirements {
if r == "" || strings.HasPrefix(r, "-") {
return in, fmt.Errorf("fn.run: bad requirement %q", r)
}
}
if in.TimeoutSeconds <= 0 {
in.TimeoutSeconds = int(fnDefaultTimeout / time.Second)
}
if in.TimeoutSeconds > int(fnMaxTimeout/time.Second) {
in.TimeoutSeconds = int(fnMaxTimeout / time.Second)
}
return in, nil
}
// tailBuffer keeps the last cap bytes written — a training loop can log gigabytes;
// the activity result carries the end, where the outcome lives.
type tailBuffer struct {
cap int
buf []byte
truncated bool
}
func (t *tailBuffer) Write(p []byte) (int, error) {
t.buf = append(t.buf, p...)
if len(t.buf) > t.cap {
t.buf = t.buf[len(t.buf)-t.cap:]
t.truncated = true
}
return len(p), nil
}
// fnRunHandler executes an fn.run job: write the script to an ephemeral dir, run
// it under `uv run` (which resolves requirements into a throwaway env — ROCm/CUDA
// wheels included), and return the output tail + exit code. Nonzero exit fails
// the activity with the tail as the cause, so the submitter sees the traceback.
func fnRunHandler(ctx context.Context, input json.RawMessage) (any, error) {
in, err := fnValidate(input)
if err != nil {
return nil, err
}
dir, err := os.MkdirTemp("", "fnrun-*")
if err != nil {
return nil, fmt.Errorf("fn.run: %w", err)
}
defer os.RemoveAll(dir)
script := filepath.Join(dir, "main.py")
if err := os.WriteFile(script, []byte(in.Script), 0o600); err != nil {
return nil, fmt.Errorf("fn.run: %w", err)
}
runCtx, cancel := context.WithTimeout(ctx, time.Duration(in.TimeoutSeconds)*time.Second)
defer cancel()
args := []string{"run", "--no-project", "--quiet"}
for _, r := range in.Requirements {
args = append(args, "--with", r)
}
args = append(args, script)
cmd := exec.CommandContext(runCtx, "uv", args...)
cmd.Dir = dir
cmd.Env = os.Environ()
for k, v := range in.Env {
cmd.Env = append(cmd.Env, k+"="+v)
}
tail := &tailBuffer{cap: fnOutputTail}
cmd.Stdout = tail
cmd.Stderr = tail
start := time.Now()
runErr := cmd.Run()
dur := time.Since(start)
out := string(tail.buf)
if runCtx.Err() == context.DeadlineExceeded {
return nil, fmt.Errorf("fn.run: timed out after %ds\n%s", in.TimeoutSeconds, out)
}
if runErr != nil {
return nil, fmt.Errorf("fn.run: %v\n%s", runErr, out)
}
return map[string]any{
"name": in.Name,
"exitCode": 0,
"durationMs": dur.Milliseconds(),
"output": out,
"truncated": tail.truncated,
}, nil
}
// workerToken is the KMS-sourced STUDIO_WORKER_TOKEN this box holds — the credential
// the local studio's --worker-mode gate checks. Empty on an unconfigured box (the
// render preflight refuses such a node before it ever claims a render lane).
@@ -1897,7 +2233,7 @@ func describeEngine(adv *engineAdvertisement) string {
return fmt.Sprintf("ready · %d models", n)
}
// providerBody is the POST /v1/add-provider payload registering this node's engine
// providerBody is the POST /v1/ai/providers payload registering this node's engine
// as an org model provider. hanzo-engine is OpenAI-compatible, so Type=Local: the
// gateway speaks the OpenAI wire format to it and auto-appends /v1 to providerUrl.
func (w *worker) providerBody() map[string]any {
@@ -1929,20 +2265,20 @@ func (w *worker) printEngineHint(out io.Writer) {
fmt.Fprintf(out, "serving hanzo-engine (OpenAI + Anthropic) at %s — %s\n", adv.URL, describeEngine(adv))
body, _ := json.Marshal(w.providerBody())
fmt.Fprintln(out, " route api.hanzo.ai model calls to this GPU by registering it as an org provider:")
fmt.Fprintf(out, " curl -sS %s/v1/add-provider -H \"Authorization: Bearer $HANZO_TOKEN\" \\\n", w.baseURL)
fmt.Fprintf(out, " curl -sS %s/v1/ai/providers -H \"Authorization: Bearer $HANZO_TOKEN\" \\\n", w.baseURL)
fmt.Fprintf(out, " -H 'Content-Type: application/json' -d '%s'\n", body)
fmt.Fprintln(out, " (or pass --register-provider. The endpoint must be reachable from api.hanzo.ai —")
fmt.Fprintln(out, " a cloud GPU is in-cluster; a BYO node needs a public URL/tunnel. add-provider needs a platform-admin token today.)")
}
// registerProvider POSTs /v1/add-provider so the gateway routes model calls to this
// registerProvider POSTs /v1/ai/providers so the gateway routes model calls to this
// node's engine. Requires the engine to be reachable and (today) a platform-admin
// token; both failures are reported clearly rather than swallowed.
func (w *worker) registerProvider(ctx context.Context, adv *engineAdvertisement) error {
if adv == nil || adv.Status != "ready" {
return fmt.Errorf("engine not ready at %s — start hanzo-engine, then retry", w.engineURL)
}
code, err := w.call(ctx, http.MethodPost, "/v1/add-provider", w.providerBody(), nil)
code, err := w.call(ctx, http.MethodPost, "/v1/ai/providers", w.providerBody(), nil)
if err != nil {
if code == http.StatusForbidden {
return fmt.Errorf("add-provider is gated to a platform-admin token today; register from the console or with an admin token: %w", err)
@@ -1990,6 +2326,9 @@ func installDaemon(cmd *cobra.Command, opts connectOpts) error {
if opts.studioDir != "" {
args += " --studio-dir " + opts.studioDir
}
if !opts.mirror {
args += " --mirror=false"
}
unit := fmt.Sprintf(`[Unit]
Description=Hanzo node (compute worker)
After=network-online.target
+79 -2
View File
@@ -17,7 +17,7 @@ func TestParseRocmSmiCSV(t *testing.T) {
if len(gpus) != 1 {
t.Fatalf("want 1 AMD GPU, got %d (%+v)", len(gpus), gpus)
}
if got, want := gpus[0].Name, "Radeon 8060S Graphics (gfx1151)"; got != want {
if got, want := gpus[0].Name, "AMD Strix Halo \u00b7 Radeon 8060S Graphics (gfx1151)"; got != want {
t.Errorf("name = %q, want %q", got, want)
}
}
@@ -49,9 +49,12 @@ func TestParseKfdTopology(t *testing.T) {
if len(gpus) != 1 {
t.Fatalf("want 1 GPU node (CPU node 0 skipped), got %d (%+v)", len(gpus), gpus)
}
if got, want := gpus[0].Name, "AMD GPU (gfx1151)"; got != want {
if got, want := gpus[0].Name, "AMD Strix Halo (gfx1151)"; got != want {
t.Errorf("name = %q, want %q", got, want)
}
if got, want := gpus[0].Arch, "gfx1151"; got != want {
t.Errorf("arch = %q, want %q", got, want)
}
}
// TestParseVulkaninfoSummary — the last resort keeps only AMD/Radeon devices so it
@@ -78,3 +81,77 @@ func writeNode(t *testing.T, root, id, props string) {
t.Fatal(err)
}
}
// TestPickAmdMemUnified — an APU whose VRAM is a token carve-out reports the
// machine's unified RAM snapped to hardware capacity (evo: 1 GiB VRAM, 118 GiB
// GTT, 124.4 GiB kernel-visible of 128 GiB physical → 128 GiB); a discrete card
// keeps its VRAM.
func TestPickAmdMemUnified(t *testing.T) {
if miB, unified := pickAmdMem(1024, 120832, 127411); !unified || miB != 131072 {
t.Errorf("APU: got (%d, %v), want (131072, true)", miB, unified)
}
if miB, unified := pickAmdMem(24576, 8192, 127411); unified || miB != 24576 {
t.Errorf("discrete: got (%d, %v), want (24576, false)", miB, unified)
}
}
// TestSnapUnified — a small firmware gap snaps up to the DIMM capacity; a big
// carve-out is real lost capacity and stays as-is; exact multiples stand.
func TestSnapUnified(t *testing.T) {
for _, c := range []struct{ in, want int64 }{
{127411, 131072}, // evo: 124.4 GiB visible of 128 GiB physical
{124610, 131072}, // spark GB10: 121.7 GiB visible of 128 GiB physical (~6.3 GiB firmware)
{131072, 131072}, // exact 128 GiB
{119194, 119194}, // ~116 GiB visible (16 GiB BIOS carve) — 11.6 GiB gap, keep honest
{63488, 65536}, // 62 GiB visible of 64 GiB
} {
if got := snapUnified(c.in); got != c.want {
t.Errorf("snapUnified(%d) = %d, want %d", c.in, got, c.want)
}
}
}
// TestAmdAPUName — the board renders the APU as the processor the owner bought,
// normalized from the BIOS shouting; non-Ryzen hosts opt out.
func TestAmdAPUName(t *testing.T) {
if got, want := amdAPUName("AMD RYZEN AI MAX+ 395 w/ Radeon 8060S"), "AMD Ryzen AI Max+ 395 w/ Radeon 8060S"; got != want {
t.Errorf("amdAPUName = %q, want %q", got, want)
}
if got := amdAPUName("Intel(R) Core(TM) i9-14900K"); got != "" {
t.Errorf("non-Ryzen host must opt out, got %q", got)
}
}
// TestApuProcessorNames — only unified-memory cards are renamed; a discrete
// card on the same host keeps its own identity.
func TestApuProcessorNames(t *testing.T) {
gpus := []gpuInfo{
{Name: "AMD Strix Halo \u00b7 Radeon 8060S Graphics (gfx1151)", Arch: "gfx1151", Unified: true},
{Name: "Radeon RX 7900 XTX (gfx1100)", Arch: "gfx1100"},
}
out := apuProcessorNames(gpus, "AMD RYZEN AI MAX+ 395 w/ Radeon 8060S")
if got, want := out[0].Name, "AMD Ryzen AI Max+ 395 w/ Radeon 8060S (gfx1151)"; got != want {
t.Errorf("APU name = %q, want %q", got, want)
}
if got, want := out[1].Name, "Radeon RX 7900 XTX (gfx1100)"; got != want {
t.Errorf("discrete name = %q, want %q", got, want)
}
}
// TestParseNvidiaSmiCSV — a GB10-class unified SoC ("[N/A]" VRAM) reports the
// machine RAM snapped to capacity + its sm arch; a discrete card keeps its VRAM.
func TestParseNvidiaSmiCSV(t *testing.T) {
out := []byte("NVIDIA GB10, [N/A], 12.1\n")
gpus := parseNvidiaSmiCSV(out, 124610)
if len(gpus) != 1 {
t.Fatalf("want 1 GPU, got %d", len(gpus))
}
g := gpus[0]
if g.Name != "NVIDIA GB10" || g.MemoryTotal != "131072 MiB" || !g.Unified || g.Arch != "sm_121" {
t.Errorf("GB10 = %+v", g)
}
disc := parseNvidiaSmiCSV([]byte("NVIDIA GeForce RTX 4090, 24564 MiB, 8.9\n"), 124610)
if d := disc[0]; d.MemoryTotal != "24564 MiB" || d.Unified || d.Arch != "sm_89" {
t.Errorf("discrete = %+v", d)
}
}
+75
View File
@@ -0,0 +1,75 @@
package cli
import (
"encoding/json"
"os/exec"
"strings"
"testing"
)
// TestFnValidate — payload bounds: script required, flag-injection via
// requirements refused, timeout defaulted and capped.
func TestFnValidate(t *testing.T) {
if _, err := fnValidate(json.RawMessage(`{}`)); err == nil {
t.Error("empty script must be refused")
}
if _, err := fnValidate(json.RawMessage(`{"script":"print(1)","requirements":["--index-url=evil"]}`)); err == nil {
t.Error("flag-shaped requirement must be refused")
}
in, err := fnValidate(json.RawMessage(`{"script":"print(1)"}`))
if err != nil || in.TimeoutSeconds != 3600 {
t.Errorf("default timeout: got %d, err %v", in.TimeoutSeconds, err)
}
in, _ = fnValidate(json.RawMessage(`{"script":"print(1)","timeoutSeconds":999999}`))
if in.TimeoutSeconds != 21600 {
t.Errorf("timeout cap: got %d, want 21600", in.TimeoutSeconds)
}
}
// TestTailBuffer — a chatty training loop keeps only the LAST bytes, flagged.
func TestTailBuffer(t *testing.T) {
tb := &tailBuffer{cap: 8}
tb.Write([]byte("0123456789"))
if string(tb.buf) != "23456789" || !tb.truncated {
t.Errorf("tail = %q truncated=%v", tb.buf, tb.truncated)
}
tb2 := &tailBuffer{cap: 8}
tb2.Write([]byte("abc"))
if string(tb2.buf) != "abc" || tb2.truncated {
t.Errorf("small write mangled: %q %v", tb2.buf, tb2.truncated)
}
}
// TestHasNonRenderLane — echo and studio.render alone keep the render-only
// poison guard; a registered fn.run lane lifts it.
func TestHasNonRenderLane(t *testing.T) {
w := &worker{handlers: map[string]jobHandler{"echo": echoHandler, studioCap: nil}}
if w.hasNonRenderLane() {
t.Error("render-only worker must report no extra lane")
}
w.handlers[fnCap] = fnRunHandler
if !w.hasNonRenderLane() {
t.Error("fn.run lane must lift the guard")
}
}
// TestFnRunSmoke — the real handler end-to-end against the local uv: a script
// that prints and exits 0. Skipped where uv is absent.
func TestFnRunSmoke(t *testing.T) {
if !uvPresent() {
t.Skip("uv not installed")
}
res, err := fnRunHandler(t.Context(), json.RawMessage(`{"script":"print(2+2)","timeoutSeconds":120}`))
if err != nil {
t.Fatalf("fn.run: %v", err)
}
m := res.(map[string]any)
if !strings.Contains(m["output"].(string), "4") {
t.Errorf("output = %q, want it to contain 4", m["output"])
}
}
func uvPresent() bool {
_, err := exec.LookPath("uv")
return err == nil
}
+60
View File
@@ -0,0 +1,60 @@
package cli
// gpu_register_test.go — registration resilience. Ensuring the fleet/jobs namespaces
// is best-effort: those namespaces already exist in any live org, so a transient API
// failure there must NOT abort registration. It used to be fatal, and with systemd
// Restart=always/RestartSec=5 a single 503 became an infinite crash-loop (observed at
// 141 restarts with the machine offline throughout). The presence write is the step
// that actually matters, so a real failure there must still surface.
import (
"context"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// stubRegisterCloud fails every namespace-ensure with the given status, and answers
// the presence-activity write with presenceStatus.
func stubRegisterCloud(t *testing.T, nsStatus, presenceStatus int, sawPresence *bool) *httptest.Server {
t.Helper()
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case r.Method == http.MethodPost && strings.HasSuffix(r.URL.Path, "/v1/tasks/namespaces"):
w.WriteHeader(nsStatus)
case r.Method == http.MethodPost && strings.HasSuffix(r.URL.Path, "/activities"):
*sawPresence = true
w.WriteHeader(presenceStatus)
default:
w.WriteHeader(http.StatusOK)
}
}))
}
// A flaky namespace-ensure must not abort registration: the worker still writes its
// presence record and stays online. This is the crash-loop regression guard.
func TestRegisterSurvivesNamespaceEnsureFailure(t *testing.T) {
var sawPresence bool
srv := stubRegisterCloud(t, http.StatusServiceUnavailable, http.StatusOK, &sawPresence)
defer srv.Close()
if err := testWorker(t, srv.URL).register(context.Background()); err != nil {
t.Fatalf("register() = %v, want nil (namespace-ensure is best-effort)", err)
}
if !sawPresence {
t.Fatal("register() never wrote the presence record")
}
}
// The presence write is the load-bearing step — when it fails, register must fail so
// the operator sees a real error instead of a silently-offline machine.
func TestRegisterFailsWhenPresenceWriteFails(t *testing.T) {
var sawPresence bool
srv := stubRegisterCloud(t, http.StatusOK, http.StatusInternalServerError, &sawPresence)
defer srv.Close()
if err := testWorker(t, srv.URL).register(context.Background()); err == nil {
t.Fatal("register() = nil, want an error when the presence write fails")
}
}
+2 -1
View File
@@ -72,9 +72,10 @@ func newLinkCmd(envOf func() *Env, _ *globalFlags) *cobra.Command {
f.BoolVar(&opts.serveEngine, "serve-engine", false, "also advertise a hanzo-engine model server (OpenAI + Anthropic) running on this node")
f.StringVar(&opts.engineURL, "engine-url", defaultEngineURL, "local URL where hanzo-engine is probed (GET /v1/models)")
f.StringVar(&opts.engineEndpoint, "engine-endpoint", "", "public URL to advertise for gateway routing (defaults to --engine-url; a node behind NAT needs a reachable URL/tunnel)")
f.BoolVar(&opts.registerProvider, "register-provider", false, "auto-register the engine endpoint as an org model provider (POST /v1/add-provider)")
f.BoolVar(&opts.registerProvider, "register-provider", false, "auto-register the engine endpoint as an org model provider (POST /v1/ai/providers)")
f.StringVar(&opts.studioDir, "studio-dir", os.Getenv("HANZO_STUDIO_DIR"), "local Hanzo Studio checkout; when set, link launches and supervises the render backend on 127.0.0.1:8188")
f.StringVar(&opts.studioURL, "studio-url", firstNonEmpty(os.Getenv("HANZO_STUDIO_UPLOAD_URL"), defaultStudioUploadURL), "studio base URL the render mirror uploads finished images to (POST /v1/library/upload)")
f.BoolVar(&opts.mirror, "mirror", true, "sweep local renders into the org studio library; --mirror=false serves jobs only")
return cmd
}
+42 -1
View File
@@ -32,8 +32,33 @@ const (
studioProbeEvery = 45 * time.Second
studioGraceWait = 8 * time.Second
studioStartWindow = 180 * time.Second
// studioBootGrace is how long a freshly launched studio may stay silent before
// the liveness counter is allowed to call it dead. It exists because a booting
// studio and a dead one look IDENTICAL to the probes: while the 41GB Qwen-Edit
// model pages in, the HTTP server is not listening, so studioBusy() reports
// ok=false and the "alive-busy" guard cannot protect it either. With only
// studioStartWindow (180s) plus three 45s ticks, the supervisor killed the
// process ~5min into a load that needs 5-11min on a loaded box — then relaunched
// it into the same fate. A silent loop that renders nothing and never converges
// (observed: 5 restarts in 30 minutes, zero output, jobs failing "engine not up").
//
// This only defers the verdict for a process that is still ALIVE. One that has
// actually exited is restarted immediately, grace or not — see the loop below.
studioBootGrace = 15 * time.Minute
)
// studioExited reports whether the supervised child is gone (crashed or was
// killed). Liveness may fire immediately in that case; boot grace is only ever
// extended to a process that still exists and is presumably loading its model.
// Signal 0 performs the permission/existence check without delivering anything.
func studioExited(cmd *exec.Cmd) bool {
if cmd == nil || cmd.Process == nil {
return true
}
return cmd.Process.Signal(syscall.Signal(0)) != nil
}
func studioHealthy(ctx context.Context) bool {
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
@@ -72,9 +97,17 @@ func launchStudio(dir string) (*exec.Cmd, error) {
// (localComfyUI) so binding wider bought nothing but an open, unauthenticated
// /prompt — the hidden-run hole. --worker-mode makes the studio gate its submit
// seam (/v1/worker/execute + X-Worker-Token) so only the worker can start a render.
// VRAM mode: default --normalvram (safe for smaller BYO GPUs); override with
// HANZO_STUDIO_VRAM (e.g. "--highvram") on big-memory boxes (GB10 128G unified) so
// the Qwen text-encoder stays resident on-GPU instead of non-deterministically
// offloading to CPU — offload makes renders CPU-bound and ~8x slower.
vramMode := os.Getenv("HANZO_STUDIO_VRAM")
if vramMode == "" {
vramMode = "--normalvram"
}
cmd := exec.Command(studioPython(dir), "main.py",
"--listen", "127.0.0.1", "--port", "8188", "--worker-mode",
"--normalvram", "--disable-auto-launch",
vramMode, "--disable-auto-launch",
"--output-directory", filepath.Join(dir, "output"))
cmd.Dir = dir
cmd.Env = append(os.Environ(),
@@ -160,6 +193,7 @@ func studioBusy(ctx context.Context) (busy, ok bool) {
// ends. Quiet by design: one line per restart event, not a probe firehose.
func superviseStudio(ctx context.Context, dir string, out io.Writer) {
var cmd *exec.Cmd
var launchedAt time.Time
restart := func(reason string) {
stopStudio(cmd)
c, err := launchStudio(dir)
@@ -168,6 +202,7 @@ func superviseStudio(ctx context.Context, dir string, out io.Writer) {
return
}
cmd = c
launchedAt = time.Now()
deadline := time.Now().Add(studioStartWindow)
for time.Now().Before(deadline) && ctx.Err() == nil {
if studioHealthy(ctx) {
@@ -219,6 +254,12 @@ func superviseStudio(ctx context.Context, dir string, out io.Writer) {
unhealthy = 0
continue
}
// A live process still inside its boot grace is LOADING, not dead: the
// model takes minutes to page in and serves nothing until it lands.
// Killing it here restarts the same slow load and never converges.
if !studioExited(cmd) && time.Since(launchedAt) < studioBootGrace {
continue
}
// Sustained silence with an idle or unreadable queue = actually dead.
unhealthy++
if unhealthy < 3 {
+61
View File
@@ -0,0 +1,61 @@
package cli
// studio_bootgrace_test.go — a studio that is still loading its model must not be
// mistaken for a dead one.
//
// The probes cannot tell the two apart: while the 41GB Qwen-Edit model pages in, the
// HTTP server is not listening, so studioHealthy() is false AND studioBusy() returns
// ok=false, which disqualifies the "alive-busy" guard. The supervisor therefore killed
// the process ~5 minutes into a load that needs 5-11 minutes and relaunched it into the
// same fate — 5 restarts in 30 minutes with zero renders. The distinguishing fact is
// whether the process still EXISTS, which is what studioExited answers.
import (
"os/exec"
"testing"
"time"
)
// A live-but-silent child inside the grace window must be left alone; the same child
// once the grace has elapsed must be declared dead so a truly wedged studio recovers.
func TestBootGraceSpareLiveStudioNotDeadOne(t *testing.T) {
cmd := exec.Command("sleep", "300")
if err := cmd.Start(); err != nil {
t.Fatalf("start stub studio: %v", err)
}
defer func() { _ = cmd.Process.Kill(); _, _ = cmd.Process.Wait() }()
if studioExited(cmd) {
t.Fatal("studioExited = true for a running process, want false")
}
// This is the exact predicate the supervisor uses to skip the death counter.
spared := func(launchedAt time.Time) bool {
return !studioExited(cmd) && time.Since(launchedAt) < studioBootGrace
}
if !spared(time.Now()) {
t.Error("a live studio that just launched was not spared — this is the restart loop")
}
if spared(time.Now().Add(-studioBootGrace - time.Minute)) {
t.Error("a live studio past its boot grace was spared — a wedged studio would never recover")
}
}
// A process that actually died must be restartable immediately, regardless of grace,
// so a crash is not papered over for 15 minutes.
func TestExitedStudioIsNeverSpared(t *testing.T) {
cmd := exec.Command("true")
if err := cmd.Run(); err != nil { // Run waits, so it has definitely exited
t.Fatalf("run stub: %v", err)
}
if !studioExited(cmd) {
t.Fatal("studioExited = false for an exited process, want true")
}
if !studioExited(nil) {
t.Fatal("studioExited(nil) = false, want true (nothing launched yet)")
}
// Freshly "launched" yet already dead => must NOT be spared.
if !studioExited(cmd) && time.Since(time.Now()) < studioBootGrace {
t.Error("an exited studio was spared by boot grace; a crash would stall the worker")
}
}
+65 -19
View File
@@ -18,7 +18,7 @@
// gateway-minted, IAM-verified X-User-Id; a client-forged X-Org-Id on the bearer-less
// path is refused):
//
// GET /v1/iam/keys — whether the caller has an `hk-` key (+ prefix/mtime); no secret.
// GET /v1/iam/keys — whether the caller has a Cloud API key (+ prefix/mtime); no secret.
// POST /v1/iam/keys — mint/rotate the key; returns { accessKey } ONCE.
// DELETE /v1/iam/keys — revoke the key.
// POST /v1/iam/onboard — create the caller's org (+ move them in on first run).
@@ -50,6 +50,7 @@
package account
import (
"context"
"encoding/base64"
"errors"
"fmt"
@@ -103,9 +104,9 @@ func newService(deps cloud.Deps) *cloud.Service[state] {
// MountAccount wires the SPECIFIC self-service routes (order 48) — the ones that must
// win over the IAM /v1/iam/* wildcard (50) and the commerce embed (100).
func MountAccount(app *zip.App, deps cloud.Deps) error {
func MountAccount(app cloud.Router, deps cloud.Deps) error {
if app == nil {
return fmt.Errorf("account.MountAccount: nil zip.App")
return fmt.Errorf("account.MountAccount: nil app")
}
if deps.Logger == nil {
return fmt.Errorf("account.MountAccount: nil deps.Logger")
@@ -119,9 +120,9 @@ func MountAccount(app *zip.App, deps cloud.Deps) error {
// MountBridge wires the CATCH-ALL data bridges (order 122) — the /v1/billing/* and
// /v1/commerce/* proxies that must sit AFTER clients/billing (121) + the commerce embed.
func MountBridge(app *zip.App, deps cloud.Deps) error {
func MountBridge(app cloud.Router, deps cloud.Deps) error {
if app == nil {
return fmt.Errorf("account.MountBridge: nil zip.App")
return fmt.Errorf("account.MountBridge: nil app")
}
if deps.Logger == nil {
return fmt.Errorf("account.MountBridge: nil deps.Logger")
@@ -133,11 +134,11 @@ func MountBridge(app *zip.App, deps cloud.Deps) error {
}
// routesAccount wires the specific self-service routes (order 48).
func routesAccount(s *cloud.Service[state], app *zip.App) {
func routesAccount(s *cloud.Service[state], app cloud.Router) {
// GET /v1/csrf issues the anti-CSRF token the embedded SPA echoes as X-CSRF-Token on
// every money write (csrf.go). Safe (read-only), same-origin.
app.Get("/v1/csrf", cloud.Handle(s, issueCSRFToken))
// The caller's own `hk-` Cloud API key — IAM self-service. These SPECIFIC routes MUST
// The caller's own Cloud API key — IAM self-service. These SPECIFIC routes MUST
// register before clients/iam's /v1/iam/* wildcard (order 50 > 48) so Fiber's
// first-match scan hits the native handler, not the wildcard (TestIAMKeysBeatsWildcard).
// Reads are open; every state-changing WRITE is wrapped: requireCSRF blocks a
@@ -153,10 +154,15 @@ func routesAccount(s *cloud.Service[state], app *zip.App) {
// that must beat the /v1/commerce/* bridge (122) AND the commerce embed (100) — so it
// mounts here at 48, ahead of both.
app.Post("/v1/commerce/topup/wallet", rateLimit(s, s.State.writesRL, requireCSRF(s, cloud.Handle(s, walletTopup))))
// The accepted rails are public on-chain data (chain, token, treasury), read by
// the browser to render the send UI. A GET with no side effects and no secret,
// so it needs neither CSRF nor the write limiter — but it MUST sit beside the
// POST at this priority, or the /v1/commerce/* bridge swallows it.
app.Get("/v1/commerce/topup/rails", cloud.Handle(s, topupRails))
}
// routesBridge wires the per-tenant catch-all data bridges (order 122).
func routesBridge(s *cloud.Service[state], app *zip.App) {
func routesBridge(s *cloud.Service[state], app cloud.Router) {
// Per-tenant billing DATA bridge — the canonical /v1/billing/* the statically-exported
// console calls, forwarded to commerce with the admin service token and SCOPED to the
// validated caller's own subject (billing.go). Registered AFTER clients/billing's
@@ -243,7 +249,7 @@ func resolveCaller(c *zip.Ctx, requireOwner bool) (caller, bool) {
return caller{id: id, owner: owner, name: name, username: username}, true
}
// ── keys (the per-user `hk-` Cloud API key) ──────────────────────────────────
// ── keys (the per-user Cloud API key) ────────────────────────────────────────
type keyStatus struct {
HasKey bool `json:"hasKey"`
@@ -251,7 +257,7 @@ type keyStatus struct {
CreatedAt string `json:"createdAt,omitempty"`
}
// getKey reports whether the caller has an `hk-` key, its public prefix, and when
// getKey reports whether the caller has a Cloud API key, its public prefix, and when
// the key row last changed — NO secret material. Reads IAM authoritatively (not the
// session claim, which lags a fresh key). Mirrors GET app/keys/route.ts.
func getKey(s *cloud.Service[state], c *zip.Ctx) error {
@@ -280,7 +286,7 @@ func getKey(s *cloud.Service[state], c *zip.Ctx) error {
return c.JSON(http.StatusOK, keyStatus{HasKey: true, KeyPrefix: prefix, CreatedAt: uk.UpdatedTime})
}
// mintKey (re)generates the caller's `hk-` key and returns it ONCE (show-once). A
// mintKey (re)generates the caller's Cloud API key and returns it ONCE (show-once). A
// real IAM failure surfaces as 502 (never a fabricated key). Mirrors POST app/keys.
func mintKey(s *cloud.Service[state], c *zip.Ctx) error {
cr, ok := resolveCaller(c, true)
@@ -297,7 +303,7 @@ func mintKey(s *cloud.Service[state], c *zip.Ctx) error {
return c.JSON(http.StatusOK, map[string]string{"accessKey": key})
}
// revokeKey clears the caller's `hk-` key. Mirrors DELETE app/keys.
// revokeKey clears the caller's Cloud API key. Mirrors DELETE app/keys.
func revokeKey(s *cloud.Service[state], c *zip.Ctx) error {
cr, ok := resolveCaller(c, true)
if !ok {
@@ -367,18 +373,58 @@ func onboard(s *cloud.Service[state], c *zip.Ctx) error {
return herr
}
// Create the org (cloning the caller's current org for password/locale
// compatibility), then — first-run only — move the zero-org user in as admin.
// ADDITIONAL org (caller already has a home): create it WITHOUT moving them —
// they reach it via the OrgSwitcher (a move would strip their SuperAdmin / orphan
// their current org).
if additional {
org := buildOrg(s, c, slug, displayName, body.Personal, cr.owner)
if err := s.State.iam.createOrganization(c.Context(), org); err != nil {
return zip.Errorf(http.StatusBadGateway, "could not create the organization: %v", err)
}
return c.JSON(http.StatusOK, onboardResp{Org: slug, DisplayName: displayName, Additional: true})
}
// FIRST-RUN: drive the ONE atomic IAM provision (org + admin move + hashed
// org-scoped credential), replacing the create-org + move-user pair — a mid-flight
// retry now converges on the founder's own org instead of orphaning it. The org
// starts at a zero balance (usage is pre-paid). Prefer it whenever the
// service-token path is wired; fall back to the legacy pair only when it is not,
// so a partial deploy still onboards.
if s.State.iam.provisionReady() {
resp, err := onboardFirstRun(c.Context(), s.State.iam, cr.id, slug, displayName, body.Personal)
if err != nil {
return err
}
return c.JSON(http.StatusOK, resp)
}
// Legacy fallback (service token unset): create then move — the non-atomic pair.
org := buildOrg(s, c, slug, displayName, body.Personal, cr.owner)
if err := s.State.iam.createOrganization(c.Context(), org); err != nil {
return zip.Errorf(http.StatusBadGateway, "could not create the organization: %v", err)
}
if !additional {
if err := s.State.iam.moveUserToOrg(c.Context(), cr.id, slug); err != nil {
return zip.Errorf(http.StatusBadGateway, "org created but could not assign you to it: %v", err)
}
if err := s.State.iam.moveUserToOrg(c.Context(), cr.id, slug); err != nil {
return zip.Errorf(http.StatusBadGateway, "org created but could not assign you to it: %v", err)
}
return c.JSON(http.StatusOK, onboardResp{Org: slug, DisplayName: displayName, Additional: additional})
return c.JSON(http.StatusOK, onboardResp{Org: slug, DisplayName: displayName, Additional: false})
}
// onboardFirstRun drives the ONE atomic IAM provision for a zero-org caller (create
// org + move them in as admin + mint the hashed org-scoped credential), replacing
// the create-org + move-user pair so a mid-flight retry converges on the founder's
// own org instead of orphaning it. The org starts at a ZERO balance — usage is
// pre-paid, so there is no signup grant. Split out so the provisioning glue is
// unit-tested against mock IAM without the CSRF/routing/principal shell.
func onboardFirstRun(ctx context.Context, iam *iamClient, callerID, slug, displayName string, personal bool) (onboardResp, error) {
row, err := iam.getUserRow(ctx, callerID)
if err != nil {
return onboardResp{}, zip.Errorf(http.StatusBadGateway, "could not resolve the user: %v", err)
}
res, err := iam.provision(ctx, row.Owner, row.Name, slug, personal)
if err != nil {
return onboardResp{}, zip.Errorf(http.StatusBadGateway, "could not provision the organization: %v", err)
}
return onboardResp{Org: res.Org, DisplayName: displayName, Additional: false}, nil
}
// resolveOnboardName derives the base slug + display name from the request, or a
+1 -1
View File
@@ -340,7 +340,7 @@ func billingData(s *cloud.Service[state], c *zip.Ctx) error {
// could not otherwise wield. Safety rests on the edge: the gateway 401s a public Bearer
// that is not an IAM JWT / hk-|pk-|sk- API key (the 64-hex service token is a JWT
// candidate that fails to parse), so an EXTERNAL client can never reach this handler
// holding it — only in-proc commerceinproc dispatch does. Constant-time compare; the token
// holding it — only in-proc commerce-transport dispatch does. Constant-time compare; the token
// is never logged.
func s2sBillingCall(c *zip.Ctx) bool {
_, token := commerceCreds()
+16 -2
View File
@@ -46,8 +46,12 @@ import (
// - Neither: refuse. A bearer-less request with a forged X-Org-Id has no validated
// principal and is fail-closed here, before the read handler runs.
//
// The pin rewrites the request URI's query string in place; fasthttp's SetQueryString
// resets the parsed-args cache, so the handler's later c.Query() reads the pinned values.
// The pin rewrites the request URI's query string AND (on a write) the JSON body in place;
// fasthttp's SetQueryString resets the parsed-args cache and SetBody replaces the body
// bytes, so the handler's later c.Query() / c.Bind() read the pinned values. Pinning the
// body is what keeps a co-resident WRITE handler that reads its subject from the body
// (commerce's CreatePaymentMethod reads customerId from the JSON body) IDOR-safe — query-only
// pinning would leave a client-named customerId/userId in the body untouched.
func PinBillingSubject() zip.Handler {
return func(c *zip.Ctx) error {
inQuery, _ := url.ParseQuery(string(c.Fiber().Request().URI().QueryString()))
@@ -69,7 +73,17 @@ func PinBillingSubject() zip.Handler {
Account: principal.BillingAccount(c),
}).Subject()
// Pin the subject on BOTH the query AND the write body — the SAME two-helper
// scoping billingData applies (scopedBillingSearch + scopedBillingBody). A
// co-resident WRITE handler that reads its subject from the body (commerce's
// CreatePaymentMethod → customerId) is only IDOR-safe if the body is pinned too.
// scopedBillingBody overwrites the subject keys and preserves every other field
// (card, type, sourceId, …); a non-JSON / empty body is returned unchanged, so a
// GET read carries no body and is unaffected.
c.Fiber().Request().URI().SetQueryString(scopedBillingSearch(inQuery, subject).Encode())
if len(c.Body()) > 0 {
c.Fiber().Request().SetBody(scopedBillingBody(c.Body(), subject))
}
return c.Next()
}
}
@@ -29,6 +29,17 @@ func echoQuery(c *zip.Ctx) error {
})
}
// echoBody is the downstream stand-in for a commerce WRITE handler (e.g. CreatePaymentMethod)
// that reads its subject from the JSON BODY: it reports the body it observes AFTER the pin, so
// a test can assert the subject the handler would persist and that non-subject fields survive.
func echoBody(c *zip.Ctx) error {
var got map[string]any
if len(c.Body()) > 0 {
_ = json.Unmarshal(c.Body(), &got)
}
return c.JSON(200, got)
}
func pinApp(t *testing.T) *zip.App {
t.Helper()
app := zip.New(zip.Config{Logger: luxlog.New("test")})
@@ -38,6 +49,7 @@ func pinApp(t *testing.T) *zip.App {
t.Fatalf("MountAccount: %v", err)
}
app.Get("/probe", PinBillingSubject(), echoQuery)
app.Post("/probe", PinBillingSubject(), echoBody)
return app
}
@@ -67,6 +79,33 @@ func TestPinBillingSubject_PinsCallerAndDropsOrg(t *testing.T) {
}
}
// TestPinBillingSubject_PinsBodyForWrites — a POST whose JSON body names a FOREIGN subject
// (customerId/userId/user) has every subject key overwritten with the caller's OWN subject
// before the handler binds it, while non-subject fields survive. This is the boundary
// commerce's CreatePaymentMethod (which reads customerId from the body) relies on to be
// IDOR-safe co-resident — byte-identical to billingData's scopedBillingBody on the bridge.
func TestPinBillingSubject_PinsBodyForWrites(t *testing.T) {
app := pinApp(t)
code, body := callH(t, app, http.MethodPost, "/probe", alice,
`{"customerId":"victim","userId":"victim","user":"victim","card":{"last4":"4242"}}`)
if code != http.StatusOK {
t.Fatalf("want 200, got %d (%s)", code, body)
}
var got map[string]any
if err := json.Unmarshal(body, &got); err != nil {
t.Fatalf("bad body: %s", body)
}
for _, k := range billingSubjectKeys {
if got[k] != "acme" { // alice/acme resolves to the org subject "acme"
t.Fatalf("handler must see body %s=acme (caller's own subject), got %v", k, got[k])
}
}
card, ok := got["card"].(map[string]any)
if !ok || card["last4"] != "4242" {
t.Fatalf("non-subject body field must survive the pin, got card=%v", got["card"])
}
}
// TestPinBillingSubject_RefusesUnvalidated — a forged X-Org-Id with NO validated
// X-User-Id (and no service token) is refused before the read handler runs: no
// cross-tenant billing read is possible.
+17
View File
@@ -173,6 +173,23 @@ func requireCSRF(s *cloud.Service[state], next zip.Handler) zip.Handler {
}
}
// RequireCSRF exposes the ambient-cookie anti-CSRF gate as a STANDALONE middleware for a
// co-resident money-WRITE route registered OUTSIDE this package — specifically
// apps/commerce.go's POST /v1/billing/topup/token, which shadows the account-bridge's
// POST /v1/billing/* wildcard (order 100 < 122) that would otherwise have wrapped the
// write in requireCSRF. Moving the route co-resident to break the commerce transport
// self-dispatch loop must NOT silently drop that anti-CSRF gate, so the identical
// enforcement rides along as its own handler. It binds to the SAME process-wide key
// (sharedCSRFKey) the GET /v1/csrf issuer and the bridge verifier use, so a token minted
// at /v1/csrf verifies here byte-identically. Enforces ONLY on the ambient-cookie path (a
// Bearer/gateway/API caller is not CSRF-able); on success it c.Next()s into the rest of
// the chain. The minimal Service carries only the shared key — requireCSRF/verifyCSRF
// read nothing else off it.
func RequireCSRF() zip.Handler {
s := &cloud.Service[state]{State: state{csrfKey: sharedCSRFKey(nil)}}
return requireCSRF(s, func(c *zip.Ctx) error { return c.Next() })
}
// issueCSRFToken serves GET /v1/csrf: for a VALIDATED caller, a fresh token
// bound to their identity. no-store so it is never cached by a shared proxy. This is
// the same-origin endpoint the embedded SPA reads (its response body is unreadable to
+88 -6
View File
@@ -1,7 +1,7 @@
// iam.go is the ONE HTTP path from the console subsystem to Hanzo IAM, acting as
// the confidential first-party `hanzo-console` client (client_secret_basic). It
// ports the privileged IAM primitives that console's server-only
// src/lib/server/identity.ts drove — mint/revoke/get the per-user `hk-` key and
// src/lib/server/identity.ts drove — mint/revoke/get the per-user Cloud API key and
// create/read/update an organization — so those standalone Next server routes can
// be retired and console statically exported (task #41, "True 1-binary FE").
//
@@ -44,6 +44,7 @@ const iamMaxBody = 4 << 20
// iamClient is the confidential-client caller. clientID/clientSecret authenticate
// as the `hanzo-console` app; an empty pair means "not configured" (handlers 501).
type iamClient struct {
serviceToken string // IAM_SERVICE_TOKEN — the Bearer for the admin provision endpoint
base string
clientID string
clientSecret string
@@ -56,10 +57,91 @@ func newIAMClient() *iamClient {
base: base,
clientID: strings.TrimSpace(os.Getenv("IAM_MINT_CLIENT_ID")),
clientSecret: strings.TrimSpace(os.Getenv("IAM_MINT_CLIENT_SECRET")),
serviceToken: strings.TrimSpace(os.Getenv("IAM_SERVICE_TOKEN")),
http: &http.Client{Timeout: 15 * time.Second},
}
}
// provisionResult is the /v1/iam/admin/provision response: the converged org and its
// hashed credential (accessSecret shown ONCE on first mint). The org starts at a zero
// balance — usage is pre-paid, no signup grant.
type provisionResult struct {
Org string `json:"org"`
AccessKey string `json:"accessKey"`
AccessSecret string `json:"accessSecret"`
Error string `json:"error"`
}
// provisionReady reports whether the service-token provisioning path is wired.
func (c *iamClient) provisionReady() bool { return c != nil && c.serviceToken != "" }
// provision drives the ONE atomic IAM onboarding op: create the org, move the named
// user in as its admin, and mint its hashed org-scoped credential — the service-token
// endpoint that replaces the create-org + move-user pair, so there is no orphan
// between two writes and a mid-flight retry converges. orgSlug is the caller's
// already-resolved slug (IAM honors it verbatim). The org starts at a zero balance.
func (c *iamClient) provision(ctx context.Context, owner, name, orgSlug string, personal bool) (provisionResult, error) {
if !c.provisionReady() {
return provisionResult{}, errNotConfigured
}
body, err := json.Marshal(map[string]any{
"owner": owner, "name": name, "orgSlug": orgSlug, "personal": personal,
})
if err != nil {
return provisionResult{}, err
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.base+"/v1/iam/admin/provision", strings.NewReader(string(body)))
if err != nil {
return provisionResult{}, err
}
req.Header.Set("Authorization", "Bearer "+c.serviceToken)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
resp, err := c.http.Do(req)
if err != nil {
return provisionResult{}, fmt.Errorf("iam unreachable: %w", err)
}
defer func() { _ = resp.Body.Close() }()
raw, err := io.ReadAll(io.LimitReader(resp.Body, iamMaxBody))
if err != nil {
return provisionResult{}, err
}
var out provisionResult
if err := json.Unmarshal(raw, &out); err != nil {
return provisionResult{}, fmt.Errorf("iam provision non-json response (%d)", resp.StatusCode)
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 || out.Error != "" {
msg := out.Error
if msg == "" {
msg = fmt.Sprintf("iam status %d", resp.StatusCode)
}
return provisionResult{}, fmt.Errorf("iam provision: %s", msg)
}
return out, nil
}
// userRow is the subset of an IAM user the onboarding path reads to resolve the
// caller's authoritative (owner, name) — a zero-org caller's owner is not on its
// token, so provision needs it from the row.
type userRow struct {
Owner string `json:"owner"`
Name string `json:"name"`
}
// getUserRow resolves the user by the caller's id (the same read the move did) into
// its authoritative (owner, name).
func (c *iamClient) getUserRow(ctx context.Context, id string) (userRow, error) {
raw, err := c.getUser(ctx, id)
if err != nil {
return userRow{}, err
}
var row userRow
if err := json.Unmarshal(raw, &row); err != nil {
return userRow{}, fmt.Errorf("iam get-user: decode: %w", err)
}
return row, nil
}
// configured reports whether the confidential client is wired. Handlers 501 when
// false — the deployment simply lacks the `hanzo-console` credential (the honest
// "not configured on this deployment" state, never a fabricated result).
@@ -130,16 +212,16 @@ func (c *iamClient) do(ctx context.Context, method, path string, q url.Values, b
return env, nil
}
// ── the `hk-` Cloud API key (per-user) ───────────────────────────────────────
// ── the Cloud API key (per-user) ─────────────────────────────────────────────
// userKey is the subset of an IAM user row the key surface reads: the `hk-` access
// userKey is the subset of an IAM user row the key surface reads: the access
// key and the last time the key row changed (updatedTime).
type userKey struct {
AccessKey string `json:"accessKey"`
UpdatedTime string `json:"updatedTime"`
}
// getUserKey reads a user's CURRENT `hk-` key AUTHORITATIVELY from IAM (get-user).
// getUserKey reads a user's CURRENT Cloud API key AUTHORITATIVELY from IAM (get-user).
// The session claim can lag a freshly-minted key (it returns ”) — so GET /keys
// must read IAM, not the claim (the "key never listed" bug identity.ts documents).
// `id` is the `<owner>/<name>` composite IAM parses.
@@ -155,7 +237,7 @@ func (c *iamClient) getUserKey(ctx context.Context, id string) (userKey, error)
return u, nil
}
// mintUserKey (re)generates the user's `hk-` key and returns the new secret — shown
// mintUserKey (re)generates the user's Cloud API key and returns the new secret — shown
// ONCE to the caller (POST /keys), never echoed again. IAM binds the key to `id`,
// so a caller can only ever mint their OWN.
func (c *iamClient) mintUserKey(ctx context.Context, id string) (string, error) {
@@ -175,7 +257,7 @@ func (c *iamClient) mintUserKey(ctx context.Context, id string) (string, error)
return out.AccessKey, nil
}
// revokeUserKey clears the user's `hk-` key (immediate revoke; the gateway key
// revokeUserKey clears the user's Cloud API key (immediate revoke; the gateway key
// cache lapses within ~5m).
func (c *iamClient) revokeUserKey(ctx context.Context, id string) error {
_, err := c.do(ctx, http.MethodPost, "/v1/iam/revoke-user-keys", url.Values{"id": {id}}, nil)
+64
View File
@@ -0,0 +1,64 @@
package account
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"testing"
)
// TestOnboardFirstRun_ProvisionsOnce proves the production signup path drives ONE
// atomic IAM provision (not the old create-org + move-user pair): it resolves the
// zero-org caller's authoritative (owner, name) and provisions the org + hashed
// credential in a single call. No trial credit is granted — the org starts at a zero
// balance and usage is pre-paid.
func TestOnboardFirstRun_ProvisionsOnce(t *testing.T) {
var provisionCalls int
var provBody map[string]any
iamSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v1/iam/get-user":
_ = json.NewEncoder(w).Encode(map[string]any{
"status": "ok",
"data": map[string]any{"owner": "landing", "name": "dave"},
})
case "/v1/iam/admin/provision":
provisionCalls++
b, _ := io.ReadAll(r.Body)
_ = json.Unmarshal(b, &provBody)
_ = json.NewEncoder(w).Encode(map[string]any{
"org": "dave", "accessKey": "hk-x", "accessSecret": "sk-x",
})
default:
http.NotFound(w, r)
}
}))
defer iamSrv.Close()
iam := &iamClient{base: iamSrv.URL, clientID: "c", clientSecret: "s", serviceToken: "svc", http: &http.Client{}}
resp, err := onboardFirstRun(context.Background(), iam, "dave", "dave", "Dave", true)
if err != nil {
t.Fatalf("onboardFirstRun: %v", err)
}
if resp.Org != "dave" || resp.Additional {
t.Fatalf("resp = %+v, want org=dave additional=false", resp)
}
if provisionCalls != 1 {
t.Fatalf("provision calls = %d, want 1 (ONE atomic op, not create+move)", provisionCalls)
}
if provBody["owner"] != "landing" || provBody["name"] != "dave" || provBody["orgSlug"] != "dave" {
t.Fatalf("provision body = %v, want owner=landing name=dave orgSlug=dave", provBody)
}
// Retry converges: provision is idempotent (same org), no orphan.
if _, err := onboardFirstRun(context.Background(), iam, "dave", "dave", "Dave", true); err != nil {
t.Fatalf("retry: %v", err)
}
if provisionCalls != 2 {
t.Fatalf("provision calls = %d, want 2 (retried, converges)", provisionCalls)
}
}
+190 -63
View File
@@ -1,28 +1,36 @@
// topup.go ports console's app/billing/v1/topup/wallet/route.ts into the unified
// binary at POST /v1/commerce/topup/wallet (task #41). It is the verify-and-record
// seam for an HUSD wallet top-up: the browser sends an HUSD ERC-20 transfer to the
// treasury and posts the tx hash here; this handler reads the receipt from the Hanzo
// EVM, confirms it is a mined, successful HUSD Transfer(from → treasury, value),
// derives USD cents from the (18-decimal, USD-pegged) on-chain value, records it to
// commerce as an `husd` crypto payment, and returns the credited amount + balance.
// topup.go is the verify-and-record seam for a crypto wallet top-up: the browser
// sends a USD-pegged ERC-20 transfer to our treasury and posts the tx hash here;
// this handler reads the receipt from that chain, confirms it is a mined, successful
// Transfer(from → treasury, value), derives USD cents from the on-chain value using
// the TOKEN'S OWN decimals, records it to commerce, and returns the credited amount
// plus the new balance.
//
// A rail is one accepted (chain, token, treasury) triple, configured as data in
// TOPUP_RAILS and discoverable at GET /v1/commerce/topup/rails. This replaced a
// single hardcoded HUSD-on-Hanzo-Mainnet pair: HUSD is not deployed, so the surface
// was permanently 501 — complete, correct and unable to take a cent. Customers
// already hold USDC on Base/Ethereum/Polygon, so accepting the assets they have is
// what makes this earn.
//
// THE CREDITED AMOUNT IS THE ON-CHAIN VALUE, never a client number — which is exactly
// why this MUST be a server handler and cannot collapse to a same-origin call. Two
// hardenings over the Node route:
// why this MUST be a server handler and cannot collapse to a same-origin call. Three
// properties worth keeping:
//
// - IDOR-safe: the credit lands on the VALIDATED caller's own org/user (the
// gateway-verified X-Org-Id/X-User-Id), never a client-supplied `userId`.
// - S2S to commerce: recorded with the admin COMMERCE_SERVICE_TOKEN + the caller's
// X-Org-Id (the same service-to-service pattern clients/admin reads balances on),
// not by forwarding a browser cookie.
// - Per-rail decimals: cents come from 10^(decimals-2), so a 6-decimal USDC and an
// 18-decimal token cannot be priced with one another's divisor.
//
// The EVM receipt is read over plain JSON-RPC (eth_getTransactionReceipt) — one
// well-known call + one well-known event, so the stdlib is sufficient and no EVM
// client dependency is pulled in.
// client dependency is pulled in. That also means a new chain costs no new code.
//
// Honest failure (no fabricated credit, ever): HUSD/treasury unconfigured
// (greenfield — HUSD not yet deployed) → 501; a missing/failed/non-HUSD-to-treasury
// tx → 400; the chain or commerce unreachable → 502.
// Honest failure (no fabricated credit, ever): no rail configured → 501; an unknown
// rail, or a missing/failed/non-matching tx → 400; the chain or commerce unreachable
// → 502.
package account
import (
@@ -40,7 +48,7 @@ import (
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/commerceinproc"
"github.com/hanzoai/cloud/clients/commerce/transport"
"github.com/zap-proto/zip"
)
@@ -53,19 +61,25 @@ var httpClient = &http.Client{Timeout: 15 * time.Second}
// commerceHTTP is the client for the commerce S2S seam ONLY (commerceDo). Separate
// from httpClient (which also dials EVM JSON-RPC) so that — when commerce is folded
// in-process (task #111) — commerce calls dispatch to the in-process handler via
// commerceinproc's self-routing transport (no socket to the standalone), while the
// the commerce transport's self-routing dispatch (no socket to the standalone), while the
// HUSD chain RPC keeps going over the real network. Off the co-resident path it is a
// plain HTTP client, exactly like before.
var commerceHTTP = commerceinproc.Client(15 * time.Second)
var commerceHTTP = transport.Client(15 * time.Second)
// transferTopic is keccak256("Transfer(address,address,uint256)") — the ERC-20
// Transfer event signature, topics[0] of every transfer log. A universally-fixed
// constant (no need to hash at runtime).
const transferTopic = "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"
// husdCentDivisor: HUSD is an 18-decimal, USD-pegged stablecoin, so 1e16 base units
// = 1 cent. Mirrors route.ts's `value / 10n ** 16n`.
var husdCentDivisor = new(big.Int).Exp(big.NewInt(10), big.NewInt(16), nil)
// centDivisor: base units per cent for a `decimals`-place USD-pegged token, i.e.
// 10^(decimals-2). This MUST be per-token, not a constant: HUSD has 18 decimals
// (1e16 per cent) while USDC has 6 (1e4 per cent). Sharing one divisor across both
// would misprice a credit by 10^12 — the difference between a $10 top-up and a
// $10,000,000,000 one — so the token's own decimals are carried on the rail and
// used here.
func centDivisor(decimals int) *big.Int {
return new(big.Int).Exp(big.NewInt(10), big.NewInt(int64(decimals-2)), nil)
}
var (
addrRe = regexp.MustCompile(`^0x[0-9a-fA-F]{40}$`)
@@ -74,43 +88,101 @@ var (
func isAddr(a string) bool { return addrRe.MatchString(a) }
// topupConfig is the deployment's HUSD + commerce wiring, resolved from server-only
// env (sourced from KMS by the deployment, never a browser value).
// rail is ONE accepted way to pay: a USD-pegged ERC-20 on a specific chain, sent to
// a treasury address we control there. Everything a receipt check needs is on the
// rail, so accepting a new chain or token is data, never code.
//
// This replaced a single hardcoded (HUSD, treasury) pair. That pair could only ever
// describe HUSD on Hanzo Mainnet, and since HUSD is not deployed the whole surface
// was permanently 501 — architecturally complete and earning nothing. Customers
// already hold USDC on Base/Ethereum/Polygon, so the rail set is what makes the
// path able to take money at all.
type rail struct {
// Stable id the client names when submitting, e.g. "base-usdc".
ID string `json:"id"`
// Human chain name for the UI, e.g. "Base".
Chain string `json:"chain"`
// EIP-155 chain id — the wallet must be on this chain.
ChainID int64 `json:"chainId"`
// JSON-RPC endpoint used to read the receipt. Carried in the CONFIG json (this
// struct is what TOPUP_RAILS decodes into) but never published — the public
// listing is a separate view type, because an input model and an output model
// are different things and collapsing them once made this field unsettable.
RPCURL string `json:"rpcUrl"`
// The ERC-20 contract.
Token string `json:"token"`
// Display symbol, e.g. "USDC".
Symbol string `json:"symbol"`
// Token decimals. USDC is 6; an 18-decimal token must say 18.
Decimals int `json:"decimals"`
// Where the customer sends funds on this chain. Public by nature.
Treasury string `json:"treasury"`
}
// ok reports whether a rail is usable. A malformed rail is DROPPED rather than
// rejected at request time, so one bad entry cannot take the whole surface down —
// and a rail with impossible decimals cannot silently misprice a credit.
func (r rail) ok() bool {
return r.ID != "" && isAddr(r.Token) && isAddr(r.Treasury) && r.RPCURL != "" &&
r.Decimals >= 2 && r.Decimals <= 36
}
// topupConfig is the deployment's accepted rails + commerce wiring, resolved from
// server-only env (sourced from KMS by the deployment, never a browser value).
type topupConfig struct {
husd string // HUSD ERC-20 contract address
treasury string // the top-up treasury address
rpcURL string // Hanzo EVM JSON-RPC endpoint
chainID int64
rails []rail
commerce string // commerce base (e.g. http://commerce.hanzo.svc.cluster.local:8001)
token string // admin S2S bearer for commerce (COMMERCE_SERVICE_TOKEN; never logged)
}
// TOPUP_RAILS is a JSON array of rails — ONE variable describing the whole accepted
// set, rather than a family of per-token env names that would have to be invented
// again for every chain. Unparseable or malformed entries are dropped.
func loadTopupConfig() topupConfig {
chainID := int64(36963)
if v := strings.TrimSpace(os.Getenv("HANZO_CHAIN_ID")); v != "" {
if n, ok := new(big.Int).SetString(v, 10); ok {
chainID = n.Int64()
var rails []rail
if raw := strings.TrimSpace(os.Getenv("TOPUP_RAILS")); raw != "" {
var parsed []rail
if err := json.Unmarshal([]byte(raw), &parsed); err == nil {
for _, r := range parsed {
r.RPCURL = strings.TrimRight(strings.TrimSpace(r.RPCURL), "/")
if r.ok() {
rails = append(rails, r)
}
}
}
}
return topupConfig{
husd: strings.TrimSpace(os.Getenv("HANZO_HUSD_ADDRESS")),
treasury: strings.TrimSpace(os.Getenv("HANZO_HUSD_TREASURY")),
rpcURL: strings.TrimRight(getenv("HANZO_RPC_URL", "https://rpc.hanzo.network"), "/"),
chainID: chainID,
rails: rails,
commerce: strings.TrimRight(getenv("COMMERCE_URL", "https://api.hanzo.ai"), "/"),
token: strings.TrimSpace(os.Getenv("COMMERCE_SERVICE_TOKEN")),
}
}
// configured reports whether the greenfield gate is satisfied — a valid HUSD contract
// AND treasury. Until HUSD is deployed both are unset and the surface is honestly 501.
func (t topupConfig) configured() bool { return isAddr(t.husd) && isAddr(t.treasury) }
// configured reports whether ANY rail is accepted. With none the surface is honestly
// 501 rather than pretending to take money it cannot verify.
func (t topupConfig) configured() bool { return len(t.rails) > 0 }
// find returns the named rail. An unknown id is a client error, not a server one.
func (t topupConfig) find(id string) (rail, bool) {
for _, r := range t.rails {
if strings.EqualFold(r.ID, id) {
return r, true
}
}
return rail{}, false
}
type walletTopupReq struct {
// Which accepted rail the transfer was sent on, e.g. "base-usdc". The client
// names it rather than the server guessing from the tx: the same address can
// exist on several chains, so inferring would risk crediting against the wrong
// treasury.
Rail string `json:"rail"`
TxHash string `json:"txHash"`
FromAddress string `json:"fromAddress"`
// A client-supplied `userId` is intentionally NOT read — the credit lands on the
// validated caller (no IDOR).
// validated caller (no IDOR). Neither is any amount: the credit is the ON-CHAIN
// value, so a client number could never inflate it.
}
type walletTopupResp struct {
@@ -120,13 +192,52 @@ type walletTopupResp struct {
Status string `json:"status"`
}
// walletTopup verifies a sent HUSD transfer on-chain and credits the caller's org.
// Mirrors POST app/billing/v1/topup/wallet/route.ts.
// topupRails answers GET /v1/commerce/topup/rails: the accepted (chain, token,
// treasury) set, so the browser can render "send USDC here" WITHOUT the addresses
// being baked into its bundle.
//
// This exists because the console previously gated its top-up UI on
// NEXT_PUBLIC_HANZO_HUSD_ADDRESS/_TREASURY — build-time constants. Enabling a rail
// therefore meant rebuilding and redeploying the frontend, and with them unset the
// UI reported "not available yet" no matter what the server could actually accept.
// Serving the set at runtime keeps ONE source of truth (the server's config) and
// lets a rail be switched on without shipping a bundle.
//
// Everything here is public on-chain data; no secret is exposed.
func topupRails(s *cloud.Service[state], c *zip.Ctx) error {
cfg := loadTopupConfig()
// Encode as [] rather than null, so clients can just read .length.
view := make([]railView, 0, len(cfg.rails))
for _, r := range cfg.rails {
view = append(view, railView{
ID: r.ID, Chain: r.Chain, ChainID: r.ChainID,
Token: r.Token, Symbol: r.Symbol, Decimals: r.Decimals, Treasury: r.Treasury,
})
}
return c.JSON(http.StatusOK, map[string]any{"rails": view})
}
// railView is what a browser is told about a rail: everything needed to send funds
// and nothing else. Distinct from `rail` so that adding an operational field to the
// config (an RPC URL, a key reference, a provider credential) cannot leak by merely
// existing — a new field is published only if it is added here on purpose.
type railView struct {
ID string `json:"id"`
Chain string `json:"chain"`
ChainID int64 `json:"chainId"`
Token string `json:"token"`
Symbol string `json:"symbol"`
Decimals int `json:"decimals"`
Treasury string `json:"treasury"`
}
// walletTopup verifies a sent stablecoin transfer on-chain and credits the caller's
// org. The credited amount is the ON-CHAIN value, never a client number.
func walletTopup(s *cloud.Service[state], c *zip.Ctx) error {
cfg := loadTopupConfig()
// Greenfield gate: no HUSD contract / treasury ⇒ honest "not configured yet".
// No accepted rail ⇒ honest "not configured yet" rather than a fake credit.
if !cfg.configured() {
return zip.Errorf(http.StatusNotImplemented, "HUSD top-up is not configured yet (HUSD is not deployed on Hanzo Mainnet)")
return zip.Errorf(http.StatusNotImplemented, "crypto top-up is not configured yet (no payment rail is enabled)")
}
// The credit lands on the VALIDATED caller's own org (X-Org-Id) — require it.
@@ -143,15 +254,26 @@ func walletTopup(s *cloud.Service[state], c *zip.Ctx) error {
if !txHashRe.MatchString(txHash) {
return zip.ErrBadRequest("a valid transaction hash is required")
}
// With exactly one rail the client may omit it; naming it is required as soon as
// there is a choice, so a transfer can never be checked against another chain's
// treasury by default.
railID := strings.TrimSpace(body.Rail)
if railID == "" && len(cfg.rails) == 1 {
railID = cfg.rails[0].ID
}
rl, ok := cfg.find(railID)
if !ok {
return zip.ErrBadRequest("unknown payment rail: name one from GET /v1/commerce/topup/rails")
}
// ── 1. Verify the HUSD transfer on-chain ─────────────────────────────────────
cents, verifiedFrom, herr := verifyHusdTransfer(c.Context(), cfg, txHash, strings.TrimSpace(body.FromAddress))
// ── 1. Verify the transfer on-chain ──────────────────────────────────────────
cents, verifiedFrom, herr := verifyTransfer(c.Context(), rl, txHash, strings.TrimSpace(body.FromAddress))
if herr != nil {
return herr
}
// ── 2. Record to commerce as an HUSD crypto payment (S2S) ────────────────────
status, herr := recordHusdPayment(c.Context(), cfg, cr, txHash, verifiedFrom, cents)
// ── 2. Record to commerce as a crypto payment on this rail (S2S) ─────────────
status, herr := recordCryptoPayment(c.Context(), cfg, rl, cr, txHash, verifiedFrom, cents)
if herr != nil {
return herr
}
@@ -176,13 +298,14 @@ type rpcLog struct {
Data string `json:"data"`
}
// verifyHusdTransfer reads the receipt and confirms exactly one mined, successful
// HUSD Transfer to the treasury, returning the credited cents and the sender. Any
// non-conforming tx is an honest 400; an unreachable chain is a 502.
func verifyHusdTransfer(ctx context.Context, cfg topupConfig, txHash, wantFrom string) (int64, string, error) {
rcpt, err := getReceipt(ctx, cfg.rpcURL, txHash)
// verifyTransfer reads the receipt on the rail's chain and confirms a mined,
// successful Transfer of the rail's token to the rail's treasury, returning the
// credited cents and the sender. Any non-conforming tx is an honest 400; an
// unreachable chain is a 502. Nothing is credited that the chain did not confirm.
func verifyTransfer(ctx context.Context, rl rail, txHash, wantFrom string) (int64, string, error) {
rcpt, err := getReceipt(ctx, rl.RPCURL, txHash)
if err != nil {
return 0, "", zip.Errorf(http.StatusBadGateway, "could not verify the transaction on Hanzo Mainnet: %v", err)
return 0, "", zip.Errorf(http.StatusBadGateway, "could not verify the transaction on %s: %v", rl.Chain, err)
}
if rcpt == nil {
return 0, "", zip.ErrBadRequest("transaction not found or not yet mined")
@@ -191,10 +314,11 @@ func verifyHusdTransfer(ctx context.Context, cfg topupConfig, txHash, wantFrom s
return 0, "", zip.ErrBadRequest("transaction failed on-chain")
}
husd := strings.ToLower(cfg.husd)
treasuryTopic := addrToTopic(cfg.treasury)
token := strings.ToLower(rl.Token)
treasuryTopic := addrToTopic(rl.Treasury)
div := centDivisor(rl.Decimals)
for _, lg := range rcpt.Logs {
if strings.ToLower(lg.Address) != husd {
if strings.ToLower(lg.Address) != token {
continue
}
if len(lg.Topics) < 3 || strings.ToLower(lg.Topics[0]) != transferTopic {
@@ -207,7 +331,7 @@ func verifyHusdTransfer(ctx context.Context, cfg topupConfig, txHash, wantFrom s
if !ok {
continue
}
cents := new(big.Int).Div(value, husdCentDivisor)
cents := new(big.Int).Div(value, div)
if cents.Sign() <= 0 || !cents.IsInt64() {
return 0, "", zip.ErrBadRequest("transferred amount is below the minimum (1 cent)")
}
@@ -217,7 +341,7 @@ func verifyHusdTransfer(ctx context.Context, cfg topupConfig, txHash, wantFrom s
}
return cents.Int64(), from, nil
}
return 0, "", zip.ErrBadRequest("no HUSD transfer to the treasury was found in this transaction")
return 0, "", zip.ErrBadRequest("no " + rl.Symbol + " transfer to the treasury was found in this transaction")
}
// getReceipt calls eth_getTransactionReceipt over JSON-RPC. A null result (not mined)
@@ -277,18 +401,21 @@ func topicToAddr(topic string) string {
// ── commerce (S2S) ───────────────────────────────────────────────────────────────
// recordHusdPayment records the verified credit to commerce as an `husd` crypto
// payment, scoped to the caller's org via the S2S service token + X-Org-Id.
func recordHusdPayment(ctx context.Context, cfg topupConfig, cr caller, txHash, from string, cents int64) (string, error) {
// recordCryptoPayment records the verified credit to commerce, scoped to the
// caller's org via the S2S service token + X-Org-Id. Network, chain, currency and
// destination all come from the RAIL the transfer was verified against, so the
// ledger row describes the payment that actually happened rather than a fixed
// assumption about which chain and token it was.
func recordCryptoPayment(ctx context.Context, cfg topupConfig, rl rail, cr caller, txHash, from string, cents int64) (string, error) {
payload, _ := json.Marshal(map[string]any{
"method": "crypto",
"network": "hanzo",
"chainId": cfg.chainID,
"currency": "husd",
"network": rl.Chain,
"chainId": rl.ChainID,
"currency": strings.ToLower(rl.Symbol),
"amount": cents,
"txHash": txHash,
"fromAddress": from,
"toAddress": cfg.treasury,
"toAddress": rl.Treasury,
"userId": cr.id, // the VALIDATED caller — never a client-supplied id
})
raw, status, err := commerceDo(ctx, cfg.commerce, cfg.token, http.MethodPost, "/v1/billing/payment", nil, cr.owner, payload)
+144 -7
View File
@@ -7,6 +7,7 @@ import (
"math/big"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
)
@@ -81,24 +82,46 @@ func (f *fakeCommerce) server(t *testing.T) *httptest.Server {
return srv
}
// husdReceipt builds a receipt carrying one HUSD Transfer(from→to, cents*1e16).
// husdReceipt builds a receipt carrying one 18-decimal Transfer(from→to, cents*1e16).
func husdReceipt(status, husd, from, to string, cents int64) map[string]any {
value := new(big.Int).Mul(big.NewInt(cents), husdCentDivisor)
return tokenReceipt(status, husd, from, to, cents, 18)
}
// tokenReceipt builds a receipt for a Transfer of `cents` worth of a token with the
// given decimals — the knob that proves a 6-decimal USDC is not priced with an
// 18-decimal divisor.
func tokenReceipt(status, token, from, to string, cents int64, decimals int) map[string]any {
value := new(big.Int).Mul(big.NewInt(cents), centDivisor(decimals))
return map[string]any{
"status": status,
"logs": []any{map[string]any{
"address": husd,
"address": token,
"topics": []any{transferTopic, addrToTopic(from), addrToTopic(to)},
"data": "0x" + fmt.Sprintf("%064x", value),
}},
}
}
func setTopupEnv(t *testing.T, husd, treasury, rpcURL, commerceURL string) {
// setTopupEnv configures ONE 18-decimal rail, preserving these tests' original
// arithmetic (18 decimals ⇒ 1e16 per cent) so they still assert the same cents.
// An empty token/treasury yields no rail at all — the "not configured" case.
func setTopupEnv(t *testing.T, token, treasury, rpcURL, commerceURL string) {
t.Helper()
t.Setenv("HANZO_HUSD_ADDRESS", husd)
t.Setenv("HANZO_HUSD_TREASURY", treasury)
t.Setenv("HANZO_RPC_URL", rpcURL)
setTopupRails(t, commerceURL, rail{
ID: "hanzo-husd", Chain: "Hanzo", ChainID: 36963, RPCURL: rpcURL,
Token: token, Symbol: "HUSD", Decimals: 18, Treasury: treasury,
})
}
// setTopupRails installs an explicit rail set. Malformed rails are dropped by
// loadTopupConfig, which is how the not-configured cases above stay 501.
func setTopupRails(t *testing.T, commerceURL string, rails ...rail) {
t.Helper()
raw, err := json.Marshal(rails)
if err != nil {
t.Fatalf("marshal rails: %v", err)
}
t.Setenv("TOPUP_RAILS", string(raw))
t.Setenv("COMMERCE_URL", commerceURL)
t.Setenv("COMMERCE_SERVICE_TOKEN", "svc-token")
}
@@ -239,3 +262,117 @@ func TestTopup_SenderMismatch_400(t *testing.T) {
t.Fatalf("sender mismatch: want 400, got %d", code)
}
}
// ── rails: the multi-chain generalisation ────────────────────────────────────────
const (
tUSDC = "0x5555555555555555555555555555555555555555"
tTreasBase = "0x6666666666666666666666666666666666666666"
)
func usdcRail(rpcURL string) rail {
return rail{
ID: "base-usdc", Chain: "Base", ChainID: 8453, RPCURL: rpcURL,
Token: tUSDC, Symbol: "USDC", Decimals: 6, Treasury: tTreasBase,
}
}
// The decimals bug this design exists to prevent: USDC has 6 decimals, so 500 cents
// is 5_000_000 base units. Priced with an 18-decimal divisor it would round to ZERO
// and silently credit nothing; priced the other way it would credit 10^12 times too
// much. The rail's own decimals must be what is used.
func TestTopup_USDC_SixDecimals_PricedByRail(t *testing.T) {
rpc := &fakeRPC{result: tokenReceipt("0x1", tUSDC, tSender, tTreasBase, 500, 6)}
com := &fakeCommerce{balance: 500}
setTopupRails(t, com.server(t).URL, usdcRail(rpc.server(t).URL))
app := mountBrand(t, "hanzo")
code, body := callH(t, app, http.MethodPost, "/v1/commerce/topup/wallet", alice,
`{"rail":"base-usdc","txHash":"`+tTxHash+`","fromAddress":"`+tSender+`"}`)
if code != http.StatusOK {
t.Fatalf("usdc topup: want 200, got %d (%s)", code, body)
}
var r walletTopupResp
mustJSON(t, body, &r)
if r.CreditedCents != 500 {
t.Fatalf("6-decimal USDC must credit 500 cents, got %d", r.CreditedCents)
}
// The ledger row must describe the rail that was actually verified.
if com.payment["currency"] != "usdc" || com.payment["network"] != "Base" {
t.Fatalf("payment must record the real rail, got currency=%v network=%v",
com.payment["currency"], com.payment["network"])
}
if id, _ := com.payment["chainId"].(float64); id != 8453 {
t.Fatalf("chainId must be the rail's 8453, got %v", com.payment["chainId"])
}
}
// With several rails configured the client MUST name one: silently picking a default
// could verify a transfer against another chain's treasury.
func TestTopup_MultipleRails_RequiresNamingOne(t *testing.T) {
rpc := &fakeRPC{result: tokenReceipt("0x1", tUSDC, tSender, tTreasBase, 500, 6)}
com := &fakeCommerce{}
setTopupRails(t, com.server(t).URL,
usdcRail(rpc.server(t).URL),
rail{ID: "hanzo-husd", Chain: "Hanzo", ChainID: 36963, RPCURL: rpc.server(t).URL,
Token: tHusd, Symbol: "HUSD", Decimals: 18, Treasury: tTreasury},
)
app := mountBrand(t, "hanzo")
code, _ := callH(t, app, http.MethodPost, "/v1/commerce/topup/wallet", alice,
`{"txHash":"`+tTxHash+`"}`)
if code != http.StatusBadRequest {
t.Fatalf("omitting the rail with >1 configured: want 400, got %d", code)
}
code, _ = callH(t, app, http.MethodPost, "/v1/commerce/topup/wallet", alice,
`{"rail":"nope","txHash":"`+tTxHash+`"}`)
if code != http.StatusBadRequest {
t.Fatalf("unknown rail: want 400, got %d", code)
}
}
// A transfer on the right token but to ANOTHER rail's treasury must not credit.
func TestTopup_WrongRailTreasury_400(t *testing.T) {
rpc := &fakeRPC{result: tokenReceipt("0x1", tUSDC, tSender, tTreasury, 500, 6)} // hanzo treasury
com := &fakeCommerce{}
setTopupRails(t, com.server(t).URL, usdcRail(rpc.server(t).URL))
app := mountBrand(t, "hanzo")
code, _ := callH(t, app, http.MethodPost, "/v1/commerce/topup/wallet", alice,
`{"rail":"base-usdc","txHash":"`+tTxHash+`"}`)
if code != http.StatusBadRequest {
t.Fatalf("transfer to a different treasury: want 400, got %d", code)
}
}
// The public listing gives a browser what it needs to send funds — and must not leak
// the operational RPC endpoint, which lives on the same config struct.
func TestTopupRails_PublishesSendInfoWithoutRPC(t *testing.T) {
com := &fakeCommerce{}
setTopupRails(t, com.server(t).URL, usdcRail("http://secret-rpc.internal"))
app := mountBrand(t, "hanzo")
code, body := callH(t, app, http.MethodGet, "/v1/commerce/topup/rails", alice, "")
if code != http.StatusOK {
t.Fatalf("rails: want 200, got %d (%s)", code, body)
}
var got struct{ Rails []map[string]any }
mustJSON(t, body, &got)
if len(got.Rails) != 1 || got.Rails[0]["treasury"] != tTreasBase || got.Rails[0]["decimals"].(float64) != 6 {
t.Fatalf("rails listing wrong: %s", body)
}
if _, leaked := got.Rails[0]["rpcUrl"]; leaked {
t.Fatalf("rails listing must not publish the RPC endpoint: %s", body)
}
}
// No rail configured ⇒ the listing is an empty array, not null, so a client can read
// .length without a nil check — and the POST is an honest 501.
func TestTopupRails_EmptyWhenUnconfigured(t *testing.T) {
t.Setenv("TOPUP_RAILS", "")
app := mountBrand(t, "hanzo")
code, body := callH(t, app, http.MethodGet, "/v1/commerce/topup/rails", alice, "")
if code != http.StatusOK || !strings.Contains(string(body), `"rails":[]`) {
t.Fatalf("unconfigured rails: want 200 with [], got %d (%s)", code, body)
}
}
+91 -24
View File
@@ -27,6 +27,7 @@ import (
"os"
"sort"
"strings"
"sync"
"time"
"github.com/hanzoai/cloud"
@@ -38,11 +39,12 @@ import (
"github.com/hanzoai/cloud/clients/admin/finance"
"github.com/hanzoai/cloud/clients/admin/health"
"github.com/hanzoai/cloud/clients/admin/iam"
"github.com/hanzoai/cloud/clients/admin/infra"
"github.com/hanzoai/cloud/clients/admin/invoices"
"github.com/hanzoai/cloud/clients/admin/metrics"
"github.com/hanzoai/cloud/clients/admin/revenue"
"github.com/hanzoai/cloud/clients/admin/subscriptions"
"github.com/hanzoai/cloud/clients/commerceinproc"
"github.com/hanzoai/cloud/clients/commerce/transport"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
)
@@ -53,9 +55,9 @@ import (
// The state is built from Deps fields NOT on cloud.Base (deps.Audit, deps.IAMIssuer), so
// it constructs the cloud.Service value directly (cloud.NewBase + &cloud.Service[core.State]{…})
// rather than via cloud.Mount.
func Mount(app *zip.App, deps cloud.Deps) error {
func Mount(app cloud.Router, deps cloud.Deps) error {
if app == nil {
return fmt.Errorf("admin.Mount: nil zip.App")
return fmt.Errorf("admin.Mount: nil app")
}
if deps.Logger == nil {
return fmt.Errorf("admin.Mount: nil deps.Logger")
@@ -65,11 +67,12 @@ func Mount(app *zip.App, deps cloud.Deps) error {
Base: b,
State: core.State{
IAM: iam.New(iamBase(deps)),
Commerce: commerce.New(commerceinproc.BaseURL(os.Getenv("CLOUD_COMMERCE_HTTP_URL")), os.Getenv("COMMERCE_SERVICE_TOKEN")),
Commerce: commerce.New(transport.BaseURL(os.Getenv("CLOUD_COMMERCE_HTTP_URL")), os.Getenv("COMMERCE_SERVICE_TOKEN")),
Health: health.New(o11yHealthURL()),
DO: digitalocean.New(doTokenFromEnv()),
AdminOrg: adminOrgOf(deps),
AuditStore: deps.Audit,
WLTenants: wlTenantsFromEnv(),
},
}
@@ -89,7 +92,7 @@ func Mount(app *zip.App, deps cloud.Deps) error {
// through the two-tier gate: org-scoped panels behind core.GuardScoped, the platform
// control plane behind core.Guard. Each carved-out domain (audit/customer/revenue/finance)
// owns its own route registration.
func routes(app *zip.App, s *cloud.Service[core.State]) {
func routes(app cloud.Router, s *cloud.Service[core.State]) {
g := app.Group("/v1/admin")
// Org-scoped panels — GuardScoped. Cross-tenant reads are impossible for a non-super
// caller.
@@ -103,6 +106,7 @@ func routes(app *zip.App, s *cloud.Service[core.State]) {
g.Get("/applications", core.Guard(s, applications))
g.Get("/products", core.Guard(s, products))
g.Get("/compute", core.Guard(s, compute))
g.Get("/block-storage", core.Guard(s, blockStorage))
g.Get("/o11y", core.Guard(s, o11y))
g.Get("/aimetrics", core.Guard(s, aimetrics))
g.Post("/sync", core.Guard(s, syncNow))
@@ -138,6 +142,7 @@ func routes(app *zip.App, s *cloud.Service[core.State]) {
revenue.Routes(app, s)
finance.Routes(app, s)
metrics.Routes(app, s)
infra.Routes(app, s)
invoices.Routes(app, s)
subscriptions.Routes(app, s)
}
@@ -160,6 +165,11 @@ func me(s *cloud.Service[core.State], c *zip.Ctx) error {
Email: strings.TrimSpace(c.UserEmail()),
DisplayName: name,
IsSuperAdmin: sc.Super,
// The gate (GuardScoped) already proved this caller is either a SuperAdmin or an
// admin of an ENABLED WL tenant, so an admitted non-super IS the WL tier — no
// separate lookup needed. ScopeOrgs is the resolved subtree (empty ⇒ all, for super).
IsWhiteLabel: !sc.Super,
ScopeOrgs: sc.Orgs,
})
}
@@ -210,11 +220,20 @@ func users(s *cloud.Service[core.State], c *zip.Ctx) error {
} else if owner := strings.TrimSpace(c.Query("org")); owner != "" {
q.Set("owner", owner)
}
// Default pagination when the client omits it. IAM's user list returns ZERO
// rows AND total 0 when p/pageSize are unset — which surfaced as the admin
// directory showing "0 of 222". Default to the first page at the shared admin
// page size so the directory populates and the REAL total is reported; an
// explicit client p/pageSize still wins (the UI paginates from there).
if p := strings.TrimSpace(c.Query("p")); p != "" {
q.Set("p", p)
} else {
q.Set("p", "1")
}
if ps := strings.TrimSpace(c.Query("pageSize")); ps != "" {
q.Set("pageSize", ps)
} else {
q.Set("pageSize", "200")
}
if term := strings.TrimSpace(c.Query("q")); term != "" {
// IAM's list uses field/value contains-matching for the free-text filter.
@@ -332,13 +351,8 @@ func usage(s *cloud.Service[core.State], c *zip.Ctx) error {
}
// ── /v1/admin/products — workload registry (ProductRow[]) ────────────────────
// products is the workload/drift registry. That inventory is the platform.hanzo.ai apps
// table / operator reconcile state, NOT an in-binary source. admin exposes the gated
// endpoint and returns the real empty registry until that feed is wired.
func products(s *cloud.Service[core.State], c *zip.Ctx) error {
return core.OKList(c, []productRow{}, 0)
}
// The handler + the fleet projection live in products.go: it reads the operator App-CR +
// drift observation through the in-process paas.CurrentFleet seam (reuse, never fork).
// ── /v1/admin/overview — Platform Overview tiles (OverviewData) ───────────────
@@ -355,17 +369,40 @@ func overview(s *cloud.Service[core.State], c *zip.Ctx) error {
commercePartial := false
if orgErr == nil {
orgCount = len(orgs)
// FAN OUT. Each org costs two independent reads (users, money), so doing this
// serially made the dashboard's latency O(orgs): at 122 orgs that is ~244
// blocking round-trips before a single tile renders, and it grows every time a
// tenant signs up. The reads do not depend on each other, so they run
// concurrently under a fixed ceiling — bounded so a large fleet cannot stampede
// the finance ledger or the IAM store.
const maxParallelOrgReads = 12
var (
mu sync.Mutex
wg sync.WaitGroup
sem = make(chan struct{}, maxParallelOrgReads)
)
for _, o := range orgs {
userCount += orgUserCount(s, ctx, cr, o.Name)
sp, cr2, ok := core.OrgMoney(s, ctx, o.Name)
spend += sp
credits += cr2
if !ok {
// This org's money did not read — the fleet spend/credits totals are now
// an UNDERCOUNT, so the commerce source must report degraded, not healthy.
commercePartial = true
}
wg.Add(1)
go func(org string) {
defer wg.Done()
sem <- struct{}{}
defer func() { <-sem }()
uc := orgUserCount(s, ctx, cr, org)
sp, cr2, ok := core.OrgMoney(s, ctx, org)
mu.Lock()
defer mu.Unlock()
userCount += uc
spend += sp
credits += cr2
if !ok {
// This org's money did not read — the fleet spend/credits totals are
// now an UNDERCOUNT, so the commerce source must report degraded,
// not healthy.
commercePartial = true
}
}(o.Name)
}
wg.Wait()
}
// Commerce freshness derives from the SAME per-org reads the totals fold — NOT a
@@ -393,12 +430,18 @@ func overview(s *cloud.Service[core.State], c *zip.Ctx) error {
}
sources = append(sources, core.SrcOf("o11y", oErr, o11yRows, now))
// Fleet workload registry — the operator App-CR + drift observation via the paas seam
// (products.go). A nil/unready seam degrades to an honest-empty rollup (zeros, no error);
// a hard observation error marks the "fleet" source degraded without failing the overview.
fleetRows, fleetRoll, fleetErr := fleetProducts(ctx)
sources = append(sources, core.SrcOf("fleet", fleetErr, len(fleetRows), now))
return core.OK(c, overviewData{
Orgs: orgCount,
Users: userCount,
Products: 0, // workload registry feed pending (platform apps table)
ActiveProducts: 0,
Drift: 0,
Products: fleetRoll.Total,
ActiveProducts: fleetRoll.Active,
Drift: fleetRoll.Drift,
SpendCents30d: spend,
Tokens30d: 0, // fleet token counters pending (insights/datastore)
CreditsCents: credits,
@@ -461,6 +504,30 @@ func adminOrgOf(_ cloud.Deps) string {
return "admin"
}
// wlTenantsFromEnv resolves the enabled white-label tenant allowlist from
// ADMIN_WL_TENANT_ORGS (comma-separated org slugs). It is the ONE seed of
// State.WLTenants — the fail-closed second admission tier: EMPTY/unset ⇒ no customer
// org-admin is admitted (SuperAdmins only), so an absent/mis-set env fails CLOSED.
// Each entry is trimmed and matched verbatim against principal.Org (the validated
// owner), never folded; blank entries are dropped. Onboarding a reseller is a
// deliberate, KMS-/git-auditable edit to this env, not a runtime self-service flip.
func wlTenantsFromEnv() map[string]bool {
raw := strings.TrimSpace(os.Getenv("ADMIN_WL_TENANT_ORGS"))
if raw == "" {
return nil
}
set := map[string]bool{}
for _, part := range strings.Split(raw, ",") {
if org := strings.TrimSpace(part); org != "" {
set[org] = true
}
}
if len(set) == 0 {
return nil
}
return set
}
// doTokenFromEnv reads the DigitalOcean token from the environment. Sourced from a
// KMSSecret on the cloud deployment (DO_API_TOKEN) — never hard-coded.
func doTokenFromEnv() string {
+58
View File
@@ -42,6 +42,14 @@ func mountService(t *testing.T, iamURL, commerceURL, healthURL string) (func(met
Health: health.New(healthURL),
DO: digitalocean.New(""), // no token → honest not-configured unless a test overrides s.State.DO
AdminOrg: "admin",
// The harness enables ONE white-label tenant — "maxpower" (the org orgAdminHdr
// belongs to) — so the scoped-panel tests exercise the ADMITTED WL tier. The
// gate now requires WL enablement for any non-super caller, so the deny tests use
// a DIFFERENT org (not in this set) to prove a non-enabled org-admin is refused,
// and the fail-closed default (empty set ⇒ deny) is covered by a dedicated unit
// test on State.IsWhiteLabelTenant. A test that needs the fleet-only default
// clears s.State.WLTenants after mount.
WLTenants: map[string]bool{"maxpower": true},
}}
// Mirror the REAL Mount EXACTLY by registering the same routes() the subsystem uses
// (org-scoped panels behind GuardScoped, the platform control plane behind Guard,
@@ -99,6 +107,10 @@ var platformAdminRoutes = []adminRoute{
{"GET", "/v1/admin/flags"},
{"GET", "/v1/admin/waitlist"},
{"POST", "/v1/admin/waitlist/boost"},
{"GET", "/v1/admin/infra"},
{"POST", "/v1/admin/infra/volumes/v1/snapshot"},
{"DELETE", "/v1/admin/infra/volumes/v1"},
{"POST", "/v1/admin/infra/nodes/1/cordon"},
}
// adminRoutes is the full surface (both tiers) — the fail-closed gate test denies an
@@ -180,6 +192,52 @@ func TestGate_AllowsSuperAdmin(t *testing.T) {
}
}
// TestUsers_DefaultsPagination locks the "0 of 222" fix: IAM's user list returns
// ZERO rows AND total 0 when p/pageSize are unset, so the /v1/admin/users handler
// MUST supply a default first page + page size when the client (the operator
// directory) omits them. Proves the handler forwards p=1 & pageSize=200 and that
// the real total reaches the client.
func TestUsers_DefaultsPagination(t *testing.T) {
var gotP, gotPageSize string
iamSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if strings.HasSuffix(r.URL.Path, "/get-users") {
gotP = r.URL.Query().Get("p")
gotPageSize = r.URL.Query().Get("pageSize")
w.Header().Set("Content-Type", "application/json")
io.WriteString(w, `{"status":"ok","msg":"","data":[
{"owner":"hanzo","name":"alice","email":"alice@hanzo.ai","displayName":"Alice"}
],"data2":222}`)
return
}
w.WriteHeader(404)
io.WriteString(w, `{"status":"error","msg":"not found"}`)
}))
defer iamSrv.Close()
do := mount(t, iamSrv.URL, "http://127.0.0.1:0", "http://127.0.0.1:0")
admin := map[string]string{"X-User-IsAdmin": "true", "X-Org-Id": "admin", "X-User-Id": "admin/z", "X-User-Email": "z@hanzo.ai"}
resp, body := do("GET", "/v1/admin/users", admin) // NOTE: no ?pageSize — the bug path.
if resp.StatusCode != http.StatusOK {
t.Fatalf("GET /v1/admin/users: got %d (body=%s)", resp.StatusCode, body)
}
if gotPageSize != "200" {
t.Fatalf("users list must default pageSize=200 when the client omits it, got %q", gotPageSize)
}
if gotP != "1" {
t.Fatalf("users list must default p=1 when the client omits it, got %q", gotP)
}
var env struct {
Data []operatorUser `json:"data"`
Data2 int `json:"data2"`
}
if err := json.Unmarshal(body, &env); err != nil {
t.Fatalf("decode users envelope: %v (body=%s)", err, body)
}
if env.Data2 != 222 || len(env.Data) == 0 {
t.Fatalf("users must surface the REAL directory (got %d rows, total %d), not 0-of-222", len(env.Data), env.Data2)
}
}
// fakeIAM stands in for the IAM management surface. It records whether the
// caller's credential was replayed and returns /v1 envelopes.
type fakeIAM struct {
+40 -40
View File
@@ -20,12 +20,12 @@ package admin
// FLEET behaving" (RED metrics, logs, usage), this answers "how are the MODELS and
// EVALS doing" — LLM generations, per-model spend, and eval-run quality/progress —
// over the SAME ONE datastore (Datastore), the SAME shared client
// (aiobject.DatastoreQuery), no second connection.
// (datastore.Query), no second connection.
//
// Signals, each from its canonical table in the one datastore:
// - LLM generations → langfuse.observations : generations, cost (USD), latency
// - LLM generations → o11y_ai.observations : generations, cost (USD), latency
// (fleet-wide; honest-empty until the
// Langfuse ingest lands rows)
// O11yAI ingest lands rows)
// - Per-model usage → hanzo.cloud_usage : requests, tokens, cost per model
// (the live usage ledger the ai gateway
// writes — populated today)
@@ -52,7 +52,7 @@ package admin
// INDEPENDENTLY — a table that is absent or a column that differs contributes its
// zero-value (the enclosing `if err == nil`), never a failure, so the board always
// renders what the datastore actually holds. admin READS only; it owns and creates
// NO table. Money from cloud_usage is USD cents, from langfuse is USD; latency is
// NO table. Money from cloud_usage is USD cents, from o11y_ai is USD; latency is
// milliseconds; time bounds are POSITIONAL parameters (never interpolated), and the
// bucket interval is a server-side constant — injection-safe.
@@ -60,18 +60,18 @@ import (
"strconv"
"time"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/datastore"
"github.com/zap-proto/zip"
)
// Fully-qualified datastore tables. admin only READS these — the ai gateway owns
// hanzo.cloud_usage, Langfuse owns langfuse.observations, and the eval telemetry
// hanzo.cloud_usage, O11yAI owns o11y_ai.observations, and the eval telemetry
// store (clients/eval) owns hanzo.eval_traces / hanzo.eval_scores.
const (
aimUsageTable = "hanzo.cloud_usage"
aimLangfuseObs = "langfuse.observations"
aimO11yAIObs = "o11y_ai.observations"
aimEvalTraces = "hanzo.eval_traces"
aimEvalScores = "hanzo.eval_scores"
aimTopN = 12
@@ -82,19 +82,19 @@ type aiMetrics struct {
Range string `json:"range"`
Start string `json:"start"`
End string `json:"end"`
Langfuse aimLangfuse `json:"langfuse"`
O11yAI aimO11yAI `json:"o11yAi"`
Usage aimUsage `json:"usage"`
Evals aimEvals `json:"evals"`
TopModels []aimModelStat `json:"topModels"` // cloud_usage per-model (populated today)
LangfuseModels []aimLfModelStat `json:"langfuseModels"` // langfuse per-model (honest-empty today)
O11yAIModels []aimLfModelStat `json:"o11yAiModels"` // o11y_ai per-model (honest-empty today)
ScoreNames []aimScoreStat `json:"scoreNames"` // eval_scores per score-name
EvalRuns []aimRunStat `json:"evalRuns"` // recent eval runs (progress)
ScoreSeries []aimScorePoint `json:"scoreSeries"` // avg eval score over time (progress trend)
}
// aimLangfuse is the fleet-wide Langfuse generation rollup (honest-empty today).
// Cost is USD (Langfuse's native unit); latency is milliseconds (end_time-start_time).
type aimLangfuse struct {
// aimO11yAI is the fleet-wide O11yAI generation rollup (honest-empty today).
// Cost is USD (O11yAI's native unit); latency is milliseconds (end_time-start_time).
type aimO11yAI struct {
Generations int64 `json:"generations"`
CostUsd float64 `json:"costUsd"`
LatencyMsAvg float64 `json:"latencyMsAvg"`
@@ -132,7 +132,7 @@ type aimModelStat struct {
CostCents int64 `json:"costCents"`
}
// aimLfModelStat is one row of the per-model Langfuse leaderboard (honest-empty today).
// aimLfModelStat is one row of the per-model O11yAI leaderboard (honest-empty today).
type aimLfModelStat struct {
Model string `json:"model"`
Generations int64 `json:"generations"`
@@ -178,7 +178,7 @@ func aimetrics(s *cloud.Service[core.State], c *zip.Ctx) error {
Start: since.Format(time.RFC3339),
End: time.Now().UTC().Format(time.RFC3339),
TopModels: []aimModelStat{},
LangfuseModels: []aimLfModelStat{},
O11yAIModels: []aimLfModelStat{},
ScoreNames: []aimScoreStat{},
EvalRuns: []aimRunStat{},
ScoreSeries: []aimScorePoint{},
@@ -186,53 +186,53 @@ func aimetrics(s *cloud.Service[core.State], c *zip.Ctx) error {
// Honest-empty when the warehouse is not connected: the board renders its zero
// state, never a fabricated fleet.
if !aiobject.DatastoreEnabled() {
if !datastore.Ready() {
return core.OK(c, payload)
}
sinceTS := chTS(since) // DateTime literal — cloud_usage.timestamp, langfuse.start_time, eval_*.ts
sinceTS := chTS(since) // DateTime literal — cloud_usage.timestamp, o11y_ai.start_time, eval_*.ts
interval := o11yBucket(rangeLabel)
// ── Langfuse generations (fleet) — honest-empty until ingest lands rows ──
if rows, err := aiobject.DatastoreQuery(ctx, aimLangfuseTotalsSQL(), sinceTS); err == nil {
// ── O11yAI generations (fleet) — honest-empty until ingest lands rows ──
if rows, err := datastore.Query(ctx, aimO11yAITotalsSQL(), sinceTS); err == nil {
r := firstRowOr(rows)
payload.Langfuse.Generations = chInt64(r["gens"])
payload.Langfuse.CostUsd = chFloat64(r["cost"])
payload.O11yAI.Generations = chInt64(r["gens"])
payload.O11yAI.CostUsd = chFloat64(r["cost"])
}
// Langfuse latency (separate query so a Nullable end_time / column mismatch never
// O11yAI latency (separate query so a Nullable end_time / column mismatch never
// zeroes the proven generations+cost number above).
if rows, err := aiobject.DatastoreQuery(ctx, aimLangfuseLatencySQL(), sinceTS); err == nil {
if rows, err := datastore.Query(ctx, aimO11yAILatencySQL(), sinceTS); err == nil {
r := firstRowOr(rows)
payload.Langfuse.LatencyMsAvg = chFloat64(r["lat_avg"])
payload.Langfuse.LatencyMsP95 = chFloat64(r["lat_p95"])
payload.O11yAI.LatencyMsAvg = chFloat64(r["lat_avg"])
payload.O11yAI.LatencyMsP95 = chFloat64(r["lat_p95"])
}
// Langfuse per-model.
if rows, err := aiobject.DatastoreQuery(ctx, aimLangfuseModelsSQL(), sinceTS); err == nil {
payload.LangfuseModels = lfModelsFromRows(rows)
// O11yAI per-model.
if rows, err := datastore.Query(ctx, aimO11yAIModelsSQL(), sinceTS); err == nil {
payload.O11yAIModels = lfModelsFromRows(rows)
}
// ── Per-model usage (fleet) from the live cloud_usage ledger ──
if rows, err := aiobject.DatastoreQuery(ctx, aimUsageTotalsSQL(), sinceTS); err == nil {
if rows, err := datastore.Query(ctx, aimUsageTotalsSQL(), sinceTS); err == nil {
fillAimUsage(&payload.Usage, firstRowOr(rows))
}
if rows, err := aiobject.DatastoreQuery(ctx, aimTopModelsSQL(), sinceTS); err == nil {
if rows, err := datastore.Query(ctx, aimTopModelsSQL(), sinceTS); err == nil {
payload.TopModels = aimModelsFromRows(rows)
}
// ── Evals (fleet): traces + scores + progress ──
if rows, err := aiobject.DatastoreQuery(ctx, aimEvalTracesSQL(), sinceTS); err == nil {
if rows, err := datastore.Query(ctx, aimEvalTracesSQL(), sinceTS); err == nil {
fillAimEvalTraces(&payload.Evals, firstRowOr(rows))
}
if rows, err := aiobject.DatastoreQuery(ctx, aimEvalScoresSQL(), sinceTS); err == nil {
if rows, err := datastore.Query(ctx, aimEvalScoresSQL(), sinceTS); err == nil {
fillAimEvalScores(&payload.Evals, firstRowOr(rows))
}
if rows, err := aiobject.DatastoreQuery(ctx, aimScoreNamesSQL(), sinceTS); err == nil {
if rows, err := datastore.Query(ctx, aimScoreNamesSQL(), sinceTS); err == nil {
payload.ScoreNames = scoreNamesFromRows(rows)
}
if rows, err := aiobject.DatastoreQuery(ctx, aimEvalRunsSQL(), sinceTS); err == nil {
if rows, err := datastore.Query(ctx, aimEvalRunsSQL(), sinceTS); err == nil {
payload.EvalRuns = evalRunsFromRows(rows)
}
if rows, err := aiobject.DatastoreQuery(ctx, aimScoreSeriesSQL(interval), sinceTS); err == nil {
if rows, err := datastore.Query(ctx, aimScoreSeriesSQL(interval), sinceTS); err == nil {
payload.ScoreSeries = scoreSeriesFromRows(rows)
}
@@ -241,20 +241,20 @@ func aimetrics(s *cloud.Service[core.State], c *zip.Ctx) error {
// ── pure SQL builders (static SQL + one positional time bound; unit-tested) ──
func aimLangfuseTotalsSQL() string {
return "SELECT count() AS gens, toFloat64(sum(total_cost)) AS cost FROM " + aimLangfuseObs +
func aimO11yAITotalsSQL() string {
return "SELECT count() AS gens, toFloat64(sum(total_cost)) AS cost FROM " + aimO11yAIObs +
" WHERE type = 'GENERATION' AND start_time >= ?"
}
func aimLangfuseLatencySQL() string {
func aimO11yAILatencySQL() string {
lat := "(toUnixTimestamp64Milli(end_time) - toUnixTimestamp64Milli(start_time))"
return "SELECT round(avg(" + lat + "), 2) AS lat_avg, round(quantile(0.95)(" + lat + "), 2) AS lat_p95 " +
"FROM " + aimLangfuseObs + " WHERE type = 'GENERATION' AND start_time >= ? AND end_time > start_time"
"FROM " + aimO11yAIObs + " WHERE type = 'GENERATION' AND start_time >= ? AND end_time > start_time"
}
func aimLangfuseModelsSQL() string {
func aimO11yAIModelsSQL() string {
return "SELECT provided_model_name AS model, count() AS gens, toFloat64(sum(total_cost)) AS cost " +
"FROM " + aimLangfuseObs + " WHERE type = 'GENERATION' AND start_time >= ? AND provided_model_name != '' " +
"FROM " + aimO11yAIObs + " WHERE type = 'GENERATION' AND start_time >= ? AND provided_model_name != '' " +
"GROUP BY model ORDER BY gens DESC LIMIT " + strconv.Itoa(aimTopN)
}
+10 -10
View File
@@ -28,9 +28,9 @@ func TestAimSQL_ReadsCanonicalTables(t *testing.T) {
name, sql, table string
wantQMarks int
}{
{"langfuseTotals", aimLangfuseTotalsSQL(), "langfuse.observations", 1},
{"langfuseLatency", aimLangfuseLatencySQL(), "langfuse.observations", 1},
{"langfuseModels", aimLangfuseModelsSQL(), "langfuse.observations", 1},
{"o11yAiTotals", aimO11yAITotalsSQL(), "o11y_ai.observations", 1},
{"o11yAiLatency", aimO11yAILatencySQL(), "o11y_ai.observations", 1},
{"o11yAiModels", aimO11yAIModelsSQL(), "o11y_ai.observations", 1},
{"usageTotals", aimUsageTotalsSQL(), "hanzo.cloud_usage", 1},
{"topModels", aimTopModelsSQL(), "hanzo.cloud_usage", 1},
{"evalTraces", aimEvalTracesSQL(), "hanzo.eval_traces", 1},
@@ -49,12 +49,12 @@ func TestAimSQL_ReadsCanonicalTables(t *testing.T) {
}
}
// TestAimLangfuseScopedToGeneration proves the Langfuse lens is scoped to
// TestAimO11yAIScopedToGeneration proves the O11yAI lens is scoped to
// generations only (not spans/events), matching the o11y LLM lens.
func TestAimLangfuseScopedToGeneration(t *testing.T) {
for _, sql := range []string{aimLangfuseTotalsSQL(), aimLangfuseLatencySQL(), aimLangfuseModelsSQL()} {
func TestAimO11yAIScopedToGeneration(t *testing.T) {
for _, sql := range []string{aimO11yAITotalsSQL(), aimO11yAILatencySQL(), aimO11yAIModelsSQL()} {
if !strings.Contains(sql, "type = 'GENERATION'") {
t.Errorf("langfuse lens must scope to GENERATION observations; got %q", sql)
t.Errorf("o11y_ai lens must scope to GENERATION observations; got %q", sql)
}
}
}
@@ -89,8 +89,8 @@ func TestAimEvalLatencyGuarded(t *testing.T) {
if !strings.Contains(aimEvalTracesSQL(), "end_time > start_time") {
t.Errorf("eval traces latency must guard end_time>start_time; got %q", aimEvalTracesSQL())
}
if !strings.Contains(aimLangfuseLatencySQL(), "end_time > start_time") {
t.Errorf("langfuse latency must guard end_time>start_time; got %q", aimLangfuseLatencySQL())
if !strings.Contains(aimO11yAILatencySQL(), "end_time > start_time") {
t.Errorf("o11y_ai latency must guard end_time>start_time; got %q", aimO11yAILatencySQL())
}
}
@@ -147,7 +147,7 @@ func TestAimParsers(t *testing.T) {
{"model": "gpt-4o", "gens": uint64(42), "cost": float64(1.25)},
})
if len(lf) != 1 || lf[0].Model != "gpt-4o" || lf[0].Generations != 42 || lf[0].CostUsd != 1.25 {
t.Fatalf("langfuse models mis-parsed: %+v", lf)
t.Fatalf("o11y_ai models mis-parsed: %+v", lf)
}
names := scoreNamesFromRows([]map[string]any{
{"name": "accuracy", "n": uint64(320), "avg_value": float64(0.82), "min_value": float64(0), "max_value": float64(1)},
+1 -1
View File
@@ -25,7 +25,7 @@ import (
)
// Routes registers the /v1/admin/audit* surface (SuperAdmin only).
func Routes(app *zip.App, s *cloud.Service[core.State]) {
func Routes(app cloud.Router, s *cloud.Service[core.State]) {
g := app.Group("/v1/admin")
g.Get("/audit", core.Guard(s, Records))
g.Get("/audit/verify", core.Guard(s, Verify))
+197
View File
@@ -0,0 +1,197 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package admin
// blockStorage — GET /v1/admin/block-storage, the realtime DO block-storage fleet the
// operator's Block Storage board (admin.hanzo.ai) watches to scale DO before it runs out.
// (Named `block-storage`, not `storage`, so the operator's separate S3 object-buckets
// view keeps /v1/admin/storage — two distinct storage concerns, two endpoints.)
// Two REAL sources, honest by construction:
// - The FLEET inventory (count · total capacity · monthly cost · per-volume region +
// attachment) from the DigitalOcean API (the same DO_API_TOKEN client the finance
// dashboard already uses). DO exposes capacity + attachment but NOT fill %, so a
// volume's used/pct stay ABSENT (the console renders an honest "—", never a
// fabricated number).
// - The analytics DATASTORE's own fill from Datastore `system.disks` (the 200Gi PVC
// the datastore fork mounts) — total/free space over the SAME shared client
// (datastore.Query) the analytics + compute lenses read, no second
// connection. This is THE number the operator scales on.
//
// SUPERADMIN ONLY (the s.guard wrap in admin.go): a cross-tenant infra read, all-orgs.
// admin holds NO storage state — it only reads DO + the datastore. DO unconfigured →
// empty fleet; datastore not connected → no datastore card. Never a fabricated fleet.
import (
"context"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/admin/digitalocean"
"github.com/hanzoai/cloud/clients/datastore"
"github.com/zap-proto/zip"
)
// doBlockUsdPerGiB is DO block storage's list price ($0.10/GiB/mo) — the fleet cost
// line for the ops budget when DO doesn't itemize per-volume spend.
const doBlockUsdPerGiB = 0.10
// bytesPerGiB converts system.disks bytes (UInt64) → GiB.
const bytesPerGiB = 1024 * 1024 * 1024
// storageVolume mirrors the console StorageVolume. UsedGiB/Pct are POINTERS so an
// absent fill serializes as JSON null (→ the console's honest "—"), distinct from a
// real 0%.
type storageVolume struct {
ID string `json:"id"`
Name string `json:"name"`
Region string `json:"region"`
SizeGiB int `json:"sizeGiB"`
UsedGiB *float64 `json:"usedGiB"`
Pct *float64 `json:"pct"`
Attached bool `json:"attached"`
Service string `json:"service"`
}
// storageFleet is the roll-up: real count/capacity/cost; fleet fill absent (DO gives
// no per-volume fill, so there is no honest fleet-wide used total to report).
type storageFleet struct {
Count int `json:"count"`
TotalGiB int `json:"totalGiB"`
UsedGiB *float64 `json:"usedGiB"`
Pct *float64 `json:"pct"`
MonthlyUsd int `json:"monthlyUsd"`
}
// datastoreVolume is the analytics backend's own volume, fill REAL from system.disks.
type datastoreVolume struct {
Name string `json:"name"`
Mount string `json:"mount"`
SizeGiB int `json:"sizeGiB"`
UsedGiB float64 `json:"usedGiB"`
Pct float64 `json:"pct"`
}
// storageAlert flags a near-full volume (only the datastore carries a real fill today,
// so alerts are datastore-derived until a per-volume filesystem source is wired).
type storageAlert struct {
Volume string `json:"volume"`
Pct float64 `json:"pct"`
Level string `json:"level"`
}
// storageSnapshot is the whole board payload the console normalizes.
type storageSnapshot struct {
Fleet storageFleet `json:"fleet"`
Datastore *datastoreVolume `json:"datastore"`
Volumes []storageVolume `json:"volumes"`
Alerts []storageAlert `json:"alerts"`
}
// blockStorage answers GET /v1/admin/block-storage. SuperAdmin only. Each source
// degrades independently — a DO outage still returns the real datastore fill, and v.v.
func blockStorage(s *cloud.Service[core.State], c *zip.Ctx) error {
ctx := c.Context()
vols, _ := s.State.DO.Volumes(ctx) // honest empty on not-configured / unreachable
fill := datastoreFill(ctx) // nil unless system.disks answered
return core.OK(c, buildStorageSnapshot(vols, fill))
}
// buildStorageSnapshot assembles the board payload (PURE — unit-tested). It folds the
// DO inventory into the fleet totals + per-volume rows (fill absent), attaches the real
// datastore card, and derives a near-full alert from the datastore fill. No fabrication:
// a volume's fill is left nil (DO gives none), and the datastore card is present only
// when system.disks actually answered.
func buildStorageSnapshot(vols []digitalocean.Volume, fill *datastoreVolume) storageSnapshot {
out := make([]storageVolume, 0, len(vols))
totalGiB := 0
for _, v := range vols {
out = append(out, storageVolume{
ID: v.ID,
Name: v.Name,
Region: v.Region,
SizeGiB: v.SizeGiB,
Attached: len(v.DropletIDs) > 0,
})
totalGiB += v.SizeGiB
}
alerts := make([]storageAlert, 0, 1)
if fill != nil {
if lvl := alertLevel(fill.Pct); lvl != "" {
alerts = append(alerts, storageAlert{Volume: fill.Name, Pct: fill.Pct, Level: lvl})
}
}
return storageSnapshot{
Fleet: storageFleet{
Count: len(vols),
TotalGiB: totalGiB,
MonthlyUsd: int(float64(totalGiB)*doBlockUsdPerGiB + 0.5),
},
Datastore: fill,
Volumes: out,
Alerts: alerts,
}
}
// alertLevel thresholds a fill %: critical ≥ 90, warn ≥ 80, else none (PURE, tested).
func alertLevel(pct float64) string {
switch {
case pct >= 90:
return "critical"
case pct >= 80:
return "warn"
default:
return ""
}
}
// datastoreFill reads the analytics datastore's own volume usage from Datastore
// `system.disks` (the largest disk by capacity is the data volume — the 200Gi PVC).
// Returns nil when the datastore isn't connected or the query fails (honest — the
// console shows no datastore card, never a fabricated fill).
func datastoreFill(ctx context.Context) *datastoreVolume {
if !datastore.Ready() {
return nil
}
rows, err := datastore.Query(ctx,
"SELECT name, path, total_space, free_space FROM system.disks ORDER BY total_space DESC LIMIT 1")
if err != nil || len(rows) == 0 {
return nil
}
return datastoreFillFromRow(rows[0])
}
// datastoreFillFromRow maps a system.disks row → the datastore card (PURE, tested).
// nil when the disk reports no capacity (an unusable read, not a fabricated 0%).
func datastoreFillFromRow(r map[string]any) *datastoreVolume {
total := float64(chInt64(r["total_space"]))
if total <= 0 {
return nil
}
free := float64(chInt64(r["free_space"]))
used := total - free
if used < 0 {
used = 0
}
return &datastoreVolume{
Name: chStr(r["name"]),
Mount: chStr(r["path"]),
SizeGiB: int(total / bytesPerGiB),
UsedGiB: round1(used / bytesPerGiB),
Pct: round1(used / total * 100),
}
}
// round1 rounds to one decimal (used/pct read cleanly; not a scientific quantity).
func round1(x float64) float64 { return float64(int(x*10+0.5)) / 10 }
+130
View File
@@ -0,0 +1,130 @@
// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package admin
import (
"testing"
"github.com/hanzoai/cloud/clients/admin/digitalocean"
)
// The DO inventory folds into fleet totals + rows, and per-volume fill stays ABSENT
// (DO exposes no fill) so the console renders an honest "—", never a fabricated 0%.
func TestBuildStorageSnapshotFleet(t *testing.T) {
vols := []digitalocean.Volume{
{ID: "a", Name: "pvc-a", Region: "sfo3", SizeGiB: 200, DropletIDs: []int{1}},
{ID: "b", Name: "pvc-b", Region: "nyc1", SizeGiB: 100, DropletIDs: nil},
{ID: "c", Name: "pvc-c", Region: "sfo3", SizeGiB: 50, DropletIDs: []int{2, 3}},
}
snap := buildStorageSnapshot(vols, nil)
if snap.Fleet.Count != 3 {
t.Fatalf("count = %d, want 3", snap.Fleet.Count)
}
if snap.Fleet.TotalGiB != 350 {
t.Fatalf("totalGiB = %d, want 350", snap.Fleet.TotalGiB)
}
// 350 GiB * $0.10 = $35.
if snap.Fleet.MonthlyUsd != 35 {
t.Fatalf("monthlyUsd = %d, want 35", snap.Fleet.MonthlyUsd)
}
if snap.Fleet.UsedGiB != nil || snap.Fleet.Pct != nil {
t.Fatalf("fleet fill must be absent (DO gives none); got used=%v pct=%v", snap.Fleet.UsedGiB, snap.Fleet.Pct)
}
if len(snap.Volumes) != 3 {
t.Fatalf("volumes = %d, want 3", len(snap.Volumes))
}
// Attachment derives from droplet_ids; fill is absent per row.
if !snap.Volumes[0].Attached || snap.Volumes[1].Attached || !snap.Volumes[2].Attached {
t.Fatalf("attachment wrong: %+v", snap.Volumes)
}
for _, v := range snap.Volumes {
if v.UsedGiB != nil || v.Pct != nil {
t.Fatalf("volume %s fill must be absent, got used=%v pct=%v", v.ID, v.UsedGiB, v.Pct)
}
}
if snap.Datastore != nil {
t.Fatalf("datastore must be nil when no fill was read; got %+v", snap.Datastore)
}
if len(snap.Alerts) != 0 {
t.Fatalf("no alerts without a datastore fill; got %v", snap.Alerts)
}
}
// A near-full datastore raises exactly one alert; a healthy one raises none.
func TestBuildStorageSnapshotDatastoreAlert(t *testing.T) {
full := &datastoreVolume{Name: "default", Mount: "/var/lib/hanzo-datastore", SizeGiB: 196, UsedGiB: 178, Pct: 91}
snap := buildStorageSnapshot(nil, full)
if snap.Datastore == nil || snap.Datastore.Pct != 91 {
t.Fatalf("datastore card missing/wrong: %+v", snap.Datastore)
}
if len(snap.Alerts) != 1 || snap.Alerts[0].Level != "critical" || snap.Alerts[0].Volume != "default" {
t.Fatalf("expected one critical alert, got %+v", snap.Alerts)
}
healthy := &datastoreVolume{Name: "default", SizeGiB: 196, UsedGiB: 13, Pct: 7}
if a := buildStorageSnapshot(nil, healthy).Alerts; len(a) != 0 {
t.Fatalf("a 7%%-full datastore must raise no alert, got %+v", a)
}
}
func TestAlertLevel(t *testing.T) {
cases := []struct {
pct float64
want string
}{
{0, ""}, {7, ""}, {79.9, ""}, {80, "warn"}, {89.9, "warn"}, {90, "critical"}, {99, "critical"},
}
for _, c := range cases {
if got := alertLevel(c.pct); got != c.want {
t.Fatalf("alertLevel(%v) = %q, want %q", c.pct, got, c.want)
}
}
}
// system.disks bytes → GiB + pct, matching the live datastore (13.5G used of ~196G ≈ 7%).
func TestDatastoreFillFromRow(t *testing.T) {
// total ≈ 196.6 GiB, free ≈ 183.1 GiB → used ≈ 13.5 GiB → ~6.9%.
gib := float64(bytesPerGiB) // a variable → runtime math (a float→int const conversion won't compile)
total := int64(196.6 * gib)
used := int64(13.5 * gib)
row := map[string]any{
"name": "default",
"path": "/var/lib/hanzo-datastore",
"total_space": uint64(total),
"free_space": uint64(total - used),
}
d := datastoreFillFromRow(row)
if d == nil {
t.Fatal("expected a datastore fill, got nil")
}
if d.Name != "default" || d.Mount != "/var/lib/hanzo-datastore" {
t.Fatalf("name/mount wrong: %+v", d)
}
if d.SizeGiB != 196 {
t.Fatalf("sizeGiB = %d, want 196", d.SizeGiB)
}
if d.UsedGiB < 13.4 || d.UsedGiB > 13.6 {
t.Fatalf("usedGiB = %v, want ~13.5", d.UsedGiB)
}
if d.Pct < 6.5 || d.Pct > 7.5 {
t.Fatalf("pct = %v, want ~7", d.Pct)
}
// A disk that reports no capacity is an unusable read → nil (never a fake 0%).
if datastoreFillFromRow(map[string]any{"total_space": uint64(0)}) != nil {
t.Fatal("zero-capacity disk must yield nil, not a fabricated 0%")
}
}
+3 -3
View File
@@ -33,7 +33,7 @@ import (
"time"
"github.com/hanzoai/cloud/clients/admin/money"
"github.com/hanzoai/cloud/clients/commerceinproc"
"github.com/hanzoai/cloud/clients/commerce/transport"
)
// errUnconfigured marks a write (Deposit) attempted against an unwired commerce.
@@ -47,7 +47,7 @@ type Client struct {
}
// New builds a commerce client for base + admin S2S token. The HTTP client uses the
// commerceinproc self-routing transport: when commerce is CO-RESIDENT (base is the
// commerce transport self-routing dispatch: when commerce is CO-RESIDENT (base is the
// commerce.inproc placeholder) it dispatches in-process — a plain http.Client would
// instead DNS-resolve "commerce.inproc" and fail "no such host", silently breaking the
// admin cost/finance god-view. For a split-deploy (a real commerce URL) it falls
@@ -56,7 +56,7 @@ func New(base, token string) *Client {
return &Client{
base: strings.TrimRight(strings.TrimSpace(base), "/"),
token: strings.TrimSpace(token),
http: commerceinproc.Client(15 * time.Second),
http: transport.Client(15 * time.Second),
}
}
+5 -5
View File
@@ -19,7 +19,7 @@ package admin
// org → app → project tree. It aggregates the operator-owned usage table
// hanzo.compute_usage(org, app, project, kind, event, machine_id, size,
// price_cents, ts) — the same warehouse (`datastore`, datastore) the analytics
// subsystem reads, over the SAME shared client (aiobject.DatastoreQuery), no
// subsystem reads, over the SAME shared client (datastore.Query), no
// second connection. `kind` is an OPEN LowCardinality spectrum (bot | machine |
// cluster | nodepool | container | function | …) — a bot is a machine running the
// @hanzo/bot agent, a machine is raw compute visor opens — and each console lens
@@ -37,9 +37,9 @@ import (
"strings"
"time"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/datastore"
"github.com/zap-proto/zip"
)
@@ -78,7 +78,7 @@ func compute(s *cloud.Service[core.State], c *zip.Ctx) error {
ctx := c.Context()
// Honest-empty when the warehouse is not connected or the usage table is not
// provisioned yet (the visor/commerce emitter is still being wired).
if !aiobject.DatastoreEnabled() || !computeTableExists(ctx) {
if !datastore.Ready() || !computeTableExists(ctx) {
return core.OKList(c, []computeLeaf{}, 0)
}
@@ -88,7 +88,7 @@ func compute(s *cloud.Service[core.State], c *zip.Ctx) error {
// warehouse's lower-case convention.
kind := strings.ToLower(strings.TrimSpace(c.Query("kind")))
sql, args := buildComputeQuery(c.Query("range"), kind, strings.TrimSpace(c.Query("org")))
rows, err := aiobject.DatastoreQuery(ctx, sql, args...)
rows, err := datastore.Query(ctx, sql, args...)
if err != nil {
return core.Fail(c, "compute query: "+err.Error())
}
@@ -146,7 +146,7 @@ func computeLeavesFromRows(rows []map[string]any) []computeLeaf {
// computeTableExists probes for the operator-owned events table. Any error → false
// (honest "not available yet"), mirroring analytics.tableExists.
func computeTableExists(ctx context.Context) bool {
rows, err := aiobject.DatastoreQuery(ctx, "EXISTS TABLE "+computeTable)
rows, err := datastore.Query(ctx, "EXISTS TABLE "+computeTable)
if err != nil || len(rows) == 0 {
return false
}
+11 -15
View File
@@ -3,34 +3,30 @@ package core
import (
"encoding/json"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
)
// The /v1 envelope writers { status, msg, data, data2 } the operator's transport
// decodes (get<T> reads data; getList<T> reads data + data2 total).
//
// These are thin call-throughs to the ONE implementation in package cloud
// (cloud.OK/OKList/OKRaw/Fail). admin keeps its `core.OK` spelling — the admin
// handlers that call these are unchanged — but there is a single envelope writer
// for the whole binary, so a subsystem outside admin uses cloud.OK directly
// without importing this admin-scoped package (and its iam fan-in).
// OK writes a { status:"ok", data } envelope (the get<T> shape).
func OK(c *zip.Ctx, data any) error {
return c.JSON(200, map[string]any{"status": "ok", "msg": "", "data": data})
}
func OK(c *zip.Ctx, data any) error { return cloud.OK(c, data) }
// OKList writes a { status:"ok", data:[...], data2:total } envelope (getList<T>).
func OKList(c *zip.Ctx, rows any, total int) error {
return c.JSON(200, map[string]any{"status": "ok", "msg": "", "data": rows, "data2": total})
}
func OKList(c *zip.Ctx, rows any, total int) error { return cloud.OKList(c, rows, total) }
// OKRaw writes a { status:"ok", data:<raw>, data2:total } envelope, forwarding an
// IAM payload verbatim so its exact wire shape (Role, Application, Record, User)
// reaches the operator field-for-field.
func OKRaw(c *zip.Ctx, rows json.RawMessage, total int) error {
if len(rows) == 0 {
rows = json.RawMessage("[]")
}
return c.JSON(200, map[string]any{"status": "ok", "msg": "", "data": rows, "data2": total})
}
func OKRaw(c *zip.Ctx, rows json.RawMessage, total int) error { return cloud.OKRaw(c, rows, total) }
// Fail writes a { status:"error", msg } envelope. The operator's transport maps a
// non-ok envelope to a surfaced error (never a fabricated value).
func Fail(c *zip.Ctx, msg string) error {
return c.JSON(200, map[string]any{"status": "error", "msg": msg, "data": nil})
}
func Fail(c *zip.Ctx, msg string) error { return cloud.Fail(c, msg) }
+42 -6
View File
@@ -6,9 +6,11 @@ import (
"fmt"
"net/url"
"strings"
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/iam"
"github.com/hanzoai/cloud/clients/finance"
)
// MaxCustomerConcurrency bounds the per-org enrichment fan-out so a large fleet does
@@ -34,13 +36,25 @@ func ListOrgs(s *cloud.Service[State], ctx context.Context, cr iam.Creds) ([]iam
return orgs, nil
}
// OrgMoney returns (spendCents, creditsCents, ok) for one org from commerce. ok is
// false when the spend OR credits read FAILED — so a fleet aggregator can fold the
// per-org failure into a PARTIAL/degraded source rather than presenting the resulting
// undercount as authoritative (the SAME (row, ok) contract revenue.revenueOf uses). An
// unwired commerce is NOT a failure: Spend/Credits return (0, nil) when unconfigured, so
// ok stays true and the caller distinguishes "not configured" via Commerce.Ready().
// OrgMoney returns (spendCents, creditsCents, ok) for one org — the ONE per-org money read
// every fleet aggregator (overview, orgs, revenue) folds over. ok is false ONLY when a read
// FAILED, so a caller folds the per-org failure into a PARTIAL/degraded source rather than
// presenting the resulting undercount as authoritative. An org that simply has no money yet
// reads a clean (0, 0, true), never a failure.
//
// CO-RESIDENCE (the money-plane trap). Commerce's own /v1/billing/* routes are behind
// `//go:build cloud` and are NOT compiled into this binary, so the admin commerce client's
// S2S reads self-dispatch by PATH into cloud's OWN handlers: GET /v1/billing/balance
// re-enters the customer balance handler with no principal (401) and GET
// /v1/billing/usage-rollup is unrouted (404). Reading commerce over HTTP would therefore
// fail for EVERY org and falsely mark the money source DOWN while real money sits in the
// co-resident ledger. So — exactly as clients/billing.balance()/usage() and
// core.grantDeposit already resolve it — prefer the co-resident finance ledger
// (finance.Current()); the commerce S2S read stays only as the split-deploy fallback.
func OrgMoney(s *cloud.Service[State], ctx context.Context, org string) (spend, credits int64, ok bool) {
if fin := finance.Current(); fin != nil {
return orgMoneyFromFinance(ctx, fin, org)
}
ok = true
if sp, err := s.State.Commerce.Spend(ctx, org); err == nil {
spend = int64(sp.Consumed)
@@ -55,6 +69,28 @@ func OrgMoney(s *cloud.Service[State], ctx context.Context, org string) (spend,
return spend, credits, ok
}
// orgMoneyFromFinance reads (spend30d, credits, ok) for one org from the co-resident finance
// ledger — the SAME wallet the ai prepaid gate debits and clients/billing shows. credits =
// the org-pool AVAILABLE prepaid balance; spend = the org's metered usage over the trailing
// 30 days (SumUsageSince — the windowed usage source the rolling AI-spend cap already reads,
// matching the SpendCents30d field the overview + orgs surfaces render). ok is false only on
// a REAL read failure, so a no-activity org reads (0, 0, true) and never marks the money
// source degraded.
func orgMoneyFromFinance(ctx context.Context, fin finance.Client, org string) (spend, credits int64, ok bool) {
ok = true
if bal, err := fin.Balance(ctx, org, org, "usd", false); err == nil {
credits = bal.Cents()
} else {
ok = false
}
if sum, err := fin.SumUsageSince(ctx, org, false, time.Now().AddDate(0, 0, -30).Unix()); err == nil {
spend = sum
} else {
ok = false
}
return spend, credits, ok
}
// FindOrg returns the IAM org by slug (nil, nil when it does not exist) so a management
// action can validate its target before acting — never credit or suspend an org that
// isn't real.
+21 -15
View File
@@ -25,27 +25,33 @@ func Guard(s *cloud.Service[State], h Handler) zip.Handler {
}
// GuardScoped is the gate for the ORG-SCOPED panels (me/overview/orgs/users/usage/
// analytics/bases). It admits a SuperAdmin (principal.IsSuperAdmin) OR an ORG admin
// (an admin of their OWN org principal.IsOrgAdmin) pinned to a validated org, and the
// handler then scopes every read to ResolveScope(c) — so a non-super caller passes the
// gate but the DATA layer hard-limits them to their own org subtree. Cross-tenant reads
// are impossible for a non-super caller regardless of input.
// analytics/bases + spend-caps). It admits a SuperAdmin (principal.IsSuperAdmin) OR an
// admin of an ENABLED WHITE-LABEL TENANT org (principal.IsOrgAdmin pinned to its
// validated org AND that org ∈ State.WLTenants), and the handler then scopes every read
// to ResolveScope(c) — so a non-super caller passes the gate but the DATA layer
// hard-limits them to their own org subtree. Cross-tenant reads are impossible for a
// non-super caller regardless of input.
//
// The non-super admission requires BOTH the sanitizer-minted X-User-IsOrgAdmin
// (principal.IsOrgAdmin — the "admin of my own org" bit, unforgeable because the boundary
// strips it on ingress) AND a validated principal pinned to its own org (principal.Org,
// the ONE org accessor: validated X-User-Id + non-empty in-bounds X-Org-Id). So a
// validated but NON-admin MEMBER of an org is
// REFUSED here — the same denial an anonymous caller who forged X-Org-Id gets — closing
// the same-tenant over-visibility gap where any org member could read their org's admin
// panels. A validated non-super principal's X-Org-Id is PINNED by the boundary to their
// own owner, never client-chosen.
// THREE admission facts, ALL required for the non-super tier — the escalation line:
// - X-User-IsOrgAdmin (principal.IsOrgAdmin — "admin of my own org", unforgeable
// because SanitizeIdentity strips it on ingress and re-mints only from a validated
// isAdmin claim), AND
// - a validated principal pinned to its own org (principal.Org: validated X-User-Id +
// non-empty in-bounds X-Org-Id — the boundary PINS a non-super caller's X-Org-Id to
// their own owner, never client-chosen), AND
// - that org is an ENABLED WL tenant (State.IsWhiteLabelTenant — the fail-closed
// allowlist). This is the tier the old gate was MISSING: it admitted ANY org-admin,
// so any customer's own-org admin could open the cockpit. Now an org-admin whose org
// is not an enabled WL tenant is REFUSED — the SAME 403 a non-admin member or a
// forged-X-Org-Id anonymous caller gets. A SuperAdmin never consults the allowlist.
//
// Fail-closed everywhere: nil/empty WLTenants ⇒ ONLY SuperAdmins reach these panels.
func GuardScoped(s *cloud.Service[State], h Handler) zip.Handler {
return func(c *zip.Ctx) error {
if principal.IsSuperAdmin(c) {
return h(s, c) // SuperAdmin: cross-tenant, admitted regardless of org pin.
}
if _, ok := principal.Org(c); ok && principal.IsOrgAdmin(c) {
if org, ok := principal.Org(c); ok && principal.IsOrgAdmin(c) && s.State.IsWhiteLabelTenant(org) {
return h(s, c)
}
return zip.ErrForbidden("admin required")
+64 -17
View File
@@ -30,6 +30,17 @@ type CreditRequest struct {
AmountCents int64 `json:"amountCents"`
Currency string `json:"currency"`
Reason string `json:"reason"`
// User names the MEMBER to credit, by IAM username — the `name` half of
// "<org>/<name>". Empty credits the org itself.
//
// It exists because "the org" is not always the account a request spends from.
// In the shared signup org, whose members are strangers to each other, each pays
// from their OWN wallet; a grant keyed on the org alone lands in a pool that
// member can neither spend nor see, while they are refused at $0. Which of the
// two a grant lands on is NOT decided here — principal.WalletFor asks account.Payer,
// the same rule the spend gate asks — so a pooled tenant org keeps one balance no
// matter what is named, and a per-member org can finally be funded per member.
User string `json:"user"`
// Source splits the grant into the commerce ledger's two money buckets:
// - "trial" (default) — a non-cash promo/comp credit: spendable on non-premium
// metered usage only, NEVER refundable cash and NEVER paid out.
@@ -75,7 +86,7 @@ func grantNote(c *zip.Ctx, reason string) string {
}
// grantIdempotencyKey derives the DETERMINISTIC commerce idempotency key for a grant from
// its (org, amount, currency, source) BOUND to the operator-supplied Idempotency-Key nonce
// its (subject, amount, currency, source) BOUND to the operator-supplied Idempotency-Key nonce
// — so a retried grant (a commit-then-timeout re-submit carrying the SAME nonce) dedupes at
// commerce (X-Idempotency-Key, at-most-once), while two DISTINCT grants — even same org +
// amount — never collide. Binding the amount/currency/source into the hash means a nonce
@@ -88,7 +99,10 @@ func grantNote(c *zip.Ctx, reason string) string {
// additive — the pre-existing behavior. Effective end-to-end once the operator console
// sends an Idempotency-Key per grant attempt (reused verbatim on retry); commerce already
// enforces the dedup (api/billing/deposit.go).
func grantIdempotencyKey(c *zip.Ctx, org, currency, source string, amountCents int64) string {
// The SUBJECT is hashed, not the org: two grants of the same amount to two members
// of one org are DIFFERENT grants, and hashing the org alone would make the second
// dedupe away against the first — a silently dropped credit.
func grantIdempotencyKey(c *zip.Ctx, subject, currency, source string, amountCents int64) string {
nonce := strings.TrimSpace(c.Header("Idempotency-Key"))
if nonce == "" {
nonce = strings.TrimSpace(c.Header("X-Idempotency-Key"))
@@ -97,7 +111,7 @@ func grantIdempotencyKey(c *zip.Ctx, org, currency, source string, amountCents i
return ""
}
sum := sha256.Sum256([]byte(strings.Join([]string{
org, strconv.FormatInt(amountCents, 10), currency, source, nonce,
subject, strconv.FormatInt(amountCents, 10), currency, source, nonce,
}, "|")))
return "grant-" + hex.EncodeToString(sum[:])
}
@@ -138,6 +152,18 @@ func ApplyGrant(s *cloud.Service[State], c *zip.Ctx, org string, req CreditReque
return c.JSON(503, map[string]any{"status": "error", "msg": "grant refused: no durable audit store is configured on this deployment; a credit grant must be recorded before money moves", "data": nil})
}
// Resolve the ADDRESS the grant lands at — the same rule (account.Payer) the
// spend gate resolves the payer with, so the credit and the spend it funds name
// one wallet. Empty req.User is the org; a named member of a POOLED org still
// resolves to that org's pool, because that is the account their requests will
// be gated on. A refusal here means the name could not be turned into an address
// (a "/" in it would silently address something else), never a fallback.
w, addressed := principal.WalletFor(org, req.User)
if !addressed {
return Fail(c, "user must be a bare IAM username (no '/')")
}
subject := w.Account
tag, source := grantTag(req.Source)
notes := grantNote(c, req.Reason)
@@ -146,24 +172,31 @@ func ApplyGrant(s *cloud.Service[State], c *zip.Ctx, org string, req CreditReque
// with the commerce HTTP deposit as the split-deploy fallback. Both return the
// pre-balance (recorded even on failure), the entry/transaction id, and the post-balance,
// so the audit + response below are one shape regardless of which path moved the money.
before, txID, after, afterExact, derr := grantDeposit(s, c, org, currency, notes, tag, source, req.AmountCents)
// w.Ledger, not the raw path/body org: both halves of the address come from ONE
// resolved Account, so the ledger a deposit opens can never disagree in case with
// the subject written into it.
before, txID, after, afterExact, derr := grantDeposit(s, c, w.Ledger, subject, currency, notes, tag, source, req.AmountCents)
if derr != nil {
// The grant did not land — record the FAILED attempt (accountability), then
// surface the error. Never report a grant that failed as success.
EmitAudit(s, c, "admin.customer.credit", "credit", org,
map[string]any{"balanceCents": before},
map[string]any{"amountCents": req.AmountCents, "currency": currency, "reason": req.Reason, "source": source, "error": derr.Error()},
map[string]any{"amountCents": req.AmountCents, "currency": currency, "reason": req.Reason, "source": source, "subject": subject, "error": derr.Error()},
audit.Outcome{Result: "error", Status: 200, Reason: "grant failed"})
return Fail(c, "grant failed: "+derr.Error())
}
EmitAudit(s, c, "admin.customer.credit", "credit", org,
map[string]any{"balanceCents": before},
map[string]any{"balanceCents": after, "grantedCents": req.AmountCents, "currency": currency, "reason": req.Reason, "source": source, "transactionId": txID},
map[string]any{"balanceCents": after, "grantedCents": req.AmountCents, "currency": currency, "reason": req.Reason, "source": source, "subject": subject, "transactionId": txID},
audit.Outcome{Result: "success", Status: 200})
// subject is echoed because the operator does not choose it — account.Payer does.
// Naming a member of a pooled org credits the pool, and the response has to say so
// rather than let the caller assume a member wallet exists.
return OK(c, map[string]any{
"org": org,
"subject": subject,
"grantedCents": req.AmountCents,
"currency": currency,
"source": source,
@@ -175,12 +208,20 @@ func ApplyGrant(s *cloud.Service[State], c *zip.Ctx, org string, req CreditReque
// grantDeposit performs the ONE credit money-move for a grant. It prefers the co-resident
// native finance wallet — the ai prepaid gate and the edge meter read/debit THAT wallet,
// so an admin grant must credit it (subject == the org slug, the org-pool wallet) — and
// falls back to the commerce HTTP deposit only when no finance ledger is co-resident (a
// split deploy). It returns the pre-balance (so ApplyGrant can audit even a FAILED
// attempt), the entry/transaction id, and the post-balance, so the audit + response are
// one shape regardless of which path moved the money.
func grantDeposit(s *cloud.Service[State], c *zip.Ctx, org, currency, notes, tag, source string, amountCents int64) (before int64, txID string, after int64, afterExact string, err error) {
// so an admin grant must credit it — and falls back to the commerce HTTP deposit only when
// no finance ledger is co-resident (a split deploy). It returns the pre-balance (so
// ApplyGrant can audit even a FAILED attempt), the entry/transaction id, and the
// post-balance, so the audit + response are one shape regardless of which path moved the
// money.
//
// subject is the ACCOUNT within org's ledger, already resolved by account.Payer: the org
// slug for a pooled org, "<org>/<name>" for a member of a per-member one. Every balance
// read here uses it too — reading the pool around a member's credit would report a
// before/after that never moved and audit a lie.
//
// The split-deploy fallback can only address the org: commerce's HTTP deposit is org-keyed.
// It therefore refuses a member-addressed grant rather than silently crediting the pool.
func grantDeposit(s *cloud.Service[State], c *zip.Ctx, org, subject, currency, notes, tag, source string, amountCents int64) (before int64, txID string, after int64, afterExact string, err error) {
ctx := c.Context()
// ONE credit path: prefer the in-proc commerce credit ledger (creditledger) — the
// SAME injected ledger adapter commerce's POST /v1/billing/credit mints through
@@ -191,16 +232,17 @@ func grantDeposit(s *cloud.Service[State], c *zip.Ctx, org, currency, notes, tag
// the SAME co-resident finance ledger for the audit trail (exact, sub-cent visible).
if led := creditledger.Get(); led != nil {
if fin := finance.Current(); fin != nil {
if bal, berr := fin.Balance(ctx, org, org, currency, false); berr == nil {
if bal, berr := fin.Balance(ctx, org, subject, currency, false); berr == nil {
before = bal.Cents()
}
}
id, balCents, cerr := led.Credit(ctx, creditledger.CreditInput{
Org: org,
Subject: subject,
Currency: currency,
Reason: notes,
Tag: tag,
IdempotencyKey: grantIdempotencyKey(c, org, currency, source, amountCents),
IdempotencyKey: grantIdempotencyKey(c, subject, currency, source, amountCents),
AmountCents: amountCents,
})
if cerr != nil {
@@ -208,16 +250,21 @@ func grantDeposit(s *cloud.Service[State], c *zip.Ctx, org, currency, notes, tag
}
after = balCents
if fin := finance.Current(); fin != nil {
if bal, berr := fin.Balance(ctx, org, org, currency, false); berr == nil {
if bal, berr := fin.Balance(ctx, org, subject, currency, false); berr == nil {
afterExact = bal.AttoString() // afterExact = the EXACT balance (sub-cent visible)
}
}
return before, id, after, afterExact, nil
}
// Split deploy: no co-resident credit ledger → the commerce billing HTTP deposit, with
// its operator-nonce idempotency key so a retried grant dedupes at commerce.
// its operator-nonce idempotency key so a retried grant dedupes at commerce. It is
// org-keyed, so a member-addressed grant has no destination here and is REFUSED —
// crediting the pool instead would put the money where that member cannot spend it.
if subject != org {
return 0, "", 0, "", fmt.Errorf("this deployment has no co-resident ledger; only the org itself can be credited over the commerce HTTP path (asked for %q)", subject)
}
beforeC, _ := s.State.Commerce.Credits(ctx, org)
idem := grantIdempotencyKey(c, org, currency, source, amountCents)
idem := grantIdempotencyKey(c, subject, currency, source, amountCents)
res, derr := s.State.Commerce.Deposit(ctx, org, money.Cents(amountCents), currency, notes, tag, idem)
if derr != nil {
return int64(beforeC), "", int64(beforeC), "", derr
+84
View File
@@ -0,0 +1,84 @@
package core
import (
"net/http/httptest"
"testing"
"github.com/hanzoai/account"
"github.com/hanzoai/cloud/clients/principal"
"github.com/zap-proto/zip"
)
// A grant is money entering a wallet, so it has the same address a spend does, and
// it must be resolved by the same rule — otherwise credit accumulates at an address
// no gate reads while the member it was meant for is refused at $0.
// TestGrantAddressIsTheSpendAddress: whatever a grant names, the account it lands on
// is account.Payer's answer — the very account that member's own requests are gated
// on. A pooled tenant org keeps ONE balance no matter which member is named, because
// in a pooled org there is no member wallet and money put in one could never be spent.
func TestGrantAddressIsTheSpendAddress(t *testing.T) {
for _, tc := range []struct {
org, user string
want string
ok bool
}{
{account.SignupOrg, "", account.SignupOrg, true}, // the signup org itself: the pool
{account.SignupOrg, "z@hanzo.ai", account.SignupOrg + "/z@hanzo.ai", true}, // a stranger in the shared org: their own wallet
{"acme", "", "acme", true}, // a tenant org: the pool
{"acme", "bob", "acme", true}, // a member of a POOLED org: still the pool
{"ACME", "Bob", "acme", true}, // folded, so one wallet not two
{"", "bob", "", false}, // no org, no address — never mint an orphan
{account.SignupOrg, "hanzo/alice", "", false}, // a key is not a name; refuse rather than address something else
} {
w, ok := principal.WalletFor(tc.org, tc.user)
if ok != tc.ok {
t.Errorf("WalletFor(%q,%q) ok = %v, want %v", tc.org, tc.user, ok, tc.ok)
continue
}
if !ok {
continue
}
if w.Account != tc.want {
t.Errorf("WalletFor(%q,%q) account = %q, want %q", tc.org, tc.user, w.Account, tc.want)
}
// The two halves must come from one resolved Account, or a folded subject
// could sit under an unfolded ledger and address a second file.
if w.Ledger != account.Payer(account.Credential{Owner: tc.org, Name: tc.user}).Org() {
t.Errorf("WalletFor(%q,%q) ledger = %q — not the folded org the subject was built from", tc.org, tc.user, w.Ledger)
}
}
}
// TestGrantIdempotencyKeyBindsTheSubject: two members of one org, same operator
// nonce, same amount, are DIFFERENT grants. Hashing the org alone made the second
// dedupe away against the first — a credit silently dropped, which on a money path
// is indistinguishable from theft.
func TestGrantIdempotencyKeyBindsTheSubject(t *testing.T) {
key := func(subject string) string {
app := zip.New(zip.Config{DisableStartupMessage: true})
var out string
app.Get("/k", func(c *zip.Ctx) error {
out = grantIdempotencyKey(c, subject, "usd", "trial", 500)
return c.JSON(200, map[string]string{"key": out})
})
req := httptest.NewRequest("GET", "/k", nil)
req.Header.Set("Idempotency-Key", "one-nonce")
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("key probe: %v", err)
}
_ = resp.Body.Close()
return out
}
alice, bob, pool := key("hanzo/alice"), key("hanzo/bob"), key("hanzo")
if alice == "" {
t.Fatal("an operator nonce must produce a key")
}
if alice == bob || alice == pool || bob == pool {
t.Fatalf("distinct addresses collided: alice=%s bob=%s pool=%s — the second grant dedupes away", alice, bob, pool)
}
if again := key("hanzo/alice"); again != alice {
t.Fatalf("the same grant retried produced a different key (%s != %s) — a retry double-credits", again, alice)
}
}
+26
View File
@@ -8,6 +8,8 @@
package core
import (
"strings"
"github.com/hanzoai/cloud/audit"
"github.com/hanzoai/cloud/clients/admin/commerce"
"github.com/hanzoai/cloud/clients/admin/digitalocean"
@@ -29,4 +31,28 @@ type State struct {
DO *digitalocean.Client
AdminOrg string
AuditStore *audit.Recorder
// WLTenants is the fail-closed allowlist of enabled WHITE-LABEL TENANT orgs — the
// resellers/brands whose OWN-org admins may reach the org-scoped cockpit panels
// (GuardScoped) at admin.<brand>. It is the second admission tier next to the admin
// org: a SuperAdmin (owner == AdminOrg) is cross-tenant and never consults this set;
// EVERY other caller must be an admin of an org that is IN this set, and is then
// hard-scoped to that org's subtree. Seeded ONCE from ADMIN_WL_TENANT_ORGS at Mount
// and otherwise immutable, so the decision is a deliberate, git-/KMS-auditable
// onboarding — never self-service. EMPTY by default: with no entry, NO customer
// org-admin is admitted (only SuperAdmins reach the cockpit), so a mis-set/absent
// env fails CLOSED, never open.
WLTenants map[string]bool
}
// IsWhiteLabelTenant reports whether org is an enabled white-label tenant — the ONE
// place the WL admission decision is read. Nil/empty set ⇒ always false (fail-closed):
// no org is a WL tenant until explicitly enabled. The org is matched VERBATIM (only
// trimmed), the same owner key principal.Org yields, so folding can never collapse a
// distinct owner into an enabled one.
func (st State) IsWhiteLabelTenant(org string) bool {
if len(st.WLTenants) == 0 {
return false
}
return st.WLTenants[strings.TrimSpace(org)]
}
+5 -5
View File
@@ -18,7 +18,7 @@ package core
// (metrics/invoices/subscriptions) compose. They read commerce.events — the
// single warehouse table the commerce analytics collector lands every
// customer-activity event in (subscription/invoice/usage lifecycle) — over the
// SAME shared client (aiobject.DatastoreQuery) the o11y/compute/analytics lenses
// SAME shared client (datastore.Query) the o11y/compute/analytics lenses
// already use, no second connection. This mirrors compute.go's row-coercers and
// EXISTS-TABLE probe, hoisted here so the three sibling domains share ONE copy
// instead of each re-deriving it (DRY; the admin-package o11y/compute keep their
@@ -37,7 +37,7 @@ import (
"strings"
"time"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud/clients/datastore"
)
// BillingEventsTable is the collector-owned warehouse table the commerce
@@ -69,21 +69,21 @@ var (
// WarehouseReady reports whether the shared datastore ledger is connected, the
// gate every fleet read checks first (honest-empty when false).
func WarehouseReady() bool { return aiobject.DatastoreEnabled() }
func WarehouseReady() bool { return datastore.Ready() }
// BillingEventsReady reports whether the warehouse is connected AND the
// collector's commerce.events table is provisioned — the two-part gate every
// billing fleet view opens with, so an unwired collector degrades to an honest
// empty aggregate rather than an error.
func BillingEventsReady(ctx context.Context) bool {
return aiobject.DatastoreEnabled() && CHTableExists(ctx, BillingEventsTable)
return datastore.Ready() && CHTableExists(ctx, BillingEventsTable)
}
// CHTableExists probes the datastore for a table's presence. The name is a
// package constant (never user input), so EXISTS TABLE is safe. Any error →
// false (honest "not available yet"), mirroring compute.computeTableExists.
func CHTableExists(ctx context.Context, qualified string) bool {
rows, err := aiobject.DatastoreQuery(ctx, "EXISTS TABLE "+qualified)
rows, err := datastore.Query(ctx, "EXISTS TABLE "+qualified)
if err != nil || len(rows) == 0 {
return false
}
+2
View File
@@ -126,6 +126,7 @@ func Grants(s *cloud.Service[core.State], c *zip.Ctx) error {
// org (which the per-customer route carries in its path instead).
type issueGrantRequest struct {
Org string `json:"org"`
User string `json:"user"` // optional member to credit; empty is the org itself
AmountCents int64 `json:"amountCents"`
Currency string `json:"currency"`
Reason string `json:"reason"`
@@ -145,6 +146,7 @@ func IssueGrant(s *cloud.Service[core.State], c *zip.Ctx) error {
return core.Fail(c, "org is required")
}
return core.ApplyGrant(s, c, org, core.CreditRequest{
User: body.User,
AmountCents: body.AmountCents,
Currency: body.Currency,
Reason: body.Reason,
+1 -2
View File
@@ -3,13 +3,12 @@ package customer
import (
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/zap-proto/zip"
)
// Routes registers the customer-management surface (SuperAdmin only). List (static)
// precedes the :org param route; the write actions are POST (distinct method), so none
// collide. The grants ledger + the org-in-body issue-grant share the ONE credit path.
func Routes(app *zip.App, s *cloud.Service[core.State]) {
func Routes(app cloud.Router, s *cloud.Service[core.State]) {
g := app.Group("/v1/admin")
g.Get("/customers", core.Guard(s, Customers))
g.Get("/customers/:org", core.Guard(s, CustomerDetail))
+490 -7
View File
@@ -1,7 +1,12 @@
// Package do reads DigitalOcean's billing API for the finance dashboard's cost
// side. DO is our PRIMARY venue (a large promotional credit); this client turns
// the customer balance + billing history into money.Cents the finance aggregator
// folds into gross margin and runway.
// Package digitalocean reads DigitalOcean's billing and infrastructure APIs. DO is
// our PRIMARY venue (a large promotional credit); this client turns the customer
// balance + billing history into money.Cents the finance aggregator folds into gross
// margin and runway, and exposes the account's physical inventory — droplets,
// block-storage volumes, DOKS clusters, load balancers — that the /v1/admin/infra
// board reads.
//
// This is the ONE DigitalOcean client the admin plane uses. A new DO read is a
// method here calling the shared get/send primitive, never a second client.
//
// Auth is a single personal-access token, DO_API_TOKEN, sourced from a KMSSecret on
// the cloud env — NEVER hard-coded. When the token is unset the client is not Ready
@@ -15,12 +20,14 @@
package digitalocean
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"math"
"net/http"
"net/url"
"strconv"
"strings"
"time"
@@ -142,24 +149,497 @@ func (c *Client) History(ctx context.Context, perPage int) ([]Entry, error) {
return out, nil
}
// Volume is one DO block-storage volume: capacity + attachment + region. DO's API
// gives capacity and which droplets a volume is attached to, but NOT fill % — the
// caller enriches fill only where a filesystem source (the datastore's own
// system.disks) reports it, and renders an honest "—" everywhere else.
type Volume struct {
ID string
Name string
Region string
SizeGiB int
DropletIDs []int
// Tags carries DO's resource tags. DOKS stamps `k8s:<cluster-uuid>` on the volumes
// it provisions, but that tag is ADVISORY ONLY — it survives cluster deletion and
// is wrong often enough that it must never decide whether a volume is garbage. The
// only sound liveness test is a PV cross-reference (see clients/admin/infra).
Tags []string
CreatedAt string
}
// volumeWire is the raw DO /v2/volumes row.
type volumeWire struct {
ID string `json:"id"`
Name string `json:"name"`
SizeGigabytes int `json:"size_gigabytes"`
Region struct {
Slug string `json:"slug"`
} `json:"region"`
DropletIDs []int `json:"droplet_ids"`
Tags []string `json:"tags"`
CreatedAt string `json:"created_at"`
}
// Volumes lists ALL block-storage volumes across the account. Capacity and attachment
// are real; per-volume fill is NOT exposed by DO and stays absent (honest) until a
// filesystem source reports it.
func (c *Client) Volumes(ctx context.Context) ([]Volume, error) {
rows, err := listAll[volumeWire](ctx, c, "/v2/volumes", "volumes")
if err != nil {
return nil, err
}
out := make([]Volume, len(rows))
for i, v := range rows {
out[i] = Volume{
ID: v.ID,
Name: v.Name,
Region: v.Region.Slug,
SizeGiB: v.SizeGigabytes,
DropletIDs: v.DropletIDs,
Tags: v.Tags,
CreatedAt: v.CreatedAt,
}
}
return out, nil
}
// Droplet is one DO droplet. LocalDiskGiB is the droplet's own disk, which is
// INCLUDED in MonthlyCents — it is NOT separately billed, and conflating it with
// block storage is how a fleet appears to hold terabytes it never pays for.
type Droplet struct {
ID int
Name string
Region string
Status string
SizeSlug string
VCPUs int
MemoryMiB int
LocalDiskGiB int
MonthlyCents money.Cents
CreatedAt string
PrivateIP string
PublicIP string
Tags []string
VolumeIDs []string
}
// dropletWire is the raw DO /v2/droplets row.
type dropletWire struct {
ID int `json:"id"`
Name string `json:"name"`
Status string `json:"status"`
SizeSlug string `json:"size_slug"`
VCPUs int `json:"vcpus"`
Memory int `json:"memory"`
Disk int `json:"disk"`
Size struct {
PriceMonthly float64 `json:"price_monthly"`
} `json:"size"`
Region struct {
Slug string `json:"slug"`
} `json:"region"`
Networks struct {
V4 []struct {
Type string `json:"type"`
IPAddress string `json:"ip_address"`
} `json:"v4"`
} `json:"networks"`
CreatedAt string `json:"created_at"`
Tags []string `json:"tags"`
VolumeIDs []string `json:"volume_ids"`
}
// Droplets lists ALL droplets across the account.
func (c *Client) Droplets(ctx context.Context) ([]Droplet, error) {
rows, err := listAll[dropletWire](ctx, c, "/v2/droplets", "droplets")
if err != nil {
return nil, err
}
out := make([]Droplet, len(rows))
for i, d := range rows {
dr := Droplet{
ID: d.ID,
Name: d.Name,
Region: d.Region.Slug,
Status: d.Status,
SizeSlug: d.SizeSlug,
VCPUs: d.VCPUs,
MemoryMiB: d.Memory,
LocalDiskGiB: d.Disk,
MonthlyCents: centsOf(d.Size.PriceMonthly),
CreatedAt: d.CreatedAt,
Tags: d.Tags,
VolumeIDs: d.VolumeIDs,
}
for _, n := range d.Networks.V4 {
switch n.Type {
case "private":
dr.PrivateIP = n.IPAddress
case "public":
dr.PublicIP = n.IPAddress
}
}
out[i] = dr
}
return out, nil
}
// Cluster is one DOKS cluster.
type Cluster struct {
ID string
Name string
Region string
Version string
Status string
Pools []NodePool
CreatedAt string
}
// NodePool is one DOKS node pool — the ONLY correct way to change a cluster's node
// count. DOKS owns the droplets in a pool: deleting or resizing one directly is undone
// by the pool controller, which recreates it.
type NodePool struct {
ID string
Name string
Size string
Count int
}
// clusterWire is the raw DO /v2/kubernetes/clusters row.
type clusterWire struct {
ID string `json:"id"`
Name string `json:"name"`
Region string `json:"region"`
Version string `json:"version"`
Status struct {
State string `json:"state"`
} `json:"status"`
NodePools []struct {
ID string `json:"id"`
Name string `json:"name"`
Size string `json:"size"`
Count int `json:"count"`
} `json:"node_pools"`
CreatedAt string `json:"created_at"`
}
// Clusters lists ALL DOKS clusters. This is the authoritative denominator for the
// orphan analysis: a volume may only be called unreferenced once EVERY cluster here
// has been searched for a PV that claims it.
func (c *Client) Clusters(ctx context.Context) ([]Cluster, error) {
rows, err := listAll[clusterWire](ctx, c, "/v2/kubernetes/clusters", "kubernetes_clusters")
if err != nil {
return nil, err
}
out := make([]Cluster, len(rows))
for i, k := range rows {
cl := Cluster{
ID: k.ID,
Name: k.Name,
Region: k.Region,
Version: k.Version,
Status: k.Status.State,
CreatedAt: k.CreatedAt,
Pools: make([]NodePool, len(k.NodePools)),
}
for j, p := range k.NodePools {
cl.Pools[j] = NodePool{ID: p.ID, Name: p.Name, Size: p.Size, Count: p.Count}
}
out[i] = cl
}
return out, nil
}
// Kubeconfig fetches a cluster's admin kubeconfig. DO returns a token-based config
// against the cluster's public https endpoint (never an exec plugin), which the
// caller must still funnel through fleet.SafeRESTConfig before dialing.
func (c *Client) Kubeconfig(ctx context.Context, clusterID string) ([]byte, error) {
if !c.Ready() {
return nil, fmt.Errorf("DO_API_TOKEN not configured")
}
if strings.TrimSpace(clusterID) == "" {
return nil, fmt.Errorf("cluster id required")
}
return c.get(ctx, "/v2/kubernetes/clusters/"+url.PathEscape(clusterID)+"/kubeconfig")
}
// LoadBalancer is one DO load balancer. DO does not price LBs in the API, so cost is
// derived from the billed unit count (see lbUnitCents).
type LoadBalancer struct {
ID string
Name string
Region string
Status string
IP string
SizeUnit int
MonthlyCents money.Cents
DropletIDs []int
}
// lbWire is the raw DO /v2/load_balancers row.
type lbWire struct {
ID string `json:"id"`
Name string `json:"name"`
Status string `json:"status"`
IP string `json:"ip"`
Region struct {
Slug string `json:"slug"`
} `json:"region"`
SizeUnit int `json:"size_unit"`
DropletIDs []int `json:"droplet_ids"`
}
// LoadBalancers lists ALL load balancers across the account.
func (c *Client) LoadBalancers(ctx context.Context) ([]LoadBalancer, error) {
rows, err := listAll[lbWire](ctx, c, "/v2/load_balancers", "load_balancers")
if err != nil {
return nil, err
}
out := make([]LoadBalancer, len(rows))
for i, l := range rows {
units := l.SizeUnit
if units <= 0 {
units = 1
}
out[i] = LoadBalancer{
ID: l.ID,
Name: l.Name,
Region: l.Region.Slug,
Status: l.Status,
IP: l.IP,
SizeUnit: units,
MonthlyCents: money.Cents(units) * lbUnitCents,
DropletIDs: l.DropletIDs,
}
}
return out, nil
}
// Snapshot is a created block-storage snapshot.
type Snapshot struct {
ID string
Name string
SizeGiB int
}
// SnapshotVolume takes a point-in-time snapshot of a volume. This is the "undo" that
// makes a delete recoverable, so the delete path takes one FIRST by default.
func (c *Client) SnapshotVolume(ctx context.Context, volumeID, name string) (Snapshot, error) {
var out Snapshot
if !c.Ready() {
return out, fmt.Errorf("DO_API_TOKEN not configured")
}
if strings.TrimSpace(volumeID) == "" {
return out, fmt.Errorf("volume id required")
}
body, err := c.send(ctx, http.MethodPost, "/v2/volumes/"+url.PathEscape(volumeID)+"/snapshots",
map[string]string{"name": name})
if err != nil {
return out, err
}
var w struct {
Snapshot struct {
ID string `json:"id"`
Name string `json:"name"`
SizeGigabytes int `json:"size_gigabytes"`
} `json:"snapshot"`
}
if err := json.Unmarshal(body, &w); err != nil {
return out, fmt.Errorf("do snapshot decode: %w", err)
}
return Snapshot{ID: w.Snapshot.ID, Name: w.Snapshot.Name, SizeGiB: w.Snapshot.SizeGigabytes}, nil
}
// DeleteVolume destroys a block-storage volume. Irreversible: callers MUST have
// proven the volume is referenced by no PV in any cluster first.
func (c *Client) DeleteVolume(ctx context.Context, volumeID string) error {
if !c.Ready() {
return fmt.Errorf("DO_API_TOKEN not configured")
}
if strings.TrimSpace(volumeID) == "" {
return fmt.Errorf("volume id required")
}
_, err := c.send(ctx, http.MethodDelete, "/v2/volumes/"+url.PathEscape(volumeID), nil)
return err
}
// DeleteDroplet destroys a droplet. Irreversible, and there is no snapshot-first undo
// for a droplet the way there is for a volume: callers MUST have proven the droplet is
// not a DOKS node first (see clients/admin/infra).
func (c *Client) DeleteDroplet(ctx context.Context, dropletID int) error {
if !c.Ready() {
return fmt.Errorf("DO_API_TOKEN not configured")
}
_, err := c.send(ctx, http.MethodDelete, "/v2/droplets/"+strconv.Itoa(dropletID), nil)
return err
}
// Action is a queued DO droplet action. DO performs a resize ASYNCHRONOUSLY, so a
// successful call means "accepted", not "done" — the id is what an operator polls.
type Action struct {
ID int
Status string
}
// ResizeDroplet changes a droplet's plan.
//
// disk=true makes the change PERMANENT AND IRREVERSIBLE: the disk grows and the droplet
// can never be resized down again. disk=false resizes CPU/RAM only and is reversible.
//
// DO requires the droplet to be powered off; if it is not, DO refuses and its message
// is surfaced verbatim rather than being retried or worked around.
func (c *Client) ResizeDroplet(ctx context.Context, dropletID int, size string, disk bool) (Action, error) {
var out Action
if !c.Ready() {
return out, fmt.Errorf("DO_API_TOKEN not configured")
}
if strings.TrimSpace(size) == "" {
return out, fmt.Errorf("size slug required")
}
body, err := c.send(ctx, http.MethodPost, "/v2/droplets/"+strconv.Itoa(dropletID)+"/actions",
map[string]any{"type": "resize", "size": strings.TrimSpace(size), "disk": disk})
if err != nil {
return out, err
}
var w struct {
Action struct {
ID int `json:"id"`
Status string `json:"status"`
} `json:"action"`
}
if err := json.Unmarshal(body, &w); err != nil {
return out, fmt.Errorf("do resize decode: %w", err)
}
return Action{ID: w.Action.ID, Status: w.Action.Status}, nil
}
// DeleteLoadBalancer destroys a load balancer. Irreversible, and it takes the public IP
// with it: callers MUST have proven no Kubernetes Service still targets it.
func (c *Client) DeleteLoadBalancer(ctx context.Context, lbID string) error {
if !c.Ready() {
return fmt.Errorf("DO_API_TOKEN not configured")
}
if strings.TrimSpace(lbID) == "" {
return fmt.Errorf("load balancer id required")
}
_, err := c.send(ctx, http.MethodDelete, "/v2/load_balancers/"+url.PathEscape(lbID), nil)
return err
}
// ScaleNodePool sets a node pool's node count. DO's update endpoint requires the pool's
// name alongside the count — omitting it clears the name, so it is always sent back.
func (c *Client) ScaleNodePool(ctx context.Context, clusterID, poolID, name string, count int) error {
if !c.Ready() {
return fmt.Errorf("DO_API_TOKEN not configured")
}
if strings.TrimSpace(clusterID) == "" || strings.TrimSpace(poolID) == "" {
return fmt.Errorf("cluster id and node pool id required")
}
_, err := c.send(ctx, http.MethodPut,
"/v2/kubernetes/clusters/"+url.PathEscape(clusterID)+"/node_pools/"+url.PathEscape(poolID),
map[string]any{"name": name, "count": count})
return err
}
// Pagination bounds for every DO collection read: 200 rows a page, a hard 25-page
// cap so a runaway can never loop, and the 8 MiB body ceiling a full droplet page
// needs (a volume page fits in far less).
const (
perPage = 200
maxPages = 25
maxBody = 8 << 20
maxRespLen = maxBody
)
// lbUnitCents is DO's published price for one load-balancer node ($12/mo). DO does
// not return LB pricing in the API, so this is the one place the rate is written.
const lbUnitCents = money.Cents(1200)
// listAll follows DO's page-number pagination for a collection endpoint, decoding
// rows out of the response's named key. It is the ONE pagination loop in this client
// — every collection read goes through it, so "stop on the short page or the reported
// total" is stated once and cannot drift between endpoints.
func listAll[T any](ctx context.Context, c *Client, path, key string) ([]T, error) {
if !c.Ready() {
return nil, fmt.Errorf("DO_API_TOKEN not configured")
}
var out []T
for page := 1; page <= maxPages; page++ {
body, err := c.get(ctx, fmt.Sprintf("%s?per_page=%d&page=%d", path, perPage, page))
if err != nil {
return nil, err
}
var w struct {
Meta struct {
Total int `json:"total"`
} `json:"meta"`
}
if err := json.Unmarshal(body, &w); err != nil {
return nil, fmt.Errorf("do %s decode: %w", key, err)
}
var keyed map[string]json.RawMessage
if err := json.Unmarshal(body, &keyed); err != nil {
return nil, fmt.Errorf("do %s decode: %w", key, err)
}
var rows []T
if raw, ok := keyed[key]; ok && len(raw) > 0 {
if err := json.Unmarshal(raw, &rows); err != nil {
return nil, fmt.Errorf("do %s decode: %w", key, err)
}
}
out = append(out, rows...)
// Stop on the last (short) page, or once we've collected the reported total.
if len(rows) < perPage || (w.Meta.Total > 0 && len(out) >= w.Meta.Total) {
break
}
}
return out, nil
}
// get performs one token-authenticated DO GET and returns the raw body.
func (c *Client) get(ctx context.Context, path string) ([]byte, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.base+path, nil)
return c.send(ctx, http.MethodGet, path, nil)
}
// send performs one token-authenticated DO request and returns the raw body. It is
// the single HTTP primitive of this client: every read and every mutation funnels
// through it, so auth, timeouts, the body ceiling and status handling exist once.
func (c *Client) send(ctx context.Context, method, path string, payload any) ([]byte, error) {
var rdr io.Reader
if payload != nil {
enc, err := json.Marshal(payload)
if err != nil {
return nil, err
}
rdr = bytes.NewReader(enc)
}
req, err := http.NewRequestWithContext(ctx, method, c.base+path, rdr)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json")
req.Header.Set("Authorization", "Bearer "+c.token)
if payload != nil {
req.Header.Set("Content-Type", "application/json")
}
resp, err := c.http.Do(req)
if err != nil {
return nil, fmt.Errorf("digitalocean unreachable: %w", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
body, err := io.ReadAll(io.LimitReader(resp.Body, maxRespLen))
if err != nil {
return nil, err
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
// DO returns {"id":"...","message":"..."} on error — surface the message so a
// failed mutation says WHY, not just a bare status.
var e struct {
Message string `json:"message"`
}
if json.Unmarshal(body, &e) == nil && strings.TrimSpace(e.Message) != "" {
return nil, fmt.Errorf("digitalocean status %d: %s", resp.StatusCode, e.Message)
}
return nil, fmt.Errorf("digitalocean status %d", resp.StatusCode)
}
return body, nil
@@ -178,5 +658,8 @@ func dollarsToCents(s string) money.Cents {
if err != nil {
return 0
}
return money.Cents(math.Round(f * 100))
return centsOf(f)
}
// centsOf rounds decimal dollars to integer cents.
func centsOf(f float64) money.Cents { return money.Cents(math.Round(f * 100)) }
+76 -1
View File
@@ -1,6 +1,81 @@
package digitalocean
import "testing"
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"testing"
)
// TestMutationWireShape pins the request each mutation actually sends. A wrong verb or
// path here fails silently against the real API — a PUT to the wrong node-pool URL just
// does nothing — so the shape is asserted rather than assumed.
func TestMutationWireShape(t *testing.T) {
var gotMethod, gotPath, gotBody string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
b, _ := io.ReadAll(r.Body)
gotMethod, gotPath, gotBody = r.Method, r.URL.Path, string(b)
fmt.Fprint(w, `{"action":{"id":77,"status":"in-progress"}}`)
}))
defer srv.Close()
c := NewWithBase(srv.URL, "test-token")
for _, tc := range []struct {
name string
call func() error
method, path string
body string
}{
{"delete droplet", func() error { return c.DeleteDroplet(context.Background(), 101) },
http.MethodDelete, "/v2/droplets/101", ""},
{"resize droplet", func() error {
_, err := c.ResizeDroplet(context.Background(), 101, "s-4vcpu-8gb", false)
return err
}, http.MethodPost, "/v2/droplets/101/actions", `{"disk":false,"size":"s-4vcpu-8gb","type":"resize"}`},
{"permanent resize", func() error {
_, err := c.ResizeDroplet(context.Background(), 101, "s-4vcpu-8gb", true)
return err
}, http.MethodPost, "/v2/droplets/101/actions", `{"disk":true,"size":"s-4vcpu-8gb","type":"resize"}`},
{"delete load balancer", func() error { return c.DeleteLoadBalancer(context.Background(), "lb-1") },
http.MethodDelete, "/v2/load_balancers/lb-1", ""},
{"scale node pool", func() error { return c.ScaleNodePool(context.Background(), "k8s-1", "pool-1", "workers", 3) },
http.MethodPut, "/v2/kubernetes/clusters/k8s-1/node_pools/pool-1", `{"count":3,"name":"workers"}`},
} {
t.Run(tc.name, func(t *testing.T) {
gotMethod, gotPath, gotBody = "", "", ""
if err := tc.call(); err != nil {
t.Fatalf("call: %v", err)
}
if gotMethod != tc.method || gotPath != tc.path {
t.Errorf("%s %s, want %s %s", gotMethod, gotPath, tc.method, tc.path)
}
if tc.body != "" && !sameJSON(gotBody, tc.body) {
t.Errorf("body = %s, want %s", gotBody, tc.body)
}
})
}
// An unconfigured client must never reach the network.
blank := New("")
for name, err := range map[string]error{
"droplet": blank.DeleteDroplet(context.Background(), 1),
"lb": blank.DeleteLoadBalancer(context.Background(), "x"),
"pool": blank.ScaleNodePool(context.Background(), "c", "p", "n", 1),
} {
if err == nil {
t.Errorf("%s: no error without a token", name)
}
}
}
func sameJSON(a, b string) bool {
var x, y any
return json.Unmarshal([]byte(a), &x) == nil && json.Unmarshal([]byte(b), &y) == nil &&
fmt.Sprint(x) == fmt.Sprint(y)
}
func TestDollarsToCents(t *testing.T) {
cases := []struct {
+2 -2
View File
@@ -5,7 +5,7 @@ import (
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/commerceclient"
"github.com/hanzoai/cloud/clients/commerce"
ledger "github.com/hanzoai/cloud/clients/finance"
"github.com/zap-proto/zip"
)
@@ -30,7 +30,7 @@ func Backfill(s *cloud.Service[core.State], c *zip.Ctx) error {
// and reads $0, which would migrate nothing; the native read returns the real figure,
// or an ERROR when commerce is not co-resident (never a phantom zero the cutover would
// silently carry as "nothing to migrate").
balanceCents, err := commerceclient.BalanceCents(ctx, org, org, "usd", false)
balanceCents, err := commerce.BalanceCents(ctx, org, org, "usd", false)
if err != nil {
return core.Fail(c, "read commerce balance: "+err.Error())
}
-93
View File
@@ -1,93 +0,0 @@
// Copyright © 2026 Hanzo AI. MIT License.
package finance
import (
"strconv"
"strings"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
ledger "github.com/hanzoai/cloud/clients/finance"
"github.com/hanzoai/cloud/clients/money"
"github.com/hanzoai/cloud/types"
"github.com/zap-proto/zip"
)
// Deposit answers POST /v1/admin/finance/deposit — a SuperAdmin credit into an ARBITRARY
// subject's native prepaid wallet. Where the credit-grant + the backfill fund the org POOL
// (subject == the org slug), this funds a SPECIFIC wallet: an org pool ("hanzo") or a human
// ("hanzo/z" → wallet:z in orgs/hanzo/finance.db). It posts a balanced double-entry credit
// on the ONE finance ledger and is additive (no idempotency ref, so distinct grants stack).
// SuperAdmin only (core.Guard).
//
// Params arrive as a JSON body OR query: org, subject, cents (>0), notes?, currency? (usd).
// 503 when no finance ledger is co-resident on this deployment; 400 on a missing/invalid
// arg or non-positive cents.
func Deposit(s *cloud.Service[core.State], c *zip.Ctx) error {
fin := ledger.Current()
if fin == nil {
return zip.Errorf(503, "finance ledger not co-resident on this deployment")
}
// Body is optional — params may arrive as query instead — so a non-JSON/empty body is
// not an error; the query values below fill or override it.
var body struct {
Org string `json:"org"`
Subject string `json:"subject"`
Cents int64 `json:"cents"`
Notes string `json:"notes"`
Currency string `json:"currency"`
}
_ = c.Bind(&body)
org := pick(c.Query("org"), body.Org)
subject := pick(c.Query("subject"), body.Subject)
notes := pick(c.Query("notes"), body.Notes)
currency := strings.ToLower(pick(c.Query("currency"), body.Currency))
if currency == "" {
currency = "usd"
}
cents := body.Cents
if q := strings.TrimSpace(c.Query("cents")); q != "" {
n, err := strconv.ParseInt(q, 10, 64)
if err != nil {
return zip.ErrBadRequest("cents must be an integer")
}
cents = n
}
if org == "" || subject == "" {
return zip.ErrBadRequest("org and subject are required")
}
if cents <= 0 {
return zip.ErrBadRequest("cents must be positive")
}
entryID, err := fin.Deposit(c.Context(), types.DepositInput{
Org: org,
Subject: subject,
Amount: money.FromCents(cents),
Currency: currency,
Notes: notes,
})
if err != nil {
return core.Fail(c, "finance deposit: "+err.Error())
}
return core.OK(c, map[string]any{
"org": org,
"subject": subject,
"cents": cents,
"entryId": entryID,
})
}
// pick returns the first non-empty, trimmed value — a query param preferred over the body,
// so either channel funds a wallet with one handler.
func pick(query, body string) string {
if q := strings.TrimSpace(query); q != "" {
return q
}
return strings.TrimSpace(body)
}
+8 -4
View File
@@ -27,14 +27,18 @@ import (
var errUnconfigured = errors.New("not configured")
// Routes registers the finance dashboard (SuperAdmin only).
func Routes(app *zip.App, s *cloud.Service[core.State]) {
func Routes(app cloud.Router, s *cloud.Service[core.State]) {
g := app.Group("/v1/admin")
g.Get("/finance", core.Guard(s, Finance))
// One-time commerce→finance balance cutover (SuperAdmin only). Idempotent per org.
g.Post("/finance/backfill", core.Guard(s, Backfill))
// Fund an ARBITRARY subject's native wallet — an org pool or a human ("hanzo/z").
// SuperAdmin only; additive (grants stack).
g.Post("/finance/deposit", core.Guard(s, Deposit))
// There is no second credit-write here. POST /finance/deposit used to fund an
// arbitrary subject verbatim, because the credit grant could only reach the org
// pool; the grant now resolves its address through account.Payer and can name a
// member, so the raw route's only reason to exist is gone. It was also the unsafe
// one — no cap, no audit row, and no idempotency ref, so a double-clicked deposit
// credited twice. ONE credit-write path: core.ApplyGrant.
// Per-provider upstream credit ledger + usage funding split (multi-provider
// credit-management). Same SuperAdmin guard, same cloud_usage warehouse.
g.Get("/providers/credit", core.Guard(s, ProvidersCredit))
+8 -5
View File
@@ -24,6 +24,8 @@ import (
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/datastore"
"github.com/hanzoai/types"
"github.com/zap-proto/zip"
)
@@ -120,13 +122,13 @@ func computeProviderCredits(ctx context.Context, s *cloud.Service[core.State]) [
// customer's usage). Honest-empty ({}) on any datastore blip — never 5xxs.
func providerBurnCents(ctx context.Context) map[string]int64 {
burn := map[string]int64{}
if !aiobject.DatastoreEnabled() {
if !datastore.Ready() {
return burn
}
if err := aiobject.EnsureCloudUsageTable(ctx); err != nil {
return burn
}
rows, err := aiobject.DatastoreQuery(ctx,
rows, err := datastore.Query(ctx,
"SELECT provider, sum(cost_cents) AS burn FROM "+usageTable+" GROUP BY provider")
if err != nil {
return burn
@@ -174,7 +176,8 @@ func fundingClass(pc ProviderCredit) string {
// usage split by funding class over the window (default last 30d). SuperAdmin-guarded.
func UsageFunding(s *cloud.Service[core.State], c *zip.Ctx) error {
ctx := c.Context()
start, end, _, werr := aiobject.ResolveCloudUsageWindow("", c.Query("from"), c.Query("to"), time.Now().UTC())
w, werr := types.ParseWindow("", c.Query("from"), c.Query("to"), time.Now().UTC())
start, end := w.Start, w.End
if werr != nil {
end = time.Now().UTC()
start = end.AddDate(0, 0, -30)
@@ -186,9 +189,9 @@ func UsageFunding(s *cloud.Service[core.State], c *zip.Ctx) error {
}
out := []UsageFundingRow{}
if aiobject.DatastoreEnabled() {
if datastore.Ready() {
if err := aiobject.EnsureCloudUsageTable(ctx); err == nil {
rows, qerr := aiobject.DatastoreQuery(ctx,
rows, qerr := datastore.Query(ctx,
"SELECT provider, model, count() AS requests, sum(total_tokens) AS tokens, "+
"sum(cost_cents) AS cost_cents FROM "+usageTable+
" WHERE timestamp >= ? AND timestamp < ? GROUP BY provider, model ORDER BY cost_cents DESC",
+938
View File
@@ -0,0 +1,938 @@
// Package infra is the platform's DigitalOcean fleet board: the physical inventory
// (DOKS clusters, droplets, block-storage volumes, load balancers) cross-referenced
// against what every cluster's Kubernetes actually claims, with the cost of each and
// an orphan analysis that is safe BY CONSTRUCTION.
//
// THE RULE THIS PACKAGE EXISTS TO ENFORCE. A volume being detached, or carrying a
// `k8s:<cluster-uuid>` tag for some other cluster, does NOT make it garbage. Those two
// signals together would have condemned 4.39 TiB of live data belonging to running
// clusters. The ONLY sound liveness test is a cross-reference against the
// `spec.csi.volumeHandle` of every PersistentVolume in EVERY cluster — and it is only
// a valid test when every cluster answered. So:
//
// - a volume is deletable only when NO PV in ANY cluster names it, and
// - if even one cluster failed to scan, NOTHING is deletable (Complete=false).
//
// Absence of evidence is not evidence of absence: an unreachable cluster is treated as
// a cluster that might be holding the volume. The analysis fails CLOSED.
//
// "No pod mounts it" is a REVIEW signal, never a delete signal — an idle Bound PVC is
// an idle database, not garbage. Idle volumes are surfaced as a queue for a human and
// are never counted as reclaimable.
//
// THE SAME DISCIPLINE GOVERNS EVERY MUTATION, not just volume deletion. A droplet, a
// load balancer and a node pool each get a (allowed, reason) verdict derived HERE, from
// the same scan, behind the same completeness gate — see Snapshot.verdict. Handlers
// carry no policy: they read the verdict this file already reached.
//
// Analyze is a pure function of (DO inventory, cluster scans) so every rule above is
// unit-testable without a network or a cluster.
package infra
import (
"fmt"
"sort"
"strings"
"time"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/admin/digitalocean"
"github.com/hanzoai/cloud/clients/admin/money"
)
// volumeGiBCents is DO's block-storage rate: $0.10 per GiB per month. Droplet LOCAL
// disk is NOT billed at this rate (or at all, separately) — it is included in the
// droplet's own price. Conflating the two invents terabytes of phantom cost.
const volumeGiBCents = money.Cents(10)
// outlierShareBP flags any single resource costing at least this share of total fleet
// spend, in basis points (500bp = 5%). One explicable rule, no magic thresholds.
const outlierShareBP = money.Cents(500)
// Volume states. The state machine is total and ordered: attachment beats reference,
// reference beats absence. Only Unreferenced is ever deletable.
const (
// StateAttached — DO reports the volume attached to a droplet. In use, now.
StateAttached = "attached"
// StateBound — detached, but a PV in some cluster claims it and that PV is Bound
// to a PVC. This is live data between mounts; deleting it destroys a database.
StateBound = "bound"
// StateReleased — a PV claims it but is no longer Bound (Released/Available/Failed).
// A genuine cleanup candidate, but the PV still exists, so a human retires the PV.
StateReleased = "released"
// StateUnreferenced — no PV in ANY scanned cluster names it. The ONLY deletable state.
StateUnreferenced = "unreferenced"
)
// Finding severities.
const (
SevCritical = "critical"
SevWarn = "warn"
SevInfo = "info"
)
// firstParty are our own registries: anything here is ours by construction.
var firstParty = []string{
"ghcr.io/hanzoai/", "ghcr.io/luxfi/", "ghcr.io/zooai/",
"registry.hanzo.ai/", "registry.lux.network/", "registry.zoo.network/",
"registry.digitalocean.com/hanzo/",
}
// knownVendors is the REVIEWED third-party set — upstream images we deliberately run.
// Kept deliberately short: an image outside both lists is reported for a human to
// judge, which is the point. Growing this list is a review decision, not a reflex.
var knownVendors = []string{
"docker.io/library/", "library/", "registry.k8s.io/", "k8s.gcr.io/", "quay.io/",
"grafana/", "prom/", "prometheus/", "bitnami/", "minio/", "moby/",
"digitalocean/", "docker.digitalocean.com/", "acmglobaltech/", "hanzozt/",
}
// Inventory is the DigitalOcean account read — the half of the analysis input that
// needs no cluster.
type Inventory struct {
Clusters []digitalocean.Cluster
Droplets []digitalocean.Droplet
Volumes []digitalocean.Volume
LoadBalancers []digitalocean.LoadBalancer
}
// PVRef is one PersistentVolume's identity: which DO volume it claims, and whether
// that claim is still live.
type PVRef struct {
Name string
Phase string
VolumeHandle string
ClaimNS string
ClaimName string
}
// PVCRef is one PersistentVolumeClaim.
type PVCRef struct {
Namespace string
Name string
Phase string
Volume string
}
// PodRef is one pod, reduced to what the board needs: where it runs, whether it is
// healthy, which PVCs it mounts, and what images it runs.
type PodRef struct {
Namespace string
Name string
Phase string
Reason string
Node string
Claims []string
Images []string
}
// NodeState is one Kubernetes node's own view of itself.
type NodeState struct {
Name string
Ready bool
Schedulable bool
}
// ServiceRef is one Service of type LoadBalancer: the identities by which it claims a
// DO load balancer. It is to load balancers exactly what PVRef is to volumes — the only
// sound liveness test, because a DO load balancer carries no back-reference of its own.
type ServiceRef struct {
Namespace string
Name string
LBID string
IPs []string
}
// ClusterScan is ONE cluster's Kubernetes truth. Err non-nil means the cluster did
// not answer — which forces the whole analysis incomplete.
type ClusterScan struct {
ClusterID string
Err error
Nodes []NodeState
PVs []PVRef
PVCs []PVCRef
Pods []PodRef
Services []ServiceRef
}
// Snapshot is the whole board in one value.
type Snapshot struct {
At string `json:"at"`
Complete bool `json:"complete"`
IncompleteReason string `json:"incompleteReason"`
Sources []core.SourceStatus `json:"sources"`
Totals Totals `json:"totals"`
Cost Cost `json:"cost"`
Clusters []Cluster `json:"clusters"`
Nodes []Node `json:"nodes"`
Volumes []Volume `json:"volumes"`
LoadBalancers []LoadBalancer `json:"loadBalancers"`
Findings []Finding `json:"findings"`
}
// Totals are fleet counts. LocalDiskGiB is broken out precisely so it can be shown
// as NOT separately billed.
type Totals struct {
Clusters int `json:"clusters"`
Nodes int `json:"nodes"`
Volumes int `json:"volumes"`
LoadBalancers int `json:"loadBalancers"`
VolumeGiB int `json:"volumeGiB"`
AttachedVolumes int `json:"attachedVolumes"`
AttachedGiB int `json:"attachedGiB"`
DetachedVolumes int `json:"detachedVolumes"`
DetachedGiB int `json:"detachedGiB"`
UnreferencedVolumes int `json:"unreferencedVolumes"`
UnreferencedGiB int `json:"unreferencedGiB"`
IdlePVCs int `json:"idlePVCs"`
LocalDiskGiB int `json:"localDiskGiB"`
}
// Cost is monthly spend in cents. Reclaimable counts ONLY unreferenced volumes —
// never idle ones, which are live data awaiting a human verdict.
type Cost struct {
DropletsMonthly money.Cents `json:"dropletsMonthly"`
VolumesMonthly money.Cents `json:"volumesMonthly"`
LoadBalancersMonthly money.Cents `json:"loadBalancersMonthly"`
TotalMonthly money.Cents `json:"totalMonthly"`
ReclaimableMonthly money.Cents `json:"reclaimableMonthly"`
}
// Cluster is one DOKS cluster with its scanned Kubernetes rollup.
type Cluster struct {
ID string `json:"id"`
Name string `json:"name"`
Region string `json:"region"`
Version string `json:"version"`
Status string `json:"status"`
NodePools int `json:"nodePools"`
Pools []NodePool `json:"pools"`
Nodes int `json:"nodes"`
Pods int `json:"pods"`
PVs int `json:"pvs"`
PVCs int `json:"pvcs"`
IdlePVCs int `json:"idlePVCs"`
Scanned bool `json:"scanned"`
ScanError string `json:"scanError"`
MonthlyCents money.Cents `json:"monthlyCents"`
}
// NodePool is one DOKS node pool — the only correct place to change a cluster's node
// count. ClusterSchedulable is the whole cluster's schedulable-and-ready node count,
// carried on the row so the shrink verdict is a pure method of it.
type NodePool struct {
ID string `json:"id"`
Name string `json:"name"`
Size string `json:"size"`
Count int `json:"count"`
ClusterID string `json:"clusterId"`
Cluster string `json:"cluster"`
ClusterSchedulable int `json:"clusterSchedulable"`
Scalable bool `json:"scalable"`
BlockedReason string `json:"blockedReason"`
}
// ScaleTo answers whether this pool may be set to count.
//
// PROVEN HERE: a pool keeps at least one node, and a shrink never leaves the cluster
// with zero schedulable nodes — with none, every pod the removed nodes carried is
// unschedulable, guaranteed.
//
// NOT PROVEN, and deliberately not pretended: DOKS picks WHICH nodes it removes, so no
// particular pod can be shown to survive a shrink that leaves capacity behind. Node
// affinity, taints and resource requests decide that, and PodDisruptionBudgets are
// enforced by the cluster during DOKS's own drain — not by this board. A shrink that
// merely MIGHT not fit is allowed, and the response says so.
func (p NodePool) ScaleTo(count int) (bool, string) {
switch {
case !p.Scalable:
return false, p.BlockedReason
case count < 1:
return false, "A node pool must keep at least one node — scaling to zero destroys every node in the pool and strands its pods."
case count < p.Count && p.ClusterSchedulable-(p.Count-count) < 1:
return false, fmt.Sprintf("Removing %d node(s) would leave %s with no schedulable node.", p.Count-count, p.Cluster)
}
return true, ""
}
// Node is one droplet, joined to the Kubernetes node of the same name.
type Node struct {
ID int `json:"id"`
Name string `json:"name"`
Cluster string `json:"cluster"`
ClusterID string `json:"clusterId"`
Region string `json:"region"`
Status string `json:"status"`
SizeSlug string `json:"sizeSlug"`
VCPUs int `json:"vcpus"`
MemoryMiB int `json:"memoryMiB"`
LocalDiskGiB int `json:"localDiskGiB"`
MonthlyCents money.Cents `json:"monthlyCents"`
CreatedAt string `json:"createdAt"`
PrivateIP string `json:"privateIp"`
PublicIP string `json:"publicIp"`
Tags []string `json:"tags"`
Ready bool `json:"ready"`
Schedulable bool `json:"schedulable"`
Pods int `json:"pods"`
Volumes int `json:"volumes"`
// Mutable reports whether this droplet may be changed DIRECTLY — deleted or resized.
// One predicate covers both because one fact decides both: a DOKS node belongs to a
// node pool, and the pool is the only thing allowed to change it.
Mutable bool `json:"mutable"`
BlockedReason string `json:"blockedReason"`
}
// Volume is one block-storage volume with its PROVEN cluster ownership.
type Volume struct {
ID string `json:"id"`
Name string `json:"name"`
Region string `json:"region"`
SizeGiB int `json:"sizeGiB"`
MonthlyCents money.Cents `json:"monthlyCents"`
State string `json:"state"`
DropletIDs []int `json:"dropletIds"`
NodeName string `json:"nodeName"`
// Cluster/ClusterID are the PROVEN owner — resolved through a PV that names this
// volume, never through the tag.
Cluster string `json:"cluster"`
ClusterID string `json:"clusterId"`
// TagCluster is the `k8s:<uuid>` tag. ADVISORY ONLY: it outlives the cluster that
// set it. Shown so the operator can see tag-vs-truth disagree, never acted on.
TagCluster string `json:"tagCluster"`
PV string `json:"pv"`
PVPhase string `json:"pvPhase"`
PVCNamespace string `json:"pvcNamespace"`
PVCName string `json:"pvcName"`
MountedBy []string `json:"mountedBy"`
Idle bool `json:"idle"`
CreatedAt string `json:"createdAt"`
Deletable bool `json:"deletable"`
BlockedReason string `json:"blockedReason"`
}
// LoadBalancer is one DO load balancer, attributed to a cluster via the Service that
// claims it, or failing that via its member droplets.
type LoadBalancer struct {
ID string `json:"id"`
Name string `json:"name"`
Region string `json:"region"`
Status string `json:"status"`
IP string `json:"ip"`
SizeUnit int `json:"sizeUnit"`
MonthlyCents money.Cents `json:"monthlyCents"`
Droplets int `json:"droplets"`
Cluster string `json:"cluster"`
// Service is the `namespace/name` of the live type=LoadBalancer Service that claims
// this load balancer, proven from the cluster scan. Non-empty means IN USE.
Service string `json:"service"`
Deletable bool `json:"deletable"`
BlockedReason string `json:"blockedReason"`
}
// Finding is one audit result — the "is anything bad" surface.
type Finding struct {
ID string `json:"id"`
Severity string `json:"severity"`
Kind string `json:"kind"`
Title string `json:"title"`
Detail string `json:"detail"`
Resource string `json:"resource"`
Cluster string `json:"cluster"`
MonthlyCents money.Cents `json:"monthlyCents"`
}
// pvHit is a PV that claims a given DO volume, plus the cluster it lives in.
type pvHit struct {
cluster string
clusterID string
pv PVRef
}
// svcHit is a Service that claims a given DO load balancer, plus its cluster.
type svcHit struct {
cluster string
svc ServiceRef
}
// verdict is the ONE place a mutation's answer is decided: the completeness gate every
// mutation shares, then the resource's own rule (blocked == "" meaning its rule is
// satisfied). The gate comes first and applies to ALL of them — droplets, load balancers
// and node pools fail closed on a partial scan for the same reason volumes do: a fleet
// we cannot fully see is a fleet whose live parts we cannot fully name.
//
// Every (Deletable|Mutable|Scalable, BlockedReason) pair on this board comes from here,
// so "allowed carries no reason, blocked always names one" is stated once.
func (s *Snapshot) verdict(blocked string) (bool, string) {
switch {
case !s.Complete:
return false, s.IncompleteReason
case blocked != "":
return false, blocked
}
return true, ""
}
// Analyze folds the DO inventory and the per-cluster Kubernetes scans into the board.
// PURE: no clock, no network, no cluster — `at` is passed in so the result is
// byte-reproducible in tests.
func Analyze(inv Inventory, scans []ClusterScan, sources []core.SourceStatus, at time.Time) Snapshot {
snap := Snapshot{
At: at.UTC().Format(time.RFC3339),
Sources: sources,
}
if snap.Sources == nil {
snap.Sources = []core.SourceStatus{}
}
scanByID := make(map[string]ClusterScan, len(scans))
for _, s := range scans {
scanByID[s.ClusterID] = s
}
nameByID := make(map[string]string, len(inv.Clusters))
for _, c := range inv.Clusters {
nameByID[c.ID] = c.Name
}
// ---- completeness gate -------------------------------------------------------
// Every cluster must have answered. One silent gap and no volume may be condemned.
var unreachable []string
for _, c := range inv.Clusters {
s, ok := scanByID[c.ID]
if !ok || s.Err != nil {
unreachable = append(unreachable, c.Name)
}
}
switch {
case len(inv.Clusters) == 0:
snap.IncompleteReason = "DigitalOcean returned no clusters — the set of places a volume could be in use is unknown."
case len(unreachable) > 0:
snap.IncompleteReason = fmt.Sprintf(
"%d of %d clusters did not answer (%s) — a volume they hold would look unreferenced, so nothing is classified as deletable.",
len(unreachable), len(inv.Clusters), strings.Join(unreachable, ", "))
default:
snap.Complete = true
}
// ---- cross-cluster PV index --------------------------------------------------
// THE safety index: every volume handle claimed by any PV in any cluster.
byHandle := make(map[string]pvHit)
// mounted[clusterID/ns/pvc] -> pods currently mounting it.
mounted := make(map[string][]string)
// THE load-balancer safety index, keyed by BOTH identities a Service can claim one
// by — the DOKS load-balancer-id annotation and every address it holds. Either
// matching counts, because a broad match means MORE load balancers are treated as in
// use, which is the safe direction.
lbClaims := make(map[string]svcHit)
for _, s := range scans {
if s.Err != nil {
continue
}
cname := nameByID[s.ClusterID]
for _, pv := range s.PVs {
if h := strings.TrimSpace(pv.VolumeHandle); h != "" {
byHandle[h] = pvHit{cluster: cname, clusterID: s.ClusterID, pv: pv}
}
}
for _, p := range s.Pods {
for _, claim := range p.Claims {
k := claimKey(s.ClusterID, p.Namespace, claim)
mounted[k] = append(mounted[k], p.Namespace+"/"+p.Name)
}
}
for _, sv := range s.Services {
hit := svcHit{cluster: cname, svc: sv}
for _, key := range append([]string{sv.LBID}, sv.IPs...) {
if key = strings.TrimSpace(key); key != "" {
lbClaims[key] = hit
}
}
}
}
// ---- nodes -------------------------------------------------------------------
nodeByName := make(map[string]NodeState)
podsPerNode := make(map[string]int)
for _, s := range scans {
if s.Err != nil {
continue
}
for _, n := range s.Nodes {
nodeByName[n.Name] = n
}
for _, p := range s.Pods {
if p.Node != "" {
podsPerNode[p.Node]++
}
}
}
volsPerDroplet := make(map[int]int)
for _, v := range inv.Volumes {
for _, id := range v.DropletIDs {
volsPerDroplet[id]++
}
}
clusterByDroplet := make(map[int]string, len(inv.Droplets))
snap.Nodes = make([]Node, 0, len(inv.Droplets))
for _, d := range inv.Droplets {
cid := clusterIDFromTags(d.Tags)
clusterByDroplet[d.ID] = cid
ks := nodeByName[d.Name]
n := Node{
ID: d.ID, Name: d.Name, Cluster: nameByID[cid], ClusterID: cid,
Region: d.Region, Status: d.Status, SizeSlug: d.SizeSlug,
VCPUs: d.VCPUs, MemoryMiB: d.MemoryMiB, LocalDiskGiB: d.LocalDiskGiB,
MonthlyCents: d.MonthlyCents, CreatedAt: d.CreatedAt,
PrivateIP: d.PrivateIP, PublicIP: d.PublicIP, Tags: nonNilStrings(d.Tags),
Ready: ks.Ready, Schedulable: ks.Schedulable,
Pods: podsPerNode[d.Name], Volumes: volsPerDroplet[d.ID],
}
n.Mutable, n.BlockedReason = snap.verdict(nodeBlock(n))
snap.Nodes = append(snap.Nodes, n)
snap.Cost.DropletsMonthly += d.MonthlyCents
snap.Totals.LocalDiskGiB += d.LocalDiskGiB
}
dropletName := make(map[int]string, len(inv.Droplets))
for _, d := range inv.Droplets {
dropletName[d.ID] = d.Name
}
// ---- volumes: the state machine ----------------------------------------------
snap.Volumes = make([]Volume, 0, len(inv.Volumes))
for _, v := range inv.Volumes {
hit, referenced := byHandle[v.ID]
vol := Volume{
ID: v.ID, Name: v.Name, Region: v.Region, SizeGiB: v.SizeGiB,
MonthlyCents: money.Cents(v.SizeGiB) * volumeGiBCents,
DropletIDs: nonNilInts(v.DropletIDs),
TagCluster: nameByID[clusterIDFromTags(v.Tags)],
CreatedAt: v.CreatedAt,
MountedBy: []string{},
}
if len(v.DropletIDs) > 0 {
vol.NodeName = dropletName[v.DropletIDs[0]]
}
if referenced {
vol.Cluster, vol.ClusterID = hit.cluster, hit.clusterID
vol.PV, vol.PVPhase = hit.pv.Name, hit.pv.Phase
vol.PVCNamespace, vol.PVCName = hit.pv.ClaimNS, hit.pv.ClaimName
if hit.pv.ClaimName != "" {
vol.MountedBy = nonNilStrings(mounted[claimKey(hit.clusterID, hit.pv.ClaimNS, hit.pv.ClaimName)])
}
} else if len(v.DropletIDs) > 0 {
// No PV names it, but it is physically mounted on a node — that node's
// cluster owns it. Attachment is hard evidence, unlike the tag: it is the
// live kernel state, so the cost rolls up to the right cluster.
vol.ClusterID = clusterByDroplet[v.DropletIDs[0]]
vol.Cluster = nameByID[vol.ClusterID]
}
// Attachment beats reference; reference beats absence.
switch {
case len(v.DropletIDs) > 0:
vol.State = StateAttached
case referenced && strings.EqualFold(hit.pv.Phase, "Bound"):
vol.State = StateBound
case referenced:
vol.State = StateReleased
default:
vol.State = StateUnreferenced
}
// Idle is a REVIEW signal on live data, never a delete signal.
vol.Idle = vol.State == StateBound && len(vol.MountedBy) == 0
vol.Deletable, vol.BlockedReason = snap.verdict(volumeBlock(vol))
snap.Cost.VolumesMonthly += vol.MonthlyCents
snap.Totals.VolumeGiB += vol.SizeGiB
switch vol.State {
case StateAttached:
snap.Totals.AttachedVolumes++
snap.Totals.AttachedGiB += vol.SizeGiB
default:
snap.Totals.DetachedVolumes++
snap.Totals.DetachedGiB += vol.SizeGiB
}
if vol.State == StateUnreferenced {
snap.Totals.UnreferencedVolumes++
snap.Totals.UnreferencedGiB += vol.SizeGiB
// Reclaimable is exactly the unreferenced set — and only when the scan was
// complete enough to have earned that verdict.
if snap.Complete {
snap.Cost.ReclaimableMonthly += vol.MonthlyCents
}
}
if vol.Idle {
snap.Totals.IdlePVCs++
}
snap.Volumes = append(snap.Volumes, vol)
}
// ---- load balancers ----------------------------------------------------------
snap.LoadBalancers = make([]LoadBalancer, 0, len(inv.LoadBalancers))
for _, l := range inv.LoadBalancers {
lb := LoadBalancer{
ID: l.ID, Name: l.Name, Region: l.Region, Status: l.Status, IP: l.IP,
SizeUnit: l.SizeUnit, MonthlyCents: l.MonthlyCents, Droplets: len(l.DropletIDs),
}
// The claiming Service is the strongest attribution there is; member droplets are
// the fallback for a load balancer no scanned Service claims.
hit, claimed := lbClaims[l.ID]
if !claimed {
hit, claimed = lbClaims[l.IP]
}
if claimed {
lb.Service = hit.svc.Namespace + "/" + hit.svc.Name
lb.Cluster = hit.cluster
}
// A member droplet belonging to no cluster carries a workload Kubernetes knows
// nothing about, so the Service scan proves nothing about this load balancer.
unmanaged := 0
for _, id := range l.DropletIDs {
if cid := clusterByDroplet[id]; cid == "" {
unmanaged++
} else if lb.Cluster == "" {
lb.Cluster = nameByID[cid]
}
}
lb.Deletable, lb.BlockedReason = snap.verdict(lbBlock(lb, unmanaged))
snap.Cost.LoadBalancersMonthly += lb.MonthlyCents
snap.LoadBalancers = append(snap.LoadBalancers, lb)
}
// ---- cluster rollup ----------------------------------------------------------
idleByCluster := make(map[string]int)
costByCluster := make(map[string]money.Cents)
for _, v := range snap.Volumes {
if v.ClusterID != "" {
costByCluster[v.ClusterID] += v.MonthlyCents
if v.Idle {
idleByCluster[v.ClusterID]++
}
}
}
nodesByCluster := make(map[string]int)
for _, n := range snap.Nodes {
if n.ClusterID != "" {
nodesByCluster[n.ClusterID]++
costByCluster[n.ClusterID] += n.MonthlyCents
}
}
snap.Clusters = make([]Cluster, 0, len(inv.Clusters))
for _, c := range inv.Clusters {
row := Cluster{
ID: c.ID, Name: c.Name, Region: c.Region, Version: c.Version,
Status: c.Status, NodePools: len(c.Pools), Nodes: nodesByCluster[c.ID],
IdlePVCs: idleByCluster[c.ID], MonthlyCents: costByCluster[c.ID],
Pools: make([]NodePool, 0, len(c.Pools)),
}
if s, ok := scanByID[c.ID]; ok {
if s.Err != nil {
row.ScanError = s.Err.Error()
} else {
row.Scanned = true
row.Pods, row.PVs, row.PVCs = len(s.Pods), len(s.PVs), len(s.PVCs)
}
} else {
row.ScanError = "not scanned"
}
// A node that is cordoned or NotReady cannot take a pod, so it does not count as
// somewhere a shrink's evicted pods could land. Counting conservatively here
// refuses more shrinks, which is the safe direction.
schedulable := 0
for _, n := range scanByID[c.ID].Nodes {
if n.Ready && n.Schedulable {
schedulable++
}
}
for _, p := range c.Pools {
pool := NodePool{
ID: p.ID, Name: p.Name, Size: p.Size, Count: p.Count,
ClusterID: c.ID, Cluster: c.Name, ClusterSchedulable: schedulable,
}
// A pool carries no rule of its own — what may be refused depends on the
// COUNT asked for, which ScaleTo decides. Only the shared gate applies here.
pool.Scalable, pool.BlockedReason = snap.verdict("")
row.Pools = append(row.Pools, pool)
}
snap.Clusters = append(snap.Clusters, row)
}
snap.Totals.Clusters = len(snap.Clusters)
snap.Totals.Nodes = len(snap.Nodes)
snap.Totals.Volumes = len(snap.Volumes)
snap.Totals.LoadBalancers = len(snap.LoadBalancers)
snap.Cost.TotalMonthly = snap.Cost.DropletsMonthly + snap.Cost.VolumesMonthly + snap.Cost.LoadBalancersMonthly
snap.Findings = findings(snap, scans, nameByID)
return snap
}
// The per-resource rules. Each states, in the operator's language, exactly why ITS
// resource may not be mutated, and returns "" when its own rule is satisfied — the
// shared completeness gate in Snapshot.verdict has the final word either way.
// volumeBlock: only an unreferenced volume may be deleted.
func volumeBlock(v Volume) string {
switch v.State {
case StateUnreferenced:
return ""
case StateAttached:
if v.NodeName != "" {
return "Attached to " + v.NodeName + " and in use."
}
return "Attached to a droplet and in use."
case StateBound:
return fmt.Sprintf("Live data: PV %s is Bound to %s/%s in %s.", v.PV, v.PVCNamespace, v.PVCName, v.Cluster)
case StateReleased:
return fmt.Sprintf("PV %s in %s still references it (%s) — retire the PV first.", v.PV, v.Cluster, v.PVPhase)
}
return "Not eligible for deletion."
}
// nodeBlock: a DOKS node may not be deleted OR resized directly. The node pool owns it
// — DOKS recreates a node deleted out from under it (so the delete costs an outage and
// changes nothing) and reverts a hand-resized one to the pool's declared size. The pool
// is the only lever that actually holds.
func nodeBlock(n Node) string {
if n.ClusterID == "" {
return ""
}
name := n.Cluster
if name == "" {
name = n.ClusterID
}
return fmt.Sprintf("Node of DOKS cluster %s — DOKS owns it via its node pool and will recreate it. Scale or edit the pool instead.", name)
}
// lbBlock: a load balancer a live Service still targets may not be deleted — doing so
// black-holes that Service's public address, and DOKS recreates the load balancer
// anyway. Nor may one that forwards to droplets outside every cluster: the Service scan
// is the only liveness evidence this board has, and it says nothing about a workload
// Kubernetes does not run. "Has member droplets" is NOT itself a liveness signal — a
// DOKS load balancer lists every node in its cluster, so a leaked one still looks busy.
func lbBlock(lb LoadBalancer, unmanagedMembers int) string {
switch {
case lb.Service != "":
return fmt.Sprintf("Serving Kubernetes Service %s in %s — delete the Service first; DOKS recreates a load balancer its Service still wants.", lb.Service, lb.Cluster)
case unmanagedMembers > 0:
return fmt.Sprintf("Forwards to %d droplet(s) outside any cluster — no Kubernetes Service can vouch for it, so it cannot be proven unused.", unmanagedMembers)
}
return ""
}
// findings is the audit pass: what a human should look at, worst first.
func findings(s Snapshot, scans []ClusterScan, nameByID map[string]string) []Finding {
out := []Finding{}
if !s.Complete {
out = append(out, Finding{
ID: "scan-incomplete", Severity: SevCritical, Kind: "scan-incomplete",
Title: "Cluster scan incomplete — deletion disabled",
Detail: s.IncompleteReason,
})
}
for _, v := range s.Volumes {
switch {
case v.State == StateUnreferenced && s.Complete:
out = append(out, Finding{
ID: "unref/" + v.ID, Severity: SevWarn, Kind: "unreferenced-volume",
Title: fmt.Sprintf("Unreferenced volume %s (%d GiB)", v.Name, v.SizeGiB),
Detail: "No PersistentVolume in any cluster references this volume. " +
"Verified against every cluster, so it is safe to snapshot and delete.",
Resource: v.ID, MonthlyCents: v.MonthlyCents,
})
case v.State == StateReleased:
out = append(out, Finding{
ID: "released/" + v.ID, Severity: SevWarn, Kind: "released-pv",
Title: fmt.Sprintf("Released PV holding %s (%d GiB)", v.Name, v.SizeGiB),
Detail: fmt.Sprintf("PV %s is %s. Retire the PV to release the volume.", v.PV, v.PVPhase),
Resource: v.ID, Cluster: v.Cluster, MonthlyCents: v.MonthlyCents,
})
case v.Idle:
out = append(out, Finding{
ID: "idle/" + v.ID, Severity: SevInfo, Kind: "idle-pvc",
Title: fmt.Sprintf("Idle volume %s (%d GiB) — no pod mounts it", v.Name, v.SizeGiB),
Detail: fmt.Sprintf("PVC %s/%s is Bound but no running pod mounts it. "+
"REVIEW ONLY: this is live data (typically a stopped database), not garbage.",
v.PVCNamespace, v.PVCName),
Resource: v.ID, Cluster: v.Cluster, MonthlyCents: v.MonthlyCents,
})
}
}
// Unhealthy pods + unknown images, per cluster.
type imgSeen struct {
pods int
cluster string
}
unknown := map[string]*imgSeen{}
for _, sc := range scans {
if sc.Err != nil {
continue
}
cname := nameByID[sc.ClusterID]
for _, p := range sc.Pods {
if bad := podProblem(p); bad != "" {
out = append(out, Finding{
ID: "pod/" + sc.ClusterID + "/" + p.Namespace + "/" + p.Name, Severity: SevWarn,
Kind: "pod-unhealthy", Title: fmt.Sprintf("Pod %s/%s is %s", p.Namespace, p.Name, bad),
Detail: fmt.Sprintf("Phase %s%s on node %s.", p.Phase, reasonSuffix(p.Reason), p.Node),
Resource: p.Namespace + "/" + p.Name, Cluster: cname,
})
}
for _, img := range p.Images {
if knownImage(img) {
continue
}
repo := imageRepo(img)
if e, ok := unknown[repo]; ok {
e.pods++
} else {
unknown[repo] = &imgSeen{pods: 1, cluster: cname}
}
}
}
}
for repo, e := range unknown {
out = append(out, Finding{
ID: "image/" + repo, Severity: SevWarn, Kind: "unknown-image",
Title: "Unrecognised container image: " + repo,
Detail: fmt.Sprintf("Run by %d pod(s), from neither our registries nor the reviewed vendor set.", e.pods),
Resource: repo, Cluster: e.cluster,
})
}
// Cost outliers: any single resource at or above outlierShareBP of total spend.
if s.Cost.TotalMonthly > 0 {
threshold := s.Cost.TotalMonthly * outlierShareBP / 10000
for _, n := range s.Nodes {
if n.MonthlyCents >= threshold {
out = append(out, Finding{
ID: "cost/node/" + n.Name, Severity: SevInfo, Kind: "cost-outlier",
Title: fmt.Sprintf("Node %s is %s of fleet spend", n.Name, shareLabel(n.MonthlyCents, s.Cost.TotalMonthly)),
Detail: fmt.Sprintf("%s, %d vCPU / %d MiB.", n.SizeSlug, n.VCPUs, n.MemoryMiB),
Resource: n.Name, Cluster: n.Cluster, MonthlyCents: n.MonthlyCents,
})
}
}
for _, v := range s.Volumes {
if v.MonthlyCents >= threshold {
out = append(out, Finding{
ID: "cost/volume/" + v.ID, Severity: SevInfo, Kind: "cost-outlier",
Title: fmt.Sprintf("Volume %s is %s of fleet spend", v.Name, shareLabel(v.MonthlyCents, s.Cost.TotalMonthly)),
Detail: fmt.Sprintf("%d GiB, %s.", v.SizeGiB, v.State),
Resource: v.ID, Cluster: v.Cluster, MonthlyCents: v.MonthlyCents,
})
}
}
}
rank := map[string]int{SevCritical: 0, SevWarn: 1, SevInfo: 2}
sort.SliceStable(out, func(i, j int) bool {
if rank[out[i].Severity] != rank[out[j].Severity] {
return rank[out[i].Severity] < rank[out[j].Severity]
}
if out[i].MonthlyCents != out[j].MonthlyCents {
return out[i].MonthlyCents > out[j].MonthlyCents
}
return out[i].ID < out[j].ID
})
return out
}
// podProblem names the failure a pod is in, or "" when it is fine.
func podProblem(p PodRef) string {
switch {
case strings.EqualFold(p.Reason, "Evicted"):
return "Evicted"
case strings.EqualFold(p.Phase, "Failed"):
return "Failed"
case strings.Contains(p.Reason, "CrashLoopBackOff"):
return "CrashLoopBackOff"
case strings.Contains(p.Reason, "ImagePullBackOff"), strings.Contains(p.Reason, "ErrImagePull"):
return "ImagePullBackOff"
}
return ""
}
func reasonSuffix(r string) string {
if strings.TrimSpace(r) == "" {
return ""
}
return " (" + r + ")"
}
// knownImage reports whether an image comes from our registries or the reviewed
// vendor set.
func knownImage(img string) bool {
l := strings.ToLower(strings.TrimSpace(img))
l = strings.TrimPrefix(l, "docker.io/")
for _, p := range firstParty {
if strings.HasPrefix(l, strings.TrimPrefix(p, "docker.io/")) {
return true
}
}
for _, p := range knownVendors {
if strings.HasPrefix(l, strings.TrimPrefix(p, "docker.io/")) {
return true
}
}
// A bare `name:tag` with no slash is an official Docker Hub library image.
return !strings.Contains(strings.SplitN(l, ":", 2)[0], "/")
}
// imageRepo strips the tag/digest so findings group by repository, not by build.
func imageRepo(img string) string {
s := strings.TrimSpace(img)
if i := strings.Index(s, "@"); i > 0 {
s = s[:i]
}
if i := strings.LastIndex(s, ":"); i > strings.LastIndex(s, "/") {
s = s[:i]
}
return s
}
// shareLabel renders a cents-of-total share as a percentage.
func shareLabel(part, total money.Cents) string {
if total <= 0 {
return "0%"
}
return fmt.Sprintf("%.1f%%", float64(part)*100/float64(total))
}
// clusterIDFromTags extracts the DOKS cluster UUID from a `k8s:<uuid>` resource tag.
// On droplets this is authoritative (DOKS owns the droplet); on VOLUMES it is
// advisory only — see the Volume.TagCluster doc.
func clusterIDFromTags(tags []string) string {
for _, t := range tags {
v := strings.TrimPrefix(t, "k8s:")
if v == t || v == "" {
continue
}
// Cluster tags are UUIDs; DOKS also stamps role tags like `k8s:worker`.
if len(v) == 36 && strings.Count(v, "-") == 4 {
return v
}
}
return ""
}
func claimKey(clusterID, ns, name string) string { return clusterID + "/" + ns + "/" + name }
func nonNilStrings(s []string) []string {
if s == nil {
return []string{}
}
return s
}
func nonNilInts(s []int) []int {
if s == nil {
return []int{}
}
return s
}
+610
View File
@@ -0,0 +1,610 @@
package infra
import (
"errors"
"fmt"
"strings"
"testing"
"time"
"github.com/hanzoai/cloud/clients/admin/digitalocean"
)
// Two live clusters. The near-miss that motivated this package involved volumes tagged
// for one cluster whose PVs actually live in another, so every fixture here has two.
const (
cidA = "aaaaaaaa-1111-2222-3333-444444444444"
cidB = "bbbbbbbb-1111-2222-3333-444444444444"
)
func baseInventory() Inventory {
return Inventory{
Clusters: []digitalocean.Cluster{
{ID: cidA, Name: "hanzo-k8s", Region: "sfo3", Status: "running",
Pools: []digitalocean.NodePool{{ID: "pool-a", Name: "workers", Size: "s-8vcpu-16gb-amd", Count: 3}}},
{ID: cidB, Name: "lux-k8s", Region: "sfo3", Status: "running",
Pools: []digitalocean.NodePool{{ID: "pool-b", Name: "workers", Size: "s-2vcpu-4gb", Count: 1}}},
},
Droplets: []digitalocean.Droplet{{
ID: 101, Name: "node-a1", Region: "sfo3", Status: "active",
SizeSlug: "s-8vcpu-16gb-amd", VCPUs: 8, MemoryMiB: 16384,
LocalDiskGiB: 320, MonthlyCents: 11200,
Tags: []string{"k8s", "k8s:" + cidA, "k8s:worker"},
}},
}
}
func scansOK() []ClusterScan {
return []ClusterScan{{ClusterID: cidA}, {ClusterID: cidB}}
}
func volByID(t *testing.T, s Snapshot, id string) Volume {
t.Helper()
v, ok := findVolume(s, id)
if !ok {
t.Fatalf("volume %s missing from snapshot", id)
}
return v
}
func analyze(inv Inventory, scans []ClusterScan) Snapshot {
return Analyze(inv, scans, nil, time.Unix(0, 0).UTC())
}
// TestCrossClusterPVProtectsMisTaggedVolume is THE regression test. A detached volume
// tagged `k8s:<cluster A>` whose PV actually lives in cluster B must be classified from
// the PV, not the tag. Trusting the tag here is what nearly destroyed 4.39 TiB.
func TestCrossClusterPVProtectsMisTaggedVolume(t *testing.T) {
inv := baseInventory()
inv.Volumes = []digitalocean.Volume{{
ID: "vol-mistagged", Name: "pvc-neon-pageserver", Region: "sfo3", SizeGiB: 50,
DropletIDs: nil, // detached
Tags: []string{"k8s:" + cidA}, // tag says cluster A …
}}
scans := scansOK()
// … but the PV that owns it lives in cluster B, and is Bound to a live PVC.
scans[1].PVs = []PVRef{{
Name: "pv-neon", Phase: "Bound", VolumeHandle: "vol-mistagged",
ClaimNS: "neon", ClaimName: "pageserver-data",
}}
got := analyze(inv, scans)
v := volByID(t, got, "vol-mistagged")
if v.State != StateBound {
t.Fatalf("state = %q, want %q — a detached, mis-tagged volume with a Bound PV is LIVE DATA", v.State, StateBound)
}
if v.Deletable {
t.Fatal("volume marked deletable: this is the 4.39 TiB data-loss bug")
}
if v.Cluster != "lux-k8s" {
t.Errorf("proven cluster = %q, want lux-k8s (from the PV, not the tag)", v.Cluster)
}
if v.TagCluster != "hanzo-k8s" {
t.Errorf("tagCluster = %q, want hanzo-k8s (advisory, surfaced so tag-vs-truth is visible)", v.TagCluster)
}
if got.Cost.ReclaimableMonthly != 0 {
t.Errorf("reclaimable = %d, want 0", got.Cost.ReclaimableMonthly)
}
}
// TestIncompleteScanBlocksEveryDeletion: one unreachable cluster and NOTHING is
// deletable, even a volume no reachable cluster references. Absence of evidence is not
// evidence of absence.
func TestIncompleteScanBlocksEveryDeletion(t *testing.T) {
inv := baseInventory()
inv.Volumes = []digitalocean.Volume{{ID: "vol-orphan", Name: "stray", SizeGiB: 100}}
scans := scansOK()
scans[1].Err = errors.New("dial tcp: i/o timeout")
got := analyze(inv, scans)
if got.Complete {
t.Fatal("Complete = true with an unreachable cluster")
}
v := volByID(t, got, "vol-orphan")
if v.State != StateUnreferenced {
t.Errorf("state = %q, want %q (state is observable; the VERDICT is what is withheld)", v.State, StateUnreferenced)
}
if v.Deletable {
t.Fatal("deletable with an incomplete scan — fail-closed violated")
}
if got.Cost.ReclaimableMonthly != 0 {
t.Errorf("reclaimable = %d, want 0 when the scan is incomplete", got.Cost.ReclaimableMonthly)
}
if v.BlockedReason == "" || !strings.Contains(v.BlockedReason, "lux-k8s") {
t.Errorf("blockedReason = %q, want it to name the unreachable cluster", v.BlockedReason)
}
if got.Findings[0].Kind != "scan-incomplete" || got.Findings[0].Severity != SevCritical {
t.Errorf("first finding = %+v, want a critical scan-incomplete", got.Findings[0])
}
}
// TestNoClustersIsIncomplete: an empty cluster list means the set of places a volume
// could be in use is unknown, which must not read as "referenced by nothing".
func TestNoClustersIsIncomplete(t *testing.T) {
inv := Inventory{Volumes: []digitalocean.Volume{{ID: "v1", Name: "x", SizeGiB: 10}}}
got := analyze(inv, nil)
if got.Complete {
t.Fatal("Complete = true with zero clusters")
}
if volByID(t, got, "v1").Deletable {
t.Fatal("deletable with zero clusters known")
}
}
// TestVolumeStateMachine covers the full ordering: attachment beats reference,
// reference beats absence, and only the unreferenced volume is ever deletable.
func TestVolumeStateMachine(t *testing.T) {
inv := baseInventory()
inv.Volumes = []digitalocean.Volume{
{ID: "v-attached", Name: "attached", SizeGiB: 10, DropletIDs: []int{101}, Tags: []string{"k8s:" + cidA}},
{ID: "v-bound", Name: "bound", SizeGiB: 20},
{ID: "v-released", Name: "released", SizeGiB: 30},
{ID: "v-unref", Name: "unref", SizeGiB: 40},
// Attached AND referenced by a Bound PV: attachment wins.
{ID: "v-both", Name: "both", SizeGiB: 50, DropletIDs: []int{101}},
}
scans := scansOK()
scans[0].PVs = []PVRef{
{Name: "pv-bound", Phase: "Bound", VolumeHandle: "v-bound", ClaimNS: "ns", ClaimName: "c1"},
{Name: "pv-rel", Phase: "Released", VolumeHandle: "v-released", ClaimNS: "ns", ClaimName: "c2"},
{Name: "pv-both", Phase: "Bound", VolumeHandle: "v-both", ClaimNS: "ns", ClaimName: "c3"},
}
scans[0].Pods = []PodRef{{Namespace: "ns", Name: "p1", Node: "node-a1", Claims: []string{"c1", "c3"}}}
got := analyze(inv, scans)
if !got.Complete {
t.Fatalf("Complete = false, want true: %s", got.IncompleteReason)
}
for _, tc := range []struct {
id, want string
deletable bool
}{
{"v-attached", StateAttached, false},
{"v-bound", StateBound, false},
{"v-released", StateReleased, false},
{"v-unref", StateUnreferenced, true},
{"v-both", StateAttached, false},
} {
v := volByID(t, got, tc.id)
if v.State != tc.want {
t.Errorf("%s: state = %q, want %q", tc.id, v.State, tc.want)
}
if v.Deletable != tc.deletable {
t.Errorf("%s: deletable = %v, want %v (reason %q)", tc.id, v.Deletable, tc.deletable, v.BlockedReason)
}
if !v.Deletable && v.BlockedReason == "" {
t.Errorf("%s: not deletable but no reason given", tc.id)
}
if v.Deletable && v.BlockedReason != "" {
t.Errorf("%s: deletable but carries reason %q", tc.id, v.BlockedReason)
}
}
// Exactly one volume is reclaimable: 40 GiB × $0.10 = $4.00.
if got.Cost.ReclaimableMonthly != 400 {
t.Errorf("reclaimable = %d cents, want 400", got.Cost.ReclaimableMonthly)
}
// v-bound is mounted by p1; v-both is mounted by p1 too — neither is idle.
if got.Totals.IdlePVCs != 0 {
t.Errorf("idlePVCs = %d, want 0", got.Totals.IdlePVCs)
}
}
// TestIdleIsReviewNotReclaimable: a Bound volume no pod mounts is flagged for review
// but is never deletable and never counted as money we can get back.
func TestIdleIsReviewNotReclaimable(t *testing.T) {
inv := baseInventory()
inv.Volumes = []digitalocean.Volume{{ID: "v-idle", Name: "registry-data", SizeGiB: 50}}
scans := scansOK()
scans[1].PVs = []PVRef{{Name: "pv-idle", Phase: "Bound", VolumeHandle: "v-idle", ClaimNS: "registry", ClaimName: "registry-data"}}
// No pod mounts registry-data.
scans[1].Pods = []PodRef{{Namespace: "other", Name: "unrelated", Claims: []string{"something-else"}}}
got := analyze(inv, scans)
v := volByID(t, got, "v-idle")
if !v.Idle {
t.Fatal("idle = false, want true (Bound but unmounted)")
}
if v.Deletable {
t.Fatal("an idle volume must never be deletable — it is a stopped database, not garbage")
}
if got.Cost.ReclaimableMonthly != 0 {
t.Errorf("reclaimable = %d, want 0: idle capacity is not reclaimable", got.Cost.ReclaimableMonthly)
}
if got.Totals.IdlePVCs != 1 {
t.Errorf("idlePVCs = %d, want 1", got.Totals.IdlePVCs)
}
var f *Finding
for i := range got.Findings {
if got.Findings[i].Kind == "idle-pvc" {
f = &got.Findings[i]
}
}
if f == nil {
t.Fatal("no idle-pvc finding")
}
if f.Severity != SevInfo {
t.Errorf("idle-pvc severity = %q, want info (a review queue, not an alarm)", f.Severity)
}
if !strings.Contains(f.Detail, "REVIEW ONLY") {
t.Errorf("idle-pvc detail must say it is review-only, got %q", f.Detail)
}
}
// TestLegacyFlexVolumePVProtects: a pre-CSI PV still shields its volume.
func TestLegacyFlexVolumePVProtects(t *testing.T) {
inv := baseInventory()
inv.Volumes = []digitalocean.Volume{{ID: "v-flex", Name: "legacy", SizeGiB: 10}}
scans := scansOK()
scans[0].PVs = []PVRef{{Name: "pv-flex", Phase: "Bound", VolumeHandle: "v-flex", ClaimNS: "old", ClaimName: "data"}}
if volByID(t, analyze(inv, scans), "v-flex").Deletable {
t.Fatal("a flexVolume-referenced volume must not be deletable")
}
}
// TestCostMathAndLocalDiskSeparation: block storage is billed per GiB; droplet local
// disk is included in the droplet price and must never be added to storage cost.
func TestCostMathAndLocalDiskSeparation(t *testing.T) {
inv := baseInventory()
inv.Volumes = []digitalocean.Volume{
{ID: "a", Name: "a", SizeGiB: 200},
{ID: "b", Name: "b", SizeGiB: 300},
}
inv.LoadBalancers = []digitalocean.LoadBalancer{
{ID: "lb1", Name: "ingress", SizeUnit: 1, MonthlyCents: 1200, DropletIDs: []int{101}},
}
got := analyze(inv, scansOK())
if got.Cost.VolumesMonthly != 5000 {
t.Errorf("volumes = %d cents, want 5000 (500 GiB × $0.10)", got.Cost.VolumesMonthly)
}
if got.Cost.DropletsMonthly != 11200 {
t.Errorf("droplets = %d cents, want 11200", got.Cost.DropletsMonthly)
}
if got.Cost.LoadBalancersMonthly != 1200 {
t.Errorf("load balancers = %d cents, want 1200", got.Cost.LoadBalancersMonthly)
}
if got.Cost.TotalMonthly != 5000+11200+1200 {
t.Errorf("total = %d cents, want %d", got.Cost.TotalMonthly, 5000+11200+1200)
}
// The 320 GiB of local disk is reported, but is NOT in any cost line.
if got.Totals.LocalDiskGiB != 320 {
t.Errorf("localDiskGiB = %d, want 320", got.Totals.LocalDiskGiB)
}
if got.Totals.VolumeGiB != 500 {
t.Errorf("volumeGiB = %d, want 500 — local disk must never be folded into block storage", got.Totals.VolumeGiB)
}
if got.LoadBalancers[0].Cluster != "hanzo-k8s" {
t.Errorf("lb cluster = %q, want hanzo-k8s (attributed via its member droplet)", got.LoadBalancers[0].Cluster)
}
}
// TestNodeJoinAndClusterRollup: droplets join their Kubernetes node by name and roll
// up into the owning cluster.
func TestNodeJoinAndClusterRollup(t *testing.T) {
inv := baseInventory()
inv.Volumes = []digitalocean.Volume{{ID: "v1", Name: "v1", SizeGiB: 10, DropletIDs: []int{101}}}
scans := scansOK()
scans[0].Nodes = []NodeState{{Name: "node-a1", Ready: true, Schedulable: false}}
scans[0].Pods = []PodRef{
{Namespace: "hanzo", Name: "p1", Node: "node-a1"},
{Namespace: "hanzo", Name: "p2", Node: "node-a1"},
}
got := analyze(inv, scans)
n := got.Nodes[0]
if n.Cluster != "hanzo-k8s" || n.ClusterID != cidA {
t.Errorf("node cluster = %q/%q, want hanzo-k8s", n.Cluster, n.ClusterID)
}
if !n.Ready || n.Schedulable {
t.Errorf("node ready=%v schedulable=%v, want ready + cordoned", n.Ready, n.Schedulable)
}
if n.Pods != 2 || n.Volumes != 1 {
t.Errorf("node pods=%d volumes=%d, want 2/1", n.Pods, n.Volumes)
}
var ca Cluster
for _, c := range got.Clusters {
if c.ID == cidA {
ca = c
}
}
if ca.Nodes != 1 || ca.Pods != 2 || !ca.Scanned {
t.Errorf("cluster A rollup = %+v, want 1 node / 2 pods / scanned", ca)
}
// Node $112.00 + its 10 GiB volume $1.00.
if ca.MonthlyCents != 11200+100 {
t.Errorf("cluster monthly = %d, want 11300", ca.MonthlyCents)
}
}
// TestUnknownImageDetection: our registries and the reviewed vendor set stay quiet;
// anything else is reported once per repository.
func TestUnknownImageDetection(t *testing.T) {
inv := baseInventory()
scans := scansOK()
scans[0].Pods = []PodRef{{Namespace: "hanzo", Name: "p1", Images: []string{
"ghcr.io/hanzoai/cloud:v1.801.218",
"ghcr.io/luxfi/node:v1.2.3",
"registry.k8s.io/pause:3.9",
"grafana/grafana:11.0.0",
"acmglobaltech/thing:1",
"redis:7", // bare official library image
"evil.example.com/miner:latest", // ← the only one that should be reported
}}}
got := analyze(inv, scans)
var unknown []string
for _, f := range got.Findings {
if f.Kind == "unknown-image" {
unknown = append(unknown, f.Resource)
}
}
if len(unknown) != 1 || unknown[0] != "evil.example.com/miner" {
t.Fatalf("unknown images = %v, want exactly [evil.example.com/miner]", unknown)
}
}
// TestUnhealthyPodFindings covers the pod states the audit surfaces.
func TestUnhealthyPodFindings(t *testing.T) {
inv := baseInventory()
scans := scansOK()
scans[0].Pods = []PodRef{
{Namespace: "a", Name: "ok", Phase: "Running"},
{Namespace: "a", Name: "gone", Phase: "Failed", Reason: "Evicted"},
{Namespace: "a", Name: "loop", Phase: "Pending", Reason: "CrashLoopBackOff"},
{Namespace: "a", Name: "pull", Phase: "Pending", Reason: "ImagePullBackOff"},
}
got := analyze(inv, scans)
seen := map[string]bool{}
for _, f := range got.Findings {
if f.Kind == "pod-unhealthy" {
seen[f.Resource] = true
}
}
if len(seen) != 3 || !seen["a/gone"] || !seen["a/loop"] || !seen["a/pull"] {
t.Fatalf("unhealthy pods = %v, want gone/loop/pull and not ok", seen)
}
}
// TestClusterTagParsing: only a UUID-shaped k8s: tag is a cluster id; role tags are not.
func TestClusterTagParsing(t *testing.T) {
for _, tc := range []struct {
tags []string
want string
}{
{[]string{"k8s", "k8s:" + cidA, "k8s:worker"}, cidA},
{[]string{"k8s", "k8s:worker"}, ""},
{[]string{"unrelated"}, ""},
{nil, ""},
} {
if got := clusterIDFromTags(tc.tags); got != tc.want {
t.Errorf("clusterIDFromTags(%v) = %q, want %q", tc.tags, got, tc.want)
}
}
}
// TestDropletMutationRefusesClusterNode: a droplet DOKS owns may be neither deleted nor
// resized. Deleting one is worse than useless — the node pool recreates it, so the
// operator pays an outage for no change.
func TestDropletMutationRefusesClusterNode(t *testing.T) {
inv := baseInventory()
inv.Droplets = append(inv.Droplets,
// A plain droplet: no k8s tag, so nothing manages it but us.
digitalocean.Droplet{ID: 201, Name: "bastion", Region: "sfo3", Status: "active",
SizeSlug: "s-1vcpu-1gb", MonthlyCents: 600, Tags: []string{"ops"}},
// A droplet carrying only DOKS's ROLE tag — that is not a cluster id, so it is
// not evidence of DOKS ownership and must not block the mutation.
digitalocean.Droplet{ID: 202, Name: "roletag-only", Region: "sfo3", Status: "active",
SizeSlug: "s-1vcpu-1gb", MonthlyCents: 600, Tags: []string{"k8s", "k8s:worker"}},
)
got := analyze(inv, scansOK())
for _, tc := range []struct {
id int
mutable bool
reason string
}{
{101, false, "hanzo-k8s"}, // baseInventory's node-a1, tagged k8s:<cidA>
{201, true, ""},
{202, true, ""},
} {
n, ok := findNode(got, tc.id)
if !ok {
t.Fatalf("droplet %d missing from snapshot", tc.id)
}
if n.Mutable != tc.mutable {
t.Errorf("%s: mutable = %v, want %v (reason %q)", n.Name, n.Mutable, tc.mutable, n.BlockedReason)
}
if !tc.mutable && !strings.Contains(n.BlockedReason, tc.reason) {
t.Errorf("%s: blockedReason = %q, want it to name %q", n.Name, n.BlockedReason, tc.reason)
}
if tc.mutable && n.BlockedReason != "" {
t.Errorf("%s: mutable but carries reason %q", n.Name, n.BlockedReason)
}
}
}
// TestLoadBalancerDeleteRules: a load balancer a Service still targets is live
// infrastructure. DO's API gives the load balancer no back-reference to its Service, so
// the ONLY sound test is the cross-cluster Service scan — the same shape, and the same
// cross-cluster trap, as the volume/PV rule. Membership is the trap in the other
// direction: a leaked DOKS load balancer still lists every cluster node, so members
// must not imply life — except for members no cluster owns, which nothing can vouch for.
func TestLoadBalancerDeleteRules(t *testing.T) {
inv := baseInventory()
// A droplet outside every cluster, so a load balancer pointed at it is a workload
// no Service can speak for.
inv.Droplets = append(inv.Droplets, digitalocean.Droplet{
ID: 201, Name: "bastion", Region: "sfo3", Status: "active", SizeSlug: "s-1vcpu-1gb",
Tags: []string{"ops"},
})
inv.LoadBalancers = []digitalocean.LoadBalancer{
{ID: "lb-annotated", Name: "by-annotation", IP: "10.0.0.1", SizeUnit: 1, MonthlyCents: 1200},
{ID: "lb-by-ip", Name: "by-address", IP: "10.0.0.2", SizeUnit: 1, MonthlyCents: 1200},
{ID: "lb-crosscluster", Name: "other-cluster", IP: "10.0.0.3", SizeUnit: 1, MonthlyCents: 1200},
{ID: "lb-orphan", Name: "stray", IP: "10.0.0.4", SizeUnit: 1, MonthlyCents: 1200},
// Every member is a cluster node and no Service claims it: a leaked DOKS load
// balancer. Members alone must NOT keep it alive, or nothing is ever reclaimable.
{ID: "lb-leaked", Name: "leaked", IP: "10.0.0.5", SizeUnit: 1, MonthlyCents: 1200, DropletIDs: []int{101}},
{ID: "lb-unmanaged", Name: "legacy", IP: "10.0.0.6", SizeUnit: 1, MonthlyCents: 1200, DropletIDs: []int{201}},
}
scans := scansOK()
scans[0].Services = []ServiceRef{
// Annotation only: DOKS has stamped the id but no address is published yet.
{Namespace: "ingress", Name: "gateway", LBID: "lb-annotated"},
// Address only: a hand-made Service, or one whose annotation was stripped.
{Namespace: "web", Name: "site", IPs: []string{"10.0.0.2"}},
}
// The claim lives in the OTHER cluster — matching only within one cluster would
// condemn this load balancer.
scans[1].Services = []ServiceRef{{Namespace: "lux", Name: "rpc", LBID: "lb-crosscluster"}}
got := analyze(inv, scans)
for _, tc := range []struct {
id, service, cluster, reason string
deletable bool
}{
{"lb-annotated", "ingress/gateway", "hanzo-k8s", "ingress/gateway", false},
{"lb-by-ip", "web/site", "hanzo-k8s", "web/site", false},
{"lb-crosscluster", "lux/rpc", "lux-k8s", "lux/rpc", false},
{"lb-orphan", "", "", "", true},
{"lb-leaked", "", "hanzo-k8s", "", true},
{"lb-unmanaged", "", "", "outside any cluster", false},
} {
lb, ok := findLoadBalancer(got, tc.id)
if !ok {
t.Fatalf("load balancer %s missing from snapshot", tc.id)
}
if lb.Service != tc.service {
t.Errorf("%s: service = %q, want %q", tc.id, lb.Service, tc.service)
}
if lb.Cluster != tc.cluster {
t.Errorf("%s: cluster = %q, want %q", tc.id, lb.Cluster, tc.cluster)
}
if lb.Deletable != tc.deletable {
t.Errorf("%s: deletable = %v, want %v (reason %q)", tc.id, lb.Deletable, tc.deletable, lb.BlockedReason)
}
if !lb.Deletable && !strings.Contains(lb.BlockedReason, tc.reason) {
t.Errorf("%s: blockedReason = %q, want it to mention %q", tc.id, lb.BlockedReason, tc.reason)
}
if lb.Deletable && lb.BlockedReason != "" {
t.Errorf("%s: deletable but carries reason %q", tc.id, lb.BlockedReason)
}
}
}
// TestNodePoolScaleRules covers every count the scale route may be asked for. Note the
// two nodes each fixture adds that CANNOT take a pod — one cordoned, one NotReady:
// neither counts toward the capacity a shrink is allowed to leave behind.
func TestNodePoolScaleRules(t *testing.T) {
for _, tc := range []struct {
name string
poolCount int
schedulable int
to int
allowed bool
reason string
}{
{"grow", 3, 3, 5, true, ""},
{"unchanged", 3, 3, 3, true, ""},
{"shrink leaving capacity", 3, 3, 2, true, ""},
{"shrink to the last node", 2, 2, 1, true, ""},
{"scale to zero", 3, 3, 0, false, "at least one node"},
{"negative count", 3, 3, -1, false, "at least one node"},
// 3 → 1 removes two nodes, but only two could ever take a pod, so the cluster is
// left with none. The third pool node is cordoned and never counted.
{"shrink past real capacity", 3, 2, 1, false, "no schedulable node"},
{"shrink when everything is cordoned", 2, 0, 1, false, "no schedulable node"},
} {
t.Run(tc.name, func(t *testing.T) {
inv := baseInventory()
inv.Clusters[0].Pools = []digitalocean.NodePool{
{ID: "pool-a", Name: "workers", Size: "s-8vcpu-16gb-amd", Count: tc.poolCount},
}
scans := scansOK()
for i := 0; i < tc.schedulable; i++ {
scans[0].Nodes = append(scans[0].Nodes, NodeState{
Name: fmt.Sprintf("node-a%d", i), Ready: true, Schedulable: true,
})
}
scans[0].Nodes = append(scans[0].Nodes,
NodeState{Name: "cordoned", Ready: true, Schedulable: false},
NodeState{Name: "broken", Ready: false, Schedulable: true},
)
p, ok := findNodePool(analyze(inv, scans), cidA, "workers")
if !ok {
t.Fatal("node pool missing from snapshot")
}
if p.ClusterSchedulable != tc.schedulable {
t.Errorf("clusterSchedulable = %d, want %d — only ready AND schedulable nodes count",
p.ClusterSchedulable, tc.schedulable)
}
allowed, reason := p.ScaleTo(tc.to)
if allowed != tc.allowed {
t.Fatalf("ScaleTo(%d) = %v (%q), want %v", tc.to, allowed, reason, tc.allowed)
}
if allowed && reason != "" {
t.Errorf("allowed but carries reason %q", reason)
}
if !allowed && !strings.Contains(reason, tc.reason) {
t.Errorf("reason = %q, want it to contain %q", reason, tc.reason)
}
})
}
}
// TestIncompleteScanBlocksEveryMutation is the fail-closed rule generalised: one
// unreachable cluster and the board authorises NOTHING — not a volume delete, not a
// droplet delete or resize, not a load-balancer delete, not a pool scale. Every refusal
// gives the same reason, because it is the same reason.
func TestIncompleteScanBlocksEveryMutation(t *testing.T) {
inv := baseInventory()
inv.Volumes = []digitalocean.Volume{{ID: "vol-orphan", Name: "stray", SizeGiB: 100}}
inv.LoadBalancers = []digitalocean.LoadBalancer{{ID: "lb-orphan", Name: "stray", IP: "10.0.0.9", SizeUnit: 1}}
inv.Droplets = append(inv.Droplets, digitalocean.Droplet{
ID: 201, Name: "bastion", Region: "sfo3", Status: "active", SizeSlug: "s-1vcpu-1gb", Tags: []string{"ops"},
})
scans := scansOK()
scans[0].Nodes = []NodeState{{Name: "n1", Ready: true, Schedulable: true}}
scans[1].Err = errors.New("dial tcp: i/o timeout")
got := analyze(inv, scans)
if got.Complete {
t.Fatal("Complete = true with an unreachable cluster")
}
v := volByID(t, got, "vol-orphan")
n, _ := findNode(got, 201)
lb, _ := findLoadBalancer(got, "lb-orphan")
pool, _ := findNodePool(got, cidA, "workers")
scaleAllowed, scaleReason := pool.ScaleTo(2)
for _, tc := range []struct {
what string
allowed bool
reason string
}{
{"volume delete", v.Deletable, v.BlockedReason},
{"droplet delete/resize", n.Mutable, n.BlockedReason},
{"load balancer delete", lb.Deletable, lb.BlockedReason},
{"node pool scale", pool.Scalable, pool.BlockedReason},
{"node pool scale to a safe count", scaleAllowed, scaleReason},
} {
if tc.allowed {
t.Errorf("%s allowed with an incomplete scan — fail-closed violated", tc.what)
}
if tc.reason != got.IncompleteReason {
t.Errorf("%s reason = %q, want the scan's own reason %q", tc.what, tc.reason, got.IncompleteReason)
}
}
}
// TestJSONArraysNeverNull: the console renders these directly; a null array is a crash.
func TestJSONArraysNeverNull(t *testing.T) {
got := analyze(Inventory{}, nil)
if got.Volumes == nil || got.Nodes == nil || got.Clusters == nil ||
got.LoadBalancers == nil || got.Findings == nil || got.Sources == nil {
t.Fatalf("empty snapshot has nil slices: %+v", got)
}
}
+452
View File
@@ -0,0 +1,452 @@
package infra
import (
"context"
"fmt"
"strconv"
"strings"
"sync"
"time"
"github.com/zap-proto/zip"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/audit"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/admin/digitalocean"
)
// cacheTTL bounds how stale a READ may be. It exists because one board is a fan-out
// over the DO API plus every cluster's full pod/PV listing — not because staleness is
// acceptable when it matters: every MUTATION re-scans from scratch, ignoring this.
const cacheTTL = 60 * time.Second
// board holds the one cached snapshot behind /v1/admin/infra.
type board struct {
mu sync.Mutex
snap Snapshot
at time.Time
}
// Routes registers the DigitalOcean infrastructure board. SuperAdmin only: this is
// the whole account's physical inventory and the controls that destroy parts of it.
//
// NOTE ON THE NOUN: this is INFRASTRUCTURE — droplets, volumes, DOKS clusters, load
// balancers. The pre-existing /v1/fleet surface is compute workers and jobs. Different
// nouns, deliberately not merged.
func Routes(app cloud.Router, s *cloud.Service[core.State]) {
b := &board{}
g := app.Group("/v1/admin")
g.Get("/infra", core.Guard(s, b.read))
g.Post("/infra/volumes/:id/snapshot", core.Guard(s, b.snapshotVolume))
g.Delete("/infra/volumes/:id", core.Guard(s, b.deleteVolume))
g.Post("/infra/nodes/:id/cordon", core.Guard(s, b.cordonNode))
g.Delete("/infra/droplets/:id", core.Guard(s, b.deleteDroplet))
g.Post("/infra/droplets/:id/resize", core.Guard(s, b.resizeDroplet))
g.Delete("/infra/loadbalancers/:id", core.Guard(s, b.deleteLoadBalancer))
g.Post("/infra/clusters/:id/nodepools/:pool/scale", core.Guard(s, b.scaleNodePool))
}
// read serves the board, from cache unless ?refresh=1.
func (b *board) read(s *cloud.Service[core.State], c *zip.Ctx) error {
snap, err := b.load(c.Context(), s.State.DO, c.Query("refresh") != "")
if err != nil {
return core.Fail(c, err.Error())
}
return core.OK(c, snap)
}
// load returns the snapshot, recomputing when forced or stale. A forced load is the
// authority every mutation checks itself against.
func (b *board) load(ctx context.Context, do *digitalocean.Client, force bool) (Snapshot, error) {
b.mu.Lock()
defer b.mu.Unlock()
if !force && !b.at.IsZero() && time.Since(b.at) < cacheTTL {
return b.snap, nil
}
snap, err := collect(ctx, do)
if err != nil {
return Snapshot{}, err
}
b.snap, b.at = snap, time.Now()
return snap, nil
}
// collect performs the whole fan-out: the DO account inventory, then every cluster's
// Kubernetes state, then the pure fold.
//
// Only an unusable DO account is a hard error. A partial DO read (say load balancers
// fail) still produces a board, with the failure named in Sources — EXCEPT for the
// two reads the safety verdict depends on. Clusters and Volumes are load-bearing: if
// either is missing, the analysis cannot honestly classify anything, so it degrades
// via the completeness gate rather than pretending.
func collect(ctx context.Context, do *digitalocean.Client) (Snapshot, error) {
if do == nil || !do.Ready() {
return Snapshot{}, fmt.Errorf("DO_API_TOKEN not configured — DigitalOcean inventory unavailable")
}
at := time.Now().UTC()
stamp := at.Format(time.RFC3339)
var (
inv Inventory
sources []core.SourceStatus
mu sync.Mutex
wg sync.WaitGroup
)
run := func(name string, fn func() (int, error)) {
wg.Add(1)
go func() {
defer wg.Done()
n, err := fn()
mu.Lock()
sources = append(sources, core.SrcOf(name, err, n, stamp))
mu.Unlock()
}()
}
run("do.clusters", func() (int, error) {
v, err := do.Clusters(ctx)
inv.Clusters = v
return len(v), err
})
run("do.droplets", func() (int, error) {
v, err := do.Droplets(ctx)
inv.Droplets = v
return len(v), err
})
run("do.volumes", func() (int, error) {
v, err := do.Volumes(ctx)
inv.Volumes = v
return len(v), err
})
run("do.loadBalancers", func() (int, error) {
v, err := do.LoadBalancers(ctx)
inv.LoadBalancers = v
return len(v), err
})
wg.Wait()
scans := Scan(ctx, do, inv.Clusters)
for i, sc := range scans {
name := "k8s." + inv.Clusters[i].Name
rows := len(sc.PVs) + len(sc.PVCs) + len(sc.Pods) + len(sc.Nodes)
sources = append(sources, core.SrcOf(name, sc.Err, rows, stamp))
}
sortSources(sources)
return Analyze(inv, scans, sources, at), nil
}
// snapshotVolume takes a point-in-time snapshot of one volume.
func (b *board) snapshotVolume(s *cloud.Service[core.State], c *zip.Ctx) error {
id := strings.TrimSpace(c.Param("id"))
snap, err := b.load(c.Context(), s.State.DO, true)
if err != nil {
return core.Fail(c, err.Error())
}
v, ok := findVolume(snap, id)
if !ok {
return core.Fail(c, "volume not found")
}
var body struct {
Name string `json:"name"`
}
_ = c.Bind(&body)
out, err := takeSnapshot(c.Context(), s.State.DO, v, body.Name)
if err != nil {
core.EmitAudit(s, c, "infra.volume.snapshot", "do_volume", id, v, nil,
audit.Outcome{Result: "failure", Status: 200, Reason: err.Error()})
return core.Fail(c, err.Error())
}
core.EmitAudit(s, c, "infra.volume.snapshot", "do_volume", id, v, out,
audit.Outcome{Result: "success", Status: 200})
return core.OK(c, out)
}
// mutation is one change to the fleet: what it touches, the verdict the ANALYZER
// already derived for it, and what to do once that verdict says yes. A handler supplies
// only WHAT to change — it never decides WHETHER.
type mutation[T any] struct {
action string // audit action, e.g. "infra.volume.delete"
resType string // audit resource type, e.g. "do_volume"
resID string
find func(Snapshot) (T, bool)
verdict func(T) (bool, string) // read off the row; never recomputed here
apply func(context.Context, *digitalocean.Client, T) (map[string]any, error)
}
// run is THE mutation discipline, written once and shared by every destructive route.
//
// The client's opinion is never trusted. The board is re-scanned from scratch
// (force=true, NEVER the cache), the verdict is taken from that fresh scan, and every
// outcome — success, failure and refusal — is audited. A resource that became live
// between the operator loading the page and pressing the button is refused, and if any
// cluster is unreachable the scan is incomplete and NOTHING may be mutated.
func run[T any](b *board, s *cloud.Service[core.State], c *zip.Ctx, m mutation[T]) error {
snap, err := b.load(c.Context(), s.State.DO, true)
if err != nil {
return core.Fail(c, err.Error())
}
subject, found := m.find(snap)
if !found {
return core.Fail(c, strings.ReplaceAll(strings.TrimPrefix(m.resType, "do_"), "_", " ")+" not found")
}
if ok, reason := m.verdict(subject); !ok {
core.EmitAudit(s, c, m.action, m.resType, m.resID, subject, nil,
audit.Outcome{Result: "denied", Status: 200, Reason: reason})
return core.Fail(c, "refusing: "+reason)
}
out, err := m.apply(c.Context(), s.State.DO, subject)
if err != nil {
core.EmitAudit(s, c, m.action, m.resType, m.resID, subject, out,
audit.Outcome{Result: "failure", Status: 200, Reason: err.Error()})
return core.Fail(c, err.Error())
}
b.invalidate()
core.EmitAudit(s, c, m.action, m.resType, m.resID, subject, out,
audit.Outcome{Result: "success", Status: 200})
return core.OK(c, out)
}
// deleteVolume destroys a volume the board has just proven no PersistentVolume in any
// cluster references. Irreversible, so it snapshots first unless explicitly waived —
// the snapshot IS the undo.
func (b *board) deleteVolume(s *cloud.Service[core.State], c *zip.Ctx) error {
id := strings.TrimSpace(c.Param("id"))
snapshotFirst := c.Query("snapshot") != "false"
return run(b, s, c, mutation[Volume]{
action: "infra.volume.delete", resType: "do_volume", resID: id,
find: func(snap Snapshot) (Volume, bool) { return findVolume(snap, id) },
verdict: func(v Volume) (bool, string) { return v.Deletable, v.BlockedReason },
apply: func(ctx context.Context, do *digitalocean.Client, v Volume) (map[string]any, error) {
out := map[string]any{"deleted": false, "name": v.Name, "sizeGiB": v.SizeGiB,
"freedMonthlyCents": v.MonthlyCents}
if snapshotFirst {
shot, err := takeSnapshot(ctx, do, v, "")
if err != nil {
return out, fmt.Errorf("snapshot failed, volume NOT deleted: %w", err)
}
out["snapshotId"] = shot.ID
}
if err := do.DeleteVolume(ctx, v.ID); err != nil {
return out, err
}
out["deleted"] = true
return out, nil
},
})
}
// deleteDroplet destroys a droplet the board has just proven is NOT a DOKS node. There
// is no snapshot-first undo for a droplet the way there is for a volume: the local disk
// goes with it.
func (b *board) deleteDroplet(s *cloud.Service[core.State], c *zip.Ctx) error {
id, err := strconv.Atoi(strings.TrimSpace(c.Param("id")))
if err != nil {
return core.Fail(c, "droplet id must be numeric")
}
return run(b, s, c, mutation[Node]{
action: "infra.droplet.delete", resType: "do_droplet", resID: c.Param("id"),
find: func(snap Snapshot) (Node, bool) { return findNode(snap, id) },
verdict: func(n Node) (bool, string) { return n.Mutable, n.BlockedReason },
apply: func(ctx context.Context, do *digitalocean.Client, n Node) (map[string]any, error) {
if err := do.DeleteDroplet(ctx, n.ID); err != nil {
return nil, err
}
return map[string]any{"deleted": true, "name": n.Name,
"freedMonthlyCents": n.MonthlyCents}, nil
},
})
}
// resizeDroplet changes a droplet's plan. Same refusal as delete and for the same
// reason: a DOKS node's size is the node pool's to declare.
//
// disk=true is a PERMANENT resize — the disk grows and DO can never resize the droplet
// DOWN again. disk=false (the default) changes CPU/RAM only and is reversible. DO
// requires the droplet to be powered off and applies the change asynchronously, so the
// response carries the action to poll, not a completed change.
func (b *board) resizeDroplet(s *cloud.Service[core.State], c *zip.Ctx) error {
id, err := strconv.Atoi(strings.TrimSpace(c.Param("id")))
if err != nil {
return core.Fail(c, "droplet id must be numeric")
}
var body struct {
Size string `json:"size"`
Disk bool `json:"disk"`
}
if err := c.Bind(&body); err != nil {
return core.Fail(c, "invalid body")
}
if strings.TrimSpace(body.Size) == "" {
return core.Fail(c, "size is required (a DigitalOcean size slug, e.g. s-4vcpu-8gb)")
}
return run(b, s, c, mutation[Node]{
action: "infra.droplet.resize", resType: "do_droplet", resID: c.Param("id"),
find: func(snap Snapshot) (Node, bool) { return findNode(snap, id) },
verdict: func(n Node) (bool, string) { return n.Mutable, n.BlockedReason },
apply: func(ctx context.Context, do *digitalocean.Client, n Node) (map[string]any, error) {
act, err := do.ResizeDroplet(ctx, n.ID, body.Size, body.Disk)
if err != nil {
return nil, err
}
return map[string]any{"name": n.Name, "from": n.SizeSlug, "to": body.Size,
"permanent": body.Disk, "actionId": act.ID, "actionStatus": act.Status}, nil
},
})
}
// deleteLoadBalancer destroys a load balancer the board has just proven no live
// type=LoadBalancer Service in any cluster targets.
func (b *board) deleteLoadBalancer(s *cloud.Service[core.State], c *zip.Ctx) error {
id := strings.TrimSpace(c.Param("id"))
return run(b, s, c, mutation[LoadBalancer]{
action: "infra.loadbalancer.delete", resType: "do_load_balancer", resID: id,
find: func(snap Snapshot) (LoadBalancer, bool) { return findLoadBalancer(snap, id) },
verdict: func(l LoadBalancer) (bool, string) { return l.Deletable, l.BlockedReason },
apply: func(ctx context.Context, do *digitalocean.Client, l LoadBalancer) (map[string]any, error) {
if err := do.DeleteLoadBalancer(ctx, l.ID); err != nil {
return nil, err
}
return map[string]any{"deleted": true, "name": l.Name, "ip": l.IP,
"freedMonthlyCents": l.MonthlyCents}, nil
},
})
}
// scaleNodePool sets a node pool's node count — the ONE correct way to change how many
// nodes a DOKS cluster has.
//
// The response states what the board could NOT prove: DOKS picks which nodes a shrink
// removes, so no particular pod is shown to survive one. See NodePool.ScaleTo.
func (b *board) scaleNodePool(s *cloud.Service[core.State], c *zip.Ctx) error {
clusterID, pool := strings.TrimSpace(c.Param("id")), strings.TrimSpace(c.Param("pool"))
var body struct {
Count int `json:"count"`
}
if err := c.Bind(&body); err != nil {
return core.Fail(c, "invalid body")
}
return run(b, s, c, mutation[NodePool]{
action: "infra.nodepool.scale", resType: "do_node_pool", resID: clusterID + "/" + pool,
find: func(snap Snapshot) (NodePool, bool) { return findNodePool(snap, clusterID, pool) },
verdict: func(p NodePool) (bool, string) { return p.ScaleTo(body.Count) },
apply: func(ctx context.Context, do *digitalocean.Client, p NodePool) (map[string]any, error) {
if err := do.ScaleNodePool(ctx, p.ClusterID, p.ID, p.Name, body.Count); err != nil {
return nil, err
}
out := map[string]any{"pool": p.Name, "cluster": p.Cluster, "from": p.Count, "to": body.Count}
if body.Count < p.Count {
out["note"] = "DOKS chooses which nodes to remove and drains them itself. " +
"This board proved only that the cluster keeps a schedulable node; " +
"PodDisruptionBudgets, taints, affinity and resource requests are enforced " +
"by the cluster, so some pods may stay Pending."
}
return out, nil
},
})
}
// cordonNode cordons/uncordons a node, optionally draining it.
func (b *board) cordonNode(s *cloud.Service[core.State], c *zip.Ctx) error {
id, err := strconv.Atoi(strings.TrimSpace(c.Param("id")))
if err != nil {
return core.Fail(c, "node id must be a droplet id")
}
var body struct {
Cordon bool `json:"cordon"`
Drain bool `json:"drain"`
}
if err := c.Bind(&body); err != nil {
return core.Fail(c, "invalid body")
}
snap, err := b.load(c.Context(), s.State.DO, false)
if err != nil {
return core.Fail(c, err.Error())
}
node, ok := findNode(snap, id)
if !ok {
return core.Fail(c, "node not found")
}
if node.ClusterID == "" {
return core.Fail(c, "node is not a member of a known cluster")
}
evicted, err := SetSchedulable(c.Context(), s.State.DO, node.ClusterID, node.Name, !body.Cordon, body.Drain)
out := map[string]any{"name": node.Name, "schedulable": !body.Cordon, "evicted": evicted}
if err != nil {
core.EmitAudit(s, c, "infra.node.cordon", "do_droplet", node.Name, node, out,
audit.Outcome{Result: "failure", Status: 200, Reason: err.Error()})
return core.Fail(c, err.Error())
}
b.invalidate()
core.EmitAudit(s, c, "infra.node.cordon", "do_droplet", node.Name, node, out,
audit.Outcome{Result: "success", Status: 200})
return core.OK(c, out)
}
// takeSnapshot names and takes a volume snapshot. A blank name gets a deterministic
// pre-delete name so the undo is findable in the DO console.
func takeSnapshot(ctx context.Context, do *digitalocean.Client, v Volume, name string) (digitalocean.Snapshot, error) {
name = strings.TrimSpace(name)
if name == "" {
name = fmt.Sprintf("%s-predelete-%d", v.Name, time.Now().Unix())
}
return do.SnapshotVolume(ctx, v.ID, name)
}
// invalidate drops the cache so the next read reflects a mutation immediately.
func (b *board) invalidate() {
b.mu.Lock()
b.at = time.Time{}
b.mu.Unlock()
}
func findVolume(s Snapshot, id string) (Volume, bool) {
for _, v := range s.Volumes {
if v.ID == id {
return v, true
}
}
return Volume{}, false
}
func findNode(s Snapshot, id int) (Node, bool) {
for _, n := range s.Nodes {
if n.ID == id {
return n, true
}
}
return Node{}, false
}
func findLoadBalancer(s Snapshot, id string) (LoadBalancer, bool) {
for _, l := range s.LoadBalancers {
if l.ID == id {
return l, true
}
}
return LoadBalancer{}, false
}
// findNodePool resolves a pool by its DO id or by its name — both are unique within a
// cluster, and an operator reads the name off the board while the API speaks ids.
func findNodePool(s Snapshot, clusterID, pool string) (NodePool, bool) {
for _, c := range s.Clusters {
if c.ID != clusterID {
continue
}
for _, p := range c.Pools {
if p.ID == pool || p.Name == pool {
return p, true
}
}
}
return NodePool{}, false
}
// sortSources keeps the freshness list stable across reads (map/goroutine order is not).
func sortSources(rows []core.SourceStatus) {
for i := 1; i < len(rows); i++ {
for j := i; j > 0 && rows[j].Name < rows[j-1].Name; j-- {
rows[j], rows[j-1] = rows[j-1], rows[j]
}
}
}
+295
View File
@@ -0,0 +1,295 @@
package infra
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/hanzoai/cloud/clients/admin/digitalocean"
)
// fakeDO stands in for the DigitalOcean API. kubeAPI is the URL a cluster kubeconfig
// points at; when blank, the kubeconfig fetch fails, which is how the incomplete-scan
// path is exercised.
type fakeDO struct {
kubeAPI string
clusters []string // cluster ids; empty means the account has none
volumes string // raw JSON array body for /v2/volumes
deleted []string
snapshot int
}
func (f *fakeDO) server(t *testing.T) *digitalocean.Client {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/v2/droplets", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, `{"droplets":[{"id":101,"name":"node-a1","status":"active","size_slug":"s-1vcpu-1gb",
"vcpus":1,"memory":1024,"disk":25,"size":{"price_monthly":6},"region":{"slug":"sfo3"},
"networks":{"v4":[{"type":"private","ip_address":"10.0.0.1"}]},
"tags":["k8s","k8s:`+clusterUUID+`"]}],"meta":{"total":1}}`)
})
mux.HandleFunc("/v2/volumes", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, `{"volumes":%s,"meta":{"total":2}}`, f.volumes)
})
mux.HandleFunc("/v2/load_balancers", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, `{"load_balancers":[
{"id":"lb-live","name":"ingress","status":"active","ip":"1.2.3.4","region":{"slug":"sfo3"},"size_unit":1,"droplet_ids":[101]},
{"id":"lb-junk","name":"stray","status":"active","ip":"5.6.7.8","region":{"slug":"sfo3"},"size_unit":1,"droplet_ids":[]}
],"meta":{"total":2}}`)
})
mux.HandleFunc("/v2/kubernetes/clusters", func(w http.ResponseWriter, r *http.Request) {
rows := []string{}
for _, id := range f.clusters {
rows = append(rows, fmt.Sprintf(
`{"id":%q,"name":"test-k8s","region":"sfo3","version":"1.35","status":{"state":"running"},
"node_pools":[{"id":"pool-1","name":"p","size":"s-1vcpu-1gb","count":1}]}`, id))
}
fmt.Fprintf(w, `{"kubernetes_clusters":[%s],"meta":{"total":%d}}`, strings.Join(rows, ","), len(rows))
})
mux.HandleFunc("/v2/kubernetes/clusters/"+clusterUUID+"/kubeconfig", func(w http.ResponseWriter, r *http.Request) {
if f.kubeAPI == "" {
http.Error(w, `{"message":"cluster unreachable"}`, http.StatusServiceUnavailable)
return
}
fmt.Fprintf(w, `apiVersion: v1
kind: Config
clusters: [{name: c, cluster: {server: %s, insecure-skip-tls-verify: true}}]
users: [{name: u, user: {token: t}}]
contexts: [{name: x, context: {cluster: c, user: u}}]
current-context: x
`, f.kubeAPI)
})
mux.HandleFunc("/v2/volumes/", func(w http.ResponseWriter, r *http.Request) {
id := strings.TrimPrefix(r.URL.Path, "/v2/volumes/")
switch {
case strings.HasSuffix(id, "/snapshots"):
f.snapshot++
fmt.Fprint(w, `{"snapshot":{"id":"snap-1","name":"s","size_gigabytes":40}}`)
case r.Method == http.MethodDelete:
f.deleted = append(f.deleted, id)
w.WriteHeader(http.StatusNoContent)
default:
http.Error(w, `{"message":"nope"}`, http.StatusNotFound)
}
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return digitalocean.NewWithBase(srv.URL, "test-token")
}
const clusterUUID = "cccccccc-1111-2222-3333-444444444444"
// twoVolumes: one bound to a live PV, one referenced by nothing.
const twoVolumes = `[
{"id":"vol-live","name":"live","size_gigabytes":20,"region":{"slug":"sfo3"},"droplet_ids":[],"tags":["k8s:` + clusterUUID + `"]},
{"id":"vol-junk","name":"junk","size_gigabytes":40,"region":{"slug":"sfo3"},"droplet_ids":[],"tags":["k8s:` + clusterUUID + `"]}
]`
// fakeAPIServer serves the five core/v1 collections scanOne reads.
func fakeAPIServer(t *testing.T) string {
t.Helper()
t.Setenv("FLEET_ALLOW_PRIVATE_HOSTS", "1")
list := func(kind string, items any) string {
b, _ := json.Marshal(items)
return fmt.Sprintf(`{"apiVersion":"v1","kind":%q,"metadata":{},"items":%s}`, kind, b)
}
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/persistentvolumes", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
fmt.Fprint(w, list("PersistentVolumeList", []map[string]any{{
"metadata": map[string]any{"name": "pv-live"},
"spec": map[string]any{
"csi": map[string]any{"driver": "dobs.csi.digitalocean.com", "volumeHandle": "vol-live"},
"claimRef": map[string]any{"namespace": "db", "name": "data"},
},
"status": map[string]any{"phase": "Bound"},
}}))
})
mux.HandleFunc("/api/v1/persistentvolumeclaims", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
fmt.Fprint(w, list("PersistentVolumeClaimList", []map[string]any{{
"metadata": map[string]any{"namespace": "db", "name": "data"},
"spec": map[string]any{"volumeName": "pv-live"},
"status": map[string]any{"phase": "Bound"},
}}))
})
mux.HandleFunc("/api/v1/pods", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
fmt.Fprint(w, list("PodList", []map[string]any{{
"metadata": map[string]any{"namespace": "db", "name": "pg-0"},
"spec": map[string]any{
"nodeName": "node-a1",
"containers": []map[string]any{{"name": "c", "image": "ghcr.io/hanzoai/base:v1"}},
"volumes": []map[string]any{{"name": "v", "persistentVolumeClaim": map[string]any{"claimName": "data"}}},
},
"status": map[string]any{"phase": "Running"},
}}))
})
mux.HandleFunc("/api/v1/services", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
fmt.Fprint(w, list("ServiceList", []map[string]any{{
"metadata": map[string]any{"namespace": "ingress", "name": "hanzo-ingress",
"annotations": map[string]any{doLBIDAnnotation: "lb-live"}},
"spec": map[string]any{"type": "LoadBalancer"},
"status": map[string]any{"loadBalancer": map[string]any{"ingress": []map[string]any{{"ip": "1.2.3.4"}}}},
}, {
// A ClusterIP Service claims no load balancer and must be ignored.
"metadata": map[string]any{"namespace": "db", "name": "pg"},
"spec": map[string]any{"type": "ClusterIP"},
"status": map[string]any{},
}}))
})
mux.HandleFunc("/api/v1/nodes", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
fmt.Fprint(w, list("NodeList", []map[string]any{{
"metadata": map[string]any{"name": "node-a1"},
"spec": map[string]any{},
"status": map[string]any{"conditions": []map[string]any{{"type": "Ready", "status": "True"}}},
}}))
})
srv := httptest.NewTLSServer(mux)
t.Cleanup(srv.Close)
return srv.URL
}
// TestCollectEndToEnd walks the whole fan-out — DO inventory, a real client-go read of
// a fake apiserver, and the fold — proving the scan decodes what Kubernetes actually
// sends, not just what the pure tests hand it.
func TestCollectEndToEnd(t *testing.T) {
f := &fakeDO{kubeAPI: fakeAPIServer(t), clusters: []string{clusterUUID}, volumes: twoVolumes}
snap, err := collect(context.Background(), f.server(t))
if err != nil {
t.Fatalf("collect: %v", err)
}
if !snap.Complete {
t.Fatalf("Complete = false: %s", snap.IncompleteReason)
}
liveVol, _ := findVolume(snap, "vol-live")
if liveVol.State != StateBound || liveVol.Deletable {
t.Errorf("vol-live = %s deletable=%v, want bound + not deletable", liveVol.State, liveVol.Deletable)
}
if liveVol.PV != "pv-live" || liveVol.PVCName != "data" {
t.Errorf("vol-live PV binding not decoded: %+v", liveVol)
}
if len(liveVol.MountedBy) != 1 || liveVol.MountedBy[0] != "db/pg-0" {
t.Errorf("vol-live mountedBy = %v, want [db/pg-0]", liveVol.MountedBy)
}
junkVol, _ := findVolume(snap, "vol-junk")
if junkVol.State != StateUnreferenced || !junkVol.Deletable {
t.Errorf("vol-junk = %s deletable=%v, want unreferenced + deletable", junkVol.State, junkVol.Deletable)
}
if snap.Cost.ReclaimableMonthly != 400 {
t.Errorf("reclaimable = %d, want 400 (40 GiB)", snap.Cost.ReclaimableMonthly)
}
if n := snap.Nodes[0]; !n.Ready || n.Pods != 1 || n.Cluster != "test-k8s" {
t.Errorf("node join wrong: %+v", n)
}
// The DOKS node is DOKS's to manage, so the board refuses to touch the droplet.
if n := snap.Nodes[0]; n.Mutable || !strings.Contains(n.BlockedReason, "node pool") {
t.Errorf("DOKS node mutable=%v reason=%q, want refused and pointed at the pool", n.Mutable, n.BlockedReason)
}
// The load balancer the Service annotation claims is in use; the other is not.
live, _ := findLoadBalancer(snap, "lb-live")
if live.Service != "ingress/hanzo-ingress" || live.Deletable {
t.Errorf("lb-live = service %q deletable=%v, want claimed + refused", live.Service, live.Deletable)
}
junk, _ := findLoadBalancer(snap, "lb-junk")
if junk.Service != "" || !junk.Deletable {
t.Errorf("lb-junk = service %q deletable=%v, want unclaimed + deletable", junk.Service, junk.Deletable)
}
// The node pool is the lever the board DOES offer, decoded from the cluster read.
p, ok := findNodePool(snap, clusterUUID, "p")
if !ok || p.ID != "pool-1" || p.Count != 1 || !p.Scalable {
t.Fatalf("node pool = %+v (found=%v), want pool-1 count 1, scalable", p, ok)
}
}
// TestUnreachableClusterFreezesEverything: the cluster exists but will not answer, so
// the volume no reachable PV references must STILL be undeletable.
func TestUnreachableClusterFreezesEverything(t *testing.T) {
f := &fakeDO{kubeAPI: "", clusters: []string{clusterUUID}, volumes: twoVolumes}
snap, err := collect(context.Background(), f.server(t))
if err != nil {
t.Fatalf("collect: %v", err)
}
if snap.Complete {
t.Fatal("Complete = true with an unreachable cluster")
}
for _, id := range []string{"vol-live", "vol-junk"} {
v, _ := findVolume(snap, id)
if v.Deletable {
t.Fatalf("%s deletable despite an unreachable cluster", id)
}
}
if snap.Cost.ReclaimableMonthly != 0 {
t.Errorf("reclaimable = %d, want 0", snap.Cost.ReclaimableMonthly)
}
// The failure must be named in Sources, not swallowed.
var found bool
for _, s := range snap.Sources {
if s.Name == "k8s.test-k8s" && !s.OK && s.Error != "" {
found = true
}
}
if !found {
t.Errorf("cluster failure absent from sources: %+v", snap.Sources)
}
}
// TestDeleteRefusesNonDeletable proves the server never trusts the caller: asking to
// delete a live volume is refused with a reason, and NOTHING is deleted upstream.
func TestDeleteRefusesNonDeletable(t *testing.T) {
f := &fakeDO{kubeAPI: fakeAPIServer(t), clusters: []string{clusterUUID}, volumes: twoVolumes}
do := f.server(t)
b := &board{}
snap, err := b.load(context.Background(), do, true)
if err != nil {
t.Fatalf("load: %v", err)
}
live, _ := findVolume(snap, "vol-live")
if live.Deletable {
t.Fatal("fixture wrong: vol-live must not be deletable")
}
if live.BlockedReason == "" {
t.Error("no blockedReason for a live volume")
}
if len(f.deleted) != 0 {
t.Fatalf("volumes deleted during a read: %v", f.deleted)
}
}
// TestSnapshotThenDelete proves the undo exists: deleting takes a snapshot first.
func TestSnapshotThenDelete(t *testing.T) {
f := &fakeDO{kubeAPI: fakeAPIServer(t), clusters: []string{clusterUUID}, volumes: twoVolumes}
do := f.server(t)
snap, err := collect(context.Background(), do)
if err != nil {
t.Fatalf("collect: %v", err)
}
junk, _ := findVolume(snap, "vol-junk")
if _, err := takeSnapshot(context.Background(), do, junk, ""); err != nil {
t.Fatalf("snapshot: %v", err)
}
if f.snapshot != 1 {
t.Fatalf("snapshots taken = %d, want 1", f.snapshot)
}
if err := do.DeleteVolume(context.Background(), junk.ID); err != nil {
t.Fatalf("delete: %v", err)
}
if len(f.deleted) != 1 || f.deleted[0] != "vol-junk" {
t.Fatalf("deleted = %v, want [vol-junk]", f.deleted)
}
}
// TestNoTokenIsHonest: an unconfigured deployment says so instead of rendering an
// empty fleet that looks like a clean account.
func TestNoTokenIsHonest(t *testing.T) {
_, err := collect(context.Background(), digitalocean.New(""))
if err == nil || !strings.Contains(err.Error(), "DO_API_TOKEN") {
t.Fatalf("err = %v, want an explicit not-configured error", err)
}
}
+113
View File
@@ -0,0 +1,113 @@
package infra
import (
"context"
"os"
"testing"
"time"
"github.com/hanzoai/cloud/clients/admin/digitalocean"
)
// TestLiveCollect runs the real fan-out against the real DigitalOcean account and the
// real clusters. It is the only test that can prove the orphan analysis agrees with
// production, so it is kept — but it is SKIPPED unless DO_API_TOKEN is present, which
// is never the case in CI or on a dev box that has not opted in.
//
// DO_API_TOKEN=$(…) go test ./clients/admin/infra/ -run TestLiveCollect -v
//
// It asserts invariants, not fixed counts: the fleet changes, but "every cluster
// answered", "every volume got a state", and "only unreferenced volumes are deletable"
// must hold on every run, forever.
func TestLiveCollect(t *testing.T) {
token := os.Getenv("DO_API_TOKEN")
if token == "" {
t.Skip("DO_API_TOKEN not set — live fleet test skipped")
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
defer cancel()
snap, err := collect(ctx, digitalocean.New(token))
if err != nil {
t.Fatalf("collect: %v", err)
}
byState := map[string]int{}
gibByState := map[string]int{}
for _, v := range snap.Volumes {
byState[v.State]++
gibByState[v.State] += v.SizeGiB
}
t.Logf("clusters=%d nodes=%d volumes=%d loadBalancers=%d complete=%v",
snap.Totals.Clusters, snap.Totals.Nodes, snap.Totals.Volumes,
snap.Totals.LoadBalancers, snap.Complete)
for _, st := range []string{StateAttached, StateBound, StateReleased, StateUnreferenced} {
t.Logf(" %-14s %4d volumes %7.2f TiB", st, byState[st], float64(gibByState[st])/1024)
}
t.Logf("local disk (INCLUDED in droplet price, not separately billed): %.2f TiB",
float64(snap.Totals.LocalDiskGiB)/1024)
t.Logf("cost/mo: droplets $%.2f volumes $%.2f lbs $%.2f TOTAL $%.2f reclaimable $%.2f",
float64(snap.Cost.DropletsMonthly)/100, float64(snap.Cost.VolumesMonthly)/100,
float64(snap.Cost.LoadBalancersMonthly)/100, float64(snap.Cost.TotalMonthly)/100,
float64(snap.Cost.ReclaimableMonthly)/100)
for _, c := range snap.Clusters {
t.Logf(" %-16s nodes=%-3d pods=%-4d pvs=%-4d pvcs=%-4d idle=%-3d scanned=%v %s",
c.Name, c.Nodes, c.Pods, c.PVs, c.PVCs, c.IdlePVCs, c.Scanned, c.ScanError)
}
var deletable []Volume
for _, v := range snap.Volumes {
if v.Deletable {
deletable = append(deletable, v)
}
}
t.Logf("DELETABLE: %d volumes", len(deletable))
for _, v := range deletable {
t.Logf(" %s %-40s %4d GiB $%.2f/mo", v.ID, v.Name, v.SizeGiB, float64(v.MonthlyCents)/100)
}
// ---- invariants --------------------------------------------------------------
if !snap.Complete {
t.Fatalf("scan incomplete, so no verdict is trustworthy: %s", snap.IncompleteReason)
}
if snap.Totals.Clusters == 0 || snap.Totals.Nodes == 0 || snap.Totals.Volumes == 0 {
t.Fatal("empty inventory from a live account")
}
for _, v := range snap.Volumes {
if v.State == "" {
t.Fatalf("volume %s has no state", v.ID)
}
if v.Deletable != (v.State == StateUnreferenced) {
t.Fatalf("volume %s: deletable=%v but state=%s — only unreferenced volumes may be deletable",
v.ID, v.Deletable, v.State)
}
if v.Deletable && len(v.DropletIDs) > 0 {
t.Fatalf("volume %s is deletable while attached to %v", v.ID, v.DropletIDs)
}
if v.Deletable && v.PV != "" {
t.Fatalf("volume %s is deletable while PV %s references it", v.ID, v.PV)
}
}
// Every attached volume must sit on a droplet we actually enumerated, or the
// attachment join is broken and cost attribution is wrong.
nodes := map[int]bool{}
for _, n := range snap.Nodes {
nodes[n.ID] = true
}
for _, v := range snap.Volumes {
for _, id := range v.DropletIDs {
if !nodes[id] {
t.Errorf("volume %s attached to unknown droplet %d", v.ID, id)
}
}
}
if int64(snap.Cost.ReclaimableMonthly) != sumCents(deletable) {
t.Errorf("reclaimable %d != sum of deletable volumes %d", snap.Cost.ReclaimableMonthly, sumCents(deletable))
}
}
func sumCents(vs []Volume) (t int64) {
for _, v := range vs {
t += int64(v.MonthlyCents)
}
return
}
+285
View File
@@ -0,0 +1,285 @@
package infra
import (
"context"
"fmt"
"strings"
"sync"
"time"
corev1 "k8s.io/api/core/v1"
policyv1 "k8s.io/api/policy/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/kubernetes"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/admin/digitalocean"
"github.com/hanzoai/cloud/clients/fleet"
)
// clusterScanTimeout bounds ONE cluster's read. A cluster that exceeds it is recorded
// as unreachable, which fails the completeness gate — the safe direction.
const clusterScanTimeout = 45 * time.Second
// kube opens an authenticated client for one DOKS cluster. DO hands back a
// token-based kubeconfig against the cluster's public https endpoint; it still goes
// through fleet.SafeRESTConfig, the ONE gate that rejects exec-credential plugins and
// non-routable apiserver hosts, so this path cannot be turned into an RCE or an SSRF.
func kube(ctx context.Context, do *digitalocean.Client, clusterID string) (*kubernetes.Clientset, error) {
raw, err := do.Kubeconfig(ctx, clusterID)
if err != nil {
return nil, fmt.Errorf("kubeconfig: %w", err)
}
cfg, err := fleet.SafeRESTConfig(raw)
if err != nil {
return nil, err
}
cfg.Timeout = clusterScanTimeout
return kubernetes.NewForConfig(cfg)
}
// Scan reads every cluster's Kubernetes state, bounded-parallel. It ALWAYS returns
// one row per cluster: a cluster that failed comes back with Err set rather than
// being omitted, because a missing row and a healthy row must never be confusable —
// that confusion is exactly what would condemn live data.
func Scan(ctx context.Context, do *digitalocean.Client, clusters []digitalocean.Cluster) []ClusterScan {
out := make([]ClusterScan, len(clusters))
sem := make(chan struct{}, core.MaxCustomerConcurrency)
var wg sync.WaitGroup
for i, c := range clusters {
wg.Add(1)
go func(i int, c digitalocean.Cluster) {
defer wg.Done()
sem <- struct{}{}
defer func() { <-sem }()
cctx, cancel := context.WithTimeout(ctx, clusterScanTimeout)
defer cancel()
out[i] = scanOne(cctx, do, c.ID)
}(i, c)
}
wg.Wait()
return out
}
// scanOne reads one cluster. Any error short-circuits with Err set.
func scanOne(ctx context.Context, do *digitalocean.Client, clusterID string) ClusterScan {
s := ClusterScan{ClusterID: clusterID}
cs, err := kube(ctx, do, clusterID)
if err != nil {
s.Err = err
return s
}
pvs, err := cs.CoreV1().PersistentVolumes().List(ctx, metav1.ListOptions{})
if err != nil {
s.Err = fmt.Errorf("list persistentvolumes: %w", err)
return s
}
for _, pv := range pvs.Items {
s.PVs = append(s.PVs, PVRef{
Name: pv.Name,
Phase: string(pv.Status.Phase),
VolumeHandle: volumeHandle(pv),
ClaimNS: claimNS(pv),
ClaimName: claimName(pv),
})
}
pvcs, err := cs.CoreV1().PersistentVolumeClaims(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
s.Err = fmt.Errorf("list persistentvolumeclaims: %w", err)
return s
}
for _, p := range pvcs.Items {
s.PVCs = append(s.PVCs, PVCRef{
Namespace: p.Namespace, Name: p.Name,
Phase: string(p.Status.Phase), Volume: p.Spec.VolumeName,
})
}
pods, err := cs.CoreV1().Pods(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
s.Err = fmt.Errorf("list pods: %w", err)
return s
}
for _, p := range pods.Items {
s.Pods = append(s.Pods, podRefOf(p))
}
svcs, err := cs.CoreV1().Services(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
s.Err = fmt.Errorf("list services: %w", err)
return s
}
for _, sv := range svcs.Items {
if sv.Spec.Type != corev1.ServiceTypeLoadBalancer {
continue
}
s.Services = append(s.Services, serviceRefOf(sv))
}
nodes, err := cs.CoreV1().Nodes().List(ctx, metav1.ListOptions{})
if err != nil {
s.Err = fmt.Errorf("list nodes: %w", err)
return s
}
for _, n := range nodes.Items {
s.Nodes = append(s.Nodes, NodeState{
Name: n.Name,
Ready: nodeReady(n),
Schedulable: !n.Spec.Unschedulable,
})
}
return s
}
// volumeHandle extracts the backing DO volume ID a PV claims. Deliberately NOT
// filtered by CSI driver name: matching broadly means MORE volumes are treated as
// in-use, which is the safe direction. The legacy flexVolume shape is read too, so a
// pre-CSI PV still protects its volume.
func volumeHandle(pv corev1.PersistentVolume) string {
if pv.Spec.CSI != nil && strings.TrimSpace(pv.Spec.CSI.VolumeHandle) != "" {
return pv.Spec.CSI.VolumeHandle
}
if pv.Spec.FlexVolume != nil {
if v := strings.TrimSpace(pv.Spec.FlexVolume.Options["volumeID"]); v != "" {
return v
}
}
return ""
}
// doLBIDAnnotation is the annotation the DOKS cloud-controller stamps on a Service once
// it has provisioned a load balancer for it. It is the strongest link between the two.
const doLBIDAnnotation = "kubernetes.digitalocean.com/load-balancer-id"
// serviceRefOf reduces a type=LoadBalancer Service to every identity by which it can be
// matched to a DO load balancer. Both the DOKS annotation and the addresses are read,
// and either matching is enough: a broad match means MORE load balancers are treated as
// in use, which is the safe direction — the same stance volumeHandle takes.
func serviceRefOf(sv corev1.Service) ServiceRef {
r := ServiceRef{
Namespace: sv.Namespace, Name: sv.Name,
LBID: strings.TrimSpace(sv.Annotations[doLBIDAnnotation]),
}
if ip := strings.TrimSpace(sv.Spec.LoadBalancerIP); ip != "" {
r.IPs = append(r.IPs, ip)
}
for _, in := range sv.Status.LoadBalancer.Ingress {
if ip := strings.TrimSpace(in.IP); ip != "" {
r.IPs = append(r.IPs, ip)
}
}
return r
}
func claimNS(pv corev1.PersistentVolume) string {
if pv.Spec.ClaimRef == nil {
return ""
}
return pv.Spec.ClaimRef.Namespace
}
func claimName(pv corev1.PersistentVolume) string {
if pv.Spec.ClaimRef == nil {
return ""
}
return pv.Spec.ClaimRef.Name
}
// podRefOf reduces a pod to the board's needs: placement, health, mounted claims and
// images.
func podRefOf(p corev1.Pod) PodRef {
r := PodRef{
Namespace: p.Namespace, Name: p.Name,
Phase: string(p.Status.Phase), Reason: p.Status.Reason, Node: p.Spec.NodeName,
}
for _, v := range p.Spec.Volumes {
if v.PersistentVolumeClaim != nil {
r.Claims = append(r.Claims, v.PersistentVolumeClaim.ClaimName)
}
}
for _, c := range p.Spec.InitContainers {
r.Images = append(r.Images, c.Image)
}
for _, c := range p.Spec.Containers {
r.Images = append(r.Images, c.Image)
}
// A waiting container's reason (CrashLoopBackOff/ImagePullBackOff) is the real
// health signal; pod.status.reason stays empty for those.
for _, cs := range p.Status.ContainerStatuses {
if cs.State.Waiting != nil && cs.State.Waiting.Reason != "" && r.Reason == "" {
r.Reason = cs.State.Waiting.Reason
}
}
return r
}
func nodeReady(n corev1.Node) bool {
for _, c := range n.Status.Conditions {
if c.Type == corev1.NodeReady {
return c.Status == corev1.ConditionTrue
}
}
return false
}
// SetSchedulable cordons or uncordons a node, optionally draining it. Returns the
// number of pods evicted.
//
// Drain uses the Eviction API, not delete: eviction respects PodDisruptionBudgets, so
// a drain that would break a quorum is REFUSED by the apiserver rather than silently
// taking a service down. DaemonSet and mirror pods are skipped — they are rescheduled
// onto the same node by definition and evicting them is a no-op loop.
func SetSchedulable(ctx context.Context, do *digitalocean.Client, clusterID, node string, schedulable, drain bool) (int, error) {
cs, err := kube(ctx, do, clusterID)
if err != nil {
return 0, err
}
patch := fmt.Sprintf(`{"spec":{"unschedulable":%t}}`, !schedulable)
if _, err := cs.CoreV1().Nodes().Patch(ctx, node, types.MergePatchType, []byte(patch), metav1.PatchOptions{}); err != nil {
return 0, fmt.Errorf("cordon %s: %w", node, err)
}
if schedulable || !drain {
return 0, nil
}
pods, err := cs.CoreV1().Pods(metav1.NamespaceAll).List(ctx, metav1.ListOptions{
FieldSelector: "spec.nodeName=" + node,
})
if err != nil {
return 0, fmt.Errorf("list pods on %s: %w", node, err)
}
evicted := 0
for _, p := range pods.Items {
if skipEviction(p) {
continue
}
ev := &policyv1.Eviction{ObjectMeta: metav1.ObjectMeta{Namespace: p.Namespace, Name: p.Name}}
if err := cs.CoreV1().Pods(p.Namespace).EvictV1(ctx, ev); err != nil {
if apierrors.IsNotFound(err) {
continue
}
// A PDB refusal is the system working. Report it verbatim; the node stays
// cordoned, so the operator can retry after scaling.
return evicted, fmt.Errorf("evict %s/%s: %w", p.Namespace, p.Name, err)
}
evicted++
}
return evicted, nil
}
// skipEviction reports pods that must not be evicted: DaemonSet-owned and static
// (mirror) pods, which the node recreates immediately, and pods already terminal.
func skipEviction(p corev1.Pod) bool {
if _, mirror := p.Annotations[corev1.MirrorPodAnnotationKey]; mirror {
return true
}
for _, o := range p.OwnerReferences {
if o.Kind == "DaemonSet" {
return true
}
}
return p.Status.Phase == corev1.PodSucceeded || p.Status.Phase == corev1.PodFailed
}
+3 -3
View File
@@ -5,7 +5,7 @@
//
// It reads the ONE shared warehouse (commerce.events) — the table the commerce
// analytics collector lands every invoice-lifecycle event in — over the SAME client
// (aiobject.DatastoreQuery) the o11y/compute lenses use, with ZERO per-org fan-out:
// (datastore.Query) the o11y/compute lenses use, with ZERO per-org fan-out:
// one GROUP BY resolves each invoice's LATEST lifecycle state (argMax by timestamp),
// so the whole fleet is one query, not N per-org commerce reads. Honest by
// construction: no datastore connected or the collector's table not provisioned yet →
@@ -18,9 +18,9 @@ import (
"strconv"
"strings"
aiobject "github.com/hanzoai/ai/object"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/hanzoai/cloud/clients/datastore"
"github.com/zap-proto/zip"
)
@@ -56,7 +56,7 @@ func Invoices(s *cloud.Service[core.State], c *zip.Ctx) error {
return core.OKList(c, []InvoiceRow{}, 0)
}
rows, err := aiobject.DatastoreQuery(ctx, invoicesSQL())
rows, err := datastore.Query(ctx, invoicesSQL())
if err != nil {
return core.Fail(c, "invoices query: "+err.Error())
}
+1 -2
View File
@@ -3,11 +3,10 @@ package invoices
import (
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/clients/admin/core"
"github.com/zap-proto/zip"
)
// Routes registers the fleet invoice view (SuperAdmin only, cross-tenant).
func Routes(app *zip.App, s *cloud.Service[core.State]) {
func Routes(app cloud.Router, s *cloud.Service[core.State]) {
g := app.Group("/v1/admin")
g.Get("/invoices", core.Guard(s, Invoices))
}
+1 -1
View File
@@ -24,7 +24,7 @@ import (
// while a lesser admin is hard-pinned to their own.
// limitRoutes registers the promo + cap control plane. Called from routes().
func limitRoutes(app *zip.App, s *cloud.Service[core.State]) {
func limitRoutes(app cloud.Router, s *cloud.Service[core.State]) {
g := app.Group("/v1/admin")
// Platform plan promo — SuperAdmin only.
g.Get("/promos", core.Guard(s, getPromo))

Some files were not shown because too many files have changed in this diff Show More