The first run on the GitHub plane gated and built the console, then the
publish stopped at "ensure project kms returned HTTP 400: slug is a
reserved subdomain and cannot be used". The admin console publishes as
hanzo-admin for the same reason; this one is hanzo-kms, and universe's
kms-static root follows it.
The caller moved to .hanzo/workflows on 2026-09-15, but the forge mirror
of a hanzoai/* repo runs no workflows, so since then nothing gated, built
or published: kms.hanzo.ai still serves the bundle from before the move.
The one caller is .github/workflows/cicd.yml again, on hanzoai/ci
build.yml@v2 with mode delegate, reading hanzo.yml as before.
Signed out, the console no longer draws "Sign in with hanzo.id". It has
the SDK mint hanzo-kms's PKCE request (its callback, an S256 challenge,
no nonce) and replaces the page with hanzo.ai/login plus that query.
hanzo.ai finishes it and the callback lands on the page that was asked
for.
Discovery is read from api.hanzo.ai and the token, revoke and logout
calls stay on this origin's /v1/iam, so nothing calls hanzo.id. Sign-out
ends the session here with the SDK's end(), which @hanzo/iam 0.21.31
has. The Status page stops printing the issuer.
zip's registration is a verb method on the group, so an operation reads as the
group answering it rather than a function taking the group, and a route that is
not a typed operation says which method it answers with. The package-level form
and the Router interface it existed to accept any target through are both gone.
The toolchain and the pin move together, deliberately. Below zip v1.36.50 the
JSON encoder was selected on a build tag, so a repo whose go directive rises
first writes a wire that is neither the old release's nor the new one's until the
pin catches up.
Co-authored-by: Hanzo Dev <[email protected]>
The encoder zip used was chosen by a build tag Go 1.27 sets by default, which
moved omitempty, nil slices and nil maps in every response. This release
speaks one wire format under either toolchain.
Co-authored-by: Hanzo Dev <[email protected]>
typescript 7.0.2 is the native compiler, so this is a pin and not a second
toolchain. frontend tsc --noEmit: 0 diagnostics before and after.
Co-authored-by: Hanzo Dev <[email protected]>
Both modules declare go 1.27.1 and the build images follow. go fix applied
the modernizers: promoted-field keys in struct literals, SplitSeq,
strings.Cut, new(expr), slices.Contains and any. The ptr helper had no
callers left once new(expr) could say it. omitzero is not applied; it
changes what the JSON says, so it is a per-field decision.
Co-authored-by: Hanzo Dev <[email protected]>
Neither the forge nor GitHub holds a run for it. A file that never
fires still evaluates on every replayed ref, so it is a hazard and
not a lane.
Co-authored-by: Hanzo Dev <[email protected]>
All six lived in .hanzo/workflows, which github.com does not read, so with
github.com canonical they dispatched nothing. They move to .github/workflows,
the one path both planes read.
That move alone would have been worse than the problem. release.yml built
three of its four targets on GitHub's hosted machines — ubuntu-24.04-arm for
linux/arm64 and macos-14 for both darwin targets. On the forge those simply
never ran, since no such runner exists there; collected by github.com they
would have started billing hosted compute on the first tag.
Each now names a host we own: linux/arm64 on spark, and both darwin targets on
dbc, which is arm64 and cross-compiles the amd64 one — the build is Go, so the
toolchain does that natively.
Release / Build binaries (darwin/amd64) (push) Failing after 0s
Release / Build binaries (darwin/arm64) (push) Failing after 0s
Release / Build binaries (linux/arm64) (push) Failing after 0s
The custody store had no way out of a running KMS: Embedded.db is
unexported and there was no accessor, so cloud could not construct one at
all. (*Embedded).Custody() closes that.
It hands out the STORE rather than the database it lives in and never the
master key: a caller that needs to enrol a node has no business reading the
secret keyspace, and the key stays inside this module. It refuses when
KMS_MASTER_KEY_B64 is absent or malformed, because nothing can be sealed
without it and a custody surface that cannot seal must refuse rather than
come up holding node wallets under something weaker.
One store per Embedded, memoized on the struct rather than in a package
variable. The store's lock is what serializes enrolment and rotation, so a
second store over the same database would let two rotations interleave and
leave a subject with two active identities.
Co-authored-by: Hanzo Dev <[email protected]>
Release / Build binaries (linux/arm64) (push) Failing after 0s
Release / Build binaries (darwin/arm64) (push) Failing after 0s
Release / Build binaries (darwin/amd64) (push) Failing after 0s
A Hanzo node is named by a wallet: one secp256k1 keypair is its identity on
the settlement L1, the account it stakes from, the account its work rewards
pay to, and the row the fleet view lists it under. One key with four uses is
a key whose copy is a forged identity and a spendable stake, so the node
does not hold it. The KMS generates it, seals it, and returns it to nobody;
the node asks for signatures.
The measurement is not compared, it is required. A key is sealed with
store.Seal at path="node", name=address, env=measurement, so the image is
the envelope's associated data, and unseal overwrites all three coordinates
with what the CALLER presents. A wrong image is a GCM authentication
failure, not a policy check that a later edit could drop.
The signing surface is a closed set. Every digest is keccak-256 over a
fixed-length preimage led by a purpose tag and committing to the address,
the sealed measurement and a strictly increasing epoch, so no digest minted
here is an RLP transaction hash: the wallet cannot be spent through this
surface, and a captured signature cannot be replayed forward. Signatures are
canonical (low-s), so one statement has one encoding.
Rotation is a change of identity, because a wallet's key cannot change under
a fixed address: the outgoing key signs a succession naming its successor,
that statement is recorded where it stays readable, and the outgoing
material is destroyed in the same transaction. Revocation destroys material
and keeps the row, so a slashed node stays attributable.
custody is a library, not a surface. It holds keys and enforces what is a
property of a key; it derives no permission from a credential and never sees
one. That follows the division this repository already made for secrets: it
answers who signed a token, and the org-scoped plane with its authorization
lives in cloud (apps/kms). Paths therefore carry no tenant — the server
reads it from the credential, as the secret surface does. custody/mount_test.go
is the reference mount, written out in full and driven by the real client, so
the serving side has a worked example rather than a description.
The contract has one definition. sdk/go/node holds the statements, the wire
types and the verification routine, with no store, transport or server
dependency; the KMS, the node daemon and the chain indexer all import it,
and custody builds its digests by calling the same package a verifier does.
Callers reach the surface through sdk/go/kmsclient rather than hand-rolling
requests, and refusals arrive as *kmsclient.Failure carrying the status.
Verification takes the payload the caller already holds — a receipt carries
the measurement it attests but never the payload it was given.
On hardware without SEV-SNP or TDX the node declares its measurement, which
pins the key to one image and detects drift but does not prove the image.
A claim of a hardware TEE with no linked verifier is refused rather than
downgraded: recording "attested" on an unchecked claim would make the fleet
view a lie. docs/custody.tex states what is enforced today against what a
TEE would add, and what a caller is assumed to do.
Co-authored-by: Hanzo Dev <[email protected]>
Release / Build binaries (linux/arm64) (push) Failing after 0s
Release / Build binaries (darwin/amd64) (push) Failing after 0s
Release / Build binaries (darwin/arm64) (push) Failing after 0s
build names the two binaries the Dockerfile ships (kmsd + kms) by calling
their existing recipes rather than repeating them; lint aliases vet, the one
static check this repo runs. help is the first target, so bare `make` now
prints the target list instead of building kmsd.
clean is unchanged — it already removed exactly the two binaries.
Co-authored-by: Hanzo Dev <[email protected]>
Both binaries are CGO_ENABLED=0 and statically linked, and Debian was supplying
three things: ca-certificates, which is data; an account, which is two files the
kernel reads to name a uid it already enforces; and curl, which existed for one
line.
That matters more here than elsewhere. This is the service holding the estate's
secrets, and the base was the largest thing in it nobody here wrote — apt, dpkg,
a shell, coreutils, glibc, each on its own upstream and CVE feed, none of it ever
executed by kmsd. A credential store should be the smallest reviewable surface in
the fleet rather than the largest.
The HEALTHCHECK is deleted rather than given a static curl: every deployment runs
under Kubernetes, whose probes ask /healthz from outside. An HTTP client inside a
secret store, so it can ask itself what the orchestrator already asks, is a poor
trade.
'make test' ran 'go test ./internal/...' and this repo has no internal/ — the
target failed setup every time, so the gate reported failure regardless of the
code and nobody could tell a real break from the missing directory. It now runs
./..., which is what the tests were always under (github.com/hanzoai/kms and
cmd/kms both have them).
Split from the zip bump on purpose: that bump needed a working gate to be
verified against, and a dependency commit should not quietly carry a build-system
fix.
Co-authored-by: Hanzo Dev <[email protected]>
Ranges drift once nothing moves them: a caret that has not been rewritten
since it was written pins whatever was current that day, and an exact pin
cannot even take a patch. Both are updated here.
One line per package, so a surface reads the same components, the same
tokens and the same face as every other one.
runs-on was a bare scalar. Jobs written that way queue and are never
picked up — link and quest have waited since 10 August, painter since the
5th, none of them with a single success — while repos writing the same
label as a sequence drain within seconds of a push on the same fleet.
Same label, same runners, different shape. It fails silently: nothing errors,
the job simply never starts, so the lane looks configured and does nothing.
The social card asked for Inter first. That is not a face this estate
licenses, ships or ever drew, so a renderer with Inter installed painted
the wrong face and one without it silently fell through to system-ui.
Zen leads now; the same tail follows it.
A docs-site template repeated across these repos named three different faces in
one place: a binding called `inter` or `geist`, constructed from next/font/google
so it actually held Inter, publishing a custom property called
`--font-geist-sans` that @hanzo/font does not set. Whichever of the three a
reader trusted, two of them were wrong.
They take Zen and ZenMono from @hanzo/font — bindings named `sans` and `mono`
for the role they fill, setting --font-zen-sans and --font-zen-mono, which is
what the package publishes.
frontend/public/favicon.svg was a #d81c33 rounded square with a hand-drawn H path. Red is the accent, not the mark. It is @hanzo/logo's dist/favicon.svg now — the same file every other Hanzo surface serves.
LLM.md is the source and stays the source; CLAUDE.md is a symlink to it, so there is
one file and it cannot drift from itself. What changes is that the link is committed
rather than left for each checkout to recreate.
Left untracked, the doc was conditional on a step outside the repo: a fresh clone had
no CLAUDE.md, nothing loaded LLM.md, and an agent worked here having never read what
this repo already explains. Most of the estate already tracks theirs; this is the rest
agreeing with them.
This package kept its own HTTP secret plane — routes, an audit log, version CAS
and a JWT gate — beside the one cloud already serves at
/v1/kms/orgs/{org}/secrets. Two doors onto the same store means two places
where a tenant boundary can be got wrong, and only one of them was reviewed as
the authorization surface.
What is left is storage and the ZAP transport, whose authorizer is a signed
consensus-authority snapshot and which stays disabled unless that snapshot is
present. This package now makes no authorization decision at all: IAM owns
identity, cloud folds the caller's validated org into the storage key so one
tenant's path cannot name another's record, and secrets are sealed at rest with
the replicator streaming encrypted backups to S3.
go.sum carried a stale pair for luxfi/vm v1.3.1 — the tag was republished, and
the entries here predated it, so nothing in the module could build. The values
now match the public checksum database, which six other repos in the estate
already agreed with.
Co-authored-by: Hanzo Dev <[email protected]>
A caller has to tell "this tenant brought no key" from "the store could not
answer": the first says fall through to another custody, the second says stop.
Both arrived as untyped errors, so the only way to ask was to read the text —
and the non-200 branch echoes the response body, so any fault whose body
happened to say "not found" was read as an absent secret and sent the caller
somewhere less guarded.
ErrSecretNotFound was already declared, with a comment promising callers would
branch via errors.Is. Wrap it on the three paths that mean a secret is genuinely
absent. A tenant miss keeps its own untyped error: it is a fault, not an absence,
and reporting it as one would restore the bug in a new place.
Co-authored-by: Hanzo Dev <[email protected]>
luxfi/kms v1.12.16 gives the hybrid handshake a second output beside the
session key: a binding that names the channel, computable only by the two
endpoints that ran it. Every envelope carries the binding of the session
it is sent on, and the server honours it only there. A request carried to
some other peer is inert — it cannot be re-signed, and no other channel
can produce the name it holds.
RequireSession goes away with it. A client without a session now has
nothing to address a request to and cannot speak at all, so the dial
agrees a hybrid session or fails; that is the shape of the code rather
than the value of a field.
List builds each answer from the coordinate the record came back with.
A path is a subtree root, so a result can sit below the one that was
asked for, and re-joining the query prefix would name it somewhere it is
not.
Co-authored-by: Hanzo Dev <[email protected]>
The HTTP path addressed /v1/kms/orgs/{org}/secrets. The server mounts
/v1/kms/secrets and takes the org from the credential the request is made with,
so every HTTP-transport call reached nothing — measured against the live edge,
where the mounted path answers a JSON 403 and the org path a bare 404.
Reading the tenant from the credential is also the only place it can be trusted
from: a segment is a name the caller picks, and one identity naming another
org's secrets should not be a request this client can compose.
The four tests that covered these calls asserted the org path against a mock
server, which answers whatever path it is sent — so they were green the whole
time the transport was broken. They now pin what the server mounts, and a test
beside them fails if an org ever steers the URL again.
Co-authored-by: Hanzo Dev <[email protected]>
Everything this client moves is a secret, so the dial now requires the
X25519 + ML-KEM-768 handshake to have produced an AEAD session, and
requires that session to have run ML-KEM rather than X25519 alone. A peer
that agrees neither gets no client and no request.
Hybrid rather than post-quantum alone because both halves have to fall
before the channel does: a break in the elliptic curve and a quantum
computer are two different futures, and a key read today is stored today.
It is a constant rather than a setting. There is no caller here for whom a
plaintext channel is the right answer, and a setting is a thing an
adversary gets to influence by making the handshake fail.
luxfi/kms v1.12.15 for the RequireSession dial, luxfi/keys v1.4.2 for a
service identity that keeps its signing key past the first signature.
Co-authored-by: Hanzo Dev <[email protected]>
Work landed on GitHub while CI reads the forge, so each side carried commits the
other did not: the forge had the module-proxy fetch, the recorded go.sum and the
sdk/go minio removal; GitHub had kms building its own zip.App. A force-push
either way would have dropped one of them silently.
go.mod conflicted only on version skew — zap v1.2.7/zapdb v1.10.0 against
v1.2.6/v1.10.6 — resolved to the higher of each, which is what version selection
picks anyway. go.sum was reconciled by re-tidying rather than by choosing a side.
Before that, every module@version the two go.sum files shared (339 in the root,
275 in sdk/go) was checked for hash disagreement: none, so no re-tag hid here.
go build ./... clean, go test ./... green.
Co-authored-by: Hanzo Dev <[email protected]>
github.com/hanzoai/cloud lost the v0.x tags it was published under — GitHub
carries v1.801.31 and nothing else — so GOPRIVATE, which implies GONOPROXY,
sent go mod download to a revision that is no longer there and build-amd64
died on "unknown revision v0.1.1". The proxy still holds it, and holds the
same bytes: the zip hashes to the h1: already in go.sum.
Dockerfile.kms-fetch never had the problem because its env assignments bind
to the git config they precede rather than to go mod download, so its graph
has always come through the proxy. Deleted there too, so both files read the
way they behave.
The coupling was one string. Mount(app, deps) took hanzoai/cloud's Deps and
read exactly two fields out of it: a logger, which luxfi/log already publishes
as a process default and which Embed has always used, and a data root.
Everything else in that struct crossed the boundary to be discarded. An init()
then pushed this package into cloud's registry so cloud could call Mount back.
That points the dependency the wrong way. It makes the graph circular, since
cloud imports kms in order to mount it, and it fixes the set of possible hosts
at one: two editions of hanzoai/cloud declare the same module path, so
cloud.Deps names a different type in each, and a package that takes it can be
composed by one of them and never the other.
kms.App(root) now builds the app and returns it, and a host composes it with
app.Use. Routes are absolute, so composition neither moves nor shadows them;
the bridge test runs through a host app with middleware in front of it to hold
that. Teardown travels with the app rather than with the process: the Embedded
server drains on a shutdown hook, and zip makes a parent drain the children it
composed, so kms.Shutdown and the package global it read are gone. The readyz
branch that answered 503 when that global was nil went with them, since a
failed Embed returns an error and no app at all.
The daemon read the host's config too, and that was a live bug: it bound
$CLOUD_LISTEN, default :8080, while its own image sets KMS_LISTEN=:8443 and its
HEALTHCHECK curls 8443. It now reads its own, through the one function that
resolves that address for both the daemon and Embed.
zip moves from v1.10.0 to v1.27.2, the version the host already selects,
because Use taking a Component is what makes an app composable at all.
middleware.Logger and the KMS_LOG_LEVEL=debug opt-in around it go with the
bump; zip logs every request itself now, so that report is no longer something
a binary can be missing.
Dropping cloud drops what it dragged in. goja, esbuild and a sourcemap parser
leave the graph: a secret store no longer links a JavaScript engine.
Both go.sum files carried a pre-retag hash for luxfi/vm v1.3.1 that no longer
matches the bits the proxy serves, which failed every build outside the
workspace. Corrected against the checksum database, which agrees with the
download.
Co-authored-by: Hanzo Dev <[email protected]>
Every build job passed and only the release job failed, with:
::error::404 page not found
`generate_release_notes: true` makes softprops/action-gh-release call
POST <api>/repos/<owner>/<repo>/releases/generate-notes
which is a GitHub-only API. Measured against this forge:
GET /v1/repos/luxfi/consensus/releases -> 200
POST /v1/repos/luxfi/consensus/releases/generate-notes -> 404 "404 page not found"
The body is Go`s bare 404 text, which the action surfaces verbatim -- so the
error names no endpoint and reads as a broken release step or a bad token. The
base path was never wrong: the forge serves its API at /v1 and v1.26.29 already
advertises that (services/actions/context.go builds api_url as AppURL + "v1").
Releases still publish; only the auto-generated notes are gone, which the forge
cannot produce anyway.
Co-authored-by: Hanzo Dev <[email protected]>
type=gha resolves its backend from an Actions runtime token that git.hanzo.ai
does not issue in a form buildx can parse, and the exporter is configured as
part of the solve -- so it fails the build rather than degrading to an uncached
one. Same defect fixed in hanzoai/o11y. ignore-error keeps a cold cache from
404ing the first build.
Co-authored-by: Hanzo Dev <[email protected]>
The docker job runs on hanzo-build-linux-amd64, so `secrets.GITHUB_TOKEN` is a
git.hanzo.ai token presented to ghcr.io, which has never seen it:
Error response from daemon: Get "https://ghcr.io/v2/": denied: denied
That pair is GitHub`s own idiom and correct there; it is simply not a credential
on a self-hosted pool. orgs/hanzoai provisions GHCR_USER + GHCR_TOKEN.
Co-authored-by: Hanzo Dev <[email protected]>
The module graph could not be loaded at all: transitive requirements name
versions whose tags were wiped or whose pseudo-versions point at commits that
history rewrites removed, so `go list -m all` fails with "unknown revision" or
"invalid pseudo-version". Builds still passed because those modules supply no
package that gets compiled — the breakage is graph-level, which is exactly the
kind that only shows up on a clean checkout.
`exclude` is the mechanism for "this version is unusable": MVS then selects the
next higher one instead of trying to fetch a revision that is gone. Bumping the
requirement alone does not work, because Go still reads the excluded version
go.mod while building the graph.
Co-authored-by: Hanzo Dev <[email protected]>
Both the zip and the go.mod hash recorded here were captured from a direct VCS
fetch made while GOPRIVATE covered github.com/luxfi/*, which switches off the
proxy and checksum verification together. The tag moved afterwards, so what we
wrote down no longer matches sum.golang.org.
proxy.golang.org still serves the originally-published bytes and they verify
against the log, so dropping the two lines and re-tidying restores the build
with no version change.
Co-authored-by: Hanzo Dev <[email protected]>
The description already said so; the README still opened by describing a live
product. A reader landing here now sees where it went before anything else.
This workflow runs on git.hanzo.ai, which presents as GHES, and the v4 action
refuses before doing any work:
GHESNotSupportedError: @actions/artifact v2.0.0+, upload-artifact@v4+ and
download-artifact@v4+ are not currently supported on GHES.
v4 moved to an artifact API the forge does not implement, so this is a hard
refusal rather than a transient upload error — retrying could never help. The
damaging part is how it presents: the build step passes, the artifact step
fails, and the job reports "failed", so a green build reads as a broken one and
the artifact is silently never produced.
Found on hanzoai/chains, where every VM in a 15-way matrix compiled and then
discarded its binary at this step. Swept the estate for the same pattern.
Co-authored-by: Hanzo Dev <[email protected]>
The first pass left three classes of config that BOTH tsc 5.9 and tsc 7
reject. Each was proven against both compilers before changing:
TS5090 A `paths` target must be relative once `baseUrl` is gone. The
first pass skipped the "./" prefix wherever baseUrl pointed at
the config's own directory, reasoning it was semantically
equivalent. It is not — without baseUrl a non-relative target
is rejected outright, by 5.9 as well as 7.
TS5110 `moduleResolution: node16` requires `module: node16`. The first
pass mapped commonjs projects to node16 resolution alone, which
BROKE those configs for the current toolchain. Both are now set.
TS5102 `downlevelIteration` is also removed in TS7; it was missing from
the dead-flag list.
Verified: repos that tsc 7 previously refused (base-studio, js-sdk, kv-js)
now report zero config errors on tsc 7 AND tsc 5.9.
Co-authored-by: Hanzo Dev <[email protected]>
TypeScript 7 is the native Go compiler and removes `baseUrl` and
`moduleResolution: node|node10`. Both appear here, so `tsc` from TS7
refuses the config outright (TS5102 / TS5108) and cannot typecheck.
`paths` targets resolve relative to `baseUrl` when it is set and relative
to the tsconfig file otherwise. Every `baseUrl` folded here already
pointed at the config's own directory, so dropping it moves nothing and
the targets are left byte-identical. Where a baseUrl pointed elsewhere,
each affected target was rewritten as join(baseUrl, target).
`moduleResolution` was chosen from the declared `module`: commonjs ->
node16, esnext/preserve -> bundler. Configs whose `module` is unset or
exotic were left alone rather than guessed at.
The result is accepted by BOTH toolchains, so nothing has to upgrade
TypeScript in lockstep. Verified on hanzo/chat packages/api: tsc 5.9
779 -> 778 errors (no regression), and tsc 7.0.2 now runs the project
in 2s where it previously refused the config.
Co-authored-by: Hanzo Dev <[email protected]>
Superseded: the Hanzo GitHub App pushes a webhook, so the forge tracks GitHub
without a per-repo workflow. This file called git.hanzo.ai/api/v1/.../mirror-sync
— a Gitea API for a system we no longer drive — and would sit inert in every repo.
One mechanism, in one place, instead of ~350 copies of a cron.
http v0.3.0 replaces the JSON header map in the ZAP frame with
length-prefixed name/value pairs ([u32 count] then [u32 nameLen][name]
[u32 valueLen][value]), taking decode to 0 allocs/op. That is a wire
break — a v0.2.x peer cannot talk to a v0.3.0 peer — so every Hanzo
module moves together. Deployment is operator-managed GitOps, so
merging aligns the code without shipping anything.
No source change was needed: mount.go already fronts the net/http
subtree with app.All("/v1/kms/*", zip.AdaptNetHTTP(h)) rather than the
removed App.Mount(prefix, http.Handler), and this module builds no ZAP
transport of its own.
That bridge was the one path the frame change touches and the one path
no test covered — every other suite calls the handler directly, in
process, so a framing regression would have landed silently.
TestMountBridge_NetHTTPOverZIP now serves the mount.go route shape over
a real TCP listener and drives it with a real net/http client, asserting
method, path, query, request headers, body, status, response headers
and repeated response headers all survive the round trip. Verified live
by mutation: drop the request header at the client and the test fails
with `request hdr = "" ... did not survive the bridge`.
Co-authored-by: Hanzo Dev <[email protected]>
zip v1.9.0 repurposes App.Mount(prefix, addr) as prefix -> remote-address
proxying (unix/tcp ZAP), so it no longer takes an http.Handler. The
in-process KMS surface goes on the documented net/http bridge instead:
a wildcard route carrying zip.AdaptNetHTTP — byte-identical to what the
old Mount expanded to (All(prefix+"/*")), so /healthz and /v1/kms/* keep
resolving exactly as before, bare prefix included (fiber marks a trailing
greedy wildcard optional).
Also pulls zap-proto/fiber/v3 v3.2.1 in place of gofiber/fiber/v3 v3.2.0
as zip's router, transitively.
go build + go test green across every package this module owns
(github.com/hanzoai/kms, cmd/{kms,kms-fetch,kmsd,smoke-zap}) and the
sdk/go module.
Co-authored-by: Hanzo Dev <[email protected]>
Four failures, both causes the same shape: state that belongs to the whole
process being read or written as if it belonged to one test.
TestEnvRequired_* (x3) failed with 401 on requests whose tokens were fine.
Embed applies ExpectedIssuer/ExpectedAudience/JWKSURL to the process-global
authConfig — that is what Embed is FOR — and stopping the instance does not put
it back. So after TestEmbed the process no longer accepted sharedIssuer, and
verification reached for a JWKS server TestEmbed had already closed. Every later
test minting a shared-issuer token got 401, and the only reason it was those
three is that they sort after TestEmbed. They passed in isolation, which is
what made it look like their problem.
jwtTestEnv.cleanup already restores exactly this; the embedded path had been
missed. Registered before jwks.Close so it runs last (t.Cleanup is LIFO).
TestConnMemory measured runtime.NumGoroutine() — process-wide — and divided it
by the connections under test. Run alone that ratio is 1.00; run inside the
package it also counts badger compactors, httptest servers and whatever else is
still up, and it read 1.15 against a [0.95, 1.05] band. The test was failing for
goroutines it does not own. It now takes a goroutine baseline and reports the
delta, which is what the heap half of the same function has always done two
lines above.
Package green, twice.
Co-authored-by: Hanzo Dev <[email protected]>
git.hanzo.ai is canonical and reads .hanzo/workflows, so build.yml,
build-kms-fetch.yml and pr-preview.yml move there by changing directory and
nothing else — same GitHub Actions syntax, same runs-on, byte-identical files.
Three more lanes had been deleted from .github/workflows without a replacement
anywhere, which silently removed this repo's only gate and the only producer of
its release binaries and semver image tags. There was no red run to show it,
because a workflow that does not exist cannot fail. Restored byte-for-byte from
history at the path the forge reads:
ci.yml go test -race ./... + builds kmsd and kms
release.yml v* tags: 4-way binary matrix, release, multi-arch image
check-fe-ts-and-lint.yml frontend typecheck on PRs
release.yml's notify-universe job is left exactly as it was; it is already
continue-on-error, so a missing UNIVERSE_DISPATCH_TOKEN on the forge degrades to
a yellow job instead of failing the release.
Not restored: validate-pr-title.yml, a conventional-commit title gate inherited
with an upstream PR template. Dropped: workflow-sanity.yml, which triggers on
paths .github/workflows/** and so can never fire again once the lanes live under
.hanzo. Neither builds, tests or publishes anything.
sync.yml is replaced with the canonical copy — the variant here predated it and
had drifted.
Co-authored-by: Hanzo Dev <[email protected]>
git.hanzo.ai mirrors this repo by PULL on a ~10-minute interval, and arcd runs
CI/CD there — so every push waited out that interval before anything built.
This asks Gitea to pull HEAD immediately.
Latency only: the repo already mirrors via the App webhook, so a missing
HANZO_GIT_TOKEN or a failed curl is non-fatal and never fails the push.
Idempotent (mirror-sync just pulls HEAD) and concurrency-coalesced.
This workflow ran on a self-hosted actions-runner-controller pool. ARC is
decommissioned: CI executes inside git.hanzo.ai (Gitea Actions, .hanzo/workflows)
and delivery is cd.hanzo.ai reconciling the reviewed image pin in
hanzoai/universe. GitHub is a mirror and runs nothing.
Removed rather than repointed — there is no GitHub-side pool to move to, and
leaving it would keep ARC load-bearing.
This workflow ran on a self-hosted actions-runner-controller pool. ARC is
decommissioned: CI executes inside git.hanzo.ai (Gitea Actions, .hanzo/workflows)
and delivery is cd.hanzo.ai reconciling the reviewed image pin in
hanzoai/universe. GitHub is a mirror and runs nothing.
Removed rather than repointed — there is no GitHub-side pool to move to, and
leaving it would keep ARC load-bearing.
This workflow ran on a self-hosted actions-runner-controller pool. ARC is
decommissioned: CI executes inside git.hanzo.ai (Gitea Actions, .hanzo/workflows)
and delivery is cd.hanzo.ai reconciling the reviewed image pin in
hanzoai/universe. GitHub is a mirror and runs nothing.
Removed rather than repointed — there is no GitHub-side pool to move to, and
leaving it would keep ARC load-bearing.
This workflow ran on a self-hosted actions-runner-controller pool. ARC is
decommissioned: CI executes inside git.hanzo.ai (Gitea Actions, .hanzo/workflows)
and delivery is cd.hanzo.ai reconciling the reviewed image pin in
hanzoai/universe. GitHub is a mirror and runs nothing.
Removed rather than repointed — there is no GitHub-side pool to move to, and
leaving it would keep ARC load-bearing.
kmsd built the OTLP HTTP exporter, which pulls gRPC and protobuf even on the
HTTP transport — otlptracehttp shares otlpconfig with the gRPC one.
luxfi/trace v1.3.0 exports over ZAP natively: a JSON SpanBatch inside a ZAP
envelope, read by hanzo/o11y/pkg/zapreceiver.
This module's graph is now free of both: otlp and google.golang.org/grpc are
each zero in go.mod. Knobs are O11Y_ENDPOINT / O11Y_TRACES_ENDPOINT /
O11Y_SERVICE_NAME, with O11Y_ENDPOINT=default resolving to o11y.hanzo.svc:4317.
Posture unchanged: opt-in, non-fatal, no dial at construction.
The Keys page labeled the two validator slots "BLS public key" and "Corona
public key". Those names do not describe the cryptography: the "BLS" slot is
secp256k1 signed via CGGMP21 threshold ECDSA (not bls12-381), and the "Corona"
slot is ed25519 signed via FROST (not post-quantum, not the Corona R-LWE
lattice protocol). Relabel to state the actual scheme so operators are not
misled into a bls12-381/PQ assumption.
Co-authored-by: Hanzo Dev <[email protected]>
Provably-dead exported accessor: no in-repo callers (only cmd/kmsd and
mount.go consume the root package) and no external importer of
github.com/hanzoai/kms across the tree uses it. Its backing httpAddr
field was write-only and its comment was wrong (stored the raw config
addr, never a resolved listener address). Removes field, assignment,
and accessor. Build+vet green.
The binaries job built with GOPRIVATE='' so go verified the first-party
luxfi/* modules against the public sumdb — force-moved upstream tags never
match, so every release failed with 'checksum mismatch / SECURITY ERROR'
(luxfi/[email protected]). Set GOPRIVATE + GONOSUMDB for luxfi/* + hanzoai/* and add a
git-config token (GH_PAT, GITHUB_TOKEN fallback) so the modules resolve directly
via git and skip the sumdb. Dockerfile: GONOSUMCHECK (removed by Go) -> GONOSUMDB,
guard the token, x-access-token form; docker job passes GH_PAT so it can read
luxfi.
Latest v1.x of the canonical KMS server. luxfi/keys (v1.4.1) and
luxfi/crypto (v1.20.2) were already at latest v1.x via the module graph.
luxfi/mpc stays out of the graph by design: threshold signing is delegated
over ZAP to an external MPC daemon (cfg.MPCAddr / MPCVaultID) through
luxfi/kms's pkg/mpc client; this module never links the standalone daemon.
GOWORK=off go build/vet/test green; test delta vs HEAD = 0.
Swap the audit-trail SQLite driver from modernc.org/sqlite to hanzoai/
sqlite, the dual-backend drop-in that registers the "sqlite" driver name
under both CGO (mattn+SQLCipher) and !cgo (modernc) builds.
- audit.go: blank import + storage doc comment rebranded.
- red_hardening_test.go: swap the test-side blank import too. Both files
had to move together: under CGO=1 hanzoai/sqlite registers "sqlite" via
mattn while a lingering direct modernc import ALSO registers it, causing
`panic: sql: Register called twice for driver sqlite` at test init.
Swapping both leaves a single registration under both build configs.
Security: SQLite in KMS backs ONLY the plaintext audit trail (KMS_AUDIT_DB);
secrets are MPC-rooted and never stored in SQLite. The audit DB is opened
with a nil key via sql.Open("sqlite", ...&_pragma=...), so it stays
plaintext under both backends — the swap does not change the at-rest path.
DSN is already modernc-style (_pragma=NAME(VALUE)); no conversion needed.
Co-authored-by: hanzo-dev <[email protected]>
The kms Go SDK module directly imports luxfi/keys (NewServiceIdentity /
ServiceIdentity). Its v1.2.0 pin was force-moved; adopt the stable v1.4.1,
regenerate go.sum from the current remote. Builds clean (CGO_ENABLED=0).
Co-authored-by: Hanzo Dev <[email protected]>
kms pinned luxfi/keys v1.4.0 whose remote tag was force-moved (content
5153d639→80a3745a → go.sum checksum mismatch), plus zap-proto/http v0.1.0
(also force-moved) and geth v1.16.98 (deleted tag). Only keys v1.4.1 is
byte-stable; adopting it floors the unified luxfi stack and regenerates
go.sum from the current remote. kmsd + kms (pure-Go CGO_ENABLED=0, ZapDB)
build clean; the ServiceIdentity surface (keys.NewServiceIdentity) is
unchanged (v1.4.1 = local v1.4.0 identity work + geth bump).
go.mod/go.sum only.
Co-authored-by: Hanzo Dev <[email protected]>
env is a first-class component of the storage key
(kms/secrets/{path}/{env}/{name}); a POST/PATCH that omitted env used to
silently default to "default", committing the write to a bucket that
project/env/path readers (kms-operator, cluster syncs) never resolve. That
split landed an IAM z-password in env=default while prod kept serving the
stale value. POST/PATCH now fail loud with 400 when env is omitted;
GET/DELETE/LIST keep a compat default for legacy readers (a read/delete
can't plant a value another reader trusts).
Root cause on the client: sdk/go/kmsclient dropped env on every HTTP op
(c.env was used only on the ZAP path), so a client configured for env=prod
still wrote to env=default over HTTP. The HTTP path now sends c.env on the
write body and as ?env= on read/delete, matching the ZAP path. Built
in-tree via a local replace so cmd/kms and cmd/kms-fetch ship the fix; the
two Dockerfiles stage sdk/go/go.mod so `go mod download` resolves the
replaced module.
Regression tests: write without env -> 400; write env=prod is readable via
the operator's project/env/path resolution (sha256 round-trip, values never
printed) and is not visible in env=default.
Co-authored-by: hanzo-dev <[email protected]>
Adopt the canonical Hanzo Node base image. Node 24 uniform across the
fleet; sqlite3 bundled (node:sqlite builtin + native better-sqlite3
toolchain). One base, one way.
Co-authored-by: Hanzo Dev <[email protected]>
The zip access-log middleware logged one Info "request" line per call —
~4k lines/min, the entire KMS log volume in ClickHouse. Startup, shutdown,
authz-denials (kms_auth_reject), and handler errors are logged elsewhere and
stay at info. Default (info) is now quiet; opt back in with KMS_LOG_LEVEL=debug.
Co-authored-by: hanzo-dev <[email protected]>
IAM moved its OAuth token endpoint to /v1/iam/oauth/token; the login proxy
(and the Go SDK client) still called the retired /v1/iam/login/oauth/access_token,
which now 404s — so /v1/kms/auth/login returned 401 for every valid machine
identity. This broke fresh KMSSecret-operator logins and the hanzoai/ci deploy-
cred fetch. Point both at /v1/iam/oauth/token (form/body unchanged); update LLM.md.
Note: requires a KMS image release + redeploy (ghcr.io/luxfi/kms) to take effect.
Env-gated on OTEL_EXPORTER_OTLP_ENDPOINT; non-fatal; clean no-op when unset (safe to ship before the collector is live). Installs the global tracer provider with a service.name resource so the console Monitoring tab filters this product. Mirrors ai/object/telemetry.go. Traces-only; metrics/logs are a tracked follow-up.
The kms.hanzo.ai "/api/v1/* returns 200" signal was a false positive: the
frontend catch-all (registerFrontend) served index.html for any unmatched
GET, including /api/v1/auth/universal-auth/login. No /api/ route is
registered anywhere in this repo — the server is already 100% /v1/kms/*
native (Go mux, both Go SDKs, and the React frontend all use /v1/kms).
The catch-all's defense-in-depth bail-out excluded /v1/, /healthz, /health
but not /api/, so an /api/-shaped GET fell through to the SPA fallback and
answered 200. This made a nonexistent legacy /api/ backend look live.
Add /api/ (and bare /api) to the bail-out so it 404s at the wire. The ONE
canonical prefix is /v1/kms; there is no /api/, and now the server says so
unambiguously. No alias, no rewrite, no backward compat.
frontend_spa_test.go locks the invariant: /api/* → 404, /v1/ and /health →
404, /healthz → 200 (real handler wins), client routes + assets → 200.
Co-authored-by: Hanzo Dev <[email protected]>
Pure move to the ZAP-family canonical framework home + the ListenHTTP rename
(verb-first, was Listen). go.sum re-recorded against the immutable public proxy.
Build green.
verifyJWT compared the token iss against the ENTIRE IAM_ISSUER string, so a
comma-list ('https://hanzo.id,https://iam.hanzo.ai,https://hanzo.ai') — the
production config — rejected every real token as wrong_iss. (The 2.x line
carried this split; it was lost in the 0.159.x re-baseline.) Mirror
checkAudience: split IAM_ISSUER on ',' into an allowlist, drop the single-value
jwt.WithIssuer, enforce via checkIssuer post-parse (trailing-slash normalized).
Single-issuer + wrong/attacker/missing-iss rejection unchanged.
Test: TestJWT_MultiIssuer_Accepted (3 listed issuers accepted, evil rejected);
existing F1 issuer-rejection + F4 multi-audience + TestList suites still green.
The image build-stage runs 'go mod download' WITHOUT GOPRIVATE in effect
(the env prefix binds to the preceding git-config, not the download), so
public luxfi modules resolve via proxy.golang.org (immutable version
content), while upstream has force-moved several luxfi tags so DIRECT git
now serves different bytes. Regenerate ALL luxfi sums via the same proxy
path the build uses (keys +AriQNM7, zap MiVRsRA8, ...). Gate: a
Dockerfile-identical 'go mod download' + CGO_ENABLED=0 build + TestList
suite verify against the regenerated go.sum.
The Dockerfile build stage runs 'go mod download' WITHOUT GOPRIVATE in effect
(shell: 'VAR=x git config ... && go mod download' applies VAR only to the
git-config, not the download), so luxfi/keys (public) resolves via
proxy.golang.org, whose v1.2.0 cache is immutable (+AriQNM7...). Upstream
later force-moved the v1.2.0 tag, so DIRECT git now serves different content
(3TAcr4tw...). The image build verifies against the proxy value; pin go.sum to
it. /go.mod hash already matches (build passed graph-load, failed only on zip).
go mod tidy and go mod download resolve the re-cut luxfi tags to different
commits; the image build uses 'go mod download', which stably resolves
luxfi/[email protected] to the +AriQNM7 content. Regenerate luxfi sums via the
build path so the Dockerfile 'go mod download' verifies. Proven in-job:
go mod verify + fresh readonly download + CGO_ENABLED=0 build of kmsd/kms
+ TestList endpoint suite all pass.
The recorded luxfi/* hashes were generated against an internal mirror
that still serves pre-force-move content; github.com (what CI and the
image build fetch) serves the re-cut tags. Regenerate luxfi sums via
'go mod tidy' against github.com so 'go mod download' verifies in the
build environment. go.mod versions unchanged; go mod verify + CGO_ENABLED=0
build of kmsd/kms + TestList endpoint suite all pass.
Add a bare list endpoint so an operator console can browse an org's secret
KEYS without ever exposing a value. Structurally incapable of leaking a value:
the scan iterates ZapDB KEYS ONLY (PrefetchValues=false) under
kms/secrets/<brand/org>/ and the row type has no value field — there is no
code path from this handler to a secret value blob.
Verified against the store: secret path/name/env are PLAINTEXT in the ZapDB
key kms/secrets/{path}/{env}/{name}; only the value blob is at-rest encrypted.
So names are honestly listable; values are never touched.
- Authz: identical authorize()+canActOnOrg() gate as get-one — never weaker.
Structurally confined to the org's brand/<org>/ namespace (stricter: cannot
enumerate another tenant's names). Optional ?prefix= (safePath + org
containment) and ?env= filters.
- Routing: bare GET coexists with the {rest...} wildcard (Go 1.22 ServeMux);
the bare path hits list, .../secrets/<path>/<name> still returns its value.
- updatedTime sourced from a new kms/mtimes/ sibling index (mirrors
versioning.go), written on put/patch, deleted on delete — never from the
value blob. Records predating this carry no mtime (updatedTime omitted),
never faked.
- TestRed3 traversal heuristic tightened: '/secrets/foo/..' path-cleans to
'/secrets' (the list), an authorized empty {secrets:[]} with no value; the
test now asserts on an actual value payload, not the substring 'secret'.
Tests: 9 new TestList_* (no-value-leak, authz fail-closed, routing
precedence, prefix injection-safety, env filter, empty org, mtime RFC3339).
VERSION 2.5.2 -> 0.159.4 (patch, v0.x server line; tag cut on merge, not now).
origin/main pins age v1.5.0 / keys v1.2.0 / zap v0.8.1 whose upstream tags
were re-cut (brand-scrub), so the recorded h1 content hashes no longer match
and `go build` fails closed with a checksum mismatch. Recompute the three
hashes from the actual module content (go, not a hand-copy) and drop pq's
stale zip hash (its code isn't compiled on this build graph — module-graph
pruning). kms stays v1.11.7 (origin/main's pin, untouched). `go mod verify`
=> all modules verified. Pre-existing drift, also fixed independently on
fix/kms-cli-build-luxfi-keys-gosum; values agree byte-for-byte.
Deliberate single-patch bump of the canonical backend to pull in the derived
<org>-kms OIDC client (KMS computes client_id/secret/state from KMS_IAM_OWNER —
no env coordination with IAM). hanzoai/kms is a thin wrapper; it pins luxfi/kms
intentionally rather than tracking latest.
Replace inline upstream-brand mentions in DEPRECATED.md, LLM.md, and a
frontend gap description with neutral 'legacy fork' wording. Add a NOTICE
file carrying the required upstream MIT attribution. The LICENSE retains
the upstream copyright notice as required by the MIT license; that is the
only place the upstream name remains.
No functional changes. Go backend and frontend build unchanged.
The current KMS is a pure Go binary. Per DEPRECATED.md the Node.js +
PostgreSQL + Redis Infisical fork was replaced — "no Node.js / Postgres /
Redis dependencies." The KMS Go runtime contains zero Redis references;
the `REDIS_URL` / `REDIS_HOST` knobs in these examples were leftover from
the deprecated stack and are never read.
Remove the orphaned Redis declarations from the three HSM compose examples
and the Zymbit k8s manifest:
- zymbit-hsm/compose.yml: redis service, REDIS_URL env, depends_on, volume
- zymbit-hsm/kubernetes-deployment.yaml: Redis Deployment/Service/PVC,
REDIS_HOST/PORT ConfigMap keys, redis egress NetworkPolicy rule
- aws-cloudhsm/compose.yml: redis service, REDIS_URL env, depends_on, volume
- google-cloud-hsm/compose.yml: redis service, REDIS_URL env, depends_on, volume
No consumer migration needed — nothing read Redis. Postgres left intact.
zeekay
