Compare commits

...
83 Commits
Author SHA1 Message Date
zeekay ddd3df4675 Merge the GitHub line: one history, so a tag move survives the sync
Hanzo CI/CD / cicd (push) Failing after 9s
CI/CD / gate (push) Failing after 10s
hanzoai/ci had two diverged mains — five commits only on github.com (the
vendormark and site-zip-publish lines) and seven only on git.hanzo.ai (the
delegate gate, the publishable rule, the registry-credential fix). Their
merge-base was f098b39, which is exactly where v1 and v2 still pointed, so the
drift was invisible: both tags named the last commit the two lines agreed on.

That is what reverted the first attempt at this. Moving v1/v2 on the forge alone
held for three minutes and was then pushed back to f098b39 by the sync that
mirrors github.com into the forge — runs 101 and 102 are the tags reappearing at
the old commit. A tag is a ref like any other; if one side of a mirrored pair
does not have the commit, the sync restores what it does have. So the histories
have to converge before the tag can move anywhere.

The single conflict was two correct changes to the same lines, and both are
kept. The registry credential is fetched UNCONDITIONALLY (this line): it is not
an object-store credential, and a repo declaring only `images:` was reaching
docker login with two empty strings. The object-store gate narrows to `bucket:`
only (the github line): the site lane no longer writes to S3 — it hands the zip
to cloud — so it no longer needs a key that is not in KMS for any org. Taking
either side alone would have re-broken the other.

Under `mode: buildx` all 24 steps still have effective conditions identical to
f098b39. All ten gates pass, including build-yml-is-one-file, which is the gate
that caught 8a415e9 patching only the .github copy.
2026-08-06 14:16:39 -07:00
zeekay b047a7a569 build: the forge lane is the lane that runs
Hanzo CI/CD / cicd (push) Successful in 1m25s
CI/CD / gate (push) Successful in 1m26s
8a415e9 moved the REGISTRY_USER/REGISTRY_PASSWORD fetch out of the
`bucket:`/`site:` gate in .github/workflows/build.yml and left
.hanzo/workflows/build.yml untouched. Those two files are one artifact spelled
twice — github.com reads only `.github/workflows`, git.hanzo.ai reads only
`.hanzo/workflows` — and this repo's own `build-yml-is-one-file` gate exists to
refuse exactly this. It would have caught it: the gate passes at a830325 and
fails at 8a415e9.

The consequence is the inverse of the fix's intent. The copy that got the fix is
read by the forge nobody builds on; the copy the runners actually execute still
asks KMS for the registry credential only when a repo declares a bucket or a
site. So a repo declaring only `images:` — the common shape — still arrives at
`docker login registry.hanzo.ai` with two empty strings, and deleting the
REGISTRY_USER/REGISTRY_PASSWORD org secrets on the strength of 8a415e9 would
have broken every image build in three orgs rather than none.

They were byte-identical at a830325 (md5 1c027d00), and the only difference the
gate tolerates is a self-naming path that neither copy actually spells
differently, so the mirror is a literal copy.

No condition changes: under `mode: buildx` all 24 steps have effective
conditions identical to f098b39, which is what the v1 tag points at today.
2026-08-06 12:52:27 -07:00
hanzo-dev 8a415e9745 build: the registry credential is not an object-store credential
Hanzo CI/CD / cicd (push) Failing after 2m17s
CI/CD / gate (push) Failing after 2m18s
REGISTRY_USER/REGISTRY_PASSWORD were fetched from KMS only inside the
`bucket:`/`site:` gate — a gate about writing bytes to hanzoai/s3, which has
nothing to do with pushing an image. A repo declaring only `images:` therefore
asked KMS for nothing, arrived at `docker login registry.hanzo.ai` with two
empty strings, and failed at the push with no error pointing anywhere near this
step.

hanzoai/bot is the repo that found it, and it will not be the last: `images:`
without `bucket:`/`site:` is the common shape, not the exotic one.

The tempting fix is to paste the two names into the forge's org secrets, and
that is precisely the CI-only copy the comment four lines down warns against —
"a second thing to rotate and the one nobody remembers". The contract is that
the ONLY things living in the forge are KMS_CLIENT_ID/SECRET. So the fetch
moves out of the gate instead of the credential moving into GitHub.

Unconditional is safe: `get` returns empty when KMS has no such key, nothing is
exported, and the login step still falls back to the kubeconfig read for repos
provisioned before these names existed. The S3 keys stay gated, because those
really are specific to the two lanes that write to a bucket.
2026-08-06 12:35:47 -07:00
zeekay a830325d31 main is not the proof branch
Hanzo CI/CD / cicd (push) Successful in 1m22s
CI/CD / gate (push) Successful in 1m22s
2c036d3 set `mode: delegate` on this repo's own caller to prove the reordered
pipeline gates before it delegates. It belongs on a PR branch, and it reached
main because the two commits after it were made without leaving that branch.

On main it is worse than untidy: this caller pins @v2, which still resolves to
the PRE-reorder build.yml, so main would have run the exact defect the reorder
exists to remove — delegate evaluated fourth, the gate skipped, green in
seconds. Restore the caller. The proof runs on a branch, pinned at the version
tag that actually carries the fix.
2026-08-06 12:01:54 -07:00
zeekay d1e6edf6d1 the pipeline carries its own rule, because its tools come from somewhere else
Hanzo CI/CD / cicd (push) Canceled after 0s
CI/CD / gate (push) Canceled after 0s
Calling "$CI_HOME/bin/publishable" looked like the DRY choice and is broken on
every forge run. The tools checkout and this workflow resolve from DIFFERENT
places: a caller pins the workflow at a ref its own forge resolves, while the
tools step derives its ref from GITHUB_WORKFLOW_REF — which git.hanzo.ai does
not set. Measured on run 36473: `GITHUB_WORKFLOW_REF=<unset>`, `derived
ref=v1`, `cloned https://github.com/hanzoai/ci@v1`. So a step calling a NEW
file under $CI_HOME cannot work until a tag moves on github.com, a host the
caller never named — and hanzoai/ci's github main has diverged from the forge's,
so moving it is not a formality either.

Inline the rule. That leaves it spelled in two files, which is the same shape
as the two copies of build.yml, so it gets the same treatment:
`publishable-rule-is-one-rule` refuses any difference between the pattern
build.yml enforces and the one bin/publishable's test proves. Proven red
against a one-token drift.
2026-08-06 12:01:09 -07:00
zeekay 2c036d3a61 proof: delegate must gate before it delegates
Hanzo CI/CD / cicd (pull_request) Canceled after 0s
CI/CD / gate (pull_request) Canceled after 0s
2026-08-06 11:50:22 -07:00
zeekay 11fcb7b145 delegate chooses where the image is built, not whether it was tested
Hanzo CI/CD / cicd (push) Successful in 1m53s
CI/CD / gate (push) Successful in 1m54s
`mode: delegate` was evaluated fourth, ahead of every toolchain and ahead of
`test:`, and every other step in the file carried `if: inputs.mode !=
'delegate'`. So a delegated run skipped the gate along with the build: it
POSTed the image to platform and exited green in seconds having proven
nothing. The seam was right — ci gates, platform builds — and the ORDER said
otherwise, which is the kind of defect that reads as correct in every diff
that touches one step at a time.

Move the delegate POST to sit after `Test (per hanzo.yml)`, and let the steps
the gate cannot run without run in both modes: the two git-auth steps, the KMS
fetch that gives them GIT_TOKEN, and the Go/C/Node/Rust toolchains. What a
delegated run skips is buildx, publish and deploy. Nothing in the fleet passes
`mode: delegate` today (0 of 258 callers), so this changes no live run — it
closes the trap before the first repo falls into it.

`gate-runs-before-delegate` locks both halves of that, because either alone can
hold while the gate still does not run: the POST must come after the gate, and
the gate must not be delegate-guarded. Proven red against both regressions,
including a reconstruction of the exact prior layout.

Delegate also now refuses a repo that declares `build_secrets`. `/v1/runner`
resolves no KMS name — platform's own secrets arrive as a KMSSecret CRD with a
statically declared key list, which structurally cannot serve a name read out
of a hanzo.yml at some SHA — so delegating such a repo builds an image with an
empty credential, on a machine whose logs the run never shows.

And bin/publishable, with its own test, refuses a `build_secrets` name that has
not declared itself public. A build_secret is handed to buildx as `--build-arg
NAME=value`, and a build-arg is in the published image: `docker history` prints
it to anyone who can pull. The one repo whose value is genuinely publishable
asserted the `pk-` prefix inside its own Dockerfile — so the check existed
once, for one image, and any repo adding a real credential got none. The rule
is the name, because the name is what a reviewer reads and what git keeps; the
fleet had already started saying it that way (docs renamed EVENT_INGEST_KEY ->
PUBLISHABLE_KEY, world's four are VITE_*). This makes the convention the
mechanism instead of a habit.
2026-08-06 11:38:34 -07:00
zandGitHub f7c2309483 Merge pull request #11 from hanzoai/ci/site-zip-publish
the site lane publishes with a credential that exists
2026-08-06 02:10:27 -07:00
zandGitHub 33ceb2c495 Merge pull request #10 from hanzoai/ci/vendormark
refuse a site-generator vendor's branding, in all four places it hides
2026-08-06 02:10:23 -07:00
zeekayandhanzo-dev 9a6647cd30 the site lane publishes with a credential that exists
`site:` has never worked. It staged the export with `mc mirror` behind
S3_ADMIN_ACCESS_KEY/S3_ADMIN_SECRET_KEY read from KMS, and those names are in
KMS for no org — so the lane's fail-closed check refused every caller that
declared one. Of the 155 hanzo.yml files across the three orgs, not one
declares a `site:`. It was dead code, and hanzoai/console grew its own
build -> zip -> publish workflow to ship at all, which is the per-repo build
logic this contract exists to prevent.

The fix is not to provision that credential. It is to stop needing it: the zip
goes to /v1/projects/<slug>/deploy and the landed prefix is promoted by
/v1/sites/<slug>/publish, both on the IAM bearer the KMS step already mints.
CI names no bucket and no org — sitePrefix(org, slug) is server-side from the
validated principal, which is what makes the prefix unforgeable.

The work is bin/sitepublish, not forty inlined lines nobody can run offline.
It verifies rather than assumes: publish returning 200 says the request was
accepted, not that the release is serving, so it re-reads the list and refuses
unless the release it just published is the one that is live.

Every assertion is written to fail when it finds nothing, because three real
defects had the opposite shape — a success case that included "I resolved
nothing":

  * an object-shaped read of a body that is not an object. `.releaseId // empty`
    yields "" for an array, a null or an error body, and every later test on ""
    passes vacuously. The shape is asserted first.
  * `.releases[]` against GET /v1/sites/<slug>/releases, which is a BARE ARRAY
    (release.go:518, `type projectsReleases []projectsRelease`). That filter
    resolves nothing, so the check could only ever refuse — it never once
    tested what it claimed. Hence `.[]`, behind a `type == "array"` guard so
    that if cloud ever wraps the list THAT goes red, by name.
  * `[ "$a" = "$b" ]` on two empty strings, which is TRUE. Non-emptiness is
    proven before the comparison, and actives are COUNTED, so "none active"
    cannot read as "yes".

Reintroducing those three turns 3, 9 and 1 of the suite's tests red.

The size boundary is the server's, not a preference: the edge caps a body at
16 MiB (GATEWAY_BODY_LIMIT) and fasthttp refuses a larger POST before any
handler runs, reporting only "Error when parsing request". So the zip is
measured and refused here, with the number and the alternative. Of the 24 built
exports in the estate 22 fit; hanzo.ai (27.9 MiB zipped, 8536 files — also past
the server's own 5000-entry cap) and trillerfest.com (76.7 MiB) go to
bin/sitedeploy, which streams per-file against a presigned grant.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-05 23:29:20 -07:00
hanzo-dev 0539ecf756 refuse a site-generator vendor's branding, in all four places it hides
Several sites in the estate began as output from a hosted "prompt to
React app" generator. It does not sign its work in a comment you would
notice; it signs it in the shape of the repo, four ways at once: an asset
folder named after the vendor, a build plugin in package.json, an
injected third-party <script>, and the commit messages, author identities
and Co-authored-by trailers of its bot.

A sweep of 1,065 repositories found the mark in five shipped sites. One
served the vendor's upload folder as its FAVICON, so the vendor's path
was in the <head> of every page and in every browser tab. Three have been
cleared; that took a history rewrite and a force-push of every branch.
This gate is so that is never needed a second time.

The fourth place is the one nothing else looked at and the one that
survives any amount of tidying: a repo can render a perfect page and
still say in `git log` who really wrote it. A commit-msg hook on one
workstation already strips these trailers, but that hook does not run on
a runner, on a teammate's laptop, or on a commit made through the web UI.
This does.

Content is judged narrowly and paths are not. The vendor's name is also
an ordinary English adjective, and the naive gate is unshippable: across
those same 1,065 repos the bare word appears innocently in tokenizer
vocabularies, word lists, ML corpora, packed public-suffix data, a
disposable-email blocklist and an upstream Clerk bundle listing other
people's dev hosts. A gate that reds a fifth of the estate is a gate
someone switches off. So the content rules are only strings that cannot
occur in prose, and two candidate rules were measured and dropped for
redding that Clerk bundle. Paths, identities and trailers are refused on
the bare word, because there no third party can put it and no tidying
reaches it.

Measured: red on exactly 4 of 1,219 checkouts, every one a true positive.
17/17 in the suite, which pins the allowances as well as the refusals.
.vendormark-allow exists for the one honest case — the history-scrubber
rule files in hanzoai/.github, which cannot do their work without
spelling the string they remove.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-05 16:56:10 -07:00
zeekayandhanzo-dev bd06e29b29 ci: @v1, not @v2 — v2 is an alias for the v1 line
hanzoai/ci is tagged v1.0.32..v1.0.35 plus a bare floating `v2` that points
at the SAME commit as v1.0.35, so v2 is not a second major — it is another
name for the line already in use. The fleet was split 13 @v1 / 7 @v2 over one
thing, which is two ways to say one version.

Runs on git.hanzo.ai's own act_runner pool (git-runner, 10 replicas). Not ARC.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-05 15:57:25 -07:00
hanzo-dev 5e0e4f906c hanzo-inc resolves to the KMS org it actually seals to
Hanzo CI/CD / cicd (push) Successful in 2m15s
CI/CD / gate (push) Successful in 2m15s
The owner -> KMS-org relation knew hanzoai, luxfi and zooai. hanzo-inc — pay
and the other company surfaces — fell through to the default branch and
became the literal owner, which is not a KMS org.

That was survivable by accident: the read takes its org from the TOKEN, so
the flat route still returned the right secret. What was wrong was the
fallback URL and, more expensively, every error message this step can print
— they named org=hanzo-inc, which sends whoever reads them looking for a
missing tenant rather than a missing credential.

One line, in the one place the comment above it already says this relation
belongs.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-05 14:40:56 -07:00
hanzo-dev f098b39ecd a documented example is not an expression to evaluate
CI/CD / gate (push) Successful in 3m1s
Hanzo CI/CD / cicd (push) Successful in 3m0s
`v1` has failed at load, on every caller in the fleet, since 0e1a56b. Not a
test failure — a startup failure: zero jobs, zero seconds, and GitHub showing
the file's PATH where its name should be, because it never got far enough to
read `name:`.

The `tests:` input's description quotes what a caller writes, and quoted it
literally:

    `tests: ${{ github.ref_type != 'tag' }}`

Prose or not, that is an expression, and it sits under `on.workflow_call.inputs`
where NO context exists. GitHub validates the whole file before it schedules
anything, refuses `github` there, and drops the run — so every repo importing
`hanzoai/ci@v1` stopped running CI entirely while reporting only a red X with
nothing inside it to read.

The irony is exact: 0e1a56b is titled "the test step has never run, and the gate
that would have said so never ran either." It made that true of every gate at
once.

Say it without the sigil. Same meaning to a reader, nothing to evaluate.
Confirmed with actionlint: clean at 23862e5c (before the break), the same error
at v1, v1.0.34 and v2, clean here.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-05 14:28:04 -07:00
hanzo-dev 674fabdcba split the server, do not strip a scheme you assumed
Hanzo CI/CD / cicd (push) Successful in 1m0s
CI/CD / gate (push) Successful in 1m0s
v1.0.33 built the forge substitution with `${SERVER#https://}`. GITHUB_SERVER_URL
on this fleet's runners is `http://hanzo-git.hanzo.svc` — the forge's in-cluster
address, plain http — so the strip matched nothing, passed the whole URL through,
and produced `https://http://hanzo-git.hanzo.svc/hanzoai/`. git then went looking
for a host called `http`:

  fatal: unable to access 'https://http://hanzo-git.hanzo.svc/hanzoai/authz/':
  Could not resolve host: http

Take the scheme and the host as the two values they are, and compare the HOST
against github.com so the guard cannot be fooled by a spelling either. Checked
against http/https, in-cluster and public, with and without a trailing slash
before pushing this time.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-05 06:34:42 -07:00
hanzo-dev 3f65c9e24f the test gate reaches our modules at the address they live at
Hanzo CI/CD / cicd (push) Successful in 1m43s
CI/CD / gate (push) Successful in 1m44s
The gate's first real execution died in `go vet`, before one test ran, on
`github.com/hanzoai/dashscopego v0.6.0` — an indirect dependency of
hanzoai/cloud that git.hanzo.ai serves and github.com answers `Repository not
found` for.

A module path is a NAME, not an address. This step authenticated to github.com
and only github.com, so on a forge runner it dialled the mirror instead of the
canonical host. Its own caller already knew better: cloud's `containment` and
`reach` jobs each make exactly this substitution, carrying a comment about the
nine consecutive releases that one sibling module's drifted GitHub ACL once
blocked. The gate that runs the tests reached the modules through a different
door, so the law was written twice and applied where it was not needed.

go.sum is what makes it safe rather than merely convenient: the forge serves the
same objects, so the fetched zip hashes to the h1: line already committed, and a
forge serving different bytes fails the build loudly instead of shipping them.

Owner-scoped and host-derived — `${{ github.repository_owner }}` against
GITHUB_SERVER_URL — so it is the same statement for any org on any forge, and a
no-op on github.com or without FORGE_TOKEN. Longest prefix wins in git, so every
other module still resolves from github.com exactly as before.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-05 06:31:02 -07:00
zeekayandhanzo-dev 8e43277aab sitedeploy self-provisions its project — first run no longer fails
Hanzo CI/CD / cicd (push) Successful in 1m10s
CI/CD / gate (push) Successful in 1m11s
Every never-before-deployed site died at the enqueue: POST /v1/projects/<slug>/deploy
404s when the project does not exist yet, so committing a workflow was never
enough — someone had to POST /v1/projects out of band first, holding a token most
sites' authors don't have. sitedeploy now creates the project itself before the
enqueue: idempotent (200/201 first run, 409 forever after), repo linked because
the git-source enqueue requires it, analytics on by default. A committed workflow
is now sufficient to stand a new site up. The plan/test seam sits above this, so
the 17 offline tests are untouched — verified green.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-04 18:54:01 -07:00
zeekayandClaude Fable 5 d3823df85a sitedeploy: one implementation of the Sites-plane publish
Hanzo CI/CD / cicd (push) Successful in 41s
CI/CD / gate (push) Successful in 42s
Every static surface on the estate was about to carry its own transcription of
enqueue → upload → complete: hanzo.ai, hanzo.app, hips, computer, and each one
after them. bin/imgver exists because that already happened once with image tags
— written eleven times, right ten times, and `sha-<short>` shipped for months.
This is that rule applied to the other half of the fleet.

The contract, once:

  POST /v1/projects/<slug>/deploy       -> 202 {id, bucket, prefix, upload}
  POST <upload.url> per object          -> the bytes, straight to S3
  POST .../deployments/<id>/complete    -> {status, files, bytes, keys}

The bytes never pass through the API: a real export is 128 MB against a 16 MiB
BodyLimit, so an artifact POST is refused by fasthttp before any handler runs and
fails as an opaque 400 that reads like a malformed payload rather than a size cap.

ONE credential. The 202 carries a presigned POST grant that is prefix-scoped
(S3 itself enforces starts-with $key), 30-minute, and size-bounded — so CI holds
no bucket key. Do not reintroduce SITES_S3_*: that was one long-lived key for a
bucket whose only tenant separation is the key prefix, so every repo holding it
could overwrite every org's site. Deletion moved server-side with it: `keys` is
the manifest cloud reconciles the prefix against, which is where
`aws s3 sync --delete` went, and both it and this script fail CLOSED on an empty
manifest rather than deleting a live site because a pipeline produced no lines.

Four defects found by running it end-to-end against hanzo.ai's real export
(8402 files, 105 MB, 54s) rather than by reading it:

  · the grant's `fields` already carries `key` — the starts-with PLACEHOLDER,
    not a destination. Forwarding it verbatim beside the real key posts `key`
    twice and S3 answers 400 for every object: 8402 files, 8402 failures.
  · `mapfile` is bash 4+; macOS ships 3.2, where it is `command not found`, the
    field array stays EMPTY, and the upload is rejected for a malformed
    X-Amz-Credential — an error pointing at the credential rather than at the
    array that was never built.
  · `[.\-_]` is how Go spells the fingerprint class; transcribed into `[[ =~ ]]`
    the backslash is literal inside a bracket expression, so it reads as the
    range \…_ and the shell rejects it. The `if` then merely evaluates false and
    every content-hashed asset silently loses `immutable`.
  · `stat -f%z || stat -c%s` cannot work: on GNU the first arm fails INSIDE a
    pipeline, awk still exits 0, the `||` never fires, and the byte count reports
    0. `wc -c` is the spelling both agree on.

Content-Type is sent per object because the policy carries no condition for it,
so what CI sends is what the object stores and what the edge serves — send
nothing and a browser downloads every page instead of rendering it. Cache-Control
mirrors cloud's apps/sites.CacheControlFor; sitedeploy_test.sh pins those exact
strings so the two cannot drift apart quietly.

Tests run offline: SITEDEPLOY_PLAN=1 stops before the first network call and
prints the manifest, so all 17 cases need no token, no bucket and no cluster.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 18:36:01 -07:00
zeekay b6b52b4a81 the gate runs before the image exists, not after it shipped
Hanzo CI/CD / cicd (push) Successful in 2m33s
CI/CD / gate (push) Successful in 2m32s
`Build & push images` sat above `test:`, guarded only by mode != delegate. So a
commit whose gates FAILED still published an image. Measured, not theorised:
hanzoai/base f9ec7067 pushed sha-f9ec706-amd64 on a run whose Test step then
went red, and hanzoai/id and hanzoai/base both show Build ahead of Test in every
run this week.

Deploy and Publish site already skip on a red gate, so nothing auto-shipped
those bytes. That is the reassuring half and also the trap: the TAG existed. A
tag in a registry is something a human pins by hand at 2am, and its existence
reads as a claim the commit passed. It did not. An artifact should not be
producible from a commit that failed its own gates — that is the same "green
that means nothing" this file spent the day removing, one step lower down.

Order is now toolchains -> Test -> Build -> Deploy. Nothing in Build depends on
the toolchains it now follows, and Deploy still follows Build, so the single
behavioural change is that a red gate produces no image at all.

Costs one thing worth naming: a repo with a broken Dockerfile and a slow suite
now waits for the suite before learning the build is broken. That is the right
trade — a late failure is cheaper than a false artifact.
2026-08-04 18:21:44 -07:00
zeekay 331e05625f push to registry.hanzo.ai by its own name, with a credential from KMS
Hanzo CI/CD / cicd (push) Successful in 2m34s
CI/CD / gate (push) Successful in 2m35s
THE 401 WAS NOT A WRONG REGISTRY. registry.hanzo.ai and oci.hanzo.ai answer the
SAME catalog behind the SAME token realm (iam.hanzo.ai/v1/iam/registry/token) —
measured, both hosts, byte-identical repository lists. They are one store with
two branded names, exactly as ~/work/CLAUDE.md describes.

What broke is narrower and stupider: docker keys credentials by HOSTNAME. This
workflow logged in to oci.hanzo.ai and pushed to whatever hanzo.yml named, so
the moment a repo repointed at the canonical registry.hanzo.ai the daemon had no
credential for that host and the push 401'd — against a registry it was already
authenticated to under its other name. So: log in to the canonical name.

CREDENTIAL FROM KMS, which is the rule and was the actual blocker. The step had
two sources, and for most repos neither fires: REGISTRY_USER/PASSWORD as GitHub
secrets (unset), else a kubeconfig read of the `registry-credentials` secret —
and the CI kubeconfig identity cannot read it (`ci-deploy-bot` grants
apps/deployments and replicasets, no secrets at all, and is bound to a service
account left over from the retired ARC). Measured consequence: the mirror step
logs "registry-credentials not readable from this kubeconfig" and MIRROR_OK is
never set, so registry.hanzo.ai holds cloud, commerce, iam, git — and NONE of
app, chat, gui, ui, design, sites, superbase, base. Those names answer
NAME_UNKNOWN. The names are now fetched from KMS beside S3_ADMIN_ACCESS_KEY,
which already works this way; the kubeconfig read stays as the fallback.

ONE NAMESPACE MAPPING, STATED ONCE. The GHCR org and the registry org are not
the same word, and the old line assumed they were —
`mrepo="oci.hanzo.ai/${repo#*/}"` mirrored ghcr.io/hanzoai/x to .../hanzoai/x.
The mapping is hanzoai->hanzo, zooai->zoo, everything else to itself:

  ghcr.io/hanzoai/chat      -> registry.hanzo.ai/hanzo/chat
  ghcr.io/zooai/app         -> registry.hanzo.ai/zoo/app
  ghcr.io/bootnode/node     -> registry.hanzo.ai/bootnode/node
  ghcr.io/lux-private/gpu   -> registry.hanzo.ai/lux-private/gpu

It lives in the push path itself so the table and the path cannot disagree —
two copies of a name mapping is how a pin and a push end up naming different
images. The ~30 repositories already mirrored under the old `hanzoai/` prefix
are left where they are; nothing pins registry.hanzo.ai yet (universe pins
ghcr.io), so they are orphaned mirror copies, not live references.

COREPACK_NPM_REGISTRY on the job. corepack fetches the pnpm/yarn tarball ITSELF
from registry.npmjs.org and does not read .npmrc to do it, so all 317 repos that
pin `packageManager` reach npmjs on every install however carefully their
registry is configured — the dependencies came from our mirror and the tool
fetching them did not. Set here for BYO runners; the git-runner fleet gets the
same value from its own config so jobs that never import this reusable are
covered too.
2026-08-04 17:57:07 -07:00
zeekay 44186bea06 conflictmarkers: the whole shape, not the first line of it
Hanzo CI/CD / cicd (push) Successful in 49s
CI/CD / gate (push) Successful in 48s
Shipped an hour ago matching a start marker alone, and hanzoai/app went red on
code that is perfectly fine. Its builder tests carry, as literal fixture data:

  <<<<<<< START_TITLE index.html >>>>>>> END_TITLE

Seven '<' and a space at column 1 — and not a merge conflict. Both markers sit
on ONE line, which is a shape git never writes.

So the gate now requires the whole conflict, in order, each part on its own
line: a start marker that does NOT also carry an end marker, a bare seven-equals
divider, and a separate end line. app's fixture fails the very first of those
and passes; a real conflict satisfies all three.

Proven against four cases before pushing, not after:

  base 42b97c9c types.d.ts  RED   23807 <<<<<<< HEAD / 23849 ======= /
                                  23850 >>>>>>> upstream/master
  app  format-pages.test.ts GREEN the fixture above
  synthetic git-shaped      RED   the exact bytes git writes on a conflict
  hanzoai/ci itself         GREEN including this file, which spells the
                                  markers out in prose, and a .rst heading
                                  underlined with seven equals signs

The divider is still only ever matched as part of that conjunction and never on
its own, because seven bare equals at column 1 is also how reStructuredText and
Setext Markdown underline a heading.

The lesson is the one this gate exists to teach, turned on itself: a check that
fires on a fragment of a pattern is not a cheap approximation of one that fires
on the pattern, it is a different check with a different, larger false-positive
surface — and the repo it fails first is the one that did nothing wrong.
2026-08-04 17:26:43 -07:00
zeekay 0e1a56bd3c the test step has never run, and the gate that would have said so never ran either
CI/CD / gate (push) Successful in 55s
Hanzo CI/CD / cicd (push) Successful in 55s
git.hanzo.ai resolves every `type: boolean` workflow_call input to false. Not
just an omitted one falling back to its default -- an explicitly passed `true`
resolves to false too. Measured in isolation (z/inputprobe runs 27365, 27382):

  type: boolean, default: true, caller omits it   -> inputs.tests = false
  type: boolean, caller passes with: {tests: true} -> inputs.tests = false
  type: string,  default: 'true', caller omits it  -> inputs.tests = true
  type: string,  default: '["hanzo-build-linux-amd64"]' -> resolves in full

So `if: inputs.mode != 'delegate' && inputs.tests` has been false on every run
this pipeline has ever served, and the `test:` block was parsed and skipped. On
2026-08-04 the last two cicd.yml runs of hanzoai/cloud, commerce, console,
gateway, hanzo.ai and this repo were all completed/success with
`Test (per hanzo.yml)` = skipped. Every test gate in the fleet was declared,
read, and never once executed -- a green over zero tests, which is the exact
failure the step's own assertion exists to refuse when a SUITE is empty, one
level up where nothing was watching.

tests: becomes a string compared against 'false'. A string sidesteps the forge
bug (proven above) and means the same thing on both planes, and comparing
against 'false' rather than testing truthiness keeps hanzoai/cloud's existing
`tests: ${{ github.ref_type != 'tag' }}` working untouched -- that expression
renders the strings "true"/"false".

The second half is what the first half was hiding. build-yml-is-one-file is a
`test:` gate, so it never ran either, and the two published copies had drifted
26 lines: .hanzo/workflows/build.yml -- the ONLY copy git.hanzo.ai executes --
was missing the Structural invariants step entirely. ignoretracked and modsize
have therefore never run on the plane that runs. The copies are one file again,
regenerated rather than hand-edited a second time, since hand-editing twice is
how the drift got in.

conflictmarkers joins that step, because it is the same defect class: `git add`
accepts a file still carrying git's own markers, and from then on they are
ordinary committed content -- status clean, diff empty, both sides of the merge
shipping. It hides in generated and vendored files that are large and
unreviewed, and no compiler objects: a .d.ts nothing imports is never parsed,
and in YAML, JSON and Markdown the markers are valid content. hanzoai/base has
carried `<<<<<<< HEAD` / `>>>>>>> upstream/master` at lines 23807 and 23850 of a
committed .d.ts. Proven red against that file and against a synthetic merge,
green on a clean tree, and it does not flag itself or a reStructuredText
underline -- the bare `=======` divider is deliberately not matched, because
seven equals signs at column 1 is also how RST and Setext underline a heading.
2026-08-04 17:05:54 -07:00
hanzo-dev 23862e5c44 two structural refusals: the repo must agree with itself
Hanzo CI/CD / cicd (push) Successful in 1m31s
CI/CD / gate (push) Successful in 1m32s
A day of production archaeology kept finding one defect class wearing
different clothes: a declaration and the reality it describes disagreeing,
with nothing in the pipeline holding an opinion about it. The framework
already refuses this shape in three places — zip refuses middleware that
wraps nothing, zipdoc refuses an op whose address it cannot resolve, Go
refuses a module past its zip ceiling. This lands the same principle for
two disagreements nothing was catching:

bin/ignoretracked — content the repo's OWN .gitignore matches but the index
still tracks. An ignore rule does not untrack what is already tracked, so
the bulk ships in every release while `git status` stays clean. The causal
story from the real outage is in the script header: the rule was removed at
12:22, 855 files landed at 12:51 inside the unguarded window, the rule came
back at 16:48. Every step was reasonable; the combined state shipped 361 MB
of orphaned cargo output in every release for 6 days.

bin/modsize — every Go module tree in the repo measured against Go's hard
500 MiB module-zip ceiling, nested modules separately. Past it, the module
is not merely large: `go get` fails in every consumer at every version that
carries it. cloud crossed at v1.801.416 and was unfetchable for nine
consecutive releases, which is why production sat pinned five releases back.

Wired as one early step in the reusable workflow, before anything builds or
pushes. Both FAIL the run — a warning nobody reads is the same silence one
level up.

Proof, not assertion: both RED against cloud@v1.801.424 (ignoretracked:
361.1 MB tracked-yet-ignored; modsize: 528.8 MiB, past the ceiling, nested
apps/platform/design measured separately at 0.2 MiB) and GREEN against the
repaired v1.801.440, hanzoai/iam, and this repo.

Scripts authored by the structural-guards lane; proofs re-run and wiring
added before landing. Callers pin @v1 — this activates when that tag moves.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-04 16:13:28 -07:00
hanzo-dev 74c176639b build: every published image names the commit it was built from
Hanzo CI/CD / cicd (push) Successful in 2m4s
CI/CD / gate (push) Successful in 2m4s
Images built by this reusable carried org.opencontainers.image.revision=unknown,
which means an image published anywhere in the fleet could not be traced to a
commit. cloud's Dockerfile declares `ARG REVISION=unknown` and stamps the label
from it; nothing here ever passed REVISION, so the default won every time.

That is cosmetic right up until two builders race for one tag — and they did.
Two images sat behind ghcr.io/hanzoai/cloud:v1.801.410, one labelled 1b8b76ed
and one labelled `unknown`, and "which of these is the release" had no answer
short of diffing layers, while the wrong one was already pinned. A version is a
receipt only if the image can name its own commit.

Passed BOTH ways, deliberately: --label sets it from outside for any Dockerfile,
--build-arg feeds `ARG REVISION` for Dockerfiles that stamp it themselves. A
Dockerfile with no ARG REVISION ignores the build-arg (buildkit warns, never
fails), so this is a no-op for every repo that has not asked for it. version and
source labels come along for the same reason.

Applied to both published copies, which are byte-identical by contract.

This retires the image -> commit holds that universe's scripts/ci/receipt.sh
currently records for iam, id and console — all three are "built by hanzoai/ci
build.yml, which does not pass REVISION", and all three are one shared fix.
2026-08-04 02:00:12 -07:00
hanzo-dev b323a1189d legal: license original work under Apache-2.0
This repo is original Hanzo work and shipped with no LICENSE, which left
it all-rights-reserved by default. Apache-2.0 per the estate policy that
original work is Apache-2.0 or BSD-3-Clause.

Verified original before licensing: not a GitHub fork, no upstream
project identity in the manifests, no third-party copyright headers in
the tree, and the root commit is Hanzo-authored.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-04 02:00:12 -07:00
zeekay 5750426c73 merge: fail closed on a declared build secret that cannot be read
Hanzo CI/CD / cicd (push) Successful in 3m28s
CI/CD / gate (push) Successful in 3m28s
An unreadable declared build_secret was a warning that baked an EMPTY value and
let the run go green — the same silent shape that shipped keyless bundles across
the fleet. A repo that declares a secret asserts the image needs it, so an
unreadable one is a failure, not a note.

Resolved against the embedded-KMS read: kept both, since one decides WHERE the
secret is read from and the other decides what happens when it is not there.
Repos declaring nothing are byte-for-byte unaffected.
2026-08-04 01:24:41 -07:00
zeekay 093b912e51 apply the embedded-KMS read to the .github copy too
Hanzo CI/CD / cicd (push) Successful in 32s
CI/CD / gate (push) Successful in 33s
There are two build.yml, and the earlier fix reached only .hanzo/workflows.
Consumers split across them: hanzo.ai imports the .hanzo copy and resolves its
key, docs imports this one and did not — which is exactly the difference
between the lane that works and the lane that ships an empty key.

Same change: embedded shape first (/v1/kms/secrets/<path>/<name>, flat .value),
standalone as fallback, default endpoint api.hanzo.ai.
2026-08-04 00:24:23 -07:00
zeekayandhanzo-dev 8edb99bb19 give both copies of the reusable the same KMS contract
The two files are one reusable and eleven repos import them by path, but only
the .hanzo copy was moved onto the embedded KMS. The .github copy — the one
docs, gateway, tasks, deploy and openapi import — still defaulted to the
standalone host and still read /v1/kms/orgs/<org>/... with .secret.value, a
shape the embedded service answers with nothing. Every one of those builds
resolved an empty credential, or the stale value the standalone held, and said
nothing about it.

They are byte-identical again: the embedded url and response shape, the
standalone fallback, and the refusal to build with an unreadable declared
build secret.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-04 00:04:59 -07:00
zeekayandhanzo-dev 9f6bfb8cff fail the build when a declared build_secret is unreadable
A repo that lists `build_secrets` in hanzo.yml has asserted its image is not
valid without them. The fetch treated every failure as survivable and exported
nothing, so buildx baked an empty value and the run went green — the artifact
was broken and only production showed it, as silence.

The declaration is now read before the two short-circuits, so an unconfigured
or unreachable KMS is an error for a repo that declares secrets instead of an
empty result that looks like "not in KMS". An unresolvable name is an error at
fetch, and the build step refuses a name that is still unset when it assembles
--build-arg.

A repo that declares no build_secrets keeps the best-effort path exactly as it
was: KMS stays optional, GHCR push still works on the workflow token.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-04 00:00:12 -07:00
zeekay a2d50bce29 fix: read build secrets from the embedded KMS
Hanzo CI/CD / cicd (push) Successful in 44s
CI/CD / gate (push) Successful in 45s
KMS is embedded in hanzoai/cloud now (HIP-0106, cloud/apps/kms). This reusable
still spoke the standalone contract, and both differences fail SILENTLY -- the
result is an empty string, which every caller here already treats as "not in
KMS", so a build that lost its secret looked exactly like one that never
declared it.

  url   /v1/kms/secrets/<path>/<name>  -- the org comes from the TOKEN, not the
        url. The old /v1/kms/orgs/<org>/... form 404s, and `curl -sf` turns that
        into empty output.
  body  {"name","env","value"} -- flat. `.secret.value` selects nothing.

Reading the OLD host is worse than reading nothing: the two instances hold
DIFFERENT values under the same name. The standalone copy of
deploy/EVENT_INGEST_KEY is stale, and cloud rejects it at ingest with 403 --
so a build that "found" a key still shipped one that cannot write, and every
pageview from that property was filed under the reserved $public tenant, which
our own org cannot read.

Tries the embedded shape first and falls back to the standalone one, so a repo
still pointed at the old host by vars.KMS_ENDPOINT keeps working while the fleet
moves. Default endpoint flipped to api.hanzo.ai.
2026-08-03 21:25:21 -07:00
hanzo-dev d31d20f679 sync: record the forge lineage as landed, since it carries nothing this does not
Hanzo CI/CD / cicd (push) Successful in 3m1s
CI/CD / gate (push) Successful in 3m2s
forge/main sat 43 commits off this history and could not fast-forward, so every
push to the mirror was rejected and the two hosts have been drifting apart while
saying nothing about it.

Nothing on it is unlanded. `git cherry origin/main forge/main` finds 42 of 42
commits already applied here by patch equivalence and none unique; no file exists
on that branch and not on this one; and the whole textual difference is prose this
side has since rewritten — the registry.hanzo.ai mirror leg, crane, corepack
provisioning and the delegate lane are all present here in their later wording. It
is this repo's own history, absorbed once already at a43f5a3 and then left behind
by 89 commits.

So this takes the lineage and NOT the tree (-s ours): the merge is a no-op on
content — verified, the resulting tree object is byte-identical to its first
parent — and its only effect is that the mirror can fast-forward again. The
alternative was a force-push, which would delete 43 commits of real history to
assert something a merge can state without deleting anything.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-02 11:03:27 -07:00
hanzo-dev f01beb8f6f ci: a repo that declares no test gate says so on the run
df6c025 made a gate that reports success without running a test fail the build.
It asks each gate that question by looping over hanzo.yml's `test:` block — so it
cannot ask the repo that declares no block at all. There the loop simply does not
run, and the build goes green having tested nothing: the exact shape that commit
exists to end, reached by declaring less rather than more.

23 repos in the fleet are in that state today, so failing here would paint the
fleet red without fixing one of them. It warns instead, into the ::warning::
annotation and the step summary this build already writes, which puts the absence
ON THE RUN rather than in a file only a reader of hanzo.yml would ever open.

The gate list is now read ONCE and asked both questions — "is it empty" and "run
each" — instead of yq being invoked twice for one fact.

Both copies of build.yml, byte-identical as aeb6adf requires (one document, two
paths, one tag serving both forges). Checked: the two files compare equal after
the edit, both parse, the step's shell passes `bash -n`, and the three shapes
behave — absent `test:` warns, `test: []` warns, a populated block runs its gates
and does not warn.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-02 11:01:56 -07:00
hanzo-dev 07a8c94b14 Merge remote-tracking branch 'origin/wip/ci-local-edits' into HEAD
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-02 11:00:42 -07:00
Claude c1bd70ca94 fix: provision Node for JS gates that live in a subtree, not just at the root
Hanzo CI/CD / cicd (push) Successful in 19s
CI/CD / gate (push) Successful in 19s
hanzoai/commerce ran 49 ginkgo suites green and then died on its last gate:

    ::group::test admin-typecheck
    bash: line 3: pnpm: command not found
    exitcode 127
    skipping post step for "Provision Node toolchain (JS test gates)";
      main step was skipped

That last line is the whole story. The step exists precisely to stop a JS gate
dying at 127 on a runner image with no Node — and it had SKIPPED, so it could
not.

Its guard was `hashFiles('package.json')`, which only ever looks at the repo
ROOT. commerce is a Go repo whose front ends live in app/, and its gate is
`cd app && pnpm install --frozen-lockfile && ... turbo run typecheck`. No root
package.json, so the guard said "not a JS caller" about a repo with four JS
workspaces.

The guard now also matches a nested one. Both patterns are passed rather than
relying on `**/` to match zero segments: this reusable has 13 callers, and a
guard that stopped matching a root package.json would break every JS repo at
once. Widening a provisioning step is safe in the other direction — a repo that
gains a Node toolchain it does not use loses nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 04:13:14 -07:00
hanzo-dev c9b7153152 ci: name the fleet that actually serves the default, and fix a delegate URL that 404s
Hanzo CI/CD / cicd (push) Successful in 19s
CI/CD / gate (push) Successful in 19s
The `runner` default has always been '["hanzo-build-linux-amd64"]', and every
runner carrying that label is StatefulSet/git-runner in ns hanzo, registered to
hanzo-git.hanzo.svc. The prose called it "the Hanzo cloud arc pool" anyway. That
is not a cosmetic mismatch: arc (arcd) is being retired, and a default described
in terms of a dead system is how the dead system comes back — the next reader
provisions one because the doc says the pipeline needs it. 156 files across three
orgs ask for this label and not one of them was ever served by arcd, which
advertised 48 labels and zero amd64.

The delegate lane was worse than stale, it was broken. `mode: delegate` POSTs to
PLATFORM_ENQUEUE_URL, defaulting to platform.hanzo.ai/v1/arcd/enqueue. Platform
renamed that route to /v1/runner; the old path now falls through to the catch-all,
which answers 401 {"message":"Unauthorized"} — distinguishable from the real route
only by the body, since /v1/runner answers 401 {"message":"Invalid enqueue token"}
and so does a path that never existed. A delegated build has therefore been
failing auth rather than enqueueing. Nothing noticed because no repo passes
`mode: delegate`, which is also why this was safe to correct in place.

Both published copies edited identically; the byte-equality gate passes at 1321
lines. The remaining occurrence of the word is deliberate — a one-line tombstone
in the input description, so the next person to look for the arc pool is told it
was retired instead of concluding the doc is merely out of date.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-08-01 21:52:03 -07:00
Claude 7cb8c69316 ci: hand the resolved spec credential to the generator too
Hanzo CI/CD / cicd (push) Successful in 36s
CI/CD / gate (push) Successful in 37s
The guard now resolves SPEC_TOKEN-or-GH_PAT, but the generator runs in a child
process and reads SPEC_TOKEN itself:

    generate.sh: no hanzoai/openapi checkout at .openapi and no SPEC_TOKEN to
    clone one

So resolving the fallback only in the guard moved the failure one line down
instead of fixing it. Exporting the resolved value under the name the script
already reads means one credential is chosen once and every consumer in the lane
sees the same one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 20:06:43 -07:00
Claude 81eee4e18d ci: the client lane falls back to GH_PAT, and says which credential it used
Hanzo CI/CD / cicd (push) Successful in 34s
CI/CD / gate (push) Successful in 35s
The lane failed every run with

    ::error::the client: lane needs SPEC_TOKEN (contents:read on hanzoai/cloud)

and there was nothing to do about it: SPEC_TOKEN is provisioned on NO org and NO
repo on this forge. Measured — orgs/hanzoai holds GHCR_USER, GHCR_TOKEN, GH_PAT,
OCI_USER, OCI_TOKEN, KMS_CLIENT_ID, KMS_CLIENT_SECRET, REGISTRY_TOKEN, and
repos/hanzoai/cli holds nothing. A missing org secret is "", not an error, so the
guard fired on every build and named a secret that does not exist anywhere.

SPEC_TOKEN stays PREFERRED: it is the least-privilege choice, a fine-grained
token scoped contents:read on one private repo, and that is worth keeping when
someone mints it. GH_PAT is the fallback because it is provisioned and already
reads private hanzoai repos.

The step now prints `spec credential: SPEC_TOKEN|GH_PAT`. A fallback that does
not say which credential it took is how a wrong one goes unnoticed — the same
defect that made luxfi/trader read as a registry problem for weeks.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 20:04:26 -07:00
Claude d9c8917e18 ci: default GITHUB_WORKFLOW_REF before expanding it — set -u killed the step
Hanzo CI/CD / cicd (push) Successful in 49s
CI/CD / gate (push) Successful in 50s
Ground truth from the run, once the step was made to report:

    GITHUB_WORKFLOW_REF=<unset>
      Failure - Main Check out this reusable`s own tools

Nothing between them. The step died on the very next line, before any clone:

    ref="${GITHUB_WORKFLOW_REF##*@}"

This forge does not set GITHUB_WORKFLOW_REF, and since bash 4.4 a `##`
expansion of an unset variable under `set -u` is an unbound-variable error. So
the step exited 1 while every host still looked blameless, and the fallback that
would have produced the right answer -- `ref="${ref:-v1}"`, two expansions later
-- was unreachable.

Reading `${GITHUB_WORKFLOW_REF:-}` into a local first makes the whole chain safe
and still derives v1, which is what every caller pins.

⚠️ Not reproducible on macOS: /bin/bash is 3.2, where the old form survives with
an empty result. Testing it there says the code is fine. The forge`s log is the
only thing that settled this, which is why the previous commit made the step
print what it derived.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 19:36:00 -07:00
Claude 31817412ec ci: the tools checkout says what it tried
CI/CD / gate (push) Successful in 33s
Hanzo CI/CD / cicd (push) Successful in 32s
The step sent both clone attempts to /dev/null and printed only

    ::error::could not fetch hanzoai/ci@$ref (bin/imgver)

which names neither the ref it derived nor which host refused, so the cause can
only be guessed at from source — and guessing produced one wrong fix already.
Now it echoes GITHUB_WORKFLOW_REF, the derived ref, and the outcome of each host
in its own group.

Also tries git.hanzo.ai/hanzoai/ci as a third host. The two org axes on that
forge are orthogonal: `hanzo` is the tenant org and `hanzoai` is the mirror
namespace, and this reusable lives under BOTH. Only the tenant path was listed.

Keeps the refs/tags/ strip from v1.0.20: every caller pins @v1, so the imported
ref is refs/tags/v1 and `--branch refs/tags/v1` names no branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 19:33:22 -07:00
hanzo-dev 2c918f5368 wip: preserve in-flight work
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
2026-07-29 16:30:04 -07:00
hanzo-dev a66bd46cb2 fix(ci): universe push retry is shallow-clone-safe (fetch+reset+reapply, not rebase)
v1.0.7's `git pull --rebase` retry never worked: the universe clone is
`--depth 1` shallow, so rebase has no merge base and fails immediately →
push stayed rejected under concurrent rolls, hard-failing the Deploy step
with the expired kubeconfig (hanzoai/world v2.4.44 stuck at 2.4.43).

Replace rebase with a shallow-safe loop: on a non-fast-forward reject,
`git fetch --depth 1 origin main` + `reset --hard FETCH_HEAD`, re-apply the
one-file CR tag change, re-commit, retry (6×). A no-op after reset (remote
already carries our tag) counts as recorded. Never force-push. Verified
against a real shallow clone with a concurrent racing push: our CR change
lands, the racer's other-file commit is preserved.

Claude-Session: https://claude.ai/code/session_01NP4ehjWW2h98FkE4YjUKuJ
2026-07-22 04:03:22 -07:00
hanzo-dev fe00b22aaf fix(ci): universe push rebases+retries on non-fast-forward — concurrent deploys no longer stall the roll
Every service's deploy commits the desired image tag to universe/main, so
when several land at once a plain `git push` loses the race with a
non-fast-forward reject. Before, that set recorded=0 and (with an expired
runner kubeconfig) hard-failed the Deploy step, leaving the CR un-updated —
exactly why hanzoai/world stuck at 2.4.41 while 2.4.42 deploys "failed".

Wrap the push in a rebase-and-retry loop (up to 6): our one-file CR change
replays cleanly onto the moved remote (other services touch other CRs). A
genuine same-CR conflict (a concurrent roll of THIS service) aborts the
rebase and stays recorded=0 — never force, so no silent backward roll.
Verified against a live racing remote: clean race → rebased + pushed,
recorded=1; the racer's commit preserved.

Claude-Session: https://claude.ai/code/session_01NP4ehjWW2h98FkE4YjUKuJ
2026-07-22 00:19:25 -07:00
hanzo-dev 123a72df80 fix(ci): deploy step is GitOps-authoritative — an expired runner kubeconfig no longer false-fails a live deploy
The Deploy step records the desired image tag durably in universe (the
in-cluster operator reconciles it every ~5min) and THEN runs runner-side
kubectl to accelerate + smoke-test the roll. Those kubectl calls were
unguarded, so when the short-lived DOKS kubeconfig from KMS expires (~7d)
every call fails "You must be logged in to the server" and the step reports
failure — even though the operator already rolled the image (hanzoai/world
v2.4.33/34/35 all showed a red Deploy while serving the new build live).

Track `recorded`: once the universe record is pushed (or already pins the
tag), runner kubectl is best-effort (warn, don't fail) because the operator
owns the rollout. With no durable record (bare-Deployment repos) kubectl is
the only path and stays fatal — the real deploy gate is preserved. Verified
under `set -euo pipefail`: recorded=1 → green on auth failure; recorded=0 → fatal.

Claude-Session: https://claude.ai/code/session_01NP4ehjWW2h98FkE4YjUKuJ
2026-07-21 17:02:48 -07:00
hanzo-dev eb8eac8c54 feat(ci): source build_secrets from KMS as --build-arg; honor static images[].args
The build loop read name/ctx/df/repo/sfx/plats but never passed images[].args,
so a repo's declared build args (e.g. sentry's required SENTRY_IMAGE pin) were
silently dropped. Now assemble --build-arg from BOTH static args and KMS-sourced
build_secrets: a repo declares 'build_secrets: [NAME]' per image; the KMS step
fetches each NAME from the same org/path/env and exports it (masked) for the
build step. Undeclared => empty => buildx line byte-for-byte unchanged.

Unblocks world Satellite/Terrain: VITE_MAPBOX_TOKEN now bakes into the Vite SPA
from KMS (hanzo/deploy, env=prod) at build. Key name IS the build-arg name.
2026-07-18 12:29:43 -07:00
hanzo-dev 7f02e0645b fix(ci): upgrade GHCR login to the KMS write:packages token (unblocks cms)
A per-job GITHUB_TOKEN or package-less GH_PAT can't push a package linked to
another repo (ghcr.io/hanzoai/cms 403). New step reads the org ghcr push token
(buildx-ghcr-auth, admin write:packages) from the cluster via the KMS kubeconfig
and re-logs ghcr with it — pushes/creates ANY org package. Fail-safe: public
forks with no KMS keep their repo-linked GITHUB_TOKEN. Native creds from KMS.
2026-07-18 10:36:58 -07:00
hanzo-dev 51b2286b79 fix(ci): GHCR login prefers KMS-backed GH_PAT — push any org package (unblocks cms)
The automatic per-job GITHUB_TOKEN only writes a package linked to THIS repo, so
it 403s on a package created/linked elsewhere (ghcr.io/hanzoai/cms). Use the
KMS-backed GH_PAT (admin:org + write:packages) when present — it pushes/creates
any <org> package — and fall back to the automatic token for public forks that
have no GH_PAT. Native creds from KMS, not GitHub's scoped token.
2026-07-18 10:30:32 -07:00
hanzo-dev 7077cec764 fix(ci): mirror-credential login is best-effort — a registry.hanzo.ai hiccup must not skip the build
Both `docker login registry.hanzo.ai` calls ran unguarded under bash -e, so a
login FAILURE (registry down / transient) aborted the step and SKIPPED the whole
build — the image never reached GHCR (the primary). A missing cred was already
fail-safe; a failed login now is too: warn + skip the mirror, push GHCR-only.
Same best-effort principle as the cloud release lane.
2026-07-17 23:46:54 -07:00
hanzo-dev 1aae2b65a3 ci: pin universe CRs to canonical semver on tag releases (not sha-<short>-amd64)
Deploy roll now pins spec.image.tag to the BARE semver (VER = ref_name w/o v)
on a tagged release — matching world 2.4.10 / cloud v1.801.62 — instead of the
sha-<short>-amd64 it hardcoded on every build. Branch/main pushes keep their
per-commit sha tag (continuous dev path), now arch-matched (bare sha for
multi-arch, fixing a latent -amd64 mismatch).

Build step also publishes the bare semver tag on tag builds (kept the -amd64
alias for back-compat). New semver backward-clobber guard: a branch build never
overwrites a service already pinned to a semver release — complements the
sha-ancestry guard (which only sees sha- tags), so neither kind of pin rolls
backward.

Emitter of the 'deploy(<svc>): sha-...' hanzo-ci commits. Consumed as @v1.
2026-07-17 23:16:48 -07:00
Hanzo AI 93eafe4f55 roll: never backward, sweep every same-repo image in the CR
Builds finish out of order — a slow build of an older commit overwrote newer
rolls (observed live: studio pinned back one merge). The roll now skips when
the CR's sha is a descendant of the builder's. And the tag field alone left
same-image sidecars on stale tags every roll (reconciled by hand four times
tonight); every same-repo image reference in the CR now moves together.
2026-07-17 13:02:41 -07:00
Hanzo AI 4fcc2e02ce fix(ci): provision Node + corepack for JS test gates (exit-127 corepack-not-found on bare arc runners)
The reusable workflow provisions Go and C toolchains for hanzo.yml test
gates but never Node — any JS caller's gate (pnpm install && pnpm lint)
died at 'corepack: command not found' before reading package.json.
Mirror the Go step: guarded to package.json callers, setup-node 22 +
corepack enable so the repo's pinned packageManager shims resolve.
2026-07-16 10:58:04 -07:00
Hanzo AI 2866795e2d fix(ci): deploy rollout-timeout configurable, default 600s (180s failed mid-pull on GB-scale images → studio deploys silently failed since 0.15.8); set-image on repo-matching containers only (never '*' wildcard clobbering rclone sidecars on bare deployments) 2026-07-15 21:59:57 -07:00
zeekay 3083524803 ci: mirror via crane (IAM token realm vs buildx multi-scope) 2026-07-15 02:59:43 -07:00
zeekay 313180e201 ci: mirror prefers direct REGISTRY_USER/PASSWORD (private repos can't see org KMS secrets on Free); kubeconfig fallback retained 2026-07-15 02:43:31 -07:00
zeekay 76cf840edb ci: remove bisect artifacts (root cause: caller-org default workflow permissions must be write for permissions: packages: write reusables) 2026-07-14 23:37:05 -07:00
zeekay 28f1664092 ci: min5/6/7 single-variable bisect 2026-07-14 23:35:39 -07:00
zeekay f3acdc8cf7 ci: min3/min4 bisect 2026-07-14 23:33:00 -07:00
zeekay 02370735a0 ci: min2 header bisect 2026-07-14 23:32:16 -07:00
zeekay 39553ee364 ci: min reusable (cross-org bisect) 2026-07-14 23:31:29 -07:00
zeekay 5ee1b857de fix(ci): test-only callers (no images:) skip the build step cleanly 2026-07-14 23:28:00 -07:00
zeekay e628b788c4 feat(ci): dual-host image push — ghcr.io + registry.hanzo.ai mirror
Public identity stays ghcr.io (GitHub imports keep working); every built tag
is also mirrored server-side (imagetools create) to OUR fleet registry so the
cluster never depends on GHCR to deploy. Credential = the cluster-synced
registry-credentials dockerconfig read via the KMS-fetched kubeconfig;
gracefully skips (GHCR-only) when unavailable. No rebuild, no extra minutes.
2026-07-14 23:26:48 -07:00
hanzo-dev 56e83218ca fix(ci): deploys record desired state in universe, then accelerate
gitops-reconcile re-applies universe CRs every ~5min, so a CR patch (or
set-image) alone is reverted on the next cycle. The deploy step now
bumps infra/k8s/operator/crs/<svc>.yaml in universe (no-op-safe commit,
same KMS git token) and keeps the CR patch/set-image only to make the
roll immediate.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 17:26:57 -07:00
hanzo-dev b5df8baca7 fix(ci): deploy patches the operator Service CR, not the Deployment
The hanzo operator reconciles Deployments from the Service CR — a bare
'kubectl set image' gets reverted on the next reconcile (observed on
world: rolled, served, reverted minutes later). Patch the CR's
spec.image when one exists; keep the Deployment fallback for
non-operator services.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 17:22:08 -07:00
hanzo-dev 3668bb9ebd fix(ci): provision static kubectl for the deploy step
Bare arc runners ship no kubectl; the deploy step died with exit 127
right after the image push. Same sudo-free static-binary pattern as
jq/yq.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 17:16:46 -07:00
hanzo-dev 86428e2f8a fix(ci): survive apt mirror rot in the cgo provision step
arc snapshot images intermittently lose archive.ubuntu.com Release files
(apt-get update exit 100 → every Go repo's build dies before its gates).
Repoint to the DO mirror (sources.list + deb822) and retry once; still a
no-op when gcc is baked in.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 17:04:19 -07:00
hanzo-dev 9f4fd9a959 fix(ci): static yq provisioning — unblocks builds on locked-down arc nodes
PyYAML can't install on arc nodes (sudo blocked by no_new_privs, no pip). Parse
hanzo.yml with a curl-installed static yq binary + jq instead. Proven by the cms
build (past provision→GHCR→KMS). Robust on bare AND pre-baked nodes.
2026-07-13 10:39:40 -07:00
f7426add1d fix(ci): provision Node for JS/TS test gates on bare arc runners (#9)
A hanzo.yml `test:` gate for a JS/TS repo (e.g. `corepack … && pnpm lint`) runs
directly on the runner, but the minimal arc runner image ships no user-PATH Node
(its bundled node is for the runner's own action execution only), so the gate died
`corepack: command not found` (exit 127) — e.g. hanzo.ai's lint gate.

Add a `Provision Node toolchain` step (the JS twin of the existing Go-toolchain
provision): `actions/setup-node@v4` pinned to LTS 22, guarded to repos with a
package.json (`hashFiles('package.json') != ''`) so non-JS callers are unaffected
and harmless if a future runner image bakes Node in. corepack then activates the
exact pnpm/npm the gate requests.

Claude-Session: https://claude.ai/code/session_016yg7GPhYdWCh9vpp4HEwLZ

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 11:20:08 -07:00
0770a28962 fix(ci): make images: optional in hanzo.yml (lint-only / deploy-elsewhere repos) (#8)
The reusable hard-subscripted `hanzo.yml['images']` in the delegate, buildx, and
deploy steps, so a repo that ships no container image (e.g. a static site deployed
via its own Cloudflare Pages deploy.yml, importing this reusable only for the
`test:` lint gate) failed with a Python KeyError before any build ran.

Use `.get('images') or []` in all three places: absent → empty list → the build
loop runs zero times and no GHCR push is attempted. Backward-compatible (every
repo with `images:` is unchanged) and it stops a cross-org repo (e.g.
hanzo-apps/hanzo.ai) from hitting `denied: permission_denied` on a vestigial push.

Claude-Session: https://claude.ai/code/session_016yg7GPhYdWCh9vpp4HEwLZ

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-10 11:08:51 -07:00
zeekayandClaude Opus 4.8 5106bdeada ci: authenticate runner git for private Go modules in the Test step
The Test step runs `go vet`/`go test` on the runner, so `go` fetches
private hanzoai/* modules (GOPRIVATE → direct) via the runner's git. Repos
that authenticate builds with the KMS `gh_token` (GIT_TOKEN) rather than an
org GH_PAT had no runner-git credential, so the gate failed with
`fatal: could not read Username for 'https://github.com'` on
hanzoai/dbx, hanzoai/tasks, hanzoai/pubsub-go, etc.

Add a guarded step (before Test) that configures git `insteadOf` with the
SAME token the image build uses — GIT_TOKEN (set by the KMS step), GH_PAT
fallback — so the runner's git can clone private modules. Gated on go.mod;
no-op when no token is present.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 10:49:15 -07:00
zeekayandClaude Opus 4.8 0339fc7dee ci: provision Go + C toolchain before the Test step on bare arc runners
hanzo.yml `test:` gates run directly on the runner (not in a build
container), but the stock arc runner image ships no Go and no C compiler.
Any Go test gate therefore died with `go: command not found` (exit 127) —
and CGO_ENABLED=1 gates would next hit `cgo: gcc not found`. This was
latent because most callers never reached the Test step (image build failed
first); hanzoai/commerce is the first to build clean and reach a Go gate.

Add two guarded steps before Test, mirroring the existing jq/PyYAML
provisioning: `actions/setup-go@v5` pinned to the repo's own go.mod version,
and a guarded gcc install. Both gated on `hashFiles('go.mod')` so pure JS/TS
callers are unaffected, and both no-op when the toolchain is already present.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 10:42:14 -07:00
hanzo-dev 0542e97a06 fix(ci): derive KMS org from GitHub owner so the deploy-cred fetch actually runs
The KMS deploy-cred step read the org only from hanzo.yml kms.org or the
KMS_ORG var. No repo sets either, so ORG was always empty and the step
short-circuited with 'KMS not configured' — /v1/kms/auth/login was never
called. Every private-dep build then fell back to the org GH_PAT, which
cannot read private hanzoai/cloud, so go mod tidy failed with git exit 128
(commerce/ai/chat images never built; #70 enforcement could not deploy).

Derive ORG from github.repository_owner (hanzoai->hanzo, luxfi->lux,
zooai->zoo; owner as-is otherwise) when unset. hanzo.yml kms.org and
KMS_ORG still override. One place, zero per-repo config. The full path
(login -> token -> GET deploy/GITHUB_TOKEN -> read hanzoai/cloud) is
verified live against kms.hanzo.ai.
2026-07-04 21:56:36 -07:00
265f807635 ci: add opt-in mode: delegate — hand the build to platform.hanzo.ai (#6)
The default `mode: buildx` path is unchanged: buildx → test → deploy ON the
arc runner. `mode: delegate` instead POSTs each hanzo.yml image to platform's
direct build webhook (POST /v1/arcd/enqueue, bearer PLATFORM_BUILD_CALLBACK_TOKEN,
body {repo, sha, image, ref, branch, dockerfile, context, os, arch}) and exits
in seconds — platform builds in-cluster with BuildKit on its own pool, pushes to
the registry, and rolls the operator Service CR. Same downstream as platform's
GitHub-App webhook (one build path, two front doors); no runner buildx, no KMS,
no runner-side deploy.

- new `mode` workflow_call input (default `buildx`) — delegation is opt-in, so
  existing repos are untouched.
- new "Delegate build to platform" step (mode == delegate): parses hanzo.yml,
  enqueues one build per (image, platform) with the buildx tag shape the deploy
  path expects (sha-<short>-<arch>[-<suffix>]); fails loud on a non-202.
- buildx/GHCR/KMS/test/deploy steps gated `mode != 'delegate'`; checkout + parse
  toolchain still run (needed to read hanzo.yml).
- endpoint override via the PLATFORM_ENQUEUE_URL repo/org var.
- README documents the delegate opt-in.

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-04 21:11:03 -07:00
zandGitHub c06948b467 Merge pull request #7 from hanzoai/fix/buildx-ghpat-fallback
fix(ci): fall back buildx gh_token to GH_PAT when KMS token is empty
2026-07-04 21:09:07 -07:00
hanzo-dev 2ef5d47f94 fix(ci): fall back buildx gh_token to GH_PAT when KMS token is empty
The buildx `gh_token` secret (for reading private cross-org Go modules like
github.com/hanzoai/cloud during `go mod tidy`) was mounted ONLY from the
KMS-fetched GIT_TOKEN. When the KMS deploy-cred login fails (or KMS is
unconfigured), GIT_TOKEN is empty, the --secret is omitted, and the buildx
stage's `go mod tidy` hits the private repo unauthenticated -> git exit 128 ->
image build fails. This broke every repo with a private cross-org dep
(hanzoai/commerce, hanzoai/ai) while the KMS login was down.

Fall back to the existing valid GH_PAT org secret — the SAME BuildKit gh_token
cloud/release.yml already uses successfully. Guarded + exported; no-op for
public-only builds when both are empty. One credential path, proven working.
2026-07-04 21:08:47 -07:00
3a9e055c45 feat(ci): opt-in multi-arch builds (amd64+arm64) via hanzo.yml platforms (#5)
DOKS has no arm64 nodes. Add per-image opt-in `platforms:` in hanzo.yml:
default stays [linux/amd64] (every existing repo's -amd64 tag shape UNCHANGED,
zero behavior change). Set [linux/amd64, linux/arm64] → buildx emits a
multi-arch manifest list (one digest, both arches); binfmt/QEMU installed for
arm64 emulation, and pure-Go CGO_ENABLED=0 Dockerfiles honoring $TARGETARCH
cross-compile natively (fast). For native-speed arm64, register a bare-metal
arm64 host (spark/GB10) as the hanzo-build-linux-arm64 runner (values exist).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-04 19:47:09 -07:00
hanzo-dev 8ae1c46625 fix(ci): KMS step must set +e (GitHub wraps run in bash -e)
The step is documented BEST-EFFORT (GHCR push uses the workflow token;
deploy creds are optional), and its 'set -uo pipefail' deliberately omits
-e. But GitHub runs 'run:' under 'bash -eo pipefail', so errexit is active
regardless — an unguarded curl (a KMS secret 404 at the caller's org/path)
aborted the step (exit 22) and failed the whole build. Explicitly 'set +e'
so KMS degrades gracefully (missing kubeconfig -> deploy simply skipped).
2026-07-03 15:42:25 -07:00
hanzo-dev 9d18d4a6ba fix(ci): provision jq + PyYAML on the runner before parsing hanzo.yml
The stock arc runner image (ghcr.io/actions/actions-runner:latest) is
minimal and ships neither jq nor python3-yaml, but the build step parses
hanzo.yml with python3+PyYAML under 'set -euo pipefail' and fails:
  ModuleNotFoundError: No module named 'yaml'
(The KMS step swallowed the same error via '|| true'.)

Add a guarded setup step (apt-get python3-yaml jq) right after checkout —
idempotent, a no-op once a runner image bakes them in. Keeps the reusable
self-contained so any org can import it onto a bare runner.
2026-07-03 15:34:24 -07:00
ecc3da75f0 fix(ci): build.yml runner defaults to online ARC pool, not offline evo (same disease as .github#11) (#3)
Co-authored-by: Hanzo AI <ai@hanzo.ai>
2026-07-02 23:38:52 -07:00
z d425a88e9a docs(brand): add hero banner 2026-06-28 20:06:09 -07:00
z fb096091b3 chore(brand): dynamic hero banner 2026-06-28 20:06:08 -07:00
hanzo-dev aadb7a04c0 ci: drop blanket GOPRIVATE — public modules use the immutable proxy+sumdb
luxfi/hanzoai/zooai Go modules are public (verified: dex, precompile, authz, pq,
age, keys all resolve unauthenticated). Setting GOPRIVATE for the whole orgs
routed them 'direct' and bypassed sum.golang.org, so a force-moved tag silently
poisoned every downstream go.sum (the recurring 'checksum mismatch' build
breakage). Dropping it lets go resolve them via proxy.golang.org + sumdb —
canonical IMMUTABLE hashes a re-publish cannot break. The GH_PAT git credential
stays for authenticated direct fallback + any genuinely-private repo (re-add via
a NARROW GOPRIVATE, never whole-org).
2026-06-28 14:10:59 -07:00
hanzo-dev 599667e100 fix(ci): tag-suffix optional — clean image tags when absent
The reusable build read ."tag-suffix" unconditionally, so any repo without it
(every repo today) got broken tags like sha-xxx-amd64-null. Default to empty in
both build and deploy so single-variant repos get clean tags; multi-variant
repos still qualify (…-ce, ce-latest). Unblocks adopting the reusable workflow.
2026-06-28 05:38:06 -07:00
hanzo-devandGitHub 2bc3b2e1a9 build: KMS secret fetch over canonical /v1/kms + buildx gh_token secret (#2)
* build: KMS secret fetch over canonical /v1/kms + buildx gh_token secret

The Infisical /api/* surface was removed when KMS migrated to luxfi/kms, so the
old /api/v1/auth/universal-auth/login + /api/v3/secrets/raw calls now 404 and
every repo's CI fails to fetch GHCR_TOKEN/KUBECONFIG. Rewrite to /v1/kms:
- auth: POST /v1/kms/auth/login {clientId,clientSecret} (IAM client_credentials
  via the <org>-kms app) -> accessToken
- fetch: per-secret GET /v1/kms/orgs/<org>/secrets/<path>/<name>?env=<env>
- org from hanzo.yml kms.org or KMS_ORG var
Also fetch GIT_TOKEN and pass it to buildx as the gh_token BuildKit secret so
Dockerfiles can clone private Go modules (luxfi/dex, luxfi/precompile). Empty-safe.

* build: one GITHUB_TOKEN in KMS for GHCR push + private-module clone (DRY)

A single GitHub PAT with repo + write:packages does both jobs, so collapse the
split GHCR_TOKEN/GIT_TOKEN to one KMS key GITHUB_TOKEN. KUBECONFIG stays separate
(different concern). Shell var GHTOKEN avoids the reserved GITHUB_ env prefix.

* build: GHCR push via automatic workflow token; KMS only for cross-org clone + kubeconfig

GitHub injects a per-job GITHUB_TOKEN scoped to the running repo; with
permissions: packages: write it can push the repo's own GHCR package (same-org),
so the push path needs no stored credential. KMS is now best-effort, supplying
only what the automatic token can't: a cross-org PAT to clone other orgs' private
Go modules (luxfi/dex, luxfi/precompile) + KUBECONFIG for deploy. Public repos
with no cross-org deps build with zero KMS dependency.
2026-06-24 10:43:03 -07:00
hanzo-devandGitHub 572c712140 ci: private Go module access (luxfi/hanzoai/zooai), guarded by GH_PAT (#1)
* ci: configure private Go module access (luxfi/hanzoai/zooai)

Any go-based `test:` step in a consumer's hanzo.yml needs to fetch
private github.com/{luxfi,hanzoai,zooai}/* modules; without auth go
hits 'git ls-remote … exit 128'. Add a guarded step that wires GH_PAT
via git insteadOf + sets GOPRIVATE job-wide. No-op when GH_PAT is
absent, so non-Go consumers (bootnode etc.) are unaffected.

* ci: make private-module auth runner-state-immune (fresh GIT_CONFIG_GLOBAL)

Upgrade the naive 'git config --global' to the robust pattern proven in
hanzoai/iam: fresh per-job GIT_CONFIG_GLOBAL + GIT_CONFIG_NOSYSTEM=1, probe
from a neutral dir. A plain global config is overridden on shared arc
runners by stale ~/.gitconfig state and actions/checkout's persisted
extraheader. Still a no-op without GH_PAT.
2026-06-22 17:21:04 -07:00
18 changed files with 3600 additions and 779 deletions
+60
View File
@@ -0,0 +1,60 @@
name: sitedeploy
description: Publish a built static export to the Hanzo PaaS Sites plane.
# For every repo whose deploy is "a static export goes live" — hanzo.ai, hanzo.app,
# hips, computer and the ones after them. Those were each about to carry their own
# copy of enqueue → upload → complete, which is how one contract becomes N
# transcriptions that drift (see the imgver action next door for the same story
# told about image tags).
#
# - uses: hanzoai/ci/.github/actions/sitedeploy@v1
# with: { slug: hanzo-ai, dir: out }
# env: { HANZO_DEPLOY_TOKEN: '${{ secrets.HANZO_DEPLOY_TOKEN }}' }
#
# A SITE IS NOT AN APP. This publishes files and stops: no image, no CR, no
# replicas, no registry. Building a container so a Go binary can serve /public is
# the shape the Sites plane exists to retire.
#
# ONE credential. The 202 hands back a prefix-scoped, 30-minute presigned POST
# grant, so CI never holds a bucket key — do NOT add SITES_S3_* here; that is the
# standing shared-bucket credential the grant replaced.
inputs:
slug:
description: The project slug on the Sites plane (POST /v1/projects/<slug>/deploy)
required: true
dir:
description: The built export directory
required: true
api:
description: Cloud API base
required: false
default: https://api.hanzo.ai
jobs:
description: Parallel uploads
required: false
default: '24'
runs:
using: composite
steps:
- name: Fetch sitedeploy
shell: bash
# The action ref is the script ref: an action pinned to @v1 runs v1's
# sitedeploy. Both forges, because this repo is served from each.
run: |
set -euo pipefail
ref="${GITHUB_ACTION_REF:-v1}"
for url in https://git.hanzo.ai/hanzoai/ci https://github.com/hanzoai/ci; do
git clone -q --depth 1 --branch "$ref" "$url" "$RUNNER_TEMP/sitedeploy-ci" 2>/dev/null && break
done
[ -x "$RUNNER_TEMP/sitedeploy-ci/bin/sitedeploy" ] \
|| { echo "::error::could not fetch hanzoai/ci@$ref (bin/sitedeploy)"; exit 1; }
- name: Deploy
shell: bash
env:
HANZO_API: ${{ inputs.api }}
SITEDEPLOY_JOBS: ${{ inputs.jobs }}
SITEDEPLOY_COMMIT: ${{ github.sha }}
SITEDEPLOY_BRANCH: ${{ github.ref_name }}
run: |
set -euo pipefail
bash "$RUNNER_TEMP/sitedeploy-ci/bin/sitedeploy" '${{ inputs.slug }}' '${{ inputs.dir }}'
File diff suppressed because it is too large Load Diff
+864 -385
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -23,5 +23,5 @@ concurrency:
cancel-in-progress: true
jobs:
gate:
uses: hanzoai/ci/.hanzo/workflows/build.yml@v2
uses: hanzoai/ci/.hanzo/workflows/build.yml@v1
secrets: inherit
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2026 Hanzo AI, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+48 -8
View File
@@ -171,10 +171,50 @@ Builds are `CGO_ENABLED=0 -trimpath`: the host that installs this runs it on
whatever base image the host is, and the digest must be a function of the
source, not of the checkout path.
## Runners — our cloud or your own
## `site:` — a static export, promoted to an immutable release
By default the build runs on the **Hanzo cloud** arc pool (we run it; metered as
build minutes). To run on **your own** self-hosted arc runners, pass their labels:
`images:` ships an OCI image a **cluster** runs; `binaries:` ships an executable
a **host** installs. `site:` ships a static export an **edge** serves — no image,
no CR, no replicas. Building a container so a Go binary can serve `/public` is
the shape this retires.
```yaml
site:
slug: hanzo-console # the project on the Sites plane
dir: out # the built export; needs index.html at its root
build: npm ci && npm run build # optional; run first
on: [main] # same branch gate as deploy.on; tags always publish
```
That is the whole configuration. **There is no credential to provision**: the
bearer is the IAM JWT the workflow already mints from `KMS_CLIENT_ID` /
`KMS_CLIENT_SECRET`, so a repo that can build can publish. CI names no bucket and
no org — the org segment is prepended server-side from the validated principal,
which is what makes the prefix unforgeable.
The export is zipped and posted to `/v1/projects/<slug>/deploy`, then that prefix
is promoted by `/v1/sites/<slug>/publish` into an **immutable release** whose id
digests its object manifest. The site's pointer is flipped to it, and the step
then re-reads the release list and refuses unless the release it just published
is the one that is live. Rollback is the same pointer aimed at an older release.
**One size boundary, and it is the server's.** cloud's public edge caps a request
body at 16 MiB (`GATEWAY_BODY_LIMIT`) and refuses a larger POST before any
handler runs — reporting only `Error when parsing request`, which names neither
size nor cause. `bin/sitepublish` therefore measures the zip and refuses *with
the number* first. Of the 24 built exports in the estate, 22 fit; the two that do
not (`hanzo.ai` at 27.9 MiB zipped and 8,536 files, `trillerfest.com` at 76.7
MiB) use [`bin/sitedeploy`](bin/sitedeploy), which streams per-file against a
presigned grant and has no body limit. `hanzo.ai` is also past the server's own
5,000-entry cap, so no transport makes that export a release.
## Runners — our fleet or your own
By default the build runs on the **Hanzo `git-runner` fleet** on git.hanzo.ai
(we run it; metered as build minutes) — the only pool that serves the default
`hanzo-build-linux-amd64` label. There is no arc pool: arc (arcd) was retired
2026-08-01 and never served any label in this default. To run on **your own**
self-hosted runners, pass their labels:
```yaml
uses: hanzoai/ci/.github/workflows/build.yml@v1
@@ -185,7 +225,7 @@ build minutes). To run on **your own** self-hosted arc runners, pass their label
## Delegate to platform (skip runner buildx)
By default the build runs buildx **on** the arc runner. To instead hand the build
By default the build runs buildx **on** the runner. To instead hand the build
to **platform.hanzo.ai** — which builds in-cluster with BuildKit and rolls the
service itself — pass `mode: delegate`:
@@ -197,7 +237,7 @@ service itself — pass `mode: delegate`:
```
The GitHub job then just POSTs each image in `hanzo.yml` to platform's direct
build webhook (`/v1/arcd/enqueue`) and exits in **seconds** — no runner buildx,
build webhook (`/v1/runner`) and exits in **seconds** — no runner buildx,
no KMS, no runner-side deploy. Platform creates the build job, launches an
in-cluster BuildKit Job on its own pool, pushes to the registry, and patches the
operator `Service` CR to roll it. It's the same build path as the platform
@@ -205,10 +245,10 @@ GitHub-App webhook — one build path, two front doors.
Requires one extra secret, `PLATFORM_BUILD_CALLBACK_TOKEN` (org- or repo-level,
picked up via `secrets: inherit`). Override the endpoint with the
`PLATFORM_ENQUEUE_URL` repo/org variable (default `https://platform.hanzo.ai/v1/arcd/enqueue`).
`PLATFORM_ENQUEUE_URL` repo/org variable (default `https://platform.hanzo.ai/v1/runner`).
`mode: buildx` (the default) is unchanged — existing repos keep running buildx on
arc, so delegation is strictly opt-in.
the fleet runner, so delegation is strictly opt-in.
## Credentials
@@ -220,6 +260,6 @@ run time. No long-lived registry or cluster credentials live in GitHub.
## Platform-native
`hanzo.yml` is also read by platform.hanzo.ai: a repo on the platform webhook
needs **only** `hanzo.yml` — the platform builds it on arc and rolls it out, no
needs **only** `hanzo.yml` — the platform builds it in-cluster and rolls it out, no
workflow file at all. This reusable is the GitHub-Actions path for repos that
trigger through GitHub instead of the platform.
+112
View File
@@ -0,0 +1,112 @@
#!/usr/bin/env bash
# conflictmarkers — refuse a repo that has committed an unresolved merge.
# One implementation, every caller.
#
# conflictmarkers [dir]
#
# WHAT THIS CATCHES
#
# `git merge` writes its disagreement INTO the file and stops. Resolving means
# editing those lines out; nothing forces you to. `git add` on a file that still
# contains them is accepted without complaint, and from that moment the markers
# are ordinary committed content — `git status` is clean, `git diff` is empty,
# and the conflict is now a permanent feature of the branch.
#
# It survives because of WHERE it lands. The file is usually generated or
# vendored — large, rarely opened, excluded from review by its own size. Nobody
# reads line 23,807 of a .d.ts. hanzoai/base carried exactly this on main:
#
# plugins/jsvm/internal/types/generated/types.d.ts:23807 <<<<<<< HEAD
# plugins/jsvm/internal/types/generated/types.d.ts:23850 >>>>>>> upstream/master
#
# from an upstream merge nobody finished, sitting in the shipped package.
#
# The compilers are no help, which is the whole problem. In TypeScript the
# markers are a syntax error — but a .d.ts nothing imports is never parsed, so
# there is no error to see. In Go they are a parse error only in a file the
# build reaches. In YAML, JSON, Markdown, SQL, HTML and every config format we
# ship, they are silently VALID content: a marker in a values file is a key
# nobody notices, and a marker in a Markdown doc renders as text.
#
# So the defect class is "both sides of a merge shipped, and nothing in the
# pipeline had an opinion". That is the same shape as its two siblings in this
# step — a declaration and reality disagreeing, with no reader positioned to
# notice — which is why it belongs here rather than in any one repo's gate.
#
# WHAT IT LOOKS FOR, AND WHY NOT MORE
#
# Only the two LABELLED markers git actually writes:
#
# ^<<<<<<< <label> the start marker, seven '<' then a space
# ^======= the divider, seven bare equals
# ^>>>>>>> <label> the end marker, seven '>' then a space
#
# ALL THREE, IN THAT ORDER, EACH ON ITS OWN LINE. Not any one of them — the
# whole shape. That is not belt-and-braces, it is the difference between a gate
# and a nuisance, and it was measured the expensive way: matching the start
# marker ALONE failed hanzoai/app, whose builder tests carry
#
# <<<<<<< START_TITLE index.html >>>>>>> END_TITLE
#
# as literal fixture data. That line opens with seven '<' and a space and is
# not a merge conflict — both markers sit on ONE line, which is a thing git
# never writes. A start marker that also carries an end marker is therefore
# excluded outright, and a file must show the divider and a separate end line
# before this refuses it.
#
# The divider is only ever matched as part of that conjunction, never alone:
# seven bare equals signs at column 1 is also how reStructuredText and Setext
# Markdown underline a heading, and failing every doc in the estate is how a
# gate gets switched off.
#
# Exactly seven, anchored at column 1: `<<<<<<<<` (eight — a heredoc, a C++
# stream) does not match, and neither does an indented mention inside a comment
# explaining conflict markers, which is what lets this file describe them
# without flagging itself.
#
# Tracked files only, via `git grep`, so a stray marker in an untracked scratch
# file or in node_modules cannot fail a build. `-I` skips binaries.
set -euo pipefail
cd "${1:-.}"
git rev-parse --is-inside-work-tree >/dev/null 2>&1 || {
echo "conflictmarkers: not a git work tree ($(pwd)) — skipping"; exit 0; }
# Built from variables rather than written literally, so this script is not its
# own first finding when it scans the repo that carries it.
L=$(printf '<%.0s' $(seq 7))
E=$(printf '=%.0s' $(seq 7))
R=$(printf '>%.0s' $(seq 7))
# Candidate files: those carrying a start marker that is NOT also an end marker
# on the same line. `-l` for names only; the line numbers come later, per file.
cands=$(git grep -I -l -E "^${L} " -- . || true)
bad=""
for f in $cands; do
grep -qE "^${L} " -- "$f" 2>/dev/null || continue
# The same-line form (app's fixture) is not a conflict. Require at least one
# start line that does not also carry an end marker.
grep -E "^${L} " -- "$f" 2>/dev/null | grep -qv -- "$R" || continue
# And require the rest of the shape: a bare divider and a separate end line.
grep -qE "^${E}$" -- "$f" 2>/dev/null || continue
grep -E "^${R} " -- "$f" 2>/dev/null | grep -qv -- "$L" || continue
bad="$bad $f"
done
if [ -n "$bad" ]; then
echo "::error::committed merge-conflict markers — an unresolved merge is in the tree"
for f in $bad; do
grep -nE "^(${L} |${E}$|${R} )" -- "$f" | while IFS= read -r line; do
echo " $f:$line"
done
echo "::error file=${f}::committed conflict marker"
done
echo
echo "Resolve the merge and commit the result. If the file is GENERATED, do not"
echo "hand-edit it — re-run its generator and commit that."
exit 1
fi
echo "OK: no committed conflict markers ($(git ls-files | wc -l) tracked files)"
+118
View File
@@ -0,0 +1,118 @@
#!/usr/bin/env bash
# ignoretracked — refuse a repo that ships bulk content its own .gitignore
# claims to be ignoring. One implementation, every caller.
#
# ignoretracked [dir]
#
# WHAT THIS CATCHES
#
# An ignore rule does not untrack what is already tracked. `.gitignore` is
# consulted when git decides whether to ADD an untracked path; a path already
# in the index is never reconsidered. So this arrangement is stable and silent:
#
# .gitignore says native/flags/target/
# the index says 855 files under native/flags/target/
# git status says nothing
#
# hanzoai/cloud lived there for 6 days and 16 hours. 362 MB of orphaned cargo
# output — three near-identical 37.8 MB staticlibs and a pile of .rlib — went
# into every release, took the module past Go's 500 MiB ceiling (see
# bin/modsize) and made it UNFETCHABLE for nine consecutive releases. The
# `git archive` of that commit is 202 MB against 12.5 MB today: 16x.
#
# The causal story is worth knowing because it is not carelessness, it is a
# race nobody could see. The ignore rule was REMOVED at 12:22 (the Rust
# staticlib was being dropped for a Go evaluator), the 855 files were added at
# 12:51 inside that 4.5-hour window when nothing was ignoring them, and the
# rule was RE-ADDED at 16:48. Every individual step was reasonable. The state
# they combined into is the defect, and no tool in the pipeline had an opinion
# about it.
#
# WHY THIS GATES ON BYTES AND NOT ON PRESENCE
#
# The obvious gate — "any tracked path matched by .gitignore fails" — is
# correct in principle and unshippable in fact. Measured across all 845 git
# repos in the three orgs: 155 of them (18%) carry at least one tracked file
# their .gitignore matches, and hanzoai/cloud's own origin/main is one of them
# (a bare, unanchored `tools` pattern on .gitignore:62 catches 27 legitimately
# tracked Go source files under apps/tools/). `CLAUDE.md` alone is ignored-and-
# tracked in 40 repos and is the ONLY hit in 29 of them. Turning that on as a
# hard fail breaks a fifth of the estate on day one, and a gate that fails what
# ought to pass is a gate that gets switched off — after which we are worse off
# than before it existed.
#
# So the gate is on the quantity that actually caused the outage. The byte
# distribution separates cleanly, which is why this threshold can be a refusal
# rather than a warning:
#
# worst legitimate repo in the estate 61.5 MB (hanzo/docs, vendored)
# ... next 48.8 MB, 45.8, 45.2, 43.8, 39.8
# repos over 100 MB ZERO
# the defect this gate exists for 362.0 MB
#
# Default ceiling 100 MB: green on all 845 repos today with 62% headroom over
# the worst honest case, and red on the real defect by 3.6x. Nothing to
# baseline, no allowlist, no per-repo exemptions — the tail is REPORTED (so the
# hygiene problem stays visible and shrinkable) and only the bulk is REFUSED.
#
# As repos are cleaned up, lower IGNORETRACKED_MAX_MB. It is a ratchet, and the
# end state is 0 — at which point this becomes the pure presence gate that was
# right all along. Do not start there.
#
# WHY core.excludesFile IS PINNED OFF
#
# `--exclude-standard` reads THREE sources: the repo's committed .gitignore
# files, .git/info/exclude, and the user's global core.excludesFile. The last
# is per-machine, so the same commit gets different verdicts on a laptop and a
# runner — on this workstation a global bare `tags` pattern matched 1,271
# Elixir source files in lux/explorer-v1 that CI would never flag. A gate whose
# answer depends on whose machine asked is not a gate. Only committed
# .gitignore is in scope, so the global file is pinned to /dev/null.
# (.git/info/exclude is per-clone and is the stock all-comments template on a
# fresh CI checkout; it cannot be overridden by -c, and is left alone.)
#
# EXIT: 0 clean or under the ceiling (findings still reported), 1 over it.
set -uo pipefail
MAX_MB=${IGNORETRACKED_MAX_MB:-100}
root=${1:-.}
cd "$root" 2>/dev/null || { echo "ignoretracked: no such directory: $root" >&2; exit 1; }
git rev-parse --git-dir >/dev/null 2>&1 || exit 0 # not a repo: nothing to check
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
# -s gives "<mode> <sha> <stage>\t<path>", which carries the blob id — so the
# size comes from cat-file and needs no working-tree stat (correct on a bare or
# sparse checkout, and unaffected by anything a build has written).
git -c core.excludesFile=/dev/null ls-files -s --cached --ignored --exclude-standard \
> "$tmp/ls" 2>/dev/null || exit 0
[ -s "$tmp/ls" ] || { echo "ignoretracked: OK — nothing tracked that .gitignore ignores"; exit 0; }
files=$(wc -l < "$tmp/ls")
bytes=$(awk '{print $2}' "$tmp/ls" \
| git cat-file --batch-check='%(objectsize)' --buffer 2>/dev/null \
| awk '/^[0-9]+$/{s+=$1} END{print s+0}')
mb=$(awk -v b="${bytes:-0}" 'BEGIN{printf "%.1f", b/1048576}')
# WHICH .gitignore LINE is doing this. Plain `check-ignore` prints nothing for a
# tracked path (it answers "would git ignore this if it were untracked", and a
# tracked path short-circuits) — --no-index is what makes it answer for the
# files we actually have. The output names source:line:pattern, so the report
# points at the line to edit instead of at 855 paths to read.
cut -f2- "$tmp/ls" | git -c core.excludesFile=/dev/null check-ignore --no-index -v --stdin \
> "$tmp/why" 2>/dev/null || true
echo "ignoretracked: ${files} tracked file(s), ${mb} MB, matched by this repo's own .gitignore"
if [ -s "$tmp/why" ]; then
echo " by rule (top 10):"
awk -F'\t' '{n[$1]++} END{for (k in n) printf "%8d %s\n", n[k], k}' "$tmp/why" \
| sort -rn | head -10
fi
limit=$(( MAX_MB * 1048576 ))
if [ "${bytes:-0}" -ge "$limit" ]; then
echo "::error::${mb} MB of tracked content is matched by this repo's own .gitignore — over the ${MAX_MB} MB ceiling. An ignore rule does not untrack what is already tracked, so this ships in every release and every module zip while \`git status\` stays clean. Fix: \`git rm -r --cached <path>\` and commit (the working tree is untouched). If the content is meant to be tracked, un-ignore it instead — negate the rule with \`!<path>\` — so the two stop disagreeing."
exit 1
fi
echo "ignoretracked: OK — ${mb} MB is under the ${MAX_MB} MB ceiling (reported, not blocking)"
exit 0
Executable
+123
View File
@@ -0,0 +1,123 @@
#!/usr/bin/env bash
# modsize — refuse a Go module that is approaching the size at which Go can no
# longer fetch it. One implementation, every caller.
#
# modsize [dir] # default: every module in the repo, from the root
#
# WHAT THIS CATCHES
#
# Go's module ceiling is not a soft limit and not a warning. From
# golang.org/x/mod/zip (and, identically, cmd/go/internal/modfetch/codehost):
#
# MaxZipFile = 500 << 20 // 524288000 bytes, 500 MiB
#
# and it is enforced THREE ways on the same number — the zip file itself, the
# total uncompressed size of the files inside it, and (in CheckDir) the running
# sum of the source tree's file sizes:
#
# "module source tree too large (max size is 524288000 bytes)"
# "total uncompressed size of module contents too large (...)"
#
# hanzoai/cloud crossed it and became UNFETCHABLE for nine consecutive
# releases. Nothing said so. The module publishes fine — the ceiling is
# enforced on the CONSUMER, at `go get`, so the failure surfaces in someone
# else's repo, at a version they did not choose, long after the release that
# caused it. There is no signal at the publishing end at all, which is why nine
# releases went out before anybody knew.
#
# WHY THIS SUMS RAW BYTES AND WHY THAT IS EXACT, NOT AN APPROXIMATION
#
# It would be reasonable to assume the limit is on the compressed zip and that
# summing uncompressed bytes over-counts. It does not: zip.CheckDir accumulates
# `info.Size()` — the raw, uncompressed size of each regular file — and fails
# when that running total exceeds MaxZipFile. So the sum of file sizes IS one
# of the three quantities Go bounds, and a git blob's size is exactly that
# file's uncompressed size. This gate therefore measures the same number Go
# measures, not a proxy for it.
#
# WHAT IT EXCLUDES, AND WHY EACH EXCLUSION IS REQUIRED FOR CORRECTNESS
#
# • Nested modules. A subdirectory with its own go.mod is a DIFFERENT module
# and its bytes are not in the parent's zip. Counting them would fail a
# repo that is nowhere near the limit — hanzoai/s3 is this shape (it builds
# s3-rdma-sidecar/ and telemetry/server/ from their own go.mod files), and
# a gate that fails a repo that would have worked is a gate people learn to
# skip. Each nested module is instead checked on its own terms.
# • Symlinks and gitlinks (submodules). zip.CheckDir skips both — symlinks
# explicitly (golang.org/issue/27093), submodules because they are not
# files. Counting a gitlink's 20-byte entry would be harmless; counting a
# symlink's target as content would not.
#
# It reads tracked content only (git ls-tree), because that is what a module
# zip is built from: the proxy serves what the VCS has at that tag, not what a
# working tree happens to contain.
#
# THE THRESHOLD IS BELOW THE CEILING ON PURPOSE
#
# Failing AT 500 MiB would be useless — at that point the module is already
# unfetchable and the only question left is how many releases shipped broken.
# The gate refuses at MODSIZE_MAX_PCT (default 80%, ~419 MiB), which is the
# "warn well before" — spelled as a refusal, because a warning about a cliff
# nobody is watching is the same defect one level up. 105 MiB of headroom is
# several releases' worth of honest growth, so this fires with room to fix it
# calmly and never fires the release it would have broken.
#
# EXIT: 0 clean, 1 when a module is at or past the threshold.
set -uo pipefail
# 500 << 20, from golang.org/x/mod/zip. Not a guess and not rounded: the
# constant is quoted so a reader can check it against the source.
CEILING=$((500 << 20))
PCT=${MODSIZE_MAX_PCT:-80}
REF=${MODSIZE_REF:-HEAD}
root=${1:-.}
cd "$root" 2>/dev/null || { echo "modsize: no such directory: $root" >&2; exit 1; }
git rev-parse --git-dir >/dev/null 2>&1 || exit 0 # not a repo: nothing to measure
# Every go.mod in the tree, at REF. The root module is "go.mod"; anything else
# is a nested module and marks a prefix the parent must not count.
mods=$(git ls-tree -r --name-only "$REF" 2>/dev/null | grep -E '(^|/)go\.mod$' | sort) || exit 0
[ -z "$mods" ] && exit 0
# path<TAB>size for every regular tracked file. Mode 100644/100755 only:
# 120000 is a symlink and 160000 a gitlink, both of which zip.CheckDir omits.
# `git ls-tree -l` separates the path with a TAB, so split on that rather than
# on whitespace: a path with spaces in it must survive intact.
sizes=$(git ls-tree -r -l "$REF" 2>/dev/null \
| sed -n 's/^\([0-9]\{6\}\) blob \([0-9a-f]*\) *\([0-9-]*\)\t\(.*\)$/\1\t\3\t\4/p' \
| awk -F'\t' '$1=="100644"||$1=="100755"{print $3 "\t" $2}')
human() { awk -v b="$1" 'BEGIN{ printf "%.1f MiB", b/1048576 }'; }
rc=0
for gm in $mods; do
if [ "$gm" = "go.mod" ]; then mdir=""; else mdir="${gm%/go.mod}/"; fi
# The prefixes this module must NOT count: every OTHER module nested under it.
nested=$(printf '%s\n' "$mods" | while read -r o; do
[ "$o" = "$gm" ] && continue
od="${o%go.mod}"
case "$od" in "$mdir"?*) printf '%s\n' "$od";; esac
done)
total=$(printf '%s\n' "$sizes" | awk -F'\t' -v m="$mdir" -v nl="$nested" '
BEGIN{ n=split(nl, arr, "\n") }
{
p=$1
if (m != "" && index(p, m) != 1) next # not in this module
for (i=1; i<=n; i++) if (arr[i] != "" && index(p, arr[i]) == 1) next # nested module
s += $2
}
END{ print s+0 }')
limit=$(( CEILING * PCT / 100 ))
name=${mdir:-./}
if [ "$total" -ge "$CEILING" ]; then
echo "::error file=${gm}::module ${name} is $(human "$total") of tracked content — PAST Go's $(human $CEILING) module ceiling. It is already unfetchable: \`go get\` fails with 'module source tree too large (max size is ${CEILING} bytes)' in every consumer, at every version that carries it. Find the bulk with: git ls-tree -r -l ${REF} | sort -k4 -n | tail -20"
rc=1
elif [ "$total" -ge "$limit" ]; then
echo "::error file=${gm}::module ${name} is $(human "$total") of tracked content — ${PCT}% of Go's $(human $CEILING) ceiling, $(human $((CEILING-total))) of headroom left. Past the ceiling the module stops being fetchable AT THE CONSUMER, so the break shows up in someone else's build and not in this one. Find the bulk with: git ls-tree -r -l ${REF} | sort -k4 -n | tail -20"
rc=1
else
echo "modsize: OK — ${name} $(human "$total") ($(( total * 100 / CEILING ))% of $(human $CEILING))"
fi
done
exit $rc
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env bash
# publishable — refuse a `build_secrets` name that does not declare itself public.
#
# Usage: publishable <hanzo.yml> (reads images[].build_secrets, rc=1 on refusal)
#
# WHY A NAME AND NOT A VALUE. A build_secret is handed to buildx as
# `--build-arg NAME=value`, and a build-arg is IN THE PUBLISHED IMAGE: `docker
# history` prints it to anyone who can pull. So the only value that may sit
# here is one that is public on purpose — a Vite/Next static export has no
# server to read an env from, so its ingest key must be inlined at build, and
# inlining it is what publishing it means.
#
# The value cannot be the test. At the moment ci reads hanzo.yml the value does
# not exist yet (KMS has not been called), and guessing secrecy from a string's
# shape is a heuristic that is wrong in both directions. The name is what a
# reviewer reads, it is in git, and it is decided by the person who knows the
# answer. So the name carries the assertion.
#
# The fleet had already started saying it this way — hanzoai/docs renamed
# EVENT_INGEST_KEY -> PUBLISHABLE_KEY, and hanzoai/world's four are VITE_*,
# which a bundler inlines by construction. This turns that convention into the
# mechanism. Before it, the ONE repo whose value is genuinely publishable
# (hanzoai/ui, a `pk-…` key) asserted the prefix inside its own Dockerfile, so
# the check existed once, for one image, and any repo adding a real credential
# got no check at all.
set -uo pipefail
f=${1:-hanzo.yml}
[ -r "$f" ] || exit 0 # no config, nothing declared, nothing to refuse
names=$(yq -r '[(.images // [])[] | (.build_secrets // [])[]] | unique | .[]' "$f" 2>/dev/null) || exit 0
[ -n "$names" ] || exit 0
rc=0
for n in $names; do
case "$n" in
# Prefixes a bundler already treats as client-side, plus an explicit
# self-declaration for everything else. Anything outside this set has not
# claimed to be public, so it is not baked.
PUBLISHABLE_*|PUBLIC_*|NEXT_PUBLIC_*|EXPO_PUBLIC_*|NUXT_PUBLIC_*|VITE_*|REACT_APP_*|*_PUBLISHABLE|*_PUBLIC) ;;
*)
echo "::error::build_secret '$n' does not declare itself publishable, and a build_secret is baked into the image as a --build-arg where \`docker history\` reveals it. Rename it (PUBLISHABLE_*, PUBLIC_*, NEXT_PUBLIC_*, VITE_*, REACT_APP_*) if the value is public on purpose; if it is a real credential it cannot be a build_secret at all." >&2
rc=1 ;;
esac
done
exit $rc
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/env bash
# Tests for bin/publishable. Offline and deterministic: every case is a
# hanzo.yml written into a temp dir. Run: bash bin/publishable_test.sh
set -uo pipefail
cd "$(dirname "$0")/.."
PUB="$PWD/bin/publishable"
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
fail=0
# t <name> <want-rc> <build_secrets yaml-inline list>
t() {
local name=$1 want=$2 list=$3
local d="$tmp/$RANDOM$RANDOM"; mkdir -p "$d"
{ echo 'images:'; echo ' - name: app'; echo ' repo: ghcr.io/hanzoai/app'
[ -n "$list" ] && echo " build_secrets: $list"; } > "$d/hanzo.yml"
out=$(bash "$PUB" "$d/hanzo.yml" 2>&1); rc=$?
if [ "$rc" = "$want" ]; then printf 'ok %-56s rc=%s\n' "$name" "$rc"
else printf 'FAIL %-56s rc=%s (want %s)\n %s\n' "$name" "$rc" "$want" "$out"; fail=1; fi
}
echo "--- refused: a name that never claimed to be public ---"
# The live case. hanzoai/ui declares exactly this, and its value IS publishable
# — but nothing outside its own Dockerfile could know that.
t "EVENT_INGEST_KEY is refused" 1 '[EVENT_INGEST_KEY]'
t "a real credential is refused" 1 '[STRIPE_SECRET_KEY]'
t "a token is refused" 1 '[GITHUB_TOKEN]'
t "a password is refused" 1 '[DB_PASSWORD]'
t "a private key is refused" 1 '[SIGNING_PRIVATE_KEY]'
t "one bad name among good ones is refused" 1 '[VITE_GTM_ID, EVENT_INGEST_KEY]'
echo "--- allowed: the name declares it ---"
# These are the fleet's real declarations, verbatim.
t "PUBLISHABLE_KEY (hanzoai/docs)" 0 '[PUBLISHABLE_KEY]'
t "VITE_MAPBOX_TOKEN (hanzoai/world)" 0 '[VITE_MAPBOX_TOKEN]'
t "VITE_SENTRY_DSN (hanzoai/world)" 0 '[VITE_SENTRY_DSN]'
t "world's four together" 0 '[VITE_MAPBOX_TOKEN, VITE_SENTRY_DSN, VITE_ANALYTICS_WEBSITE_ID, VITE_GTM_ID]'
t "NEXT_PUBLIC_ prefix" 0 '[NEXT_PUBLIC_INGEST_KEY]'
t "REACT_APP_ prefix" 0 '[REACT_APP_MAP_KEY]'
t "EXPO_PUBLIC_ prefix" 0 '[EXPO_PUBLIC_API_KEY]'
t "NUXT_PUBLIC_ prefix" 0 '[NUXT_PUBLIC_API_KEY]'
t "PUBLIC_ prefix" 0 '[PUBLIC_ANALYTICS_ID]'
t "_PUBLISHABLE suffix" 0 '[STRIPE_PUBLISHABLE]'
t "_PUBLIC suffix" 0 '[ANALYTICS_ID_PUBLIC]'
echo "--- silent: nothing declared, nothing to say ---"
# 44 of the fleet's 47 repos are this case and must be byte-for-byte unchanged.
t "no build_secrets key at all" 0 ''
t "empty build_secrets list" 0 '[]'
echo "--- a missing file is not a refusal ---"
out=$(bash "$PUB" "$tmp/does-not-exist.yml" 2>&1); rc=$?
if [ "$rc" = 0 ]; then printf 'ok %-56s rc=0\n' "absent hanzo.yml is silent"
else printf 'FAIL %-56s rc=%s\n' "absent hanzo.yml is silent" "$rc"; fail=1; fi
echo "--- the refusal says what to do about it ---"
d="$tmp/msg"; mkdir -p "$d"
printf 'images:\n - name: app\n build_secrets: [EVENT_INGEST_KEY]\n' > "$d/hanzo.yml"
out=$(bash "$PUB" "$d/hanzo.yml" 2>&1)
for pat in "EVENT_INGEST_KEY" "docker history" "PUBLISHABLE_" "cannot be a build_secret"; do
if printf '%s' "$out" | grep -qF "$pat"; then printf 'ok %-56s\n' "message names '$pat'"
else printf 'FAIL %-56s\n got: %s\n' "message names '$pat'" "$out"; fail=1; fi
done
[ "$fail" = 0 ] && echo "PASS" || echo "FAIL"
exit $fail
Executable
+238
View File
@@ -0,0 +1,238 @@
#!/usr/bin/env bash
# sitedeploy — publish a built static export to the Hanzo PaaS Sites plane.
# One implementation, every static site.
#
# sitedeploy <slug> <dir> # e.g. sitedeploy hanzo-ai out
#
# WHY THIS IS A SCRIPT AND NOT INLINE SHELL: the same forty lines of enqueue →
# upload → complete were about to be pasted into hanzo.ai, hanzo.app, hips,
# computer and every static surface after them. Written N times it is right N-1
# times and then wrong once — which is exactly how `sha-<short>` became the only
# tag eleven repos ever published (see bin/imgver). The plane's contract lives
# here, once.
#
# THE THREE STEPS, and why the bytes never pass through the API:
#
# POST /v1/projects/<slug>/deploy -> 202 {id, bucket, prefix, upload}
# POST <upload.url> per file -> the bytes, straight to S3
# POST /v1/projects/<slug>/deployments/<id>/complete
#
# A real export is large — hanzo.ai is 128 MB across 8403 files — and cloud's
# BodyLimit is 16 MiB, so an artifact POST is refused by fasthttp BEFORE any
# handler runs, and it fails as an opaque 400 "Error when parsing request" that
# reads like a malformed payload rather than a size cap. The git source is the
# documented route for exactly this shape.
#
# NO STANDING S3 CREDENTIAL. The 202 carries `upload`: a presigned POST policy
# that is prefix-scoped (starts-with $key "<org>/<slug>/", enforced by S3 itself),
# short-lived (30 min) and size-bounded. That replaced handing every repo the
# bucket's own long-lived access key — one key for a bucket whose only tenant
# separation is the key prefix, so every repo holding it could overwrite EVERY
# org's site (cloud apps/projects/grant.go). Do not reintroduce SITES_S3_* here.
# HANZO_DEPLOY_TOKEN is the ONE credential this needs.
#
# DELETION IS THE SERVER'S. The grant authorizes writes only, so CI cannot remove
# a file; `keys` in the completion is the manifest cloud reconciles the prefix
# against, and it deletes what the build no longer produces. That is where
# `aws s3 sync --delete` went. It fails CLOSED on an empty manifest, and so does
# this script — a build that enumerated nothing has failed, and honouring it
# literally would delete the live site.
#
# ENV: HANZO_DEPLOY_TOKEN (required) the org-scoped `sk-` key; mint at POST /v1/keys
# HANZO_API (https://api.hanzo.ai)
# SITEDEPLOY_JOBS (24) parallel uploads
# SITEDEPLOY_COMMIT / SITEDEPLOY_BRANCH recorded on the deployment
# SITEDEPLOY_PLAN=1 print the manifest and exit; no network. The test seam.
set -euo pipefail
slug="${1:?usage: sitedeploy <slug> <dir>}"
dir="${2:?usage: sitedeploy <slug> <dir>}"
api="${HANZO_API:-https://api.hanzo.ai}"
jobs="${SITEDEPLOY_JOBS:-24}"
[ -d "$dir" ] || { echo "::error::$dir is not a directory — the build produced no export"; exit 1; }
# ---- content type -----------------------------------------------------------
# The presigned POST carries no Content-Type condition, so whatever CI sends is
# what the object stores — and what it stores is what the edge serves. Send
# nothing and every page is application/octet-stream, which a browser DOWNLOADS
# instead of rendering: a green deploy that serves an unusable site.
ctype() {
case "${1##*.}" in
html|htm) echo 'text/html; charset=utf-8' ;;
css) echo 'text/css; charset=utf-8' ;;
js|mjs) echo 'text/javascript; charset=utf-8' ;;
json) echo 'application/json; charset=utf-8' ;;
xml) echo 'application/xml; charset=utf-8' ;;
txt) echo 'text/plain; charset=utf-8' ;;
svg) echo 'image/svg+xml' ;;
png) echo 'image/png' ;;
jpg|jpeg) echo 'image/jpeg' ;;
gif) echo 'image/gif' ;;
webp) echo 'image/webp' ;;
avif) echo 'image/avif' ;;
ico) echo 'image/x-icon' ;;
woff) echo 'font/woff' ;;
woff2) echo 'font/woff2' ;;
ttf) echo 'font/ttf' ;;
otf) echo 'font/otf' ;;
wasm) echo 'application/wasm' ;;
pdf) echo 'application/pdf' ;;
webmanifest) echo 'application/manifest+json' ;;
map) echo 'application/json' ;;
*) echo 'application/octet-stream' ;;
esac
}
# ---- cache control ----------------------------------------------------------
# Mirrors cloud's apps/sites.CacheControlFor so a site cached the same whichever
# route its bytes took. bin/sitedeploy_test.sh pins these strings; if the server
# changes its policy the test is what catches the drift.
#
# A fingerprinted asset (Vite/Next/webpack emit app.4f3a9c21.js) is immutable: a
# new build changes the hash, so the old URL can be cached forever.
#
# The class is written `[._-]` with the dash LAST and the pattern held in a
# variable. Go's regexp spells the same set `[.\-_]`, and transcribing that
# literally into `[[ =~ ]]` is a silent defect twice over: a backslash is literal
# inside a POSIX bracket expression, so `\-_` reads as the RANGE \…_ and the shell
# rejects it as "invalid character range" — and because the `if` merely evaluates
# false, every fingerprinted asset quietly fell back to max-age=3600 instead of
# immutable. An unquoted inline pattern also has its own quoting hazards; the
# variable form is the one that is read as a regex rather than a glob.
fingerprint_re='[._-][0-9a-fA-F]{8,}\.[a-z0-9]+$'
cachectl() {
local k="$1" base="${1##*/}"
case "${k##*.}" in
html|htm) echo 'public, max-age=60, s-maxage=86400' ;;
js|mjs|css|woff|woff2|png|jpg|jpeg|gif|svg|webp|avif|ico|ttf|otf|wasm|data|pck|unityweb|mem)
if [[ "$base" =~ $fingerprint_re ]]; then
echo 'public, max-age=31536000, immutable'
else
echo 'public, max-age=3600'
fi ;;
*) echo 'public, max-age=3600' ;;
esac
}
# ---- the manifest -----------------------------------------------------------
# Paths RELATIVE to the export root, which is exactly what the completion's
# `keys` must carry (cloud reconciles `keep[rel]` against them).
#
# CNAME does not travel. It is a GitHub Pages artifact that means nothing to S3
# and would ship a stale hostname claim into the bucket.
manifest=$(cd "$dir" && find . -type f ! -name CNAME | sed 's|^\./||' | LC_ALL=C sort)
count=$(printf '%s' "$manifest" | grep -c . || true)
[ "$count" -gt 0 ] || { echo "::error::$dir contains no files — refusing to deploy an empty manifest"; exit 1; }
if [ -n "${SITEDEPLOY_PLAN:-}" ]; then
printf 'slug=%s dir=%s files=%s\n' "$slug" "$dir" "$count"
while IFS= read -r k; do [ -n "$k" ] && printf '%s\t%s\t%s\n' "$k" "$(ctype "$k")" "$(cachectl "$k")"; done <<< "$manifest"
exit 0
fi
: "${HANZO_DEPLOY_TOKEN:?HANZO_DEPLOY_TOKEN is unset. It is the ONE credential this needs; set it on the forge (git.hanzo.ai), which is what reads .hanzo/workflows — not on GitHub. Mint: POST /v1/keys {\"type\":\"secret\"}}"
# ---- 0. ensure the project exists (self-provision) --------------------------
# A brand-new slug has no project, and the enqueue below 404s on a missing one —
# which is why every never-before-deployed site failed its first run. Create it
# idempotently here instead: 200/201 the first time, 409 every time after, both
# success. So a new site needs no out-of-band `POST /v1/projects`; committing the
# workflow is enough. The repo is linked because the git-source enqueue requires
# a linked project (deployGit rejects an unlinked one); SITEDEPLOY_REPO overrides
# the origin for a forge checkout whose remote is not the canonical repo.
repo="${SITEDEPLOY_REPO:-$(git remote get-url origin 2>/dev/null || true)}"
pcode=$(curl -sS -o /tmp/sd-proj.json -w '%{http_code}' \
-X POST "$api/v1/projects" \
-H "Authorization: Bearer $HANZO_DEPLOY_TOKEN" -H 'Content-Type: application/json' \
-d "$(jq -nc --arg s "$slug" --arg u "$repo" --arg b "${SITEDEPLOY_BRANCH:-main}" \
'{slug:$s, name:$s} + (if $u=="" then {} else {repo:{url:$u, branch:$b}} end)')")
case "$pcode" in
200|201|409) : ;; # created now, or already there — either is the state we need
*) echo "::error::ensure project $slug returned HTTP $pcode"; head -c 400 /tmp/sd-proj.json; echo; exit 1 ;;
esac
# ---- 1. enqueue -------------------------------------------------------------
# 202 Accepted is the success code: the deployment is queued, not live. bucket
# and prefix come FROM cloud (sitePrefix(org, slug) is server-side) — never guess
# them, or the upload lands where nothing is served the moment an org or slug
# changes.
code=$(curl -sS -o /tmp/sd-enq.json -w '%{http_code}' \
-X POST "$api/v1/projects/$slug/deploy" \
-H "Authorization: Bearer $HANZO_DEPLOY_TOKEN" -H 'Content-Type: application/json' \
-d "{\"source\":\"git\",\"commit\":\"${SITEDEPLOY_COMMIT:-}\",\"branch\":\"${SITEDEPLOY_BRANCH:-main}\"}")
if [ "$code" != "202" ]; then
echo "::error::enqueue $api/v1/projects/$slug/deploy returned HTTP $code"; head -c 600 /tmp/sd-enq.json; echo; exit 1
fi
dep=$(jq -r '.id' /tmp/sd-enq.json)
prefix=$(jq -r '.prefix' /tmp/sd-enq.json)
upload_url=$(jq -r '.upload.url // empty' /tmp/sd-enq.json)
if [ -z "$upload_url" ]; then
echo "::error::the 202 carried no upload grant, so there is no way to write the bytes."
echo " cloud mints one only when presigning is configured (S3_ADMIN_* on the cloud deployment)."
exit 1
fi
jq -c '{id,version,status,bucket,prefix}' /tmp/sd-enq.json
# A build that dies after this point would leave the deployment "queued" and the
# project stuck "building" forever. Report the failure so cloud records an honest
# terminal state instead of a lie by omission.
fail() {
curl -sS -X POST "$api/v1/projects/$slug/deployments/$dep/complete" \
-H "Authorization: Bearer $HANZO_DEPLOY_TOKEN" -H 'Content-Type: application/json' \
-d '{"status":"error","message":"CI upload failed"}' >/dev/null 2>&1 || true
echo "::error::marked deployment $dep as error"
}
trap 'fail' ERR
# ---- 2. the bytes -----------------------------------------------------------
# `file` goes LAST: S3 ignores every field after the file part, so a grant field
# trailing the body is silently dropped and the signature check fails.
#
# `key` is DROPPED from the grant's fields and re-sent per object. The grant
# carries key="<org>/<slug>/" — the starts-with PLACEHOLDER, not a destination —
# and forwarding it verbatim alongside the real key posts `key` twice, which S3
# answers 400 for every object. That is the whole of the first end-to-end run:
# 8403 files, 8403 400s. Everything else in the map (bucket, policy, x-amz-*) is
# covered by the signature and must travel untouched.
jq -r '.upload.fields | to_entries[] | select(.key != "key") | "-F\n\(.key)=\(.value)"' /tmp/sd-enq.json > /tmp/sd-fields
put() {
local rel="$1" args=() line
# A read loop, NOT `mapfile`: mapfile is bash 4+, and macOS ships bash 3.2, so
# on a dev box it fails as `command not found`, the array stays EMPTY, and every
# upload goes out with no policy or signature at all. S3 then rejects it for a
# malformed X-Amz-Credential — an error that points at the credential rather
# than at the array that never got built.
while IFS= read -r line; do args+=("$line"); done < /tmp/sd-fields
curl -sS --fail-with-body -o /dev/null \
-X POST "$UP_URL" \
-F "key=$PREFIX/$rel" "${args[@]}" \
-F "Content-Type=$(ctype "$rel")" -F "Cache-Control=$(cachectl "$rel")" \
-F "file=@$DIR/$rel" \
|| { echo "::error::upload failed: $rel"; return 1; }
}
export -f put ctype cachectl
export UP_URL="$upload_url" PREFIX="$prefix" DIR="$dir" fingerprint_re
echo "uploading $count files to s3://$(jq -r .bucket /tmp/sd-enq.json)/$prefix ($jobs parallel)"
printf '%s\n' "$manifest" | grep . | xargs -P "$jobs" -I{} bash -c 'put "$@"' _ {}
# ---- 3. flip it live --------------------------------------------------------
# `wc -c`, not `stat`: the size flag is spelled -f%z on BSD and -c%s on GNU, and a
# `stat -f%z || stat -c%s` fallback does not work — on Linux the first arm fails
# INSIDE the pipeline, awk still exits 0, and the `||` never fires, so the byte
# count silently reports 0. wc is the one spelling both agree on.
bytes=$(cd "$dir" && find . -type f ! -name CNAME -exec wc -c {} + | awk '$2!="total"{s+=$1} END{print s+0}')
jq -n --arg c "${SITEDEPLOY_COMMIT:-}" --argjson f "$count" --argjson b "${bytes:-0}" \
--args '{status:"live",commit:$c,files:$f,bytes:$b,keys:$ARGS.positional}' \
$(printf '%s\n' "$manifest" | grep .) > /tmp/sd-done.json
code=$(curl -sS -o /tmp/sd-resp.json -w '%{http_code}' \
-X POST "$api/v1/projects/$slug/deployments/$dep/complete" \
-H "Authorization: Bearer $HANZO_DEPLOY_TOKEN" -H 'Content-Type: application/json' \
--data-binary @/tmp/sd-done.json)
trap - ERR
if [ "$code" != "200" ]; then
echo "::error::complete returned HTTP $code"; head -c 600 /tmp/sd-resp.json; echo; fail; exit 1
fi
jq -c '{status,liveUrl,version,files,bytes}' /tmp/sd-resp.json
+80
View File
@@ -0,0 +1,80 @@
#!/usr/bin/env bash
# Tests for bin/sitedeploy. Runs OFFLINE: SITEDEPLOY_PLAN=1 stops the script
# before the first network call and prints the manifest it would upload, so every
# case here is deterministic and needs no token, no bucket and no cluster.
# Run: bash bin/sitedeploy_test.sh
set -uo pipefail
cd "$(dirname "$0")/.."
SD="$PWD/bin/sitedeploy"
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
fail=0
plan() { SITEDEPLOY_PLAN=1 bash "$SD" a-slug "$1" 2>&1; }
t() { # t <name> <got> <want>
if [ "$2" = "$3" ]; then printf 'ok %-56s -> %s\n' "$1" "$2"
else printf 'FAIL %-56s -> %s (want %s)\n' "$1" "$2" "$3"; fail=1; fi
}
# field <dir> <key> <col> — the ctype (2) or cachectl (3) column for one key
field() { plan "$1" | awk -F'\t' -v k="$2" -v c="$3" '$1==k{print $c}'; }
site="$tmp/site"; mkdir -p "$site/assets" "$site/nested/deep"
echo '<h1>hi</h1>' > "$site/index.html"
echo 'body{}' > "$site/assets/app.4f3a9c21.css"
echo 'x' > "$site/assets/plain.css"
echo 'y' > "$site/assets/chunk-AB12CD34.js"
echo '{}' > "$site/data.json"
echo 'z' > "$site/nested/deep/page.html"
echo 'hanzo.ai' > "$site/CNAME"
# --- the manifest ------------------------------------------------------------
# Keys are RELATIVE to the export root: cloud reconciles keep[rel] against them,
# so a leading ./ or an absolute path would match nothing and the completion
# would prune the entire live site.
t "keys are relative, no leading ./" "$(plan "$site" | awk -F'\t' 'NR>1&&$1~/^\.?\//{print "ABS"}' | head -1)" ""
t "nested paths keep their subdirs" "$(plan "$site" | awk -F'\t' '$1=="nested/deep/page.html"{print "yes"}')" "yes"
# CNAME is a GitHub Pages artifact: it means nothing to S3 and would ship a stale
# hostname claim into the bucket.
t "CNAME does not travel" "$(plan "$site" | awk -F'\t' '$1=="CNAME"{print "leaked"}')" ""
t "file count excludes CNAME" "$(plan "$site" | head -1 | grep -o 'files=[0-9]*')" "files=6"
# --- content type ------------------------------------------------------------
# The presigned POST carries no Content-Type condition, so what CI sends is what
# the object stores and what the edge serves. Send nothing and a browser
# DOWNLOADS every page instead of rendering it.
t "html" "$(field "$site" index.html 2)" "text/html; charset=utf-8"
t "css" "$(field "$site" assets/plain.css 2)" "text/css; charset=utf-8"
t "js" "$(field "$site" assets/chunk-AB12CD34.js 2)" "text/javascript; charset=utf-8"
t "json" "$(field "$site" data.json 2)" "application/json; charset=utf-8"
# --- cache control: mirrors cloud apps/sites.CacheControlFor -----------------
# These strings are the SERVER's policy, pinned here so the two cannot drift
# apart silently. If cloud changes CacheControlFor, this is what goes red.
t "html is short-lived, long at the edge" "$(field "$site" index.html 3)" "public, max-age=60, s-maxage=86400"
t "unfingerprinted asset is an hour" "$(field "$site" assets/plain.css 3)" "public, max-age=3600"
# The regression this pins: Go spells the class [.\-_], and transcribing that
# into [[ =~ ]] makes the shell reject it as an invalid character range. The `if`
# then merely evaluates false, so every hashed asset silently lost `immutable`.
t "fingerprinted .hash. is immutable" "$(field "$site" assets/app.4f3a9c21.css 3)" "public, max-age=31536000, immutable"
t "fingerprinted -HASH- is immutable" "$(field "$site" assets/chunk-AB12CD34.js 3)" "public, max-age=31536000, immutable"
# --- fail closed -------------------------------------------------------------
# reconcilePrefix deletes whatever the manifest omits, so an empty manifest is a
# request to delete the live site. A build that enumerated nothing has failed.
empty="$tmp/empty"; mkdir -p "$empty"
plan "$empty" >/dev/null 2>&1
t "empty export is refused" "$?" "1"
only_cname="$tmp/onlycname"; mkdir -p "$only_cname"; echo x > "$only_cname/CNAME"
plan "$only_cname" >/dev/null 2>&1
t "a dir holding only CNAME is empty too" "$?" "1"
plan "$tmp/does-not-exist" >/dev/null 2>&1
t "missing export dir is refused" "$?" "1"
# --- the credential ----------------------------------------------------------
# Not in PLAN mode (that is the offline seam), but a real run must refuse to
# start rather than enqueue a deployment it cannot complete.
out=$(HANZO_DEPLOY_TOKEN= bash "$SD" a-slug "$site" 2>&1); rc=$?
t "no token: exits non-zero" "$rc" "1"
t "no token: says which secret" "$(printf '%s' "$out" | grep -c HANZO_DEPLOY_TOKEN)" "1"
[ $fail -eq 0 ] && echo "PASS" || echo "FAIL"
exit $fail
+220
View File
@@ -0,0 +1,220 @@
#!/usr/bin/env bash
# sitepublish — publish a built static export as an immutable Release on the
# Hanzo Sites plane. One implementation, every `site:` in the fleet.
#
# sitepublish <slug> <dir> # e.g. sitepublish hanzo-console out
#
# WHY THIS EXISTS: the reusable's `site:` lane used to stage the export with
# `mc mirror` against hanzoai/s3, which needed S3_ADMIN_ACCESS_KEY and
# S3_ADMIN_SECRET_KEY. Those names are not in KMS for any org, and the lane
# fails closed without them — so `site:` refused for every caller that ever
# declared it, and hanzoai/console had to grow its own build→zip→publish
# workflow to ship at all. Per-repo build logic is the one thing this repo
# exists to prevent. The bytes now travel the SAME route console proved, and
# the credential is the one CI already holds.
#
# THE TWO STEPS:
#
# POST /v1/projects/<slug>/deploy Content-Type: application/zip, zip as body
# POST /v1/sites/<slug>/publish {"source":"<slug>"}
#
# The first lands the export at the org's site prefix; the second promotes that
# prefix into an immutable Release and flips it live. `source` is ORG-RELATIVE
# and the org segment is prepended server-side from the validated principal
# (cloud apps/projects/blob.go:60, `sitePrefix(org, slug) = org + "/" + slug`),
# which is why "<slug>" is the whole of it and why a caller cannot address
# another tenant's bytes by writing a longer path.
#
# ONE CREDENTIAL, ALREADY MINTED. HANZO_API_TOKEN is the IAM JWT the workflow's
# KMS step already holds: cloud's /v1/kms/auth/login is a broker that performs
# the IAM client_credentials exchange and returns IAM's own JWT verbatim, so the
# token that reads this org's secrets is the token that publishes this org's
# site. api.hanzo.ai validates it and mints X-Org-Id from the `owner` claim —
# identity is never a header this script sends, so there is nothing here to
# spoof. Do NOT reintroduce S3_ADMIN_*; a standing bucket key was what this
# replaced.
#
# THE SIZE BOUNDARY IS THE SERVER'S, NOT A PREFERENCE. cloud's public edge sets
# BodyLimit from GATEWAY_BODY_LIMIT, default 16 MiB (cloud config.go:312), and
# fasthttp refuses an oversized POST BEFORE any handler runs — it surfaces as an
# opaque 400 "Error when parsing request" that reads like a malformed payload
# rather than a size cap (cloud apps/projects/grant.go:5 tells that story about
# a ~170 MB export). So the zip is measured HERE and refused HERE, with the
# number and the alternative, instead of being sent to fail unreadably. Measured
# across the estate's 24 built exports, 22 fit; hanzo.ai (27.9 MiB zipped, 8536
# files) and trillerfest.com (76.7 MiB) do not. Those go to bin/sitedeploy,
# which streams per-file against a presigned grant and has no body limit —
# hanzo.ai is over the server's own 5000-entry cap anyway, so no transport makes
# it a Release.
#
# ENV: HANZO_API_TOKEN (required) the IAM bearer; the workflow's KMS step mints it
# HANZO_API (https://api.hanzo.ai)
# SITEPUBLISH_MAX_ZIP (16777216) mirror of the server's GATEWAY_BODY_LIMIT
# SITEPUBLISH_PLAN=1 print what it would send and exit; no network. Test seam.
set -euo pipefail
slug="${1:?usage: sitepublish <slug> <dir>}"
dir="${2:?usage: sitepublish <slug> <dir>}"
api="${HANZO_API:-https://api.hanzo.ai}"
# Server-side caps, mirrored so a breach is named in CI instead of arriving as a
# 413 (or, for the body limit, as an unreadable 400). cloud apps/projects/blob.go
# :29-31. bin/sitepublish_test.sh pins these numbers; if cloud moves them the
# test is what catches the drift.
max_zip="${SITEPUBLISH_MAX_ZIP:-16777216}" # gateway BodyLimit, 16 MiB
max_files=5000 # maxFiles
max_file_bytes=$((64 << 20)) # maxFileBytes, 64 MiB
max_total_bytes=$((512 << 20)) # maxTotalBytes uncompressed, 512 MiB
die() { echo "::error::$*"; exit 1; }
# The slug is a URL path segment AND an S3 key segment, so it is held to the
# grammar cloud validates project slugs with — checked before anything is built
# or sent, because a bad slug is a typo to fix and not a 404 to interpret.
echo "$slug" | grep -qE '^[a-z0-9]([a-z0-9-]{0,38}[a-z0-9])?$' \
|| die "slug '$slug' is not a project slug (^[a-z0-9]([a-z0-9-]{0,38}[a-z0-9])?\$)"
[ -d "$dir" ] || die "'$dir' is not a directory — the build produced no export"
# A site is a thing with an index. cloud enforces it however the bytes arrive;
# checking here turns a failed build into a failed publish with a readable cause
# rather than a promoted release that 404s at its own root.
[ -f "$dir/index.html" ] || die "'$dir' has no index.html at its root — /v1/sites refuses a source without one"
# CNAME does not travel: a GitHub Pages artifact that means nothing to S3 and
# would ship a stale hostname claim into the bucket. Same exclusion bin/sitedeploy
# makes, for the same reason.
files=$(cd "$dir" && find . -type f ! -name CNAME | wc -l)
[ "$files" -gt 0 ] || die "'$dir' contains no files — refusing to publish an empty release"
[ "$files" -le "$max_files" ] \
|| die "'$dir' holds $files files; cloud caps one artifact at $max_files (apps/projects/blob.go maxFiles). Use bin/sitedeploy — but note the Releases plane has the same cap, so this export cannot become a Release."
bytes=$(cd "$dir" && find . -type f ! -name CNAME -exec wc -c {} + | awk '$2!="total"{s+=$1} END{print s+0}')
[ "$bytes" -le "$max_total_bytes" ] \
|| die "'$dir' is $bytes bytes uncompressed; cloud caps an artifact at $max_total_bytes (maxTotalBytes)"
# Per-file cap, checked with find rather than a loop so a 9000-file export costs
# one traversal. -size uses 512-byte blocks with `c` for bytes; +N c is "strictly
# greater than N bytes", which is the cap's own boundary.
big=$(cd "$dir" && find . -type f ! -name CNAME -size +${max_file_bytes}c -printf '%P (%s bytes)\n' | head -3)
[ -z "$big" ] || die "these files exceed cloud's ${max_file_bytes}-byte per-file cap (maxFileBytes):
$big"
if [ -n "${SITEPUBLISH_PLAN:-}" ]; then
printf 'slug=%s dir=%s files=%s bytes=%s api=%s source=%s\n' \
"$slug" "$dir" "$files" "$bytes" "$api" "$slug"
exit 0
fi
# No apostrophe in this message: inside "${VAR:?word}" a single quote opens a
# quoted section for the PARSER, and the script dies at EOF with "unexpected EOF
# while looking for matching quote" — a syntax error reported at the last line,
# nowhere near the one that caused it.
: "${HANZO_API_TOKEN:?HANZO_API_TOKEN is unset. It is the IAM bearer minted by the KMS step of the reusable workflow, from KMS_CLIENT_ID/KMS_CLIENT_SECRET; there is no second credential to seal.}"
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
zipf="$tmp/site.zip"
# -X drops extra file attributes (uid/gid/timestamps beyond the DOS fields) so
# the same tree zips to the same bytes on any runner. Entries are stored
# RELATIVE to the export root — cloud strips a leading "./" and refuses absolute
# or ".."-escaping names (blob.go:362-370), and strips a common top-level prefix
# if one exists, so a flat archive is what it expects and what this sends.
( cd "$dir" && zip -qXr "$zipf" . -x './CNAME' ) || die "zip of '$dir' failed"
zbytes=$(wc -c < "$zipf")
# The refusal the whole size story is about. Above this the POST dies at the
# edge with a 400 that names nothing.
[ "$zbytes" -le "$max_zip" ] || die "the zipped export is $zbytes bytes, past cloud's ${max_zip}-byte edge BodyLimit (GATEWAY_BODY_LIMIT, cloud config.go). fasthttp refuses the POST before any handler runs and reports only 'Error when parsing request', so this is refused here where the number is visible. Publish this export with bin/sitedeploy, which streams per-file against a presigned grant and has no body limit."
echo "publishing $slug — $files files, $bytes bytes ($zbytes zipped)"
# ---- 1. the bytes -----------------------------------------------------------
# Every response body is KEPT and printed on failure. cloud answers these routes
# with a specific status per cause — 402 hosting gate, 403 wrong org, 404 no such
# site for this tenant, 409 the source moved mid-publish, 413 past the caps, 503
# storage unconfigured — and each is a different fix, so discarding the body
# discards the answer.
code=$(curl -sS -o "$tmp/deploy.json" -w '%{http_code}' \
-X POST "$api/v1/projects/$slug/deploy" \
-H "Authorization: Bearer $HANZO_API_TOKEN" \
-H 'Content-Type: application/zip' \
--data-binary "@$zipf") \
|| die "POST $api/v1/projects/$slug/deploy did not complete"
case "$code" in
2??) ;;
*) echo "::error::upload $slug → HTTP $code"; head -c 800 "$tmp/deploy.json"; echo; exit 1 ;;
esac
# ---- 2. promote + activate --------------------------------------------------
# `source` is the org-relative prefix the upload just landed at. Content types
# and cache policy are not ours to set: copyRelease rewrites the metadata on
# every object it promotes, so a release serves identically however its bytes
# were staged.
code=$(curl -sS -o "$tmp/publish.json" -w '%{http_code}' \
-X POST "$api/v1/sites/$slug/publish" \
-H "Authorization: Bearer $HANZO_API_TOKEN" \
-H 'Content-Type: application/json' \
-d "$(jq -nc --arg s "$slug" '{source:$s}')") \
|| die "POST $api/v1/sites/$slug/publish did not complete"
case "$code" in
2??) ;;
*) echo "::error::publish $slug → HTTP $code"; head -c 800 "$tmp/publish.json"; echo; exit 1 ;;
esac
# Guard the SHAPE before reading a field out of it. `jq -r '.releaseId // empty'`
# on an error body, an array, or a null yields "" and every later test on it
# passes vacuously — which is the defect class this whole script is written
# against. Assert it is an object carrying the field, then read it.
jq -e 'type == "object" and (.releaseId | type) == "string" and (.releaseId | length) > 0' \
"$tmp/publish.json" >/dev/null \
|| { echo "::error::publish answered $code but the body is not a release object with a releaseId:"; head -c 800 "$tmp/publish.json"; echo; exit 1; }
rid=$(jq -r '.releaseId' "$tmp/publish.json")
objs=$(jq -r '.objects // 0' "$tmp/publish.json")
url=$(jq -r '.url // ""' "$tmp/publish.json")
# ---- 3. verify the release we just made is the one that is live -------------
# A 200 from publish says the request was accepted, not that this release is
# serving. The list is the only thing that can say so.
#
# THE LIST IS A BARE JSON ARRAY — `type projectsReleases []projectsRelease`
# (cloud apps/projects/release.go:518). `.releases[]` against it resolves
# NOTHING, and an assertion that resolves nothing is an assertion that cannot
# fail for the reason it was written. Hence `.[]`, and hence the shape guard
# first: if cloud ever wraps this in an object, THAT is what goes red, loudly,
# instead of the check quietly matching zero releases forever.
code=$(curl -sS -o "$tmp/releases.json" -w '%{http_code}' \
"$api/v1/sites/$slug/releases" \
-H "Authorization: Bearer $HANZO_API_TOKEN") \
|| die "GET $api/v1/sites/$slug/releases did not complete"
case "$code" in
2??) ;;
*) echo "::error::list releases for $slug → HTTP $code"; head -c 800 "$tmp/releases.json"; echo; exit 1 ;;
esac
jq -e 'type == "array" and length > 0' "$tmp/releases.json" >/dev/null \
|| { echo "::error::GET /v1/sites/$slug/releases did not answer a non-empty JSON array (it is the bare-array shape from release.go:518). Body:"; head -c 800 "$tmp/releases.json"; echo; exit 1; }
# Exactly one release is active. Counting first is what makes "none active"
# distinguishable from "the wrong one is active" — a bare grep for '"active":true'
# matches ANY release in the list and would pass on both.
nactive=$(jq '[.[] | select(.active == true)] | length' "$tmp/releases.json")
[ "$nactive" = 1 ] \
|| { echo "::error::expected exactly 1 active release for $slug, found $nactive:"; jq -c '[.[]|{releaseId,active}]' "$tmp/releases.json"; exit 1; }
active=$(jq -r 'map(select(.active == true))[0].releaseId // ""' "$tmp/releases.json")
# Both sides proven non-empty BEFORE they are compared. `[ "$a" = "$b" ]` with
# two empty strings is TRUE, so an equality test on unguarded values reports
# success precisely when it learned nothing.
[ -n "$active" ] || { echo "::error::the active release for $slug carries no releaseId:"; jq -c '.' "$tmp/releases.json"; exit 1; }
[ -n "$rid" ] || die "publish returned no releaseId to verify against"
[ "$active" = "$rid" ] \
|| { echo "::error::published $rid but $active is live for $slug — the flip did not take"; exit 1; }
echo "live: $rid ($objs objects) → ${url:-no public host — that slug is claimed; rename the site or bind a domain}"
if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
{ echo "### Site — \`$slug\`"; echo
echo "| release | objects | files | zipped | url |"
echo "|---|---|---|---|---|"
echo "| \`$rid\` | $objs | $files | $zbytes B | ${url:-—} |"
} >> "$GITHUB_STEP_SUMMARY"
fi
+168
View File
@@ -0,0 +1,168 @@
#!/usr/bin/env bash
# Tests for bin/sitepublish. Runs OFFLINE and deterministically: a `curl` shim on
# PATH answers the three routes from fixtures, so every case here — including the
# ones that must FAIL — needs no token, no bucket and no cluster.
# Run: bash bin/sitepublish_test.sh
#
# The suite is weighted toward REFUSALS on purpose. A publish step that reports
# success when it verified nothing is worse than one that does not verify at all,
# because it is indistinguishable from a working one until a site silently stops
# updating. Three real defects of that exact shape are pinned below by name:
#
# • an object-shaped read (`.releaseId`) against a body that is not an object
# • `.releases[]` against the BARE ARRAY /v1/sites/<slug>/releases returns,
# which resolves nothing and therefore can never fail for its stated reason
# • `[ "$a" = "$b" ]` on two values that are both empty, which is TRUE
set -uo pipefail
cd "$(dirname "$0")/.."
SP="$PWD/bin/sitepublish"
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
fail=0
t() { # t <name> <want_rc> <got_rc> [<must-contain> <output>]
local name="$1" want="$2" got="$3" needle="${4:-}" out="${5:-}"
if [ "$got" != "$want" ]; then
printf 'FAIL %-58s rc=%s (want %s)\n' "$name" "$got" "$want"; fail=1; return
fi
if [ -n "$needle" ] && ! printf '%s' "$out" | grep -qF -- "$needle"; then
printf 'FAIL %-58s rc=%s but missing %q\n' "$name" "$got" "$needle"; fail=1
printf ' got: %s\n' "$(printf '%s' "$out" | head -c 300)"; return
fi
printf 'ok %-58s rc=%s\n' "$name" "$got"
}
# ---- a curl shim -------------------------------------------------------------
# Dispatches on the URL and writes the fixture the scenario names to the file the
# real curl would have written, then prints the status the way `-w %{http_code}`
# does. Everything the script does with a response goes through this, so the
# tests exercise the REAL parsing, not a mock of it.
shim="$tmp/bin"; mkdir -p "$shim"
cat > "$shim/curl" <<'SHIM'
#!/usr/bin/env bash
out=/dev/null url=
while [ $# -gt 0 ]; do
case "$1" in
-o) out="$2"; shift 2 ;;
http*|https*) url="$1"; shift ;;
*) shift ;;
esac
done
case "$url" in
*/deploy) printf '%s' "${T_DEPLOY_BODY:-{\"ok\":true\}}" > "$out"; printf '%s' "${T_DEPLOY_CODE:-200}" ;;
*/publish) printf '%s' "${T_PUBLISH_BODY:-}" > "$out"; printf '%s' "${T_PUBLISH_CODE:-200}" ;;
*/releases) printf '%s' "${T_LIST_BODY:-}" > "$out"; printf '%s' "${T_LIST_CODE:-200}" ;;
*) echo "shim: unexpected url $url" >&2; exit 9 ;;
esac
SHIM
chmod +x "$shim/curl"
site="$tmp/site"; mkdir -p "$site/assets"
echo '<h1>hi</h1>' > "$site/index.html"
echo 'body{}' > "$site/assets/app.css"
echo 'hanzo.ai' > "$site/CNAME"
run() { # run <dir> — publish with the shim on PATH, current T_* scenario
PATH="$shim:$PATH" HANZO_API_TOKEN=tok HANZO_API=https://api.test \
bash "$SP" a-slug "$1" 2>&1
}
OK_PUB='{"releaseId":"rel-1","slug":"a-slug","objects":2,"active":true,"url":"https://a-slug.hanzo.page"}'
OK_LIST='[{"releaseId":"rel-1","active":true},{"releaseId":"rel-0","active":false}]'
# ---- the plan seam: no network, no token ------------------------------------
out=$(SITEPUBLISH_PLAN=1 bash "$SP" a-slug "$site" 2>&1); rc=$?
t "plan prints the manifest and exits" 0 $rc "files=2" "$out"
# CNAME is a GitHub Pages artifact: it means nothing to S3 and would ship a stale
# hostname claim. Two files, not three.
t "plan excludes CNAME from the count" 0 $rc "files=2 " "$out"
t "plan sends source=<slug>, org-relative" 0 $rc "source=a-slug" "$out"
# ---- refusals that need no network ------------------------------------------
out=$(SITEPUBLISH_PLAN=1 bash "$SP" 'Bad_Slug' "$site" 2>&1); rc=$?
t "an invalid slug is refused before anything" 1 $rc "is not a project slug" "$out"
out=$(SITEPUBLISH_PLAN=1 bash "$SP" a-slug "$tmp/nope" 2>&1); rc=$?
t "a missing export dir is refused" 1 $rc "is not a directory" "$out"
noidx="$tmp/noidx"; mkdir -p "$noidx"; echo x > "$noidx/page.html"
out=$(SITEPUBLISH_PLAN=1 bash "$SP" a-slug "$noidx" 2>&1); rc=$?
t "an export with no index.html is refused" 1 $rc "no index.html at its root" "$out"
empty="$tmp/empty"; mkdir -p "$empty"; echo x > "$empty/index.html"; rm "$empty/index.html"
out=$(SITEPUBLISH_PLAN=1 bash "$SP" a-slug "$empty" 2>&1); rc=$?
t "an empty export is refused" 1 $rc "no index.html" "$out"
# The size refusal is the whole reason this script measures before it sends:
# past the edge BodyLimit fasthttp rejects the POST before any handler runs and
# reports only "Error when parsing request", which names neither size nor cause.
out=$(SITEPUBLISH_MAX_ZIP=1 PATH="$shim:$PATH" HANZO_API_TOKEN=tok \
bash "$SP" a-slug "$site" 2>&1); rc=$?
t "a zip past the edge BodyLimit is refused here" 1 $rc "BodyLimit" "$out"
t " ...and the refusal names bin/sitedeploy" 1 $rc "bin/sitedeploy" "$out"
# The server caps one artifact at 5000 entries; a 5001-file export can never
# become a Release by ANY transport, so saying so here beats a 413 later.
many="$tmp/many"; mkdir -p "$many"; echo x > "$many/index.html"
( cd "$many" && touch f{1..5001} )
out=$(SITEPUBLISH_PLAN=1 bash "$SP" a-slug "$many" 2>&1); rc=$?
t "an export past maxFiles=5000 is refused" 1 $rc "cloud caps one artifact at 5000" "$out"
# ---- the happy path ----------------------------------------------------------
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY="$OK_LIST" run "$site"); rc=$?
t "publish + verified flip succeeds" 0 $rc "live: rel-1" "$out"
# ---- transport failures carry the server's answer ---------------------------
# Each status is a different fix, so the body is printed rather than swallowed.
out=$(T_DEPLOY_CODE=413 T_DEPLOY_BODY='{"error":"artifact exceeds"}' run "$site"); rc=$?
t "a non-2xx upload fails and prints the body" 1 $rc "artifact exceeds" "$out"
out=$(T_PUBLISH_CODE=402 T_PUBLISH_BODY='{"error":"hosting not enabled"}' run "$site"); rc=$?
t "a non-2xx publish fails and prints the body" 1 $rc "hosting not enabled" "$out"
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_CODE=503 T_LIST_BODY='{"error":"storage"}' run "$site"); rc=$?
t "a non-2xx release list fails" 1 $rc "list releases" "$out"
# ---- defect 1: an object-shaped read against a non-object -------------------
# `jq -r '.releaseId // empty'` yields "" for an array, a null, or an error body,
# and every later test on "" passes vacuously. The shape is asserted first.
out=$(T_PUBLISH_BODY='[{"releaseId":"rel-1"}]' T_LIST_BODY="$OK_LIST" run "$site"); rc=$?
t "publish answering an ARRAY is refused" 1 $rc "not a release object" "$out"
out=$(T_PUBLISH_BODY='{"ok":true}' T_LIST_BODY="$OK_LIST" run "$site"); rc=$?
t "publish answering no releaseId is refused" 1 $rc "not a release object" "$out"
out=$(T_PUBLISH_BODY='{"releaseId":""}' T_LIST_BODY="$OK_LIST" run "$site"); rc=$?
t "publish answering an EMPTY releaseId is refused" 1 $rc "not a release object" "$out"
out=$(T_PUBLISH_BODY='not json' T_LIST_BODY="$OK_LIST" run "$site"); rc=$?
t "publish answering non-JSON is refused" 1 $rc "not a release object" "$out"
# ---- defect 2: the list is a BARE ARRAY -------------------------------------
# release.go:518 is `type projectsReleases []projectsRelease`. A `.releases[]`
# filter resolves NOTHING against that, so an assertion built on it can only ever
# refuse — it never once tested what it claimed to. The guard demands the array
# shape, so the day cloud wraps the list THAT is what goes red, by name.
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='{"releases":[{"releaseId":"rel-1","active":true}]}' run "$site"); rc=$?
t "a WRAPPED release list is refused, loudly" 1 $rc "bare-array shape" "$out"
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='[]' run "$site"); rc=$?
t "an EMPTY release list is refused" 1 $rc "non-empty JSON array" "$out"
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='null' run "$site"); rc=$?
t "a null release list is refused" 1 $rc "non-empty JSON array" "$out"
# ---- defect 3: counting, so "none" cannot read as "yes" ---------------------
# A bare `grep '"active":true'` matches ANY release in the list, so it passes
# both when the wrong release is live and when the list merely mentions one.
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='[{"releaseId":"rel-1","active":false}]' run "$site"); rc=$?
t "ZERO active releases is refused" 1 $rc "expected exactly 1 active release" "$out"
out=$(T_PUBLISH_BODY="$OK_PUB" \
T_LIST_BODY='[{"releaseId":"rel-1","active":true},{"releaseId":"rel-2","active":true}]' run "$site"); rc=$?
t "TWO active releases is refused" 1 $rc "expected exactly 1 active release" "$out"
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='[{"releaseId":"rel-9","active":true}]' run "$site"); rc=$?
t "a DIFFERENT release being live is refused" 1 $rc "the flip did not take" "$out"
# The both-empty comparison, head on: an active entry whose releaseId is "".
# `[ "$active" = "$rid" ]` with both empty is TRUE and would report success on a
# release that does not exist. Non-emptiness is proven before the comparison.
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='[{"releaseId":"","active":true}]' run "$site"); rc=$?
t "an active release with an EMPTY id is refused" 1 $rc "carries no releaseId" "$out"
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='[{"active":true}]' run "$site"); rc=$?
t "an active release with NO id field is refused" 1 $rc "carries no releaseId" "$out"
# ---- the credential is required ---------------------------------------------
out=$(PATH="$shim:$PATH" HANZO_API_TOKEN= bash "$SP" a-slug "$site" 2>&1); rc=$?
t "a missing bearer refuses before any request" 1 $rc "HANZO_API_TOKEN is unset" "$out"
echo
[ "$fail" = 0 ] && echo "sitepublish: all tests passed" || echo "sitepublish: FAILURES"
exit "$fail"
Executable
+161
View File
@@ -0,0 +1,161 @@
#!/usr/bin/env bash
# vendormark — refuse a repo that carries a site-generator vendor's branding.
# One implementation, every caller.
#
# vendormark [dir] [git-range]
#
# WHAT THIS CATCHES
#
# Several sites in the estate began life as output from a hosted "prompt to
# React app" generator. The generator does not sign its work in a comment you
# would notice — it signs it in the SHAPE of the repo, in four places at once:
#
# 1. an asset folder named after the vendor, e.g. public/<vendor>-uploads/,
# so every image the browser requests carries the vendor's name in its URL
# 2. a build plugin dependency in package.json and the lockfiles
# 3. an injected third-party <script> tag in index.html
# 4. commit messages, author identities and Co-authored-by trailers written
# by the vendor's bot
#
# The fourth is the one nothing else looks at, and the one that survives every
# amount of tidying the working tree: a repo can render a perfect page and
# still say, in `git log`, who really wrote it.
#
# This was not a hypothetical. A sweep of 1,065 repositories across the three
# orgs found the mark in five shipped sites — one of them serving the vendor's
# upload folder as its FAVICON, so the vendor's path was in the <head> of every
# page and in every browser tab. Purging it needed a history rewrite and a
# force-push of every branch. This gate exists so that is never needed twice.
#
# WHY THE CONTENT RULES ARE SO SPECIFIC
#
# The vendor's name is also an ordinary English adjective, and the naive gate —
# case-insensitive substring of that word — is unshippable. Measured across all
# 1,065 repos, the bare word appears legitimately in 20+ of them and NONE of
# them are contaminated:
#
# tokenizer vocab.json (3 repos) "lovable</w>": 38565
# English word lists words_alpha.txt, faker adjectives
# ML training/eval corpora alpaca, kto, BAGEL eval prompts
# x/net publicsuffix packed data a substring of concatenated domains
# vendored prompt corpora third-party collections that quote it
# our own marketing copy a competitor named in a case study
#
# A gate that reds a fifth of the estate on day one is a gate someone switches
# off, and then we are worse off than before it existed. So the content rules
# are only the strings that CANNOT occur in prose: a path segment, an npm
# package name, an injected script URL, a generator meta tag, a bot's email.
#
# Two candidate rules were DROPPED after measuring them, and it is worth saying
# why: `lovableproject.com` and `lovable.app` red an upstream Clerk SDK bundle,
# which ships a list of dev-preview host suffixes naming several generators. A
# third party's honest list of other people's hosts is not our contamination.
# Those two strings are still refused in commit messages and git identities,
# where no third party can put them.
#
# PATHS AND IDENTITIES ARE ABSOLUTE
#
# Content is judged narrowly; paths and git identities are not. There is no
# legitimate reason for the vendor's name to appear in a filename we track or
# in an author, committer or Co-authored-by line we write. Those are refused on
# the bare word, no exceptions, because that is where the mark is load-bearing
# and where tidying the working tree never reaches.
#
# THE ESCAPE HATCH
#
# `.vendormark-allow` — one path glob per line, `#` comments ignored. It exists
# for exactly one honest case: a repo whose JOB is to name these vendors, such
# as the history-scrubber rule files in hanzoai/.github, which cannot do their
# work without spelling the string they remove. If you are reaching for it for
# any other reason, you are about to ship the mark.
#
# EXIT: 0 clean, 1 mark found.
set -uo pipefail
root=${1:-.}
range=${2:-}
cd "$root" 2>/dev/null || { echo "vendormark: no such directory: $root" >&2; exit 1; }
git rev-parse --git-dir >/dev/null 2>&1 || exit 0 # not a repo: nothing to check
# PATHS: the bare vendor name, no exceptions. Add a vendor here and its
# filenames are refused everywhere.
VENDORS='lovable'
# CONTENT: only strings that cannot occur in prose — an asset folder, an npm
# package, the injected script, a generator meta tag, the bot's address. Spelled
# out in full rather than composed from $VENDORS, because half of them do not
# contain the vendor's name at all. See the header for the two that were
# measured against the estate and dropped.
CONTENT_SIG='lovable-uploads/|lovable-tagger|@lovable\.dev|cdn\.gpteng\.co|gptengineer\.js|content="[Ll]ovable"'
# IDENTITIES AND MESSAGES: the bare word plus the vendor hosts, because nothing
# outside our own commits can write these.
IDENT_SIG='lovable|gptengineer|gpt-engineer'
fail=0
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
# --- allowlist ------------------------------------------------------------
: > "$tmp/allow"
if [ -f .vendormark-allow ]; then
grep -vE '^\s*(#|$)' .vendormark-allow > "$tmp/allow" || true
fi
allowed() { # $1 = path
[ -s "$tmp/allow" ] || return 1
while IFS= read -r g; do
# shellcheck disable=SC2254
case "$1" in $g) return 0;; esac
done < "$tmp/allow"
return 1
}
# --- 1. tracked paths -----------------------------------------------------
git ls-files 2>/dev/null | grep -iE "$VENDORS" > "$tmp/paths" || true
if [ -s "$tmp/paths" ]; then
n=0
while IFS= read -r p; do allowed "$p" || { echo " path: $p"; n=$((n+1)); }; done < "$tmp/paths"
if [ "$n" -gt 0 ]; then
echo "::error::${n} tracked path(s) carry a site-generator vendor's name. Every one of them is a URL the browser requests, so the vendor's branding is in the page. Move the assets to a neutral folder (public/img/) and repoint the references in the same change — a favicon or logo that a <link> or <img> depends on must be REPLACED with the real brand mark, not merely deleted."
fail=1
fi
fi
# --- 2. tracked content ---------------------------------------------------
git grep -Iin -E "$CONTENT_SIG" HEAD 2>/dev/null | sed 's#^HEAD:##' > "$tmp/content" || true
if [ -s "$tmp/content" ]; then
n=0
while IFS= read -r line; do
p=${line%%:*}
allowed "$p" || { echo " content: $(echo "$line" | cut -c1-140)"; n=$((n+1)); }
done < "$tmp/content"
if [ "$n" -gt 0 ]; then
echo "::error::${n} tracked line(s) reference a site-generator vendor's asset folder, build plugin or injected script. Drop the dependency and the script tag; repoint the asset URLs."
fail=1
fi
fi
# --- 3. commit messages, identities, trailers -----------------------------
# Default scope is HEAD alone, which is what a fetch-depth:1 CI checkout has and
# is enough to refuse the commit being pushed. Pass a range (or set
# VENDORMARK_RANGE) to judge more, e.g. origin/main..HEAD for a whole PR.
range=${range:-${VENDORMARK_RANGE:-}}
if [ -n "$range" ]; then set -- "$range"; else set -- -1 HEAD; fi
git log "$@" --format='%H%x01%an <%ae>%x01%cn <%ce>%x01%B%x01%(trailers:unfold=true)%x02' 2>/dev/null \
| tr -d '\n' | tr '\002' '\n' > "$tmp/log" || true
if [ -s "$tmp/log" ]; then
n=0
while IFS= read -r c; do
echo "$c" | grep -qiE "$IDENT_SIG" || continue
echo " commit: $(echo "$c" | tr '\001' ' ' | cut -c1-150)"
n=$((n+1))
done < "$tmp/log"
if [ "$n" -gt 0 ]; then
echo "::error::${n} commit(s) name a site-generator vendor in the message, the author/committer identity or a Co-authored-by trailer. A working tree can be tidied; \`git log\` cannot be, short of a history rewrite and a force-push of every branch. Reword before pushing. Describe the work — \"generated placeholder assets\" — and never credit the generator."
fail=1
fi
fi
if [ "$fail" -eq 0 ]; then
echo "vendormark: OK — no site-generator branding in paths, tracked content, messages or identities"
fi
exit "$fail"
+142
View File
@@ -0,0 +1,142 @@
#!/usr/bin/env bash
# vendormark_test.sh — the refusals AND the allowances, both pinned.
#
# A gate is only worth having if it is exact in both directions. A false
# negative lets the vendor's mark back into a shipped site; a false positive
# reds a repo that was always fine, and a gate that reds honest repos is a gate
# someone switches off. So this suite asserts BOTH halves, and the allowance
# half is the larger one on purpose: the vendor's name is also an ordinary
# English adjective, and it occurs innocently in tokenizer vocabularies, word
# lists, ML corpora, packed public-suffix data and third-party host lists all
# over the estate.
#
# Offline and deterministic: temp repos, no network.
set -uo pipefail
BIN=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/vendormark
pass=0; fail=0
V=$(printf 'lo%s' 'vable') # not spelled literally, so this file does not
# itself trip the gate it is testing
newrepo() {
d=$(mktemp -d)
git -C "$d" init -q
git -C "$d" config user.email dev@hanzo.ai
git -C "$d" config user.name "Hanzo Dev"
printf 'x\n' > "$d/README.md"
git -C "$d" add -A
git -C "$d" commit -qm "initial"
echo "$d"
}
check() { # name expected_rc dir [range]
local name=$1 want=$2 dir=$3 range=${4:-}
"$BIN" "$dir" $range >/dev/null 2>&1; local got=$?
if [ "$got" = "$want" ]; then pass=$((pass+1)); echo " ok $name"
else fail=$((fail+1)); echo " FAIL $name (want rc=$want, got rc=$got)"; fi
}
echo "REFUSALS — the mark in each of the four places it hides"
d=$(newrepo)
check "clean repo is green" 0 "$d"
mkdir -p "$d/public/$V-uploads"
printf 'PNG\n' > "$d/public/$V-uploads/28d53ec4.png"
git -C "$d" add -A && git -C "$d" commit -qm "add an image"
check "MUTATION: tracked path named for the vendor" 1 "$d"
git -C "$d" rm -rq "public/$V-uploads" && git -C "$d" commit -qm "move the image"
check "MUTATION REVERTED: green again" 0 "$d"
rm -rf "$d"
d=$(newrepo)
printf '<img src="/%s-uploads/28d53ec4.png" />\n' "$V" > "$d/index.html"
git -C "$d" add -A && git -C "$d" commit -qm "a page"
check "content: asset folder referenced in markup" 1 "$d"
rm -rf "$d"
d=$(newrepo)
printf '{"devDependencies":{"%s-tagger":"^1.1.3"}}\n' "$V" > "$d/package.json"
git -C "$d" add -A && git -C "$d" commit -qm "deps"
check "content: the vendor's build plugin in package.json" 1 "$d"
rm -rf "$d"
d=$(newrepo)
printf '<script src="https://cdn.gpteng.co/gptengineer.js"></script>\n' > "$d/index.html"
git -C "$d" add -A && git -C "$d" commit -qm "a page"
check "content: the vendor's injected script tag" 1 "$d"
rm -rf "$d"
d=$(newrepo)
printf '<meta name="generator" content="Lovable" />\n' > "$d/index.html"
git -C "$d" add -A && git -C "$d" commit -qm "a page"
check "content: the generator meta tag" 1 "$d"
rm -rf "$d"
d=$(newrepo)
printf 'y\n' > "$d/a.txt"; git -C "$d" add -A
git -C "$d" commit -qm "Update $V project template"
check "MUTATION: the vendor named in a commit message" 1 "$d"
git -C "$d" commit -q --amend -m "update the project template"
check "MUTATION REVERTED: reworded message is green" 0 "$d"
rm -rf "$d"
d=$(newrepo)
printf 'y\n' > "$d/a.txt"; git -C "$d" add -A
# --no-verify ON PURPOSE. A commit-msg hook on the workstation
# (~/.githooks/commit-msg) already strips vendor co-author trailers, and with it
# enabled this case cannot be constructed — the hook silently rewrites the
# trailer to ours and the assertion passes for the wrong reason. That hook is
# per-machine: it does not run on a runner, on a teammate's laptop, or on a
# commit made through the GitHub web UI. This gate is the layer that does. So
# the test bypasses the hook to prove the GATE catches what the hook would have.
git -C "$d" commit -q --no-verify -m "a change
Co-authored-by: $V bot <bot@$V.dev>"
check "MUTATION: Co-authored-by credits the vendor" 1 "$d"
git -C "$d" commit -q --amend --no-verify -m "a change"
check "MUTATION REVERTED: trailer dropped is green" 0 "$d"
rm -rf "$d"
d=$(newrepo)
printf 'y\n' > "$d/a.txt"; git -C "$d" add -A
git -C "$d" -c user.name="$V" -c user.email="bot@$V.dev" commit -qm "a change"
check "MUTATION: the vendor as author identity" 1 "$d"
rm -rf "$d"
d=$(newrepo)
printf 'y\n' > "$d/a.txt"; git -C "$d" add -A; git -C "$d" commit -qm "clean subject"
printf 'z\n' > "$d/b.txt"; git -C "$d" add -A; git -C "$d" commit -qm "Visual edit in $V"
printf 'w\n' > "$d/c.txt"; git -C "$d" add -A; git -C "$d" commit -qm "clean again"
check "default scope (HEAD only) misses an older bad message" 0 "$d"
check "explicit range catches it" 1 "$d" "HEAD~3..HEAD"
rm -rf "$d"
echo
echo "ALLOWANCES — the same word, innocently, as it really occurs in the estate"
d=$(newrepo)
printf ' "%s</w>": 38565,\n' "$V" > "$d/vocab.json"
printf '%s\n' "$V" > "$d/words_alpha.txt"
printf 'Tell me a story about a %s character.\n' "$V" > "$d/harmless.txt"
printf 'caseStudy: %s and v0 activate users by turning a prompt into an app\n' "${V^}" > "$d/guide.yaml"
printf 'mail2%s.com\n' "$V" > "$d/generic_emails.txt"
printf 'let e=[".%s.app",".%sproject.com",".webcontainer-api.io"];\n' "$V" "$V" > "$d/clerk-bundle.js"
printf 'A collection of UI components. Integrate them in v0, %s, Bolt.\n' "${V^}" > "$d/registries.json"
git -C "$d" add -A && git -C "$d" commit -qm "corpora, word lists and third-party bundles"
check "tokenizer vocab / word list / ML corpus / marketing prose / blocklist / upstream host list" 0 "$d"
rm -rf "$d"
echo
echo "ESCAPE HATCH — a repo whose job is to name the vendor"
d=$(newrepo)
mkdir -p "$d/tools/rules"
printf 'Hanzo Dev <dev@hanzo.ai> <bot@%s.dev>\n' "$V" > "$d/tools/rules/mailmap.txt"
git -C "$d" add -A && git -C "$d" commit -qm "scrubber rules"
check "scrubber rule file is refused by default" 1 "$d"
printf 'tools/rules/*\n' > "$d/.vendormark-allow"
git -C "$d" add -A && git -C "$d" commit -qm "declare the rule files"
check "...and allowed once declared in .vendormark-allow" 0 "$d"
rm -rf "$d"
echo
echo "vendormark_test: $pass passed, $fail failed"
[ "$fail" -eq 0 ]
+88
View File
@@ -59,6 +59,60 @@ test:
exit 1
fi
echo "OK: .github and .hanzo copies are one file ($(wc -l < .github/workflows/build.yml) lines)"
- name: gate-runs-before-delegate
# `mode: delegate` chooses WHERE the image is built. It must never choose
# WHETHER the commit was tested. It did: the delegate step was evaluated
# fourth, ahead of every toolchain and ahead of `test:`, and every other step
# carried `if: inputs.mode != 'delegate'` — so a delegated run skipped the
# gate along with the build and finished green in seconds having proven
# nothing. The seam was right and the ORDER was wrong, which is the kind of
# defect that reads as correct in every diff that touches one step at a time.
#
# Two assertions, because either one alone can be satisfied while the gate
# still does not run: the delegate POST must come AFTER the test gate, and
# the test gate must not be delegate-guarded.
run: |
set -e
f=.github/workflows/build.yml
t=$(grep -n '^ - name: Test (per hanzo.yml)$' "$f" | cut -d: -f1)
d=$(grep -n '^ - name: Delegate build to platform (mode=delegate)$' "$f" | cut -d: -f1)
[ -n "$t" ] && [ -n "$d" ] || { echo "::error::cannot find the test gate ($t) or the delegate step ($d) — a rename broke this assertion, fix the assertion"; exit 1; }
[ "$d" -gt "$t" ] || { echo "::error::the delegate step (line $d) runs BEFORE the test gate (line $t) — a delegated build would ship an ungated commit"; exit 1; }
if sed -n "${t}p;$((t+1)),$((t+80))p" "$f" | grep -m1 '^ if: ' | grep -q "mode != 'delegate'"; then
echo "::error::the test gate is guarded by \`inputs.mode != 'delegate'\` — delegate would skip it"; exit 1
fi
echo "OK: test gate at line $t, delegate POST at line $d, gate not delegate-guarded"
- name: publishable
# bin/publishable refuses a `build_secrets` name that has not declared itself
# public. That matters because a build_secret is baked in as a --build-arg,
# which `docker history` prints to anyone who can pull — so the key is a
# publishing decision wearing the word "secret". Until this, the only check
# in the fleet lived inside hanzoai/ui's own Dockerfile, for one image.
run: bash bin/publishable_test.sh
- name: publishable-rule-is-one-rule
# The rule is spelled twice on purpose, and this is what keeps it one rule.
#
# build.yml cannot call bin/publishable, because the tools checkout and the
# workflow resolve from DIFFERENT places: a caller pins the workflow at a ref
# its own forge resolves, while the tools step derives its ref from
# GITHUB_WORKFLOW_REF — which git.hanzo.ai does not set, so it falls back to
# `v1` and clones github.com. Measured on run 36473: `derived ref=v1`,
# `cloned https://github.com/hanzoai/ci@v1`. So a step calling a NEW file
# under $CI_HOME is broken on every forge run until a tag moves on a host the
# caller never named. The pipeline has to carry its own rule.
#
# Which leaves the pattern in two files, which is the same shape as the two
# copies of build.yml and gets the same treatment: demand they are identical
# and let the gate, not a reader, be the thing that notices.
run: |
set -e
a=$(grep -oE '^[[:space:]]*PUBLISHABLE_\*\|[^)]*\)' .github/workflows/build.yml | tr -d '[:space:]')
b=$(grep -oE '^[[:space:]]*PUBLISHABLE_\*\|[^)]*\)' bin/publishable | tr -d '[:space:]')
[ -n "$a" ] && [ -n "$b" ] || { echo "::error::publishable rule not found in build.yml ($a) or bin/publishable ($b) — an edit renamed it, fix this assertion"; exit 1; }
[ "$a" = "$b" ] || { echo "::error::the publishable rule differs between build.yml and bin/publishable — build.yml is what runs, bin/publishable is what is tested, so a difference means the tested rule is not the enforced one:
build.yml: $a
bin/publishable: $b"; exit 1; }
echo "OK: one publishable rule, two spellings — $a"
- name: imgver
# bin/imgver decides the version EVERY image in the fleet publishes — this
# workflow's build lane calls it, and so does the imgver composite action the
@@ -67,6 +121,24 @@ test:
# IfNotPresent never picks up and no reader can see. Offline and
# deterministic: the registry floor is injected, so it needs no network.
run: bash bin/imgver_test.sh
- name: vendormark
# bin/vendormark is the gate the build lane runs against every checkout: it
# refuses a site-generator vendor's branding in a tracked path, in tracked
# content, in a commit message, in an author identity or in a Co-authored-by
# trailer. Five shipped sites carried it — one serving the vendor's upload
# folder as its favicon — and clearing them took a history rewrite and a
# force-push of every branch.
#
# The suite pins the refusals AND the allowances, and the allowances are the
# larger half on purpose. The vendor's name is also an ordinary English
# adjective: across the 1,065 repos swept it appears innocently in tokenizer
# vocabularies, word lists, ML corpora, packed public-suffix data and an
# upstream Clerk bundle's list of other people's dev hosts. A gate that reds
# a fifth of the estate is a gate someone switches off, so the content rules
# are only strings that cannot occur in prose, and two candidate rules were
# measured and DROPPED for redding that Clerk bundle. Offline and
# deterministic: temp repos, no network.
run: bash bin/vendormark_test.sh
- name: gover
# bin/gover is the gate the build lane runs against every Dockerfile before
# it builds: it refuses a Go builder image older than the go.mod it
@@ -83,6 +155,22 @@ test:
# and a multi-module repo is judged by its NEAREST go.mod. Offline and
# deterministic: temp dirs, no registry, no network.
run: bash bin/gover_test.sh
- name: sitepublish
# bin/sitepublish is the whole of the `site:` lane. The lane it replaced was
# dead: it staged with `mc mirror` behind S3_ADMIN_ACCESS_KEY/SECRET_KEY,
# names KMS holds for no org, so it failed closed for every caller — and of
# the 155 hanzo.yml files across the three orgs, not one declared a `site:`.
#
# The suite is weighted toward REFUSALS because the failure that matters here
# is a green publish that verified nothing — indistinguishable from a working
# one until a site quietly stops updating. Three defects of exactly that
# shape are pinned by name: an object-shaped read of a body that is not an
# object, `.releases[]` against the BARE ARRAY the list route returns (which
# resolves nothing and so can never fail for its stated reason), and
# `[ "$a" = "$b" ]` on two empty strings, which is TRUE. Reintroducing them
# turns 3, 9 and 1 of these tests red respectively. Offline and
# deterministic: a curl shim answers the routes from fixtures, no network.
run: bash bin/sitepublish_test.sh
# No `deploy:` ON PURPOSE. Rollout is a reviewed tag pin in hanzoai/universe
# (infra/k8s/operator/crs/ci.yaml), the same rule cloud and git follow: a