Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 zeekay
|
ddd3df4675 | ||
|
zeekay
|
b047a7a569 | ||
 hanzo-dev
|
8a415e9745 | ||
|
zeekay
|
a830325d31 | ||
|
zeekay
|
d1e6edf6d1 | ||
|
zeekay
|
2c036d3a61 | ||
|
zeekay
|
11fcb7b145 | ||
|
zandGitHub
|
f7c2309483 | ||
|
zandGitHub
|
33ceb2c495 | ||
|
zeekayandhanzo-dev
|
9a6647cd30 | ||
|
hanzo-dev
|
0539ecf756 | ||
|
zeekayandhanzo-dev
|
bd06e29b29 | ||
|
hanzo-dev
|
5e0e4f906c | ||
|
hanzo-dev
|
f098b39ecd |
+247
-245
@@ -29,14 +29,17 @@ on:
|
||||
default: '["hanzo-build-linux-amd64"]'
|
||||
mode:
|
||||
description: >-
|
||||
Build execution mode. `buildx` (default) runs the full buildx →
|
||||
test → deploy pipeline ON the runner. `delegate` instead POSTs the
|
||||
build to platform.hanzo.ai (`/v1/runner`) — platform builds
|
||||
in-cluster with BuildKit and rolls the operator Service CR itself, so
|
||||
the GitHub job finishes in seconds with no runner buildx. A repo opts
|
||||
in by passing `with: { mode: delegate }`; everything else is unchanged.
|
||||
Requires the `PLATFORM_BUILD_CALLBACK_TOKEN` secret (via secrets:
|
||||
inherit).
|
||||
Where the IMAGE is built. `buildx` (default) builds it on this runner.
|
||||
`delegate` POSTs it to platform.hanzo.ai (`/v1/runner`), which builds
|
||||
in-cluster with BuildKit and rolls the operator Service CR itself.
|
||||
A repo opts in with `with: { mode: delegate }` and needs the
|
||||
`PLATFORM_BUILD_CALLBACK_TOKEN` secret (via secrets: inherit).
|
||||
|
||||
THE GATE RUNS IN BOTH MODES, and that is the whole point of the input
|
||||
being named for the build. `delegate` used to be evaluated fourth, ahead
|
||||
of every toolchain and ahead of `test:`, so it skipped the gate along
|
||||
with the build — 20 steps skipped, 0 executed, green. What a delegated
|
||||
run skips is buildx, publish and deploy. It does not skip being tested.
|
||||
type: string
|
||||
default: buildx
|
||||
tests:
|
||||
@@ -73,7 +76,7 @@ on:
|
||||
A string sidesteps it on both planes with identical meaning, and the
|
||||
comparison is against 'false' rather than a truthiness test so that the
|
||||
expression callers already write — hanzoai/cloud's
|
||||
`tests: ${{ github.ref_type != 'tag' }}`, which renders the strings
|
||||
`tests:` set from `github.ref_type != 'tag'`, which renders the strings
|
||||
"true"/"false" — keeps working unchanged.
|
||||
type: string
|
||||
default: 'true'
|
||||
@@ -174,11 +177,23 @@ jobs:
|
||||
# on they are ordinary content: status clean, diff empty, both sides
|
||||
# of the merge shipping. It hides in generated and vendored files,
|
||||
# which are large, rarely opened and effectively unreviewed —
|
||||
# hanzoai/base carried `<<<<<<< HEAD` / `>>>>>>> upstream/master` at
|
||||
# hanzoai/base carried ` / `>>>>>>> upstream/master` at
|
||||
# lines 23807 and 23850 of a committed .d.ts. No compiler catches it
|
||||
# there: a .d.ts nothing imports is never parsed, and in YAML, JSON,
|
||||
# Markdown and every config format we ship the markers are silently
|
||||
# VALID content.
|
||||
# vendormark — a site-generator vendor's branding left in the tree.
|
||||
# Several sites began as output from a hosted "prompt to React app"
|
||||
# generator, which signs its work in four places at once: an asset
|
||||
# folder named after the vendor (so its name is in every image URL,
|
||||
# and on one site in the FAVICON, so it was in every browser tab), a
|
||||
# build plugin in package.json, an injected third-party <script>,
|
||||
# and the commit messages, author identities and Co-authored-by
|
||||
# trailers of its bot. The last survives any amount of tidying the
|
||||
# working tree — a repo can render a perfect page and still say in
|
||||
# `git log` who really wrote it. Removing it needed a history
|
||||
# rewrite and a force-push of every branch; this is so that is never
|
||||
# needed twice.
|
||||
#
|
||||
# Both proven RED against cloud@v1.801.424 (the real outage) and GREEN
|
||||
# against the repaired tree before landing here. They read the checkout
|
||||
@@ -190,6 +205,7 @@ jobs:
|
||||
"$RUNNER_TEMP/ci/bin/ignoretracked" .
|
||||
"$RUNNER_TEMP/ci/bin/modsize" .
|
||||
"$RUNNER_TEMP/ci/bin/conflictmarkers" .
|
||||
"$RUNNER_TEMP/ci/bin/vendormark" .
|
||||
|
||||
- name: Provision parse toolchain (jq + PyYAML)
|
||||
# This reusable parses the caller's hanzo.yml with python3 + PyYAML and
|
||||
@@ -222,64 +238,7 @@ jobs:
|
||||
jq --version
|
||||
yq --version
|
||||
|
||||
- name: Delegate build to platform (mode=delegate)
|
||||
# The GHA-escape fast path: instead of running buildx on this runner, POST
|
||||
# each image in hanzo.yml to platform.hanzo.ai's direct-enqueue webhook
|
||||
# (`/v1/runner`). Platform creates a build_job row, launches an
|
||||
# in-cluster BuildKit Job on its own pool, pushes to the registry, and —
|
||||
# for a system service — patches the operator Service CR to roll it. The
|
||||
# downstream is IDENTICAL to the platform GitHub-App webhook path (one
|
||||
# build path, two front doors), so a delegated build behaves exactly like
|
||||
# a platform-native one. This job then exits in seconds — no buildx, no
|
||||
# KMS, no runner-side deploy.
|
||||
if: inputs.mode == 'delegate'
|
||||
env:
|
||||
ENQUEUE_URL: ${{ vars.PLATFORM_ENQUEUE_URL || 'https://platform.hanzo.ai/v1/runner' }}
|
||||
ENQUEUE_TOKEN: ${{ secrets.PLATFORM_BUILD_CALLBACK_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${ENQUEUE_TOKEN:-}" ]; then
|
||||
echo "::error::mode=delegate needs the PLATFORM_BUILD_CALLBACK_TOKEN secret (secrets: inherit)"; exit 1
|
||||
fi
|
||||
REPO="${{ github.repository }}"
|
||||
SHA="${{ github.sha }}"
|
||||
SHORT=$(echo "$SHA" | cut -c1-7)
|
||||
REF="${{ github.ref }}"
|
||||
BRANCH="${{ github.ref_name }}"
|
||||
# One enqueue per (image, platform), mirroring the buildx tag shape the
|
||||
# deploy path expects (`sha-<short>-<arch>[-<suffix>]`). Default arch is
|
||||
# amd64 (single-arch), so an existing repo's tag shape is unchanged.
|
||||
yq -o=json -I=0 '.images' hanzo.yml | jq -c '.[]' | while read -r img; do
|
||||
name=$(echo "$img"|jq -r .name); repo=$(echo "$img"|jq -r .repo)
|
||||
ctx=$(echo "$img"|jq -r .context); df=$(echo "$img"|jq -r '.dockerfile // (.context+"/Dockerfile")')
|
||||
sfx=$(echo "$img"|jq -r '."tag-suffix" // ""')
|
||||
# Same Go-builder check the buildx lane runs. It belongs here too:
|
||||
# delegating the build moves WHERE it runs, not whether the image
|
||||
# can compile the module, and a mismatch enqueued to platform fails
|
||||
# on a machine whose logs this run never shows. Cheap to check on
|
||||
# the runner that already has the checkout; expensive to discover
|
||||
# from the in-cluster builder.
|
||||
"$CI_HOME/bin/gover" "$df" "$ctx"
|
||||
echo "$img" | jq -r '(.platforms // ["linux/amd64"])[]' | while read -r plat; do
|
||||
arch="${plat##*/}"
|
||||
image="${repo}:sha-${SHORT}-${arch}${sfx:+-$sfx}"
|
||||
body=$(jq -nc \
|
||||
--arg repo "$REPO" --arg sha "$SHA" --arg image "$image" \
|
||||
--arg ref "$REF" --arg branch "$BRANCH" \
|
||||
--arg dockerfile "$df" --arg context "$ctx" --arg arch "$arch" \
|
||||
'{repo:$repo,sha:$sha,image:$image,ref:$ref,branch:$branch,dockerfile:$dockerfile,context:$context,os:"linux",arch:$arch}')
|
||||
echo "::group::delegate $name → $image"
|
||||
code=$(curl -sS -o /tmp/enqueue.out -w '%{http_code}' -X POST "$ENQUEUE_URL" \
|
||||
-H "Authorization: Bearer $ENQUEUE_TOKEN" -H 'Content-Type: application/json' -d "$body")
|
||||
cat /tmp/enqueue.out; echo
|
||||
# 202 Accepted = queued; 409 = no live runner for the pool (surface it loud).
|
||||
if [ "$code" != "202" ]; then echo "::error::enqueue $image failed (HTTP $code)"; exit 1; fi
|
||||
echo "::endgroup::"
|
||||
done
|
||||
done
|
||||
|
||||
- name: Authenticated git for go modules (rate-limit + any private repo)
|
||||
if: inputs.mode != 'delegate'
|
||||
# luxfi/hanzoai/zooai Go modules are PUBLIC, so `go` resolves them through
|
||||
# the default public proxy (proxy.golang.org) + checksum db (sum.golang.org)
|
||||
# — canonical, IMMUTABLE hashes that a force-moved tag can no longer break.
|
||||
@@ -335,7 +294,6 @@ jobs:
|
||||
|
||||
- name: Fetch deploy credentials from KMS
|
||||
id: kms
|
||||
if: inputs.mode != 'delegate'
|
||||
env:
|
||||
KMS_CLIENT_ID: ${{ secrets.KMS_CLIENT_ID }}
|
||||
KMS_CLIENT_SECRET: ${{ secrets.KMS_CLIENT_SECRET }}
|
||||
@@ -375,6 +333,15 @@ jobs:
|
||||
if [ -z "$ORG" ]; then
|
||||
case "${{ github.repository_owner }}" in
|
||||
hanzoai) ORG=hanzo ;;
|
||||
# hanzo-inc is the same TENANT as hanzoai — pay, commerce and the
|
||||
# rest of the company surfaces bill and seal to KMS org `hanzo`.
|
||||
# Without this line ORG fell through to the literal owner,
|
||||
# "hanzo-inc", which is not a KMS org. That still LOOKED fine,
|
||||
# because the read below takes its org from the TOKEN, not the URL
|
||||
# — so the only things that were wrong were the fallback URL and
|
||||
# every error message, which named an org that does not exist and
|
||||
# sent the reader looking for a tenant instead of a credential.
|
||||
hanzo-inc) ORG=hanzo ;;
|
||||
luxfi) ORG=lux ;;
|
||||
zooai) ORG=zoo ;;
|
||||
*) ORG="${{ github.repository_owner }}" ;;
|
||||
@@ -391,6 +358,40 @@ jobs:
|
||||
# how a stale key survived: every failure looked like "not in KMS", and
|
||||
# "not in KMS" was survivable.
|
||||
DECLARED="$(yq -r '[(.images // [])[] | (.build_secrets // [])[]] | unique | .[]' hanzo.yml 2>/dev/null || true)"
|
||||
# A build_secret becomes `--build-arg NAME=value` below, and a build-arg
|
||||
# is IN THE PUBLISHED IMAGE — `docker history` prints it to anyone who
|
||||
# can pull. So `build_secrets` can only ever carry values that are
|
||||
# public on purpose: a Vite/Next SPA has no server to read an env from,
|
||||
# so its ingest key must be inlined at build, and inlining it is what
|
||||
# publishing it means. Every other value must not be here at all.
|
||||
#
|
||||
# Nothing checked that. The one repo whose value is genuinely
|
||||
# publishable (hanzoai/ui, a `pk-…` ingest key) asserted the `pk-`
|
||||
# prefix INSIDE ITS OWN Dockerfile — so the check existed once, for one
|
||||
# image, and any repo that added a real credential got no check at all.
|
||||
# Per-repo vigilance is not a mechanism; the assertion belongs here,
|
||||
# where the build-arg is assembled, once, for every caller.
|
||||
#
|
||||
# INLINE, not "$CI_HOME/bin/publishable", and that is forced rather than
|
||||
# chosen. The tools checkout and this workflow resolve from DIFFERENT
|
||||
# places: the caller pins the workflow at a ref its forge resolves, while
|
||||
# the tools step derives its ref from GITHUB_WORKFLOW_REF — which
|
||||
# git.hanzo.ai does not set, so it falls back to `v1` and clones
|
||||
# github.com. Measured on run 36473: `GITHUB_WORKFLOW_REF=<unset>`,
|
||||
# `derived ref=v1`, `cloned https://github.com/hanzoai/ci@v1`. A step
|
||||
# that calls a NEW file under $CI_HOME is therefore broken on every forge
|
||||
# run until a tag moves on a host this repo is not even pinned to.
|
||||
#
|
||||
# The rule is also spelled in bin/publishable, which the `publishable`
|
||||
# gate tests. Two spellings of one rule is the same shape as the two
|
||||
# copies of this file, and it gets the same treatment: a gate refuses
|
||||
# any difference between them. See `publishable-rule-is-one-rule`.
|
||||
for bs in $DECLARED; do
|
||||
case "$bs" in
|
||||
PUBLISHABLE_*|PUBLIC_*|NEXT_PUBLIC_*|EXPO_PUBLIC_*|NUXT_PUBLIC_*|VITE_*|REACT_APP_*|*_PUBLISHABLE|*_PUBLIC) ;;
|
||||
*) echo "::error::build_secret '$bs' does not declare itself publishable, and a build_secret is baked into the image as a --build-arg where \`docker history\` reveals it. Rename it (PUBLISHABLE_*, PUBLIC_*, NEXT_PUBLIC_*, VITE_*, REACT_APP_*) if the value is public on purpose; if it is a real credential it cannot be a build_secret at all."; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
required() { [ -n "$DECLARED" ]; }
|
||||
if [ -z "${KMS_CLIENT_ID:-}" ] || [ -z "$ORG" ]; then
|
||||
if required; then
|
||||
@@ -425,28 +426,45 @@ jobs:
|
||||
if [ -n "$GIT_TOKEN" ]; then echo "::add-mask::$GIT_TOKEN"; echo "GIT_TOKEN=$GIT_TOKEN" >> "$GITHUB_ENV"; fi
|
||||
KUBECONFIG_B64=$(get KUBECONFIG)
|
||||
if [ -n "$KUBECONFIG_B64" ]; then echo "$KUBECONFIG_B64" | base64 -d > "$RUNNER_TEMP/kubeconfig"; echo "kubeconfig=$RUNNER_TEMP/kubeconfig" >> "$GITHUB_OUTPUT"; fi
|
||||
# Object-store credential, for the two lanes that write bytes to
|
||||
# hanzoai/s3: a repo publishing `binaries:` to a bucket:, and a repo
|
||||
# publishing a `site:` (whose build output has to land in the org's own
|
||||
# storage space before /v1/sites can promote it). Same four names the
|
||||
# services read at run time, from the same org/path/env — a CI-only copy
|
||||
# of a credential is a second thing to rotate and the one nobody
|
||||
# remembers. One fetch serves both lanes; a repo declaring both does not
|
||||
# ask KMS twice.
|
||||
# The REGISTRY credential, and it is fetched UNCONDITIONALLY because
|
||||
# every repo that publishes an image needs it — which is most of them.
|
||||
#
|
||||
# It used to sit inside the `bucket:`/`site:` gate below, and that gate
|
||||
# is about writing bytes to hanzoai/s3, a completely unrelated lane. So
|
||||
# a repo declaring only `images:` — hanzoai/bot, and it is not alone —
|
||||
# asked KMS for nothing and reached the docker login step with empty
|
||||
# credentials. The build then failed at the push with no clue pointing
|
||||
# here, and the obvious "fix" is to paste REGISTRY_USER/REGISTRY_PASSWORD
|
||||
# into the forge's org secrets, which is exactly the CI-only copy the
|
||||
# comment below warns about: a second thing to rotate, and the one
|
||||
# nobody remembers. The contract is that the ONLY things living in the
|
||||
# forge are KMS_CLIENT_ID/SECRET, so the fetch moves out of the gate
|
||||
# rather than the credential moving into GitHub.
|
||||
#
|
||||
# Empty is harmless: nothing is exported when KMS has no such key, and
|
||||
# the login step already falls back to the kubeconfig read for repos
|
||||
# provisioned before these names existed.
|
||||
for k in REGISTRY_USER REGISTRY_PASSWORD; do
|
||||
v=$(get "$k")
|
||||
if [ -n "$v" ]; then
|
||||
[ "$k" = REGISTRY_PASSWORD ] && echo "::add-mask::$v"
|
||||
echo "$k=$v" >> "$GITHUB_ENV"
|
||||
fi
|
||||
done
|
||||
# Object-store credential, for the ONE lane that writes bytes to
|
||||
# hanzoai/s3 itself: a repo publishing `binaries:` to a bucket:. Same
|
||||
# names the services read at run time, from the same org/path/env — a
|
||||
# CI-only copy of a credential is a second thing to rotate and the one
|
||||
# nobody remembers.
|
||||
#
|
||||
# `site:` used to be fetched here too, and that is exactly what killed
|
||||
# it: S3_ADMIN_ACCESS_KEY/S3_ADMIN_SECRET_KEY are not in KMS for any
|
||||
# org, so the site lane's fail-closed check refused every caller that
|
||||
# ever declared one. The site lane no longer touches S3 — it hands the
|
||||
# zip to cloud and cloud writes the bytes — so the credential it cannot
|
||||
# have is one it no longer needs.
|
||||
SITE_DECL="$(yq -r '.site | type' hanzo.yml 2>/dev/null || echo '!!null')"
|
||||
if [ -n "$(yq -r '.bucket // ""' hanzo.yml 2>/dev/null || true)" ] || [ "$SITE_DECL" = '!!map' ]; then
|
||||
# The registry credential comes from KMS like every other secret —
|
||||
# the ONLY things that live in GitHub are KMS_CLIENT_ID/SECRET.
|
||||
# Named here so a repo with no kubeconfig (most of them) can still
|
||||
# reach registry.hanzo.ai; the kubeconfig read below stays as the
|
||||
# fallback for repos provisioned before these names existed.
|
||||
for k in REGISTRY_USER REGISTRY_PASSWORD; do
|
||||
v=$(get "$k")
|
||||
if [ -n "$v" ]; then
|
||||
[ "$k" = REGISTRY_PASSWORD ] && echo "::add-mask::$v"
|
||||
echo "$k=$v" >> "$GITHUB_ENV"
|
||||
fi
|
||||
done
|
||||
if [ -n "$(yq -r '.bucket // ""' hanzo.yml 2>/dev/null || true)" ]; then
|
||||
for k in S3_ADMIN_ACCESS_KEY S3_ADMIN_SECRET_KEY; do
|
||||
v=$(get "$k")
|
||||
if [ -n "$v" ]; then echo "::add-mask::$v"; echo "$k=$v" >> "$GITHUB_ENV"; fi
|
||||
@@ -458,38 +476,30 @@ jobs:
|
||||
if [ -n "$v" ]; then echo "$k=$v" >> "$GITHUB_ENV"; fi
|
||||
done
|
||||
fi
|
||||
# `site:` needs two things beyond the S3 credential above, and BOTH come
|
||||
# from this step — there is one KMS login in this workflow and the site
|
||||
# lane does not add a second.
|
||||
# `site:` needs exactly ONE thing from this step, and it is a thing this
|
||||
# step already has — there is one KMS login in this workflow and the
|
||||
# site lane does not add a second.
|
||||
#
|
||||
# CLOUD_PROJECTS_BUCKET — the bucket /v1/sites lists the build output
|
||||
# in. The SERVER owns it (it is never in a publish request, which is
|
||||
# what makes the org prefix unforgeable), so CI cannot be told it by
|
||||
# hanzo.yml; it reads the same KMS name the cloud binary reads, and
|
||||
# falls back to the same default. One name, one place.
|
||||
# HANZO_API_TOKEN — the bearer for /v1/sites. It is the token THIS STEP
|
||||
# ALREADY MINTED: cloud's /v1/kms/auth/login is a broker that
|
||||
# exchanges clientId/clientSecret at IAM's client_credentials
|
||||
# endpoint and "returns IAM's own JWT verbatim" (cloud
|
||||
# apps/kms/login.go), scoped by its `owner` claim to exactly this
|
||||
# org. api.hanzo.ai validates that same issuer and mints X-Org-Id
|
||||
# from it. So the credential that reads this org's secrets and the
|
||||
# credential that publishes this org's site are ONE machine identity,
|
||||
# with one lifetime and one thing to rotate. It is masked here,
|
||||
# before GITHUB_ENV is the first place it could surface.
|
||||
# HANZO_API_TOKEN — the bearer for /v1/projects and /v1/sites. It is
|
||||
# the token THIS STEP ALREADY MINTED: cloud's /v1/kms/auth/login is
|
||||
# a broker that performs the IAM client_credentials exchange and
|
||||
# "returns IAM's own JWT verbatim" (cloud apps/kms/login.go), scoped
|
||||
# by its `owner` claim to exactly this org. api.hanzo.ai validates
|
||||
# that same issuer and mints X-Org-Id from it. So the credential
|
||||
# that reads this org's secrets and the credential that publishes
|
||||
# this org's site are ONE machine identity, with one lifetime and
|
||||
# one thing to rotate. It is masked here, before GITHUB_ENV is the
|
||||
# first place it could surface.
|
||||
#
|
||||
# HANZO_ORG is exported with them because the S3 upload has to name the
|
||||
# org segment the server will prepend to `source`. That it AGREES with
|
||||
# the publish identity is not an assumption: every `get` above read
|
||||
# /v1/kms/orgs/$ORG/..., and cloud re-checks the token's `owner` against
|
||||
# that :org on each read — so an ORG that disagreed would have fetched no
|
||||
# credential at all, and this lane fails closed below rather than
|
||||
# uploading into a prefix nobody will look in.
|
||||
# Neither the bucket nor the org is exported any more, because neither
|
||||
# is CI's to name. The org segment is prepended SERVER-side from the
|
||||
# validated principal and the bucket never appears in a request at all
|
||||
# (cloud apps/projects/blob.go:60, `sitePrefix(org, slug)`) — which is
|
||||
# precisely what makes the prefix unforgeable. CI passing them was CI
|
||||
# restating something it could only get wrong.
|
||||
if [ "$SITE_DECL" = '!!map' ]; then
|
||||
v=$(get CLOUD_PROJECTS_BUCKET)
|
||||
if [ -n "$v" ]; then echo "CLOUD_PROJECTS_BUCKET=$v" >> "$GITHUB_ENV"; fi
|
||||
echo "::add-mask::$TOKEN"
|
||||
{ echo "HANZO_API_TOKEN=$TOKEN"; echo "HANZO_ORG=$ORG"; } >> "$GITHUB_ENV"
|
||||
echo "HANZO_API_TOKEN=$TOKEN" >> "$GITHUB_ENV"
|
||||
fi
|
||||
# Build-time secrets → --build-arg. A repo declares per-image
|
||||
# `build_secrets: [NAME, ...]` in hanzo.yml; each NAME is fetched from
|
||||
@@ -601,7 +611,7 @@ jobs:
|
||||
# OWN Go version from go.mod so the toolchain matches the module exactly.
|
||||
# Guarded to Go repos (go.mod present) so pure-JS/TS callers are
|
||||
# unaffected; harmless if a future runner image bakes Go in.
|
||||
if: inputs.mode != 'delegate' && hashFiles('go.mod') != ''
|
||||
if: hashFiles('go.mod') != ''
|
||||
uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version-file: go.mod
|
||||
@@ -615,7 +625,7 @@ jobs:
|
||||
# fails (snapshot mirror rot: "no longer has a Release file"),
|
||||
# repoint archive.ubuntu.com at the DO mirror — both sources.list and
|
||||
# noble's deb822 ubuntu.sources — and retry once.
|
||||
if: inputs.mode != 'delegate' && hashFiles('go.mod') != ''
|
||||
if: hashFiles('go.mod') != ''
|
||||
run: |
|
||||
command -v gcc >/dev/null 2>&1 && exit 0
|
||||
sudo apt-get update -qq || {
|
||||
@@ -640,12 +650,12 @@ jobs:
|
||||
# root-only hashFiles SKIPPED this step and the gate died at
|
||||
# bash: line 3: pnpm: command not found (exit 127)
|
||||
# after 49 ginkgo suites had already passed.
|
||||
if: inputs.mode != 'delegate' && hashFiles('package.json', '**/package.json') != ''
|
||||
if: hashFiles('package.json', '**/package.json') != ''
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 22
|
||||
- name: Enable corepack (pnpm/yarn shims for JS test gates)
|
||||
if: inputs.mode != 'delegate' && hashFiles('package.json', '**/package.json') != ''
|
||||
if: hashFiles('package.json', '**/package.json') != ''
|
||||
run: corepack enable
|
||||
|
||||
- name: Provision Rust toolchain (cargo test gates)
|
||||
@@ -657,7 +667,7 @@ jobs:
|
||||
# the exact failure this workflow refuses everywhere else.
|
||||
# Guarded to Rust callers (Cargo.toml present); a no-op once a runner
|
||||
# image bakes rustup in.
|
||||
if: inputs.mode != 'delegate' && hashFiles('Cargo.toml') != ''
|
||||
if: hashFiles('Cargo.toml') != ''
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
|
||||
- name: Provision Java toolchain (client codegen)
|
||||
@@ -876,7 +886,7 @@ jobs:
|
||||
# `http`. Take the scheme and the host as the two values they are, and
|
||||
# compare the HOST against github.com so the guard cannot be fooled by a
|
||||
# spelling either.
|
||||
if: inputs.mode != 'delegate' && hashFiles('go.mod') != ''
|
||||
if: hashFiles('go.mod') != ''
|
||||
env:
|
||||
GH_PAT: ${{ secrets.GH_PAT }}
|
||||
FORGE_TOKEN: ${{ secrets.FORGE_TOKEN }}
|
||||
@@ -927,7 +937,7 @@ jobs:
|
||||
# it ran nothing AND nothing shows as having run. A gate that is not a test
|
||||
# gate — vet, lint, a build, a codegen-freshness check — says neither and is
|
||||
# untouched. To silence it legitimately, make the gate run a test.
|
||||
if: inputs.mode != 'delegate' && inputs.tests != 'false'
|
||||
if: inputs.tests != 'false'
|
||||
run: |
|
||||
set -euo pipefail
|
||||
NOTHING='\[no test files\]|\[no tests to run\]|no tests ran|collected 0 items|No tests found|no test specified|running 0 tests|Tests:[[:space:]]+0 total|(^|[^0-9])0 (passing|passed)'
|
||||
@@ -956,6 +966,78 @@ jobs:
|
||||
fi
|
||||
done
|
||||
|
||||
- name: Delegate build to platform (mode=delegate)
|
||||
# Delegate moves WHERE the image is built. It does not move whether the
|
||||
# commit was gated — so this step sits AFTER `Test (per hanzo.yml)`, and
|
||||
# every step the gate needs runs in both modes. It used to sit fourth,
|
||||
# before any of them, and `mode: delegate` therefore skipped the gate
|
||||
# along with the build: 20 steps skipped, 0 executed, green. "ci gates,
|
||||
# platform builds" was the design and the ordering said otherwise.
|
||||
#
|
||||
# What it does: POST each image in hanzo.yml to platform.hanzo.ai's
|
||||
# direct-enqueue webhook (`/v1/runner`). Platform creates a build_job
|
||||
# row, launches an in-cluster BuildKit Job on its own pool, pushes to the
|
||||
# registry, and — for a system service — patches the operator Service CR
|
||||
# to roll it. Downstream is IDENTICAL to the platform webhook path, so a
|
||||
# delegated build behaves exactly like a platform-native one. What this
|
||||
# job skips is buildx, publish and deploy — not the gate.
|
||||
if: inputs.mode == 'delegate'
|
||||
env:
|
||||
ENQUEUE_URL: ${{ vars.PLATFORM_ENQUEUE_URL || 'https://platform.hanzo.ai/v1/runner' }}
|
||||
ENQUEUE_TOKEN: ${{ secrets.PLATFORM_BUILD_CALLBACK_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${ENQUEUE_TOKEN:-}" ]; then
|
||||
echo "::error::mode=delegate needs the PLATFORM_BUILD_CALLBACK_TOKEN secret (secrets: inherit)"; exit 1
|
||||
fi
|
||||
# `/v1/runner` has no way to carry a build_secret: its wire body accepts
|
||||
# buildArgs, and platform resolves NO KMS name — its own secrets arrive
|
||||
# as a KMSSecret CRD with a statically declared key list, which cannot
|
||||
# serve a name discovered by reading a hanzo.yml at some SHA. A repo
|
||||
# that declares build_secrets and delegates gets an image built without
|
||||
# them: green run, empty credential, exactly the silence the KMS step
|
||||
# refuses. Say so here rather than downstream, where this run shows no
|
||||
# logs at all.
|
||||
if [ -n "$(yq -r '[(.images // [])[] | (.build_secrets // [])[]] | unique | .[]' hanzo.yml 2>/dev/null || true)" ]; then
|
||||
echo "::error::hanzo.yml declares build_secrets, which mode=delegate cannot deliver — platform builds with no KMS. Build on the runner (drop mode: delegate), or drop the build_secrets."; exit 1
|
||||
fi
|
||||
REPO="${{ github.repository }}"
|
||||
SHA="${{ github.sha }}"
|
||||
SHORT=$(echo "$SHA" | cut -c1-7)
|
||||
REF="${{ github.ref }}"
|
||||
BRANCH="${{ github.ref_name }}"
|
||||
# One enqueue per (image, platform), mirroring the buildx tag shape the
|
||||
# deploy path expects (`sha-<short>-<arch>[-<suffix>]`). Default arch is
|
||||
# amd64 (single-arch), so an existing repo's tag shape is unchanged.
|
||||
yq -o=json -I=0 '.images' hanzo.yml | jq -c '.[]' | while read -r img; do
|
||||
name=$(echo "$img"|jq -r .name); repo=$(echo "$img"|jq -r .repo)
|
||||
ctx=$(echo "$img"|jq -r .context); df=$(echo "$img"|jq -r '.dockerfile // (.context+"/Dockerfile")')
|
||||
sfx=$(echo "$img"|jq -r '."tag-suffix" // ""')
|
||||
# Same Go-builder check the buildx lane runs. It belongs here too:
|
||||
# delegating the build moves WHERE it runs, not whether the image
|
||||
# can compile the module, and a mismatch enqueued to platform fails
|
||||
# on a machine whose logs this run never shows. Cheap to check on
|
||||
# the runner that already has the checkout; expensive to discover
|
||||
# from the in-cluster builder.
|
||||
"$CI_HOME/bin/gover" "$df" "$ctx"
|
||||
echo "$img" | jq -r '(.platforms // ["linux/amd64"])[]' | while read -r plat; do
|
||||
arch="${plat##*/}"
|
||||
image="${repo}:sha-${SHORT}-${arch}${sfx:+-$sfx}"
|
||||
body=$(jq -nc \
|
||||
--arg repo "$REPO" --arg sha "$SHA" --arg image "$image" \
|
||||
--arg ref "$REF" --arg branch "$BRANCH" \
|
||||
--arg dockerfile "$df" --arg context "$ctx" --arg arch "$arch" \
|
||||
'{repo:$repo,sha:$sha,image:$image,ref:$ref,branch:$branch,dockerfile:$dockerfile,context:$context,os:"linux",arch:$arch}')
|
||||
echo "::group::delegate $name → $image"
|
||||
code=$(curl -sS -o /tmp/enqueue.out -w '%{http_code}' -X POST "$ENQUEUE_URL" \
|
||||
-H "Authorization: Bearer $ENQUEUE_TOKEN" -H 'Content-Type: application/json' -d "$body")
|
||||
cat /tmp/enqueue.out; echo
|
||||
# 202 Accepted = queued; 409 = no live runner for the pool (surface it loud).
|
||||
if [ "$code" != "202" ]; then echo "::error::enqueue $image failed (HTTP $code)"; exit 1; fi
|
||||
echo "::endgroup::"
|
||||
done
|
||||
done
|
||||
|
||||
- name: Build & push images (per hanzo.yml)
|
||||
# AFTER the gate, deliberately. This step used to run before `test:`,
|
||||
# so a repo whose gates FAILED still published an image: hanzoai/base
|
||||
@@ -1607,52 +1689,40 @@ jobs:
|
||||
# that an EDGE serves, which is why this is a third verb and not a flag on
|
||||
# one of those two.
|
||||
#
|
||||
# The plane it publishes onto is hanzoai/cloud's Projects/sites API
|
||||
# (`/v1/sites`), whose model is VALUES PLUS A POINTER: a build output is
|
||||
# promoted into an immutable Release whose id is a digest of its object
|
||||
# manifest, and the site's pointer is flipped to it. `publish` is
|
||||
# promote+activate in one call — the 99% path — and rollback is the same
|
||||
# pointer aimed at an older release. Every first-party Hanzo site is moving
|
||||
# onto this, so the publish step belongs here ONCE rather than as a
|
||||
# hand-rolled sync in each site repo.
|
||||
# The plane it publishes onto is hanzoai/cloud's Projects/sites API,
|
||||
# whose model is VALUES PLUS A POINTER: a build output is promoted into an
|
||||
# immutable Release whose id is a digest of its object manifest, and the
|
||||
# site's pointer is flipped to it. `publish` is promote+activate in one
|
||||
# call — the 99% path — and rollback is the same pointer aimed at an older
|
||||
# release.
|
||||
#
|
||||
# HOW THE BYTES GET THERE. Not through the API. cloud's own artifact path
|
||||
# says so in as many words — a tar(.gz) through `/v1/projects/:slug/deploy`
|
||||
# is the small builder-one-click case, and "large sites use the git/CI path
|
||||
# that syncs to S3 directly and never streams bytes through this handler"
|
||||
# (apps/projects/blob.go). A first-party site is ~125 MiB across ~8,400
|
||||
# files; streaming that through a gateway body limit is not a path. So CI
|
||||
# syncs the export into the org's OWN storage space and then asks the
|
||||
# server to promote it, which is also why no object body ever crosses the
|
||||
# API and no browser ever holds an S3 credential.
|
||||
# WHY THIS LANE CHANGED. It used to stage the export with `mc mirror`
|
||||
# against hanzoai/s3, gated on S3_ADMIN_ACCESS_KEY + S3_ADMIN_SECRET_KEY
|
||||
# read from KMS. Those names are in KMS for NO org, and the step fails
|
||||
# closed without them — so `site:` refused every caller that declared one
|
||||
# and was dead code no repo could use. Measured before this change: of
|
||||
# the 155 hanzo.yml files across the three orgs, NOT ONE carried a
|
||||
# `site:` block, and hanzoai/console had grown its own build→zip→publish
|
||||
# workflow to ship at all. Per-repo build logic is the one thing this contract exists to
|
||||
# prevent, so the fix is not to provision that credential — it is to stop
|
||||
# needing it.
|
||||
#
|
||||
# The sync client is `mc`, provisioned the way this file provisions every
|
||||
# other tool it needs on a bare runner (jq, yq, kubectl, crane): ONE curl
|
||||
# of ONE pinned static binary into ~/.local/bin, no sudo, no archive, no
|
||||
# package manager. It is also the CLI of the SAME client family the server
|
||||
# uses — hanzoai/s3-go is built on the MinIO Go client — so CI and cloud
|
||||
# talk to hanzoai/s3 through one lineage instead of two. `mirror
|
||||
# --overwrite --remove` makes the staging prefix EXACTLY the export: a file
|
||||
# deleted from the build cannot linger into a release, and unchanged
|
||||
# objects are not re-sent.
|
||||
# The bytes now travel the route console proved: the zip goes to
|
||||
# /v1/projects/<slug>/deploy and the landed prefix is promoted by
|
||||
# /v1/sites/<slug>/publish, both on the IAM bearer the KMS step above
|
||||
# ALREADY minted. CI holds no bucket key, names no bucket and names no
|
||||
# org — the org segment is prepended server-side from the validated
|
||||
# principal, which is what makes the prefix unforgeable. Nothing is left
|
||||
# for an org to be missing.
|
||||
#
|
||||
# IDEMPOTENCY IS BY CONSTRUCTION, not by a remembered request key. The
|
||||
# release id is a SHA-256 over the sorted manifest of (key, size, etag), so
|
||||
# the same bytes produce the same id; cloud finds that release row already
|
||||
# present and short-circuits — no copy, no new release, no second billable
|
||||
# deploy. Re-running this job on an unchanged export is therefore a no-op
|
||||
# that still ends green and still prints the release it left live. Nothing
|
||||
# in this step needs to remember that it ran.
|
||||
#
|
||||
# The staging prefix is ONE per site (`<org>/.ci/<slug>`), a sibling of the
|
||||
# live site prefix and of the server's own `<org>/.releases` space — a
|
||||
# project slug can never be ".ci" because slugs forbid '.', and neither a
|
||||
# deploy purge nor a project delete reaches it. Bounded, not per-commit:
|
||||
# the org's storage holds the last export, not every export ever made. Two
|
||||
# publishes racing on one site is the one thing that costs: cloud copies
|
||||
# each object conditional on the exact ETag it digested, so a source that
|
||||
# moves mid-publish is a 409 and the release is abandoned rather than
|
||||
# half-made. That fails this step loudly; the next push converges.
|
||||
# THE WORK IS IN bin/sitepublish, NOT HERE. Forty lines of zip → upload →
|
||||
# promote → verify inlined in a workflow is forty lines nobody can run
|
||||
# offline, and this file already learned that with imgver and gover.
|
||||
# bin/sitepublish_test.sh drives it through a curl shim with no network,
|
||||
# and is weighted toward the REFUSALS — a publish step that reports
|
||||
# success when it verified nothing is worse than one that never verified,
|
||||
# because it is indistinguishable from a working one until a site quietly
|
||||
# stops updating.
|
||||
if: inputs.mode != 'delegate' && github.event_name != 'pull_request'
|
||||
env:
|
||||
API: ${{ vars.API_ENDPOINT || 'https://api.hanzo.ai' }}
|
||||
@@ -1678,87 +1748,19 @@ jobs:
|
||||
if [ -z "$SLUG" ] || [ -z "$DIR" ]; then
|
||||
echo "::error::site: needs both slug: (the /v1/sites project) and dir: (the built export)"; exit 1
|
||||
fi
|
||||
# The slug is a URL path segment AND an S3 key segment here, so it is held
|
||||
# to the SAME grammar cloud validates project slugs with. Checked before
|
||||
# anything is uploaded: a bad slug is a typo to fix, not a 404 to read.
|
||||
echo "$SLUG" | grep -qE '^[a-z0-9]([a-z0-9-]{0,38}[a-z0-9])?$' \
|
||||
|| { echo "::error::site.slug '$SLUG' is not a project slug (^[a-z0-9]([a-z0-9-]{0,38}[a-z0-9])?\$)"; exit 1; }
|
||||
if [ -n "$BUILD" ]; then
|
||||
echo "::group::build $SLUG ($BUILD)"
|
||||
sh -c "$BUILD"
|
||||
echo "::endgroup::"
|
||||
fi
|
||||
# A site is a thing with an index — the one contract cloud enforces
|
||||
# however content arrives. Checked HERE, before the mirror, because
|
||||
# `--remove` on a failed build would otherwise empty the staging prefix
|
||||
# and turn a broken build into a broken publish.
|
||||
[ -f "$DIR/index.html" ] \
|
||||
|| { echo "::error::site.dir '$DIR' has no index.html at its root — /v1/sites refuses a source without one"; exit 1; }
|
||||
# Fail closed. Publishing with no credential is not a warning: it would
|
||||
# leave the site on its previous release with a green run claiming
|
||||
# otherwise. (All three come from the KMS step above — never from git.)
|
||||
if [ -z "${S3_ADMIN_ACCESS_KEY:-}" ] || [ -z "${S3_ADMIN_SECRET_KEY:-}" ] \
|
||||
|| [ -z "${HANZO_API_TOKEN:-}" ] || [ -z "${HANZO_ORG:-}" ]; then
|
||||
echo "::error::hanzo.yml declares site: but KMS supplied no publish credential (S3_ADMIN_ACCESS_KEY/SECRET_KEY + an IAM token for $API) — refusing to publish"; exit 1
|
||||
# Fail closed on the ONE thing that can be missing, and say which one.
|
||||
# An empty token here means the KMS login did not happen — not that a
|
||||
# site credential was never provisioned, which is what the old message
|
||||
# said and what sent people looking for a secret that does not exist.
|
||||
if [ -z "${HANZO_API_TOKEN:-}" ]; then
|
||||
echo "::error::hanzo.yml declares site: but the KMS step minted no IAM token for $API — refusing to publish. That is a KMS_CLIENT_ID/KMS_CLIENT_SECRET problem on this repo, not a missing site credential."; exit 1
|
||||
fi
|
||||
command -v mc >/dev/null 2>&1 || {
|
||||
mkdir -p "$HOME/.local/bin"; export PATH="$HOME/.local/bin:$PATH"
|
||||
curl -fsSL https://dl.min.io/client/mc/release/linux-amd64/archive/mc.RELEASE.2025-08-13T08-35-41Z \
|
||||
-o "$HOME/.local/bin/mc" && chmod +x "$HOME/.local/bin/mc"
|
||||
}
|
||||
# Same public endpoint + scheme the binaries lane publishes through, from
|
||||
# the same KMS names, so CI reaches hanzoai/s3 by ONE address.
|
||||
SCHEME=https
|
||||
if [ "${S3_PUBLIC_SECURE:-true}" = "false" ]; then SCHEME=http; fi
|
||||
HOST="${S3_PUBLIC_ENDPOINT:-s3.hanzo.ai}"
|
||||
BUCKET="${CLOUD_PROJECTS_BUCKET:-hanzo-sites}"
|
||||
# The credential reaches mc through MC_HOST_<alias> — never `mc alias set`,
|
||||
# whose key and secret would sit in argv for anything reading /proc, and
|
||||
# never a config file on the runner. URL-encoded with jq so a secret
|
||||
# containing '/' or '+' cannot re-parse the alias URL, and masked as a
|
||||
# whole because the composed string carries the secret verbatim.
|
||||
MC_HOST_hanzosite="$SCHEME://$(jq -rn --arg v "$S3_ADMIN_ACCESS_KEY" '$v|@uri'):$(jq -rn --arg v "$S3_ADMIN_SECRET_KEY" '$v|@uri')@$HOST"
|
||||
echo "::add-mask::$MC_HOST_hanzosite"
|
||||
export MC_HOST_hanzosite
|
||||
# `source` is RELATIVE to the org's own storage root. The org segment is
|
||||
# prepended SERVER-side from the validated principal and the bucket never
|
||||
# appears in the request at all (cloud apps/projects/release.go,
|
||||
# sourcePrefix) — an absolute path or a URL is refused — so the worst a
|
||||
# source can address is something this org already owns. CI names the
|
||||
# same prefix on the S3 side, org included, because there it IS writing a
|
||||
# key.
|
||||
SRC=".ci/$SLUG"
|
||||
echo "::group::sync $DIR → $BUCKET/$HANZO_ORG/$SRC"
|
||||
mc --no-color mirror --overwrite --remove --summary "$DIR" "hanzosite/$BUCKET/$HANZO_ORG/$SRC"
|
||||
echo "::endgroup::"
|
||||
# Promote + activate. Content types and cache policy are NOT our problem:
|
||||
# copyRelease replaces the metadata on every object it promotes, so a
|
||||
# release serves identically however its bytes were staged.
|
||||
code=$(curl -sS -o "$RUNNER_TEMP/publish.json" -w '%{http_code}' \
|
||||
-X POST "$API/v1/sites/$SLUG/publish" \
|
||||
-H "Authorization: Bearer $HANZO_API_TOKEN" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "$(jq -nc --arg s "$SRC" '{source:$s}')") \
|
||||
|| { echo "::error::POST $API/v1/sites/$SLUG/publish did not complete"; exit 1; }
|
||||
# Non-2xx is fatal and PRINTS ITS BODY. cloud answers this route with a
|
||||
# specific status per cause — 402 hosting gate, 403 wrong org, 404 no such
|
||||
# site for this tenant, 409 the source moved mid-publish, 413 past the
|
||||
# object/byte caps, 503 storage unconfigured — and every one of them is a
|
||||
# different fix, so swallowing the body would be discarding the answer.
|
||||
case "$code" in
|
||||
2??) ;;
|
||||
*) echo "::error::publish $SLUG → HTTP $code"; cat "$RUNNER_TEMP/publish.json"; echo; exit 1 ;;
|
||||
esac
|
||||
rid=$(jq -r '.releaseId // empty' "$RUNNER_TEMP/publish.json")
|
||||
url=$(jq -r '.url // empty' "$RUNNER_TEMP/publish.json")
|
||||
objs=$(jq -r '.objects // 0' "$RUNNER_TEMP/publish.json")
|
||||
[ -n "$rid" ] \
|
||||
|| { echo "::error::publish answered $code with no releaseId:"; cat "$RUNNER_TEMP/publish.json"; echo; exit 1; }
|
||||
# An empty url is a real answer, not a failure: the bare <slug> host is
|
||||
# first-come across every org, so a site whose slug is already claimed is
|
||||
# live at its own prefix with no public subdomain and is told to rename.
|
||||
echo "published $rid ($objs objects) → ${url:-no public host — that slug is claimed; rename the site or bind a domain}"
|
||||
{ echo "### Site — \`$SLUG\`"; echo
|
||||
echo "| release | objects | url |"; echo "|---|---|---|"
|
||||
echo "| \`$rid\` | $objs | ${url:-—} |"
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
# Slug grammar, index.html, the server's file/byte caps and the edge
|
||||
# BodyLimit are all checked inside sitepublish, before anything is sent,
|
||||
# so each refusal names its own number instead of arriving as a 4xx.
|
||||
HANZO_API="$API" "$CI_HOME/bin/sitepublish" "$SLUG" "$DIR"
|
||||
|
||||
+247
-245
@@ -29,14 +29,17 @@ on:
|
||||
default: '["hanzo-build-linux-amd64"]'
|
||||
mode:
|
||||
description: >-
|
||||
Build execution mode. `buildx` (default) runs the full buildx →
|
||||
test → deploy pipeline ON the runner. `delegate` instead POSTs the
|
||||
build to platform.hanzo.ai (`/v1/runner`) — platform builds
|
||||
in-cluster with BuildKit and rolls the operator Service CR itself, so
|
||||
the GitHub job finishes in seconds with no runner buildx. A repo opts
|
||||
in by passing `with: { mode: delegate }`; everything else is unchanged.
|
||||
Requires the `PLATFORM_BUILD_CALLBACK_TOKEN` secret (via secrets:
|
||||
inherit).
|
||||
Where the IMAGE is built. `buildx` (default) builds it on this runner.
|
||||
`delegate` POSTs it to platform.hanzo.ai (`/v1/runner`), which builds
|
||||
in-cluster with BuildKit and rolls the operator Service CR itself.
|
||||
A repo opts in with `with: { mode: delegate }` and needs the
|
||||
`PLATFORM_BUILD_CALLBACK_TOKEN` secret (via secrets: inherit).
|
||||
|
||||
THE GATE RUNS IN BOTH MODES, and that is the whole point of the input
|
||||
being named for the build. `delegate` used to be evaluated fourth, ahead
|
||||
of every toolchain and ahead of `test:`, so it skipped the gate along
|
||||
with the build — 20 steps skipped, 0 executed, green. What a delegated
|
||||
run skips is buildx, publish and deploy. It does not skip being tested.
|
||||
type: string
|
||||
default: buildx
|
||||
tests:
|
||||
@@ -73,7 +76,7 @@ on:
|
||||
A string sidesteps it on both planes with identical meaning, and the
|
||||
comparison is against 'false' rather than a truthiness test so that the
|
||||
expression callers already write — hanzoai/cloud's
|
||||
`tests: ${{ github.ref_type != 'tag' }}`, which renders the strings
|
||||
`tests:` set from `github.ref_type != 'tag'`, which renders the strings
|
||||
"true"/"false" — keeps working unchanged.
|
||||
type: string
|
||||
default: 'true'
|
||||
@@ -174,11 +177,23 @@ jobs:
|
||||
# on they are ordinary content: status clean, diff empty, both sides
|
||||
# of the merge shipping. It hides in generated and vendored files,
|
||||
# which are large, rarely opened and effectively unreviewed —
|
||||
# hanzoai/base carried `<<<<<<< HEAD` / `>>>>>>> upstream/master` at
|
||||
# hanzoai/base carried ` / `>>>>>>> upstream/master` at
|
||||
# lines 23807 and 23850 of a committed .d.ts. No compiler catches it
|
||||
# there: a .d.ts nothing imports is never parsed, and in YAML, JSON,
|
||||
# Markdown and every config format we ship the markers are silently
|
||||
# VALID content.
|
||||
# vendormark — a site-generator vendor's branding left in the tree.
|
||||
# Several sites began as output from a hosted "prompt to React app"
|
||||
# generator, which signs its work in four places at once: an asset
|
||||
# folder named after the vendor (so its name is in every image URL,
|
||||
# and on one site in the FAVICON, so it was in every browser tab), a
|
||||
# build plugin in package.json, an injected third-party <script>,
|
||||
# and the commit messages, author identities and Co-authored-by
|
||||
# trailers of its bot. The last survives any amount of tidying the
|
||||
# working tree — a repo can render a perfect page and still say in
|
||||
# `git log` who really wrote it. Removing it needed a history
|
||||
# rewrite and a force-push of every branch; this is so that is never
|
||||
# needed twice.
|
||||
#
|
||||
# Both proven RED against cloud@v1.801.424 (the real outage) and GREEN
|
||||
# against the repaired tree before landing here. They read the checkout
|
||||
@@ -190,6 +205,7 @@ jobs:
|
||||
"$RUNNER_TEMP/ci/bin/ignoretracked" .
|
||||
"$RUNNER_TEMP/ci/bin/modsize" .
|
||||
"$RUNNER_TEMP/ci/bin/conflictmarkers" .
|
||||
"$RUNNER_TEMP/ci/bin/vendormark" .
|
||||
|
||||
- name: Provision parse toolchain (jq + PyYAML)
|
||||
# This reusable parses the caller's hanzo.yml with python3 + PyYAML and
|
||||
@@ -222,64 +238,7 @@ jobs:
|
||||
jq --version
|
||||
yq --version
|
||||
|
||||
- name: Delegate build to platform (mode=delegate)
|
||||
# The GHA-escape fast path: instead of running buildx on this runner, POST
|
||||
# each image in hanzo.yml to platform.hanzo.ai's direct-enqueue webhook
|
||||
# (`/v1/runner`). Platform creates a build_job row, launches an
|
||||
# in-cluster BuildKit Job on its own pool, pushes to the registry, and —
|
||||
# for a system service — patches the operator Service CR to roll it. The
|
||||
# downstream is IDENTICAL to the platform GitHub-App webhook path (one
|
||||
# build path, two front doors), so a delegated build behaves exactly like
|
||||
# a platform-native one. This job then exits in seconds — no buildx, no
|
||||
# KMS, no runner-side deploy.
|
||||
if: inputs.mode == 'delegate'
|
||||
env:
|
||||
ENQUEUE_URL: ${{ vars.PLATFORM_ENQUEUE_URL || 'https://platform.hanzo.ai/v1/runner' }}
|
||||
ENQUEUE_TOKEN: ${{ secrets.PLATFORM_BUILD_CALLBACK_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${ENQUEUE_TOKEN:-}" ]; then
|
||||
echo "::error::mode=delegate needs the PLATFORM_BUILD_CALLBACK_TOKEN secret (secrets: inherit)"; exit 1
|
||||
fi
|
||||
REPO="${{ github.repository }}"
|
||||
SHA="${{ github.sha }}"
|
||||
SHORT=$(echo "$SHA" | cut -c1-7)
|
||||
REF="${{ github.ref }}"
|
||||
BRANCH="${{ github.ref_name }}"
|
||||
# One enqueue per (image, platform), mirroring the buildx tag shape the
|
||||
# deploy path expects (`sha-<short>-<arch>[-<suffix>]`). Default arch is
|
||||
# amd64 (single-arch), so an existing repo's tag shape is unchanged.
|
||||
yq -o=json -I=0 '.images' hanzo.yml | jq -c '.[]' | while read -r img; do
|
||||
name=$(echo "$img"|jq -r .name); repo=$(echo "$img"|jq -r .repo)
|
||||
ctx=$(echo "$img"|jq -r .context); df=$(echo "$img"|jq -r '.dockerfile // (.context+"/Dockerfile")')
|
||||
sfx=$(echo "$img"|jq -r '."tag-suffix" // ""')
|
||||
# Same Go-builder check the buildx lane runs. It belongs here too:
|
||||
# delegating the build moves WHERE it runs, not whether the image
|
||||
# can compile the module, and a mismatch enqueued to platform fails
|
||||
# on a machine whose logs this run never shows. Cheap to check on
|
||||
# the runner that already has the checkout; expensive to discover
|
||||
# from the in-cluster builder.
|
||||
"$CI_HOME/bin/gover" "$df" "$ctx"
|
||||
echo "$img" | jq -r '(.platforms // ["linux/amd64"])[]' | while read -r plat; do
|
||||
arch="${plat##*/}"
|
||||
image="${repo}:sha-${SHORT}-${arch}${sfx:+-$sfx}"
|
||||
body=$(jq -nc \
|
||||
--arg repo "$REPO" --arg sha "$SHA" --arg image "$image" \
|
||||
--arg ref "$REF" --arg branch "$BRANCH" \
|
||||
--arg dockerfile "$df" --arg context "$ctx" --arg arch "$arch" \
|
||||
'{repo:$repo,sha:$sha,image:$image,ref:$ref,branch:$branch,dockerfile:$dockerfile,context:$context,os:"linux",arch:$arch}')
|
||||
echo "::group::delegate $name → $image"
|
||||
code=$(curl -sS -o /tmp/enqueue.out -w '%{http_code}' -X POST "$ENQUEUE_URL" \
|
||||
-H "Authorization: Bearer $ENQUEUE_TOKEN" -H 'Content-Type: application/json' -d "$body")
|
||||
cat /tmp/enqueue.out; echo
|
||||
# 202 Accepted = queued; 409 = no live runner for the pool (surface it loud).
|
||||
if [ "$code" != "202" ]; then echo "::error::enqueue $image failed (HTTP $code)"; exit 1; fi
|
||||
echo "::endgroup::"
|
||||
done
|
||||
done
|
||||
|
||||
- name: Authenticated git for go modules (rate-limit + any private repo)
|
||||
if: inputs.mode != 'delegate'
|
||||
# luxfi/hanzoai/zooai Go modules are PUBLIC, so `go` resolves them through
|
||||
# the default public proxy (proxy.golang.org) + checksum db (sum.golang.org)
|
||||
# — canonical, IMMUTABLE hashes that a force-moved tag can no longer break.
|
||||
@@ -335,7 +294,6 @@ jobs:
|
||||
|
||||
- name: Fetch deploy credentials from KMS
|
||||
id: kms
|
||||
if: inputs.mode != 'delegate'
|
||||
env:
|
||||
KMS_CLIENT_ID: ${{ secrets.KMS_CLIENT_ID }}
|
||||
KMS_CLIENT_SECRET: ${{ secrets.KMS_CLIENT_SECRET }}
|
||||
@@ -375,6 +333,15 @@ jobs:
|
||||
if [ -z "$ORG" ]; then
|
||||
case "${{ github.repository_owner }}" in
|
||||
hanzoai) ORG=hanzo ;;
|
||||
# hanzo-inc is the same TENANT as hanzoai — pay, commerce and the
|
||||
# rest of the company surfaces bill and seal to KMS org `hanzo`.
|
||||
# Without this line ORG fell through to the literal owner,
|
||||
# "hanzo-inc", which is not a KMS org. That still LOOKED fine,
|
||||
# because the read below takes its org from the TOKEN, not the URL
|
||||
# — so the only things that were wrong were the fallback URL and
|
||||
# every error message, which named an org that does not exist and
|
||||
# sent the reader looking for a tenant instead of a credential.
|
||||
hanzo-inc) ORG=hanzo ;;
|
||||
luxfi) ORG=lux ;;
|
||||
zooai) ORG=zoo ;;
|
||||
*) ORG="${{ github.repository_owner }}" ;;
|
||||
@@ -391,6 +358,40 @@ jobs:
|
||||
# how a stale key survived: every failure looked like "not in KMS", and
|
||||
# "not in KMS" was survivable.
|
||||
DECLARED="$(yq -r '[(.images // [])[] | (.build_secrets // [])[]] | unique | .[]' hanzo.yml 2>/dev/null || true)"
|
||||
# A build_secret becomes `--build-arg NAME=value` below, and a build-arg
|
||||
# is IN THE PUBLISHED IMAGE — `docker history` prints it to anyone who
|
||||
# can pull. So `build_secrets` can only ever carry values that are
|
||||
# public on purpose: a Vite/Next SPA has no server to read an env from,
|
||||
# so its ingest key must be inlined at build, and inlining it is what
|
||||
# publishing it means. Every other value must not be here at all.
|
||||
#
|
||||
# Nothing checked that. The one repo whose value is genuinely
|
||||
# publishable (hanzoai/ui, a `pk-…` ingest key) asserted the `pk-`
|
||||
# prefix INSIDE ITS OWN Dockerfile — so the check existed once, for one
|
||||
# image, and any repo that added a real credential got no check at all.
|
||||
# Per-repo vigilance is not a mechanism; the assertion belongs here,
|
||||
# where the build-arg is assembled, once, for every caller.
|
||||
#
|
||||
# INLINE, not "$CI_HOME/bin/publishable", and that is forced rather than
|
||||
# chosen. The tools checkout and this workflow resolve from DIFFERENT
|
||||
# places: the caller pins the workflow at a ref its forge resolves, while
|
||||
# the tools step derives its ref from GITHUB_WORKFLOW_REF — which
|
||||
# git.hanzo.ai does not set, so it falls back to `v1` and clones
|
||||
# github.com. Measured on run 36473: `GITHUB_WORKFLOW_REF=<unset>`,
|
||||
# `derived ref=v1`, `cloned https://github.com/hanzoai/ci@v1`. A step
|
||||
# that calls a NEW file under $CI_HOME is therefore broken on every forge
|
||||
# run until a tag moves on a host this repo is not even pinned to.
|
||||
#
|
||||
# The rule is also spelled in bin/publishable, which the `publishable`
|
||||
# gate tests. Two spellings of one rule is the same shape as the two
|
||||
# copies of this file, and it gets the same treatment: a gate refuses
|
||||
# any difference between them. See `publishable-rule-is-one-rule`.
|
||||
for bs in $DECLARED; do
|
||||
case "$bs" in
|
||||
PUBLISHABLE_*|PUBLIC_*|NEXT_PUBLIC_*|EXPO_PUBLIC_*|NUXT_PUBLIC_*|VITE_*|REACT_APP_*|*_PUBLISHABLE|*_PUBLIC) ;;
|
||||
*) echo "::error::build_secret '$bs' does not declare itself publishable, and a build_secret is baked into the image as a --build-arg where \`docker history\` reveals it. Rename it (PUBLISHABLE_*, PUBLIC_*, NEXT_PUBLIC_*, VITE_*, REACT_APP_*) if the value is public on purpose; if it is a real credential it cannot be a build_secret at all."; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
required() { [ -n "$DECLARED" ]; }
|
||||
if [ -z "${KMS_CLIENT_ID:-}" ] || [ -z "$ORG" ]; then
|
||||
if required; then
|
||||
@@ -425,28 +426,45 @@ jobs:
|
||||
if [ -n "$GIT_TOKEN" ]; then echo "::add-mask::$GIT_TOKEN"; echo "GIT_TOKEN=$GIT_TOKEN" >> "$GITHUB_ENV"; fi
|
||||
KUBECONFIG_B64=$(get KUBECONFIG)
|
||||
if [ -n "$KUBECONFIG_B64" ]; then echo "$KUBECONFIG_B64" | base64 -d > "$RUNNER_TEMP/kubeconfig"; echo "kubeconfig=$RUNNER_TEMP/kubeconfig" >> "$GITHUB_OUTPUT"; fi
|
||||
# Object-store credential, for the two lanes that write bytes to
|
||||
# hanzoai/s3: a repo publishing `binaries:` to a bucket:, and a repo
|
||||
# publishing a `site:` (whose build output has to land in the org's own
|
||||
# storage space before /v1/sites can promote it). Same four names the
|
||||
# services read at run time, from the same org/path/env — a CI-only copy
|
||||
# of a credential is a second thing to rotate and the one nobody
|
||||
# remembers. One fetch serves both lanes; a repo declaring both does not
|
||||
# ask KMS twice.
|
||||
# The REGISTRY credential, and it is fetched UNCONDITIONALLY because
|
||||
# every repo that publishes an image needs it — which is most of them.
|
||||
#
|
||||
# It used to sit inside the `bucket:`/`site:` gate below, and that gate
|
||||
# is about writing bytes to hanzoai/s3, a completely unrelated lane. So
|
||||
# a repo declaring only `images:` — hanzoai/bot, and it is not alone —
|
||||
# asked KMS for nothing and reached the docker login step with empty
|
||||
# credentials. The build then failed at the push with no clue pointing
|
||||
# here, and the obvious "fix" is to paste REGISTRY_USER/REGISTRY_PASSWORD
|
||||
# into the forge's org secrets, which is exactly the CI-only copy the
|
||||
# comment below warns about: a second thing to rotate, and the one
|
||||
# nobody remembers. The contract is that the ONLY things living in the
|
||||
# forge are KMS_CLIENT_ID/SECRET, so the fetch moves out of the gate
|
||||
# rather than the credential moving into GitHub.
|
||||
#
|
||||
# Empty is harmless: nothing is exported when KMS has no such key, and
|
||||
# the login step already falls back to the kubeconfig read for repos
|
||||
# provisioned before these names existed.
|
||||
for k in REGISTRY_USER REGISTRY_PASSWORD; do
|
||||
v=$(get "$k")
|
||||
if [ -n "$v" ]; then
|
||||
[ "$k" = REGISTRY_PASSWORD ] && echo "::add-mask::$v"
|
||||
echo "$k=$v" >> "$GITHUB_ENV"
|
||||
fi
|
||||
done
|
||||
# Object-store credential, for the ONE lane that writes bytes to
|
||||
# hanzoai/s3 itself: a repo publishing `binaries:` to a bucket:. Same
|
||||
# names the services read at run time, from the same org/path/env — a
|
||||
# CI-only copy of a credential is a second thing to rotate and the one
|
||||
# nobody remembers.
|
||||
#
|
||||
# `site:` used to be fetched here too, and that is exactly what killed
|
||||
# it: S3_ADMIN_ACCESS_KEY/S3_ADMIN_SECRET_KEY are not in KMS for any
|
||||
# org, so the site lane's fail-closed check refused every caller that
|
||||
# ever declared one. The site lane no longer touches S3 — it hands the
|
||||
# zip to cloud and cloud writes the bytes — so the credential it cannot
|
||||
# have is one it no longer needs.
|
||||
SITE_DECL="$(yq -r '.site | type' hanzo.yml 2>/dev/null || echo '!!null')"
|
||||
if [ -n "$(yq -r '.bucket // ""' hanzo.yml 2>/dev/null || true)" ] || [ "$SITE_DECL" = '!!map' ]; then
|
||||
# The registry credential comes from KMS like every other secret —
|
||||
# the ONLY things that live in GitHub are KMS_CLIENT_ID/SECRET.
|
||||
# Named here so a repo with no kubeconfig (most of them) can still
|
||||
# reach registry.hanzo.ai; the kubeconfig read below stays as the
|
||||
# fallback for repos provisioned before these names existed.
|
||||
for k in REGISTRY_USER REGISTRY_PASSWORD; do
|
||||
v=$(get "$k")
|
||||
if [ -n "$v" ]; then
|
||||
[ "$k" = REGISTRY_PASSWORD ] && echo "::add-mask::$v"
|
||||
echo "$k=$v" >> "$GITHUB_ENV"
|
||||
fi
|
||||
done
|
||||
if [ -n "$(yq -r '.bucket // ""' hanzo.yml 2>/dev/null || true)" ]; then
|
||||
for k in S3_ADMIN_ACCESS_KEY S3_ADMIN_SECRET_KEY; do
|
||||
v=$(get "$k")
|
||||
if [ -n "$v" ]; then echo "::add-mask::$v"; echo "$k=$v" >> "$GITHUB_ENV"; fi
|
||||
@@ -458,38 +476,30 @@ jobs:
|
||||
if [ -n "$v" ]; then echo "$k=$v" >> "$GITHUB_ENV"; fi
|
||||
done
|
||||
fi
|
||||
# `site:` needs two things beyond the S3 credential above, and BOTH come
|
||||
# from this step — there is one KMS login in this workflow and the site
|
||||
# lane does not add a second.
|
||||
# `site:` needs exactly ONE thing from this step, and it is a thing this
|
||||
# step already has — there is one KMS login in this workflow and the
|
||||
# site lane does not add a second.
|
||||
#
|
||||
# CLOUD_PROJECTS_BUCKET — the bucket /v1/sites lists the build output
|
||||
# in. The SERVER owns it (it is never in a publish request, which is
|
||||
# what makes the org prefix unforgeable), so CI cannot be told it by
|
||||
# hanzo.yml; it reads the same KMS name the cloud binary reads, and
|
||||
# falls back to the same default. One name, one place.
|
||||
# HANZO_API_TOKEN — the bearer for /v1/sites. It is the token THIS STEP
|
||||
# ALREADY MINTED: cloud's /v1/kms/auth/login is a broker that
|
||||
# exchanges clientId/clientSecret at IAM's client_credentials
|
||||
# endpoint and "returns IAM's own JWT verbatim" (cloud
|
||||
# apps/kms/login.go), scoped by its `owner` claim to exactly this
|
||||
# org. api.hanzo.ai validates that same issuer and mints X-Org-Id
|
||||
# from it. So the credential that reads this org's secrets and the
|
||||
# credential that publishes this org's site are ONE machine identity,
|
||||
# with one lifetime and one thing to rotate. It is masked here,
|
||||
# before GITHUB_ENV is the first place it could surface.
|
||||
# HANZO_API_TOKEN — the bearer for /v1/projects and /v1/sites. It is
|
||||
# the token THIS STEP ALREADY MINTED: cloud's /v1/kms/auth/login is
|
||||
# a broker that performs the IAM client_credentials exchange and
|
||||
# "returns IAM's own JWT verbatim" (cloud apps/kms/login.go), scoped
|
||||
# by its `owner` claim to exactly this org. api.hanzo.ai validates
|
||||
# that same issuer and mints X-Org-Id from it. So the credential
|
||||
# that reads this org's secrets and the credential that publishes
|
||||
# this org's site are ONE machine identity, with one lifetime and
|
||||
# one thing to rotate. It is masked here, before GITHUB_ENV is the
|
||||
# first place it could surface.
|
||||
#
|
||||
# HANZO_ORG is exported with them because the S3 upload has to name the
|
||||
# org segment the server will prepend to `source`. That it AGREES with
|
||||
# the publish identity is not an assumption: every `get` above read
|
||||
# /v1/kms/orgs/$ORG/..., and cloud re-checks the token's `owner` against
|
||||
# that :org on each read — so an ORG that disagreed would have fetched no
|
||||
# credential at all, and this lane fails closed below rather than
|
||||
# uploading into a prefix nobody will look in.
|
||||
# Neither the bucket nor the org is exported any more, because neither
|
||||
# is CI's to name. The org segment is prepended SERVER-side from the
|
||||
# validated principal and the bucket never appears in a request at all
|
||||
# (cloud apps/projects/blob.go:60, `sitePrefix(org, slug)`) — which is
|
||||
# precisely what makes the prefix unforgeable. CI passing them was CI
|
||||
# restating something it could only get wrong.
|
||||
if [ "$SITE_DECL" = '!!map' ]; then
|
||||
v=$(get CLOUD_PROJECTS_BUCKET)
|
||||
if [ -n "$v" ]; then echo "CLOUD_PROJECTS_BUCKET=$v" >> "$GITHUB_ENV"; fi
|
||||
echo "::add-mask::$TOKEN"
|
||||
{ echo "HANZO_API_TOKEN=$TOKEN"; echo "HANZO_ORG=$ORG"; } >> "$GITHUB_ENV"
|
||||
echo "HANZO_API_TOKEN=$TOKEN" >> "$GITHUB_ENV"
|
||||
fi
|
||||
# Build-time secrets → --build-arg. A repo declares per-image
|
||||
# `build_secrets: [NAME, ...]` in hanzo.yml; each NAME is fetched from
|
||||
@@ -601,7 +611,7 @@ jobs:
|
||||
# OWN Go version from go.mod so the toolchain matches the module exactly.
|
||||
# Guarded to Go repos (go.mod present) so pure-JS/TS callers are
|
||||
# unaffected; harmless if a future runner image bakes Go in.
|
||||
if: inputs.mode != 'delegate' && hashFiles('go.mod') != ''
|
||||
if: hashFiles('go.mod') != ''
|
||||
uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version-file: go.mod
|
||||
@@ -615,7 +625,7 @@ jobs:
|
||||
# fails (snapshot mirror rot: "no longer has a Release file"),
|
||||
# repoint archive.ubuntu.com at the DO mirror — both sources.list and
|
||||
# noble's deb822 ubuntu.sources — and retry once.
|
||||
if: inputs.mode != 'delegate' && hashFiles('go.mod') != ''
|
||||
if: hashFiles('go.mod') != ''
|
||||
run: |
|
||||
command -v gcc >/dev/null 2>&1 && exit 0
|
||||
sudo apt-get update -qq || {
|
||||
@@ -640,12 +650,12 @@ jobs:
|
||||
# root-only hashFiles SKIPPED this step and the gate died at
|
||||
# bash: line 3: pnpm: command not found (exit 127)
|
||||
# after 49 ginkgo suites had already passed.
|
||||
if: inputs.mode != 'delegate' && hashFiles('package.json', '**/package.json') != ''
|
||||
if: hashFiles('package.json', '**/package.json') != ''
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 22
|
||||
- name: Enable corepack (pnpm/yarn shims for JS test gates)
|
||||
if: inputs.mode != 'delegate' && hashFiles('package.json', '**/package.json') != ''
|
||||
if: hashFiles('package.json', '**/package.json') != ''
|
||||
run: corepack enable
|
||||
|
||||
- name: Provision Rust toolchain (cargo test gates)
|
||||
@@ -657,7 +667,7 @@ jobs:
|
||||
# the exact failure this workflow refuses everywhere else.
|
||||
# Guarded to Rust callers (Cargo.toml present); a no-op once a runner
|
||||
# image bakes rustup in.
|
||||
if: inputs.mode != 'delegate' && hashFiles('Cargo.toml') != ''
|
||||
if: hashFiles('Cargo.toml') != ''
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
|
||||
- name: Provision Java toolchain (client codegen)
|
||||
@@ -876,7 +886,7 @@ jobs:
|
||||
# `http`. Take the scheme and the host as the two values they are, and
|
||||
# compare the HOST against github.com so the guard cannot be fooled by a
|
||||
# spelling either.
|
||||
if: inputs.mode != 'delegate' && hashFiles('go.mod') != ''
|
||||
if: hashFiles('go.mod') != ''
|
||||
env:
|
||||
GH_PAT: ${{ secrets.GH_PAT }}
|
||||
FORGE_TOKEN: ${{ secrets.FORGE_TOKEN }}
|
||||
@@ -927,7 +937,7 @@ jobs:
|
||||
# it ran nothing AND nothing shows as having run. A gate that is not a test
|
||||
# gate — vet, lint, a build, a codegen-freshness check — says neither and is
|
||||
# untouched. To silence it legitimately, make the gate run a test.
|
||||
if: inputs.mode != 'delegate' && inputs.tests != 'false'
|
||||
if: inputs.tests != 'false'
|
||||
run: |
|
||||
set -euo pipefail
|
||||
NOTHING='\[no test files\]|\[no tests to run\]|no tests ran|collected 0 items|No tests found|no test specified|running 0 tests|Tests:[[:space:]]+0 total|(^|[^0-9])0 (passing|passed)'
|
||||
@@ -956,6 +966,78 @@ jobs:
|
||||
fi
|
||||
done
|
||||
|
||||
- name: Delegate build to platform (mode=delegate)
|
||||
# Delegate moves WHERE the image is built. It does not move whether the
|
||||
# commit was gated — so this step sits AFTER `Test (per hanzo.yml)`, and
|
||||
# every step the gate needs runs in both modes. It used to sit fourth,
|
||||
# before any of them, and `mode: delegate` therefore skipped the gate
|
||||
# along with the build: 20 steps skipped, 0 executed, green. "ci gates,
|
||||
# platform builds" was the design and the ordering said otherwise.
|
||||
#
|
||||
# What it does: POST each image in hanzo.yml to platform.hanzo.ai's
|
||||
# direct-enqueue webhook (`/v1/runner`). Platform creates a build_job
|
||||
# row, launches an in-cluster BuildKit Job on its own pool, pushes to the
|
||||
# registry, and — for a system service — patches the operator Service CR
|
||||
# to roll it. Downstream is IDENTICAL to the platform webhook path, so a
|
||||
# delegated build behaves exactly like a platform-native one. What this
|
||||
# job skips is buildx, publish and deploy — not the gate.
|
||||
if: inputs.mode == 'delegate'
|
||||
env:
|
||||
ENQUEUE_URL: ${{ vars.PLATFORM_ENQUEUE_URL || 'https://platform.hanzo.ai/v1/runner' }}
|
||||
ENQUEUE_TOKEN: ${{ secrets.PLATFORM_BUILD_CALLBACK_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${ENQUEUE_TOKEN:-}" ]; then
|
||||
echo "::error::mode=delegate needs the PLATFORM_BUILD_CALLBACK_TOKEN secret (secrets: inherit)"; exit 1
|
||||
fi
|
||||
# `/v1/runner` has no way to carry a build_secret: its wire body accepts
|
||||
# buildArgs, and platform resolves NO KMS name — its own secrets arrive
|
||||
# as a KMSSecret CRD with a statically declared key list, which cannot
|
||||
# serve a name discovered by reading a hanzo.yml at some SHA. A repo
|
||||
# that declares build_secrets and delegates gets an image built without
|
||||
# them: green run, empty credential, exactly the silence the KMS step
|
||||
# refuses. Say so here rather than downstream, where this run shows no
|
||||
# logs at all.
|
||||
if [ -n "$(yq -r '[(.images // [])[] | (.build_secrets // [])[]] | unique | .[]' hanzo.yml 2>/dev/null || true)" ]; then
|
||||
echo "::error::hanzo.yml declares build_secrets, which mode=delegate cannot deliver — platform builds with no KMS. Build on the runner (drop mode: delegate), or drop the build_secrets."; exit 1
|
||||
fi
|
||||
REPO="${{ github.repository }}"
|
||||
SHA="${{ github.sha }}"
|
||||
SHORT=$(echo "$SHA" | cut -c1-7)
|
||||
REF="${{ github.ref }}"
|
||||
BRANCH="${{ github.ref_name }}"
|
||||
# One enqueue per (image, platform), mirroring the buildx tag shape the
|
||||
# deploy path expects (`sha-<short>-<arch>[-<suffix>]`). Default arch is
|
||||
# amd64 (single-arch), so an existing repo's tag shape is unchanged.
|
||||
yq -o=json -I=0 '.images' hanzo.yml | jq -c '.[]' | while read -r img; do
|
||||
name=$(echo "$img"|jq -r .name); repo=$(echo "$img"|jq -r .repo)
|
||||
ctx=$(echo "$img"|jq -r .context); df=$(echo "$img"|jq -r '.dockerfile // (.context+"/Dockerfile")')
|
||||
sfx=$(echo "$img"|jq -r '."tag-suffix" // ""')
|
||||
# Same Go-builder check the buildx lane runs. It belongs here too:
|
||||
# delegating the build moves WHERE it runs, not whether the image
|
||||
# can compile the module, and a mismatch enqueued to platform fails
|
||||
# on a machine whose logs this run never shows. Cheap to check on
|
||||
# the runner that already has the checkout; expensive to discover
|
||||
# from the in-cluster builder.
|
||||
"$CI_HOME/bin/gover" "$df" "$ctx"
|
||||
echo "$img" | jq -r '(.platforms // ["linux/amd64"])[]' | while read -r plat; do
|
||||
arch="${plat##*/}"
|
||||
image="${repo}:sha-${SHORT}-${arch}${sfx:+-$sfx}"
|
||||
body=$(jq -nc \
|
||||
--arg repo "$REPO" --arg sha "$SHA" --arg image "$image" \
|
||||
--arg ref "$REF" --arg branch "$BRANCH" \
|
||||
--arg dockerfile "$df" --arg context "$ctx" --arg arch "$arch" \
|
||||
'{repo:$repo,sha:$sha,image:$image,ref:$ref,branch:$branch,dockerfile:$dockerfile,context:$context,os:"linux",arch:$arch}')
|
||||
echo "::group::delegate $name → $image"
|
||||
code=$(curl -sS -o /tmp/enqueue.out -w '%{http_code}' -X POST "$ENQUEUE_URL" \
|
||||
-H "Authorization: Bearer $ENQUEUE_TOKEN" -H 'Content-Type: application/json' -d "$body")
|
||||
cat /tmp/enqueue.out; echo
|
||||
# 202 Accepted = queued; 409 = no live runner for the pool (surface it loud).
|
||||
if [ "$code" != "202" ]; then echo "::error::enqueue $image failed (HTTP $code)"; exit 1; fi
|
||||
echo "::endgroup::"
|
||||
done
|
||||
done
|
||||
|
||||
- name: Build & push images (per hanzo.yml)
|
||||
# AFTER the gate, deliberately. This step used to run before `test:`,
|
||||
# so a repo whose gates FAILED still published an image: hanzoai/base
|
||||
@@ -1607,52 +1689,40 @@ jobs:
|
||||
# that an EDGE serves, which is why this is a third verb and not a flag on
|
||||
# one of those two.
|
||||
#
|
||||
# The plane it publishes onto is hanzoai/cloud's Projects/sites API
|
||||
# (`/v1/sites`), whose model is VALUES PLUS A POINTER: a build output is
|
||||
# promoted into an immutable Release whose id is a digest of its object
|
||||
# manifest, and the site's pointer is flipped to it. `publish` is
|
||||
# promote+activate in one call — the 99% path — and rollback is the same
|
||||
# pointer aimed at an older release. Every first-party Hanzo site is moving
|
||||
# onto this, so the publish step belongs here ONCE rather than as a
|
||||
# hand-rolled sync in each site repo.
|
||||
# The plane it publishes onto is hanzoai/cloud's Projects/sites API,
|
||||
# whose model is VALUES PLUS A POINTER: a build output is promoted into an
|
||||
# immutable Release whose id is a digest of its object manifest, and the
|
||||
# site's pointer is flipped to it. `publish` is promote+activate in one
|
||||
# call — the 99% path — and rollback is the same pointer aimed at an older
|
||||
# release.
|
||||
#
|
||||
# HOW THE BYTES GET THERE. Not through the API. cloud's own artifact path
|
||||
# says so in as many words — a tar(.gz) through `/v1/projects/:slug/deploy`
|
||||
# is the small builder-one-click case, and "large sites use the git/CI path
|
||||
# that syncs to S3 directly and never streams bytes through this handler"
|
||||
# (apps/projects/blob.go). A first-party site is ~125 MiB across ~8,400
|
||||
# files; streaming that through a gateway body limit is not a path. So CI
|
||||
# syncs the export into the org's OWN storage space and then asks the
|
||||
# server to promote it, which is also why no object body ever crosses the
|
||||
# API and no browser ever holds an S3 credential.
|
||||
# WHY THIS LANE CHANGED. It used to stage the export with `mc mirror`
|
||||
# against hanzoai/s3, gated on S3_ADMIN_ACCESS_KEY + S3_ADMIN_SECRET_KEY
|
||||
# read from KMS. Those names are in KMS for NO org, and the step fails
|
||||
# closed without them — so `site:` refused every caller that declared one
|
||||
# and was dead code no repo could use. Measured before this change: of
|
||||
# the 155 hanzo.yml files across the three orgs, NOT ONE carried a
|
||||
# `site:` block, and hanzoai/console had grown its own build→zip→publish
|
||||
# workflow to ship at all. Per-repo build logic is the one thing this contract exists to
|
||||
# prevent, so the fix is not to provision that credential — it is to stop
|
||||
# needing it.
|
||||
#
|
||||
# The sync client is `mc`, provisioned the way this file provisions every
|
||||
# other tool it needs on a bare runner (jq, yq, kubectl, crane): ONE curl
|
||||
# of ONE pinned static binary into ~/.local/bin, no sudo, no archive, no
|
||||
# package manager. It is also the CLI of the SAME client family the server
|
||||
# uses — hanzoai/s3-go is built on the MinIO Go client — so CI and cloud
|
||||
# talk to hanzoai/s3 through one lineage instead of two. `mirror
|
||||
# --overwrite --remove` makes the staging prefix EXACTLY the export: a file
|
||||
# deleted from the build cannot linger into a release, and unchanged
|
||||
# objects are not re-sent.
|
||||
# The bytes now travel the route console proved: the zip goes to
|
||||
# /v1/projects/<slug>/deploy and the landed prefix is promoted by
|
||||
# /v1/sites/<slug>/publish, both on the IAM bearer the KMS step above
|
||||
# ALREADY minted. CI holds no bucket key, names no bucket and names no
|
||||
# org — the org segment is prepended server-side from the validated
|
||||
# principal, which is what makes the prefix unforgeable. Nothing is left
|
||||
# for an org to be missing.
|
||||
#
|
||||
# IDEMPOTENCY IS BY CONSTRUCTION, not by a remembered request key. The
|
||||
# release id is a SHA-256 over the sorted manifest of (key, size, etag), so
|
||||
# the same bytes produce the same id; cloud finds that release row already
|
||||
# present and short-circuits — no copy, no new release, no second billable
|
||||
# deploy. Re-running this job on an unchanged export is therefore a no-op
|
||||
# that still ends green and still prints the release it left live. Nothing
|
||||
# in this step needs to remember that it ran.
|
||||
#
|
||||
# The staging prefix is ONE per site (`<org>/.ci/<slug>`), a sibling of the
|
||||
# live site prefix and of the server's own `<org>/.releases` space — a
|
||||
# project slug can never be ".ci" because slugs forbid '.', and neither a
|
||||
# deploy purge nor a project delete reaches it. Bounded, not per-commit:
|
||||
# the org's storage holds the last export, not every export ever made. Two
|
||||
# publishes racing on one site is the one thing that costs: cloud copies
|
||||
# each object conditional on the exact ETag it digested, so a source that
|
||||
# moves mid-publish is a 409 and the release is abandoned rather than
|
||||
# half-made. That fails this step loudly; the next push converges.
|
||||
# THE WORK IS IN bin/sitepublish, NOT HERE. Forty lines of zip → upload →
|
||||
# promote → verify inlined in a workflow is forty lines nobody can run
|
||||
# offline, and this file already learned that with imgver and gover.
|
||||
# bin/sitepublish_test.sh drives it through a curl shim with no network,
|
||||
# and is weighted toward the REFUSALS — a publish step that reports
|
||||
# success when it verified nothing is worse than one that never verified,
|
||||
# because it is indistinguishable from a working one until a site quietly
|
||||
# stops updating.
|
||||
if: inputs.mode != 'delegate' && github.event_name != 'pull_request'
|
||||
env:
|
||||
API: ${{ vars.API_ENDPOINT || 'https://api.hanzo.ai' }}
|
||||
@@ -1678,87 +1748,19 @@ jobs:
|
||||
if [ -z "$SLUG" ] || [ -z "$DIR" ]; then
|
||||
echo "::error::site: needs both slug: (the /v1/sites project) and dir: (the built export)"; exit 1
|
||||
fi
|
||||
# The slug is a URL path segment AND an S3 key segment here, so it is held
|
||||
# to the SAME grammar cloud validates project slugs with. Checked before
|
||||
# anything is uploaded: a bad slug is a typo to fix, not a 404 to read.
|
||||
echo "$SLUG" | grep -qE '^[a-z0-9]([a-z0-9-]{0,38}[a-z0-9])?$' \
|
||||
|| { echo "::error::site.slug '$SLUG' is not a project slug (^[a-z0-9]([a-z0-9-]{0,38}[a-z0-9])?\$)"; exit 1; }
|
||||
if [ -n "$BUILD" ]; then
|
||||
echo "::group::build $SLUG ($BUILD)"
|
||||
sh -c "$BUILD"
|
||||
echo "::endgroup::"
|
||||
fi
|
||||
# A site is a thing with an index — the one contract cloud enforces
|
||||
# however content arrives. Checked HERE, before the mirror, because
|
||||
# `--remove` on a failed build would otherwise empty the staging prefix
|
||||
# and turn a broken build into a broken publish.
|
||||
[ -f "$DIR/index.html" ] \
|
||||
|| { echo "::error::site.dir '$DIR' has no index.html at its root — /v1/sites refuses a source without one"; exit 1; }
|
||||
# Fail closed. Publishing with no credential is not a warning: it would
|
||||
# leave the site on its previous release with a green run claiming
|
||||
# otherwise. (All three come from the KMS step above — never from git.)
|
||||
if [ -z "${S3_ADMIN_ACCESS_KEY:-}" ] || [ -z "${S3_ADMIN_SECRET_KEY:-}" ] \
|
||||
|| [ -z "${HANZO_API_TOKEN:-}" ] || [ -z "${HANZO_ORG:-}" ]; then
|
||||
echo "::error::hanzo.yml declares site: but KMS supplied no publish credential (S3_ADMIN_ACCESS_KEY/SECRET_KEY + an IAM token for $API) — refusing to publish"; exit 1
|
||||
# Fail closed on the ONE thing that can be missing, and say which one.
|
||||
# An empty token here means the KMS login did not happen — not that a
|
||||
# site credential was never provisioned, which is what the old message
|
||||
# said and what sent people looking for a secret that does not exist.
|
||||
if [ -z "${HANZO_API_TOKEN:-}" ]; then
|
||||
echo "::error::hanzo.yml declares site: but the KMS step minted no IAM token for $API — refusing to publish. That is a KMS_CLIENT_ID/KMS_CLIENT_SECRET problem on this repo, not a missing site credential."; exit 1
|
||||
fi
|
||||
command -v mc >/dev/null 2>&1 || {
|
||||
mkdir -p "$HOME/.local/bin"; export PATH="$HOME/.local/bin:$PATH"
|
||||
curl -fsSL https://dl.min.io/client/mc/release/linux-amd64/archive/mc.RELEASE.2025-08-13T08-35-41Z \
|
||||
-o "$HOME/.local/bin/mc" && chmod +x "$HOME/.local/bin/mc"
|
||||
}
|
||||
# Same public endpoint + scheme the binaries lane publishes through, from
|
||||
# the same KMS names, so CI reaches hanzoai/s3 by ONE address.
|
||||
SCHEME=https
|
||||
if [ "${S3_PUBLIC_SECURE:-true}" = "false" ]; then SCHEME=http; fi
|
||||
HOST="${S3_PUBLIC_ENDPOINT:-s3.hanzo.ai}"
|
||||
BUCKET="${CLOUD_PROJECTS_BUCKET:-hanzo-sites}"
|
||||
# The credential reaches mc through MC_HOST_<alias> — never `mc alias set`,
|
||||
# whose key and secret would sit in argv for anything reading /proc, and
|
||||
# never a config file on the runner. URL-encoded with jq so a secret
|
||||
# containing '/' or '+' cannot re-parse the alias URL, and masked as a
|
||||
# whole because the composed string carries the secret verbatim.
|
||||
MC_HOST_hanzosite="$SCHEME://$(jq -rn --arg v "$S3_ADMIN_ACCESS_KEY" '$v|@uri'):$(jq -rn --arg v "$S3_ADMIN_SECRET_KEY" '$v|@uri')@$HOST"
|
||||
echo "::add-mask::$MC_HOST_hanzosite"
|
||||
export MC_HOST_hanzosite
|
||||
# `source` is RELATIVE to the org's own storage root. The org segment is
|
||||
# prepended SERVER-side from the validated principal and the bucket never
|
||||
# appears in the request at all (cloud apps/projects/release.go,
|
||||
# sourcePrefix) — an absolute path or a URL is refused — so the worst a
|
||||
# source can address is something this org already owns. CI names the
|
||||
# same prefix on the S3 side, org included, because there it IS writing a
|
||||
# key.
|
||||
SRC=".ci/$SLUG"
|
||||
echo "::group::sync $DIR → $BUCKET/$HANZO_ORG/$SRC"
|
||||
mc --no-color mirror --overwrite --remove --summary "$DIR" "hanzosite/$BUCKET/$HANZO_ORG/$SRC"
|
||||
echo "::endgroup::"
|
||||
# Promote + activate. Content types and cache policy are NOT our problem:
|
||||
# copyRelease replaces the metadata on every object it promotes, so a
|
||||
# release serves identically however its bytes were staged.
|
||||
code=$(curl -sS -o "$RUNNER_TEMP/publish.json" -w '%{http_code}' \
|
||||
-X POST "$API/v1/sites/$SLUG/publish" \
|
||||
-H "Authorization: Bearer $HANZO_API_TOKEN" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "$(jq -nc --arg s "$SRC" '{source:$s}')") \
|
||||
|| { echo "::error::POST $API/v1/sites/$SLUG/publish did not complete"; exit 1; }
|
||||
# Non-2xx is fatal and PRINTS ITS BODY. cloud answers this route with a
|
||||
# specific status per cause — 402 hosting gate, 403 wrong org, 404 no such
|
||||
# site for this tenant, 409 the source moved mid-publish, 413 past the
|
||||
# object/byte caps, 503 storage unconfigured — and every one of them is a
|
||||
# different fix, so swallowing the body would be discarding the answer.
|
||||
case "$code" in
|
||||
2??) ;;
|
||||
*) echo "::error::publish $SLUG → HTTP $code"; cat "$RUNNER_TEMP/publish.json"; echo; exit 1 ;;
|
||||
esac
|
||||
rid=$(jq -r '.releaseId // empty' "$RUNNER_TEMP/publish.json")
|
||||
url=$(jq -r '.url // empty' "$RUNNER_TEMP/publish.json")
|
||||
objs=$(jq -r '.objects // 0' "$RUNNER_TEMP/publish.json")
|
||||
[ -n "$rid" ] \
|
||||
|| { echo "::error::publish answered $code with no releaseId:"; cat "$RUNNER_TEMP/publish.json"; echo; exit 1; }
|
||||
# An empty url is a real answer, not a failure: the bare <slug> host is
|
||||
# first-come across every org, so a site whose slug is already claimed is
|
||||
# live at its own prefix with no public subdomain and is told to rename.
|
||||
echo "published $rid ($objs objects) → ${url:-no public host — that slug is claimed; rename the site or bind a domain}"
|
||||
{ echo "### Site — \`$SLUG\`"; echo
|
||||
echo "| release | objects | url |"; echo "|---|---|---|"
|
||||
echo "| \`$rid\` | $objs | ${url:-—} |"
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
# Slug grammar, index.html, the server's file/byte caps and the edge
|
||||
# BodyLimit are all checked inside sitepublish, before anything is sent,
|
||||
# so each refusal names its own number instead of arriving as a 4xx.
|
||||
HANZO_API="$API" "$CI_HOME/bin/sitepublish" "$SLUG" "$DIR"
|
||||
|
||||
@@ -23,5 +23,5 @@ concurrency:
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
gate:
|
||||
uses: hanzoai/ci/.hanzo/workflows/build.yml@v2
|
||||
uses: hanzoai/ci/.hanzo/workflows/build.yml@v1
|
||||
secrets: inherit
|
||||
|
||||
@@ -171,6 +171,43 @@ Builds are `CGO_ENABLED=0 -trimpath`: the host that installs this runs it on
|
||||
whatever base image the host is, and the digest must be a function of the
|
||||
source, not of the checkout path.
|
||||
|
||||
## `site:` — a static export, promoted to an immutable release
|
||||
|
||||
`images:` ships an OCI image a **cluster** runs; `binaries:` ships an executable
|
||||
a **host** installs. `site:` ships a static export an **edge** serves — no image,
|
||||
no CR, no replicas. Building a container so a Go binary can serve `/public` is
|
||||
the shape this retires.
|
||||
|
||||
```yaml
|
||||
site:
|
||||
slug: hanzo-console # the project on the Sites plane
|
||||
dir: out # the built export; needs index.html at its root
|
||||
build: npm ci && npm run build # optional; run first
|
||||
on: [main] # same branch gate as deploy.on; tags always publish
|
||||
```
|
||||
|
||||
That is the whole configuration. **There is no credential to provision**: the
|
||||
bearer is the IAM JWT the workflow already mints from `KMS_CLIENT_ID` /
|
||||
`KMS_CLIENT_SECRET`, so a repo that can build can publish. CI names no bucket and
|
||||
no org — the org segment is prepended server-side from the validated principal,
|
||||
which is what makes the prefix unforgeable.
|
||||
|
||||
The export is zipped and posted to `/v1/projects/<slug>/deploy`, then that prefix
|
||||
is promoted by `/v1/sites/<slug>/publish` into an **immutable release** whose id
|
||||
digests its object manifest. The site's pointer is flipped to it, and the step
|
||||
then re-reads the release list and refuses unless the release it just published
|
||||
is the one that is live. Rollback is the same pointer aimed at an older release.
|
||||
|
||||
**One size boundary, and it is the server's.** cloud's public edge caps a request
|
||||
body at 16 MiB (`GATEWAY_BODY_LIMIT`) and refuses a larger POST before any
|
||||
handler runs — reporting only `Error when parsing request`, which names neither
|
||||
size nor cause. `bin/sitepublish` therefore measures the zip and refuses *with
|
||||
the number* first. Of the 24 built exports in the estate, 22 fit; the two that do
|
||||
not (`hanzo.ai` at 27.9 MiB zipped and 8,536 files, `trillerfest.com` at 76.7
|
||||
MiB) use [`bin/sitedeploy`](bin/sitedeploy), which streams per-file against a
|
||||
presigned grant and has no body limit. `hanzo.ai` is also past the server's own
|
||||
5,000-entry cap, so no transport makes that export a release.
|
||||
|
||||
## Runners — our fleet or your own
|
||||
|
||||
By default the build runs on the **Hanzo `git-runner` fleet** on git.hanzo.ai
|
||||
|
||||
Executable
+46
@@ -0,0 +1,46 @@
|
||||
#!/usr/bin/env bash
|
||||
# publishable — refuse a `build_secrets` name that does not declare itself public.
|
||||
#
|
||||
# Usage: publishable <hanzo.yml> (reads images[].build_secrets, rc=1 on refusal)
|
||||
#
|
||||
# WHY A NAME AND NOT A VALUE. A build_secret is handed to buildx as
|
||||
# `--build-arg NAME=value`, and a build-arg is IN THE PUBLISHED IMAGE: `docker
|
||||
# history` prints it to anyone who can pull. So the only value that may sit
|
||||
# here is one that is public on purpose — a Vite/Next static export has no
|
||||
# server to read an env from, so its ingest key must be inlined at build, and
|
||||
# inlining it is what publishing it means.
|
||||
#
|
||||
# The value cannot be the test. At the moment ci reads hanzo.yml the value does
|
||||
# not exist yet (KMS has not been called), and guessing secrecy from a string's
|
||||
# shape is a heuristic that is wrong in both directions. The name is what a
|
||||
# reviewer reads, it is in git, and it is decided by the person who knows the
|
||||
# answer. So the name carries the assertion.
|
||||
#
|
||||
# The fleet had already started saying it this way — hanzoai/docs renamed
|
||||
# EVENT_INGEST_KEY -> PUBLISHABLE_KEY, and hanzoai/world's four are VITE_*,
|
||||
# which a bundler inlines by construction. This turns that convention into the
|
||||
# mechanism. Before it, the ONE repo whose value is genuinely publishable
|
||||
# (hanzoai/ui, a `pk-…` key) asserted the prefix inside its own Dockerfile, so
|
||||
# the check existed once, for one image, and any repo adding a real credential
|
||||
# got no check at all.
|
||||
set -uo pipefail
|
||||
|
||||
f=${1:-hanzo.yml}
|
||||
[ -r "$f" ] || exit 0 # no config, nothing declared, nothing to refuse
|
||||
|
||||
names=$(yq -r '[(.images // [])[] | (.build_secrets // [])[]] | unique | .[]' "$f" 2>/dev/null) || exit 0
|
||||
[ -n "$names" ] || exit 0
|
||||
|
||||
rc=0
|
||||
for n in $names; do
|
||||
case "$n" in
|
||||
# Prefixes a bundler already treats as client-side, plus an explicit
|
||||
# self-declaration for everything else. Anything outside this set has not
|
||||
# claimed to be public, so it is not baked.
|
||||
PUBLISHABLE_*|PUBLIC_*|NEXT_PUBLIC_*|EXPO_PUBLIC_*|NUXT_PUBLIC_*|VITE_*|REACT_APP_*|*_PUBLISHABLE|*_PUBLIC) ;;
|
||||
*)
|
||||
echo "::error::build_secret '$n' does not declare itself publishable, and a build_secret is baked into the image as a --build-arg where \`docker history\` reveals it. Rename it (PUBLISHABLE_*, PUBLIC_*, NEXT_PUBLIC_*, VITE_*, REACT_APP_*) if the value is public on purpose; if it is a real credential it cannot be a build_secret at all." >&2
|
||||
rc=1 ;;
|
||||
esac
|
||||
done
|
||||
exit $rc
|
||||
Executable
+65
@@ -0,0 +1,65 @@
|
||||
#!/usr/bin/env bash
|
||||
# Tests for bin/publishable. Offline and deterministic: every case is a
|
||||
# hanzo.yml written into a temp dir. Run: bash bin/publishable_test.sh
|
||||
set -uo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
PUB="$PWD/bin/publishable"
|
||||
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
|
||||
fail=0
|
||||
|
||||
# t <name> <want-rc> <build_secrets yaml-inline list>
|
||||
t() {
|
||||
local name=$1 want=$2 list=$3
|
||||
local d="$tmp/$RANDOM$RANDOM"; mkdir -p "$d"
|
||||
{ echo 'images:'; echo ' - name: app'; echo ' repo: ghcr.io/hanzoai/app'
|
||||
[ -n "$list" ] && echo " build_secrets: $list"; } > "$d/hanzo.yml"
|
||||
out=$(bash "$PUB" "$d/hanzo.yml" 2>&1); rc=$?
|
||||
if [ "$rc" = "$want" ]; then printf 'ok %-56s rc=%s\n' "$name" "$rc"
|
||||
else printf 'FAIL %-56s rc=%s (want %s)\n %s\n' "$name" "$rc" "$want" "$out"; fail=1; fi
|
||||
}
|
||||
|
||||
echo "--- refused: a name that never claimed to be public ---"
|
||||
# The live case. hanzoai/ui declares exactly this, and its value IS publishable
|
||||
# — but nothing outside its own Dockerfile could know that.
|
||||
t "EVENT_INGEST_KEY is refused" 1 '[EVENT_INGEST_KEY]'
|
||||
t "a real credential is refused" 1 '[STRIPE_SECRET_KEY]'
|
||||
t "a token is refused" 1 '[GITHUB_TOKEN]'
|
||||
t "a password is refused" 1 '[DB_PASSWORD]'
|
||||
t "a private key is refused" 1 '[SIGNING_PRIVATE_KEY]'
|
||||
t "one bad name among good ones is refused" 1 '[VITE_GTM_ID, EVENT_INGEST_KEY]'
|
||||
|
||||
echo "--- allowed: the name declares it ---"
|
||||
# These are the fleet's real declarations, verbatim.
|
||||
t "PUBLISHABLE_KEY (hanzoai/docs)" 0 '[PUBLISHABLE_KEY]'
|
||||
t "VITE_MAPBOX_TOKEN (hanzoai/world)" 0 '[VITE_MAPBOX_TOKEN]'
|
||||
t "VITE_SENTRY_DSN (hanzoai/world)" 0 '[VITE_SENTRY_DSN]'
|
||||
t "world's four together" 0 '[VITE_MAPBOX_TOKEN, VITE_SENTRY_DSN, VITE_ANALYTICS_WEBSITE_ID, VITE_GTM_ID]'
|
||||
t "NEXT_PUBLIC_ prefix" 0 '[NEXT_PUBLIC_INGEST_KEY]'
|
||||
t "REACT_APP_ prefix" 0 '[REACT_APP_MAP_KEY]'
|
||||
t "EXPO_PUBLIC_ prefix" 0 '[EXPO_PUBLIC_API_KEY]'
|
||||
t "NUXT_PUBLIC_ prefix" 0 '[NUXT_PUBLIC_API_KEY]'
|
||||
t "PUBLIC_ prefix" 0 '[PUBLIC_ANALYTICS_ID]'
|
||||
t "_PUBLISHABLE suffix" 0 '[STRIPE_PUBLISHABLE]'
|
||||
t "_PUBLIC suffix" 0 '[ANALYTICS_ID_PUBLIC]'
|
||||
|
||||
echo "--- silent: nothing declared, nothing to say ---"
|
||||
# 44 of the fleet's 47 repos are this case and must be byte-for-byte unchanged.
|
||||
t "no build_secrets key at all" 0 ''
|
||||
t "empty build_secrets list" 0 '[]'
|
||||
|
||||
echo "--- a missing file is not a refusal ---"
|
||||
out=$(bash "$PUB" "$tmp/does-not-exist.yml" 2>&1); rc=$?
|
||||
if [ "$rc" = 0 ]; then printf 'ok %-56s rc=0\n' "absent hanzo.yml is silent"
|
||||
else printf 'FAIL %-56s rc=%s\n' "absent hanzo.yml is silent" "$rc"; fail=1; fi
|
||||
|
||||
echo "--- the refusal says what to do about it ---"
|
||||
d="$tmp/msg"; mkdir -p "$d"
|
||||
printf 'images:\n - name: app\n build_secrets: [EVENT_INGEST_KEY]\n' > "$d/hanzo.yml"
|
||||
out=$(bash "$PUB" "$d/hanzo.yml" 2>&1)
|
||||
for pat in "EVENT_INGEST_KEY" "docker history" "PUBLISHABLE_" "cannot be a build_secret"; do
|
||||
if printf '%s' "$out" | grep -qF "$pat"; then printf 'ok %-56s\n' "message names '$pat'"
|
||||
else printf 'FAIL %-56s\n got: %s\n' "message names '$pat'" "$out"; fail=1; fi
|
||||
done
|
||||
|
||||
[ "$fail" = 0 ] && echo "PASS" || echo "FAIL"
|
||||
exit $fail
|
||||
Executable
+220
@@ -0,0 +1,220 @@
|
||||
#!/usr/bin/env bash
|
||||
# sitepublish — publish a built static export as an immutable Release on the
|
||||
# Hanzo Sites plane. One implementation, every `site:` in the fleet.
|
||||
#
|
||||
# sitepublish <slug> <dir> # e.g. sitepublish hanzo-console out
|
||||
#
|
||||
# WHY THIS EXISTS: the reusable's `site:` lane used to stage the export with
|
||||
# `mc mirror` against hanzoai/s3, which needed S3_ADMIN_ACCESS_KEY and
|
||||
# S3_ADMIN_SECRET_KEY. Those names are not in KMS for any org, and the lane
|
||||
# fails closed without them — so `site:` refused for every caller that ever
|
||||
# declared it, and hanzoai/console had to grow its own build→zip→publish
|
||||
# workflow to ship at all. Per-repo build logic is the one thing this repo
|
||||
# exists to prevent. The bytes now travel the SAME route console proved, and
|
||||
# the credential is the one CI already holds.
|
||||
#
|
||||
# THE TWO STEPS:
|
||||
#
|
||||
# POST /v1/projects/<slug>/deploy Content-Type: application/zip, zip as body
|
||||
# POST /v1/sites/<slug>/publish {"source":"<slug>"}
|
||||
#
|
||||
# The first lands the export at the org's site prefix; the second promotes that
|
||||
# prefix into an immutable Release and flips it live. `source` is ORG-RELATIVE
|
||||
# and the org segment is prepended server-side from the validated principal
|
||||
# (cloud apps/projects/blob.go:60, `sitePrefix(org, slug) = org + "/" + slug`),
|
||||
# which is why "<slug>" is the whole of it and why a caller cannot address
|
||||
# another tenant's bytes by writing a longer path.
|
||||
#
|
||||
# ONE CREDENTIAL, ALREADY MINTED. HANZO_API_TOKEN is the IAM JWT the workflow's
|
||||
# KMS step already holds: cloud's /v1/kms/auth/login is a broker that performs
|
||||
# the IAM client_credentials exchange and returns IAM's own JWT verbatim, so the
|
||||
# token that reads this org's secrets is the token that publishes this org's
|
||||
# site. api.hanzo.ai validates it and mints X-Org-Id from the `owner` claim —
|
||||
# identity is never a header this script sends, so there is nothing here to
|
||||
# spoof. Do NOT reintroduce S3_ADMIN_*; a standing bucket key was what this
|
||||
# replaced.
|
||||
#
|
||||
# THE SIZE BOUNDARY IS THE SERVER'S, NOT A PREFERENCE. cloud's public edge sets
|
||||
# BodyLimit from GATEWAY_BODY_LIMIT, default 16 MiB (cloud config.go:312), and
|
||||
# fasthttp refuses an oversized POST BEFORE any handler runs — it surfaces as an
|
||||
# opaque 400 "Error when parsing request" that reads like a malformed payload
|
||||
# rather than a size cap (cloud apps/projects/grant.go:5 tells that story about
|
||||
# a ~170 MB export). So the zip is measured HERE and refused HERE, with the
|
||||
# number and the alternative, instead of being sent to fail unreadably. Measured
|
||||
# across the estate's 24 built exports, 22 fit; hanzo.ai (27.9 MiB zipped, 8536
|
||||
# files) and trillerfest.com (76.7 MiB) do not. Those go to bin/sitedeploy,
|
||||
# which streams per-file against a presigned grant and has no body limit —
|
||||
# hanzo.ai is over the server's own 5000-entry cap anyway, so no transport makes
|
||||
# it a Release.
|
||||
#
|
||||
# ENV: HANZO_API_TOKEN (required) the IAM bearer; the workflow's KMS step mints it
|
||||
# HANZO_API (https://api.hanzo.ai)
|
||||
# SITEPUBLISH_MAX_ZIP (16777216) mirror of the server's GATEWAY_BODY_LIMIT
|
||||
# SITEPUBLISH_PLAN=1 print what it would send and exit; no network. Test seam.
|
||||
set -euo pipefail
|
||||
|
||||
slug="${1:?usage: sitepublish <slug> <dir>}"
|
||||
dir="${2:?usage: sitepublish <slug> <dir>}"
|
||||
api="${HANZO_API:-https://api.hanzo.ai}"
|
||||
|
||||
# Server-side caps, mirrored so a breach is named in CI instead of arriving as a
|
||||
# 413 (or, for the body limit, as an unreadable 400). cloud apps/projects/blob.go
|
||||
# :29-31. bin/sitepublish_test.sh pins these numbers; if cloud moves them the
|
||||
# test is what catches the drift.
|
||||
max_zip="${SITEPUBLISH_MAX_ZIP:-16777216}" # gateway BodyLimit, 16 MiB
|
||||
max_files=5000 # maxFiles
|
||||
max_file_bytes=$((64 << 20)) # maxFileBytes, 64 MiB
|
||||
max_total_bytes=$((512 << 20)) # maxTotalBytes uncompressed, 512 MiB
|
||||
|
||||
die() { echo "::error::$*"; exit 1; }
|
||||
|
||||
# The slug is a URL path segment AND an S3 key segment, so it is held to the
|
||||
# grammar cloud validates project slugs with — checked before anything is built
|
||||
# or sent, because a bad slug is a typo to fix and not a 404 to interpret.
|
||||
echo "$slug" | grep -qE '^[a-z0-9]([a-z0-9-]{0,38}[a-z0-9])?$' \
|
||||
|| die "slug '$slug' is not a project slug (^[a-z0-9]([a-z0-9-]{0,38}[a-z0-9])?\$)"
|
||||
[ -d "$dir" ] || die "'$dir' is not a directory — the build produced no export"
|
||||
|
||||
# A site is a thing with an index. cloud enforces it however the bytes arrive;
|
||||
# checking here turns a failed build into a failed publish with a readable cause
|
||||
# rather than a promoted release that 404s at its own root.
|
||||
[ -f "$dir/index.html" ] || die "'$dir' has no index.html at its root — /v1/sites refuses a source without one"
|
||||
|
||||
# CNAME does not travel: a GitHub Pages artifact that means nothing to S3 and
|
||||
# would ship a stale hostname claim into the bucket. Same exclusion bin/sitedeploy
|
||||
# makes, for the same reason.
|
||||
files=$(cd "$dir" && find . -type f ! -name CNAME | wc -l)
|
||||
[ "$files" -gt 0 ] || die "'$dir' contains no files — refusing to publish an empty release"
|
||||
[ "$files" -le "$max_files" ] \
|
||||
|| die "'$dir' holds $files files; cloud caps one artifact at $max_files (apps/projects/blob.go maxFiles). Use bin/sitedeploy — but note the Releases plane has the same cap, so this export cannot become a Release."
|
||||
|
||||
bytes=$(cd "$dir" && find . -type f ! -name CNAME -exec wc -c {} + | awk '$2!="total"{s+=$1} END{print s+0}')
|
||||
[ "$bytes" -le "$max_total_bytes" ] \
|
||||
|| die "'$dir' is $bytes bytes uncompressed; cloud caps an artifact at $max_total_bytes (maxTotalBytes)"
|
||||
|
||||
# Per-file cap, checked with find rather than a loop so a 9000-file export costs
|
||||
# one traversal. -size uses 512-byte blocks with `c` for bytes; +N c is "strictly
|
||||
# greater than N bytes", which is the cap's own boundary.
|
||||
big=$(cd "$dir" && find . -type f ! -name CNAME -size +${max_file_bytes}c -printf '%P (%s bytes)\n' | head -3)
|
||||
[ -z "$big" ] || die "these files exceed cloud's ${max_file_bytes}-byte per-file cap (maxFileBytes):
|
||||
$big"
|
||||
|
||||
if [ -n "${SITEPUBLISH_PLAN:-}" ]; then
|
||||
printf 'slug=%s dir=%s files=%s bytes=%s api=%s source=%s\n' \
|
||||
"$slug" "$dir" "$files" "$bytes" "$api" "$slug"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# No apostrophe in this message: inside "${VAR:?word}" a single quote opens a
|
||||
# quoted section for the PARSER, and the script dies at EOF with "unexpected EOF
|
||||
# while looking for matching quote" — a syntax error reported at the last line,
|
||||
# nowhere near the one that caused it.
|
||||
: "${HANZO_API_TOKEN:?HANZO_API_TOKEN is unset. It is the IAM bearer minted by the KMS step of the reusable workflow, from KMS_CLIENT_ID/KMS_CLIENT_SECRET; there is no second credential to seal.}"
|
||||
|
||||
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
|
||||
zipf="$tmp/site.zip"
|
||||
|
||||
# -X drops extra file attributes (uid/gid/timestamps beyond the DOS fields) so
|
||||
# the same tree zips to the same bytes on any runner. Entries are stored
|
||||
# RELATIVE to the export root — cloud strips a leading "./" and refuses absolute
|
||||
# or ".."-escaping names (blob.go:362-370), and strips a common top-level prefix
|
||||
# if one exists, so a flat archive is what it expects and what this sends.
|
||||
( cd "$dir" && zip -qXr "$zipf" . -x './CNAME' ) || die "zip of '$dir' failed"
|
||||
zbytes=$(wc -c < "$zipf")
|
||||
|
||||
# The refusal the whole size story is about. Above this the POST dies at the
|
||||
# edge with a 400 that names nothing.
|
||||
[ "$zbytes" -le "$max_zip" ] || die "the zipped export is $zbytes bytes, past cloud's ${max_zip}-byte edge BodyLimit (GATEWAY_BODY_LIMIT, cloud config.go). fasthttp refuses the POST before any handler runs and reports only 'Error when parsing request', so this is refused here where the number is visible. Publish this export with bin/sitedeploy, which streams per-file against a presigned grant and has no body limit."
|
||||
|
||||
echo "publishing $slug — $files files, $bytes bytes ($zbytes zipped)"
|
||||
|
||||
# ---- 1. the bytes -----------------------------------------------------------
|
||||
# Every response body is KEPT and printed on failure. cloud answers these routes
|
||||
# with a specific status per cause — 402 hosting gate, 403 wrong org, 404 no such
|
||||
# site for this tenant, 409 the source moved mid-publish, 413 past the caps, 503
|
||||
# storage unconfigured — and each is a different fix, so discarding the body
|
||||
# discards the answer.
|
||||
code=$(curl -sS -o "$tmp/deploy.json" -w '%{http_code}' \
|
||||
-X POST "$api/v1/projects/$slug/deploy" \
|
||||
-H "Authorization: Bearer $HANZO_API_TOKEN" \
|
||||
-H 'Content-Type: application/zip' \
|
||||
--data-binary "@$zipf") \
|
||||
|| die "POST $api/v1/projects/$slug/deploy did not complete"
|
||||
case "$code" in
|
||||
2??) ;;
|
||||
*) echo "::error::upload $slug → HTTP $code"; head -c 800 "$tmp/deploy.json"; echo; exit 1 ;;
|
||||
esac
|
||||
|
||||
# ---- 2. promote + activate --------------------------------------------------
|
||||
# `source` is the org-relative prefix the upload just landed at. Content types
|
||||
# and cache policy are not ours to set: copyRelease rewrites the metadata on
|
||||
# every object it promotes, so a release serves identically however its bytes
|
||||
# were staged.
|
||||
code=$(curl -sS -o "$tmp/publish.json" -w '%{http_code}' \
|
||||
-X POST "$api/v1/sites/$slug/publish" \
|
||||
-H "Authorization: Bearer $HANZO_API_TOKEN" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "$(jq -nc --arg s "$slug" '{source:$s}')") \
|
||||
|| die "POST $api/v1/sites/$slug/publish did not complete"
|
||||
case "$code" in
|
||||
2??) ;;
|
||||
*) echo "::error::publish $slug → HTTP $code"; head -c 800 "$tmp/publish.json"; echo; exit 1 ;;
|
||||
esac
|
||||
|
||||
# Guard the SHAPE before reading a field out of it. `jq -r '.releaseId // empty'`
|
||||
# on an error body, an array, or a null yields "" and every later test on it
|
||||
# passes vacuously — which is the defect class this whole script is written
|
||||
# against. Assert it is an object carrying the field, then read it.
|
||||
jq -e 'type == "object" and (.releaseId | type) == "string" and (.releaseId | length) > 0' \
|
||||
"$tmp/publish.json" >/dev/null \
|
||||
|| { echo "::error::publish answered $code but the body is not a release object with a releaseId:"; head -c 800 "$tmp/publish.json"; echo; exit 1; }
|
||||
rid=$(jq -r '.releaseId' "$tmp/publish.json")
|
||||
objs=$(jq -r '.objects // 0' "$tmp/publish.json")
|
||||
url=$(jq -r '.url // ""' "$tmp/publish.json")
|
||||
|
||||
# ---- 3. verify the release we just made is the one that is live -------------
|
||||
# A 200 from publish says the request was accepted, not that this release is
|
||||
# serving. The list is the only thing that can say so.
|
||||
#
|
||||
# THE LIST IS A BARE JSON ARRAY — `type projectsReleases []projectsRelease`
|
||||
# (cloud apps/projects/release.go:518). `.releases[]` against it resolves
|
||||
# NOTHING, and an assertion that resolves nothing is an assertion that cannot
|
||||
# fail for the reason it was written. Hence `.[]`, and hence the shape guard
|
||||
# first: if cloud ever wraps this in an object, THAT is what goes red, loudly,
|
||||
# instead of the check quietly matching zero releases forever.
|
||||
code=$(curl -sS -o "$tmp/releases.json" -w '%{http_code}' \
|
||||
"$api/v1/sites/$slug/releases" \
|
||||
-H "Authorization: Bearer $HANZO_API_TOKEN") \
|
||||
|| die "GET $api/v1/sites/$slug/releases did not complete"
|
||||
case "$code" in
|
||||
2??) ;;
|
||||
*) echo "::error::list releases for $slug → HTTP $code"; head -c 800 "$tmp/releases.json"; echo; exit 1 ;;
|
||||
esac
|
||||
jq -e 'type == "array" and length > 0' "$tmp/releases.json" >/dev/null \
|
||||
|| { echo "::error::GET /v1/sites/$slug/releases did not answer a non-empty JSON array (it is the bare-array shape from release.go:518). Body:"; head -c 800 "$tmp/releases.json"; echo; exit 1; }
|
||||
|
||||
# Exactly one release is active. Counting first is what makes "none active"
|
||||
# distinguishable from "the wrong one is active" — a bare grep for '"active":true'
|
||||
# matches ANY release in the list and would pass on both.
|
||||
nactive=$(jq '[.[] | select(.active == true)] | length' "$tmp/releases.json")
|
||||
[ "$nactive" = 1 ] \
|
||||
|| { echo "::error::expected exactly 1 active release for $slug, found $nactive:"; jq -c '[.[]|{releaseId,active}]' "$tmp/releases.json"; exit 1; }
|
||||
|
||||
active=$(jq -r 'map(select(.active == true))[0].releaseId // ""' "$tmp/releases.json")
|
||||
# Both sides proven non-empty BEFORE they are compared. `[ "$a" = "$b" ]` with
|
||||
# two empty strings is TRUE, so an equality test on unguarded values reports
|
||||
# success precisely when it learned nothing.
|
||||
[ -n "$active" ] || { echo "::error::the active release for $slug carries no releaseId:"; jq -c '.' "$tmp/releases.json"; exit 1; }
|
||||
[ -n "$rid" ] || die "publish returned no releaseId to verify against"
|
||||
[ "$active" = "$rid" ] \
|
||||
|| { echo "::error::published $rid but $active is live for $slug — the flip did not take"; exit 1; }
|
||||
|
||||
echo "live: $rid ($objs objects) → ${url:-no public host — that slug is claimed; rename the site or bind a domain}"
|
||||
|
||||
if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
||||
{ echo "### Site — \`$slug\`"; echo
|
||||
echo "| release | objects | files | zipped | url |"
|
||||
echo "|---|---|---|---|---|"
|
||||
echo "| \`$rid\` | $objs | $files | $zbytes B | ${url:-—} |"
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
fi
|
||||
Executable
+168
@@ -0,0 +1,168 @@
|
||||
#!/usr/bin/env bash
|
||||
# Tests for bin/sitepublish. Runs OFFLINE and deterministically: a `curl` shim on
|
||||
# PATH answers the three routes from fixtures, so every case here — including the
|
||||
# ones that must FAIL — needs no token, no bucket and no cluster.
|
||||
# Run: bash bin/sitepublish_test.sh
|
||||
#
|
||||
# The suite is weighted toward REFUSALS on purpose. A publish step that reports
|
||||
# success when it verified nothing is worse than one that does not verify at all,
|
||||
# because it is indistinguishable from a working one until a site silently stops
|
||||
# updating. Three real defects of that exact shape are pinned below by name:
|
||||
#
|
||||
# • an object-shaped read (`.releaseId`) against a body that is not an object
|
||||
# • `.releases[]` against the BARE ARRAY /v1/sites/<slug>/releases returns,
|
||||
# which resolves nothing and therefore can never fail for its stated reason
|
||||
# • `[ "$a" = "$b" ]` on two values that are both empty, which is TRUE
|
||||
set -uo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
SP="$PWD/bin/sitepublish"
|
||||
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
|
||||
fail=0
|
||||
|
||||
t() { # t <name> <want_rc> <got_rc> [<must-contain> <output>]
|
||||
local name="$1" want="$2" got="$3" needle="${4:-}" out="${5:-}"
|
||||
if [ "$got" != "$want" ]; then
|
||||
printf 'FAIL %-58s rc=%s (want %s)\n' "$name" "$got" "$want"; fail=1; return
|
||||
fi
|
||||
if [ -n "$needle" ] && ! printf '%s' "$out" | grep -qF -- "$needle"; then
|
||||
printf 'FAIL %-58s rc=%s but missing %q\n' "$name" "$got" "$needle"; fail=1
|
||||
printf ' got: %s\n' "$(printf '%s' "$out" | head -c 300)"; return
|
||||
fi
|
||||
printf 'ok %-58s rc=%s\n' "$name" "$got"
|
||||
}
|
||||
|
||||
# ---- a curl shim -------------------------------------------------------------
|
||||
# Dispatches on the URL and writes the fixture the scenario names to the file the
|
||||
# real curl would have written, then prints the status the way `-w %{http_code}`
|
||||
# does. Everything the script does with a response goes through this, so the
|
||||
# tests exercise the REAL parsing, not a mock of it.
|
||||
shim="$tmp/bin"; mkdir -p "$shim"
|
||||
cat > "$shim/curl" <<'SHIM'
|
||||
#!/usr/bin/env bash
|
||||
out=/dev/null url=
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
-o) out="$2"; shift 2 ;;
|
||||
http*|https*) url="$1"; shift ;;
|
||||
*) shift ;;
|
||||
esac
|
||||
done
|
||||
case "$url" in
|
||||
*/deploy) printf '%s' "${T_DEPLOY_BODY:-{\"ok\":true\}}" > "$out"; printf '%s' "${T_DEPLOY_CODE:-200}" ;;
|
||||
*/publish) printf '%s' "${T_PUBLISH_BODY:-}" > "$out"; printf '%s' "${T_PUBLISH_CODE:-200}" ;;
|
||||
*/releases) printf '%s' "${T_LIST_BODY:-}" > "$out"; printf '%s' "${T_LIST_CODE:-200}" ;;
|
||||
*) echo "shim: unexpected url $url" >&2; exit 9 ;;
|
||||
esac
|
||||
SHIM
|
||||
chmod +x "$shim/curl"
|
||||
|
||||
site="$tmp/site"; mkdir -p "$site/assets"
|
||||
echo '<h1>hi</h1>' > "$site/index.html"
|
||||
echo 'body{}' > "$site/assets/app.css"
|
||||
echo 'hanzo.ai' > "$site/CNAME"
|
||||
|
||||
run() { # run <dir> — publish with the shim on PATH, current T_* scenario
|
||||
PATH="$shim:$PATH" HANZO_API_TOKEN=tok HANZO_API=https://api.test \
|
||||
bash "$SP" a-slug "$1" 2>&1
|
||||
}
|
||||
OK_PUB='{"releaseId":"rel-1","slug":"a-slug","objects":2,"active":true,"url":"https://a-slug.hanzo.page"}'
|
||||
OK_LIST='[{"releaseId":"rel-1","active":true},{"releaseId":"rel-0","active":false}]'
|
||||
|
||||
# ---- the plan seam: no network, no token ------------------------------------
|
||||
out=$(SITEPUBLISH_PLAN=1 bash "$SP" a-slug "$site" 2>&1); rc=$?
|
||||
t "plan prints the manifest and exits" 0 $rc "files=2" "$out"
|
||||
# CNAME is a GitHub Pages artifact: it means nothing to S3 and would ship a stale
|
||||
# hostname claim. Two files, not three.
|
||||
t "plan excludes CNAME from the count" 0 $rc "files=2 " "$out"
|
||||
t "plan sends source=<slug>, org-relative" 0 $rc "source=a-slug" "$out"
|
||||
|
||||
# ---- refusals that need no network ------------------------------------------
|
||||
out=$(SITEPUBLISH_PLAN=1 bash "$SP" 'Bad_Slug' "$site" 2>&1); rc=$?
|
||||
t "an invalid slug is refused before anything" 1 $rc "is not a project slug" "$out"
|
||||
out=$(SITEPUBLISH_PLAN=1 bash "$SP" a-slug "$tmp/nope" 2>&1); rc=$?
|
||||
t "a missing export dir is refused" 1 $rc "is not a directory" "$out"
|
||||
noidx="$tmp/noidx"; mkdir -p "$noidx"; echo x > "$noidx/page.html"
|
||||
out=$(SITEPUBLISH_PLAN=1 bash "$SP" a-slug "$noidx" 2>&1); rc=$?
|
||||
t "an export with no index.html is refused" 1 $rc "no index.html at its root" "$out"
|
||||
empty="$tmp/empty"; mkdir -p "$empty"; echo x > "$empty/index.html"; rm "$empty/index.html"
|
||||
out=$(SITEPUBLISH_PLAN=1 bash "$SP" a-slug "$empty" 2>&1); rc=$?
|
||||
t "an empty export is refused" 1 $rc "no index.html" "$out"
|
||||
|
||||
# The size refusal is the whole reason this script measures before it sends:
|
||||
# past the edge BodyLimit fasthttp rejects the POST before any handler runs and
|
||||
# reports only "Error when parsing request", which names neither size nor cause.
|
||||
out=$(SITEPUBLISH_MAX_ZIP=1 PATH="$shim:$PATH" HANZO_API_TOKEN=tok \
|
||||
bash "$SP" a-slug "$site" 2>&1); rc=$?
|
||||
t "a zip past the edge BodyLimit is refused here" 1 $rc "BodyLimit" "$out"
|
||||
t " ...and the refusal names bin/sitedeploy" 1 $rc "bin/sitedeploy" "$out"
|
||||
|
||||
# The server caps one artifact at 5000 entries; a 5001-file export can never
|
||||
# become a Release by ANY transport, so saying so here beats a 413 later.
|
||||
many="$tmp/many"; mkdir -p "$many"; echo x > "$many/index.html"
|
||||
( cd "$many" && touch f{1..5001} )
|
||||
out=$(SITEPUBLISH_PLAN=1 bash "$SP" a-slug "$many" 2>&1); rc=$?
|
||||
t "an export past maxFiles=5000 is refused" 1 $rc "cloud caps one artifact at 5000" "$out"
|
||||
|
||||
# ---- the happy path ----------------------------------------------------------
|
||||
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY="$OK_LIST" run "$site"); rc=$?
|
||||
t "publish + verified flip succeeds" 0 $rc "live: rel-1" "$out"
|
||||
|
||||
# ---- transport failures carry the server's answer ---------------------------
|
||||
# Each status is a different fix, so the body is printed rather than swallowed.
|
||||
out=$(T_DEPLOY_CODE=413 T_DEPLOY_BODY='{"error":"artifact exceeds"}' run "$site"); rc=$?
|
||||
t "a non-2xx upload fails and prints the body" 1 $rc "artifact exceeds" "$out"
|
||||
out=$(T_PUBLISH_CODE=402 T_PUBLISH_BODY='{"error":"hosting not enabled"}' run "$site"); rc=$?
|
||||
t "a non-2xx publish fails and prints the body" 1 $rc "hosting not enabled" "$out"
|
||||
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_CODE=503 T_LIST_BODY='{"error":"storage"}' run "$site"); rc=$?
|
||||
t "a non-2xx release list fails" 1 $rc "list releases" "$out"
|
||||
|
||||
# ---- defect 1: an object-shaped read against a non-object -------------------
|
||||
# `jq -r '.releaseId // empty'` yields "" for an array, a null, or an error body,
|
||||
# and every later test on "" passes vacuously. The shape is asserted first.
|
||||
out=$(T_PUBLISH_BODY='[{"releaseId":"rel-1"}]' T_LIST_BODY="$OK_LIST" run "$site"); rc=$?
|
||||
t "publish answering an ARRAY is refused" 1 $rc "not a release object" "$out"
|
||||
out=$(T_PUBLISH_BODY='{"ok":true}' T_LIST_BODY="$OK_LIST" run "$site"); rc=$?
|
||||
t "publish answering no releaseId is refused" 1 $rc "not a release object" "$out"
|
||||
out=$(T_PUBLISH_BODY='{"releaseId":""}' T_LIST_BODY="$OK_LIST" run "$site"); rc=$?
|
||||
t "publish answering an EMPTY releaseId is refused" 1 $rc "not a release object" "$out"
|
||||
out=$(T_PUBLISH_BODY='not json' T_LIST_BODY="$OK_LIST" run "$site"); rc=$?
|
||||
t "publish answering non-JSON is refused" 1 $rc "not a release object" "$out"
|
||||
|
||||
# ---- defect 2: the list is a BARE ARRAY -------------------------------------
|
||||
# release.go:518 is `type projectsReleases []projectsRelease`. A `.releases[]`
|
||||
# filter resolves NOTHING against that, so an assertion built on it can only ever
|
||||
# refuse — it never once tested what it claimed to. The guard demands the array
|
||||
# shape, so the day cloud wraps the list THAT is what goes red, by name.
|
||||
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='{"releases":[{"releaseId":"rel-1","active":true}]}' run "$site"); rc=$?
|
||||
t "a WRAPPED release list is refused, loudly" 1 $rc "bare-array shape" "$out"
|
||||
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='[]' run "$site"); rc=$?
|
||||
t "an EMPTY release list is refused" 1 $rc "non-empty JSON array" "$out"
|
||||
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='null' run "$site"); rc=$?
|
||||
t "a null release list is refused" 1 $rc "non-empty JSON array" "$out"
|
||||
|
||||
# ---- defect 3: counting, so "none" cannot read as "yes" ---------------------
|
||||
# A bare `grep '"active":true'` matches ANY release in the list, so it passes
|
||||
# both when the wrong release is live and when the list merely mentions one.
|
||||
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='[{"releaseId":"rel-1","active":false}]' run "$site"); rc=$?
|
||||
t "ZERO active releases is refused" 1 $rc "expected exactly 1 active release" "$out"
|
||||
out=$(T_PUBLISH_BODY="$OK_PUB" \
|
||||
T_LIST_BODY='[{"releaseId":"rel-1","active":true},{"releaseId":"rel-2","active":true}]' run "$site"); rc=$?
|
||||
t "TWO active releases is refused" 1 $rc "expected exactly 1 active release" "$out"
|
||||
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='[{"releaseId":"rel-9","active":true}]' run "$site"); rc=$?
|
||||
t "a DIFFERENT release being live is refused" 1 $rc "the flip did not take" "$out"
|
||||
|
||||
# The both-empty comparison, head on: an active entry whose releaseId is "".
|
||||
# `[ "$active" = "$rid" ]` with both empty is TRUE and would report success on a
|
||||
# release that does not exist. Non-emptiness is proven before the comparison.
|
||||
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='[{"releaseId":"","active":true}]' run "$site"); rc=$?
|
||||
t "an active release with an EMPTY id is refused" 1 $rc "carries no releaseId" "$out"
|
||||
out=$(T_PUBLISH_BODY="$OK_PUB" T_LIST_BODY='[{"active":true}]' run "$site"); rc=$?
|
||||
t "an active release with NO id field is refused" 1 $rc "carries no releaseId" "$out"
|
||||
|
||||
# ---- the credential is required ---------------------------------------------
|
||||
out=$(PATH="$shim:$PATH" HANZO_API_TOKEN= bash "$SP" a-slug "$site" 2>&1); rc=$?
|
||||
t "a missing bearer refuses before any request" 1 $rc "HANZO_API_TOKEN is unset" "$out"
|
||||
|
||||
echo
|
||||
[ "$fail" = 0 ] && echo "sitepublish: all tests passed" || echo "sitepublish: FAILURES"
|
||||
exit "$fail"
|
||||
Executable
+161
@@ -0,0 +1,161 @@
|
||||
#!/usr/bin/env bash
|
||||
# vendormark — refuse a repo that carries a site-generator vendor's branding.
|
||||
# One implementation, every caller.
|
||||
#
|
||||
# vendormark [dir] [git-range]
|
||||
#
|
||||
# WHAT THIS CATCHES
|
||||
#
|
||||
# Several sites in the estate began life as output from a hosted "prompt to
|
||||
# React app" generator. The generator does not sign its work in a comment you
|
||||
# would notice — it signs it in the SHAPE of the repo, in four places at once:
|
||||
#
|
||||
# 1. an asset folder named after the vendor, e.g. public/<vendor>-uploads/,
|
||||
# so every image the browser requests carries the vendor's name in its URL
|
||||
# 2. a build plugin dependency in package.json and the lockfiles
|
||||
# 3. an injected third-party <script> tag in index.html
|
||||
# 4. commit messages, author identities and Co-authored-by trailers written
|
||||
# by the vendor's bot
|
||||
#
|
||||
# The fourth is the one nothing else looks at, and the one that survives every
|
||||
# amount of tidying the working tree: a repo can render a perfect page and
|
||||
# still say, in `git log`, who really wrote it.
|
||||
#
|
||||
# This was not a hypothetical. A sweep of 1,065 repositories across the three
|
||||
# orgs found the mark in five shipped sites — one of them serving the vendor's
|
||||
# upload folder as its FAVICON, so the vendor's path was in the <head> of every
|
||||
# page and in every browser tab. Purging it needed a history rewrite and a
|
||||
# force-push of every branch. This gate exists so that is never needed twice.
|
||||
#
|
||||
# WHY THE CONTENT RULES ARE SO SPECIFIC
|
||||
#
|
||||
# The vendor's name is also an ordinary English adjective, and the naive gate —
|
||||
# case-insensitive substring of that word — is unshippable. Measured across all
|
||||
# 1,065 repos, the bare word appears legitimately in 20+ of them and NONE of
|
||||
# them are contaminated:
|
||||
#
|
||||
# tokenizer vocab.json (3 repos) "lovable</w>": 38565
|
||||
# English word lists words_alpha.txt, faker adjectives
|
||||
# ML training/eval corpora alpaca, kto, BAGEL eval prompts
|
||||
# x/net publicsuffix packed data a substring of concatenated domains
|
||||
# vendored prompt corpora third-party collections that quote it
|
||||
# our own marketing copy a competitor named in a case study
|
||||
#
|
||||
# A gate that reds a fifth of the estate on day one is a gate someone switches
|
||||
# off, and then we are worse off than before it existed. So the content rules
|
||||
# are only the strings that CANNOT occur in prose: a path segment, an npm
|
||||
# package name, an injected script URL, a generator meta tag, a bot's email.
|
||||
#
|
||||
# Two candidate rules were DROPPED after measuring them, and it is worth saying
|
||||
# why: `lovableproject.com` and `lovable.app` red an upstream Clerk SDK bundle,
|
||||
# which ships a list of dev-preview host suffixes naming several generators. A
|
||||
# third party's honest list of other people's hosts is not our contamination.
|
||||
# Those two strings are still refused in commit messages and git identities,
|
||||
# where no third party can put them.
|
||||
#
|
||||
# PATHS AND IDENTITIES ARE ABSOLUTE
|
||||
#
|
||||
# Content is judged narrowly; paths and git identities are not. There is no
|
||||
# legitimate reason for the vendor's name to appear in a filename we track or
|
||||
# in an author, committer or Co-authored-by line we write. Those are refused on
|
||||
# the bare word, no exceptions, because that is where the mark is load-bearing
|
||||
# and where tidying the working tree never reaches.
|
||||
#
|
||||
# THE ESCAPE HATCH
|
||||
#
|
||||
# `.vendormark-allow` — one path glob per line, `#` comments ignored. It exists
|
||||
# for exactly one honest case: a repo whose JOB is to name these vendors, such
|
||||
# as the history-scrubber rule files in hanzoai/.github, which cannot do their
|
||||
# work without spelling the string they remove. If you are reaching for it for
|
||||
# any other reason, you are about to ship the mark.
|
||||
#
|
||||
# EXIT: 0 clean, 1 mark found.
|
||||
set -uo pipefail
|
||||
|
||||
root=${1:-.}
|
||||
range=${2:-}
|
||||
cd "$root" 2>/dev/null || { echo "vendormark: no such directory: $root" >&2; exit 1; }
|
||||
git rev-parse --git-dir >/dev/null 2>&1 || exit 0 # not a repo: nothing to check
|
||||
|
||||
# PATHS: the bare vendor name, no exceptions. Add a vendor here and its
|
||||
# filenames are refused everywhere.
|
||||
VENDORS='lovable'
|
||||
|
||||
# CONTENT: only strings that cannot occur in prose — an asset folder, an npm
|
||||
# package, the injected script, a generator meta tag, the bot's address. Spelled
|
||||
# out in full rather than composed from $VENDORS, because half of them do not
|
||||
# contain the vendor's name at all. See the header for the two that were
|
||||
# measured against the estate and dropped.
|
||||
CONTENT_SIG='lovable-uploads/|lovable-tagger|@lovable\.dev|cdn\.gpteng\.co|gptengineer\.js|content="[Ll]ovable"'
|
||||
|
||||
# IDENTITIES AND MESSAGES: the bare word plus the vendor hosts, because nothing
|
||||
# outside our own commits can write these.
|
||||
IDENT_SIG='lovable|gptengineer|gpt-engineer'
|
||||
|
||||
fail=0
|
||||
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
|
||||
|
||||
# --- allowlist ------------------------------------------------------------
|
||||
: > "$tmp/allow"
|
||||
if [ -f .vendormark-allow ]; then
|
||||
grep -vE '^\s*(#|$)' .vendormark-allow > "$tmp/allow" || true
|
||||
fi
|
||||
allowed() { # $1 = path
|
||||
[ -s "$tmp/allow" ] || return 1
|
||||
while IFS= read -r g; do
|
||||
# shellcheck disable=SC2254
|
||||
case "$1" in $g) return 0;; esac
|
||||
done < "$tmp/allow"
|
||||
return 1
|
||||
}
|
||||
|
||||
# --- 1. tracked paths -----------------------------------------------------
|
||||
git ls-files 2>/dev/null | grep -iE "$VENDORS" > "$tmp/paths" || true
|
||||
if [ -s "$tmp/paths" ]; then
|
||||
n=0
|
||||
while IFS= read -r p; do allowed "$p" || { echo " path: $p"; n=$((n+1)); }; done < "$tmp/paths"
|
||||
if [ "$n" -gt 0 ]; then
|
||||
echo "::error::${n} tracked path(s) carry a site-generator vendor's name. Every one of them is a URL the browser requests, so the vendor's branding is in the page. Move the assets to a neutral folder (public/img/) and repoint the references in the same change — a favicon or logo that a <link> or <img> depends on must be REPLACED with the real brand mark, not merely deleted."
|
||||
fail=1
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- 2. tracked content ---------------------------------------------------
|
||||
git grep -Iin -E "$CONTENT_SIG" HEAD 2>/dev/null | sed 's#^HEAD:##' > "$tmp/content" || true
|
||||
if [ -s "$tmp/content" ]; then
|
||||
n=0
|
||||
while IFS= read -r line; do
|
||||
p=${line%%:*}
|
||||
allowed "$p" || { echo " content: $(echo "$line" | cut -c1-140)"; n=$((n+1)); }
|
||||
done < "$tmp/content"
|
||||
if [ "$n" -gt 0 ]; then
|
||||
echo "::error::${n} tracked line(s) reference a site-generator vendor's asset folder, build plugin or injected script. Drop the dependency and the script tag; repoint the asset URLs."
|
||||
fail=1
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- 3. commit messages, identities, trailers -----------------------------
|
||||
# Default scope is HEAD alone, which is what a fetch-depth:1 CI checkout has and
|
||||
# is enough to refuse the commit being pushed. Pass a range (or set
|
||||
# VENDORMARK_RANGE) to judge more, e.g. origin/main..HEAD for a whole PR.
|
||||
range=${range:-${VENDORMARK_RANGE:-}}
|
||||
if [ -n "$range" ]; then set -- "$range"; else set -- -1 HEAD; fi
|
||||
git log "$@" --format='%H%x01%an <%ae>%x01%cn <%ce>%x01%B%x01%(trailers:unfold=true)%x02' 2>/dev/null \
|
||||
| tr -d '\n' | tr '\002' '\n' > "$tmp/log" || true
|
||||
if [ -s "$tmp/log" ]; then
|
||||
n=0
|
||||
while IFS= read -r c; do
|
||||
echo "$c" | grep -qiE "$IDENT_SIG" || continue
|
||||
echo " commit: $(echo "$c" | tr '\001' ' ' | cut -c1-150)"
|
||||
n=$((n+1))
|
||||
done < "$tmp/log"
|
||||
if [ "$n" -gt 0 ]; then
|
||||
echo "::error::${n} commit(s) name a site-generator vendor in the message, the author/committer identity or a Co-authored-by trailer. A working tree can be tidied; \`git log\` cannot be, short of a history rewrite and a force-push of every branch. Reword before pushing. Describe the work — \"generated placeholder assets\" — and never credit the generator."
|
||||
fail=1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$fail" -eq 0 ]; then
|
||||
echo "vendormark: OK — no site-generator branding in paths, tracked content, messages or identities"
|
||||
fi
|
||||
exit "$fail"
|
||||
Executable
+142
@@ -0,0 +1,142 @@
|
||||
#!/usr/bin/env bash
|
||||
# vendormark_test.sh — the refusals AND the allowances, both pinned.
|
||||
#
|
||||
# A gate is only worth having if it is exact in both directions. A false
|
||||
# negative lets the vendor's mark back into a shipped site; a false positive
|
||||
# reds a repo that was always fine, and a gate that reds honest repos is a gate
|
||||
# someone switches off. So this suite asserts BOTH halves, and the allowance
|
||||
# half is the larger one on purpose: the vendor's name is also an ordinary
|
||||
# English adjective, and it occurs innocently in tokenizer vocabularies, word
|
||||
# lists, ML corpora, packed public-suffix data and third-party host lists all
|
||||
# over the estate.
|
||||
#
|
||||
# Offline and deterministic: temp repos, no network.
|
||||
set -uo pipefail
|
||||
BIN=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/vendormark
|
||||
pass=0; fail=0
|
||||
V=$(printf 'lo%s' 'vable') # not spelled literally, so this file does not
|
||||
# itself trip the gate it is testing
|
||||
|
||||
newrepo() {
|
||||
d=$(mktemp -d)
|
||||
git -C "$d" init -q
|
||||
git -C "$d" config user.email dev@hanzo.ai
|
||||
git -C "$d" config user.name "Hanzo Dev"
|
||||
printf 'x\n' > "$d/README.md"
|
||||
git -C "$d" add -A
|
||||
git -C "$d" commit -qm "initial"
|
||||
echo "$d"
|
||||
}
|
||||
check() { # name expected_rc dir [range]
|
||||
local name=$1 want=$2 dir=$3 range=${4:-}
|
||||
"$BIN" "$dir" $range >/dev/null 2>&1; local got=$?
|
||||
if [ "$got" = "$want" ]; then pass=$((pass+1)); echo " ok $name"
|
||||
else fail=$((fail+1)); echo " FAIL $name (want rc=$want, got rc=$got)"; fi
|
||||
}
|
||||
|
||||
echo "REFUSALS — the mark in each of the four places it hides"
|
||||
|
||||
d=$(newrepo)
|
||||
check "clean repo is green" 0 "$d"
|
||||
mkdir -p "$d/public/$V-uploads"
|
||||
printf 'PNG\n' > "$d/public/$V-uploads/28d53ec4.png"
|
||||
git -C "$d" add -A && git -C "$d" commit -qm "add an image"
|
||||
check "MUTATION: tracked path named for the vendor" 1 "$d"
|
||||
git -C "$d" rm -rq "public/$V-uploads" && git -C "$d" commit -qm "move the image"
|
||||
check "MUTATION REVERTED: green again" 0 "$d"
|
||||
rm -rf "$d"
|
||||
|
||||
d=$(newrepo)
|
||||
printf '<img src="/%s-uploads/28d53ec4.png" />\n' "$V" > "$d/index.html"
|
||||
git -C "$d" add -A && git -C "$d" commit -qm "a page"
|
||||
check "content: asset folder referenced in markup" 1 "$d"
|
||||
rm -rf "$d"
|
||||
|
||||
d=$(newrepo)
|
||||
printf '{"devDependencies":{"%s-tagger":"^1.1.3"}}\n' "$V" > "$d/package.json"
|
||||
git -C "$d" add -A && git -C "$d" commit -qm "deps"
|
||||
check "content: the vendor's build plugin in package.json" 1 "$d"
|
||||
rm -rf "$d"
|
||||
|
||||
d=$(newrepo)
|
||||
printf '<script src="https://cdn.gpteng.co/gptengineer.js"></script>\n' > "$d/index.html"
|
||||
git -C "$d" add -A && git -C "$d" commit -qm "a page"
|
||||
check "content: the vendor's injected script tag" 1 "$d"
|
||||
rm -rf "$d"
|
||||
|
||||
d=$(newrepo)
|
||||
printf '<meta name="generator" content="Lovable" />\n' > "$d/index.html"
|
||||
git -C "$d" add -A && git -C "$d" commit -qm "a page"
|
||||
check "content: the generator meta tag" 1 "$d"
|
||||
rm -rf "$d"
|
||||
|
||||
d=$(newrepo)
|
||||
printf 'y\n' > "$d/a.txt"; git -C "$d" add -A
|
||||
git -C "$d" commit -qm "Update $V project template"
|
||||
check "MUTATION: the vendor named in a commit message" 1 "$d"
|
||||
git -C "$d" commit -q --amend -m "update the project template"
|
||||
check "MUTATION REVERTED: reworded message is green" 0 "$d"
|
||||
rm -rf "$d"
|
||||
|
||||
d=$(newrepo)
|
||||
printf 'y\n' > "$d/a.txt"; git -C "$d" add -A
|
||||
# --no-verify ON PURPOSE. A commit-msg hook on the workstation
|
||||
# (~/.githooks/commit-msg) already strips vendor co-author trailers, and with it
|
||||
# enabled this case cannot be constructed — the hook silently rewrites the
|
||||
# trailer to ours and the assertion passes for the wrong reason. That hook is
|
||||
# per-machine: it does not run on a runner, on a teammate's laptop, or on a
|
||||
# commit made through the GitHub web UI. This gate is the layer that does. So
|
||||
# the test bypasses the hook to prove the GATE catches what the hook would have.
|
||||
git -C "$d" commit -q --no-verify -m "a change
|
||||
|
||||
Co-authored-by: $V bot <bot@$V.dev>"
|
||||
check "MUTATION: Co-authored-by credits the vendor" 1 "$d"
|
||||
git -C "$d" commit -q --amend --no-verify -m "a change"
|
||||
check "MUTATION REVERTED: trailer dropped is green" 0 "$d"
|
||||
rm -rf "$d"
|
||||
|
||||
d=$(newrepo)
|
||||
printf 'y\n' > "$d/a.txt"; git -C "$d" add -A
|
||||
git -C "$d" -c user.name="$V" -c user.email="bot@$V.dev" commit -qm "a change"
|
||||
check "MUTATION: the vendor as author identity" 1 "$d"
|
||||
rm -rf "$d"
|
||||
|
||||
d=$(newrepo)
|
||||
printf 'y\n' > "$d/a.txt"; git -C "$d" add -A; git -C "$d" commit -qm "clean subject"
|
||||
printf 'z\n' > "$d/b.txt"; git -C "$d" add -A; git -C "$d" commit -qm "Visual edit in $V"
|
||||
printf 'w\n' > "$d/c.txt"; git -C "$d" add -A; git -C "$d" commit -qm "clean again"
|
||||
check "default scope (HEAD only) misses an older bad message" 0 "$d"
|
||||
check "explicit range catches it" 1 "$d" "HEAD~3..HEAD"
|
||||
rm -rf "$d"
|
||||
|
||||
echo
|
||||
echo "ALLOWANCES — the same word, innocently, as it really occurs in the estate"
|
||||
|
||||
d=$(newrepo)
|
||||
printf ' "%s</w>": 38565,\n' "$V" > "$d/vocab.json"
|
||||
printf '%s\n' "$V" > "$d/words_alpha.txt"
|
||||
printf 'Tell me a story about a %s character.\n' "$V" > "$d/harmless.txt"
|
||||
printf 'caseStudy: %s and v0 activate users by turning a prompt into an app\n' "${V^}" > "$d/guide.yaml"
|
||||
printf 'mail2%s.com\n' "$V" > "$d/generic_emails.txt"
|
||||
printf 'let e=[".%s.app",".%sproject.com",".webcontainer-api.io"];\n' "$V" "$V" > "$d/clerk-bundle.js"
|
||||
printf 'A collection of UI components. Integrate them in v0, %s, Bolt.\n' "${V^}" > "$d/registries.json"
|
||||
git -C "$d" add -A && git -C "$d" commit -qm "corpora, word lists and third-party bundles"
|
||||
check "tokenizer vocab / word list / ML corpus / marketing prose / blocklist / upstream host list" 0 "$d"
|
||||
rm -rf "$d"
|
||||
|
||||
echo
|
||||
echo "ESCAPE HATCH — a repo whose job is to name the vendor"
|
||||
|
||||
d=$(newrepo)
|
||||
mkdir -p "$d/tools/rules"
|
||||
printf 'Hanzo Dev <dev@hanzo.ai> <bot@%s.dev>\n' "$V" > "$d/tools/rules/mailmap.txt"
|
||||
git -C "$d" add -A && git -C "$d" commit -qm "scrubber rules"
|
||||
check "scrubber rule file is refused by default" 1 "$d"
|
||||
printf 'tools/rules/*\n' > "$d/.vendormark-allow"
|
||||
git -C "$d" add -A && git -C "$d" commit -qm "declare the rule files"
|
||||
check "...and allowed once declared in .vendormark-allow" 0 "$d"
|
||||
rm -rf "$d"
|
||||
|
||||
echo
|
||||
echo "vendormark_test: $pass passed, $fail failed"
|
||||
[ "$fail" -eq 0 ]
|
||||
@@ -59,6 +59,60 @@ test:
|
||||
exit 1
|
||||
fi
|
||||
echo "OK: .github and .hanzo copies are one file ($(wc -l < .github/workflows/build.yml) lines)"
|
||||
- name: gate-runs-before-delegate
|
||||
# `mode: delegate` chooses WHERE the image is built. It must never choose
|
||||
# WHETHER the commit was tested. It did: the delegate step was evaluated
|
||||
# fourth, ahead of every toolchain and ahead of `test:`, and every other step
|
||||
# carried `if: inputs.mode != 'delegate'` — so a delegated run skipped the
|
||||
# gate along with the build and finished green in seconds having proven
|
||||
# nothing. The seam was right and the ORDER was wrong, which is the kind of
|
||||
# defect that reads as correct in every diff that touches one step at a time.
|
||||
#
|
||||
# Two assertions, because either one alone can be satisfied while the gate
|
||||
# still does not run: the delegate POST must come AFTER the test gate, and
|
||||
# the test gate must not be delegate-guarded.
|
||||
run: |
|
||||
set -e
|
||||
f=.github/workflows/build.yml
|
||||
t=$(grep -n '^ - name: Test (per hanzo.yml)$' "$f" | cut -d: -f1)
|
||||
d=$(grep -n '^ - name: Delegate build to platform (mode=delegate)$' "$f" | cut -d: -f1)
|
||||
[ -n "$t" ] && [ -n "$d" ] || { echo "::error::cannot find the test gate ($t) or the delegate step ($d) — a rename broke this assertion, fix the assertion"; exit 1; }
|
||||
[ "$d" -gt "$t" ] || { echo "::error::the delegate step (line $d) runs BEFORE the test gate (line $t) — a delegated build would ship an ungated commit"; exit 1; }
|
||||
if sed -n "${t}p;$((t+1)),$((t+80))p" "$f" | grep -m1 '^ if: ' | grep -q "mode != 'delegate'"; then
|
||||
echo "::error::the test gate is guarded by \`inputs.mode != 'delegate'\` — delegate would skip it"; exit 1
|
||||
fi
|
||||
echo "OK: test gate at line $t, delegate POST at line $d, gate not delegate-guarded"
|
||||
- name: publishable
|
||||
# bin/publishable refuses a `build_secrets` name that has not declared itself
|
||||
# public. That matters because a build_secret is baked in as a --build-arg,
|
||||
# which `docker history` prints to anyone who can pull — so the key is a
|
||||
# publishing decision wearing the word "secret". Until this, the only check
|
||||
# in the fleet lived inside hanzoai/ui's own Dockerfile, for one image.
|
||||
run: bash bin/publishable_test.sh
|
||||
- name: publishable-rule-is-one-rule
|
||||
# The rule is spelled twice on purpose, and this is what keeps it one rule.
|
||||
#
|
||||
# build.yml cannot call bin/publishable, because the tools checkout and the
|
||||
# workflow resolve from DIFFERENT places: a caller pins the workflow at a ref
|
||||
# its own forge resolves, while the tools step derives its ref from
|
||||
# GITHUB_WORKFLOW_REF — which git.hanzo.ai does not set, so it falls back to
|
||||
# `v1` and clones github.com. Measured on run 36473: `derived ref=v1`,
|
||||
# `cloned https://github.com/hanzoai/ci@v1`. So a step calling a NEW file
|
||||
# under $CI_HOME is broken on every forge run until a tag moves on a host the
|
||||
# caller never named. The pipeline has to carry its own rule.
|
||||
#
|
||||
# Which leaves the pattern in two files, which is the same shape as the two
|
||||
# copies of build.yml and gets the same treatment: demand they are identical
|
||||
# and let the gate, not a reader, be the thing that notices.
|
||||
run: |
|
||||
set -e
|
||||
a=$(grep -oE '^[[:space:]]*PUBLISHABLE_\*\|[^)]*\)' .github/workflows/build.yml | tr -d '[:space:]')
|
||||
b=$(grep -oE '^[[:space:]]*PUBLISHABLE_\*\|[^)]*\)' bin/publishable | tr -d '[:space:]')
|
||||
[ -n "$a" ] && [ -n "$b" ] || { echo "::error::publishable rule not found in build.yml ($a) or bin/publishable ($b) — an edit renamed it, fix this assertion"; exit 1; }
|
||||
[ "$a" = "$b" ] || { echo "::error::the publishable rule differs between build.yml and bin/publishable — build.yml is what runs, bin/publishable is what is tested, so a difference means the tested rule is not the enforced one:
|
||||
build.yml: $a
|
||||
bin/publishable: $b"; exit 1; }
|
||||
echo "OK: one publishable rule, two spellings — $a"
|
||||
- name: imgver
|
||||
# bin/imgver decides the version EVERY image in the fleet publishes — this
|
||||
# workflow's build lane calls it, and so does the imgver composite action the
|
||||
@@ -67,6 +121,24 @@ test:
|
||||
# IfNotPresent never picks up and no reader can see. Offline and
|
||||
# deterministic: the registry floor is injected, so it needs no network.
|
||||
run: bash bin/imgver_test.sh
|
||||
- name: vendormark
|
||||
# bin/vendormark is the gate the build lane runs against every checkout: it
|
||||
# refuses a site-generator vendor's branding in a tracked path, in tracked
|
||||
# content, in a commit message, in an author identity or in a Co-authored-by
|
||||
# trailer. Five shipped sites carried it — one serving the vendor's upload
|
||||
# folder as its favicon — and clearing them took a history rewrite and a
|
||||
# force-push of every branch.
|
||||
#
|
||||
# The suite pins the refusals AND the allowances, and the allowances are the
|
||||
# larger half on purpose. The vendor's name is also an ordinary English
|
||||
# adjective: across the 1,065 repos swept it appears innocently in tokenizer
|
||||
# vocabularies, word lists, ML corpora, packed public-suffix data and an
|
||||
# upstream Clerk bundle's list of other people's dev hosts. A gate that reds
|
||||
# a fifth of the estate is a gate someone switches off, so the content rules
|
||||
# are only strings that cannot occur in prose, and two candidate rules were
|
||||
# measured and DROPPED for redding that Clerk bundle. Offline and
|
||||
# deterministic: temp repos, no network.
|
||||
run: bash bin/vendormark_test.sh
|
||||
- name: gover
|
||||
# bin/gover is the gate the build lane runs against every Dockerfile before
|
||||
# it builds: it refuses a Go builder image older than the go.mod it
|
||||
@@ -83,6 +155,22 @@ test:
|
||||
# and a multi-module repo is judged by its NEAREST go.mod. Offline and
|
||||
# deterministic: temp dirs, no registry, no network.
|
||||
run: bash bin/gover_test.sh
|
||||
- name: sitepublish
|
||||
# bin/sitepublish is the whole of the `site:` lane. The lane it replaced was
|
||||
# dead: it staged with `mc mirror` behind S3_ADMIN_ACCESS_KEY/SECRET_KEY,
|
||||
# names KMS holds for no org, so it failed closed for every caller — and of
|
||||
# the 155 hanzo.yml files across the three orgs, not one declared a `site:`.
|
||||
#
|
||||
# The suite is weighted toward REFUSALS because the failure that matters here
|
||||
# is a green publish that verified nothing — indistinguishable from a working
|
||||
# one until a site quietly stops updating. Three defects of exactly that
|
||||
# shape are pinned by name: an object-shaped read of a body that is not an
|
||||
# object, `.releases[]` against the BARE ARRAY the list route returns (which
|
||||
# resolves nothing and so can never fail for its stated reason), and
|
||||
# `[ "$a" = "$b" ]` on two empty strings, which is TRUE. Reintroducing them
|
||||
# turns 3, 9 and 1 of these tests red respectively. Offline and
|
||||
# deterministic: a curl shim answers the routes from fixtures, no network.
|
||||
run: bash bin/sitepublish_test.sh
|
||||
|
||||
# No `deploy:` ON PURPOSE. Rollout is a reviewed tag pin in hanzoai/universe
|
||||
# (infra/k8s/operator/crs/ci.yaml), the same rule cloud and git follow: a
|
||||
|
||||
Reference in New Issue
Block a user