Compare commits

...
Author SHA1 Message Date
Hanzo AI 2c5eeb6352 fix(ci): build.yml runner defaults to online ARC pool, not offline evo (same disease as .github#11) 2026-07-02 23:38:48 -07:00
z d07661c0cc docs(brand): add hero banner 2026-06-28 20:06:09 -07:00
z c67f9fbfcd chore(brand): dynamic hero banner 2026-06-28 20:06:08 -07:00
hanzo-dev c380563306 ci: drop blanket GOPRIVATE — public modules use the immutable proxy+sumdb
luxfi/hanzoai/zooai Go modules are public (verified: dex, precompile, authz, pq,
age, keys all resolve unauthenticated). Setting GOPRIVATE for the whole orgs
routed them 'direct' and bypassed sum.golang.org, so a force-moved tag silently
poisoned every downstream go.sum (the recurring 'checksum mismatch' build
breakage). Dropping it lets go resolve them via proxy.golang.org + sumdb —
canonical IMMUTABLE hashes a re-publish cannot break. The GH_PAT git credential
stays for authenticated direct fallback + any genuinely-private repo (re-add via
a NARROW GOPRIVATE, never whole-org).
2026-06-28 14:10:59 -07:00
hanzo-dev daacc3ca9b fix(ci): tag-suffix optional — clean image tags when absent
The reusable build read ."tag-suffix" unconditionally, so any repo without it
(every repo today) got broken tags like sha-xxx-amd64-null. Default to empty in
both build and deploy so single-variant repos get clean tags; multi-variant
repos still qualify (…-ce, ce-latest). Unblocks adopting the reusable workflow.
2026-06-28 05:38:06 -07:00
hanzo-devandGitHub 9231a46d2b build: KMS secret fetch over canonical /v1/kms + buildx gh_token secret (#2)
* build: KMS secret fetch over canonical /v1/kms + buildx gh_token secret

The Infisical /api/* surface was removed when KMS migrated to luxfi/kms, so the
old /api/v1/auth/universal-auth/login + /api/v3/secrets/raw calls now 404 and
every repo's CI fails to fetch GHCR_TOKEN/KUBECONFIG. Rewrite to /v1/kms:
- auth: POST /v1/kms/auth/login {clientId,clientSecret} (IAM client_credentials
  via the <org>-kms app) -> accessToken
- fetch: per-secret GET /v1/kms/orgs/<org>/secrets/<path>/<name>?env=<env>
- org from hanzo.yml kms.org or KMS_ORG var
Also fetch GIT_TOKEN and pass it to buildx as the gh_token BuildKit secret so
Dockerfiles can clone private Go modules (luxfi/dex, luxfi/precompile). Empty-safe.

* build: one GITHUB_TOKEN in KMS for GHCR push + private-module clone (DRY)

A single GitHub PAT with repo + write:packages does both jobs, so collapse the
split GHCR_TOKEN/GIT_TOKEN to one KMS key GITHUB_TOKEN. KUBECONFIG stays separate
(different concern). Shell var GHTOKEN avoids the reserved GITHUB_ env prefix.

* build: GHCR push via automatic workflow token; KMS only for cross-org clone + kubeconfig

GitHub injects a per-job GITHUB_TOKEN scoped to the running repo; with
permissions: packages: write it can push the repo's own GHCR package (same-org),
so the push path needs no stored credential. KMS is now best-effort, supplying
only what the automatic token can't: a cross-org PAT to clone other orgs' private
Go modules (luxfi/dex, luxfi/precompile) + KUBECONFIG for deploy. Public repos
with no cross-org deps build with zero KMS dependency.
2026-06-24 10:43:03 -07:00
hanzo-devandGitHub be43b6f323 ci: private Go module access (luxfi/hanzoai/zooai), guarded by GH_PAT (#1)
* ci: configure private Go module access (luxfi/hanzoai/zooai)

Any go-based `test:` step in a consumer's hanzo.yml needs to fetch
private github.com/{luxfi,hanzoai,zooai}/* modules; without auth go
hits 'git ls-remote … exit 128'. Add a guarded step that wires GH_PAT
via git insteadOf + sets GOPRIVATE job-wide. No-op when GH_PAT is
absent, so non-Go consumers (bootnode etc.) are unaffected.

* ci: make private-module auth runner-state-immune (fresh GIT_CONFIG_GLOBAL)

Upgrade the naive 'git config --global' to the robust pattern proven in
hanzoai/iam: fresh per-job GIT_CONFIG_GLOBAL + GIT_CONFIG_NOSYSTEM=1, probe
from a neutral dir. A plain global config is overridden on shared arc
runners by stale ~/.gitconfig state and actions/checkout's persisted
extraheader. Still a no-op without GH_PAT.
2026-06-22 17:21:04 -07:00
3 changed files with 99 additions and 21 deletions
+9
View File
@@ -0,0 +1,9 @@
<svg xmlns="http://www.w3.org/2000/svg" width="1280" height="640" viewBox="0 0 1280 640" role="img" aria-label="ci">
<rect width="1280" height="640" fill="#0A0A0A"/>
<svg x="96" y="215" width="210" height="210" viewBox="0 0 67 67"><path d="M22.21 67V44.6369H0V67H22.21Z" fill="#fff"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z" fill="#fff"/><path d="M22.21 0H0V22.3184H22.21V0Z" fill="#fff"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z" fill="#fff"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z" fill="#fff"/></svg>
<text x="378" y="276" font-family="Inter,system-ui,-apple-system,sans-serif" font-size="78" font-weight="800" letter-spacing="-2" fill="#ffffff">ci</text>
<text x="378" y="322" font-family="Inter,system-ui,sans-serif" font-size="30" fill="#ffffff" opacity=".66">Reusable CI/CD: build/test/deploy any repo from its hanzo.yml on arc</text>
<rect x="378" y="338" width="806" height="3" rx="1.5" fill="#ffffff" opacity=".9"/>
<text x="378" y="390" font-family="Inter,system-ui,sans-serif" font-size="24" font-weight="600" fill="#ffffff" opacity=".5">github.com/hanzoai</text>
<text x="1184" y="390" text-anchor="end" font-family="Inter,system-ui,sans-serif" font-size="24" font-weight="600" fill="#ffffff" opacity=".5">hanzo.ai</text>
</svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

+88 -21
View File
@@ -15,7 +15,7 @@ on:
the build, metered as build minutes). Bring-your-own: pass the labels
of your own self-hosted arc runners instead.
type: string
default: '["self-hosted","linux","amd64"]'
default: '["hanzo-build-linux-amd64"]'
permissions:
contents: read
@@ -27,28 +27,89 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Authenticated git for go modules (rate-limit + any private repo)
# luxfi/hanzoai/zooai Go modules are PUBLIC, so `go` resolves them through
# the default public proxy (proxy.golang.org) + checksum db (sum.golang.org)
# — canonical, IMMUTABLE hashes that a force-moved tag can no longer break.
# We deliberately do NOT set GOPRIVATE: that would route these public
# modules `direct` and bypass the sumdb, re-introducing the re-publish
# poisoning we just removed. This step only adds a GH_PAT git credential so
# the proxy's `direct` fallback (and any genuinely-private repo added later,
# via a narrow GOPRIVATE) authenticates instead of hitting the anon rate
# limit. No-op when GH_PAT is absent.
#
# Robustness (learned the hard way in hanzoai/iam): a plain
# `git config --global` is defeated on shared self-hosted arc runners by
# (a) a stale insteadOf/credential left in ~/.gitconfig by a prior run and
# (b) actions/checkout's persisted http.<github>.extraheader (the repo-
# scoped GITHUB_TOKEN) in the checked-out repo's local config. So use a
# FRESH per-job GIT_CONFIG_GLOBAL + GIT_CONFIG_NOSYSTEM=1 and verify from
# a NEUTRAL dir (matches where go clones modules — GOMODCACHE).
env:
GH_PAT: ${{ secrets.GH_PAT }}
run: |
if [ -z "${GH_PAT:-}" ]; then
echo "GH_PAT not set — public proxy+sumdb handles everything"; exit 0
fi
CFG="$RUNNER_TEMP/gitconfig-private"; : > "$CFG"
GIT_CONFIG_GLOBAL="$CFG" git config --global \
url."https://x-access-token:${GH_PAT}@github.com/".insteadOf "https://github.com/"
{
echo "GIT_CONFIG_GLOBAL=$CFG"
echo "GIT_CONFIG_NOSYSTEM=1"
} >> "$GITHUB_ENV"
( cd "$RUNNER_TEMP" && GIT_CONFIG_GLOBAL="$CFG" GIT_CONFIG_NOSYSTEM=1 \
git ls-remote https://github.com/hanzoai/authz >/dev/null 2>&1 ) \
&& echo "git auth OK" \
|| echo "::warning::GH_PAT set but repo probe failed"
- name: Log in to GHCR (automatic workflow token)
# GHCR push needs no stored credential: GitHub injects a per-job
# GITHUB_TOKEN scoped to THIS repo, and `permissions: packages: write`
# (above) lets it push the repo's own package (e.g. zooai/node →
# ghcr.io/zooai/node, same org). First push creates the package
# repo-linked. No KMS, no PAT for the push path.
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u "${{ github.actor }}" --password-stdin
- name: Fetch deploy credentials from KMS
id: kms
env:
KMS_CLIENT_ID: ${{ secrets.KMS_CLIENT_ID }}
KMS_CLIENT_SECRET: ${{ secrets.KMS_CLIENT_SECRET }}
KMS_ENDPOINT: ${{ vars.KMS_ENDPOINT || 'https://kms.hanzo.ai' }}
KMS_WORKSPACE: ${{ vars.KMS_WORKSPACE }}
KMS_ORG: ${{ vars.KMS_ORG }}
run: |
set -euo pipefail
TOKEN=$(curl -sf "$KMS_ENDPOINT/api/v1/auth/universal-auth/login" \
set -uo pipefail
# Canonical luxfi/kms surface — /v1/kms (the Infisical /api/* surface was
# removed when KMS migrated to luxfi/kms, MPC-rooted). Auth = an IAM
# client_credentials JWT minted by the per-org `<org>-kms` application;
# secrets are org-scoped and fetched one at a time (no bulk /secrets/raw).
#
# This step is BEST-EFFORT: the automatic workflow token already
# authorizes GHCR push, so a repo with no cross-org private deps and no
# deploy can build without KMS. KMS provides only:
# GITHUB_TOKEN — a cross-org PAT for cloning OTHER orgs' private Go
# modules (luxfi/dex, luxfi/precompile from a zooai build), which the
# repo-scoped automatic token cannot read. Handed to buildx as the
# `gh_token` BuildKit secret.
# KUBECONFIG — cluster access for the deploy step.
ORG="$(python3 -c "import yaml;print((yaml.safe_load(open('hanzo.yml')).get('kms') or {}).get('org','') )" 2>/dev/null || true)"
: "${ORG:=${KMS_ORG:-}}"
ENV="$(python3 -c "import yaml;print((yaml.safe_load(open('hanzo.yml')).get('kms') or {}).get('environment','prod'))" 2>/dev/null || echo prod)"
PATHQ="$(python3 -c "import yaml;print((yaml.safe_load(open('hanzo.yml')).get('kms') or {}).get('path','/deploy').strip('/'))" 2>/dev/null || echo deploy)"
if [ -z "${KMS_CLIENT_ID:-}" ] || [ -z "$ORG" ]; then
echo "::notice::KMS not configured (no KMS_CLIENT_ID or org) — skipping; GHCR push uses the workflow token"; exit 0
fi
TOKEN=$(curl -sf "$KMS_ENDPOINT/v1/kms/auth/login" \
-H 'Content-Type: application/json' \
-d "{\"clientId\":\"$KMS_CLIENT_ID\",\"clientSecret\":\"$KMS_CLIENT_SECRET\"}" | jq -r '.accessToken')
ENV="$(python3 -c "import yaml;print((yaml.safe_load(open('hanzo.yml')).get('kms') or {}).get('environment','prod'))")"
PATHQ="$(python3 -c "import yaml;print((yaml.safe_load(open('hanzo.yml')).get('kms') or {}).get('path','/deploy'))")"
SECRETS=$(curl -sf "$KMS_ENDPOINT/api/v3/secrets/raw?workspaceId=$KMS_WORKSPACE&secretPath=$PATHQ&environment=$ENV" \
-H "Authorization: Bearer $TOKEN")
get() { echo "$SECRETS" | jq -r ".secrets[]|select(.secretKey==\"$1\")|.secretValue // empty"; }
GHCR_USER=$(get GHCR_USER); GHCR_TOKEN=$(get GHCR_TOKEN); KUBECONFIG_B64=$(get KUBECONFIG)
: "${GHCR_USER:=${{ github.actor }}}"
if [ -z "$GHCR_TOKEN" ]; then echo "::error::GHCR_TOKEN not found in KMS"; exit 1; fi
echo "::add-mask::$GHCR_TOKEN"
echo "$GHCR_TOKEN" | docker login ghcr.io -u "$GHCR_USER" --password-stdin
-d "{\"clientId\":\"$KMS_CLIENT_ID\",\"clientSecret\":\"$KMS_CLIENT_SECRET\"}" | jq -r '.accessToken // empty')
if [ -z "$TOKEN" ]; then echo "::warning::KMS login failed (org=$ORG client=$KMS_CLIENT_ID) — cross-org private deps & deploy unavailable"; exit 0; fi
get() { curl -sf "$KMS_ENDPOINT/v1/kms/orgs/$ORG/secrets/$PATHQ/$1?env=$ENV" -H "Authorization: Bearer $TOKEN" | jq -r '.secret.value // empty'; }
# Cross-org private Go module read token → buildx `gh_token` secret.
# GIT_TOKEN is not a reserved name, so it's safe in GITHUB_ENV.
GIT_TOKEN=$(get GITHUB_TOKEN)
if [ -n "$GIT_TOKEN" ]; then echo "::add-mask::$GIT_TOKEN"; echo "GIT_TOKEN=$GIT_TOKEN" >> "$GITHUB_ENV"; fi
KUBECONFIG_B64=$(get KUBECONFIG)
if [ -n "$KUBECONFIG_B64" ]; then echo "$KUBECONFIG_B64" | base64 -d > "$RUNNER_TEMP/kubeconfig"; echo "kubeconfig=$RUNNER_TEMP/kubeconfig" >> "$GITHUB_OUTPUT"; fi
- name: Build & push images (per hanzo.yml)
@@ -60,11 +121,17 @@ jobs:
python3 -c "import yaml,json;print(json.dumps(yaml.safe_load(open('hanzo.yml'))['images']))" | jq -c '.[]' | while read -r img; do
name=$(echo "$img"|jq -r .name); ctx=$(echo "$img"|jq -r .context)
df=$(echo "$img"|jq -r '.dockerfile // (.context+"/Dockerfile")'); repo=$(echo "$img"|jq -r .repo)
sfx=$(echo "$img"|jq -r '."tag-suffix"')
TAGS="-t $repo:sha-${SHORT}-amd64-${sfx} -t $repo:${sfx}-latest"
[ "$IS_TAG" = 1 ] && TAGS="$TAGS -t $repo:${VER}-amd64-${sfx}"
# tag-suffix is OPTIONAL (most repos ship a single variant). When set
# (e.g. "ce"/"ee") it qualifies every tag; when absent the tags are
# clean (no trailing dash). Build + deploy must agree on this shape.
sfx=$(echo "$img"|jq -r '."tag-suffix" // ""')
TAGS="-t $repo:sha-${SHORT}-amd64${sfx:+-$sfx} -t $repo:${sfx:+$sfx-}latest"
[ "$IS_TAG" = 1 ] && TAGS="$TAGS -t $repo:${VER}-amd64${sfx:+-$sfx}"
echo "::group::build $name → $repo (${sfx})"
docker buildx build --platform linux/amd64 --push $TAGS -f "$df" "$ctx"
# GIT_TOKEN (from KMS, via GITHUB_ENV) is passed as the `gh_token`
# BuildKit secret so Dockerfiles can clone private Go modules; omitted
# cleanly when absent (public-only builds unaffected).
docker buildx build --platform linux/amd64 ${GIT_TOKEN:+--secret id=gh_token,env=GIT_TOKEN} --push $TAGS -f "$df" "$ctx"
echo "::endgroup::"
done
@@ -94,8 +161,8 @@ jobs:
python3 -c "import yaml,json;print(json.dumps(yaml.safe_load(open('hanzo.yml'))['deploy']['services']))" | jq -c '.[]' | while read -r s; do
svc=$(echo "$s"|jq -r .name); imgname=$(echo "$s"|jq -r .image)
repo=$(jq -r --arg n "$imgname" '.[]|select(.name==$n)|.repo' /tmp/imgs.json)
sfx=$(jq -r --arg n "$imgname" '.[]|select(.name==$n)|."tag-suffix"' /tmp/imgs.json)
ref="$repo:sha-${SHORT}-amd64-${sfx}"
sfx=$(jq -r --arg n "$imgname" '.[]|select(.name==$n)|."tag-suffix" // ""' /tmp/imgs.json)
ref="$repo:sha-${SHORT}-amd64${sfx:+-$sfx}"
echo "rolling $svc → $ref"
kubectl -n "$NS" set image "deployment/$svc" "*=$ref"
kubectl -n "$NS" rollout status "deployment/$svc" --timeout=180s
+2
View File
@@ -1,3 +1,5 @@
<p align="center"><img src=".github/hero.svg" alt="ci" width="880"></p>
# hanzoai/ci
One reusable CI/CD workflow for every Hanzo / Lux / Zoo repo. Build + test +