the test step has never run, and the gate that would have said so never ran either
CI/CD / gate (push) Successful in 55s
Hanzo CI/CD / cicd (push) Successful in 55s

git.hanzo.ai resolves every `type: boolean` workflow_call input to false. Not
just an omitted one falling back to its default -- an explicitly passed `true`
resolves to false too. Measured in isolation (z/inputprobe runs 27365, 27382):

  type: boolean, default: true, caller omits it   -> inputs.tests = false
  type: boolean, caller passes with: {tests: true} -> inputs.tests = false
  type: string,  default: 'true', caller omits it  -> inputs.tests = true
  type: string,  default: '["hanzo-build-linux-amd64"]' -> resolves in full

So `if: inputs.mode != 'delegate' && inputs.tests` has been false on every run
this pipeline has ever served, and the `test:` block was parsed and skipped. On
2026-08-04 the last two cicd.yml runs of hanzoai/cloud, commerce, console,
gateway, hanzo.ai and this repo were all completed/success with
`Test (per hanzo.yml)` = skipped. Every test gate in the fleet was declared,
read, and never once executed -- a green over zero tests, which is the exact
failure the step's own assertion exists to refuse when a SUITE is empty, one
level up where nothing was watching.

tests: becomes a string compared against 'false'. A string sidesteps the forge
bug (proven above) and means the same thing on both planes, and comparing
against 'false' rather than testing truthiness keeps hanzoai/cloud's existing
`tests: ${{ github.ref_type != 'tag' }}` working untouched -- that expression
renders the strings "true"/"false".

The second half is what the first half was hiding. build-yml-is-one-file is a
`test:` gate, so it never ran either, and the two published copies had drifted
26 lines: .hanzo/workflows/build.yml -- the ONLY copy git.hanzo.ai executes --
was missing the Structural invariants step entirely. ignoretracked and modsize
have therefore never run on the plane that runs. The copies are one file again,
regenerated rather than hand-edited a second time, since hand-editing twice is
how the drift got in.

conflictmarkers joins that step, because it is the same defect class: `git add`
accepts a file still carrying git's own markers, and from then on they are
ordinary committed content -- status clean, diff empty, both sides of the merge
shipping. It hides in generated and vendored files that are large and
unreviewed, and no compiler objects: a .d.ts nothing imports is never parsed,
and in YAML, JSON and Markdown the markers are valid content. hanzoai/base has
carried `<<<<<<< HEAD` / `>>>>>>> upstream/master` at lines 23807 and 23850 of a
committed .d.ts. Proven red against that file and against a synthetic merge,
green on a clean tree, and it does not flag itself or a reStructuredText
underline -- the bare `=======` divider is deliberately not matched, because
seven equals signs at column 1 is also how RST and Setext underline a heading.
This commit is contained in:
zeekay
2026-08-04 17:05:54 -07:00
parent 23862e5c44
commit 0e1a56bd3c
3 changed files with 198 additions and 27 deletions
+42 -5
View File
@@ -42,15 +42,41 @@ on:
tests:
description: >-
Run the `test:` block. Default true, and a caller should leave it that
way. false asserts that THIS EXACT COMMIT was already gated before this
way. 'false' asserts that THIS EXACT COMMIT was already gated before this
run — it does not mean "ship untested". The one shape that holds today
is hanzoai/cloud's release: clients/platform/release.go mints the v* tag
only after that SHA passed the gate on main AND built AND smoked, so the
tag build would re-test a commit already proven, at the cost of a
3108-package link storm. Passing false there runs the gate once instead
3108-package link storm. Passing 'false' there runs the gate once instead
of twice; passing it anywhere else runs it zero times.
type: boolean
default: true
STRING, NOT BOOLEAN, AND THAT IS LOAD-BEARING. git.hanzo.ai resolves
EVERY `type: boolean` workflow_call input to false — not just an omitted
one carrying a default, but an explicitly passed `true` as well. Measured
in isolation (z/inputprobe, runs 27365 and 27382): with `type: boolean,
default: true` and a caller that omits it, `inputs.tests` printed
`false`; a caller passing `with: {tests: true}` printed `false` too;
inverting the polarity to a `skip_tests` boolean printed `false` when
`true` was passed. String inputs in the SAME probe resolved correctly —
`mode` printed `buildx` and `runner` printed its full JSON array from
their declared defaults.
So for as long as this input was a boolean, `if: ... && inputs.tests` was
false on EVERY run, and the step below never executed. That is not a
hypothetical: on 2026-08-04 the last two `cicd.yml` runs of hanzoai/cloud,
commerce, console, gateway, hanzo.ai AND hanzoai/ci itself all reported
completed/success with `Test (per hanzo.yml)` = skipped. Every `test:`
block in the fleet was declared, was read, and had never once run — a
green over zero tests, which the step below exists specifically to refuse
when a SUITE is empty, and which had swallowed the whole gate.
A string sidesteps it on both planes with identical meaning, and the
comparison is against 'false' rather than a truthiness test so that the
expression callers already write — hanzoai/cloud's
`tests: ${{ github.ref_type != 'tag' }}`, which renders the strings
"true"/"false" — keeps working unchanged.
type: string
default: 'true'
submodules:
description: >-
Passed straight to actions/checkout. Default '' is checkout's own
@@ -135,6 +161,16 @@ jobs:
# nested modules separately. Past it, `go get` fails everywhere
# with 'module source tree too large'; near it, this is the only
# warning anyone gets before the first consumer breaks.
# conflictmarkers — a merge that was committed unresolved. `git add`
# accepts a file that still carries git's own markers, and from then
# on they are ordinary content: status clean, diff empty, both sides
# of the merge shipping. It hides in generated and vendored files,
# which are large, rarely opened and effectively unreviewed —
# hanzoai/base carried `<<<<<<< HEAD` / `>>>>>>> upstream/master` at
# lines 23807 and 23850 of a committed .d.ts. No compiler catches it
# there: a .d.ts nothing imports is never parsed, and in YAML, JSON,
# Markdown and every config format we ship the markers are silently
# VALID content.
#
# Both proven RED against cloud@v1.801.424 (the real outage) and GREEN
# against the repaired tree before landing here. They read the checkout
@@ -145,6 +181,7 @@ jobs:
set -euo pipefail
"$RUNNER_TEMP/ci/bin/ignoretracked" .
"$RUNNER_TEMP/ci/bin/modsize" .
"$RUNNER_TEMP/ci/bin/conflictmarkers" .
- name: Provision parse toolchain (jq + PyYAML)
# This reusable parses the caller's hanzo.yml with python3 + PyYAML and
@@ -1008,7 +1045,7 @@ jobs:
# it ran nothing AND nothing shows as having run. A gate that is not a test
# gate — vet, lint, a build, a codegen-freshness check — says neither and is
# untouched. To silence it legitimately, make the gate run a test.
if: inputs.mode != 'delegate' && inputs.tests
if: inputs.mode != 'delegate' && inputs.tests != 'false'
run: |
set -euo pipefail
NOTHING='\[no test files\]|\[no tests to run\]|no tests ran|collected 0 items|No tests found|no test specified|running 0 tests|Tests:[[:space:]]+0 total|(^|[^0-9])0 (passing|passed)'
+73 -22
View File
@@ -42,15 +42,41 @@ on:
tests:
description: >-
Run the `test:` block. Default true, and a caller should leave it that
way. false asserts that THIS EXACT COMMIT was already gated before this
way. 'false' asserts that THIS EXACT COMMIT was already gated before this
run — it does not mean "ship untested". The one shape that holds today
is hanzoai/cloud's release: clients/platform/release.go mints the v* tag
only after that SHA passed the gate on main AND built AND smoked, so the
tag build would re-test a commit already proven, at the cost of a
3108-package link storm. Passing false there runs the gate once instead
3108-package link storm. Passing 'false' there runs the gate once instead
of twice; passing it anywhere else runs it zero times.
type: boolean
default: true
STRING, NOT BOOLEAN, AND THAT IS LOAD-BEARING. git.hanzo.ai resolves
EVERY `type: boolean` workflow_call input to false — not just an omitted
one carrying a default, but an explicitly passed `true` as well. Measured
in isolation (z/inputprobe, runs 27365 and 27382): with `type: boolean,
default: true` and a caller that omits it, `inputs.tests` printed
`false`; a caller passing `with: {tests: true}` printed `false` too;
inverting the polarity to a `skip_tests` boolean printed `false` when
`true` was passed. String inputs in the SAME probe resolved correctly —
`mode` printed `buildx` and `runner` printed its full JSON array from
their declared defaults.
So for as long as this input was a boolean, `if: ... && inputs.tests` was
false on EVERY run, and the step below never executed. That is not a
hypothetical: on 2026-08-04 the last two `cicd.yml` runs of hanzoai/cloud,
commerce, console, gateway, hanzo.ai AND hanzoai/ci itself all reported
completed/success with `Test (per hanzo.yml)` = skipped. Every `test:`
block in the fleet was declared, was read, and had never once run — a
green over zero tests, which the step below exists specifically to refuse
when a SUITE is empty, and which had swallowed the whole gate.
A string sidesteps it on both planes with identical meaning, and the
comparison is against 'false' rather than a truthiness test so that the
expression callers already write — hanzoai/cloud's
`tests: ${{ github.ref_type != 'tag' }}`, which renders the strings
"true"/"false" — keeps working unchanged.
type: string
default: 'true'
submodules:
description: >-
Passed straight to actions/checkout. Default '' is checkout's own
@@ -120,6 +146,43 @@ jobs:
|| { echo "::error::could not fetch hanzoai/ci@${ref} (bin/imgver) from any host"; exit 1; }
echo "CI_HOME=$RUNNER_TEMP/ci" >> "$GITHUB_ENV"
- name: Structural invariants (the repo agrees with itself)
# Two cheap refusals, run before anything is built or pushed, for a
# defect class no later stage can see: a declaration and the reality it
# describes disagreeing in silence.
#
# ignoretracked — content the repo's OWN .gitignore matches but the
# index still tracks. An ignore rule does not untrack what is
# already tracked, so the bulk ships in every release while
# `git status` stays clean. hanzoai/cloud carried 361 MB this way
# for 6 days — past Go's 500 MiB module-zip ceiling, making nine
# consecutive releases unfetchable by every consumer.
# modsize — each Go module tree measured against that same ceiling,
# nested modules separately. Past it, `go get` fails everywhere
# with 'module source tree too large'; near it, this is the only
# warning anyone gets before the first consumer breaks.
# conflictmarkers — a merge that was committed unresolved. `git add`
# accepts a file that still carries git's own markers, and from then
# on they are ordinary content: status clean, diff empty, both sides
# of the merge shipping. It hides in generated and vendored files,
# which are large, rarely opened and effectively unreviewed —
# hanzoai/base carried `<<<<<<< HEAD` / `>>>>>>> upstream/master` at
# lines 23807 and 23850 of a committed .d.ts. No compiler catches it
# there: a .d.ts nothing imports is never parsed, and in YAML, JSON,
# Markdown and every config format we ship the markers are silently
# VALID content.
#
# Both proven RED against cloud@v1.801.424 (the real outage) and GREEN
# against the repaired tree before landing here. They read the checkout
# only — no network, subsecond on a healthy repo — and they FAIL the
# run rather than annotate it, because a warning nobody reads is the
# same silence one level up.
run: |
set -euo pipefail
"$RUNNER_TEMP/ci/bin/ignoretracked" .
"$RUNNER_TEMP/ci/bin/modsize" .
"$RUNNER_TEMP/ci/bin/conflictmarkers" .
- name: Provision parse toolchain (jq + PyYAML)
# This reusable parses the caller's hanzo.yml with python3 + PyYAML and
# slices JSON with jq. A minimal runner image
@@ -336,23 +399,11 @@ jobs:
fi
echo "::warning::KMS login failed (org=$ORG client=$KMS_CLIENT_ID) — cross-org private deps & deploy unavailable"; exit 0
fi
# KMS is embedded in hanzoai/cloud now (HIP-0106, cloud/apps/kms), and its
# contract differs from the standalone service in two ways that BOTH fail
# silently — you get an empty string, which every caller below already
# treats as "not in KMS":
#
# url /v1/kms/secrets/<path>/<name> — the org comes from the TOKEN,
# not the url. The old /v1/kms/orgs/<org>/... form 404s, and -sf
# turns that into empty output.
# body {"name","env","value"} — flat. `.secret.value` selects nothing.
#
# Reading the OLD host is worse than reading nothing: the two instances
# hold DIFFERENT values under the same name, and the standalone copy of
# deploy/EVENT_INGEST_KEY is stale — cloud rejects it at ingest with 403,
# so a build that "found" a key still ships one that cannot write.
#
# Try the embedded shape first and fall back to the standalone one, so a
# repo still pointed at the old host keeps working while the fleet moves.
# Embedded KMS (HIP-0106): org from the token, flat {"value"}. The old
# /v1/kms/orgs/<org>/... + .secret.value pair 404s and `curl -sf` turns
# that into "" — which every caller reads as "not in KMS". The standalone
# host also holds a STALE key cloud 403s, so reading it ships a dead one.
# Embedded first, standalone as fallback.
get() {
v=$(curl -sf "$KMS_ENDPOINT/v1/kms/secrets/$PATHQ/$1?env=$ENV" -H "Authorization: Bearer $TOKEN" | jq -r '.value // empty')
if [ -z "$v" ]; then
@@ -994,7 +1045,7 @@ jobs:
# it ran nothing AND nothing shows as having run. A gate that is not a test
# gate — vet, lint, a build, a codegen-freshness check — says neither and is
# untouched. To silence it legitimately, make the gate run a test.
if: inputs.mode != 'delegate' && inputs.tests
if: inputs.mode != 'delegate' && inputs.tests != 'false'
run: |
set -euo pipefail
NOTHING='\[no test files\]|\[no tests to run\]|no tests ran|collected 0 items|No tests found|no test specified|running 0 tests|Tests:[[:space:]]+0 total|(^|[^0-9])0 (passing|passed)'
+83
View File
@@ -0,0 +1,83 @@
#!/usr/bin/env bash
# conflictmarkers — refuse a repo that has committed an unresolved merge.
# One implementation, every caller.
#
# conflictmarkers [dir]
#
# WHAT THIS CATCHES
#
# `git merge` writes its disagreement INTO the file and stops. Resolving means
# editing those lines out; nothing forces you to. `git add` on a file that still
# contains them is accepted without complaint, and from that moment the markers
# are ordinary committed content — `git status` is clean, `git diff` is empty,
# and the conflict is now a permanent feature of the branch.
#
# It survives because of WHERE it lands. The file is usually generated or
# vendored — large, rarely opened, excluded from review by its own size. Nobody
# reads line 23,807 of a .d.ts. hanzoai/base carried exactly this on main:
#
# plugins/jsvm/internal/types/generated/types.d.ts:23807 <<<<<<< HEAD
# plugins/jsvm/internal/types/generated/types.d.ts:23850 >>>>>>> upstream/master
#
# from an upstream merge nobody finished, sitting in the shipped package.
#
# The compilers are no help, which is the whole problem. In TypeScript the
# markers are a syntax error — but a .d.ts nothing imports is never parsed, so
# there is no error to see. In Go they are a parse error only in a file the
# build reaches. In YAML, JSON, Markdown, SQL, HTML and every config format we
# ship, they are silently VALID content: a marker in a values file is a key
# nobody notices, and a marker in a Markdown doc renders as text.
#
# So the defect class is "both sides of a merge shipped, and nothing in the
# pipeline had an opinion". That is the same shape as its two siblings in this
# step — a declaration and reality disagreeing, with no reader positioned to
# notice — which is why it belongs here rather than in any one repo's gate.
#
# WHAT IT LOOKS FOR, AND WHY NOT MORE
#
# Only the two LABELLED markers git actually writes:
#
# ^<<<<<<< <label> the start marker, seven '<' then a space
# ^>>>>>>> <label> the end marker, seven '>' then a space
#
# The middle `=======` divider is deliberately NOT matched. It is seven bare
# equals signs at the start of a line, which is also how reStructuredText and
# Setext-style Markdown underline a heading — a real and common shape in our
# docs. Matching it would fail repos that have nothing wrong with them, and a
# gate that fails what ought to pass is a gate that gets switched off. The two
# angle-bracket forms carry a label and have no such collision.
#
# Exactly seven, anchored at column 1, followed by a space: `<<<<<<<<` (eight,
# a heredoc or a C++ stream) does not match, and neither does an indented
# mention inside a comment explaining conflict markers — which is what lets this
# file describe them without flagging itself.
#
# Tracked files only, via `git grep`, so a stray marker in an untracked scratch
# file or in node_modules cannot fail a build. `-I` skips binaries.
set -euo pipefail
cd "${1:-.}"
git rev-parse --is-inside-work-tree >/dev/null 2>&1 || {
echo "conflictmarkers: not a git work tree ($(pwd)) — skipping"; exit 0; }
# Built from a variable rather than written literally, so this script is not its
# own first finding when it scans the repo that carries it.
L=$(printf '<%.0s' $(seq 7))
R=$(printf '>%.0s' $(seq 7))
hits=$(git grep -I -n -E "^(${L}|${R}) " -- . || true)
if [ -n "$hits" ]; then
echo "::error::committed merge-conflict markers — an unresolved merge is in the tree"
echo "$hits" | while IFS= read -r line; do
echo " $line"
echo "::error file=${line%%:*}::committed conflict marker"
done
echo
echo "Resolve the merge and commit the result. If the file is GENERATED, do not"
echo "hand-edit it — re-run its generator and commit that."
exit 1
fi
echo "OK: no committed conflict markers ($(git ls-files | wc -l) tracked files)"