two structural refusals: the repo must agree with itself

A day of production archaeology kept finding one defect class wearing
different clothes: a declaration and the reality it describes disagreeing,
with nothing in the pipeline holding an opinion about it. The framework
already refuses this shape in three places — zip refuses middleware that
wraps nothing, zipdoc refuses an op whose address it cannot resolve, Go
refuses a module past its zip ceiling. This lands the same principle for
two disagreements nothing was catching:

bin/ignoretracked — content the repo's OWN .gitignore matches but the index
still tracks. An ignore rule does not untrack what is already tracked, so
the bulk ships in every release while `git status` stays clean. The causal
story from the real outage is in the script header: the rule was removed at
12:22, 855 files landed at 12:51 inside the unguarded window, the rule came
back at 16:48. Every step was reasonable; the combined state shipped 361 MB
of orphaned cargo output in every release for 6 days.

bin/modsize — every Go module tree in the repo measured against Go's hard
500 MiB module-zip ceiling, nested modules separately. Past it, the module
is not merely large: `go get` fails in every consumer at every version that
carries it. cloud crossed at v1.801.416 and was unfetchable for nine
consecutive releases, which is why production sat pinned five releases back.

Wired as one early step in the reusable workflow, before anything builds or
pushes. Both FAIL the run — a warning nobody reads is the same silence one
level up.

Proof, not assertion: both RED against cloud@v1.801.424 (ignoretracked:
361.1 MB tracked-yet-ignored; modsize: 528.8 MiB, past the ceiling, nested
apps/platform/design measured separately at 0.2 MiB) and GREEN against the
repaired v1.801.440, hanzoai/iam, and this repo.

Scripts authored by the structural-guards lane; proofs re-run and wiring
added before landing. Callers pin @v1 — this activates when that tag moves.

Co-authored-by: Hanzo Dev <dev@hanzo.ai>

.github/workflows/build.yml

@@ -120,6 +120,32 @@ jobs:
             || { echo "::error::could not fetch hanzoai/ci@${ref} (bin/imgver) from any host"; exit 1; }
           echo "CI_HOME=$RUNNER_TEMP/ci" >> "$GITHUB_ENV"
 
+      - name: Structural invariants (the repo agrees with itself)
+        # Two cheap refusals, run before anything is built or pushed, for a
+        # defect class no later stage can see: a declaration and the reality it
+        # describes disagreeing in silence.
+        #
+        #   ignoretracked — content the repo's OWN .gitignore matches but the
+        #     index still tracks. An ignore rule does not untrack what is
+        #     already tracked, so the bulk ships in every release while
+        #     `git status` stays clean. hanzoai/cloud carried 361 MB this way
+        #     for 6 days — past Go's 500 MiB module-zip ceiling, making nine
+        #     consecutive releases unfetchable by every consumer.
+        #   modsize — each Go module tree measured against that same ceiling,
+        #     nested modules separately. Past it, `go get` fails everywhere
+        #     with 'module source tree too large'; near it, this is the only
+        #     warning anyone gets before the first consumer breaks.
+        #
+        # Both proven RED against cloud@v1.801.424 (the real outage) and GREEN
+        # against the repaired tree before landing here. They read the checkout
+        # only — no network, subsecond on a healthy repo — and they FAIL the
+        # run rather than annotate it, because a warning nobody reads is the
+        # same silence one level up.
+        run: |
+          set -euo pipefail
+          "$RUNNER_TEMP/ci/bin/ignoretracked" .
+          "$RUNNER_TEMP/ci/bin/modsize" .
+
       - name: Provision parse toolchain (jq + PyYAML)
         # This reusable parses the caller's hanzo.yml with python3 + PyYAML and
         # slices JSON with jq. A minimal runner image

bin/ignoretracked

@@ -0,0 +1,118 @@
+#!/usr/bin/env bash
+# ignoretracked — refuse a repo that ships bulk content its own .gitignore
+# claims to be ignoring. One implementation, every caller.
+#
+#   ignoretracked [dir]
+#
+# WHAT THIS CATCHES
+#
+# An ignore rule does not untrack what is already tracked. `.gitignore` is
+# consulted when git decides whether to ADD an untracked path; a path already
+# in the index is never reconsidered. So this arrangement is stable and silent:
+#
+#   .gitignore says   native/flags/target/
+#   the index says    855 files under native/flags/target/
+#   git status says   nothing
+#
+# hanzoai/cloud lived there for 6 days and 16 hours. 362 MB of orphaned cargo
+# output — three near-identical 37.8 MB staticlibs and a pile of .rlib — went
+# into every release, took the module past Go's 500 MiB ceiling (see
+# bin/modsize) and made it UNFETCHABLE for nine consecutive releases. The
+# `git archive` of that commit is 202 MB against 12.5 MB today: 16x.
+#
+# The causal story is worth knowing because it is not carelessness, it is a
+# race nobody could see. The ignore rule was REMOVED at 12:22 (the Rust
+# staticlib was being dropped for a Go evaluator), the 855 files were added at
+# 12:51 inside that 4.5-hour window when nothing was ignoring them, and the
+# rule was RE-ADDED at 16:48. Every individual step was reasonable. The state
+# they combined into is the defect, and no tool in the pipeline had an opinion
+# about it.
+#
+# WHY THIS GATES ON BYTES AND NOT ON PRESENCE
+#
+# The obvious gate — "any tracked path matched by .gitignore fails" — is
+# correct in principle and unshippable in fact. Measured across all 845 git
+# repos in the three orgs: 155 of them (18%) carry at least one tracked file
+# their .gitignore matches, and hanzoai/cloud's own origin/main is one of them
+# (a bare, unanchored `tools` pattern on .gitignore:62 catches 27 legitimately
+# tracked Go source files under apps/tools/). `CLAUDE.md` alone is ignored-and-
+# tracked in 40 repos and is the ONLY hit in 29 of them. Turning that on as a
+# hard fail breaks a fifth of the estate on day one, and a gate that fails what
+# ought to pass is a gate that gets switched off — after which we are worse off
+# than before it existed.
+#
+# So the gate is on the quantity that actually caused the outage. The byte
+# distribution separates cleanly, which is why this threshold can be a refusal
+# rather than a warning:
+#
+#     worst legitimate repo in the estate   61.5 MB   (hanzo/docs, vendored)
+#     ... next                              48.8 MB, 45.8, 45.2, 43.8, 39.8
+#     repos over 100 MB                     ZERO
+#     the defect this gate exists for      362.0 MB
+#
+# Default ceiling 100 MB: green on all 845 repos today with 62% headroom over
+# the worst honest case, and red on the real defect by 3.6x. Nothing to
+# baseline, no allowlist, no per-repo exemptions — the tail is REPORTED (so the
+# hygiene problem stays visible and shrinkable) and only the bulk is REFUSED.
+#
+# As repos are cleaned up, lower IGNORETRACKED_MAX_MB. It is a ratchet, and the
+# end state is 0 — at which point this becomes the pure presence gate that was
+# right all along. Do not start there.
+#
+# WHY core.excludesFile IS PINNED OFF
+#
+# `--exclude-standard` reads THREE sources: the repo's committed .gitignore
+# files, .git/info/exclude, and the user's global core.excludesFile. The last
+# is per-machine, so the same commit gets different verdicts on a laptop and a
+# runner — on this workstation a global bare `tags` pattern matched 1,271
+# Elixir source files in lux/explorer-v1 that CI would never flag. A gate whose
+# answer depends on whose machine asked is not a gate. Only committed
+# .gitignore is in scope, so the global file is pinned to /dev/null.
+# (.git/info/exclude is per-clone and is the stock all-comments template on a
+# fresh CI checkout; it cannot be overridden by -c, and is left alone.)
+#
+# EXIT: 0 clean or under the ceiling (findings still reported), 1 over it.
+set -uo pipefail
+
+MAX_MB=${IGNORETRACKED_MAX_MB:-100}
+root=${1:-.}
+cd "$root" 2>/dev/null || { echo "ignoretracked: no such directory: $root" >&2; exit 1; }
+git rev-parse --git-dir >/dev/null 2>&1 || exit 0   # not a repo: nothing to check
+
+tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
+
+# -s gives "<mode> <sha> <stage>\t<path>", which carries the blob id — so the
+# size comes from cat-file and needs no working-tree stat (correct on a bare or
+# sparse checkout, and unaffected by anything a build has written).
+git -c core.excludesFile=/dev/null ls-files -s --cached --ignored --exclude-standard \
+  > "$tmp/ls" 2>/dev/null || exit 0
+[ -s "$tmp/ls" ] || { echo "ignoretracked: OK — nothing tracked that .gitignore ignores"; exit 0; }
+
+files=$(wc -l < "$tmp/ls")
+bytes=$(awk '{print $2}' "$tmp/ls" \
+  | git cat-file --batch-check='%(objectsize)' --buffer 2>/dev/null \
+  | awk '/^[0-9]+$/{s+=$1} END{print s+0}')
+mb=$(awk -v b="${bytes:-0}" 'BEGIN{printf "%.1f", b/1048576}')
+
+# WHICH .gitignore LINE is doing this. Plain `check-ignore` prints nothing for a
+# tracked path (it answers "would git ignore this if it were untracked", and a
+# tracked path short-circuits) — --no-index is what makes it answer for the
+# files we actually have. The output names source:line:pattern, so the report
+# points at the line to edit instead of at 855 paths to read.
+cut -f2- "$tmp/ls" | git -c core.excludesFile=/dev/null check-ignore --no-index -v --stdin \
+  > "$tmp/why" 2>/dev/null || true
+
+echo "ignoretracked: ${files} tracked file(s), ${mb} MB, matched by this repo's own .gitignore"
+if [ -s "$tmp/why" ]; then
+  echo "  by rule (top 10):"
+  awk -F'\t' '{n[$1]++} END{for (k in n) printf "%8d  %s\n", n[k], k}' "$tmp/why" \
+    | sort -rn | head -10
+fi
+
+limit=$(( MAX_MB * 1048576 ))
+if [ "${bytes:-0}" -ge "$limit" ]; then
+  echo "::error::${mb} MB of tracked content is matched by this repo's own .gitignore — over the ${MAX_MB} MB ceiling. An ignore rule does not untrack what is already tracked, so this ships in every release and every module zip while \`git status\` stays clean. Fix: \`git rm -r --cached <path>\` and commit (the working tree is untouched). If the content is meant to be tracked, un-ignore it instead — negate the rule with \`!<path>\` — so the two stop disagreeing."
+  exit 1
+fi
+echo "ignoretracked: OK — ${mb} MB is under the ${MAX_MB} MB ceiling (reported, not blocking)"
+exit 0

bin/modsize

@@ -0,0 +1,123 @@
+#!/usr/bin/env bash
+# modsize — refuse a Go module that is approaching the size at which Go can no
+# longer fetch it. One implementation, every caller.
+#
+#   modsize [dir]        # default: every module in the repo, from the root
+#
+# WHAT THIS CATCHES
+#
+# Go's module ceiling is not a soft limit and not a warning. From
+# golang.org/x/mod/zip (and, identically, cmd/go/internal/modfetch/codehost):
+#
+#   MaxZipFile = 500 << 20     // 524288000 bytes, 500 MiB
+#
+# and it is enforced THREE ways on the same number — the zip file itself, the
+# total uncompressed size of the files inside it, and (in CheckDir) the running
+# sum of the source tree's file sizes:
+#
+#   "module source tree too large (max size is 524288000 bytes)"
+#   "total uncompressed size of module contents too large (...)"
+#
+# hanzoai/cloud crossed it and became UNFETCHABLE for nine consecutive
+# releases. Nothing said so. The module publishes fine — the ceiling is
+# enforced on the CONSUMER, at `go get`, so the failure surfaces in someone
+# else's repo, at a version they did not choose, long after the release that
+# caused it. There is no signal at the publishing end at all, which is why nine
+# releases went out before anybody knew.
+#
+# WHY THIS SUMS RAW BYTES AND WHY THAT IS EXACT, NOT AN APPROXIMATION
+#
+# It would be reasonable to assume the limit is on the compressed zip and that
+# summing uncompressed bytes over-counts. It does not: zip.CheckDir accumulates
+# `info.Size()` — the raw, uncompressed size of each regular file — and fails
+# when that running total exceeds MaxZipFile. So the sum of file sizes IS one
+# of the three quantities Go bounds, and a git blob's size is exactly that
+# file's uncompressed size. This gate therefore measures the same number Go
+# measures, not a proxy for it.
+#
+# WHAT IT EXCLUDES, AND WHY EACH EXCLUSION IS REQUIRED FOR CORRECTNESS
+#
+#   • Nested modules. A subdirectory with its own go.mod is a DIFFERENT module
+#     and its bytes are not in the parent's zip. Counting them would fail a
+#     repo that is nowhere near the limit — hanzoai/s3 is this shape (it builds
+#     s3-rdma-sidecar/ and telemetry/server/ from their own go.mod files), and
+#     a gate that fails a repo that would have worked is a gate people learn to
+#     skip. Each nested module is instead checked on its own terms.
+#   • Symlinks and gitlinks (submodules). zip.CheckDir skips both — symlinks
+#     explicitly (golang.org/issue/27093), submodules because they are not
+#     files. Counting a gitlink's 20-byte entry would be harmless; counting a
+#     symlink's target as content would not.
+#
+# It reads tracked content only (git ls-tree), because that is what a module
+# zip is built from: the proxy serves what the VCS has at that tag, not what a
+# working tree happens to contain.
+#
+# THE THRESHOLD IS BELOW THE CEILING ON PURPOSE
+#
+# Failing AT 500 MiB would be useless — at that point the module is already
+# unfetchable and the only question left is how many releases shipped broken.
+# The gate refuses at MODSIZE_MAX_PCT (default 80%, ~419 MiB), which is the
+# "warn well before" — spelled as a refusal, because a warning about a cliff
+# nobody is watching is the same defect one level up. 105 MiB of headroom is
+# several releases' worth of honest growth, so this fires with room to fix it
+# calmly and never fires the release it would have broken.
+#
+# EXIT: 0 clean, 1 when a module is at or past the threshold.
+set -uo pipefail
+
+# 500 << 20, from golang.org/x/mod/zip. Not a guess and not rounded: the
+# constant is quoted so a reader can check it against the source.
+CEILING=$((500 << 20))
+PCT=${MODSIZE_MAX_PCT:-80}
+REF=${MODSIZE_REF:-HEAD}
+
+root=${1:-.}
+cd "$root" 2>/dev/null || { echo "modsize: no such directory: $root" >&2; exit 1; }
+git rev-parse --git-dir >/dev/null 2>&1 || exit 0   # not a repo: nothing to measure
+
+# Every go.mod in the tree, at REF. The root module is "go.mod"; anything else
+# is a nested module and marks a prefix the parent must not count.
+mods=$(git ls-tree -r --name-only "$REF" 2>/dev/null | grep -E '(^|/)go\.mod$' | sort) || exit 0
+[ -z "$mods" ] && exit 0
+
+# path<TAB>size for every regular tracked file. Mode 100644/100755 only:
+# 120000 is a symlink and 160000 a gitlink, both of which zip.CheckDir omits.
+# `git ls-tree -l` separates the path with a TAB, so split on that rather than
+# on whitespace: a path with spaces in it must survive intact.
+sizes=$(git ls-tree -r -l "$REF" 2>/dev/null \
+  | sed -n 's/^\([0-9]\{6\}\) blob \([0-9a-f]*\) *\([0-9-]*\)\t\(.*\)$/\1\t\3\t\4/p' \
+  | awk -F'\t' '$1=="100644"||$1=="100755"{print $3 "\t" $2}')
+
+human() { awk -v b="$1" 'BEGIN{ printf "%.1f MiB", b/1048576 }'; }
+
+rc=0
+for gm in $mods; do
+  if [ "$gm" = "go.mod" ]; then mdir=""; else mdir="${gm%/go.mod}/"; fi
+  # The prefixes this module must NOT count: every OTHER module nested under it.
+  nested=$(printf '%s\n' "$mods" | while read -r o; do
+    [ "$o" = "$gm" ] && continue
+    od="${o%go.mod}"
+    case "$od" in "$mdir"?*) printf '%s\n' "$od";; esac
+  done)
+  total=$(printf '%s\n' "$sizes" | awk -F'\t' -v m="$mdir" -v nl="$nested" '
+    BEGIN{ n=split(nl, arr, "\n") }
+    {
+      p=$1
+      if (m != "" && index(p, m) != 1) next          # not in this module
+      for (i=1; i<=n; i++) if (arr[i] != "" && index(p, arr[i]) == 1) next  # nested module
+      s += $2
+    }
+    END{ print s+0 }')
+  limit=$(( CEILING * PCT / 100 ))
+  name=${mdir:-./}
+  if [ "$total" -ge "$CEILING" ]; then
+    echo "::error file=${gm}::module ${name} is $(human "$total") of tracked content — PAST Go's $(human $CEILING) module ceiling. It is already unfetchable: \`go get\` fails with 'module source tree too large (max size is ${CEILING} bytes)' in every consumer, at every version that carries it. Find the bulk with: git ls-tree -r -l ${REF} | sort -k4 -n | tail -20"
+    rc=1
+  elif [ "$total" -ge "$limit" ]; then
+    echo "::error file=${gm}::module ${name} is $(human "$total") of tracked content — ${PCT}% of Go's $(human $CEILING) ceiling, $(human $((CEILING-total))) of headroom left. Past the ceiling the module stops being fetchable AT THE CONSUMER, so the break shows up in someone else's build and not in this one. Find the bulk with: git ls-tree -r -l ${REF} | sort -k4 -n | tail -20"
+    rc=1
+  else
+    echo "modsize: OK — ${name} $(human "$total") ($(( total * 100 / CEILING ))% of $(human $CEILING))"
+  fi
+done
+exit $rc