Every request to an object-store root cost a HEAD and a GET to S3, 80 ms at
the median, before a byte left the ingress. One process-wide cache, bounded at
256 MiB and shared by every site, now holds objects up to 2 MiB with their
absence: an entry checked within the last second is served from memory, an
older one costs one HEAD, and the bytes are read again only when the ETag
moved. Sites publish in place, so a publish is served within a second.
Concurrent misses ask the store once; a store that stops answering leaves held
objects serving.
Text is held brotli-encoded beside its bytes and sent that way to a client
that accepts br, as its own representation (ETag "<tag>-br", Vary). Cloudflare
was pulling 557 KB of docs HTML uncompressed across the country per miss.
Everything under /_next/static/ is immutable whatever its extension. Every
middleware shares one S3 client and its connection pool.
Co-authored-by: Hanzo Dev <[email protected]>
OWASP Coraza with the Core Rule Set v4 embedded, declared as `waf` on
every provider: file, labels and the Middleware CR. Rules load Core Rule
Set, then files, then inline directives.
The engine state is written after every rule. The Core Rule Set's own
recommended configuration opens with SecRuleEngine DetectionOnly, so a
state written first is overridden by the rules it governs and the
firewall loads all of CRS and refuses nothing; detectionOnly on the
middleware is the one place the mode is decided.
geoBlock rode along: it was declared in dynamic config only, absent from
MiddlewareSpec, both deep copies and the CRD schema, so a Middleware CR
carrying it produced a middleware without it. The +groupName marker now
matches the registered group, which is what makes controller-gen's output
land where code-gen.sh reads it; the stale traefik.io_*.yaml dumps go.
CI asks for the arm64 runner git.hanzo.ai actually has and builds both
platforms.
The ACME store holds the account key and every leaf private key the edge has
been issued. It was JSON at mode 0600 on a node, which protects it from another
UID on that node and from nothing else. It is now sealed under a key fetched
from KMS at startup and held only in memory, so the file on the node is inert.
pkg/kms speaks the same luxfi/kms contract the gateway's routes loader does, so
the estate has one KMS conversation rather than two.
The list held command strings, so the reusable read .name off a string and
the step died before running anything:
jq: error (at <stdin>:1): Cannot index string with string "name"
Same commands, in the shape the schema declares.
Mount took a cloud.Deps in order to read two fields out of it, Brand and
Domain, and handed both straight back on /_/ingress/config. Nothing else in
that struct was ever touched, and carrying it cost the whole dependency:
hanzoai/cloud ships two editions under one module path, so cloud.Deps is a
different type in each and a package that names it can be composed by exactly
one of them. This repo has already been on both sides of that. Days ago its
go.mod pinned cloud v1.801.293, where Deps has no Logger field and there is no
package-level Register at all; today it pins v0.1.1, where both exist. One file,
two truths, depending on which edition happened to be pinned. So ingress takes
the two strings now and cloud is out of go.mod.
The other half of the coupling was direction. An init() pushed ingress into
cloud.Registry, so the host had to be imported for the subsystem to announce
itself, and the graph closed on itself. App builds and returns its own *zip.App
instead: a host holds what it gets back and mounts it where it likes, and
ingress no longer knows a registry exists. The logger came out of the same
struct and is the process default now, which cannot be nil and so needs no
check.
None of this was ever compiled. It sat behind a `cloud` build tag that no make
target, no CI job and no Dockerfile passed, which is how a file gets to be
valid against one edition of its dependency and dead against the other without
anyone noticing. The tag protected nothing either -- cmd/ingress does not
import this package, and `go list -deps ./cmd/ingress` still shows no zip -- so
it is gone, and `.` joins test-unit and the hanzo.yml test list. A surface that
nothing builds is how this rotted the first time.
ingress was the last core repo still building through .github/workflows/ci.yml
on GitHub Actions, so the thing that terminates TLS for the whole estate was
outside git.hanzo.ai / ci.hanzo.ai / cd.hanzo.ai. Two pipelines for one fleet.
It is also why the single-writer ACME commit (4f3f6d299) had been on main for
hours with no image anywhere: no native build watches this repo, and probing
sha-4f3f6d2, sha-4f3f6d299, 4f3f6d299 and main all returned 'not found'. The
missing image is the sole blocker on enabling SharedStore, which is the durable
fix for the hostPath acme.json divergence that just cost this estate an
88-error Let's Encrypt rate-limit storm.
hanzo.yml declares the one image; .hanzo/workflows/cicd.yml is the canonical
seven-line caller on hanzoai/ci@v1 (v2 is a floating alias for the same commit
as v1.0.35, not a second major). Builds run on the git-runner act_runner pool;
there is no ARC in this fleet.
The Dockerfile needs no extra asset step -- its first stage builds the webui
bundle with pnpm, which is exactly what webui/embed.go's go:embed of
webui/static requires and why a bare local 'go build ./cmd/ingress' fails.
The GitHub test job is ported into hanzo.yml's test: block, webui build
included, for the same reason.
Both GitHub workflows go. ci.yml is superseded. workflow-sanity.yml only ever
triggered on changes to .github/workflows/** and now guards nothing but itself.
The old 'deploy' job is deliberately not recreated: it ran kubectl rollout
restart from a runner with no kubectl and failed every time, and cd.hanzo.ai
reconciles from the universe pin regardless.
Rollout stays a reviewed DIGEST pin in universe --
infra/k8s/ingress/deployment.yaml already pins 1.10.6@sha256:59ec99dd, and
desired matches live exactly. A digest cannot be re-pointed; keep it that way.
Co-authored-by: Hanzo Dev <[email protected]>
zeekayandHanzo Dev