The estate was spread across ten zip versions while the zipdoc/graft refactor
landed at v1.18.22. Measured before bumping: the API is additive across that
range -- 57 exported symbols at v1.17.2, 83 at v1.18.22, with exactly one
removal (`PluginStatus`), and no repo that pins zip references it.
Verified by building this module against the new version, not by assuming.
Co-authored-by: Hanzo Dev <[email protected]>
The official golang images set GOTOOLCHAIN=local, so a go.mod `go` directive
newer than the base image fails the build at the first go command:
go: go.mod requires go >= 1.26.5 (running go 1.26.4; GOTOOLCHAIN=local)
Two orthogonal changes. Pinning the base to the exact patch go.mod names makes
the build hermetic — the versions match, so no toolchain is downloaded.
GOTOOLCHAIN=auto is the resilience half: when go.mod next moves ahead of the
pin, the toolchain is fetched and checksum-verified against sum.golang.org
instead of failing the build outright.
Co-authored-by: Hanzo Dev <[email protected]>
Ingress already described itself correctly: the L7 front door, a real product,
distinct from gateway. The defects were factual.
- Build badge pointed at .github/workflows/build.yaml, which does not exist;
the repo's workflows are ci.yml and workflow-sanity.yml.
- License badge linked blob/master/LICENSE.md — wrong branch (default is main)
and unreachable anyway, since the repo is private. Made it relative.
- Four `kubectl apply -f https://raw.githubusercontent.com/...` lines. All 404
on both master and main because raw.githubusercontent does not serve private
repos. Replaced with `kubectl apply -f k8s/hanzo/` from a clone, and said why.
- The binary download used releases/latest/download/hanzo-ingress_linux_amd64
.tar.gz. Real assets embed the version: hanzo-ingress_v1.7.36_linux_amd64
.tar.gz. The documented URL 404s.
- The IngressClass example claimed name `hanzo` with controller
`hanzo.ai/ingress-controller`. k8s/hanzo/ingressclass.yaml says name
`ingress`, controller `ingress.k8s.io/ingress-controller`. Fixed all six
places, including the `ingressClassName` in the sample Ingress resource —
copying that one would have produced an Ingress no controller claims.
- Added the Traefik lineage. The repo ships Traefik's MIT LICENSE.md and its
entire middleware vocabulary while never naming it; saying so is the honest
direction, and it tells a reader which half is ours.
- Described gateway as the trust boundary rather than "endpoint routing", to
match what gateway's own README and source now say.
Co-authored-by: Hanzo Dev <[email protected]>
luxfi is a public org, and naming it in GOPRIVATE sends Go to direct git and
past the checksum database. That matters here specifically because luxfi
force-re-cuts published tags: the same version gets republished with different
content, so one version legitimately carries two truthful hashes.
Our go.sum holds the proxy/sumdb value. So a direct fetch of a re-cut tag fails
with SECURITY ERROR on whichever module was re-cut most recently, and after that
one is worked around it walks to the next module on the next build — which is
why it reads as an isolated one-module problem rather than a policy error.
Every luxfi module in this go.mod was verified present on proxy.golang.org
before this changed, so nothing loses a resolution path. hanzoai stays private:
those modules genuinely cannot be proxied and keep their token-authenticated
direct fetch.
The doc comment listed a banned-lineage domain as an example tenant
override. It is an example, not a dependency, so it costs nothing to name a
neutral domain — and that lineage must not appear in this tree at all.
Worth noting where it hid: inside a // comment. The security-guards helper that
scans for banned strings pipes through drop_comments, so a scan built on it
scores this line clean. The poison guard added in universe deliberately does not
use that helper.
Co-authored-by: Hanzo Dev <[email protected]>
v1.18.0 added OpScope() to the Router interface and deleted zip/zaprpc; ingress
implements neither and imports neither.
Verified with the repo's own 'make test-unit' on both sides, counted the same
way (the full 'make test' is not runnable here: test-ui-unit builds a Docker
image and test-integration wants a live cluster). Baseline v1.17.2 fails 12,
v1.18.2 fails 11, and the 11 are a strict SUBSET of the 12 — the difference is
TestHTTP30RTT, which is flaky QUIC timing, not a fix. No new failure appears.
The remaining failures (Test_addRoute/Host_IPv6, Test_createConfiguration,
Test_FlushAfterWriteNil/brotli, TestInternalServices, TestNegotiation,
TestWebSocketUpgradeCase) are all pre-existing on an untouched main.
Co-authored-by: Hanzo Dev <[email protected]>
GOPRIVATE switches off the module proxy AND checksum verification together.
Covering github.com/luxfi/* meant every luxfi module was fetched straight from
git and trusted without verification — which is how moved tags got captured into
module caches and served from them indefinitely, making the same commit build on
one machine and fail on another.
Nearly all of luxfi is public and on proxy.golang.org, so it belongs under
normal verification. The private org is lux-private.
Twelve luxfi repos are still private AND still 404 on the proxy, so they are
listed individually rather than by wildcard — coverage now matches what actually
needs a direct fetch, and every other luxfi module is verified against
sum.golang.org again:
bridge broker compliance erc20-go fasthttp graph hsm indexer
mpc ordering password pool
The right end state is those repos moving to lux-private; until then the
explicit list keeps the exemption honest instead of blanket.
Co-authored-by: Hanzo Dev <[email protected]>
[email protected] published a go.mod requiring
vulcand/oxy/[email protected] — the placeholder Go writes
when a replace directive has no resolvable target. Nothing can resolve that, so
the module graph would not load at all. Current cloud requires oxy/v2 v2.2.0
outright.
deps: drop go.sum lines that disagree with the checksum log
Recorded from direct VCS fetches made while GOPRIVATE covered
github.com/luxfi/*, which switches off the module proxy and checksum
verification together. The tags moved afterwards, so these hashes no longer
match what sum.golang.org holds.
proxy.golang.org still serves the originally-published bytes and they verify
against the log, so dropping only the disagreeing lines and re-tidying restores
a verified module graph with no version change. Lines that agree are untouched.
Co-authored-by: Hanzo Dev <[email protected]>
go.mod pins github.com/zap-proto/http v0.3.0, which replaced
`NewTransport(addr)` with `Dial(network, addr)` — mirroring net.Dial so the
network is a value ("tcp", "unix") rather than a family of constructors. The
call site was never updated, so the package does not compile:
pkg/server/service/zap_backend.go:176:15: undefined: zaphttp.NewTransport
This went unnoticed locally because pkg/server/service transitively imports
webui, whose //go:embed static fails before the compiler ever type-checks this
file unless the Vite assets have been built. In a container build the assets
exist, so the Go stage is the first place the breakage surfaces — which is why
it only ever failed in the image build.
Semantics are unchanged: transportFor already keyed one Transport per
"host:port" backend address, which is exactly Dial's tcp case.
Co-authored-by: Hanzo Dev <[email protected]>
163a30c81 removed `"baseUrl": "./src"` on the premise that "every baseUrl
folded here already pointed at the config's own directory, so dropping it moves
nothing". That premise does not hold for webui: its baseUrl pointed at ./src,
not at the tsconfig directory, and it was the only thing making the app's
src-rooted bare imports resolvable — `contexts/version`, `hooks/use-theme`,
`layout/ErrorSuspenseWrapper`, `pages`, and friends.
vite.config.ts resolves those through vite-tsconfig-paths, which reads exactly
this tsconfig. So dropping baseUrl did not just affect `tsc`; it broke the
production bundle:
[vite]: Rollup failed to resolve import "contexts/version" from
"/src/webui/src/App.tsx"
That is stage 1 of the Dockerfile, so it has been failing every ingress image
build since it landed — the binary cannot be produced at all.
TypeScript 7 removes `baseUrl`, so restoring it is not an option. `paths`
targets resolve relative to the tsconfig's own directory when baseUrl is
absent, so `"*": ["./src/*"]` expresses the same resolution in the form both
toolchains accept, and it stays correct as new directories appear under src/
instead of enumerating them.
Co-authored-by: Hanzo Dev <[email protected]>
An exhausted pool is transient back-pressure, but we were reporting it as a
bare `http.Error` 503 with no retry signal and no cache directive, and we were
doing it in four separate places.
WHY this matters operationally: *.hanzo.app routes 163 sites through a single
k8s Service. Every rolling deploy briefly drains that Service to zero ready
endpoints, and with `allowEmptyServices=true` the router still exists, so the
balancer answers. In the last 24h that produced 191 such responses on the
hanzo-app wildcard router (access log: `503 20 ... "-" 0ms` — no upstream, 0ms).
Those responses carried:
- no Retry-After, so clients and probes could not tell "come back in 2s" from
"this is broken", and retried immediately against an already-empty pool;
- no Cache-Control, so any intermediary was free to retain the error. A
cached capacity error outlives the outage that produced it and turns a
two-second deploy gap into a sticky one.
The deeper problem was that "what to say when there is no server" lived in four
balancers (wrr, p2c, hrw, leasttime), each with its own copy of the error value
and its own copy of the write. Four copies is four chances to drift, and a
capacity error that drifts into a success status is exactly how a broken fleet
reports itself healthy.
So: one error value and one writer in the parent loadbalancer package, and all
four strategies funnel through it. Status is written before the body so a
saturated pool can never be mistaken for a healthy one — a status-code monitor
must see 503.
Tests assert 503 + Retry-After + no-store at the writer and through all four
strategies' ServeHTTP; they fail against the previous http.Error behaviour.
Co-authored-by: Hanzo Dev <[email protected]>
TypeScript 7 is the native Go compiler and removes `baseUrl` and
`moduleResolution: node|node10`. Both appear here, so `tsc` from TS7
refuses the config outright (TS5102 / TS5108) and cannot typecheck.
`paths` targets resolve relative to `baseUrl` when it is set and relative
to the tsconfig file otherwise. Every `baseUrl` folded here already
pointed at the config's own directory, so dropping it moves nothing and
the targets are left byte-identical. Where a baseUrl pointed elsewhere,
each affected target was rewritten as join(baseUrl, target).
`moduleResolution` was chosen from the declared `module`: commonjs ->
node16, esnext/preserve -> bundler. Configs whose `module` is unset or
exotic were left alone rather than guessed at.
The result is accepted by BOTH toolchains, so nothing has to upgrade
TypeScript in lockstep. Verified on hanzo/chat packages/api: tsc 5.9
779 -> 778 errors (no regression), and tsc 7.0.2 now runs the project
in 2s where it previously refused the config.
Co-authored-by: Hanzo Dev <[email protected]>
Superseded: the Hanzo GitHub App pushes a webhook, so the forge tracks GitHub
without a per-repo workflow. This file called git.hanzo.ai/api/v1/.../mirror-sync
— a Gitea API for a system we no longer drive — and would sit inert in every repo.
One mechanism, in one place, instead of ~350 copies of a cron.
zip v1.10.0 carries zap-proto/http v0.3.0, where wire headers are length-prefixed
pairs instead of JSON — a break against v1.9.x, so every ZAP service moves together.
`go build ./...` still fails here on webui/embed.go's `pattern static`, which needs
the web assets built first. That fails identically on clean main and is unrelated.
Co-authored-by: Hanzo Dev <[email protected]>
git.hanzo.ai mirrors this repo by PULL on a ~10-minute interval, and arcd runs
CI/CD there — so every push waited out that interval before anything built.
This asks Gitea to pull HEAD immediately.
Latency only: the repo already mirrors via the App webhook, so a missing
HANZO_GIT_TOKEN or a failed curl is non-fatal and never fails the push.
Idempotent (mirror-sync just pulls HEAD) and concurrency-coalesced.
Inherited upstream workflow, repointed at a self-hosted ARC pool at fork time.
ARC is being decommissioned: CI runs inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.
This repo has no .hanzo/workflows today; it is a fork whose upstream CI we do
not rely on. If it needs a build, it gets a native pipeline — not an ARC pool.
Inherited upstream workflow, repointed at a self-hosted ARC pool at fork time.
ARC is being decommissioned: CI runs inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.
This repo has no .hanzo/workflows today; it is a fork whose upstream CI we do
not rely on. If it needs a build, it gets a native pipeline — not an ARC pool.
Inherited upstream workflow, repointed at a self-hosted ARC pool at fork time.
ARC is being decommissioned: CI runs inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.
This repo has no .hanzo/workflows today; it is a fork whose upstream CI we do
not rely on. If it needs a build, it gets a native pipeline — not an ARC pool.
Inherited upstream workflow, repointed at a self-hosted ARC pool at fork time.
ARC is being decommissioned: CI runs inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.
This repo has no .hanzo/workflows today; it is a fork whose upstream CI we do
not rely on. If it needs a build, it gets a native pipeline — not an ARC pool.
Inherited upstream workflow, repointed at a self-hosted ARC pool at fork time.
ARC is being decommissioned: CI runs inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.
This repo has no .hanzo/workflows today; it is a fork whose upstream CI we do
not rely on. If it needs a build, it gets a native pipeline — not an ARC pool.
Inherited upstream workflow, repointed at a self-hosted ARC pool at fork time.
ARC is being decommissioned: CI runs inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.
This repo has no .hanzo/workflows today; it is a fork whose upstream CI we do
not rely on. If it needs a build, it gets a native pipeline — not an ARC pool.
Inherited upstream workflow, repointed at a self-hosted ARC pool at fork time.
ARC is being decommissioned: CI runs inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.
This repo has no .hanzo/workflows today; it is a fork whose upstream CI we do
not rely on. If it needs a build, it gets a native pipeline — not an ARC pool.
Inherited upstream workflow, repointed at a self-hosted ARC pool at fork time.
ARC is being decommissioned: CI runs inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.
This repo has no .hanzo/workflows today; it is a fork whose upstream CI we do
not rely on. If it needs a build, it gets a native pipeline — not an ARC pool.
Inherited upstream workflow, repointed at a self-hosted ARC pool at fork time.
ARC is being decommissioned: CI runs inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.
This repo has no .hanzo/workflows today; it is a fork whose upstream CI we do
not rely on. If it needs a build, it gets a native pipeline — not an ARC pool.
Inherited upstream workflow, repointed at a self-hosted ARC pool at fork time.
ARC is being decommissioned: CI runs inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.
This repo has no .hanzo/workflows today; it is a fork whose upstream CI we do
not rely on. If it needs a build, it gets a native pipeline — not an ARC pool.
Inherited upstream workflow, repointed at a self-hosted ARC pool at fork time.
ARC is being decommissioned: CI runs inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.
This repo has no .hanzo/workflows today; it is a fork whose upstream CI we do
not rely on. If it needs a build, it gets a native pipeline — not an ARC pool.
Inherited upstream workflow, repointed at a self-hosted ARC pool at fork time.
ARC is being decommissioned: CI runs inside git.hanzo.ai (Gitea Actions,
.hanzo/workflows) and delivery is cd.hanzo.ai reconciling the reviewed image
pin in hanzoai/universe. GitHub is a mirror and runs nothing.
This repo has no .hanzo/workflows today; it is a fork whose upstream CI we do
not rely on. If it needs a build, it gets a native pipeline — not an ARC pool.
NewLoggerProvider refused, because the only way to push logs was OTLP and OTLP
carries protobuf and gRPC on either flavour. luxfi/log v1.6.0 added the third
wire — a JSON LogBatch in a ZAP envelope, MsgLogBatch = 3 — so that sentence is
no longer true and the refusal is replaced by the real exporter.
Traces, metrics and logs now all leave this binary the same way.
Metrics chose between an OTLP gRPC and an OTLP HTTP exporter — a choice that
never avoided either dependency, since otlpmetrichttp and otlpmetricgrpc share
otlpconfig. luxfi/metric v1.9.0 adapts the OTel metric SDK onto the ZAP wire
(ResourceMetrics -> MetricFamily), so one newExporter replaces both and the
meter provider is otherwise untouched.
Logs have no ZAP wire: traces ride MsgSpanBatch and metrics ride the
MetricFamily batch, but nothing on the o11y side receives a log batch. OTLP was
the only way to push them, so NewLoggerProvider now refuses with that reason
rather than reaching for it. ingress writes to stdout and the platform collects
it; inventing a log protocol to satisfy a gated experimental flag would be the
wrong trade.
otlp is zero in go.mod. Traces, metrics and logs are all off it.
Setup chose between an OTLP gRPC and an OTLP HTTP exporter. Both pull gRPC and
protobuf, because otlptracehttp and otlptracegrpc share otlpconfig, so the
choice never avoided either — it only picked a flavour.
setupExporter replaces both with the ZAP span exporter from luxfi/trace: a JSON
SpanBatch inside a ZAP envelope, read by o11y/pkg/zapreceiver. The endpoint is
read from whichever config block is set, and trimmed to host:port — a scheme
and path carry no meaning on this wire.
Metrics and logs still export OTLP; those are separate exporters.
Production runs exactly three providers — Kubernetes Ingress, Kubernetes CRD
and File. Everything else was inherited from upstream and carried forward
untouched: Docker, Swarm, Consul Catalog, and the KV config backends, plus
their docs, integration fixtures and compose files.
Removes 111 files across docs/content/{providers,routing,reference,user-guides,
expose}, integration/{resources/compose,fixtures/consul_catalog} and
pkg/config/kv, and updates the README to state the three supported providers
rather than upstream's full matrix.
Dead surface is not free: it is documentation that describes behaviour nobody
here can exercise, integration fixtures that shape CI time, and config code
that has to keep compiling through every refactor.
Verified: go build ./... exits 0 and go vet is clean after the deletions —
pkg/config/kv had no remaining importers.
hanzokv/go v9.22.0 renames its Go package from `redis` to `kv`; the import
path is unchanged. Update the call sites and rename the middleware's own
client vocabulary to match:
redis_limiter.go -> kv_limiter.go
redisLimiter -> kvLimiter
newRedisLimiter -> newKVLimiter
redisPrefix -> kvPrefix (key prefix value "rate:" is unchanged)
Rediser -> KVClient
mockRedisClient -> mockKVClient
Three things deliberately keep the redis name because they are not ours
to rename:
- `redis.call(...)` inside AllowTokenBucketRaw, and the matching
state.SetGlobal("redis", mod) in the test's Lua VM. That is the KV
server's scripting API — a wire contract, not vocabulary.
- dynamic.Redis / config.Redis. That field is CRD schema: it has
generated deepcopy and applyconfiguration code and it is what a
Middleware CR spells today. Renaming it breaks every deployed CR and
needs a codegen pass, so it belongs in its own change.
- pkg/provider/kv/redis rides github.com/kvtools/redis, a foreign
client, and never touched hanzokv/go.
Verified with go vet ./pkg/middlewares/ratelimiter/... (clean) and
CGO_ENABLED=0 go test ./pkg/middlewares/ratelimiter/... (ok, 121s). A
full `go build ./...` cannot run on darwin/arm64 here: luxfi/accel's cgo
fails with 'stdlib.h' file not found, and linking the test binary fails
with ld: library 'resolv' not found. Both are local toolchain gaps that
predate this change.
Co-authored-by: Hanzo Dev <[email protected]>
Five luxfi versions each have TWO different contents recorded across this
workspace, because published tags were moved instead of a new patch being
cut: [email protected], [email protected], [email protected], [email protected], [email protected].
Adjudicated before editing, since "checksum mismatch / SECURITY ERROR" is
also exactly what a real supply-chain attack looks like. It is not one:
sum.golang.org holds the OLD hash, while proxy.golang.org and a direct
fetch BOTH serve the same NEW bytes. Two independent transports agreeing
means nothing is rewriting content in flight — the tag moved at source.
The sumdb entry is a fossil: GOPRIVATE covers github.com/luxfi/* with
GOSUMDB=off, so our own modules never consult the checksum DB and a moved
tag splits consumers silently instead of failing at publish time.
The old bytes are served by nothing now, so a stale pin can never build.
Corrected the recorded hash to the only content that exists, in BOTH line
forms (h1: and /go.mod h1:) — Go reports these one at a time, so a partial
fix just relocates the error.
Deliberately no `go mod tidy` here: this changes no selected version, only
the recorded hash of versions already chosen.
Verified: `go list -m all` resolves with no checksum error.
The durable fix is upstream and NOT in this commit: never move a published
tag, cut x.y.z+1 instead. Moving a tag mutates a value other repos have
already recorded, and with GOSUMDB off no consumer can detect it.
This fork's own code talked to Hanzo KV through upstream go-redis. It now uses
our client, which is a drop-in — same package name and API, only the import path
changes. Pinned to v9.21.1, the first release whose go.mod declares the new
path.
Note: `go mod tidy` cannot complete in this module for an unrelated reason —
github.com/luxfi/[email protected] fails checksum verification, the same upstream
re-tag class as luxfi/[email protected], luxfi/[email protected] and luxfi/[email protected]. That
predates this change.
Every job in validate.yaml failed the checksum-database lookup for
github.com/luxfi/[email protected]. ci.yml:16 exempts github.com/{hanzoai,luxfi,zap-proto}/*
and passes; validate.yaml set no GOPRIVATE at all, so it alone hit the sumdb.
The underlying cause is real and permanent, not a flake: that tag was
force-re-tagged upstream. sum.golang.org holds
h1:gFDyzMJPTLLheMEIeduvc1Qw23awKumXiLjGXhaToO0= while the proxy, the origin and
our go.sum all serve h1:B3R960OXA49XL3DQrhynv57JZivMii5sBClNdCVItwM= for the same
tag and commit 40ba2173ad38. The /go.mod hash is IDENTICAL across both, which is
the signature of a source-content re-tag rather than a different module. The
transparency log is append-only, so the mismatch never ages out.
This aligns the two workflows; it does NOT paper over the re-tag. go.sum is
untouched (the only lines added by v3.7.10 were the four fork entries) and
luxfi/zap still resolves at v0.8.8. The real remedy is upstream: stop mutable
version tags, and pin an immutable pseudo-version.
Co-authored-by: Hanzo Dev <[email protected]>
Everything here is a string a human or a customer could read. Three of
them were also stale enough to be wrong against the current runtime:
- nomad fixtures tagged `traefik.enable=true` while the provider's
default prefix has been `ingress` for some time -- the tags matched
nothing. Now `ingress.*`, consistent with nomad_test's own cases.
- integration rawdata fixtures expected `"entryPoints": ["traefik"]`,
but DefaultInternalEntryPointName is `ingress`.
- the digestauth compose fixture carried htdigest hashes computed over
realm `traefik`, while auth.defaultRealm is `ingress` -- so those
credentials could not have authenticated. Recomputed against the real
realm: md5(test:ingress:test) and md5(test2:ingress:test2).
Also: the CRD fixture that integration actually applies (its
`description:` text reaches customers through `kubectl explain`), the
repo hero image, a test comment, and a cert-generation note. The CRD
descriptions are set to exactly what the (already clean) Go doc comments
generate, so a fixed regeneration is a no-op diff.
LLM.md is rewritten to match: the ownership table now records direct
requires instead of `replace`, and the "intentionally stays" register is
corrected -- the digest-realm entry was stale (that file has used realm
`ingresser` for a while), and the entries that genuinely stay are stated
with the reason they stay: licence attribution, k3s's `--disable=traefik`
flag, upstream Docker Hub image addresses, and one upstream issue URL.
Two remaining leaks are tracked rather than papered over: the stale
`traefik.io_*.yaml` CRD dumps (blocked on a broken code-gen.sh glob) and
`OU=Traefik` in the acme test certificate.
Standing fork rule: a hanzoai fork declares its OWN module path and is
required directly -- never resolved via `replace` against the upstream
path. Both of these violated it, and that is precisely why the upstream
brand was still printed on every import line in pkg/plugins and
pkg/middlewares/grpcweb.
github.com/traefik/yaegi v0.16.1 -> github.com/hanzoai/yaegi v0.16.2
github.com/traefik/grpc-web v0.16.0 -> github.com/hanzoai/grpc-web v0.16.1
The forks now declare those module paths themselves (hanzoai/yaegi
c54ababa, hanzoai/grpc-web 7f097a6), so both `replace` directives are
deleted rather than repointed. Resolution is unchanged -- same source,
same code, one patch bump each to carry the new module line.
go.sum gains only the four new hashes; nothing else is rewritten, so the
pre-existing luxfi/zap checksum entry is untouched.
Verified: go build over the non-webui package set, plus ./pkg/plugins/...
and ./pkg/middlewares/grpcweb/... green.
Ported from upstream Traefik #13227, which this fork predates.
The K8s Ingress provider builds router rules by interpolating the Ingress path
into a rule expression delimited by backticks. Backtick-delimited operands have
no escape sequence, so a path containing a backtick terminated its operand and
the remainder was parsed as rule syntax. A path of
/x`) || Host(`iam.hanzo.ai
turned PathPrefix(`...`) into a second matcher for an unrelated host. The
pathMatcher annotation was worse: it names a bare identifier, so it could not be
quoted at all and was interpolated verbatim.
Both inputs are chosen by whoever creates the Ingress. Kubernetes only validates
that a path begins with "/" and applies no validation to the annotation, so in a
cluster where namespace-scoped users can create Ingresses this let one namespace
attach matchers for hosts and paths owned by another — a cross-tenant traffic
hijack, since routers all land in the same dynamic configuration.
Paths are now interpolated with %q, so an injected backtick stays inert data
inside a quoted operand. The matcher, which cannot be quoted, is restricted to
Path, PathPrefix and PathRegexp; anything else fails the path and is logged
rather than silently shaping the rule. loadRouter returns an error for that,
matching upstream's signature.
The muxer has always accepted both quoted and backtick operands, so this is a
change of emitted syntax only. Two consequences worth knowing:
- Every generated path rule changes textually. Where routers collide, the key
carries a hash of the rule (makeRouterKeyWithHash), so those generated router
names change. Four test expectations pin the new hashes.
- Host operands still use backticks. Hosts are validated by the API server as
RFC 1123 names, which excludes every rule metacharacter, so they are not an
injection vector and are left alone to limit the blast radius of this change.
TestRuleInjection covers both vectors and asserts through the real muxer that
the injected host does not match, that an ordinary path still routes, and that
the three legitimate matchers are still accepted. The muxer assertions run the
request through requestdecorator, without which Host matchers never evaluate and
the test would pass vacuously.
Co-authored-by: Hanzo Dev <[email protected]>
An Ingress selecting this controller via the spec.ingressClassName field was
dropped whole — routers and spec.TLS together — while the same Ingress using
the deprecated kubernetes.io/ingress.class annotation worked. Two ways to say
one thing, with different outcomes.
shouldProcessIngress resolved the field form against the IngressClass objects
discovered by the client, which narrows them to spec.controller ==
IngressClassController. That value is deployment-specific: it defaults to
"ingress.k8s.io/ingress-controller" and is only overridden by
INGRESS_CONTROLLER_NAME, which the production Deployment never set — while the
cluster's IngressClass declares "hanzo.ai/ingress-controller". No class matched,
the discovered set was empty, and every field-form Ingress was disowned. The
annotation form never consulted that set, so it kept working, which is why
manifests were converted to the annotation as a workaround.
An explicit ingressClass option already names the class this provider serves.
It now decides ownership for the field form exactly as it does for the
annotation form, so both select the same Ingresses. Discovery still applies when
no ingressClass is configured, which is what keeps a sibling controller's class
from being claimed.
Two supporting changes:
- filterIngressClassByController is now the single implementation of the
controller narrowing, called by both the live client and the test double. The
double previously returned every IngressClass regardless of spec.controller,
so the controller-matching path — where this bug lived — was never exercised
by any test. Existing fixtures are unaffected; they already declare a
controller this build serves.
- Skipping an Ingress now logs the class it asked for and the class we serve.
The failure was silent, and its visible symptom is a missing certificate,
which points nowhere near class selection.
TestIngressClassSelectionParity reproduces the production topology — two
IngressClasses on non-default controllers, the provider pinned to one, and
Ingresses selecting it both ways — and asserts both forms produce their
certificate. It fails before this change with one router and one certificate
instead of two. It also asserts the sibling gateway controller's Ingress is
still not claimed.
Co-authored-by: Hanzo Dev <[email protected]>
The image path in ci.yml (docker job -> hanzoai/.github/docker-build.yml@main)
has been a 0s startup_failure since inception, and release.yaml builds only
binaries, so no ghcr.io/hanzoai/ingress image has been produced since ~1.7.41.
This adds one self-contained workflow that builds and pushes the image on our
own hanzo-build-linux-amd64 ARC runners: on a v* tag push, and manually via
workflow_dispatch (recovery lane). linux/amd64 to match the deployed image.
No reusable dependency, no deploy step.
- Reject an empty prefix (s3://cdn or s3://cdn/): serving a whole bucket would
expose every other site. newObjectFS fails closed, same as missing creds.
- Endpoint/region come only from the ingress environment (S3_ENDPOINT/S3_REGION),
never from the Middleware — a CR can no longer point the shared credential at
an arbitrary host (SSRF) and the schema carries only root/spaMode/... Removed
the endpoint/region fields from StaticFiles and the CRD schema.
- SPA fallback is gated: a missing asset (non-HTML extension) returns 404 instead
of the 200 HTML shell, so a broken deploy is visible; extensionless client
routes still fall back to the shell.
- Set X-Content-Type-Options: nosniff on served responses (this plane serves
third-party bundles).
- Map errors by cause: object-store outage -> 502 (availability), access-denied
-> 403, missing -> 404/SPA; error responses are not cached.
- Bind object reads to the request context so a client disconnect cancels the
upstream read; the 5-minute backstop remains.
- index.html (the shell) defaults to Cache-Control: no-cache (revalidated via
ETag/Last-Modified); other assets keep the long default / cacheControl override.
Tests: +empty-prefix fail-closed, +SPA asset 404 vs navigation fallback,
+error mapping (502/403/404), +request-context binding, +nosniff, +shell no-cache.
19 unit + 2 shadow green (incl -race). CRD schema slimmed and re-validated.
Root becomes a union: a local path serves from disk (unchanged), and an
"s3://bucket/prefix" URL serves from an object store through the same handler,
so index resolution, spaMode fallback, errorPage404 and cacheControl behave
identically. spaMode stays the only difference between an SPA site and a plain
static site.
- s3.go: an http.FileSystem over hanzos3/go-sdk. Serving delegates to the SDK's
seekable object, so http.ServeContent streams and answers Range requests
without buffering the whole bundle. Keys are joined traversal-safe under the
configured prefix; ETag comes from the object. Credentials come only from the
process environment (AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY); endpoint/region
default from S3_ENDPOINT/S3_REGION. A missing endpoint or credentials fails
the middleware build rather than serving open.
- serveFile (SPA index / 404 page) now opens through the root http.FileSystem
instead of os.Open, so the fallback works over either origin.
- Middleware CRD: model staticFiles (+ endpoint, region) on MiddlewareSpec so a
Middleware CR carrying spec.staticFiles is accepted by the apiserver instead
of pruned. Regenerated the CRD schema block, deepcopy and applyconfig.
Tests: hermetic handler/origin tests over an in-memory store (index, spaMode,
404, cache, ETag, Range, traversal, prefix isolation, fail-closed) plus a
shadow proof (real object store, real bundle, 60 MiB streamed) behind the
`shadow` build tag. Local-origin behavior is covered to prove it is unchanged.
Credentialed-CORS hardening. Two independent bugs in the CORS header
middleware let a credentialed response reach a foreign origin.
header.go:
- anchorOriginRegex wraps every AccessControlAllowOriginListRegex pattern in
^(?:…)$ so the shipped `https://([a-z0-9-]+\.)?hanzo\.ai` can no longer
substring-match the attacker origin `https://hanzo.ai.evil.com`.
regexp.MatchString is unanchored, so the pattern reflected any origin that
merely prefixed the brand. The (?:…) group keeps top-level alternation
intact; pre-existing anchors are stripped so the result never double-anchors.
- Access-Control-Allow-Credentials is now emitted ONLY alongside a matched
Access-Control-Allow-Origin — on both the response path
(PostRequestModifyResponseHeaders) and the preflight path
(processCorsHeaders). It was set unconditionally, so ACAC:true could pair
with a reflected/foreign origin set by another layer (backend or CDN) — the
near-account-takeover class.
zap_backend.go (INGRESS_ZAP_BACKENDS inert by default; pre-enable hardening):
- Cap the buffered request body (io.LimitReader; default 64 MiB, override
INGRESS_ZAP_MAX_BODY_BYTES). Unbounded io.ReadAll let one oversized request
OOM the ingress.
- Honour req.Context() cancellation before dialing.
- Correct the false "PQ-secure" doc: v0.1.0 dials plaintext TCP; note the
absent request-write deadline.
Tests: anchored-regex + credential-gating regressions (attacker suffix, null
origin, allowlisted subdomain) on both response and preflight paths; body-cap
and parseMaxBody unit tests.
Close the #29 framework-ownership tail. The reverse-proxy lib
github.com/vulcand/oxy/v2 was still resolving to upstream github.com/traefik/oxy
via replace; move it onto a hanzoai source-fork, exactly like the yaegi and
grpc-web forks landed in #8.
hanzoai/oxy is a straight source-mirror of traefik/oxy (itself a fork of
vulcand/oxy) pinned to the exact commit previously resolved
(v2.0.0-20260126093803-fb11d60e0fdf). The mirror keeps module
github.com/vulcand/oxy/v2, so it is a drop-in — no import lines change, only
resolution moves to a path hanzoai controls (immune to upstream re-tag/deletion).
- go.mod: drop vulcand/oxy => traefik/oxy from the Containous-forks block;
add vulcand/oxy => hanzoai/oxy to the owned framework-deps block.
- go.sum: swap traefik/oxy -> hanzoai/oxy; tidy also prunes the now-dead
traefik/yaegi + traefik/grpc-web entries (unreferenced once replaced).
- go build ./... = 0; oxy-consuming middleware tests (buffering, auth,
customerrors) pass; go list -m shows all three framework deps => hanzoai/*.
Co-authored-by: Hanzo Dev <[email protected]>
cd9fab39 bumped zap-proto/http v0.0.0->v0.1.0, which replaced the
net/http transport RoundTrip(*http.Request) (*http.Response, error)
with the fasthttp Do(*fasthttp.Request, *fasthttp.Response) error.
pkg/server/service/zap_backend.go still called the removed RoundTrip,
so the whole server half and the hanzo-ingress binary failed to
compile, blocking the next ingress deploy.
Migrate the ZAP-HTTP dispatch to Do: translate the inbound
*http.Request to a *fasthttp.Request (method, origin-form target,
protocol, headers, host, body, declared trailers), run the PQ-secure
round trip via Transport.Do, and translate the filled *fasthttp.Response
back to the *http.Response the RoundTripper contract expects (status,
reason, headers, body, Content-Length via the field, declared trailers
lifted into Response.Trailer, not duplicated into Response.Header).
Header/trailer separation mirrors the ZAP-HTTP codec and net/http's own
Transport, so the reconstructed response matches what the prior net/http
codec produced. Do fully buffers the response and returns its pooled
connection internally, so there is no caller-visible body/connection
lifecycle as there was with the old RoundTrip.
Also migrate zap_backend_test.go's ZAP server handler to a
fasthttp.RequestHandler (v0.1 Server.Handler type) and extend it to
cover GET, a body POST echo, a non-200 status, and a response trailer
so this cannot silently regress.
go build ./... , go vet ./pkg/server/service, and the package tests
(-race) all pass.
Co-authored-by: Hanzo Dev <[email protected]>
zeekayandHanzo Dev