zeekayandHanzo Dev
4d86d17ea1
explorer.testnet.lux.network (502), explorer.devnet.lux.network (404) and both
per-env bridge UIs (TLS handshake failure) are now 200 and render their own app,
verified through a real browser, not a status code.
ROOT CAUSE 1 — `certResolver: letsencrypt` is a no-op on this cluster.
The lux-system hanzo-ingress DaemonSet is started with no
--certificatesresolvers.* argument, so the named resolver does not exist and
every host whose only TLS source was a certResolver was served the self-signed
"INGRESS DEFAULT CERT":
openssl s_client -servername bridge.lux-dev.network
subject=CN=INGRESS DEFAULT CERT issuer=CN=INGRESS DEFAULT CERT
The four bridge hosts had a correct IngressRoute and a healthy 1/1 backend the
whole time — `curl -k` returned 200 and <title>Bridge</title> — so every probe
that skipped verification called them fine. cert-manager (ClusterIssuer
letsencrypt-prod, Cloudflare DNS-01) is the one mechanism here that issues real
certificates, and IngressRoute tls.secretName consumes it (precedent:
lux-ns/exchange-proxy). So: one Certificate per route, tls.secretName on the
route, certResolver deleted.
subject=CN=bridge.lux-test.network issuer=C=US, O=Let's Encrypt, CN=YR2
ROOT CAUSE 2 — explorer.{testnet,devnet}.lux.network were routed to the API.
Neither host appeared in any IngressRoute. Their only route was the legacy k8s
Ingress lux-explorer/explorer-{testnet,devnet} -> ExternalName
explorer-{testnet,devnet}-v2 -> explorer.lux-{testnet,devnet}.svc:8090 — the Go
indexer, which has no `/` handler. Devnet therefore 404'd with the app 1/1 and
"routing fine"; testnet 502'd because that Service additionally had zero
endpoints. Mainnet looked healthy only because a SECOND route,
lux-mainnet/lux-explorer-mainnet, claims explorer.lux.network for the frontend
and outranks the Ingress.
Both hosts are added to their network's existing IngressRoute — the same
frontend-plus-data-path shape mainnet already uses — and the two legacy
Ingresses are deleted. One host, one route, all three networks identical.
ROOT CAUSE 3 — lux-testnet explorer + explore-fe-lux were scaled to 0.
git declared replicas: 1 for explorer and said nothing at all about
explore-fe-lux, so nothing put either back. Scaled to 1; both reached Ready on
the first attempt with no restarts, so the zero was drift, not a crashloop
being papered over. explore-fe.yaml commits the frontend (and its
explore-env-lux ConfigMap) so this cannot silently recur.
Also certified the api-explore.* hosts on the same two routes: they answered
`curl -k .../health` 200 but failed verification for the same reason, and
leaving them out would have been a half-fix.
Rendered, not just 200 (Playwright, page inner text):
explorer.testnet.lux.network "LUX TESTNET Total blocks 981 Total
transactions 971 Wallet addresses 44"
explorer.devnet.lux.network "LUX DEVNET Total blocks 174 Total
transactions 173 Wallet addresses 10"
bridge.lux-test.network "Teleport / From / Source / Connect Wallet"
bridge.lux-dev.network same
No regression: bridge.lux.network, bridge-api/zoo/pars, explore.lux.network,
explorer.lux.network, faucet.lux-test.network all still 200 with valid TLS.
All five files are `kubectl diff -f` clean against live, and lux-testnet's
kustomization now tracks ingress.yaml (its exclusion note described a strip-csp
middleware the live object does not reference) plus the new explore-fe.yaml.
Known, NOT fixed here: the zoo/hanzo/pars devnet+testnet explorer routes carry
the same dead certResolver and their hosts still fail TLS; api-explore.lux-dev.network
is claimed by two routes and the losing one serves it.
Co-authored-by: Hanzo Dev <[email protected]>
104 lines
5.0 KiB
YAML
104 lines
5.0 KiB
YAML
---
|
|
###############################################################################
|
|
# IngressRoutes: Lux Testnet explorer (lux-k8s / lux-testnet)
|
|
#
|
|
# This file was three route-shapes behind live: it still described
|
|
# explore-test.lux.network -> lux-frontend-testnet:3000 and
|
|
# api-explore-test.lux.network -> explorer-testnet:4000, neither of which has
|
|
# existed for months. It is now the verbatim live object plus the forward delta
|
|
# below, so `kubectl diff -f` is clean and kustomization.yaml can track it.
|
|
#
|
|
# Shape (identical to lux-mainnet/lux-explorer-mainnet and
|
|
# lux-devnet/lux-explorer-devnet — one way for all three networks):
|
|
# frontend host + SPA data path (/v1/*, /health, /envs.js, icons) -> explorer:8090
|
|
# frontend host, everything else -> explore-fe-lux:80
|
|
# api host -> explorer:8090
|
|
#
|
|
# FORWARD DELTA (2026-07-26), two defects, both measured:
|
|
#
|
|
# 1. explorer.testnet.lux.network answered 502. Its ONLY route was the legacy
|
|
# k8s Ingress lux-explorer/explorer-testnet -> ExternalName
|
|
# explorer-testnet-v2 -> explorer.lux-testnet.svc:8090, and that Service had
|
|
# zero endpoints because the lux-testnet `explorer` and `explore-fe-lux`
|
|
# Deployments were both scaled to 0. Scaling them back (git declares
|
|
# replicas: 1 for explorer; both came up Ready first try, no crashloop —
|
|
# the zero was drift, not a crash symptom) fixes the 502, but the host would
|
|
# then reach the Go API, which has no `/` handler and 404s — exactly the
|
|
# failure mode devnet was in. So the host is added to the frontend rule
|
|
# here, mirroring explorer.lux.network on mainnet, and the legacy Ingress
|
|
# is deleted. One host, one route.
|
|
#
|
|
# 2. `certResolver: letsencrypt` names a resolver that DOES NOT EXIST. The
|
|
# lux-system hanzo-ingress DaemonSet is started with no
|
|
# --certificatesresolvers.* argument, so the setting is inert and every host
|
|
# that depends on it is served the self-signed "INGRESS DEFAULT CERT"
|
|
# (verified: explore.lux-test.network failed verification while
|
|
# explorer.lux.network, which draws its cert from a cert-manager Ingress,
|
|
# did not). cert-manager + ClusterIssuer letsencrypt-prod (Cloudflare
|
|
# DNS-01) is the one mechanism on this cluster that issues real
|
|
# certificates; tls.secretName below consumes it.
|
|
###############################################################################
|
|
apiVersion: cert-manager.io/v1
|
|
kind: Certificate
|
|
metadata:
|
|
name: explorer-testnet-tls
|
|
namespace: lux-testnet
|
|
spec:
|
|
secretName: explorer-testnet-tls
|
|
issuerRef: { group: cert-manager.io, kind: ClusterIssuer, name: letsencrypt-prod }
|
|
# Every host this IngressRoute matches. The api-explore.* hosts were reachable
|
|
# (`curl -k .../health` -> 200) but unusable from any normal client for the
|
|
# same certResolver reason, so they are certified here too rather than left
|
|
# half-fixed.
|
|
dnsNames:
|
|
- explore.lux-test.network
|
|
- explorer.lux-test.network
|
|
- explore-test.lux.network
|
|
- explorer.testnet.lux.network
|
|
- api-explore.lux-test.network
|
|
- api-explorer.lux-test.network
|
|
- api-explore-test.lux.network
|
|
---
|
|
apiVersion: hanzo.ai/v1alpha1
|
|
kind: IngressRoute
|
|
metadata:
|
|
name: lux-explorer-testnet
|
|
namespace: lux-testnet
|
|
labels:
|
|
app: lux-explorer
|
|
network: testnet
|
|
spec:
|
|
entryPoints:
|
|
- websecure
|
|
tls:
|
|
secretName: explorer-testnet-tls
|
|
routes:
|
|
# SPA data paths on a frontend host -> indexer backend
|
|
- kind: Rule
|
|
priority: 10000
|
|
match: (Host(`explore.lux-test.network`) || Host(`explorer.lux-test.network`) || Host(`explore-test.lux.network`) || Host(`explorer.testnet.lux.network`)) && (PathPrefix(`/v1/`) || Path(`/health`) || Path(`/envs.js`) || Path(`/icon.svg`) || Path(`/logo.svg`) || Path(`/icon-zoo.svg`) || Path(`/icon-lux.svg`) || Path(`/logo-zoo.svg`) || Path(`/logo-lux.svg`))
|
|
middlewares:
|
|
- { name: cors-allow-all }
|
|
services:
|
|
- { name: explorer, port: 8090 }
|
|
# Frontend host -> Lux-brand SPA
|
|
- kind: Rule
|
|
match: Host(`explore.lux-test.network`) || Host(`explorer.lux-test.network`) || Host(`explore-test.lux.network`) || Host(`explorer.testnet.lux.network`)
|
|
services:
|
|
- { name: explore-fe-lux, port: 80 }
|
|
# API host -> backend, scoped paths first
|
|
- kind: Rule
|
|
match: (Host(`api-explore.lux-test.network`) || Host(`api-explorer.lux-test.network`) || Host(`api-explore-test.lux.network`)) && (PathPrefix(`/v1/graph`) || PathPrefix(`/v1/explorer`) || PathPrefix(`/v1/base/realtime`) || PathPrefix(`/v1/indexer`) || Path(`/health`))
|
|
middlewares:
|
|
- { name: cors-allow-all }
|
|
services:
|
|
- { name: explorer, port: 8090 }
|
|
- kind: Rule
|
|
match: Host(`api-explore.lux-test.network`) || Host(`api-explorer.lux-test.network`) || Host(`api-explore-test.lux.network`)
|
|
middlewares:
|
|
- { name: mainpage-rewrite }
|
|
- { name: blockscout-prefix-cchain }
|
|
- { name: cors-allow-all }
|
|
services:
|
|
- { name: explorer, port: 8090 }
|