Settings() raised SettingsError constructing ALLOWED_ORIGINS. The field is
declared list[str] and documented "comma-separated in env", but
pydantic-settings decodes complex types from the environment as JSON before
any validator runs, so the plain `a,b,c` that every Deployment manifest, .env
and compose file writes was fatal — not a bad value, an unconstructable
Settings.
That made every route depending on it 500: require_admin builds an IAMClient,
which calls get_settings(). The lux.cloud fleet overview returned HTTP 500 the
moment the admin gate shipped, and the failure was in the gate, nowhere near
the collector it guarded.
NoDecode hands the raw string to one validator that splits it, for both list
settings. One variable, one format, decided in one place.
Co-authored-by: Hanzo Dev <[email protected]>
lux-operator is scaled to 0, so the LuxNetwork CR's image tag is only a
declaration and can be arbitrarily stale. Mainnet's CR says v1.34.0 — a tag
that does not exist in the registry — while all five nodes answer
info.getNodeVersion with luxd/1.36.2; testnet's says v1.32.12 against a
running 1.36.24.
Reporting the CR tag as `version` would put a fabricated number on the
dashboard. `version` is now the observed value and `image` stays as declared,
so a drift between intent and reality is visible instead of hidden.
Co-authored-by: Hanzo Dev <[email protected]>
luxd serves /v1 only — server/http/server.go pins baseURL="/v1" — so every
/ext call in this repo was a silent no-op. The fleet collector's _rpc()
returns None on non-200, so the lux.cloud dashboard rendered nodes_rpc_ok:0,
version:null and every node rpc_reachable:false while the fleet was in fact
running luxd/1.36.2 (mainnet), 1.36.24 (testnet), 1.36.25 (devnet).
Fixing the path alone would not have helped: the collector addressed each
validator by a hardcoded public LoadBalancer IP, and there are now zero
LoadBalancer Services in lux-mainnet/testnet/devnet. Those IPs time out. So
node addressing is no longer configured — it is derived from the cluster on
every collection, and the one way to reach a validator is the lux-k8s API
server's pod proxy, since luxd RPC is cluster-internal.
That collapses the old MODE A / MODE B split into a single path: read the CR
for the declared shape, the pods for phase/readiness/IP, then each pod's
JSON-RPC through the proxy. One credential authenticates both. Deleting the
endpoint ConfigMap and its 15 baked-in IPs removes the failure mode where
infrastructure moves and a dashboard quietly reports zeros.
Probes move to /v1/health/liveness, not /v1/health: a degraded-but-running
validator answers /v1/health with 503, so probing it would restart nodes that
are merely behind. Any fleet from charts/lux could never have gone Ready.
LuxNode.external_ip becomes .ip — with no public per-pod endpoint, the honest
value is the pod IP.
Avalanche endpoints keep /ext; that is their API, not ours.
Measured, with controls: pod-proxy POST /v1/info -> 200 on all 15 pods;
/ext/health -> 404; /zzz/health -> 404; nonexistent pod -> 404. A test asserts
every probe path is under /v1 and fails if /ext returns.
Co-authored-by: Hanzo Dev <[email protected]>
Deny-by-default on the ONE cross-tenant infra route, enforced on both the
standalone fleet_app and the mounted api/fleets router:
unauthenticated -> 401 · invalid token -> 401 · non-admin -> 403 · admin -> 200
Adversarial test exercises the gate end-to-end over real HTTP; only JWKS
verification and the luxd RPC collector are stubbed.
Turn lux.cloud into a live graphical fleet dashboard for the managed luxd
fleet (devnet/testnet/mainnet, 5 validators each on do-sfo3-lux-k8s).
Backend (read-only collector, no mutations, no monolith dependency):
- core/chains/lux_fleet.py: reads the operator LuxNetwork CR (lux.cloud/v1)
for validators/version/plugin-set/phase, lists luxd-N pods + per-pod LB
Services, and queries luxd JSON-RPC per node (info.getNodeVersion/getNodeID/
peers/isBootstrapped, C-Chain eth_blockNumber, P-Chain platform.getHeight),
plus a native D-Chain DEX summary. Aggregates all 3 nets concurrently.
- fleet_app.py: minimal standalone FastAPI exposing GET /v1/fleets/lux/overview
(public, read-only). Reuses the bootnode api image; isolated from the
(currently import-broken) monolith.
- fleets/__init__.py: same overview endpoint also mounted on the main router.
- Uses a READ-ONLY lux-k8s kubeconfig from KMS; only public chain data leaves.
Frontend (Next.js 16, Lux white-label, auto by hostname):
- lib/fleet.ts: typed client + health/status derivations.
- components/lux/fleet-dashboard.tsx: graphical multi-net dashboard — totals,
world topology map, per-network cards (phase, version, plugin-set, peers,
C-Chain height, DEX state, per-chain C/P/X/D bootstrap), expandable per-node
tables (node ID, version, peers, heights). Auto-refresh 15s.
- app/fleet/page.tsx: public /fleet page (the lux.cloud fleet dashboard).
- components/lux/fleet-strip.tsx + page.tsx: live fleet status strip on the
landing hero.
- navbar + dashboard nav: Fleet links; register existing Lux fleet mgmt page.
- brand.ts: add skillsOrg (fixes pre-existing typecheck/build break).
Deploy: infra/k8s/lux-fleet.yaml — lux-fleet Deployment/Service + ExternalSecret
(lux-k8s RO kubeconfig from KMS) + ingress path /v1/fleets/lux/overview on
api.cloud.lux.network.
Verified: collector live against all 3 nets (real heights/peers/versions/DEX);
api + collector tests pass (8/8); web typecheck + production build green;
/fleet renders live data in browser (screenshot).
bootnode is its own GitHub org — its images belong in its own GHCR namespace,
per the never-mix-registries rule. This also removes cross-org friction: the
repo's own GITHUB_TOKEN pushes to ghcr.io/bootnode/* (same org), so once the
arcd App is on the bootnode org, build+push needs no cross-org token.
These two repos share root commit 8999f3c3 ("Initial commit") but
diverged long ago — hanzoai/bootnode evolved the recent stack
(api-go, brand package, etc.) while bootnode-org/main retained the
mainline 2026 history (175 commits we don't have).
Brand-leak fix: this work belongs in github.com/bootnode/bootnode,
not github.com/hanzoai/bootnode. Per the no-cross-brand-leak policy,
merge both histories together and push the result back to
bootnode/bootnode as the canonical home.
After this lands:
- bootnode/bootnode is the canonical repo
- hanzoai/bootnode gets archived with a redirect notice
- @bootnode-io/brand npm package's `repository.url` already points at
bootnode/bootnode (set in the earlier package.json scaffold)
# Conflicts:
# .github/workflows/ci.yml
# api/bootnode/api/__init__.py
# api/bootnode/api/billing/__init__.py
# api/bootnode/config.py
# api/bootnode/core/billing/__init__.py
# api/bootnode/db/models.py
# api/bootnode/main.py
# api/tests/test_billing.py
# web/app/dashboard/infrastructure/page.tsx
# web/app/dashboard/organization/billing/page.tsx
# web/app/dashboard/organization/page.tsx
# web/app/dashboard/page.tsx
# web/app/layout.tsx
# web/components/navbar.tsx
# web/components/unified-navigation.tsx
# web/lib/brand.ts
# web/public/logo/zoo-icon.svg
# web/public/logo/zoo-logo-white.svg
# web/public/logo/zoo-logo.svg
# web/tsconfig.tsbuildinfo
The K8s DATABASE_URL env var uses postgresql:// which makes
create_async_engine try to load psycopg2 (sync driver). Add
_ensure_async_dsn() to convert postgresql:// and postgres:// to
postgresql+asyncpg:// at engine creation time. asyncpg is already
in dependencies.
- Replace eval() with json.loads() in usage tracker (security fix)
- Webhook verification fails-closed in production when secret missing
- Add Commerce env vars to K8s deployment (URL, API key, webhook secret)
- Expose ZAP port 9999 in Dockerfile and K8s service
- Replace Stripe env vars with Square in compose.yml
- Update privacy/terms pages: Stripe→Square via Hanzo Commerce
- Add full billing API client module to web/lib/api.ts (17 endpoints)
- Fix hardcoded localhost URLs in dashboard/analytics/organization pages
- Add production webhook rejection test (57 tests passing)
Updated all container image references across compose, K8s manifests,
cloud deploy scripts, and tests to use the ghcr.io/hanzoai/bootnode
registry format.
zeekayandHanzo Dev