zeekayandHanzo Dev
0bf2051754
`ghcr.io/luxfi/dex` carries a second binary, `gateway`, and nothing ran it, so every quote a Lux interface asked for reached nothing. It runs here as `trade` — not `gateway`, because this namespace already has one in front of the validators' RPC. Two routers on one host and the chain's is the more specific: /v1/chain keeps priority 300 and this takes 200, above the bare-host router in the ingress file provider that has been answering these paths out of hanzoai/cloud. The paths are an allowlist. `PathPrefix(/v1)` would be shorter and would publish /v1/admin/pool/freeze, which the binary authorises on a request header the caller writes; those handlers stay reachable from inside the cluster and from nowhere else, and the header is stripped on the way in as well. 96369 is read through the V3 quoter that landed with the AMM today, and read over the cluster Service rather than back through Cloudflare — same endpoint, 15ms instead of 149ms, and no dependency on the edge that is serving the answer. Co-authored-by: Hanzo Dev <[email protected]>
deploy — the fleet, and the only thing that deploys it
Every service is one values file:
deploy/<namespace>/<name>.yaml
That is the whole inventory. Adding a service is adding a file; there is no
second list to keep in step with it, and pin.sh --which reads the answer to
"who runs this image" out of the directory rather than out of a table.
Each file is values for one shared chart, referenced by version:
oci://oci.hanzo.ai/charts/app:0.1.11
helm template <name> oci://oci.hanzo.ai/charts/app --version 0.1.11 \
-n <namespace> -f deploy/<namespace>/<name>.yaml
What actually runs
The lux-fleet generator and its fence live in hanzo/universe infra/k8s/hanzo-cd/project-lux.yaml; they read this repo from GitHub and deploy to the cluster named lux-cluster.
a git generator over deploy/*/*.yaml, one Application per file, destination
lux-cluster, and a templatePatch that reads each file's own cd.automated:
syncPolicy:
automated:
prune: false
selfHeal: true
19 of the 122 values files set cd.automated: true: the static sites,
each verified answering from the edge on the current cluster. Every pod-based
file is false until it is shown to run there; a pin bump does not opt a file
back in. This file previously said the ApplicationSet "deliberately has no
automated block" and that "there is no CD in this repository yet"; neither has
been true since the templatePatch was added, and reading it that way is how a
hand-edited object gets healed back and nobody knows why.
prune: false throughout. Nothing here deletes: a values file that stops
declaring an object leaves that object running, which is why removing a service
is a deliberate act on the cluster and not a git rm.
Moving an image — pin.sh
deploy/pin.sh <namespace>/<service> <version> # a release
deploy/pin.sh --verify [<service>…] # every pin, against its registry
deploy/pin.sh --which <repository> # who runs this image
deploy/pin.sh --freeze <service>… # a moving tag names bytes
cd.hanzo.ai does not watch the registry — it watches this directory. Publishing
an image changes nothing; moving the pin here is the deploy. .hanzo/workflows/ image-published.yml is the only automated caller, and it does nothing else.
A pin is two fields and one fact. tag is how a human says which release;
digest is what the kubelet pulls. The chart renders repo:tag@digest, and a
reference carrying both resolves by digest — so a tag written from one
registry read and a digest from another can name two different releases, and the
pod then reports the tag while running the digest. pin.sh resolves both from
one HEAD /v2/<path>/manifests/<tag> and writes them in one edit.
That is not hypothetical here. zoo/exchange-api pinned 2.3.34 to a digest
that 2.3.34 no longer resolves to, beside three other deployments of the same
version with no digest at all: four pods, one version number, two sets of bytes.
These files were generated from what is RUNNING
Not from the manifests in k8s/. Those were applied by hand over a long period
and their currency is unknown. The running fleet is the only statement of what is
true, so it is what was read.
Where the two disagreed, k8s/ was the stale one — and where k8s/ held
something these files did not (an IngressRoute, a ConfigMap, a claim, a Role),
that object moved here and the manifest was deleted. See k8s/README.md for the
two objects that could not move and why.
selectorLabels — why most files have one
spec.selector cannot be changed after a workload is created. Most services here
select on app: <name>, not the chart's app.kubernetes.io/name + instance,
so each file records the selector its workload already carries. Without it the
chart would emit its own and the apply would be refused.
It is the selector only — the full label set on metadata is still the chart's.
Verifying, before changing anything
# render every file and diff it against the live object
python3 <universe>/charts/app/hack/diff-live.py --context <ctx> --values deploy
- BREAKING — the apiserver would refuse the apply. Only a workload's
spec.selectorqualifies; it is immutable. - DRIFT — live differs from the file. Someone changed one of them.
- BENIGN — an intended consequence of moving to the chart.
What is NOT here
Chain CRs (LuxChain, LuxNetwork, LuxIndexer, LuxGateway, LuxMPC) are
not services and have their own controllers. They stay in k8s/.
Secrets are not here and never will be. The chart has no way to carry a secret
value: kmsSecrets names a path in KMS and the operator syncs it.
lux-system/s3 is deliberately absent. hack/live2values.py generated one from
Deployment/s3 and both halves were wrong: that Deployment is a 0-replica husk,
and Service/s3 selects app: seaweedfs with a single endpoint on a
StatefulSet — so a sync would have repointed it at a selector matching zero pods
and taken S3 down for every in-cluster consumer. Correcting it means picking one
name for the thing, and renaming a StatefulSet is a recreate against its bound
volume. Until that lands, lux-system S3 is visibly outside CD, which is the
truthful state; a values file claiming otherwise is worse than an absence.
Why the chart is pinned at 0.1.11 and not 0.1.13
One values file does not render under 0.1.13:
deploy/lux-cloud/price.yaml
kms: Additional property hostAPI is not allowed
kms: Additional property credentialsSecret is not allowed
0.1.13 moved KMSSecrets to a ZAP transport: transport + zapAddr replace
hostAPI and the per-CR credential, and the rendered object's group changes from
secrets.lux.network/v1alpha1 to kms.hanzo.ai/v1. That is a cluster-side
change, not a values edit — the lux KMS operator has to serve the new group, and
the ApplicationSet's ignoreDifferences still names the old one. The chart's
default zapAddr is cloud.hanzo.svc:9653, which is a hanzo-cluster address;
Lux's KMS plane is kms.lux.cloud, and what its ZAP endpoint is has not been
established.
So the bump is a migration with a cluster prerequisite, and it is not done here.
The rest of 0.1.13 is already satisfied: image.digest is present wherever the
chart's claim guard requires it, and lux-explorer/lux-faucet no longer carries
its version inside image.repository. Under 0.1.11 all 122 files render.
Pins still to move
pin.sh --verify reads 121 pins and 116 agree with their registry. The five that
do not are credentials rather than defects:
| Pin | Why it cannot be read |
|---|---|
hanzo/lux-admin |
registry.digitalocean.com, no DO registry credential |
lux-devnet/explore-fe-pars, lux-testnet/explore-fe-pars, lux-mainnet/explore-fe-pars |
ghcr.io/parsdao/explore answers 403 to a luxfi token |
About a hundred services still carry a concrete version with no digest. Those are
not ambiguous the way :latest was — the tag names a release — but a registry
tag is mutable, which is how zoo/exchange-api came to be pinned at bytes its
own version no longer meant. The end state is a digest beside every tag, written
by the release that built it. pin.sh <service> <version> does that for any
service whose repo dispatches on publish; --freeze is the interim for one that
does not.