Hanzo Dev
cb6a7d86fa
zoo-explorer-explorer had restarted 600+ times in both zoo-system and
zoo-devnet. The operator (<=0.1.3) rendered the ZooExplorer CR as
container args --rpc/--chain/--port, but ghcr.io/luxfi/explorer is a Go
binary whose stdlib `flag` set defines only -config/-data/-http/-mdns/
-version. Every start therefore ended in:
flag provided but not defined: -rpc
Usage of explorer: ...
i.e. exit 2 before the listener ever bound. operator 0.1.4
(src/controller.rs::reconcile_explorer) renders the CR into a chains.yaml
ConfigMap and passes --config=/etc/explorer/chains.yaml instead. 62864bd
already pinned 0.1.4 for the readiness fix, so only the rationale is
added here — but that pin is what stops the crash, and the comment should
say so or the next person re-floats the tag.
The crash was hiding two further defects. Both pods now come up Running
and are still useless without these:
* base rpcEndpoint pointed at zoo-chain-rpc.zoo.svc.cluster.local. There
is no `zoo` namespace on do-sfo3-zoo-k8s (`kubectl -n zoo get svc` ->
"No resources found"), so the indexer logs `no such host` every 30s and
has indexed zero blocks since it stopped crashing. `zoo` is not a
blockchain alias either — the EVM answers on /ext/bc/C/rpc. The CR is
applied into zoo-<env>, so the address is namespace-relative;
zoo-testnet/zoo-mainnet keep their own tracked-blockchainID overrides.
* the devnet validator ran without --http-allowed-hosts, so luxd's
localhost-only default 403s every in-cluster caller — the same
eth_chainId POST returns 0x17872 with `Host: localhost` and 403
Forbidden without it. spec.config renders as --kebab-case flags
(operator 0.1.3+), so HTTP_ALLOWED_HOSTS there is the one source of
truth; the unreferenced zoo-devnet/statefulset-patch.yaml is not.
Co-authored-by: Hanzo Dev <[email protected]>
157 lines
5.6 KiB
YAML
157 lines
5.6 KiB
YAML
# zoo-operator — reconciles ZooNetwork / ZooChain / ZooExplorer / ZooGateway.
|
|
#
|
|
# Topology: ONE operator runs in zoo-system, watches every namespace
|
|
# (--namespace ""). zoo-mainnet/zoo-testnet/zoo-devnet overlays scale
|
|
# this Deployment to 0 so only zoo-system holds the lease.
|
|
#
|
|
# OPERATOR_NAMESPACE comes from the pod's own namespace (fieldRef) so
|
|
# the lease lives in zoo-system — see src/main.rs line 106 + src/leader.rs.
|
|
# That fieldRef is load-bearing: main.rs falls back to the literal
|
|
# "zoo-system", so a replica running WITHOUT this env writes zoo-system's
|
|
# lease from whatever namespace it is in. That is exactly how a stale
|
|
# zoo-testnet pod (no env, floating :main) held zoo-system's lease and
|
|
# starved the real operator. Scaling the overlays to 0 is the invariant,
|
|
# not a tidiness preference — apply them or a stray replica survives.
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: zoo-operator
|
|
labels:
|
|
app: zoo-operator
|
|
app.kubernetes.io/name: zoo-operator
|
|
app.kubernetes.io/part-of: zoo
|
|
spec:
|
|
replicas: 1
|
|
selector:
|
|
matchLabels:
|
|
app: zoo-operator
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: zoo-operator
|
|
spec:
|
|
serviceAccountName: zoo-operator
|
|
imagePullSecrets:
|
|
- name: ghcr-secret
|
|
- name: hanzo
|
|
containers:
|
|
- name: operator
|
|
# Pinned to the immutable semver — the floating :main this used to
|
|
# track is what let a stale replica outlive its manifest (see above).
|
|
# 0.1.3 renders the zoo-chain StatefulSet with luxd CLI flags derived
|
|
# from the ZooNetwork CR (earlier images emitted only ZOO_* env, which
|
|
# luxd's viper EnvPrefix ignores). 0.1.4 fixes two more: it unbraids
|
|
# readiness from leadership (/readyz used to 503 on every non-leader,
|
|
# so a healthy standby was NotReady forever and zoo-system sat 0/1 for
|
|
# two days), and it renders the ZooExplorer Deployment with
|
|
# --config=/etc/explorer/chains.yaml. Earlier images passed
|
|
# --rpc/--chain/--port, which luxfi/explorer's stdlib `flag` set does
|
|
# not define (only -config/-data/-http/-mdns/-version), so the binary
|
|
# printed usage and exited 2 on every start — 600+ restarts of
|
|
# zoo-explorer-explorer in zoo-system and zoo-devnet.
|
|
# Bump x.y.z+ on every operator change; never float.
|
|
image: ghcr.io/zooai/operator:0.1.4
|
|
imagePullPolicy: IfNotPresent
|
|
args: ["--namespace", ""]
|
|
env:
|
|
- name: OPERATOR_NAMESPACE
|
|
valueFrom:
|
|
fieldRef:
|
|
fieldPath: metadata.namespace
|
|
ports:
|
|
- containerPort: 8081
|
|
name: health
|
|
- containerPort: 8443
|
|
name: metrics
|
|
resources:
|
|
requests:
|
|
cpu: 10m
|
|
memory: 64Mi
|
|
limits:
|
|
cpu: 500m
|
|
memory: 256Mi
|
|
livenessProbe:
|
|
httpGet:
|
|
path: /healthz
|
|
port: 8081
|
|
initialDelaySeconds: 15
|
|
periodSeconds: 20
|
|
readinessProbe:
|
|
httpGet:
|
|
path: /readyz
|
|
port: 8081
|
|
initialDelaySeconds: 5
|
|
periodSeconds: 10
|
|
---
|
|
apiVersion: v1
|
|
kind: ServiceAccount
|
|
metadata:
|
|
name: zoo-operator
|
|
labels:
|
|
app: zoo-operator
|
|
app.kubernetes.io/part-of: zoo
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: ClusterRole
|
|
metadata:
|
|
name: zoo-operator
|
|
rules:
|
|
- apiGroups: ["zoo.network"]
|
|
resources: ["zoonetworks", "zoonetworks/status", "zoonetworks/finalizers", "zoochains", "zoochains/status", "zooexplorers", "zooexplorers/status", "zoogateways", "zoogateways/status"]
|
|
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
|
|
- apiGroups: [""]
|
|
resources: ["pods", "services", "configmaps", "secrets", "events", "persistentvolumeclaims"]
|
|
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
|
|
- apiGroups: ["apps"]
|
|
resources: ["deployments", "statefulsets"]
|
|
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
|
|
- apiGroups: ["coordination.k8s.io"]
|
|
resources: ["leases"]
|
|
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
|
|
---
|
|
# ClusterRoleBinding is cluster-scoped, so a single object of this name is
|
|
# shared by every overlay. The operator runs with --namespace "" (all
|
|
# namespaces) behind ONE cluster-global leader-election lease, so whichever
|
|
# replica wins the lease — in ANY of these namespaces — must be able to
|
|
# list/watch the zoo.network CRs cluster-wide. Therefore bind EVERY per-env
|
|
# operator ServiceAccount here (one binding, many subjects). Overlays must NOT
|
|
# rewrite a single subject's namespace: that turned this into a
|
|
# last-apply-wins collision that left only one SA bound (the leader's SA was
|
|
# unbound → 403 on every reconcile → /readyz 503).
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: ClusterRoleBinding
|
|
metadata:
|
|
name: zoo-operator
|
|
roleRef:
|
|
apiGroup: rbac.authorization.k8s.io
|
|
kind: ClusterRole
|
|
name: zoo-operator
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: zoo-operator
|
|
namespace: zoo-mainnet
|
|
- kind: ServiceAccount
|
|
name: zoo-operator
|
|
namespace: zoo-testnet
|
|
- kind: ServiceAccount
|
|
name: zoo-operator
|
|
namespace: zoo-devnet
|
|
- kind: ServiceAccount
|
|
name: zoo-operator
|
|
namespace: zoo-system
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: zoo-operator
|
|
labels:
|
|
app: zoo-operator
|
|
app.kubernetes.io/part-of: zoo
|
|
spec:
|
|
selector:
|
|
app: zoo-operator
|
|
ports:
|
|
- port: 8443
|
|
targetPort: 8443
|
|
name: metrics
|