The Rust zoo-operator configures each validator by passing the ZooNetwork CR config map as ZOO_* env vars. luxd (ghcr.io/luxfi/node, CMD ["./luxd"]) does NOT read http-host / bootstrap / track-chains from ZOO_* env — only from real CLI flags. So luxd bound its HTTP/health API to the default 127.0.0.1: the kubelet liveness probe targets the pod IP, never reaches /ext/health, and SIGTERMs the node ~90s after boot — an endless restart loop that kept testnet and devnet validators from ever stabilizing (and would have hit mainnet too). Inject the flags directly onto the operator-rendered StatefulSet via a per-overlay statefulset-patch.yaml, applied by deploy-all.sh right after the kustomize apply (the operator owns the StatefulSet, so it is not in the kustomize set and cannot be patched through the overlay; the operator does not re-render an existing pod template, so the patch persists). - --http-host=0.0.0.0 + --http-allowed-hosts=* so the probe can reach health - --bootstrap-ips/--bootstrap-ids/--network-id/--sybil-protection-enabled as real flags (the ZOO_* env equivalents were silently ignored: nodes booted with 'Adding bootstrap nodes ... count=0') - devnet: drop the bogus --track-chains=zoo (and TRACK_CHAINS env) — luxd rejects the bare 'zoo' alias; the devnet Zoo EVM chain isn't created yet
124 lines
3.9 KiB
Bash
Executable File
124 lines
3.9 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Deploy every service from manifests/services.yml to the target namespace.
|
|
#
|
|
# Usage:
|
|
# ./deploy-all.sh <env>
|
|
#
|
|
# <env> ∈ {dev,test,main} → namespace ∈ {zoo-devnet, zoo-testnet, zoo-mainnet}.
|
|
# Cluster is always do-sfo3-zoo-k8s (set in the manifest).
|
|
|
|
set -euo pipefail
|
|
|
|
ENV="${1:?Usage: deploy-all.sh <dev|test|main>}"
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
ROOT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)"
|
|
MANIFEST="${ROOT_DIR}/manifests/services.yml"
|
|
|
|
if [[ ! -f "${MANIFEST}" ]]; then
|
|
echo "::error::Manifest not found: ${MANIFEST}"
|
|
exit 1
|
|
fi
|
|
|
|
case "${ENV}" in
|
|
main) NS="zoo-mainnet" ;;
|
|
test) NS="zoo-testnet" ;;
|
|
dev) NS="zoo-devnet" ;;
|
|
*) echo "::error::Unknown env '${ENV}' (expect dev/test/main)"; exit 1 ;;
|
|
esac
|
|
|
|
OVERLAY_DIR="${ROOT_DIR}/k8s/${NS}"
|
|
|
|
echo "========================================================"
|
|
echo " Deploying Zoo universe: ${ENV}"
|
|
echo " Namespace: ${NS}"
|
|
echo " Overlay: ${OVERLAY_DIR}"
|
|
echo "========================================================"
|
|
|
|
if [[ -d "${ROOT_DIR}/k8s/crds" ]]; then
|
|
echo "Applying CRDs..."
|
|
kubectl apply -f "${ROOT_DIR}/k8s/crds/"
|
|
fi
|
|
|
|
if [[ ! -d "${OVERLAY_DIR}" ]]; then
|
|
echo "::error::Overlay directory not found: ${OVERLAY_DIR}"
|
|
exit 1
|
|
fi
|
|
|
|
echo "Applying overlay: ${NS}..."
|
|
kubectl apply -k "${OVERLAY_DIR}"
|
|
|
|
# Inject luxd CLI flags onto the operator-rendered zoo-chain StatefulSet.
|
|
# The Rust zoo-operator owns this StatefulSet (it is not in the kustomize set,
|
|
# so it can't be patched via the overlay) and passes config as ZOO_* env vars,
|
|
# which luxd ignores for http-host / bootstrap / track-chains. Without
|
|
# --http-host=0.0.0.0 luxd binds its HTTP/health API to 127.0.0.1 and the
|
|
# kubelet liveness probe (pod IP) can never reach it → ~90s SIGTERM restart loop.
|
|
# The operator does not re-render an existing StatefulSet's pod template, so this
|
|
# patch persists across reconciles; re-running this deploy re-asserts it.
|
|
STS_PATCH="${OVERLAY_DIR}/statefulset-patch.yaml"
|
|
if [[ -f "${STS_PATCH}" ]]; then
|
|
echo "Patching luxd flags onto statefulset/zoo-chain..."
|
|
kubectl -n "${NS}" patch statefulset zoo-chain --type=strategic --patch-file "${STS_PATCH}"
|
|
fi
|
|
|
|
# Patch each manifest image onto its deployment so service-built dispatches
|
|
# take effect without committing image pins into the base.
|
|
python3 - "${MANIFEST}" "${NS}" <<'PYEOF'
|
|
import sys, subprocess
|
|
|
|
manifest_path = sys.argv[1]
|
|
namespace = sys.argv[2]
|
|
|
|
in_services = False
|
|
current_key = ""
|
|
current_image = ""
|
|
|
|
def apply(svc, image):
|
|
if not svc or not image:
|
|
return
|
|
print(f" set image {svc} -> {image}")
|
|
subprocess.run([
|
|
"kubectl", "set", "image",
|
|
f"deployment/{svc}", f"{svc}={image}",
|
|
"-n", namespace
|
|
], capture_output=True)
|
|
|
|
with open(manifest_path) as f:
|
|
for line in f:
|
|
stripped = line.strip()
|
|
if stripped == "services:":
|
|
in_services = True
|
|
continue
|
|
if not in_services:
|
|
continue
|
|
if not stripped or stripped.startswith("#"):
|
|
continue
|
|
indent = len(line) - len(line.lstrip())
|
|
if indent == 2 and stripped.endswith(":"):
|
|
if current_key:
|
|
apply(current_key, current_image)
|
|
current_key = stripped.rstrip(":")
|
|
current_image = ""
|
|
elif indent == 4 and current_key:
|
|
key, _, val = stripped.partition(":")
|
|
val = val.strip().strip('"').strip("'")
|
|
if key == "image":
|
|
current_image = val
|
|
if current_key:
|
|
apply(current_key, current_image)
|
|
PYEOF
|
|
|
|
echo ""
|
|
echo "Waiting for rollouts in ${NS}..."
|
|
kubectl rollout status deployment --all -n "${NS}" --timeout=300s 2>/dev/null || true
|
|
|
|
echo ""
|
|
echo "=== Pod Status ==="
|
|
kubectl get pods -n "${NS}" -o wide 2>/dev/null || true
|
|
|
|
echo ""
|
|
echo "========================================================"
|
|
echo " Deploy complete: ${ENV} (${NS})"
|
|
echo "========================================================"
|