Say where the mainnet validators actually run, and how to roll them
The five Lux mainnet validators are docker containers on the LAN host evo, started by hand: no compose file, no unit, no script. deploy/lux-mainnet/luxd.yaml addresses a Service on a cluster this estate no longer has, so the only record of the flags they carry was the running processes themselves. scripts/validators.sh carries those flags and rolls the fleet one node at a time, verifying each is back on the version asked for with four peers before touching the next -- sample 5 / quorum 4 means two down is the network. Co-authored-by: Hanzo Dev <[email protected]>
This commit is contained in:
Zach Kelling
co-authored by
Hanzo Dev
Hanzo Dev
parent
5e387e1772
commit
8d70cb136f
@@ -32,17 +32,23 @@ e2e/ Playwright E2E tests (~488 cases)
|
||||
|
||||
## Networks
|
||||
|
||||
All networks are namespaces on the **single** `do-sfo3-lux-k8s` cluster. There
|
||||
is no `lux-test-k8s` or `lux-dev-k8s` cluster — no such kube context exists.
|
||||
The validators run as docker containers on the LAN host `evo`, not in a
|
||||
cluster. `do-sfo3-lux-k8s` is a DigitalOcean context and that account is gone;
|
||||
`deploy/lux-*/luxd.yaml` still addresses
|
||||
`luxd-headless.lux-mainnet.svc.cluster.local:9630`, a Service no cluster here
|
||||
serves. Read those files as the image pin and chain config they carry, not as a
|
||||
description of where anything runs.
|
||||
|
||||
| Network | Network ID | C-Chain ID | Namespace (do-sfo3-lux-k8s) | luxd |
|
||||
|---------|-----------|------------|------------------------------|------|
|
||||
| Mainnet | 1 | 96369 | `lux-mainnet` | 5/5 |
|
||||
| Testnet | 2 | 96368 | `lux-testnet` | 5/5 |
|
||||
| Devnet | 3 | 96367 | `lux-devnet` | 5/5 |
|
||||
Each network runs a 5-replica `luxd` StatefulSet in its namespace. The only
|
||||
declarative source for those StatefulSets is `deploy/lux-{mainnet,testnet,devnet}/luxd.yaml`,
|
||||
reconciled by the matching `lux-*-luxd` CD Application.
|
||||
| Network | Network ID | C-Chain ID | HTTP ports on evo | luxd |
|
||||
|---------|-----------|------------|-------------------|------|
|
||||
| Mainnet | 1 | 96369 | 9730 9740 9750 9760 9770 | 5/5 |
|
||||
| Testnet | 2 | 96368 | 9830 9840 9850 9860 9870 | 5/5 |
|
||||
| Devnet | 3 | 96367 | 9930 9940 9950 9960 9970 | 5/5 |
|
||||
|
||||
Node i answers on 9730+10i, stakes on 9731+10i and keeps its ledger in
|
||||
`/var/lib/luxd/mainnet/data-i`. `scripts/validators.sh` is the recipe: it holds
|
||||
every flag the containers carry and rolls them one at a time, which is the only
|
||||
safe order at sample 5 / quorum 4.
|
||||
|
||||
### app chain EVM Chain IDs
|
||||
|
||||
|
||||
Executable
+148
@@ -0,0 +1,148 @@
|
||||
#!/usr/bin/env bash
|
||||
# The five validators of Lux mainnet, where they actually run, and the one way
|
||||
# to move them to a new luxd.
|
||||
#
|
||||
# They are NOT in a Kubernetes cluster. deploy/lux-mainnet/luxd.yaml describes a
|
||||
# StatefulSet against luxd-headless.lux-mainnet.svc.cluster.local:9630, and no
|
||||
# such Service exists on the k3s cluster this estate has: the DOKS cluster that
|
||||
# file was written for is gone. The validators are five docker containers on the
|
||||
# LAN host `evo`, started by hand, with no compose file, no unit and no script —
|
||||
# which is why finding them took an afternoon. This file is that missing script.
|
||||
#
|
||||
# Ports are positional: node i answers HTTP on 9730+10i and stakes on 9731+10i,
|
||||
# with its ledger at /var/lib/luxd/mainnet/data-i. Every node bootstraps off the
|
||||
# staking ports of the other four. Consensus is sample 5 / quorum 4, so four
|
||||
# must be up to decide anything: roll ONE at a time, and only when the one
|
||||
# before it is back, bootstrapped and connected to the other four.
|
||||
#
|
||||
# scripts/validators.sh list
|
||||
# scripts/validators.sh roll 1.37.1
|
||||
set -euo pipefail
|
||||
|
||||
host=${LUXD_HOST:-evo}
|
||||
root=/var/lib/luxd/mainnet
|
||||
repo=ghcr.io/luxfi/node
|
||||
chain=0x17871 # 96369, Lux mainnet C-Chain
|
||||
|
||||
# index:name — the name is the node's identity, the index is its port and ledger.
|
||||
# Three of the five answered to hanzo-*/zoo-* names for a while; they are five
|
||||
# validators of one network (all mount the same genesis) and are named for it.
|
||||
nodes=(0:lux-mainnet-0 1:lux-mainnet-1 2:lux-mainnet-2 3:lux-mainnet-3 4:lux-mainnet-4)
|
||||
|
||||
http() { echo $((9730 + 10 * $1)); }
|
||||
stake() { echo $((9731 + 10 * $1)); }
|
||||
|
||||
# beacons are the staking ports of every node except this one.
|
||||
beacons() {
|
||||
local self=$1 i out=
|
||||
for i in 0 1 2 3 4; do
|
||||
[ "$i" = "$self" ] && continue
|
||||
out="${out:+$out,}127.0.0.1:$(stake "$i")"
|
||||
done
|
||||
echo "$out"
|
||||
}
|
||||
|
||||
# run recreates node $1 on image $2, keeping its ledger. Every flag the running
|
||||
# fleet carries is here; nothing is defaulted silently.
|
||||
run() {
|
||||
local i=$1 name=$2 image=$3
|
||||
ssh "$host" docker rm -f "$name" 2>/dev/null || true
|
||||
ssh "$host" docker run -d --name "$name" --network host --restart unless-stopped \
|
||||
-v "$root/genesis.json:/genesis/genesis.json:ro" \
|
||||
-v "$root/chainconf:/chainconf:ro" \
|
||||
-v "$root/data-$i:/data" \
|
||||
"$image" /bin/sh -c "'
|
||||
mkdir -p /data/plugins
|
||||
cp /luxd/build/plugins/* /data/plugins/ 2>/dev/null || true
|
||||
exec /luxd/build/luxd --network-id=1 --genesis-file=/genesis/genesis.json \
|
||||
--chain-config-dir=/chainconf --data-dir=/data --db-type=zapdb --plugin-dir=/data/plugins \
|
||||
--staking-tls-cert-file=/data/staking/staker.crt \
|
||||
--staking-tls-key-file=/data/staking/staker.key \
|
||||
--staking-signer-key-file=/data/staking/signer.key \
|
||||
--http-host=0.0.0.0 --http-port=$(http "$i") --http-allowed-hosts=\"*\" --staking-port=$(stake "$i") \
|
||||
--sybil-protection-enabled=true \
|
||||
--consensus-sample-size=5 --consensus-quorum-size=4 \
|
||||
--consensus-preference-quorum-size=4 --consensus-confidence-quorum-size=4 \
|
||||
--consensus-commit-threshold=2 \
|
||||
--network-require-validator-to-connect=true --network-allow-private-ips=true \
|
||||
--bootstrap-nodes=\"$(beacons "$i")\" --skip-bootstrap=false --track-all-chains=true \
|
||||
--public-ip=127.0.0.1 --api-admin-enabled=true --index-enabled=true --log-level=info
|
||||
'" >/dev/null
|
||||
}
|
||||
|
||||
# ask prints "version peers head bootstrapped prefix" for the node on port $1.
|
||||
#
|
||||
# It reads both generations because a roll leaves the fleet mixed. Up to
|
||||
# luxfi/constants v1.6.3 a chain hung off /v1/bc and info was JSON-RPC at
|
||||
# /v1/info; from v1.6.4 the segment is /v1/chain and info is typed operations
|
||||
# under /v1/info/ops. The prefix a node answers on is the fact this rolls for,
|
||||
# so it is reported, not assumed.
|
||||
ask() {
|
||||
local p=$1 v peers head boot prefix
|
||||
v=$(ssh "$host" "curl -s -m 6 http://127.0.0.1:$p/v1/info/ops/node/version" 2>/dev/null |
|
||||
sed -n 's/.*"version":"\([^"]*\)".*/\1/p')
|
||||
if [ -n "$v" ]; then
|
||||
prefix=chain
|
||||
peers=$(ssh "$host" "curl -s -m 6 http://127.0.0.1:$p/v1/info/ops/peers" 2>/dev/null |
|
||||
sed -n 's/.*"numPeers":"\([0-9]*\)".*/\1/p')
|
||||
boot=$(ssh "$host" "curl -s -m 6 'http://127.0.0.1:$p/v1/info/ops/chain/bootstrapped?chain=C'" 2>/dev/null |
|
||||
sed -n 's/.*"isBootstrapped":\([a-z]*\).*/\1/p')
|
||||
else
|
||||
prefix=bc
|
||||
v=$(ssh "$host" "curl -s -m 6 -X POST -H 'content-type: application/json' --data '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"info.getNodeVersion\",\"params\":[]}' http://127.0.0.1:$p/v1/info" 2>/dev/null |
|
||||
sed -n 's/.*"version":"\([^"]*\)".*/\1/p')
|
||||
peers=$(ssh "$host" "curl -s -m 6 -X POST -H 'content-type: application/json' --data '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"info.peers\",\"params\":[]}' http://127.0.0.1:$p/v1/info" 2>/dev/null |
|
||||
sed -n 's/.*"numPeers":"\([0-9]*\)".*/\1/p')
|
||||
boot=$(ssh "$host" "curl -s -m 6 -X POST -H 'content-type: application/json' --data '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"info.isBootstrapped\",\"params\":{\"chain\":\"C\"}}' http://127.0.0.1:$p/v1/info" 2>/dev/null |
|
||||
sed -n 's/.*"isBootstrapped":\([a-z]*\).*/\1/p')
|
||||
fi
|
||||
head=$(ssh "$host" "curl -s -m 6 -X POST -H 'content-type: application/json' --data '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"eth_blockNumber\",\"params\":[]}' http://127.0.0.1:$p/v1/$prefix/C/rpc" 2>/dev/null |
|
||||
sed -n 's/.*"result":"\([^"]*\)".*/\1/p')
|
||||
echo "${v:-down} ${peers:-0} ${head:--} ${boot:-false} ${prefix}"
|
||||
}
|
||||
|
||||
list() {
|
||||
printf '%-16s %-6s %-18s %-6s %-10s %-6s %s\n' NODE PORT VERSION PEERS HEAD BOOT PREFIX
|
||||
local e i name
|
||||
for e in "${nodes[@]}"; do
|
||||
i=${e%%:*}; name=${e##*:}
|
||||
# shellcheck disable=SC2046
|
||||
printf '%-16s %-6s %-18s %-6s %-10s %-6s %s\n' "$name" "$(http "$i")" $(ask "$(http "$i")")
|
||||
done
|
||||
}
|
||||
|
||||
# roll moves every node to $1, one at a time. A node that does not come back on
|
||||
# the new prefix with four peers stops the roll where it stands: four of five is
|
||||
# quorum exactly, so a second unhealthy node is the network.
|
||||
roll() {
|
||||
local want=${1#v} image e i name p line v peers prefix waited
|
||||
image="$repo:v$want"
|
||||
ssh "$host" docker pull -q "$image" >/dev/null
|
||||
for e in "${nodes[@]}"; do
|
||||
i=${e%%:*}; name=${e##*:}; p=$(http "$i")
|
||||
echo "== $name ($p) before: $(ask "$p")"
|
||||
run "$i" "$name" "$image"
|
||||
waited=0
|
||||
while [ "$waited" -lt 300 ]; do
|
||||
line=$(ask "$p"); v=${line%% *}
|
||||
case "$v" in luxd/"$want") break ;; esac
|
||||
sleep 10; waited=$((waited + 10))
|
||||
done
|
||||
line=$(ask "$p")
|
||||
set -- $line; v=$1; peers=$2; prefix=$5
|
||||
echo "== $name after: $line"
|
||||
if [ "$v" != "luxd/$want" ] || [ "$prefix" != chain ] || [ "${peers:-0}" -lt 4 ]; then
|
||||
echo "STOP: $name did not come back as luxd/$want on /v1/chain with four peers" >&2
|
||||
exit 1
|
||||
fi
|
||||
ssh "$host" "curl -s -m 6 -X POST -H 'content-type: application/json' --data '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"eth_chainId\",\"params\":[]}' http://127.0.0.1:$p/v1/chain/C/rpc" |
|
||||
grep -q "$chain" || { echo "STOP: $name does not answer $chain on /v1/chain/C/rpc" >&2; exit 1; }
|
||||
done
|
||||
list
|
||||
}
|
||||
|
||||
case "${1:-list}" in
|
||||
list) list ;;
|
||||
roll) roll "${2:?usage: validators.sh roll X.Y.Z}" ;;
|
||||
*) echo "usage: validators.sh [list | roll X.Y.Z]" >&2; exit 2 ;;
|
||||
esac
|
||||
Reference in New Issue
Block a user