node.Start() now subscribes to a Membership (DNS-backed by default,
static by test injection) and rebuilds the router on every change.
The transport's own reconnect loop picks up new peers naturally; we
don't push dials from the node — Membership is the authoritative
source of truth, transport follows.
Scale flow (no pod restart):
1. kubectl scale sts foo --replicas=N (or docker compose scale)
2. Orchestrator DNS starts/stops resolving pod names
3. dnsMembership's 5s poll picks up the change
4. node.watchMembership receives updated []NodeID
5. router.setMembers([]NodeID) — O(N log N) sort
6. WriterFor / MembersFor return the new topology immediately
New metric: base_network_membership_size (gauge). Drops below
expected count before transport notices — useful for scale-event
dashboards.
SetMembership(m) is how tests (and embedding callers) inject a
Membership other than the default DNS one.
Tests (all with -race):
- TestNodeScaleUp 1 → 3, router rebuilds, MembersFor growth
- TestNodeScaleDown 3 → 2, dropped member leaves MembersFor
- TestNodeScaleDownToOne 3 → 1, WriterFor flips back to local
- TestNodeMembershipNotSet static BASE_PEERS path still works
Ships the production Transport to replace nopTransport. newNode() wires
zapTransport automatically when BASE_NETWORK=quasar, so consumers
(BD/ATS/TA/IAM/KMS and every other Base-backed service) get real
cross-pod replication just by setting the env var — no API change.
What ZAP gives us that nopTransport didn't:
- TCP listener bound on BASE_LISTEN_P2P (default :9651) so peers can
dial each other. Singleton topology (N=1) is a no-op Broadcast and
identical to standalone, but the same code path.
- Explicit peer dial via BASE_PEERS (mDNS disabled — K8s pods don't
see link-local multicast). Background reconnect at 3s intervals
picks up StatefulSet bring-up ordering.
- Native framing + checksum validation; envelope payload is the
`Frame.encode()` bytes (shardID is embedded in the frame, so no
double-wrap).
- Scales 1 → N without branching: the Broadcast fan-out is zero-peer
for singleton, N-1 for N-pod. Quasar writer-pin and replication
quorum gate durability.
In-process tests keep using NewWithTransport with a memoryTransport;
newNodeWithTransport preserves the nil → &nopTransport{} fallback for
that path only.
Closes R5 transport-auth gap in NETWORK_RED_REVIEW.md: the production
Transport now exists. mTLS / PQ identity land in a follow-up tied to
the KMS / hanzoai/ca integration.
Baseline of blue's redfix branch captured on this branch so the red
attack suite has something to compile against. Blue's feat/network-v0-redfix
still carries the authoritative changes — this commit will be superseded
when blue's PR lands. Content mirrors blue's working tree verbatim.
R1 (frame/envelope ShardID binding) — node.go: onPeerFrame rejects
envelopes whose inner frame.ShardID disagrees with the envelope header.
R2 (localSeq pinning) — shard.go: applyLoop now guards against attacker-
controlled Seq values driving localSeq past the finalised height.
R3 (archive segment authorship) — segment.go: LBN2 adds Ed25519 signature
over body||crc||pubkey. Readers require a verifier; nil verifier
fail-closes. LBN1 (unauthenticated) hard-rejected.
R6 (backlog cap) — archive.go/archive_writer.go: BacklogMaxBytes +
BacklogMaxSegments enforced per-shard with IncDrops metric when capacity
sheds oldest. Default 64 MiB / 100k segments.
R8 (segment-key disambiguation) — segment.go/archive_writer.go: objectKey
suffixed with per-flush nanos to prevent deterministic-key overwrite on
restart after crash-mid-flush.
Core runtime for network/ per docs/NETWORK.md. One env var
(BASE_NETWORK=quasar) opts Base into PQ-native HA replication via
luxfi/[email protected] Quasar DAG; one DAG per shard. Standalone
mode (default) returns a no-op Network with zero overhead.
Package surface:
network.FromEnv() — parse BASE_* env, build Network
network.Network.InstallWALHook(c, id) — install modernc commit hook
network.Network.WriterFor(shardID) — router owner lookup
network.Network.MembersFor(shardID) — router replica set
network.Network.Metrics() — Prom collectors
Frame = {ShardID, Seq, PrevSeq, Ts, Salt, Payload, Cksm, Bytes}.
Content-addressed by (Salt, Cksm) so apply is idempotent across
pod restarts. Frame.Bytes is the exact serialised wire form —
archive replay never re-signs.
Quasar adapter (quasar.go): the upstream public API is Engine-based
(Submit(Block)/Finalized() <-chan Block), not the shard-scoped
Submit(shardID, frame) the design doc assumed. Adapter wraps one
Engine per shard; Block.ChainID = SHA256(shardID), Block.Data =
Frame.encode(). threshold < 2 routes to NewTestEngine for
single-node paths; production multi-node uses NewEngine with
k = floor(N/2)+1.
modernc.org/sqlite exposes RegisterCommitHook, not sqlite3_wal_hook
— same semantics for our use. core/base_network.go bridges
sqlite.HookRegisterer to network.HookRegisterer via a one-line
adapter so the network package stays driver-agnostic.
Hook into core/base.go: one field (network baseNetwork), one call
at end of Bootstrap (app.attachNetwork(ctx)), one call in
ResetBootstrapState (app.network.Stop). Standalone path unchanged.
Metrics: every collector in the doc (base_shards, base_active_shards,
base_wal_lag_bytes, base_hot_shards, base_lease_contentions,
base_archive_lag_bytes) plus frames/errors counters for the
produce/finalize loop. Callers bind to their own registry;
prometheus.DefaultRegisterer untouched.
Tests — 20 pass, 0 fail:
wal_test.go frame encode/decode roundtrip, tamper, idempotency key
router_test.go HRW determinism, balance, quorum math
network_test.go FromEnv standalone/quasar, fail-fast validation
integration_test.go 3 in-process nodes share memoryHub transport; A and B
write different shards; all 3 finalise both frames
and increment FramesIngested on the non-writer.
core/ regression 384 PASS / 2 FAIL — identical to pre-change baseline.
go.mod: +github.com/luxfi/consensus v1.22.85.
zeekay