Files
base/network/quasar.go
zeekay 658d7d181d feat(network): Quasar-replicated WAL frames for Base (Phase 1)
Core runtime for network/ per docs/NETWORK.md. One env var
(BASE_NETWORK=quasar) opts Base into PQ-native HA replication via
luxfi/[email protected] Quasar DAG; one DAG per shard. Standalone
mode (default) returns a no-op Network with zero overhead.

Package surface:

  network.FromEnv()                      — parse BASE_* env, build Network
  network.Network.InstallWALHook(c, id)  — install modernc commit hook
  network.Network.WriterFor(shardID)     — router owner lookup
  network.Network.MembersFor(shardID)    — router replica set
  network.Network.Metrics()              — Prom collectors

Frame = {ShardID, Seq, PrevSeq, Ts, Salt, Payload, Cksm, Bytes}.
Content-addressed by (Salt, Cksm) so apply is idempotent across
pod restarts. Frame.Bytes is the exact serialised wire form —
archive replay never re-signs.

Quasar adapter (quasar.go): the upstream public API is Engine-based
(Submit(Block)/Finalized() <-chan Block), not the shard-scoped
Submit(shardID, frame) the design doc assumed. Adapter wraps one
Engine per shard; Block.ChainID = SHA256(shardID), Block.Data =
Frame.encode(). threshold < 2 routes to NewTestEngine for
single-node paths; production multi-node uses NewEngine with
k = floor(N/2)+1.

modernc.org/sqlite exposes RegisterCommitHook, not sqlite3_wal_hook
— same semantics for our use. core/base_network.go bridges
sqlite.HookRegisterer to network.HookRegisterer via a one-line
adapter so the network package stays driver-agnostic.

Hook into core/base.go: one field (network baseNetwork), one call
at end of Bootstrap (app.attachNetwork(ctx)), one call in
ResetBootstrapState (app.network.Stop). Standalone path unchanged.

Metrics: every collector in the doc (base_shards, base_active_shards,
base_wal_lag_bytes, base_hot_shards, base_lease_contentions,
base_archive_lag_bytes) plus frames/errors counters for the
produce/finalize loop. Callers bind to their own registry;
prometheus.DefaultRegisterer untouched.

Tests — 20 pass, 0 fail:
  wal_test.go           frame encode/decode roundtrip, tamper, idempotency key
  router_test.go        HRW determinism, balance, quorum math
  network_test.go       FromEnv standalone/quasar, fail-fast validation
  integration_test.go   3 in-process nodes share memoryHub transport; A and B
                        write different shards; all 3 finalise both frames
                        and increment FramesIngested on the non-writer.
  core/ regression      384 PASS / 2 FAIL — identical to pre-change baseline.

go.mod: +github.com/luxfi/consensus v1.22.85.
2026-04-22 13:39:01 -07:00

126 lines
3.1 KiB
Go

package network
import (
"context"
"crypto/sha256"
"fmt"
"time"
"github.com/luxfi/consensus/protocol/quasar"
)
// quasarEngine is the adapter over github.com/luxfi/consensus/protocol/quasar.
// It narrows the upstream Engine to exactly the surface this package needs:
//
// Submit(shardID, frame) — push a local commit into the DAG.
// Finalized() chan Frame — stream of frames that quorum has acked.
// Members() []NodeID — current voting set for the shard.
// Stop() — graceful shutdown.
//
// One quasarEngine per shard; one Quasar DAG per shard, per the design doc.
type quasarEngine struct {
shardID string
chainID [32]byte
members []NodeID
threshold int
inner quasar.Engine
finalized chan Frame
stop context.CancelFunc
}
// newQuasarEngine builds a Quasar engine scoped to one shard. threshold is
// the number of acks needed (k = ⌊N/2⌋+1 in quorum mode, or just N when
// N ≤ 2). Single-node (N=1) uses NewTestEngine — Quasar requires threshold
// ≥ 2 for production multi-node and we honour that cleanly.
func newQuasarEngine(ctx context.Context, shardID string, members []NodeID, threshold int) (*quasarEngine, error) {
cfg := quasar.Config{QThreshold: threshold, QuasarTimeout: 30}
var (
inner quasar.Engine
err error
)
if threshold < 2 {
inner, err = quasar.NewTestEngine(cfg)
} else {
inner, err = quasar.NewEngine(cfg)
}
if err != nil {
return nil, fmt.Errorf("quasar: %w", err)
}
engCtx, cancel := context.WithCancel(ctx)
if err := inner.Start(engCtx); err != nil {
cancel()
return nil, fmt.Errorf("quasar start: %w", err)
}
q := &quasarEngine{
shardID: shardID,
chainID: sha256.Sum256([]byte(shardID)),
members: append([]NodeID(nil), members...),
threshold: threshold,
inner: inner,
finalized: make(chan Frame, 1024),
stop: cancel,
}
go q.forwardFinalized(engCtx)
return q, nil
}
// Submit wraps a Frame in a quasar.Block and hands it to the engine. The
// Block's ID is the frame checksum so the DAG keys on content.
func (q *quasarEngine) Submit(f Frame) error {
blk := &quasar.Block{
ID: f.blockID(),
ChainID: q.chainID,
ChainName: q.shardID,
Height: f.Seq,
Timestamp: time.Unix(0, f.Timestamp),
Data: f.encode(),
}
return q.inner.Submit(blk)
}
// Finalized returns the stream of frames finalised by quorum.
func (q *quasarEngine) Finalized() <-chan Frame { return q.finalized }
// Members returns the voting set for the shard.
func (q *quasarEngine) Members() []NodeID { return q.members }
// Stop halts the engine and closes the finalized channel.
func (q *quasarEngine) Stop() error {
q.stop()
return q.inner.Stop()
}
func (q *quasarEngine) forwardFinalized(ctx context.Context) {
src := q.inner.Finalized()
for {
select {
case <-ctx.Done():
close(q.finalized)
return
case blk, ok := <-src:
if !ok {
close(q.finalized)
return
}
if blk == nil {
continue
}
f, err := decodeFrame(blk.Data)
if err != nil {
continue
}
select {
case q.finalized <- f:
case <-ctx.Done():
close(q.finalized)
return
}
}
}
}