admin_importChain rewrites the chain from a file on disk. It defaulted to
enabled, with a comment saying it was on so that importChain would work, so
every node carried the ability to have its history replaced by anything that
could reach its RPC port -- to spare an operator one line of config during a
restore that happens rarely and is supervised when it does.
A node that is restoring sets admin-api-enabled in its chain config for as long
as the restore takes. Nothing else needs it, and nothing else should offer it.
The tests set the field explicitly and are unaffected.
Co-authored-by: Hanzo Dev <[email protected]>
State sync is skipped when the chain is not far enough ahead of local state,
because bootstrapping would be faster. That trade only exists while
bootstrapping can run at all. It replays blocks against the state beneath them,
so a node missing the state at its own head cannot replay a single block — it
fails to materialize the next one and stops there, whatever the gap is.
On such a node the threshold declines the one mechanism that could repair it,
and small-gap corruption becomes permanent: it cannot execute forward, and it is
not far enough behind to be allowed to sync. The threshold now applies only when
the node holds the state its head is built on. A chain that cannot be read
answers no — the cost of being wrong that way is a sync that was not strictly
needed, against a node that could otherwise never come back.
Co-authored-by: Hanzo Dev <[email protected]>
It was the zero value, so a chain got it only by carrying a per-chain config
file that said so — and most deployments have no chain-config path at all, which
made the one mechanism for rejoining after a deep gap unreachable on whole
fleets. There is no posture where refusing it and re-executing instead is the
right answer, so it is not a decision to hand an operator. The bound is
StateSyncMinBlocks, which lives inside the mechanism: a smaller gap replays and
never reaches this path.
Co-authored-by: Hanzo Dev <[email protected]>
A node was discarding 948 builds every ten minutes on a HEALTHY chain, and had
spent 41 hours doing it on a halted one. The cause was a barrier this fork
dropped.
Upstream holds the builder until the transaction pool has finished resetting onto
the accepted head: `for !needToBuild() || pendingPoolUpdate()`, where
pendingPoolUpdate is chainHeadHash != mempoolHeadHash. We kept the loop and
deleted the second half, along with both hashes and the two setters that maintain
them.
Without it the builder wakes in the window between a block being accepted and the
pool reorganising onto it. The pool still lists what that block just mined, so
needToBuild says there is work; the miner assembles against the new head, finds
nothing left, and returns an empty block; the anti-empty policy discards it.
Nothing changed, so the builder wakes again — immediately, and forever, until the
pool happens to reset.
Restored rather than reinvented: the same two hashes, the same comparison, the
same loop condition, fed by the pool's own reorg event on one side and Accept on
the other. Both are initialised to the current head so a fresh or restarting node
starts converged instead of gated shut.
This also removes proposeEmptyBuild, added earlier today. It built whenever the
pool was non-empty, which is exactly the spin state — it would have turned 948
discarded builds into 948 empty blocks on chain. The barrier removes the
condition it was guarding against, so the guard goes.
Co-authored-by: Hanzo Dev <[email protected]>
An assembled block with no transactions is two conditions wearing one shape. No
demand is the demand-driven property working — discard it, or an idle chain grows
forever. Demand the assembler failed to collect is a bug, and discarding THAT
makes it permanent: no block means no nonce advances, so the pending transaction
is never mined, so the pool never drains, so the condition under which a block
would be built can never return.
The gate read the assembled block, which cannot tell the two apart, so it took
the second for the first. Measured on lux-devnet: the chain stopped at 20:38 UTC
on 2026-08-11 holding ONE executable transaction, sat 41 hours, and behind it the
traffic generator logged 7,100 consecutive failures while luxd-0 and luxd-3
logged errEmptyBlock ~2000 times per six hours. Every node was healthy. Nothing
was wrong with consensus — CERTs gossiped the whole time.
It reads the pool now. Nothing pending is idleness and the block is discarded;
anything pending builds, whatever assembly managed to collect, and logs the
mismatch where somebody can see it. Liveness stops depending on the assembly path
being correct.
The rule moves into proposeEmptyBuild so it can be tested without standing up a
VM, which is why it had no test before.
This does not fix the proximate cause, and is not meant to: the transaction was
in exactly one node's pool (luxd-2) and never reached either proposer, so EVM
transaction gossip is the next thing to look at. What changes here is that such a
failure degrades throughput instead of halting the chain.
Co-authored-by: Hanzo Dev <[email protected]>
The comment claimed Block.Accept passes a batch to sm.Apply(reqs, batch) so the
marker and the shared-memory mutation land in one atomic write. It does not, and
deliberately: a batch cannot cross the plugin's process boundary, atomiczap
refuses one, and passing it made every settling block a fatal Accept.
Accept commits the state first and applies second, which orders a crash toward a
skipped op rather than a replayed one. A comment describing the opposite reads
as a bug on the money path and invites reordering the writes back.
Co-authored-by: Hanzo Dev <[email protected]>
Two defects, both fatal, both on the C<->D atomic settlement path.
1. Accept handed the versiondb batch to sm.Apply unconditionally. The
C-Chain EVM is an out-of-process plugin, so its SharedMemory is the ZAP
client, and that client refuses any batch — a database.Batch cannot
cross a process boundary. Every block staging an atomic op would have
failed Accept, which is fatal. Commit the versiondb first, then Apply
with no batch. Without a shared batch exactly-once is unreachable; the
choice is at-most-once or at-least-once, and replay is unsound here (a
duplicate put is fatal, and after the peer consumes the object it
re-creates it). A skipped op is survivable: C's settlement replay guard
is EVM state, not the Remove, and a skipped intent Put is refunded by
reclaimIntent. The residual single-node cost is stated in the code.
2. The settle path read the object from shared memory DURING EVM
execution while the consuming Remove landed at Accept. A node
re-executing that block — bootstrapping, state-syncing, re-tracing —
found the object gone, computed a reverted receipt where the network
computed a successful one, and died on the receipt root. The seam
settled live and wedged every node that tried to sync past it.
Execution now binds to the object bytes the transaction carries and
declares the consumption in its logs; this adds the block-level rule
that proves the declaration against shared memory at Verify, on both
the consensus path and the producer's own pre-proposal verify, so a
forged object costs a block rather than forking a receipt root. The
check is gated on the bootstrap frontier, which the shared-memory layer
requires of any chain consuming cross-chain state.
Co-authored-by: Hanzo Dev <[email protected]>
The --import-chain-data startup import is re-entered on every boot, because
production pod specs keep the flag set. Once the chain is already at or above
the file's tip, importBlocksFromFile imports 0 blocks and returns
"no blocks imported (parsed=0)"; Initialize treated that as fatal, so the
C-Chain failed to initialise on every restart of an already-imported node —
/v1/bc/C/rpc 404, node contributes no consensus vote.
This was fixed once. Commit 15e75f43 ("startup RLP-import consensus-wedge +
idempotency") landed BOTH halves on hotfix/v1.104.9. Only the consensus-refresh
half was forward-ported to main (it is here as "FIX (v1.104.11)"); the
idempotency half was not, and isNothingToImportError does not exist on main.
Verified: git merge-base --is-ancestor 15e75f43 main => false; the commit is
reachable only from hotfix/v1.104.9 and release/v1.104.9-hotfix.4.
Observed live on lux-testnet 96368 today rolling node v1.36.35, which is built
from evm main: two nodes came back with a dead C-Chain on exactly this error.
The v1.36.24 plugin binary in the same cluster still contains the string
"Startup import: chain already at/above import tip; nothing to import"; the
v1.36.35 binary does not.
Restores isNothingToImportError and the guard, unchanged in meaning from
15e75f43: idempotent ONLY when the chain already holds accepted content
(curHead > 0). On a fresh chain the same messages still mean a genuinely
empty or corrupt file and stay fatal.
startup_import_idempotency_test.go locks the contract, both RLP shapes plus the
fresh-chain safety case. Red without the fix (undefined: isNothingToImportError,
build failed); green with it:
--- PASS: TestStartupImportIdempotency_AlreadyAtTip (0.02s)
ok github.com/luxfi/evm/plugin/evm 1.260s
Co-authored-by: Hanzo Dev <[email protected]>
luxd pins baseURL="/v1" (node/server/http/server.go:33) and has never
served /ext, so both exported constructors — plugin/evm/client.NewClient
and warp.NewClient — built a path no node answers. They are the seam
every out-of-tree caller goes through: cli/pkg/utils/staking.go,
sdk/validatormanager/uptime.go and netrunner/api all hand them a bare
node URI, and staking.go's own SplitRPCURI already parsed /v1/bc/<id>/rpc.
Those callers were already on /v1; this package was the lagging outlier,
so fixing it here fixes all of them without touching one.
Also corrects the simulator's default websocket endpoint, the hardhat
default, and the docs — including api.testnet.lux.network, which does not
serve RPC at all. The testnet host is api.lux-test.network (chainId
0x17870). Prefix-only sweeps miss that class of error.
Tests pin the constructed path against an httptest server, with both an
alias and a blockchain ID, so a future prefix change fails loudly instead
of silently 404ing in production.
Co-authored-by: Hanzo Dev <[email protected]>
The first cut registered the cevm checkpoint store over init.DB before
initializeDBs ran. That is wrong in standalone-database mode: initializeDBs points
the chain state at a separate standalone DB (vm.db), so checkpoints would land in
the base avaDB, not alongside the chain state. It also predated vm.db being set.
Fix: register AFTER initializeDBs, over vm.db — the RAW durable base DB
(NOT the versiondb), standalone-DB-aware (vm.db == the standalone chain-state DB
when UseStandaloneDatabase is set). This is exactly the pattern warpDB / validatorsDB
use (prefixdb.New(prefix, db) over the raw base) for state that need not commit
atomically with the last accepted block: cevm checkpoints now persist on WRITE and
survive a restart, with no versiondb pending-commit dependency — the durability gap
noted when the feature landed.
Still opt-in (cevm-resident-store, default false), inert without -tags cevm, and
safe by construction (a missing/stale checkpoint falls back to the Go-dump seed;
every root is verified). Default plugin build green; resident lazy-reload test green.
Co-authored-by: Hanzo Dev <[email protected]>
Wires StateLoadFromLazy into the resident applier so the C-Chain, on restart,
resumes cevm from a persisted checkpoint (bounded-RAM lazy load) instead of dumping
the full Go state into cevm — the "own disk-backed state" evolution the seed-cap
comment names. Removes the post-restart declined-too-large seed dump.
- core/parallel/cevm_resident_store.go: SetCevmResidentStore(db) (the VM registers a
durable KV; nil = off, unchanged behavior). tryLazyLoadCheckpoint pages the account
trie from disk via cevmbridge.StateLoadFromLazy and sets the resident to the
checkpoint height; persistCheckpoint writes StatePersist records + the height at a
bounded cadence (default 4096 blocks).
- cevm_shadow.go: getResident tries the checkpoint before a fresh handle;
persistCheckpoint runs after a block's cevm root is VERIFIED against the header.
- plugin/evm: CevmResidentStore config flag (default false) → registers a "cevmstate"-
prefixed sub-DB before chain init.
- cevmbridge/bridge_stub.go: StateLoadFromLazy no-op for the non-cevm build.
Safe by construction + triple-gated: nothing runs unless built -tags cevm, the VM
registered a store, AND a checkpoint exists; and the applier still verifies EVERY
cevm root against the header, so a stale/wrong checkpoint only forces a resync (a
Go-dump seed next block), never a wrong commit. nil store (the default) is the prior
in-memory, seed-from-Go behavior byte-for-byte.
Proof (cevm_resident_store_cevm_test.go, //go:build cevm): seed + apply a block,
persistCheckpoint, simulate restart (free handles, retain store), getResident
LAZY-LOADS the checkpoint (height + root recovered, no Go dump), then the next block
applies to a byte-identical FULL root vs a full-reseed reference. Full parallel cevm
suite green; default (non-cevm) plugin build unchanged.
Co-authored-by: Hanzo Dev <[email protected]>
The startup --import-chain-data path (vm.go, after importBlocksFromFile) advanced
core.BlockChain + acceptedBlockDB to the imported tip but never refreshed the
in-memory consensus wrapper vm.State (components/chain), which initChainState had
built at the pre-import genesis tip. On the FIRST import boot the consensus engine
then reads genesis while the EVM head is the imported tip and the proposervm wedges
("BuildBlock: preferred block not fetchable", preferred=empty id) until a restart
re-runs initChainState off acceptedBlockDB.
admin_importChain runs on a LIVE node whose vm.State is already current, so only the
startup path needs this. Now syncs vm.State via the existing, boot-safe
chain.State.SetLastAcceptedBlock, fail-closed (a boot that cannot establish the
imported tip stops loudly rather than serving consensus at a stale genesis).
Empirically proven on lux-testnet 96368 / node v1.36.17 (luxfi/evm v1.104.9): first
import boot wedged at tip 251, healed only on restart; with this the first boot
finalizes past the import tip with no restart. Imported-state execution / stateRoot
unchanged.
Co-authored-by: Hanzo Dev <[email protected]>
The message/validators/warp marshalers are hand-rolled per-type packers
(marshalValue type switch) that ignore struct tags — the linearcodec that read
them is gone. Pure dead metadata. Golden round-trip tests green (byte-identical).
Co-authored-by: Hanzo Dev <[email protected]>
Two-tier finality (Nova = local majority accept, drives VM.Accept; Quasar =
⅔-by-stake export) requires that NO export surface ever names a block above the
served accept tip. RED's final pass approved SHIP with three LOW items to close
first.
LOW-1 — DRY the accept-tip clamp into ONE place. New eth.Ethereum.ClampedQuasarHeight()
= min(LastQuasarHeight, LastAccepted.Number) is the single clamp; BOTH export
surfaces read it: EthAPIBackend.quasarHeight() (RPC finalized/safe) and
warpBlockClient.LastQuasarHeight() (warp export gate, via the BlockClient
interface). Removes the second inlined clamp copy that could drift. LastQuasarHeight()
is documented as RAW (node boot re-seed only, where reconcile has already run).
LOW-2 — exactly ONE admin_importChain. Deleted the eth-package AdminAPI.ImportChain
(which neither reconciled the export height nor skipped <=head blocks); the plugin VM's
(plugin/evm/admin_api.go) is the sole one — it runs under vmLock and calls
reconcileQuasarWithAccept after each SetLastAcceptedBlockDirect. Left a NOTE so it is
not re-added and reopens the rewind hole. Dropped the now-unused rlp import.
LOW-3 — exclude concurrent export-height writers + prove the real boot wiring.
SetLastQuasarFinalized now takes vmLock (the same lock reconcileQuasarWithAccept's
admin_importChain caller holds), so the observer callback's durable Put+Store can never
interleave with a re-import's Reset(0)+Delete on the same key. Fixed a boot ordering
gap: the startup auto-import (importBlocksFromFile) runs AFTER initializeChain's
reconcile, so a --import-chain-data rewind could leave a stale export height — added a
reconcile right after the auto-import. New eth.NewTestEthereum test-support ctor.
Tests (all green): eth -race adds TestClampedQuasarHeight_RealRewind (genuine
SetLastAcceptedBlockDirect rewind; belt clamps finalized/safe to the rewound tip while
raw is unchanged); plugin/evm adds TestBootReconcile_RealLastAccepted_ClearsStaleQuasarOnRewind
(drives the REAL restored LastAcceptedBlock through belt + suspenders) and
TestSetLastQuasarFinalized_LockedAgainstReconcile (concurrency: durable key <-> in-mem
always consistent). gofmt/vet/ontology clean.
Co-authored-by: Hanzo Dev <[email protected]>
Follow-up to the export-boundary re-point, closing RED's re-review MEDIUM (a finality-safety hole
reachable via our own EVM-rewind→re-import restore runbook) and LOW.
MEDIUM — the rewind path. A rewind / RLP re-import (SetLastAcceptedBlockDirect) can lower the accept
tip BELOW the persisted EXPORT-FINAL (Quasar) height; the monotone SetLastQuasarHeight + advance-only
SyncQuasarFrontier then keep it stuck high, and after the chain rebuilds past that height with a
DIFFERENT canonical, `finalized` would name a block that was NEVER ⅔-certified — reported export-final
to every bridge/DEX consumer. Two parts, both needed:
- Belt: api_backend quasarHeight() resolves finalized/safe at min(LastQuasarHeight, LastAccepted) —
finalized/safe can NEVER exceed the served accept tip, even with a stale-high durable key.
- Suspenders: VM.reconcileQuasarWithAccept clears the durable quasarHeightKey + the in-mem height
(eth.ResetLastQuasarHeight, bypassing the monotone guard) when the accept tip is below the export
height — the ONE legal regression of the otherwise-irreversible frontier. Conservative reset to 0
(nothing export-final until the rebuilt chain re-certifies). Called on boot (against the restored
accept tip — catches an offline rewind) and after admin_importChain (the live re-import). On the
next boot the node re-seeds consensus' export frontier from the cleared (0) height, so the
advance-only SyncQuasarFrontier yields to the rewind without touching the four shipped fixes.
LOW — SetLastQuasarFinalized now PERSISTS the durable key BEFORE advancing the in-mem height, so the
durable key is the source of truth: a crash between the two can only leave in-mem at/behind the
durable value on boot, never ahead of it (no monotonicity dip). A persist failure does not advance.
Tests: augmented TestFinalizedSafeResolveToQuasarNeverAboveIt with the belt case (seed export height
ABOVE the accept tip → finalized CLAMPS to the tip — the case RED flagged as false confidence, now
ENFORCED); new TestReconcileQuasarWithAccept_ClearsStaleHeightOnRewind (durable key + in-mem cleared
on an accept regression) and TestSetLastQuasarFinalized_PersistsBeforeAdvancing (persist-first,
no-advance-on-failure). eth suite green under -race.
Co-authored-by: Hanzo Dev <[email protected]>
The two-tier consensus (v1.36) advances VM.Accept at a BARE MAJORITY (Nova), which is reorgable
and can fork under f=1 equivocation. The Ethereum `finalized`/`safe` tags and warp cross-chain
exports were still reading the accept tip — the "semantic collapse" the split exists to prevent.
Re-point every EXPORT surface onto the export-final (Quasar, ⅔-by-stake) frontier consensus
already exposes.
#1 CRITICAL — eth `finalized`/`safe` tags:
- eth.Ethereum gains a DURABLE, monotone lastQuasarHeight (SetLastQuasarHeight/LastQuasarHeight),
pushed by consensus strictly AFTER accept. The plugin VM's SetLastQuasarFinalized persists it to
acceptedBlockDB and reloads it on boot (readLastQuasarHeight), so the export surface never
regresses on restart.
- api_backend HeaderByNumber/BlockByNumber resolve Finalized(-3)/Safe(-4) to GetBlockByNumber(
LastQuasarHeight) BEFORE the IsAccepted() catch-all — NEVER a block above the Quasar height.
`latest`/`pending` stay on the accept tip (correct — not finality tags).
- Test: TestFinalizedSafeResolveToQuasarNeverAboveIt proves finalized == the Quasar height,
monotone, and never above the accepted tip.
#2 HIGH — warp cross-chain export double-spend:
- warp.verifyBlockMessage now requires the block be EXPORT-final (height ≤ BlockClient.
LastQuasarHeight), so a validator signs a block hash only at Quasar finality, never on a
reorgable Nova accept (two conflicting Nova forks could otherwise each aggregate a valid export).
- WarpQuorumNumeratorMinimum floored 33 → 67 (⅔): any warp message needs ≥⅔ stake to verify, so at
most one of two conflicting exports can validate (two ⅔ sets intersect on >⅓ honest stake) —
restoring the ⅔-accept safety the pre-split accept tier gave for free.
The consensus engine pushes the frontier via NetworkConfig.QuasarObserver; the node bridges it into
the VM (separate node commit). Nova-only chains (no export surface) are unaffected.
Co-authored-by: Hanzo Dev <[email protected]>
Three deterministic test reds, all test-only (no production source change):
- plugin/evm/message: regenerate 6 LE golden vectors (utils LP-023 BE→LE
codec migration @v1.2.0; oracle-verified, not tautological) + fix stale
big-endian doc comments in codec.go.
- plugin/evm/validators/state: convert TestParseValidator fixtures BE→LE
(same codec migration) + doc comments.
- core/txpool/legacypool: decouple testTxPoolConfig from the production
DefaultConfig (which was deliberately bumped in fdc840067 for throughput);
pin canonical geth test limits. NOT a regression — production tx-acceptance
is correct; the test baseline was self-contradictory (per-account min >
global max). Also fixes a 600s package timeout → 20s.
Co-authored-by: Hanzo Dev <[email protected]>
rpcChainVM protocol unchanged (42) — node compatibility preserved. Pairs with
the genesis-injection fix (0bd9927a2) for evm module tag v1.104.1.
Co-authored-by: Hanzo Dev <[email protected]>
Forward-integration onto main (v1.103.0 line) of the accept-empty recovery fix
released as v1.101.4. errEmptyBlock moves from SyntacticVerify (parse/verify/
accept) to the build path only, so a consensus-finalized empty block Accepts
while a demand-driven proposer still never produces one.
Co-authored-by: Hanzo Dev <[email protected]>
ROOT CAUSE of the C-Chain bootstrap ancestry-descent stall: ParseBlock was STATEFUL.
BlockGasCost is NOT in the RLP block — it is DERIVED from the parent header. UnmarshalBlock
computed it via vm.blockChain.GetHeaderByHash(parent), and SyntacticVerify REQUIRED it non-nil
(errNilBlockGasCostEVM). For a bootstrap descent parsing ancestry blocks AHEAD of the accepted
height the parent is not present, so every such block failed to parse — masked by the ZAP
errorToZAP mapping as the misleading "malformed block id (does not connect to accepted chain)".
The Lux bootstrapper indexes the whole fetched batch by (id,height,parent) up front, so it could
not index the ancestry, never found the frontier tip in the batch, and re-requested forever ->
ErrStalled. Confirmed on devnet: "syntactic block verification failed: nil blockGasCost is
invalid after EVM activation".
FIX — decomplect parse from verify (matches upstream avalanchego coreth: parse decodes, Verify
enforces connectivity):
- block.go: new ensureBlockGasCost() fills BlockGasCost from the parent header WHEN PRESENT
(SetHeaderExtra is hash-keyed, so it persists for a block first parsed while its parent was
absent). Best-effort at parse; authoritative at Verify (AcceptBootstrapBlock calls blk.Verify
only after the contiguity gate height==finalized+1, so the parent is ACCEPTED/present).
- block_verification.go: SyntacticVerify no longer requires BlockGasCost (stateless,
block-intrinsic checks only); keeps the uint64 sanity check when it IS set.
- vm.go UnmarshalBlock: replace the inline parent-lookup with block.ensureBlockGasCost().
SAFETY (parse != accept): parse only decodes + stateless syntactic checks; no non-connecting or
gas-cost-invalid block can be ACCEPTED just because it now parses. Verify/Accept
(InsertBlockManual -> consensus/dummy header verification) recomputes the expected BlockGasCost
from the (present) parent and rejects a mismatch; the bootstrap contiguity gate accepts blocks
oldest-first (parent finalized). A wrong-hash block is rejected off-path by the content-addressed
descent. BlockGasCost is not in the block hash -> no malleability.
evm plugin/evm tests pass.
Co-authored-by: Hanzo Dev <[email protected]>
Pre-existing idle-builder wake race (not caused by QCv2): on a demand-driven
C-Chain with no empty blocks, a tx arriving while no builder is parked in
waitForNeedToBuild broadcasts pendingSignal to no waiter and is lost; the tx is
then only caught on the NEXT needToBuild() poll, which — depending on the
proposervm slot loop and engine re-call cadence — could leave the first tx after
idle pending for a long/indefinite time (the mainnet treasury-sweep-stuck and
v1.32.1 devnet DEXFill-didn't-complete symptoms).
Fix: startPendingTxPoll — a node-lifetime ticker (builderPollInterval = 500ms)
that periodically broadcasts pendingSignal so any parked builder re-evaluates
needToBuild() against the live mempool. It is decomplected from the tx-submit
subscription (awaitSubmittedTxs remains the primary, immediate wake) so the two
wake sources are independent. It NEVER forces an empty block — a wake only
produces a build when needToBuild() is true — so demand-driven chains stay
empty-block-free, and it changes wake TIMING only, never block CONTENTS, so
validators still build byte-identical blocks (deterministic). needToBuild now
routes through an injectable pendingTxs predicate so the wake path is testable
without a live txpool.
Regression: TestBuilderWakesOnPendingTxAfterIdle reproduces the node bridge's
exact waitForEvent call on an idle builder, delivers a pending tx with NO submit
broadcast, and asserts it returns PendingTxs within a bounded deadline. It FAILS
on the pre-fix code (times out — the lost wakeup) and passes after.
TestBuilderPollBackstopStopsOnShutdown asserts the ticker exits cleanly. Builder
tests green under -race.
Co-authored-by: Hanzo Dev <[email protected]>
Decomplect the build-pacing decision out of waitForEvent into a pure
nextBuildTime(earliestBuild, lastBuildTime) function (no clock read), so the
logic the network-halt fix depends on is deterministically testable. The
wall-clock comparison stays in waitForEvent.
TestNextBuildTime covers all 5 cases (first-build paces to target rate; past
earliestBuild builds immediately; target rate dominates retry floor; retry
floor prevents hot-loop; boundary tie-break). TestNextBuildTime_NeverEarlier
ThanTargetRate locks the invariant the fix relies on: the next build is NEVER
scheduled before parent.Time+TargetBlockRate, so a too-early high-fee block can
never re-introduce the small-tip stall.
Second fresh-net block-production bug, exposed once WaitForEvent was unblocked.
The hand-rolled node notification bridge calls WaitForEvent in a tight loop and
builds a block the instant any tx is pending — with no pacing (ava's proposervm,
which Lux does not wrap, normally provides it). Two consequences:
1. Block gas cost: cost = parentCost + step·(TargetBlockRate − elapsed). A block
built immediately after its parent (elapsed≈0) requires step·TargetBlockRate
(e.g. 200000·2 = 400000) of tips to pass verifyBlockFee. A large deploy
covers it; a single small transfer (~100000) cannot → BuildBlock fails →
the tx never mines.
2. Equivocation: with all 5 validators building instantly off slightly different
mempool snapshots, they produce different blocks at the same height; two can
each gather an α-of-K cert → conflicting finality certs → Logger.Crit → the
node exits(1) and crash-loops.
Fix: pace waitForEvent to parent.Time + TargetBlockRate (the point at which the
block gas cost decays back to the parent's level, 0 from genesis), mirroring the
timing ava's coreth builder applies. Small-tip txs now mine, and the ~rate-long
window lets the mempool gossip-converge so validators build the IDENTICAL block
(same ID, one cert) instead of racing to finalize conflicting blocks. Retry
floor 1ms→100ms so a failed BuildBlock cannot hot-loop.
Fresh-start block-production stall root cause. The node-side notification
bridge (node/chains/manager.go) calls VM.WaitForEvent in a loop with a
non-cancellable context.Background(), starting BEFORE SetState(NormalOp)
initializes vm.builder. The first WaitForEvent therefore parks in the
builder==nil select { ctx.Done | stateSyncDone | shutdownChan } and, since the
context never cancels and state-sync is off, blocks FOREVER -- the serial
bridge loop never re-calls WaitForEvent to observe the now-ready builder, so no
PendingTxs ever reaches the engine and no block is ever built.
ava avoids this with common.NotificationForwarder + cn.OnChange=CheckForEvent,
which cancels the outstanding WaitForEvent and re-subscribes on VM state change.
Lux replaced that with the hand-rolled bridge and lost the recovery path.
Fix (self-contained, VM-side): add builderReady chan, closed once by
onNormalOperationsStarted after vm.builder is set, and add a select case so a
WaitForEvent that parked early re-fetches the ready builder and proceeds.
Makes WaitForEvent correct for any caller, cancellable or not.
go.sum: re-record luxfi/age,pq checksums after upstream re-tag (verify ON).
Enable-everything builder surface (public permissionless launch). The geth-layer
strict-PQ profile (LuxStrictPQ) refused ecrecover / p256Verify / sha256 /
ripemd160 / blake2f / bls12381 / kzg under vm.config.PQ — but ecrecover
(secp256k1) is THE Ethereum signature path every dapp depends on, and the rest
are verify-only / hash, key-safe. Refusing them breaks the builder surface.
The chain now installs NO forbidding pq.Profile: chainConfig.PQ stays nil,
which pq.RefuseUnder(nil) treats as permissive. The generic geth mechanism
(chainConfig.PQ + runPrecompile.RefuseUnder + legacypool nil-check) is kept for
any chain that wants it; Lux installs nothing.
This completes the decomplect started in luxfi/precompile: consensus PQ and the
EVM precompile profile were complected through config.PQ / StrictPQTimestamp.
After precompile dropped contract.RefuseUnderStrictPQ from every verify-only
precompile, the whole strict-PQ projection here was dead. Removed:
- plugin/evm/pq_profile.go (LuxStrictPQ)
- plugin/evm/vm.go: the `if vm.config.PQ { chainConfig.PQ = LuxStrictPQ() }`
install + the unconditional StrictPQTimestamp=0 genesis pin
- plugin/evm/config: the config.PQ bool field (no code writer; was JSON-opt-in)
- params/extras/network_upgrades.go: StrictPQTimestamp field + apply + IsStrictPQ
- precompile/registry/bridge.go + core/precompile_overrider.go: the dead
IsStrictPQ adapters (only existed to feed the deleted StrictPQReporter)
- tests: pq_test.go, strict_pq_activation_test.go, and the two StrictPQ
assertions (+ orphaned helper) in precompile_upgrade_rollout_test.go
Lux's PQ security is enforced at consensus / identity (quasar/p3q), not by
refusing an EVM verifier a dapp asked for. The live lux-mainnet upgrade.json
never actually set strictPQTimestamp, so this matches production.
Build: full module green. Tests: params/extras, plugin/evm/config,
precompile/registry pass.
Rename across the evm warp backend and the warp precompile: warp.Core ->
warp.Message, NewCore -> NewMessage, ParseCore -> ParseMessage, the .Core field
-> .Message, and local core/msgCore/testCore vars -> msg/testMsg. deps: warp
v1.21.0 -> v1.22.0, precompile v0.11.0 -> v0.12.0.
geth's core/types and core/vm imports are untouched — the rename is scoped to
the warp message type, never the geth core package. ./warp/... and
./precompile/contracts/warp/... green, including TestWarpWrongNetworkID.
Track the upstream rename across the evm warp backend
(warp/{service,backend,verifier_backend}) and the warp precompile
(precompile/contracts/warp/{config,contract,contract_warp_handler},
precompileconfig): SignedCore->Core, WarpEnvelope->Envelope.
Identifiers only — the digest D, ZAP c14n bytes and BLS Beam / Corona Pulse /
ML-DSA signing domains are unchanged. ./warp/... and
./precompile/contracts/warp/... green, including the 10k-validator predicate
sign->aggregate->verify.
Hard-fork the last evm warp consumers onto the canonical ZAP message format
(one and one way only). warp v1.20.0 deleted UnsignedMessage; the signed
subject is now warp.SignedCore (D = keccak256(coreDST ‖ zap_c14n)) and the
signed wrapper is warp.WarpEnvelope. The BLS Beam signs warp.BeamSigningBytes(D).
deps: warp v1.19.5 -> v1.20.0, precompile v0.5.59 -> v0.10.0.
- warp/{backend,verifier_backend,service}: UnsignedMessage -> SignedCore;
NewMessage -> NewWarpEnvelope; sign/verify over BeamSigningBytes(core.ID()).
- precompile/contracts/warp: ParseMessage -> ParseWarpEnvelope; .UnsignedMessage
-> .Core; .Signature.(*BitSetSignature) -> .Beam (concrete). VerifyPredicate
verifies the Beam over D with exported primitives (Bits + bls +
BeamSigningBytes) because the precompile quorum denominator must include
keyless validators, which warp.VerifyEnvelope/GetCanonicalValidatorSet reject.
- precompileconfig.WarpMessageWriter.AddMessage(*SignedCore) (mirrors precompile v0.10.0).
- plugin/evm/vm.go: warpVerifierAdapter over *SignedCore; collapse the duplicate
external-warp import alias (warptypes -> luxwarp).
- remove redundant warp/{aggregator,bls_adapter,signer_adapter}.go: provably-dead
duplicates of warp.SignatureAggregator / warp.NewSigner (zero production refs).
- tests migrated to ZAP; fixed a pre-existing offchain message-key test bug;
revived the predicate suite (validator state injected via Runtime, the
production path, replacing the decoupled WithValidatorState key).
Verified: go build ./... green; go test ./warp/... ./precompile/contracts/warp/...
green (incl. 10k-validator TestWarpPredicate). grep UnsignedMessage = 0.
First-run network, no legacy, no pre-activation history to protect, so the
DexSettleActivationTime dated-fork gate is pure complexity. Delete the concept of
an activation time: 0x9999 dispatches unconditionally and its EXTCODESIZE marker
installs at genesis (parentTimestamp==nil); StrictPQ pinned active-from-genesis for
the Lux plugin (whole ecosystem is PQ-native — removed the per-VM config conditional).
Deleted: DexSettleActivationTime const + pointer/accessor/predicates, the cross-repo
mirror, the drift-guard test, and the 'behaves-as-plain-account-before-activation'
replay-safety branches/prose. One activation path; less code. Genesis determinism via
the single core/genesis.go toBlock() derivation. Feature tests green (0x9999 registered/
dispatched/marked-at-genesis; StrictPQ genesis+nil-inert; plugin PQ).
The 0x9999 DEX value path is identity-gated: the asset resolver refuses to
admit a market unless the running chain's identity matches. core.NewBlockChain
re-executes accepted blocks during construction (loadLastState recovery /
populateMissingTries backfill), but the identity was only bound onto the chain
AFTER eth.New returned (vm.go SetConsensusContext) — so an unclean restart after
a 0x9999 swap re-executed that block with networkID 0 / C-Chain Empty, reverted
the committed swap, failed ValidateState, and the node could not boot.
Fix is purely timing: hand NewBlockChain the *runtime.Runtime the VM already
holds (eth.Settings.ChainRuntime -> core.NewBlockChain) and bind the consensus
context BEFORE reprocess. No new identity type — networkID/chainID already live
on runtime.Runtime; the context is just the last-mile transport to geth's
ConsensusContext API. stdlib runtime aliased stdruntime in blockchain.go.
HIGH-3 host side. The always-on 0x9999 settlement precompile resolves its halt +
pot-seeding authority from contract.AtomicState.GovernanceController(); this is the
production wiring that supplies it — a per-network governance CONTRACT, never a
hardcoded mnemonic-derivable EOA.
- dex_value_wiring.go (installDEXValuePath): bind rt.GovernanceController =
dexGovernanceFor(networkID) onto the chain runtime at boot (the ONE production
site), alongside the asset resolver + native D client, and log it. dexGovernanceFor
is the genesis-set per-network governance-contract seam: it returns ids.ShortEmpty
for every network TODAY (no Governor/Timelock is deployed at a known address on any
Lux network yet), so every network runs FAIL-CLOSED (no halt authority — provably no
single key can DoS the DEX) until its real governance CONTRACT address is wired in,
per net. The directive is explicit: each entry MUST be a governance contract, NEVER a
dev-mnemonic EOA, and NEVER the retired 0x9011…94714. The boot log makes an unset
authority loud ("<unset: halt/seed FAIL-CLOSED — deploy + wire a governance contract>").
- core/precompile_overrider.go + precompile/registry/bridge.go: accessibleStateAdapter
and accessibleStateBridge implement GovernanceController() (ids.ShortID→common.Address,
exact 20-byte conversion); the internal twin interface gains the method. Returns the
zero address (fail-closed) when the runtime/atomic capability is absent.
- precompile/contract/atomic.go (evm-internal twin): AtomicState gains GovernanceController().
Builds clean (CGO off) across plugin/evm, core, precompile/registry; the dexsettle_guard
parity test passes. Normal trading is unaffected (halt switches are emergency controls,
off by default); only the ability to halt is gated — which previously only a mnemonic-EOA
holder had (a vuln), so no legitimate capability is lost and a censorship/DoS vector is removed.
This is the SOLE production install of the always-on 0x9999 DEX value-path trust
root — the seam Red flagged as fail-closed-by-absence (InstallAssetResolver /
InstallDChainClient had zero production callers, so every swap reverted
ErrNoAssetResolver). vm.Initialize now constructs and installs, from the node's REAL
chain context (never a constant):
- a registry-backed dexcore.AssetResolver bound to (networkID, cChainID): it derives
the canonical AssetID with the node's own ids and resolves it in a registry built
from the per-network manifest (EmbeddedManifestFor by EVM chainID, or the runtime-
rooted localnet native manifest). The manifest is identity-bound via
NewRuntimeVerifier (wrong-net/wrong-chain refused) and admitted under the locked-down
DefaultDexAssetPolicy synthetic-config deny-scan.
- the local in-process NativeDChainClient for the book/ledger custody seam (NOT a
remote keeper/venue/ZAP engine).
Wired for ALL networks (mainnet 96369 / testnet 96368 / devnet 96370 / localnet 1337)
so the DEX is live out-of-the-box. FAIL-CLOSED, NEVER FATAL: a construction failure is
logged and the resolver is left UNINSTALLED (swaps revert), never opened; the node
still boots for all non-DEX usage. The install binds the SAME (networkID, cChainID)
the swap path's AtomicState cross-checks, so a mismatch is fail-closed at swap time.
- core/precompile_overrider.go: forward GetCodeSize (EXTCODESIZE) on the concrete
stateDBAdapter so the value path's live-code verifier reads authoritative state.
The host side of the 0x9999 native C<->D atomic seam, and the consensus-liveness
fix Red found.
THE BUG: Block.Accept applied the staged atomic ops to shared memory FIRST
(sm.Apply, no batch -> committed immediately) and advanced the node-local seq
marker LATER, in a separate versiondb.Commit(). atomic.SharedMemory.Apply is NOT
idempotent (duplicate Put errors). A crash BETWEEN them left shared memory
mutated but the marker unadvanced; on restart the same parent->current window
re-collected, sm.Apply re-applied the present Put, errored, and Accept returned
fatal -> the chain HALTED.
THE FIX (mirrors chains/dexvm commitAtomic): flushDexAtomic -> stageDexAtomic
stages the advanced dexAtomicSeqKey marker INTO the versiondb in-memory layer and
returns (reqs, sm). Block.Accept snapshots the versiondb (marker + lastAccepted)
via CommitBatch() and passes it to sm.Apply(reqs, batch) — its WriteAll commits
the shared-memory mutation AND the batch in ONE database write. The marker advance
and the SM mutation are now all-or-nothing: a crash applies-and-marks both or
neither, and a re-accept of an applied window sees an advanced marker -> empty
window -> no duplicate Put -> no halt. With no atomic ops the plain
versiondb.Commit() path is unchanged.
Also: core/precompile_overrider.go wires CallIndex() through the EVM accessible-
state adapter; precompile/registry/bridge.go + precompile/contract/atomic.go
forward the optional contract.AtomicState capability to luxfi/precompile.
Regression for the closed halt window lives in precompile
native_haltwindow_test.go (unit). The live-path test (real versiondb + crash
injection) is CI-only (CGO + luxfi/accel LUXCPP). Local symbol-resolution proof:
CGO_ENABLED=0 go build ./plugin/evm/ compiles clean.
The C-Chain DEX settles certified D-Chain fill receipts at the 0x9999 precompile
(LP-9999); it never queries a live matcher. Remove the deprecated
DexZapEndpoint -> SetBackend(NewZAPEngine()) wiring from vm.go and the
DexZapEndpoint config knob: a synchronous in-block ZAP query against the
D-Chain's moving book forks consensus (each validator observes differently-timed
fills -> divergent StateRoot, proven by chains/dexvm
TestRED_PerValidatorRelay_SplitsConsensus). ZAP survives only as the dexvm's own
build-path transport, never as a C-Chain live backend. Receipts ride in V4 swap
hookData and are verified inline (deterministic BLS).
Bump github.com/luxfi/precompile v0.5.47 -> v0.5.49 to embed the complete,
red-swarm-hardened 0x9999 receipt-settlement surface.
Task 2 (Red H2): StrictPQTimestamp was wired from config.PQ (vm.go) but the
gate was unproven inert. The canonical lux-mainnet config already sets
pq:true and upgrade.json sets strictPQTimestamp=Quasar-TS; added behavioral
tests proving NetworkUpgrades.IsStrictPQ activates (genesis when pq:true ->
StrictPQTimestamp=0; at the timestamp on running chains; Override carries it).
GUARDRAIL fix: vm.Initialize installed gethvm.AllForbidden() (=pq.Strict),
which forbids the standard bn256 precompiles 0x06-0x08 — breaking
Ethereum-compat dapps. Now installs the EVM-owned LuxStrictPQ() projection
(pq_profile.go): every classical family forbidden EXCEPT bn256. Lux's
security-critical pairing/DLOG is in CUSTOM precompiles (0x0900, 0x22
Pedersen, zkvm verifiers) + the consensus cert, each gated by the SAME
profile bit (config.PQ also pins StrictPQTimestamp=0 -> RefuseUnderStrictPQ).
Tests (b): IsStrictPQ true on a prod strict chain (genesis + timestamp +
Override paths), false when nil. (f): LuxStrictPQ admits bn256 0x06-0x08 and
still refuses every other family; differs from AllForbidden only by bn256.
Previous ExportChain returned (bool, error) and would re-export from
scratch every call, leaving partial files with no marker on Ctrl-C.
Now returns *ExportChainResult with first/last block hashes and a Status
field ("ok" | "noop" | "interrupted"). Behavior:
* Atomic JSON sentinel at <file>.sentinel (tmp+rename) updated every
1024 blocks and on terminal states.
* Idempotency: if file exists with a "done" sentinel matching
{first, last, firstHash, lastHash}, returns Status=noop without
re-exporting. Safe to call from a cron CronJob.
* Interrupt: ctx.Done() at every block boundary writes a sentinel
with Status=interrupted + HighestExported so the caller can resume.
* Block iterator still streams (no whole-chain buffering); gzip path
unchanged.
Symmetric with admin_importChain which returns *ImportChainResult, and
with the cmd/rlp-export CLI shipping alongside.
Server-side test coverage: writeSentinel atomicity, readSentinel
absent + corrupt fall-through, JSON shape pinning for both the
sentinel and ExportChainResult, exportProgressInterval stability.
Quasar Edition rip: there is one and only one upgrade-key namespace.
The strict decoder forces parse-time failure on any unknown field —
which catches typos and stale aliases (e.g. etnaTimestamp) at config
roll instead of silently muting forks at activation.
Why this matters: encoding/json by default ignores unknown JSON keys
and zero-initializes the corresponding struct fields. After the
EtnaTimestamp → QuasarTimestamp rename, any deployed upgrade.json
still carrying etnaTimestamp would parse cleanly with the new field
left nil, and the fork would never activate. The strict decoder
turns that silent-disable failure mode into a loud boot-time error.
Pairs with: luxfi/genesis kill-etna sweep + node v1.28.22 plugin bump.
On lux-mainnet 2026-06-02 every newly-created L2 EVM chain (hanzo, zoo,
spc, pars) panicked at chain create with:
panic in eth.New: unable to commit genesis block to statedb:
triedb parent [0x56e81f17171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421] layer missing
Root cause: the operator does not specify state-scheme in chain config, so
vm.ethConfig.StateScheme = "" enters eth.New. backend.go then calls
customrawdb.ParseStateSchemeExt("", chainDb) which delegates to
geth/rawdb.ParseStateScheme — upstream returns PathScheme as the implicit
default for any database with no stored scheme (every fresh L2). That
silently re-enables path mode even though vm.go's preflight at the top
of Initialize hard-refuses it. Genesis commit then runs through pathdb's
Update with parentRoot=EmptyRootHash and dies the moment the layer tree
is consulted.
Fix: extract the scheme normalisation into resolveStateScheme() and lock
the empty-string default to HashScheme here, before backend.go can inherit
geth's path-by-default. Path explicit is still refused as before, so this
is purely an operator-default fix.
Adds:
- plugin/evm/vm_statescheme_test.go: TestResolveStateScheme covering all
four input cases (empty→hash, hash passthrough, path refused, custom
passthrough).
- core/genesis_pathdb_test.go: TestGenesisCommit_PathDB_{FreshDB,
SetupGenesisBlock_FreshDB} document the upstream pathdb genesis-commit
invariant on a fresh in-memory DB so future drift here is caught
immediately.
Also bumps compatibility.json to register v0.18.4-v0.18.18 with the
existing RPC chain VM protocol (42); restores TestCompatibility (which
was failing on main since v0.18.4 because the table was never updated).
Version: v0.18.17 -> v0.18.18.
luxfi/genesis is now indirect (no direct EVM imports), but tidy pulled in
fresher minimum versions of its transitive deps:
- constants v1.5.2 -> v1.5.7
- crypto v1.19.6 -> v1.19.15
- metric v1.5.1 -> v1.5.5
- precompile v0.5.23 -> v0.5.27
- otel v1.42.0 -> v1.43.0
Plus drops StephenButtolph/canoto, fatih/structtag, grpc-prometheus from
indirect block (no longer transitively required).
No functional changes; this is purely the dependency cleanup that follows
the luxfi/genesis v1.13.8 bump.
Bug fix (closure-swarm v1.28.15 boot blocker):
PQ gate at plugin/evm/vm.go was setting vm.chainConfig.PQ BEFORE
vm.chainConfig was assigned from g.Config — chainConfig was nil at
that point, panic on dereference. Moved the gate AFTER assignment.
Repro: fresh-PVC bootstrap with chain-config.json pq:true caused
"runtime error: invalid memory address or nil pointer dereference"
in the EVM ZAP plugin → C-Chain never bootstraps. Discovered during
localnet 1337 bring-up against v1.28.15.
Fix: move the gate to AFTER `vm.chainConfig = g.Config`. PQ profile
is now correctly installed on a non-nil chainConfig.
Added:
- cmd/rlp-verify/main.go — decodes C-Chain RLP export, verifies
genesis hash matches genesis JSON, each block's ParentHash chains
cleanly. What admin.importChain does sans state execution; sufficient
proof that an RLP+genesis pair is wireable to a luxd boot.
Indirect dep bumps:
- github.com/ChainSafe/go-schnorrkel v1.1.0 (sr25519Verify precompile)
- github.com/cosmos/go-bip39 v1.0.0 (bridgeRegistrarConfig precompile)
- github.com/luxfi/ai v0.1.0 (aiMiningConfig precompile)
Build tag: drop `gpu` tag on darwin gpu_bridge.go (parallel/gpu is one
concept; the darwin gpu bridge always compiles when cgo+darwin).
The original tests called precompile.Run() directly and expected the
PQ refusal to happen inside the precompile body. That contract no
longer holds: after CR-1, every classical precompile's Run() body is
strict-PQ-agnostic — the gate lives in (*EVM).runPrecompile and reads
the per-chain Profile from ChainConfig.PQ.
This rewrite tests the two contracts the plugin layer is actually
responsible for:
1. AllForbidden() == pq.Strict() (Hash() identity check) — every
classical primitive family flagged.
2. (*Profile).RefuseUnder(op) returns the family-specific sentinel
error for every op a strict-PQ chain must refuse.
3. nil profile (the default for non-PQ chains) admits every op.
Cross-precompile integration coverage (the strict-PQ chain runs an
EVM that actually refuses ecrecover at the boundary) lives in
geth/core/vm/pq_gate_test.go — closer to the dispatch chokepoint and
not duplicated here.
zeekayandHanzo Dev