A node killed while admin_importChain was accepting a round could not start
its C-Chain again: "required historical state unavailable (reexec=2)". The
import wrote the acceptor tip for the whole round first and then committed the
round's tries one block at a time, so a kill in between left a tip on disk
above the accepted pointer, naming blocks whose state was never written.
reprocessState re-executes from the acceptor tip, which it takes to be at or
below the block it recovers, and searches back from there only 2x the commit
interval for a state to start from.
The import now accepts each block the way the acceptor does, through the one
acceptDurably both call: tip, then trie, a block at a time. And reprocessState
moves an acceptor tip that is above the accepted block back to it, which is
what recovers a database an earlier release left that way.
TestImportArchive_KilledAnywhereRestarts kills an import at every write it
makes, reopens from the accepted pointer the VM would read, resumes, and
restarts once more. On the previous code, at commit interval 1, the kill after
write 11 of 48 does not start again.
Refs #152
Co-authored-by: Hanzo Dev <[email protected]>
admin_importChain and --import-chain-data validate blocks with InsertChain
and then move the accepted tip with SetLastAcceptedBlockDirect. Neither step
does what the acceptor does for a block consensus accepts: index its
transactions and hand its trie to the state manager. An archive node therefore
committed only the roots the import force-committed at round ends and indexed
only each round's last block. After a restart every other height's state was a
missing trie node, eth_getTransactionByHash found one transaction per round,
and with commit-interval 1 the startup head repair could not re-execute the tip
from the height before it, so the C-Chain did not start.
AcceptImported does the acceptor's durable work for each inserted block, in
order, through acceptState, which the acceptor now calls too.
TestImportArchive_HistorySurvivesRestart imports in rounds shorter than the
chain, reopens it on the same database, and checks every height's state and
every transaction's lookup. On the previous code it fails at height 1, and with
commit-interval 1 the chain does not reopen.
A database an earlier release imported keeps the gaps; only a fresh import
under this one fills them.
Fixes#152
Co-authored-by: Hanzo Dev <[email protected]>
The C++ and Rust EVMs are separate luxd plugins. The host spawns every VM with
VM_TRANSPORT=zap (node/vms/rpcchainvm/factory.go) and the cevm binary is
exercised that way cross-process today. Choosing an EVM is choosing which plugin
binary runs -- not a switch inside the Go EVM.
This tree carried a second mechanism for the same choice: an in-process cgo
bridge that linked C++ into the Go binary behind -tags cevm/revm, with a
per-transaction executor registry, an evm_setBackend RPC to flip between
backends at runtime, a resident-store config knob, and a startup log naming the
chosen backend. Its executor returned nil, nil; its tagged build did not compile
(cevm_resident_store.go referenced symbols cevmbridge never defined); and no
build produced a binary that used it. 3,961 lines, gone.
What stays is what the Go EVM actually has: the BlockExecutor seam and
Block-STM behind it (blockstm.go), and the GPU accelerator seam with its
darwin/cgo bridge into luxfi/gpu (gpu_bridge.go). The state processor consults
both and falls through to sequential execution when neither is registered.
Note for the record: nothing in the shipped binary registers a BlockExecutor,
so blocks run sequentially on the Go interpreter. The removed startup log said
"Block-STM across all cores"; it was not describing this binary. Wiring
Block-STM into the production path changes consensus execution and is a
separate decision, made deliberately, with a parity gate.
Co-authored-by: Hanzo Dev <[email protected]>
Three things, all in the import path.
The tree did not build. plugin/evm/vm_database.go names AncientShared and
FreezeThreshold on rawdb.OpenOptions; luxfi/geth v1.20.2 has neither and v1.20.3
has both, so a clean checkout failed to compile before any change. Pinned to
v1.20.3.
The blocks-per-round was the literal 2500 inside importBlocksFromFile, so the
only way to change it was to edit and rebuild. It is a parameter now:
admin_importChain takes an optional count, --import-chain-batch sets it for the
startup path, and defaultImportBatch stands in when neither says. The right
number is a property of the machine -- a host with memory to spare amortises one
InsertChain call over far more blocks -- so it does not belong in the source.
Both importers wrote their own copy of the same closure to advance the accepted
marker and commit. There is one persistAcceptedBlock now and both hand it to the
importer. What genuinely differs between them -- the startup path reconciles the
export frontier and syncs consensus state afterwards, because it runs before the
engine is live -- stays where it belongs.
Co-authored-by: Hanzo Dev <[email protected]>
github.com/luxfi/utils is now github.com/luxfi/util. Import paths and the
module requirement follow the rename, and the luxUtils alias becomes luxUtil.
The old module stays as an indirect requirement that published dependencies
still pull in.
Co-authored-by: Hanzo Dev <[email protected]>
constants.BlackholeAddr is a plain [20]byte, so constants no longer imports
geth. The fourteen sites here all want a geth common.Address, and each file
already imports geth/common, so each takes the free conversion itself.
Only one site was a compile error: the .Bytes() method call in
rewardmanager.DisableFeeRewards. Everywhere else [20]byte is assignable to
common.Address, so the change was invisible to the compiler, and the two
require.Equal assertions in rewardmanager would have compiled and then failed
at run time, because reflect.DeepEqual compares types before bytes. Those two
are wrapped for that reason, not for style.
The two failures in core, TestHeadPin_Across4096Boundary_NoLeak and the
invalid-chainID-0 panic in TestCreateThenDeletePostByzantium, reproduce
identically with these lines reverted and are unrelated.
Co-authored-by: Hanzo Dev <[email protected]>
luxd serves chain routes under /v1/chain, and the node, cli and sdk releases
already ship it. The deployed gateways still answer the old prefix, so remote
reads break until they are redeployed; the estate moves together rather than
sitting half converted.
Lines that record an observation - a probe, a status code, a past release -
keep the prefix they were written against, as do URLs on hosts that are not
ours. Avalanche endpoints move to /ext/bc, which is the path that host serves.
Co-authored-by: Hanzo Dev <[email protected]>
The rename to /v1/chain is real in luxd source, but the deployed fleet has not
taken it yet: probed just now, api.lux.network, api.lux-test.network,
api.zoo.network and api.hanzo.network each answer 200 on /v1/bc/C/rpc and 404 on
/v1/chain/C/rpc. Composing /v1/chain against those hosts reaches nothing, which
is the failure the rename set out to prevent.
The move belongs with the node redeploy, not ahead of it.
Co-authored-by: Hanzo Dev <[email protected]>
luxd serves chain routes under /v1/chain; the /v1/bc prefix it replaced is
gone, so anything composing it reaches nothing. Lines that record an
observation - a probe, a status code, a past release - keep the prefix they
were written against, as do URLs on hosts that are not ours.
Co-authored-by: Hanzo Dev <[email protected]>
luxd serves chain APIs under one segment from constants.ChainAliasPrefix
(constants v1.6.4, node v1.36.181):
/v1/bc/<alias>/... -> /v1/chain/<alias>/...
/ext/bc/<alias>/... -> /v1/chain/<alias>/...
Observations keep their original addresses: a dated note or a recorded status
code is a fact about when it was written, not an instruction to follow.
A receipt root that disagrees stops the node where it stands — that is the
property the placement of this rule exists to prevent, and it holds whether or
not it has ever been reached.
Comment only.
Co-authored-by: Hanzo Dev <[email protected]>
The comment explained the placement by recounting what a node hit when the rule
was enforced during execution. The reason it belongs here does not depend on
that having happened: execution has to be a function of the block alone, and
reading an object out of shared memory while the matching Remove lands at accept
makes it a function of when the block is executed instead.
Behaviour unchanged; comment only.
Co-authored-by: Hanzo Dev <[email protected]>
The ancient store existed in geth and its flags existed on the node, and nothing
joined them: the VM always opened its chain database straight over the key-value
store, so --cchain-ancient reached the C-Chain config and stopped there.
openChainDatabase is that join. Without an ancient directory the key-value store
is the database, as before, and every block the node ever accepted stays in it.
With one, blocks past the freeze threshold live in an append-only store on that
path: the node that owns the directory writes them there and drops its own
copies, and nodes that share the directory read history from it instead of each
keeping a copy. On a machine running many nodes that is the difference between
one copy of the chain and one per node.
The config test was written against an earlier shape, where ancient-read-only
and ancient-shared each required the other — a pair that named one arrangement
twice and was removed. It now pins what is actually enforced: sharing needs a
directory to share, and a store needs a hot window for the block being built to
find its parent. A comment about two writers interleaving one file had been left
sitting above the threshold check, which is not what it describes.
Co-authored-by: Hanzo Dev <[email protected]>
The two settings each required the other, so neither could be given alone and
the pair named one arrangement twice. Reading a store another node writes
already means not writing to it. Nothing branched on ancient-read-only.
Co-authored-by: Hanzo Dev <[email protected]>
parseGenesis names each genesis config key it copies into the extras
config. feeSplitTimestamp was not on that list, so a configured split was
silently dropped and every fee kept burning whole. Name it beside the
other fork timestamps. verifyFeeSplit already refuses a split with no
governed destination, so a misconfigured node fails at parse rather than
at a validator — TestParseGenesis{ReadsFeeSplitTimestamp,RefusesSplit
WithoutRewardManager} pin both halves.
Point the fee-governance test's DAO address at the deployed CREATE2 DAO
Safe 0xF66B025b so the reward destination has one source of truth.
Co-authored-by: Hanzo Dev <[email protected]>
admin_importChain rewrites the chain from a file on disk. It defaulted to
enabled, with a comment saying it was on so that importChain would work, so
every node carried the ability to have its history replaced by anything that
could reach its RPC port -- to spare an operator one line of config during a
restore that happens rarely and is supervised when it does.
A node that is restoring sets admin-api-enabled in its chain config for as long
as the restore takes. Nothing else needs it, and nothing else should offer it.
The tests set the field explicitly and are unaffected.
Co-authored-by: Hanzo Dev <[email protected]>
State sync is skipped when the chain is not far enough ahead of local state,
because bootstrapping would be faster. That trade only exists while
bootstrapping can run at all. It replays blocks against the state beneath them,
so a node missing the state at its own head cannot replay a single block — it
fails to materialize the next one and stops there, whatever the gap is.
On such a node the threshold declines the one mechanism that could repair it,
and small-gap corruption becomes permanent: it cannot execute forward, and it is
not far enough behind to be allowed to sync. The threshold now applies only when
the node holds the state its head is built on. A chain that cannot be read
answers no — the cost of being wrong that way is a sync that was not strictly
needed, against a node that could otherwise never come back.
Co-authored-by: Hanzo Dev <[email protected]>
It was the zero value, so a chain got it only by carrying a per-chain config
file that said so — and most deployments have no chain-config path at all, which
made the one mechanism for rejoining after a deep gap unreachable on whole
fleets. There is no posture where refusing it and re-executing instead is the
right answer, so it is not a decision to hand an operator. The bound is
StateSyncMinBlocks, which lives inside the mechanism: a smaller gap replays and
never reaches this path.
Co-authored-by: Hanzo Dev <[email protected]>
A node was discarding 948 builds every ten minutes on a HEALTHY chain, and had
spent 41 hours doing it on a halted one. The cause was a barrier this fork
dropped.
Upstream holds the builder until the transaction pool has finished resetting onto
the accepted head: `for !needToBuild() || pendingPoolUpdate()`, where
pendingPoolUpdate is chainHeadHash != mempoolHeadHash. We kept the loop and
deleted the second half, along with both hashes and the two setters that maintain
them.
Without it the builder wakes in the window between a block being accepted and the
pool reorganising onto it. The pool still lists what that block just mined, so
needToBuild says there is work; the miner assembles against the new head, finds
nothing left, and returns an empty block; the anti-empty policy discards it.
Nothing changed, so the builder wakes again — immediately, and forever, until the
pool happens to reset.
Restored rather than reinvented: the same two hashes, the same comparison, the
same loop condition, fed by the pool's own reorg event on one side and Accept on
the other. Both are initialised to the current head so a fresh or restarting node
starts converged instead of gated shut.
This also removes proposeEmptyBuild, added earlier today. It built whenever the
pool was non-empty, which is exactly the spin state — it would have turned 948
discarded builds into 948 empty blocks on chain. The barrier removes the
condition it was guarding against, so the guard goes.
Co-authored-by: Hanzo Dev <[email protected]>
An assembled block with no transactions is two conditions wearing one shape. No
demand is the demand-driven property working — discard it, or an idle chain grows
forever. Demand the assembler failed to collect is a bug, and discarding THAT
makes it permanent: no block means no nonce advances, so the pending transaction
is never mined, so the pool never drains, so the condition under which a block
would be built can never return.
The gate read the assembled block, which cannot tell the two apart, so it took
the second for the first. Measured on lux-devnet: the chain stopped at 20:38 UTC
on 2026-08-11 holding ONE executable transaction, sat 41 hours, and behind it the
traffic generator logged 7,100 consecutive failures while luxd-0 and luxd-3
logged errEmptyBlock ~2000 times per six hours. Every node was healthy. Nothing
was wrong with consensus — CERTs gossiped the whole time.
It reads the pool now. Nothing pending is idleness and the block is discarded;
anything pending builds, whatever assembly managed to collect, and logs the
mismatch where somebody can see it. Liveness stops depending on the assembly path
being correct.
The rule moves into proposeEmptyBuild so it can be tested without standing up a
VM, which is why it had no test before.
This does not fix the proximate cause, and is not meant to: the transaction was
in exactly one node's pool (luxd-2) and never reached either proposer, so EVM
transaction gossip is the next thing to look at. What changes here is that such a
failure degrades throughput instead of halting the chain.
Co-authored-by: Hanzo Dev <[email protected]>
The comment claimed Block.Accept passes a batch to sm.Apply(reqs, batch) so the
marker and the shared-memory mutation land in one atomic write. It does not, and
deliberately: a batch cannot cross the plugin's process boundary, atomiczap
refuses one, and passing it made every settling block a fatal Accept.
Accept commits the state first and applies second, which orders a crash toward a
skipped op rather than a replayed one. A comment describing the opposite reads
as a bug on the money path and invites reordering the writes back.
Co-authored-by: Hanzo Dev <[email protected]>
Two defects, both fatal, both on the C<->D atomic settlement path.
1. Accept handed the versiondb batch to sm.Apply unconditionally. The
C-Chain EVM is an out-of-process plugin, so its SharedMemory is the ZAP
client, and that client refuses any batch — a database.Batch cannot
cross a process boundary. Every block staging an atomic op would have
failed Accept, which is fatal. Commit the versiondb first, then Apply
with no batch. Without a shared batch exactly-once is unreachable; the
choice is at-most-once or at-least-once, and replay is unsound here (a
duplicate put is fatal, and after the peer consumes the object it
re-creates it). A skipped op is survivable: C's settlement replay guard
is EVM state, not the Remove, and a skipped intent Put is refunded by
reclaimIntent. The residual single-node cost is stated in the code.
2. The settle path read the object from shared memory DURING EVM
execution while the consuming Remove landed at Accept. A node
re-executing that block — bootstrapping, state-syncing, re-tracing —
found the object gone, computed a reverted receipt where the network
computed a successful one, and died on the receipt root. The seam
settled live and wedged every node that tried to sync past it.
Execution now binds to the object bytes the transaction carries and
declares the consumption in its logs; this adds the block-level rule
that proves the declaration against shared memory at Verify, on both
the consensus path and the producer's own pre-proposal verify, so a
forged object costs a block rather than forking a receipt root. The
check is gated on the bootstrap frontier, which the shared-memory layer
requires of any chain consuming cross-chain state.
Co-authored-by: Hanzo Dev <[email protected]>
The --import-chain-data startup import is re-entered on every boot, because
production pod specs keep the flag set. Once the chain is already at or above
the file's tip, importBlocksFromFile imports 0 blocks and returns
"no blocks imported (parsed=0)"; Initialize treated that as fatal, so the
C-Chain failed to initialise on every restart of an already-imported node —
/v1/bc/C/rpc 404, node contributes no consensus vote.
This was fixed once. Commit 15e75f43 ("startup RLP-import consensus-wedge +
idempotency") landed BOTH halves on hotfix/v1.104.9. Only the consensus-refresh
half was forward-ported to main (it is here as "FIX (v1.104.11)"); the
idempotency half was not, and isNothingToImportError does not exist on main.
Verified: git merge-base --is-ancestor 15e75f43 main => false; the commit is
reachable only from hotfix/v1.104.9 and release/v1.104.9-hotfix.4.
Observed live on lux-testnet 96368 today rolling node v1.36.35, which is built
from evm main: two nodes came back with a dead C-Chain on exactly this error.
The v1.36.24 plugin binary in the same cluster still contains the string
"Startup import: chain already at/above import tip; nothing to import"; the
v1.36.35 binary does not.
Restores isNothingToImportError and the guard, unchanged in meaning from
15e75f43: idempotent ONLY when the chain already holds accepted content
(curHead > 0). On a fresh chain the same messages still mean a genuinely
empty or corrupt file and stay fatal.
startup_import_idempotency_test.go locks the contract, both RLP shapes plus the
fresh-chain safety case. Red without the fix (undefined: isNothingToImportError,
build failed); green with it:
--- PASS: TestStartupImportIdempotency_AlreadyAtTip (0.02s)
ok github.com/luxfi/evm/plugin/evm 1.260s
Co-authored-by: Hanzo Dev <[email protected]>
luxd pins baseURL="/v1" (node/server/http/server.go:33) and has never
served /ext, so both exported constructors — plugin/evm/client.NewClient
and warp.NewClient — built a path no node answers. They are the seam
every out-of-tree caller goes through: cli/pkg/utils/staking.go,
sdk/validatormanager/uptime.go and netrunner/api all hand them a bare
node URI, and staking.go's own SplitRPCURI already parsed /v1/bc/<id>/rpc.
Those callers were already on /v1; this package was the lagging outlier,
so fixing it here fixes all of them without touching one.
Also corrects the simulator's default websocket endpoint, the hardhat
default, and the docs — including api.testnet.lux.network, which does not
serve RPC at all. The testnet host is api.lux-test.network (chainId
0x17870). Prefix-only sweeps miss that class of error.
Tests pin the constructed path against an httptest server, with both an
alias and a blockchain ID, so a future prefix change fails loudly instead
of silently 404ing in production.
Co-authored-by: Hanzo Dev <[email protected]>
The first cut registered the cevm checkpoint store over init.DB before
initializeDBs ran. That is wrong in standalone-database mode: initializeDBs points
the chain state at a separate standalone DB (vm.db), so checkpoints would land in
the base avaDB, not alongside the chain state. It also predated vm.db being set.
Fix: register AFTER initializeDBs, over vm.db — the RAW durable base DB
(NOT the versiondb), standalone-DB-aware (vm.db == the standalone chain-state DB
when UseStandaloneDatabase is set). This is exactly the pattern warpDB / validatorsDB
use (prefixdb.New(prefix, db) over the raw base) for state that need not commit
atomically with the last accepted block: cevm checkpoints now persist on WRITE and
survive a restart, with no versiondb pending-commit dependency — the durability gap
noted when the feature landed.
Still opt-in (cevm-resident-store, default false), inert without -tags cevm, and
safe by construction (a missing/stale checkpoint falls back to the Go-dump seed;
every root is verified). Default plugin build green; resident lazy-reload test green.
Co-authored-by: Hanzo Dev <[email protected]>
Wires StateLoadFromLazy into the resident applier so the C-Chain, on restart,
resumes cevm from a persisted checkpoint (bounded-RAM lazy load) instead of dumping
the full Go state into cevm — the "own disk-backed state" evolution the seed-cap
comment names. Removes the post-restart declined-too-large seed dump.
- core/parallel/cevm_resident_store.go: SetCevmResidentStore(db) (the VM registers a
durable KV; nil = off, unchanged behavior). tryLazyLoadCheckpoint pages the account
trie from disk via cevmbridge.StateLoadFromLazy and sets the resident to the
checkpoint height; persistCheckpoint writes StatePersist records + the height at a
bounded cadence (default 4096 blocks).
- cevm_shadow.go: getResident tries the checkpoint before a fresh handle;
persistCheckpoint runs after a block's cevm root is VERIFIED against the header.
- plugin/evm: CevmResidentStore config flag (default false) → registers a "cevmstate"-
prefixed sub-DB before chain init.
- cevmbridge/bridge_stub.go: StateLoadFromLazy no-op for the non-cevm build.
Safe by construction + triple-gated: nothing runs unless built -tags cevm, the VM
registered a store, AND a checkpoint exists; and the applier still verifies EVERY
cevm root against the header, so a stale/wrong checkpoint only forces a resync (a
Go-dump seed next block), never a wrong commit. nil store (the default) is the prior
in-memory, seed-from-Go behavior byte-for-byte.
Proof (cevm_resident_store_cevm_test.go, //go:build cevm): seed + apply a block,
persistCheckpoint, simulate restart (free handles, retain store), getResident
LAZY-LOADS the checkpoint (height + root recovered, no Go dump), then the next block
applies to a byte-identical FULL root vs a full-reseed reference. Full parallel cevm
suite green; default (non-cevm) plugin build unchanged.
Co-authored-by: Hanzo Dev <[email protected]>
The startup --import-chain-data path (vm.go, after importBlocksFromFile) advanced
core.BlockChain + acceptedBlockDB to the imported tip but never refreshed the
in-memory consensus wrapper vm.State (components/chain), which initChainState had
built at the pre-import genesis tip. On the FIRST import boot the consensus engine
then reads genesis while the EVM head is the imported tip and the proposervm wedges
("BuildBlock: preferred block not fetchable", preferred=empty id) until a restart
re-runs initChainState off acceptedBlockDB.
admin_importChain runs on a LIVE node whose vm.State is already current, so only the
startup path needs this. Now syncs vm.State via the existing, boot-safe
chain.State.SetLastAcceptedBlock, fail-closed (a boot that cannot establish the
imported tip stops loudly rather than serving consensus at a stale genesis).
Empirically proven on lux-testnet 96368 / node v1.36.17 (luxfi/evm v1.104.9): first
import boot wedged at tip 251, healed only on restart; with this the first boot
finalizes past the import tip with no restart. Imported-state execution / stateRoot
unchanged.
Co-authored-by: Hanzo Dev <[email protected]>
The message/validators/warp marshalers are hand-rolled per-type packers
(marshalValue type switch) that ignore struct tags — the linearcodec that read
them is gone. Pure dead metadata. Golden round-trip tests green (byte-identical).
Co-authored-by: Hanzo Dev <[email protected]>
Two-tier finality (Nova = local majority accept, drives VM.Accept; Quasar =
⅔-by-stake export) requires that NO export surface ever names a block above the
served accept tip. RED's final pass approved SHIP with three LOW items to close
first.
LOW-1 — DRY the accept-tip clamp into ONE place. New eth.Ethereum.ClampedQuasarHeight()
= min(LastQuasarHeight, LastAccepted.Number) is the single clamp; BOTH export
surfaces read it: EthAPIBackend.quasarHeight() (RPC finalized/safe) and
warpBlockClient.LastQuasarHeight() (warp export gate, via the BlockClient
interface). Removes the second inlined clamp copy that could drift. LastQuasarHeight()
is documented as RAW (node boot re-seed only, where reconcile has already run).
LOW-2 — exactly ONE admin_importChain. Deleted the eth-package AdminAPI.ImportChain
(which neither reconciled the export height nor skipped <=head blocks); the plugin VM's
(plugin/evm/admin_api.go) is the sole one — it runs under vmLock and calls
reconcileQuasarWithAccept after each SetLastAcceptedBlockDirect. Left a NOTE so it is
not re-added and reopens the rewind hole. Dropped the now-unused rlp import.
LOW-3 — exclude concurrent export-height writers + prove the real boot wiring.
SetLastQuasarFinalized now takes vmLock (the same lock reconcileQuasarWithAccept's
admin_importChain caller holds), so the observer callback's durable Put+Store can never
interleave with a re-import's Reset(0)+Delete on the same key. Fixed a boot ordering
gap: the startup auto-import (importBlocksFromFile) runs AFTER initializeChain's
reconcile, so a --import-chain-data rewind could leave a stale export height — added a
reconcile right after the auto-import. New eth.NewTestEthereum test-support ctor.
Tests (all green): eth -race adds TestClampedQuasarHeight_RealRewind (genuine
SetLastAcceptedBlockDirect rewind; belt clamps finalized/safe to the rewound tip while
raw is unchanged); plugin/evm adds TestBootReconcile_RealLastAccepted_ClearsStaleQuasarOnRewind
(drives the REAL restored LastAcceptedBlock through belt + suspenders) and
TestSetLastQuasarFinalized_LockedAgainstReconcile (concurrency: durable key <-> in-mem
always consistent). gofmt/vet/ontology clean.
Co-authored-by: Hanzo Dev <[email protected]>
Follow-up to the export-boundary re-point, closing RED's re-review MEDIUM (a finality-safety hole
reachable via our own EVM-rewind→re-import restore runbook) and LOW.
MEDIUM — the rewind path. A rewind / RLP re-import (SetLastAcceptedBlockDirect) can lower the accept
tip BELOW the persisted EXPORT-FINAL (Quasar) height; the monotone SetLastQuasarHeight + advance-only
SyncQuasarFrontier then keep it stuck high, and after the chain rebuilds past that height with a
DIFFERENT canonical, `finalized` would name a block that was NEVER ⅔-certified — reported export-final
to every bridge/DEX consumer. Two parts, both needed:
- Belt: api_backend quasarHeight() resolves finalized/safe at min(LastQuasarHeight, LastAccepted) —
finalized/safe can NEVER exceed the served accept tip, even with a stale-high durable key.
- Suspenders: VM.reconcileQuasarWithAccept clears the durable quasarHeightKey + the in-mem height
(eth.ResetLastQuasarHeight, bypassing the monotone guard) when the accept tip is below the export
height — the ONE legal regression of the otherwise-irreversible frontier. Conservative reset to 0
(nothing export-final until the rebuilt chain re-certifies). Called on boot (against the restored
accept tip — catches an offline rewind) and after admin_importChain (the live re-import). On the
next boot the node re-seeds consensus' export frontier from the cleared (0) height, so the
advance-only SyncQuasarFrontier yields to the rewind without touching the four shipped fixes.
LOW — SetLastQuasarFinalized now PERSISTS the durable key BEFORE advancing the in-mem height, so the
durable key is the source of truth: a crash between the two can only leave in-mem at/behind the
durable value on boot, never ahead of it (no monotonicity dip). A persist failure does not advance.
Tests: augmented TestFinalizedSafeResolveToQuasarNeverAboveIt with the belt case (seed export height
ABOVE the accept tip → finalized CLAMPS to the tip — the case RED flagged as false confidence, now
ENFORCED); new TestReconcileQuasarWithAccept_ClearsStaleHeightOnRewind (durable key + in-mem cleared
on an accept regression) and TestSetLastQuasarFinalized_PersistsBeforeAdvancing (persist-first,
no-advance-on-failure). eth suite green under -race.
Co-authored-by: Hanzo Dev <[email protected]>
The two-tier consensus (v1.36) advances VM.Accept at a BARE MAJORITY (Nova), which is reorgable
and can fork under f=1 equivocation. The Ethereum `finalized`/`safe` tags and warp cross-chain
exports were still reading the accept tip — the "semantic collapse" the split exists to prevent.
Re-point every EXPORT surface onto the export-final (Quasar, ⅔-by-stake) frontier consensus
already exposes.
#1 CRITICAL — eth `finalized`/`safe` tags:
- eth.Ethereum gains a DURABLE, monotone lastQuasarHeight (SetLastQuasarHeight/LastQuasarHeight),
pushed by consensus strictly AFTER accept. The plugin VM's SetLastQuasarFinalized persists it to
acceptedBlockDB and reloads it on boot (readLastQuasarHeight), so the export surface never
regresses on restart.
- api_backend HeaderByNumber/BlockByNumber resolve Finalized(-3)/Safe(-4) to GetBlockByNumber(
LastQuasarHeight) BEFORE the IsAccepted() catch-all — NEVER a block above the Quasar height.
`latest`/`pending` stay on the accept tip (correct — not finality tags).
- Test: TestFinalizedSafeResolveToQuasarNeverAboveIt proves finalized == the Quasar height,
monotone, and never above the accepted tip.
#2 HIGH — warp cross-chain export double-spend:
- warp.verifyBlockMessage now requires the block be EXPORT-final (height ≤ BlockClient.
LastQuasarHeight), so a validator signs a block hash only at Quasar finality, never on a
reorgable Nova accept (two conflicting Nova forks could otherwise each aggregate a valid export).
- WarpQuorumNumeratorMinimum floored 33 → 67 (⅔): any warp message needs ≥⅔ stake to verify, so at
most one of two conflicting exports can validate (two ⅔ sets intersect on >⅓ honest stake) —
restoring the ⅔-accept safety the pre-split accept tier gave for free.
The consensus engine pushes the frontier via NetworkConfig.QuasarObserver; the node bridges it into
the VM (separate node commit). Nova-only chains (no export surface) are unaffected.
Co-authored-by: Hanzo Dev <[email protected]>
Three deterministic test reds, all test-only (no production source change):
- plugin/evm/message: regenerate 6 LE golden vectors (utils LP-023 BE→LE
codec migration @v1.2.0; oracle-verified, not tautological) + fix stale
big-endian doc comments in codec.go.
- plugin/evm/validators/state: convert TestParseValidator fixtures BE→LE
(same codec migration) + doc comments.
- core/txpool/legacypool: decouple testTxPoolConfig from the production
DefaultConfig (which was deliberately bumped in fdc840067 for throughput);
pin canonical geth test limits. NOT a regression — production tx-acceptance
is correct; the test baseline was self-contradictory (per-account min >
global max). Also fixes a 600s package timeout → 20s.
Co-authored-by: Hanzo Dev <[email protected]>
rpcChainVM protocol unchanged (42) — node compatibility preserved. Pairs with
the genesis-injection fix (0bd9927a2) for evm module tag v1.104.1.
Co-authored-by: Hanzo Dev <[email protected]>
Forward-integration onto main (v1.103.0 line) of the accept-empty recovery fix
released as v1.101.4. errEmptyBlock moves from SyntacticVerify (parse/verify/
accept) to the build path only, so a consensus-finalized empty block Accepts
while a demand-driven proposer still never produces one.
Co-authored-by: Hanzo Dev <[email protected]>
ROOT CAUSE of the C-Chain bootstrap ancestry-descent stall: ParseBlock was STATEFUL.
BlockGasCost is NOT in the RLP block — it is DERIVED from the parent header. UnmarshalBlock
computed it via vm.blockChain.GetHeaderByHash(parent), and SyntacticVerify REQUIRED it non-nil
(errNilBlockGasCostEVM). For a bootstrap descent parsing ancestry blocks AHEAD of the accepted
height the parent is not present, so every such block failed to parse — masked by the ZAP
errorToZAP mapping as the misleading "malformed block id (does not connect to accepted chain)".
The Lux bootstrapper indexes the whole fetched batch by (id,height,parent) up front, so it could
not index the ancestry, never found the frontier tip in the batch, and re-requested forever ->
ErrStalled. Confirmed on devnet: "syntactic block verification failed: nil blockGasCost is
invalid after EVM activation".
FIX — decomplect parse from verify (matches upstream avalanchego coreth: parse decodes, Verify
enforces connectivity):
- block.go: new ensureBlockGasCost() fills BlockGasCost from the parent header WHEN PRESENT
(SetHeaderExtra is hash-keyed, so it persists for a block first parsed while its parent was
absent). Best-effort at parse; authoritative at Verify (AcceptBootstrapBlock calls blk.Verify
only after the contiguity gate height==finalized+1, so the parent is ACCEPTED/present).
- block_verification.go: SyntacticVerify no longer requires BlockGasCost (stateless,
block-intrinsic checks only); keeps the uint64 sanity check when it IS set.
- vm.go UnmarshalBlock: replace the inline parent-lookup with block.ensureBlockGasCost().
SAFETY (parse != accept): parse only decodes + stateless syntactic checks; no non-connecting or
gas-cost-invalid block can be ACCEPTED just because it now parses. Verify/Accept
(InsertBlockManual -> consensus/dummy header verification) recomputes the expected BlockGasCost
from the (present) parent and rejects a mismatch; the bootstrap contiguity gate accepts blocks
oldest-first (parent finalized). A wrong-hash block is rejected off-path by the content-addressed
descent. BlockGasCost is not in the block hash -> no malleability.
evm plugin/evm tests pass.
Co-authored-by: Hanzo Dev <[email protected]>
Pre-existing idle-builder wake race (not caused by QCv2): on a demand-driven
C-Chain with no empty blocks, a tx arriving while no builder is parked in
waitForNeedToBuild broadcasts pendingSignal to no waiter and is lost; the tx is
then only caught on the NEXT needToBuild() poll, which — depending on the
proposervm slot loop and engine re-call cadence — could leave the first tx after
idle pending for a long/indefinite time (the mainnet treasury-sweep-stuck and
v1.32.1 devnet DEXFill-didn't-complete symptoms).
Fix: startPendingTxPoll — a node-lifetime ticker (builderPollInterval = 500ms)
that periodically broadcasts pendingSignal so any parked builder re-evaluates
needToBuild() against the live mempool. It is decomplected from the tx-submit
subscription (awaitSubmittedTxs remains the primary, immediate wake) so the two
wake sources are independent. It NEVER forces an empty block — a wake only
produces a build when needToBuild() is true — so demand-driven chains stay
empty-block-free, and it changes wake TIMING only, never block CONTENTS, so
validators still build byte-identical blocks (deterministic). needToBuild now
routes through an injectable pendingTxs predicate so the wake path is testable
without a live txpool.
Regression: TestBuilderWakesOnPendingTxAfterIdle reproduces the node bridge's
exact waitForEvent call on an idle builder, delivers a pending tx with NO submit
broadcast, and asserts it returns PendingTxs within a bounded deadline. It FAILS
on the pre-fix code (times out — the lost wakeup) and passes after.
TestBuilderPollBackstopStopsOnShutdown asserts the ticker exits cleanly. Builder
tests green under -race.
Co-authored-by: Hanzo Dev <[email protected]>
Decomplect the build-pacing decision out of waitForEvent into a pure
nextBuildTime(earliestBuild, lastBuildTime) function (no clock read), so the
logic the network-halt fix depends on is deterministically testable. The
wall-clock comparison stays in waitForEvent.
TestNextBuildTime covers all 5 cases (first-build paces to target rate; past
earliestBuild builds immediately; target rate dominates retry floor; retry
floor prevents hot-loop; boundary tie-break). TestNextBuildTime_NeverEarlier
ThanTargetRate locks the invariant the fix relies on: the next build is NEVER
scheduled before parent.Time+TargetBlockRate, so a too-early high-fee block can
never re-introduce the small-tip stall.
Second fresh-net block-production bug, exposed once WaitForEvent was unblocked.
The hand-rolled node notification bridge calls WaitForEvent in a tight loop and
builds a block the instant any tx is pending — with no pacing (ava's proposervm,
which Lux does not wrap, normally provides it). Two consequences:
1. Block gas cost: cost = parentCost + step·(TargetBlockRate − elapsed). A block
built immediately after its parent (elapsed≈0) requires step·TargetBlockRate
(e.g. 200000·2 = 400000) of tips to pass verifyBlockFee. A large deploy
covers it; a single small transfer (~100000) cannot → BuildBlock fails →
the tx never mines.
2. Equivocation: with all 5 validators building instantly off slightly different
mempool snapshots, they produce different blocks at the same height; two can
each gather an α-of-K cert → conflicting finality certs → Logger.Crit → the
node exits(1) and crash-loops.
Fix: pace waitForEvent to parent.Time + TargetBlockRate (the point at which the
block gas cost decays back to the parent's level, 0 from genesis), mirroring the
timing ava's coreth builder applies. Small-tip txs now mine, and the ~rate-long
window lets the mempool gossip-converge so validators build the IDENTICAL block
(same ID, one cert) instead of racing to finalize conflicting blocks. Retry
floor 1ms→100ms so a failed BuildBlock cannot hot-loop.
Fresh-start block-production stall root cause. The node-side notification
bridge (node/chains/manager.go) calls VM.WaitForEvent in a loop with a
non-cancellable context.Background(), starting BEFORE SetState(NormalOp)
initializes vm.builder. The first WaitForEvent therefore parks in the
builder==nil select { ctx.Done | stateSyncDone | shutdownChan } and, since the
context never cancels and state-sync is off, blocks FOREVER -- the serial
bridge loop never re-calls WaitForEvent to observe the now-ready builder, so no
PendingTxs ever reaches the engine and no block is ever built.
ava avoids this with common.NotificationForwarder + cn.OnChange=CheckForEvent,
which cancels the outstanding WaitForEvent and re-subscribes on VM state change.
Lux replaced that with the hand-rolled bridge and lost the recovery path.
Fix (self-contained, VM-side): add builderReady chan, closed once by
onNormalOperationsStarted after vm.builder is set, and add a select case so a
WaitForEvent that parked early re-fetches the ready builder and proceeds.
Makes WaitForEvent correct for any caller, cancellable or not.
go.sum: re-record luxfi/age,pq checksums after upstream re-tag (verify ON).
Enable-everything builder surface (public permissionless launch). The geth-layer
strict-PQ profile (LuxStrictPQ) refused ecrecover / p256Verify / sha256 /
ripemd160 / blake2f / bls12381 / kzg under vm.config.PQ — but ecrecover
(secp256k1) is THE Ethereum signature path every dapp depends on, and the rest
are verify-only / hash, key-safe. Refusing them breaks the builder surface.
The chain now installs NO forbidding pq.Profile: chainConfig.PQ stays nil,
which pq.RefuseUnder(nil) treats as permissive. The generic geth mechanism
(chainConfig.PQ + runPrecompile.RefuseUnder + legacypool nil-check) is kept for
any chain that wants it; Lux installs nothing.
This completes the decomplect started in luxfi/precompile: consensus PQ and the
EVM precompile profile were complected through config.PQ / StrictPQTimestamp.
After precompile dropped contract.RefuseUnderStrictPQ from every verify-only
precompile, the whole strict-PQ projection here was dead. Removed:
- plugin/evm/pq_profile.go (LuxStrictPQ)
- plugin/evm/vm.go: the `if vm.config.PQ { chainConfig.PQ = LuxStrictPQ() }`
install + the unconditional StrictPQTimestamp=0 genesis pin
- plugin/evm/config: the config.PQ bool field (no code writer; was JSON-opt-in)
- params/extras/network_upgrades.go: StrictPQTimestamp field + apply + IsStrictPQ
- precompile/registry/bridge.go + core/precompile_overrider.go: the dead
IsStrictPQ adapters (only existed to feed the deleted StrictPQReporter)
- tests: pq_test.go, strict_pq_activation_test.go, and the two StrictPQ
assertions (+ orphaned helper) in precompile_upgrade_rollout_test.go
Lux's PQ security is enforced at consensus / identity (quasar/p3q), not by
refusing an EVM verifier a dapp asked for. The live lux-mainnet upgrade.json
never actually set strictPQTimestamp, so this matches production.
Build: full module green. Tests: params/extras, plugin/evm/config,
precompile/registry pass.
Rename across the evm warp backend and the warp precompile: warp.Core ->
warp.Message, NewCore -> NewMessage, ParseCore -> ParseMessage, the .Core field
-> .Message, and local core/msgCore/testCore vars -> msg/testMsg. deps: warp
v1.21.0 -> v1.22.0, precompile v0.11.0 -> v0.12.0.
geth's core/types and core/vm imports are untouched — the rename is scoped to
the warp message type, never the geth core package. ./warp/... and
./precompile/contracts/warp/... green, including TestWarpWrongNetworkID.
Track the upstream rename across the evm warp backend
(warp/{service,backend,verifier_backend}) and the warp precompile
(precompile/contracts/warp/{config,contract,contract_warp_handler},
precompileconfig): SignedCore->Core, WarpEnvelope->Envelope.
Identifiers only — the digest D, ZAP c14n bytes and BLS Beam / Corona Pulse /
ML-DSA signing domains are unchanged. ./warp/... and
./precompile/contracts/warp/... green, including the 10k-validator predicate
sign->aggregate->verify.
Hard-fork the last evm warp consumers onto the canonical ZAP message format
(one and one way only). warp v1.20.0 deleted UnsignedMessage; the signed
subject is now warp.SignedCore (D = keccak256(coreDST ‖ zap_c14n)) and the
signed wrapper is warp.WarpEnvelope. The BLS Beam signs warp.BeamSigningBytes(D).
deps: warp v1.19.5 -> v1.20.0, precompile v0.5.59 -> v0.10.0.
- warp/{backend,verifier_backend,service}: UnsignedMessage -> SignedCore;
NewMessage -> NewWarpEnvelope; sign/verify over BeamSigningBytes(core.ID()).
- precompile/contracts/warp: ParseMessage -> ParseWarpEnvelope; .UnsignedMessage
-> .Core; .Signature.(*BitSetSignature) -> .Beam (concrete). VerifyPredicate
verifies the Beam over D with exported primitives (Bits + bls +
BeamSigningBytes) because the precompile quorum denominator must include
keyless validators, which warp.VerifyEnvelope/GetCanonicalValidatorSet reject.
- precompileconfig.WarpMessageWriter.AddMessage(*SignedCore) (mirrors precompile v0.10.0).
- plugin/evm/vm.go: warpVerifierAdapter over *SignedCore; collapse the duplicate
external-warp import alias (warptypes -> luxwarp).
- remove redundant warp/{aggregator,bls_adapter,signer_adapter}.go: provably-dead
duplicates of warp.SignatureAggregator / warp.NewSigner (zero production refs).
- tests migrated to ZAP; fixed a pre-existing offchain message-key test bug;
revived the predicate suite (validator state injected via Runtime, the
production path, replacing the decoupled WithValidatorState key).
Verified: go build ./... green; go test ./warp/... ./precompile/contracts/warp/...
green (incl. 10k-validator TestWarpPredicate). grep UnsignedMessage = 0.
First-run network, no legacy, no pre-activation history to protect, so the
DexSettleActivationTime dated-fork gate is pure complexity. Delete the concept of
an activation time: 0x9999 dispatches unconditionally and its EXTCODESIZE marker
installs at genesis (parentTimestamp==nil); StrictPQ pinned active-from-genesis for
the Lux plugin (whole ecosystem is PQ-native — removed the per-VM config conditional).
Deleted: DexSettleActivationTime const + pointer/accessor/predicates, the cross-repo
mirror, the drift-guard test, and the 'behaves-as-plain-account-before-activation'
replay-safety branches/prose. One activation path; less code. Genesis determinism via
the single core/genesis.go toBlock() derivation. Feature tests green (0x9999 registered/
dispatched/marked-at-genesis; StrictPQ genesis+nil-inert; plugin PQ).
The 0x9999 DEX value path is identity-gated: the asset resolver refuses to
admit a market unless the running chain's identity matches. core.NewBlockChain
re-executes accepted blocks during construction (loadLastState recovery /
populateMissingTries backfill), but the identity was only bound onto the chain
AFTER eth.New returned (vm.go SetConsensusContext) — so an unclean restart after
a 0x9999 swap re-executed that block with networkID 0 / C-Chain Empty, reverted
the committed swap, failed ValidateState, and the node could not boot.
Fix is purely timing: hand NewBlockChain the *runtime.Runtime the VM already
holds (eth.Settings.ChainRuntime -> core.NewBlockChain) and bind the consensus
context BEFORE reprocess. No new identity type — networkID/chainID already live
on runtime.Runtime; the context is just the last-mile transport to geth's
ConsensusContext API. stdlib runtime aliased stdruntime in blockchain.go.
zeekayandHanzo Dev