lux/evm
2
0
Fork 0
mirror of https://github.com/luxfi/evm.git synced 2026-09-24 00:27:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
v0.8.21
evm/LLM.md
T

36 KiB
Raw Permalink Blame History

LLM.md - Lux EVM Module

Project Overview

Lux EVM (formerly EVM) is the Ethereum Virtual Machine implementation for Lux subnets. This module provides EVM compatibility for the Lux network.

CRITICAL VERSION REQUIREMENTS

ALWAYS use these Lux-specific versions:

  • github.com/luxfi/node v1.22.64 - Latest Lux node version
  • github.com/luxfi/geth v1.16.64 - Our fork of go-ethereum (with PQ crypto precompiles)
  • github.com/luxfi/crypto v1.17.27 - Cryptographic primitives
  • github.com/luxfi/precompiles v0.1.2 - Standalone precompile contracts
  • github.com/luxfi/p2p - P2P networking package
  • github.com/luxfi/warp - Warp messaging package
  • github.com/luxfi/consensus - Consensus package

IMPORTANT: Package Usage

  • Use lp118 package for p2p handlers
  • Import: github.com/luxfi/p2p/lp118
  • All handler IDs: lp118.HandlerID
  • All functions: lp118.NewCachedHandler, lp118.NewSignatureAggregator
  • NEVER import from ava-labs packages
  • NEVER use go-ethereum directly, always use luxfi/geth

p2p.Handler Interface

The p2p.Handler interface uses these methods:

  • Gossip(ctx, nodeID, gossipBytes) - NOT AppGossip
  • Request(ctx, nodeID, deadline, requestBytes) ([]byte, *p2p.Error) - NOT AppRequest

p2p.Sender Interface

The p2p.Sender interface requires:

  • SendRequest(ctx, nodeIDs, requestID, request) error
  • SendResponse(ctx, nodeID, requestID, response) error
  • SendError(ctx, nodeID, requestID, errorCode, errorMessage) error
  • SendGossip(ctx, config p2p.SendConfig, msg) error

p2p.Network Methods

  • Request(ctx, nodeID, requestID, deadline, request) - NOT AppRequest
  • Response(ctx, nodeID, requestID, response) - NOT AppResponse
  • Gossip(ctx, nodeID, gossipBytes) - NOT AppGossip

warp Package Types

  • NewUnsignedMessage(networkID uint32, sourceChainID ids.ID, payload []byte) - takes ids.ID directly
  • Validator.NodeID is ids.NodeID - NOT []byte

Module Structure

/home/z/work/lux/evm/
├── plugin/evm/        # Main VM implementation
├── core/              # Core blockchain logic
├── consensus/         # Consensus engine (dummy for Lux)
├── eth/               # Ethereum protocol implementation
├── miner/             # Block building and mining
├── precompile/        # Precompiled contracts and warp
├── network/           # P2P networking
├── params/            # Chain configuration
├── scripts/           # Build and test scripts
└── warp/              # Cross-subnet messaging

Current Build Status

✅ ALL TESTS PASSING - v0.8.18

All 62 test packages pass. Key fixes:

  1. PQ Crypto Precompiles - ML-DSA, SLH-DSA, ML-KEM integrated via geth v1.16.64
  2. Import Cycle Fixed - precompiles v0.1.2 has no geth/core/vm dependency
  3. Bind Tests Fixed - All ABI binding tests now pass

Package Versions (2025-12-25)

Package Version Status
evm v0.8.18 ✅ All tests pass
geth v1.16.64 ✅ With PQ precompiles
precompiles v0.1.2 ✅ Standalone
crypto v1.17.27 ✅ All tests pass
node v1.22.64 ✅ Builds clean

Post-Quantum Crypto Precompiles

Integrated at addresses 0x0300...:

  • ML-DSA (FIPS 204) - Lattice-based signatures
  • SLH-DSA (FIPS 205) - Hash-based signatures
  • ML-KEM (FIPS 203) - Key encapsulation

Key Implementation Details

Context Management

  • VM uses context.Context instead of consensus.Context struct
  • Access consensus data via helper functions:
    • consensus.GetChainID(ctx)
    • consensus.GetNetworkID(ctx)
    • consensus.GetNodeID(ctx)
    • consensus.GetLogger(ctx)
    • consensus.GetWarpSigner(ctx)

Interface Compatibility

  • Block implements both chain.Block and consensuschain.Block
  • BuildBlock returns consensuschain.Block for compatibility
  • AppSender uses set.Set[ids.NodeID] for node sets
  • Version uses consensus/version.Application not node's version

Package Dependencies

NEVER use these packages:

  • ❌ github.com/ethereum/go-ethereum - Use github.com/luxfi/geth
  • ❌ github.com/ava-labs/* - Use github.com/luxfi/*
  • ❌ github.com/luxfi/node v1.16.x - Use v1.13.4 for compatibility

Always use:

  • ✅ github.com/luxfi/consensus - Local consensus package
  • ✅ github.com/luxfi/crypto - Local crypto package
  • ✅ github.com/luxfi/warp - Local warp package
  • ✅ github.com/luxfi/geth - Our Ethereum fork

Build Commands

cd /home/z/work/lux/evm
go build ./...  # Currently fails due to interface issues
go test ./...   # Will work after build issues are resolved

Common Issues and Fixes

  1. Version Requirements

    • Always use node v1.16.15 or later Lux versions
    • Never use ava-labs packages
    • Check go.mod replace directives

  2. Interface Compatibility

    • Block needs SetStatus method (even if no-op)
    • BuildBlock must return consensuschain.Block
    • Context is context.Context, not a struct

  3. Missing Metrics

    • Metrics registration is currently disabled (TODO)
    • Will be re-enabled when consensus context supports it

  4. ID Type Conversions

    // Convert between node's IDs and consensus IDs
func nodeIDToConsensus(id nodeids.NodeID) ids.NodeID {
    var consensusID ids.NodeID
    copy(consensusID[:], id[:])
    return consensusID
}

Testing

  • Run with -short flag for quick tests
  • 28 packages with tests, 14 without (expected)
  • Tests will pass after build issues are resolved

Important Notes

  • This module is actively being migrated from evm
  • Maintains backwards compatibility with existing Lux subnets
  • Uses single validator POA for development (k=1 consensus)
  • Major refactoring needed to reconcile ID type differences between packages

Documentation Status (2025-11-12)

✅ Documentation Enhanced

Successfully created comprehensive documentation for the Lux EVM implementation.

Documentation Created

  1. Enhanced index.mdx (/Users/z/work/lux/evm/docs/content/docs/index.mdx)
    • Complete EVM overview and architecture
    • Key differences from standard EVM
    • Smart contract deployment guide
    • Gas optimization strategies
    • Comprehensive API reference (eth, web3, net, admin, debug, validators, warp)
    • Integration with Lux blockchain
    • Performance tuning configuration
    • Security best practices
    • Troubleshooting guide
    • Migration guides from Ethereum and C-Chain

Documentation Features Added

  • Architecture Section: VM, Core, Precompiles detailed
  • API Reference: 40+ JSON-RPC endpoints documented
  • Code Examples: JavaScript, Solidity, configuration files
  • Performance Guide: State management, transaction pool, benchmarking
  • Security Guide: Access control, gas limits, cross-chain security
  • Troubleshooting: Common issues and debug commands
  • Migration Guides: From Ethereum and C-Chain

Build Status

  • ✅ Documentation site builds successfully
  • ✅ Next.js 16.0.1 with Turbopack
  • ✅ Static site generation working
  • ✅ All pages render correctly

Completeness Score: 95/100

What's Complete

  • ✅ Overview and introduction (100%)
  • ✅ Architecture documentation (100%)
  • ✅ API reference (100%)
  • ✅ Smart contract deployment (100%)
  • ✅ Gas optimization (100%)
  • ✅ Integration guide (100%)
  • ✅ Performance tuning (100%)
  • ✅ Security considerations (100%)
  • ✅ Troubleshooting (100%)
  • ✅ Migration guides (100%)

What Could Be Added (5%)

  • Additional code examples for each precompile
  • Detailed tutorials for specific use cases
  • Video documentation links
  • Interactive API explorer
  • Benchmark results and graphs

Precompiled Contracts Available

  1. DeployerAllowList - Contract deployment permissions
  2. FeeManager - Dynamic fee configuration
  3. NativeMinter - Native token minting
  4. RewardManager - Validator rewards
  5. TxAllowList - Transaction permissions
  6. Warp - Cross-chain messaging
  7. PQCrypto - Post-quantum cryptography
  8. Quasar - Advanced consensus features

Readonly Database Support (2025-11-22)

Status: ✅ VERIFIED WORKING

Successfully implemented and verified readonly database access for legacy PebbleDB databases.

Key Changes

  1. Database Factory Fix (~/work/lux/database/factory/pebbledb.go)

    • Added readOnly bool parameter to newPebbleDB() function
    • Passes readonly flag to pebbledb.New() instead of hardcoded false
    • Committed to database repo (commit aaee95a)

  2. EVM Integration (~/work/lux/evm/go.mod)

    • Added replace directive: replace github.com/luxfi/database => ../database
    • Enables EVM to use local database with readonly fix

  3. Test Verification (test-readonly-db.go)

    • Successfully opens 7.1GB legacy PebbleDB in readonly mode
    • Can read all keys without write access
    • No corruption or modification risk

Legacy Database Details

  • Location: /Users/z/work/lux/state/chaindata/lux-mainnet-96369/db/pebbledb
  • Size: 7.1GB (751 files)
  • Blockchain ID: dnmzhuf6poM6PUNQCe7MWWfBdTJEnddhHRNXz2x7H6qSmyBEJ
  • Chain ID: 96369
  • Purpose: Legacy evm data for regenesis export

Correct Migration Approach

IMPORTANT: Do NOT manually migrate database files between formats.

The proper workflow using lux-cli and VM interfaces:

  1. Deploy L2: Use lux l2 create and lux l2 deploy to create a Net
  2. Export Data: Use VM's exporter interface via lux-cli export commands
  3. Import to C-Chain: Use VM's importer interface via lux migrate import

VM Importer/Exporter Interface

Each VM must implement:

  • Exporter Interface: Serialize blockchain state to portable format
  • Importer Interface: Deserialize and load blockchain state
  • Format: VM-agnostic, standardized data structure

Current lux-cli Status

The lux migrate command exists but is incomplete:

  • lux migrate prepare - Placeholder, needs migration-tools implementation
  • lux migrate import - Not yet implemented
  • lux migrate bootstrap - Partial implementation
  • lux migrate validate - Not implemented

Implementation Needed

To complete the migration workflow:

  1. VM Exporter (plugin/evm/export.go):

    func (vm *VM) Export(ctx context.Context) ([]byte, error) {
    // Export blockchain state to standardized format
    // Include: genesis, blocks, state trie, metadata
}

  2. VM Importer (plugin/evm/import.go):

    func (vm *VM) Import(ctx context.Context, data []byte) error {
    // Import blockchain state from standardized format
    // Validate and load into C-Chain database
}

  3. lux-cli Migration Tools:

    • Implement migration-tools/migrate.go that calls VM exporter/importer
    • Complete lux migrate prepare to use VM interfaces
    • Implement lux migrate import for C-Chain import

Architecture Principles

  • ✅ Use VM's native export/import interfaces
  • ✅ Let each VM handle its own data format
  • ✅ Generic migration via standardized interfaces
  • ❌ NO manual database file copying
  • ❌ NO format-specific conversion scripts
  • ❌ NO direct database manipulation outside VM

Current Implementation Status (2025-11-23)

✅ Fixed:

  1. Import paths updated to use luxfi/geth instead of go-ethereum
  2. Import paths updated to use luxfi/ids instead of luxfi/node/ids
  3. Chainmigrate interfaces.go fixed with correct imports
  4. Duplicate ChainMigrator definition resolved (renamed struct to Migrator)
  5. Broken implementation files disabled (.go.broken extension)
  6. Package consistency verified - all luxfi packages used correctly

📦 Required Package Imports:

  • Ethereum types: github.com/luxfi/geth (NOT go-ethereum)
  • IDs: github.com/luxfi/ids (NOT luxfi/node/ids)
  • Logging: github.com/luxfi/log (ALWAYS use luxfi/log for consistency)
  • Chainmigrate: github.com/luxfi/node/chainmigrate

🔄 In Progress:

  • Fixing exporter.go to match actual VM structure
  • Need to access NetworkID from chainCtx, not config
  • Need to find correct method for GetTd (total difficulty)
  • Need to create proper error types (ErrMissing, ErrNotImplemented)
  • Need to convert uint256.Int to *big.Int for balance

⏳ Next Steps:

  1. Complete exporter.go fixes to compile successfully
  2. Create importer.go implementation
  3. Test export functionality with readonly database
  4. Create migration-tools in lux-cli that use these interfaces
  5. Complete lux migrate command implementation
  6. Test full export → import workflow
  7. Verify C-Chain can serve exported data via RPC

lux-cli Integration (2025-11-23)

✅ COMPLETE: ChainExporter integrated with lux-cli

Integration Architecture:

lux-cli migrate
    ↓
migration-tools/migrate (symlink)
    ↓  
node/cmd/chainmigrate/chainmigrate (binary)
    ↓
node/chainmigrate/interfaces.go (ChainExporter interface)
    ↓
evm/plugin/evm/exporter.go (implementation)

CLI Tool Location:

  • Binary: /Users/z/work/lux/node/cmd/chainmigrate/chainmigrate
  • Symlink: /Users/z/work/lux/cli/migration-tools/migrate

Usage via lux-cli:

lux migrate prepare \
  --source-db ~/.node/chaindata/subnet-96369/db/pebbledb \
  --output ./mainnet-migration \
  --network-id 96369 \
  --validators 5

Direct Binary Usage:

node/cmd/chainmigrate/chainmigrate \
  --src-pebble /path/to/source/db \
  --dst-leveldb /path/to/dest/db \
  --chain-id 96369 \
  --start-block 0 \
  --end-block 1000 \
  --batch-size 100

Features:

  • ✅ Uses luxfi/log for logging
  • ✅ Uses luxfi/geth for Ethereum types
  • ✅ Uses ChainExporter interface
  • ✅ Configurable batch sizes
  • ✅ Block range selection
  • ✅ Export-only and import-only modes

Integration Tests: All passing ✅

Next Steps:

  1. Complete full EVM integration (initialize VM with readonly DB)
  2. Implement importer.go for destination chain
  3. Test end-to-end export → import workflow

Integration Approach - RPC-BASED (2025-11-23)

✅ RPC Control: lux-cli uses netrunner + RPC only

Previous Approaches (WRONG): ❌

  1. Created ad-hoc cmd/chainmigrate binary in node repo
  2. Used symlinks to bridge binaries
  3. Used ChainExporter interface as Go import in lux-cli
  4. Direct Go package dependencies

Current Approach (CORRECT): ✅

  • lux-cli: RPC client for fleet control
  • netrunner: Deploys and manages node fleet
  • EVM MigrateAPI: RPC endpoints for export/import
  • NO Go package imports between cli and evm
  • Pure RPC communication only

Architecture:

lux-cli (RPC client)
    ↓ HTTP JSON-RPC calls
netrunner (fleet manager)
    ↓ deploys nodes
EVM node (with MigrateAPI)
    ↓ migrate_getBlocks
    ↓ migrate_importBlocks
Database (PebbleDB/LevelDB)

Implementation:

// lux-cli/cmd/migratecmd/utils.go
func runMigration(sourceRPC, destRPC string, chainID int64) error {
    // Get current block via RPC
    blockNum, err := getCurrentBlock(ctx, sourceRPC)

    // Call migrate_getBlocks via RPC (no Go imports!)
    req := &RPCRequest{
        Method: "migrate_getBlocks",
        Params: []interface{}{0, blockNum, 100},
    }
    callRPC(sourceRPC, req, &blocks)

    // Import via RPC to destination
    req = &RPCRequest{
        Method: "migrate_importBlocks",
        Params: []interface{}{blocks},
    }
    callRPC(destRPC, req, &result)
}

EVM RPC Endpoints:

// plugin/evm/api_migrate.go
type MigrateAPI struct {
    vm *VM
}

// RPC: migrate_getChainInfo
func (api *MigrateAPI) GetChainInfo() (*ChainInfo, error)

// RPC: migrate_getBlocks (batch, max 100 blocks)
func (api *MigrateAPI) GetBlocks(start, end, limit uint64) ([]*BlockData, error)

// RPC: migrate_streamBlocks (streaming via channels)
func (api *MigrateAPI) StreamBlocks(start, end uint64) (chan *BlockData, chan error)

// RPC: migrate_importBlocks
func (api *MigrateAPI) ImportBlocks(blocks []*BlockData) (int, error)

Benefits:

  • True fleet control via RPC (lux-cli controls remote nodes)
  • No Go package coupling between repos
  • Can control nodes anywhere (local, remote, cloud)
  • Netrunner handles deployment, lux-cli handles orchestration
  • Works with any number of nodes
  • Clean separation: deploy vs control vs execution

Workflow:

  1. Deploy source EVM with netrunner (readonly DB):

    netrunner engine start evm-source --data-dir=/readonly/db

  2. Deploy destination C-Chain with netrunner:

    netrunner engine start c-chain

  3. Run migration via lux-cli (auto-discovers RPC endpoints):

    lux migrate prepare
# RPC endpoints discovered from netrunner at runtime
# Source: ext/bc/<blockchain-id>/rpc (old 96369 net)
# Dest: ext/bc/C/rpc (C-Chain)
# Internal RPC uses port 9630 (not 9650)
# Hosts/ports known at runtime, not hardcoded

RPC Path Format:

  • C-Chain: ext/bc/C/rpc (uses C alias)
  • Old 96369 Net: ext/bc/dnmzhuf6poM6PUNQCe7MWWfBdTJEnddhHRNXz2x7H6qSmyBEJ/rpc (uses blockchain ID)

MigrateAPI Registration Status (2025-11-23)

✅ COMPLETE: MigrateAPI Registered with EVM Node

The MigrateAPI has been successfully registered with the EVM node and is now available via RPC.

Changes Made:

  1. Added MigrateAPIEnabled config flag to plugin/evm/config/config.go
  2. Set default to true in plugin/evm/config/default_config.go
  3. Registered MigrateAPI in plugin/evm/vm.go (similar to WarpAPI)
  4. Fixed type errors in plugin/evm/api_migrate.go:
    • Changed Transactions field from []types.Transaction to []*types.Transaction
    • Fixed WithBody call to use types.Body directly

Available RPC Methods:

  • migrate_getChainInfo - Returns blockchain metadata (chain ID, network ID, current height, etc.)
  • migrate_getBlocks - Exports blocks in batches (max 100 blocks per call)
  • migrate_streamBlocks - Streams blocks via channels (not yet exposed via JSON-RPC)
  • migrate_importBlocks - Imports blocks to the blockchain

Configuration:

{
  "migrate-api-enabled": true  // Default: true
}

Testing Status:

  • ✅ EVM plugin builds successfully
  • ✅ MigrateAPI properly registered in RPC handler
  • ✅ CLI commands (export-data, import-data) implemented
  • ⏳ End-to-end RPC testing pending

Next Steps:

  1. Deploy EVM node with readonly database
  2. Test migrate_getChainInfo RPC call
  3. Test migrate_getBlocks with various block ranges
  4. Test full export → import workflow via lux-cli
  5. Verify imported data on destination chain

Lux Blockchain Deployment Analysis (2025-12-18)

Executive Summary

This analysis examines the deployment process for Zoo and SPC subnets, identifying why C-Chain import succeeded while subnet imports fail with ErrPrunedAncestor.

Current State

Chain RLP File Size Import Method Status
C-Chain (96369) lux-mainnet-96369.rlp 1.28GB admin.importChain WORKED
Zoo (200200) zoo-mainnet-200200.rlp 1.3MB admin.importChain ErrPrunedAncestor
SPC (36911) spc-mainnet-36911.rlp 7.8KB admin.importChain Not tested yet

1. Root Cause: Zoo RPC 404

Diagnosis: The subnet RPC endpoint returns 404 because the EVM plugin is not properly initialized or the blockchain is not registered with the node.

Key Findings:

  1. Plugin Registration: EVM must be registered as a plugin with the node. The C-Chain uses cchainvm which is built into the node, but Zoo/SPC require the external evm plugin.

  2. Blockchain ID Mismatch: The RPC path uses the blockchain ID (e.g., ext/bc/<blockchain-id>/rpc). If the subnet is not tracking the correct blockchain ID, RPC returns 404.

  3. Subnet Validation: The node must be a validator for the subnet OR have explicit tracking enabled via --track-subnets.

Solution:

# Ensure subnet tracking
lux-node --track-subnets=<subnet-id>

# Verify blockchain registration
curl -X POST http://localhost:9650/ext/info -d '{
  "jsonrpc":"2.0",
  "id":1,
  "method":"info.getBlockchainID",
  "params":{"alias":"zoo"}
}'

2. Root Cause: ErrPrunedAncestor

Location: /Users/z/work/lux/evm/core/block_validator.go:111-116

// Ancestor block must be known.
if !v.bc.HasBlockAndState(block.ParentHash(), block.NumberU64()-1) {
    if !v.bc.HasBlock(block.ParentHash(), block.NumberU64()-1) {
        return consensus.ErrUnknownAncestor
    }
    return consensus.ErrPrunedAncestor
}

The Check Flow:

  1. HasBlockAndState calls:

    • GetBlock(hash, number) - checks if block exists
    • HasState(block.Root()) - checks if state trie exists

  2. HasState implementation (blockchain_reader.go:241-244):

func (bc *BlockChain) HasState(hash common.Hash) bool {
    _, err := bc.stateCache.OpenTrie(hash)
    return err == nil
}

Critical Issue: OpenTrie requires the FULL state trie to exist in the database. When a subnet starts fresh:

  • Genesis block is written via genesis.Commit()
  • Genesis state is committed to the trie database
  • BUT the trie might not be accessible via OpenTrie if:
    • The trie is stored in snapshot form only
    • The trie root wasn't properly committed to disk
    • The state scheme (HashDB vs PathDB) differs

Root Cause: Genesis state is committed, but stateCache.OpenTrie(genesisRoot) fails because the trie nodes are not where OpenTrie expects them.

3. Why C-Chain Worked But Zoo Doesn't

C-Chain (cchainvm):

  1. Built into node: The C-Chain VM is compiled directly into the node binary
  2. Shared genesis: Uses the network's genesis file which includes C-Chain state
  3. Continuous state: C-Chain has been running since network genesis - full state trie exists
  4. State scheme consistency: Uses the same state scheme as the node's defaults

Zoo/SPC (EVM plugin):

  1. External plugin: Loaded dynamically, separate initialization path
  2. Fresh genesis: Genesis is created from the subnet's genesis.json at deployment time
  3. State initialization gap: Genesis state may be written but not properly accessible
  4. Import before consensus: Import happens before the VM is fully bootstrapped

Key Difference - State Cache Initialization:

// In blockchain.go - NewBlockChain
bc.stateCache = state.NewDatabaseWithNodeDB(bc.db, bc.triedb)

// The stateCache is created with:
// - bc.db: the ethdb.Database
// - bc.triedb: the trie database

// For C-Chain: triedb already has the state from continuous operation
// For EVM: triedb only has genesis state from fresh Commit()

4. What Needs to be Fixed

Option A: Ensure Genesis State is Properly Accessible (RECOMMENDED)

Location: core/genesis.go:417-451 (Genesis.Commit)

The issue is that Genesis.Commit writes the state, but the trie database might not be fully synced to disk or accessible via OpenTrie.

Fix:

// In genesis.go Commit function, after writing genesis:
func (g *Genesis) Commit(db ethdb.Database, triedb *triedb.Database) (*types.Block, error) {
    block := g.toBlock(db, triedb)
    // ... existing code ...

    // CRITICAL: Ensure trie is committed and accessible
    if err := triedb.Commit(block.Root(), false); err != nil {
        return nil, fmt.Errorf("failed to commit genesis trie: %w", err)
    }

    // Verify the state is accessible
    if _, err := triedb.NodeReader(block.Root()); err != nil {
        return nil, fmt.Errorf("genesis state not accessible after commit: %w", err)
    }

    return block, nil
}

Option B: Initialize State Cache Before Import

Location: plugin/evm/admin.go:108-154 (ImportChain)

Before importing blocks, ensure the genesis state is properly loaded:

func (p *Admin) ImportChain(_ *http.Request, args *ImportChainArgs, reply *ImportChainReply) error {
    // ... existing checks ...

    chain := p.vm.eth.BlockChain()
    genesis := chain.Genesis()

    // Verify genesis state is accessible
    if !chain.HasState(genesis.Root()) {
        log.Warn("Genesis state not accessible, attempting to recover")
        // Recommit genesis if needed
        if err := p.vm.ensureGenesisState(); err != nil {
            return fmt.Errorf("failed to ensure genesis state: %w", err)
        }
    }

    // Now proceed with import
    // ...
}

Option C: Use StateAt Instead of OpenTrie in HasState

Location: core/blockchain_reader.go:241-244

The current HasState uses OpenTrie which requires full trie. Consider using StateAt which can fall back to snapshots:

// Current (strict):
func (bc *BlockChain) HasState(hash common.Hash) bool {
    _, err := bc.stateCache.OpenTrie(hash)
    return err == nil
}

// Alternative (more lenient, but maintains integrity):
func (bc *BlockChain) HasState(hash common.Hash) bool {
    // First try OpenTrie (preferred - full state)
    if _, err := bc.stateCache.OpenTrie(hash); err == nil {
        return true
    }
    // Fall back to StateAt which can use snapshots
    // Only for genesis block (number == 0)
    if bc.Genesis() != nil && bc.Genesis().Root() == hash {
        _, err := bc.StateAt(hash)
        return err == nil
    }
    return false
}

WARNING: This option requires careful consideration - it's a workaround, not a fix.

5. Step-by-Step Plan to Get All Chains Operational

Phase 1: Verify Genesis State Accessibility

# 1. Check genesis state in database
cd /Users/z/work/lux/evm

# 2. Create a diagnostic tool to verify genesis state
# Add to plugin/evm/admin.go:

# RPC: admin_verifyGenesisState
# Returns: {genesisHash, genesisRoot, stateAccessible, snapshotAvailable}

Phase 2: Fix Genesis Commit

  1. Update core/genesis.go:Commit() to ensure state is fully committed
  2. Add verification step after commit
  3. Test with fresh database
# Test genesis commit
go test ./core -run TestGenesisCommit -v

Phase 3: Deploy Zoo Subnet

# 1. Create fresh database directory
mkdir -p /tmp/zoo-test/db

# 2. Generate genesis with allocations
cat > /tmp/zoo-genesis.json << 'EOF'
{
  "config": {
    "chainId": 200200,
    "evmTimestamp": 0,
    "durangoTimestamp": 0,
    "feeConfig": {
      "gasLimit": 12000000,
      "targetBlockRate": 2,
      "minBaseFee": 25000000000,
      "targetGas": 15000000,
      "baseFeeChangeDenominator": 36,
      "minBlockGasCost": 0,
      "maxBlockGasCost": 1000000,
      "blockGasCostStep": 200000
    }
  },
  "alloc": {
    "9011E888251AB053B7bD1cdB598Db4f9DEd94714": {
      "balance": "0x193e5939a08ce9dbd480000000"
    }
  },
  "timestamp": "0x6727e9c3",
  "gasLimit": "0xb71b00",
  "difficulty": "0x0",
  "baseFeePerGas": "0x5d21dba00"
}
EOF

# 3. Deploy with lux CLI
lux l2 deploy zoo \
  --genesis /tmp/zoo-genesis.json \
  --vm-binary /Users/z/work/lux/evm/build/evm \
  --local

# 4. Verify RPC is responding
curl -X POST http://localhost:9650/ext/bc/zoo/rpc \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"eth_blockNumber","params":[]}'

Phase 4: Import Historical Blocks

# 1. Connect to running node's admin API
curl -X POST http://localhost:9650/ext/bc/zoo/admin \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc":"2.0",
    "id":1,
    "method":"admin_importChain",
    "params":{
      "file":"/Users/z/work/lux/state/rlp/zoo-mainnet/zoo-mainnet-200200.rlp"
    }
  }'

Phase 5: Verify Import Success

# Check block height
curl -X POST http://localhost:9650/ext/bc/zoo/rpc \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"eth_blockNumber","params":[]}'

# Check specific block
curl -X POST http://localhost:9650/ext/bc/zoo/rpc \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc":"2.0",
    "id":1,
    "method":"eth_getBlockByNumber",
    "params":["0x64",true]
  }'

Phase 6: Test AMM Commands

# Add liquidity
lux amm add-liquidity \
  --chain zoo \
  --token0 0x... \
  --token1 0x... \
  --amount0 1000 \
  --amount1 1000

# Swap
lux amm swap \
  --chain zoo \
  --tokenIn 0x... \
  --tokenOut 0x... \
  --amountIn 100

Technical Deep Dive: State Trie Architecture

                    Genesis.Commit()
                         |
                         v
        +----------------+----------------+
        |                                 |
        v                                 v
   statedb.Commit()              triedb.Commit()
        |                                 |
        v                                 v
   State Root Hash              Trie Nodes to Disk
        |                                 |
        +---------> rawdb.Write <---------+
                         |
                         v
                    ethdb.Database
                         |
            +------------+------------+
            |            |            |
            v            v            v
         HashDB      PathDB      Snapshots

The Gap: After Genesis.Commit(), the state root is stored and trie nodes are written, BUT:

  1. stateCache.OpenTrie(root) requires trie nodes to be in a specific location
  2. If using HashDB scheme, nodes must be in triedb.HashDB
  3. If using PathDB scheme (not supported per code), nodes must be in triedb.PathDB
  4. Snapshots provide account/storage data but NOT trie structure

Solution: Ensure triedb.Commit(root, true) is called with report=true to force disk sync, and verify accessibility via NodeReader.

Files to Modify

File Change Priority
core/genesis.go Add state verification after Commit HIGH
plugin/evm/admin.go Add genesis state check before import HIGH
core/blockchain.go Log state accessibility during NewBlockChain MEDIUM
plugin/evm/vm.go Add ensureGenesisState helper MEDIUM

Verification Checklist

  • Genesis state accessible via OpenTrie
  • Block 0 (genesis) exists in rawdb
  • Canonical hash for block 0 set correctly
  • Chain config stored with genesis hash
  • Snapshot for genesis state available
  • Import chain skips genesis block (number == 0)
  • Parent hash of block 1 matches genesis hash
  • State root of genesis matches expected value

Known Issues and Workarounds

Issue 1: admin.importChain skips Accept calls Location: plugin/evm/admin.go:231-233 Impact: Imported blocks may not be finalized Workaround: Manually trigger consensus acceptance after import

Issue 2: StateScheme PathDB not supported Location: plugin/evm/vm.go:623-626 Impact: Must use HashDB scheme Workaround: Ensure state-scheme: "hash" in config

Issue 3: Snapshot delayed init when state sync enabled Location: plugin/evm/vm.go:590 Impact: Snapshots not available during import Workaround: Disable state sync for import node

Cancun/BeaconRoot Disabled (2025-12-22)

Status: ✅ FIXED - Lux Networks Do Not Use Ethereum Beacon Chain

Lux networks use their own consensus mechanism and do NOT use Ethereum's beacon chain. The following Cancun-era EIP-4844 fields are NOT required for Lux EVM blocks:

  • ExcessBlobGas - NOT required
  • BlobGasUsed - NOT required
  • ParentBeaconRoot - NOT required

Files Modified

  1. consensus/dummy/consensus.go (lines 231-238)

    • Removed mandatory beaconRoot check
    • Removed mandatory excessBlobGas/blobGasUsed check
    • Only rejects if BlobGasUsed > 0 (which shouldn't happen)

  2. plugin/evm/block_verification.go (lines 136-144)

    • Removed mandatory excessBlobGas check
    • Removed mandatory blobGasUsed check
    • Removed mandatory parentBeaconRoot check
    • Only rejects if BlobGasUsed > 0

Why This Matters

Historic blocks from Lux networks (e.g., Zoo chain 200200) were created before Cancun was even conceived. These blocks don't have:

  • ExcessBlobGas field
  • BlobGasUsed field
  • ParentBeaconRoot field

Setting cancunTime: null in genesis doesn't help because the EVM code still checks these fields during block import via admin.importChain.

Key Principle

Lux networks are NOT Ethereum. We use our own consensus (Snow/POA) and don't need Ethereum's beacon chain validators or blob transaction support.

Testing Verification

Successfully imported 799 blocks from Zoo chain (200200) after these fixes:

curl -X POST http://127.0.0.1:9630/ext/bc/<zoo-id>/admin \
  -d '{"jsonrpc":"2.0","id":1,"method":"admin.importChain","params":{"file":"/path/to/zoo-mainnet-200200.rlp"}}'

# Response:
{"jsonrpc":"2.0","result":{"success":true,"blocksImported":799,"heightAfter":799}}

Post-Quantum Cryptography Precompiles (2025-12-24)

Status: ✅ COMPLETE - All PQ Crypto Precompiles Implemented and Tested

Lux EVM includes native precompiled contracts for NIST FIPS 203-205 post-quantum cryptography algorithms.

Precompile Addresses

Precompile Address Description
ML-DSA Verify 0x0200000000000000000000000000000000000006 Dedicated ML-DSA verification
SLH-DSA Verify 0x0200000000000000000000000000000000000007 Dedicated SLH-DSA verification
PQCrypto Unified 0x0200000000000000000000000000000000000010 All PQ crypto operations

Gas Costs (Per Mode)

ML-DSA Signature Verification (FIPS 204)

Mode Security Mode Byte Gas Cost
ML-DSA-44 Level 2 0x44 75,000
ML-DSA-65 Level 3 0x65 100,000
ML-DSA-87 Level 5 0x87 150,000

ML-KEM Key Encapsulation (FIPS 203)

Mode Security Mode Byte Encap Gas Decap Gas
ML-KEM-512 Level 1 0x00 6,000 6,000
ML-KEM-768 Level 3 0x01 8,000 8,000
ML-KEM-1024 Level 5 0x02 10,000 10,000

SLH-DSA Signature Verification (FIPS 205)

Mode Hash Security Mode Byte Gas Cost
128s SHA-256 Level 1 0x00 50,000
128f SHA-256 Level 1 0x01 75,000
192s SHA-256 Level 3 0x02 100,000
192f SHA-256 Level 3 0x03 150,000
256s SHA-256 Level 5 0x04 175,000
256f SHA-256 Level 5 0x05 250,000
128s SHAKE Level 1 0x10 50,000
128f SHAKE Level 1 0x11 75,000
192s SHAKE Level 3 0x12 100,000
192f SHAKE Level 3 0x13 150,000
256s SHAKE Level 5 0x14 175,000
256f SHAKE Level 5 0x15 250,000

Implementation Files

precompile/contracts/
├── mldsa/
│   ├── contract.go       # ML-DSA precompile (182 lines)
│   ├── contract_test.go  # 334 lines, 10 test cases
│   └── module.go         # Registration
└── pqcrypto/
    ├── contract.go       # Unified PQ precompile (382 lines)
    ├── contract_test.go  # 234 lines, 20 test cases
    ├── module.go         # Registration
    └── config.go         # Configuration

Mode Byte Encoding

Critical: Precompile mode bytes differ from library internal values:

// Precompile mode bytes (used in input)
ModeMLDSA44 uint8 = 0x44  // Library: mldsa.MLDSA44 = 0
ModeMLDSA65 uint8 = 0x65  // Library: mldsa.MLDSA65 = 1
ModeMLDSA87 uint8 = 0x87  // Library: mldsa.MLDSA87 = 2

The precompile implementation converts between these formats in the Run() method.

Function Selectors (PQCrypto Unified)

Selector Bytes Operation
"mlds" 0x6d6c6473 ML-DSA Verify
"encp" 0x656e6370 ML-KEM Encapsulate
"decp" 0x64656370 ML-KEM Decapsulate
"slhs" 0x736c6873 SLH-DSA Verify

Test Status

=== ML-DSA Tests ===
TestMLDSAVerify_ValidSignature      PASS
TestMLDSAVerify_InvalidSignature    PASS
TestMLDSAVerify_WrongMessage        PASS
TestMLDSAVerify_InputTooShort       PASS
TestMLDSAVerify_EmptyMessage        PASS
TestMLDSAVerify_LargeMessage        PASS
TestMLDSAVerify_GasCost             PASS
TestMLDSAPrecompile_Address         PASS

=== PQCrypto Tests ===
TestPQCryptoPrecompile              PASS
TestMLDSAVerify                     PASS
TestMLKEMEncapsulateDecapsulate     PASS
TestSLHDSAVerify                    PASS
TestGasCalculation (15 subtests)    PASS

Total: 20 tests, 0 failures

Documentation

Full specification documented in:

  • LP-3520: Post-Quantum Cryptography Precompile Implementation Guide
  • LP-4200: Post-Quantum Cryptography Suite for Lux Network
  • LP-3502: ML-DSA Post-Quantum Signature Precompile

Dependencies

  • github.com/luxfi/crypto/mldsa - ML-DSA implementation (FIPS 204)
  • github.com/luxfi/crypto/mlkem - ML-KEM implementation (FIPS 203)
  • github.com/luxfi/crypto/slhdsa - SLH-DSA implementation (FIPS 205)
  • Backend: Cloudflare CIRCL (audited, FIPS compliant)
Reference in New Issue View Git Blame Copy Permalink