indexer/LLM.md

Explorer

Overview

Single-binary block explorer. Indexes EVM and multi-chain networks, serves explorer API v2, embeds the frontend. SQLite by default, PQ encrypted streaming backups to S3. No PostgreSQL, no Elixir, no external services.

Package Information

• Module: github.com/luxfi/explorer
• Binary: explorer (NOT lux-explorer)
• Go: 1.26.1
• Image: ghcr.io/luxfi/explorer

Architecture

explorer (single binary)
│
├── Indexer goroutines      Write → SQLite (WAL, sole writer)
├── Base HTTP server        Read → SQLite (RO, 8 conns)
│   ├── /v1/explorer/*      Explorer API v2 REST
│   ├── /v1/explorer/graphql GraphQL
│   ├── /v1/explorer/ws     WebSocket/SSE realtime
│   └── /*                  Embedded Next.js frontend (go:embed)
├── Replicate sidecar       WAL → S3 (age PQ encrypted)
└── ZapDB KV             Fast hash/height lookups

Storage

• SQLite WAL: default, zero-config, concurrent readers
• ZapDB: KV layer for O(1) hash lookups
• PostgreSQL: optional via -tags postgres
• No Redis: ZapDB replaces it

PQ Encrypted Streaming Backups

Every SQLite WAL write is streamed to S3, encrypted with post-quantum cryptography:

• Algorithm: ML-KEM-768 + X25519 hybrid (NIST FIPS 203)
• Library: luxfi/age v1.4.0
• Transport: hanzoai/replicate WAL streaming
• Chunk size: 64KB ChaCha20-Poly1305 AEAD per chunk
• Forward secrecy: fresh ephemeral keys per WAL segment
• Restore: point-in-time to any TXID from S3

SQLite WAL → Replicate → age (ML-KEM-768 ∥ X25519) → S3

Enable: REPLICATE_S3_ENDPOINT + REPLICATE_AGE_RECIPIENT env vars.

Rust Services (Optional)

Pure Go implementations exist for everything. Rust originals available as static library via CGO:

explorer-rs/ffi/ → libexplorer_ffi.a (staticlib)
  ├── smart-contract-verifier  C ABI: explorer_verify_solidity()
  ├── sig-provider             C ABI: explorer_lookup_signature()
  └── memory mgmt              C ABI: explorer_free()

rustffi/verifier.go            Go CGO bindings (build tag: rustffi)

Build without Rust: go build ./cmd/explorer/ (default, pure Go) Build with Rust: CGO_ENABLED=1 go build -tags rustffi ./cmd/explorer/

Directory Structure

cmd/explorer/           Unified binary entry point
cmd/indexer/            Per-chain indexer (legacy)
cmd/evmchains/          Multi-EVM (shared DB)
cmd/multichain/         100+ chain parallel indexer

explorer/               API layer (imports hanzoai/base as library)
  explorer.go           Plugin registration, read-only SQLite conn
  handlers.go           /v1/explorer/* route handlers (30 endpoints)
  format.go             Explorer v2 response formatters
  collections.go        User data: watchlists, tags, custom ABIs
  testutil/factory.go   Test data factory

evm/                    EVM indexer
  adapter.go            Main indexer (2330 lines)
  api/server.go         REST + GraphQL + WebSocket + JSON-RPC
  api/repository.go     Database queries
  api/graphql.go        GraphQL schema
  contracts/            Verification, proxy detection, ABI decode
  defi/                 DeFi protocol indexing
  account/              User accounts, API keys
  erc4337.go            Account Abstraction
  mev.go                MEV detection
  blob.go               EIP-4844

storage/                Dual-layer storage
  unified.go            KV + Query combined
  kv/                   ZapDB
  query/sqlite.go       SQLite (default)
  query/postgres.go     PostgreSQL (build tag)

rustffi/                Optional CGO bindings to Rust static lib
{x,a,b,q,t,z,k}chain/  DAG chain adapters
pchain/                 Platform chain adapter
chain/, dag/            Shared libraries
migrations/             SQL schema (4 files)
e2e/                    End-to-end tests

API Routes

All under /v1/explorer/:

Endpoint	Description
GET /blocks	List blocks (cursor pagination)
GET /blocks/{id}	Block by hash or number
GET /blocks/{id}/transactions	Block's transactions
GET /transactions	List transactions
GET /transactions/{hash}	Transaction details
GET /transactions/{hash}/token-transfers	Token transfers in tx
GET /transactions/{hash}/internal-transactions	Internal calls
GET /transactions/{hash}/logs	Event logs
GET /addresses/{hash}	Address details + balance
GET /addresses/{hash}/transactions	Address transactions
GET /addresses/{hash}/tokens	Token balances
GET /tokens	List tokens
GET /tokens/{addr}/holders	Token holders
GET /smart-contracts/{addr}	Verified contract + ABI
POST /smart-contracts/{addr}/verify	Submit verification
GET /search	Full-text search
GET /stats	Chain statistics
POST /graphql	GraphQL queries
WS /ws	Realtime subscriptions

White-Label

Zero branding in code. Configure at runtime:

explorer --chain-name="My Chain" --coin=MYC --chain-id=12345

EVM Index Integrity

evm/indexer.go holds each chain as a contiguous run of heights, each with all of its transactions:

• A block row marks its height held and is written last, after the block's transactions, logs and transfers. The head loop resumes from MAX+1.
• A null block, or null receipts for a block with transactions, is ErrNoBlock: a backend behind a load-balanced RPC that does not hold it yet. The height is retried, never skipped.
• A transaction without a hash is refused with its block. No row is ever keyed by the empty hash.
• Receipts come from eth_getBlockReceipts. A node answering MethodNotFound (-32601, transport.RPCError) is read per transaction with eth_getTransactionReceipt from then on (zood). A node serving neither yields transactions without results.
• indexBlock can be repeated safely. Rows are replaced, and address tx counts and token balances move only for a transaction or transfer not already stored. Rows at the height under another block hash are deleted.
• The audit (at start, then every 5 minutes) deletes hashless rows, fills missing heights, and reads again every block whose transaction rows do not match its tx_count or its hash. It reads only above the last clean audit, compares 10000 heights per pass, and compares totals before any per-block query. A pass reads at most 256 heights again. While work remains, a pass follows every head poll.
• A relaunched chain is emptied and reindexed from its new blocks. The check starts when the head stays more than reorgDepth (128) under the cursor for regenesisConfirmations (3) polls running. It is a relaunch if block 0's hash changed, or if the chain's block at the lowest height above genesis the index holds is a hash the index does not hold. The second test catches a restart from the same genesis. It compares against the stored rows, so it holds across an explorer restart. A lagging node serves that block unchanged, and a null proves nothing, so neither erases anything. All eight tables in evmTables are emptied, including evm_token_instances.

EVM Feature Parity

What the Go indexer covers, and where it stops:

• Block indexing (consensus, uncle, EIP-4844 blobs)
• Transaction types 0-3 (legacy, access list, EIP-1559, EIP-4844)
• Internal transaction tracing (3 tracers: callTracer, jsTracer, parityTracer)
• Token tracking: ERC-20, ERC-721, ERC-1155 (single + batch), ERC-404. One decoder reads them all — evm/decodeTransfers — so a movement has one reading whichever indexing path saw the block. ERC-20 and ERC-721 share the Transfer signature and are told apart by topic arity: the ERC-721 id is indexed and rides in topics[3].
• NFT items: /tokens/{addr}/instances and /instances/{id} — id, current owner (the destination of the item's last transfer), and the URI the contract gives it. The URI is an eth_call; the document behind it is not fetched, so uri_state says whether the chain answered ("ok"), named nothing ("absent"), or has not been asked ("unread"). image_url and metadata stay null rather than assert media nobody read.
• Contract verification: Solidity (all versions), Vyper, Standard JSON
• Proxy detection: EIP-1967, EIP-1822, EIP-1167, Gnosis Safe, OpenZeppelin
• Account Abstraction: ERC-4337 UserOps, bundlers, paymasters
• MEV detection: sandwich, frontrun, backrun, liquidation
• DeFi: AMM V2/V3, perps, lending, staking, bridges, CLOB orderbook
• evm/defi additionally carries event signatures for Seaport, LooksRare, Blur, X2Y2, Rarible and Zora. Nothing imports the package and no Lux chain has a marketplace contract deployed, so no marketplace is indexed today. migrations/004_nft_marketplace.sql is that same unbuilt venue's schema and is not applied — the live API reads the SQLite tables evm/indexer.go creates.
• Charts, statistics, gas oracle
• User accounts, API keys, watchlists, address tags

Testing

go test ./...                    # All tests (431+ across 22 packages)
go test ./explorer/...           # API layer (35 tests)
go test ./evm/...                # EVM indexer
go test ./storage/...            # Storage backends

Dependencies

Package	Purpose
hanzoai/base	HTTP server, collections, SSE, IAM
hanzoai/replicate	SQLite WAL streaming to S3
luxfi/age	PQ encryption (ML-KEM-768 + X25519)
luxfi/database	ZapDB KV wrapper
luxfi/zap	Zero-copy wire protocol

What This Replaced

Before (6 containers)	After (1 binary)
Elixir backend	evm/ Go package
Rust verifier	evm/contracts/verifier.go
Rust sig-provider	evm/contracts/abi.go
PostgreSQL	SQLite WAL
Redis	ZapDB KV
Graph Node + subgraphs	G-Chain native GraphQL