Structural regression lock for the consensus write-path fix (luxfi/consensus
fix/fetch-on-unknown-vote). The wedge was a SHARED-engine bug, not a dexvm bug:
the dexvm has — and must keep — ZERO networking/gossip primitives. It parses,
verifies, accepts, and matches blocks under consensus but NEVER propagates them;
propagation, vote distribution, and ancestor fetch live in luxfi/consensus.
TestDChainVM_IsPureBlockChainVM walks the package's own non-test .go files and
fails (with file:line) if any references AppGossip, SendAppGossip, PushQuery,
PullQuery, SendChits, common.Sender, or RequestAncestors. Verified to FAIL on an
injected sentinel and PASS on the real tree (12 source files, zero hits).
CGO_ENABLED=0 go test ./pkg/dchain/... green.
dexfillproof compared after.Markets[].PoolID against ids.ID.String() (cb58),
but pkg/dchain/read.go encodes poolId as lowercase hex of the 32-byte id, so
the read-back market-presence assertion never matched and the tool exited 1
even after a real FILL landed. Compare on hex (the canonical wire form). Also
rename the residual clob_ doc/error strings to dex_ — the public surface is
dex_* (the engine's internal clobMethods identifiers stay as-is).
The D-Chain DEX wire methods now use the dex_ namespace (dex_place,
dex_submit, dex_cancel, dex_ensure_market, dex_open_market, dex_deposit,
dex_withdraw, dex_get_markets/book/orders/trades) per the luxd
service-prefix convention (eth_/avm./platform.). On the D-Chain the DEX
IS the CLOB (the AMM lives in C-Chain contracts, not this VM), so the
public wire exposes the chain's purpose, not the internal engine name.
The internal Go identifiers (clobMethods, MethodPlace, the CLOB engine
name) are unchanged — only the on-wire string VALUES move. Covers BOTH
transports via the one clobMethods table (ZAP RegisterCLOB + HTTP
ingest.go httpHandlers) plus read.go GET endpoints. Forwards-only (DEX
not live yet). dexseed + dexfillproof clients aligned.
zapwire + dchain tests green; four-path + venue E2E pass.
The serve goroutine deliberately uses context.Background() (its lifetime is
the VM's, cancelled by Shutdown via zapIngestCancel, not the short-lived
Initialize ctx), so the ctx parameter was never used. Remove it — no slop.
Functionally identical to v1.5.17; the param carried no behavior.
The in-luxd native D-Chain plugin served HTTP only (the dev left the ZAP
socket to standalone cmd/dvenue: handler.go said 'the in-luxd plugin does
NOT use this path'). Owner transport decision: the ZAP socket is the
CANONICAL native ingestion in-luxd (the HFT path — binary wire, streaming,
co-located on the pod), HTTP stays as the compat surface (web/exchange-api/
debug). Both fold over the ONE clobMethods core; transport is orthogonal to
consensus, so it is safe to serve both — an order from either transport hits
the identical submitTx -> mempool.Add -> Verify-match path, and validators
replay from accepted block BYTES, never the wire.
zapingest.go: the VM owns its transports. Initialize parses init.Config for
an optional zapIngestAddr and, when set, opens rpc.Listen co-located in the
plugin process bound to clobMethods (reusing RegisterCLOB, the existing
socket seam) — served concurrently, closed on Shutdown. Enabled by DATA (the
chain config the operator/genesis sets), NOT a build tag or forked entry
point: one binary, one core, the transport set is a per-environment knob.
Empty addr => HTTP-compat-only (in-process tests + cmd/dvenue, which runs its
own socket, are unaffected). Malformed config fails fast at Initialize.
TDD (all !cgo):
- TestZAPIngestCrossingOrdersMatchInBlock: an order over the VM's OWN
config-started socket (no manual RegisterCLOB) -> consensus -> committed
fill.
- TestZAPIngestSharesOneCoreWithHTTP: maker rests over ZAP, taker crosses
over HTTP against the SAME book -> proves one core, two transports.
- TestParseConfig / TestNoZAPIngestWhenUnset / TestZAPIngestStopsOnShutdown:
config contract (forwards-only), no-op path, clean idempotent teardown.
Additive: RegisterCLOB unchanged; CreateHandlers/ingest.go unchanged; the
runner (cmd/dchain) unchanged — the socket is a VM concern driven by config.
The mess was three layers defending a self-inflicted footgun:
(a) cmd/clobverify + cmd/fourpath-live SEED ascii-of-symbol synthetic
markets (poolID = ASCII of "LUX/LUSD"), orders with no orderuser row;
(b) internal/localguard refused those tools off-loopback;
(c) VM.assertOrderUserCoverage BRICKED the chain at boot if a custody
market had a resting order with no orderuser row.
Decomplect-perfect (NOT another guard):
1. DELETE cmd/clobverify + cmd/fourpath-live. Real markets come from
permissionless OpenMarket + real ERC-20s + the maker; the real-market
e2e supersedes them. cmd/dexseed (canonical, structurally synthetic-safe)
stays.
2. DELETE internal/localguard (+ test) — it existed ONLY to fence the
deleted seeders.
3. The settlement-identity invariant already holds BY CONSTRUCTION at order
creation: lockOrderSpend writes orderuser: and putOrderRow writes order:
to the SAME versiondb overlay, committed in ONE atomic Block.Accept
overlay.Commit(); an unfunded place never rests. So a resting custody
order without its orderuser row is UNREPRESENTABLE in committed state ->
the boot scan was provably dead -> DELETE VM.assertOrderUserCoverage.
The single remaining settlement-identity guard is the consensus-time
fail-closed in settleOrderEffects (ErrMissingSettlementUser), which can
never fire on honest committed state.
TDD:
- TestCommittedStateNeverHasOrderWithoutIdentity: drives the real commit
path (rest + reject + fill); committed order: set == orderuser: set.
- TestRestartDoesNotBrickBoot: hand-corrupts state the commit path cannot
produce; boot now SUCCEEDS (no brick); consensus-time guard still fails
closed when a fill touches the corrupt order.
- All prior settlement-identity regression tests still pass.
Result: zero guards, illegal state unrepresentable. Removes the footgun
instead of guarding the footgun we created.
cmd/dexseed seeds REAL markets onto the live in-luxd native D-Chain CLOB
ingestion seam (POST /ext/bc/D/dex/clob_* with FROZEN zapwire frames),
using consensus-native asset identities for every market side.
Identity rule (the property that makes this seeder SAFE against a shared
cluster, unlike the loopback-gated cmd/clobverify): every asset handle is
the dexcore canonical AssetID = DeriveAssetID(networkID, C-chainID, kind,
canonicalRef) — the SAME id the chain's resolver derives — NEVER an
ascii-of-symbol id (the synthetic form, e.g. 0x4c5558 'LUX', that bricked
the chain). A synthetic/malformed ref is refused at frame-build time,
before a single frame is sent; the chain's permissionless resolver then
proves each side is backed by live on-chain code (EXTCODESIZE) before
binding the market.
End-to-end smoke path for one real LUX/<ERC20> market: clob_open_market
(bind real base+quote ids) -> clob_deposit (fund maker base) -> clob_place
(rest maker ask) -> clob_get_markets + clob_get_book (read back & confirm).
Pure, testable core (seed.go): assetSpec.resolveID, poolIDFor (canonical
domain-separated SHA-256 over the derived ids, NON-ascii), depositRefFor,
realMarketFrames. Tests (main_test.go): golden AssetID vectors, OpenMarket
96B + slot equality, synthetic/malformed-ref rejection, deterministic
non-ascii poolID, frozen-frame field round-trips. CGO_ENABLED=0, pure-Go,
luxfi packages only.
clobverify and fourpath-live seed markets with ASCII-of-symbol SYNTHETIC
identities (poolID = symbol bytes; asset ids = ascii of LUX/LUSD). When those
frames reached the live devnet validator set they persisted into the
replication snapshot and BRICKED the native D-Chain VM: dchain.assertOrderUserCoverage
sees a resting order on market 4c55582f4c555344 ("LUX/LUSD") with no orderuser
row at boot and refuses the degraded settlement path.
Add internal/localguard.AssertLocalOnly: each seeding tool now refuses to write
unless (1) the explicit --i-understand-this-seeds-synthetic-test-data flag is
set AND (2) every target (validator base URLs / venue / cchain-rpc) resolves
entirely to loopback. The check runs BEFORE any frame is sent, so the
poison-a-shared-chain path is impossible while local single-process testing is
unchanged. clobverify -read-only is exempt (it never seeds).
IsLoopbackTarget is a pure, fail-closed classifier (URL or host:port; rejects
on parse error, unresolvable name, or any non-loopback resolved IP). Doc
comments at both tool tops mark them LOCAL-ONLY with the brick reason.
Tests: internal/localguard covers (a) non-loopback rejected, (b) loopback
without opt-in rejected, (c) loopback + opt-in allowed, plus k8s-dns /
lux-dev.network / public-ip / 0.0.0.0 classification.
HIGH-1/HIGH-2 (integrity): assetid.go was UNTRACKED — it defines AssetKind, the
{EVMNative,ERC20,UTXO} consts, EVMNativeMarker, DeriveAssetID, and CanonicalRefFor:
the foundational real-asset identity primitive the whole value path keys on. A fresh
worktree at the prior dexcore commit failed to build (undefined: AssetKind ×5 in
place.go/resolver.go). Commit it (+ assetid_test.go) so pkg/dexcore COMPILES and the
value-path branch is reproducible.
assetid.go derives the consensus-native 32-byte AssetID from where an asset really
lives — (networkID, sourceChainID, kind, canonicalRef) — via a length-prefixed,
domain-separated ("lux:dex:asset:v1") SHA-256 fold, byte-identical to
chains/dexvm/registry.DeriveAssetID, so a swap-derived id (this home) and a registered
id (the registry) name the SAME asset by the SAME id, by construction.
MED-1 (golden KAT): assetid_test.go pins AssetIDGoldenVectors — fixed
(networkID, chainID, kind, ref) → expected 32-byte id for all three kinds (incl UTXO).
The mirror test in chains/dexvm/registry asserts the IDENTICAL bytes, locking the
byte-identity the resolve↔register equivalence depends on; a drift in either home's
preimage discipline fails CI in both. Domain-separation test confirms ERC20 vs
EVM_NATIVE never collide and a zero-address ERC-20 / empty source chain is refused.
Verified: CGO_ENABLED=0 go build ./pkg/dexcore clean; full dexcore suite green.
The value-path admission port becomes PERMISSIONLESS + PUBLIC per the owner's clean
model: an asset trades IFF its canonical identity verifies on-chain, with NO
manifest-membership requirement.
- resolver.go: AssetResolver.ResolveEnabled -> ResolveAsset (proves canonical identity
permissionlessly; refuses only malformed/wrong-network/explicitly-disabled/synthetic,
never 'not pre-registered'). RequireEnabledAsset -> RequireResolvedAsset.
ErrAssetNotAdmitted -> ErrAssetNotResolved (identity-resolve refusal). The EXTCODESIZE
OnChainAssetVerifier is now documented as the AUTHORITATIVE reality check (a synthetic
asset has no code -> ErrAssetNotOnChain). Fail-closed errors (no resolver/verifier)
unchanged.
- place.go OpenMarketChecked: same two orthogonal steps (resolve identity -> verify
on-chain reality), now permissionless: any REAL (base, quote) pair opens; only
malformed/synthetic/disabled refused. Value keying (left-pad) unchanged.
- resolver_test.go: replace the membership-allowlist fakeResolver with a permissionless
resolver. New positive proof (fails without the change): a REAL unregistered ERC-20
opens a market. Synthetic base now refused by the reality gate (ErrAssetNotOnChain),
preserving the safety property via the authoritative check, not an allowlist. Disabled
(kill switch) and malformed refused at resolve; nil resolver/verifier fail closed.
DeriveAssetID untouched (byte-identical). Conservation keying untouched.
The CLOB source's SettleFills credits the taker's dexcore available with the proceeds;
the AMM source did not (only SpendLocked + journal record), leaving the two sources
asymmetric on the D-surface. A caller draining the taker's transient by the journal
amount then over-drained for an AMM leg (the proceeds were never in available). Credit
the taker's available in the AMM source too, so after ANY swap the taker's available
holds exactly the journal-recorded proceeds. Conservation holds across (ledger + pool
reserves): input left the taker's locked into the pool base reserve; output left the
pool quote reserve into the taker's available — both legs balance to the unit.
The cEVM home's taker is a 20-byte EVM address; the prior 16-byte AccountID would
fold a 20-byte address lossily, so two distinct addresses sharing a leading 16-byte
prefix could collide as one dexcore account and drain each other's resting-order
proceeds (cross-user drain on the account axis). Widen AccountID to the full 32-byte
injective settlement identity both homes map into without a lossy fold: the cEVM
left-pads the 20-byte address (accountFromAddress), dchain left-pads its 16-byte wire
user. 32 bytes is birthday-safe at 2^128 and injective for every real identity.
matcherHint renders the full 32 account bytes as the matcher's User/UserID string —
the hint is ONLY a matching collision handle, never authoritative for value (settlement
resolves the taker from the explicit AccountID and the maker from the orderuser: row).
Replaces the trimTrailingZeros round-trip (no longer needed: settlement never reads the
hint back).
The caller-agnostic core both DEX homes call (the standalone D-Chain VM over a
versiondb overlay, and the in-process 0x9999 cEVM precompile over 0x9999 storage),
so there is ONE settlement algorithm and ONE router in ONE place:
- ledger.go: custody schema + conserving balance primitives (available/locked) over
a generic Store, full-injective AccountID(16)/AssetID(32) keys.
- settle.go: integer-exact fill conservation + fail-closed maker identity
(orderuser: index, ErrMissingSettlementUser, never the 8-byte handle).
- router.go: the on-chain smart-order-router — CLOB-first, AMM-fallthrough,
best-execution across V4+AMM with split; taker-authenticated MEV floor. Source
abstraction leaves room for an RFQ/external-CEX source LATER without reshaping.
RouteClass{PublicCLOB,FHEPrivate} reserves the staged private-pool route.
- journal.go: the JOINT JOURNAL — C-surface deltas recorded for the caller to mirror
onto EVM balances; D-surface writes go straight to the Store. Both-or-neither.
- source_clob.go / source_amm.go: the two liquidity sources (in-process matcher;
in-cEVM V2/V3 constant-product, 128-bit exact).
- place.go: maker resting-order place/cancel (custody-gated, fail-closed cancel).
- book.go / root.go: book schema + rebuild; RFC-6962 DEX execution root.
REAL assets only (assetID = left-pad of the 20-byte token address); no synthetic
ticker ids anywhere. 16 tests pass race-clean: conservation, fail-closed ownership
(handle-collision non-value-bearing), deterministic replay across independent
stores, two-layer MEV floor, CLOB/AMM routing + split, BUY-requires-limit.
Reword comparison docs, comments, and design prose to Lux's own terms
(native DEX core, EVM adapter / 0x9999 V4 PoolManager, native HFT surface,
one consensus-finalized C+D transition). Rename the test broker-venue label
and oracle source weight to neutral names. Behavior-preserving: no logic,
ABI, or wire format changes.
Every d-chain write is deduped by the hash of its frame (seen: key); re-sending
identical deposit/place/submit frames across runs returns the FIRST run's cached
outcome (stale orderId, empty fills) and proves nothing. Thread a wall-clock
nonce into the order SIZE (ask+buy same size so they fully cross) and the deposit
AMOUNT so each invocation is a genuinely new, funded cross. Proven on lux-devnet:
fresh fills submitted to luxd-0/2/4 each committed identically on all 5 validators.
A pure-Go (CGO=0) tool that audits the native D-Chain across a validator set:
submit a crossing order to ONE validator over the in-luxd HTTP ingestion seam
(POST /ext/bc/D/dex/clob_* with frozen zapwire frames), then read the committed
trade log (GET /ext/bc/D/dex/clob_get_trades) from EVERY validator and compare
the trade rows + accepted head root.
Verdict:
- all nodes show the identical fill + identical root -> PROVEN replicated under
the chain's quorum (not just the proposer).
- only the submit-target shows it -> SINGLE-PROPOSER (replication did not reach
followers).
- partial / roots diverge -> incomplete.
Links only pkg/zapwire (the one source of frame truth) + stdlib; decodes the
read responses as open JSON (exercises the v1.5.15 read schema exactly as an
external consumer would). Builds static for in-cluster runs.
The D-Chain VM exposed only write methods (clob_place/submit/cancel/...) over
its HTTP surface; there was no way to read committed chain state, so a fill
proven on the proposer could not be verified as replicated across validators,
and native fills (trade: rows) could not feed markets-display.
Add a READ surface (read.go) orthogonal to the write surface (handler.go):
- clob_get_trades?since=<h>&limit=<n> committed trade: log (markets feed +
cross-validator replication probe)
- clob_get_orders?market=<hex> resting order: rows for a market
- clob_get_markets every market: id/symbol/assets/summary
- clob_get_book?market=<hex>&depth=<n> aggregated bid/ask ladder + totals
Reads are JSON, GET, served via CreateHandlers on the SAME /ext/bc/<D>/dex/
route group as the writes. They read committed state directly under vm.mu —
NO mempool, NO block, NO consensus round — so a read can never change a fill or
balance (proven: TestRead_ZeroConsensusImpact). Every response carries the
accepted-block head (height, lastAccepted, root) so a cross-validator diff is
anchored to an exact block: two nodes at the same height+root MUST return
identical trades.
Trades/markets read durable rows (the committed truth, identical on every node
at a height); orders/book read the in-RAM book (a deterministic fold of the
committed order: rows). The 80-byte trade row carries no poolID (frozen GPU
ABI), so clob_get_trades is the GLOBAL height-ordered log — exactly right for a
single-market devnet (the verification case) and for an indexer that scopes
fills to a market by the order ids it tracks.
State helpers (state.go): readTrades (height-ordered durable scan), listMarkets.
lx: export DEXPrice.Float() + FixedQtyToFloat (one conversion; the internal
fixedPriceToFloat/fixedQtyToFloat now delegate).
TDD (read_test.go, all green, race-clean):
- TradesAfterCommittedFill: a matched cross is queryable with right px/size/side
- MarketsAndOrdersAndBook: the three other endpoints reflect committed state
- ZeroConsensusImpact: 25 reads advance neither height nor mempool
- CrossValidatorIdenticalTrades: two INDEPENDENT VMs (separate DBs) that
Accept the SAME block bytes return BYTE-IDENTICAL trades JSON + same root —
the in-process model of the live 5-validator replication question
- GenesisEmpty / BadParams: clean empty answers; boundary validation (400/405)
Removed the temporary zzz_fillrepro_test.go (its header said DELETE after;
superseded by read_test.go's committed-fill coverage).
database v1.20.4 fixes prefixdb NewIteratorWithStartAndPrefix synthesizing a
pre-prefix start for a nil-start prefix scan, which returned ZERO rows over a
prefixdb-wrapped store (every plugin VM's chain DB via vm/rpc) when sub-prefix
keys sorted before the target. This stranded the native D-Chain: rebuildBookFromDB
iterates order:<pool> alongside asset:/balance:/locked:/meta: rows, folded an
EMPTY book, and every taker cross produced 0 fills despite durably-committed
asks. With the bump the order book refolds correctly and crosses settle.
The luxd plugin transport (vm/rpc) resolves Block.Accept by ID via VM.GetBlock —
unlike Block.Verify, which carries the full block bytes (ParseBlock). The dexvm's
GetBlock only indexed ACCEPTED blocks, so GetBlock(builtID) for a block the engine
had just built returned ErrNotFound; the engine's self-finalize Accept
(finalizeOwnProposal) became a silent no-op, the clob_* submitTx waiter
(handler.go) hung forever, and the consensus forwarder wedged inside the hung
Accept so NO further D-Chain blocks were ever built. The EVM survives the same
ID-only Accept because geth resolves built-but-unaccepted blocks from its block
cache; the dexvm had no equivalent.
Add a processingBlocks index (built/verified-but-unaccepted blocks): BuildBlock
and ParseBlock record into it (ParseBlock returns the stable instance so Verify's
stashed overlay is the one Accept commits), GetBlock consults it after
acceptedBlocks, and Accept/Reject prune it. Smallest change that makes the
plugin Accept(ID) round-trip resolve the same instance built/verified — no
transport, engine, or matcher change.
TDD: TestVMAcceptViaGetBlock drives the exact plugin path BuildBlock ->
GetBlock(id) -> Accept (proven to FAIL without the lookup: 'not found, the
production stall'); TestVMParseVerifyAcceptByID covers the follower
ParseBlock -> Verify -> GetBlock(id) -> Accept. Full pkg/dchain suite green
CGO=0. Every prior test accepted the *Block from BuildBlock directly, skipping
the ID-resolve path the live node uses.
The block store is the in-RAM acceptedBlocks map; only bootstrapGenesis and
Accept populate it. On restart loadHead restored the head META (lastAccepted id,
height, root) but NOT the block object, so once the chain advanced past genesis
the engine's GetBlock(lastAccepted) — called immediately after Initialize —
returned ErrNotFound and luxd failed VM init with 'zap initialize: remote error:
get last accepted block: not found', taking the D-Chain down on the first
validator to restart.
Persist the head block's canonical bytes (metaHeadBlock) in the SAME atomic
overlay/batch as the head meta (Accept) and in bootstrapGenesis. loadHead now
reconstructs the head block via parseBlock (id/height asserted to match the head
pointer) and puts it back into acceptedBlocks. A legacy DB with no persisted head
block recovers genesis from the deterministic genesisBlock() (DRY with
bootstrapGenesis) and errors loudly for a non-genesis head rather than silently
resetting the chain.
TDD: TestVMRestartGetBlockHead advances two blocks, restarts (fresh VM, same DB),
and asserts GetBlock(head) returns the block — the exact engine call that failed;
proven to FAIL without the acceptedBlocks repopulation ('not found'), pass with.
TestVMRestartLegacyGenesisDB covers the no-metaHeadBlock genesis recovery. Full
pkg/dchain suite green CGO=0.
The in-luxd native D-Chain VM (pkg/dchain) bootstrapped consensus but never
exposed order ingestion: VM.CreateHandlers returned an empty map, so luxd
mounted nothing and /ext/bc/D/* 404'd. RegisterCLOB (the clob_* -> submitTx ->
mempool -> consensus -> Verify-match path) was only reachable over the ZAP
socket transport, which only cmd/dvenue (the abandoned standalone venue) wires.
Wire ingestion the idiomatic luxd way: CreateHandlers now returns one HTTP
handler per CLOB method (/dex/clob_place, /dex/clob_submit, ...), each adapting
the EXACT same rpc.RawHandler RegisterCLOB binds (one method->handler table,
two transports — vm.clobMethods). luxd mounts each as an exact route under
/ext/bc/<chainID> (and the D alias) through the node's own router. An order
POSTed to /ext/bc/D/dex/clob_submit flows submitTx -> mempool.Add -> consensus
PendingTxs -> BuildBlock -> Block.Verify (the matcher) -> Block.Accept, and the
response is the CONSENSUS-COMPUTED outcome. No external matcher, no relay, no
side port, no node image change — the chain is the matcher.
TDD: TestIngestCrossingOrdersMatchInBlock proves two crossing orders submitted
over the HTTP handler produce a fill committed as D-Chain state (trade: row,
height advances). CGO=0; full pkg/dchain suite green; plugin cross-builds
linux/amd64.
The D-Chain DEX VM (pkg/dchain) is the native consensus matcher: it runs
the lx.OrderBook matcher inside luxd consensus (Block.Verify over a
versiondb overlay) and is the chain VM luxd loads from PluginDir at the
dexvm vmID slot — exactly like the EVM plugin. There is no standalone
venue and no ZAP relay in the trading path.
cmd/dchain + runner carried a //go:build dchain && cgo gate from the old
'venue accelerator' framing, but they import ONLY pkg/dchain (pure-Go),
log, and vm/rpc — zero native deps. The gate excluded the plugin from
every CGO=0 build, which is exactly how the node Dockerfile builds chain
VM plugins. Drop the tag so cmd/dchain is the canonical CGO=0 native
plugin (same harness as chains/*/cmd/plugin). The optional GPU AMM
accelerator in pkg/lx stays gated by its own cuda/metal tags and is not
linked by this consensus plugin.
Proof: CGO_ENABLED=0 go build/vet ./cmd/dchain clean; native VM suite
(TestLocalnetVenueE2E, TestFourPath, conservation, ownership) passes.
The D-Chain (dexvm) is its own chain: trading is its consensus-driven state
transition (matcher-at-Verify in pkg/dchain). There is no off-chain keeper
driving 0x9999 — 0x9999 is only the C<->D settlement adapter. Remove the
keeper and its footprint:
- cmd/dexkeeper/ (the off-chain relayer)
- Dockerfile.dexkeeper
- .github/workflows/docker-dexkeeper.yml + docker.yml reference
- k8s/devnet-keeper/ (never deployed)
- go.mod: drop luxfi/chains dep (was pinned at v1.3.16 ONLY for the keeper;
nothing else in dex imports chains — go mod tidy removes it)
Native dchain VM + matcher build pure-Go (CGO_ENABLED=0) and the full
pkg/dchain suite passes (consensus match->Verify->Accept, conservation,
LUX/LUSD + LUX/LETH venue e2e, ownership red-team).
The hanzoai/.github reusable docker-build.yml is unreachable cross-org from
luxfi/* (GitHub 'workflow was not found' at validation -> 0s fail; the real
reason dex's docker.yml never ran). Give dexkeeper its own inline workflow on
the lux-build ARC pool, mirroring luxfi/node's proven Docker workflow exactly
(PAT resolve -> docker/build-push-action with secret id=ghtok). Align
Dockerfile.dexkeeper to consume id=ghtok. Remove the dexkeeper matrix entry
from the (cross-org-broken) reusable docker.yml.
The reusable docker-build.yml jobs declare id-token: write; a caller whose
top-level permissions are narrower is rejected by GitHub at workflow validation
(the 0s 'workflow file issue' failure — present since the workflow was added).
Every working caller of this reusable workflow grants id-token: write. This is
the actual parse-stage fix (the runner-input + secret fixes were also needed for
the build to RUN, but this is why it never started).
The docker.yml runner-amd64/arm64/deploy overrides were bare strings
(luxfi-amd64, …); the reusable workflow fromJson()-decodes them, so they
produced a 0s 'workflow file issue' failure (broken since 2026-06-19). Drop
the overrides -> canonical [self-hosted,linux,<arch>] defaults (the lux arcd
pool's labels; org runner-group + arcd allowlist isolate). Dockerfile.dexkeeper
now consumes the org PAT via --mount=type=secret,id=gh_token (the secret the
reusable workflow actually supplies) for the private luxfi/* module fetch,
ARG GITHUB_TOKEN kept as a local-build fallback.
cmd/dexkeeper watches the C-Chain 0x9999 IntentSubmitted event, submits the dexvm
ImportTx + settling RelayOrderTx (dex.submitTx), polls dex.getSettlement for the
D->C proceeds object, then has the maker sign+send the C-side Phase-B settle that
emits DEXFill. Maker key derived in-process from KMS-synced LUX_MNEMONIC at
m/44'/9000'/0'/0/i. Relay sent UNSIGNED (authority binds to escrow owner).
Dockerfile.dexkeeper (CGO=0) + docker.yml matrix entry (ghcr.io/luxfi/dexkeeper)
+ k8s/devnet-keeper manifests. Tests pass.
- Dockerfile.dexkeeper: pure-Go (CGO off) static keeper image; documents the
zap>=v0.8.10 + chains-RPC-seam tag prereqs.
- docker.yml: add ghcr.io/luxfi/dexkeeper to the build matrix.
- k8s/devnet-keeper/kmssecret-mnemonic.yaml: SANCTIONED maker-key path — a KMSSecret
(operator-reconciled) that syncs the devnet mnemonic from lux KMS (lux-infra/devnet,
/staking key=mnemonic, the proven luxd-staking-kms-sync scope) into LUX_MNEMONIC. No
plaintext handled out-of-band. Server-dry-run validated against the live CRD.
- deployment.yaml: the keeper in watch mode against devnet C/D RPC + dchain-venue:9099,
maker key from the synced secret, single replica (proposer-relays-once).
- job-drive-first-swap.yaml: one-shot seed + Phase-A swap to kick the first DEXFill.
- README.md: the full deploy sequence + verification (DEXFill eth_getLogs, subgraph, UI).
DEVNET ONLY. All manifests dry-run clean.
Two corrections to the Phase-B path:
1. CollateralRef bug: the settling RelayOrderTx now binds CollateralRef = ev.IntentID
(the C->D object's UTXOID), NOT importTx.ID(). executeImport keys the escrow by
tx.ImportedInputs[0].UTXOID (== intentID) and settleFromFills looks it up by
RelayOrderTx.CollateralRef; with importTx.ID() the escrow lookup missed (proceeds-only,
no refund, degraded authority bind). Now they agree.
2. getExportedOutput is no longer a stub: it polls dex.getSettlement(collateralRef) (the
new dexvm RPC) until the proposer relays + the settling block accepts and records the
proceeds object's (outputID, amount), then returns them for the Phase-B DS01 settlement.
~60s poll budget; on a zero-fill (no proceeds) it reports the reclaim path.
Builds linux/amd64 static (CGO off); keeper tests pass.
cmd/dexkeeper watches the C-Chain for IntentSubmitted (Phase-A intent: a maker
swap to 0x9999 with DI01 hookData), submits the ImportTx (consume the staged
C->D atomic object, fund the D order) + the settling RelayOrderTx (clob_submit,
collateral-bound) to the dexvm dex.submitTx RPC, then drives the Phase-B
settlement swap (DS01 hookData) that consumes the D->C proceeds object and emits
DEXFill. The keeper holds NO custody authority: value is locked in shared memory
by C before it acts, and the D->C object pays its recorded owner regardless.
- maker.go: BIP44 m/44'/9000'/0'/0/i maker-key derivation, pinned byte-for-byte
against genesis.LoadBIP44WalletKeysFromMnemonic. Uses the STANDARD EVM address
(keccak256(X||Y)[12:]) everywhere — NOT secp256k1.PrivateKey.EVMAddress(),
which hashes the compressed pubkey and yields a non-standard address.
- calldata.go: Phase-A/B 0x9999 calldata via zap/dexsession, intent-id
derivation mirroring precompile/dex DeriveIntentID, IntentSubmitted + DEXFill
decoders (fail-closed at the boundary).
- venue.go: ZAP maker-liquidity seeding, modeled on cmd/fourpath-live.
- keeper.go: watch loop + import/relay orchestration. Settling relay is sent
UNSIGNED with From = escrow owner (the architecturally-correct build).
- Modes: seed | phase-a | watch. Maker key from LUX_MNEMONIC.
Unit tests (10, all passing) cover the pure logic: maker derivation vs the
genesis authority + a known vector, Phase-A/B calldata byte-identity + DI01/DS01
shape, intent-id determinism/injectivity, and synthetic event decode (+ negative
cases). Builds CGO_ENABLED=0 for linux/amd64 (static ELF).
Known gap (documented stub): getExportedOutput — the D->C export discovery needs
a dexvm settlement-coordinate query (dex.getSettlement) or a C-Chain importable-
UTXO view; neither is exposed today. Phase-A path is fully wired.
SubmitMarketable and MatchOrders derived their returned fills by re-slicing
the shared trade log (ob.Trades[startOffset:]) with an offset captured BEFORE
matching. ob.Trades self-truncates at 100k entries (keeps the last 50k), so a
match that crosses that boundary mid-match trims the log and leaves the
pre-match offset pointing past the end -> panic 'slice bounds out of range'.
The log feeds settlement, so fills had to stay exact.
tryMatchImmediateLocked / MatchOrders now capture each fill as it is produced,
independent of ob.Trades, and return that slice. Decomplects 'this match's
fills' (a value the caller needs) from 'the trade log' (a bounded-retention
buffer): correct across the truncation boundary by construction, deterministic
across validators.
Adds orderbook_truncation_test.go: three regression cases that cross the
boundary through the production match path (single submits, one sweeping
submit, legacy MatchOrders) with exact-fill and value-conservation asserts.
They panic on the old code, pass on the fixed code. No sleeps, no stubs.
Also: 0x9999 D-matches/C-settles docs (LLM.md, README, ARCHITECTURE),
venue-canonical naming, fourpath-live proof tool, drop dead snowman-api key
from run-lux-testnet.sh.
Adds the missing SERVER side of the FIX trading path and a unified e2e
proving REAL fills over all four DEX access paths against ONE book.
The decomplection: the D-Chain CLOB (pkg/dchain) is the single matcher,
reached over the single frozen clob_* ZAP surface (pkg/zapwire). Every
access path is a thin wire dialect onto that surface — none owns a
matcher. This commit adds the two missing dialects (FIX, a focused WS
venue handler) and the shared seam they translate through.
New:
- pkg/venue: ONE Venue interface (ensure/place/cancel/submit = the
clob_* ops) + ONE production ZAP-backed impl. The seam FIX, WS, and
the 0x9010 precompile all route through. Dials the SAME rpc.Listen
socket dvenue serves.
- pkg/fix: dependency-free FIX 4.4 codec (BodyLength tag 9 + CheckSum
tag 10 computed AND validated; spec-correct emit/parse) + a TCP
acceptor (Logon/Heartbeat/TestRequest/Logout, monotonic seq,
NewOrderSingle->clob_submit->ExecutionReport, OrderCancelRequest->
clob_cancel). Pure protocol front-end onto Venue; no matcher.
- pkg/api/venue_ws.go: minimal gorilla/websocket /ws handler that
translates a JSON order to Venue.Submit and streams fills back —
the WS dialect onto the SAME venue (kept orthogonal to the existing
TradingEngine product WS server).
- cmd/fix-gateway: runs the acceptor in front of a live venue
(-venue clob_* endpoint), the production form of the FIX path.
- pkg/dchain/fourpath_e2e_test.go: stands up ONE venue + ONE LUX/LUSD
book and drives a real limit-order -> match -> FILL over EACH path
(V4 precompile engine_zap frame | ZAP native | FIX | WS) against the
SAME resting asks, asserting real consensus fills, per-path custody
changes, and GLOBAL conservation (available+locked invariant).
V4 leg: exercised at the engine_zap WIRE boundary — the exact 66-byte
clob_submit frame engine_zap.Swap emits (asserted byte-identical to
zapwire.EncodeSubmit) + the V4 BalanceDelta sign convention derived from
the consensus fills. A live luxd EVM (StateDB replay-idempotency, gas,
PoolManager) is what the full on-chain precompile call additionally
needs; the matcher boundary is proven in-process here.
No deps added; no existing files modified (go.mod/go.sum untouched).
Builds CGO_ENABLED=0; race-clean.
Drop 31 demo/benchmark/experiment one-off cmds whose coverage lives in
package tests (bench-all, dag-network, dclient, dexe2e, e2e-fix-zmq,
fix-benchmark, hybrid-auto, latency-benchmark, memory-*, mlx-memory-analysis,
multi-node, multi-node-test, nats-*, perf-test, stress-test, test-client,
zmq-*, dvenue-cancel, dvenue-custody, venueprobe, empty api-server/server/trader).
Keep production binaries: dvenue, dchain, dex-server, dex-api-server, gateway,
price-feed, monitor, persistent-server, demo (each referenced by a
Dockerfile/Makefile/script/k8s manifest or a documented entrypoint).
Consolidate cmd/dvenue + cmd/dvenue-cpu into one tag-free cmd/dvenue: the two
mains were byte-identical except a build tag and a log label; the GPU-vs-CPU
matcher is already selected inside pkg/lx by its own cgo-tagged files, so the
main needs no tag. CGO_ENABLED alone picks the matcher; one binary builds in
both modes. engine_{cgo,nocgo}.go carry only the engine display label. dvenue
now builds under the default CGO_ENABLED=0 gate (was tag-excluded before).
Dockerfile.dvenue: drop the dvenue-custody build+copy (cmd removed).
Makefile build-tools: point at real tools (price-feed, monitor) instead of the
removed cmd/trader and never-existent cmd/lx-metrics.
pkg/price: fix the three CChainSource tests. They were gated on requireLuxd
(a luxd TCP dial) but CChainSource never dials luxd — fetch() is pure simulate().
The gate skipped only when luxd was down, masking the real defect on hosts where
luxd runs. Rewrite the tests to drive the source's true contract deterministically
via src.poll() (no ticker race) and assert against the symbols the source actually
polls (cchainTokens), plus not-found for untracked symbols. Remove the dead
requireLuxd helper and its net import.
orderuser->full 16-byte user is the only settlement identity; 8-byte UserID is a matcher/STP handle only, never a money key. Missing orderuser fails closed (ErrMissingSettlementUser), no matcher-handle fallback. Ownership invariant harness alongside conservation (catches value-conserving theft). txHash-ref custody idempotency closes vault-drain/strand. Reject zero-lock orders; persist orderuser before book insert; restart asserts coverage.
Adds runIDOR — the live devnet counterpart to the precompile unit test
TestZAPCancelAuthRejectsOthersAcrossMultiExec. A maker rests an ask; a DIFFERENT
account attempts to cancel it with the same tick band + salt; asserts the attacker's
cancel REVERTS, the maker's lock survives, and the maker can still cancel its own
order. Proves the durable cancel-authorization binding (precompile RED H4) closes the
EVM-path cancel revert WITHOUT opening an IDOR.
bootVenue opened the on-disk chainstate via badgerdb.New; use zapdb.New
(same signature, same on-disk format). Bump luxfi/vm v1.2.3→v1.2.4 to pick
up the matching rpc/zap zapdb harness. Transitive api v1.0.12→v1.0.15,
warp v1.18.6→v1.19.2 (v1.x). go build+vet ./pkg/dchain/ clean.
NEW cmd/dvenue-cpu: //go:build dchain (no cgo) — the FREE pure-Go CPU D-Chain venue daemon
(drives dchain.VM's CPU matcher; builds CGO=0; published as ghcr.io/luxfi/dchain:cpu). The
cgo cmd/dvenue stays the LICENSED GPU venue (//go:build dchain && cgo). Both now open the DB
via github.com/luxfi/database/zapdb (NOT badgerdb directly — per the zapdb-only rule).
The venue image has its own cadence and must NOT build on every luxfi/dex SDK
v* tag (would fail + waste runner time). Use a dedicated venue-v* tag prefix;
strip it for the image tag via type=match group=1 so the image is :v0.1.0.
The missing artifact that blocks D-Chain turn-on: ghcr.io/luxfi/dchain-venue,
the private build that serves the P-chain-registered D-Chain (the public node
image excludes the DexVM). Bundles two binaries the operator venue StatefulSet
runs: dvenue (the ZAP CLOB matcher, -tags dchain,cgo, links the CUDA kernels +
secp256k1 via the luxcpp/dex venue dep bundle) and the dchain-tagged luxd
(-tags nattraversal,dchain, CGO=0 pure-Go proxy) + the EVM C-Chain plugin.
arm64-only: the matcher's cgo+linux path (orderbook_cuda.go + amm_gpu_cuda.go)
has no non-CUDA fallback on any arch, and the only CUDA arcd runner is spark
(GB10, aarch64). Deliberately omits the grpc tag (it flips zapdb/pb to the
protobuf variant which drops Marshal/Unmarshal — does not compile vs zapdb
v1.10.1; verified). Mirrors node/Dockerfile's private-dep fetch + node/docker.yml
PAT pattern. Prereq: luxcpp/dex dex-gpu-release must publish the bundle first.
CRITICAL: the prior taker-only settlement debited the maker's resting order but NEVER credited
the maker the taker's payment — a value hole on every two-sided fill (old test only audited the
taker leg). FIX: per-account ledger in pkg/dchain (balance:/locked:/asset:/orderasset:; checked-int
lockFromAvailable/unlockToAvailable/spendLocked) — a fill is a pure intra-D-Chain balance move,
wei-exact on BOTH legs. settle.go (orderLock/settleFills/quoteUnitsCeil: lock>=spend), block.go
custody gate (lock-before-match, unfunded→reject) + settleOrderEffects. rebuildBookFromDB now
restoreIntegerLane() — RowToOrder didn't round-trip SizeUnits/RemainingUnits → makers from disk
had a broken integer lane → nil-units fills settle refused. zapwire ADDITIVE custody frames
(clob_deposit/withdraw/open_market); 4 frozen CLOB frames byte-untouched. Conservation tests
(real VM) + live 1337 e2e (deposit→cross→maker-credited→withdraw 1100==1100→restart) green.
pkg/dchain: store original outcome under seen:<txID> and return it verbatim on replay so a
deduped clob_place is distinguishable from a genuine reject (was returning StatusRejected →
reverted the EVM tx that re-executes ~5x). BookDepth read accessor. cmd/dvenue = the dial-able
venue D-Chain ZAP listener (badgerdb + vm.Initialize + rpc.Listen + RegisterCLOB + production
sealer) the 0x9010 precompile + dexvm proxy point at; cmd/dclient = native-ZAP e2e driver.
zapdb v1.10.0→v1.10.1 (cloud-free, cgo blocker resolved on the v1 lineage). Tests green.
zeekay